AI-generated and doctors' answers to health-related questions
Tidsskrift for Den norske legeforening · 2025
DOI: 10.4045/tidsskr.24.0402 ↗Kurz gefasst
Fachartikel in der Zeitschrift Tidsskrift for Den norske legeforening, 2025. Verfasst von Tiril Egset Mork und 6 weiteren Autor:innen. Beteiligte Einrichtung: Det medisinske fakultet, Universitetet i Bergen. Hauptthemen laut MEDLINE: Physicians, Artificial Intelligence. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 5 Literaturangaben.
Abstract (Originalsprache)
Background
Several studies have investigated how large language models answer health-related questions. In a study from 2023, responses to health-related questions in English generated by the language model GPT-3.5 were perceived as more empathetic and informative than responses from doctors. We wanted to apply the newer language model GPT-4 in Norwegian to investigate how respondents with a healthcare background rated responses to health-related questions from doctors and those generated by the language model. Material and method: A total of 192 health-related questions with corresponding answers from doctors were sourced from the website Studenterspør.no. The language model GPT-4 was used to generate a new set of answers to the same questions. Both sets of answers were evaluated by 344 respondents with a background in health care. The respondents, who were blinded to whether the answer was generated by a doctor or the language model, were asked to rate the empathy, quality of information and helpfulness of the answers.
Results
The survey consisted of 344 respondents and 192 questions. The average number of evaluations per answer was 5.7. There was a significant difference between doctors' answers and those generated by GPT-4 in terms of perceived empathy (p < 0.001), quality of information (p < 0.001) and helpfulness (p < 0.001).
Interpretation
The answers generated by GPT-4 were rated as more empathetic, informative and helpful than the answers from doctors. This suggests that AI could serve as an aid to healthcare personnel by drafting good responses to health-related questions.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Det medisinske fakultet, Universitetet i Bergen.
- Høgskulen på Vestlandet.
- Institutt for datateknologi, elektroteknologi og realfag, Høgskulen på Vestlandet.
- Institutt for Biomedisin, Universitetet i Bergen.
- Kirurgisk serviceklinikk, Haukeland universitetssjukehus.
Zitieren
APA
Mork, T. E., Mjøs, H. G., Nilsen, H. G., Kjelsrud, S., Lundervold, A. S., Lundervold, A., & Jammer, I. (2025). AI-generated and doctors' answers to health-related questions. Tidsskrift for Den norske legeforening. https://doi.org/10.4045/tidsskr.24.0402
Vancouver
Mork TE, Mjøs HG, Nilsen HG, Kjelsrud S, Lundervold AS, Lundervold A, et al. AI-generated and doctors' answers to health-related questions. Tidsskrift for Den norske legeforening. 2025. doi:10.4045/tidsskr.24.0402
BibTeX
@article{Mork2025,
author = {Mork, Tiril Egset and Mjøs, Håkon Garnes and Nilsen, Harald Giskegjerde and Kjelsrud, Sindre and Lundervold, Alexander Selvikvåg and Lundervold, Arvid and Jammer, Ib},
title = {AI-generated and doctors' answers to health-related questions},
journal = {Tidsskrift for Den norske legeforening},
year = {2025},
doi = {10.4045/tidsskr.24.0402}
}
EUVIMED