EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures

Eric Einspänner iD, Roland Schwab iD, Sebastian Hupfeld iD, Maximilian Thormann iD, Erelle Fuchs, Matthias Gawlitza, Jan Borggrefe iD, Daniel Behme iD

British Journal of Radiology · 2025

DOI: 10.1093/bjr/tqaf225 ↗

Kurz gefasst

Fachartikel in der Zeitschrift British Journal of Radiology, 2025. Verfasst von Eric Einspänner und 7 weiteren Autor:innen. Beteiligte Einrichtung: Clinic for Neuroradiology, Otto von Guericke University Magdeburg Medical Faculty, Magdeburg, 39120, Germany. Hauptthemen laut MEDLINE: Informed Consent, Patient Education as Topic, Language, Radiology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

This study evaluated 3 LLM chatbots (GPT-3.5-turbo, GPT-4-turbo, and GPT-4o) on their effectiveness in supporting patient education by answering common patient questions for CT, MRI, and DSA informed consent, assessing their accuracy and clarity.

Methods

Two radiologists formulated 90 questions categorized as general, clinical, or technical. Each LLM answered every question 5×. Radiologists then rated the responses for medical accuracy and clarity, while medical physicists assessed technical accuracy using a Likert scale. Semantic similarity was analyzed with SBERT and cosine similarity.

Results

Ratings improved with newer model versions. Linear mixed-effects models revealed that GPT-4 models were rated significantly higher than GPT-3.5 (P < .001) by both physicians and physicists. However, physicians’ ratings for GPT-4 models showed a significant performance decrease for complex modalities like DSA and MRI (P < .01), a pattern not observed in physicists’ ratings. SBERT analysis revealed high internal consistency across all models. SBERT analysis revealed high internal consistency across all models.

Conclusion

Variability in ratings revealed that while models effectively handled general and technical questions, they struggled with contextually complex medical inquiries requiring personalized responses and nuanced understanding. Statistical analysis confirms that while newer models are superior, their performance is modality-dependent and perceived differently by clinical and technical experts. Advances in knowledge This study evaluates the potential of LLMs to enhance informed consent in radiology, highlighting strengths in general and technical questions while noting limitations with complex clinical inquiries, with performance varying significantly by model type and imaging modality.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Informed ConsentPatient Education as TopicLanguageRadiologyHumansMagnetic Resonance ImagingLarge Language Models

Stichwörter der Autor:innen

AIChatGPTLLMNLPradiology

Autor:innen und Institutionen

  • Clinic for Neuroradiology, Otto von Guericke University Magdeburg Medical Faculty, Magdeburg, 39120, Germany.
  • Department of Radiation Protection Magdeburg, University Hospital, Otto-von-Guericke-University, Magdeburg, 39120, Germany.
  • Department of Nuclear Medicine, Charité Universitaetsmedizin Berlin, Berlin, 10117, Germany.
  • Section Neuroradiology, University Hospital Jena Institute of Diagnostic and Interventional Radiology, Jena, 07747, Germany.
  • Department of Radiology, Neuroradiology, and Nuclear Medicine, Johannes-Wesling-Hospital, Ruhr-University-Bochum, Bochum, 44789, Germany.
  • STIMULATE Research Campus, Otto von Guericke University Magdeburg, Magdeburg, 39106, Germany.

ORCID-Profile: Eric Einspänner, Roland Schwab, Sebastian Hupfeld, Maximilian Thormann, Jan Borggrefe, Daniel Behme

Zitieren

APA

Einspänner, E., Schwab, R., Hupfeld, S., Thormann, M., Fuchs, E., Gawlitza, M., Borggrefe, J., & Behme, D. (2025). Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures. British Journal of Radiology. https://doi.org/10.1093/bjr/tqaf225

Vancouver

Einspänner E, Schwab R, Hupfeld S, Thormann M, Fuchs E, Gawlitza M, et al. Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures. British Journal of Radiology. 2025. doi:10.1093/bjr/tqaf225

BibTeX
@article{Einspnner2025,
  author = {Einspänner, Eric and Schwab, Roland and Hupfeld, Sebastian and Thormann, Maximilian and Fuchs, Erelle and Gawlitza, Matthias and Borggrefe, Jan and Behme, Daniel},
  title = {Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures},
  journal = {British Journal of Radiology},
  year = {2025},
  doi = {10.1093/bjr/tqaf225}
}
Kostenlos starten →