Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures
British Journal of Radiology · 2025
DOI: 10.1093/bjr/tqaf225 ↗Kurz gefasst
Fachartikel in der Zeitschrift British Journal of Radiology, 2025. Verfasst von Eric Einspänner und 7 weiteren Autor:innen. Beteiligte Einrichtung: Clinic for Neuroradiology, Otto von Guericke University Magdeburg Medical Faculty, Magdeburg, 39120, Germany. Hauptthemen laut MEDLINE: Informed Consent, Patient Education as Topic, Language, Radiology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Objectives
This study evaluated 3 LLM chatbots (GPT-3.5-turbo, GPT-4-turbo, and GPT-4o) on their effectiveness in supporting patient education by answering common patient questions for CT, MRI, and DSA informed consent, assessing their accuracy and clarity.
Methods
Two radiologists formulated 90 questions categorized as general, clinical, or technical. Each LLM answered every question 5×. Radiologists then rated the responses for medical accuracy and clarity, while medical physicists assessed technical accuracy using a Likert scale. Semantic similarity was analyzed with SBERT and cosine similarity.
Results
Ratings improved with newer model versions. Linear mixed-effects models revealed that GPT-4 models were rated significantly higher than GPT-3.5 (P < .001) by both physicians and physicists. However, physicians’ ratings for GPT-4 models showed a significant performance decrease for complex modalities like DSA and MRI (P < .01), a pattern not observed in physicists’ ratings. SBERT analysis revealed high internal consistency across all models. SBERT analysis revealed high internal consistency across all models.
Conclusion
Variability in ratings revealed that while models effectively handled general and technical questions, they struggled with contextually complex medical inquiries requiring personalized responses and nuanced understanding. Statistical analysis confirms that while newer models are superior, their performance is modality-dependent and perceived differently by clinical and technical experts. Advances in knowledge This study evaluates the potential of LLMs to enhance informed consent in radiology, highlighting strengths in general and technical questions while noting limitations with complex clinical inquiries, with performance varying significantly by model type and imaging modality.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Clinic for Neuroradiology, Otto von Guericke University Magdeburg Medical Faculty, Magdeburg, 39120, Germany.
- Department of Radiation Protection Magdeburg, University Hospital, Otto-von-Guericke-University, Magdeburg, 39120, Germany.
- Department of Nuclear Medicine, Charité Universitaetsmedizin Berlin, Berlin, 10117, Germany.
- Section Neuroradiology, University Hospital Jena Institute of Diagnostic and Interventional Radiology, Jena, 07747, Germany.
- Department of Radiology, Neuroradiology, and Nuclear Medicine, Johannes-Wesling-Hospital, Ruhr-University-Bochum, Bochum, 44789, Germany.
- STIMULATE Research Campus, Otto von Guericke University Magdeburg, Magdeburg, 39106, Germany.
ORCID-Profile: Eric Einspänner, Roland Schwab, Sebastian Hupfeld, Maximilian Thormann, Jan Borggrefe, Daniel Behme
Zitieren
APA
Einspänner, E., Schwab, R., Hupfeld, S., Thormann, M., Fuchs, E., Gawlitza, M., Borggrefe, J., & Behme, D. (2025). Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures. British Journal of Radiology. https://doi.org/10.1093/bjr/tqaf225
Vancouver
Einspänner E, Schwab R, Hupfeld S, Thormann M, Fuchs E, Gawlitza M, et al. Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures. British Journal of Radiology. 2025. doi:10.1093/bjr/tqaf225
BibTeX
@article{Einspnner2025,
author = {Einspänner, Eric and Schwab, Roland and Hupfeld, Sebastian and Thormann, Maximilian and Fuchs, Erelle and Gawlitza, Matthias and Borggrefe, Jan and Behme, Daniel},
title = {Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures},
journal = {British Journal of Radiology},
year = {2025},
doi = {10.1093/bjr/tqaf225}
}
EUVIMED