The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation
European Radiology · 2024
DOI: 10.1007/s00330-024-10727-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift European Radiology, 2024. Verfasst von Robert Siepmann und 11 weiteren Autor:innen. Beteiligte Einrichtung: Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. Hauptthemen laut MEDLINE: Artificial Intelligence, Radiologists. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Horizon 2020 Framework Programme, Deutsche Forschungsgemeinschaft, BMBF.
Abstract (Originalsprache)
Objectives
Large language models (LLMs) have shown potential in radiology, but their ability to aid radiologists in interpreting imaging studies remains unexplored. We investigated the effects of a state-of-the-art LLM (GPT-4) on the radiologists’ diagnostic workflow. Materials and methods In this retrospective study, six radiologists of different experience levels read 40 selected radiographic [ n = 10], CT [ n = 10], MRI [ n = 10], and angiographic [ n = 10] studies unassisted (session one) and assisted by GPT-4 (session two). Each imaging study was presented with demographic data, the chief complaint, and associated symptoms, and diagnoses were registered using an online survey tool. The impact of Artificial Intelligence (AI) on diagnostic accuracy, confidence, user experience, input prompts, and generated responses was assessed. False information was registered. Linear mixed-effect models were used to quantify the factors (fixed: experience, modality, AI assistance; random: radiologist) influencing diagnostic accuracy and confidence.
Results
When assessing if the correct diagnosis was among the top-3 differential diagnoses, diagnostic accuracy improved slightly from 181/240 (75.4%, unassisted) to 188/240 (78.3%, AI-assisted). Similar improvements were found when only the top differential diagnosis was considered. AI assistance was used in 77.5% of the readings. Three hundred nine prompts were generated, primarily involving differential diagnoses (59.1%) and imaging features of specific conditions (27.5%). Diagnostic confidence was significantly higher when readings were AI-assisted ( p > 0.001). Twenty-three responses (7.4%) were classified as hallucinations, while two (0.6%) were misinterpretations.
Conclusion
Integrating GPT-4 in the diagnostic process improved diagnostic accuracy slightly and diagnostic confidence significantly. Potentially harmful hallucinations and misinterpretations call for caution and highlight the need for further safeguarding measures. Clinical relevance statement Using GPT-4 as a virtual assistant when reading images made six radiologists of different experience levels feel more confident and provide more accurate diagnoses; yet, GPT-4 gave factually incorrect and potentially harmful information in 7.4% of its responses.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany.
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. snebelung@ukaachen.de.
Förderung
- Horizon 2020 Framework Programme
- Deutsche Forschungsgemeinschaft
- BMBF
Zitieren
APA
Siepmann, R., Huppertz, M., Rastkhiz, A., Reen, M., Corban, E., Schmidt, C., Wilke, S., Schad, P., Yüksel, C., Kuhl, C., Truhn, D., & Nebelung, S. (2024). The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation. European Radiology. https://doi.org/10.1007/s00330-024-10727-2
Vancouver
Siepmann R, Huppertz M, Rastkhiz A, Reen M, Corban E, Schmidt C, et al. The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation. European Radiology. 2024. doi:10.1007/s00330-024-10727-2
BibTeX
@article{Siepmann2024,
author = {Siepmann, Robert and Huppertz, Marc and Rastkhiz, Annika and Reen, Matthias and Corban, Eric and Schmidt, Christian and Wilke, Stephan and Schad, Philipp and Yüksel, Can and Kuhl, Christiane and Truhn, Daniel and Nebelung, Sven},
title = {The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation},
journal = {European Radiology},
year = {2024},
doi = {10.1007/s00330-024-10727-2}
}
EUVIMED