EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation

Robert Siepmann, Marc Huppertz, Annika Rastkhiz, Matthias Reen, Eric Corban, Christian Schmidt, Stephan Wilke, Philipp Schad, Can Yüksel, Christiane Kuhl, Daniel Truhn, Sven Nebelung

European Radiology · 2024

DOI: 10.1007/s00330-024-10727-2 ↗

Kurz gefasst

Fachartikel in der Zeitschrift European Radiology, 2024. Verfasst von Robert Siepmann und 11 weiteren Autor:innen. Beteiligte Einrichtung: Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. Hauptthemen laut MEDLINE: Artificial Intelligence, Radiologists. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Horizon 2020 Framework Programme, Deutsche Forschungsgemeinschaft, BMBF.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

Large language models (LLMs) have shown potential in radiology, but their ability to aid radiologists in interpreting imaging studies remains unexplored. We investigated the effects of a state-of-the-art LLM (GPT-4) on the radiologists’ diagnostic workflow. Materials and methods In this retrospective study, six radiologists of different experience levels read 40 selected radiographic [ n = 10], CT [ n = 10], MRI [ n = 10], and angiographic [ n = 10] studies unassisted (session one) and assisted by GPT-4 (session two). Each imaging study was presented with demographic data, the chief complaint, and associated symptoms, and diagnoses were registered using an online survey tool. The impact of Artificial Intelligence (AI) on diagnostic accuracy, confidence, user experience, input prompts, and generated responses was assessed. False information was registered. Linear mixed-effect models were used to quantify the factors (fixed: experience, modality, AI assistance; random: radiologist) influencing diagnostic accuracy and confidence.

Results

When assessing if the correct diagnosis was among the top-3 differential diagnoses, diagnostic accuracy improved slightly from 181/240 (75.4%, unassisted) to 188/240 (78.3%, AI-assisted). Similar improvements were found when only the top differential diagnosis was considered. AI assistance was used in 77.5% of the readings. Three hundred nine prompts were generated, primarily involving differential diagnoses (59.1%) and imaging features of specific conditions (27.5%). Diagnostic confidence was significantly higher when readings were AI-assisted ( p > 0.001). Twenty-three responses (7.4%) were classified as hallucinations, while two (0.6%) were misinterpretations.

Conclusion

Integrating GPT-4 in the diagnostic process improved diagnostic accuracy slightly and diagnostic confidence significantly. Potentially harmful hallucinations and misinterpretations call for caution and highlight the need for further safeguarding measures. Clinical relevance statement Using GPT-4 as a virtual assistant when reading images made six radiologists of different experience levels feel more confident and provide more accurate diagnoses; yet, GPT-4 gave factually incorrect and potentially harmful information in 7.4% of its responses.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceRadiologistsHumansRetrospective StudiesFemaleMaleImage Interpretation, Computer-AssistedDiagnostic ImagingClinical CompetenceRadiologyMiddle Aged

Stichwörter der Autor:innen

Artificial intelligenceDiagnostic errorsDiagnostic imagingRadiology

Autor:innen und Institutionen

  • Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany.
  • Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. snebelung@ukaachen.de.

Förderung

  • Horizon 2020 Framework Programme
  • Deutsche Forschungsgemeinschaft
  • BMBF

Zitieren

APA

Siepmann, R., Huppertz, M., Rastkhiz, A., Reen, M., Corban, E., Schmidt, C., Wilke, S., Schad, P., Yüksel, C., Kuhl, C., Truhn, D., & Nebelung, S. (2024). The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation. European Radiology. https://doi.org/10.1007/s00330-024-10727-2

Vancouver

Siepmann R, Huppertz M, Rastkhiz A, Reen M, Corban E, Schmidt C, et al. The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation. European Radiology. 2024. doi:10.1007/s00330-024-10727-2

BibTeX
@article{Siepmann2024,
  author = {Siepmann, Robert and Huppertz, Marc and Rastkhiz, Annika and Reen, Matthias and Corban, Eric and Schmidt, Christian and Wilke, Stephan and Schad, Philipp and Yüksel, Can and Kuhl, Christiane and Truhn, Daniel and Nebelung, Sven},
  title = {The virtual reference radiologist: comprehensive AI assistance for clinical image reading and interpretation},
  journal = {European Radiology},
  year = {2024},
  doi = {10.1007/s00330-024-10727-2}
}
Kostenlos starten →