EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Assessing GPT-4 multimodal performance in radiological image analysis

Dana Brin, Vera Sorin, Yiftach Barash, Eli Konen, Benjamin S Glicksberg, Girish N Nadkarni, Eyal Klang

European Radiology · 30.08.2024 · 35 (4) · 1959-1965

DOI: 10.1007/s00330-024-11035-5 ↗

Kurz gefasst

Fachartikel in der Zeitschrift European Radiology, online erschienen am 30.08.2024. Verfasst von Dana Brin und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Diagnostic Imaging, Chaim Sheba Medical Center, Tel Hashomer, Israel. dannabrin@gmail.com. Hauptthemen laut MEDLINE: Artificial Intelligence, Image Interpretation, Computer-Assisted. Die Arbeit wurde bisher 110-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

This study aims to assess the performance of a multimodal artificial intelligence (AI) model capable of analyzing both images and textual data (GPT-4V), in interpreting radiological images. It focuses on a range of modalities, anatomical regions, and pathologies to explore the potential of zero-shot generative AI in enhancing diagnostic processes in radiology.

Methods

We analyzed 230 anonymized emergency room diagnostic images, consecutively collected over 1 week, using GPT-4V. Modalities included ultrasound (US), computerized tomography (CT), and X-ray images. The interpretations provided by GPT-4V were then compared with those of senior radiologists. This comparison aimed to evaluate the accuracy of GPT-4V in recognizing the imaging modality, anatomical region, and pathology present in the images.

Results

GPT-4V identified the imaging modality correctly in 100% of cases (221/221), the anatomical region in 87.1% (189/217), and the pathology in 35.2% (76/216). However, the model’s performance varied significantly across different modalities, with anatomical region identification accuracy ranging from 60.9% (39/64) in US images to 97% (98/101) and 100% (52/52) in CT and X-ray images ( p < 0.001). Similarly, pathology identification ranged from 9.1% (6/66) in US images to 36.4% (36/99) in CT and 66.7% (34/51) in X-ray images ( p < 0.001). These variations indicate inconsistencies in GPT-4V’s ability to interpret radiological images accurately.

Conclusion

While the integration of AI in radiology, exemplified by multimodal GPT-4, offers promising avenues for diagnostic enhancement, the current capabilities of GPT-4V are not yet reliable for interpreting radiological images. This study underscores the necessity for ongoing development to achieve dependable performance in radiology diagnostics. Clinical relevance statement Although GPT-4V shows promise in radiological image interpretation, its high diagnostic hallucination rate (> 40%) indicates it cannot be trusted for clinical use as a standalone tool. Improvements are necessary to enhance its reliability and ensure patient safety. Key Points GPT-4V’s capability in analyzing images offers new clinical possibilities in radiology . GPT-4V excels in identifying imaging modalities but demonstrates inconsistent anatomy and pathology detection . Ongoing AI advancements are necessary to enhance diagnostic reliability in radiological applications .

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceImage Interpretation, Computer-AssistedHumansTomography, X-Ray ComputedFemaleUltrasonographyMaleReproducibility of ResultsMiddle AgedAdult

Stichwörter der Autor:innen

Artificial intelligenceComputed tomography (x-ray)Diagnostic imagingRadiologyUltrasonography

Autor:innen und Institutionen

  • Department of Diagnostic Imaging, Chaim Sheba Medical Center, Tel Hashomer, Israel. dannabrin@gmail.com.
  • Faculty of Medicine, Tel-Aviv University, Tel Aviv-Yafo, Israel. dannabrin@gmail.com.
  • Department of Diagnostic Imaging, Chaim Sheba Medical Center, Tel Hashomer, Israel.
  • Faculty of Medicine, Tel-Aviv University, Tel Aviv-Yafo, Israel.
  • DeepVision Lab, Chaim Sheba Medical Center, Tel Hashomer, Israel.
  • Hasso Plattner Institute for Digital Health, Icahn School of Medicine at Mount Sinai, New York, New York, USA.

Zitieren

APA

Brin, D., Sorin, V., Barash, Y., Konen, E., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2024). Assessing GPT-4 multimodal performance in radiological image analysis. European Radiology, 35(4), 1959-1965. https://doi.org/10.1007/s00330-024-11035-5

Vancouver

Brin D, Sorin V, Barash Y, Konen E, Glicksberg BS, Nadkarni GN, et al. Assessing GPT-4 multimodal performance in radiological image analysis. European Radiology. 2024;35(4):1959-1965. doi:10.1007/s00330-024-11035-5

BibTeX
@article{Brin2024,
  author = {Brin, Dana and Sorin, Vera and Barash, Yiftach and Konen, Eli and Glicksberg, Benjamin S and Nadkarni, Girish N and Klang, Eyal},
  title = {Assessing GPT-4 multimodal performance in radiological image analysis},
  journal = {European Radiology},
  year = {2024},
  volume = {35},
  number = {4},
  pages = {1959--1965},
  doi = {10.1007/s00330-024-11035-5}
}
Kostenlos starten →