EUVIMEDEUVIMED entdecken
VergleichsstudieEvaluationsstudieOpen Access · CC BY 4.0

Intra-axial primary brain tumor differentiation: comparing large language models on structured MRI reports vs. radiologists on images

Takeshi Nakaura, Hiroyuki Uetani, Naofumi Yoshida, Naoki Kobayashi, Yasunori Nagayama, Masafumi Kidoh, Jun-Ichiro Kuroda, Akitake Mukasa, Toshinori Hirai

European Radiology · 22.08.2025 · 36 (2) · 1594-1604

DOI: 10.1007/s00330-025-11924-3 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift European Radiology, online erschienen am 22.08.2025. Verfasst von Takeshi Nakaura und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Diagnostic Radiology, Graduate School of Medical Sciences, Kumamoto University, Kumamoto, Japan. kff00712@nifty.com. Hauptthemen laut MEDLINE: Large Language Models, Magnetic Resonance Imaging, Image Processing, Computer-Assisted, Brain Neoplasms. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Aimed to evaluate the potential of large language models (LLMs) in differentiating intra-axial primary brain tumors using structured magnetic resonance imaging (MRI) reports and compare their performance with radiologists. Materials and methods Structured reports of preoperative MRI findings from 137 surgically confirmed intra-axial primary brain tumors, including Glioblastoma ( n = 77), Central Nervous System (CNS) Lymphoma ( n = 22), Astrocytoma ( n = 9), Oligodendroglioma ( n = 9), and others ( n = 20), were analyzed by multiple LLMs, including GPT-4, Claude-3-Opus, Claude-3-Sonnet, GPT-3.5, Llama-2-70B, Qwen1.5-72B, and Gemini-Pro-1.0. The models provided the top 5 differential diagnoses based on the preoperative MRI findings, and their top 1, 3, and 5 accuracies were compared with board-certified neuroradiologists’ interpretations of the actual preoperative MRI images.

Results

Radiologists achieved top 1, 3, and 5 accuracies of 85.4%, 94.9%, and 94.9%, respectively. Among the LLMs, GPT-4 performed best with top 1, 3, and 5 accuracies of 65.7%, 84.7%, and 90.5%, respectively. Notably, GPT-4’s top 3 accuracy of 84.7% approached the radiologists’ top 1 accuracy of 85.4%. Other LLMs showed varying performance levels, with average accuracies ranging from 62.3% to 75.9%. LLMs demonstrated high accuracy for Glioblastoma but struggled with CNS Lymphoma and other less common tumors, particularly in top 1 accuracy.

Conclusion

LLMs show promise as assistive tools for differentiating intra-axial primary brain tumors using structured MRI reports. However, a significant gap remains between their performance and that of board-certified neuroradiologists interpreting actual images. The choice of LLM and tumor type significantly influences the results. Key Points Question How do Large Language Models (LLM) perform when differentiating complex intra-axial primary brain tumors from structured MRI reports compared to radiologists interpreting images?

Findings

Radiologists outperformed all tested LLMs in diagnostic accuracy. The best model, GPT-4, showed promise but lagged considerably behind radiologists, particularly for less common tumors . Clinical relevance LLMs show potential as assistive tools for generating differential diagnoses from structured MRI reports, particularly for non-specialists, but they cannot currently replace the nuanced diagnostic expertise of a board-certified radiologist interpreting the primary image data . Graphical Abstract

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsMagnetic Resonance ImagingImage Processing, Computer-AssistedBrain NeoplasmsRetrospective StudiesHumansMaleFemaleAdultMiddle AgedAgedRadiologistsDiagnosis, Differential

Stichwörter der Autor:innen

Artificial intelligenceBrain tumorsLarge language modelsMagnetic resonance imagingStructured reports

Autor:innen und Institutionen

  • Department of Diagnostic Radiology, Graduate School of Medical Sciences, Kumamoto University, Kumamoto, Japan. kff00712@nifty.com.
  • Department of Diagnostic Radiology, Graduate School of Medical Sciences, Kumamoto University, Kumamoto, Japan.
  • Department of Neurosurgery, Graduate School of Medical Sciences, Kumamoto University, Kumamoto, Japan.

Zitieren

APA

Nakaura, T., Uetani, H., Yoshida, N., Kobayashi, N., Nagayama, Y., Kidoh, M., Kuroda, J. I., Mukasa, A., & Hirai, T. (2025). Intra-axial primary brain tumor differentiation: comparing large language models on structured MRI reports vs. radiologists on images. European Radiology, 36(2), 1594-1604. https://doi.org/10.1007/s00330-025-11924-3

Vancouver

Nakaura T, Uetani H, Yoshida N, Kobayashi N, Nagayama Y, Kidoh M, et al. Intra-axial primary brain tumor differentiation: comparing large language models on structured MRI reports vs. radiologists on images. European Radiology. 2025;36(2):1594-1604. doi:10.1007/s00330-025-11924-3

BibTeX
@article{Nakaura2025,
  author = {Nakaura, Takeshi and Uetani, Hiroyuki and Yoshida, Naofumi and Kobayashi, Naoki and Nagayama, Yasunori and Kidoh, Masafumi and Kuroda, Jun-Ichiro and Mukasa, Akitake and Hirai, Toshinori},
  title = {Intra-axial primary brain tumor differentiation: comparing large language models on structured MRI reports vs. radiologists on images},
  journal = {European Radiology},
  year = {2025},
  volume = {36},
  number = {2},
  pages = {1594--1604},
  doi = {10.1007/s00330-025-11924-3}
}
Kostenlos starten →