Performance Evaluation and Implications of Large Language Models in Radiology Board Exams: Prospective Comparative Analysis
JMIR Medical Education · 2025
DOI: 10.2196/64284 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift JMIR Medical Education, 2025. Verfasst von Boxiong Wei. Beteiligte Einrichtung: Department of Ultrasound, Peking University First Hospital, 8 Xishiku Rd, Xicheng District, Beijing, 100034, China, 86 13132150190, 86 314521. Hauptthemen laut MEDLINE: Radiology, Educational Measurement, Artificial Intelligence. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Artificial intelligence advancements have enabled large language models to significantly impact radiology education and diagnostic accuracy.
Objective
This study evaluates the performance of mainstream large language models, including GPT-4, Claude, Bard, Tongyi Qianwen, and Gemini Pro, in radiology board exams.
Methods
A comparative analysis of 150 multiple-choice questions from radiology board exams without images was conducted. Models were assessed on their accuracy for text-based questions and were categorized by cognitive levels and medical specialties using χ 2 tests and ANOVA.
Results
GPT-4 achieved the highest accuracy (83.3%, 125/150), significantly outperforming all other models. Specifically, Claude achieved an accuracy of 62% (93/150; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Ultrasound, Peking University First Hospital, 8 Xishiku Rd, Xicheng District, Beijing, 100034, China, 86 13132150190, 86 314521.
ORCID-Profile: Boxiong Wei
Zitieren
APA
Wei, B. (2025). Performance Evaluation and Implications of Large Language Models in Radiology Board Exams: Prospective Comparative Analysis. JMIR Medical Education. https://doi.org/10.2196/64284
Vancouver
Wei B. Performance Evaluation and Implications of Large Language Models in Radiology Board Exams: Prospective Comparative Analysis. JMIR Medical Education. 2025. doi:10.2196/64284
BibTeX
@article{Wei2025,
author = {Wei, Boxiong},
title = {Performance Evaluation and Implications of Large Language Models in Radiology Board Exams: Prospective Comparative Analysis},
journal = {JMIR Medical Education},
year = {2025},
doi = {10.2196/64284}
}
EUVIMED