EUVIMEDOpdag EUVIMED
VergleichsstudieOpen access

Performance Evaluation and Implications of Large Language Models in Radiology Board Exams: Prospective Comparative Analysis

Boxiong Wei iD

JMIR Medical Education · 2025

DOI: 10.2196/64284 ↗

At a glance

Vergleichsstudie in JMIR Medical Education, 2025. Written by Boxiong Wei. Institution: Department of Ultrasound, Peking University First Hospital, 8 Xishiku Rd, Xicheng District, Beijing, 100034, China, 86 13132150190, 86 314521. Major MEDLINE topics: Radiology, Educational Measurement, Artificial Intelligence. Cited 24 times so far (Crossref); 28 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Artificial intelligence advancements have enabled large language models to significantly impact radiology education and diagnostic accuracy.

Objective

This study evaluates the performance of mainstream large language models, including GPT-4, Claude, Bard, Tongyi Qianwen, and Gemini Pro, in radiology board exams.

Methods

A comparative analysis of 150 multiple-choice questions from radiology board exams without images was conducted. Models were assessed on their accuracy for text-based questions and were categorized by cognitive levels and medical specialties using χ 2 tests and ANOVA.

Results

GPT-4 achieved the highest accuracy (83.3%, 125/150), significantly outperforming all other models. Specifically, Claude achieved an accuracy of 62% (93/150; P

Subjects

MeSH terms (MEDLINE)

RadiologyEducational MeasurementArtificial IntelligenceHumansProspective StudiesClinical CompetenceLarge Language Models

Author keywords

AIGPT-4LLMartificial intelligencediagnosticslarge language modelsmedical educationmedical trainingradiologyradiology examsultrasound

Authors and institutions

  • Department of Ultrasound, Peking University First Hospital, 8 Xishiku Rd, Xicheng District, Beijing, 100034, China, 86 13132150190, 86 314521.

ORCID profiles: Boxiong Wei

Cite

APA

Wei, B. (2025). Performance Evaluation and Implications of Large Language Models in Radiology Board Exams: Prospective Comparative Analysis. JMIR Medical Education. https://doi.org/10.2196/64284

Vancouver

Wei B. Performance Evaluation and Implications of Large Language Models in Radiology Board Exams: Prospective Comparative Analysis. JMIR Medical Education. 2025. doi:10.2196/64284

BibTeX
@article{Wei2025,
  author = {Wei, Boxiong},
  title = {Performance Evaluation and Implications of Large Language Models in Radiology Board Exams: Prospective Comparative Analysis},
  journal = {JMIR Medical Education},
  year = {2025},
  doi = {10.2196/64284}
}
Start for free →