Performance of large language models in medical licensing examinations: a systematic review and meta-analysis
Journal of Educational Evaluation for Health Professions · 18.11.2025 · 22 · 36
DOI: 10.3352/jeehp.2025.22.36 ↗Kurz gefasst
Systematische Übersicht in der Zeitschrift Journal of Educational Evaluation for Health Professions, online erschienen am 18.11.2025. Verfasst von Haniyeh Nouri und 5 weiteren Autor:innen. Beteiligte Einrichtung: Student Research Committee, School of Medicine, Ardabil University of Medical Sciences, Ardabil, Iran. Hauptthemen laut MEDLINE: Educational Measurement, Licensure, Medical, Language, Education, Medical. Die Arbeit wurde bisher 15-mal zitiert (Crossref). Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Purpose
This study systematically evaluates and compares the performance of large language models (LLMs) in answering medical licensing examination questions. By conducting subgroup analyses based on language, question format, and model type, this meta-analysis aims to provide a comprehensive overview of LLM capabilities in medical education and clinical decision-making.
Methods
This systematic review, registered in PROSPERO and following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, searched MEDLINE (PubMed), Scopus, and Web of Science for relevant articles published up to February 1, 2025. The search strategy included Medical Subject Headings (MeSH) terms and keywords related to (“ChatGPT” OR “GPT” OR “LLM variants”) AND (“medical licensing exam*” OR “medical exam*” OR “medical education” OR “radiology exam*”). Eligible studies evaluated LLM accuracy on medical licensing examination questions. Pooled accuracy was estimated using a random-effects model, with subgroup analyses by LLM type, language, and question format. Publication bias was assessed using Egger’s regression test.
Results
This systematic review identified 2,404 studies. After removing duplicates and excluding irrelevant articles through title and abstract screening, 36 studies were included after full-text review. The pooled accuracy was 72% (95% confidence interval, 70.0% to 75.0%) with high heterogeneity (I2=99%, P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Student Research Committee, School of Medicine, Ardabil University of Medical Sciences, Ardabil, Iran.
- Department of Health Information Management, School of Medicine, Ardabil University of Medical Sciences, Ardabil, Iran.
- Department of Physiology, School of Medicine, Ardabil University of Medical Sciences, Ardabil, Iran.
Zitieren
APA
Nouri, H., Mahdavi, A., Abedi, A., Mohammadnia, A., Hamedan, M., & Amanzadeh, M. (2025). Performance of large language models in medical licensing examinations: a systematic review and meta-analysis. Journal of Educational Evaluation for Health Professions, 22, 36. https://doi.org/10.3352/jeehp.2025.22.36
Vancouver
Nouri H, Mahdavi A, Abedi A, Mohammadnia A, Hamedan M, Amanzadeh M. Performance of large language models in medical licensing examinations: a systematic review and meta-analysis. Journal of Educational Evaluation for Health Professions. 2025;22:36. doi:10.3352/jeehp.2025.22.36
BibTeX
@article{Nouri2025,
author = {Nouri, Haniyeh and Mahdavi, Abdollah and Abedi, Ali and Mohammadnia, Alireza and Hamedan, Mahnaz and Amanzadeh, Masoud},
title = {Performance of large language models in medical licensing examinations: a systematic review and meta-analysis},
journal = {Journal of Educational Evaluation for Health Professions},
year = {2025},
volume = {22},
pages = {36},
doi = {10.3352/jeehp.2025.22.36}
}
EUVIMED