EUVIMEDEUVIMED entdecken
Systematische ÜbersichtMetaanalyseOpen Access

Performance of large language models in medical licensing examinations: a systematic review and meta-analysis

Haniyeh Nouri, Abdollah Mahdavi, Ali Abedi, Alireza Mohammadnia, Mahnaz Hamedan, Masoud Amanzadeh

Journal of Educational Evaluation for Health Professions · 18.11.2025 · 22 · 36

DOI: 10.3352/jeehp.2025.22.36 ↗

Kurz gefasst

Systematische Übersicht in der Zeitschrift Journal of Educational Evaluation for Health Professions, online erschienen am 18.11.2025. Verfasst von Haniyeh Nouri und 5 weiteren Autor:innen. Beteiligte Einrichtung: Student Research Committee, School of Medicine, Ardabil University of Medical Sciences, Ardabil, Iran. Hauptthemen laut MEDLINE: Educational Measurement, Licensure, Medical, Language, Education, Medical. Die Arbeit wurde bisher 15-mal zitiert (Crossref). Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

This study systematically evaluates and compares the performance of large language models (LLMs) in answering medical licensing examination questions. By conducting subgroup analyses based on language, question format, and model type, this meta-analysis aims to provide a comprehensive overview of LLM capabilities in medical education and clinical decision-making.

Methods

This systematic review, registered in PROSPERO and following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, searched MEDLINE (PubMed), Scopus, and Web of Science for relevant articles published up to February 1, 2025. The search strategy included Medical Subject Headings (MeSH) terms and keywords related to (“ChatGPT” OR “GPT” OR “LLM variants”) AND (“medical licensing exam*” OR “medical exam*” OR “medical education” OR “radiology exam*”). Eligible studies evaluated LLM accuracy on medical licensing examination questions. Pooled accuracy was estimated using a random-effects model, with subgroup analyses by LLM type, language, and question format. Publication bias was assessed using Egger’s regression test.

Results

This systematic review identified 2,404 studies. After removing duplicates and excluding irrelevant articles through title and abstract screening, 36 studies were included after full-text review. The pooled accuracy was 72% (95% confidence interval, 70.0% to 75.0%) with high heterogeneity (I2=99%, P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Educational MeasurementLicensure, MedicalLanguageEducation, MedicalHumansClinical CompetenceLarge Language Models

Stichwörter der Autor:innen

Artificial intelligenceLarge language modelsMedical educationMedical examination

Autor:innen und Institutionen

  • Student Research Committee, School of Medicine, Ardabil University of Medical Sciences, Ardabil, Iran.
  • Department of Health Information Management, School of Medicine, Ardabil University of Medical Sciences, Ardabil, Iran.
  • Department of Physiology, School of Medicine, Ardabil University of Medical Sciences, Ardabil, Iran.

Zitieren

APA

Nouri, H., Mahdavi, A., Abedi, A., Mohammadnia, A., Hamedan, M., & Amanzadeh, M. (2025). Performance of large language models in medical licensing examinations: a systematic review and meta-analysis. Journal of Educational Evaluation for Health Professions, 22, 36. https://doi.org/10.3352/jeehp.2025.22.36

Vancouver

Nouri H, Mahdavi A, Abedi A, Mohammadnia A, Hamedan M, Amanzadeh M. Performance of large language models in medical licensing examinations: a systematic review and meta-analysis. Journal of Educational Evaluation for Health Professions. 2025;22:36. doi:10.3352/jeehp.2025.22.36

BibTeX
@article{Nouri2025,
  author = {Nouri, Haniyeh and Mahdavi, Abdollah and Abedi, Ali and Mohammadnia, Alireza and Hamedan, Mahnaz and Amanzadeh, Masoud},
  title = {Performance of large language models in medical licensing examinations: a systematic review and meta-analysis},
  journal = {Journal of Educational Evaluation for Health Professions},
  year = {2025},
  volume = {22},
  pages = {36},
  doi = {10.3352/jeehp.2025.22.36}
}
Kostenlos starten →