Artificial Intelligence Scores Differ from Candidate Scores on Board Exams: A Critical Appraisal
CommonHealth · 2026
DOI: 10.15367/8s8ky475 ↗Kurz gefasst
Fachartikel in der Zeitschrift CommonHealth, 2026. Verfasst von Jamie Mansell und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Context
Recent research has investigated Artificial Intelligence’s (AI) ability to answer items such as board-style medical exam questions. On board style exam questions, do scores differ between AI and humans?
Methods
PubMed was searched in May and June of 2024 using the Boolean phrase: Medical examination OR medical education AND artificial intelligence. Inclusion criteria comprised of English-language studies no older than 3 years, evaluating standardized/validated board-style exam questions and compared AI to human examinees. Studies were excluded if they evaluated open-ended or case-study style questions, prompted AI or failed to prevent learning effects of AI, or were systematic reviews or meta-analyses. Studies were evaluated using the STROBE checklist. Outcome measures included scores on board-style exam questions.
Results
An original search revealed 4,973 studies. 61 studies met title criteria and were screened more thoroughly. Five studies met all criteria for inclusion and were analyzed. In a study of 11,733 medical residents, ChatGPT-4 outperformed the examinees 70.1% to 55.8%. In another study, ChatGPT-4 outperformed examinees on a pre-internship examination 68.5% to 57%. On a national licensing medical examination, 1,025 first-year medical students were compared to ChatGPT-3.5 and version 4. Both versions of ChatGPT outscored humans 77% (v3.5) and 86%(v4) compared to 55%. On a licensing examination, human examinees were compared to ChatGPT-4 and version 3.5, ChatGPT-4 outperformed humans, v3.5 did not, 85%(v4) and 58% (v3.5) compared to 76%. 736 urologists were compared to ChatGPT-3.5 and version 4, and Google’s Bard. ChatGPT-4 outperformed the examinees, Bard performed at examinee-level, ChatGPT-3.5 performed below-level of humans. STROBE scores were 20, 19, 20, 21, and 15, respectively.
Conclusions
AI surpassed human examinees on medical exams. While AI exceeded humans on these exams, it did not score 100%. Caution should be applied when considering using AI as a tool for study sessions or individualized learning plans. SOR = A.
Zitieren
APA
Mansell, J., Russ, A., Tierney, R., & Hayes, R. (2026). Artificial Intelligence Scores Differ from Candidate Scores on Board Exams: A Critical Appraisal. CommonHealth. https://doi.org/10.15367/8s8ky475
Vancouver
Mansell J, Russ A, Tierney R, Hayes R. Artificial Intelligence Scores Differ from Candidate Scores on Board Exams: A Critical Appraisal. CommonHealth. 2026. doi:10.15367/8s8ky475
BibTeX
@article{Mansell2026,
author = {Mansell, Jamie and Russ, Anne and Tierney, Ryan and Hayes, Reece},
title = {Artificial Intelligence Scores Differ from Candidate Scores on Board Exams: A Critical Appraisal},
journal = {CommonHealth},
year = {2026},
doi = {10.15367/8s8ky475}
}
EUVIMED