EUVIMEDEUVIMED entdecken
Crossref

Artificial Intelligence Scores Differ from Candidate Scores on Board Exams: A Critical Appraisal

Jamie Mansell, Anne Russ, Ryan Tierney, Reece Hayes

CommonHealth · 2026

DOI: 10.15367/8s8ky475 ↗

Kurz gefasst

Fachartikel in der Zeitschrift CommonHealth, 2026. Verfasst von Jamie Mansell und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Context

Recent research has investigated Artificial Intelligence’s (AI) ability to answer items such as board-style medical exam questions. On board style exam questions, do scores differ between AI and humans?

Methods

PubMed was searched in May and June of 2024 using the Boolean phrase: Medical examination OR medical education AND artificial intelligence. Inclusion criteria comprised of English-language studies no older than 3 years, evaluating standardized/validated board-style exam questions and compared AI to human examinees. Studies were excluded if they evaluated open-ended or case-study style questions, prompted AI or failed to prevent learning effects of AI, or were systematic reviews or meta-analyses. Studies were evaluated using the STROBE checklist. Outcome measures included scores on board-style exam questions.

Results

An original search revealed 4,973 studies. 61 studies met title criteria and were screened more thoroughly. Five studies met all criteria for inclusion and were analyzed. In a study of 11,733 medical residents, ChatGPT-4 outperformed the examinees 70.1% to 55.8%. In another study, ChatGPT-4 outperformed examinees on a pre-internship examination 68.5% to 57%. On a national licensing medical examination, 1,025 first-year medical students were compared to ChatGPT-3.5 and version 4. Both versions of ChatGPT outscored humans 77% (v3.5) and 86%(v4) compared to 55%. On a licensing examination, human examinees were compared to ChatGPT-4 and version 3.5, ChatGPT-4 outperformed humans, v3.5 did not, 85%(v4) and 58% (v3.5) compared to 76%. 736 urologists were compared to ChatGPT-3.5 and version 4, and Google’s Bard. ChatGPT-4 outperformed the examinees, Bard performed at examinee-level, ChatGPT-3.5 performed below-level of humans. STROBE scores were 20, 19, 20, 21, and 15, respectively.

Conclusions

AI surpassed human examinees on medical exams. While AI exceeded humans on these exams, it did not score 100%. Caution should be applied when considering using AI as a tool for study sessions or individualized learning plans. SOR = A.

Zitieren

APA

Mansell, J., Russ, A., Tierney, R., & Hayes, R. (2026). Artificial Intelligence Scores Differ from Candidate Scores on Board Exams: A Critical Appraisal. CommonHealth. https://doi.org/10.15367/8s8ky475

Vancouver

Mansell J, Russ A, Tierney R, Hayes R. Artificial Intelligence Scores Differ from Candidate Scores on Board Exams: A Critical Appraisal. CommonHealth. 2026. doi:10.15367/8s8ky475

BibTeX
@article{Mansell2026,
  author = {Mansell, Jamie and Russ, Anne and Tierney, Ryan and Hayes, Reece},
  title = {Artificial Intelligence Scores Differ from Candidate Scores on Board Exams: A Critical Appraisal},
  journal = {CommonHealth},
  year = {2026},
  doi = {10.15367/8s8ky475}
}
Kostenlos starten →