EUVIMEDObjevte EUVIMED
Open access

Performance of DeepSeek and GPT Models on Pediatric Board Preparation Questions: Comparative Evaluation

Masab Mansoor, Andrew Ibrahim, Ali Hamide

JMIR AI · 2025-08-27 · 4 · e76056-e76056

DOI: 10.2196/76056 ↗

At a glance

Journal article in JMIR AI, published online on 2025-08-27. Written by Masab Mansoor and 2 co-authors. Institution: Louisiana Campus, Edward Via College of Osteopathic Medicine, 4408 Bon Aire Dr, Monroe, LA, 71203, United States, 1 5045213500. Cited 9 times so far (Crossref); 13 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Limited research exists evaluating artificial intelligence (AI) performance on standardized pediatric assessments. This study evaluated 3 leading AI models on pediatric board preparation questions.

Objective

The aim of this study is to evaluate and compare the performance of 3 leading large language models (LLMs) on pediatric board examination preparation questions and contextualize their performance against human physician benchmarks.

Methods

We analyzed DeepSeek-R1, ChatGPT-4, and ChatGPT-4.5 using 266 multiple-choice questions from the 2023 PREP Self-Assessment. Performance was compared to published American Board of Pediatrics first-time pass rates.

Results

DeepSeek-R1 exhibited the highest accuracy at 98.1% (261/266 correct responses). ChatGPT-4.5 achieved 96.6% accuracy (257/266), performing at the upper threshold of human performance. ChatGPT-4 demonstrated 82.7% accuracy (220/266), comparable to the lower range of human pass rates. Error pattern analysis revealed that AI models most commonly struggled with questions requiring integration of complex clinical presentations with rare disease knowledge.

Conclusions

DeepSeek-R1 demonstrated exceptional performance exceeding typical American Board of Pediatrics pass rates, suggesting potential applications in medical education and clinical support, though further research on complex clinical reasoning is needed.

Subjects

Author keywords

ChatGPTDeepSeekartificial intelligenceboard examinationlarge language modelsmedical educationpediatrics

Authors and institutions

  • Louisiana Campus, Edward Via College of Osteopathic Medicine, 4408 Bon Aire Dr, Monroe, LA, 71203, United States, 1 5045213500.
  • School of Medicine, Texas Tech University Health Sciences Center, Lubbock, TX, United States.

Cite

APA

Mansoor, M., Ibrahim, A., & Hamide, A. (2025). Performance of DeepSeek and GPT Models on Pediatric Board Preparation Questions: Comparative Evaluation. JMIR AI, 4, e76056-e76056. https://doi.org/10.2196/76056

Vancouver

Mansoor M, Ibrahim A, Hamide A. Performance of DeepSeek and GPT Models on Pediatric Board Preparation Questions: Comparative Evaluation. JMIR AI. 2025;4:e76056-e76056. doi:10.2196/76056

BibTeX
@article{Mansoor2025,
  author = {Mansoor, Masab and Ibrahim, Andrew and Hamide, Ali},
  title = {Performance of DeepSeek and GPT Models on Pediatric Board Preparation Questions: Comparative Evaluation},
  journal = {JMIR AI},
  year = {2025},
  volume = {4},
  pages = {e76056--e76056},
  doi = {10.2196/76056}
}
Start for free →