EUVIMEDEUVIMED entdecken
Open Access

Performance of DeepSeek and GPT Models on Pediatric Board Preparation Questions: Comparative Evaluation

Masab Mansoor, Andrew Ibrahim, Ali Hamide

JMIR AI · 27.08.2025 · 4 · e76056-e76056

DOI: 10.2196/76056 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR AI, online erschienen am 27.08.2025. Verfasst von Masab Mansoor und 2 weiteren Autor:innen. Beteiligte Einrichtung: Louisiana Campus, Edward Via College of Osteopathic Medicine, 4408 Bon Aire Dr, Monroe, LA, 71203, United States, 1 5045213500. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 13 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Limited research exists evaluating artificial intelligence (AI) performance on standardized pediatric assessments. This study evaluated 3 leading AI models on pediatric board preparation questions.

Objective

The aim of this study is to evaluate and compare the performance of 3 leading large language models (LLMs) on pediatric board examination preparation questions and contextualize their performance against human physician benchmarks.

Methods

We analyzed DeepSeek-R1, ChatGPT-4, and ChatGPT-4.5 using 266 multiple-choice questions from the 2023 PREP Self-Assessment. Performance was compared to published American Board of Pediatrics first-time pass rates.

Results

DeepSeek-R1 exhibited the highest accuracy at 98.1% (261/266 correct responses). ChatGPT-4.5 achieved 96.6% accuracy (257/266), performing at the upper threshold of human performance. ChatGPT-4 demonstrated 82.7% accuracy (220/266), comparable to the lower range of human pass rates. Error pattern analysis revealed that AI models most commonly struggled with questions requiring integration of complex clinical presentations with rare disease knowledge.

Conclusions

DeepSeek-R1 demonstrated exceptional performance exceeding typical American Board of Pediatrics pass rates, suggesting potential applications in medical education and clinical support, though further research on complex clinical reasoning is needed.

Schlagwörter

Stichwörter der Autor:innen

ChatGPTDeepSeekartificial intelligenceboard examinationlarge language modelsmedical educationpediatrics

Autor:innen und Institutionen

  • Louisiana Campus, Edward Via College of Osteopathic Medicine, 4408 Bon Aire Dr, Monroe, LA, 71203, United States, 1 5045213500.
  • School of Medicine, Texas Tech University Health Sciences Center, Lubbock, TX, United States.

Zitieren

APA

Mansoor, M., Ibrahim, A., & Hamide, A. (2025). Performance of DeepSeek and GPT Models on Pediatric Board Preparation Questions: Comparative Evaluation. JMIR AI, 4, e76056-e76056. https://doi.org/10.2196/76056

Vancouver

Mansoor M, Ibrahim A, Hamide A. Performance of DeepSeek and GPT Models on Pediatric Board Preparation Questions: Comparative Evaluation. JMIR AI. 2025;4:e76056-e76056. doi:10.2196/76056

BibTeX
@article{Mansoor2025,
  author = {Mansoor, Masab and Ibrahim, Andrew and Hamide, Ali},
  title = {Performance of DeepSeek and GPT Models on Pediatric Board Preparation Questions: Comparative Evaluation},
  journal = {JMIR AI},
  year = {2025},
  volume = {4},
  pages = {e76056--e76056},
  doi = {10.2196/76056}
}
Kostenlos starten →