EUVIMEDEUVIMED entdecken
VergleichsstudieQuerschnittstudieOpen Access

Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study

Weiping Wang iD, Yuchen Zhou iD, Jingxuan Fu iD, Ke Hu iD

JMIR Medical Education · 14.11.2025 · 11 · e73469

DOI: 10.2196/73469 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Medical Education, online erschienen am 14.11.2025. Verfasst von Weiping Wang und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing,…. Hauptthemen laut MEDLINE: Licensure, Medical, Educational Measurement, Language. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Deepseek-R1, an open-source large language model (LLM), has generated significant global interest in the past months.

Objective

This study aimed to compare the performance of DeepSeek and OpenAI LLMs on the Chinese National Medical Licensing Examination (NMLE) and evaluate their potential in medical education.

Methods

This cross-sectional study assessed 2 DeepSeek models (DeepSeek-R1 and DeepSeek-V3), 3 OpenAI models (ChatGPT-o1 pro, ChatGPT-o3 mini, and GPT-4o), and 2 additional Chinese LLMs (ERNIE 4.5 Turbo and Qwen 3) using the 2021 NMLE. Model performance was evaluated based on overall accuracy, accuracy across question types (A1, A2, A3 and A4, and B1), case analysis and non–case analysis questions, medical specialties, and accuracy consensus between different model combinations.

Results

All LLMs successfully passed the NMLE. DeepSeek-R1 achieved the highest accuracy (573/597, 96%), followed by DeepSeek-V3 (558/600, 93%), both of which significantly outperformed ChatGPT-o1 pro (450/600, 75%), ChatGPT-o3 mini (455/600, 75.8%), and GPT-4o (452/600, 75.3%; P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Licensure, MedicalEducational MeasurementLanguageCross-Sectional StudiesHumansChina

Stichwörter der Autor:innen

AIChinese Medical Licensing ExaminationDeepSeekOpenAIartificial intelligencelarge language models

Autor:innen und Institutionen

  • Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
  • Tsinghua Medicine, School of Medicine, Tsinghua University, Beijing, China.
  • Department of clinical laboratory, Xuanwu Hospital, Capital Medical University, Beijing, China.

ORCID-Profile: Weiping Wang, Yuchen Zhou, Jingxuan Fu, Ke Hu

Zitieren

APA

Wang, W., Zhou, Y., Fu, J., & Hu, K. (2025). Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study. JMIR Medical Education, 11, e73469. https://doi.org/10.2196/73469

Vancouver

Wang W, Zhou Y, Fu J, Hu K. Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study. JMIR Medical Education. 2025;11:e73469. doi:10.2196/73469

BibTeX
@article{Wang2025,
  author = {Wang, Weiping and Zhou, Yuchen and Fu, Jingxuan and Hu, Ke},
  title = {Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study},
  journal = {JMIR Medical Education},
  year = {2025},
  volume = {11},
  pages = {e73469},
  doi = {10.2196/73469}
}
Kostenlos starten →