Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study
JMIR Medical Education · 14.11.2025 · 11 · e73469
DOI: 10.2196/73469 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift JMIR Medical Education, online erschienen am 14.11.2025. Verfasst von Weiping Wang und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing,…. Hauptthemen laut MEDLINE: Licensure, Medical, Educational Measurement, Language. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Deepseek-R1, an open-source large language model (LLM), has generated significant global interest in the past months.
Objective
This study aimed to compare the performance of DeepSeek and OpenAI LLMs on the Chinese National Medical Licensing Examination (NMLE) and evaluate their potential in medical education.
Methods
This cross-sectional study assessed 2 DeepSeek models (DeepSeek-R1 and DeepSeek-V3), 3 OpenAI models (ChatGPT-o1 pro, ChatGPT-o3 mini, and GPT-4o), and 2 additional Chinese LLMs (ERNIE 4.5 Turbo and Qwen 3) using the 2021 NMLE. Model performance was evaluated based on overall accuracy, accuracy across question types (A1, A2, A3 and A4, and B1), case analysis and non–case analysis questions, medical specialties, and accuracy consensus between different model combinations.
Results
All LLMs successfully passed the NMLE. DeepSeek-R1 achieved the highest accuracy (573/597, 96%), followed by DeepSeek-V3 (558/600, 93%), both of which significantly outperformed ChatGPT-o1 pro (450/600, 75%), ChatGPT-o3 mini (455/600, 75.8%), and GPT-4o (452/600, 75.3%; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
- Tsinghua Medicine, School of Medicine, Tsinghua University, Beijing, China.
- Department of clinical laboratory, Xuanwu Hospital, Capital Medical University, Beijing, China.
ORCID-Profile: Weiping Wang, Yuchen Zhou, Jingxuan Fu, Ke Hu
Zitieren
APA
Wang, W., Zhou, Y., Fu, J., & Hu, K. (2025). Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study. JMIR Medical Education, 11, e73469. https://doi.org/10.2196/73469
Vancouver
Wang W, Zhou Y, Fu J, Hu K. Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study. JMIR Medical Education. 2025;11:e73469. doi:10.2196/73469
BibTeX
@article{Wang2025,
author = {Wang, Weiping and Zhou, Yuchen and Fu, Jingxuan and Hu, Ke},
title = {Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study},
journal = {JMIR Medical Education},
year = {2025},
volume = {11},
pages = {e73469},
doi = {10.2196/73469}
}
EUVIMED