Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study
JMIR Medical Education · 2025-11-14 · 11 · e73469
DOI: 10.2196/73469 ↗At a glance
Vergleichsstudie in JMIR Medical Education, published online on 2025-11-14. Written by Weiping Wang and 3 co-authors. Institution: Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing,…. Major MEDLINE topics: Licensure, Medical, Educational Measurement, Language. Cited 22 times so far (Crossref); 22 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Deepseek-R1, an open-source large language model (LLM), has generated significant global interest in the past months.
Objective
This study aimed to compare the performance of DeepSeek and OpenAI LLMs on the Chinese National Medical Licensing Examination (NMLE) and evaluate their potential in medical education.
Methods
This cross-sectional study assessed 2 DeepSeek models (DeepSeek-R1 and DeepSeek-V3), 3 OpenAI models (ChatGPT-o1 pro, ChatGPT-o3 mini, and GPT-4o), and 2 additional Chinese LLMs (ERNIE 4.5 Turbo and Qwen 3) using the 2021 NMLE. Model performance was evaluated based on overall accuracy, accuracy across question types (A1, A2, A3 and A4, and B1), case analysis and non–case analysis questions, medical specialties, and accuracy consensus between different model combinations.
Results
All LLMs successfully passed the NMLE. DeepSeek-R1 achieved the highest accuracy (573/597, 96%), followed by DeepSeek-V3 (558/600, 93%), both of which significantly outperformed ChatGPT-o1 pro (450/600, 75%), ChatGPT-o3 mini (455/600, 75.8%), and GPT-4o (452/600, 75.3%; P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
- Tsinghua Medicine, School of Medicine, Tsinghua University, Beijing, China.
- Department of clinical laboratory, Xuanwu Hospital, Capital Medical University, Beijing, China.
ORCID profiles: Weiping Wang, Yuchen Zhou, Jingxuan Fu, Ke Hu
Cite
APA
Wang, W., Zhou, Y., Fu, J., & Hu, K. (2025). Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study. JMIR Medical Education, 11, e73469. https://doi.org/10.2196/73469
Vancouver
Wang W, Zhou Y, Fu J, Hu K. Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study. JMIR Medical Education. 2025;11:e73469. doi:10.2196/73469
BibTeX
@article{Wang2025,
author = {Wang, Weiping and Zhou, Yuchen and Fu, Jingxuan and Hu, Ke},
title = {Evaluating the Performance of DeepSeek-R1 and DeepSeek-V3 Versus OpenAI Models in the Chinese National Medical Licensing Examination: Cross-Sectional Comparative Study},
journal = {JMIR Medical Education},
year = {2025},
volume = {11},
pages = {e73469},
doi = {10.2196/73469}
}
EUVIMED