Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022)
PLOS One · 19.11.2026 · 21 (11) · e0361855
DOI: 10.1371/journal.pone.0361855 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, online erschienen am 19.11.2026. Verfasst von Qian-Qian Liao und 7 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
DeepSeek, a cutting-edge artificial intelligence (AI) platform developed in China, has attracted growing interest for potential applications in clinical medicine and medical education. DeepSeek-V3 (MoE architecture) and DeepSeek-R1 (reinforcement learning-optimized for reasoning) are its flagship models, but the performance difference between DeepSeek‑R1 and DeepSeek‑V3 remains inconclusive. The objective of this study was to compare the performance of DeepSeek-R1 and DeepSeek-V3 on the Chinese National Medical Licensing Examination (NMLE).
Methods
This study employed DeepSeek-R1-0528 (released on May 28, 2025) and DeepSeek-V3-0324 (released on March 24, 2025) to perform the NMLE testing, and the subject area for the examination was Clinical Medicine. We collected the questions from NMLE 2004 through NMLE 2022, comprising 600 multiple-choice questions (MCQs) per annual examination, for a total of 11,400 MCQs. The NMLE tests were carried out using the free web-based version of DeepSeek, with DeepSeek-R1-0528 and DeepSeek-V3-0324 evaluated on separate computer systems. Model performance was assessed based on accuracy rates.
Results
The annual average accuracy rates for DeepSeek-R1-0528 and DeepSeek-V3-0324 were 91.2% (95% CI: 89.6% – 92.8%) and 91.1% (95% CI: 89.6% – 92.6%), respectively, with no significant difference observed between them ( P = 0.591, Cohen’s d = 0.086). The regression analysis showed that DeepSeek-R1-0528 showed no significant changes in accuracy rates from NMLE 2004 to NMLE 2013, while DeepSeek-V3-0324 showed a significant declining trend. Moreover, from NMLE 2014 to NMLE 2022, both models demonstrated significant declining trends in the accuracy rates. DeepSeek-R1-0528 and DeepSeek-V3-0324 answered 999 and 1,014 MCQs incorrectly, respectively, with 715 MCQs answered incorrectly by both models.
Conclusions
DeepSeek-R1-0528 and DeepSeek-V3-0324 demonstrated near-identical and excellent performance on the Chinese NMLE.
Zitieren
APA
Liao, Q., Tang, Y., Ge, R., Zhu, K., Xie, Z., Lin, Y., Li, Y., & Li, Z. (2026). Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022). PLOS One, 21(11), e0361855. https://doi.org/10.1371/journal.pone.0361855
Vancouver
Liao Q, Tang Y, Ge R, Zhu K, Xie Z, Lin Y, et al. Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022). PLOS One. 2026;21(11):e0361855. doi:10.1371/journal.pone.0361855
BibTeX
@article{Liao2026,
author = {Liao, Qian-Qian and Tang, Yan-Feng and Ge, Run and Zhu, Ke-Wei and Xie, Zhou-Ni and Lin, Yu-Jiao and Li, Yang-Chun and Li, Zhuo-Jun},
title = {Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022)},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {11},
pages = {e0361855},
doi = {10.1371/journal.pone.0361855}
}
EUVIMED