EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022)

Qian-Qian Liao, Yan-Feng Tang, Run Ge, Ke-Wei Zhu, Zhou-Ni Xie, Yu-Jiao Lin, Yang-Chun Li, Zhuo-Jun Li

PLOS One · 19.11.2026 · 21 (11) · e0361855

DOI: 10.1371/journal.pone.0361855 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 19.11.2026. Verfasst von Qian-Qian Liao und 7 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

DeepSeek, a cutting-edge artificial intelligence (AI) platform developed in China, has attracted growing interest for potential applications in clinical medicine and medical education. DeepSeek-V3 (MoE architecture) and DeepSeek-R1 (reinforcement learning-optimized for reasoning) are its flagship models, but the performance difference between DeepSeek‑R1 and DeepSeek‑V3 remains inconclusive. The objective of this study was to compare the performance of DeepSeek-R1 and DeepSeek-V3 on the Chinese National Medical Licensing Examination (NMLE).

Methods

This study employed DeepSeek-R1-0528 (released on May 28, 2025) and DeepSeek-V3-0324 (released on March 24, 2025) to perform the NMLE testing, and the subject area for the examination was Clinical Medicine. We collected the questions from NMLE 2004 through NMLE 2022, comprising 600 multiple-choice questions (MCQs) per annual examination, for a total of 11,400 MCQs. The NMLE tests were carried out using the free web-based version of DeepSeek, with DeepSeek-R1-0528 and DeepSeek-V3-0324 evaluated on separate computer systems. Model performance was assessed based on accuracy rates.

Results

The annual average accuracy rates for DeepSeek-R1-0528 and DeepSeek-V3-0324 were 91.2% (95% CI: 89.6% – 92.8%) and 91.1% (95% CI: 89.6% – 92.6%), respectively, with no significant difference observed between them ( P = 0.591, Cohen’s d = 0.086). The regression analysis showed that DeepSeek-R1-0528 showed no significant changes in accuracy rates from NMLE 2004 to NMLE 2013, while DeepSeek-V3-0324 showed a significant declining trend. Moreover, from NMLE 2014 to NMLE 2022, both models demonstrated significant declining trends in the accuracy rates. DeepSeek-R1-0528 and DeepSeek-V3-0324 answered 999 and 1,014 MCQs incorrectly, respectively, with 715 MCQs answered incorrectly by both models.

Conclusions

DeepSeek-R1-0528 and DeepSeek-V3-0324 demonstrated near-identical and excellent performance on the Chinese NMLE.

Zitieren

APA

Liao, Q., Tang, Y., Ge, R., Zhu, K., Xie, Z., Lin, Y., Li, Y., & Li, Z. (2026). Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022). PLOS One, 21(11), e0361855. https://doi.org/10.1371/journal.pone.0361855

Vancouver

Liao Q, Tang Y, Ge R, Zhu K, Xie Z, Lin Y, et al. Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022). PLOS One. 2026;21(11):e0361855. doi:10.1371/journal.pone.0361855

BibTeX
@article{Liao2026,
  author = {Liao, Qian-Qian and Tang, Yan-Feng and Ge, Run and Zhu, Ke-Wei and Xie, Zhou-Ni and Lin, Yu-Jiao and Li, Yang-Chun and Li, Zhuo-Jun},
  title = {Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022)},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {11},
  pages = {e0361855},
  doi = {10.1371/journal.pone.0361855}
}
Kostenlos starten →