EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance of ChatGPT-o1 and DeepSeek-R1 on the health law related questions in the Chinese national licensing examination: a comparative study

Jihui Liu, Qingtian Zhu, Yadong Fan, Chenchen Yuan, Xiaowu Dong, Chengpeng Li, Weiwei Chen, Guotao Lu

Frontiers in Education · 05.03.2026 · 11

DOI: 10.3389/feduc.2026.1749606 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Education, online erschienen am 05.03.2026. Verfasst von Jihui Liu und 7 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

This study aimed to compare the performance of two advanced large language models (LLMs), DeepSeek-R1 and ChatGPT-o1, in addressing health law–related questions from the Chinese National Medical Licensing Examination (CNMLE), thereby evaluating their applicability in medical education in non-English contexts.

Methods

A total of 400 health law questions were randomly selected from the official CNMLE guidebook. Each question was independently administered to DeepSeek-R1 and ChatGPT-o1 via standardized Application Programming Interfaces (API) prompts to minimize hallucination and memory effects. Model responses were compared against the official answers, and statistical analyses were conducted using McNemar’s test, with p < 0.05 indicating significance.

Results

DeepSeek-R1 achieved an overall accuracy of 93.5% (374/400), significantly higher than ChatGPT-o1’s 79.5% (318/400, p < 0.001). Subgroup analysis revealed that DeepSeek-R1 consistently outperformed ChatGPT-o1 across most legal categories, including medical institutions, infectious disease prevention, malpractice liability, and pharmaceutical regulation. Both models performed comparably in categories such as blood donation and maternal–child health law. DeepSeek-R1 also achieved perfect accuracy in smaller domains such as public health emergencies and occupational disease control.

Conclusion

DeepSeek-R1 demonstrated superior performance compared with ChatGPT-o1 in answering health law questions within the CNMLE, highlighting its potential as a reliable tool for medical education in China. The findings underscore the influence of linguistic and cultural context on LLM performance. Future work should expand evaluation to open-ended and case-based questions and explore fine-tuning strategies to enhance the accuracy in healthcare settings.

Zitieren

APA

Liu, J., Zhu, Q., Fan, Y., Yuan, C., Dong, X., Li, C., Chen, W., & Lu, G. (2026). Performance of ChatGPT-o1 and DeepSeek-R1 on the health law related questions in the Chinese national licensing examination: a comparative study. Frontiers in Education, 11. https://doi.org/10.3389/feduc.2026.1749606

Vancouver

Liu J, Zhu Q, Fan Y, Yuan C, Dong X, Li C, et al. Performance of ChatGPT-o1 and DeepSeek-R1 on the health law related questions in the Chinese national licensing examination: a comparative study. Frontiers in Education. 2026;11. doi:10.3389/feduc.2026.1749606

BibTeX
@article{Liu2026,
  author = {Liu, Jihui and Zhu, Qingtian and Fan, Yadong and Yuan, Chenchen and Dong, Xiaowu and Li, Chengpeng and Chen, Weiwei and Lu, Guotao},
  title = {Performance of ChatGPT-o1 and DeepSeek-R1 on the health law related questions in the Chinese national licensing examination: a comparative study},
  journal = {Frontiers in Education},
  year = {2026},
  volume = {11},
  doi = {10.3389/feduc.2026.1749606}
}
Kostenlos starten →