Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning
PLOS One · 08.05.2026 · 21 (5) · e0346078
DOI: 10.1371/journal.pone.0346078 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, online erschienen am 08.05.2026. Verfasst von Yang Wang und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Emergency Medicine, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China. Hauptthemen laut MEDLINE: Clinical Decision-Making, Clinical Reasoning. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 9 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Artificial intelligence (AI) is undergoing an era of transformative advancement, particularly through the emergence of Transformer-based large language models (LLMs). While these systems demonstrate strong reasoning and generalization capabilities, their clinical applicability, particularly in emergency and critical care decision-making, remains underexplored.. In time-sensitive settings, diagnostic reasoning must align rigorously with evidence-based standards and ensure the relevance of timing to clinical decisions.
Objective
This study aims to provide a preliminary evaluation of the decision-support performance of the DeepSeek model in acute medical scenarios. We systematically evaluate its diagnostic reasoning, temporal consistency of recommendations, and adherence to evidence-based critical care protocols using standardized case-based assessments.
Methods
Twenty-nine representative clinical cases were extracted from the Merck Manual of Diagnosis and Therapy, a widely used medical reference providing standardized case descriptions. The model’s outputs were evaluated across four decision-making dimensions: differential diagnosis, diagnostic testing, final diagnosis, and management planning. Human raters scored each response for accuracy, and multivariable linear regression was applied to assess associations between performance and case parameters (age, gender, and Rapid Emergency Medicine Score [REMS]).
Results
DeepSeek achieved an overall mean accuracy of 82.9% (95% CI: 80.2–85.6%) across all cases. Accuracy peaked in final diagnosis (97.7%), but declined in differential diagnosis (73.0%). Model performance showed no significant variation across demographic or severity strata.
Conclusions
DeepSeek shows promising performance in structured case-based diagnostic tasks, particularly in confirmatory diagnostic reasoning. However, its early-stage reasoning and handling of ambiguous cases require enhancement. Future studies using larger and more diverse clinical datasets are needed to further evaluate the model’s robustness and potential clinical applicability.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Emergency Medicine, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China.
- School of Medicine, Shaoxing University, Shaoxing, Zhejiang, China.
- Department of Neurosurgery, Affiliated Jinhua Hospital, Zhejiang University School of Medicine, Jinhua, China.
- Department of Critical Care Medicine, Affiliated Jinhua Hospital, Zhejiang University School of Medicine, Jinhua, China.
- Provincial Key Laboratory of Precise Diagnosis and Treatment of Abdominal Infection, School of Medicine, Sir Run Run Shaw Hospital, Zhejiang University, Zhejiang, China.
- Longquan Industrial Innovation Research Institute, Lishui, China.
ORCID-Profile: Zhongheng Zhang
Zitieren
APA
Wang, Y., He, Y., Qin, X., Hong, Y., Chen, L., Zhang, J., Ni, H., & Zhang, Z. (2026). Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning. PLOS One, 21(5), e0346078. https://doi.org/10.1371/journal.pone.0346078
Vancouver
Wang Y, He Y, Qin X, Hong Y, Chen L, Zhang J, et al. Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning. PLOS One. 2026;21(5):e0346078. doi:10.1371/journal.pone.0346078
BibTeX
@article{Wang2026,
author = {Wang, Yang and He, Yang and Qin, Xuchang and Hong, Yucai and Chen, Lin and Zhang, Jing and Ni, Hongying and Zhang, Zhongheng},
title = {Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {5},
pages = {e0346078},
doi = {10.1371/journal.pone.0346078}
}
EUVIMED