EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning

Yang Wang, Yang He, Xuchang Qin, Yucai Hong, Lin Chen, Jing Zhang, Hongying Ni, Zhongheng Zhang iD

PLOS One · 08.05.2026 · 21 (5) · e0346078

DOI: 10.1371/journal.pone.0346078 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 08.05.2026. Verfasst von Yang Wang und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Emergency Medicine, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China. Hauptthemen laut MEDLINE: Clinical Decision-Making, Clinical Reasoning. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 9 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Artificial intelligence (AI) is undergoing an era of transformative advancement, particularly through the emergence of Transformer-based large language models (LLMs). While these systems demonstrate strong reasoning and generalization capabilities, their clinical applicability, particularly in emergency and critical care decision-making, remains underexplored.. In time-sensitive settings, diagnostic reasoning must align rigorously with evidence-based standards and ensure the relevance of timing to clinical decisions.

Objective

This study aims to provide a preliminary evaluation of the decision-support performance of the DeepSeek model in acute medical scenarios. We systematically evaluate its diagnostic reasoning, temporal consistency of recommendations, and adherence to evidence-based critical care protocols using standardized case-based assessments.

Methods

Twenty-nine representative clinical cases were extracted from the Merck Manual of Diagnosis and Therapy, a widely used medical reference providing standardized case descriptions. The model’s outputs were evaluated across four decision-making dimensions: differential diagnosis, diagnostic testing, final diagnosis, and management planning. Human raters scored each response for accuracy, and multivariable linear regression was applied to assess associations between performance and case parameters (age, gender, and Rapid Emergency Medicine Score [REMS]).

Results

DeepSeek achieved an overall mean accuracy of 82.9% (95% CI: 80.2–85.6%) across all cases. Accuracy peaked in final diagnosis (97.7%), but declined in differential diagnosis (73.0%). Model performance showed no significant variation across demographic or severity strata.

Conclusions

DeepSeek shows promising performance in structured case-based diagnostic tasks, particularly in confirmatory diagnostic reasoning. However, its early-stage reasoning and handling of ambiguous cases require enhancement. Future studies using larger and more diverse clinical datasets are needed to further evaluate the model’s robustness and potential clinical applicability.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Clinical Decision-MakingClinical ReasoningHumansLarge Language ModelsFemaleMaleArtificial IntelligenceDiagnosis, DifferentialAdultMiddle Aged

Autor:innen und Institutionen

  • Department of Emergency Medicine, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China.
  • School of Medicine, Shaoxing University, Shaoxing, Zhejiang, China.
  • Department of Neurosurgery, Affiliated Jinhua Hospital, Zhejiang University School of Medicine, Jinhua, China.
  • Department of Critical Care Medicine, Affiliated Jinhua Hospital, Zhejiang University School of Medicine, Jinhua, China.
  • Provincial Key Laboratory of Precise Diagnosis and Treatment of Abdominal Infection, School of Medicine, Sir Run Run Shaw Hospital, Zhejiang University, Zhejiang‌‌, China.
  • Longquan Industrial Innovation Research Institute, Lishui, China.

ORCID-Profile: Zhongheng Zhang

Zitieren

APA

Wang, Y., He, Y., Qin, X., Hong, Y., Chen, L., Zhang, J., Ni, H., & Zhang, Z. (2026). Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning. PLOS One, 21(5), e0346078. https://doi.org/10.1371/journal.pone.0346078

Vancouver

Wang Y, He Y, Qin X, Hong Y, Chen L, Zhang J, et al. Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning. PLOS One. 2026;21(5):e0346078. doi:10.1371/journal.pone.0346078

BibTeX
@article{Wang2026,
  author = {Wang, Yang and He, Yang and Qin, Xuchang and Hong, Yucai and Chen, Lin and Zhang, Jing and Ni, Hongying and Zhang, Zhongheng},
  title = {Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {5},
  pages = {e0346078},
  doi = {10.1371/journal.pone.0346078}
}
Kostenlos starten →