Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions
Clinical Kidney Journal · 02.11.2026
DOI: 10.1093/ckj/sfag363 ↗Kurz gefasst
Fachartikel in der Zeitschrift Clinical Kidney Journal, online erschienen am 02.11.2026. Verfasst von Youlu Zhao und 10 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Background
Large language models (LLMs) demonstrate significant potential in medical applications. DeepSeek, as an open-source and customizable LLM, offers potential for nephrology applications through flexible deployment, potential cost reduction, and enhanced data privacy. However, its specific performance and applicability in the nephrology specialty remain to be systematically evaluated.
Methods
We evaluated six large language models (DeepSeek-R1, DeepSeek-V3, GPT-4, GPT-4o, Qwen-2.5, and Gemini-2.5) using 1 294 nephrology multiple-choice questions derived from the ASN NephSAP and KSAP educational programs. The DeepSeek-R1, DeepSeek-V3, GPT-4o, and Gemini-2.5 models were further enhanced through a retrieval-augmented generation (RAG) framework incorporating a nephrology comorbidity knowledge base (NCKB). The NCKB, reviewed by board-certified nephrologists, integrated authoritative sources including standard textbooks, clinical guidelines, and peer-reviewed literature. The NCKB included 50 347 structured knowledge units. Model accuracy, response time, and error types were analyzed using chi-square tests and visualized with bar and radar charts.
Results
A total of 1 294 nephrology exam questions were used for evaluation. The knowledge-enhanced DeepSeek models demonstrated significantly improved performance compared with their base versions (both P < 0.001), with DSR1 + NCKB and DSV3 + NCKB achieving accuracies of 81.22% and 79.13%, respectively. The performance difference between DSR1 + NCKB and DSV3 + NCKB did not reach statistical significance (P = 0.171). Knowledge gaps were the main cause of DSR1 + NCKB’s errors (52.26%), followed by reasoning and comprehension errors (20.16%).
Conclusion
Integrating an expert-curated NCKB significantly improves performance on standardized nephrology questions compared with baseline models. The magnitude of improvement varies across models, with DSR1 + NCKB achieving performance comparable to or approaching that of advanced proprietary models evaluated in this study.
Zitieren
APA
Zhao, Y., Yang, H., Zheng, X., Xu, D., Xu, L., Zhou, Q., Li, M., Zhang, Y., Liu, L., Lv, J., & Yang, L. (2026). Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions. Clinical Kidney Journal. https://doi.org/10.1093/ckj/sfag363
Vancouver
Zhao Y, Yang H, Zheng X, Xu D, Xu L, Zhou Q, et al. Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions. Clinical Kidney Journal. 2026. doi:10.1093/ckj/sfag363
BibTeX
@article{Zhao2026,
author = {Zhao, Youlu and Yang, Hongyu and Zheng, Xizi and Xu, Damin and Xu, Lingyi and Zhou, Qingqing and Li, Mengrui and Zhang, Yuhui and Liu, Lijun and Lv, Jicheng and Yang, Li},
title = {Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions},
journal = {Clinical Kidney Journal},
year = {2026},
doi = {10.1093/ckj/sfag363}
}
EUVIMED