EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions

Youlu Zhao, Hongyu Yang, Xizi Zheng, Damin Xu, Lingyi Xu, Qingqing Zhou, Mengrui Li, Yuhui Zhang, Lijun Liu, Jicheng Lv, Li Yang

Clinical Kidney Journal · 02.11.2026

DOI: 10.1093/ckj/sfag363 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Clinical Kidney Journal, online erschienen am 02.11.2026. Verfasst von Youlu Zhao und 10 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) demonstrate significant potential in medical applications. DeepSeek, as an open-source and customizable LLM, offers potential for nephrology applications through flexible deployment, potential cost reduction, and enhanced data privacy. However, its specific performance and applicability in the nephrology specialty remain to be systematically evaluated.

Methods

We evaluated six large language models (DeepSeek-R1, DeepSeek-V3, GPT-4, GPT-4o, Qwen-2.5, and Gemini-2.5) using 1 294 nephrology multiple-choice questions derived from the ASN NephSAP and KSAP educational programs. The DeepSeek-R1, DeepSeek-V3, GPT-4o, and Gemini-2.5 models were further enhanced through a retrieval-augmented generation (RAG) framework incorporating a nephrology comorbidity knowledge base (NCKB). The NCKB, reviewed by board-certified nephrologists, integrated authoritative sources including standard textbooks, clinical guidelines, and peer-reviewed literature. The NCKB included 50 347 structured knowledge units. Model accuracy, response time, and error types were analyzed using chi-square tests and visualized with bar and radar charts.

Results

A total of 1 294 nephrology exam questions were used for evaluation. The knowledge-enhanced DeepSeek models demonstrated significantly improved performance compared with their base versions (both P < 0.001), with DSR1 + NCKB and DSV3 + NCKB achieving accuracies of 81.22% and 79.13%, respectively. The performance difference between DSR1 + NCKB and DSV3 + NCKB did not reach statistical significance (P = 0.171). Knowledge gaps were the main cause of DSR1 + NCKB’s errors (52.26%), followed by reasoning and comprehension errors (20.16%).

Conclusion

Integrating an expert-curated NCKB significantly improves performance on standardized nephrology questions compared with baseline models. The magnitude of improvement varies across models, with DSR1 + NCKB achieving performance comparable to or approaching that of advanced proprietary models evaluated in this study.

Zitieren

APA

Zhao, Y., Yang, H., Zheng, X., Xu, D., Xu, L., Zhou, Q., Li, M., Zhang, Y., Liu, L., Lv, J., & Yang, L. (2026). Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions. Clinical Kidney Journal. https://doi.org/10.1093/ckj/sfag363

Vancouver

Zhao Y, Yang H, Zheng X, Xu D, Xu L, Zhou Q, et al. Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions. Clinical Kidney Journal. 2026. doi:10.1093/ckj/sfag363

BibTeX
@article{Zhao2026,
  author = {Zhao, Youlu and Yang, Hongyu and Zheng, Xizi and Xu, Damin and Xu, Lingyi and Zhou, Qingqing and Li, Mengrui and Zhang, Yuhui and Liu, Lijun and Lv, Jicheng and Yang, Li},
  title = {Evaluation of knowledge-augmented DeepSeek models on nephrology examination questions},
  journal = {Clinical Kidney Journal},
  year = {2026},
  doi = {10.1093/ckj/sfag363}
}
Kostenlos starten →