Enhancing LLM-based medical decision-making by test-time knowledge acquisition
Health Information Science and Systems · 06.04.2026 · 14 (1)
DOI: 10.1007/s13755-026-00449-8 ↗Kurz gefasst
Fachartikel in der Zeitschrift Health Information Science and Systems, online erschienen am 06.04.2026. Verfasst von Shipeng Li und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Intelligent Science and Technology, Nanjing University, Taihu Avenue, Suzhou, 215163 Jiangsu China. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Purpose
Medical decision-making (MDM) is a complex clinical reasoning process that requires the systematic integration of multidisciplinary knowledge and evidence. Current approaches based on large language models (LLMs) are constrained by their reliance on static training corpora and often exhibit limited domain-specific adaptation, which can compromise diagnostic accuracy and reliability. This study aims to overcome these limitations by developing a framework that enables LLMs to dynamically acquire and refine knowledge during test time, thereby enhancing the robustness and precision of MDM systems.
Methods
We propose a test-time optimization framework that refines a frozen LLM's diagnostic reasoning through test-time knowledge acquisition and integration. For each medical query, the model generates multiple trajectories that are synthesized into a pseudo reference answer, whose self-consistency score separates confident from unconfident cases. Confident cases enable reward-guided reflection to extract reliable diagnostic heuristics, while unconfident cases undergo unsupervised reflection to reveal reasoning gaps and uncertainty patterns. The extracted knowledge is continually incorporated into an evolving, capacity-controlled knowledge base through operations that add, modify, or merge knowledge. This updated knowledge base then guides subsequent inference, allowing the model to adapt its reasoning strategy during test time without updating any parameters.
Results
Experimental evaluations on three public medical decision-making benchmarks-MedQA, NEJMQA, and MMLU-Pro-Health-show that the proposed framework consistently improves the performance of the state-of-the-art LLM, DeepSeekv3.2 Exp 671B. For example, on the MMLU-Pro-Health dataset, our method achieved an average accuracy of 79.22%, surpassing DeepSeekv3.2 Exp 671B by 1.84 percentage points, thus demonstrating the effectiveness of the framework in enhancing diagnostic decision-making.
Conclusion
By leveraging inference-time self-evaluation and experience accumulation, this work introduces a new paradigm for building reliable, adaptive, and context-aware medical AI systems. It underscores the critical role of continual knowledge evolution in advancing trustworthy artificial intelligence for clinical decision support and lays the foundation for future developments in dynamic and responsive medical reasoning tools.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Intelligent Science and Technology, Nanjing University, Taihu Avenue, Suzhou, 215163 Jiangsu China.
- School of Automation, Hangzhou Dianzi University, Wenyi Road, Hangzhou, 310018 Zhejiang China.
- Department of Computer Science, Hong Kong Baptist University, Waterloo Road, Kowloon Tong, Hong Kong China.
- Department of Dermatology, Renji Hospital, Shanghai Jiao Tong University School of Medicine, Pujian Road, Pudong, 200127 Shanghai China.
Zitieren
APA
Li, S., Bao, L., Li, S., & Wan, B. (2026). Enhancing LLM-based medical decision-making by test-time knowledge acquisition. Health Information Science and Systems, 14(1). https://doi.org/10.1007/s13755-026-00449-8
Vancouver
Li S, Bao L, Li S, Wan B. Enhancing LLM-based medical decision-making by test-time knowledge acquisition. Health Information Science and Systems. 2026;14(1). doi:10.1007/s13755-026-00449-8
BibTeX
@article{Li2026,
author = {Li, Shipeng and Bao, Liuxin and Li, Shikun and Wan, Bo},
title = {Enhancing LLM-based medical decision-making by test-time knowledge acquisition},
journal = {Health Information Science and Systems},
year = {2026},
volume = {14},
number = {1},
doi = {10.1007/s13755-026-00449-8}
}
EUVIMED