CPGPrompt: translating clinical guidelines into large language model-executable decision support
Journal of the American Medical Informatics Association · 01.04.2026 · 33 (4) · 855-862
DOI: 10.1093/jamia/ocag026 ↗Kurz gefasst
Leitlinie in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 26.02.2026. Verfasst von Ruiqi Deng und 8 weiteren Autor:innen. Beteiligte Einrichtung: Information Science (Health Tech), Cornell Tech, Cornell University, New York, NY 10044, United States. Hauptthemen laut MEDLINE: Large Language Models, Decision Support Systems, Clinical, Practice Guidelines as Topic, Decision Trees. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 16 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NLM NIH HHS, National Library of Medicine.
Abstract (Originalsprache)
Objective
Clinical practice guidelines (CPGs) provide evidence-based recommendations for patient care; however, integrating them into artificial intelligence (AI) remains challenging. Previous approaches, such as rule-based systems or black-box AI models, face significant limitations, including poor interpretability, inconsistent adherence to guidelines, and narrow domain applicability. To address this, we develop and validate CPGPrompt, an auto-prompting system that converts narrative clinical guidelines into large language models (LLMs). Materials and Methods Our framework translates CPGs into structured decision trees and utilizes an LLM to dynamically navigate them for patient case evaluation. Synthetic vignettes were generated across 3 domains—headache, lower back pain, and prostate cancer—and distributed into 4 categories to test different decision scenarios. System performance was assessed on both binary specialty referral decisions and fine-grained pathway classification tasks.
Results
The binary specialty referral classification achieved consistently strong performance across all domains (F1: 0.85-1.00), with high recall (1.00 ± 0.00). In contrast, multiclass pathway assignment showed reduced performance, with domain-specific variations: headache (F1: 0.47), lower back pain (F1: 0.72), and prostate cancer (F1: 0.77).
Discussion
Domain-specific performance differences reflected the structure of each guideline. The headache guideline highlighted challenges with negation handling. The lower back pain guideline required temporal reasoning. In contrast, prostate cancer pathways benefited from quantifiable laboratory tests, resulting in more reliable decision-making.
Conclusion
CPGPrompt demonstrates generalizability across diverse clinical domains while maintaining high sensitivity for referral decisions. Its transparent, auditable framework enables the systematic identification of failure modes and provides advantages over black-box AI approaches. However, persistent challenges with subjective clinical assessments indicate a need for targeted improvements and greater clinical robustness.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Information Science (Health Tech), Cornell Tech, Cornell University, New York, NY 10044, United States.
- Systems Engineering, Cornell University, Ithaca, NY 14853, United States.
- Population Health Sciences, Weill Cornell Medicine, New York, NY 10065, United States.
- Computer and Information Science, Cornell University, Ithaca, NY 14853, United States.
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, United States.
- Department of Medicine, Weill Cornell Medicine, New York, NY 10065, United States.
ORCID-Profile: Ruiqi Deng, Geoffrey Martin, Gongbo Zhang, Yi Liu, Chunhua Weng, Justin F Rousseau, Yifan Peng
Förderung
- NLM NIH HHS
- National Library of Medicine
Zitieren
APA
Deng, R., Martin, G., Wang, T., Zhang, G., Liu, Y., Weng, C., Wang, Y., Rousseau, J. F., & Peng, Y. (2026). CPGPrompt: translating clinical guidelines into large language model-executable decision support. Journal of the American Medical Informatics Association, 33(4), 855-862. https://doi.org/10.1093/jamia/ocag026
Vancouver
Deng R, Martin G, Wang T, Zhang G, Liu Y, Weng C, et al. CPGPrompt: translating clinical guidelines into large language model-executable decision support. Journal of the American Medical Informatics Association. 2026;33(4):855-862. doi:10.1093/jamia/ocag026
BibTeX
@article{Deng2026,
author = {Deng, Ruiqi and Martin, Geoffrey and Wang, Tony and Zhang, Gongbo and Liu, Yi and Weng, Chunhua and Wang, Yanshan and Rousseau, Justin F and Peng, Yifan},
title = {CPGPrompt: translating clinical guidelines into large language model-executable decision support},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
volume = {33},
number = {4},
pages = {855--862},
doi = {10.1093/jamia/ocag026}
}
EUVIMED