EUVIMEDEUVIMED entdecken
LeitlinieOpen Access

CPGPrompt: translating clinical guidelines into large language model-executable decision support

Ruiqi Deng iD, Geoffrey Martin iD, Tony Wang, Gongbo Zhang iD, Yi Liu iD, Chunhua Weng iD, Yanshan Wang, Justin F Rousseau iD, Yifan Peng iD

Journal of the American Medical Informatics Association · 01.04.2026 · 33 (4) · 855-862

DOI: 10.1093/jamia/ocag026 ↗

Kurz gefasst

Leitlinie in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 26.02.2026. Verfasst von Ruiqi Deng und 8 weiteren Autor:innen. Beteiligte Einrichtung: Information Science (Health Tech), Cornell Tech, Cornell University, New York, NY 10044, United States. Hauptthemen laut MEDLINE: Large Language Models, Decision Support Systems, Clinical, Practice Guidelines as Topic, Decision Trees. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 16 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NLM NIH HHS, National Library of Medicine.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Clinical practice guidelines (CPGs) provide evidence-based recommendations for patient care; however, integrating them into artificial intelligence (AI) remains challenging. Previous approaches, such as rule-based systems or black-box AI models, face significant limitations, including poor interpretability, inconsistent adherence to guidelines, and narrow domain applicability. To address this, we develop and validate CPGPrompt, an auto-prompting system that converts narrative clinical guidelines into large language models (LLMs). Materials and Methods Our framework translates CPGs into structured decision trees and utilizes an LLM to dynamically navigate them for patient case evaluation. Synthetic vignettes were generated across 3 domains—headache, lower back pain, and prostate cancer—and distributed into 4 categories to test different decision scenarios. System performance was assessed on both binary specialty referral decisions and fine-grained pathway classification tasks.

Results

The binary specialty referral classification achieved consistently strong performance across all domains (F1: 0.85-1.00), with high recall (1.00 ± 0.00). In contrast, multiclass pathway assignment showed reduced performance, with domain-specific variations: headache (F1: 0.47), lower back pain (F1: 0.72), and prostate cancer (F1: 0.77).

Discussion

Domain-specific performance differences reflected the structure of each guideline. The headache guideline highlighted challenges with negation handling. The lower back pain guideline required temporal reasoning. In contrast, prostate cancer pathways benefited from quantifiable laboratory tests, resulting in more reliable decision-making.

Conclusion

CPGPrompt demonstrates generalizability across diverse clinical domains while maintaining high sensitivity for referral decisions. Its transparent, auditable framework enables the systematic identification of failure modes and provides advantages over black-box AI approaches. However, persistent challenges with subjective clinical assessments indicate a need for targeted improvements and greater clinical robustness.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsDecision Support Systems, ClinicalPractice Guidelines as TopicDecision TreesArtificial IntelligenceHumansProstatic NeoplasmsMaleLow Back PainReferral and ConsultationIntelligent Systems

Stichwörter der Autor:innen

AIclinical decision supportclinical practice guidelinesdecision treeslarge language models

Autor:innen und Institutionen

  • Information Science (Health Tech), Cornell Tech, Cornell University, New York, NY 10044, United States.
  • Systems Engineering, Cornell University, Ithaca, NY 14853, United States.
  • Population Health Sciences, Weill Cornell Medicine, New York, NY 10065, United States.
  • Computer and Information Science, Cornell University, Ithaca, NY 14853, United States.
  • Department of Biomedical Informatics, Columbia University, New York, NY 10032, United States.
  • Department of Medicine, Weill Cornell Medicine, New York, NY 10065, United States.

ORCID-Profile: Ruiqi Deng, Geoffrey Martin, Gongbo Zhang, Yi Liu, Chunhua Weng, Justin F Rousseau, Yifan Peng

Förderung

  • NLM NIH HHS
  • National Library of Medicine

Zitieren

APA

Deng, R., Martin, G., Wang, T., Zhang, G., Liu, Y., Weng, C., Wang, Y., Rousseau, J. F., & Peng, Y. (2026). CPGPrompt: translating clinical guidelines into large language model-executable decision support. Journal of the American Medical Informatics Association, 33(4), 855-862. https://doi.org/10.1093/jamia/ocag026

Vancouver

Deng R, Martin G, Wang T, Zhang G, Liu Y, Weng C, et al. CPGPrompt: translating clinical guidelines into large language model-executable decision support. Journal of the American Medical Informatics Association. 2026;33(4):855-862. doi:10.1093/jamia/ocag026

BibTeX
@article{Deng2026,
  author = {Deng, Ruiqi and Martin, Geoffrey and Wang, Tony and Zhang, Gongbo and Liu, Yi and Weng, Chunhua and Wang, Yanshan and Rousseau, Justin F and Peng, Yifan},
  title = {CPGPrompt: translating clinical guidelines into large language model-executable decision support},
  journal = {Journal of the American Medical Informatics Association},
  year = {2026},
  volume = {33},
  number = {4},
  pages = {855--862},
  doi = {10.1093/jamia/ocag026}
}
Kostenlos starten →