A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation
Journal of the American Medical Informatics Association · 03.11.2026
DOI: 10.1093/jamia/ocag178 ↗Kurz gefasst
Leitlinie in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 03.11.2026. Verfasst von Vangelis G Alexiou und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Vascular Surgery, University of Patras School of Medicine, Patras, Greece. Die Arbeit nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Microsoft Corporation Azure Sponsorship.
Abstract (Originalsprache)
Background
The clinical practice guidelines (CPGs) published by the European Society for Vascular Surgery (ESVS) span approximately 1100 pages whose simultaneous cross-referencing is impractical at the point of care. Unguarded large language models (LLMs) carry hallucination risk, motivating a guideline-locked design.
Methods
We describe the architecture and development of a context-aware clinical agent encompassing the full ESVS corpus of 14 CPGs. The system extends a validated single-guideline GraphRAG prototype with an LLM-driven Guideline Router, an embedded Context Gate that withholds retrieval when decision-critical parameters are absent, dual-channel retrieval, and GPT-5 synthesis under a mandatory citation template. An automated benchmark assessed the final system on 55 vignettes, scored by 2 blinded LLM judges over 3 runs (330 judgments), with clinician adjudication of every flagged claim.
Results
The system stayed within its locked corpus, declined all out-of-scope queries, and achieved 98.4% guideline-routing recall. The Context Gate showed 91.9% specificity and 78.6% sensitivity for underspecified queries, and all 259 cited recommendations were verifiable within the locked corpus. Automated judges flagged an unsupported claim in 18.2% of items under a sensitive screening definition; on adjudication by 2 vascular surgeons, 7.3% of items contained a clinically consequential error, none graded capable of causing harm. Clinical acceptability of complete answers was not scored and is reserved for prospective expert evaluation.
Conclusion
These results support the feasibility of a multi-guideline context-aware clinical agent for vascular surgery; citation traceability and locked-corpus provenance facilitate future auditability.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Vascular Surgery, University of Patras School of Medicine, Patras, Greece.
- Alfa Institute of Biomedical Sciences (AIBS), Athens, Greece.
- Department of Vascular Surgery, Attikon University Hospital, Athens, Greece.
- Department of Vascular Surgery, Yale University School of Medicine, New Haven, CT, United States.
- School of Medicine, National and Kapodistrian University of Athens, Athens, Greece.
ORCID-Profile: Vangelis G Alexiou
Förderung
- Microsoft Corporation Azure Sponsorship
Zitieren
APA
Alexiou, V. G., Vassiliou, A., Sumpio, B. E., Geroulakos, G., & Kakkos, S. K. (2026). A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocag178
Vancouver
Alexiou VG, Vassiliou A, Sumpio BE, Geroulakos G, Kakkos SK. A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation. Journal of the American Medical Informatics Association. 2026. doi:10.1093/jamia/ocag178
BibTeX
@article{Alexiou2026,
author = {Alexiou, Vangelis G and Vassiliou, Areti and Sumpio, Bauer E and Geroulakos, George and Kakkos, Stavros K},
title = {A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
doi = {10.1093/jamia/ocag178}
}
EUVIMED