EUVIMEDEUVIMED entdecken
LeitlinieOpen Access · CC BY 4.0

A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation

Vangelis G Alexiou iD, Areti Vassiliou, Bauer E Sumpio, George Geroulakos, Stavros K Kakkos

Journal of the American Medical Informatics Association · 03.11.2026

DOI: 10.1093/jamia/ocag178 ↗

Kurz gefasst

Leitlinie in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 03.11.2026. Verfasst von Vangelis G Alexiou und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Vascular Surgery, University of Patras School of Medicine, Patras, Greece. Die Arbeit nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Microsoft Corporation Azure Sponsorship.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The clinical practice guidelines (CPGs) published by the European Society for Vascular Surgery (ESVS) span approximately 1100 pages whose simultaneous cross-referencing is impractical at the point of care. Unguarded large language models (LLMs) carry hallucination risk, motivating a guideline-locked design.

Methods

We describe the architecture and development of a context-aware clinical agent encompassing the full ESVS corpus of 14 CPGs. The system extends a validated single-guideline GraphRAG prototype with an LLM-driven Guideline Router, an embedded Context Gate that withholds retrieval when decision-critical parameters are absent, dual-channel retrieval, and GPT-5 synthesis under a mandatory citation template. An automated benchmark assessed the final system on 55 vignettes, scored by 2 blinded LLM judges over 3 runs (330 judgments), with clinician adjudication of every flagged claim.

Results

The system stayed within its locked corpus, declined all out-of-scope queries, and achieved 98.4% guideline-routing recall. The Context Gate showed 91.9% specificity and 78.6% sensitivity for underspecified queries, and all 259 cited recommendations were verifiable within the locked corpus. Automated judges flagged an unsupported claim in 18.2% of items under a sensitive screening definition; on adjudication by 2 vascular surgeons, 7.3% of items contained a clinically consequential error, none graded capable of causing harm. Clinical acceptability of complete answers was not scored and is reserved for prospective expert evaluation.

Conclusion

These results support the feasibility of a multi-guideline context-aware clinical agent for vascular surgery; citation traceability and locked-corpus provenance facilitate future auditability.

Schlagwörter

Stichwörter der Autor:innen

ESVSGraphRAGagentic AIartificial intelligenceclinical decision supportclinical practice guidelinescontext-aware clinical agentquery validationretrieval-augmented generationvascular surgery

Autor:innen und Institutionen

  • Department of Vascular Surgery, University of Patras School of Medicine, Patras, Greece.
  • Alfa Institute of Biomedical Sciences (AIBS), Athens, Greece.
  • Department of Vascular Surgery, Attikon University Hospital, Athens, Greece.
  • Department of Vascular Surgery, Yale University School of Medicine, New Haven, CT, United States.
  • School of Medicine, National and Kapodistrian University of Athens, Athens, Greece.

ORCID-Profile: Vangelis G Alexiou

Förderung

  • Microsoft Corporation Azure Sponsorship

Zitieren

APA

Alexiou, V. G., Vassiliou, A., Sumpio, B. E., Geroulakos, G., & Kakkos, S. K. (2026). A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocag178

Vancouver

Alexiou VG, Vassiliou A, Sumpio BE, Geroulakos G, Kakkos SK. A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation. Journal of the American Medical Informatics Association. 2026. doi:10.1093/jamia/ocag178

BibTeX
@article{Alexiou2026,
  author = {Alexiou, Vangelis G and Vassiliou, Areti and Sumpio, Bauer E and Geroulakos, George and Kakkos, Stavros K},
  title = {A multi-guideline clinical agent for vascular surgery: architecture, implementation, and benchmark evaluation},
  journal = {Journal of the American Medical Informatics Association},
  year = {2026},
  doi = {10.1093/jamia/ocag178}
}
Kostenlos starten →