Verification feedback and self-revision for omissions in clinical recommendations by large language models: A paired evaluation
Artificial Intelligence in Health · 17.11.2026 · 026370137
DOI: 10.36922/aih026370137 ↗Kurz gefasst
Fachartikel in der Zeitschrift Artificial Intelligence in Health, online erschienen am 17.11.2026. Verfasst von Sanjay Basu und 4 weiteren Autor:innen.
Abstract (Originalsprache)
Large language models (LLMs) in clinical decision support can omit required actions (omission) or recommend wrong ones (commission). Verification layers check responses, and models can revise without feedback. Based on 176 test cases, we delivered the first responses of three LLMs (GPT-6 Astra, Gemini 3.1 Pro, Claude Opus 5.5) in six versions: no feedback, the full findings of a rule-and-agent verification framework, or alerts calibrated to a 10% false-alert budget, each appended or returned to the model for revision. Three LLM judges scored the share of high-grade (high-certainty) items on LLM-generated checklists that each text omitted; physicians reviewed samples. Revision without feedback (self-revision) lowered omission from 8.1% to 7.1% (−0.9 percentage points; 95% confidence interval [CI]: −1.7 to −0.1). In post hoc comparisons, revision with the full findings omitted 2.2 percentage points more than self-revision (95% CI: 1.3 to 3.2), or 0.8 (95% CI: −0.1 to 1.8) without the Gemini 3.1 Pro revisions that replaced content with status phrases. Revision with the alerts, which reached 56 of 528 first responses, did not differ from self-revision (0.5 percentage points; 95% CI: −0.3 to 1.3). Appended, the full findings changed judged omission by −0.6 percentage points (95% CI: −1.20 to −0.04). Based on 354 responses to physician-reviewed commission probes, both deliveries of the full findings lowered commission, mostly citations to nonexistent guidelines, from 6.8% to 3.4% (appended) and 4.2% (revised); with hedged commission counted, only revision lowered it (post hoc). Revision with verification feedback did not reduce omission beyond self-revision, which evaluations should include as a baseline.
Zitieren
APA
Basu, S., Patel, S., Sheth, P., Morgan, J., & Batniji, R. (2026). Verification feedback and self-revision for omissions in clinical recommendations by large language models: A paired evaluation. Artificial Intelligence in Health, 0(0), 026370137. https://doi.org/10.36922/aih026370137
Vancouver
Basu S, Patel S, Sheth P, Morgan J, Batniji R. Verification feedback and self-revision for omissions in clinical recommendations by large language models: A paired evaluation. Artificial Intelligence in Health. 2026;0(0):026370137. doi:10.36922/aih026370137
BibTeX
@article{Basu2026,
author = {Basu, Sanjay and Patel, Sadiq and Sheth, Parth and Morgan, John and Batniji, Rajaie},
title = {Verification feedback and self-revision for omissions in clinical recommendations by large language models: A paired evaluation},
journal = {Artificial Intelligence in Health},
year = {2026},
volume = {0},
number = {0},
pages = {026370137},
doi = {10.36922/aih026370137}
}
EUVIMED