Risk-Calibrated Medical LLM Triage with Physician-Rubric Self-Verification and Patient-Facing Evidence Cards: An Empirical Health Bench Study
Stout in Medical Science, Biology and Chemistry · 18.02.2026 · 2 (1) · 36-55
DOI: 10.61424/k0qrad59 ↗Kurz gefasst
Fachartikel in der Zeitschrift Stout in Medical Science, Biology and Chemistry, online erschienen am 18.02.2026. Verfasst von Oscar Liu.
Abstract (Originalsprache)
Medical large language models can answer patient questions fluently while remaining unreliable about urgency, uncertainty, and safe escalation. This study evaluated a risk-controlled post-generation architecture on HealthBench 2025, comprising 5,000 physician-annotated medical conversations, 57,237 weighted rubric criteria, 4,206 physician ideal responses, and 11,112 stored answers across 2,778 four-candidate cases. The architecture combined a word-level triage classifier, fold-local rubric retrieval, a word–character–retrieval ensemble with temperature scaling, deterministic answer reranking, triage-policy self-verification, selective escalation, and patient-facing evidence cards. Five-fold out-of-fold evaluation on 453 physician-labeled emergency-referral cases produced a macro-F1 of 0.645 for the self-verified ensemble versus 0.620 for direct TF-IDF; expected calibration error fell from 0.126 to 0.058, and under-triage fell from 18.5% to 16.1%. At a 0.70 confidence threshold, coverage was 32.0%, selective risk was 22.1%, and urgent-route recall reached 98.1%. On the 2,778 candidate-selection cases, rubric-RAG raised weighted rubric-alignment coverage from 0.801 to 0.845. Self-verification preserved 0.845 alignment while reducing the response-level triage-policy error proxy from 53.8% to 42.3%. The results show that retrieval improves content coverage, whereas a distinct safety check is needed to convert content quality into safer disposition behavior. Confidence-aware abstention and concise evidence cards provide an auditable interface for patient-facing use, while the remaining uncertainty supports clinician oversight rather than autonomous deployment.
Zitieren
APA
Liu, O. (2026). Risk-Calibrated Medical LLM Triage with Physician-Rubric Self-Verification and Patient-Facing Evidence Cards: An Empirical Health Bench Study. Stout in Medical Science, Biology and Chemistry, 2(1), 36-55. https://doi.org/10.61424/k0qrad59
Vancouver
Liu O. Risk-Calibrated Medical LLM Triage with Physician-Rubric Self-Verification and Patient-Facing Evidence Cards: An Empirical Health Bench Study. Stout in Medical Science, Biology and Chemistry. 2026;2(1):36-55. doi:10.61424/k0qrad59
BibTeX
@article{Liu2026,
author = {Liu, Oscar},
title = {Risk-Calibrated Medical LLM Triage with Physician-Rubric Self-Verification and Patient-Facing Evidence Cards: An Empirical Health Bench Study},
journal = {Stout in Medical Science, Biology and Chemistry},
year = {2026},
volume = {2},
number = {1},
pages = {36--55},
doi = {10.61424/k0qrad59}
}
EUVIMED