Understanding Clinical Urgency Representations for Safer Medical Language Models
Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 1195-1202
DOI: 10.1609/aaaiss.v10i1.43544 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Mehar Prateek Kalra und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Large language models (LLMs) are increasingly used in patient-facing medical settings, including triage advice about whether a case can be managed at home or needs urgent or emergency care. Trustworthy medical LLMs must not only provide accurate answers but also appropriately recognize the severity of clinical situations. Yet current evaluations judge these systems almost entirely by their generated text, where models can easily produce a cautious-sounding reply without understanding the severity of the case. We investigate whether medical LLMs internally register clinical urgency before response generation and whether such internal representations are faithfully reflected in model responses. We probe the last-token hidden state activations of 284 CARE-Bench training cases using linear, PCA, and SVM classifiers under 5-fold stratified cross-validation and evaluate the results on a holdout test set of 50 prompts. We find that clinical urgency is linearly recoverable from hidden states and is not fully captured by its next-token preferences. Our probes reach a cross-validated AUROC of about 0.73 and holdout AUROCs of 0.77--0.86, remaining well above chance under 95% bootstrap intervals, and consistently score higher than a KL-divergence next-token baseline. Comparing the probe's urgency estimate with an independent LLM judge's rating of the generated reply, 70% of held-out cases are aligned, while 30% are flagged for review. Of the 7 inappropriate responses, the probe–judge comparison catches 5, while aligned responses are correct 94% of the time. These results do not indicate whether the vignette is truly urgent; rather, they assess whether the model's internal sense of urgency is consistent with the advice it provides to the patient.
Zitieren
APA
Kalra, M. P., Devulapalli, U., & Narayan, A. (2026). Understanding Clinical Urgency Representations for Safer Medical Language Models. Proceedings of the AAAI Symposium Series, 10(1), 1195-1202. https://doi.org/10.1609/aaaiss.v10i1.43544
Vancouver
Kalra MP, Devulapalli U, Narayan A. Understanding Clinical Urgency Representations for Safer Medical Language Models. Proceedings of the AAAI Symposium Series. 2026;10(1):1195-1202. doi:10.1609/aaaiss.v10i1.43544
BibTeX
@article{Kalra2026,
author = {Kalra, Mehar Prateek and Devulapalli, Uday and Narayan, Apurva},
title = {Understanding Clinical Urgency Representations for Safer Medical Language Models},
journal = {Proceedings of the AAAI Symposium Series},
year = {2026},
volume = {10},
number = {1},
pages = {1195--1202},
doi = {10.1609/aaaiss.v10i1.43544}
}
EUVIMED