LLM-guided counterfactual reasoning for explainable clinical risk prediction
Intelligent Hospital · 2026-12 · 2 (4) · 100113
DOI: 10.1016/j.inhs.2026.100113 ↗Kurz gefasst
Fachartikel in der Zeitschrift Intelligent Hospital, 2026-12. Verfasst von Syed Mohammad und 4 weiteren Autor:innen. Die Arbeit nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Qatar National Library, Hamad Medical Corporation.
Abstract (Originalsprache)
Deep learning models have demonstrated promising performance in clinical natural language processing (NLP), yet their black-box nature limits clinical interpretability and trustworthiness. Although post-hoc explainability methods such as attention visualization, SHAP, and LIME identify influential features, they provide limited insight into how clinically meaningful modifications to a patient’s narrative would alter the model prediction. This paper proposes a structured large language model (LLM)-guided counterfactual explanation framework for interpreting black-box ClinicalBERT models predicting opioid-induced respiratory depression (OIRD). Unlike conventional single-prompt prompting strategies, the proposed methodology decomposes counterfactual generation into three sequential reasoning stages: (i) latent clinical concept identification, (ii) localization of the corresponding textual evidence, and (iii) controlled generation of minimally modified counterfactual clinical narratives. This structured prompting strategy improves the transparency and controllability of the explanation process while preserving the semantic coherence and clinical plausibility of the generated narratives. The proposed framework was evaluated using three state-of-the-art open-source LLMs, namely DeepSeek-R1 (14B), Qwen2.5-Coder (32B), and Llama-3.1 (8B)-Instruct, for explaining the predictions of a fine-tuned ClinicalBERT OIRD classifier. Experimental results demonstrate that the proposed framework consistently generates clinically meaningful counterfactual explanations capable of altering model predictions through minimal and targeted textual modifications. Among the evaluated models, Qwen2.5-Coder (32B) achieved the highest counterfactual generation success rate, while DeepSeek-R1 (14B) produced explanations with superior semantic preservation and clinical coherence. The comparative analysis further demonstrates that the proposed framework is model-agnostic and can be readily integrated with different LLMs without modifying the underlying prediction model. These findings suggest that the proposed structured prompting framework provides a practical and generalizable approach for improving explainability and transparency in clinical NLP systems and has the potential to support trustworthy AI-assisted clinical decision making.
Förderung
- Qatar National Library
- Hamad Medical Corporation
Zitieren
APA
Mohammad, S., Paul, S., Mohamed, N. M. F., Kumar, N., & Dakua, S. P. (2026). LLM-guided counterfactual reasoning for explainable clinical risk prediction. Intelligent Hospital, 2(4), 100113. https://doi.org/10.1016/j.inhs.2026.100113
Vancouver
Mohammad S, Paul S, Mohamed NMF, Kumar N, Dakua SP. LLM-guided counterfactual reasoning for explainable clinical risk prediction. Intelligent Hospital. 2026;2(4):100113. doi:10.1016/j.inhs.2026.100113
BibTeX
@article{Mohammad2026,
author = {Mohammad, Syed and Paul, Santu and Mohamed, Nahla Mohamed Farid and Kumar, Neeraj and Dakua, Sarada Prasad},
title = {LLM-guided counterfactual reasoning for explainable clinical risk prediction},
journal = {Intelligent Hospital},
year = {2026},
volume = {2},
number = {4},
pages = {100113},
doi = {10.1016/j.inhs.2026.100113}
}
EUVIMED