EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Explainable AI for mental health emergency returns: integrating large language models with predictive modeling

Abdulaziz Ahmed, Mohammad Saleem, Mohammed Alzeen iD, Badari Birur, Rachel E Fargason, Bradley G Burk, Ahmed Alhassan, Mohammed Ali Al-Garadi

JAMIA Open · 05.05.2026 · 9 (3)

DOI: 10.1093/jamiaopen/ooag065 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 19.06.2026. Verfasst von Abdulaziz Ahmed und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health Services Administration, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL 35233, United States. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Emergency department (ED) returns for mental health (MH) conditions pose a substantial burden on healthcare systems. Traditional machine learning (ML) models have shown promise in predicting ED returns but often fall short in clinical interpretability. This study evaluates whether integrating Large Language Models (LLMs) with ML methods can enhance both predictive performance and interpretability.

Methods

We conducted a retrospective analysis of 42 464 ED visits involving 27 904 unique MH patients at an academic medical center in the deep south (2018-2022). Two primary outcomes were assessed: (1) 30-day ED return prediction, and (2) interpretability of model outputs. We employed LLaMA 3 (8B) for few-shot classification of chief complaints and social determinants of health (SDoH) features, including tobacco, alcohol, substance use, exercise, and home environment. These LLM-derived features were integrated into an XGBoost model. Model explanation was enhanced using a novel LLM-SHAP hybrid framework.

Results

The LLM-based chief complaint classifier achieved 0.882 accuracy, 0.95 precision, 0.88 recall, and 0.86 F1-score, outperforming traditional models. SDoH classifiers reached F1-scores ranging from 0.67 to 0.96 across different categories. Incorporating LLM-extracted features into XGBoost improved AUC from 0.74 to 0.76 and AUC-PR from 0.58 to 0.61. The LLM-SHAP framework provided enhanced contextualized explanations for risk predictions, improving interpretability at both patient and population levels.

Conclusion

Integrating LLMs with traditional ML models modestly improved predictive performance but substantially enhanced interpretability. This hybrid approach offers a promising path toward more actionable and explainable clinical decision support tools in emergency psychiatry.

Schlagwörter

Stichwörter der Autor:innen

30 days emergency returnEmergency departmentExplainable AIlarge language modelmachine learning

Autor:innen und Institutionen

  • Department of Health Services Administration, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL 35233, United States.
  • Department of Biomedical Informatics and Data Science, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL 35233, United States.
  • Department of Psychiatry and Behavioral Neurobiology, University of Alabama at Birmingham, Birmingham, AL 35233, United States.
  • Department of Pharmacy, University of Alabama at Birmingham, Birmingham, AL 35233, United States.
  • Department of Biomedical Informatics, School of Medicine, Vanderbilt University Medical Center, Nashville, TN 37203, United States.

ORCID-Profile: Mohammed Alzeen

Zitieren

APA

Ahmed, A., Saleem, M., Alzeen, M., Birur, B., Fargason, R. E., Burk, B. G., Alhassan, A., & Al-Garadi, M. A. (2026). Explainable AI for mental health emergency returns: integrating large language models with predictive modeling. JAMIA Open, 9(3). https://doi.org/10.1093/jamiaopen/ooag065

Vancouver

Ahmed A, Saleem M, Alzeen M, Birur B, Fargason RE, Burk BG, et al. Explainable AI for mental health emergency returns: integrating large language models with predictive modeling. JAMIA Open. 2026;9(3). doi:10.1093/jamiaopen/ooag065

BibTeX
@article{Ahmed2026,
  author = {Ahmed, Abdulaziz and Saleem, Mohammad and Alzeen, Mohammed and Birur, Badari and Fargason, Rachel E and Burk, Bradley G and Alhassan, Ahmed and Al-Garadi, Mohammed Ali},
  title = {Explainable AI for mental health emergency returns: integrating large language models with predictive modeling},
  journal = {JAMIA Open},
  year = {2026},
  volume = {9},
  number = {3},
  doi = {10.1093/jamiaopen/ooag065}
}
Kostenlos starten →