Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features
Vorhersage der Behandlungsbindung bei Medikation gegen Opioidkonsumstörung: Machine-Learning-Ansatz mit NLP- und LLM-abgeleiteten klinischen Merkmalen
Journal of the American Medical Informatics Association · 01.12.2025 · 32 (12) · 1865-1876
DOI: 10.1093/jamia/ocaf157 ↗Auf einen Blick
LLM-abgeleitete Merkmale aus Freitextnotizen verbesserten die 6-Monats-Prognose der BUP-NAL-Behandlungsbindung nur moderat. Die besten Modelle erreichten eine ROC-AUC von 0,65.
Fragestellung
Verbessern aus Freitextnotizen mit LLMs gewonnene Merkmale die Vorhersage der 6-Monats-Behandlungsbindung unter Buprenorphin-Naloxon?
Methode
Entwicklung und interne Validierung mit EHR-Daten von Stanford Health Care (STARR), externe Validierung mit NeuroBlu; Klassifikations- und Überlebensmodelle mit strukturierten und 13 Freitext-Merkmalen.
Ergebnisse
XGBoost erzielte die beste Klassifikationsleistung (ROC-AUC 0,65). LLM-Merkmale verbesserten alle Modellarchitekturen; bei Zeit-bis-Ereignis-Modellen lag der höchste C-index bei etwa 0,65.
Schlussfolgerung der Autor:innen
Die Kombination aus strukturierten EHR-Daten und LLM-extrahierten Merkmalen verbessert die Vorhersage der BUP-NAL-Behandlungsbindung moderat und erhöht die Interpretierbarkeit.
Kernaussagen
- Beste Klassifikationsleistung: ROC-AUC 0,65
- LLM-Merkmale verbesserten alle Modellarchitekturen
- Genannte Prädiktoren: chronischer Schmerz, Lebererkrankung, Major Depression
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 22.09.2025. Verfasst von Fateme Nateghi Haredasht und 15 weiteren Autor:innen. Beteiligte Einrichtung: Stanford Center for Biomedical Informatics Research, Stanford, CA 94305, United States. Hauptthemen laut MEDLINE: Machine Learning, Opioid-Related Disorders, Natural Language Processing, Buprenorphine, Naloxone Drug Combination. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Institute of Allergy and Infectious Diseases, Google, Inc., Betty Moore Foundation.
Themen: Sucht und Abhängigkeit, Künstliche Intelligenz in der Pflege
Abstract (Originalsprache)
Objective
Building upon our previous work on predicting treatment retention in medications for opioid use disorder, we aimed to improve 6-month retention prediction in buprenorphine-naloxone (BUP-NAL) therapy by incorporating features derived from large language models (LLMs) applied to unstructured clinical notes. Materials and Methods We used de-identified electronic health record (EHR) data from Stanford Health Care (STARR) for model development and internal validation, and the NeuroBlu behavioral health database for external validation. Structured features were supplemented with 13 clinical and psychosocial features extracted from free-text notes using the CLinical Entity Augmented Retrieval pipeline, which combines named entity recognition with LLM-based classification to provide contextual interpretation. We trained classification (Logistic Regression, Random Forest, XGBoost) and survival models (CoxPH, Random Survival Forest, Survival XGBoost), evaluated using Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) and C-index.
Results
XGBoost achieved the highest classification performance (ROC-AUC = 0.65). Incorporating LLM-derived features improved model performance across all architectures, with the largest gains observed in simpler models such as Logistic Regression. In time-to-event analysis, Random Survival Forest and Survival XGBoost reached the highest C-index (≈0.65). SHapley Additive exPlanations analysis identified LLM-extracted features like Chronic Pain, Liver Disease, and Major Depression as key predictors. We also developed an interactive web tool for real-time clinical use.
Discussion
Features extracted using NLP and LLM-assisted methods improved model accuracy and interpretability, revealing valuable psychosocial risks not captured in structured EHRs.
Conclusion
Combining structured EHR data with LLM-extracted features moderately improves BUP-NAL retention prediction, enabling personalized risk stratification and advancing AI-driven care for substance use disorders.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Substanzen
Autor:innen und Institutionen
- Stanford Center for Biomedical Informatics Research, Stanford, CA 94305, United States.
- Stanford University School of Medicine, Stanford, CA 94305, United States.
- Department of Biomedical Data Science, Stanford, CA 94305, United States.
- Department of Psychiatry & Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94305, United States.
- Department of Electrical Engineering (ESAT), STADIUS Center, KU Leuven, 3001 Leuven, Belgium.
- KKT Technologies, Pte. Ltd., 139951, Singapore.
ORCID-Profile: Ivan Lopez, Tina Hernandez-Boussard
Förderung
- National Institute of Allergy and Infectious Diseases
- Google, Inc.
- Betty Moore Foundation
- Human-Centered Artificial Intelligence
- NIH
- NIH/National Institute on Drug Abuse Clinical Trials Network
- Stanford Artificial Intelligence in Medicine and Imaging-Human-Centered Artificial Intelligence (AIMI-HAI) Partnership
- NIH/National Institute of Allergy and Infectious Diseases
Zitieren
APA
Nateghi Haredasht, F., Lopez, I., Tate, S., Ashtari, P., Chan, M. M., Kulkarni, D., Chen, C. A., Vangala, M., Griffith, K., Bunning, B., Miner, A. S., Hernandez-Boussard, T., Humphreys, K., Lembke, A., Vance, L. A., & Chen, J. H. (2025). Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features. Journal of the American Medical Informatics Association, 32(12), 1865-1876. https://doi.org/10.1093/jamia/ocaf157
Vancouver
Nateghi Haredasht F, Lopez I, Tate S, Ashtari P, Chan MM, Kulkarni D, et al. Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features. Journal of the American Medical Informatics Association. 2025;32(12):1865-1876. doi:10.1093/jamia/ocaf157
BibTeX
@article{NateghiHaredasht2025,
author = {Nateghi Haredasht, Fateme and Lopez, Ivan and Tate, Steven and Ashtari, Pooya and Chan, Min Min and Kulkarni, Deepali and Chen, Chwen-Yuen Angie and Vangala, Maithri and Griffith, Kira and Bunning, Bryan and Miner, Adam S and Hernandez-Boussard, Tina and Humphreys, Keith and Lembke, Anna and Vance, L Alexander and Chen, Jonathan H},
title = {Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features},
journal = {Journal of the American Medical Informatics Association},
year = {2025},
volume = {32},
number = {12},
pages = {1865--1876},
doi = {10.1093/jamia/ocaf157}
}
EUVIMED