EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features

Vorhersage der Behandlungsbindung bei Medikation gegen Opioidkonsumstörung: Machine-Learning-Ansatz mit NLP- und LLM-abgeleiteten klinischen Merkmalen

Fateme Nateghi Haredasht, Ivan Lopez iD, Steven Tate, Pooya Ashtari, Min Min Chan, Deepali Kulkarni, Chwen-Yuen Angie Chen, Maithri Vangala, Kira Griffith, Bryan Bunning, Adam S Miner, Tina Hernandez-Boussard iD und 4 weitere

Journal of the American Medical Informatics Association · 01.12.2025 · 32 (12) · 1865-1876

DOI: 10.1093/jamia/ocaf157 ↗

Auf einen Blick

LLM-abgeleitete Merkmale aus Freitextnotizen verbesserten die 6-Monats-Prognose der BUP-NAL-Behandlungsbindung nur moderat. Die besten Modelle erreichten eine ROC-AUC von 0,65.

Fragestellung

Verbessern aus Freitextnotizen mit LLMs gewonnene Merkmale die Vorhersage der 6-Monats-Behandlungsbindung unter Buprenorphin-Naloxon?

Methode

Entwicklung und interne Validierung mit EHR-Daten von Stanford Health Care (STARR), externe Validierung mit NeuroBlu; Klassifikations- und Überlebensmodelle mit strukturierten und 13 Freitext-Merkmalen.

Ergebnisse

XGBoost erzielte die beste Klassifikationsleistung (ROC-AUC 0,65). LLM-Merkmale verbesserten alle Modellarchitekturen; bei Zeit-bis-Ereignis-Modellen lag der höchste C-index bei etwa 0,65.

Schlussfolgerung der Autor:innen

Die Kombination aus strukturierten EHR-Daten und LLM-extrahierten Merkmalen verbessert die Vorhersage der BUP-NAL-Behandlungsbindung moderat und erhöht die Interpretierbarkeit.

Kernaussagen

  • Beste Klassifikationsleistung: ROC-AUC 0,65
  • LLM-Merkmale verbesserten alle Modellarchitekturen
  • Genannte Prädiktoren: chronischer Schmerz, Lebererkrankung, Major Depression

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 22.09.2025. Verfasst von Fateme Nateghi Haredasht und 15 weiteren Autor:innen. Beteiligte Einrichtung: Stanford Center for Biomedical Informatics Research, Stanford, CA 94305, United States. Hauptthemen laut MEDLINE: Machine Learning, Opioid-Related Disorders, Natural Language Processing, Buprenorphine, Naloxone Drug Combination. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Institute of Allergy and Infectious Diseases, Google, Inc., Betty Moore Foundation.

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Building upon our previous work on predicting treatment retention in medications for opioid use disorder, we aimed to improve 6-month retention prediction in buprenorphine-naloxone (BUP-NAL) therapy by incorporating features derived from large language models (LLMs) applied to unstructured clinical notes. Materials and Methods We used de-identified electronic health record (EHR) data from Stanford Health Care (STARR) for model development and internal validation, and the NeuroBlu behavioral health database for external validation. Structured features were supplemented with 13 clinical and psychosocial features extracted from free-text notes using the CLinical Entity Augmented Retrieval pipeline, which combines named entity recognition with LLM-based classification to provide contextual interpretation. We trained classification (Logistic Regression, Random Forest, XGBoost) and survival models (CoxPH, Random Survival Forest, Survival XGBoost), evaluated using Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) and C-index.

Results

XGBoost achieved the highest classification performance (ROC-AUC = 0.65). Incorporating LLM-derived features improved model performance across all architectures, with the largest gains observed in simpler models such as Logistic Regression. In time-to-event analysis, Random Survival Forest and Survival XGBoost reached the highest C-index (≈0.65). SHapley Additive exPlanations analysis identified LLM-extracted features like Chronic Pain, Liver Disease, and Major Depression as key predictors. We also developed an interactive web tool for real-time clinical use.

Discussion

Features extracted using NLP and LLM-assisted methods improved model accuracy and interpretability, revealing valuable psychosocial risks not captured in structured EHRs.

Conclusion

Combining structured EHR data with LLM-extracted features moderately improves BUP-NAL retention prediction, enabling personalized risk stratification and advancing AI-driven care for substance use disorders.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningOpioid-Related DisordersNatural Language ProcessingBuprenorphine, Naloxone Drug CombinationOpiate Substitution TreatmentNarcotic AntagonistsHumansElectronic Health RecordsFemaleMaleROC CurveAdultLogistic ModelsBuprenorphine

Stichwörter der Autor:innen

electronic health recordslarge language modelsmachine learningnatural language processingopioid use disorderpredictive modelingtreatment attrition

Substanzen

Buprenorphine, Naloxone Drug CombinationNarcotic AntagonistsBuprenorphine

Autor:innen und Institutionen

  • Stanford Center for Biomedical Informatics Research, Stanford, CA 94305, United States.
  • Stanford University School of Medicine, Stanford, CA 94305, United States.
  • Department of Biomedical Data Science, Stanford, CA 94305, United States.
  • Department of Psychiatry & Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94305, United States.
  • Department of Electrical Engineering (ESAT), STADIUS Center, KU Leuven, 3001 Leuven, Belgium.
  • KKT Technologies, Pte. Ltd., 139951, Singapore.

ORCID-Profile: Ivan Lopez, Tina Hernandez-Boussard

Förderung

  • National Institute of Allergy and Infectious Diseases
  • Google, Inc.
  • Betty Moore Foundation
  • Human-Centered Artificial Intelligence
  • NIH
  • NIH/National Institute on Drug Abuse Clinical Trials Network
  • Stanford Artificial Intelligence in Medicine and Imaging-Human-Centered Artificial Intelligence (AIMI-HAI) Partnership
  • NIH/National Institute of Allergy and Infectious Diseases

Zitieren

APA

Nateghi Haredasht, F., Lopez, I., Tate, S., Ashtari, P., Chan, M. M., Kulkarni, D., Chen, C. A., Vangala, M., Griffith, K., Bunning, B., Miner, A. S., Hernandez-Boussard, T., Humphreys, K., Lembke, A., Vance, L. A., & Chen, J. H. (2025). Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features. Journal of the American Medical Informatics Association, 32(12), 1865-1876. https://doi.org/10.1093/jamia/ocaf157

Vancouver

Nateghi Haredasht F, Lopez I, Tate S, Ashtari P, Chan MM, Kulkarni D, et al. Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features. Journal of the American Medical Informatics Association. 2025;32(12):1865-1876. doi:10.1093/jamia/ocaf157

BibTeX
@article{NateghiHaredasht2025,
  author = {Nateghi Haredasht, Fateme and Lopez, Ivan and Tate, Steven and Ashtari, Pooya and Chan, Min Min and Kulkarni, Deepali and Chen, Chwen-Yuen Angie and Vangala, Maithri and Griffith, Kira and Bunning, Bryan and Miner, Adam S and Hernandez-Boussard, Tina and Humphreys, Keith and Lembke, Anna and Vance, L Alexander and Chen, Jonathan H},
  title = {Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features},
  journal = {Journal of the American Medical Informatics Association},
  year = {2025},
  volume = {32},
  number = {12},
  pages = {1865--1876},
  doi = {10.1093/jamia/ocaf157}
}
Kostenlos starten →