EUVIMEDEUVIMED entdecken
Open Access

From benchmark to bedside: transfer learning from social media to patient-provider text messages for suicide risk prediction

Hannah A Burkhardt iD, Xiruo Ding, Amanda Kerbrat, Katherine Anne Comtois, Trevor Cohen

Journal of the American Medical Informatics Association · 19.05.2023 · 30 (6) · 1068-1078

DOI: 10.1093/jamia/ocad062 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 12.04.2023. Verfasst von Hannah A Burkhardt und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, Washington, USA. Hauptthemen laut MEDLINE: Social Media, Text Messaging, Suicide. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NLM NIH HHS, Garvey Institute for Brain Health Solutions Innovation, Informatics-Supported Authorship for Caring.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Compared to natural language processing research investigating suicide risk prediction with social media (SM) data, research utilizing data from clinical settings are scarce. However, the utility of models trained on SM data in text from clinical settings remains unclear. In addition, commonly used performance metrics do not directly translate to operational value in a real-world deployment. The objectives of this study were to evaluate the utility of SM-derived training data for suicide risk prediction in a clinical setting and to develop a metric of the clinical utility of automated triage of patient messages for suicide risk. Materials and Methods Using clinical data, we developed a Bidirectional Encoder Representations from Transformers-based suicide risk detection model to identify messages indicating potential suicide risk. We used both annotated and unlabeled suicide-related SM posts for multi-stage transfer learning, leveraging customized contemporary learning rate schedules. We also developed a novel metric estimating predictive models’ potential to reduce follow-up delays with patients in distress and used it to assess model utility.

Results

Multi-stage transfer learning from SM data outperformed baseline approaches by traditional classification performance metrics, improving performance from 0.734 to a best F1 score of 0.797. Using this approach for automated triage could reduce response times by 15 minutes per urgent message.

Discussion

Despite differences in data characteristics and distribution, publicly available SM data benefit clinical suicide risk prediction when used in conjunction with contemporary transfer learning techniques. Estimates of time saved due to automated triage indicate the potential for the practical impact of such models when deployed as part of established suicide prevention interventions.

Conclusions

This work demonstrates a pathway for leveraging publicly available SM data toward improving risk assessment, paving the way for better clinical care and improved clinical outcomes.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Social MediaText MessagingSuicideHumansBenchmarkingMachine Learning

Stichwörter der Autor:innen

artificial intelligencecomputer assisteddecision-makingdelivery of health carenatural language processingsocial mediasuicide prevention

Autor:innen und Institutionen

  • Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, Washington, USA.
  • Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, Washington, USA.

ORCID-Profile: Hannah A Burkhardt

Förderung

  • NLM NIH HHS
  • Garvey Institute for Brain Health Solutions Innovation
  • Informatics-Supported Authorship for Caring
  • Military Suicide Research Consortiu
  • Office of the Assistant Secretary of Defense for Health Affairs
  • Department of Defense
  • Military Suicide Research Consortium

Zitieren

APA

Burkhardt, H. A., Ding, X., Kerbrat, A., Comtois, K. A., & Cohen, T. (2023). From benchmark to bedside: transfer learning from social media to patient-provider text messages for suicide risk prediction. Journal of the American Medical Informatics Association, 30(6), 1068-1078. https://doi.org/10.1093/jamia/ocad062

Vancouver

Burkhardt HA, Ding X, Kerbrat A, Comtois KA, Cohen T. From benchmark to bedside: transfer learning from social media to patient-provider text messages for suicide risk prediction. Journal of the American Medical Informatics Association. 2023;30(6):1068-1078. doi:10.1093/jamia/ocad062

BibTeX
@article{Burkhardt2023,
  author = {Burkhardt, Hannah A and Ding, Xiruo and Kerbrat, Amanda and Comtois, Katherine Anne and Cohen, Trevor},
  title = {From benchmark to bedside: transfer learning from social media to patient-provider text messages for suicide risk prediction},
  journal = {Journal of the American Medical Informatics Association},
  year = {2023},
  volume = {30},
  number = {6},
  pages = {1068--1078},
  doi = {10.1093/jamia/ocad062}
}
Kostenlos starten →