EUVIMEDEUVIMED entdecken
Open Access

Machine Learning–Based Evaluation of Suicide Risk Assessment in Crisis Counseling Calls

Zac E Imel, Brian Pace, Brad Pendergraft, Jordan Pruett, Michael Tanana, Christina S Soma, Kate A Comtois, David C Atkins

Psychiatric Services · 01.11.2024 · 75 (11) · 1068-1074

DOI: 10.1176/appi.ps.20230648 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Psychiatric Services, erschienen am 01.11.2024. Verfasst von Zac E Imel und 7 weiteren Autor:innen. Beteiligte Einrichtung: Lyssn.io, Seattle (Imel, Pace, Pruett, Tanana, Soma, Atkins); Protocall Services, Portland, Oregon (Pendergraft); Harborview Medical Center, Universi…. Hauptthemen laut MEDLINE: Machine Learning, Counseling. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIMH NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

OBJECTIVE: Counselor assessment of suicide risk is one key component of crisis counseling, and standards require risk assessment in every crisis counseling conversation. Efforts to increase risk assessment frequency are limited by quality improvement tools that rely on human evaluation of conversations, which is labor intensive, slow, and impossible to scale. Advances in machine learning (ML) have made possible the development of tools that can automatically and immediately detect the presence of risk assessment in crisis counseling conversations. METHODS: To train models, a coding team labeled every statement in 476 crisis counseling calls (193,257 statements) for a core element of risk assessment. The authors then fine-tuned a transformer-based ML model with the labeled data, utilizing separate training, validation, and test data sets. RESULTS: Generally, the evaluated ML model was highly consistent with human raters. For detecting any risk assessment, ML model agreement with human ratings was 98% of human interrater agreement. Across specific labels, average F1 (the harmonic mean of precision and recall) was 0.86 at the call level and 0.66 at the statement level and often varied as a result of a low base rate for some risk labels. CONCLUSIONS: ML models can reliably detect the presence of suicide risk assessment in crisis counseling conversations, presenting an opportunity to scale quality improvement efforts.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningCounselingHumansRisk AssessmentSuicide PreventionCrisis InterventionSuicide

Stichwörter der Autor:innen

AIcrisis counselingmachine learningquality assurancesuicide

Autor:innen und Institutionen

  • Lyssn.io, Seattle (Imel, Pace, Pruett, Tanana, Soma, Atkins); Protocall Services, Portland, Oregon (Pendergraft); Harborview Medical Center, University of Washington, Seattle (Comtois).

Förderung

  • NIMH NIH HHS

Zitieren

APA

Imel, Z. E., Pace, B., Pendergraft, B., Pruett, J., Tanana, M., Soma, C. S., Comtois, K. A., & Atkins, D. C. (2024). Machine Learning–Based Evaluation of Suicide Risk Assessment in Crisis Counseling Calls. Psychiatric Services, 75(11), 1068-1074. https://doi.org/10.1176/appi.ps.20230648

Vancouver

Imel ZE, Pace B, Pendergraft B, Pruett J, Tanana M, Soma CS, et al. Machine Learning–Based Evaluation of Suicide Risk Assessment in Crisis Counseling Calls. Psychiatric Services. 2024;75(11):1068-1074. doi:10.1176/appi.ps.20230648

BibTeX
@article{Imel2024,
  author = {Imel, Zac E and Pace, Brian and Pendergraft, Brad and Pruett, Jordan and Tanana, Michael and Soma, Christina S and Comtois, Kate A and Atkins, David C},
  title = {Machine Learning–Based Evaluation of Suicide Risk Assessment in Crisis Counseling Calls},
  journal = {Psychiatric Services},
  year = {2024},
  volume = {75},
  number = {11},
  pages = {1068--1074},
  doi = {10.1176/appi.ps.20230648}
}
Kostenlos starten →