EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Potential of ChatGPT in youth mental health emergency triage: Comparative analysis with clinicians

Samanvith Thotapalli iD, Musa Yilanli iD, Ian McKay, William Leever, Eric Youngstrom, Karah Harvey-Nuckles, Kimberly Lowder, Steffanie Schweitzer, Erin Sunderland, Daniel I Jackson, Emre Sezgin

Psychiatry and Clinical Neurosciences Reports · 2025-09 · 4 (3)

DOI: 10.1002/pcn5.70159 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Psychiatry and Clinical Neurosciences Reports, online erschienen am 15.07.2025. Verfasst von Samanvith Thotapalli und 10 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychiatry and Behavioral Health The Ohio State University Columbus Ohio USA. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 61 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aim

Large language models, such as GPT‐4, are increasingly integrated into healthcare to support clinicians in making informed decisions. Given ChatGPT's potential, it is necessary to explore such applications as a support tool, particularly within mental health telephone triage services. This study evaluates whether GPT Models can accurately triage psychiatric emergency vignettes and compares its performance to that of clinicians.

Methods

A cross‐sectional study was performed to assess the performance of three different GPT‐4 models (GPT‐4o, GPT‐4o Mini, and GPT‐4 Legacy) in psychiatric emergency triage. Twenty‐two psychiatric emergency vignettes, intended to represent realistic prehospital triage scenarios, were initially drafted using ChatGPT and subsequently reviewed and refined by the research team to ensure clinical accuracy and relevance. The GPT‐4 models independently generated clinical responses to the vignettes over three iterations to ensure consistency. Thereafter, two advanced practice nurse practitioners independently assessed these responses utilizing a 3‐point Likert‐type scale for the main triage criteria: risk level ( Low = 1 to High = 3), necessity of hospital admission ( Ye s = 1; No = 2), and urgency of clinical evaluation ( Low = 1 to High = 3). Additionally, the nurse practitioners provided their clinical judgments independently for each vignette. Interrater reliability was evaluated by comparing responses generated by the GPT Models with the independent clinical assessments of nurse practitioners, and agreement was evaluated using Cohen's Kappa. The clinical expert committee ( n = 3) conducted qualitative analyses of responses from both GPT Models using a systematic coding method to evaluate triage accuracy, clarity, completeness, and total score. The evaluation of responses focused on three key triage criteria: risk ( Low = 1 to High = 3), admission necessity ( Ye s = 1; No = 2), and urgency of clinical evaluation ( Low = 1 to High = 3).

Results

GPT Models had an average admission score of 1.73 (standard deviation [SD] = 0.45; scale: Yes = 1, No = 2), indicating a general trend toward recommending against hospital admission. Risk (mean = 2.12, SD = 0.83) and urgency (mean = 2.27, SD = 0.44) assessments suggested moderate‐to‐high perceived risk and urgency (scale: Low = 1, High = 3), reflecting conservative decision‐making. Interrater reliability between clinicians and GPT‐4 models was substantial, with Cohen's Kappa values of 0.77 (admission), 0.78 (risk), and 0.76 (urgency). GPT Models’ responses tended toward slight over‐triage, indicated by four false‐positive admission recommendations and zero false negatives. Substantial interrater reliability was observed between clinicians and GPT‐4 responses across the three triage criteria (Cohen's Kappa: admission = 0.77; risk = 0.78; urgency = 0.76).The mean scores for triage criteria responses between GPT‐4 models and clinicians exhibited consistent patterns with minimal variability. Overall, GPT Models had a tendency to over‐triage patients as indicated by four total false positives and zero false negatives for admissions.

Conclusion

This study indicates that GPT Models may serve as supportive decision‐support tools in mental health telephone triage, particularly for psychiatric emergencies. Although response variability across iterations was minimal, most discrepancies in admission decisions were identified as false positives, reflecting that GPT Models may have a tendency to over‐triage relative to clinician judgment. Further investigation is needed to establish robust structure to increase alignment with clinical decisions and response relevance in clinical practice.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligenceemergencymental healthsuicidetriage

Autor:innen und Institutionen

  • Department of Psychiatry and Behavioral Health The Ohio State University Columbus Ohio USA.
  • Nationwide Children's Hospital Columbus Ohio USA.
  • The Abigail Wexner Research Institute, Nationwide Children's Hospital Columbus Ohio USA.

ORCID-Profile: Samanvith Thotapalli, Musa Yilanli

Zitieren

APA

Thotapalli, S., Yilanli, M., McKay, I., Leever, W., Youngstrom, E., Harvey-Nuckles, K., Lowder, K., Schweitzer, S., Sunderland, E., Jackson, D. I., & Sezgin, E. (2025). Potential of ChatGPT in youth mental health emergency triage: Comparative analysis with clinicians. Psychiatry and Clinical Neurosciences Reports, 4(3). https://doi.org/10.1002/pcn5.70159

Vancouver

Thotapalli S, Yilanli M, McKay I, Leever W, Youngstrom E, Harvey-Nuckles K, et al. Potential of ChatGPT in youth mental health emergency triage: Comparative analysis with clinicians. Psychiatry and Clinical Neurosciences Reports. 2025;4(3). doi:10.1002/pcn5.70159

BibTeX
@article{Thotapalli2025,
  author = {Thotapalli, Samanvith and Yilanli, Musa and McKay, Ian and Leever, William and Youngstrom, Eric and Harvey-Nuckles, Karah and Lowder, Kimberly and Schweitzer, Steffanie and Sunderland, Erin and Jackson, Daniel I and Sezgin, Emre},
  title = {Potential of ChatGPT in youth mental health emergency triage: Comparative analysis with clinicians},
  journal = {Psychiatry and Clinical Neurosciences Reports},
  year = {2025},
  volume = {4},
  number = {3},
  doi = {10.1002/pcn5.70159}
}
Kostenlos starten →