Evaluating Diagnostic Accuracy and Treatment Efficacy in Mental Health: A Comparative Analysis of Large Language Model Tools and Mental Health Professionals
European Journal of Investigation in Health, Psychology and Education · 18.01.2025 · 15 (1) · 9
DOI: 10.3390/ejihpe15010009 ↗Kurz gefasst
Fachartikel in der Zeitschrift European Journal of Investigation in Health, Psychology and Education, online erschienen am 18.01.2025. Verfasst von Inbar Levkovich. Beteiligte Einrichtung: Faculty of Education, Tel-Hai Academic College, Upper Galilee 2208, Israel. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Large language models (LLMs) offer promising possibilities in mental health, yet their ability to assess disorders and recommend treatments remains underexplored. This quantitative cross-sectional study evaluated four LLMs (Gemini (Gemini 2.0 Flash Experimental), Claude (Claude 3.5 Sonnet), ChatGPT-3.5, and ChatGPT-4) using text vignettes representing conditions such as depression, suicidal ideation, early and chronic schizophrenia, social phobia, and PTSD. Each model’s diagnostic accuracy, treatment recommendations, and predicted outcomes were compared with norms established by mental health professionals. Findings indicated that for certain conditions, including depression and PTSD, models like ChatGPT-4 achieved higher diagnostic accuracy compared to human professionals. However, in more complex cases, such as early schizophrenia, LLM performance varied, with ChatGPT-4 achieving only 55% accuracy, while other LLMs and professionals performed better. LLMs tended to suggest a broader range of proactive treatments, whereas professionals recommended more targeted psychiatric consultations and specific medications. In terms of outcome predictions, professionals were generally more optimistic regarding full recovery, especially with treatment, while LLMs predicted lower full recovery rates and higher partial recovery rates, particularly in untreated cases. While LLMs recommend a broader treatment range, their conservative recovery predictions, particularly for complex conditions, highlight the need for professional oversight. LLMs provide valuable support in diagnostics and treatment planning but cannot replace professional discretion.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Education, Tel-Hai Academic College, Upper Galilee 2208, Israel.
ORCID-Profile: Inbar Levkovich
Zitieren
APA
Levkovich, I. (2025). Evaluating Diagnostic Accuracy and Treatment Efficacy in Mental Health: A Comparative Analysis of Large Language Model Tools and Mental Health Professionals. European Journal of Investigation in Health, Psychology and Education, 15(1), 9. https://doi.org/10.3390/ejihpe15010009
Vancouver
Levkovich I. Evaluating Diagnostic Accuracy and Treatment Efficacy in Mental Health: A Comparative Analysis of Large Language Model Tools and Mental Health Professionals. European Journal of Investigation in Health, Psychology and Education. 2025;15(1):9. doi:10.3390/ejihpe15010009
BibTeX
@article{Levkovich2025,
author = {Levkovich, Inbar},
title = {Evaluating Diagnostic Accuracy and Treatment Efficacy in Mental Health: A Comparative Analysis of Large Language Model Tools and Mental Health Professionals},
journal = {European Journal of Investigation in Health, Psychology and Education},
year = {2025},
volume = {15},
number = {1},
pages = {9},
doi = {10.3390/ejihpe15010009}
}
EUVIMED