Large Language Model–Based Chatbots and Agentic AI for Mental Health Counseling: Systematic Review of Methodologies, Evaluation Frameworks, and Ethical Safeguards
Large-Language-Model-basierte Chatbots und agentische KI für die psychische Gesundheitsberatung: systematische Übersichtsarbeit zu Methodik, Bewertungsrahmen und ethischen Schutzmaßnahmen
JMIR AI · 2026-03-13 · 5 · e80348
DOI: 10.2196/80348 ↗Auf einen Blick
20 Studien zu LLM-basierten Chatbots für die psychische Gesundheitsberatung zeigen heterogene Methodik und häufige Nutzung von Bewertungsverfahren. Externe Validierung sowie Angaben zu Ethik und Sicherheit waren oft unzureichend.
Fragestellung
Welche Methodik, Bewertungspraktiken und ethischen bzw. Governance-Rahmen wurden bei LLM-basierten Chatbots für die psychische Gesundheitsberatung verwendet, und wo bestehen Lücken?
Methode
Systematische Übersicht; Suche in Google Scholar, PubMed, IEEE Xplore und ACM Digital Library für Studien von Januar 2020 bis Mai 2025; narrative Synthese nach PRISMA 2020; Qualitätsbewertung mit Ampelrahmen in 5 Domänen.
Ergebnisse
20 Studien. GPT-basierte Modelle in 45 % (9/20), fein abgestimmte oder domänenspezifische Modelle in 90 % (18/20). Qualitative Bewertungen 65 % (13/20), quantitative Sprachmetriken 90 % (18/20).
Einschränkungen
Recurrende Lücken bei externer Validierung sowie bei der Berichterstattung zu Ethik und Sicherheit; keine registrierten RCTs oder unabhängige klinische Validierung in realen Versorgungssituationen.
Schlussfolgerung der Autor:innen
LLM-basierte Chatbots für die psychische Gesundheitsberatung erscheinen vielversprechend, die Evidenz ist jedoch durch heterogene Designs, geringe externe Validierung und inkonsistente Angaben zu Sicherheit und Governance begrenzt.
Kernaussagen
- 20 Studien wurden eingeschlossen
- Externe Validierung und Sicherheitsangaben waren häufig unzureichend
- Keine registrierten RCTs oder unabhängige klinische Validierung in der Versorgung
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
At a glance
Übersichtsarbeit in JMIR AI, published online on 2026-03-13. Written by Ha Na Cho and 3 co-authors. Institution: Department of Informatics, University of California, Irvine, Irvine, CA, United States. Cited 22 times so far (Crossref); 45 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Large language model (LLM)–based chatbots have rapidly emerged as tools for digital mental health (MH) counseling. However, evidence on their methodological quality, evaluation rigor, and ethical safeguards remains fragmented, limiting interpretation of clinical readiness and deployment safety.
Objective
This systematic review aimed to synthesize the methodologies, evaluation practices, and ethical or governance frameworks of LLM-based chatbots developed for MH counseling and to identify gaps affecting validity, reproducibility, and translation.
Methods
We searched Google Scholar, PubMed, IEEE Xplore, and ACM Digital Library for studies published between January 2020 and May 2025. Eligible studies reported original development or empirical evaluation of LLM-driven MH counseling chatbots. We excluded studies that did not involve LLM-based conversational agents, were not focused on counseling or supportive MH communication, or lacked evaluable system outputs or outcomes. Screening and data extraction were conducted in Covidence (Veritas Health Innovation) following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines. Study quality was appraised using a structured traffic light framework across 5 methodological domains (design, dataset reporting, evaluation metrics, external validation, and ethics), with an overall judgment derived across domains. We used narrative synthesis with descriptive aggregation to summarize methodological trends, evaluation metrics, and governance considerations.
Results
Twenty studies met the inclusion criteria. GPT-based models (GPT-2/3/4) were used in 45% (9/20) of studies, while 90% (18/20) used fine-tuned or domain-adaptation models such as LLaMa, ChatGLM, or Qwen. Reported deployment types were not mutually exclusive; standalone apps were most common (18/20, 90%), and some systems were also implemented as virtual agents (4/20, 20%) or delivered via existing platforms (2/20, 10%). Evaluation approaches were frequently mixed, with qualitative assessment (13/20, 65%), such as thematic analysis or rubric-based scoring, often complemented by quantitative language metrics (18/20, 90%), including BLEU (Bilingual Evaluation Understudy), ROUGE (Recall-Oriented Understudy for Gisting Evaluation), or perplexity. Quality appraisal indicated consistently low risk for dataset reporting and evaluation metrics, but recurring limitations were observed in external validation and reporting on ethics and safety, including incomplete documentation of safety safeguards and governance practices. No included study reported registered randomized controlled trials or independent clinical validation in real-world care settings.
Conclusions
LLM-based MH counseling chatbots show promise for scalable and personalized support, but current evidence is limited by heterogeneous study designs, minimal external validation, and inconsistent reporting of safety and governance practices. Future work should prioritize clinically grounded evaluation frameworks, transparent reporting of model and prompt configurations, and stronger validation using standardized outcomes to support safe, reliable, and regulatory-ready deployment.
Subjects
Author keywords
Authors and institutions
- Department of Informatics, University of California, Irvine, Irvine, CA, United States.
ORCID profiles: Ha Na Cho, Jiayuan Wang, Di Hu, Kai Zheng
Cite
APA
Cho, H. N., Wang, J., Hu, D., & Zheng, K. (2026). Large Language Model–Based Chatbots and Agentic AI for Mental Health Counseling: Systematic Review of Methodologies, Evaluation Frameworks, and Ethical Safeguards. JMIR AI, 5, e80348. https://doi.org/10.2196/80348
Vancouver
Cho HN, Wang J, Hu D, Zheng K. Large Language Model–Based Chatbots and Agentic AI for Mental Health Counseling: Systematic Review of Methodologies, Evaluation Frameworks, and Ethical Safeguards. JMIR AI. 2026;5:e80348. doi:10.2196/80348
BibTeX
@article{Cho2026,
author = {Cho, Ha Na and Wang, Jiayuan and Hu, Di and Zheng, Kai},
title = {Large Language Model–Based Chatbots and Agentic AI for Mental Health Counseling: Systematic Review of Methodologies, Evaluation Frameworks, and Ethical Safeguards},
journal = {JMIR AI},
year = {2026},
volume = {5},
pages = {e80348},
doi = {10.2196/80348}
}
EUVIMED