EUVIMEDScopri EUVIMED
ÜbersichtsarbeitSystematische ÜbersichtOpen access

Large Language Model–Based Chatbots and Agentic AI for Mental Health Counseling: Systematic Review of Methodologies, Evaluation Frameworks, and Ethical Safeguards

Large-Language-Model-basierte Chatbots und agentische KI für die psychische Gesundheitsberatung: systematische Übersichtsarbeit zu Methodik, Bewertungsrahmen und ethischen Schutzmaßnahmen

Ha Na Cho iD, Jiayuan Wang iD, Di Hu iD, Kai Zheng iD

JMIR AI · 2026-03-13 · 5 · e80348

DOI: 10.2196/80348 ↗

Auf einen Blick

20 Studien zu LLM-basierten Chatbots für die psychische Gesundheitsberatung zeigen heterogene Methodik und häufige Nutzung von Bewertungsverfahren. Externe Validierung sowie Angaben zu Ethik und Sicherheit waren oft unzureichend.

Fragestellung

Welche Methodik, Bewertungspraktiken und ethischen bzw. Governance-Rahmen wurden bei LLM-basierten Chatbots für die psychische Gesundheitsberatung verwendet, und wo bestehen Lücken?

Methode

Systematische Übersicht; Suche in Google Scholar, PubMed, IEEE Xplore und ACM Digital Library für Studien von Januar 2020 bis Mai 2025; narrative Synthese nach PRISMA 2020; Qualitätsbewertung mit Ampelrahmen in 5 Domänen.

Ergebnisse

20 Studien. GPT-basierte Modelle in 45 % (9/20), fein abgestimmte oder domänenspezifische Modelle in 90 % (18/20). Qualitative Bewertungen 65 % (13/20), quantitative Sprachmetriken 90 % (18/20).

Einschränkungen

Recurrende Lücken bei externer Validierung sowie bei der Berichterstattung zu Ethik und Sicherheit; keine registrierten RCTs oder unabhängige klinische Validierung in realen Versorgungssituationen.

Schlussfolgerung der Autor:innen

LLM-basierte Chatbots für die psychische Gesundheitsberatung erscheinen vielversprechend, die Evidenz ist jedoch durch heterogene Designs, geringe externe Validierung und inkonsistente Angaben zu Sicherheit und Governance begrenzt.

Kernaussagen

  • 20 Studien wurden eingeschlossen
  • Externe Validierung und Sicherheitsangaben waren häufig unzureichend
  • Keine registrierten RCTs oder unabhängige klinische Validierung in der Versorgung

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Übersichtsarbeit in JMIR AI, published online on 2026-03-13. Written by Ha Na Cho and 3 co-authors. Institution: Department of Informatics, University of California, Irvine, Irvine, CA, United States. Cited 22 times so far (Crossref); 45 references. The full text is freely available in PubMed Central.

Auf einen BlickAbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Background

Large language model (LLM)–based chatbots have rapidly emerged as tools for digital mental health (MH) counseling. However, evidence on their methodological quality, evaluation rigor, and ethical safeguards remains fragmented, limiting interpretation of clinical readiness and deployment safety.

Objective

This systematic review aimed to synthesize the methodologies, evaluation practices, and ethical or governance frameworks of LLM-based chatbots developed for MH counseling and to identify gaps affecting validity, reproducibility, and translation.

Methods

We searched Google Scholar, PubMed, IEEE Xplore, and ACM Digital Library for studies published between January 2020 and May 2025. Eligible studies reported original development or empirical evaluation of LLM-driven MH counseling chatbots. We excluded studies that did not involve LLM-based conversational agents, were not focused on counseling or supportive MH communication, or lacked evaluable system outputs or outcomes. Screening and data extraction were conducted in Covidence (Veritas Health Innovation) following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines. Study quality was appraised using a structured traffic light framework across 5 methodological domains (design, dataset reporting, evaluation metrics, external validation, and ethics), with an overall judgment derived across domains. We used narrative synthesis with descriptive aggregation to summarize methodological trends, evaluation metrics, and governance considerations.

Results

Twenty studies met the inclusion criteria. GPT-based models (GPT-2/3/4) were used in 45% (9/20) of studies, while 90% (18/20) used fine-tuned or domain-adaptation models such as LLaMa, ChatGLM, or Qwen. Reported deployment types were not mutually exclusive; standalone apps were most common (18/20, 90%), and some systems were also implemented as virtual agents (4/20, 20%) or delivered via existing platforms (2/20, 10%). Evaluation approaches were frequently mixed, with qualitative assessment (13/20, 65%), such as thematic analysis or rubric-based scoring, often complemented by quantitative language metrics (18/20, 90%), including BLEU (Bilingual Evaluation Understudy), ROUGE (Recall-Oriented Understudy for Gisting Evaluation), or perplexity. Quality appraisal indicated consistently low risk for dataset reporting and evaluation metrics, but recurring limitations were observed in external validation and reporting on ethics and safety, including incomplete documentation of safety safeguards and governance practices. No included study reported registered randomized controlled trials or independent clinical validation in real-world care settings.

Conclusions

LLM-based MH counseling chatbots show promise for scalable and personalized support, but current evidence is limited by heterogeneous study designs, minimal external validation, and inconsistent reporting of safety and governance practices. Future work should prioritize clinically grounded evaluation frameworks, transparent reporting of model and prompt configurations, and stronger validation using standardized outcomes to support safe, reliable, and regulatory-ready deployment.

Subjects

Author keywords

conversational agentdigital healthdigital mental health interventionlarge language model chatbotspersonalized health care

Authors and institutions

  • Department of Informatics, University of California, Irvine, Irvine, CA, United States.

ORCID profiles: Ha Na Cho, Jiayuan Wang, Di Hu, Kai Zheng

Cite

APA

Cho, H. N., Wang, J., Hu, D., & Zheng, K. (2026). Large Language Model–Based Chatbots and Agentic AI for Mental Health Counseling: Systematic Review of Methodologies, Evaluation Frameworks, and Ethical Safeguards. JMIR AI, 5, e80348. https://doi.org/10.2196/80348

Vancouver

Cho HN, Wang J, Hu D, Zheng K. Large Language Model–Based Chatbots and Agentic AI for Mental Health Counseling: Systematic Review of Methodologies, Evaluation Frameworks, and Ethical Safeguards. JMIR AI. 2026;5:e80348. doi:10.2196/80348

BibTeX
@article{Cho2026,
  author = {Cho, Ha Na and Wang, Jiayuan and Hu, Di and Zheng, Kai},
  title = {Large Language Model–Based Chatbots and Agentic AI for Mental Health Counseling: Systematic Review of Methodologies, Evaluation Frameworks, and Ethical Safeguards},
  journal = {JMIR AI},
  year = {2026},
  volume = {5},
  pages = {e80348},
  doi = {10.2196/80348}
}
Start for free →