EUVIMEDObjevte EUVIMED
Systematische ÜbersichtMetaanalyseOpen access

The Efficacy of Rule-Based Versus Large Language Model-Based Chatbots in Alleviating Symptoms of Depression and Anxiety: Systematic Review and Meta-Analysis

Die Wirksamkeit regelbasierter versus auf Large Language Models basierender Chatbots zur Linderung von Depressions- und Angstsymptomen: systematische Übersicht und Metaanalyse

Qiuxue Du iD, Yongliang Ren iD, Ze-Long Meng iD, Han He iD, Shasha Meng iD

Journal of Medical Internet Research · 2025-12-04 · 27 · e78186-e78186

DOI: 10.2196/78186 ↗

Auf einen Blick

15 Studien zeigen: Regelbasierte Chatbots hatten einen kleinen signifikanten Effekt auf Depressionssymptome, LLM-basierte Chatbots nicht. Für Angst wurden keine signifikanten Effekte berichtet.

Fragestellung

Wie unterscheiden sich LLM-basierte und regelbasierte Chatbots in ihrer Wirksamkeit gegen Depressions- und Angstsymptome?

Methode

Systematische Suche in 7 Datenbanken; 15 Studien von 2020 bis 2025. RVE und random-effects-Modell mit REML; Publikationsbias mit Egger-Test, Funnel Plot und fail-safe N.

Ergebnisse

Depression: regelbasiert g=0,266 (95 %-KI 0,020–0,512; P=.04), LLM-basiert g=0,407 (95 %-KI −0,734 bis 1,550; P=.17). Angst: regelbasiert g=0,147 (95 %-KI −0,073 bis 0,367; P=.15), LLM-basiert g=0,711 (95 %-KI −0,334 bis 1,760; P=.13).

Einschränkungen

Kleine Stichprobe; robuste Evidenz für LLM-basierte Chatbot-Interventionen fehlt.

Schlussfolgerung der Autor:innen

Regelbasierte Chatbots haben einen moderaten Effekt auf depressive Symptome und eignen sich für Umgebungen mit knappen psychologischen Ressourcen; 4–8 Wochen könnten ein kritisches Zeitfenster sein.

Kernaussagen

  • Regelbasierte Chatbots verbesserten depressive Symptome signifikant.
  • LLM-basierte Chatbots zeigten keine signifikanten Effekte.
  • Interventionsdauer und Alter beeinflussten die Wirksamkeit nicht signifikant.

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Systematische Übersicht in Journal of Medical Internet Research, published online on 2025-12-04. Written by Qiuxue Du and 4 co-authors. Institution: Beijing Yuxin Technology Co., Ltd, Room 2-5, 13th Floor, Building 2, No. 48, Zhichun Road, Haidian District, Beijing, 100086, China, 86 010-81377053. Major MEDLINE topics: Depression, Anxiety, Language. Cited 25 times so far (Crossref); 82 references. The full text is freely available in PubMed Central.

Auf einen BlickAbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

The global mental health crisis is becoming increasingly severe. Due to the shortage of mental health professionals, high treatment costs, and insufficient accessibility of services, there is an urgent need for scalable and low-cost intervention methods. In recent years, chatbots have shown potential for psychological interventions. The efficacy differences between large language model (LLM)-based and rule-based chatbots have not been systematically evaluated, with few studies directly comparing the two; existing meta-analyses have notable limitations: there is high heterogeneity in intervention design (eg, dialogue structure, interaction frequency, and duration) across studies, and there is a lack of direct comparison of differentiated intervention effects on depressive and anxiety symptoms, making it difficult to integrate conclusions.

Objective

By integrating studies from the past five years, this research evaluates the differences in effectiveness between LLM-based and rule-based chatbots in alleviating depressive and anxiety symptoms. It also analyzes the impacts of control group type, intervention duration, and age on intervention outcomes. By analyzing chatbot functionality, the study aims to provide evidence-based technological pathway options and optimization recommendations for differentiated interventions for depression and anxiety.

Methods

A systematic search of 7 databases included 15 studies published between 2020 and 2025. Robust variance estimation (RVE) was used to account for non-independent effect sizes, and standardized mean differences (SMDs) were calculated using Hedges g . Based on the expectation of clinical and methodological heterogeneity among studies, a random-effects model was preselected, and the pooled effect size was estimated using restricted maximum likelihood estimation (REML) and interpreted according to Cohen criteria. Publication bias was assessed using the RVE-adjusted Egger test, funnel plot asymmetry, and a fail-safe N.

Results

For depression, rule-based intervention achieved a small but significant effect ( g =0.266; 95% CI 0.020-0.512; P =.04), while LLM-based intervention showed a nonsignificant effect with wide confidence intervals ( g =0.407; 95% CI −0.734 to 1.550; P =.17). For anxiety, rule-based intervention did not yield a significant effect ( g =0.147; 95% CI −0.073 to 0.367; P =.15). Similarly, LLM-based intervention showed a higher point estimate but also with nonsignificance and wide confidence intervals ( g =0.711; 95% CI −0.334 to 1.760; P =.13). Subgroup analysis showed that the rule-based chatbot was more effective than the blank control for depression, with the greatest effect in the medium term (4‐8 weeks).

Conclusions

Rule-based chatbots have a modest effect on improving depressive symptoms and are suitable for environments with limited psychological resources; 4‐8 weeks may be a critical intervention window. Intervention duration and participant age did not significantly influence intervention effectiveness. Limited by the sample size, robust evidence supporting the effectiveness of LLM-based chatbot interventions is lacking, and further sample size expansion is warranted.

Subjects

MeSH terms (MEDLINE)

DepressionAnxietyLanguageHumansLarge Language ModelsGenerative Artificial Intelligence

Author keywords

PRISMAPreferred Reporting Items for Systematic Reviews and Meta-Analysesanxietychatbotsdepressionlarge language modelsmental health

Authors and institutions

  • Beijing Yuxin Technology Co., Ltd, Room 2-5, 13th Floor, Building 2, No. 48, Zhichun Road, Haidian District, Beijing, 100086, China, 86 010-81377053.
  • Department of Psychology, School of Humanities and Social Sciences, Beijing Forestry University, Beijing, China.

ORCID profiles: Qiuxue Du, Yongliang Ren, Ze-Long Meng, Han He, Shasha Meng

Cite

APA

Du, Q., Ren, Y., Meng, Z. L., He, H., & Meng, S. (2025). The Efficacy of Rule-Based Versus Large Language Model-Based Chatbots in Alleviating Symptoms of Depression and Anxiety: Systematic Review and Meta-Analysis. Journal of Medical Internet Research, 27, e78186-e78186. https://doi.org/10.2196/78186

Vancouver

Du Q, Ren Y, Meng ZL, He H, Meng S. The Efficacy of Rule-Based Versus Large Language Model-Based Chatbots in Alleviating Symptoms of Depression and Anxiety: Systematic Review and Meta-Analysis. Journal of Medical Internet Research. 2025;27:e78186-e78186. doi:10.2196/78186

BibTeX
@article{Du2025,
  author = {Du, Qiuxue and Ren, Yongliang and Meng, Ze-Long and He, Han and Meng, Shasha},
  title = {The Efficacy of Rule-Based Versus Large Language Model-Based Chatbots in Alleviating Symptoms of Depression and Anxiety: Systematic Review and Meta-Analysis},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e78186--e78186},
  doi = {10.2196/78186}
}
Start for free →