EUVIMEDEUVIMED entdecken
Systematische ÜbersichtOpen Access

Empathic Conversational Agent Platform Designs and Their Evaluation in the Context of Mental Health: Systematic Review

Ruvini Sanjeewa iD, Ravi Iyer iD, Pragalathan Apputhurai iD, Nilmini Wickramasinghe iD, Denny Meyer iD

JMIR Mental Health · 09.09.2024 · 11 · e58974

DOI: 10.2196/58974 ↗

Kurz gefasst

Systematische Übersicht in der Zeitschrift JMIR Mental Health, online erschienen am 09.09.2024. Verfasst von Ruvini Sanjeewa und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Health Sciences, Swinburne University of Technology, Hawthorn, Australia. Hauptthemen laut MEDLINE: Empathy, Mental Health Services. Die Arbeit wurde bisher 65-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The demand for mental health (MH) services in the community continues to exceed supply. At the same time, technological developments make the use of artificial intelligence–empowered conversational agents (CAs) a real possibility to help fill this gap.

Objective

The objective of this review was to identify existing empathic CA design architectures within the MH care sector and to assess their technical performance in detecting and responding to user emotions in terms of classification accuracy. In addition, the approaches used to evaluate empathic CAs within the MH care sector in terms of their acceptability to users were considered. Finally, this review aimed to identify limitations and future directions for empathic CAs in MH care.

Methods

A systematic literature search was conducted across 6 academic databases to identify journal articles and conference proceedings using search terms covering 3 topics: “conversational agents,” “mental health,” and “empathy.” Only studies discussing CA interventions for the MH care domain were eligible for this review, with both textual and vocal characteristics considered as possible data inputs. Quality was assessed using appropriate risk of bias and quality tools.

Results

A total of 19 articles met all inclusion criteria. Most (12/19, 63%) of these empathic CA designs in MH care were machine learning (ML) based, with 26% (5/19) hybrid engines and 11% (2/19) rule-based systems. Among the ML-based CAs, 47% (9/19) used neural networks, with transformer-based architectures being well represented (7/19, 37%). The remaining 16% (3/19) of the ML models were unspecified. Technical assessments of these CAs focused on response accuracies and their ability to recognize, predict, and classify user emotions. While single-engine CAs demonstrated good accuracy, the hybrid engines achieved higher accuracy and provided more nuanced responses. Of the 19 studies, human evaluations were conducted in 16 (84%), with only 5 (26%) focusing directly on the CA’s empathic features. All these papers used self-reports for measuring empathy, including single or multiple (scale) ratings or qualitative feedback from in-depth interviews. Only 1 (5%) paper included evaluations by both CA users and experts, adding more value to the process.

Conclusions

The integration of CA design and its evaluation is crucial to produce empathic CAs. Future studies should focus on using a clear definition of empathy and standardized scales for empathy measurement, ideally including expert assessment. In addition, the diversity in measures used for technical assessment and evaluation poses a challenge for comparing CA performances, which future research should also address. However, CAs with good technical and empathic performance are already available to users of MH care services, showing promise for new applications, such as helpline services.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

EmpathyMental Health ServicesHumansArtificial Intelligence

Stichwörter der Autor:innen

chatbotsconversational agentsemotionally awareempathymental healthmental well-beingvirtual assistants

Autor:innen und Institutionen

  • School of Health Sciences, Swinburne University of Technology, Hawthorn, Australia.
  • School of Computing, Engineering and Mathematical Sciences, La Trobe University, Bundoora, Australia.

ORCID-Profile: Ruvini Sanjeewa, Ravi Iyer, Pragalathan Apputhurai, Nilmini Wickramasinghe, Denny Meyer

Zitieren

APA

Sanjeewa, R., Iyer, R., Apputhurai, P., Wickramasinghe, N., & Meyer, D. (2024). Empathic Conversational Agent Platform Designs and Their Evaluation in the Context of Mental Health: Systematic Review. JMIR Mental Health, 11, e58974. https://doi.org/10.2196/58974

Vancouver

Sanjeewa R, Iyer R, Apputhurai P, Wickramasinghe N, Meyer D. Empathic Conversational Agent Platform Designs and Their Evaluation in the Context of Mental Health: Systematic Review. JMIR Mental Health. 2024;11:e58974. doi:10.2196/58974

BibTeX
@article{Sanjeewa2024,
  author = {Sanjeewa, Ruvini and Iyer, Ravi and Apputhurai, Pragalathan and Wickramasinghe, Nilmini and Meyer, Denny},
  title = {Empathic Conversational Agent Platform Designs and Their Evaluation in the Context of Mental Health: Systematic Review},
  journal = {JMIR Mental Health},
  year = {2024},
  volume = {11},
  pages = {e58974},
  doi = {10.2196/58974}
}
Kostenlos starten →