EUVIMEDScopri EUVIMED
Randomisierte kontrollierte StudieOpen access

Usability Comparison Among Healthy Participants of an Anthropomorphic Digital Human and a Text-Based Chatbot as a Responder to Questions on Mental Health: Randomized Controlled Trial

Almira Osmanovic Thunström iD, Hanne Krage Carlsen iD, Lilas Ali iD, Tomas Larson iD, Andreas Hellström iD, Steinn Steingrimsson iD

JMIR Human Factors · 2024

DOI: 10.2196/54581 ↗

At a glance

Randomisierte kontrollierte Studie in JMIR Human Factors, 2024. Written by Almira Osmanovic Thunström and 5 co-authors. Institution: Region Västra Götaland, Psychiatric Department, Sahlgrenska University Hospital, Gothenburg, Sweden. Major MEDLINE topics: User-Computer Interface. Cited 25 times so far (Crossref); 48 references. The full text is freely available in PubMed Central.

AbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Background

The use of chatbots in mental health support has increased exponentially in recent years, with studies showing that they may be effective in treating mental health problems. More recently, the use of visual avatars called digital humans has been introduced. Digital humans have the capability to use facial expressions as another dimension in human-computer interactions. It is important to study the difference in emotional response and usability preferences between text-based chatbots and digital humans for interacting with mental health services.

Objective

This study aims to explore to what extent a digital human interface and a text-only chatbot interface differed in usability when tested by healthy participants, using BETSY (Behavior, Emotion, Therapy System, and You) which uses 2 distinct interfaces: a digital human with anthropomorphic features and a text-only user interface. We also set out to explore how chatbot-generated conversations on mental health (specific to each interface) affected self-reported feelings and biometrics.

Methods

We explored to what extent a digital human with anthropomorphic features differed from a traditional text-only chatbot regarding perception of usability through the System Usability Scale, emotional reactions through electroencephalography, and feelings of closeness. Healthy participants (n=45) were randomized to 2 groups that used a digital human with anthropomorphic features (n=25) or a text-only chatbot with no such features (n=20). The groups were compared by linear regression analysis and t tests.

Results

No differences were observed between the text-only and digital human groups regarding demographic features. The mean System Usability Scale score was 75.34 (SD 10.01; range 57-90) for the text-only chatbot versus 64.80 (SD 14.14; range 40-90) for the digital human interface. Both groups scored their respective chatbot interfaces as average or above average in usability. Women were more likely to report feeling annoyed by BETSY.

Conclusions

The text-only chatbot was perceived as significantly more user-friendly than the digital human, although there were no significant differences in electroencephalography measurements. Male participants exhibited lower levels of annoyance with both interfaces, contrary to previously reported findings.

Subjects

MeSH terms (MEDLINE)

User-Computer InterfaceHumansFemaleMaleAdultHealthy VolunteersMental HealthElectroencephalographyEmotions

Author keywords

AIMLNLPalgorithmalgorithmsartificial intelligencechat-botchat-botschatbotchatbotsdigital healthinterfacemachine learningmental diseasemental diseases

Authors and institutions

  • Region Västra Götaland, Psychiatric Department, Sahlgrenska University Hospital, Gothenburg, Sweden.
  • Section of Psychiatry and Neurochemistry, Institute of Neuroscience and Physiology, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
  • Region Västra Götaland, Centre of Registers, Gothenburg, Sweden.
  • Institute of Health Care Sciences, Centre for Person-Centred Care, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
  • Centre for Person-Centred Care, University of Gothenburg, Gothenburg, Sweden.
  • Department of Technology Management and Economics, Chalmers University of Technology, Gothenburg, Sweden.

ORCID profiles: Almira Osmanovic Thunström, Hanne Krage Carlsen, Lilas Ali, Tomas Larson, Andreas Hellström, Steinn Steingrimsson

Cite

APA

Thunström, A. O., Carlsen, H. K., Ali, L., Larson, T., Hellström, A., & Steingrimsson, S. (2024). Usability Comparison Among Healthy Participants of an Anthropomorphic Digital Human and a Text-Based Chatbot as a Responder to Questions on Mental Health: Randomized Controlled Trial. JMIR Human Factors. https://doi.org/10.2196/54581

Vancouver

Thunström AO, Carlsen HK, Ali L, Larson T, Hellström A, Steingrimsson S. Usability Comparison Among Healthy Participants of an Anthropomorphic Digital Human and a Text-Based Chatbot as a Responder to Questions on Mental Health: Randomized Controlled Trial. JMIR Human Factors. 2024. doi:10.2196/54581

BibTeX
@article{Thunstrm2024,
  author = {Thunström, Almira Osmanovic and Carlsen, Hanne Krage and Ali, Lilas and Larson, Tomas and Hellström, Andreas and Steingrimsson, Steinn},
  title = {Usability Comparison Among Healthy Participants of an Anthropomorphic Digital Human and a Text-Based Chatbot as a Responder to Questions on Mental Health: Randomized Controlled Trial},
  journal = {JMIR Human Factors},
  year = {2024},
  doi = {10.2196/54581}
}
Start for free →