Development and Preliminary Evaluation of a Conversational Agent Delivering Problem-Solving Therapy for Family Caregivers of Children With a Chronic Health Condition: Multiphase Mixed Methods Study
Journal of Medical Internet Research · 2026-10-06 · 28 · e94175-e94175
DOI: 10.2196/94175 ↗At a glance
Mixed-Methods-Studie in Journal of Medical Internet Research, published online on 2026-10-06. Written by Weichao Yuwen and 6 co-authors. Institution: School of Nursing & Healthcare Leadership, University of Washington Tacoma, 1900 Commerce Street, Campus Box 358421, Tacoma, WA, 98402, United States. Major MEDLINE topics: Caregivers, Problem Solving, Communication. 55 references. The full text is freely available in PubMed Central. Funded by: NIH HHS, NINR NIH HHS.
Topics: Pflegende Angehörige, Kinderkrankenpflege
Abstract (original language)
Background
Family caregivers of children with chronic health conditions experience substantial physical and mental health burdens, including burnout, anxiety, depression, fatigue, and sleep disturbances. Despite this need, validated digital mental health tools tailored to family caregivers remain limited. AI-powered conversational agents offer a promising approach for delivering on-demand, personalized mental health support, yet development and evaluation frameworks for this population are lacking.
Objective
This paper describes the iterative development and formative evaluation of COCO (Caring of Caregivers Online), a conversational agent designed for family caregivers of children with chronic health conditions. COCO integrates problem-solving therapy (PST) and motivational interviewing (MI) within a human-in-the-loop development framework that progressed from rule-based interactions to a large language model (LLM)–powered conversational agent.
Methods
COCO was developed across four phases: (1) caregiver persona and dialogue development based on PST and MI; (2) usability testing of a low-fidelity prototype with standardized patients in a single session of PST; (3) usability testing of a high-fidelity prototype with caregivers in a single session of PST (n=38); (4) integration of an LLM into COCO. The Wizard-of-Oz method was used across phases 2 and 3 to collect naturalistic dialogues and refine COCO’s conversational design. In phase 3, usability of COCO was assessed using the System Usability Scale (SUS). Caregiver emotions were measured before and after the session using 6 subscales of the PANAS-X. In phase 4, GPT-4 was integrated into COCO with few-shot learning and evaluated by research team members using the caregiver personas. Descriptive statistics were used to summarize quantitative measures. The MI principles and techniques used by COCO across the 4 phases were coded using the Motivational Interviewing Treatment Integrity Coding Manual .
Results
In phase 1, 4 gold-standard dialogues were developed using caregiver personas. In phase 2, standardized patients described COCO as validating and identified its problem-solving and on-demand support as helpful for caregivers. In phase 3, COCO-Wizard-of-Oz achieved a mean SUS score of 75.6% (SD 12.9%), reflecting acceptable usability. Participants demonstrated significant improvement in negative affect, sadness, guilt, and fatigue following PST sessions ( P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- School of Nursing & Healthcare Leadership, University of Washington Tacoma, 1900 Commerce Street, Campus Box 358421, Tacoma, WA, 98402, United States.
- Florida State University, Institute on Digital Health and Innovation, College of Nursing, Florida State University,, Tallahassee, FL, United States, 1 (850) 644-3296.
- University of Washington, Seattle, WA, United States.
Funding
- NIH HHS
- NINR NIH HHS
Cite
APA
Yuwen, W., Wang, L., Xie, S. J., Divina, M., Fan, X., Kearns, W., & Ward, T. M. (2026). Development and Preliminary Evaluation of a Conversational Agent Delivering Problem-Solving Therapy for Family Caregivers of Children With a Chronic Health Condition: Multiphase Mixed Methods Study. Journal of Medical Internet Research, 28, e94175-e94175. https://doi.org/10.2196/94175
Vancouver
Yuwen W, Wang L, Xie SJ, Divina M, Fan X, Kearns W, et al. Development and Preliminary Evaluation of a Conversational Agent Delivering Problem-Solving Therapy for Family Caregivers of Children With a Chronic Health Condition: Multiphase Mixed Methods Study. Journal of Medical Internet Research. 2026;28:e94175-e94175. doi:10.2196/94175
BibTeX
@article{Yuwen2026,
author = {Yuwen, Weichao and Wang, Liying and Xie, Serena Jinchen and Divina, Myra and Fan, Xuehong and Kearns, William and Ward, Teresa M},
title = {Development and Preliminary Evaluation of a Conversational Agent Delivering Problem-Solving Therapy for Family Caregivers of Children With a Chronic Health Condition: Multiphase Mixed Methods Study},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e94175--e94175},
doi = {10.2196/94175}
}
EUVIMED