Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial
Sicherheit und Nutzererfahrung einer generativen KI-basierten digitalen Intervention für die psychische Gesundheit: explorative randomisierte kontrollierte Studie
Journal of Medical Internet Research · 23.05.2025 · 27 · e67365
DOI: 10.2196/67365 ↗Auf einen Blick
In einer 2-wöchigen RCT mit 160 Erwachsenen waren Nutzerbeziehung und Erfahrung in beiden Gruppen insgesamt ähnlich. Die generative KI zeigte höhere Genauigkeit bei empathischen Antworten.
Fragestellung
Untersucht wurde, wie sich eine generative KI in einer dialogbasierten DMHI im Vergleich zu einer regelbasierten Intervention auf Nutzerbeziehung, Erfahrung, Sicherheit und technische Schutzmechanismen auswirkt.
Methode
2-wöchige explorative RCT mit 160 Erwachsenen; generative KI (n=81) vs. regelbasiert (n=79); Erfassung von Nutzerbeziehung, Erfahrung, unerwünschten Ereignissen und technischen Schutzmechanismen.
Ergebnisse
Die meisten Maße der Nutzerbeziehung und Erfahrung waren in beiden Armen ähnlich. Empathieerkennung/-antworten: 98 % vs. 69 %. Keine schweren oder gerätebezogenen unerwünschten Ereignisse; Schutzmechanismen 100 % erfolgreich.
Schlussfolgerung der Autor:innen
Die Studie liefert erste Evidenz, dass generative KI mit passenden Schutzmechanismen in einer DMHI erfolgreich eingesetzt werden kann, ohne Nutzererfahrung und Beziehung zu beeinträchtigen.
Kernaussagen
- Nutzerbeziehung und Erfahrung waren in beiden Armen insgesamt ähnlich.
- Empathische Antworten waren im generativen Arm genauer (98 % vs. 69 %).
- Keine schweren oder gerätebezogenen unerwünschten Ereignisse; Schutzmechanismen 100 % erfolgreich.
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Randomisierte kontrollierte Studie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 23.05.2025. Verfasst von Timothy R Campellone und 14 weiteren Autor:innen. Beteiligte Einrichtung: Woebot Health, San Francisco, CA, United States. Hauptthemen laut MEDLINE: Artificial Intelligence, Mental Health. Die Arbeit wurde bisher 42-mal zitiert (Crossref) und nennt 13 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
General awareness and exposure to generative artificial intelligence (AI) have increased recently. This transformative technology has the potential to create a more dynamic and engaging user experience in digital mental health interventions (DMHIs). However, if not appropriately used and controlled, it can introduce risks to users that may result in harm and erode trust. At the time of conducting this trial, there had not been a rigorous evaluation of an approach to safely implementing generative AI in a DMHI.
Objective
This study aims to explore the user relationship, experience, safety, and technical guardrails of a DMHI using generative AI compared with a rules-based intervention.
Methods
We conducted a 2-week exploratory randomized controlled trial (RCT) with 160 adult participants randomized to receive a generative AI (n=81) or rules-based (n=79) version of a conversation-based DMHI. Self-report measures of the user relationship (client satisfaction, working alliance bond, and accuracy of empathic listening and reflection) and experience (engagement metrics, adverse events, and technical guardrail success) were collected. Descriptions and validation of technical guardrails for handling user inputs (eg, detecting potentially concerning language and off-topic responses) and model outputs (eg, not providing medical advice and not providing a diagnosis) are provided, along with examples to illustrate how they worked. Safety monitoring was conducted throughout the trial for adverse events, and the success of technical guardrails created for the generative arm was assessed post trial.
Results
In general, the majority of measures of user relationship and experience appeared to be similar in both the generative and rules-based arms. The generative arm appeared to be more accurate at detecting and responding to user statements with empathy (98% accuracy vs 69%). There were no serious or device-related adverse events, and technical guardrails were shown to be 100% successful in posttrial review of generated statements. A majority of participants in both groups reported an increase in positive sentiment (62% and 66%) about AI at the end of the trial.
Conclusions
This trial provides initial evidence that, with the right guardrails and process, generative AI can be successfully used in a digital mental health intervention (DMHI) while maintaining the user experience and relationship. It also provides an initial blueprint for approaches to technical and conversational guardrails that can be replicated to build a safe DMHI. Trial Registration ClinicalTrials.gov NCT05948670; https://clinicaltrials.gov/study/NCT05948670
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Woebot Health, San Francisco, CA, United States.
ORCID-Profile: Timothy R Campellone, Megan Flom, Robert M Montgomery, Lauren Bullard, Maddison C Pirner, Aaron Pavez, Michelle Morales, Devin Harper, Catherine Oddy, Tom O'Connor, Jade Daniels, Stephanie Eaneff, Valerie L Forman-Hoffman, Casey Sackett, Alison Darcy
Zitieren
APA
Campellone, T. R., Flom, M., Montgomery, R. M., Bullard, L., Pirner, M. C., Pavez, A., Morales, M., Harper, D., Oddy, C., O'Connor, T., Daniels, J., Eaneff, S., Forman-Hoffman, V. L., Sackett, C., & Darcy, A. (2025). Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial. Journal of Medical Internet Research, 27, e67365. https://doi.org/10.2196/67365
Vancouver
Campellone TR, Flom M, Montgomery RM, Bullard L, Pirner MC, Pavez A, et al. Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial. Journal of Medical Internet Research. 2025;27:e67365. doi:10.2196/67365
BibTeX
@article{Campellone2025,
author = {Campellone, Timothy R and Flom, Megan and Montgomery, Robert M and Bullard, Lauren and Pirner, Maddison C and Pavez, Aaron and Morales, Michelle and Harper, Devin and Oddy, Catherine and O'Connor, Tom and Daniels, Jade and Eaneff, Stephanie and Forman-Hoffman, Valerie L and Sackett, Casey and Darcy, Alison},
title = {Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial},
journal = {Journal of Medical Internet Research},
year = {2025},
volume = {27},
pages = {e67365},
doi = {10.2196/67365}
}
EUVIMED