EUVIMEDOpdag EUVIMED
Randomisierte kontrollierte StudieOpen access

Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial

Sicherheit und Nutzererfahrung einer generativen KI-basierten digitalen Intervention für die psychische Gesundheit: explorative randomisierte kontrollierte Studie

Timothy R Campellone iD, Megan Flom iD, Robert M Montgomery iD, Lauren Bullard iD, Maddison C Pirner iD, Aaron Pavez iD, Michelle Morales iD, Devin Harper iD, Catherine Oddy iD, Tom O'Connor iD, Jade Daniels iD, Stephanie Eaneff iD and 3 more

Journal of Medical Internet Research · 2025-05-23 · 27 · e67365

DOI: 10.2196/67365 ↗

Auf einen Blick

In einer 2-wöchigen RCT mit 160 Erwachsenen waren Nutzerbeziehung und Erfahrung in beiden Gruppen insgesamt ähnlich. Die generative KI zeigte höhere Genauigkeit bei empathischen Antworten.

Fragestellung

Untersucht wurde, wie sich eine generative KI in einer dialogbasierten DMHI im Vergleich zu einer regelbasierten Intervention auf Nutzerbeziehung, Erfahrung, Sicherheit und technische Schutzmechanismen auswirkt.

Methode

2-wöchige explorative RCT mit 160 Erwachsenen; generative KI (n=81) vs. regelbasiert (n=79); Erfassung von Nutzerbeziehung, Erfahrung, unerwünschten Ereignissen und technischen Schutzmechanismen.

Ergebnisse

Die meisten Maße der Nutzerbeziehung und Erfahrung waren in beiden Armen ähnlich. Empathieerkennung/-antworten: 98 % vs. 69 %. Keine schweren oder gerätebezogenen unerwünschten Ereignisse; Schutzmechanismen 100 % erfolgreich.

Schlussfolgerung der Autor:innen

Die Studie liefert erste Evidenz, dass generative KI mit passenden Schutzmechanismen in einer DMHI erfolgreich eingesetzt werden kann, ohne Nutzererfahrung und Beziehung zu beeinträchtigen.

Kernaussagen

  • Nutzerbeziehung und Erfahrung waren in beiden Armen insgesamt ähnlich.
  • Empathische Antworten waren im generativen Arm genauer (98 % vs. 69 %).
  • Keine schweren oder gerätebezogenen unerwünschten Ereignisse; Schutzmechanismen 100 % erfolgreich.

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Randomisierte kontrollierte Studie in Journal of Medical Internet Research, published online on 2025-05-23. Written by Timothy R Campellone and 14 co-authors. Institution: Woebot Health, San Francisco, CA, United States. Major MEDLINE topics: Artificial Intelligence, Mental Health. Cited 42 times so far (Crossref); 13 references. The full text is freely available in PubMed Central.

Auf einen BlickAbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

General awareness and exposure to generative artificial intelligence (AI) have increased recently. This transformative technology has the potential to create a more dynamic and engaging user experience in digital mental health interventions (DMHIs). However, if not appropriately used and controlled, it can introduce risks to users that may result in harm and erode trust. At the time of conducting this trial, there had not been a rigorous evaluation of an approach to safely implementing generative AI in a DMHI.

Objective

This study aims to explore the user relationship, experience, safety, and technical guardrails of a DMHI using generative AI compared with a rules-based intervention.

Methods

We conducted a 2-week exploratory randomized controlled trial (RCT) with 160 adult participants randomized to receive a generative AI (n=81) or rules-based (n=79) version of a conversation-based DMHI. Self-report measures of the user relationship (client satisfaction, working alliance bond, and accuracy of empathic listening and reflection) and experience (engagement metrics, adverse events, and technical guardrail success) were collected. Descriptions and validation of technical guardrails for handling user inputs (eg, detecting potentially concerning language and off-topic responses) and model outputs (eg, not providing medical advice and not providing a diagnosis) are provided, along with examples to illustrate how they worked. Safety monitoring was conducted throughout the trial for adverse events, and the success of technical guardrails created for the generative arm was assessed post trial.

Results

In general, the majority of measures of user relationship and experience appeared to be similar in both the generative and rules-based arms. The generative arm appeared to be more accurate at detecting and responding to user statements with empathy (98% accuracy vs 69%). There were no serious or device-related adverse events, and technical guardrails were shown to be 100% successful in posttrial review of generated statements. A majority of participants in both groups reported an increase in positive sentiment (62% and 66%) about AI at the end of the trial.

Conclusions

This trial provides initial evidence that, with the right guardrails and process, generative AI can be successfully used in a digital mental health intervention (DMHI) while maintaining the user experience and relationship. It also provides an initial blueprint for approaches to technical and conversational guardrails that can be replicated to build a safe DMHI. Trial Registration ClinicalTrials.gov NCT05948670; https://clinicaltrials.gov/study/NCT05948670

Subjects

MeSH terms (MEDLINE)

Artificial IntelligenceMental HealthHumansAdultFemaleMaleMiddle AgedPatient SatisfactionYoung AdultGenerative Artificial Intelligence

Author keywords

AIDMHIRCTartificial intelligencechatbotscontrolled trialsdigital healthdigital mental health interventionexploratorygenerativegenerative AImental healthrandomizedrandomized controlled trialrelationship

Authors and institutions

  • Woebot Health, San Francisco, CA, United States.

ORCID profiles: Timothy R Campellone, Megan Flom, Robert M Montgomery, Lauren Bullard, Maddison C Pirner, Aaron Pavez, Michelle Morales, Devin Harper, Catherine Oddy, Tom O'Connor, Jade Daniels, Stephanie Eaneff, Valerie L Forman-Hoffman, Casey Sackett, Alison Darcy

Cite

APA

Campellone, T. R., Flom, M., Montgomery, R. M., Bullard, L., Pirner, M. C., Pavez, A., Morales, M., Harper, D., Oddy, C., O'Connor, T., Daniels, J., Eaneff, S., Forman-Hoffman, V. L., Sackett, C., & Darcy, A. (2025). Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial. Journal of Medical Internet Research, 27, e67365. https://doi.org/10.2196/67365

Vancouver

Campellone TR, Flom M, Montgomery RM, Bullard L, Pirner MC, Pavez A, et al. Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial. Journal of Medical Internet Research. 2025;27:e67365. doi:10.2196/67365

BibTeX
@article{Campellone2025,
  author = {Campellone, Timothy R and Flom, Megan and Montgomery, Robert M and Bullard, Lauren and Pirner, Maddison C and Pavez, Aaron and Morales, Michelle and Harper, Devin and Oddy, Catherine and O'Connor, Tom and Daniels, Jade and Eaneff, Stephanie and Forman-Hoffman, Valerie L and Sackett, Casey and Darcy, Alison},
  title = {Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e67365},
  doi = {10.2196/67365}
}
Start for free →