EUVIMEDEUVIMED entdecken
Qualitative StudieOpen Access

Stakeholder Criteria for Trust in Artificial Intelligence–Based Computer Perception Tools in Health Care: Qualitative Interview Study

Ansh Rai iD, Meghan E Hurley iD, John Herrington iD, Eric A Storch iD, Casey J Zampella iD, Julia Parrish-Morris iD, Anika Sonig iD, Gabriel Lázaro-Muñoz iD, Kristin Kostick-Quenet iD

Journal of Medical Internet Research · 2025

DOI: 10.2196/78757 ↗

Kurz gefasst

Qualitative Studie in der Zeitschrift Journal of Medical Internet Research, 2025. Verfasst von Ansh Rai und 8 weiteren Autor:innen. Beteiligte Einrichtung: Center for Medical Ethics and Health Policy, Baylor College of Medicine, Houston, United States. Hauptthemen laut MEDLINE: Artificial Intelligence, Trust, Delivery of Health Care. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Computer perception (CP) technologies hold significant promise for advancing precision mental health care systems, given their ability to leverage algorithmic analysis of continuous, passive sensing data from wearables and smartphones (eg, behavioral activity, geolocation, vocal features, and ambient environmental data) to infer clinically meaningful behavioral and physiological states. However, successful implementation critically depends on cultivating well-founded stakeholder trust.

Objective

This study aims to investigate, across adolescents, caregivers, clinicians, and developers, the contingencies under which CP technologies are deemed trustworthy in health care.

Methods

We conducted 80 semistructured interviews with a purposive sample of adolescents (n=20) diagnosed with autism, Tourette syndrome, anxiety, obsessive-compulsive disorder, or attention-deficit/hyperactivity disorder and their caregivers (n=20); practicing clinicians across psychiatry, psychology, and pediatrics (n=20); and CP system developers (n=20). Interview transcripts were coded by 2 independent coders and analyzed using multistage, inductive thematic content analysis to identify prominent themes.

Results

Across stakeholder groups, 5 core criteria emerged as prerequisites for trust in CP outputs: (1) epistemic alignment—consistency between system outputs, personal experience, and existing diagnostic frameworks; (2) demonstrable rigor—training on representative data and validation in real-world contexts; (3) explainability—transparent communication of input variables, thresholds, and decision logic; (4) sensitivity to complexity—the capacity to accommodate heterogeneity and comorbidity in symptom expression; and (5) a nonsubstitutive role—technologies must augment, rather than supplant, clinical judgment. A novel and cautionary finding was that epistemic alignment—whether outputs affirmed participants’ preexisting beliefs, diagnostic expectations, or internal states—was a dominant factor in determining whether the tool was perceived as trustworthy. Participants also expressed relational trust, placing confidence in CP systems based on endorsements from respected peers, academic institutions, or regulatory agencies. However, both trust strategies raise significant concerns: confirmation bias may lead users to overvalue outputs that align with their assumptions, while surrogate trust may be misapplied in the absence of robust performance validation.

Conclusions

This study advances empirical understanding of how trust is formed and calibrated around artificial intelligence–based CP technologies. While trust is commonly framed as a function of technical performance, our findings show that it is deeply shaped by cognitive heuristics, social relationships, and alignment with entrenched epistemologies. These dynamics can facilitate intuitive verification but may also constrain the transformative potential of CP systems by reinforcing existing beliefs. To address this, we recommend a dual strategy: (1) embedding CP tools within institutional frameworks that uphold rigorous validation, ethical oversight, and transparent design; and (2) providing clinicians with training and interface designs that support critical appraisal and minimize susceptibility to cognitive bias. Recalibrating trust to reflect actual system capacities—rather than familiarity or endorsement—is essential for ethically sound and clinically meaningful integration of CP technologies.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceTrustDelivery of Health CareHumansAdolescentFemaleMaleInterviews as TopicQualitative ResearchAdult

Stichwörter der Autor:innen

artificial intelligencecomputingmachine learningpsychiatrytechnologytrusttrustworthiness

Autor:innen und Institutionen

  • Center for Medical Ethics and Health Policy, Baylor College of Medicine, Houston, United States.
  • Children's Hospital of Philadelphia, Philadelphia, United States.
  • Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, United States.
  • Center for Bioethics, Harvard Medical School, Boston, MA, United States.
  • Department of Psychiatry and Behavioral Sciences, Massachusetts General Hospital, Boston, MA, United States.

ORCID-Profile: Ansh Rai, Meghan E Hurley, John Herrington, Eric A Storch, Casey J Zampella, Julia Parrish-Morris, Anika Sonig, Gabriel Lázaro-Muñoz, Kristin Kostick-Quenet

Zitieren

APA

Rai, A., Hurley, M. E., Herrington, J., Storch, E. A., Zampella, C. J., Parrish-Morris, J., Sonig, A., Lázaro-Muñoz, G., & Kostick-Quenet, K. (2025). Stakeholder Criteria for Trust in Artificial Intelligence–Based Computer Perception Tools in Health Care: Qualitative Interview Study. Journal of Medical Internet Research. https://doi.org/10.2196/78757

Vancouver

Rai A, Hurley ME, Herrington J, Storch EA, Zampella CJ, Parrish-Morris J, et al. Stakeholder Criteria for Trust in Artificial Intelligence–Based Computer Perception Tools in Health Care: Qualitative Interview Study. Journal of Medical Internet Research. 2025. doi:10.2196/78757

BibTeX
@article{Rai2025,
  author = {Rai, Ansh and Hurley, Meghan E and Herrington, John and Storch, Eric A and Zampella, Casey J and Parrish-Morris, Julia and Sonig, Anika and Lázaro-Muñoz, Gabriel and Kostick-Quenet, Kristin},
  title = {Stakeholder Criteria for Trust in Artificial Intelligence–Based Computer Perception Tools in Health Care: Qualitative Interview Study},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  doi = {10.2196/78757}
}
Kostenlos starten →