EUVIMEDOdkrijte EUVIMED
Multizentrische StudieRandomisierte kontrollierte StudieOpen access

Effect of Artificial Intelligence Helpfulness and Uncertainty on Cognitive Interactions with Pharmacists: Randomized Controlled Trial

Einfluss von Nützlichkeit und Unsicherheit von KI auf kognitive Interaktionen mit Apotheker:innen: randomisierte kontrollierte Studie

Chuan-Ching Tsai iD, Jin Yong Kim iD, Qiyuan Chen iD, Brigid Rowell iD, X Jessie Yang iD, Raed Kontar iD, Megan Whitaker iD, Corey Lester iD

Journal of Medical Internet Research · 2025-01-31 · 27 · e59946

DOI: 10.2196/59946 ↗

Auf einen Blick

30 Apotheker:innen bearbeiteten Verifikationsaufgaben mit und ohne KI-Unterstützung. Unnütze KI und angezeigte Unsicherheit verlängerten die Verarbeitungszeit.

Fragestellung

Welche kognitiven Interaktionsmuster zeigen Apotheker:innen bei der Medikamentenverifikation mit einem KI-Prototyp, und wie wirken Hilfreichkeit und Unsicherheitsanzeige der KI?

Methode

Randomisierte kontrollierte Studie; 30 Apotheker:innen aus beruflichen Netzwerken; 200 Verifikationsaufgaben pro Person; 100 ohne und 100 mit KI-Unterstützung; Eye-Tracking online.

Ergebnisse

Apotheker:innen verlagerten 19 %–26 % ihrer Fixationen auf KI-generierte Bereiche. Die Zahl der Fixationen auf Füllbilder nahm nicht ab. Unnütze KI und Unsicherheitsanzeige führten zu längeren Dwell-Times.

Schlussfolgerung der Autor:innen

Unnütze KI erhöht die kognitive Verarbeitungszeit in den Originalbildern. Transparenz ist bei „Black-Box“-Systemen nötig, mehr Information kann aber die kognitive Belastung erhöhen.

Kernaussagen

  • 19 %–26 % der Fixationen entfielen auf KI-generierte Bereiche
  • Unnütze KI verlängerte die Dwell-Time auf Referenz- und Füllbildern
  • Unsicherheitsanzeige ging mit längeren Verarbeitungszeiten einher

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Multizentrische Studie in Journal of Medical Internet Research, published online on 2025-01-31. Written by Chuan-Ching Tsai and 7 co-authors. Institution: Department of Clinical Pharmacy, College of Pharmacy, University of Michigan, Ann Arbor, MI, United States. Major MEDLINE topics: Artificial Intelligence, Cognition, Pharmacists. Cited 18 times so far (Crossref); 54 references. The full text is freely available in PubMed Central. Funded by: NLM NIH HHS.

Auf einen BlickPovzetekSubjectsBibliografski podatkiCiteRelated studies

Abstract (original language)

Background

Clinical decision support systems leveraging artificial intelligence (AI) are increasingly integrated into health care practices, including pharmacy medication verification. Communicating uncertainty in an AI prediction is viewed as an important mechanism for boosting human collaboration and trust. Yet, little is known about the effects on human cognition as a result of interacting with such types of AI advice.

Objective

This study aimed to evaluate the cognitive interaction patterns of pharmacists during medication product verification when using an AI prototype. Moreover, we examine the impact of AI’s assistance, both helpful and unhelpful, and the communication of uncertainty of AI-generated results on pharmacists’ cognitive interaction with the prototype.

Methods

In a randomized controlled trial, 30 pharmacists from professional networks each performed 200 medication verification tasks while their eye movements were recorded using an online eye tracker. Participants completed 100 verifications without AI assistance and 100 with AI assistance (either with black box help without uncertainty information or uncertainty-aware help, which displays AI uncertainty). Fixation patterns (first and last areas fixated, number of fixations, fixation duration, and dwell times) were analyzed in relation to AI help type and helpfulness.

Results

Pharmacists shifted 19%-26% of their total fixations to AI-generated regions when these were available, suggesting the integration of AI advice in decision-making. AI assistance did not reduce the number of fixations on fill images, which remained the primary focus area. Unhelpful AI advice led to longer dwell times on reference and fill images, indicating increased cognitive processing. Displaying AI uncertainty led to longer cognitive processing times as measured by dwell times in original images.

Conclusions

Unhelpful AI increases cognitive processing time in the original images. Transparency in AI is needed in “black box” systems, but showing more information can add a cognitive burden. Therefore, the communication of uncertainty should be optimized and integrated into clinical workflows using user-centered design to avoid increasing cognitive load or impeding clinicians’ original workflow. Trial Registration ClinicalTrials.gov NCT06795477; https://clinicaltrials.gov/study/NCT06795477

Subjects

MeSH terms (MEDLINE)

Artificial IntelligenceCognitionPharmacistsAdultFemaleHumansMaleMiddle AgedDecision Support Systems, ClinicalUncertainty

Author keywords

AI helpfulness and accuracyCDSSartificial intelligenceclinical decision support systemcognitioncognitive interactionscognitive processingdecision-makingeye-trackinginteractionmedicationmedication verificationpharmacistsuncertainty visualization

Authors and institutions

  • Department of Clinical Pharmacy, College of Pharmacy, University of Michigan, Ann Arbor, MI, United States.
  • Department of Industrial and Operations Engineering, College of Engineering, University of Michigan, Ann Arbor, MI, United States.

ORCID profiles: Chuan-Ching Tsai, Jin Yong Kim, Qiyuan Chen, Brigid Rowell, X Jessie Yang, Raed Kontar, Megan Whitaker, Corey Lester

Funding

  • NLM NIH HHS

Cite

APA

Tsai, C. C., Kim, J. Y., Chen, Q., Rowell, B., Yang, X. J., Kontar, R., Whitaker, M., & Lester, C. (2025). Effect of Artificial Intelligence Helpfulness and Uncertainty on Cognitive Interactions with Pharmacists: Randomized Controlled Trial. Journal of Medical Internet Research, 27, e59946. https://doi.org/10.2196/59946

Vancouver

Tsai CC, Kim JY, Chen Q, Rowell B, Yang XJ, Kontar R, et al. Effect of Artificial Intelligence Helpfulness and Uncertainty on Cognitive Interactions with Pharmacists: Randomized Controlled Trial. Journal of Medical Internet Research. 2025;27:e59946. doi:10.2196/59946

BibTeX
@article{Tsai2025,
  author = {Tsai, Chuan-Ching and Kim, Jin Yong and Chen, Qiyuan and Rowell, Brigid and Yang, X Jessie and Kontar, Raed and Whitaker, Megan and Lester, Corey},
  title = {Effect of Artificial Intelligence Helpfulness and Uncertainty on Cognitive Interactions with Pharmacists: Randomized Controlled Trial},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e59946},
  doi = {10.2196/59946}
}
Start for free →