The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study
JMIR Human Factors · 2025-02-11 · 12 · e60273
DOI: 10.2196/60273 ↗At a glance
Journal article in JMIR Human Factors, published online on 2025-02-11. Written by Jin Yong Kim and 8 co-authors. Institution: Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI, United States. Major MEDLINE topics: Pharmacists, Trust, Artificial Intelligence, Medication Errors. Cited 15 times so far (Crossref); 71 references. The full text is freely available in PubMed Central. Funded by: NLM NIH HHS.
Abstract (original language)
Background
Dispensing errors significantly contribute to adverse drug events, resulting in substantial health care costs and patient harm. Automated pill verification technologies have been developed to aid pharmacists with medication dispensing. However, pharmacists’ trust in such automated technologies remains unexplored.
Objective
This study aims to investigate pharmacists’ trust in automated pill verification technology designed to support medication dispensing.
Methods
Thirty licensed pharmacists in the United States performed a web-based simulated pill verification task to determine whether an image of a filled medication bottle matched a known reference image. Participants completed a block of 100 verification trials without any help, and another block of 100 trials with the help of an imperfect artificial intelligence (AI) aid recommending acceptance or rejection of a filled medication bottle. The experiment used a mixed subjects design. The between-subjects factor was the AI aid type, with or without an AI uncertainty plot. The within-subjects factor was the four potential verification outcomes: (1) the AI rejects the incorrect drug, (2) the AI rejects the correct drug, (3) the AI approves the incorrect drug, and (4) the AI approves the correct drug. Participants’ trust in the AI system was measured. Mixed model (generalized linear models) tests were conducted with 2-tailed t tests to compare the means between the 2 AI aid types for each verification outcome.
Results
Participants had an average trust propensity score of 72 (SD 18.08) out of 100, indicating a positive attitude toward trusting automated technologies. The introduction of an uncertainty plot to the AI aid significantly enhanced pharmacists’ end trust (t28=–1.854; P=.04). Trust dynamics were influenced by AI aid type and verification outcome. Specifically, pharmacists using the AI aid with the uncertainty plot had a significantly larger trust increment when the AI approved the correct drug (t78.98=3.93; P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI, United States.
- College of Pharmacy, University of Michigan, Ann Arbor, MI, United States.
- Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, MI, United States.
ORCID profiles: Jin Yong Kim, Vincent D Marshall, Brigid Rowell, Qiyuan Chen, Yifan Zheng, John D Lee, Raed Al Kontar, Corey Lester, Xi Jessie Yang
Funding
- NLM NIH HHS
Cite
APA
Kim, J. Y., Marshall, V. D., Rowell, B., Chen, Q., Zheng, Y., Lee, J. D., Kontar, R. A., Lester, C., & Yang, X. J. (2025). The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study. JMIR Human Factors, 12, e60273. https://doi.org/10.2196/60273
Vancouver
Kim JY, Marshall VD, Rowell B, Chen Q, Zheng Y, Lee JD, et al. The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study. JMIR Human Factors. 2025;12:e60273. doi:10.2196/60273
BibTeX
@article{Kim2025,
author = {Kim, Jin Yong and Marshall, Vincent D and Rowell, Brigid and Chen, Qiyuan and Zheng, Yifan and Lee, John D and Kontar, Raed Al and Lester, Corey and Yang, Xi Jessie},
title = {The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study},
journal = {JMIR Human Factors},
year = {2025},
volume = {12},
pages = {e60273},
doi = {10.2196/60273}
}
EUVIMED