EUVIMEDObjevte EUVIMED
Open access

The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study

Jin Yong Kim iD, Vincent D Marshall iD, Brigid Rowell iD, Qiyuan Chen iD, Yifan Zheng iD, John D Lee iD, Raed Al Kontar iD, Corey Lester iD, Xi Jessie Yang iD

JMIR Human Factors · 2025-02-11 · 12 · e60273

DOI: 10.2196/60273 ↗

At a glance

Journal article in JMIR Human Factors, published online on 2025-02-11. Written by Jin Yong Kim and 8 co-authors. Institution: Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI, United States. Major MEDLINE topics: Pharmacists, Trust, Artificial Intelligence, Medication Errors. Cited 15 times so far (Crossref); 71 references. The full text is freely available in PubMed Central. Funded by: NLM NIH HHS.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Dispensing errors significantly contribute to adverse drug events, resulting in substantial health care costs and patient harm. Automated pill verification technologies have been developed to aid pharmacists with medication dispensing. However, pharmacists’ trust in such automated technologies remains unexplored.

Objective

This study aims to investigate pharmacists’ trust in automated pill verification technology designed to support medication dispensing.

Methods

Thirty licensed pharmacists in the United States performed a web-based simulated pill verification task to determine whether an image of a filled medication bottle matched a known reference image. Participants completed a block of 100 verification trials without any help, and another block of 100 trials with the help of an imperfect artificial intelligence (AI) aid recommending acceptance or rejection of a filled medication bottle. The experiment used a mixed subjects design. The between-subjects factor was the AI aid type, with or without an AI uncertainty plot. The within-subjects factor was the four potential verification outcomes: (1) the AI rejects the incorrect drug, (2) the AI rejects the correct drug, (3) the AI approves the incorrect drug, and (4) the AI approves the correct drug. Participants’ trust in the AI system was measured. Mixed model (generalized linear models) tests were conducted with 2-tailed t tests to compare the means between the 2 AI aid types for each verification outcome.

Results

Participants had an average trust propensity score of 72 (SD 18.08) out of 100, indicating a positive attitude toward trusting automated technologies. The introduction of an uncertainty plot to the AI aid significantly enhanced pharmacists’ end trust (t28=–1.854; P=.04). Trust dynamics were influenced by AI aid type and verification outcome. Specifically, pharmacists using the AI aid with the uncertainty plot had a significantly larger trust increment when the AI approved the correct drug (t78.98=3.93; P

Subjects

MeSH terms (MEDLINE)

PharmacistsTrustArtificial IntelligenceMedication ErrorsHumansFemaleMaleAdultUncertaintyMiddle AgedUnited States

Author keywords

artificial intelligenceartificial intelligence aidautomationhuman-computer interactionmedication errorspharmacistspill verificationsafetyuncertainty communicationvisualization

Authors and institutions

  • Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI, United States.
  • College of Pharmacy, University of Michigan, Ann Arbor, MI, United States.
  • Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, MI, United States.

ORCID profiles: Jin Yong Kim, Vincent D Marshall, Brigid Rowell, Qiyuan Chen, Yifan Zheng, John D Lee, Raed Al Kontar, Corey Lester, Xi Jessie Yang

Funding

  • NLM NIH HHS

Cite

APA

Kim, J. Y., Marshall, V. D., Rowell, B., Chen, Q., Zheng, Y., Lee, J. D., Kontar, R. A., Lester, C., & Yang, X. J. (2025). The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study. JMIR Human Factors, 12, e60273. https://doi.org/10.2196/60273

Vancouver

Kim JY, Marshall VD, Rowell B, Chen Q, Zheng Y, Lee JD, et al. The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study. JMIR Human Factors. 2025;12:e60273. doi:10.2196/60273

BibTeX
@article{Kim2025,
  author = {Kim, Jin Yong and Marshall, Vincent D and Rowell, Brigid and Chen, Qiyuan and Zheng, Yifan and Lee, John D and Kontar, Raed Al and Lester, Corey and Yang, Xi Jessie},
  title = {The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study},
  journal = {JMIR Human Factors},
  year = {2025},
  volume = {12},
  pages = {e60273},
  doi = {10.2196/60273}
}
Start for free →