EUVIMEDEUVIMED entdecken
Vergleichsstudie

Comparison of Drug Information Questions' Responses Between Drug Information Pharmacists and Artificial Intelligence Technologies: A Real‐World Analysis in a Tertiary Care Hospital

Lamia Alzahrani, Arwa Alkhalaf iD, Rayd Almehizia, Mohamed Ahmed, Fahad Alkhuzaee iD, Amina Alkhalaf iD, Alhanouf Almoussa iD, Wafa Alshehre iD, Maram Alenazi, Abdulaziz Alaqil, Maryam Bamakhramah

JACCP: JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · 2026

DOI: 10.1002/jac5.70208 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JACCP: JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY, 2026. Verfasst von Lamia Alzahrani und 10 weiteren Autor:innen. Beteiligte Einrichtung: Pharmaceutical Care Division, King Faisal Specialist Hospital and Research Center, Jeddah, Saudi Arabia. Hauptthemen laut MEDLINE: Artificial Intelligence, Tertiary Care Centers, Pharmacists, Drug Information Services. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 20 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Drug Information Centers (DICs) play a critical role in health care by providing accurate, evidence‐based responses to drug‐related inquiries. The integration of artificial intelligence (AI) into this domain offers opportunities to enhance efficiency and decision‐making. This study aims to compare the accuracy and safety of drug information responses provided by drug information pharmacists and artificial intelligence (AI) technologies, including GPT‐4o, Gemini, Copilot, and Medical Chat (OpenAI, San Francisco, CA, USA; Google, Mountain View, CA, USA; Microsoft, Redmond, WA, USA; Medical Chat, developer information not publicly specified).

Methods

A retrospective, real‐world study was conducted over 3 months at a tertiary hospital. The end points of this study are the statistical differences in overall accuracy of responses obtained from generative AI technologies and drug information pharmacists to drug information questions' responses measured in six dimensions.

Results

GPT‐4o and Medical Chat achieved composite scores comparable to DIC pharmacists (mean [standard deviation (SD)]: 18.3 [2.5], 17.5 [3.1], and 17.7 [3.5], respectively), while Gemini and Copilot showed lower performance (16.7 [3.5] and 16.4 [2.9]). Welch's analysis of variance (ANOVA) demonstrated a significant difference in composite scores among groups ( p < 0.001), with GPT‐4o outperforming Gemini and Copilot in pairwise comparisons. Differences were also observed across question categories, with routine inquiries such as administration and dosing being more common than complex categories. Inter‐rater agreement was low (Fleiss' kappa = 0.015, p = 0.297), indicating variability in scoring among evaluators.

Conclusion

Some AI tools demonstrated performance comparable with pharmacists for routine drug information questions, while others showed lower reliability, particularly for clinically nuanced responses. Variability among tools and low rater agreement highlight the need for cautious interpretation and continued pharmacist oversight when using AI in drug information practice.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceTertiary Care CentersPharmacistsDrug Information ServicesPharmacy Service, HospitalHumansRetrospective StudiesGenerative Artificial Intelligence

Stichwörter der Autor:innen

artificial intelligencedruggenerative AIsafety

Autor:innen und Institutionen

  • Pharmaceutical Care Division, King Faisal Specialist Hospital and Research Center, Jeddah, Saudi Arabia.
  • Measurement and Evaluation, Faculty of Education, King Abdulaziz University, Jeddah, Saudi Arabia.
  • Pharmaceutical Care Division, King Faisal Specialist Hospital and Research Center, Riyadh, Saudi Arabia.
  • Pharmaceutical Administration Services, King Abdullah Medical City in Holy Capital, Makkah, Saudi Arabia.
  • Faculty of Pharmacy, King Abdulaziz University, Jeddah, Saudi Arabia.
  • Pharmaceutical Care Division, Ministry of Health, Riyadh, Saudi Arabia.

ORCID-Profile: Arwa Alkhalaf, Fahad Alkhuzaee, Amina Alkhalaf, Alhanouf Almoussa, Wafa Alshehre

Zitieren

APA

Alzahrani, L., Alkhalaf, A., Almehizia, R., Ahmed, M., Alkhuzaee, F., Alkhalaf, A., Almoussa, A., Alshehre, W., Alenazi, M., Alaqil, A., & Bamakhramah, M. (2026). Comparison of Drug Information Questions' Responses Between Drug Information Pharmacists and Artificial Intelligence Technologies: A Real‐World Analysis in a Tertiary Care Hospital. JACCP: JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY. https://doi.org/10.1002/jac5.70208

Vancouver

Alzahrani L, Alkhalaf A, Almehizia R, Ahmed M, Alkhuzaee F, Alkhalaf A, et al. Comparison of Drug Information Questions' Responses Between Drug Information Pharmacists and Artificial Intelligence Technologies: A Real‐World Analysis in a Tertiary Care Hospital. JACCP: JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY. 2026. doi:10.1002/jac5.70208

BibTeX
@article{Alzahrani2026,
  author = {Alzahrani, Lamia and Alkhalaf, Arwa and Almehizia, Rayd and Ahmed, Mohamed and Alkhuzaee, Fahad and Alkhalaf, Amina and Almoussa, Alhanouf and Alshehre, Wafa and Alenazi, Maram and Alaqil, Abdulaziz and Bamakhramah, Maryam},
  title = {Comparison of Drug Information Questions' Responses Between Drug Information Pharmacists and Artificial Intelligence Technologies: A Real‐World Analysis in a Tertiary Care Hospital},
  journal = {JACCP: JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY},
  year = {2026},
  doi = {10.1002/jac5.70208}
}
Kostenlos starten →