Accuracy and Safety of ChatGPT-3.5 in Assessing Over-the-Counter Medication Use During Pregnancy: A Descriptive Comparative Study
Pharmacy · 30.07.2025 · 13 (4) · 104
DOI: 10.3390/pharmacy13040104 ↗Kurz gefasst
Fachartikel in der Zeitschrift Pharmacy, online erschienen am 30.07.2025. Verfasst von Bernadette Cornelison und 6 weiteren Autor:innen. Beteiligte Einrichtung: R. Ken Coit College of Pharmacy, University of Arizona, 1295 N. Martin Ave., Tucson, AZ 85721, USA. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
As artificial intelligence (AI) becomes increasingly utilized to perform tasks requiring human intelligence, patients who are pregnant may turn to AI for advice on over-the-counter (OTC) medications. However, medications used in pregnancy may pose profound safety concerns limited by data availability. This study focuses on a chatbot’s ability to accurately provide information regarding OTC medications as it relates to patients that are pregnant. A prospective, descriptive design was used to compare the responses generated by the Chat Generative Pre-Trained Transformer 3.5 (ChatGPT-3.5) to the information provided by UpToDate®. Eighty-seven of the top pharmacist-recommended OTC drugs in the United States (U.S.) as identified by Pharmacy Times were assessed for safe use in pregnancy using ChatGPT-3.5. A piloted, standard prompt was input into ChatGPT-3.5, and the responses were recorded. Two groups independently rated the responses compared to UpToDate on their correctness, completeness, and safety using a 5-point Likert scale. After independent evaluations, the groups discussed the findings to reach a consensus, with a third independent investigator giving final ratings. For correctness, the median score was 5 (interquartile range [IQR]: 5–5). For completeness, the median score was 4 (IQR: 4–5). For safety, the median score was 5 (IQR: 5–5). Despite high overall scores, the safety errors in 9% of the evaluations (n = 8), including omissions that pose a risk of serious complications, currently renders the chatbot an unsafe standalone resource for this purpose.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- R. Ken Coit College of Pharmacy, University of Arizona, 1295 N. Martin Ave., Tucson, AZ 85721, USA.
ORCID-Profile: Bernadette Cornelison, David R Axon, Kristen A Root
Zitieren
APA
Cornelison, B., Axon, D. R., Abbott, B., Bishop, C., Jebara, C., Kumar, A., & Root, K. A. (2025). Accuracy and Safety of ChatGPT-3.5 in Assessing Over-the-Counter Medication Use During Pregnancy: A Descriptive Comparative Study. Pharmacy, 13(4), 104. https://doi.org/10.3390/pharmacy13040104
Vancouver
Cornelison B, Axon DR, Abbott B, Bishop C, Jebara C, Kumar A, et al. Accuracy and Safety of ChatGPT-3.5 in Assessing Over-the-Counter Medication Use During Pregnancy: A Descriptive Comparative Study. Pharmacy. 2025;13(4):104. doi:10.3390/pharmacy13040104
BibTeX
@article{Cornelison2025,
author = {Cornelison, Bernadette and Axon, David R and Abbott, Bryan and Bishop, Carter and Jebara, Cindy and Kumar, Anjali and Root, Kristen A},
title = {Accuracy and Safety of ChatGPT-3.5 in Assessing Over-the-Counter Medication Use During Pregnancy: A Descriptive Comparative Study},
journal = {Pharmacy},
year = {2025},
volume = {13},
number = {4},
pages = {104},
doi = {10.3390/pharmacy13040104}
}
EUVIMED