EUVIMEDEUVIMED entdecken
Crossref

Performance of ChatGPT on Factual Knowledge Questions Regarding Clinical Pharmacy

Merel van Nuland iD, Abdullah Erdogan, Cenkay Aςar, Ramon Contrucci, Sven Hilbrants, Lamyae Maanach, Toine Egberts, Paul D van der Linden

The Journal of Clinical Pharmacology · 2024-09 · 64 (9) · 1095-1100

DOI: 10.1002/jcph.2443 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Journal of Clinical Pharmacology, online erschienen am 16.04.2024. Verfasst von Merel van Nuland und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Clinical Pharmacy, Tergooi Medical Center, Hilversum, The Netherlands. Hauptthemen laut MEDLINE: Pharmacists. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 11 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

ChatGPT is a language model that was trained on a large dataset including medical literature. Several studies have described the performance of ChatGPT on medical exams. In this study, we examine its performance in answering factual knowledge questions regarding clinical pharmacy. Questions were obtained from a Dutch application that features multiple‐choice questions to maintain a basic knowledge level for clinical pharmacists. In total, 264 clinical pharmacy‐related questions were presented to ChatGPT and responses were evaluated for accuracy, concordance, quality of the substantiation, and reproducibility. Accuracy was defined as the correctness of the answer, and results were compared to the overall score by pharmacists over 2022. Responses were marked concordant if no contradictions were present. The quality of the substantiation was graded by two independent pharmacists using a 4‐point scale. Reproducibility was established by presenting questions multiple times and on various days. ChatGPT yielded accurate responses for 79% of the questions, surpassing pharmacists' accuracy of 66%. Concordance was 95%, and the quality of the substantiation was deemed good or excellent for 73% of the questions. Reproducibility was consistently high, both within day and between days (>92%), as well as across different users. ChatGPT demonstrated a higher accuracy and reproducibility to factual knowledge questions related to clinical pharmacy practice than pharmacists. Consequently, we posit that ChatGPT could serve as a valuable resource to pharmacists. We hope the technology will further improve, which may lead to enhanced future performance.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PharmacistsHumansReproducibility of ResultsPharmacy Service, HospitalSurveys and QuestionnairesEducational Measurement

Stichwörter der Autor:innen

ChatGPTartificial intelligenceclinical pharmacologyexam questionslanguage model

Autor:innen und Institutionen

  • Department of Clinical Pharmacy, Tergooi Medical Center, Hilversum, The Netherlands.
  • Department of Clinical Pharmacy, Amphia Hospital, Breda, The Netherlands.
  • Department of Clinical Pharmacy, Leeuwarden Medical Center, Leeuwarden, The Netherlands.
  • Department of Clinical Pharmacy, Haga Hospital, The Hague, The Netherlands.
  • Department of Clinical Pharmacy, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
  • Division of Pharmacoepidemiology and Clinical Pharmacology, Department of Pharmaceutical Sciences, Faculty of Science, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands.

ORCID-Profile: Merel van Nuland

Zitieren

APA

van Nuland, M., Erdogan, A., Aςar, C., Contrucci, R., Hilbrants, S., Maanach, L., Egberts, T., & van der Linden, P. D. (2024). Performance of ChatGPT on Factual Knowledge Questions Regarding Clinical Pharmacy. The Journal of Clinical Pharmacology, 64(9), 1095-1100. https://doi.org/10.1002/jcph.2443

Vancouver

van Nuland M, Erdogan A, Aςar C, Contrucci R, Hilbrants S, Maanach L, et al. Performance of ChatGPT on Factual Knowledge Questions Regarding Clinical Pharmacy. The Journal of Clinical Pharmacology. 2024;64(9):1095-1100. doi:10.1002/jcph.2443

BibTeX
@article{vanNuland2024,
  author = {van Nuland, Merel and Erdogan, Abdullah and Aςar, Cenkay and Contrucci, Ramon and Hilbrants, Sven and Maanach, Lamyae and Egberts, Toine and van der Linden, Paul D},
  title = {Performance of ChatGPT on Factual Knowledge Questions Regarding Clinical Pharmacy},
  journal = {The Journal of Clinical Pharmacology},
  year = {2024},
  volume = {64},
  number = {9},
  pages = {1095--1100},
  doi = {10.1002/jcph.2443}
}
Kostenlos starten →