EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Evaluating and leveraging large language models in clinical pharmacology and therapeutics assessment: From exam takers to exam shapers

Alexandre O Gérard iD, Diane Merino iD, Marc Labriffe iD, Fanny Rocher, Delphine Viard iD, Laurence Zemori, Thibaud Lavrut, Erik M Donker iD, Joost D Piët iD, Jean-Paul Fournier iD, Milou-Daniel Drici iD, Alexandre Destere iD

British Journal of Clinical Pharmacology · 2026-08 · 92 (8) · 2574-2583

DOI: 10.1002/bcp.70137 ↗

Kurz gefasst

Fachartikel in der Zeitschrift British Journal of Clinical Pharmacology, online erschienen am 10.06.2025. Verfasst von Alexandre O Gérard und 11 weiteren Autor:innen. Beteiligte Einrichtung: Université Côte d'Azur Medical Centre, Department of Clinical Pharmacology and Pharmacovigilance Center, Nice, France. Hauptthemen laut MEDLINE: Large Language Models, Educational Measurement, Pharmacology, Clinical, Education, Medical, Undergraduate. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aims

In medical education, the ability of large language models (LLMs) to match human performance raises questions about their potential as educational tools. This study evaluates LLMs' performance on Clinical Pharmacology and Therapeutics (CPT) exams, comparing their results to medical students and exploring their ability to identify poorly formulated multiple‐choice questions (MCQs).

Methods

ChatGPT‐4 Omni, Gemini Advanced, Le Chat and DeepSeek R1 were tested on local CPT exams (third year of bachelor's degree, first/second year of master's degree) and the European Prescribing Exam (EuroPE + ). The exams included MCQs and open‐ended questions assessing knowledge and prescribing skills. LLM results were analysed using the same scoring system as students. A confusion matrix was used to evaluate the ability of ChatGPT and Gemini to identify ambiguous/erroneous MCQs.

Results

LLMs achieved comparable or superior results to medical students across all levels. For local exams, LLMs outperformed M1 students and matched L3 and M2 students. In EuroPE + , LLMs significantly outperformed students in both the knowledge and prescribing skills sections. All LLM errors in EuroPE + were genuine (100%), whereas local exam errors were frequently due to ambiguities or correction flaws (24.3%). When both ChatGPT and Gemini provided the same incorrect answer to an MCQ, the specificity for detecting ambiguous questions was 92.9%, with a negative predictive value of 85.5%.

Conclusion

LLMs demonstrate capabilities comparable to or exceeding medical students in CPT exams. Their ability to flag potentially flawed MCQs highlights their value not only as educational tools but also as quality control instruments in exam preparation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsEducational MeasurementPharmacology, ClinicalEducation, Medical, UndergraduateHumansStudents, MedicalClinical Competence

Stichwörter der Autor:innen

large language modelmedical studypedagogypharmacology, therapeutics

Autor:innen und Institutionen

  • Université Côte d'Azur Medical Centre, Department of Clinical Pharmacology and Pharmacovigilance Center, Nice, France.
  • Pharmacology and Transplantation, INSERM U1248, Université de Limoges, Limoges, France.
  • Department of Pharmacology, Toxicology and Pharmacovigilance, University Hospital of Limoges, Limoges, France.
  • Research and Expertise Centre in Pharmacotherapy Education (RECIPE), Amsterdam, The Netherlands.
  • Department of Internal Medicine, Unit Pharmacotherapy, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands.
  • Université Côte d'Azur, Faculty of Medicine of Nice, Medical Simulation Center, Nice, France.

ORCID-Profile: Alexandre O Gérard, Diane Merino, Marc Labriffe, Delphine Viard, Erik M Donker, Joost D Piët, Jean-Paul Fournier, Milou-Daniel Drici, Alexandre Destere

Zitieren

APA

Gérard, A. O., Merino, D., Labriffe, M., Rocher, F., Viard, D., Zemori, L., Lavrut, T., Donker, E. M., Piët, J. D., Fournier, J. P., Drici, M. D., & Destere, A. (2026). Evaluating and leveraging large language models in clinical pharmacology and therapeutics assessment: From exam takers to exam shapers. British Journal of Clinical Pharmacology, 92(8), 2574-2583. https://doi.org/10.1002/bcp.70137

Vancouver

Gérard AO, Merino D, Labriffe M, Rocher F, Viard D, Zemori L, et al. Evaluating and leveraging large language models in clinical pharmacology and therapeutics assessment: From exam takers to exam shapers. British Journal of Clinical Pharmacology. 2026;92(8):2574-2583. doi:10.1002/bcp.70137

BibTeX
@article{Grard2026,
  author = {Gérard, Alexandre O and Merino, Diane and Labriffe, Marc and Rocher, Fanny and Viard, Delphine and Zemori, Laurence and Lavrut, Thibaud and Donker, Erik M and Piët, Joost D and Fournier, Jean-Paul and Drici, Milou-Daniel and Destere, Alexandre},
  title = {Evaluating and leveraging large language models in clinical pharmacology and therapeutics assessment: From exam takers to exam shapers},
  journal = {British Journal of Clinical Pharmacology},
  year = {2026},
  volume = {92},
  number = {8},
  pages = {2574--2583},
  doi = {10.1002/bcp.70137}
}
Kostenlos starten →