EUVIMEDEUVIMED entdecken
Vergleichsstudie

Off‐the‐Shelf Large Language Models for Guiding Pharmacoepidemiological Study Design

Gerard Ompad iD, Keele Wurst iD, Darmendra Ramcharran iD, Anders Hviid, Andrew Bate iD, Maurizio Sessa iD

Clinical Pharmacology & Therapeutics · 2025

DOI: 10.1002/cpt.70039 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Clinical Pharmacology & Therapeutics, 2025. Verfasst von Gerard Ompad und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Drug Design and Pharmacology, University of Copenhagen, Copenhagen, Denmark. Hauptthemen laut MEDLINE: Large Language Models, Pharmacoepidemiology, Research Design, Clinical Trial Protocols as Topic. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 39 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study aimed to assess the ability of two off‐the‐shelf large language models, ChatGPT and Gemini, to support the design of pharmacoepidemiological studies. We assessed 48 study protocols of pharmacoepidemiological studies published between 2018 and 2024, covering various study types, including disease epidemiology, drug utilization, safety, and effectiveness. The coherence (i.e., “Is the response coherent and well‐formed, or is it difficult to understand?”) and relevance (i.e., “Is the response relevant and informative, or is it lacking in substance?”) of the large language models' responses were evaluated by human experts across seven key study design components. Coding accuracy was assessed. Both large language models demonstrated high coherence, with over 90% of study components rated as “Strongly agree” by experts for most categories. ChatGPT achieved the highest coherence for “Index date” (97.9%) and “Study design” (95.8%). Gemini excelled in “Study outcome” (93.9%) and “Study exposure” (95.9%). Relevance, however, was more variable, with ChatGPT aligning with expert recommendations in over 90% of cases for “Index date” and “Study design” but showing lower agreement for covariates (65%) and follow‐up (70%). Coding agreement percentages reveal varying levels of concordance, with the Anatomical Therapeutic Chemical classification system coding system demonstrating the highest agreement at 50% with experts. In contrast, the Current Procedural Terminology and International Classification of Diseases systems showed agreements of 22.2% and 20%, respectively. While ChatGPT and Gemini show promise in certain tasks supporting pharmacoepidemiological study design, their limitations in relevance and coding accuracy highlight the need for critical oversight by domain experts.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsPharmacoepidemiologyResearch DesignClinical Trial Protocols as TopicGenerative Artificial IntelligenceHumansReproducibility of Results

Autor:innen und Institutionen

  • Department of Drug Design and Pharmacology, University of Copenhagen, Copenhagen, Denmark.
  • Epidemiology, GSK, Research Triangle Park, North Carolina, USA.
  • Safety and Quantitative Innovation, GSK, Waltham, Massachusetts, USA.
  • Department of Pharmacy Practice, School of Pharmacy, University of Rhode Island, Kingston, Rhode Island, USA.
  • Department of Epidemiology Research, Statens Serum Institute, Copenhagen, Denmark.
  • Global Safety, GSK, Brentford, UK.

ORCID-Profile: Gerard Ompad, Keele Wurst, Darmendra Ramcharran, Andrew Bate, Maurizio Sessa

Zitieren

APA

Ompad, G., Wurst, K., Ramcharran, D., Hviid, A., Bate, A., & Sessa, M. (2025). Off‐the‐Shelf Large Language Models for Guiding Pharmacoepidemiological Study Design. Clinical Pharmacology & Therapeutics. https://doi.org/10.1002/cpt.70039

Vancouver

Ompad G, Wurst K, Ramcharran D, Hviid A, Bate A, Sessa M. Off‐the‐Shelf Large Language Models for Guiding Pharmacoepidemiological Study Design. Clinical Pharmacology & Therapeutics. 2025. doi:10.1002/cpt.70039

BibTeX
@article{Ompad2025,
  author = {Ompad, Gerard and Wurst, Keele and Ramcharran, Darmendra and Hviid, Anders and Bate, Andrew and Sessa, Maurizio},
  title = {Off‐the‐Shelf Large Language Models for Guiding Pharmacoepidemiological Study Design},
  journal = {Clinical Pharmacology & Therapeutics},
  year = {2025},
  doi = {10.1002/cpt.70039}
}
Kostenlos starten →