EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

GP or ChatGPT? Ability of large language models (LLMs) to support general practitioners when prescribing antibiotics

Oanh Ngoc Nguyen, Doaa Amin, James Bennett, Øystein Hetlevik, Sara Malik, Andrew Tout, Heike Vornhagen, Akke Vellinga iD

Journal of Antimicrobial Chemotherapy · 2025

DOI: 10.1093/jac/dkaf077 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Antimicrobial Chemotherapy, 2025. Verfasst von Oanh Ngoc Nguyen und 7 weiteren Autor:innen. Beteiligte Einrichtung: CARA Network, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland. Hauptthemen laut MEDLINE: Anti-Bacterial Agents, General Practitioners, Practice Patterns, Physicians', Drug Prescriptions. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Leader, Health Research Board, Ireland, SPHeRE.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Large language models (LLMs) are becoming ubiquitous and widely implemented. LLMs could also be used for diagnosis and treatment. National antibiotic prescribing guidelines are customized and informed by local laboratory data on antimicrobial resistance.

Methods

Based on 24 vignettes with information on type of infection, gender, age group and comorbidities, GPs and LLMs were prompted to provide a treatment. Four countries (Ireland, UK, USA and Norway) were included and a GP from each country and six LLMs (ChatGPT, Gemini, Copilot, Mistral AI, Claude and Llama 3.1) were provided with the vignettes, including their location (country). Responses were compared with the country’s national prescribing guidelines. In addition, limitations of LLMs such as hallucination, toxicity and data leakage were assessed.

Results

GPs’ answers to the vignettes showed high accuracy in relation to diagnosis (96%–100%) and yes/no antibiotic prescribing (83%–92%). GPs referenced (100%) and prescribed (58%–92%) according to national guidelines, but dose/duration of treatment was less accurate (50%–75%). Overall, the GPs’ accuracy had a mean of 74%. LLMs scored high in relation to diagnosis (92%–100%), antibiotic prescribing (88%–100%) and the choice of antibiotic (59%–100%) but correct referencing often failed (38%–96%), in particular for the Norwegian guidelines (0%–13%). Data leakage was shown to be an issue as personal information was repeated in the models’ responses to the vignettes.

Conclusions

LLMs may be safe to guide antibiotic prescribing in general practice. However, to interpret vignettes, apply national guidelines and prescribe the right dose and duration, GPs remain best placed.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Anti-Bacterial AgentsGeneral PractitionersPractice Patterns, Physicians'Drug PrescriptionsHumansMaleFemaleMiddle AgedAdultNorwayUnited KingdomLarge Language ModelsGenerative Artificial Intelligence

Substanzen

Anti-Bacterial Agents

Autor:innen und Institutionen

  • CARA Network, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland.
  • NIHR In Practice Fellow, Hull York Medical School, University of Hull, Hull HU6 7RX, UK.
  • Department of Global Public Health and Primary Care, University of Bergen, Bergen, Norway.
  • Midleton Medi Center, Midleton, Co Cork, Ireland.
  • Division of General Internal Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
  • CARA Network, Insight Centre for Data Analytics, University of Galway, Galway, Ireland.

ORCID-Profile: Akke Vellinga

Förderung

  • Research Leader
  • Health Research Board, Ireland
  • SPHeRE

Zitieren

APA

Ngoc Nguyen, O., Amin, D., Bennett, J., Hetlevik, Ø., Malik, S., Tout, A., Vornhagen, H., & Vellinga, A. (2025). GP or ChatGPT? Ability of large language models (LLMs) to support general practitioners when prescribing antibiotics. Journal of Antimicrobial Chemotherapy. https://doi.org/10.1093/jac/dkaf077

Vancouver

Ngoc Nguyen O, Amin D, Bennett J, Hetlevik Ø, Malik S, Tout A, et al. GP or ChatGPT? Ability of large language models (LLMs) to support general practitioners when prescribing antibiotics. Journal of Antimicrobial Chemotherapy. 2025. doi:10.1093/jac/dkaf077

BibTeX
@article{NgocNguyen2025,
  author = {Ngoc Nguyen, Oanh and Amin, Doaa and Bennett, James and Hetlevik, Øystein and Malik, Sara and Tout, Andrew and Vornhagen, Heike and Vellinga, Akke},
  title = {GP or ChatGPT? Ability of large language models (LLMs) to support general practitioners when prescribing antibiotics},
  journal = {Journal of Antimicrobial Chemotherapy},
  year = {2025},
  doi = {10.1093/jac/dkaf077}
}
Kostenlos starten →