EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Comparing large language models for antibiotic prescribing in different clinical scenarios: which performs better?

Andrea De Vito, Nicholas Geremia, Davide Fiore Bavaro, Susan K Seo, Justin Laracy, Maria Mazzitelli, Andrea Marino, Alberto Enrico Maraolo, Antonio Russo, Agnese Colpani, Michele Bartoletti, Anna Maria Cattelan und 5 weitere

Clinical Microbiology and Infection · 2025-08 · 31 (8) · 1336-1342

DOI: 10.1016/j.cmi.2025.03.002 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Clinical Microbiology and Infection, 2025-08. Verfasst von Andrea De Vito und 16 weiteren Autor:innen. Beteiligte Einrichtung: Unit of Infectious Diseases, Department of Medicine, Surgery and Pharmacy, Sassari, Italy. Electronic address: andreadevitoaho@gmail.com. Hauptthemen laut MEDLINE: Anti-Bacterial Agents, Drug Prescriptions, Practice Patterns, Physicians'. Die Arbeit wurde bisher 54-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

OBJECTIVES: Large language models (LLMs) show promise in clinical decision-making, but comparative evaluations of their antibiotic prescribing accuracy are limited. This study assesses the performance of various LLMs in recommending antibiotic treatments across diverse clinical scenarios. METHODS: Fourteen LLMs, including standard and premium versions of ChatGPT, Claude, Copilot, Gemini, Le Chat, Grok, Perplexity, and Pi.ai, were evaluated using 60 clinical cases with antibiograms covering 10 infection types. A standardized prompt was used for antibiotic recommendations focusing on drug choice, dosage, and treatment duration. Responses were anonymized and reviewed by a blinded expert panel assessing antibiotic appropriateness, dosage correctness, and duration adequacy. RESULTS: A total of 840 responses were collected and analysed. ChatGPT-o1 demonstrated the highest accuracy in antibiotic prescriptions, with 71.7% (43/60) of its recommendations classified as correct and only one (1.7%) incorrect. Gemini and Claude 3 Opus had the lowest accuracy. Dosage correctness was highest for ChatGPT-o1 (96.7%, 58/60), followed by Perplexity Pro (90.0%, 54/60) and Claude 3.5 Sonnet (91.7%, 55/60). In treatment duration, Gemini provided the most appropriate recommendations (75.0%, 45/60), whereas Claude 3.5 Sonnet tended to over-prescribe duration. Performance declined with increasing case complexity, particularly for difficult-to-treat microorganisms. DISCUSSION: There is significant variability among LLMs in prescribing appropriate antibiotics, dosages, and treatment durations. ChatGPT-o1 outperformed other models, indicating the potential of advanced LLMs as decision-support tools in antibiotic prescribing. However, decreased accuracy in complex cases and inconsistencies among models highlight the need for careful validation before clinical utilization.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Anti-Bacterial AgentsDrug PrescriptionsPractice Patterns, Physicians'HumansClinical Decision-MakingAntimicrobial StewardshipMicrobial Sensitivity TestsLarge Language Models

Stichwörter der Autor:innen

Antibiotic treatmentAntimicrobial susceptibility testingChatGPT-o1Difficult-to-treat infectionLLMsLarge language models

Substanzen

Anti-Bacterial Agents

Autor:innen und Institutionen

  • Unit of Infectious Diseases, Department of Medicine, Surgery and Pharmacy, Sassari, Italy. Electronic address: andreadevitoaho@gmail.com.
  • Unit of Infectious Diseases, Department of Clinical Medicine, Ospedale dell'Angelo, Venice, Italy; Unit of Infectious Diseases, Department of Clinical Medicine, Ospedale Civile S.S. Giovanni e Paolo, Venice, Italy.
  • Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Milan, Italy; Infectious Diseases Unit - Department of Biomedical Sciences - Istituti di Ricovero e Cura a Carattere Scientifico (IRCCS) Humanitas…
  • Infectious Diseases Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, USA.
  • Infectious and Tropical Diseases Unit, Department of Molecular Medicine, Padua University Hospital, Padua, Italy.
  • Unit of Infectious Diseases, Department of Clinical and Experimental Medicine, Azienda Ospedaliera di Rilievo Nazionale e di Alta Specializzazione (ARNAS), Garibaldi Hospital, University of Catania, Catania, Italy.

Zitieren

APA

De Vito, A., Geremia, N., Bavaro, D. F., Seo, S. K., Laracy, J., Mazzitelli, M., Marino, A., Maraolo, A. E., Russo, A., Colpani, A., Bartoletti, M., Cattelan, A. M., Mussini, C., Parisi, S. G., Vaira, L. A., Nunnari, G., & Madeddu, G. (2025). Comparing large language models for antibiotic prescribing in different clinical scenarios: which performs better?. Clinical Microbiology and Infection, 31(8), 1336-1342. https://doi.org/10.1016/j.cmi.2025.03.002

Vancouver

De Vito A, Geremia N, Bavaro DF, Seo SK, Laracy J, Mazzitelli M, et al. Comparing large language models for antibiotic prescribing in different clinical scenarios: which performs better?. Clinical Microbiology and Infection. 2025;31(8):1336-1342. doi:10.1016/j.cmi.2025.03.002

BibTeX
@article{DeVito2025,
  author = {De Vito, Andrea and Geremia, Nicholas and Bavaro, Davide Fiore and Seo, Susan K and Laracy, Justin and Mazzitelli, Maria and Marino, Andrea and Maraolo, Alberto Enrico and Russo, Antonio and Colpani, Agnese and Bartoletti, Michele and Cattelan, Anna Maria and Mussini, Cristina and Parisi, Saverio Giuseppe and Vaira, Luigi Angelo and Nunnari, Giuseppe and Madeddu, Giordano},
  title = {Comparing large language models for antibiotic prescribing in different clinical scenarios: which performs better?},
  journal = {Clinical Microbiology and Infection},
  year = {2025},
  volume = {31},
  number = {8},
  pages = {1336--1342},
  doi = {10.1016/j.cmi.2025.03.002}
}
Kostenlos starten →