Pre‐implementation safety evaluation of an AI decision‐support system for surgical antimicrobial prophylaxis
Internal Medicine Journal · 2026-08 · 56 (8) · 1349-1355
DOI: 10.1111/imj.70470 ↗Kurz gefasst
Fachartikel in der Zeitschrift Internal Medicine Journal, online erschienen am 10.06.2026. Verfasst von Joshua Andrews und 4 weiteren Autor:innen. Beteiligte Einrichtung: Central Adelaide Local Health Network, Adelaide, South Australia, Australia. Hauptthemen laut MEDLINE: Antimicrobial Stewardship, Antibiotic Prophylaxis, Decision Support Systems, Clinical, Artificial Intelligence. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Background
Large language models (LLMs) show potential to support antimicrobial prescribing but require simulation‐based, institution‐specific safety evaluation prior to any consideration of clinical use. In Australia, antimicrobial prescribing represents a high‐risk domain for digital decision‐support systems due to patient safety and antimicrobial resistance implications.
Aim
To characterise prescribing accuracy, error phenotypes and antimicrobial stewardship risk associated with a LLM that was provided with publicly available surgical prophylaxis guidelines during inference (without fine‐tuning or model modification) across 20 simulated surgical scenarios.
Methods
Twenty simulated surgical scenarios were tested using a LLM that was prompt‐conditioned with publicly available guideline text during inference, without any fine‐tuning or modification of model weights. For each case, the model generated recommendations for agent, dose, timing, re‐dosing and guideline citation. Outputs were independently assessed by two local clinicians familiar with the guideline, with accuracy scored across five domains and harm classified using a modified National Coordinating Council for Medical Error Reporting and Prevention (NCC MERP) Index.
Results
Clinically significant antimicrobial prescribing risk was identified in 10% of simulated scenarios (2/20), recognising wide confidence intervals due to the small sample size. These included omission of required anaerobic coverage and failure to redose prophylaxis in prolonged procedures. Overall guideline concordance was 4/5, with perfect dose accuracy but lower performance for timing (70%) and guideline citation (45%).
Conclusions
This study demonstrates the feasibility of constructing institutionally governed, guideline‐based AI systems while identifying stewardship‐relevant safety risks that currently preclude clinical use without further validation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Substanzen
Autor:innen und Institutionen
- Central Adelaide Local Health Network, Adelaide, South Australia, Australia.
- Adelaide Medical School, The University of Adelaide, Adelaide, South Australia, Australia.
- Amsterdam University Medical Centre, Meibergdreef, Netherlands.
- Harvard Medical School, Boston, Massachusetts, USA.
ORCID-Profile: Joshua Andrews
Zitieren
APA
Andrews, J., Kanjilal, S., Kotton, C., Stretton, B., & Leslie, A. (2026). Pre‐implementation safety evaluation of an AI decision‐support system for surgical antimicrobial prophylaxis. Internal Medicine Journal, 56(8), 1349-1355. https://doi.org/10.1111/imj.70470
Vancouver
Andrews J, Kanjilal S, Kotton C, Stretton B, Leslie A. Pre‐implementation safety evaluation of an AI decision‐support system for surgical antimicrobial prophylaxis. Internal Medicine Journal. 2026;56(8):1349-1355. doi:10.1111/imj.70470
BibTeX
@article{Andrews2026,
author = {Andrews, Joshua and Kanjilal, Sanjat and Kotton, Camile and Stretton, Brandon and Leslie, Alasdair},
title = {Pre‐implementation safety evaluation of an AI decision‐support system for surgical antimicrobial prophylaxis},
journal = {Internal Medicine Journal},
year = {2026},
volume = {56},
number = {8},
pages = {1349--1355},
doi = {10.1111/imj.70470}
}
EUVIMED