EUVIMEDEUVIMED entdecken
Crossref

Testing knowledge boundaries: adversarial evaluation of large language models for antimicrobial stewardship

Ángela Abejez-Arrizabalaga, Galadriel Pellejero-Sagastizabal, Rocío Aznar-Gimeno, Beatriz Franco, Santiago Letona-Giménez, Elena Morte-Romea, Julia Origüen, María Teresa Pérez-Rodríguez, Pilar Retamar, Esmeé Ruizendaal, Teske Schoffelen, Jeroen Schouten und 3 weitere

Clinical Microbiology and Infection · 2026-08

DOI: 10.1016/j.cmi.2026.08.025 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Clinical Microbiology and Infection, 2026-08. Verfasst von Ángela Abejez-Arrizabalaga und 14 weiteren Autor:innen. Beteiligte Einrichtung: Division of Infectious Diseases, Hospital Clínico Universitario Lozano Blesa, Zaragoza, Spain; Clinical Research Group on Infectious Diseases (GIIS06…. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 15 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

OBJECTIVES: Evaluate whether general-purpose large language models (LLMs) demonstrate competencies suitable for antimicrobial stewardship (AMS) support and characterize their failure modes. METHODS: Cross-sectional evaluation of seven LLMs (GPT-5, Claude Sonnet 4.5, Gemini 2.5 Pro, Grok 4, Llama-3.3-70b-instruct, Qwen 2.5-72b-instruct, and DeepSeek-chat-v3.1) using 30 clinical scenarios mapped to European Society of Clinical Microbiology and Infectious Diseases AMS competency frameworks. Scenarios included deliberate traps for fabrication and dangerous recommendations. Six AMS experts from the Netherlands and Spain performed blinded dual evaluation using content scores (0-5 scale) and binary safety flags for fabrication and danger. Standard and incentivizing prompt framings were compared. RESULTS: Four commercial models achieved mean content scores above 3.9/5.0: Claude Sonnet 4.5 (4.06), Gemini 2.5 Pro (3.96), Grok 4 (3.96), and GPT-5 (3.94). Open-weight models scored significantly lower (2.94-3.57). No model achieved more than 63% responses free of fabrication or danger flags. However, fabrication did not impair clinical utility in nontrap scenarios (all within-category comparisons p > 0.20). Danger flags ranged from 6.7% to 16.7% across models, with no significant difference between commercial and open-weight models. Incentivizing prompts were associated with a consistent 0.48-point-content score improvement (p 0.006), although significance attenuated after accounting for scenario-level clustering. Evaluators endorsed LLMs as useful AMS support tools with moderate supervision (5/6), identifying documentation preparation and trainee education as promising applications. CONCLUSIONS: Medically untrained LLMs demonstrate competencies suitable for supervised AMS support. Fabrication remains the central safety challenge and requires verification workflows; danger, although less frequent (6.7-16.7%), concentrated in identifiable and therefore mitigable failure modes. Nonclinical stewardship tasks (education, documentation, communication) can benefit now, whereas clinical recommendations require expert oversight. Mapping these boundaries allows AMS teams, particularly those understaffed or without on-site infectious diseases expertise, to decide where LLM support adds value rather than risk.

Schlagwörter

Stichwörter der Autor:innen

Antimicrobial stewardshipArtificial intelligenceClinical-decision supportFabricationLarge language modelsPatient safety

Autor:innen und Institutionen

  • Division of Infectious Diseases, Hospital Clínico Universitario Lozano Blesa, Zaragoza, Spain; Clinical Research Group on Infectious Diseases (GIIS069), Instituto de Investigación Sanitaria Aragón, Zaragoza, Spain; Cent…
  • Division of Infectious Diseases, Hospital Clínico Universitario Lozano Blesa, Zaragoza, Spain; Clinical Research Group on Infectious Diseases (GIIS069), Instituto de Investigación Sanitaria Aragón, Zaragoza, Spain.
  • Department of Big Data and Cognitive Systems, Instituto Tecnológico de Aragón (ITA), Zaragoza, Spain.
  • Infectious Diseases Unit, Internal Medicine Department, Complexo Hospitalario Universitario de Vigo, Vigo, Spain; Infectious Diseases Research, Galicia Sur Health Research Institute, Vigo, Spain.
  • Centro de Investigación Biomédica en Red de Enfermedades Infecciosas (CIBERINFEC), Instituto de Salud Carlos III, Madrid, Spain; Division of Infectious Diseases, Hospital Universitario Virgen Macarena, Sevilla, Spain; D…
  • Department of Medical Microbiology, Radboudumc, Nijmegen, The Netherlands.

Zitieren

APA

Abejez-Arrizabalaga, Á., Pellejero-Sagastizabal, G., Aznar-Gimeno, R., Franco, B., Letona-Giménez, S., Morte-Romea, E., Origüen, J., Pérez-Rodríguez, M. T., Retamar, P., Ruizendaal, E., Schoffelen, T., Schouten, J., Sisamón, N., Hoyo, R. D., & Paño-Pardo, J. R. (2026). Testing knowledge boundaries: adversarial evaluation of large language models for antimicrobial stewardship. Clinical Microbiology and Infection. https://doi.org/10.1016/j.cmi.2026.08.025

Vancouver

Abejez-Arrizabalaga Á, Pellejero-Sagastizabal G, Aznar-Gimeno R, Franco B, Letona-Giménez S, Morte-Romea E, et al. Testing knowledge boundaries: adversarial evaluation of large language models for antimicrobial stewardship. Clinical Microbiology and Infection. 2026. doi:10.1016/j.cmi.2026.08.025

BibTeX
@article{AbejezArrizabalaga2026,
  author = {Abejez-Arrizabalaga, Ángela and Pellejero-Sagastizabal, Galadriel and Aznar-Gimeno, Rocío and Franco, Beatriz and Letona-Giménez, Santiago and Morte-Romea, Elena and Origüen, Julia and Pérez-Rodríguez, María Teresa and Retamar, Pilar and Ruizendaal, Esmeé and Schoffelen, Teske and Schouten, Jeroen and Sisamón, Natalia and Hoyo, Rafael Del and Paño-Pardo, José Ramón},
  title = {Testing knowledge boundaries: adversarial evaluation of large language models for antimicrobial stewardship},
  journal = {Clinical Microbiology and Infection},
  year = {2026},
  doi = {10.1016/j.cmi.2026.08.025}
}
Kostenlos starten →