EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Bedside Triage by Large Language Models in Acute Pancreatitis: A Scenario‐Based Comparative Evaluation of GPT‐4, GPT‐5, and Gemini

Yahya Kemal Çalışkan iD, Fatih Başak iD, Olgun Erdem iD

World Journal of Surgery · 2026-08 · 50 (8) · 2321-2328

DOI: 10.1002/wjs.70492 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift World Journal of Surgery, online erschienen am 09.07.2026. Verfasst von Yahya Kemal Çalışkan und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of General Surgery, University of Health Sciences, Kanuni Training and Research Hospital, Istanbul, Turkey. Hauptthemen laut MEDLINE: Large Language Models, Pancreatitis, Triage, Clinical Decision-Making. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 16 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Early decision‐making in acute pancreatitis (AP) involves diagnostic confirmation, early severity triage, escalation thresholds, and initiation of guideline‐concordant management under time pressure and incomplete information. Large language models (LLMs) may support structured bedside reasoning, but their clinical usefulness cannot be inferred from guideline knowledge alone.

Methods

A cross‐sectional, scenario‐based comparative evaluation was conducted in January 2026 using 20 AP scenarios: 15 refined hypothetical vignettes and 5 de‐identified, privacy‐modified real‐life case patterns. GPT‐4, GPT‐5, and Gemini received identical single‐turn prompts. Model access was through OpenAI API gpt‐4‐0613, OpenAI API gpt‐5, and Google Vertex AI Gemini 1.0 Pro; temperature was set to 0.0, and each prompt was repeated three times per model. Outputs were scored by two independent clinician‐raters using a prespecified 1–5 ordinal rubric across guideline concordance, safety, actionability, and data‐synthesis quality. Two senior board‐certified surgeons independently generated expert reference pathways for comparison.

Results

GPT‐5 achieved the highest guideline concordance (4.28 ± 0.38) and safety (4.20 ± 0.45) profiles. GPT‐4 provided the clearest stepwise actionability (4.15 ± 0.48), whereas Gemini showed the strongest data‐synthesis quality (4.22 ± 0.52). With deterministic settings, internal consistency across three repeated runs was 100%. All models demonstrated clinically relevant failure modes, particularly unwarranted certainty under missing data; this occurred in 12/20 GPT‐4, 7/20 GPT‐5, and 15/20 Gemini outputs.

Conclusion

No model should be used as a stand‐alone bedside decision‐maker for AP. In this scenario‐based early evaluation, GPT‐5 was the most safety‐aligned model, GPT‐4 was the most operationally actionable, and Gemini was strongest for synthesis, but all require clinician oversight, prospective validation, and governance before clinical deployment.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsPancreatitisTriageClinical Decision-MakingHumansCross-Sectional StudiesAcute DiseaseGenerative Artificial Intelligence

Stichwörter der Autor:innen

acute pancreatitisclinical decision supportguideline concordancelarge language modelssafetyscenario‐based evaluationtriage

Autor:innen und Institutionen

  • Department of General Surgery, University of Health Sciences, Kanuni Training and Research Hospital, Istanbul, Turkey.
  • Department of General Surgery, University of Health Sciences, Umraniye Training and Research Hospital, Istanbul, Turkey.

ORCID-Profile: Yahya Kemal Çalışkan, Fatih Başak, Olgun Erdem

Zitieren

APA

Çalışkan, Y. K., Başak, F., & Erdem, O. (2026). Bedside Triage by Large Language Models in Acute Pancreatitis: A Scenario‐Based Comparative Evaluation of GPT‐4, GPT‐5, and Gemini. World Journal of Surgery, 50(8), 2321-2328. https://doi.org/10.1002/wjs.70492

Vancouver

Çalışkan YK, Başak F, Erdem O. Bedside Triage by Large Language Models in Acute Pancreatitis: A Scenario‐Based Comparative Evaluation of GPT‐4, GPT‐5, and Gemini. World Journal of Surgery. 2026;50(8):2321-2328. doi:10.1002/wjs.70492

BibTeX
@article{alkan2026,
  author = {Çalışkan, Yahya Kemal and Başak, Fatih and Erdem, Olgun},
  title = {Bedside Triage by Large Language Models in Acute Pancreatitis: A Scenario‐Based Comparative Evaluation of GPT‐4, GPT‐5, and Gemini},
  journal = {World Journal of Surgery},
  year = {2026},
  volume = {50},
  number = {8},
  pages = {2321--2328},
  doi = {10.1002/wjs.70492}
}
Kostenlos starten →