EUVIMEDEUVIMED entdecken
VergleichsstudieValidierungsstudieMultizentrische StudieOpen Access · CC BY-NC 4.0

Comparison of large language models and conventional machine learning in postoperative outcome prediction: a retrospective, multi-national development and validation study

Jipyeong Lee, Hyeonsik Kim, Luke Kim, Leerang Lim, Hyung-Chul Lee, Hyeonhoon Lee

Korean Journal of Anesthesiology · 2026

DOI: 10.4097/kja.25646 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Korean Journal of Anesthesiology, 2026. Verfasst von Jipyeong Lee und 5 weiteren Autor:innen. Beteiligte Einrichtung: Healthcare AI Research Institute, Seoul National University Hospital, Seoul, Korea. Hauptthemen laut MEDLINE: Large Language Models, Machine Learning, Postoperative Complications. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Korea Health Industry Development Institute, Ministry of Health and Welfare, NAVER Corporation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Conventional machine learning (ML) models for predicting surgical outcomes have limitations in generalizability. We explored large language models (LLMs) as scalable alternatives to conventional ML models in predicting postoperative outcomes, including in-hospital 30-day mortality, intensive care unit (ICU) admission, and acute kidney injury (AKI).

Methods

This study utilized the Informative Surgical Patient for Innovative Research Environment (INSPIRE) dataset (n = 80 985) from South Korea for model development and internal validation, and the Medical Informatics Operating Room Vitals and Events Repository (MOVER) dataset (n = 6265) from the United States for external validation. The study compared three different LLMs—Generative Pre-trained Transformer [GPT]-4o, Llama-3-70B, and OpenBioLLM-70B—against MLs using various prompt engineering approaches. LLMs were evaluated with different model parameter quantizations (4-bit normalized floating point vs. 16-bit brain floating point).

Results

OpenBioLLM-70B was comparable to eXtreme Gradient Boosting (XGBoost) across all tasks (in-hospital 30-day mortality: area under receiver operating characteristic curve [AUROC] 0.782 [95% CI, 0.748–0.813] vs. 0.791 [95% CI, 0.753–0.825]; ICU admission: AUROC 0.595 [95% CI, 0.581–0.609] vs. 0.594 [95% CI, 0.580–0.608]; AKI: AUROC 0.830 [95% CI, 0.802–0.855] vs. 0.823 [95% CI, 0.792–0.851]) during external validation. Open-source LLMs maintained performance with 4-bit quantization, reducing computational requirements by 75%.

Conclusions

The findings support the versatility and efficiency of LLMs for clinical decision support through on-premises compatibility, addressing data privacy. Further validation with diverse datasets is needed to ensure their reliability and applicability across different perioperative settings.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsMachine LearningPostoperative ComplicationsAgedFemaleHumansMaleMiddle AgedHospital MortalityPredictive Learning ModelsRepublic of KoreaRetrospective StudiesUnited States

Stichwörter der Autor:innen

Acute kidney injuryClinical decision support systemsLarge language modelsMachine learningMortalityPatient readmissionPerioperative medicine

Autor:innen und Institutionen

  • Healthcare AI Research Institute, Seoul National University Hospital, Seoul, Korea.
  • Department of Interdisciplinary Program in Medical Informatics, Seoul National University, Seoul, Korea.
  • Monash School of Medicine, Monash University, Victoria, Australia.
  • Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Korea.
  • Healthcare AI Research Institute, Seoul National University Hospital, Seoul, Korea. hhoon@snu.ac.kr.
  • Department of Medicine, Seoul National University College of Medicine, Seoul, Korea. hhoon@snu.ac.kr.

Förderung

  • Korea Health Industry Development Institute
  • Ministry of Health and Welfare
  • NAVER Corporation

Zitieren

APA

Lee, J., Kim, H., Kim, L., Lim, L., Lee, H. C., & Lee, H. (2026). Comparison of large language models and conventional machine learning in postoperative outcome prediction: a retrospective, multi-national development and validation study. Korean Journal of Anesthesiology. https://doi.org/10.4097/kja.25646

Vancouver

Lee J, Kim H, Kim L, Lim L, Lee HC, Lee H. Comparison of large language models and conventional machine learning in postoperative outcome prediction: a retrospective, multi-national development and validation study. Korean Journal of Anesthesiology. 2026. doi:10.4097/kja.25646

BibTeX
@article{Lee2026,
  author = {Lee, Jipyeong and Kim, Hyeonsik and Kim, Luke and Lim, Leerang and Lee, Hyung-Chul and Lee, Hyeonhoon},
  title = {Comparison of large language models and conventional machine learning in postoperative outcome prediction: a retrospective, multi-national development and validation study},
  journal = {Korean Journal of Anesthesiology},
  year = {2026},
  doi = {10.4097/kja.25646}
}
Kostenlos starten →