EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access

American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study

Christabelle Pabalan, Alvin Stewart, Gabrielle Fisher, Andrew Fisher

Journal of Medical Internet Research · 05.11.2026 · 28 · e90724-e90724

DOI: 10.2196/90724 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 05.11.2026. Verfasst von Christabelle Pabalan und 3 weiteren Autor:innen. Beteiligte Einrichtung: Qventus Inc, Mountain View, CA, United States. Hauptthemen laut MEDLINE: Machine Learning, Anesthesiologists. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The American Society of Anesthesiologists Physical Status (ASA-PS) classification system fundamentally shapes perioperative care delivery but suffers from poor interrater reliability (0.4‐0.6). Machine learning (ML) models and large language models (LLMs) offer complementary remedies: ML processes structured data consistently but cannot interpret the nuance of clinical narratives, while LLMs parse narrative text and generate explanations but adhere inconsistently to diagnostic criteria.

Objective

This study aimed to develop and evaluate a parallel ML-LLM ensemble that combines the complementary strengths of both approaches for automated ASA-PS classification.

Methods

We retrospectively analyzed 3000 adult surgical encounters from the University of Arkansas for Medical Sciences (UAMS) between August 2024 and May 2025. Cases were randomly allocated to training (n=2000), validation (N=500), and held-out test sets (N=500). The validation set was used exclusively for architecture comparison and hyperparameter tuning, and the held-out test set was reserved for final model performance validation. We developed multiple architectures, including traditional ML models (Extreme Gradient Boosting [XGBoost], Light Gradient Boosting Machine [LightGBM], ExtraTrees), an LLM-only baseline (Generative Pretrained Transformer-4o [GPT-4o]), and hybrid approaches. The parallel ensemble processed structured data through XGBoost and unstructured clinical notes through GPT-4o independently, with outputs combined via weighted averaging. Model performance was evaluated using macro- F 1 -score, exact match accuracy, and ±1-class accuracy.

Results

A theoretical performance ceiling of macro- F 1 -score=0.59 was determined through evaluation by an expert panel of 2 board-certified anesthesiologists (AF and GF) who independently rated 50 patient charts for ASA-PS scores. On the true held-out test set (N=500), the parallel ensemble ( α =0.30) achieved a macro- F 1 -score of 0.530 (95% CI 0.350‐0.600), exact match accuracy of 0.630 (95% CI 0.580‐0.670), and ±1-class accuracy of 0.980 (95% CI 0.968‐0.992), exceeding the frozen LLM-only baseline evaluated on the same set (macro- F 1 -score 0.493, exact match accuracy 0.621,±1-class accuracy 0.981). On the validation set used for architecture selection, the ensemble outperformed structured-only ML (XGBoost macro- F 1 -score=0.333, P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningAnesthesiologistsLarge Language ModelsRetrospective StudiesHumansBoosting Machine Learning AlgorithmsSocieties, MedicalPredictive Learning Models

Stichwörter der Autor:innen

ASA physical status classificationASA-PSAmerican Society of Anesthesiologists Physical Status classificationlarge language modelsmachine learningperioperative careperioperative risk stratification

Autor:innen und Institutionen

  • Qventus Inc, Mountain View, CA, United States.
  • Department of Anesthesiology, University of Arkansas Medical Center, Little Rock, AR, United States.
  • Department Anesthesiology and Perioperative Medicine, Medical University of South Carolina, 135 Rutledge Ave, Charleston, SC, 29425, United States, +1 (843) 792-2322.

Zitieren

APA

Pabalan, C., Stewart, A., Fisher, G., & Fisher, A. (2026). American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study. Journal of Medical Internet Research, 28, e90724-e90724. https://doi.org/10.2196/90724

Vancouver

Pabalan C, Stewart A, Fisher G, Fisher A. American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study. Journal of Medical Internet Research. 2026;28:e90724-e90724. doi:10.2196/90724

BibTeX
@article{Pabalan2026,
  author = {Pabalan, Christabelle and Stewart, Alvin and Fisher, Gabrielle and Fisher, Andrew},
  title = {American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e90724--e90724},
  doi = {10.2196/90724}
}
Kostenlos starten →