EUVIMEDEUVIMED entdecken
Open Access

Temporal Generalizability of Machine Learning Models for Predicting Postoperative Delirium Using Electronic Health Record Data: Model Development and Validation Study

Koutarou Matsumoto iD, Yasunobu Nohara iD, Mikako Sakaguchi iD, Yohei Takayama iD, Syota Fukushige iD, Hidehisa Soejima iD, Naoki Nakashima iD, Masahiro Kamouchi iD

JMIR Perioperative Medicine · 26.10.2023 · 6 · e50895

DOI: 10.2196/50895 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Perioperative Medicine, online erschienen am 26.10.2023. Verfasst von Koutarou Matsumoto und 7 weiteren Autor:innen. Beteiligte Einrichtung: Biostatistics Center, Kurume University, Kurume, Japan. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Although machine learning models demonstrate significant potential in predicting postoperative delirium, the advantages of their implementation in real-world settings remain unclear and require a comparison with conventional models in practical applications.

Objective

The objective of this study was to validate the temporal generalizability of decision tree ensemble and sparse linear regression models for predicting delirium after surgery compared with that of the traditional logistic regression model.

Methods

The health record data of patients hospitalized at an advanced emergency and critical care medical center in Kumamoto, Japan, were collected electronically. We developed a decision tree ensemble model using extreme gradient boosting (XGBoost) and a sparse linear regression model using least absolute shrinkage and selection operator (LASSO) regression. To evaluate the predictive performance of the model, we used the area under the receiver operating characteristic curve (AUROC) and the Matthews correlation coefficient (MCC) to measure discrimination and the slope and intercept of the regression between predicted and observed probabilities to measure calibration. The Brier score was evaluated as an overall performance metric. We included 11,863 consecutive patients who underwent surgery with general anesthesia between December 2017 and February 2022. The patients were divided into a derivation cohort before the COVID-19 pandemic and a validation cohort during the COVID-19 pandemic. Postoperative delirium was diagnosed according to the confusion assessment method.

Results

A total of 6497 patients (68.5, SD 14.4 years, women n=2627, 40.4%) were included in the derivation cohort, and 5366 patients (67.8, SD 14.6 years, women n=2105, 39.2%) were included in the validation cohort. Regarding discrimination, the XGBoost model (AUROC 0.87-0.90 and MCC 0.34-0.44) did not significantly outperform the LASSO model (AUROC 0.86-0.89 and MCC 0.34-0.41). The logistic regression model (AUROC 0.84-0.88, MCC 0.33-0.40, slope 1.01-1.19, intercept –0.16 to 0.06, and Brier score 0.06-0.07), with 8 predictors (age, intensive care unit, neurosurgery, emergency admission, anesthesia time, BMI, blood loss during surgery, and use of an ambulance) achieved good predictive performance.

Conclusions

The XGBoost model did not significantly outperform the LASSO model in predicting postoperative delirium. Furthermore, a parsimonious logistic model with a few important predictors achieved comparable performance to machine learning models in predicting postoperative delirium.

Schlagwörter

Stichwörter der Autor:innen

electronic health record datamachine learningpostoperative deliriumprediction modeltemporal generalizability

Autor:innen und Institutionen

  • Biostatistics Center, Kurume University, Kurume, Japan.
  • Big Data Science and Technology, Faculty of Advanced Science and Technology, Kumamoto University, Kumamoto, Japan.
  • Department of Nursing, Saiseikai Kumamoto Hospital, Kumamoto, Japan.
  • Department of Inspection, Saiseikai Kumamoto Hospital, Kumamoto, Japan.
  • Institute for Medical Information Research and Analysis, Saiseikai Kumamoto Hospital, Kumamoto, Japan.
  • Medical Information Center, Kyushu University Hospital, Fukuoka, Japan.

ORCID-Profile: Koutarou Matsumoto, Yasunobu Nohara, Mikako Sakaguchi, Yohei Takayama, Syota Fukushige, Hidehisa Soejima, Naoki Nakashima, Masahiro Kamouchi

Zitieren

APA

Matsumoto, K., Nohara, Y., Sakaguchi, M., Takayama, Y., Fukushige, S., Soejima, H., Nakashima, N., & Kamouchi, M. (2023). Temporal Generalizability of Machine Learning Models for Predicting Postoperative Delirium Using Electronic Health Record Data: Model Development and Validation Study. JMIR Perioperative Medicine, 6, e50895. https://doi.org/10.2196/50895

Vancouver

Matsumoto K, Nohara Y, Sakaguchi M, Takayama Y, Fukushige S, Soejima H, et al. Temporal Generalizability of Machine Learning Models for Predicting Postoperative Delirium Using Electronic Health Record Data: Model Development and Validation Study. JMIR Perioperative Medicine. 2023;6:e50895. doi:10.2196/50895

BibTeX
@article{Matsumoto2023,
  author = {Matsumoto, Koutarou and Nohara, Yasunobu and Sakaguchi, Mikako and Takayama, Yohei and Fukushige, Syota and Soejima, Hidehisa and Nakashima, Naoki and Kamouchi, Masahiro},
  title = {Temporal Generalizability of Machine Learning Models for Predicting Postoperative Delirium Using Electronic Health Record Data: Model Development and Validation Study},
  journal = {JMIR Perioperative Medicine},
  year = {2023},
  volume = {6},
  pages = {e50895},
  doi = {10.2196/50895}
}
Kostenlos starten →