Optimising Retraining Frequency for a Paediatric Emergency Department Admission Prediction Model: Development and Temporal Validation Using Real‐World Data
Emergency Medicine Australasia · 2026-06 · 38 (3)
DOI: 10.1111/1742-6723.70271 ↗Kurz gefasst
Fachartikel in der Zeitschrift Emergency Medicine Australasia, online erschienen am 06.05.2026. Verfasst von Ethan Williams und 3 weiteren Autor:innen. Beteiligte Einrichtung: Perth Children's Hospital Emergency Department, Nedlands, Australia. Hauptthemen laut MEDLINE: Emergency Service, Hospital, Triage, Patient Admission, Machine Learning. Die Arbeit nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Australian Government Research Training Program, Noton Dale Medical Research Scholarship.
Abstract (Originalsprache)
Objective
To analyse temporal performance drift and optimal retraining frequency for an ensemble machine learning model to predict inpatient admission from paediatric emergency department (ED) triage data.
Methods
This study utilised 409,307 ED presentations from 1 July 2018 to 30 June 2024 at Perth Children's Hospital. An ensemble stacking model (XGBoost, TabNet, multi‐layer perceptron and logistic regression base learners with a logistic regression meta‐learner) incorporated structured triage features and tuned BioClinicalBERT‐derived embeddings from free‐text notes. The model ran prospectively through a 5‐year rolling‐window simulation, testing nine retraining cadences from weekly to triennial and a static model. Training, retraining and validation datasets were temporally separate and prior to the test set. Primary outcomes were discrimination via the area under the receiver operator characteristic (AUROC) and calibration as absolute mean daily bed error (AMDBE).
Results
Weekly retraining achieved a mean AUROC of 0.843 (SD 0.016) and AMDBE of 2.57 (SD 1.79) over the 5‐year simulation. Fortnightly and monthly cadences were non‐inferior (AMDBE 2.61 and 2.73), whereas longer intervals showed progressive calibration degradation ( p < 0.001) and stable AUROC. Concept drift was most pronounced in the static model, with a mean AMDBE of 10.6 in 2024 compared to 1.79 for the weekly model. Notably, monthly retraining required only 25% of the weekly computational burden with non‐inferior performance.
Conclusion
Monthly, or more frequent, model retraining sustains discrimination and calibration for paediatric ED admission prediction. This effectively mitigated concept drift and enabled accurate simulated daily bed‐demand forecasting, providing evidence to support the clinical testing of such modelling.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Perth Children's Hospital Emergency Department, Nedlands, Australia.
- The University of Notre Dame Australia, School of Medicine, Fremantle, Australia.
ORCID-Profile: Ethan Williams, Toshi Sinha
Förderung
- Australian Government Research Training Program
- Noton Dale Medical Research Scholarship
Zitieren
APA
Williams, E., Sinha, T., Lyttle, M., & Kanagasingam, Y. (2026). Optimising Retraining Frequency for a Paediatric Emergency Department Admission Prediction Model: Development and Temporal Validation Using Real‐World Data. Emergency Medicine Australasia, 38(3). https://doi.org/10.1111/1742-6723.70271
Vancouver
Williams E, Sinha T, Lyttle M, Kanagasingam Y. Optimising Retraining Frequency for a Paediatric Emergency Department Admission Prediction Model: Development and Temporal Validation Using Real‐World Data. Emergency Medicine Australasia. 2026;38(3). doi:10.1111/1742-6723.70271
BibTeX
@article{Williams2026,
author = {Williams, Ethan and Sinha, Toshi and Lyttle, Mark and Kanagasingam, Yogesan},
title = {Optimising Retraining Frequency for a Paediatric Emergency Department Admission Prediction Model: Development and Temporal Validation Using Real‐World Data},
journal = {Emergency Medicine Australasia},
year = {2026},
volume = {38},
number = {3},
doi = {10.1111/1742-6723.70271}
}
EUVIMED