EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Optimising Retraining Frequency for a Paediatric Emergency Department Admission Prediction Model: Development and Temporal Validation Using Real‐World Data

Ethan Williams iD, Toshi Sinha iD, Mark Lyttle, Yogesan Kanagasingam

Emergency Medicine Australasia · 2026-06 · 38 (3)

DOI: 10.1111/1742-6723.70271 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Emergency Medicine Australasia, online erschienen am 06.05.2026. Verfasst von Ethan Williams und 3 weiteren Autor:innen. Beteiligte Einrichtung: Perth Children's Hospital Emergency Department, Nedlands, Australia. Hauptthemen laut MEDLINE: Emergency Service, Hospital, Triage, Patient Admission, Machine Learning. Die Arbeit nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Australian Government Research Training Program, Noton Dale Medical Research Scholarship.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To analyse temporal performance drift and optimal retraining frequency for an ensemble machine learning model to predict inpatient admission from paediatric emergency department (ED) triage data.

Methods

This study utilised 409,307 ED presentations from 1 July 2018 to 30 June 2024 at Perth Children's Hospital. An ensemble stacking model (XGBoost, TabNet, multi‐layer perceptron and logistic regression base learners with a logistic regression meta‐learner) incorporated structured triage features and tuned BioClinicalBERT‐derived embeddings from free‐text notes. The model ran prospectively through a 5‐year rolling‐window simulation, testing nine retraining cadences from weekly to triennial and a static model. Training, retraining and validation datasets were temporally separate and prior to the test set. Primary outcomes were discrimination via the area under the receiver operator characteristic (AUROC) and calibration as absolute mean daily bed error (AMDBE).

Results

Weekly retraining achieved a mean AUROC of 0.843 (SD 0.016) and AMDBE of 2.57 (SD 1.79) over the 5‐year simulation. Fortnightly and monthly cadences were non‐inferior (AMDBE 2.61 and 2.73), whereas longer intervals showed progressive calibration degradation ( p < 0.001) and stable AUROC. Concept drift was most pronounced in the static model, with a mean AMDBE of 10.6 in 2024 compared to 1.79 for the weekly model. Notably, monthly retraining required only 25% of the weekly computational burden with non‐inferior performance.

Conclusion

Monthly, or more frequent, model retraining sustains discrimination and calibration for paediatric ED admission prediction. This effectively mitigated concept drift and enabled accurate simulated daily bed‐demand forecasting, providing evidence to support the clinical testing of such modelling.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Emergency Service, HospitalTriagePatient AdmissionMachine LearningHumansPredictive Learning ModelsPrediction AlgorithmsChildROC CurvePediatricsLogistic ModelsEmergency Room Visits

Stichwörter der Autor:innen

admission predictionartificial intelligenceconcept driftmachine learningpaediatric emergency medicine

Autor:innen und Institutionen

  • Perth Children's Hospital Emergency Department, Nedlands, Australia.
  • The University of Notre Dame Australia, School of Medicine, Fremantle, Australia.

ORCID-Profile: Ethan Williams, Toshi Sinha

Förderung

  • Australian Government Research Training Program
  • Noton Dale Medical Research Scholarship

Zitieren

APA

Williams, E., Sinha, T., Lyttle, M., & Kanagasingam, Y. (2026). Optimising Retraining Frequency for a Paediatric Emergency Department Admission Prediction Model: Development and Temporal Validation Using Real‐World Data. Emergency Medicine Australasia, 38(3). https://doi.org/10.1111/1742-6723.70271

Vancouver

Williams E, Sinha T, Lyttle M, Kanagasingam Y. Optimising Retraining Frequency for a Paediatric Emergency Department Admission Prediction Model: Development and Temporal Validation Using Real‐World Data. Emergency Medicine Australasia. 2026;38(3). doi:10.1111/1742-6723.70271

BibTeX
@article{Williams2026,
  author = {Williams, Ethan and Sinha, Toshi and Lyttle, Mark and Kanagasingam, Yogesan},
  title = {Optimising Retraining Frequency for a Paediatric Emergency Department Admission Prediction Model: Development and Temporal Validation Using Real‐World Data},
  journal = {Emergency Medicine Australasia},
  year = {2026},
  volume = {38},
  number = {3},
  doi = {10.1111/1742-6723.70271}
}
Kostenlos starten →