EUVIMEDEUVIMED entdecken
Open Access

Multisite evaluation of prediction models for emergency department crowding before and during the COVID-19 pandemic

Ari J Smith iD, Brian W Patterson iD, Michael S Pulia, John Mayer, Rebecca J Schwei, Radha Nagarajan, Frank Liao iD, Manish N Shah, Justin J Boutilier iD

Journal of the American Medical Informatics Association · 2023

DOI: 10.1093/jamia/ocac214 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2023. Verfasst von Ari J Smith und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin, USA. Hauptthemen laut MEDLINE: COVID-19, Emergency Service, Hospital, Crowding. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: UW School of Medicine and Public Health, the Wisconsin Partnership Program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To develop a machine learning framework to forecast emergency department (ED) crowding and to evaluate model performance under spatial and temporal data drift. Materials and Methods We obtained 4 datasets, identified by the location: 1—large academic hospital and 2—rural hospital, and time period: pre-coronavirus disease (COVID) (January 1, 2019–February 1, 2020) and COVID-era (May 15, 2020–February 1, 2021). Our primary target was a binary outcome that is equal to 1 if the number of patients with acute respiratory illness that were ED boarding for more than 4 h was above a prescribed historical percentile. We trained a random forest and used the area under the curve (AUC) to evaluate out-of-sample performance for 2 experiments: (1) we evaluated the impact of sudden temporal drift by training models using pre-COVID data and testing them during the COVID-era, (2) we evaluated the impact of spatial drift by testing models trained at location 1 on data from location 2, and vice versa.

Results

The baseline AUC values for ED boarding ranged from 0.54 (pre-COVID at location 2) to 0.81 (COVID-era at location 1). Models trained with pre-COVID data performed similarly to COVID-era models (0.82 vs 0.78 at location 1). Models that were transferred from location 2 to location 1 performed worse than models trained at location 1 (0.51 vs 0.78). Discussion and Conclusion Our results demonstrate that ED boarding is a predictable metric for ED crowding, models were not significantly impacted by temporal data drift, and any attempts at implementation must consider spatial data drift.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

COVID-19Emergency Service, HospitalCrowdingHumansForecastingPandemicsRetrospective Studies

Stichwörter der Autor:innen

COVID-19data driftemergency department boardingemergency medicinemachine learning

Autor:innen und Institutionen

  • Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin, USA.
  • BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, Wisconsin, USA.
  • Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, Wisconsin, USA.
  • Marshfield Clinic Research Institute, Marshfield, Wisconsin, USA.
  • Applied Data Science, Information Services, UW-Health, Madison, Wisconsin, USA.

ORCID-Profile: Ari J Smith, Brian W Patterson, Frank Liao, Justin J Boutilier

Förderung

  • UW School of Medicine and Public Health
  • the Wisconsin Partnership Program

Zitieren

APA

Smith, A. J., Patterson, B. W., Pulia, M. S., Mayer, J., Schwei, R. J., Nagarajan, R., Liao, F., Shah, M. N., & Boutilier, J. J. (2023). Multisite evaluation of prediction models for emergency department crowding before and during the COVID-19 pandemic. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocac214

Vancouver

Smith AJ, Patterson BW, Pulia MS, Mayer J, Schwei RJ, Nagarajan R, et al. Multisite evaluation of prediction models for emergency department crowding before and during the COVID-19 pandemic. Journal of the American Medical Informatics Association. 2023. doi:10.1093/jamia/ocac214

BibTeX
@article{Smith2023,
  author = {Smith, Ari J and Patterson, Brian W and Pulia, Michael S and Mayer, John and Schwei, Rebecca J and Nagarajan, Radha and Liao, Frank and Shah, Manish N and Boutilier, Justin J},
  title = {Multisite evaluation of prediction models for emergency department crowding before and during the COVID-19 pandemic},
  journal = {Journal of the American Medical Informatics Association},
  year = {2023},
  doi = {10.1093/jamia/ocac214}
}
Kostenlos starten →