Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage
JAMIA Open · 04.10.2023 · 6 (4)
DOI: 10.1093/jamiaopen/ooad107 ↗Kurz gefasst
Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 20.12.2023. Verfasst von Stephanie Teeple und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA 19143, United Sta…. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 78 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Library of Medicine, National Institutes of Health, Agency for Healthcare Research and Quality.
Abstract (Originalsprache)
Objective
To investigate how missing data in the patient problem list may impact racial disparities in the predictive performance of a machine learning (ML) model for emergency department (ED) triage. Materials and Methods Racial disparities may exist in the missingness of EHR data (eg, systematic differences in access, testing, and/or treatment) that can impact model predictions across racialized patient groups. We use an ML model that predicts patients’ risk for adverse events to produce triage-level recommendations, patterned after a clinical decision support tool deployed at multiple EDs. We compared the model’s predictive performance on sets of observed (problem list data at the point of triage) versus manipulated (updated to the more complete problem list at the end of the encounter) test data. These differences were compared between Black and non-Hispanic White patient groups using multiple performance measures relevant to health equity.
Results
There were modest, but significant, changes in predictive performance comparing the observed to manipulated models across both Black and non-Hispanic White patient groups; c-statistic improvement ranged between 0.027 and 0.058. The manipulation produced no between-group differences in c-statistic by race. However, there were small between-group differences in other performance measures, with greater change for non-Hispanic White patients.
Discussion
Problem list missingness impacted model performance for both patient groups, with marginal differences detected by race.
Conclusion
Further exploration is needed to examine how missingness may contribute to racial disparities in clinical model predictions across settings. The novel manipulation method demonstrated may aid future research.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA 19143, United States.
- Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA 19104, United States.
- Department of Emergency Medicine, Johns Hopkins University, Baltimore, MD 21218, United States.
- Clinical Decision Support Solutions, Beckman Coulter, Brea, CA 92821, United States.
- Division of Pulmonary, Allergy and Critical Care, Department of Medicine at the Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States.
- Department of Pediatric Emergency Medicine, Johns Hopkins University, Baltimore, MD 21218, United States.
Förderung
- National Library of Medicine
- National Institutes of Health
- Agency for Healthcare Research and Quality
- Department of Health and Human Services
Zitieren
APA
Teeple, S., Smith, A., Toerper, M., Levin, S., Halpern, S., Badaki-Makun, O., & Hinson, J. (2023). Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage. JAMIA Open, 6(4). https://doi.org/10.1093/jamiaopen/ooad107
Vancouver
Teeple S, Smith A, Toerper M, Levin S, Halpern S, Badaki-Makun O, et al. Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage. JAMIA Open. 2023;6(4). doi:10.1093/jamiaopen/ooad107
BibTeX
@article{Teeple2023,
author = {Teeple, Stephanie and Smith, Aria and Toerper, Matthew and Levin, Scott and Halpern, Scott and Badaki-Makun, Oluwakemi and Hinson, Jeremiah},
title = {Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage},
journal = {JAMIA Open},
year = {2023},
volume = {6},
number = {4},
doi = {10.1093/jamiaopen/ooad107}
}
EUVIMED