Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study
Statistical Methods in Medical Research · 2023-08 · 32 (8) · 1461-1477
DOI: 10.1177/09622802231165001 ↗Kurz gefasst
Fachartikel in der Zeitschrift Statistical Methods in Medical Research, online erschienen am 27.04.2023. Verfasst von Rose Sisk und 4 weiteren Autor:innen. Beteiligte Einrichtung: Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of …. Hauptthemen laut MEDLINE: Research Design, Critical Care. Die Arbeit wurde bisher 106-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Medical Research Council, Alan Turing Institute, Gendius Ltd.
Abstract (Originalsprache)
Background
In clinical prediction modelling, missing data can occur at any stage of the model pipeline; development, validation or deployment. Multiple imputation is often recommended yet challenging to apply at deployment; for example, the outcome cannot be in the imputation model, as recommended under multiple imputation. Regression imputation uses a fitted model to impute the predicted value of missing predictors from observed data, and could offer a pragmatic alternative at deployment. Moreover, the use of missing indicators has been proposed to handle informative missingness, but it is currently unknown how well this method performs in the context of clinical prediction models.
Methods
We simulated data under various missing data mechanisms to compare the predictive performance of clinical prediction models developed using both imputation methods. We consider deployment scenarios where missing data is permitted or prohibited, imputation models that use or omit the outcome, and clinical prediction models that include or omit missing indicators. We assume that the missingness mechanism remains constant across the model pipeline. We also apply the proposed strategies to critical care data.
Results
With complete data available at deployment, our findings were in line with existing recommendations; that the outcome should be used to impute development data when using multiple imputation and omitted under regression imputation. When missingness is allowed at deployment, omitting the outcome from the imputation model at the development was preferred. Missing indicators improved model performance in many cases but can be harmful under outcome-dependent missingness.
Conclusion
We provide evidence that commonly taught principles of handling missing data via multiple imputation may not apply to clinical prediction models, particularly when data can be missing at deployment. We observed comparable predictive performance under multiple imputation and regression imputation. The performance of the missing data handling method must be evaluated on a study-by-study basis, and the most appropriate strategy for handling missing data at development should consider whether missing data are allowed at deployment. Some guidance is provided.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK.
- Gendius Ltd, Macclesfield, UK.
- Alan Turing Institute, London, UK.
- NIHR Manchester Biomedical Research Centre, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK.
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands.
ORCID-Profile: Rose Sisk, Matthew Sperrin, Glen Philip Martin
Förderung
- Medical Research Council
- Alan Turing Institute
- Gendius Ltd
Zitieren
APA
Sisk, R., Sperrin, M., Peek, N., van Smeden, M., & Martin, G. P. (2023). Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study. Statistical Methods in Medical Research, 32(8), 1461-1477. https://doi.org/10.1177/09622802231165001
Vancouver
Sisk R, Sperrin M, Peek N, van Smeden M, Martin GP. Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study. Statistical Methods in Medical Research. 2023;32(8):1461-1477. doi:10.1177/09622802231165001
BibTeX
@article{Sisk2023,
author = {Sisk, Rose and Sperrin, Matthew and Peek, Niels and van Smeden, Maarten and Martin, Glen Philip},
title = {Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study},
journal = {Statistical Methods in Medical Research},
year = {2023},
volume = {32},
number = {8},
pages = {1461--1477},
doi = {10.1177/09622802231165001}
}
EUVIMED