Propensity‐based standardization to enhance the validation and interpretation of prediction model discrimination for a target population
Statistics in Medicine · 30.08.2023 · 42 (19) · 3508-3528
DOI: 10.1002/sim.9817 ↗Kurz gefasst
Fachartikel in der Zeitschrift Statistics in Medicine, online erschienen am 13.06.2023. Verfasst von Valentijn M T de Jong und 5 weiteren Autor:innen. Beteiligte Einrichtung: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands. Hauptthemen laut MEDLINE: Diagnosis-Related Groups, Benchmarking. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: European Commission.
Abstract (Originalsprache)
External validation of the discriminative ability of prediction models is of key importance. However, the interpretation of such evaluations is challenging, as the ability to discriminate depends on both the sample characteristics (ie, case‐mix) and the generalizability of predictor coefficients, but most discrimination indices do not provide any insight into their respective contributions. To disentangle differences in discriminative ability across external validation samples due to a lack of model generalizability from differences in sample characteristics, we propose propensity‐weighted measures of discrimination. These weighted metrics, which are derived from propensity scores for sample membership, are standardized for case‐mix differences between the model development and validation samples, allowing for a fair comparison of discriminative ability in terms of model characteristics in a target population of interest. We illustrate our methods with the validation of eight prediction models for deep vein thrombosis in 12 external validation data sets and assess our methods in a simulation study. In the illustrative example, propensity score standardization reduced between‐study heterogeneity of discrimination, indicating that between‐study variability was partially attributable to case‐mix. The simulation study showed that only flexible propensity‐score methods (allowing for non‐linear effects) produced unbiased estimates of model discrimination in the target population, and only when the positivity assumption was met. Propensity score‐based standardization may facilitate the interpretation of (heterogeneity in) discriminative ability of a prediction model as observed across multiple studies, and may guide model updating strategies for a particular target population. Careful propensity score modeling with attention for non‐linear relations is recommended.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
- Data Analytics and Methods Task Force, European Medicines Agency, Amsterdam, The Netherlands.
- Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Amsterdam, The Netherlands.
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
- Section of Epidemiology, Department of Public Health, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
- Smart Data Analysis and Statistics, Utrecht, The Netherlands.
ORCID-Profile: Valentijn M T de Jong, Jeroen Hoogland, Richard D Riley, Tri-Long Nguyen, Thomas P A Debray
Förderung
- European Commission
Zitieren
APA
de Jong, V. M. T., Hoogland, J., Moons, K. G. M., Riley, R. D., Nguyen, T. L., & Debray, T. P. A. (2023). Propensity‐based standardization to enhance the validation and interpretation of prediction model discrimination for a target population. Statistics in Medicine, 42(19), 3508-3528. https://doi.org/10.1002/sim.9817
Vancouver
de Jong VMT, Hoogland J, Moons KGM, Riley RD, Nguyen TL, Debray TPA. Propensity‐based standardization to enhance the validation and interpretation of prediction model discrimination for a target population. Statistics in Medicine. 2023;42(19):3508-3528. doi:10.1002/sim.9817
BibTeX
@article{deJong2023,
author = {de Jong, Valentijn M T and Hoogland, Jeroen and Moons, Karel G M and Riley, Richard D and Nguyen, Tri-Long and Debray, Thomas P A},
title = {Propensity‐based standardization to enhance the validation and interpretation of prediction model discrimination for a target population},
journal = {Statistics in Medicine},
year = {2023},
volume = {42},
number = {19},
pages = {3508--3528},
doi = {10.1002/sim.9817}
}
EUVIMED