Bootstrapping the cross-validation estimate
The Annals of Applied Statistics · 01.12.2025 · 19 (4) · 2981-3002
DOI: 10.1214/25-aoas2036 ↗Kurz gefasst
Fachartikel in der Zeitschrift The Annals of Applied Statistics, erschienen am 01.12.2025. Verfasst von Bryan Cai und 8 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 23 Literaturangaben.
Abstract (Originalsprache)
Cross-validation is a widely used technique for evaluating the performance of prediction models, ranging from simple binary classification to complex precision medicine strategies. It helps correct for optimism bias in error estimates, which can be significant for models built using complex statistical learning algorithms. However, since the cross-validation estimate is a random value dependent on observed data, it is essential to accurately quantify the uncertainty associated with the estimate. This is especially important when comparing the performance of two models using cross-validation, as one must determine whether differences in estimated error are due to chance. Although various methods have been developed to make inferences on cross-validation estimates, they often have many limitations, such as requiring stringent model assumptions. This paper proposes a fast bootstrap method that quickly estimates the standard error of the cross-validation estimate and produces valid confidence intervals for a population parameter measuring average model performance. Our method overcomes the computational challenges inherent in bootstrapping a cross-validation estimate by estimating the variance component within a random-effects model. It is also as flexible as the cross-validation procedure itself. To showcase the effectiveness of our approach, we conducted comprehensive simulations and real-data analysis across two applications.
Zitieren
APA
Cai, B., Luo, Y., Guo, X., Pellegrini, F., Pang, M., Moor, C. D., Shen, C., Charu, V., & Tian, L. (2025). Bootstrapping the cross-validation estimate. The Annals of Applied Statistics, 19(4), 2981-3002. https://doi.org/10.1214/25-aoas2036
Vancouver
Cai B, Luo Y, Guo X, Pellegrini F, Pang M, Moor CD, et al. Bootstrapping the cross-validation estimate. The Annals of Applied Statistics. 2025;19(4):2981-3002. doi:10.1214/25-aoas2036
BibTeX
@article{Cai2025,
author = {Cai, Bryan and Luo, Yuanhui and Guo, Xinzhou and Pellegrini, Fabio and Pang, Menglan and Moor, Carl de and Shen, Changyu and Charu, Vivek and Tian, Lu},
title = {Bootstrapping the cross-validation estimate},
journal = {The Annals of Applied Statistics},
year = {2025},
volume = {19},
number = {4},
pages = {2981--3002},
doi = {10.1214/25-aoas2036}
}
EUVIMED