EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Efficient estimation and correction of selection-induced bias with order statistics

Yann McLatchie, Aki Vehtari

Statistics and Computing · 2024-08 · 34 (4)

DOI: 10.1007/s11222-024-10442-4 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Statistics and Computing, online erschienen am 12.06.2024. Verfasst von Yann McLatchie und 1 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 88 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Aalto University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Model selection aims to identify a sufficiently well performing model that is possibly simpler than the most complex model among a pool of candidates. However, the decision-making process itself can inadvertently introduce non-negligible bias when the cross-validation estimates of predictive performance are marred by excessive noise. In finite data regimes, cross-validated estimates can encourage the statistician to select one model over another when it is not actually better for future data. While this bias remains negligible in the case of few models, when the pool of candidates grows, and model selection decisions are compounded (as in step-wise selection), the expected magnitude of selection-induced bias is likely to grow too. This paper introduces an efficient approach to estimate and correct selection-induced bias based on order statistics. Numerical experiments demonstrate the reliability of our approach in estimating both selection-induced bias and over-fitting along compounded model selection decisions, with specific application to forward search. This work represents a light-weight alternative to more computationally expensive approaches to correcting selection-induced bias, such as nested cross-validation and the bootstrap. Our approach rests on several theoretic assumptions, and we provide a diagnostic to help understand when these may not be valid and when to fall back on safer, albeit more computationally expensive approaches. The accompanying code facilitates its practical implementation and fosters further exploration in this area.

Förderung

  • Aalto University

Zitieren

APA

McLatchie, Y., & Vehtari, A. (2024). Efficient estimation and correction of selection-induced bias with order statistics. Statistics and Computing, 34(4). https://doi.org/10.1007/s11222-024-10442-4

Vancouver

McLatchie Y, Vehtari A. Efficient estimation and correction of selection-induced bias with order statistics. Statistics and Computing. 2024;34(4). doi:10.1007/s11222-024-10442-4

BibTeX
@article{McLatchie2024,
  author = {McLatchie, Yann and Vehtari, Aki},
  title = {Efficient estimation and correction of selection-induced bias with order statistics},
  journal = {Statistics and Computing},
  year = {2024},
  volume = {34},
  number = {4},
  doi = {10.1007/s11222-024-10442-4}
}
Kostenlos starten →