Testing for no effect in regression problems: A permutation approach
Statistica Neerlandica · 2025-02 · 79 (1)
DOI: 10.1111/stan.12346 ↗At a glance
Journal article in Statistica Neerlandica, published online on 2024-06-21. Written by Michał G. Ciszewski and 4 co-authors. Cited 3 times so far (Crossref); 20 references. The full text is freely available (open access, CC BY 4.0). Funded by: Nederlandse Organisatie voor Wetenschappelijk Onderzoek.
Abstract (original language)
Often the question arises whether can be predicted based on using a certain model. Especially for highly flexible models such as neural networks one may ask whether a seemingly good prediction is actually better than fitting pure noise or whether it has to be attributed to the flexibility of the model. This paper proposes a rigorous permutation test to assess whether the prediction is better than the prediction of pure noise. The test avoids any sample splitting and is based instead on generating new pairings of . It introduces a new formulation of the null hypothesis and rigorous justification for the test, which distinguishes it from the previous literature. The theoretical findings are applied both to simulated data and to sensor data of tennis serves in an experimental context. The simulation study underscores how the available information affects the test. It shows that the less informative the predictors, the lower the probability of rejecting the null hypothesis of fitting pure noise and emphasizes that detecting weaker dependence between variables requires a sufficient sample size.
Funding
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
Cite
APA
Ciszewski, M. G., Söhl, J., Leenen, T., Trigt, B. V., & Jongbloed, G. (2025). Testing for no effect in regression problems: A permutation approach. Statistica Neerlandica, 79(1). https://doi.org/10.1111/stan.12346
Vancouver
Ciszewski MG, Söhl J, Leenen T, Trigt BV, Jongbloed G. Testing for no effect in regression problems: A permutation approach. Statistica Neerlandica. 2025;79(1). doi:10.1111/stan.12346
BibTeX
@article{Ciszewski2025,
author = {Ciszewski, Michał G. and Söhl, Jakob and Leenen, Ton and Trigt, Bart van and Jongbloed, Geurt},
title = {Testing for no effect in regression problems: A permutation approach},
journal = {Statistica Neerlandica},
year = {2025},
volume = {79},
number = {1},
doi = {10.1111/stan.12346}
}
EUVIMED