EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Testing for no effect in regression problems: A permutation approach

Michał G. Ciszewski, Jakob Söhl, Ton Leenen, Bart van Trigt, Geurt Jongbloed

Statistica Neerlandica · 2025-02 · 79 (1)

DOI: 10.1111/stan.12346 ↗

At a glance

Journal article in Statistica Neerlandica, published online on 2024-06-21. Written by Michał G. Ciszewski and 4 co-authors. Cited 3 times so far (Crossref); 20 references. The full text is freely available (open access, CC BY 4.0). Funded by: Nederlandse Organisatie voor Wetenschappelijk Onderzoek.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Often the question arises whether can be predicted based on using a certain model. Especially for highly flexible models such as neural networks one may ask whether a seemingly good prediction is actually better than fitting pure noise or whether it has to be attributed to the flexibility of the model. This paper proposes a rigorous permutation test to assess whether the prediction is better than the prediction of pure noise. The test avoids any sample splitting and is based instead on generating new pairings of . It introduces a new formulation of the null hypothesis and rigorous justification for the test, which distinguishes it from the previous literature. The theoretical findings are applied both to simulated data and to sensor data of tennis serves in an experimental context. The simulation study underscores how the available information affects the test. It shows that the less informative the predictors, the lower the probability of rejecting the null hypothesis of fitting pure noise and emphasizes that detecting weaker dependence between variables requires a sufficient sample size.

Funding

  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Cite

APA

Ciszewski, M. G., Söhl, J., Leenen, T., Trigt, B. V., & Jongbloed, G. (2025). Testing for no effect in regression problems: A permutation approach. Statistica Neerlandica, 79(1). https://doi.org/10.1111/stan.12346

Vancouver

Ciszewski MG, Söhl J, Leenen T, Trigt BV, Jongbloed G. Testing for no effect in regression problems: A permutation approach. Statistica Neerlandica. 2025;79(1). doi:10.1111/stan.12346

BibTeX
@article{Ciszewski2025,
  author = {Ciszewski, Michał G. and Söhl, Jakob and Leenen, Ton and Trigt, Bart van and Jongbloed, Geurt},
  title = {Testing for no effect in regression problems: A permutation approach},
  journal = {Statistica Neerlandica},
  year = {2025},
  volume = {79},
  number = {1},
  doi = {10.1111/stan.12346}
}
Start for free →