Distributional bias compromises leave-one-out cross-validation
Science Advances · 28.11.2025 · 11 (48)
DOI: 10.1126/sciadv.adx6976 ↗Kurz gefasst
Fachartikel in der Zeitschrift Science Advances, erschienen am 28.11.2025. Verfasst von George I Austin und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY, USA. Die Arbeit wurde bisher 30-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NICHD NIH HHS, NLM NIH HHS, NIGMS NIH HHS.
Abstract (Originalsprache)
Cross-validation is a common method for evaluating machine learning models. “Leave-one-out cross-validation,” in which each data instance is used to test a model trained on all other instances, is often used in data-scarce regimes. As common metrics such as the R 2 score cannot be calculated for a single prediction, predictions are commonly aggregated across folds for performance evaluation. Here, we prove that this creates “distributional bias”: a negative correlation between the average label of each training fold and the label of its corresponding test instance. As machine learning models tend to regress to the mean of their training data, this bias tends to negatively affect performance evaluation and hyperparameter optimization. We demonstrate that distributional bias exists across diverse tasks, models, and evaluation approaches, and can bias against stronger regularization. To address it, we developed a generalizable rebalanced cross-validation that is robust to distributional bias in both classification and regression, and demonstrates improved performance in simulations, machine learning benchmarks, and several published analyses.
Autor:innen und Institutionen
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY, USA.
- Program for Mathematical Genomics, Department of Systems Biology, Columbia University Irving Medical Center, New York, NY, USA.
- Department of Computer Science, Columbia University, New York, NY, USA.
- Department of Obstetrics and Gynecology, Columbia University Irving Medical Center, New York, NY, USA.
ORCID-Profile: George I Austin, Itsik Pe'er, Tal Korem
Förderung
- NICHD NIH HHS
- NLM NIH HHS
- NIGMS NIH HHS
- National Institutes of Health
- Program for Mathematical Genomics at Columbia University
Zitieren
APA
Austin, G. I., Pe'er, I., & Korem, T. (2025). Distributional bias compromises leave-one-out cross-validation. Science Advances, 11(48). https://doi.org/10.1126/sciadv.adx6976
Vancouver
Austin GI, Pe'er I, Korem T. Distributional bias compromises leave-one-out cross-validation. Science Advances. 2025;11(48). doi:10.1126/sciadv.adx6976
BibTeX
@article{Austin2025,
author = {Austin, George I and Pe'er, Itsik and Korem, Tal},
title = {Distributional bias compromises leave-one-out cross-validation},
journal = {Science Advances},
year = {2025},
volume = {11},
number = {48},
doi = {10.1126/sciadv.adx6976}
}
EUVIMED