EUVIMEDEUVIMED entdecken
Open Access

Distributional bias compromises leave-one-out cross-validation

George I Austin iD, Itsik Pe'er iD, Tal Korem iD

Science Advances · 28.11.2025 · 11 (48)

DOI: 10.1126/sciadv.adx6976 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Science Advances, erschienen am 28.11.2025. Verfasst von George I Austin und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY, USA. Die Arbeit wurde bisher 30-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NICHD NIH HHS, NLM NIH HHS, NIGMS NIH HHS.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Cross-validation is a common method for evaluating machine learning models. “Leave-one-out cross-validation,” in which each data instance is used to test a model trained on all other instances, is often used in data-scarce regimes. As common metrics such as the R 2 score cannot be calculated for a single prediction, predictions are commonly aggregated across folds for performance evaluation. Here, we prove that this creates “distributional bias”: a negative correlation between the average label of each training fold and the label of its corresponding test instance. As machine learning models tend to regress to the mean of their training data, this bias tends to negatively affect performance evaluation and hyperparameter optimization. We demonstrate that distributional bias exists across diverse tasks, models, and evaluation approaches, and can bias against stronger regularization. To address it, we developed a generalizable rebalanced cross-validation that is robust to distributional bias in both classification and regression, and demonstrates improved performance in simulations, machine learning benchmarks, and several published analyses.

Autor:innen und Institutionen

  • Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY, USA.
  • Program for Mathematical Genomics, Department of Systems Biology, Columbia University Irving Medical Center, New York, NY, USA.
  • Department of Computer Science, Columbia University, New York, NY, USA.
  • Department of Obstetrics and Gynecology, Columbia University Irving Medical Center, New York, NY, USA.

ORCID-Profile: George I Austin, Itsik Pe'er, Tal Korem

Förderung

  • NICHD NIH HHS
  • NLM NIH HHS
  • NIGMS NIH HHS
  • National Institutes of Health
  • Program for Mathematical Genomics at Columbia University

Zitieren

APA

Austin, G. I., Pe'er, I., & Korem, T. (2025). Distributional bias compromises leave-one-out cross-validation. Science Advances, 11(48). https://doi.org/10.1126/sciadv.adx6976

Vancouver

Austin GI, Pe'er I, Korem T. Distributional bias compromises leave-one-out cross-validation. Science Advances. 2025;11(48). doi:10.1126/sciadv.adx6976

BibTeX
@article{Austin2025,
  author = {Austin, George I and Pe'er, Itsik and Korem, Tal},
  title = {Distributional bias compromises leave-one-out cross-validation},
  journal = {Science Advances},
  year = {2025},
  volume = {11},
  number = {48},
  doi = {10.1126/sciadv.adx6976}
}
Kostenlos starten →