Mitigating Feature Overfitting in Barlow Twins via Mixed-Sample Regularization for Stable Long-Horizon Representation Learning
Computational Discovery and Intelligent Systems (CDIS) · 25.04.2026 · 3 (1) · 71-90
DOI: 10.66279/7scztv14 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computational Discovery and Intelligent Systems (CDIS), online erschienen am 25.04.2026. Verfasst von Arwa Saad und 3 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In self-supervised learning, feature overfitting during extended training is still a significant problem, especially in redundancy-reduction frameworks like Barlow Twins. Barlow Twins performs well at first, but after prolonged training (e.g., after 600 epochs), its representation quality deteriorates. This is primarily because of limited data diversity and overfitting to feature correlations. An improved Mixed Barlow Twins framework that incorporates mixed-sample regularization via linear interpolation in the input space is presented to overcome this restriction. This method facilitates simpler feature matrices and mitigates redundancy-induced overfitting by ensuring consistency between mixed inputs and their corresponding embeddings. Stable optimization without performance degradation is demonstrated by extensive experiments on CIFAR-10 with a ResNet-50 backbone over 1000 training epochs. In longer-term scenarios, the proposed approach outperforms the standard Barlow Twins baseline, achieving a k-NN classification accuracy of 92.1%. With only 7.2 GB of GPU memory and about 15 hours of training time, the technique also maintains high computational efficiency with little overhead. These findings indicate that mixed-sample regularization is a simple yet effective method for improving representation robustness and training stability in self-supervised learning.
Zitieren
APA
Saad, A., Chakrabarti, P., Abdelrahman, M. A., & Ravi, V. (2026). Mitigating Feature Overfitting in Barlow Twins via Mixed-Sample Regularization for Stable Long-Horizon Representation Learning. Computational Discovery and Intelligent Systems (CDIS), 3(1), 71-90. https://doi.org/10.66279/7scztv14
Vancouver
Saad A, Chakrabarti P, Abdelrahman MA, Ravi V. Mitigating Feature Overfitting in Barlow Twins via Mixed-Sample Regularization for Stable Long-Horizon Representation Learning. Computational Discovery and Intelligent Systems (CDIS). 2026;3(1):71-90. doi:10.66279/7scztv14
BibTeX
@article{Saad2026,
author = {Saad, Arwa and Chakrabarti, Prasun and Abdelrahman, Mona Ali and Ravi, Vinayakumar},
title = {Mitigating Feature Overfitting in Barlow Twins via Mixed-Sample Regularization for Stable Long-Horizon Representation Learning},
journal = {Computational Discovery and Intelligent Systems (CDIS)},
year = {2026},
volume = {3},
number = {1},
pages = {71--90},
doi = {10.66279/7scztv14}
}
EUVIMED