Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space
Big Data and Cognitive Computing · 05.03.2026 · 10 (3) · 78
DOI: 10.3390/bdcc10030078 ↗Kurz gefasst
Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 05.03.2026. Verfasst von M. Hadi Sepanj und 3 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Sciences and Engineering Research Council of Canada (NSERC) through the Discovery Grants Program.
Abstract (Originalsprache)
Self-supervised learning (SSL) has emerged as a powerful paradigm for representation learning by optimizing geometric objectives, such as invariance to augmentations, variance preservation, and feature decorrelation, without requiring labels. However, most existing methods operate in Euclidean space, limiting their ability to capture nonlinear dependencies and geometric structures. In this work, we propose Kernel VICReg, a novel self-supervised learning framework that pulls the VICReg objective into a Reproducing Kernel Hilbert Space (RKHS). By kernelizing each term of the loss, variance, invariance, and covariance, we obtain a general formulation that operates on double-centered kernel matrices and Hilbert–Schmidt norms, enabling nonlinear feature learning without explicit mappings. We demonstrate that Kernel VICReg mitigates the risk of representational collapse under challenging conditions and improves performance on datasets exhibiting nonlinear structure or limited sample regimes. Empirical evaluations across MNIST, CIFAR-10, STL-10, TinyImageNet, and ImageNet100 show consistent gains over Euclidean VICReg, with particularly strong improvements on datasets where nonlinear structures are prominent. UMAP visualizations are provided only as a qualitative illustration of embedding geometry and are not used as a calibration or statistical validation. Our results suggest that kernelizing SSL objectives is a promising direction for bridging classical kernel methods with modern representation learning.
Förderung
- Natural Sciences and Engineering Research Council of Canada (NSERC) through the Discovery Grants Program
Zitieren
APA
Sepanj, M. H., Ghojogh, B., Moradi, S., & Fieguth, P. (2026). Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space. Big Data and Cognitive Computing, 10(3), 78. https://doi.org/10.3390/bdcc10030078
Vancouver
Sepanj MH, Ghojogh B, Moradi S, Fieguth P. Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space. Big Data and Cognitive Computing. 2026;10(3):78. doi:10.3390/bdcc10030078
BibTeX
@article{Sepanj2026,
author = {Sepanj, M. Hadi and Ghojogh, Benyamin and Moradi, Saed and Fieguth, Paul},
title = {Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space},
journal = {Big Data and Cognitive Computing},
year = {2026},
volume = {10},
number = {3},
pages = {78},
doi = {10.3390/bdcc10030078}
}
EUVIMED