EUVIMEDEUVIMED entdecken
Crossref

High-dimensional ridge regression with random features for non-identically distributed data with a variance profile

Issa-Mbenard Dabo, Jérémie Bigot

Information and Inference: A Journal of the IMA · 2026

DOI: 10.1093/imaiai/iaag033 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information and Inference: A Journal of the IMA, 2026. Verfasst von Issa-Mbenard Dabo und 1 weiteren Autor:innen. Die Arbeit nennt 33 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Random feature ridge regression is often analyzed in the high-dimensional regime under the homogeneous sampling model $x_{i}=\varSigma ^{1/2}x_{i}^{\prime}$, where the vectors $x_{i}^{\prime}$ have iid entries and the same covariance matrix $\varSigma$ is shared by all samples. In this paper, we move beyond this setting and study non-identically distributed data through a variance-profile model in which the training and test covariates have row-dependent diagonal covariance matrices $\varSigma _{i}=\text{diag}(\gamma _{i1}^{2},\ldots ,\gamma _{ip}^{2})$ and $\widetilde{\varSigma }_{i}=\text{diag}(\tilde \gamma _{i1}^{2},\ldots ,\tilde \gamma _{ip}^{2})$. Our main contribution is the derivation of asymptotic equivalents for the training and test risks of ridge regression with random features when $n$, $p$ and $m$ grow proportionally. The first set of equivalents is obtained by combining the linear-plus-chaos approximation with traffic-probability arguments, whereas the second set is deterministic and follows from operator-valued free probability through an amalgamation-over-the-diagonal argument. These equivalents are sharp in numerical experiments. They also reveal how heterogeneous variance profiles, including mixture-type profiles inspired by MNIST, can modify generalization and exhibit double-descent behavior when the ridge parameter is small.

Zitieren

APA

Dabo, I., & Bigot, J. (2026). High-dimensional ridge regression with random features for non-identically distributed data with a variance profile. Information and Inference: A Journal of the IMA. https://doi.org/10.1093/imaiai/iaag033

Vancouver

Dabo I, Bigot J. High-dimensional ridge regression with random features for non-identically distributed data with a variance profile. Information and Inference: A Journal of the IMA. 2026. doi:10.1093/imaiai/iaag033

BibTeX
@article{Dabo2026,
  author = {Dabo, Issa-Mbenard and Bigot, Jérémie},
  title = {High-dimensional ridge regression with random features for non-identically distributed data with a variance profile},
  journal = {Information and Inference: A Journal of the IMA},
  year = {2026},
  doi = {10.1093/imaiai/iaag033}
}
Kostenlos starten →