EUVIMEDEUVIMED entdecken
Crossref

Input–Label Correlation Governs a Linear-to-Nonlinear Transition in Random Features under Spiked Covariance

Samet Demir, Zafer Doğan

SIAM Journal on Mathematics of Data Science · 31.12.2026 · 8 (4) · 1322-1349

DOI: 10.1137/26m1853313 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Mathematics of Data Science, online erschienen am 06.11.2026. Verfasst von Samet Demir und 1 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Gefördert von: TUBITAK-ARDEB 1001, Koc University & Is Bank Artificial Intelligence, TUBITAK.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Random feature models (RFMs)—two-layer networks with a randomly initialized fixed first layer and a trained linear readout—are among the simplest nonlinear predictors. Prior asymptotic analyses in the proportional high-dimensional regime show that, under isotropic data, RFMs reduce to noisy linear models and offer no advantage over classical linear methods such as ridge regression. Yet RFMs frequently outperform linear baselines on structured real data. We show that this tension is explained by a correlation-driven phase transition: under spiked-covariance designs, the interaction between anisotropy and input–label correlation determines whether the RFM behaves as an effectively linear predictor or exhibits genuinely nonlinear gains. Concretely, we establish a universality principle under anisotropy and characterize the RFM generalization error via an equivalent noisy polynomial model. The effective degree of this polynomial—equivalently, which Hermite orders of the activation survive—is governed by the strength of input–label correlation, yielding an explicit boundary in the correlation–spike-magnitude plane. Below the boundary, the RFM collapses to a linear surrogate and can underperform strong linear baselines; above it, higher-order terms persist and the RFM achieves a clear nonlinear advantage. Numerical simulations and real-data experiments corroborate the theory and delineate the transition between these two regimes.

Förderung

  • TUBITAK-ARDEB 1001
  • Koc University & Is Bank Artificial Intelligence
  • TUBITAK

Zitieren

APA

Demir, S., & Doğan, Z. (2026). Input–Label Correlation Governs a Linear-to-Nonlinear Transition in Random Features under Spiked Covariance. SIAM Journal on Mathematics of Data Science, 8(4), 1322-1349. https://doi.org/10.1137/26m1853313

Vancouver

Demir S, Doğan Z. Input–Label Correlation Governs a Linear-to-Nonlinear Transition in Random Features under Spiked Covariance. SIAM Journal on Mathematics of Data Science. 2026;8(4):1322-1349. doi:10.1137/26m1853313

BibTeX
@article{Demir2026,
  author = {Demir, Samet and Doğan, Zafer},
  title = {Input–Label Correlation Governs a Linear-to-Nonlinear Transition in Random Features under Spiked Covariance},
  journal = {SIAM Journal on Mathematics of Data Science},
  year = {2026},
  volume = {8},
  number = {4},
  pages = {1322--1349},
  doi = {10.1137/26m1853313}
}
Kostenlos starten →