EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Infinite‐width limit of deep linear neural networks

Lénaïc Chizat, Maria Colombo, Xavier Fernández‐Real, Alessio Figalli

Communications on Pure and Applied Mathematics · 2024-10 · 77 (10) · 3958-4007

DOI: 10.1002/cpa.22200 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Communications on Pure and Applied Mathematics, online erschienen am 06.05.2024. Verfasst von Lénaïc Chizat und 3 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Stavros Niarchos Foundation, Agencia Estatal de Investigación, European Research Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper studies the infinite‐width limit of deep linear neural networks (NNs) initialized with random parameters. We obtain that, when the number of parameters diverges, the training dynamics converge (in a precise sense) to the dynamics obtained from a gradient descent on an infinitely wide deterministic linear NN. Moreover, even if the weights remain random, we get their precise law along the training dynamics, and prove a quantitative convergence result of the linear predictor in terms of the number of parameters. We finally study the continuous‐time limit obtained for infinitely wide linear NNs and show that the linear predictors of the NN converge at an exponential rate to the minimal ‐norm minimizer of the risk.

Förderung

  • Stavros Niarchos Foundation
  • Agencia Estatal de Investigación
  • European Research Council

Zitieren

APA

Chizat, L., Colombo, M., Fernández‐Real, X., & Figalli, A. (2024). Infinite‐width limit of deep linear neural networks. Communications on Pure and Applied Mathematics, 77(10), 3958-4007. https://doi.org/10.1002/cpa.22200

Vancouver

Chizat L, Colombo M, Fernández‐Real X, Figalli A. Infinite‐width limit of deep linear neural networks. Communications on Pure and Applied Mathematics. 2024;77(10):3958-4007. doi:10.1002/cpa.22200

BibTeX
@article{Chizat2024,
  author = {Chizat, Lénaïc and Colombo, Maria and Fernández‐Real, Xavier and Figalli, Alessio},
  title = {Infinite‐width limit of deep linear neural networks},
  journal = {Communications on Pure and Applied Mathematics},
  year = {2024},
  volume = {77},
  number = {10},
  pages = {3958--4007},
  doi = {10.1002/cpa.22200}
}
Kostenlos starten →