EUVIMEDEUVIMED entdecken
Crossref

Convergence rates for shallow neural networks learned by gradient descent

Alina Braun, Michael Kohler, Sophie Langer, Harro Walk

Bernoulli · 01.02.2024 · 30 (1) · 475-502

DOI: 10.3150/23-bej1605 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bernoulli, erschienen am 01.02.2024. Verfasst von Alina Braun und 3 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 56 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this paper we analyze the L2 error of neural network regression estimates with one hidden layer. Under the assumption that the Fourier transform of the regression function decays suitably fast, we show that an estimate, where all initial weights are chosen according to proper uniform distributions and where the weights are learned by gradient descent, achieves a rate of convergence of 1∕n (up to a logarithmic factor). Our statistical analysis implies that the key aspect behind this result is the proper choice of the initial inner weights and the adjustment of the outer weights via gradient descent. This indicates that we can also simply use linear least squares to choose the outer weights. We prove a corresponding theoretical result and compare our new linear least squares neural network estimate with standard neural network estimates via simulated data. Our simulations show that our theoretical considerations lead to an estimate with an improved performance in many cases.

Zitieren

APA

Braun, A., Kohler, M., Langer, S., & Walk, H. (2024). Convergence rates for shallow neural networks learned by gradient descent. Bernoulli, 30(1), 475-502. https://doi.org/10.3150/23-bej1605

Vancouver

Braun A, Kohler M, Langer S, Walk H. Convergence rates for shallow neural networks learned by gradient descent. Bernoulli. 2024;30(1):475-502. doi:10.3150/23-bej1605

BibTeX
@article{Braun2024,
  author = {Braun, Alina and Kohler, Michael and Langer, Sophie and Walk, Harro},
  title = {Convergence rates for shallow neural networks learned by gradient descent},
  journal = {Bernoulli},
  year = {2024},
  volume = {30},
  number = {1},
  pages = {475--502},
  doi = {10.3150/23-bej1605}
}
Kostenlos starten →