EUVIMEDEUVIMED entdecken
Crossref

Towards Lower Bounds on the Depth of ReLU Neural Networks

Christoph Hertrich, Amitabh Basu, Marco Di Summa, Martin Skutella

SIAM Journal on Discrete Mathematics · 30.06.2023 · 37 (2) · 997-1029

DOI: 10.1137/22m1489332 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Discrete Mathematics, online erschienen am 14.06.2023. Verfasst von Christoph Hertrich und 3 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 77 Literaturangaben. Gefördert von: Air Force Office of Scientific Research, Deutsche Forschungsgemeinschaft, National Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We contribute to a better understanding of the class of functions that can be represented by a neural network with ReLU activations and a given architecture. Using techniques from mixed-integer optimization, polyhedral theory, and tropical geometry, we provide a mathematical counterbalance to the universal approximation theorems which suggest that a single hidden layer is sufficient for learning any function. In particular, we investigate whether the class of exactly representable functions strictly increases by adding more layers (with no restrictions on size). As a by-product of our investigations, we settle an old conjecture about piecewise linear functions by Wang and Sun [ IEEE Trans. Inform. Theory, 51 (2005), pp. 4425–4431] in the affirmative. We also present upper bounds on the sizes of neural networks required to represent functions with logarithmic depth.

Förderung

  • Air Force Office of Scientific Research
  • Deutsche Forschungsgemeinschaft
  • National Science Foundation

Zitieren

APA

Hertrich, C., Basu, A., Summa, M. D., & Skutella, M. (2023). Towards Lower Bounds on the Depth of ReLU Neural Networks. SIAM Journal on Discrete Mathematics, 37(2), 997-1029. https://doi.org/10.1137/22m1489332

Vancouver

Hertrich C, Basu A, Summa MD, Skutella M. Towards Lower Bounds on the Depth of ReLU Neural Networks. SIAM Journal on Discrete Mathematics. 2023;37(2):997-1029. doi:10.1137/22m1489332

BibTeX
@article{Hertrich2023,
  author = {Hertrich, Christoph and Basu, Amitabh and Summa, Marco Di and Skutella, Martin},
  title = {Towards Lower Bounds on the Depth of ReLU Neural Networks},
  journal = {SIAM Journal on Discrete Mathematics},
  year = {2023},
  volume = {37},
  number = {2},
  pages = {997--1029},
  doi = {10.1137/22m1489332}
}
Kostenlos starten →