EUVIMEDEUVIMED entdecken
Crossref

Geometry of Linear Convolutional Networks

Kathlén Kohn, Thomas Merkh, Guido Montúfar, Matthew Trager

SIAM Journal on Applied Algebra and Geometry · 2022-09 · 6 (3) · 368-406

DOI: 10.1137/21m1441183 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SIAM Journal on Applied Algebra and Geometry, online erschienen am 02.08.2022. Verfasst von Kathlén Kohn und 3 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 37 Literaturangaben. Gefördert von: Horizon 2020 Framework Programme, Knut och Alice Wallenbergs Stiftelse.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We study the family of functions that are represented by a linear convolutional network (LCN). These functions form a semi-algebraic subset of the set of linear maps from input space to output space. In contrast, the families of functions represented by fully connected linear networks form algebraic sets. We observe that the functions represented by LCNs can be identified with polynomials that admit certain factorizations, and we use this perspective to describe the impact of the network's architecture on the geometry of the resulting function space. We further study the optimization of an objective function over an LCN, analyzing critical points in function space and in parameter space and describing dynamical invariants for gradient descent. Overall, our theory predicts that the optimized parameters of an LCN will often correspond to repeated filters across layers, or filters that can be decomposed as repeated filters. We also conduct numerical and symbolic experiments that illustrate our results and present an in-depth analysis of the landscape for small architectures.

Förderung

  • Horizon 2020 Framework Programme
  • Knut och Alice Wallenbergs Stiftelse

Zitieren

APA

Kohn, K., Merkh, T., Montúfar, G., & Trager, M. (2022). Geometry of Linear Convolutional Networks. SIAM Journal on Applied Algebra and Geometry, 6(3), 368-406. https://doi.org/10.1137/21m1441183

Vancouver

Kohn K, Merkh T, Montúfar G, Trager M. Geometry of Linear Convolutional Networks. SIAM Journal on Applied Algebra and Geometry. 2022;6(3):368-406. doi:10.1137/21m1441183

BibTeX
@article{Kohn2022,
  author = {Kohn, Kathlén and Merkh, Thomas and Montúfar, Guido and Trager, Matthew},
  title = {Geometry of Linear Convolutional Networks},
  journal = {SIAM Journal on Applied Algebra and Geometry},
  year = {2022},
  volume = {6},
  number = {3},
  pages = {368--406},
  doi = {10.1137/21m1441183}
}
Kostenlos starten →