Geometry of Linear Convolutional Networks
SIAM Journal on Applied Algebra and Geometry · 2022-09 · 6 (3) · 368-406
DOI: 10.1137/21m1441183 ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Journal on Applied Algebra and Geometry, online erschienen am 02.08.2022. Verfasst von Kathlén Kohn und 3 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 37 Literaturangaben. Gefördert von: Horizon 2020 Framework Programme, Knut och Alice Wallenbergs Stiftelse.
Abstract (Originalsprache)
We study the family of functions that are represented by a linear convolutional network (LCN). These functions form a semi-algebraic subset of the set of linear maps from input space to output space. In contrast, the families of functions represented by fully connected linear networks form algebraic sets. We observe that the functions represented by LCNs can be identified with polynomials that admit certain factorizations, and we use this perspective to describe the impact of the network's architecture on the geometry of the resulting function space. We further study the optimization of an objective function over an LCN, analyzing critical points in function space and in parameter space and describing dynamical invariants for gradient descent. Overall, our theory predicts that the optimized parameters of an LCN will often correspond to repeated filters across layers, or filters that can be decomposed as repeated filters. We also conduct numerical and symbolic experiments that illustrate our results and present an in-depth analysis of the landscape for small architectures.
Förderung
- Horizon 2020 Framework Programme
- Knut och Alice Wallenbergs Stiftelse
Zitieren
APA
Kohn, K., Merkh, T., Montúfar, G., & Trager, M. (2022). Geometry of Linear Convolutional Networks. SIAM Journal on Applied Algebra and Geometry, 6(3), 368-406. https://doi.org/10.1137/21m1441183
Vancouver
Kohn K, Merkh T, Montúfar G, Trager M. Geometry of Linear Convolutional Networks. SIAM Journal on Applied Algebra and Geometry. 2022;6(3):368-406. doi:10.1137/21m1441183
BibTeX
@article{Kohn2022,
author = {Kohn, Kathlén and Merkh, Thomas and Montúfar, Guido and Trager, Matthew},
title = {Geometry of Linear Convolutional Networks},
journal = {SIAM Journal on Applied Algebra and Geometry},
year = {2022},
volume = {6},
number = {3},
pages = {368--406},
doi = {10.1137/21m1441183}
}
EUVIMED