Notes on the Symmetries of 2-Layer ReLU-Networks
Proceedings of the Northern Lights Deep Learning Workshop · 2020
DOI: 10.7557/18.5150 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the Northern Lights Deep Learning Workshop, 2020. Verfasst von Henning Petzka und 2 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 9 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Symmetries in neural networks allow different weight configurations leading to the same network function. For odd activation functions, the set of transformations mapping between such configurations have been studied extensively, but less is known for neural networks with ReLU activation functions. We give a complete characterization for fully-connected networks with two layers. Apart from two well-known transformations, only degenerated situations allow additional transformations that leave the network function unchanged. Reduction steps can remove only part of the degenerated cases. Finally, we present a non-degenerate situation for deep neural networks leading to new transformations leaving the network function intact.
Zitieren
APA
Petzka, H., Trimmel, M., & Sminchisescu, C. (2020). Notes on the Symmetries of 2-Layer ReLU-Networks. Proceedings of the Northern Lights Deep Learning Workshop. https://doi.org/10.7557/18.5150
Vancouver
Petzka H, Trimmel M, Sminchisescu C. Notes on the Symmetries of 2-Layer ReLU-Networks. Proceedings of the Northern Lights Deep Learning Workshop. 2020. doi:10.7557/18.5150
BibTeX
@article{Petzka2020,
author = {Petzka, Henning and Trimmel, Martin and Sminchisescu, Cristian},
title = {Notes on the Symmetries of 2-Layer ReLU-Networks},
journal = {Proceedings of the Northern Lights Deep Learning Workshop},
year = {2020},
doi = {10.7557/18.5150}
}
EUVIMED