Compositional Function Spaces for Deep Learning
SIAM Review · 28.02.2026 · 68 (1) · 127-149
DOI: 10.1137/25m1802948 ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Review, online erschienen am 09.02.2026. Verfasst von Rahul Parhi und 1 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 78 Literaturangaben. Gefördert von: ONR MURI, AFOSR/AFRL, National Science Foundation.
Abstract (Originalsprache)
We present a variational framework for studying functions learned by deep neural networks with rectified linear unit nonlinearities. We introduce a function space built from compositions of functions of second-order Radon-domain bounded variation. The compositional form of these functions captures the structure of deep neural networks. We prove a representer theorem that shows that deep neural networks with finite width solve regularized data-fitting problems over this space. The critical width is controlled by the square of the number of training data. This perspective explains the effect of weight-decay regularization in neural network training, the importance of skip connections, and the role of sparsity in neural networks. By considering the function-space perspective, we provide sharp links between deep learning and variational methods.
Förderung
- ONR MURI
- AFOSR/AFRL
- National Science Foundation
Zitieren
APA
Parhi, R., & Nowak, R. D. (2026). Compositional Function Spaces for Deep Learning. SIAM Review, 68(1), 127-149. https://doi.org/10.1137/25m1802948
Vancouver
Parhi R, Nowak RD. Compositional Function Spaces for Deep Learning. SIAM Review. 2026;68(1):127-149. doi:10.1137/25m1802948
BibTeX
@article{Parhi2026,
author = {Parhi, Rahul and Nowak, Robert D.},
title = {Compositional Function Spaces for Deep Learning},
journal = {SIAM Review},
year = {2026},
volume = {68},
number = {1},
pages = {127--149},
doi = {10.1137/25m1802948}
}
EUVIMED