Deep Learning Meets Sparse Regularization: A signal processing perspective
IEEE Signal Processing Magazine · 2023-09 · 40 (6) · 63-74
DOI: 10.1109/msp.2023.3286988 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Signal Processing Magazine, 2023-09. Verfasst von Rahul Parhi und 1 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: National Science Foundation, European Research Council, U.S. Office of Naval Research.
Abstract (Originalsprache)
Deep learning (DL) has been wildly successful in practice, and most of the state-of-the-art machine learning methods are based on neural networks (NNs). Lacking, however, is a rigorous mathematical theory that adequately explains the amazing performance of deep NNs (DNNs). In this article, we present a relatively new mathematical framework that provides the beginning of a deeper understanding of DL. This framework precisely characterizes the functional properties of NNs that are trained to fit to data. The key mathematical tools that support this framework include transform-domain sparse regularization, the Radon transform of computed tomography, and approximation theory, which are all techniques deeply rooted in signal processing. This framework explains the effect of weight decay regularization in NN training, use of skip connections and low-rank weight matrices in network architectures, role of sparsity in NNs, and explains why NNs can perform well in high-dimensional problems.
Förderung
- National Science Foundation
- European Research Council
- U.S. Office of Naval Research
- Air Force Office of Scientific Research/Air Force Research Laboratory
Zitieren
APA
Parhi, R., & Nowak, R. D. (2023). Deep Learning Meets Sparse Regularization: A signal processing perspective. IEEE Signal Processing Magazine, 40(6), 63-74. https://doi.org/10.1109/msp.2023.3286988
Vancouver
Parhi R, Nowak RD. Deep Learning Meets Sparse Regularization: A signal processing perspective. IEEE Signal Processing Magazine. 2023;40(6):63-74. doi:10.1109/msp.2023.3286988
BibTeX
@article{Parhi2023,
author = {Parhi, Rahul and Nowak, Robert D.},
title = {Deep Learning Meets Sparse Regularization: A signal processing perspective},
journal = {IEEE Signal Processing Magazine},
year = {2023},
volume = {40},
number = {6},
pages = {63--74},
doi = {10.1109/msp.2023.3286988}
}
EUVIMED