Unveiling Hidden Convexity in Deep Learning: A sparse signal processing perspective [Special Issue on the Mathematics of Deep Learning]
IEEE Signal Processing Magazine · 2026-05 · 43 (3) · 86-101
DOI: 10.1109/msp.2026.3677783 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Signal Processing Magazine, 2026-05. Verfasst von Emi Zeger und 1 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Gefördert von: National Science Foundation CAREER Award, National Institutes of Health, Office of Naval Research.
Abstract (Originalsprache)
Deep neural networks (DNNs) have achieved groundbreaking success in various fields. Yet, the inherent nonconvexity of their training objectives and the complexity of their architectures present challenges for optimization and theoretical understanding. Our article examines the hidden convex structure within DNNs by drawing upon frameworks from sparse signal processing, specifically least absolute shrinkage and selection operator (LASSO), group LASSO, and nuclear-norm regularized models. By leveraging these established techniques from sparse linear models, we demonstrate how convex formulations can yield global optima and facilitate the interpretability of deep networks. Furthermore, we explore convolutional neural networks (CNNs), transformers, diffusion models, and generative adversarial networks from this convex regularization framework. We show how hypercomplex structures, such as geometric algebra, emerge in this analysis, offering new insights into the generalization to unseen data. We aim to engage the signal processing community in exploring these convex perspectives to better understand deep learning and its practical applications.
Förderung
- National Science Foundation CAREER Award
- National Institutes of Health
- Office of Naval Research
- Defense Advanced Research Projects Agency
- National Science Foundation Graduate Research Fellowship
Zitieren
APA
Zeger, E., & Pilanci, M. (2026). Unveiling Hidden Convexity in Deep Learning: A sparse signal processing perspective [Special Issue on the Mathematics of Deep Learning]. IEEE Signal Processing Magazine, 43(3), 86-101. https://doi.org/10.1109/msp.2026.3677783
Vancouver
Zeger E, Pilanci M. Unveiling Hidden Convexity in Deep Learning: A sparse signal processing perspective [Special Issue on the Mathematics of Deep Learning]. IEEE Signal Processing Magazine. 2026;43(3):86-101. doi:10.1109/msp.2026.3677783
BibTeX
@article{Zeger2026,
author = {Zeger, Emi and Pilanci, Mert},
title = {Unveiling Hidden Convexity in Deep Learning: A sparse signal processing perspective [Special Issue on the Mathematics of Deep Learning]},
journal = {IEEE Signal Processing Magazine},
year = {2026},
volume = {43},
number = {3},
pages = {86--101},
doi = {10.1109/msp.2026.3677783}
}
EUVIMED