EUVIMEDEUVIMED entdecken
Crossref

Unveiling Hidden Convexity in Deep Learning: A sparse signal processing perspective [Special Issue on the Mathematics of Deep Learning]

Emi Zeger, Mert Pilanci

IEEE Signal Processing Magazine · 2026-05 · 43 (3) · 86-101

DOI: 10.1109/msp.2026.3677783 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Signal Processing Magazine, 2026-05. Verfasst von Emi Zeger und 1 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Gefördert von: National Science Foundation CAREER Award, National Institutes of Health, Office of Naval Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep neural networks (DNNs) have achieved groundbreaking success in various fields. Yet, the inherent nonconvexity of their training objectives and the complexity of their architectures present challenges for optimization and theoretical understanding. Our article examines the hidden convex structure within DNNs by drawing upon frameworks from sparse signal processing, specifically least absolute shrinkage and selection operator (LASSO), group LASSO, and nuclear-norm regularized models. By leveraging these established techniques from sparse linear models, we demonstrate how convex formulations can yield global optima and facilitate the interpretability of deep networks. Furthermore, we explore convolutional neural networks (CNNs), transformers, diffusion models, and generative adversarial networks from this convex regularization framework. We show how hypercomplex structures, such as geometric algebra, emerge in this analysis, offering new insights into the generalization to unseen data. We aim to engage the signal processing community in exploring these convex perspectives to better understand deep learning and its practical applications.

Förderung

  • National Science Foundation CAREER Award
  • National Institutes of Health
  • Office of Naval Research
  • Defense Advanced Research Projects Agency
  • National Science Foundation Graduate Research Fellowship

Zitieren

APA

Zeger, E., & Pilanci, M. (2026). Unveiling Hidden Convexity in Deep Learning: A sparse signal processing perspective [Special Issue on the Mathematics of Deep Learning]. IEEE Signal Processing Magazine, 43(3), 86-101. https://doi.org/10.1109/msp.2026.3677783

Vancouver

Zeger E, Pilanci M. Unveiling Hidden Convexity in Deep Learning: A sparse signal processing perspective [Special Issue on the Mathematics of Deep Learning]. IEEE Signal Processing Magazine. 2026;43(3):86-101. doi:10.1109/msp.2026.3677783

BibTeX
@article{Zeger2026,
  author = {Zeger, Emi and Pilanci, Mert},
  title = {Unveiling Hidden Convexity in Deep Learning: A sparse signal processing perspective [Special Issue on the Mathematics of Deep Learning]},
  journal = {IEEE Signal Processing Magazine},
  year = {2026},
  volume = {43},
  number = {3},
  pages = {86--101},
  doi = {10.1109/msp.2026.3677783}
}
Kostenlos starten →