Low-Rank Tensor Decompositions for the Theory of Neural Networks: Understanding deep learning through the lens of tensor formats [Special Issue on the Mathematics of Deep Learning]
IEEE Signal Processing Magazine · 2026-03 · 43 (2) · 107-121
DOI: 10.1109/msp.2025.3646570 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Signal Processing Magazine, 2026-03. Verfasst von Ricardo Borsoi und 2 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Gefördert von: French National Research Agency, National Science Foundation.
Abstract (Originalsprache)
The groundbreaking performance of deep neural networks (NNs) promoted a surge of interest in providing a mathematical basis to deep learning theory. Low-rank tensor decompositions are specially fit for this task due to their close connection to NNs and their rich theoretical results. Different tensor decompositions have strong uniqueness guarantees, which allow for a direct interpretation of their factors, and polynomial time algorithms have been proposed to compute them. Through the connections between tensors and NNs, such results supported many important advances in the theory of NNs. In this review article, we show how low-rank tensor methods—which have been a core tool in the signal processing and machine learning communities—play a fundamental role in theoretically explaining different aspects of the performance of deep NNs, including their expressivity, algorithmic learnability and computational hardness, generalization, and identifiability. Our goal is to give an accessible overview of existing approaches (developed by different communities, ranging from computer science to mathematics) in a coherent and unified way and to open a broader perspective on the use of low-rank tensor decompositions for the theory of deep NNs.
Förderung
- French National Research Agency
- National Science Foundation
Zitieren
APA
Borsoi, R., Usevich, K., & Clausel, M. (2026). Low-Rank Tensor Decompositions for the Theory of Neural Networks: Understanding deep learning through the lens of tensor formats [Special Issue on the Mathematics of Deep Learning]. IEEE Signal Processing Magazine, 43(2), 107-121. https://doi.org/10.1109/msp.2025.3646570
Vancouver
Borsoi R, Usevich K, Clausel M. Low-Rank Tensor Decompositions for the Theory of Neural Networks: Understanding deep learning through the lens of tensor formats [Special Issue on the Mathematics of Deep Learning]. IEEE Signal Processing Magazine. 2026;43(2):107-121. doi:10.1109/msp.2025.3646570
BibTeX
@article{Borsoi2026,
author = {Borsoi, Ricardo and Usevich, Konstantin and Clausel, Marianne},
title = {Low-Rank Tensor Decompositions for the Theory of Neural Networks: Understanding deep learning through the lens of tensor formats [Special Issue on the Mathematics of Deep Learning]},
journal = {IEEE Signal Processing Magazine},
year = {2026},
volume = {43},
number = {2},
pages = {107--121},
doi = {10.1109/msp.2025.3646570}
}
EUVIMED