Deep Neural Network Compression via Data-Driven Low-Rank Singular Value Decomposition
Journal of Artificial Intelligence Research · 23.07.2026 · 86
DOI: 10.1613/jair.1.18694 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Artificial Intelligence Research, online erschienen am 23.07.2026. Verfasst von Abdelfattah Toulaoui und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Deep neural network compression focuses on reducing the number of parameters and computational complexity of neural networks, enabling their use in resource-constrained environments where smaller networks, faster response times, and lower energy consumption are critical. In this paper, we propose a novel compression method that uses Singular Value Decomposition (SVD) informed by data examples. Unlike existing SVD-based approaches that operate solely on network weights, our method leverages input data to more effectively preserve essential information during compression. This data-driven approach significantly enhances performance compared to state-of-the-art techniques. Through experiments on standard benchmark datasets, we show that our method achieves substantial reductions in model size and computation with minimal impact on accuracy. Furthermore, we demonstrate its applicability in compressing deep neural networks to a fraction of their original size, achieving competitive accuracy of networks with a compression ratio of less than 3× with no fine-tuning, and at most 25 epochs of fine-tuning for higher compression ratios.
Zitieren
APA
Toulaoui, A., Khalfi, H., & Hafidi, I. (2026). Deep Neural Network Compression via Data-Driven Low-Rank Singular Value Decomposition. Journal of Artificial Intelligence Research, 86. https://doi.org/10.1613/jair.1.18694
Vancouver
Toulaoui A, Khalfi H, Hafidi I. Deep Neural Network Compression via Data-Driven Low-Rank Singular Value Decomposition. Journal of Artificial Intelligence Research. 2026;86. doi:10.1613/jair.1.18694
BibTeX
@article{Toulaoui2026,
author = {Toulaoui, Abdelfattah and Khalfi, Hamza and Hafidi, Imad},
title = {Deep Neural Network Compression via Data-Driven Low-Rank Singular Value Decomposition},
journal = {Journal of Artificial Intelligence Research},
year = {2026},
volume = {86},
doi = {10.1613/jair.1.18694}
}
EUVIMED