Adaptive A-Semilogarithmic Gradient Quantization for Efficient Deep Neural Network Training
Algorithms · 29.06.2026 · 19 (7) · 521
DOI: 10.3390/a19070521 ↗Kurz gefasst
Fachartikel in der Zeitschrift Algorithms, online erschienen am 29.06.2026. Verfasst von Stefan Panić und 3 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science, Technological Development and Innovation of the Republic of Serbia.
Abstract (Originalsprache)
This paper introduces an adaptive A-semilogarithmic gradient quantization framework aimed at reducing memory overhead and computational complexity during the training of deep neural networks. The approach employs a semilogarithmic companding function parameterized by a dynamically adjusted scaling factor A, which evolves in response to the statistical properties of gradients throughout the training process. Two distinct quantization strategies are proposed and evaluated: The switching piecewise A-quantizer, which adaptively toggles between low-bit uniform and high-bit semilogarithmic quantization according to an exponentially weighted moving-average (EMA) estimate of gradient variance; and the hybrid A-quantizer, which statically partitions the gradient domain, applying uniform quantization in low-magnitude regions and semilogarithmic companding in high-magnitude regions. The proposed methods are empirically evaluated on both multilayer perceptron (MLP) and convolutional neural network (CNN) architectures using tabular and image-classification benchmarks, including DCCC, CIFAR-10, CIFAR-100, and ImageNet. Quantitative results demonstrate that both models achieve comparable classification accuracy to full-precision (FP32) baselines while significantly reducing gradient reconstruction error. Notably, the hybrid A-quantizer consistently yields better validation accuracy, reduced RMSE, and improved convergence behavior relative to its switching counterpart. These findings underscore the effectiveness of hybrid semilogarithmic quantization as a robust and efficient solution for training deep models in resource-constrained or bandwidth-limited environments, with strong potential for scalable deployment across diverse hardware platforms.
Förderung
- Ministry of Science, Technological Development and Innovation of the Republic of Serbia
Zitieren
APA
Panić, S., Dubljanin, M., Savić, M., & Smilić, M. (2026). Adaptive A-Semilogarithmic Gradient Quantization for Efficient Deep Neural Network Training. Algorithms, 19(7), 521. https://doi.org/10.3390/a19070521
Vancouver
Panić S, Dubljanin M, Savić M, Smilić M. Adaptive A-Semilogarithmic Gradient Quantization for Efficient Deep Neural Network Training. Algorithms. 2026;19(7):521. doi:10.3390/a19070521
BibTeX
@article{Pani2026,
author = {Panić, Stefan and Dubljanin, Milan and Savić, Milan and Smilić, Marko},
title = {Adaptive A-Semilogarithmic Gradient Quantization for Efficient Deep Neural Network Training},
journal = {Algorithms},
year = {2026},
volume = {19},
number = {7},
pages = {521},
doi = {10.3390/a19070521}
}
EUVIMED