EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Adaptive A-Semilogarithmic Gradient Quantization for Efficient Deep Neural Network Training

Stefan Panić, Milan Dubljanin, Milan Savić, Marko Smilić

Algorithms · 29.06.2026 · 19 (7) · 521

DOI: 10.3390/a19070521 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Algorithms, online erschienen am 29.06.2026. Verfasst von Stefan Panić und 3 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science, Technological Development and Innovation of the Republic of Serbia.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper introduces an adaptive A-semilogarithmic gradient quantization framework aimed at reducing memory overhead and computational complexity during the training of deep neural networks. The approach employs a semilogarithmic companding function parameterized by a dynamically adjusted scaling factor A, which evolves in response to the statistical properties of gradients throughout the training process. Two distinct quantization strategies are proposed and evaluated: The switching piecewise A-quantizer, which adaptively toggles between low-bit uniform and high-bit semilogarithmic quantization according to an exponentially weighted moving-average (EMA) estimate of gradient variance; and the hybrid A-quantizer, which statically partitions the gradient domain, applying uniform quantization in low-magnitude regions and semilogarithmic companding in high-magnitude regions. The proposed methods are empirically evaluated on both multilayer perceptron (MLP) and convolutional neural network (CNN) architectures using tabular and image-classification benchmarks, including DCCC, CIFAR-10, CIFAR-100, and ImageNet. Quantitative results demonstrate that both models achieve comparable classification accuracy to full-precision (FP32) baselines while significantly reducing gradient reconstruction error. Notably, the hybrid A-quantizer consistently yields better validation accuracy, reduced RMSE, and improved convergence behavior relative to its switching counterpart. These findings underscore the effectiveness of hybrid semilogarithmic quantization as a robust and efficient solution for training deep models in resource-constrained or bandwidth-limited environments, with strong potential for scalable deployment across diverse hardware platforms.

Förderung

  • Ministry of Science, Technological Development and Innovation of the Republic of Serbia

Zitieren

APA

Panić, S., Dubljanin, M., Savić, M., & Smilić, M. (2026). Adaptive A-Semilogarithmic Gradient Quantization for Efficient Deep Neural Network Training. Algorithms, 19(7), 521. https://doi.org/10.3390/a19070521

Vancouver

Panić S, Dubljanin M, Savić M, Smilić M. Adaptive A-Semilogarithmic Gradient Quantization for Efficient Deep Neural Network Training. Algorithms. 2026;19(7):521. doi:10.3390/a19070521

BibTeX
@article{Pani2026,
  author = {Panić, Stefan and Dubljanin, Milan and Savić, Milan and Smilić, Marko},
  title = {Adaptive A-Semilogarithmic Gradient Quantization for Efficient Deep Neural Network Training},
  journal = {Algorithms},
  year = {2026},
  volume = {19},
  number = {7},
  pages = {521},
  doi = {10.3390/a19070521}
}
Kostenlos starten →