Quantization-Induced Degradation Scaling Laws: A Unified Mathematical Framework
AIR Journal of Engineering and Technology · 18.05.2026 · 2026
DOI: 10.65737/airjet2026717 ↗Kurz gefasst
Fachartikel in der Zeitschrift AIR Journal of Engineering and Technology, online erschienen am 18.05.2026. Verfasst von Mohammad Hawwari und 6 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben.
Abstract (Originalsprache)
Quantization is a primary technique for efficient deployment of large language models (LLMs), yet the performance degradation it introduces—quantization-induced degradation (QiD)—is understood only through empirically fitted scaling laws. No first-principles derivation connecting bit-width, model size, and training tokens to QiD currently exists. In this paper, we provide the first analytical derivation of QiD scaling laws from first principles. By modeling uniform quantization as an additive perturbation on the trained weight tensor and expanding the Chinchilla loss surface to second order, we derive a closed-form QiD Master Equation: δL(b, N, D) = Γ · 2⁻²ᵇ · Nμ · D⁻ᵝ, where b is the bit-width, N the parameter count, D the training tokens, and Γ, μ, β are analytically determined constants. We prove that QiD converges to zero at rate O(4⁻ᵇ) per additional bit, establish an information-theoretic lower bound on achievable QiD via rate-distortion arguments, and demonstrate that the empirical scaling laws of Kumar et al. (2024), Ouyang et al. (2024), and Frantar et al. (2025) emerge as special cases of our unified framework. Numerical validation against published data confirms the analytical predictions. The framework provides design rules for minimum bit-width selection and reveals fundamental limits on quantization efficiency.
Zitieren
APA
Hawwari, M., Yilmazer, Y., Inan, K., Turkmen, G., Cetintas, Z., Lee, J., & Baburoglu, I. (2026). Quantization-Induced Degradation Scaling Laws: A Unified Mathematical Framework. AIR Journal of Engineering and Technology, 2026. https://doi.org/10.65737/airjet2026717
Vancouver
Hawwari M, Yilmazer Y, Inan K, Turkmen G, Cetintas Z, Lee J, et al. Quantization-Induced Degradation Scaling Laws: A Unified Mathematical Framework. AIR Journal of Engineering and Technology. 2026;2026. doi:10.65737/airjet2026717
BibTeX
@article{Hawwari2026,
author = {Hawwari, Mohammad and Yilmazer, Yusuf and Inan, Kivanc and Turkmen, Gokdeniz and Cetintas, Zehra and Lee, Joohyun and Baburoglu, Ilayda},
title = {Quantization-Induced Degradation Scaling Laws: A Unified Mathematical Framework},
journal = {AIR Journal of Engineering and Technology},
year = {2026},
volume = {2026},
doi = {10.65737/airjet2026717}
}
EUVIMED