EUVIMEDEUVIMED entdecken
Crossref

Quantization-Induced Degradation Scaling Laws: A Unified Mathematical Framework

Mohammad Hawwari, Yusuf Yilmazer, Kivanc Inan, Gokdeniz Turkmen, Zehra Cetintas, Joohyun Lee, Ilayda Baburoglu

AIR Journal of Engineering and Technology · 18.05.2026 · 2026

DOI: 10.65737/airjet2026717 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AIR Journal of Engineering and Technology, online erschienen am 18.05.2026. Verfasst von Mohammad Hawwari und 6 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Quantization is a primary technique for efficient deployment of large language models (LLMs), yet the performance degradation it introduces—quantization-induced degradation (QiD)—is understood only through empirically fitted scaling laws. No first-principles derivation connecting bit-width, model size, and training tokens to QiD currently exists. In this paper, we provide the first analytical derivation of QiD scaling laws from first principles. By modeling uniform quantization as an additive perturbation on the trained weight tensor and expanding the Chinchilla loss surface to second order, we derive a closed-form QiD Master Equation: δL(b, N, D) = Γ · 2⁻²ᵇ · Nμ · D⁻ᵝ, where b is the bit-width, N the parameter count, D the training tokens, and Γ, μ, β are analytically determined constants. We prove that QiD converges to zero at rate O(4⁻ᵇ) per additional bit, establish an information-theoretic lower bound on achievable QiD via rate-distortion arguments, and demonstrate that the empirical scaling laws of Kumar et al. (2024), Ouyang et al. (2024), and Frantar et al. (2025) emerge as special cases of our unified framework. Numerical validation against published data confirms the analytical predictions. The framework provides design rules for minimum bit-width selection and reveals fundamental limits on quantization efficiency.

Zitieren

APA

Hawwari, M., Yilmazer, Y., Inan, K., Turkmen, G., Cetintas, Z., Lee, J., & Baburoglu, I. (2026). Quantization-Induced Degradation Scaling Laws: A Unified Mathematical Framework. AIR Journal of Engineering and Technology, 2026. https://doi.org/10.65737/airjet2026717

Vancouver

Hawwari M, Yilmazer Y, Inan K, Turkmen G, Cetintas Z, Lee J, et al. Quantization-Induced Degradation Scaling Laws: A Unified Mathematical Framework. AIR Journal of Engineering and Technology. 2026;2026. doi:10.65737/airjet2026717

BibTeX
@article{Hawwari2026,
  author = {Hawwari, Mohammad and Yilmazer, Yusuf and Inan, Kivanc and Turkmen, Gokdeniz and Cetintas, Zehra and Lee, Joohyun and Baburoglu, Ilayda},
  title = {Quantization-Induced Degradation Scaling Laws: A Unified Mathematical Framework},
  journal = {AIR Journal of Engineering and Technology},
  year = {2026},
  volume = {2026},
  doi = {10.65737/airjet2026717}
}
Kostenlos starten →