EUVIMEDEUVIMED entdecken
Crossref

SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization

Zhixiong Zhao, Fangxin Liu, Junjie Wang, Chenyang Guan, Zongwu Wang, Li Jiang, Haibing Guan

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (34) · 28786-28794

DOI: 10.1609/aaai.v40i34.40112 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zhixiong Zhao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The emergence of accurate open large language models (LLMs) has sparked a push for advanced quantization techniques to enable efficient deployment on end-user devices. In this paper, we revisit the challenge of extreme LLM compression---targeting ultra-low-bit quantization for both activations and weights---from a Fourier frequency domain perspective. We propose SpecQuant, a two-stage framework that tackles activation outliers and cross-channel variance. In the first stage, activation outliers are smoothed and transferred into the weight matrix to simplify downstream quantization. In the second stage, we apply channel-wise low-frequency Fourier truncation to suppress high-frequency components while preserving essential signal energy, improving quantization robustness. Our method builds on the principle that most of the weight energy is concentrated in low-frequency components, which can be retained with minimal impact on model accuracy. To enable runtime adaptability, we introduce a lightweight truncation module during inference that adjusts truncation thresholds based on channel characteristics. On LLaMA-3 8B, SpecQuant achieves 4-bit quantization for both weights and activations, narrowing the zero-shot accuracy gap to only 1.5% compared to full precision, while delivering 2× faster inference and 3× lower memory usage.

Zitieren

APA

Zhao, Z., Liu, F., Wang, J., Guan, C., Wang, Z., Jiang, L., & Guan, H. (2026). SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization. Proceedings of the AAAI Conference on Artificial Intelligence, 40(34), 28786-28794. https://doi.org/10.1609/aaai.v40i34.40112

Vancouver

Zhao Z, Liu F, Wang J, Guan C, Wang Z, Jiang L, et al. SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(34):28786-28794. doi:10.1609/aaai.v40i34.40112

BibTeX
@article{Zhao2026,
  author = {Zhao, Zhixiong and Liu, Fangxin and Wang, Junjie and Guan, Chenyang and Wang, Zongwu and Jiang, Li and Guan, Haibing},
  title = {SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {34},
  pages = {28786--28794},
  doi = {10.1609/aaai.v40i34.40112}
}
Kostenlos starten →