AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration
GetMobile: Mobile Computing and Communications · 20.01.2025 · 28 (4) · 12-17
DOI: 10.1145/3714983.3714987 ↗Kurz gefasst
Fachartikel in der Zeitschrift GetMobile: Mobile Computing and Communications, online erschienen am 21.01.2025. Verfasst von Ji Lin und 5 weiteren Autor:innen. Die Arbeit wurde bisher 225-mal zitiert (Crossref) und nennt 10 Literaturangaben.
Abstract (Originalsprache)
Large language models (LLMs) have transformed numerous AI applications. On-device LLM is becoming increasingly important: running LLMs locally on edge devices can reduce cloud computing costs and protect users' privacy. However, the astronomical model size and the limited hardware resources pose significant deployment challenges. To solve these issues, we propose Activation-aware Weight Quantization (AWQ) and TinyChat, an algorithm-system full-stack solution for efficient on-device LLM deployment. AWQ is a novel quantization method that identifies and protects salient weights based on activation distribution, significantly reducing model size while preserving performance. TinyChat, an optimized inference framework, translates AWQ's theoretical memory savings into practical speedups through techniques such as on-the-fly dequantization, SIMD-aware weight packing, and kernel fusion. Together, they enable 4x model size reduction and 3-4x acceleration across various edge platforms, from high-end desktop GPUs to resource-constrained IoT devices. This solution democratizes on-device LLM deployment, offering privacy-preserving, low-latency AI capabilities across a wide range of applications.
Zitieren
APA
Lin, J., Tang, J., Tang, H., Yang, S., Xiao, G., & Han, S. (2025). AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. GetMobile: Mobile Computing and Communications, 28(4), 12-17. https://doi.org/10.1145/3714983.3714987
Vancouver
Lin J, Tang J, Tang H, Yang S, Xiao G, Han S. AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. GetMobile: Mobile Computing and Communications. 2025;28(4):12-17. doi:10.1145/3714983.3714987
BibTeX
@article{Lin2025,
author = {Lin, Ji and Tang, Jiaming and Tang, Haotian and Yang, Shang and Xiao, Guangxuan and Han, Song},
title = {AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration},
journal = {GetMobile: Mobile Computing and Communications},
year = {2025},
volume = {28},
number = {4},
pages = {12--17},
doi = {10.1145/3714983.3714987}
}
EUVIMED