OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (12) · 13355-13364
DOI: 10.1609/aaai.v38i12.29237 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Changhun Lee und 4 weiteren Autor:innen. Die Arbeit wurde bisher 75-mal zitiert (Crossref).
Abstract (Originalsprache)
Large language models (LLMs) with hundreds of billions of parameters require powerful server-grade GPUs for inference, limiting their practical deployment. To address this challenge, we introduce the outlier-aware weight quantization (OWQ) method, which aims to minimize LLM's footprint through low-precision representation. OWQ prioritizes a small subset of structured weights sensitive to quantization, storing them in high-precision, while applying highly tuned quantization to the remaining dense weights. This sensitivity-aware mixed-precision scheme reduces the quantization error notably, and extensive experiments demonstrate that 3.1-bit models using OWQ perform comparably to 4-bit models optimized by OPTQ. Furthermore, OWQ incorporates a parameter-efficient fine-tuning for task-specific adaptation, called weak column tuning (WCT), enabling accurate task-specific LLM adaptation with minimal memory overhead in the optimized format. OWQ represents a notable advancement in the flexibility, efficiency, and practicality of LLM optimization literature. The source code is available at https://github.com/xvyaward/owq.
Zitieren
APA
Lee, C., Jin, J., Kim, T., Kim, H., & Park, E. (2024). OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 38(12), 13355-13364. https://doi.org/10.1609/aaai.v38i12.29237
Vancouver
Lee C, Jin J, Kim T, Kim H, Park E. OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(12):13355-13364. doi:10.1609/aaai.v38i12.29237
BibTeX
@article{Lee2024,
author = {Lee, Changhun and Jin, Jungyu and Kim, Taesu and Kim, Hyungjun and Park, Eunhyeok},
title = {OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {12},
pages = {13355--13364},
doi = {10.1609/aaai.v38i12.29237}
}
EUVIMED