HALO: Hardware-Aware Quantization with Low Critical-Path-Delay Weights for LLM Acceleration
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (27) · 22472-22481
DOI: 10.1609/aaai.v40i27.39406 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Rohan Juneja und 4 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref).
Abstract (Originalsprache)
Quantization is critical for efficiently deploying large language models (LLMs). Yet conventional methods remain hardware-agnostic, limited to bit-width constraints, and do not account for intrinsic circuit characteristics such as the timing behaviors and energy profiles of Multiply-Accumulate (MAC) units. This disconnect from circuit-level behavior limits the ability to exploit available timing margins and energy-saving opportunities, reducing the overall efficiency of deployment on modern accelerators. To address these limitations, we propose HALO, a versatile framework for Hardware-Aware Post-Training Quantization (PTQ). Unlike traditional methods, HALO explicitly incorporates detailed hardware characteristics, including critical-path timing and power consumption, into its quantization approach. HALO strategically selects weights with low critical-path-delays enabling higher operational frequencies and dynamic frequency scaling without disrupting the architecture's dataflow. Remarkably, HALO achieves these improvements with only a few dynamic voltage and frequency scaling (DVFS) adjustments, ensuring simplicity and practicality in deployment. Additionally, by reducing switching activity within the MAC units, HALO effectively lowers energy consumption. Evaluations on accelerators such as Tensor Processing Units (TPUs) and Graphics Processing Units (GPUs) demonstrate that HALO significantly enhances inference efficiency, achieving average performance improvements of 270% and energy savings of 51% over baseline quantization methods, all with minimal impact on accuracy.
Zitieren
APA
Juneja, R., Aggarwal, S., Huda, S., Mitra, T., & Peh, L. (2026). HALO: Hardware-Aware Quantization with Low Critical-Path-Delay Weights for LLM Acceleration. Proceedings of the AAAI Conference on Artificial Intelligence, 40(27), 22472-22481. https://doi.org/10.1609/aaai.v40i27.39406
Vancouver
Juneja R, Aggarwal S, Huda S, Mitra T, Peh L. HALO: Hardware-Aware Quantization with Low Critical-Path-Delay Weights for LLM Acceleration. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(27):22472-22481. doi:10.1609/aaai.v40i27.39406
BibTeX
@article{Juneja2026,
author = {Juneja, Rohan and Aggarwal, Shivam and Huda, Safeen and Mitra, Tulika and Peh, Li-Shiuan},
title = {HALO: Hardware-Aware Quantization with Low Critical-Path-Delay Weights for LLM Acceleration},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {27},
pages = {22472--22481},
doi = {10.1609/aaai.v40i27.39406}
}
EUVIMED