EUVIMEDEUVIMED entdecken
Crossref

HALO: Hardware-Aware Quantization with Low Critical-Path-Delay Weights for LLM Acceleration

Rohan Juneja, Shivam Aggarwal, Safeen Huda, Tulika Mitra, Li-Shiuan Peh

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (27) · 22472-22481

DOI: 10.1609/aaai.v40i27.39406 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Rohan Juneja und 4 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Quantization is critical for efficiently deploying large language models (LLMs). Yet conventional methods remain hardware-agnostic, limited to bit-width constraints, and do not account for intrinsic circuit characteristics such as the timing behaviors and energy profiles of Multiply-Accumulate (MAC) units. This disconnect from circuit-level behavior limits the ability to exploit available timing margins and energy-saving opportunities, reducing the overall efficiency of deployment on modern accelerators. To address these limitations, we propose HALO, a versatile framework for Hardware-Aware Post-Training Quantization (PTQ). Unlike traditional methods, HALO explicitly incorporates detailed hardware characteristics, including critical-path timing and power consumption, into its quantization approach. HALO strategically selects weights with low critical-path-delays enabling higher operational frequencies and dynamic frequency scaling without disrupting the architecture's dataflow. Remarkably, HALO achieves these improvements with only a few dynamic voltage and frequency scaling (DVFS) adjustments, ensuring simplicity and practicality in deployment. Additionally, by reducing switching activity within the MAC units, HALO effectively lowers energy consumption. Evaluations on accelerators such as Tensor Processing Units (TPUs) and Graphics Processing Units (GPUs) demonstrate that HALO significantly enhances inference efficiency, achieving average performance improvements of 270% and energy savings of 51% over baseline quantization methods, all with minimal impact on accuracy.

Zitieren

APA

Juneja, R., Aggarwal, S., Huda, S., Mitra, T., & Peh, L. (2026). HALO: Hardware-Aware Quantization with Low Critical-Path-Delay Weights for LLM Acceleration. Proceedings of the AAAI Conference on Artificial Intelligence, 40(27), 22472-22481. https://doi.org/10.1609/aaai.v40i27.39406

Vancouver

Juneja R, Aggarwal S, Huda S, Mitra T, Peh L. HALO: Hardware-Aware Quantization with Low Critical-Path-Delay Weights for LLM Acceleration. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(27):22472-22481. doi:10.1609/aaai.v40i27.39406

BibTeX
@article{Juneja2026,
  author = {Juneja, Rohan and Aggarwal, Shivam and Huda, Safeen and Mitra, Tulika and Peh, Li-Shiuan},
  title = {HALO: Hardware-Aware Quantization with Low Critical-Path-Delay Weights for LLM Acceleration},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {27},
  pages = {22472--22481},
  doi = {10.1609/aaai.v40i27.39406}
}
Kostenlos starten →