EUVIMEDEUVIMED entdecken
Crossref

HAPE: Hardware-Aware LLM Pruning For Efficient On-Device Inference Optimization

Wenqian Zhao, Lancheng Zou, Zixiao Wang, Xufeng Yao, Bei Yu

ACM Transactions on Design Automation of Electronic Systems · 31.07.2025 · 30 (4) · 1-18

DOI: 10.1145/3744244 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Design Automation of Electronic Systems, online erschienen am 09.07.2025. Verfasst von Wenqian Zhao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 40 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Over the past few years, large language models (LLMs) have demonstrated remarkable performance and versatility across a variety of complex tasks. However, their deployment has been challenged by their substantial model size and computational requirements. Pruning is a effective approach to make the model parameters sparse, thereby acquire inference acceleration. While not everyone requires training or fine-tuning large models, the diverse range of applications necessitates the deployment of LLMs on different devices. Model pruning and compression have emerged as areas of deep research interest to address these challenges. In consideration of versatility and practicality, we have designed a hardware-aware pruning process for general-purpose hardware/edge devices to enable efficient deployment and inference of LLMs. Instead of considering sparse ratio alone, we are motivated to design a pruning framework that incorporates genuine inference speed-up sensitivity from each pruning structure. Moreover, our framework breaks the layer-by-layer pruning setting and fuse several layers into one pruning stage to allow cross-layer optimization. Apart from that, we hold pragmatism by conducting compilation optimization during pruning. This step is critical because most sparsity patterns barely show distinct speed acceleration with corresponding dataflow and memory optimization. Our process operates within a post-training framework, obviating the need for additional training and thereby reducing resource requirements, while ensuring diverse inference speed and accuracy requirements on hardware.

Zitieren

APA

Zhao, W., Zou, L., Wang, Z., Yao, X., & Yu, B. (2025). HAPE: Hardware-Aware LLM Pruning For Efficient On-Device Inference Optimization. ACM Transactions on Design Automation of Electronic Systems, 30(4), 1-18. https://doi.org/10.1145/3744244

Vancouver

Zhao W, Zou L, Wang Z, Yao X, Yu B. HAPE: Hardware-Aware LLM Pruning For Efficient On-Device Inference Optimization. ACM Transactions on Design Automation of Electronic Systems. 2025;30(4):1-18. doi:10.1145/3744244

BibTeX
@article{Zhao2025,
  author = {Zhao, Wenqian and Zou, Lancheng and Wang, Zixiao and Yao, Xufeng and Yu, Bei},
  title = {HAPE: Hardware-Aware LLM Pruning For Efficient On-Device Inference Optimization},
  journal = {ACM Transactions on Design Automation of Electronic Systems},
  year = {2025},
  volume = {30},
  number = {4},
  pages = {1--18},
  doi = {10.1145/3744244}
}
Kostenlos starten →