LLMSGHD: Co-Designing Large Language Model Software and Hardware for Efficient Inference
ACM Transactions on Architecture and Code Optimization · 30.09.2026 · 23 (3) · 1-25
DOI: 10.1145/3820045 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Architecture and Code Optimization, online erschienen am 06.08.2026. Verfasst von Qingshan Xue und 8 weiteren Autor:innen. Die Arbeit nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Over the past two years, the rapid evolution of Large Language Models (LLMs) and the relatively slower progress in hardware development have led to the emergence of many operator-level optimization techniques and theories. These software-based methods accelerate model inference by improving memory management and computational efficiency, and they have demonstrated empirical effectiveness. However, in current computer architecture research, hardware designers often focus on microarchitectural optimizations or the performance of individual operations. They tend to adopt existing software optimizations passively, without actively leveraging them as design principles. Constrained by this limited perspective, researchers may overlook opportunities to systematically integrate insights from software into hardware design, potentially hindering more efficient and flexible architectural innovations. To combine operator-level optimization methods with hardware design, we introduce a hardware evaluation platform called LLMSGHD ( L arge L anguage M odel S oftware- G uided H ardware D esign), which focuses on operator-level optimized LLM inference workloads. LLMSGHD simulates hardware inference behavior while aiming for broad applicability and high efficiency. LLMSGHD integrates advanced software optimization techniques to offer more insightful analysis for hardware design, particularly regarding computational density variations in inference and their interaction with software-level optimizations. Based on LLMSGHD, we develope a heterogeneous LLM inference platform targeting high throughput and low cost.
Zitieren
APA
Xue, Q., Ma, S., Xu, R., Shi, Y., Wang, B., Luo, S., Zhang, X., Huang, L., & Li, T. (2026). LLMSGHD: Co-Designing Large Language Model Software and Hardware for Efficient Inference. ACM Transactions on Architecture and Code Optimization, 23(3), 1-25. https://doi.org/10.1145/3820045
Vancouver
Xue Q, Ma S, Xu R, Shi Y, Wang B, Luo S, et al. LLMSGHD: Co-Designing Large Language Model Software and Hardware for Efficient Inference. ACM Transactions on Architecture and Code Optimization. 2026;23(3):1-25. doi:10.1145/3820045
BibTeX
@article{Xue2026,
author = {Xue, Qingshan and Ma, Sheng and Xu, Rui and Shi, Yihao and Wang, Bo and Luo, Shengbai and Zhang, Xueyi and Huang, Libo and Li, Tiejun},
title = {LLMSGHD: Co-Designing Large Language Model Software and Hardware for Efficient Inference},
journal = {ACM Transactions on Architecture and Code Optimization},
year = {2026},
volume = {23},
number = {3},
pages = {1--25},
doi = {10.1145/3820045}
}
EUVIMED