EUVIMEDEUVIMED entdecken
Crossref

EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Models

Mingqiang Huang, Ao Shen, Kai Li, Haoxiang Peng, Boyu Li, Yupeng Su, Hao Yu

IEEE Transactions on Circuits and Systems I: Regular Papers · 2025-07 · 72 (7) · 3352-3365

DOI: 10.1109/tcsi.2025.3546256 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems I: Regular Papers, 2025-07. Verfasst von Mingqiang Huang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 73-mal zitiert (Crossref) und nennt 37 Literaturangaben. Gefördert von: Scientific and Technological Innovation 2030-Major Projects, National Natural Science Foundation of China, Natural Science Foundation of Guangdong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rapid advancements in artificial intelligence (AI), particularly the Large Language Models (LLMs), have profoundly affected our daily work and communication forms. However, it is still a challenge to deploy LLMs on resource-constrained edge devices (such as robots), due to the intensive computation requirements, heavy memory access, diverse operator types and difficulties in compilation. In this work, we proposed EdgeLLM to address the above issues. Firstly, focusing on the computation, we designed mix-precision processing element array together with group systolic architecture, that can efficiently support both FP $16\ast $ FP16 for the MHA block (Multi-Head Attention) and FP $16\ast $ INT4 for the FFN layer (Feed-Forward Network). Meanwhile specific optimization on log-scale structured weight sparsity, has been used to further increase the efficiency. Secondly, to address the compilation and deployment issue, we analyzed the whole operators within LLM models and developed a universal data parallelism scheme, by which all of the input and output features maintain the same data shape, enabling to process different operators without any data rearrangement. Then we proposed an end-to-end compiler to map the whole LLM model on CPU-FPGA heterogeneous system (AMD Xilinx VCU128 FPGA). The accelerator achieves $1.91\times $ higher throughput and $7.55\times $ higher energy efficiency than the commercial GPU (NVIDIA A100-SXM4-80G). When compared with state-of-the-art FPGA accelerator of FlightLLM, it shows 10-24% better performance in terms of HBM bandwidth utilization, energy efficiency and LLM throughput.

Förderung

  • Scientific and Technological Innovation 2030-Major Projects
  • National Natural Science Foundation of China
  • Natural Science Foundation of Guangdong Province
  • Shenzhen Science and Technology Program
  • Science and Technology Innovation Key Research and Development Program of Chongqing

Zitieren

APA

Huang, M., Shen, A., Li, K., Peng, H., Li, B., Su, Y., & Yu, H. (2025). EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Models. IEEE Transactions on Circuits and Systems I: Regular Papers, 72(7), 3352-3365. https://doi.org/10.1109/tcsi.2025.3546256

Vancouver

Huang M, Shen A, Li K, Peng H, Li B, Su Y, et al. EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Models. IEEE Transactions on Circuits and Systems I: Regular Papers. 2025;72(7):3352-3365. doi:10.1109/tcsi.2025.3546256

BibTeX
@article{Huang2025,
  author = {Huang, Mingqiang and Shen, Ao and Li, Kai and Peng, Haoxiang and Li, Boyu and Su, Yupeng and Yu, Hao},
  title = {EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Models},
  journal = {IEEE Transactions on Circuits and Systems I: Regular Papers},
  year = {2025},
  volume = {72},
  number = {7},
  pages = {3352--3365},
  doi = {10.1109/tcsi.2025.3546256}
}
Kostenlos starten →