EUVIMEDEUVIMED entdecken
Crossref

Understanding the Potential of FPGA-based Spatial Acceleration for Large Language Model Inference

Hongzheng Chen, Jiahao Zhang, Yixiao Du, Shaojie Xiang, Zichao Yue, Niansong Zhang, Yaohui Cai, Zhiru Zhang

ACM Transactions on Reconfigurable Technology and Systems · 31.03.2025 · 18 (1) · 1-29

DOI: 10.1145/3656177 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Reconfigurable Technology and Systems, online erschienen am 17.12.2024. Verfasst von Hongzheng Chen und 7 weiteren Autor:innen. Die Arbeit wurde bisher 83-mal zitiert (Crossref) und nennt 105 Literaturangaben. Gefördert von: ACE, Semiconductor Research Corporation, DARPA and NSF.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advancements in large language models (LLMs) boasting billions of parameters have generated a significant demand for efficient deployment in inference workloads. While hardware accelerators for Transformer-based models have been extensively studied, the majority of existing approaches rely on temporal architectures that reuse hardware units for different network layers and operators. However, these methods often encounter challenges in achieving low latency due to considerable memory access overhead. This article investigates the feasibility and potential of model-specific spatial acceleration for LLM inference on field-programmable gate arrays (FPGAs). Our approach involves the specialization of distinct hardware units for specific operators or layers, facilitating direct communication between them through a dataflow architecture while minimizing off-chip memory accesses. We introduce a comprehensive analytical model for estimating the performance of a spatial LLM accelerator, taking into account the on-chip compute and memory resources available on an FPGA. This model can be extended to multi-FPGA settings for distributed inference. Through our analysis, we can identify the most effective parallelization and buffering schemes for the accelerator and, crucially, determine the scenarios in which FPGA-based spatial acceleration can outperform its GPU-based counterpart. To enable more productive implementations of an LLM model on FPGAs, we further provide a library of high-level synthesis (HLS) kernels that are composable and reusable. This library will be made available as open-source. To validate the effectiveness of both our analytical model and HLS library, we have implemented Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformers (GPT2) on an AMD Xilinx Alveo U280 FPGA device. Experimental results demonstrate our approach can achieve up to 13.4× speedup when compared to previous FPGA-based accelerators for the BERT model. For GPT generative inference, we attain a 2.2× speedup compared to Design for Excellence, an FPGA overlay, in the prefill stage, while achieving a 1.9× speedup and a 5.7× improvement in energy efficiency compared to the NVIDIA A100 GPU in the decode stage.

Förderung

  • ACE
  • Semiconductor Research Corporation
  • DARPA and NSF

Zitieren

APA

Chen, H., Zhang, J., Du, Y., Xiang, S., Yue, Z., Zhang, N., Cai, Y., & Zhang, Z. (2025). Understanding the Potential of FPGA-based Spatial Acceleration for Large Language Model Inference. ACM Transactions on Reconfigurable Technology and Systems, 18(1), 1-29. https://doi.org/10.1145/3656177

Vancouver

Chen H, Zhang J, Du Y, Xiang S, Yue Z, Zhang N, et al. Understanding the Potential of FPGA-based Spatial Acceleration for Large Language Model Inference. ACM Transactions on Reconfigurable Technology and Systems. 2025;18(1):1-29. doi:10.1145/3656177

BibTeX
@article{Chen2025,
  author = {Chen, Hongzheng and Zhang, Jiahao and Du, Yixiao and Xiang, Shaojie and Yue, Zichao and Zhang, Niansong and Cai, Yaohui and Zhang, Zhiru},
  title = {Understanding the Potential of FPGA-based Spatial Acceleration for Large Language Model Inference},
  journal = {ACM Transactions on Reconfigurable Technology and Systems},
  year = {2025},
  volume = {18},
  number = {1},
  pages = {1--29},
  doi = {10.1145/3656177}
}
Kostenlos starten →