EUVIMEDEUVIMED entdecken
Crossref

MCoreOPU: An FPGA-based Multi-Core Overlay Processor for Transformer-based Models

Shaoqiang Lu, Tiandong Zhao, Ting-Jung Lin, Rumin Zhang, Chen Wu, Lei He

ACM Transactions on Reconfigurable Technology and Systems · 30.09.2025 · 18 (3) · 1-27

DOI: 10.1145/3742437 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Reconfigurable Technology and Systems, online erschienen am 18.08.2025. Verfasst von Shaoqiang Lu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 49 Literaturangaben. Gefördert von: IDT Young Doctoral Innovation Program, Ningbo NSF.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transformer-based models have achieved extensive success with increasingly large numbers of parameters and computations, for which many multi-core accelerators have been developed. Nevertheless, they suffer from limited throughput due to either low operating frequency or high communication overhead between cores. This article proposes an FPGA-based multi-core overlay processor, named MCoreOPU, to optimize intra-core computation and inter-core communication. First, we boost the operating frequency of the processing element (PE) array to double the rest of the processor to improve the intra-core throughput. Second, we develop on-chip synchronization routers to reduce off-chip memory traffic, where only the partial sum and maximum are communicated between cores rather than entire vectors for layer normalization and softmax. Moreover, we pipeline synchronization to reduce synchronization latency and develop a bypass of the interconnect bus to reduce the off-chip memory access latency. Finally, we optimize the multi-core model allocation and scheduling to minimize the inter-core communications and maximize the intra-core computation efficiency. The MCoreOPU is implemented in 8-bit fixed-point precision with four cores and four DDRs on the Xilinx U200 FPGA, where the PE array runs at 600 MHz while the rest runs at 300 MHz. Experimental results show that the throughput per MAC of MCoreOPU for BERT, ViT, GPT-2, and LLaMA inference is 1.31 \(\times\) –7.18 \(\times\) higher than other FPGA-based accelerators. Compared with the A100 GPU, the throughput per equivalent MAC efficiency is improved by 22.52 \(\times\) –27.12 \(\times\) .

Förderung

  • IDT Young Doctoral Innovation Program
  • Ningbo NSF

Zitieren

APA

Lu, S., Zhao, T., Lin, T., Zhang, R., Wu, C., & He, L. (2025). MCoreOPU: An FPGA-based Multi-Core Overlay Processor for Transformer-based Models. ACM Transactions on Reconfigurable Technology and Systems, 18(3), 1-27. https://doi.org/10.1145/3742437

Vancouver

Lu S, Zhao T, Lin T, Zhang R, Wu C, He L. MCoreOPU: An FPGA-based Multi-Core Overlay Processor for Transformer-based Models. ACM Transactions on Reconfigurable Technology and Systems. 2025;18(3):1-27. doi:10.1145/3742437

BibTeX
@article{Lu2025,
  author = {Lu, Shaoqiang and Zhao, Tiandong and Lin, Ting-Jung and Zhang, Rumin and Wu, Chen and He, Lei},
  title = {MCoreOPU: An FPGA-based Multi-Core Overlay Processor for Transformer-based Models},
  journal = {ACM Transactions on Reconfigurable Technology and Systems},
  year = {2025},
  volume = {18},
  number = {3},
  pages = {1--27},
  doi = {10.1145/3742437}
}
Kostenlos starten →