Hardware-Friendly Fully Quantized Mamba-2 Model and Its FPGA-Based Accelerator
IEEE Access · 2026 · 14 · 78920-78930
DOI: 10.1109/access.2026.3696226 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Hiroshi Fuketa und 1 weiteren Autor:innen. Die Arbeit nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: JSPS KAKENHI.
Abstract (Originalsprache)
In this paper, we propose a fully quantized Mamba-2 model to enable efficient large language model (LLM) processing for edge AI. As a promising next-generation architecture, Mamba offers a scalable alternative to Transformer-based models with better suitability for edge AI. In the proposed model, the weights and activations are quantized to ternary and 8-bit integer (INT8) precision, respectively, eliminating the need for floating-point operations. Additionally, the non-linear functions included in the original Mamba model, such as SiLU, softplus, exponential, and normalization functions, are replaced with hardware-friendly alternatives, including ReLU, power-of-two functions, and table-lookup-based normalization. These substitutions substantially reduce computational and hardware complexity while largely preserving model accuracy through the proposed two-stage quantization-aware training. Furthermore, we propose an FPGA-based accelerator tailored to the proposed fully quantized Mamba-2 model. The proposed accelerator is implemented on an Ultra96-V2 board and consists primarily of a LUT-based vector-matrix multiplication unit and a DSP-based vector-vector multiplication unit. The former computes the products of INT8-precision vectors and ternary-precision matrices, whereas the latter computes the products of INT8-precision vectors. Implementation and measurement results show that the proposed accelerator achieves a 17% reduction in the FPGA resource cost and a 3.6-fold improvement in energy efficiency compared with conventional FPGA-based LLM accelerators, owing to the hardware-friendly design.
Förderung
- JSPS KAKENHI
Zitieren
APA
Fuketa, H., & Hioki, M. (2026). Hardware-Friendly Fully Quantized Mamba-2 Model and Its FPGA-Based Accelerator. IEEE Access, 14, 78920-78930. https://doi.org/10.1109/access.2026.3696226
Vancouver
Fuketa H, Hioki M. Hardware-Friendly Fully Quantized Mamba-2 Model and Its FPGA-Based Accelerator. IEEE Access. 2026;14:78920-78930. doi:10.1109/access.2026.3696226
BibTeX
@article{Fuketa2026,
author = {Fuketa, Hiroshi and Hioki, Masakazu},
title = {Hardware-Friendly Fully Quantized Mamba-2 Model and Its FPGA-Based Accelerator},
journal = {IEEE Access},
year = {2026},
volume = {14},
pages = {78920--78930},
doi = {10.1109/access.2026.3696226}
}
EUVIMED