TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture
ACM Transactions on Reconfigurable Technology and Systems · 31.03.2025 · 18 (1) · 1-31
DOI: 10.1145/3714416 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Reconfigurable Technology and Systems, online erschienen am 14.03.2025. Verfasst von Jiajun Wu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Grants Council (RGC) of Hong Kong under the Research Impact Fund, Theme-based Research Scheme (TRS) Project, AI Chip Center for Emerging Smart Systems.
Abstract (Originalsprache)
Modern transformer-based deep neural networks present unique technical challenges for effective acceleration in real-world applications. Apart from the vast amount of linear operations needed due to their sizes, modern transformer models are increasingly reliance on precise non-linear computations that make traditional low-bitwidth quantization methods and fixed-dataflow matrix accelerators ineffective for end-to-end acceleration. To address this need to accelerate both linear and non-linear operations in a unified and programmable framework, this article introduces TATAA. TATAA employs 8-bit integer ( int8 ) arithmetic for quantized linear layer operations through post-training quantization, while it relies on bfloat16 floating-point arithmetic to approximate non-linear layers of a transformer model. TATAA hardware features a transformable arithmetic architecture that supports both formats during runtime with minimal overhead, enabling it to switch between a systolic array mode for int8 matrix multiplications and a SIMD mode for vectorized bfloat16 operations. An end-to-end compiler is presented to enable flexible mapping from emerging transformer models to the proposed hardware. Experimental results indicate that our mixed-precision design incurs only 0.14% to 1.16% accuracy drop when compared with the pre-trained single-precision transformer models across a range of vision, language, and generative text applications. Our prototype implementation on the Alveo U280 FPGA currently achieves 2,935.2 GOPS throughput on linear layers and a maximum of 189.5 GFLOPS for non-linear operations, outperforming related works by up to \(1.45\times\) in end-to-end throughput and \(2.29\times\) in DSP efficiency, while achieving \(2.19\times\) higher power efficiency than modern NVIDIA RTX4090 GPU.
Förderung
- Research Grants Council (RGC) of Hong Kong under the Research Impact Fund
- Theme-based Research Scheme (TRS) Project
- AI Chip Center for Emerging Smart Systems
Zitieren
APA
Wu, J., Song, M., Zhao, J., Gao, Y., Li, J., & So, H. K. (2025). TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture. ACM Transactions on Reconfigurable Technology and Systems, 18(1), 1-31. https://doi.org/10.1145/3714416
Vancouver
Wu J, Song M, Zhao J, Gao Y, Li J, So HK. TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture. ACM Transactions on Reconfigurable Technology and Systems. 2025;18(1):1-31. doi:10.1145/3714416
BibTeX
@article{Wu2025,
author = {Wu, Jiajun and Song, Mo and Zhao, Jingmin and Gao, Yizhao and Li, Jia and So, Hayden Kwok-Hay},
title = {TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture},
journal = {ACM Transactions on Reconfigurable Technology and Systems},
year = {2025},
volume = {18},
number = {1},
pages = {1--31},
doi = {10.1145/3714416}
}
EUVIMED