A Neuromorphic Transformer Architecture Enabling Hardware-Friendly Edge Computing
IEEE Transactions on Circuits and Systems I: Regular Papers · 2025-06 · 72 (6) · 2676-2689
DOI: 10.1109/tcsi.2025.3557955 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems I: Regular Papers, 2025-06. Verfasst von P. J. Zhou und 9 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 57 Literaturangaben. Gefördert von: Scientific and Technological Innovation 2030-Major Projects, Sichuan Provincial Science and Technology Program.
Abstract (Originalsprache)
The transformer model has demonstrated significant capabilities in various intelligent tasks, attracting widespread attention in recent years. However, it involves numerous complex operations, including large-bit-width multiplication, division, matrix transposition, and exponentiation. These require substantial storage and computational resources, making it challenging to deploy on edge devices. This work introduces a neuromorphic transformer architecture with low hardware cost for AI edge computing (AI-EC). At the structural level, it absorbs scaling factors within the self-attention mechanism into weight matrixes, thereby eliminating the division caused by the scaling operation. Additionally, a transposition calculation method is proposed to perform matrix transposition using dedicated memory access strategies and optimized data flow designs, which reduces logic resource overhead and avoids memory access discontinuities. At the computing paradigm level, the architecture employs spike-driven computing, substituting multi-bit multipliers with AND logic for synaptic operations. The paradigm introduces high sparsity to computational data, which is effectively exploited to reduce the computational workload of the architecture. The results indicate that the architecture successfully eliminates high-cost operators and significantly reduces computational expenses. Eventually, this architecture is verified as a prototype using a 28 nm CMOS process library, demonstrating a compact logic area of sub-0.2 mm2 and a high energy efficiency of 0.34 pJ/SOP @ 50MHz. This work is expected to promote the application of transformers in edge computing and the development of intelligent edge applications.
Förderung
- Scientific and Technological Innovation 2030-Major Projects
- Sichuan Provincial Science and Technology Program
Zitieren
APA
Zhou, P. J., Ma, R. C., Chen, Y. C., Liu, Z. T., Liu, C. Y., Meng, L. W., Qiao, G. C., Liu, Y., Yu, Q., & Hu, S. G. (2025). A Neuromorphic Transformer Architecture Enabling Hardware-Friendly Edge Computing. IEEE Transactions on Circuits and Systems I: Regular Papers, 72(6), 2676-2689. https://doi.org/10.1109/tcsi.2025.3557955
Vancouver
Zhou PJ, Ma RC, Chen YC, Liu ZT, Liu CY, Meng LW, et al. A Neuromorphic Transformer Architecture Enabling Hardware-Friendly Edge Computing. IEEE Transactions on Circuits and Systems I: Regular Papers. 2025;72(6):2676-2689. doi:10.1109/tcsi.2025.3557955
BibTeX
@article{Zhou2025,
author = {Zhou, P. J. and Ma, R. C. and Chen, Y. C. and Liu, Z. T. and Liu, C. Y. and Meng, L. W. and Qiao, G. C. and Liu, Y. and Yu, Q. and Hu, S. G.},
title = {A Neuromorphic Transformer Architecture Enabling Hardware-Friendly Edge Computing},
journal = {IEEE Transactions on Circuits and Systems I: Regular Papers},
year = {2025},
volume = {72},
number = {6},
pages = {2676--2689},
doi = {10.1109/tcsi.2025.3557955}
}
EUVIMED