Hardware-Friendly and Efficient Vision Transformer for Deployment on Low-Power Embedded Device
Journal of Low Power Electronics and Applications · 22.12.2025 · 16 (1) · 1
DOI: 10.3390/jlpea16010001 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Low Power Electronics and Applications, online erschienen am 22.12.2025. Verfasst von Ziyang Chen und 6 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Key R&D Program of Guangdong Province, Jiangsu Provincial Science and Technology Major Special Project.
Abstract (Originalsprache)
The Transformer architecture has achieved remarkable success across numerous computer vision tasks due to its superior capability for global dependency modeling. However, the high computational complexity and hardware-unfriendly operations such as Layer Normalization (LN), Softmax, and GELU severely hinder its deployment on resource-constrained platforms. To address these challenges, this paper proposes a hardware-friendly CNN-Transformer hybrid pyramid architecture that effectively balances accuracy, efficiency, and deployability. The proposed model integrates convolutional bottlenecks with Transformer encoders to capture both local and global contextual information while maintaining low computational cost. A pyramid feature extraction structure is further introduced to enhance multi-scale semantic representation. To improve hardware efficiency, we redesign key nonlinear components by introducing hardware-friendly activation, normalization, and Softmax approximations. Specifically, GELU and LN are replaced by ReLU and Batch Normalization (BN), and a simplified logarithmic-exponential formulation termed Softmax2 is proposed, which eliminates complex exponential and division operations, significantly reducing hardware implementation cost. Extensive experiments demonstrate the effectiveness of the proposed framework. The experimental results validate that the proposed architecture offers a promising and practical solution for real-time and embedded vision applications.
Förderung
- Key R&D Program of Guangdong Province
- Jiangsu Provincial Science and Technology Major Special Project
Zitieren
APA
Chen, Z., Hao, M., Cao, X., Zhang, J., Shen, C., Li, G., & Zhang, M. (2025). Hardware-Friendly and Efficient Vision Transformer for Deployment on Low-Power Embedded Device. Journal of Low Power Electronics and Applications, 16(1), 1. https://doi.org/10.3390/jlpea16010001
Vancouver
Chen Z, Hao M, Cao X, Zhang J, Shen C, Li G, et al. Hardware-Friendly and Efficient Vision Transformer for Deployment on Low-Power Embedded Device. Journal of Low Power Electronics and Applications. 2025;16(1):1. doi:10.3390/jlpea16010001
BibTeX
@article{Chen2025,
author = {Chen, Ziyang and Hao, Ming and Cao, Xinye and Zhang, Jingwei and Shen, Chaoyao and Li, Guoqing and Zhang, Meng},
title = {Hardware-Friendly and Efficient Vision Transformer for Deployment on Low-Power Embedded Device},
journal = {Journal of Low Power Electronics and Applications},
year = {2025},
volume = {16},
number = {1},
pages = {1},
doi = {10.3390/jlpea16010001}
}
EUVIMED