Optimus: An Operator Fusion Framework for Deep Neural Networks
ACM Transactions on Embedded Computing Systems · 31.01.2023 · 22 (1) · 1-26
DOI: 10.1145/3520142 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Embedded Computing Systems, online erschienen am 29.10.2022. Verfasst von Xuyi Cai und 2 weiteren Autor:innen. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Strategic Priority Research Program of Chinese Academy of Science.
Abstract (Originalsprache)
The reduction of neural parameters and operations for the applications on embedded and IoT platforms in current deep neural network (DNN) architectures has received increasing attention. Relatively, the intermediate feature maps of such lightweight neural networks begin to grow and usually outsize the on-chip memory as the new bottleneck, which introduces considerable power-consuming off-chip memory accesses. To reduce the feature-induced memory accesses, operator fusion has been proposed to parallelize the execution of multiple convolutional layers and shown significant reduction of off-chip memory accesses. However, how to fuse the neural operators is still a challenging issue that heavily depends on both the neural network (NN) topology and the specific DNN accelerator configuration. In this work, we observed prior operator fusion approaches fail to guarantee memory-level optimality as they search in the constrained operator fusion design space. Considering the complexity of the NN topologies and the constrained resources of the DNN accelerators, we develop a novel operator fusion framework, Optimus. Optimus includes an accurate memory cost model dedicated to the scheduler to evaluate the potential operator-fusion schemes and a directed acyclic graph-based operator fusion algorithm for both off-line and on-line workload deployment scenarios, which altogether generates high-efficiency operator-fusion solutions for arbitrary network models running on DNN accelerators. The experimental results show that Optimus reduces 17–75% off-chip memory accesses and obtains 1.86×–3.66× energy efficiency on state-of-the-art DNN workloads when compared to the baselines and brings significant power-efficiency boost to the DNN accelerators of different architectures and dataflows.
Förderung
- National Natural Science Foundation of China
- Strategic Priority Research Program of Chinese Academy of Science
Zitieren
APA
Cai, X., Wang, Y., & Zhang, L. (2023). Optimus: An Operator Fusion Framework for Deep Neural Networks. ACM Transactions on Embedded Computing Systems, 22(1), 1-26. https://doi.org/10.1145/3520142
Vancouver
Cai X, Wang Y, Zhang L. Optimus: An Operator Fusion Framework for Deep Neural Networks. ACM Transactions on Embedded Computing Systems. 2023;22(1):1-26. doi:10.1145/3520142
BibTeX
@article{Cai2023,
author = {Cai, Xuyi and Wang, Ying and Zhang, Lei},
title = {Optimus: An Operator Fusion Framework for Deep Neural Networks},
journal = {ACM Transactions on Embedded Computing Systems},
year = {2023},
volume = {22},
number = {1},
pages = {1--26},
doi = {10.1145/3520142}
}
EUVIMED