Breaking the Edge: Enabling Efficient Neural Network Inference on Integrated Edge Devices
IEEE Transactions on Cloud Computing · 2025-04 · 13 (2) · 694-710
DOI: 10.1109/tcc.2025.3559346 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Cloud Computing, 2025-04. Verfasst von Feng Zhang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 98-mal zitiert (Crossref) und nennt 100 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China, Beijing Nova Program.
Abstract (Originalsprache)
Edge computing has gained widespread attention in cloud computing due to the increasing demands of AIoT applications and the evolution of edge architectures. One prevalent application in this domain is neural network inference on edge for computing and processing. This article presents an in-depth exploration of inference on integrated edge devices and introduces EdgeNN, a groundbreaking solution for inference specifically designed for CPU-GPU integrated edge devices. EdgeNN offers three key innovations. First, EdgeNN adaptively employs zero-copy optimization by harnessing unified physical memory. Second, EdgeNN introduces an innovative approach to CPU-GPU hybrid execution tailored for inference tasks. This technique enables concurrent CPU and GPU operation, effectively leveraging edge platforms’ computational capabilities. Third, EdgeNN adopts a finely tuned adaptive inference tuning technique that analyzes complex inference structures. It divides computations into sub-tasks, intelligently assigning them to the two processors for better performance. Experimental results demonstrate EdgeNN's superiority across six popular neural network inference processing. EdgeNN delivers average speed improvements of 3.97×, 4.10×, 3.12×, and 8.80× when compared to inference on four distinct edge CPUs. Furthermore, EdgeNN achieves significant time advantages compared to the direct execution of original programs. This improvement is attributed to better unified memory utilization (44.37%) and the innovative CPU-GPU hybrid execution approach (17.91%). Additionally, EdgeNN exhibits superior energy efficiency, providing 29.14× higher energy efficiency than edge CPUs and 5.70× higher energy efficiency than discrete GPUs. EdgeNN is now open source at https://github.com/ChenyangZhang-cs/EdgeNN.
Förderung
- National Key R&D Program of China
- National Natural Science Foundation of China
- Beijing Nova Program
- State Key Laboratory of Computer Architecture
Zitieren
APA
Zhang, F., Zhang, C., Guan, J., Zhou, Q., Chen, K., Zhang, X., He, B., Zhai, J., & Du, X. (2025). Breaking the Edge: Enabling Efficient Neural Network Inference on Integrated Edge Devices. IEEE Transactions on Cloud Computing, 13(2), 694-710. https://doi.org/10.1109/tcc.2025.3559346
Vancouver
Zhang F, Zhang C, Guan J, Zhou Q, Chen K, Zhang X, et al. Breaking the Edge: Enabling Efficient Neural Network Inference on Integrated Edge Devices. IEEE Transactions on Cloud Computing. 2025;13(2):694-710. doi:10.1109/tcc.2025.3559346
BibTeX
@article{Zhang2025,
author = {Zhang, Feng and Zhang, Chenyang and Guan, Jiawei and Zhou, Qiangjun and Chen, Kuangyu and Zhang, Xiao and He, Bingsheng and Zhai, Jidong and Du, Xiaoyong},
title = {Breaking the Edge: Enabling Efficient Neural Network Inference on Integrated Edge Devices},
journal = {IEEE Transactions on Cloud Computing},
year = {2025},
volume = {13},
number = {2},
pages = {694--710},
doi = {10.1109/tcc.2025.3559346}
}
EUVIMED