EUVIMEDEUVIMED entdecken
Crossref

Breaking the Edge: Enabling Efficient Neural Network Inference on Integrated Edge Devices

Feng Zhang, Chenyang Zhang, Jiawei Guan, Qiangjun Zhou, Kuangyu Chen, Xiao Zhang, Bingsheng He, Jidong Zhai, Xiaoyong Du

IEEE Transactions on Cloud Computing · 2025-04 · 13 (2) · 694-710

DOI: 10.1109/tcc.2025.3559346 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Cloud Computing, 2025-04. Verfasst von Feng Zhang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 98-mal zitiert (Crossref) und nennt 100 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China, Beijing Nova Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Edge computing has gained widespread attention in cloud computing due to the increasing demands of AIoT applications and the evolution of edge architectures. One prevalent application in this domain is neural network inference on edge for computing and processing. This article presents an in-depth exploration of inference on integrated edge devices and introduces EdgeNN, a groundbreaking solution for inference specifically designed for CPU-GPU integrated edge devices. EdgeNN offers three key innovations. First, EdgeNN adaptively employs zero-copy optimization by harnessing unified physical memory. Second, EdgeNN introduces an innovative approach to CPU-GPU hybrid execution tailored for inference tasks. This technique enables concurrent CPU and GPU operation, effectively leveraging edge platforms’ computational capabilities. Third, EdgeNN adopts a finely tuned adaptive inference tuning technique that analyzes complex inference structures. It divides computations into sub-tasks, intelligently assigning them to the two processors for better performance. Experimental results demonstrate EdgeNN's superiority across six popular neural network inference processing. EdgeNN delivers average speed improvements of 3.97×, 4.10×, 3.12×, and 8.80× when compared to inference on four distinct edge CPUs. Furthermore, EdgeNN achieves significant time advantages compared to the direct execution of original programs. This improvement is attributed to better unified memory utilization (44.37%) and the innovative CPU-GPU hybrid execution approach (17.91%). Additionally, EdgeNN exhibits superior energy efficiency, providing 29.14× higher energy efficiency than edge CPUs and 5.70× higher energy efficiency than discrete GPUs. EdgeNN is now open source at https://github.com/ChenyangZhang-cs/EdgeNN.

Förderung

  • National Key R&D Program of China
  • National Natural Science Foundation of China
  • Beijing Nova Program
  • State Key Laboratory of Computer Architecture

Zitieren

APA

Zhang, F., Zhang, C., Guan, J., Zhou, Q., Chen, K., Zhang, X., He, B., Zhai, J., & Du, X. (2025). Breaking the Edge: Enabling Efficient Neural Network Inference on Integrated Edge Devices. IEEE Transactions on Cloud Computing, 13(2), 694-710. https://doi.org/10.1109/tcc.2025.3559346

Vancouver

Zhang F, Zhang C, Guan J, Zhou Q, Chen K, Zhang X, et al. Breaking the Edge: Enabling Efficient Neural Network Inference on Integrated Edge Devices. IEEE Transactions on Cloud Computing. 2025;13(2):694-710. doi:10.1109/tcc.2025.3559346

BibTeX
@article{Zhang2025,
  author = {Zhang, Feng and Zhang, Chenyang and Guan, Jiawei and Zhou, Qiangjun and Chen, Kuangyu and Zhang, Xiao and He, Bingsheng and Zhai, Jidong and Du, Xiaoyong},
  title = {Breaking the Edge: Enabling Efficient Neural Network Inference on Integrated Edge Devices},
  journal = {IEEE Transactions on Cloud Computing},
  year = {2025},
  volume = {13},
  number = {2},
  pages = {694--710},
  doi = {10.1109/tcc.2025.3559346}
}
Kostenlos starten →