EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MemoriaNova: Optimizing Memory-Aware Model Inference for Edge Computing

Renjun Zhang, Tianming Zhang, Zinuo Cai, Dongmei Li, Ruhui Ma, Buyya Rajkumar

ACM Transactions on Architecture and Code Optimization · 31.03.2025 · 22 (1) · 1-25

DOI: 10.1145/3701997 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Architecture and Code Optimization, online erschienen am 19.03.2025. Verfasst von Renjun Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, deploying deep learning models on edge devices has become pervasive, driven by the increasing demand for intelligent edge computing solutions across various industries. From industrial automation to intelligent surveillance and healthcare, edge devices are being leveraged for real-time analytics and decision-making. Existing methods face two challenges when deploying machine learning models on edge devices. The first challenge is handling the execution order of operators with a simple strategy, which can lead to a potential waste of memory resources when dealing with directed acyclic graph structure models. The second challenge is that they usually process operators of a model one by one to optimize the inference latency, which may lead to the optimization problem getting trapped in local optima. We present MemoriaNova, comprising BTSearch and GenEFlow, to solve these two problems. BTSearch is a graph state backtracking algorithm with efficient pruning and hashing strategies designed to minimize memory overhead during inference and enlarge latency optimization search space. GenEFlow, based on genetic algorithms (GA), integrates latency modeling, and memory constraints to optimize distributed inference latency. This innovative approach considers a comprehensive search space for model partitioning, ensuring robust and adaptable solutions. We implement BTSearch and GenEFlow and test them on 11 deep-learning models with different structures and scales. The results show that BTSearch can reach 12% memory optimization compared with the widely used random execution strategy. At the same time, GenEFlow reduces inference latency by 33.9% in distributed systems with four-edge devices.

Zitieren

APA

Zhang, R., Zhang, T., Cai, Z., Li, D., Ma, R., & Rajkumar, B. (2025). MemoriaNova: Optimizing Memory-Aware Model Inference for Edge Computing. ACM Transactions on Architecture and Code Optimization, 22(1), 1-25. https://doi.org/10.1145/3701997

Vancouver

Zhang R, Zhang T, Cai Z, Li D, Ma R, Rajkumar B. MemoriaNova: Optimizing Memory-Aware Model Inference for Edge Computing. ACM Transactions on Architecture and Code Optimization. 2025;22(1):1-25. doi:10.1145/3701997

BibTeX
@article{Zhang2025,
  author = {Zhang, Renjun and Zhang, Tianming and Cai, Zinuo and Li, Dongmei and Ma, Ruhui and Rajkumar, Buyya},
  title = {MemoriaNova: Optimizing Memory-Aware Model Inference for Edge Computing},
  journal = {ACM Transactions on Architecture and Code Optimization},
  year = {2025},
  volume = {22},
  number = {1},
  pages = {1--25},
  doi = {10.1145/3701997}
}
Kostenlos starten →