Keep in Balance: Runtime-reconfigurable Intermittent Deep Inference
ACM Transactions on Embedded Computing Systems · 31.10.2023 · 22 (5s) · 1-25
DOI: 10.1145/3607918 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Embedded Computing Systems, online erschienen am 09.09.2023. Verfasst von Chih-Hsuan Yen und 3 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 54 Literaturangaben. Gefördert von: Ministry of Science and Technology, Taiwan.
Abstract (Originalsprache)
Intermittent deep neural network (DNN) inference is a promising technique to enable intelligent applications on tiny devices powered by ambient energy sources. Nonetheless, intermittent execution presents inherent challenges, primarily involving accumulating progress across power cycles and having to refetch volatile data lost due to power loss in each power cycle. Existing approaches typically optimize the inference configuration to maximize data reuse. However, we observe that such a fixed configuration may be significantly inefficient due to the fluctuating balance point between data reuse and data refetch caused by the dynamic nature of ambient energy. This work proposes DynBal , an approach to dynamically reconfigure the inference engine at runtime. DynBal is realized as a middleware plugin that improves inference performance by exploring the interplay between data reuse and data refetch to maintain their balance with respect to the changing level of intermittency. An indirect metric is developed to easily evaluate an inference configuration considering the variability in intermittency, and a lightweight reconfiguration algorithm is employed to efficiently optimize the configuration at runtime. We evaluate the improvement brought by integrating DynBal into a recent intermittent inference approach that uses a fixed configuration. Evaluations were conducted on a Texas Instruments device with various network models and under varied intermittent power strengths. Our experimental results demonstrate that DynBal can speed up intermittent inference by 3.26 times, achieving a greater improvement for a large network under high intermittency and a large gap between memory and computation performance.
Förderung
- Ministry of Science and Technology, Taiwan
Zitieren
APA
Yen, C., Mendis, H. R., Kuo, T., & Hsiu, P. (2023). Keep in Balance: Runtime-reconfigurable Intermittent Deep Inference. ACM Transactions on Embedded Computing Systems, 22(5s), 1-25. https://doi.org/10.1145/3607918
Vancouver
Yen C, Mendis HR, Kuo T, Hsiu P. Keep in Balance: Runtime-reconfigurable Intermittent Deep Inference. ACM Transactions on Embedded Computing Systems. 2023;22(5s):1-25. doi:10.1145/3607918
BibTeX
@article{Yen2023,
author = {Yen, Chih-Hsuan and Mendis, Hashan Roshantha and Kuo, Tei-Wei and Hsiu, Pi-Cheng},
title = {Keep in Balance: Runtime-reconfigurable Intermittent Deep Inference},
journal = {ACM Transactions on Embedded Computing Systems},
year = {2023},
volume = {22},
number = {5s},
pages = {1--25},
doi = {10.1145/3607918}
}
EUVIMED