DaCapo: An On-Device Learning Scheme for Memory-Constrained Embedded Systems
ACM Transactions on Embedded Computing Systems · 31.10.2023 · 22 (5s) · 1-23
DOI: 10.1145/3609121 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Embedded Computing Systems, online erschienen am 09.09.2023. Verfasst von Osama Khan und 2 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 40 Literaturangaben. Gefördert von: Institute of Information and Communications Technology Planning and Evaluation, Development of Core Technology for Autonomous Energy-driven Computing System SW in Power-Instable Environment, National Research Foundation of Korea.
Abstract (Originalsprache)
The use of deep neural network (DNN) applications in microcontroller unit (MCU) embedded systems is getting popular. However, the DNN models in such systems frequently suffer from accuracy loss due to the dataset shift problem. On-device learning resolves this problem by updating the model parameters on-site with the real-world data, thus localizing the model to its surroundings. However, the backpropagation step during on-device learning requires the output of every layer computed during the forward pass to be stored in memory. This is usually infeasible in MCU devices as they are equipped only with a few KBs of SRAM. Given their energy limitation and the timeliness requirements, using flash memory to store the output of every layer is not practical either. Although there have been proposed a few research results to enable on-device learning under stringent memory conditions, they require the modification of the target models or the use of non-conventional gradient computation strategies. This paper proposes DaCapo, a backpropagation scheme that enables on-device learning in memory-constrained embedded systems. DaCapo stores only the output of certain layers, known as checkpoints, in SRAM, and discards the others. The discarded outputs are recomputed during backpropagation from the nearest checkpoint in front of them. In order to minimize the recomputation occurrences, DaCapo optimally plans the checkpoints to be stored in the SRAM area at a particular phase of the backpropagation and thus replaces the checkpoints stored in memory as the backpropagation progresses. We implemented the proposed scheme in an STM32F429ZI board and evaluated it with five representative DNN models. Our evaluation showed that DaCapo improved backpropagation time by up to 22% and saved energy consumption by up to 28% in comparison to AIfES, a machine learning platform optimized for MCU devices. In addition, our proposed approach enabled the training of MobileNet, which the MCU device had been previously unable to train.
Förderung
- Institute of Information and Communications Technology Planning and Evaluation
- Development of Core Technology for Autonomous Energy-driven Computing System SW in Power-Instable Environment
- National Research Foundation of Korea
Zitieren
APA
Khan, O., Park, G., & Seo, E. (2023). DaCapo: An On-Device Learning Scheme for Memory-Constrained Embedded Systems. ACM Transactions on Embedded Computing Systems, 22(5s), 1-23. https://doi.org/10.1145/3609121
Vancouver
Khan O, Park G, Seo E. DaCapo: An On-Device Learning Scheme for Memory-Constrained Embedded Systems. ACM Transactions on Embedded Computing Systems. 2023;22(5s):1-23. doi:10.1145/3609121
BibTeX
@article{Khan2023,
author = {Khan, Osama and Park, Gwanjong and Seo, Euiseong},
title = {DaCapo: An On-Device Learning Scheme for Memory-Constrained Embedded Systems},
journal = {ACM Transactions on Embedded Computing Systems},
year = {2023},
volume = {22},
number = {5s},
pages = {1--23},
doi = {10.1145/3609121}
}
EUVIMED