Hardware–Software Co-Design for Real-Time Latency–Accuracy Navigation in Tiny Machine Learning Applications
IEEE Micro · 2023-11 · 43 (6) · 93-101
DOI: 10.1109/mm.2023.3317243 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Micro, 2023-11. Verfasst von Payman Behnam und 8 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 6 Literaturangaben. Gefördert von: National Science Foundation.
Abstract (Originalsprache)
Tiny machine learning (TinyML) applications increasingly operate in dynamically changing deployment scenarios, requiring optimization for both accuracy and latency. Existing methods mainly target a single point in the accuracy/latency tradeoff space, which is insufficient as no single static point can be optimal under variable conditions. We draw on a recently proposed weight-shared SuperNet mechanism to enable serving a stream of queries that activates different SubNets within a SuperNet. This creates an opportunity to exploit the inherent temporal locality of different queries that use the same SuperNet. We propose a hardware–software co-design called SUSHI that introduces a novel SubGraph Stationary optimization. SUSHI consists of a novel field-programmable gate array implementation and a software scheduler that controls which SubNets to serve and which SubGraph to cache in real time. SUSHI yields up to a 32% improvement in latency, 0.98% increase in served accuracy, and achieves up to 78.7% off-chip energy saved across several neural network architectures.
Förderung
- National Science Foundation
Zitieren
APA
Behnam, P., Tong, J., Khare, A., Chen, Y., Pan, Y., Gadikar, P., Bambhaniya, A., Krishna, T., & Tumanov, A. (2023). Hardware–Software Co-Design for Real-Time Latency–Accuracy Navigation in Tiny Machine Learning Applications. IEEE Micro, 43(6), 93-101. https://doi.org/10.1109/mm.2023.3317243
Vancouver
Behnam P, Tong J, Khare A, Chen Y, Pan Y, Gadikar P, et al. Hardware–Software Co-Design for Real-Time Latency–Accuracy Navigation in Tiny Machine Learning Applications. IEEE Micro. 2023;43(6):93-101. doi:10.1109/mm.2023.3317243
BibTeX
@article{Behnam2023,
author = {Behnam, Payman and Tong, Jianming and Khare, Alind and Chen, Yangyu and Pan, Yue and Gadikar, Pranav and Bambhaniya, Abhimanyu and Krishna, Tushar and Tumanov, Alexey},
title = {Hardware–Software Co-Design for Real-Time Latency–Accuracy Navigation in Tiny Machine Learning Applications},
journal = {IEEE Micro},
year = {2023},
volume = {43},
number = {6},
pages = {93--101},
doi = {10.1109/mm.2023.3317243}
}
EUVIMED