A Transistor Operations Model for Deep Learning Energy Consumption Scaling Law
IEEE Transactions on Artificial Intelligence · 2024-01 · 5 (1) · 192-204
DOI: 10.1109/tai.2022.3229280 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Artificial Intelligence, 2024-01. Verfasst von Chen Li und 2 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 61 Literaturangaben. Gefördert von: EC H2020 DAWN4IoE - Data Aware Wireless Network for Internet-of-Everything.
Abstract (Originalsprache)
Deep neural networks (DNNs) have transformed the automation of a wide range of industries and found increasing ubiquity in society. The high complexity of DNN models and its widespread adoption has led to global energy consumption doubling every 3–4 months. Current energy consumption measures largely monitor system wide consumption or make linear assumptions of DNN models. The former approach captures other unrelated energy consumption anomalies, whilst the latter does not accurately reflect nonlinear computations. In this article, we are the first to develop a bottom-up transistor operations (TOs) approach to expose the role of nonlinear activation functions and neural network structure. As there will be inevitable energy measurement errors at the core level, we statistically model the energy scaling laws as opposed to absolute consumption values. We offer models for both feedforward DNNs and convolution neural networks on a variety of datasets and hardware configurations—achieving a 93.6%–99.5% precision. This outperforms existing floating-point operations (FLOPs)-based methods and our TOs method can be further extended to other DNN models.
Förderung
- EC H2020 DAWN4IoE - Data Aware Wireless Network for Internet-of-Everything
Zitieren
APA
Li, C., Tsourdos, A., & Guo, W. (2024). A Transistor Operations Model for Deep Learning Energy Consumption Scaling Law. IEEE Transactions on Artificial Intelligence, 5(1), 192-204. https://doi.org/10.1109/tai.2022.3229280
Vancouver
Li C, Tsourdos A, Guo W. A Transistor Operations Model for Deep Learning Energy Consumption Scaling Law. IEEE Transactions on Artificial Intelligence. 2024;5(1):192-204. doi:10.1109/tai.2022.3229280
BibTeX
@article{Li2024,
author = {Li, Chen and Tsourdos, Antonios and Guo, Weisi},
title = {A Transistor Operations Model for Deep Learning Energy Consumption Scaling Law},
journal = {IEEE Transactions on Artificial Intelligence},
year = {2024},
volume = {5},
number = {1},
pages = {192--204},
doi = {10.1109/tai.2022.3229280}
}
EUVIMED