Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI
Sensors · 11.05.2023 · 23 (10) · 4667
DOI: 10.3390/s23104667 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 11.05.2023. Verfasst von Mara Pistellato und 6 weiteren Autor:innen. Beteiligte Einrichtung: Dipartimento di Scienze Ambientali, Informatica e Statistica (DAIS), Università Ca'Foscari di Venezia, Via Torino 155, 30170 Venezia, Italy. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: SMACT Competence Center scpa, SMACT Competence Center scpa, project CoNNeCT.
Abstract (Originalsprache)
Over the past few years, several applications have been extensively exploiting the advantages of deep learning, in particular when using convolutional neural networks (CNNs). The intrinsic flexibility of such models makes them widely adopted in a variety of practical applications, from medical to industrial. In this latter scenario, however, using consumer Personal Computer (PC) hardware is not always suitable for the potential harsh conditions of the working environment and the strict timing that industrial applications typically have. Therefore, the design of custom FPGA (Field Programmable Gate Array) solutions for network inference is gaining massive attention from researchers and companies as well. In this paper, we propose a family of network architectures composed of three kinds of custom layers working with integer arithmetic with a customizable precision (down to just two bits). Such layers are designed to be effectively trained on classical GPUs (Graphics Processing Units) and then synthesized to FPGA hardware for real-time inference. The idea is to provide a trainable quantization layer, called Requantizer, acting both as a non-linear activation for neurons and a value rescaler to match the desired bit precision. This way, the training is not only quantization-aware, but also capable of estimating the optimal scaling coefficients to accommodate both the non-linear nature of the activations and the constraints imposed by the limited precision. In the experimental section, we test the performance of this kind of model while working both on classical PC hardware and a case-study implementation of a signal peak detection device running on a real FPGA. We employ TensorFlow Lite for training and comparison, and use Xilinx FPGAs and Vivado for synthesis and implementation. The results show an accuracy of the quantized networks close to the floating point version, without the need for representative data for calibration as in other approaches, and performance that is better than dedicated peak detection algorithms. The FPGA implementation is able to run in real time at a rate of four gigapixels per second with moderate hardware resources, while achieving a sustained efficiency of 0.5 TOPS/W (tera operations per second per watt), in line with custom integrated hardware accelerators.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Dipartimento di Scienze Ambientali, Informatica e Statistica (DAIS), Università Ca'Foscari di Venezia, Via Torino 155, 30170 Venezia, Italy.
- Dipartimento di Management, Università Ca'Foscari di Venezia, Cannaregio 873, 30121 Venezia, Italy.
- Covision Lab SCARL, Via Durst 4, 39042 Bressanone, Italy.
- Dipartimento di Ingegneria e Scienza dell'Informazione (DISI), University of Trento, Via Sommarive 9, 38123 Trento, Italy.
ORCID-Profile: Mara Pistellato, Filippo Bergamasco, Gianluca Bigaglia, Andrea Gasparetto, Andrea Albarelli, Roberto Passerone
Förderung
- SMACT Competence Center scpa
- SMACT Competence Center scpa, project CoNNeCT
Zitieren
APA
Pistellato, M., Bergamasco, F., Bigaglia, G., Gasparetto, A., Albarelli, A., Boschetti, M., & Passerone, R. (2023). Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI. Sensors, 23(10), 4667. https://doi.org/10.3390/s23104667
Vancouver
Pistellato M, Bergamasco F, Bigaglia G, Gasparetto A, Albarelli A, Boschetti M, et al. Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI. Sensors. 2023;23(10):4667. doi:10.3390/s23104667
BibTeX
@article{Pistellato2023,
author = {Pistellato, Mara and Bergamasco, Filippo and Bigaglia, Gianluca and Gasparetto, Andrea and Albarelli, Andrea and Boschetti, Marco and Passerone, Roberto},
title = {Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI},
journal = {Sensors},
year = {2023},
volume = {23},
number = {10},
pages = {4667},
doi = {10.3390/s23104667}
}
EUVIMED