Compact Neural Network via Stacking Hybrid Units
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-01 · 46 (1) · 103-116
DOI: 10.1109/tpami.2023.3323496 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-01. Verfasst von Weichao Lan und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 89 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NSFC and Research Grants Council, General Research Fund of RGC, Hong Kong Baptist University.
Abstract (Originalsprache)
As an effective tool for network compression, pruning techniques have been widely used to reduce the large number of parameters in deep neural networks (NNs). Nevertheless, unstructured pruning has the limitation of dealing with the sparse and irregular weights. By contrast, structured pruning can help eliminate this drawback but it requires complex criteria to determine which components to be pruned. Therefore, this paper presents a new method termed BUnit-Net, which directly constructs compact NNs by stacking designed basic units, without requiring additional judgement criteria anymore. Given the basic units of various architectures, they are combined and stacked systematically to build up compact NNs which involve fewer weight parameters due to the independence among the units. In this way, BUnit-Net can achieve the same compression effect as unstructured pruning while the weight tensors can still remain regular and dense. We formulate BUnit-Net in diverse popular backbones in comparison with the state-of-the-art pruning methods on different benchmark datasets. Moreover, two new metrics are proposed to evaluate the trade-off of compression performance. Experiment results show that BUnit-Net can achieve comparable classification accuracy while saving around 80% FLOPs and 73% parameters. That is, stacking basic units provides a new promising way for network compression.
Förderung
- NSFC and Research Grants Council
- General Research Fund of RGC
- Hong Kong Baptist University
Zitieren
APA
Lan, W., Cheung, Y. M., Jiang, J., Hu, Z., & Li, M. (2024). Compact Neural Network via Stacking Hybrid Units. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1), 103-116. https://doi.org/10.1109/tpami.2023.3323496
Vancouver
Lan W, Cheung YM, Jiang J, Hu Z, Li M. Compact Neural Network via Stacking Hybrid Units. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(1):103-116. doi:10.1109/tpami.2023.3323496
BibTeX
@article{Lan2024,
author = {Lan, Weichao and Cheung, Yiu-Ming and Jiang, Juyong and Hu, Zhikai and Li, Mengke},
title = {Compact Neural Network via Stacking Hybrid Units},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {1},
pages = {103--116},
doi = {10.1109/tpami.2023.3323496}
}
EUVIMED