EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Compact Neural Network via Stacking Hybrid Units

Weichao Lan, Yiu-Ming Cheung, Juyong Jiang, Zhikai Hu, Mengke Li

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-01 · 46 (1) · 103-116

DOI: 10.1109/tpami.2023.3323496 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-01. Verfasst von Weichao Lan und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 89 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NSFC and Research Grants Council, General Research Fund of RGC, Hong Kong Baptist University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As an effective tool for network compression, pruning techniques have been widely used to reduce the large number of parameters in deep neural networks (NNs). Nevertheless, unstructured pruning has the limitation of dealing with the sparse and irregular weights. By contrast, structured pruning can help eliminate this drawback but it requires complex criteria to determine which components to be pruned. Therefore, this paper presents a new method termed BUnit-Net, which directly constructs compact NNs by stacking designed basic units, without requiring additional judgement criteria anymore. Given the basic units of various architectures, they are combined and stacked systematically to build up compact NNs which involve fewer weight parameters due to the independence among the units. In this way, BUnit-Net can achieve the same compression effect as unstructured pruning while the weight tensors can still remain regular and dense. We formulate BUnit-Net in diverse popular backbones in comparison with the state-of-the-art pruning methods on different benchmark datasets. Moreover, two new metrics are proposed to evaluate the trade-off of compression performance. Experiment results show that BUnit-Net can achieve comparable classification accuracy while saving around 80% FLOPs and 73% parameters. That is, stacking basic units provides a new promising way for network compression.

Förderung

  • NSFC and Research Grants Council
  • General Research Fund of RGC
  • Hong Kong Baptist University

Zitieren

APA

Lan, W., Cheung, Y. M., Jiang, J., Hu, Z., & Li, M. (2024). Compact Neural Network via Stacking Hybrid Units. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1), 103-116. https://doi.org/10.1109/tpami.2023.3323496

Vancouver

Lan W, Cheung YM, Jiang J, Hu Z, Li M. Compact Neural Network via Stacking Hybrid Units. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(1):103-116. doi:10.1109/tpami.2023.3323496

BibTeX
@article{Lan2024,
  author = {Lan, Weichao and Cheung, Yiu-Ming and Jiang, Juyong and Hu, Zhikai and Li, Mengke},
  title = {Compact Neural Network via Stacking Hybrid Units},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {1},
  pages = {103--116},
  doi = {10.1109/tpami.2023.3323496}
}
Kostenlos starten →