EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Channel Pruning Method Based on Decoupling Feature Scale Distribution in Batch Normalization Layers

Zijie Qiu, Peng Wei, Mingwei Yao, Rui Zhang, Yingchun Kuang

IEEE Access · 2024 · 12 · 48865-48880

DOI: 10.1109/access.2024.3382994 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2024. Verfasst von Zijie Qiu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Science Foundation of China, Shanghai University IPv6 Scale Deployment Innovation Integration Demonstration Application and IPv6 Network Support Capability Improvement Project, Based on 5G+AI Data-Driven 020 Integrated Immersive Teaching Platform Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Pruning and compression of models are practical approaches for deploying and applying deep convolutional neural networks in scenarios with limited memory and computational resources. To mitigate the impact of pruning on model accuracy and enhance the stability of pruning (defined as the negligible drop in test accuracy immediately following pruning), an algorithm for reward-penalty decoupling is introduced in this study to achieve automated sparse training and channel pruning. During sparse training, the influence of unimportant channels is automatically identified and reduced, thereby preserving the ability of the important channels for feature recognition. First, by utilizing the gradient information learned through network backpropagation, the feature scaling factors of the batch normalization layers are combined with the gradient to determine the importance threshold for the network channels. Subsequently, a two-stage sparse training algorithm is proposed based on the reward-penalty decoupling strategy, applying different loss function strategies to the feature scaling factors of “important” and “unimportant” channels during decoupled sparse training. This approach has been experimentally validated across various tasks, baselines, and datasets, demonstrating its superiority over the previous state-of-the-art methods. The results indicate that the effect of pruning on model accuracy is significantly alleviated by the proposed method, and pruned models require only limited fine-tuning to achieve excellent performance.

Förderung

  • National Science Foundation of China
  • Shanghai University IPv6 Scale Deployment Innovation Integration Demonstration Application and IPv6 Network Support Capability Improvement Project
  • Based on 5G+AI Data-Driven 020 Integrated Immersive Teaching Platform Project

Zitieren

APA

Qiu, Z., Wei, P., Yao, M., Zhang, R., & Kuang, Y. (2024). Channel Pruning Method Based on Decoupling Feature Scale Distribution in Batch Normalization Layers. IEEE Access, 12, 48865-48880. https://doi.org/10.1109/access.2024.3382994

Vancouver

Qiu Z, Wei P, Yao M, Zhang R, Kuang Y. Channel Pruning Method Based on Decoupling Feature Scale Distribution in Batch Normalization Layers. IEEE Access. 2024;12:48865-48880. doi:10.1109/access.2024.3382994

BibTeX
@article{Qiu2024,
  author = {Qiu, Zijie and Wei, Peng and Yao, Mingwei and Zhang, Rui and Kuang, Yingchun},
  title = {Channel Pruning Method Based on Decoupling Feature Scale Distribution in Batch Normalization Layers},
  journal = {IEEE Access},
  year = {2024},
  volume = {12},
  pages = {48865--48880},
  doi = {10.1109/access.2024.3382994}
}
Kostenlos starten →