Channel Pruning Method Based on Decoupling Feature Scale Distribution in Batch Normalization Layers
IEEE Access · 2024 · 12 · 48865-48880
DOI: 10.1109/access.2024.3382994 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2024. Verfasst von Zijie Qiu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Science Foundation of China, Shanghai University IPv6 Scale Deployment Innovation Integration Demonstration Application and IPv6 Network Support Capability Improvement Project, Based on 5G+AI Data-Driven 020 Integrated Immersive Teaching Platform Project.
Abstract (Originalsprache)
Pruning and compression of models are practical approaches for deploying and applying deep convolutional neural networks in scenarios with limited memory and computational resources. To mitigate the impact of pruning on model accuracy and enhance the stability of pruning (defined as the negligible drop in test accuracy immediately following pruning), an algorithm for reward-penalty decoupling is introduced in this study to achieve automated sparse training and channel pruning. During sparse training, the influence of unimportant channels is automatically identified and reduced, thereby preserving the ability of the important channels for feature recognition. First, by utilizing the gradient information learned through network backpropagation, the feature scaling factors of the batch normalization layers are combined with the gradient to determine the importance threshold for the network channels. Subsequently, a two-stage sparse training algorithm is proposed based on the reward-penalty decoupling strategy, applying different loss function strategies to the feature scaling factors of “important” and “unimportant” channels during decoupled sparse training. This approach has been experimentally validated across various tasks, baselines, and datasets, demonstrating its superiority over the previous state-of-the-art methods. The results indicate that the effect of pruning on model accuracy is significantly alleviated by the proposed method, and pruned models require only limited fine-tuning to achieve excellent performance.
Förderung
- National Science Foundation of China
- Shanghai University IPv6 Scale Deployment Innovation Integration Demonstration Application and IPv6 Network Support Capability Improvement Project
- Based on 5G+AI Data-Driven 020 Integrated Immersive Teaching Platform Project
Zitieren
APA
Qiu, Z., Wei, P., Yao, M., Zhang, R., & Kuang, Y. (2024). Channel Pruning Method Based on Decoupling Feature Scale Distribution in Batch Normalization Layers. IEEE Access, 12, 48865-48880. https://doi.org/10.1109/access.2024.3382994
Vancouver
Qiu Z, Wei P, Yao M, Zhang R, Kuang Y. Channel Pruning Method Based on Decoupling Feature Scale Distribution in Batch Normalization Layers. IEEE Access. 2024;12:48865-48880. doi:10.1109/access.2024.3382994
BibTeX
@article{Qiu2024,
author = {Qiu, Zijie and Wei, Peng and Yao, Mingwei and Zhang, Rui and Kuang, Yingchun},
title = {Channel Pruning Method Based on Decoupling Feature Scale Distribution in Batch Normalization Layers},
journal = {IEEE Access},
year = {2024},
volume = {12},
pages = {48865--48880},
doi = {10.1109/access.2024.3382994}
}
EUVIMED