EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Matching the Ideal Pruning Method with Knowledge Distillation for Optimal Compression

Leila Malihi, Gunther Heidemann

Applied System Innovation · 29.06.2024 · 7 (4) · 56

DOI: 10.3390/asi7040056 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied System Innovation, online erschienen am 29.06.2024. Verfasst von Leila Malihi und 1 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Osnabrück University “Open Access Publizieren” of the “Deutsche Forschungsgemeinschaft” (DFG).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, model compression techniques have gained significant attention as a means to reduce the computational and memory requirements of deep neural networks. Knowledge distillation and pruning are two prominent approaches in this domain, each offering unique advantages in achieving model efficiency. This paper investigates the combined effects of knowledge distillation and two pruning strategies, weight pruning and channel pruning, on enhancing compression efficiency and model performance. The study introduces a metric called “Performance Efficiency” to evaluate the impact of these pruning strategies on model compression and performance. Our research is conducted on the popular datasets CIFAR-10 and CIFAR-100. We compared diverse model architectures, including ResNet, DenseNet, EfficientNet, and MobileNet. The results emphasize the efficacy of both weight and channel pruning in achieving model compression. However, a significant distinction emerges, with weight pruning showing superior performance across all four architecture types. We realized that the weight pruning method better adapts to knowledge distillation than channel pruning. Pruned models show a significant reduction in parameters without a significant reduction in accuracy.

Förderung

  • Osnabrück University “Open Access Publizieren” of the “Deutsche Forschungsgemeinschaft” (DFG)

Zitieren

APA

Malihi, L., & Heidemann, G. (2024). Matching the Ideal Pruning Method with Knowledge Distillation for Optimal Compression. Applied System Innovation, 7(4), 56. https://doi.org/10.3390/asi7040056

Vancouver

Malihi L, Heidemann G. Matching the Ideal Pruning Method with Knowledge Distillation for Optimal Compression. Applied System Innovation. 2024;7(4):56. doi:10.3390/asi7040056

BibTeX
@article{Malihi2024,
  author = {Malihi, Leila and Heidemann, Gunther},
  title = {Matching the Ideal Pruning Method with Knowledge Distillation for Optimal Compression},
  journal = {Applied System Innovation},
  year = {2024},
  volume = {7},
  number = {4},
  pages = {56},
  doi = {10.3390/asi7040056}
}
Kostenlos starten →