Pruning Strategies of Vision Transformers: A Specialized Survey
IEEE Open Journal of the Computer Society · 2026 · 7 · 1083-1105
DOI: 10.1109/ojcs.2026.3692927 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Open Journal of the Computer Society, 2026. Verfasst von Setareh Ahsaei und 1 weiteren Autor:innen. Die Arbeit nennt 114 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Vision Transformer (ViT) models are rapidly emerging as a preferred choice for solving various computer vision challenges. ViTs have a high parameter count, leading to substantial computational and memory demands, often exceeding those of traditional convolutional neural networks (CNNs). While compression techniques have proven effective in reducing these burdens, fundamental differences between CNNs and ViTs, such as architectural structure, feature extraction mechanisms, and input dependency, mean that CNN-based pruning schemes cannot be directly applied to ViTs. In this paper, a comprehensive examination of ViT pruning techniques is presented, making this the first survey focused exclusively on pruning in ViTs. Existing methods are analyzed from multiple perspectives, including pruning targets, granularity, pruning schedules, acceleration objectives, and selection criteria, to provide a structured understanding of their effectiveness. The differences between CNN and ViT pruning methods are elucidated, highlighting why traditional CNN strategies are insufficient for ViTs. A quantitative comparison of representative approaches is conducted to evaluate trade-offs among computational cost, model size reduction, and accuracy preservation. Finally, practical pruning guidelines are offered for deployment on resource-constrained devices, and future research directions are discussed to encourage further advancements in efficient, architecture-aware pruning strategies.
Zitieren
APA
Ahsaei, S., & Raji, M. (2026). Pruning Strategies of Vision Transformers: A Specialized Survey. IEEE Open Journal of the Computer Society, 7, 1083-1105. https://doi.org/10.1109/ojcs.2026.3692927
Vancouver
Ahsaei S, Raji M. Pruning Strategies of Vision Transformers: A Specialized Survey. IEEE Open Journal of the Computer Society. 2026;7:1083-1105. doi:10.1109/ojcs.2026.3692927
BibTeX
@article{Ahsaei2026,
author = {Ahsaei, Setareh and Raji, Mohsen},
title = {Pruning Strategies of Vision Transformers: A Specialized Survey},
journal = {IEEE Open Journal of the Computer Society},
year = {2026},
volume = {7},
pages = {1083--1105},
doi = {10.1109/ojcs.2026.3692927}
}
EUVIMED