Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract)
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (48) · 41255-41257
DOI: 10.1609/aaai.v40i48.42233 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Suwoong Lee und 3 weiteren Autor:innen.
Abstract (Originalsprache)
We propose Tailored ViT Slimming (TVS), a budget-aware multi-dimensional pruning framework for Vision Transformers. TVS injects learnable masks into MHSA and MLP modules and applies adaptive non-convex sparsity regularization to achieve maximal utilization of parameters under strict module-wise budgets. In addition, by retaining scaled masks after pruning, TVS avoids abrupt accuracy drops and provides stable initialization for fine-tuning. On ImageNet-1k with DeiT-S and DeiT-B, TVS consistently outperforms prior ViT compression methods. This result empirically shows that the non-convex sparsity regularizer is effective not only in CNNs but also in ViTs.
Zitieren
APA
Lee, S., Lee, S., Jeon, Y., & Kim, J. (2026). Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 40(48), 41255-41257. https://doi.org/10.1609/aaai.v40i48.42233
Vancouver
Lee S, Lee S, Jeon Y, Kim J. Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(48):41255-41257. doi:10.1609/aaai.v40i48.42233
BibTeX
@article{Lee2026,
author = {Lee, Suwoong and Lee, Seungjae and Jeon, Yunho and Kim, Junmo},
title = {Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract)},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {48},
pages = {41255--41257},
doi = {10.1609/aaai.v40i48.42233}
}
EUVIMED