EUVIMEDEUVIMED entdecken
Crossref

Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract)

Suwoong Lee, Seungjae Lee, Yunho Jeon, Junmo Kim

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (48) · 41255-41257

DOI: 10.1609/aaai.v40i48.42233 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Suwoong Lee und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We propose Tailored ViT Slimming (TVS), a budget-aware multi-dimensional pruning framework for Vision Transformers. TVS injects learnable masks into MHSA and MLP modules and applies adaptive non-convex sparsity regularization to achieve maximal utilization of parameters under strict module-wise budgets. In addition, by retaining scaled masks after pruning, TVS avoids abrupt accuracy drops and provides stable initialization for fine-tuning. On ImageNet-1k with DeiT-S and DeiT-B, TVS consistently outperforms prior ViT compression methods. This result empirically shows that the non-convex sparsity regularizer is effective not only in CNNs but also in ViTs.

Zitieren

APA

Lee, S., Lee, S., Jeon, Y., & Kim, J. (2026). Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 40(48), 41255-41257. https://doi.org/10.1609/aaai.v40i48.42233

Vancouver

Lee S, Lee S, Jeon Y, Kim J. Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(48):41255-41257. doi:10.1609/aaai.v40i48.42233

BibTeX
@article{Lee2026,
  author = {Lee, Suwoong and Lee, Seungjae and Jeon, Yunho and Kim, Junmo},
  title = {Tailored ViT Slimming: Budget-Aware Multi-Dimensional Sparsity Regularization for Vision Transformers Pruning (Student Abstract)},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {48},
  pages = {41255--41257},
  doi = {10.1609/aaai.v40i48.42233}
}
Kostenlos starten →