Joint MLP and Token Pruning for Personalizing Vision Transformers
Big Data and Cognitive Computing · 09.08.2026 · 10 (8) · 265
DOI: 10.3390/bdcc10080265 ↗Kurz gefasst
Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 09.08.2026. Verfasst von Zhiyue Li und 4 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
ViTs have achieved excellent performance in image recognition tasks, but their large parameter counts and high computational complexity limit their deployment on resource-constrained devices. Most existing ViT pruning methods adopt class-agnostic pruning strategies, which fail to distinguish the diverse structural requirements of different target classes. As a result, they are prone to removing critical features, leading to class-wise accuracy imbalance in practical deployment. To address this issue, this paper proposes a class-aware joint pruning framework for ViTs, which collaboratively compresses the model from two orthogonal dimensions: MLP neurons and visual tokens. Specifically, (1) based on first-order Taylor expansion, we quantify the contribution of each MLP neuron to the target classes and adaptively prune redundant neurons to achieve structured compression, followed by lightweight fine-tuning on the target class subset; (2) we propose a Class-Guided Token Selection (CGTS) method, which constructs class prototype vectors using a few support samples of the target classes and then dynamically selects patch tokens that are semantically highly relevant to the target classes during inference in a zero-shot manner, requiring no additional training or fine-tuning. The two modules complement each other, achieving dual compression from the parameter dimension and the inference data dimension. Experiments on CIFAR-100 and TinyImageNet datasets using DeiT-Tiny/Small models demonstrate that, compared with state-of-the-art pruning methods, our method reduces GMACs on target class subsets by up to 48%, improves inference speed by nearly 50%, and requires only 0.8 KB of additional storage overhead per subset, ultimately achieving a superior trade-off among accuracy, computational efficiency, and storage overhead.
Zitieren
APA
Li, Z., Liu, T., Huang, F., Huang, X., & Zou, Z. (2026). Joint MLP and Token Pruning for Personalizing Vision Transformers. Big Data and Cognitive Computing, 10(8), 265. https://doi.org/10.3390/bdcc10080265
Vancouver
Li Z, Liu T, Huang F, Huang X, Zou Z. Joint MLP and Token Pruning for Personalizing Vision Transformers. Big Data and Cognitive Computing. 2026;10(8):265. doi:10.3390/bdcc10080265
BibTeX
@article{Li2026,
author = {Li, Zhiyue and Liu, Tong and Huang, Feng and Huang, Xinzhi and Zou, Zhihao},
title = {Joint MLP and Token Pruning for Personalizing Vision Transformers},
journal = {Big Data and Cognitive Computing},
year = {2026},
volume = {10},
number = {8},
pages = {265},
doi = {10.3390/bdcc10080265}
}
EUVIMED