EUVIMEDEUVIMED entdecken
Crossref

Sparse Tuning Enhances Plasticity in PTM-based Continual Learning

Huan Zhang, Shenghua Fan, Shuyu Dong, Yujin Zheng, Dingwen Wang, Fan Lyu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (33) · 28230-28238

DOI: 10.1609/aaai.v40i33.40050 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Huan Zhang und 5 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Continual Learning with Pre-trained Models holds great promise for efficient adaptation across sequential tasks. However, most existing approaches freeze PTMs and rely on auxiliary modules like prompts or adapters, limiting model plasticity and leading to suboptimal generalization when facing significant distribution shifts. While full fine-tuning can improve adaptability, it risks disrupting crucial pre-trained knowledge. In this paper, we propose Mutual Information-guided Sparse Tuning (MIST), a plug-and-play method that selectively updates a small subset of PTM parameters, less than 5%, based on sensitivity to mutual information objectives. MIST enables effective task-specific adaptation while preserving generalization. To further reduce interference, we introduce strong sparsity regularization by randomly dropping gradients during tuning, resulting in fewer than 0.5% of parameters being updated per step. Applied before standard freeze-based methods, MIST consistently boosts performance across diverse continual learning benchmarks. Experiments show that integrating our method into multiple baselines yields significant performance gains.

Zitieren

APA

Zhang, H., Fan, S., Dong, S., Zheng, Y., Wang, D., & Lyu, F. (2026). Sparse Tuning Enhances Plasticity in PTM-based Continual Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(33), 28230-28238. https://doi.org/10.1609/aaai.v40i33.40050

Vancouver

Zhang H, Fan S, Dong S, Zheng Y, Wang D, Lyu F. Sparse Tuning Enhances Plasticity in PTM-based Continual Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(33):28230-28238. doi:10.1609/aaai.v40i33.40050

BibTeX
@article{Zhang2026,
  author = {Zhang, Huan and Fan, Shenghua and Dong, Shuyu and Zheng, Yujin and Wang, Dingwen and Lyu, Fan},
  title = {Sparse Tuning Enhances Plasticity in PTM-based Continual Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {33},
  pages = {28230--28238},
  doi = {10.1609/aaai.v40i33.40050}
}
Kostenlos starten →