EUVIMEDEUVIMED entdecken
Crossref

Attention Retention for Continual Learning with Vision Transformers

Yue Lu, Xiangyu Zhou, Shizhou Zhang, Yinghui Xing, Guoqiang Liang, Wencong Zhang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (29) · 24133-24141

DOI: 10.1609/aaai.v40i29.39592 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yue Lu und 5 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Continual learning (CL) empowers AI systems to progressively acquire knowledge from non-stationary data streams. However, catastrophic forgetting remains a critical challenge. In this work, we identify attention drift in Vision Transformers as a primary source of catastrophic forgetting, where the attention to previously learned visual concepts shifts significantly after learning new tasks. Inspired by neuroscientific insights into the selective attention in the human visual system, we propose a novel attention-retaining framework to mitigate forgetting in CL. Our method constrains attention drift by explicitly modifying gradients during backpropagation through a two-step process: 1) extracting attention maps of the previous task using a layer-wise rollout mechanism and generating instance-adaptive binary masks, and 2) when learning a new task, applying these masks to zero out gradients associated with previous attention regions, thereby preventing disruption of learned visual concepts. For compatibility with modern optimizers, the gradient masking process is further enhanced by scaling parameter updates proportionally to maintain their relative magnitudes. Experiments and visualizations demonstrate the effectiveness of our method in mitigating catastrophic forgetting and preserving visual concepts. It achieves state-of-the-art performance and exhibits robust generalizability across diverse CL scenarios.

Zitieren

APA

Lu, Y., Zhou, X., Zhang, S., Xing, Y., Liang, G., & Zhang, W. (2026). Attention Retention for Continual Learning with Vision Transformers. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29), 24133-24141. https://doi.org/10.1609/aaai.v40i29.39592

Vancouver

Lu Y, Zhou X, Zhang S, Xing Y, Liang G, Zhang W. Attention Retention for Continual Learning with Vision Transformers. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(29):24133-24141. doi:10.1609/aaai.v40i29.39592

BibTeX
@article{Lu2026,
  author = {Lu, Yue and Zhou, Xiangyu and Zhang, Shizhou and Xing, Yinghui and Liang, Guoqiang and Zhang, Wencong},
  title = {Attention Retention for Continual Learning with Vision Transformers},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {29},
  pages = {24133--24141},
  doi = {10.1609/aaai.v40i29.39592}
}
Kostenlos starten →