Class Incremental Learning With Deep Contrastive Learning and Attention Distillation
IEEE Signal Processing Letters · 2024 · 31 · 1224-1228
DOI: 10.1109/lsp.2024.3392678 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Signal Processing Letters, 2024. Verfasst von Jitao Zhu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: Shenzhen Science and Technology Program.
Abstract (Originalsprache)
Class incremental learning can solve the issue of catastrophic forgetting when the trained model is used to learn a new task in which the model may forget part of previous knowledge learned. The key issue is to alleviate the stability-plasticity dilemma and maintain the balance between preventing old knowledge from being forgotten and learning new knowledge. In this letter, a new class incremental learning method with deep contrastive learning and attention distillation is proposed. The deep contrastive learning can optimize last layer of the model to learn strong-semantic information and the intermediate layers to learn weak-semantic information when new data is coming so that the plasticity can be improved. To ensure a balance between plasticity and stability, a new attention distillation approach is developed to learn and distill the latent attention information of the feature maps and the similarity between different samples. Our analysis and experimental results indicate that the proposed method can improve the learning and memorization ability and the average accuracy of the model compared with the previous methods, and obtain competitive results in class increment scenarios for image classification.
Förderung
- Shenzhen Science and Technology Program
Zitieren
APA
Zhu, J., Luo, G., Duan, B., & Zhu, Y. (2024). Class Incremental Learning With Deep Contrastive Learning and Attention Distillation. IEEE Signal Processing Letters, 31, 1224-1228. https://doi.org/10.1109/lsp.2024.3392678
Vancouver
Zhu J, Luo G, Duan B, Zhu Y. Class Incremental Learning With Deep Contrastive Learning and Attention Distillation. IEEE Signal Processing Letters. 2024;31:1224-1228. doi:10.1109/lsp.2024.3392678
BibTeX
@article{Zhu2024,
author = {Zhu, Jitao and Luo, Guibo and Duan, Baishan and Zhu, Yuesheng},
title = {Class Incremental Learning With Deep Contrastive Learning and Attention Distillation},
journal = {IEEE Signal Processing Letters},
year = {2024},
volume = {31},
pages = {1224--1228},
doi = {10.1109/lsp.2024.3392678}
}
EUVIMED