EUVIMEDEUVIMED entdecken
Crossref

Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning

Yan Fan, Yu Wang, Pengfei Zhu, Qinghua Hu

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (11) · 11927-11935

DOI: 10.1609/aaai.v38i11.29079 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yan Fan und 3 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Continual learning (CL) has shown promising results and comparable performance to learning at once in a fully supervised manner. However, CL strategies typically require a large number of labeled samples, making their real-life deployment challenging. In this work, we focus on semi-supervised continual learning (SSCL), where the model progressively learns from partially labeled data with unknown categories. We provide a comprehensive analysis of SSCL and demonstrate that unreliable distributions of unlabeled data lead to unstable training and refinement of the progressing stages. This problem severely impacts the performance of SSCL. To address the limitations, we propose a novel approach called Dynamic Sub-Graph Distillation (DSGD) for semi-supervised continual learning, which leverages both semantic and structural information to achieve more stable knowledge distillation on unlabeled data and exhibit robustness against distribution bias. Firstly, we formalize a general model of structural distillation and design a dynamic graph construction for the continual learning progress. Next, we define a structure distillation vector and design a dynamic sub-graph distillation algorithm, which enables end-to-end training and adaptability to scale up tasks. The entire proposed method is adaptable to various CL methods and supervision settings. Finally, experiments conducted on three datasets CIFAR10, CIFAR100, and ImageNet-100, with varying supervision ratios, demonstrate the effectiveness of our proposed approach in mitigating the catastrophic forgetting problem in semi-supervised continual learning scenarios. Our code is available: https://github.com/fanyan0411/DSGD.

Zitieren

APA

Fan, Y., Wang, Y., Zhu, P., & Hu, Q. (2024). Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(11), 11927-11935. https://doi.org/10.1609/aaai.v38i11.29079

Vancouver

Fan Y, Wang Y, Zhu P, Hu Q. Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(11):11927-11935. doi:10.1609/aaai.v38i11.29079

BibTeX
@article{Fan2024,
  author = {Fan, Yan and Wang, Yu and Zhu, Pengfei and Hu, Qinghua},
  title = {Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {11},
  pages = {11927--11935},
  doi = {10.1609/aaai.v38i11.29079}
}
Kostenlos starten →