EUVIMEDEUVIMED entdecken
Crossref

MoE-Adapters++: Toward More Efficient Continual Learning of Vision-Language Models Via Dynamic Mixture-of-Experts Adapters

Jiazuo Yu, Zichen Huang, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang, Huchuan Lu, You He

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-12 · 47 (12) · 11912-11928

DOI: 10.1109/tpami.2025.3597942 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-12. Verfasst von Jiazuo Yu und 7 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 102 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, Basic Scientific Research Funding of the Central Universities of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this paper, we first propose MoE-Adapters, a parameter-efficient training framework to alleviate long-term forgetting issues in incremental learning with Vision-Language Models (VLM). Our MoE-Adapters leverages incrementally added routers to activate and integrate exclusive expert adapters from a pre-defined static expert set, enabling the pre-trained CLIP to efficiently adapt to new tasks. To preserve the zero-shot capability of VLM, a Distribution Discriminative Auto-Selector (DDAS) is introduced that automatically routes in-distribution and out-of-distribution inputs to the MoE-Adapters and the original CLIP, respectively. However, relying on a static expert set and a separate distribution selector can lead to parameter redundancy and increased training complexity. In response, we further extend an MoE-Adapters++ framework by introducing dynamic MoE-adapters, which allows experts to be adaptively involved during the continual learning process. Additionally, a Latent Embedding Auto-Selector (LEAS) is proposed that incorporates distribution selection within CLIP to create a more unified architecture. Extensive experiments across diverse settings demonstrate that the proposed method consistently surpasses previous state-of-the-art approaches while concurrently improving training efficiency.

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • Basic Scientific Research Funding of the Central Universities of China
  • Liaoning Provincial Science and Technology Joint Program Project

Zitieren

APA

Yu, J., Huang, Z., Zhuge, Y., Zhang, L., Hu, P., Wang, D., Lu, H., & He, Y. (2025). MoE-Adapters++: Toward More Efficient Continual Learning of Vision-Language Models Via Dynamic Mixture-of-Experts Adapters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(12), 11912-11928. https://doi.org/10.1109/tpami.2025.3597942

Vancouver

Yu J, Huang Z, Zhuge Y, Zhang L, Hu P, Wang D, et al. MoE-Adapters++: Toward More Efficient Continual Learning of Vision-Language Models Via Dynamic Mixture-of-Experts Adapters. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(12):11912-11928. doi:10.1109/tpami.2025.3597942

BibTeX
@article{Yu2025,
  author = {Yu, Jiazuo and Huang, Zichen and Zhuge, Yunzhi and Zhang, Lu and Hu, Ping and Wang, Dong and Lu, Huchuan and He, You},
  title = {MoE-Adapters++: Toward More Efficient Continual Learning of Vision-Language Models Via Dynamic Mixture-of-Experts Adapters},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {12},
  pages = {11912--11928},
  doi = {10.1109/tpami.2025.3597942}
}
Kostenlos starten →