MoE-Adapters++: Toward More Efficient Continual Learning of Vision-Language Models Via Dynamic Mixture-of-Experts Adapters
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-12 · 47 (12) · 11912-11928
DOI: 10.1109/tpami.2025.3597942 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-12. Verfasst von Jiazuo Yu und 7 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 102 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, Basic Scientific Research Funding of the Central Universities of China.
Abstract (Originalsprache)
In this paper, we first propose MoE-Adapters, a parameter-efficient training framework to alleviate long-term forgetting issues in incremental learning with Vision-Language Models (VLM). Our MoE-Adapters leverages incrementally added routers to activate and integrate exclusive expert adapters from a pre-defined static expert set, enabling the pre-trained CLIP to efficiently adapt to new tasks. To preserve the zero-shot capability of VLM, a Distribution Discriminative Auto-Selector (DDAS) is introduced that automatically routes in-distribution and out-of-distribution inputs to the MoE-Adapters and the original CLIP, respectively. However, relying on a static expert set and a separate distribution selector can lead to parameter redundancy and increased training complexity. In response, we further extend an MoE-Adapters++ framework by introducing dynamic MoE-adapters, which allows experts to be adaptively involved during the continual learning process. Additionally, a Latent Embedding Auto-Selector (LEAS) is proposed that incorporates distribution selection within CLIP to create a more unified architecture. Extensive experiments across diverse settings demonstrate that the proposed method consistently surpasses previous state-of-the-art approaches while concurrently improving training efficiency.
Förderung
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities
- Basic Scientific Research Funding of the Central Universities of China
- Liaoning Provincial Science and Technology Joint Program Project
Zitieren
APA
Yu, J., Huang, Z., Zhuge, Y., Zhang, L., Hu, P., Wang, D., Lu, H., & He, Y. (2025). MoE-Adapters++: Toward More Efficient Continual Learning of Vision-Language Models Via Dynamic Mixture-of-Experts Adapters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(12), 11912-11928. https://doi.org/10.1109/tpami.2025.3597942
Vancouver
Yu J, Huang Z, Zhuge Y, Zhang L, Hu P, Wang D, et al. MoE-Adapters++: Toward More Efficient Continual Learning of Vision-Language Models Via Dynamic Mixture-of-Experts Adapters. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(12):11912-11928. doi:10.1109/tpami.2025.3597942
BibTeX
@article{Yu2025,
author = {Yu, Jiazuo and Huang, Zichen and Zhuge, Yunzhi and Zhang, Lu and Hu, Ping and Wang, Dong and Lu, Huchuan and He, You},
title = {MoE-Adapters++: Toward More Efficient Continual Learning of Vision-Language Models Via Dynamic Mixture-of-Experts Adapters},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {12},
pages = {11912--11928},
doi = {10.1109/tpami.2025.3597942}
}
EUVIMED