KSS-MoE: Knowledge Space Synergy Framework in Mixture of Experts for Continual Visual Instruction Tuning
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25536-25544
DOI: 10.1609/aaai.v40i30.39749 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Lingyun Song und 8 weiteren Autor:innen.
Abstract (Originalsprache)
Multimodal Large Language Models (MLLMs) employing the Mixture-of-Experts (MoE) structure exhibit encouraging results in visual language tasks. However, they struggle with catastrophic forgetting due to a lack of effective collaboration among experts and negative transfer across tasks. This happens because the router typically employed in MoE for managing expert assignments is inadequate when there are significant shifts in data distribution across various tasks. A drop in the effectiveness of earlier tasks is caused by negative transfer, which occurs due to conflicts in shared knowledge between tasks, disturbing the knowledge already acquired. To address these issues, we propose the Knowledge Space Synergy Framework in Mixture of Experts (KSS-MoE) for Continual Visual Instruction Tuning (CVIT). It dynamically combines the knowledge subspaces of experts to improve the integration of fine-grained complementary knowledge and collaborative abilities of experts, thus addressing the limitations of the basic router. Furthermore, we introduce a general expert that maintains orthogonal subspaces for shared knowledge, enabling effective cross-task knowledge utilization while reducing negative transfer. Extensive experiments conducted on eight CVIT tasks confirm the excellence of KSS-MoE, showcasing its top-tier performance.
Zitieren
APA
Song, L., Chen, Z., Pan, K., Han, X., Gan, X., Pan, Y., Sun, X., Wang, X., & Shang, X. (2026). KSS-MoE: Knowledge Space Synergy Framework in Mixture of Experts for Continual Visual Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25536-25544. https://doi.org/10.1609/aaai.v40i30.39749
Vancouver
Song L, Chen Z, Pan K, Han X, Gan X, Pan Y, et al. KSS-MoE: Knowledge Space Synergy Framework in Mixture of Experts for Continual Visual Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25536-25544. doi:10.1609/aaai.v40i30.39749
BibTeX
@article{Song2026,
author = {Song, Lingyun and Chen, Ziyao and Pan, Kang and Han, Xiaolin and Gan, Xinbiao and Pan, Yudai and Sun, Xiaofan and Wang, Xiaoqi and Shang, Xuequn},
title = {KSS-MoE: Knowledge Space Synergy Framework in Mixture of Experts for Continual Visual Instruction Tuning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {30},
pages = {25536--25544},
doi = {10.1609/aaai.v40i30.39749}
}
EUVIMED