EUVIMEDEUVIMED entdecken
Crossref

KSS-MoE: Knowledge Space Synergy Framework in Mixture of Experts for Continual Visual Instruction Tuning

Lingyun Song, Ziyao Chen, Kang Pan, Xiaolin Han, Xinbiao Gan, Yudai Pan, Xiaofan Sun, Xiaoqi Wang, Xuequn Shang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25536-25544

DOI: 10.1609/aaai.v40i30.39749 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Lingyun Song und 8 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal Large Language Models (MLLMs) employing the Mixture-of-Experts (MoE) structure exhibit encouraging results in visual language tasks. However, they struggle with catastrophic forgetting due to a lack of effective collaboration among experts and negative transfer across tasks. This happens because the router typically employed in MoE for managing expert assignments is inadequate when there are significant shifts in data distribution across various tasks. A drop in the effectiveness of earlier tasks is caused by negative transfer, which occurs due to conflicts in shared knowledge between tasks, disturbing the knowledge already acquired. To address these issues, we propose the Knowledge Space Synergy Framework in Mixture of Experts (KSS-MoE) for Continual Visual Instruction Tuning (CVIT). It dynamically combines the knowledge subspaces of experts to improve the integration of fine-grained complementary knowledge and collaborative abilities of experts, thus addressing the limitations of the basic router. Furthermore, we introduce a general expert that maintains orthogonal subspaces for shared knowledge, enabling effective cross-task knowledge utilization while reducing negative transfer. Extensive experiments conducted on eight CVIT tasks confirm the excellence of KSS-MoE, showcasing its top-tier performance.

Zitieren

APA

Song, L., Chen, Z., Pan, K., Han, X., Gan, X., Pan, Y., Sun, X., Wang, X., & Shang, X. (2026). KSS-MoE: Knowledge Space Synergy Framework in Mixture of Experts for Continual Visual Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25536-25544. https://doi.org/10.1609/aaai.v40i30.39749

Vancouver

Song L, Chen Z, Pan K, Han X, Gan X, Pan Y, et al. KSS-MoE: Knowledge Space Synergy Framework in Mixture of Experts for Continual Visual Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25536-25544. doi:10.1609/aaai.v40i30.39749

BibTeX
@article{Song2026,
  author = {Song, Lingyun and Chen, Ziyao and Pan, Kang and Han, Xiaolin and Gan, Xinbiao and Pan, Yudai and Sun, Xiaofan and Wang, Xiaoqi and Shang, Xuequn},
  title = {KSS-MoE: Knowledge Space Synergy Framework in Mixture of Experts for Continual Visual Instruction Tuning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {30},
  pages = {25536--25544},
  doi = {10.1609/aaai.v40i30.39749}
}
Kostenlos starten →