Towards Multimodal Continual Knowledge Embedding with Modality Forgetting Modulation
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (17) · 14946-14954
DOI: 10.1609/aaai.v40i17.38516 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xiaowen Jiang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
The continuous emergence of new entities, relations, triples, and multimodal information drives the dynamic evolution of multimodal knowledge graph (MMKG). However, existing MMKG embedding models follow a static setting, where training from scratch for growing MMKG wastes learned knowledge, while fine-tuning on new knowledge easily leads to catastrophic forgetting, severely limiting their applicability in real-world scenarios. To address this, we propose a multimodal continual representation learning framework (MoFot) for growing MMKG. Unlike existing static multimodal embedding methods, MoFot focuses on alleviating catastrophic forgetting rather than retraining to adapt to new knowledge. Specifically, MoFot effectively mitigates catastrophic forgetting caused by parameter updates and differing forgetting rates across modalities through a multimodal collaborative modulation mechanism. The mechanism ensures consistent retention of previously learned multimodal knowledge across snapshots through multimodal weight modulation and multimodal feature modulation. MoFot outperforms existing MMKG embedding, KG continual learning, and MMKG inductive models. Experimental results demonstrate that MoFot not only avoids forgetting but also enhances old knowledge by learning new knowledge, achieving adaptation to new knowledge while mitigating forgetting of old knowledge.
Zitieren
APA
Jiang, X., Yang, J., Yang, S., Gao, Y., Jiang, X., Yang, L. T., & Yang, J. (2026). Towards Multimodal Continual Knowledge Embedding with Modality Forgetting Modulation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(17), 14946-14954. https://doi.org/10.1609/aaai.v40i17.38516
Vancouver
Jiang X, Yang J, Yang S, Gao Y, Jiang X, Yang LT, et al. Towards Multimodal Continual Knowledge Embedding with Modality Forgetting Modulation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(17):14946-14954. doi:10.1609/aaai.v40i17.38516
BibTeX
@article{Jiang2026,
author = {Jiang, Xiaowen and Yang, Jing and Yang, ShunDong and Gao, Yuan and Jiang, Xinfa and Yang, Laurence Tianruo and Yang, Jieming},
title = {Towards Multimodal Continual Knowledge Embedding with Modality Forgetting Modulation},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {17},
pages = {14946--14954},
doi = {10.1609/aaai.v40i17.38516}
}
EUVIMED