EUVIMEDEUVIMED entdecken
Crossref

A Unified Self-Distillation Framework for Multimodal Sentiment Analysis with Uncertain Missing Modalities

Mingcheng Li, Dingkang Yang, Yuxuan Lei, Shunli Wang, Shuaibing Wang, Liuzhen Su, Kun Yang, Yuzheng Wang, Mingyang Sun, Lihua Zhang

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (9) · 10074-10082

DOI: 10.1609/aaai.v38i9.28871 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Mingcheng Li und 9 weiteren Autor:innen. Die Arbeit wurde bisher 74-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal Sentiment Analysis (MSA) has attracted widespread research attention recently. Most MSA studies are based on the assumption of modality completeness. However, many inevitable factors in real-world scenarios lead to uncertain missing modalities, which invalidate the fixed multimodal fusion approaches. To this end, we propose a Unified multimodal Missing modality self-Distillation Framework (UMDF) to handle the problem of uncertain missing modalities in MSA. Specifically, a unified self-distillation mechanism in UMDF drives a single network to automatically learn robust inherent representations from the consistent distribution of multimodal data. Moreover, we present a multi-grained crossmodal interaction module to deeply mine the complementary semantics among modalities through coarse- and fine-grained crossmodal attention. Eventually, a dynamic feature integration module is introduced to enhance the beneficial semantics in incomplete modalities while filtering the redundant information therein to obtain a refined and robust multimodal representation. Comprehensive experiments on three datasets demonstrate that our framework significantly improves MSA performance under both uncertain missing-modality and complete-modality testing conditions.

Zitieren

APA

Li, M., Yang, D., Lei, Y., Wang, S., Wang, S., Su, L., Yang, K., Wang, Y., Sun, M., & Zhang, L. (2024). A Unified Self-Distillation Framework for Multimodal Sentiment Analysis with Uncertain Missing Modalities. Proceedings of the AAAI Conference on Artificial Intelligence, 38(9), 10074-10082. https://doi.org/10.1609/aaai.v38i9.28871

Vancouver

Li M, Yang D, Lei Y, Wang S, Wang S, Su L, et al. A Unified Self-Distillation Framework for Multimodal Sentiment Analysis with Uncertain Missing Modalities. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(9):10074-10082. doi:10.1609/aaai.v38i9.28871

BibTeX
@article{Li2024,
  author = {Li, Mingcheng and Yang, Dingkang and Lei, Yuxuan and Wang, Shunli and Wang, Shuaibing and Su, Liuzhen and Yang, Kun and Wang, Yuzheng and Sun, Mingyang and Zhang, Lihua},
  title = {A Unified Self-Distillation Framework for Multimodal Sentiment Analysis with Uncertain Missing Modalities},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {9},
  pages = {10074--10082},
  doi = {10.1609/aaai.v38i9.28871}
}
Kostenlos starten →