EUVIMEDEUVIMED entdecken
Crossref

FINE: Factorized Multimodal Sentiment Analysis via Mutual INformation Estimation

Yadong Liu, Shangfei Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (3) · 1955-1963

DOI: 10.1609/aaai.v40i3.37176 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yadong Liu und 1 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal sentiment analysis remains a challenging task due to the inherent heterogeneity across modalities. Such heterogeneity often manifests as asynchronous signals, imbalanced information between modalities, and interference from task-irrelevant noise, hindering the learning of robust and accurate sentiment representations. To address these issues, we propose a factorized multimodal fusion framework that first disentangles each modality into shared and unique representations, and then suppresses task-irrelevant noise within both to retain only sentiment-critical representations. This fine-grained decomposition improves representation quality by reducing redundancy, prompting cross-modal complementarity, and isolating task-relevant sentiment cues. Rather than manipulating the feature space directly, we adopt a mutual information–based optimization strategy to guide the factorization process in a more stable and principled manner. To further support feature extraction and long-term temporal modeling, we introduce two auxiliary modules: a Mixture of Q-Formers, placed before factorization, which precedes the factorization and uses learnable queries to extract fine-grained affective features from multiple modalities, and a Dynamic Contrastive Queue, placed after factorization, which stores latest high-level representations for contrastive learning, enabling the model to capture long-range discriminative patterns and improve class-level separability. Extensive experiments on multiple public datasets demonstrate that our method consistently outperforms existing approaches, validating the effecti veness and robustness of the proposed framework.

Zitieren

APA

Liu, Y., & Wang, S. (2026). FINE: Factorized Multimodal Sentiment Analysis via Mutual INformation Estimation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(3), 1955-1963. https://doi.org/10.1609/aaai.v40i3.37176

Vancouver

Liu Y, Wang S. FINE: Factorized Multimodal Sentiment Analysis via Mutual INformation Estimation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(3):1955-1963. doi:10.1609/aaai.v40i3.37176

BibTeX
@article{Liu2026,
  author = {Liu, Yadong and Wang, Shangfei},
  title = {FINE: Factorized Multimodal Sentiment Analysis via Mutual INformation Estimation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {3},
  pages = {1955--1963},
  doi = {10.1609/aaai.v40i3.37176}
}
Kostenlos starten →