Cognition-driven Adaptive Semantic Decoding Framework for Multimodal Sentiment Analysis
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.06.2026 · 22 (6) · 1-23
DOI: 10.1145/3802587 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 21.05.2026. Verfasst von Jiajia Tang und 5 weiteren Autor:innen. Die Arbeit nennt 60 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Zhejiang Provincial Natural Science Foundation of China, Key Research and Development Project of Zhejiang Province.
Abstract (Originalsprache)
In real-world scenarios, multimodal sentiment analysis faces significant challenges, particularly in cross-scenario generalization. Existing works fail to effectively deal with the variability in evaluation frameworks and modality combinations, which results in poor transfer performance across different application contexts. In this article, the cognition-driven adaptive semantic decoding framework (CASDF) is proposed to realize an evaluation system and modality-independent multimodal sentiment analysis. Specifically, the adaptive modality association module is proposed to construct the adaptive modality mapping space, which allows us to dynamically adapt to arbitrary modality combinations. This indeed breaks through the limitation of the modality number and effectively deals with the modality gap. Furthermore, similar to the human hierarchical cognition (“perception-concept-decision”), the evaluation system progressive alignment module is presented to establish the unified evaluation system. This consists of the perception, concept, and decision analysis, which contributes to the adaptive cross-task analysis from the discrete sentiment space to the continuous sentiment space. The above joint analysis of the evaluation system and modality number indeed leads to the more flexible and generable multimodal sentiment semantic decoding paradigm. The experiments demonstrate that our sentiment semantic analysis network can achieve state-of-the-art performance.
Förderung
- National Natural Science Foundation of China
- Zhejiang Provincial Natural Science Foundation of China
- Key Research and Development Project of Zhejiang Province
Zitieren
APA
Tang, J., Xie, G., Liu, H., Jin, X., Zhang, J., & Kong, W. (2026). Cognition-driven Adaptive Semantic Decoding Framework for Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(6), 1-23. https://doi.org/10.1145/3802587
Vancouver
Tang J, Xie G, Liu H, Jin X, Zhang J, Kong W. Cognition-driven Adaptive Semantic Decoding Framework for Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(6):1-23. doi:10.1145/3802587
BibTeX
@article{Tang2026,
author = {Tang, Jiajia and Xie, Gang and Liu, Honggang and Jin, Xuanyu and Zhang, Jianhai and Kong, Wanzeng},
title = {Cognition-driven Adaptive Semantic Decoding Framework for Multimodal Sentiment Analysis},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2026},
volume = {22},
number = {6},
pages = {1--23},
doi = {10.1145/3802587}
}
EUVIMED