PaSE: Prototype-aligned Calibration and Shapley-based Equilibrium for Multimodal Sentiment Analysis
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 30960-30968
DOI: 10.1609/aaai.v40i37.40355 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Kang He und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref).
Abstract (Originalsprache)
Multimodal Sentiment Analysis (MSA) seeks to understand human emotions by integrating textual, acoustic, and visual signals. Although multimodal fusion is designed to leverage cross-modal complementarity, real-world scenarios often exhibit modality competition: dominant modalities tend to overshadow weaker ones, leading to suboptimal performance. In this paper, we propose PaSE, a novel Prototype-aligned Calibration and Shapley-optimized Equilibrium framework, which enhances collaboration while explicitly mitigating modality competition. PaSE first applies Prototype-guided Calibration Learning (PCL) to refine unimodal representations and align them through an Entropic Optimal Transport mechanism that ensures semantic consistency. To further stabilize optimization, we introduce a Dual-Phase Optimization strategy. A prototype-gated fusion module is first used to extract shared representations, followed by Shapley-based Gradient Modulation (SGM), which adaptively adjusts gradients according to the contribution of each modality. Extensive experiments on IEMOCAP, MOSI, and MOSEI confirm that PaSE achieves the superior performance and effectively alleviates modality competition.
Zitieren
APA
He, K., Chen, B., Ding, Y., Li, F., Teng, C., & Ji, D. (2026). PaSE: Prototype-aligned Calibration and Shapley-based Equilibrium for Multimodal Sentiment Analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 30960-30968. https://doi.org/10.1609/aaai.v40i37.40355
Vancouver
He K, Chen B, Ding Y, Li F, Teng C, Ji D. PaSE: Prototype-aligned Calibration and Shapley-based Equilibrium for Multimodal Sentiment Analysis. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):30960-30968. doi:10.1609/aaai.v40i37.40355
BibTeX
@article{He2026,
author = {He, Kang and Chen, Boyu and Ding, Yuzhe and Li, Fei and Teng, Chong and Ji, Donghong},
title = {PaSE: Prototype-aligned Calibration and Shapley-based Equilibrium for Multimodal Sentiment Analysis},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {37},
pages = {30960--30968},
doi = {10.1609/aaai.v40i37.40355}
}
EUVIMED