CrossSent: Cross-Modal Attention with Pairwise Ranking Regularization for Multi-Modal Sentiment
Electronics · 11.03.2026 · 15 (6) · 1157
DOI: 10.3390/electronics15061157 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 11.03.2026. Verfasst von Jiaxiong Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Multi-modal sentiment analysis (MSA) aims to accurately identify users’ emotional states by integrating textual, acoustic, and visual modalities. However, existing methods often suffer from insufficient cross-modal interaction, rigid fusion strategies, and limited sensitivity to subtle sentiment-level differences, which severely restrict model generalization and robustness. To address these issues, this paper proposes CrossSent, a multi-modal sentiment analysis framework that combines cross-modal attention with pairwise ranking regularization. Specifically, a Gated Multi-modal Residual Adapter (GMRA) is introduced to dynamically integrate heterogeneous features through gated residual connections, effectively mitigating modality asynchrony and noise interference. Meanwhile, a Monotonic Pairwise Ranking (MPR) regularization enhances discrimination among fine-grained sentiment levels. Furthermore, an Error-Interval Ordinal Inconsistency (EIOI) loss is designed to tolerate small prediction deviations, improving both stability and robustness. Experimental results on CMU-MOSI, CMU-MOSEI, and CH-SIMS demonstrate that CrossSent consistently surpasses state-of-the-art baselines across key metrics. For instance, it achieves 89.78% binary accuracy and 52.1% seven-class accuracy on CMU-MOSI, 87.72% and 54.7% on CMU-MOSEI, and 80.41%, 62.36%, and 43.54% for three- and five-level CH-SIMS tasks, with reduced mean absolute errors of 0.563, 0.513, and 0.408, respectively. We further report ordinal-consistency measures (QWK and level-jump statistics) to complement conventional metrics and quantify level-wise agreement. These results validate the effectiveness and generalization capability of the proposed framework.
Zitieren
APA
Liu, J., Qi, K., Liao, Z., Yuan, F., & Zhuo, W. (2026). CrossSent: Cross-Modal Attention with Pairwise Ranking Regularization for Multi-Modal Sentiment. Electronics, 15(6), 1157. https://doi.org/10.3390/electronics15061157
Vancouver
Liu J, Qi K, Liao Z, Yuan F, Zhuo W. CrossSent: Cross-Modal Attention with Pairwise Ranking Regularization for Multi-Modal Sentiment. Electronics. 2026;15(6):1157. doi:10.3390/electronics15061157
BibTeX
@article{Liu2026,
author = {Liu, Jiaxiong and Qi, Ke and Liao, Zhiwen and Yuan, Feixiang and Zhuo, Wen},
title = {CrossSent: Cross-Modal Attention with Pairwise Ranking Regularization for Multi-Modal Sentiment},
journal = {Electronics},
year = {2026},
volume = {15},
number = {6},
pages = {1157},
doi = {10.3390/electronics15061157}
}
EUVIMED