EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CrossSent: Cross-Modal Attention with Pairwise Ranking Regularization for Multi-Modal Sentiment

Jiaxiong Liu, Ke Qi, Zhiwen Liao, Feixiang Yuan, Wen Zhuo

Electronics · 11.03.2026 · 15 (6) · 1157

DOI: 10.3390/electronics15061157 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 11.03.2026. Verfasst von Jiaxiong Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-modal sentiment analysis (MSA) aims to accurately identify users’ emotional states by integrating textual, acoustic, and visual modalities. However, existing methods often suffer from insufficient cross-modal interaction, rigid fusion strategies, and limited sensitivity to subtle sentiment-level differences, which severely restrict model generalization and robustness. To address these issues, this paper proposes CrossSent, a multi-modal sentiment analysis framework that combines cross-modal attention with pairwise ranking regularization. Specifically, a Gated Multi-modal Residual Adapter (GMRA) is introduced to dynamically integrate heterogeneous features through gated residual connections, effectively mitigating modality asynchrony and noise interference. Meanwhile, a Monotonic Pairwise Ranking (MPR) regularization enhances discrimination among fine-grained sentiment levels. Furthermore, an Error-Interval Ordinal Inconsistency (EIOI) loss is designed to tolerate small prediction deviations, improving both stability and robustness. Experimental results on CMU-MOSI, CMU-MOSEI, and CH-SIMS demonstrate that CrossSent consistently surpasses state-of-the-art baselines across key metrics. For instance, it achieves 89.78% binary accuracy and 52.1% seven-class accuracy on CMU-MOSI, 87.72% and 54.7% on CMU-MOSEI, and 80.41%, 62.36%, and 43.54% for three- and five-level CH-SIMS tasks, with reduced mean absolute errors of 0.563, 0.513, and 0.408, respectively. We further report ordinal-consistency measures (QWK and level-jump statistics) to complement conventional metrics and quantify level-wise agreement. These results validate the effectiveness and generalization capability of the proposed framework.

Zitieren

APA

Liu, J., Qi, K., Liao, Z., Yuan, F., & Zhuo, W. (2026). CrossSent: Cross-Modal Attention with Pairwise Ranking Regularization for Multi-Modal Sentiment. Electronics, 15(6), 1157. https://doi.org/10.3390/electronics15061157

Vancouver

Liu J, Qi K, Liao Z, Yuan F, Zhuo W. CrossSent: Cross-Modal Attention with Pairwise Ranking Regularization for Multi-Modal Sentiment. Electronics. 2026;15(6):1157. doi:10.3390/electronics15061157

BibTeX
@article{Liu2026,
  author = {Liu, Jiaxiong and Qi, Ke and Liao, Zhiwen and Yuan, Feixiang and Zhuo, Wen},
  title = {CrossSent: Cross-Modal Attention with Pairwise Ranking Regularization for Multi-Modal Sentiment},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {6},
  pages = {1157},
  doi = {10.3390/electronics15061157}
}
Kostenlos starten →