EUVIMEDDescubrir EUVIMED
Open access · CC BY 4.0

Learning Adaptive Cross-Modal Interactions for Multimodal Sentiment Analysis

Chuhan Cheng, Hangcheng Wu, Junqiao Wang, Yuqi Ouyang

Data · 2026-08-20 · 11 (8) · 209

DOI: 10.3390/data11080209 ↗

At a glance

Journal article in Data, published online on 2026-08-20. Written by Chuhan Cheng and 3 co-authors. 61 references. The full text is freely available (open access, CC BY 4.0).

ResumenDatos bibliográficosCiteRelated studies

Abstract (original language)

Multimodal sentiment analysis aims to integrate heterogeneous textual, visual, and acoustic information for effective emotion understanding. However, existing methods often suffer from insufficient cross-modal interaction modeling, limited adaptability in multimodal fusion, and inadequate suppression of modality-specific noise under complex conversational scenarios. To address these challenges, this paper proposes a framework for learning adaptive cross-modal interactions for multimodal sentiment analysis. The proposed framework consists of three stages: modality-aware preprocessing, heterogeneous representation learning, and adaptive multimodal fusion. First, a unified preprocessing strategy is designed to improve cross-modal consistency through textual normalization, speaker-aware visual alignment, and utterance-level acoustic representation enhancement. Second, modality-specific encoders are constructed to capture complementary semantic, spatial, and utterance-level acoustic characteristics from textual, visual, and acoustic modalities, respectively. Third, an adaptive fusion framework is introduced to explicitly model cross-modal interactions, dynamically estimate the importance of different modality combinations, and further calibrate discriminative feature channels through channel attention. By jointly performing modality-level interaction learning and channel-wise feature refinement, the proposed framework effectively enhances multimodal representation capability for sentiment classification. Extensive experiments conducted on the CMU-MOSI and MELD benchmark datasets demonstrate that our framework consistently outperforms previous methods. In particular, the proposed model achieves 90.27% accuracy and 90.26% F1-score on CMU-MOSI, together with 66.57% accuracy and 66.21% F1-score on MELD. Additional ablation studies and qualitative analyses further validate the effectiveness of the proposed preprocessing strategy, modality-specific representation learning, and adaptive fusion mechanism.

Cite

APA

Cheng, C., Wu, H., Wang, J., & Ouyang, Y. (2026). Learning Adaptive Cross-Modal Interactions for Multimodal Sentiment Analysis. Data, 11(8), 209. https://doi.org/10.3390/data11080209

Vancouver

Cheng C, Wu H, Wang J, Ouyang Y. Learning Adaptive Cross-Modal Interactions for Multimodal Sentiment Analysis. Data. 2026;11(8):209. doi:10.3390/data11080209

BibTeX
@article{Cheng2026,
  author = {Cheng, Chuhan and Wu, Hangcheng and Wang, Junqiao and Ouyang, Yuqi},
  title = {Learning Adaptive Cross-Modal Interactions for Multimodal Sentiment Analysis},
  journal = {Data},
  year = {2026},
  volume = {11},
  number = {8},
  pages = {209},
  doi = {10.3390/data11080209}
}
Start for free →