EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A modality-aware contrastive learning framework for multimodal sentiment analysis

Lusha Zhu

PeerJ Computer Science · 11.06.2026 · 12 · e3887

DOI: 10.7717/peerj-cs.3887 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 11.06.2026. Verfasst von Lusha Zhu. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Scientific Research Project of the Hunan Provincial Department of Education.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal sentiment analysis (MSA) is essential for human-computer interaction. It combines textual, visual, and acoustic signals to improve the precision of emotion recognition. Despite recent advances, current methods still struggle with fine-grained feature fusion and insufficient modelling of complex cross-modal relationships. To address these challenges, we propose a novel Multimodal-Aware Contrastive Learning (MACL) framework. MACL proposes a Dynamic Multi-Scale Attention (DMSA) mechanism that adaptively captures multi-level temporal and spatial features within each modality, thereby enhancing the fidelity of intra-modal feature representations and improving sensitivity to subtle emotional cues. MACL incorporates a Modality-Aware Representation Learning (MARL) module that jointly learns both modality-shared and modality-specific representations, enabling the model to preserve fine-grained local details while aligning global semantic information across heterogeneous modalities. Furthermore, an Information Noise-Contrastive Estimation (InfoNCE)-based contrastive learning strategy is incorporated to maintain semantic consistency. Experimental results on the benchmark CMU Multimodal Opinion-Level Sentiment Intensity(CMU-MOSI) and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) datasets demonstrate that MACL consistently outperforms existing state-of-the-art approaches, validating its robustness and superior generalization.

Förderung

  • Scientific Research Project of the Hunan Provincial Department of Education

Zitieren

APA

Zhu, L. (2026). A modality-aware contrastive learning framework for multimodal sentiment analysis. PeerJ Computer Science, 12, e3887. https://doi.org/10.7717/peerj-cs.3887

Vancouver

Zhu L. A modality-aware contrastive learning framework for multimodal sentiment analysis. PeerJ Computer Science. 2026;12:e3887. doi:10.7717/peerj-cs.3887

BibTeX
@article{Zhu2026,
  author = {Zhu, Lusha},
  title = {A modality-aware contrastive learning framework for multimodal sentiment analysis},
  journal = {PeerJ Computer Science},
  year = {2026},
  volume = {12},
  pages = {e3887},
  doi = {10.7717/peerj-cs.3887}
}
Kostenlos starten →