A modality-aware contrastive learning framework for multimodal sentiment analysis
PeerJ Computer Science · 11.06.2026 · 12 · e3887
DOI: 10.7717/peerj-cs.3887 ↗Kurz gefasst
Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 11.06.2026. Verfasst von Lusha Zhu. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Scientific Research Project of the Hunan Provincial Department of Education.
Abstract (Originalsprache)
Multimodal sentiment analysis (MSA) is essential for human-computer interaction. It combines textual, visual, and acoustic signals to improve the precision of emotion recognition. Despite recent advances, current methods still struggle with fine-grained feature fusion and insufficient modelling of complex cross-modal relationships. To address these challenges, we propose a novel Multimodal-Aware Contrastive Learning (MACL) framework. MACL proposes a Dynamic Multi-Scale Attention (DMSA) mechanism that adaptively captures multi-level temporal and spatial features within each modality, thereby enhancing the fidelity of intra-modal feature representations and improving sensitivity to subtle emotional cues. MACL incorporates a Modality-Aware Representation Learning (MARL) module that jointly learns both modality-shared and modality-specific representations, enabling the model to preserve fine-grained local details while aligning global semantic information across heterogeneous modalities. Furthermore, an Information Noise-Contrastive Estimation (InfoNCE)-based contrastive learning strategy is incorporated to maintain semantic consistency. Experimental results on the benchmark CMU Multimodal Opinion-Level Sentiment Intensity(CMU-MOSI) and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) datasets demonstrate that MACL consistently outperforms existing state-of-the-art approaches, validating its robustness and superior generalization.
Förderung
- Scientific Research Project of the Hunan Provincial Department of Education
Zitieren
APA
Zhu, L. (2026). A modality-aware contrastive learning framework for multimodal sentiment analysis. PeerJ Computer Science, 12, e3887. https://doi.org/10.7717/peerj-cs.3887
Vancouver
Zhu L. A modality-aware contrastive learning framework for multimodal sentiment analysis. PeerJ Computer Science. 2026;12:e3887. doi:10.7717/peerj-cs.3887
BibTeX
@article{Zhu2026,
author = {Zhu, Lusha},
title = {A modality-aware contrastive learning framework for multimodal sentiment analysis},
journal = {PeerJ Computer Science},
year = {2026},
volume = {12},
pages = {e3887},
doi = {10.7717/peerj-cs.3887}
}
EUVIMED