Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.01.2025 · 21 (1) · 1-19
DOI: 10.1145/3702996 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 23.12.2024. Verfasst von Nan Wang und 1 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 49 Literaturangaben. Gefördert von: National Social Science Foundation of China.
Abstract (Originalsprache)
Advancements in Multimodal Sentiment Analysis (MSA) have predominantly focused on leveraging the interdependence of text, acoustic, and visual modalities to enhance sentiment prediction. However, efficiently integrating these modalities remains a challenge. In response, we investigate the design of a practical approach to control the information flow more accurately and propose DWGCI. This multimodal model employs systematic cross-modal interactions and gating mechanisms to analyze sentiment. It ensures comprehensive feature extraction across modalities by utilizing BERT for text features, COVAREP for acoustic features, and CNNs for visual features. To improve the common feature expression of each modality and increase the dynamic interaction between modalities, we introduced a Text-Guided Multimodal Fusion Module (TGMFM). Another integral part of our model is a unique Dynamic Gating Module (DGM) strategically positioned to follow cross-modal interactions. This mechanism represents a significant innovation that can adapt to modality-specific sentiment cues with unprecedented precision dynamically. The model can make nuanced distinctions between modalities, which significantly enhances the accuracy of sentiment prediction. Our DWGCI has been subjected to rigorous testing on two widely used datasets. It demonstrated its superior performance, established a benchmark in MSA for gate fusion innovation, and highlighted its transformative potential.
Förderung
- National Social Science Foundation of China
Zitieren
APA
Wang, N., & Wang, Q. (2025). Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications, 21(1), 1-19. https://doi.org/10.1145/3702996
Vancouver
Wang N, Wang Q. Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications. 2025;21(1):1-19. doi:10.1145/3702996
BibTeX
@article{Wang2025,
author = {Wang, Nan and Wang, Qi},
title = {Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2025},
volume = {21},
number = {1},
pages = {1--19},
doi = {10.1145/3702996}
}
EUVIMED