EUVIMEDEUVIMED entdecken
Crossref

Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis

Nan Wang, Qi Wang

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.01.2025 · 21 (1) · 1-19

DOI: 10.1145/3702996 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 23.12.2024. Verfasst von Nan Wang und 1 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 49 Literaturangaben. Gefördert von: National Social Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Advancements in Multimodal Sentiment Analysis (MSA) have predominantly focused on leveraging the interdependence of text, acoustic, and visual modalities to enhance sentiment prediction. However, efficiently integrating these modalities remains a challenge. In response, we investigate the design of a practical approach to control the information flow more accurately and propose DWGCI. This multimodal model employs systematic cross-modal interactions and gating mechanisms to analyze sentiment. It ensures comprehensive feature extraction across modalities by utilizing BERT for text features, COVAREP for acoustic features, and CNNs for visual features. To improve the common feature expression of each modality and increase the dynamic interaction between modalities, we introduced a Text-Guided Multimodal Fusion Module (TGMFM). Another integral part of our model is a unique Dynamic Gating Module (DGM) strategically positioned to follow cross-modal interactions. This mechanism represents a significant innovation that can adapt to modality-specific sentiment cues with unprecedented precision dynamically. The model can make nuanced distinctions between modalities, which significantly enhances the accuracy of sentiment prediction. Our DWGCI has been subjected to rigorous testing on two widely used datasets. It demonstrated its superior performance, established a benchmark in MSA for gate fusion innovation, and highlighted its transformative potential.

Förderung

  • National Social Science Foundation of China

Zitieren

APA

Wang, N., & Wang, Q. (2025). Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications, 21(1), 1-19. https://doi.org/10.1145/3702996

Vancouver

Wang N, Wang Q. Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications. 2025;21(1):1-19. doi:10.1145/3702996

BibTeX
@article{Wang2025,
  author = {Wang, Nan and Wang, Qi},
  title = {Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2025},
  volume = {21},
  number = {1},
  pages = {1--19},
  doi = {10.1145/3702996}
}
Kostenlos starten →