Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion
Sensors · 01.03.2023 · 23 (5) · 2679
DOI: 10.3390/s23052679 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 01.03.2023. Verfasst von Huiru Wang und 4 weiteren Autor:innen. Beteiligte Einrichtung: Xinjiang Key Laboratory of Signal Detection and Processing, College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China. Die Arbeit wurde bisher 38-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, the open project of key laboratory, Xinjiang Uygur Autonomous Region, key laboratory, Xinjiang Uygur Autonomous Region.
Abstract (Originalsprache)
Multimodal sentiment analysis has gained popularity as a research field for its ability to predict users’ emotional tendencies more comprehensively. The data fusion module is a critical component of multimodal sentiment analysis, as it allows for integrating information from multiple modalities. However, it is challenging to combine modalities and remove redundant information effectively. In our research, we address these challenges by proposing a multimodal sentiment analysis model based on supervised contrastive learning, which leads to more effective data representation and richer multimodal features. Specifically, we introduce the MLFC module, which utilizes a convolutional neural network (CNN) and Transformer to solve the redundancy problem of each modal feature and reduce irrelevant information. Moreover, our model employs supervised contrastive learning to enhance its ability to learn standard sentiment features from data. We evaluate our model on three widely-used datasets, namely MVSA-single, MVSA-multiple, and HFM, demonstrating that our model outperforms the state-of-the-art model. Finally, we conduct ablation experiments to validate the efficacy of our proposed method.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Xinjiang Key Laboratory of Signal Detection and Processing, College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China.
- College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China.
Förderung
- National Natural Science Foundation of China
- the open project of key laboratory, Xinjiang Uygur Autonomous Region
- key laboratory, Xinjiang Uygur Autonomous Region
- National Science Foundation of China
Zitieren
APA
Wang, H., Li, X., Ren, Z., Wang, M., & Ma, C. (2023). Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion. Sensors, 23(5), 2679. https://doi.org/10.3390/s23052679
Vancouver
Wang H, Li X, Ren Z, Wang M, Ma C. Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion. Sensors. 2023;23(5):2679. doi:10.3390/s23052679
BibTeX
@article{Wang2023,
author = {Wang, Huiru and Li, Xiuhong and Ren, Zenyu and Wang, Min and Ma, Chunming},
title = {Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion},
journal = {Sensors},
year = {2023},
volume = {23},
number = {5},
pages = {2679},
doi = {10.3390/s23052679}
}
EUVIMED