EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion

Huiru Wang, Xiuhong Li, Zenyu Ren, Min Wang, Chunming Ma

Sensors · 01.03.2023 · 23 (5) · 2679

DOI: 10.3390/s23052679 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 01.03.2023. Verfasst von Huiru Wang und 4 weiteren Autor:innen. Beteiligte Einrichtung: Xinjiang Key Laboratory of Signal Detection and Processing, College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China. Die Arbeit wurde bisher 38-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, the open project of key laboratory, Xinjiang Uygur Autonomous Region, key laboratory, Xinjiang Uygur Autonomous Region.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal sentiment analysis has gained popularity as a research field for its ability to predict users’ emotional tendencies more comprehensively. The data fusion module is a critical component of multimodal sentiment analysis, as it allows for integrating information from multiple modalities. However, it is challenging to combine modalities and remove redundant information effectively. In our research, we address these challenges by proposing a multimodal sentiment analysis model based on supervised contrastive learning, which leads to more effective data representation and richer multimodal features. Specifically, we introduce the MLFC module, which utilizes a convolutional neural network (CNN) and Transformer to solve the redundancy problem of each modal feature and reduce irrelevant information. Moreover, our model employs supervised contrastive learning to enhance its ability to learn standard sentiment features from data. We evaluate our model on three widely-used datasets, namely MVSA-single, MVSA-multiple, and HFM, demonstrating that our model outperforms the state-of-the-art model. Finally, we conduct ablation experiments to validate the efficacy of our proposed method.

Schlagwörter

Stichwörter der Autor:innen

MLFCSCSupConmultimodalmultimodal sentiment analysissupervised contrastive learning

Autor:innen und Institutionen

  • Xinjiang Key Laboratory of Signal Detection and Processing, College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China.
  • College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China.

Förderung

  • National Natural Science Foundation of China
  • the open project of key laboratory, Xinjiang Uygur Autonomous Region
  • key laboratory, Xinjiang Uygur Autonomous Region
  • National Science Foundation of China

Zitieren

APA

Wang, H., Li, X., Ren, Z., Wang, M., & Ma, C. (2023). Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion. Sensors, 23(5), 2679. https://doi.org/10.3390/s23052679

Vancouver

Wang H, Li X, Ren Z, Wang M, Ma C. Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion. Sensors. 2023;23(5):2679. doi:10.3390/s23052679

BibTeX
@article{Wang2023,
  author = {Wang, Huiru and Li, Xiuhong and Ren, Zenyu and Wang, Min and Ma, Chunming},
  title = {Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {5},
  pages = {2679},
  doi = {10.3390/s23052679}
}
Kostenlos starten →