Text-image semantic relevance identification for aspect-based multimodal sentiment analysis
PeerJ Computer Science · 12.04.2024 · 10 · e1904
DOI: 10.7717/peerj-cs.1904 ↗Kurz gefasst
Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 12.04.2024. Verfasst von Tianzhi Zhang und 5 weiteren Autor:innen. Beteiligte Einrichtung: Information Engineering University, Zhengzhou, Henan, China. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Department of Science and Technology of Henan Province.
Abstract (Originalsprache)
Aspect-based multimodal sentiment analysis (ABMSA) is an emerging task in the research of multimodal sentiment analysis, which aims to identify the sentiment of each aspect mentioned in multimodal sample. Although recent research on ABMSA has achieved some success, most existing models only adopt attention mechanism to interact aspect with text and image respectively and obtain sentiment output through multimodal concatenation, they often neglect to consider that some samples may not have semantic relevance between text and image. In this article, we propose a Text-Image Semantic Relevance Identification (TISRI) model for ABMSA to address the problem. Specifically, we introduce a multimodal feature relevance identification module to calculate the semantic similarity between text and image, and then construct an image gate to dynamically control the input image information. On this basis, an image auxiliary information is provided to enhance the semantic expression ability of visual feature representation to generate more intuitive image representation. Furthermore, we employ attention mechanism during multimodal feature fusion to obtain the text-aware image representation through text-image interaction to prevent irrelevant image information interfering our model. Experiments demonstrate that TISRI achieves competitive results on two ABMSA Twitter datasets, and then validate the effectiveness of our methods.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Information Engineering University, Zhengzhou, Henan, China.
Förderung
- Department of Science and Technology of Henan Province
Zitieren
APA
Zhang, T., Zhou, G., Lu, J., Li, Z., Wu, H., & Liu, S. (2024). Text-image semantic relevance identification for aspect-based multimodal sentiment analysis. PeerJ Computer Science, 10, e1904. https://doi.org/10.7717/peerj-cs.1904
Vancouver
Zhang T, Zhou G, Lu J, Li Z, Wu H, Liu S. Text-image semantic relevance identification for aspect-based multimodal sentiment analysis. PeerJ Computer Science. 2024;10:e1904. doi:10.7717/peerj-cs.1904
BibTeX
@article{Zhang2024,
author = {Zhang, Tianzhi and Zhou, Gang and Lu, Jicang and Li, Zhibo and Wu, Hao and Liu, Shuo},
title = {Text-image semantic relevance identification for aspect-based multimodal sentiment analysis},
journal = {PeerJ Computer Science},
year = {2024},
volume = {10},
pages = {e1904},
doi = {10.7717/peerj-cs.1904}
}
EUVIMED