EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Text-image semantic relevance identification for aspect-based multimodal sentiment analysis

Tianzhi Zhang, Gang Zhou, Jicang Lu, Zhibo Li, Hao Wu, Shuo Liu

PeerJ Computer Science · 12.04.2024 · 10 · e1904

DOI: 10.7717/peerj-cs.1904 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 12.04.2024. Verfasst von Tianzhi Zhang und 5 weiteren Autor:innen. Beteiligte Einrichtung: Information Engineering University, Zhengzhou, Henan, China. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Department of Science and Technology of Henan Province.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aspect-based multimodal sentiment analysis (ABMSA) is an emerging task in the research of multimodal sentiment analysis, which aims to identify the sentiment of each aspect mentioned in multimodal sample. Although recent research on ABMSA has achieved some success, most existing models only adopt attention mechanism to interact aspect with text and image respectively and obtain sentiment output through multimodal concatenation, they often neglect to consider that some samples may not have semantic relevance between text and image. In this article, we propose a Text-Image Semantic Relevance Identification (TISRI) model for ABMSA to address the problem. Specifically, we introduce a multimodal feature relevance identification module to calculate the semantic similarity between text and image, and then construct an image gate to dynamically control the input image information. On this basis, an image auxiliary information is provided to enhance the semantic expression ability of visual feature representation to generate more intuitive image representation. Furthermore, we employ attention mechanism during multimodal feature fusion to obtain the text-aware image representation through text-image interaction to prevent irrelevant image information interfering our model. Experiments demonstrate that TISRI achieves competitive results on two ABMSA Twitter datasets, and then validate the effectiveness of our methods.

Schlagwörter

Stichwörter der Autor:innen

Image auxiliary informationImage gateSemantic relevance identificationAspect-based multimodal sentiment analysis

Autor:innen und Institutionen

  • Information Engineering University, Zhengzhou, Henan, China.

Förderung

  • Department of Science and Technology of Henan Province

Zitieren

APA

Zhang, T., Zhou, G., Lu, J., Li, Z., Wu, H., & Liu, S. (2024). Text-image semantic relevance identification for aspect-based multimodal sentiment analysis. PeerJ Computer Science, 10, e1904. https://doi.org/10.7717/peerj-cs.1904

Vancouver

Zhang T, Zhou G, Lu J, Li Z, Wu H, Liu S. Text-image semantic relevance identification for aspect-based multimodal sentiment analysis. PeerJ Computer Science. 2024;10:e1904. doi:10.7717/peerj-cs.1904

BibTeX
@article{Zhang2024,
  author = {Zhang, Tianzhi and Zhou, Gang and Lu, Jicang and Li, Zhibo and Wu, Hao and Liu, Shuo},
  title = {Text-image semantic relevance identification for aspect-based multimodal sentiment analysis},
  journal = {PeerJ Computer Science},
  year = {2024},
  volume = {10},
  pages = {e1904},
  doi = {10.7717/peerj-cs.1904}
}
Kostenlos starten →