EUVIMEDEUVIMED entdecken
Crossref

Enriching Multimodal Sentiment Analysis Through Textual Emotional Descriptions of Visual-Audio Content

Sheng Wu, Dongxiao He, Xiaobao Wang, Longbiao Wang, Jianwu Dang

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (2) · 1601-1609

DOI: 10.1609/aaai.v39i2.32152 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Sheng Wu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 68-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal Sentiment Analysis (MSA) stands as a critical research frontier, seeking to comprehensively unravel human emotions by amalgamating text, audio, and visual data. Yet, discerning subtle emotional nuances within audio and video expressions poses a formidable challenge, particularly when emotional polarities across various segments appear similar. In this paper, our objective is to spotlight emotion-relevant attributes of audio and visual modalities to facilitate multimodal fusion in the context of nuanced emotional shifts in visual-audio scenarios. To this end, we introduce DEVA, a progressive fusion framework founded on textual sentiment descriptions aimed at accentuating emotional features of visual-audio content. DEVA employs an Emotional Description Generator (EDG) to transmute raw audio and visual data into textualized sentiment descriptions, thereby amplifying their emotional characteristics. These descriptions are then integrated with the source data to yield richer, enhanced features. Furthermore, DEVA incorporates the Text-guided Progressive Fusion Module (TPF), leveraging varying levels of text as a core modality guide. This module progressively fuses visual-audio minor modalities to alleviate disparities between text and visual-audio modalities. Experimental results on widely used sentiment analysis benchmark datasets, including MOSI, MOSEI, and CH-SIMS, underscore significant enhancements compared to state-of-the-art models. Moreover, fine-grained emotion experiments corroborate the robust sensitivity of DEVA to subtle emotional variations.

Zitieren

APA

Wu, S., He, D., Wang, X., Wang, L., & Dang, J. (2025). Enriching Multimodal Sentiment Analysis Through Textual Emotional Descriptions of Visual-Audio Content. Proceedings of the AAAI Conference on Artificial Intelligence, 39(2), 1601-1609. https://doi.org/10.1609/aaai.v39i2.32152

Vancouver

Wu S, He D, Wang X, Wang L, Dang J. Enriching Multimodal Sentiment Analysis Through Textual Emotional Descriptions of Visual-Audio Content. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(2):1601-1609. doi:10.1609/aaai.v39i2.32152

BibTeX
@article{Wu2025,
  author = {Wu, Sheng and He, Dongxiao and Wang, Xiaobao and Wang, Longbiao and Dang, Jianwu},
  title = {Enriching Multimodal Sentiment Analysis Through Textual Emotional Descriptions of Visual-Audio Content},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {2},
  pages = {1601--1609},
  doi = {10.1609/aaai.v39i2.32152}
}
Kostenlos starten →