Image–Text Multimodal Sentiment Analysis with Large Model-Generated Descriptive Semantics and Difference-Aware Gated Fusion
Electronics · 15.08.2026 · 15 (16) · 3639
DOI: 10.3390/electronics15163639 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 15.08.2026. Verfasst von Hengyuan Zhang und 4 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Xinjiang Uygur Autonomous Region Innovation Environment (Talent and Base) Construction Special Program—Natural Science Program, Undergraduate Education and Teaching Research and Reform Project of Universities in the Xinjiang Uygur Autonomous Region, Xinjiang Normal University.
Abstract (Originalsprache)
Image–text multimodal sentiment analysis aims to integrate textual and visual information to comprehensively understand sentiment expressions in complex scenarios. However, existing methods focus on cross-modal feature interaction and fusion, and still have difficulty capturing effective sentiment cues in scenarios involving insufficient textual semantics, implicit visual affective cues, and inconsistent sentiment expressions between text and image. To address these issues, this paper proposes an image–text multimodal sentiment analysis method with large model-generated descriptive semantics and difference-aware gated fusion. Specifically, a large model generates semantic descriptions for image–text pairs, from which an enhanced semantic view is constructed to supplement implicit or insufficiently expressed sentiment cues in the original modalities. An original-enhanced dual-branch structure models the original image–text evidence and enhanced semantic evidence separately. To improve semantic consistency between the two branches, a cross-branch semantic alignment mechanism is introduced to reduce semantic shifts caused by enhanced information. In the fusion stage, difference-aware gated fusion and residual compensation are employed to adaptively balance branch contributions while preserving discriminative branch differences. Experimental results on the MVSA-Single and MVSA-Multiple datasets show that the proposed method improves performance in image–text multimodal sentiment classification, thereby validating the effectiveness of combining semantic enhancement with difference-aware modeling.
Förderung
- Xinjiang Uygur Autonomous Region Innovation Environment (Talent and Base) Construction Special Program—Natural Science Program
- Undergraduate Education and Teaching Research and Reform Project of Universities in the Xinjiang Uygur Autonomous Region
- Xinjiang Normal University
- National Natural Science Foundation of China
- National Social Science Foundation of China
Zitieren
APA
Zhang, H., Yusufu, A., Liu, J., Ainiwaer, A., & Yusufu, A. (2026). Image–Text Multimodal Sentiment Analysis with Large Model-Generated Descriptive Semantics and Difference-Aware Gated Fusion. Electronics, 15(16), 3639. https://doi.org/10.3390/electronics15163639
Vancouver
Zhang H, Yusufu A, Liu J, Ainiwaer A, Yusufu A. Image–Text Multimodal Sentiment Analysis with Large Model-Generated Descriptive Semantics and Difference-Aware Gated Fusion. Electronics. 2026;15(16):3639. doi:10.3390/electronics15163639
BibTeX
@article{Zhang2026,
author = {Zhang, Hengyuan and Yusufu, Aizihaierjiang and Liu, Jiang and Ainiwaer, Abidan and Yusufu, Aizierguli},
title = {Image–Text Multimodal Sentiment Analysis with Large Model-Generated Descriptive Semantics and Difference-Aware Gated Fusion},
journal = {Electronics},
year = {2026},
volume = {15},
number = {16},
pages = {3639},
doi = {10.3390/electronics15163639}
}
EUVIMED