EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis

Wenjuan Gong, Jiarui Li, Tingbo Shi, Chunhan Li, Chenglizhao Chen, José M. Álvarez

ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2026 · 22 (11) · 1-23

DOI: 10.1145/3829080 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 11.11.2026. Verfasst von Wenjuan Gong und 5 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Natural Science Foundation of Shandong Province, Shandong Data Open Innovative Application Laboratory.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sentiment analysis, crucial in fields like human–computer interaction and mental health, has advanced through multimodal fusion and large-scale language models. People exhibit significant variability in emotional expressions due to behavioral differences, yet existing methods often apply generic models across populations. By focusing primarily on objective features, these models overlook “personal” cues that may lead to inconsistencies between the extracted features and the corresponding emotion categories. Additionally, at the common level, this issue is aggravated by the accumulation of personal cues mapped to a common emotion category. Existing approaches tend to fuse these cues indiscriminately, whereas features that align consistently with the target emotion category should be emphasized. The attention mechanism, as the dominant fusion method, cannot adjust the misallocated weights it assigns. To address these limitations, we propose a “personal behavior-robust” sentiment analysis approach that disentangles personal cues from common ones through decoupling and learns the relative importance of features across modalities via a corrective feedback-based “interpretable” fusion strategy to adjust misallocated weights assigned by the attention mechanism. The proposed model effectively differentiates and prioritizes relevant information from multiple cues, guiding the model to focus on the cue that aligns with the overall emotion. Experimental results demonstrate that the proposed methods achieve good performance on two public datasets: CMU-MOSI and CMU-MOSEI.

Förderung

  • Natural Science Foundation of Shandong Province
  • Shandong Data Open Innovative Application Laboratory

Zitieren

APA

Gong, W., Li, J., Shi, T., Li, C., Chen, C., & Álvarez, J. M. (2026). Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(11), 1-23. https://doi.org/10.1145/3829080

Vancouver

Gong W, Li J, Shi T, Li C, Chen C, Álvarez JM. Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(11):1-23. doi:10.1145/3829080

BibTeX
@article{Gong2026,
  author = {Gong, Wenjuan and Li, Jiarui and Shi, Tingbo and Li, Chunhan and Chen, Chenglizhao and Álvarez, José M.},
  title = {Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2026},
  volume = {22},
  number = {11},
  pages = {1--23},
  doi = {10.1145/3829080}
}
Kostenlos starten →