Noise-Aware Causal Orthogonal Gating Network for Robust and Interpretable Multimodal Sentiment Analysis
Frontiers in Computing and Intelligent Systems · 01.06.2026 · 16 (2) · 181-192
DOI: 10.54097/qy8p7839 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Computing and Intelligent Systems, online erschienen am 01.06.2026. Verfasst von Junrui Li und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Across online social platforms, the sheer expansion of multimodal content has made Multimodal Sentiment Analysis (MSA) both indispensable and technically difficult within computational social systems. Cross-modal heterogeneity is the core of the whole problem: textual, visual, and acoustic data streams almost never align completely in a clean, simple way. On top of this basic mismatch, things like noise, missing types of modality, and wrong inter-modal connections end up hurting both how reliable existing models are, and also how easy they are to understand. Given all these issues, this study puts forward the Noise-Aware Causal Orthogonal Gating Network (NCOG), which is a new framework made to add causal inference theory into adaptive multimodal fusion work. It does not use the traditional unimodal nonlinear gating that most methods rely on; instead, NCOG splits the gating work into two separate submodules that are orthogonal but can work together to complement each other. One of these submodules is called the Reliability Gate, and it filters signals using clear noise proxies — things like inter-frame variation, pitch energy fluctuation, and ASR confidence. The other one, named the Causal Contribution Gate, does not work the same way as this front-end screening step. It uses timestep-level counterfactual masking to work out the real causal effects between different modalities, and at the same time it gets rid of spurious correlations that do not mean anything. Along with this two-gate design, there is also a cross-modal attention module that dynamically adjusts how much weight each modality gets. When the input data has noise or some parts are missing, this module also makes the model more stable and harder to break. When we tested it on real data, NCOG-MSA got 88.12 percent accuracy, 87.11 percent F1 score, 0.681 for MAE, and 0.809 for PCC on the CMU-MOSI dataset. For the CMU-MOSEI dataset, the corresponding numbers are 87.44 percent accuracy, 89.04 percent F1, 0.483 MAE, and 0.816 PCC. If we compare it with the best existing fusion baselines, which include ACMG, TMFN, and HCAN, our framework shows better results on every single one of these evaluation metrics. Putting all these test results together, we can see that combining explicit noise modeling and causal reasoning makes multimodal sentiment analysis more robust, easier to understand, and works better on new data. This also gives a solid basic framework for building affective intelligence that can work in complex real-world social systems.
Zitieren
APA
Li, J., Chen, J., Li, C., & Zhang, W. (2026). Noise-Aware Causal Orthogonal Gating Network for Robust and Interpretable Multimodal Sentiment Analysis. Frontiers in Computing and Intelligent Systems, 16(2), 181-192. https://doi.org/10.54097/qy8p7839
Vancouver
Li J, Chen J, Li C, Zhang W. Noise-Aware Causal Orthogonal Gating Network for Robust and Interpretable Multimodal Sentiment Analysis. Frontiers in Computing and Intelligent Systems. 2026;16(2):181-192. doi:10.54097/qy8p7839
BibTeX
@article{Li2026,
author = {Li, Junrui and Chen, Jing and Li, Chang and Zhang, Wei},
title = {Noise-Aware Causal Orthogonal Gating Network for Robust and Interpretable Multimodal Sentiment Analysis},
journal = {Frontiers in Computing and Intelligent Systems},
year = {2026},
volume = {16},
number = {2},
pages = {181--192},
doi = {10.54097/qy8p7839}
}
EUVIMED