EUVIMEDEUVIMED entdecken
Crossref

Relational graph-driven differential denoising and diffusion attention fusion for multimodal conversational emotion recognition

Ying Liu, Yuntao Shou, Wei Ai, Tao Meng, Keqin Li

Neurocomputing · 2026-06 · 681 · 133306

DOI: 10.1016/j.neucom.2026.133306 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neurocomputing, 2026-06. Verfasst von Ying Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 48 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In real-world scenarios, audio and video signals are often subject to environmental noise and limited acquisition conditions, resulting in extracted features containing excessive noise. Furthermore, there is an imbalance in data quality and information carrying capacity between different modalities. These two issues together lead to information distortion and weight bias during the fusion phase, impairing overall recognition performance. Most existing methods neglect the impact of noisy modalities and rely on implicit weighting to model modality importance, thereby failing to explicitly account for the predominant contribution of the textual modality in emotion understanding. To address these issues, we propose a relation-aware denoising and diffusion attention fusion model for MCER. Specifically, we first design a differential Transformer that explicitly computes the differences between two attention maps, thereby enhancing temporally consistent information while suppressing time-irrelevant noise, which leads to effective denoising in both audio and video modalities. Second, we construct modality-specific and cross-modality relation subgraphs to capture speaker-dependent emotional dependencies, enabling fine-grained modeling of intra- and inter-modal relationships. Finally, we introduce a text-guided cross-modal diffusion mechanism that leverages self-attention to model intra-modal dependencies and adaptively diffuses audiovisual information into the textual stream, ensuring more robust and semantically aligned multimodal fusion.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Liu, Y., Shou, Y., Ai, W., Meng, T., & Li, K. (2026). Relational graph-driven differential denoising and diffusion attention fusion for multimodal conversational emotion recognition. Neurocomputing, 681, 133306. https://doi.org/10.1016/j.neucom.2026.133306

Vancouver

Liu Y, Shou Y, Ai W, Meng T, Li K. Relational graph-driven differential denoising and diffusion attention fusion for multimodal conversational emotion recognition. Neurocomputing. 2026;681:133306. doi:10.1016/j.neucom.2026.133306

BibTeX
@article{Liu2026,
  author = {Liu, Ying and Shou, Yuntao and Ai, Wei and Meng, Tao and Li, Keqin},
  title = {Relational graph-driven differential denoising and diffusion attention fusion for multimodal conversational emotion recognition},
  journal = {Neurocomputing},
  year = {2026},
  volume = {681},
  pages = {133306},
  doi = {10.1016/j.neucom.2026.133306}
}
Kostenlos starten →