A Dynamic Multimodal Causal Graph framework for standardized Emotion Recognition in Conversations
Computer Standards & Interfaces · 2026-04 · 97 · 104130
DOI: 10.1016/j.csi.2026.104130 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computer Standards & Interfaces, 2026-04. Verfasst von Ronghao Pan und 2 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: European Regional Development Fund, Ministerio de Ciencia e Innovación, State Agency of Research.
Abstract (Originalsprache)
Understanding emotions in conversations is a fundamental challenge in affective computing. Emotional expressions evolve dynamically across dialogue turns and depend on multimodal cues such as speech, text, and facial behavior. However, existing multimodal models often rely on global attention mechanisms that overlook causal constraints. This allows information leakage from future turns and neglect of the speaker’s emotional evolution. To address these limitations, we propose the Dynamic Multimodal Causal Graph Emotion System (DMCGES). DMCGES integrates a restricted dynamic causal graph to ensure temporal coherence, as well as a speaker-specific memory module to capture affective trajectories and enhance multimodal alignment and robustness. The framework aligns with the IEEE 7010-2020 standard, which emphasizes integrating human well-being as a fundamental design principle in autonomous and intelligent systems. Experiments on the IEMOCAP and MELD benchmark datasets demonstrate that DMCGES outperforms state-of-the-art approaches in terms of accuracy and F1 score. On the IEMOCAP dataset, DMCGES achieved an accuracy of 69.36% and an F1 score of 69.49%, representing relative improvements of 1.95% and 2.39%, respectively. On the MELD dataset, our model achieved an accuracy of 62.38% and an F1 score of 62.03%, improving upon SACCMA’s results by 0.1% in accuracy and 2.73% in F1 score. • Creation of a Dynamic Multimodal Causal Graph for Emotion Recognition in Conversations. • Evaluation of textual, audio, and video features for speech emotion recognition. • Use of a standardized, JSONL-based, multimodal interface for reproducibility and interoperability. • Evaluation using two benchmark datasets: IEMOCAP and MELD. • DMCGES improves overall performance compared to unimodal, bimodal, and other state-of-the-art approaches.
Förderung
- European Regional Development Fund
- Ministerio de Ciencia e Innovación
- State Agency of Research
Zitieren
APA
Pan, R., García-Díaz, J. A., & Valencia-García, R. (2026). A Dynamic Multimodal Causal Graph framework for standardized Emotion Recognition in Conversations. Computer Standards & Interfaces, 97, 104130. https://doi.org/10.1016/j.csi.2026.104130
Vancouver
Pan R, García-Díaz JA, Valencia-García R. A Dynamic Multimodal Causal Graph framework for standardized Emotion Recognition in Conversations. Computer Standards & Interfaces. 2026;97:104130. doi:10.1016/j.csi.2026.104130
BibTeX
@article{Pan2026,
author = {Pan, Ronghao and García-Díaz, José Antonio and Valencia-García, Rafael},
title = {A Dynamic Multimodal Causal Graph framework for standardized Emotion Recognition in Conversations},
journal = {Computer Standards & Interfaces},
year = {2026},
volume = {97},
pages = {104130},
doi = {10.1016/j.csi.2026.104130}
}
EUVIMED