EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

A Dynamic Multimodal Causal Graph framework for standardized Emotion Recognition in Conversations

Ronghao Pan, José Antonio García-Díaz, Rafael Valencia-García

Computer Standards & Interfaces · 2026-04 · 97 · 104130

DOI: 10.1016/j.csi.2026.104130 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computer Standards & Interfaces, 2026-04. Verfasst von Ronghao Pan und 2 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: European Regional Development Fund, Ministerio de Ciencia e Innovación, State Agency of Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Understanding emotions in conversations is a fundamental challenge in affective computing. Emotional expressions evolve dynamically across dialogue turns and depend on multimodal cues such as speech, text, and facial behavior. However, existing multimodal models often rely on global attention mechanisms that overlook causal constraints. This allows information leakage from future turns and neglect of the speaker’s emotional evolution. To address these limitations, we propose the Dynamic Multimodal Causal Graph Emotion System (DMCGES). DMCGES integrates a restricted dynamic causal graph to ensure temporal coherence, as well as a speaker-specific memory module to capture affective trajectories and enhance multimodal alignment and robustness. The framework aligns with the IEEE 7010-2020 standard, which emphasizes integrating human well-being as a fundamental design principle in autonomous and intelligent systems. Experiments on the IEMOCAP and MELD benchmark datasets demonstrate that DMCGES outperforms state-of-the-art approaches in terms of accuracy and F1 score. On the IEMOCAP dataset, DMCGES achieved an accuracy of 69.36% and an F1 score of 69.49%, representing relative improvements of 1.95% and 2.39%, respectively. On the MELD dataset, our model achieved an accuracy of 62.38% and an F1 score of 62.03%, improving upon SACCMA’s results by 0.1% in accuracy and 2.73% in F1 score. • Creation of a Dynamic Multimodal Causal Graph for Emotion Recognition in Conversations. • Evaluation of textual, audio, and video features for speech emotion recognition. • Use of a standardized, JSONL-based, multimodal interface for reproducibility and interoperability. • Evaluation using two benchmark datasets: IEMOCAP and MELD. • DMCGES improves overall performance compared to unimodal, bimodal, and other state-of-the-art approaches.

Förderung

  • European Regional Development Fund
  • Ministerio de Ciencia e Innovación
  • State Agency of Research

Zitieren

APA

Pan, R., García-Díaz, J. A., & Valencia-García, R. (2026). A Dynamic Multimodal Causal Graph framework for standardized Emotion Recognition in Conversations. Computer Standards & Interfaces, 97, 104130. https://doi.org/10.1016/j.csi.2026.104130

Vancouver

Pan R, García-Díaz JA, Valencia-García R. A Dynamic Multimodal Causal Graph framework for standardized Emotion Recognition in Conversations. Computer Standards & Interfaces. 2026;97:104130. doi:10.1016/j.csi.2026.104130

BibTeX
@article{Pan2026,
  author = {Pan, Ronghao and García-Díaz, José Antonio and Valencia-García, Rafael},
  title = {A Dynamic Multimodal Causal Graph framework for standardized Emotion Recognition in Conversations},
  journal = {Computer Standards & Interfaces},
  year = {2026},
  volume = {97},
  pages = {104130},
  doi = {10.1016/j.csi.2026.104130}
}
Kostenlos starten →