EUVIMEDEUVIMED entdecken
Crossref

Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognition

Jiang Li, Xiaoping Wang, Zhigang Zeng

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-10 · 47 (10) · 8786-8803

DOI: 10.1109/tpami.2025.3581236 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-10. Verfasst von Jiang Li und 2 weiteren Autor:innen. Hauptthemen laut MEDLINE: Emotions, Pattern Recognition, Automated, Natural Language Processing. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 66 Literaturangaben. Gefördert von: Interdisciplinary Research Program of HUST, National Natural Science Foundation of China, Outstanding Research Groups of Hubei Province of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal emotion recognition in conversation (MERC) has garnered substantial research attention recently. Existing MERC methods face several challenges: (1) they fail to fully harness direct inter-modal cues, possibly leading to less-than-thorough cross-modal modeling; (2) they concurrently extract information from the same and different modalities at each network layer, potentially triggering conflicts from the fusion of multi-source data; (3) they lack the agility required to detect dynamic sentimental changes, perhaps resulting in inaccurate classification of utterances with abrupt sentiment shifts. To address these issues, a novel approach named GraphSmile is proposed for tracking intricate emotional cues in multimodal dialogues. GraphSmile comprises two key components, i.e., GSF and SDP modules. GSF ingeniously leverages graph structures to alternately assimilate inter-modal and intra-modal emotional dependencies layer by layer, adequately capturing cross-modal cues while effectively circumventing fusion conflicts. SDP is an auxiliary task to explicitly delineate the sentiment dynamics between utterances, promoting the model's ability to distinguish sentimental discrepancies. GraphSmile is effortlessly applied to multimodal sentiment analysis in conversation (MSAC), thus enabling simultaneous execution of MERC and MSAC tasks. Empirical results on multiple benchmarks demonstrate that GraphSmile can handle complex emotional and sentimental patterns, significantly outperforming baseline models.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

EmotionsPattern Recognition, AutomatedNatural Language ProcessingHumansCuesAlgorithms

Förderung

  • Interdisciplinary Research Program of HUST
  • National Natural Science Foundation of China
  • Outstanding Research Groups of Hubei Province of China
  • 111 Project on Computational Intelligence and Intelligent Control
  • Fundamental Research Funds for the Central Universities of HUST

Zitieren

APA

Li, J., Wang, X., & Zeng, Z. (2025). Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(10), 8786-8803. https://doi.org/10.1109/tpami.2025.3581236

Vancouver

Li J, Wang X, Zeng Z. Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(10):8786-8803. doi:10.1109/tpami.2025.3581236

BibTeX
@article{Li2025,
  author = {Li, Jiang and Wang, Xiaoping and Zeng, Zhigang},
  title = {Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognition},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {10},
  pages = {8786--8803},
  doi = {10.1109/tpami.2025.3581236}
}
Kostenlos starten →