Multimodal Emotion Recognition in Conversation Based on Hypergraphs
Electronics · 19.11.2023 · 12 (22) · 4703
DOI: 10.3390/electronics12224703 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 19.11.2023. Verfasst von Jiaze Li und 3 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Liaoning Education Department Scientific Research Project, General project of Liaoning Provincial Department of Education.
Abstract (Originalsprache)
In recent years, sentiment analysis in conversation has garnered increasing attention due to its widespread applications in areas such as social media analytics, sentiment mining, and electronic healthcare. Existing research primarily focuses on sequence learning and graph-based approaches, yet they overlook the high-order interactions between different modalities and the long-term dependencies within each modality. To address these problems, this paper proposes a novel hypergraph-based method for multimodal emotion recognition in conversation (MER-HGraph). MER-HGraph extracts features from three modalities: acoustic, text, and visual. It treats each modality utterance in a conversation as a node and constructs intra-modal hypergraphs (Intra-HGraph) and inter-modal hypergraphs (Inter-HGraph) using hyperedges. The hypergraphs are then updated using hypergraph convolutional networks. Additionally, to mitigate noise in acoustic data and mitigate the impact of fixed time scales, we introduce a dynamic time window module to capture local-global information from acoustic signals. Extensive experiments on the IEMOCAP and MELD datasets demonstrate that MER-HGraph outperforms existing models in multimodal emotion recognition tasks, leveraging high-order information from multimodal data to enhance recognition capabilities.
Förderung
- Liaoning Education Department Scientific Research Project
- General project of Liaoning Provincial Department of Education
Zitieren
APA
Li, J., Mei, H., Jia, L., & Zhang, X. (2023). Multimodal Emotion Recognition in Conversation Based on Hypergraphs. Electronics, 12(22), 4703. https://doi.org/10.3390/electronics12224703
Vancouver
Li J, Mei H, Jia L, Zhang X. Multimodal Emotion Recognition in Conversation Based on Hypergraphs. Electronics. 2023;12(22):4703. doi:10.3390/electronics12224703
BibTeX
@article{Li2023,
author = {Li, Jiaze and Mei, Hongyan and Jia, Liyun and Zhang, Xing},
title = {Multimodal Emotion Recognition in Conversation Based on Hypergraphs},
journal = {Electronics},
year = {2023},
volume = {12},
number = {22},
pages = {4703},
doi = {10.3390/electronics12224703}
}
EUVIMED