EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Adaptive Graph Learning with Multimodal Fusion for Emotion Recognition in Conversation

Jian Liu iD, Jian Li, Jiawei Dong, Zifan Mo, Na Liu, Qingdu Li iD, Ye Yuan

Biomimetics · 25.06.2025 · 10 (7) · 414

DOI: 10.3390/biomimetics10070414 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biomimetics, online erschienen am 25.06.2025. Verfasst von Jian Liu und 6 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Machine Intelligence, University of Shanghai for Science and Technology, Shanghai 200093, China. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National key Research & Development Plan of Ministry of Science and Technology of China, Young Scientists Fund of the National Natural Science Foundation of China, National key Research & Development Plan of Ministry of Science and Technology of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Robust emotion recognition is a prerequisite for natural, fluid human–computer interaction, yet conversational settings remain challenging because emotions are shaped simultaneously by global topic flow and local speaker-to-speaker dependencies. Here, we introduce GASMER—Graph-Adaptive Structure for Multimodal Emotion Recognition—a unified architecture that tackles both issues. It uses the correlation structure based on graph neural networks (GNNs) to model the complex dependencies in the conversation, while adaptively learning the graph structure for GNNs. The experiments indicate that our model has strong performance that outperforms all existing graph-based approaches, and remains competitive when compared to recent multimodal fusion models, underscoring the importance of combining fine-grained multimodal fusion with adaptive graph learning for conversational emotion recognition. On the IEMOCAP dataset, GASMER improves accuracy by 2.7% and the weighted F1-score by 3.6% compared to the best baseline. On the MOSEI dataset, it achieves a 1.2% gain in binary classification accuracy (ACC-2).

Schlagwörter

Stichwörter der Autor:innen

adaptive graph structure learningconversational AIemotion recognitiongraph neural networkstransformer-based fusion

Autor:innen und Institutionen

  • Institute of Machine Intelligence, University of Shanghai for Science and Technology, Shanghai 200093, China.
  • Department of Information Technology, National Technical University Kharkiv Polytechnic Institute, 61000 Kharkiv, Ukraine.
  • School of Automation and Electronic Information, Xiangtan University, Xiangtan 411105, China.

ORCID-Profile: Jian Liu, Qingdu Li

Förderung

  • National key Research & Development Plan of Ministry of Science and Technology of China
  • Young Scientists Fund of the National Natural Science Foundation of China
  • National key Research & Development Plan of Ministry of Science and Technology of China

Zitieren

APA

Liu, J., Li, J., Dong, J., Mo, Z., Liu, N., Li, Q., & Yuan, Y. (2025). Adaptive Graph Learning with Multimodal Fusion for Emotion Recognition in Conversation. Biomimetics, 10(7), 414. https://doi.org/10.3390/biomimetics10070414

Vancouver

Liu J, Li J, Dong J, Mo Z, Liu N, Li Q, et al. Adaptive Graph Learning with Multimodal Fusion for Emotion Recognition in Conversation. Biomimetics. 2025;10(7):414. doi:10.3390/biomimetics10070414

BibTeX
@article{Liu2025,
  author = {Liu, Jian and Li, Jian and Dong, Jiawei and Mo, Zifan and Liu, Na and Li, Qingdu and Yuan, Ye},
  title = {Adaptive Graph Learning with Multimodal Fusion for Emotion Recognition in Conversation},
  journal = {Biomimetics},
  year = {2025},
  volume = {10},
  number = {7},
  pages = {414},
  doi = {10.3390/biomimetics10070414}
}
Kostenlos starten →