EUVIMEDEUVIMED entdecken
Crossref

Multimodal Emotion-Cause Pair Extraction in Conversations

Fanfan Wang, Zixiang Ding, Rui Xia, Zhaoyu Li, Jianfei Yu

IEEE Transactions on Affective Computing · 01.07.2023 · 14 (3) · 1832-1844

DOI: 10.1109/taffc.2022.3226559 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, erschienen am 01.07.2023. Verfasst von Fanfan Wang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 61-mal zitiert (Crossref) und nennt 66 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Jiangsu Province for Young Scholars, Distinguished Young Scholars.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Conversation is an important form of human communication and contains a large number of emotions. It is interesting to discover emotions and their causes in conversations. Conversation in its natural form is multimodal. Many studies have been carried out on multimodal emotion recognition in conversations, yet there is still a lack of work on multimodal emotion cause analysis. In this article, we introduce a new task named Multimodal Emotion-Cause Pair Extraction in Conversations, aiming to jointly extract emotions and the corresponding causes from conversations reflected in multiple modalities (i.e., text, audio and video). We accordingly construct a multimodal conversational emotion cause dataset, Emotion-Cause-in-Friends, which contains 9,794 multimodal emotion-cause pairs among 13,619 utterances in theFriendssitcom. We benchmark the task by establishing two baseline systems including a heuristic approach considering inherent patterns in the location of causes and emotions and a deep learning approach that incorporates multimodal features for emotion-cause pair extraction, and conduct the human performance test for comparison. Furthermore, we investigate the effect of multimodal information, explore the potential of incorporating commonsense knowledge, and perform the task under both Static and Real-time settings.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Jiangsu Province for Young Scholars
  • Distinguished Young Scholars

Zitieren

APA

Wang, F., Ding, Z., Xia, R., Li, Z., & Yu, J. (2023). Multimodal Emotion-Cause Pair Extraction in Conversations. IEEE Transactions on Affective Computing, 14(3), 1832-1844. https://doi.org/10.1109/taffc.2022.3226559

Vancouver

Wang F, Ding Z, Xia R, Li Z, Yu J. Multimodal Emotion-Cause Pair Extraction in Conversations. IEEE Transactions on Affective Computing. 2023;14(3):1832-1844. doi:10.1109/taffc.2022.3226559

BibTeX
@article{Wang2023,
  author = {Wang, Fanfan and Ding, Zixiang and Xia, Rui and Li, Zhaoyu and Yu, Jianfei},
  title = {Multimodal Emotion-Cause Pair Extraction in Conversations},
  journal = {IEEE Transactions on Affective Computing},
  year = {2023},
  volume = {14},
  number = {3},
  pages = {1832--1844},
  doi = {10.1109/taffc.2022.3226559}
}
Kostenlos starten →