EUVIMEDEUVIMED entdecken
Crossref

CAMEL: Capturing Metaphorical Alignment with Context Disentangling for Multimodal Emotion Recognition

Linhao Zhang, Li Jin, Guangluan Xu, Xiaoyu Li, Cai Xu, Kaiwen Wei, Nayu Liu, Haonan Liu

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (8) · 9341-9349

DOI: 10.1609/aaai.v38i8.28787 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Linhao Zhang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Understanding the emotional polarity of multimodal content with metaphorical characteristics, such as memes, poses a significant challenge in Multimodal Emotion Recognition (MER). Previous MER researches have overlooked the phenomenon of metaphorical alignment in multimedia content, which involves non-literal associations between concepts to convey implicit emotional tones. Metaphor-agnostic MER methods may be misinformed by the isolated unimodal emotions, which are distinct from the real emotions blended in multimodal metaphors. Moreover, contextual semantics can further affect the emotions associated with similar metaphors, leading to the challenge of maintaining contextual compatibility. To address the issue of metaphorical alignment in MER, we propose to leverage a conditional generative approach for capturing metaphorical analogies. Our approach formulates schematic prompts and corresponding references based on theoretical foundations, which allows the model to better grasp metaphorical nuances. In order to maintain contextual sensitivity, we incorporate a disentangled contrastive matching mechanism, which undergoes curricular adjustment to regulate its intensity during the learning process. The automatic and human evaluation experiments on two benchmarks prove that, our model provides considerable and stable improvements in recognizing multimodal emotion with metaphor attributes.

Zitieren

APA

Zhang, L., Jin, L., Xu, G., Li, X., Xu, C., Wei, K., Liu, N., & Liu, H. (2024). CAMEL: Capturing Metaphorical Alignment with Context Disentangling for Multimodal Emotion Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 38(8), 9341-9349. https://doi.org/10.1609/aaai.v38i8.28787

Vancouver

Zhang L, Jin L, Xu G, Li X, Xu C, Wei K, et al. CAMEL: Capturing Metaphorical Alignment with Context Disentangling for Multimodal Emotion Recognition. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(8):9341-9349. doi:10.1609/aaai.v38i8.28787

BibTeX
@article{Zhang2024,
  author = {Zhang, Linhao and Jin, Li and Xu, Guangluan and Li, Xiaoyu and Xu, Cai and Wei, Kaiwen and Liu, Nayu and Liu, Haonan},
  title = {CAMEL: Capturing Metaphorical Alignment with Context Disentangling for Multimodal Emotion Recognition},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {8},
  pages = {9341--9349},
  doi = {10.1609/aaai.v38i8.28787}
}
Kostenlos starten →