Metaphor recognition based on cross-modal multi-level information fusion
Complex & Intelligent Systems · 2025-01 · 11 (1)
DOI: 10.1007/s40747-024-01684-w ↗Kurz gefasst
Fachartikel in der Zeitschrift Complex & Intelligent Systems, online erschienen am 28.12.2024. Verfasst von Qimeng Yang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Natural Science Foundation of Xinjiang Uygur Autonomous Region, Xinjiang Uygur Autonomous Region Universities Fundamental Research Funds Scientific Research Project, Postgraduate Innovation Projects in the Autonomous Region, China.
Abstract (Originalsprache)
The metaphor is a pervasive linguistic device that has become an active research topic in the computer field because of its essential role in language's cognitive and communicative processes. Currently, the rapid expansion of social media encourages the development of multimodal. As the most popular communication method in social media, memes have attracted the attention of many linguists, who believe that metaphors contain rich metaphorical information. However, multimodal metaphor detection suffers from insufficient information due to the short text of memes and lacks effective multimodal fusion methods. To address these problems, we utilize a single-pass non-autoregressive text generation method to convert images into text to provide additional textual information for the model. In addition, the information of different modes is fused by a multi-layer fusion module consisting of a prefix guide module and a similarity-aware aggregator. The module can reduce the heterogeneity between modes, learn fine-grained information, and better integrate the characteristic information of different modes. We conducted many experiments on the Met-Meme dataset. Compared with the strong baseline model in the experiment, the weighted F1 of our model on three data types of the MET-Meme dataset improved by 1.95%, 1.55%, and 1.72%, respectively. To further demonstrate the effectiveness of the proposed method, we also conducted experiments on a multimodal sarcasm dataset and obtained competitive results.
Förderung
- Natural Science Foundation of Xinjiang Uygur Autonomous Region
- Xinjiang Uygur Autonomous Region Universities Fundamental Research Funds Scientific Research Project
- Postgraduate Innovation Projects in the Autonomous Region, China
Zitieren
APA
Yang, Q., Yan, Y., He, X., & Guo, S. (2025). Metaphor recognition based on cross-modal multi-level information fusion. Complex & Intelligent Systems, 11(1). https://doi.org/10.1007/s40747-024-01684-w
Vancouver
Yang Q, Yan Y, He X, Guo S. Metaphor recognition based on cross-modal multi-level information fusion. Complex & Intelligent Systems. 2025;11(1). doi:10.1007/s40747-024-01684-w
BibTeX
@article{Yang2025,
author = {Yang, Qimeng and Yan, Yuanbo and He, Xiaoyu and Guo, Shisong},
title = {Metaphor recognition based on cross-modal multi-level information fusion},
journal = {Complex & Intelligent Systems},
year = {2025},
volume = {11},
number = {1},
doi = {10.1007/s40747-024-01684-w}
}
EUVIMED