Transformer-based models for multimodal irony detection
Journal of Ambient Intelligence and Humanized Computing · 2023-06 · 14 (6) · 7399-7410
DOI: 10.1007/s12652-022-04447-y ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Ambient Intelligence and Humanized Computing, online erschienen am 19.10.2022. Verfasst von David Tomás und 4 weiteren Autor:innen. Die Arbeit wurde bisher 46-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Spanish Ministry of Science and Innovation, Universidad de Alicante.
Abstract (Originalsprache)
Irony is nowadays a pervasive phenomenon in social networks. The multimodal functionalities of these platforms (i.e., the possibility to attach audio, video, and images to textual information) are increasingly leading their users to employ combinations of information in different formats to express their ironic thoughts. The present work focuses on the study of irony detection in social media posts involving image and text. To this end, a transformer architecture for the fusion of textual and image information is proposed. The model leverages disentangled text attention with visual transformers, improving F1-score up to 9% over previous existing works in the field and current state-of-the-art visio-linguistic transformers. The proposed architecture was evaluated in three different multimodal datasets gathered from Twitter and Tumblr. The results revealed that, in many situations, the text-only version of the architecture was able to capture the ironic nature of the message without using visual information. This phenomenon was further analysed, leading to the identification of linguistic patterns that could provide the context necessary for irony detection without the need for additional visual information.
Förderung
- Spanish Ministry of Science and Innovation
- Universidad de Alicante
Zitieren
APA
Tomás, D., Ortega-Bueno, R., Zhang, G., Rosso, P., & Schifanella, R. (2023). Transformer-based models for multimodal irony detection. Journal of Ambient Intelligence and Humanized Computing, 14(6), 7399-7410. https://doi.org/10.1007/s12652-022-04447-y
Vancouver
Tomás D, Ortega-Bueno R, Zhang G, Rosso P, Schifanella R. Transformer-based models for multimodal irony detection. Journal of Ambient Intelligence and Humanized Computing. 2023;14(6):7399-7410. doi:10.1007/s12652-022-04447-y
BibTeX
@article{Toms2023,
author = {Tomás, David and Ortega-Bueno, Reynier and Zhang, Guobiao and Rosso, Paolo and Schifanella, Rossano},
title = {Transformer-based models for multimodal irony detection},
journal = {Journal of Ambient Intelligence and Humanized Computing},
year = {2023},
volume = {14},
number = {6},
pages = {7399--7410},
doi = {10.1007/s12652-022-04447-y}
}
EUVIMED