Research on multimodal hate speech detection based on self-attention mechanism feature fusion
The Journal of Supercomputing · 2025-01 · 81 (1)
DOI: 10.1007/s11227-024-06602-y ↗Kurz gefasst
Fachartikel in der Zeitschrift The Journal of Supercomputing, online erschienen am 18.10.2024. Verfasst von Junjie Mao und 2 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 32 Literaturangaben.
Abstract (Originalsprache)
The widespread rise of multimedia social platforms has diversified the ways in which people communicate and the content they share. Hate speech, as a threat to societal harmony, has also shifted its manifestation from a singular textual to a multimodal one. Previously, most methods for detecting hate speech were limited to the text modality, making it difficult to identify and classify newly emerging multimodal hate speech that combines text and images. This paper proposes a novel multimodal hate speech detection model to respond to the above-mentioned needs for multimodal hate speech detection. The proposed joint model can use moving windows to extract multi-level visual features and extract text features based on the RoBERTa pretraining model and introduces a multi-head self-attention mechanism in the later fusion process for image and text feature fusion. This article also conducted experiments on the multimodal benchmark dataset hateful memes. The model achieved an accuracy of 0.8780, precision of 0.9135, F1-score of 0.8237, and AUCROC of 0.8532, defeating the SOTA multimodal hate speech recognition model.
Zitieren
APA
Mao, J., Shi, H., & Li, X. (2025). Research on multimodal hate speech detection based on self-attention mechanism feature fusion. The Journal of Supercomputing, 81(1). https://doi.org/10.1007/s11227-024-06602-y
Vancouver
Mao J, Shi H, Li X. Research on multimodal hate speech detection based on self-attention mechanism feature fusion. The Journal of Supercomputing. 2025;81(1). doi:10.1007/s11227-024-06602-y
BibTeX
@article{Mao2025,
author = {Mao, Junjie and Shi, Hanxiao and Li, Xiaojun},
title = {Research on multimodal hate speech detection based on self-attention mechanism feature fusion},
journal = {The Journal of Supercomputing},
year = {2025},
volume = {81},
number = {1},
doi = {10.1007/s11227-024-06602-y}
}
EUVIMED