Self-Enhanced Attention for Image Captioning
Neural Processing Letters · 01.04.2024 · 56 (2)
DOI: 10.1007/s11063-024-11527-x ↗Kurz gefasst
Fachartikel in der Zeitschrift Neural Processing Letters, online erschienen am 01.04.2024. Verfasst von Qingyu Sun und 3 weiteren Autor:innen. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Image captioning, which involves automatically generating textual descriptions based on the content of images, has garnered increasing attention from researchers. Recently, Transformers have emerged as the preferred choice for the language model in image captioning models. Transformers leverage self-attention mechanisms to address gradient accumulation issues and eliminate the risk of gradient explosion commonly associated with RNN networks. However, a challenge arises when the input features of the self-attention mechanism belong to different categories, as it may result in ineffective highlighting of important features. To address this issue, our paper proposes a novel attention mechanism called Self-Enhanced Attention (SEA), which replaces the self-attention mechanism in the decoder part of the Transformer model. In our proposed SEA, after generating the attention weight matrix, it further adjusts the matrix based on its own distribution to effectively highlight important features. To evaluate the effectiveness of SEA, we conducted experiments on the COCO dataset, comparing the results with different visual models and training strategies. The experimental results demonstrate that when using SEA, the CIDEr score is significantly higher compared to the scores obtained without using SEA. This indicates the successful addressing of the challenge of effectively highlighting important features with our proposed mechanism.
Zitieren
APA
Sun, Q., Zhang, J., Fang, Z., & Gao, Y. (2024). Self-Enhanced Attention for Image Captioning. Neural Processing Letters, 56(2). https://doi.org/10.1007/s11063-024-11527-x
Vancouver
Sun Q, Zhang J, Fang Z, Gao Y. Self-Enhanced Attention for Image Captioning. Neural Processing Letters. 2024;56(2). doi:10.1007/s11063-024-11527-x
BibTeX
@article{Sun2024,
author = {Sun, Qingyu and Zhang, Juan and Fang, Zhijun and Gao, Yongbin},
title = {Self-Enhanced Attention for Image Captioning},
journal = {Neural Processing Letters},
year = {2024},
volume = {56},
number = {2},
doi = {10.1007/s11063-024-11527-x}
}
EUVIMED