EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Self-Enhanced Attention for Image Captioning

Qingyu Sun, Juan Zhang, Zhijun Fang, Yongbin Gao

Neural Processing Letters · 01.04.2024 · 56 (2)

DOI: 10.1007/s11063-024-11527-x ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neural Processing Letters, online erschienen am 01.04.2024. Verfasst von Qingyu Sun und 3 weiteren Autor:innen. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Image captioning, which involves automatically generating textual descriptions based on the content of images, has garnered increasing attention from researchers. Recently, Transformers have emerged as the preferred choice for the language model in image captioning models. Transformers leverage self-attention mechanisms to address gradient accumulation issues and eliminate the risk of gradient explosion commonly associated with RNN networks. However, a challenge arises when the input features of the self-attention mechanism belong to different categories, as it may result in ineffective highlighting of important features. To address this issue, our paper proposes a novel attention mechanism called Self-Enhanced Attention (SEA), which replaces the self-attention mechanism in the decoder part of the Transformer model. In our proposed SEA, after generating the attention weight matrix, it further adjusts the matrix based on its own distribution to effectively highlight important features. To evaluate the effectiveness of SEA, we conducted experiments on the COCO dataset, comparing the results with different visual models and training strategies. The experimental results demonstrate that when using SEA, the CIDEr score is significantly higher compared to the scores obtained without using SEA. This indicates the successful addressing of the challenge of effectively highlighting important features with our proposed mechanism.

Zitieren

APA

Sun, Q., Zhang, J., Fang, Z., & Gao, Y. (2024). Self-Enhanced Attention for Image Captioning. Neural Processing Letters, 56(2). https://doi.org/10.1007/s11063-024-11527-x

Vancouver

Sun Q, Zhang J, Fang Z, Gao Y. Self-Enhanced Attention for Image Captioning. Neural Processing Letters. 2024;56(2). doi:10.1007/s11063-024-11527-x

BibTeX
@article{Sun2024,
  author = {Sun, Qingyu and Zhang, Juan and Fang, Zhijun and Gao, Yongbin},
  title = {Self-Enhanced Attention for Image Captioning},
  journal = {Neural Processing Letters},
  year = {2024},
  volume = {56},
  number = {2},
  doi = {10.1007/s11063-024-11527-x}
}
Kostenlos starten →