EUVIMEDEUVIMED entdecken
Crossref

RVAIC: Refined visual attention for improved image captioning

Majjed Al-Qatf, Ammar Hawbani, XingFu Wang, Amr Abdusallam, Saeed Alsamhi, Mohammed Alhabib, Edward Curry

Journal of Intelligent & Fuzzy Systems · 14.02.2024 · 46 (2) · 3447-3459

DOI: 10.3233/jifs-233004 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Intelligent & Fuzzy Systems, erschienen am 14.02.2024. Verfasst von Majjed Al-Qatf und 6 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 14 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual attention has emerged as a prominent approach for improving the effectiveness of image captioning, as it enables the decoder network to focus selectively on the most salient regions in the image content, thereby facilitating the generation of precise and informative captions. Although visual attention achieves the improvement, the small numerical values of its input have a negative impact on its softmax, decreasing its effectiveness. To address this limitation, we propose a refined visual attention (RVA) framework that internally reweights visual attention by leveraging the language context of previously generated words. We first feed the language context into a fully connected layer to obtain appropriate dimensions for the visual features. Then, we use a sigmoid function to obtain a probability distribution to reweight the softmax’s input by applying the multiplication process. Experiments conducted on the MS COCO dataset demonstrate that RVA outperforms traditional visual attention and other existing image captioning methods, highlighting its effectiveness in enhancing the accuracy and informativeness of image captions.

Zitieren

APA

Al-Qatf, M., Hawbani, A., Wang, X., Abdusallam, A., Alsamhi, S., Alhabib, M., & Curry, E. (2024). RVAIC: Refined visual attention for improved image captioning. Journal of Intelligent & Fuzzy Systems, 46(2), 3447-3459. https://doi.org/10.3233/jifs-233004

Vancouver

Al-Qatf M, Hawbani A, Wang X, Abdusallam A, Alsamhi S, Alhabib M, et al. RVAIC: Refined visual attention for improved image captioning. Journal of Intelligent & Fuzzy Systems. 2024;46(2):3447-3459. doi:10.3233/jifs-233004

BibTeX
@article{AlQatf2024,
  author = {Al-Qatf, Majjed and Hawbani, Ammar and Wang, XingFu and Abdusallam, Amr and Alsamhi, Saeed and Alhabib, Mohammed and Curry, Edward},
  title = {RVAIC: Refined visual attention for improved image captioning},
  journal = {Journal of Intelligent & Fuzzy Systems},
  year = {2024},
  volume = {46},
  number = {2},
  pages = {3447--3459},
  doi = {10.3233/jifs-233004}
}
Kostenlos starten →