Integration of the latent variable knowledge into deep image captioning with Bayesian modeling
IET Image Processing · 2023-05 · 17 (7) · 2256-2271
DOI: 10.1049/ipr2.12790 ↗Kurz gefasst
Fachartikel in der Zeitschrift IET Image Processing, online erschienen am 25.03.2023. Verfasst von Alireza Barati und 2 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Automatic image captioning systems assign one or more sentences to images to describe their visual content. Most of these systems use attention‐based deep convolutional neural networks and recurrent neural networks (CNN‐RNN‐Att). However, they must optimally use latent variables and side information within the image concepts. This study aims to integrate a latent variable into image captioning using CNN‐RNN‐Att. A Bayesian modeling framework is used for this work. As an instance of a latent variable, High‐Level Semantic Concepts (HLSCs) of tags are used to implement the proposed model. The Bayesian model output interpretation is to localize the entire image description process and breaks it down into sub‐problems. Thus, a baseline descriptor subnet is trained independently for each sub‐problem, and it is the only expert in captioning for a given HLSC. The final output is the caption derived from the subnet; its HLSC is closest to the image content. The results indicate that CNN‐RNN‐Att applied to data localized using HLSCs improves the captioning accuracy of the proposed method, which can be compared to the latest state‐of‐the‐art and most accurate captioning systems.
Zitieren
APA
Barati, A., Farsi, H., & Mohamadzadeh, S. (2023). Integration of the latent variable knowledge into deep image captioning with Bayesian modeling. IET Image Processing, 17(7), 2256-2271. https://doi.org/10.1049/ipr2.12790
Vancouver
Barati A, Farsi H, Mohamadzadeh S. Integration of the latent variable knowledge into deep image captioning with Bayesian modeling. IET Image Processing. 2023;17(7):2256-2271. doi:10.1049/ipr2.12790
BibTeX
@article{Barati2023,
author = {Barati, Alireza and Farsi, Hassan and Mohamadzadeh, Sajad},
title = {Integration of the latent variable knowledge into deep image captioning with Bayesian modeling},
journal = {IET Image Processing},
year = {2023},
volume = {17},
number = {7},
pages = {2256--2271},
doi = {10.1049/ipr2.12790}
}
EUVIMED