EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Integration of the latent variable knowledge into deep image captioning with Bayesian modeling

Alireza Barati, Hassan Farsi, Sajad Mohamadzadeh

IET Image Processing · 2023-05 · 17 (7) · 2256-2271

DOI: 10.1049/ipr2.12790 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IET Image Processing, online erschienen am 25.03.2023. Verfasst von Alireza Barati und 2 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automatic image captioning systems assign one or more sentences to images to describe their visual content. Most of these systems use attention‐based deep convolutional neural networks and recurrent neural networks (CNN‐RNN‐Att). However, they must optimally use latent variables and side information within the image concepts. This study aims to integrate a latent variable into image captioning using CNN‐RNN‐Att. A Bayesian modeling framework is used for this work. As an instance of a latent variable, High‐Level Semantic Concepts (HLSCs) of tags are used to implement the proposed model. The Bayesian model output interpretation is to localize the entire image description process and breaks it down into sub‐problems. Thus, a baseline descriptor subnet is trained independently for each sub‐problem, and it is the only expert in captioning for a given HLSC. The final output is the caption derived from the subnet; its HLSC is closest to the image content. The results indicate that CNN‐RNN‐Att applied to data localized using HLSCs improves the captioning accuracy of the proposed method, which can be compared to the latest state‐of‐the‐art and most accurate captioning systems.

Zitieren

APA

Barati, A., Farsi, H., & Mohamadzadeh, S. (2023). Integration of the latent variable knowledge into deep image captioning with Bayesian modeling. IET Image Processing, 17(7), 2256-2271. https://doi.org/10.1049/ipr2.12790

Vancouver

Barati A, Farsi H, Mohamadzadeh S. Integration of the latent variable knowledge into deep image captioning with Bayesian modeling. IET Image Processing. 2023;17(7):2256-2271. doi:10.1049/ipr2.12790

BibTeX
@article{Barati2023,
  author = {Barati, Alireza and Farsi, Hassan and Mohamadzadeh, Sajad},
  title = {Integration of the latent variable knowledge into deep image captioning with Bayesian modeling},
  journal = {IET Image Processing},
  year = {2023},
  volume = {17},
  number = {7},
  pages = {2256--2271},
  doi = {10.1049/ipr2.12790}
}
Kostenlos starten →