Image and audio caps: automated captioning of background sounds and images using deep learning
Multimedia Systems · 2023-10 · 29 (5) · 2951-2959
DOI: 10.1007/s00530-022-00902-0 ↗Kurz gefasst
Fachartikel in der Zeitschrift Multimedia Systems, online erschienen am 26.02.2022. Verfasst von M. Poongodi und 2 weiteren Autor:innen. Die Arbeit wurde bisher 48-mal zitiert (Crossref) und nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Hamad bin Khalifa University.
Abstract (Originalsprache)
Image recognition based on computers is something human beings have been working on for many years. It is one of the most difficult tasks in the field of computer science, and improvements to this system are made when we speak. In this paper, we propose a methodology to automatically propose an appropriate title and add a specific sound to the image. Two models have been extensively trained and combined to achieve this effect. Sounds are recommended based on the image scene and the headings are generated using a combination of natural language processing and state-of-the-art computer vision models. A Top 5 accuracy of 67% and a Top 1 accuracy of 53% have been achieved. It is also worth mentioning that this is also the first model of its kind to make this forecast.
Förderung
- Hamad bin Khalifa University
Zitieren
APA
Poongodi, M., Hamdi, M., & Wang, H. (2023). Image and audio caps: automated captioning of background sounds and images using deep learning. Multimedia Systems, 29(5), 2951-2959. https://doi.org/10.1007/s00530-022-00902-0
Vancouver
Poongodi M, Hamdi M, Wang H. Image and audio caps: automated captioning of background sounds and images using deep learning. Multimedia Systems. 2023;29(5):2951-2959. doi:10.1007/s00530-022-00902-0
BibTeX
@article{Poongodi2023,
author = {Poongodi, M. and Hamdi, Mounir and Wang, Huihui},
title = {Image and audio caps: automated captioning of background sounds and images using deep learning},
journal = {Multimedia Systems},
year = {2023},
volume = {29},
number = {5},
pages = {2951--2959},
doi = {10.1007/s00530-022-00902-0}
}
EUVIMED