A comprehensive review of CNN and transformer based visual feature extraction for automatic video captioning
International Journal of Advances in Intelligent Informatics · 31.08.2026 · 12 (3) · 788
DOI: 10.26555/ijain.v12i3.2280 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Advances in Intelligent Informatics, online erschienen am 31.08.2026. Verfasst von Hemel Sharker Akash und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
Automatic Video Captions are increasingly important to use in applications in the accessibility, search, education, and content moderation areas. Generating accurate and context-aware captions is difficult for lengthy and complex videos. Recent surveys focus more on language models than on the vision side. This means that the contribution of visual feature extraction towards the performance of the model is not studied properly. Here we attempt to fill this gap by conducting reviews of video captioning literature from the vision viewpoint, comparing CNNs-based and transformer-based feature extraction, and detecting trends, pros and cons. Approximately 100 papers published between 2017 - 2025 were reviewed and analyzed based on base vision, dataset and evaluation metrics. The review classified the selected papers into CNN-based pipelines and transformer-based techniques. Moreover, it compared their rank measured by application popularity, dataset usage, and performance scores across popular datasets/benchmarks. The review shows that CNNs were dominated before 2021, but transformer-based approaches now lead due to their ability to capture long-range temporal dependencies and multimodal interactions. However, CNNs continue to perform well for short videos. As for Dataset usage, patterns show that ActivityNet Captions and YouCook2 are suited for long-form reasoning because of timely manner captioning, while MSR-VTT and MSVD are effective for short clips (one sentence caption). In terms of Special semantic quality metrics, CIDEr and METEOR are more semantically appropriate compared to n-gram quality metrics like BLEU. This review emphasizes transformer-based architecture, particularly multimodal models, which will be the future of video captioning. However, challenges remain in handling lengthy videos, adapting across domains and languages, and ensuring efficient deployment (small, optimized model). Addressing these gaps will add value by making video captioning systems more reliable, interpretable, and broadly usable in real-world contexts.
Zitieren
APA
Akash, H. S., Raja, J. E., & Hossein, M. J. (2026). A comprehensive review of CNN and transformer based visual feature extraction for automatic video captioning. International Journal of Advances in Intelligent Informatics, 12(3), 788. https://doi.org/10.26555/ijain.v12i3.2280
Vancouver
Akash HS, Raja JE, Hossein MJ. A comprehensive review of CNN and transformer based visual feature extraction for automatic video captioning. International Journal of Advances in Intelligent Informatics. 2026;12(3):788. doi:10.26555/ijain.v12i3.2280
BibTeX
@article{Akash2026,
author = {Akash, Hemel Sharker and Raja, Joseph Emerson and Hossein, Md. Jakir},
title = {A comprehensive review of CNN and transformer based visual feature extraction for automatic video captioning},
journal = {International Journal of Advances in Intelligent Informatics},
year = {2026},
volume = {12},
number = {3},
pages = {788},
doi = {10.26555/ijain.v12i3.2280}
}
EUVIMED