Multi‐Gated Dual‐Stream Visual Feature Fusion for Image Captioning
Computer Graphics Forum · 28.02.2026
DOI: 10.1111/cgf.70398 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computer Graphics Forum, online erschienen am 28.02.2026. Verfasst von Yuzhe Lu und 7 weiteren Autor:innen. Die Arbeit nennt 50 Literaturangaben.
Abstract (Originalsprache)
As a task at the intersection of computer vision and natural language processing, image captioning offers significant application value in domains such as intelligent human–computer interaction, accessibility support and multimedia content retrieval. The primary objective is to generate natural language descriptions by interpreting visual features, traditionally relying on heterogeneous single‐stream grid features and region features. However, existing approaches face limitations: grid features struggle to balance global semantic perception with local detail analysis, and region features exhibit weakened spatial modelling efficacy due to sparse semantic correlations. Furthermore, fusing heterogeneous visual features often lacks effective control over complementarity and redundancy, leading to descriptions prone to semantic bias or detail omission. To address these challenges, we propose a novel Multi‐Gated Dual‐Stream Visual Feature Fusion (MGDSF) for Image Captioning. Our approach enhances the semantic accuracy and completeness of generated captions through dual‐stream feature extraction and a multi‐gated fusion (MGF) mechanism. First, we employ a Mamba‐like linear attention mechanism to construct a grid feature network with hierarchical positional awareness. This network achieves global modelling while maintaining local sensitivity by dynamically modulating information flow. Second, based on the Detection Transformer (DETR) framework, we design a region feature extractor to provide complementary local object visual information. Finally, we introduce a MGF module that balances the complementarity of dual‐stream visual features and suppresses cross‐modal information redundancy via multiple context‐aware gates, thereby achieving fine‐grained visual‐semantic alignment. Experiments on MS COCO demonstrate that MGDSF surpasses existing methods on multiple evaluation metrics, achieving METEOR, ROUGE‐L and CIDEr scores of 30.0%, 59.8% and 140.1%, respectively. These results validate the effectiveness of our proposed method and indicate its broad application potential.
Zitieren
APA
Lu, Y., Zhu, Z., Ge, D., Tu, R., Bai, Y., Yang, Y., Wang, Y., & Yu, Z. (2026). Multi‐Gated Dual‐Stream Visual Feature Fusion for Image Captioning. Computer Graphics Forum. https://doi.org/10.1111/cgf.70398
Vancouver
Lu Y, Zhu Z, Ge D, Tu R, Bai Y, Yang Y, et al. Multi‐Gated Dual‐Stream Visual Feature Fusion for Image Captioning. Computer Graphics Forum. 2026. doi:10.1111/cgf.70398
BibTeX
@article{Lu2026,
author = {Lu, Yuzhe and Zhu, Zhongjie and Ge, Di and Tu, Renwei and Bai, Yongqiang and Yang, Yueping and Wang, Yuer and Yu, Zhijing},
title = {Multi‐Gated Dual‐Stream Visual Feature Fusion for Image Captioning},
journal = {Computer Graphics Forum},
year = {2026},
doi = {10.1111/cgf.70398}
}
EUVIMED