Uncertainty-Guided Spatiotemporal Consistency Fusion Network for Infrared-Visible Video Fusion Under Extremely Low-Light Conditions
IEEE Transactions on Image Processing · 2026 · 35 · 8894-8909
DOI: 10.1109/tip.2026.3719477 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2026. Verfasst von Cheng Zhao und 7 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: Shenzhen Medical Research Special Fund Project, National Natural Science Foundation of China, Basic and Applied Basic Research Foundation of Guangdong Province.
Abstract (Originalsprache)
Infrared-visible video fusion under extremely low-light conditions is critically important yet remains underexplored, largely due to the scarcity of high-quality datasets and challenges posed by spatiotemporal uncertainty and modality bias. To address the dataset shortage, we built a dataset of 4,739 infrared and visible registration video pairs captured under extremely low-light conditions, spanning 5 scene types and 17 subcategories. Further, we proposed an Uncertainty-guided Spatiotemporal Consistency Fusion Network, termed USCFNet, for the infrared-visible video fusion. At each layer of the encoder, an Entropy-Gated SpatioTemporal Attention (EGSTA) module is introduced to capture temporal instability and spatial reliability variations through entropy-aware attention modulation, thereby enhancing feature spatiotemporal consistency. The refined infrared and visible features are then fused via a Difference-Guided Fusion (DGF) module, which adaptively exploits their content and edge differences to improve structural integrity and detail clarity. By progressively connecting DGF modules from shallow to deep layers, the network achieves the synergistic fusion of shallow textures and deep semantics. Subsequently, the output of the last DGF module is fused with the modality features of the last layer through a hierarchical mixture-of-experts fusion module. This module enables the balanced integration of modality information while preserving fine local details. Finally, the fusion feature is fed into the decoder to produce the final fused video. Extensive experiments on our dataset and two public datasets show that USCFNet outperforms competing methods, achieving lower distortion and stronger spatiotemporal consistency. The source code and dataset are available at https://github.com/Zhaocheng1/ELVID.
Förderung
- Shenzhen Medical Research Special Fund Project
- National Natural Science Foundation of China
- Basic and Applied Basic Research Foundation of Guangdong Province
- Earth System Big Data Platform of the School of Earth Sciences
Zitieren
APA
Zhao, C., Song, T., Wu, Z., Wang, T., Gabbouj, M., Yue, G., Lei, B., & Zhou, W. (2026). Uncertainty-Guided Spatiotemporal Consistency Fusion Network for Infrared-Visible Video Fusion Under Extremely Low-Light Conditions. IEEE Transactions on Image Processing, 35, 8894-8909. https://doi.org/10.1109/tip.2026.3719477
Vancouver
Zhao C, Song T, Wu Z, Wang T, Gabbouj M, Yue G, et al. Uncertainty-Guided Spatiotemporal Consistency Fusion Network for Infrared-Visible Video Fusion Under Extremely Low-Light Conditions. IEEE Transactions on Image Processing. 2026;35:8894-8909. doi:10.1109/tip.2026.3719477
BibTeX
@article{Zhao2026,
author = {Zhao, Cheng and Song, Tianyun and Wu, Zhiliang and Wang, Tianfu and Gabbouj, Moncef and Yue, Guanghui and Lei, Baiying and Zhou, Wei},
title = {Uncertainty-Guided Spatiotemporal Consistency Fusion Network for Infrared-Visible Video Fusion Under Extremely Low-Light Conditions},
journal = {IEEE Transactions on Image Processing},
year = {2026},
volume = {35},
pages = {8894--8909},
doi = {10.1109/tip.2026.3719477}
}
EUVIMED