EUVIMEDEUVIMED entdecken
Crossref

Uncertainty-Guided Spatiotemporal Consistency Fusion Network for Infrared-Visible Video Fusion Under Extremely Low-Light Conditions

Cheng Zhao, Tianyun Song, Zhiliang Wu, Tianfu Wang, Moncef Gabbouj, Guanghui Yue, Baiying Lei, Wei Zhou

IEEE Transactions on Image Processing · 2026 · 35 · 8894-8909

DOI: 10.1109/tip.2026.3719477 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2026. Verfasst von Cheng Zhao und 7 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: Shenzhen Medical Research Special Fund Project, National Natural Science Foundation of China, Basic and Applied Basic Research Foundation of Guangdong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Infrared-visible video fusion under extremely low-light conditions is critically important yet remains underexplored, largely due to the scarcity of high-quality datasets and challenges posed by spatiotemporal uncertainty and modality bias. To address the dataset shortage, we built a dataset of 4,739 infrared and visible registration video pairs captured under extremely low-light conditions, spanning 5 scene types and 17 subcategories. Further, we proposed an Uncertainty-guided Spatiotemporal Consistency Fusion Network, termed USCFNet, for the infrared-visible video fusion. At each layer of the encoder, an Entropy-Gated SpatioTemporal Attention (EGSTA) module is introduced to capture temporal instability and spatial reliability variations through entropy-aware attention modulation, thereby enhancing feature spatiotemporal consistency. The refined infrared and visible features are then fused via a Difference-Guided Fusion (DGF) module, which adaptively exploits their content and edge differences to improve structural integrity and detail clarity. By progressively connecting DGF modules from shallow to deep layers, the network achieves the synergistic fusion of shallow textures and deep semantics. Subsequently, the output of the last DGF module is fused with the modality features of the last layer through a hierarchical mixture-of-experts fusion module. This module enables the balanced integration of modality information while preserving fine local details. Finally, the fusion feature is fed into the decoder to produce the final fused video. Extensive experiments on our dataset and two public datasets show that USCFNet outperforms competing methods, achieving lower distortion and stronger spatiotemporal consistency. The source code and dataset are available at https://github.com/Zhaocheng1/ELVID.

Förderung

  • Shenzhen Medical Research Special Fund Project
  • National Natural Science Foundation of China
  • Basic and Applied Basic Research Foundation of Guangdong Province
  • Earth System Big Data Platform of the School of Earth Sciences

Zitieren

APA

Zhao, C., Song, T., Wu, Z., Wang, T., Gabbouj, M., Yue, G., Lei, B., & Zhou, W. (2026). Uncertainty-Guided Spatiotemporal Consistency Fusion Network for Infrared-Visible Video Fusion Under Extremely Low-Light Conditions. IEEE Transactions on Image Processing, 35, 8894-8909. https://doi.org/10.1109/tip.2026.3719477

Vancouver

Zhao C, Song T, Wu Z, Wang T, Gabbouj M, Yue G, et al. Uncertainty-Guided Spatiotemporal Consistency Fusion Network for Infrared-Visible Video Fusion Under Extremely Low-Light Conditions. IEEE Transactions on Image Processing. 2026;35:8894-8909. doi:10.1109/tip.2026.3719477

BibTeX
@article{Zhao2026,
  author = {Zhao, Cheng and Song, Tianyun and Wu, Zhiliang and Wang, Tianfu and Gabbouj, Moncef and Yue, Guanghui and Lei, Baiying and Zhou, Wei},
  title = {Uncertainty-Guided Spatiotemporal Consistency Fusion Network for Infrared-Visible Video Fusion Under Extremely Low-Light Conditions},
  journal = {IEEE Transactions on Image Processing},
  year = {2026},
  volume = {35},
  pages = {8894--8909},
  doi = {10.1109/tip.2026.3719477}
}
Kostenlos starten →