EUVIMEDEUVIMED entdecken
Crossref

GText-IF: Leveraging Text-Driven Semantics for Degradation-Aware Image Fusion

Xunpeng Yi, Han Xu, Yibing Zhang, Hao Zhang, Linfeng Tang, Jiayi Ma

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-11 · 48 (11) · 15440-15456

DOI: 10.1109/tpami.2026.3722579 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-11. Verfasst von Xunpeng Yi und 5 weiteren Autor:innen. Die Arbeit nennt 55 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Image fusion aims to aggregate information from multiple source images to generate the fused image that provides a comprehensive representation. In the presence of degradations in source images, current image fusion methods struggle to adaptively accommodate them, resulting in residual degradations in the fused images and thereby impairing information representation. To address this issue, we propose an innovative degradation-aware image fusion framework via leveraging text-driven semantics, termed GText-IF. It extends the field of image fusion to the domain of text and image fusion and explores the feasibility of leveraging deep text and image coupling to guide degradation-aware fusion. First, by leveraging the powerful comprehension capability of vision-language models together with human prior knowledge, textual information is incorporated into degradation-aware image fusion as the semantic guidance. In this manner, text serves as a shared medium connecting humans and machines while providing foundational semantic cues. Second, semantic segmentation is ingeniously introduced as a bridge between textual and visual modalities, exploiting their intrinsic semantic consistency. The spatial representational properties of content information described by text are consistent with semantic segmentation, while the effective representation of semantic features in image content is also achieved through semantic segmentation. By integrating multiple decoders with a degradation prompt module, GText-IF can adaptively mitigate degradations in the fused results with the assistance of textual prompts and semantic guidance. Extensive experiments demonstrate that GText-IF can perform effective degradation-aware fusion in real degraded image scenarios, exhibiting remarkable advancement, robustness, and practicality.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Yi, X., Xu, H., Zhang, Y., Zhang, H., Tang, L., & Ma, J. (2026). GText-IF: Leveraging Text-Driven Semantics for Degradation-Aware Image Fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(11), 15440-15456. https://doi.org/10.1109/tpami.2026.3722579

Vancouver

Yi X, Xu H, Zhang Y, Zhang H, Tang L, Ma J. GText-IF: Leveraging Text-Driven Semantics for Degradation-Aware Image Fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(11):15440-15456. doi:10.1109/tpami.2026.3722579

BibTeX
@article{Yi2026,
  author = {Yi, Xunpeng and Xu, Han and Zhang, Yibing and Zhang, Hao and Tang, Linfeng and Ma, Jiayi},
  title = {GText-IF: Leveraging Text-Driven Semantics for Degradation-Aware Image Fusion},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {11},
  pages = {15440--15456},
  doi = {10.1109/tpami.2026.3722579}
}
Kostenlos starten →