Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation
IEEE Transactions on Geoscience and Remote Sensing · 2025 · 63 · 1-11
DOI: 10.1109/tgrs.2024.3522293 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Geoscience and Remote Sensing, 2025. Verfasst von Sen Lei und 5 weiteren Autor:innen. Die Arbeit wurde bisher 61-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Sichuan Province, Fellowship of China National Postdoctoral Program for Innovative Talents.
Abstract (Originalsprache)
Given a language expression, referring remote sensing image segmentation (RRSIS) aims to identify ground objects and assign pixelwise labels within the imagery. One of the key challenges for this task is to capture discriminative multimodal features via image-text alignment. However, the existing RRSIS methods use one vanilla and coarse alignment, where the language expression is directly extracted to be fused with the visual features. In this article, we argue that a “fine-grained image-text alignment” can improve the extraction of multimodal information. To this point, we propose a new RRSIS method to fully exploit the visual and linguistic representations. Specifically, the original referring expression is regarded as context text, which is further decoupled into the ground object and spatial position texts. The proposed fine-grained image-text alignment module (FIAM) would simultaneously leverage the features of the input image and the corresponding texts, obtaining better discriminative multimodal representation. Meanwhile, to handle the various scales of ground objects in remote sensing, we introduce a text-aware multiscale enhancement module (TMEM) to adaptively perform cross-scale fusion and intersections. We evaluate the effectiveness of the proposed method on two public referring remote sensing datasets including RefSegRS and RRSIS-D, and our method obtains superior performance over several state-of-the-art methods. The code will be publicly available at https://github.com/Shaosifan/FIANet.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Sichuan Province
- Fellowship of China National Postdoctoral Program for Innovative Talents
- Ant Group Research Fund
Zitieren
APA
Lei, S., Xiao, X., Zhang, T., Li, H., Shi, Z., & Zhu, Q. (2025). Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation. IEEE Transactions on Geoscience and Remote Sensing, 63, 1-11. https://doi.org/10.1109/tgrs.2024.3522293
Vancouver
Lei S, Xiao X, Zhang T, Li H, Shi Z, Zhu Q. Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation. IEEE Transactions on Geoscience and Remote Sensing. 2025;63:1-11. doi:10.1109/tgrs.2024.3522293
BibTeX
@article{Lei2025,
author = {Lei, Sen and Xiao, Xinyu and Zhang, Tianlin and Li, Heng-Chao and Shi, Zhenwei and Zhu, Qing},
title = {Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation},
journal = {IEEE Transactions on Geoscience and Remote Sensing},
year = {2025},
volume = {63},
pages = {1--11},
doi = {10.1109/tgrs.2024.3522293}
}
EUVIMED