RS-DeCLIP: Gated Dual-Source DeCLIP Adaptation with Side-Branch Query-Based Local Fusion for Remote Sensing Image–Text Retrieval
Remote Sensing · 19.11.2026 · 18 (22) · 3986
DOI: 10.3390/rs18223986 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 19.11.2026. Verfasst von Yun Liao und 5 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Yunnan Provincial Graduate Supervisor Team Construction Project, Yunnan University.
Abstract (Originalsprache)
Remote sensing image–text retrieval requires diverse textual supervision and fine-grained visual representations. Redundant captions limit linguistic diversity, while rewriting can introduce semantic drift. A single global image descriptor may also underrepresent small objects and local spatial arrangements, limiting fine-grained discrimination. To address these challenges, we propose RS-DeCLIP, a two-stage retrieval framework. Stage 1 adapts a RemoteCLIP ViT-B/32 dual encoder with DeCLIP-style multi-view supervision using official captions and captions polished by a large language model. A semantic consistency gate downweights text self-supervision (TSS) between less consistent original and rewritten captions. Similarity-aware negative downweighting and text-to-image weighting address ambiguous negatives and directional imbalance, while training-only fine-grained alignment strengthens detail sensitivity. Stage 2 targets the fine-grained visual information underrepresented by the global descriptor. Five learnable visual query vectors, conditioned on the global image embedding and guided by soft spatial preferences, attend to a dense P2 feature map to extract local evidence. Consistency-gated orthogonal residual fusion integrates this evidence into the frozen global image embedding. These visual queries are distinct from the user’s text query: image feature extraction and fusion do not use the retrieval text, so fused image descriptors can be precomputed and compared with independently encoded text vectors. On the Remote Sensing Image Captioning Dataset (RSICD), Stage 1 improves mean recall (MR) from 35.26% to 38.54%, and Stage 2 further increases MR to 39.27%. On the Remote Sensing Image–Text Match Dataset (RSITMD), Stage 2 improves MR from 50.90% to 52.58%. The final model exceeds the reported GeoRSCLIP-FT (RET-2) results by 0.40 and 0.77 percentage points on RSICD and RSITMD, respectively, and the Representation Discrepancy Bridging (RDB) method by 1.18 and 0.54 percentage points.
Förderung
- Yunnan Provincial Graduate Supervisor Team Construction Project
- Yunnan University
Zitieren
APA
Liao, Y., Liao, C., Liu, J., Cui, Q., Yang, C., & Duan, Q. (2026). RS-DeCLIP: Gated Dual-Source DeCLIP Adaptation with Side-Branch Query-Based Local Fusion for Remote Sensing Image–Text Retrieval. Remote Sensing, 18(22), 3986. https://doi.org/10.3390/rs18223986
Vancouver
Liao Y, Liao C, Liu J, Cui Q, Yang C, Duan Q. RS-DeCLIP: Gated Dual-Source DeCLIP Adaptation with Side-Branch Query-Based Local Fusion for Remote Sensing Image–Text Retrieval. Remote Sensing. 2026;18(22):3986. doi:10.3390/rs18223986
BibTeX
@article{Liao2026,
author = {Liao, Yun and Liao, Cheng and Liu, Junhui and Cui, Qi and Yang, Chang and Duan, Qing},
title = {RS-DeCLIP: Gated Dual-Source DeCLIP Adaptation with Side-Branch Query-Based Local Fusion for Remote Sensing Image–Text Retrieval},
journal = {Remote Sensing},
year = {2026},
volume = {18},
number = {22},
pages = {3986},
doi = {10.3390/rs18223986}
}
EUVIMED