EUVIMEDEUVIMED entdecken
Crossref

RSCaMa: Remote Sensing Image Change Captioning With State Space Model

Chenyang Liu, Keyan Chen, Bowen Chen, Haotian Zhang, Zhengxia Zou, Zhenwei Shi

IEEE Geoscience and Remote Sensing Letters · 2024 · 21 · 1-5

DOI: 10.1109/lgrs.2024.3404604 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Geoscience and Remote Sensing Letters, 2024. Verfasst von Chenyang Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 100-mal zitiert (Crossref) und nennt 21 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Key Research and Development Program of China, Beijing Natural Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Remote sensing image change captioning (RSICC) aims to describe surface changes between multitemporal remote sensing images in language, including the changed object categories, locations, and dynamics of changing objects (e.g., added or disappeared). This poses challenges to spatial and temporal modeling of bi-temporal features. Despite previous methods progressing in the spatial change perception, there are still weaknesses in joint spatial-temporal modeling. To address this, in this letter, we propose a novel RSCaMa model, which achieves efficient joint spatial-temporal modeling through multiple CaMa layers, enabling iterative refinement of bi-temporal features. To achieve efficient spatial modeling, we introduce the recently popular Mamba [a state space model (SSM)] with a global receptive field and linear complexity into the RSICC task and propose the Spatial Difference-aware SSM (SD-SSM), overcoming limitations of previous convolutional neural network (CNN)- and Transformer-based methods in the receptive field and computational complexity. SD-SSM enhances the model’s ability to capture spatial changes sharply. In terms of efficient temporal modeling, considering the potential correlation between the temporal scanning characteristics of Mamba and the temporality of the RSICC, we propose the Temporal-Traversing SSM (TT-SSM), which scans bi-temporal features in a temporal crosswise manner, enhancing the model’s temporal understanding and information interaction. Experiments validate the effectiveness of the efficient joint spatial-temporal modeling and demonstrate the outstanding performance of RSCaMa and the potential of the Mamba in the RSICC task. Additionally, we systematically compare three different language decoders, including Mamba, generative pre-trained Transformer (GPT)-style decoder, and Transformer decoder, providing valuable insights for future RSICC research. The code will be available at https://github.com/Chen-Yang-Liu/RSCaMa.

Förderung

  • National Natural Science Foundation of China
  • National Key Research and Development Program of China
  • Beijing Natural Science Foundation
  • Fundamental Research Funds for Central Universities

Zitieren

APA

Liu, C., Chen, K., Chen, B., Zhang, H., Zou, Z., & Shi, Z. (2024). RSCaMa: Remote Sensing Image Change Captioning With State Space Model. IEEE Geoscience and Remote Sensing Letters, 21, 1-5. https://doi.org/10.1109/lgrs.2024.3404604

Vancouver

Liu C, Chen K, Chen B, Zhang H, Zou Z, Shi Z. RSCaMa: Remote Sensing Image Change Captioning With State Space Model. IEEE Geoscience and Remote Sensing Letters. 2024;21:1-5. doi:10.1109/lgrs.2024.3404604

BibTeX
@article{Liu2024,
  author = {Liu, Chenyang and Chen, Keyan and Chen, Bowen and Zhang, Haotian and Zou, Zhengxia and Shi, Zhenwei},
  title = {RSCaMa: Remote Sensing Image Change Captioning With State Space Model},
  journal = {IEEE Geoscience and Remote Sensing Letters},
  year = {2024},
  volume = {21},
  pages = {1--5},
  doi = {10.1109/lgrs.2024.3404604}
}
Kostenlos starten →