RSPS-SAM: A Remote Sensing Image Panoptic Segmentation Method Based on SAM
Remote Sensing · 28.10.2024 · 16 (21) · 4002
DOI: 10.3390/rs16214002 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 28.10.2024. Verfasst von Zhuoran Liu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Open Research Fund of the State Key Laboratory of Space–Earth Integrated Information Technology, Key Laboratory of Land Satellite Remote Sensing Application, Ministry of Natural Resources of the People’s Republic of China.
Abstract (Originalsprache)
Satellite remote sensing images contain complex and diverse ground object information and the images exhibit spatial multi-scale characteristics, making the panoptic segmentation of satellite remote sensing images a highly challenging task. Due to the lack of large-scale annotated datasets for panoramic segmentation, existing methods still suffer from weak model generalization capabilities. To mitigate this issue, this paper leverages the advantages of the Segment Anything Model (SAM), which can segment any object in remote sensing images without requiring any annotations and proposes a high-resolution remote sensing image panoptic segmentation method called Remote Sensing Panoptic Segmentation SAM (RSPS-SAM). Firstly, to address the problem of global information loss caused by cropping large remote sensing images for training, a Batch Attention Pyramid was designed to extract multi-scale features from remote sensing images and capture long-range contextual information between cropped patches, thereby enhancing the semantic understanding of remote sensing images. Secondly, we constructed a Mask Decoder to address the limitation of SAM requiring manual input prompts and its inability to output category information. This decoder utilized mask-based attention for mask segmentation, enabling automatic prompt generation and category prediction of segmented objects. Finally, the effectiveness of the proposed method was validated on the high-resolution remote sensing image airport scene dataset RSAPS-ASD. The results demonstrate that the proposed method achieves segmentation and recognition of foreground instances and background regions in high-resolution remote sensing images without the need for prompt input, while providing smooth segmentation boundaries with a panoptic segmentation quality (PQ) of 57.2, outperforming current mainstream methods.
Förderung
- National Natural Science Foundation of China
- Open Research Fund of the State Key Laboratory of Space–Earth Integrated Information Technology
- Key Laboratory of Land Satellite Remote Sensing Application, Ministry of Natural Resources of the People’s Republic of China
- Civil Space Advance Research Project of China
Zitieren
APA
Liu, Z., Li, Z., Liang, Y., Persello, C., Sun, B., He, G., & Ma, L. (2024). RSPS-SAM: A Remote Sensing Image Panoptic Segmentation Method Based on SAM. Remote Sensing, 16(21), 4002. https://doi.org/10.3390/rs16214002
Vancouver
Liu Z, Li Z, Liang Y, Persello C, Sun B, He G, et al. RSPS-SAM: A Remote Sensing Image Panoptic Segmentation Method Based on SAM. Remote Sensing. 2024;16(21):4002. doi:10.3390/rs16214002
BibTeX
@article{Liu2024,
author = {Liu, Zhuoran and Li, Zizhen and Liang, Ying and Persello, Claudio and Sun, Bo and He, Guangjun and Ma, Lei},
title = {RSPS-SAM: A Remote Sensing Image Panoptic Segmentation Method Based on SAM},
journal = {Remote Sensing},
year = {2024},
volume = {16},
number = {21},
pages = {4002},
doi = {10.3390/rs16214002}
}
EUVIMED