EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Self-Supervised Learning across the Spectrum

Jayanth Shenoy, Xingjian Davis Zhang, Bill Tao, Shlok Mehrotra, Rem Yang, Han Zhao, Deepak Vasisht

Remote Sensing · 19.09.2024 · 16 (18) · 3470

DOI: 10.3390/rs16183470 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 19.09.2024. Verfasst von Jayanth Shenoy und 6 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 70 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NSF, Cisco Systems Inc., IBM-IL Discovery Accelerator Institute.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Satellite image time series (SITS) segmentation is crucial for many applications, like environmental monitoring, land cover mapping, and agricultural crop type classification. However, training models for SITS segmentation remains a challenging task due to the lack of abundant training data, which requires fine-grained annotation. We propose S4, a new self-supervised pretraining approach that significantly reduces the requirement for labeled training data by utilizing two key insights of satellite imagery: (a) Satellites capture images in different parts of the spectrum, such as radio frequencies and visible frequencies. (b) Satellite imagery is geo-registered, allowing for fine-grained spatial alignment. We use these insights to formulate pretraining tasks in S4. To the best of our knowledge, S4 is the first multimodal and temporal approach for SITS segmentation. S4’s novelty stems from leveraging multiple properties required for SITS self-supervision: (1) multiple modalities, (2) temporal information, and (3) pixel-level feature extraction. We also curate m2s2-SITS, a large-scale dataset of unlabeled, spatially aligned, multimodal, and geographic-specific SITS that serves as representative pretraining data for S4. Finally, we evaluate S4 on multiple SITS segmentation datasets and demonstrate its efficacy against competing baselines while using limited labeled data. Through a series of extensive comparisons and ablation studies, we demonstrate S4’s ability as an effective feature extractor for downstream semantic segmentation.

Förderung

  • NSF
  • Cisco Systems Inc.
  • IBM-IL Discovery Accelerator Institute
  • CloudBank
  • National Science Foundation Graduate Research Fellowship

Zitieren

APA

Shenoy, J., Zhang, X. D., Tao, B., Mehrotra, S., Yang, R., Zhao, H., & Vasisht, D. (2024). Self-Supervised Learning across the Spectrum. Remote Sensing, 16(18), 3470. https://doi.org/10.3390/rs16183470

Vancouver

Shenoy J, Zhang XD, Tao B, Mehrotra S, Yang R, Zhao H, et al. Self-Supervised Learning across the Spectrum. Remote Sensing. 2024;16(18):3470. doi:10.3390/rs16183470

BibTeX
@article{Shenoy2024,
  author = {Shenoy, Jayanth and Zhang, Xingjian Davis and Tao, Bill and Mehrotra, Shlok and Yang, Rem and Zhao, Han and Vasisht, Deepak},
  title = {Self-Supervised Learning across the Spectrum},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {18},
  pages = {3470},
  doi = {10.3390/rs16183470}
}
Kostenlos starten →