EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model’s Generalizability in Permafrost Mapping

Wenwen Li, Chia-Yu Hsu, Sizhe Wang, Yezhou Yang, Hyunho Lee, Anna Liljedahl, Chandi Witharana, Yili Yang, Brendan M. Rogers, Samantha T. Arundel, Matthew B. Jones, Kenton McHenry und 1 weitere

Remote Sensing · 24.02.2024 · 16 (5) · 797

DOI: 10.3390/rs16050797 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 24.02.2024. Verfasst von Wenwen Li und 12 weiteren Autor:innen. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, Google.org’s Impact Challenge for Climate Innovation Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper assesses trending AI foundation models, especially emerging computer vision foundation models and their performance in natural landscape feature segmentation. While the term foundation model has quickly garnered interest from the geospatial domain, its definition remains vague. Hence, this paper will first introduce AI foundation models and their defining characteristics. Built upon the tremendous success achieved by Large Language Models (LLMs) as the foundation models for language tasks, this paper discusses the challenges of building foundation models for geospatial artificial intelligence (GeoAI) vision tasks. To evaluate the performance of large AI vision models, especially Meta’s Segment Anything Model (SAM), we implemented different instance segmentation pipelines that minimize the changes to SAM to leverage its power as a foundation model. A series of prompt strategies were developed to test SAM’s performance regarding its theoretical upper bound of predictive accuracy, zero-shot performance, and domain adaptability through fine-tuning. The analysis used two permafrost feature datasets, ice-wedge polygons and retrogressive thaw slumps because (1) these landform features are more challenging to segment than man-made features due to their complicated formation mechanisms, diverse forms, and vague boundaries; (2) their presence and changes are important indicators for Arctic warming and climate change. The results show that although promising, SAM still has room for improvement to support AI-augmented terrain mapping. The spatial and domain generalizability of this finding is further validated using a more general dataset EuroCrops for agricultural field mapping. Finally, we discuss future research directions that strengthen SAM’s applicability in challenging geospatial domains.

Förderung

  • National Science Foundation
  • Google.org’s Impact Challenge for Climate Innovation Program

Zitieren

APA

Li, W., Hsu, C., Wang, S., Yang, Y., Lee, H., Liljedahl, A., Witharana, C., Yang, Y., Rogers, B. M., Arundel, S. T., Jones, M. B., McHenry, K., & Solis, P. (2024). Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model’s Generalizability in Permafrost Mapping. Remote Sensing, 16(5), 797. https://doi.org/10.3390/rs16050797

Vancouver

Li W, Hsu C, Wang S, Yang Y, Lee H, Liljedahl A, et al. Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model’s Generalizability in Permafrost Mapping. Remote Sensing. 2024;16(5):797. doi:10.3390/rs16050797

BibTeX
@article{Li2024,
  author = {Li, Wenwen and Hsu, Chia-Yu and Wang, Sizhe and Yang, Yezhou and Lee, Hyunho and Liljedahl, Anna and Witharana, Chandi and Yang, Yili and Rogers, Brendan M. and Arundel, Samantha T. and Jones, Matthew B. and McHenry, Kenton and Solis, Patricia},
  title = {Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model’s Generalizability in Permafrost Mapping},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {5},
  pages = {797},
  doi = {10.3390/rs16050797}
}
Kostenlos starten →