A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts
Permafrost and Periglacial Processes · 2025-01 · 36 (1) · 83-92
DOI: 10.1002/ppp.2249 ↗Kurz gefasst
Fachartikel in der Zeitschrift Permafrost and Periglacial Processes, online erschienen am 20.10.2024. Verfasst von Ingmar Nitze und 13 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: German Academic Exchange Service, National Aeronautics and Space Administration, Helmholtz Association.
Abstract (Originalsprache)
Deep‐learning (DL) models have become increasingly beneficial for the detection of retrogressive thaw slumps (RTS) in the permafrost domain. However, comparing accuracy metrics is challenging due to unstandardized labeling guidelines. To address this, we conducted an experiment with 12 international domain experts from a broad range of scientific backgrounds. Using 3 m PlanetScope multispectral imagery, they digitized RTS footprints in two sites. We evaluated label uncertainty by comparing manually outlined RTS labels using Intersection‐over‐Union (IoU) and F1 metrics. At the Canadian Peel Plateau site, we see good agreement, particularly in the active parts of RTS. Differences were observed in the interpretation of the debris tongue and the stable vegetated sections of RTS. At the Russian Bykovsky site, we observed a larger mismatch. Here, the same differences were documented, but several participants mistakenly identified non‐RTS features. This emphasizes the importance of site‐specific knowledge for reliable label creation. The experiment highlights the need for standardized labeling procedures and definition of their scientific purpose. The most similar expert labels outperformed the accuracy metrics reported in the literature, highlighting human labeling capabilities with proper training, site knowledge, and clear guidelines. These findings lay the groundwork for DL‐based RTS monitoring in the pan‐Arctic.
Förderung
- German Academic Exchange Service
- National Aeronautics and Space Administration
- Helmholtz Association
- National Science Foundation
- European Space Agency
- International Permafrost Association
Zitieren
APA
Nitze, I., Sluijs, J. V. D., Barth, S., Bernhard, P., Huang, L., Kizyakov, A., Lara, M. J., Nesterova, N., Runge, A., Veremeeva, A., Jones, M. W., Witharana, C., Xia, Z., & Liljedahl, A. K. (2025). A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts. Permafrost and Periglacial Processes, 36(1), 83-92. https://doi.org/10.1002/ppp.2249
Vancouver
Nitze I, Sluijs JVD, Barth S, Bernhard P, Huang L, Kizyakov A, et al. A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts. Permafrost and Periglacial Processes. 2025;36(1):83-92. doi:10.1002/ppp.2249
BibTeX
@article{Nitze2025,
author = {Nitze, Ingmar and Sluijs, Jurjen Van der and Barth, Sophia and Bernhard, Philipp and Huang, Lingcao and Kizyakov, Alexander and Lara, Mark J. and Nesterova, Nina and Runge, Alexandra and Veremeeva, Alexandra and Jones, Melissa Ward and Witharana, Chandi and Xia, Zhuoxuan and Liljedahl, Anna K.},
title = {A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts},
journal = {Permafrost and Periglacial Processes},
year = {2025},
volume = {36},
number = {1},
pages = {83--92},
doi = {10.1002/ppp.2249}
}
EUVIMED