EUVIMEDObjevte EUVIMED
Open access · CC BY 4.0

A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts

Ingmar Nitze, Jurjen Van der Sluijs, Sophia Barth, Philipp Bernhard, Lingcao Huang, Alexander Kizyakov, Mark J. Lara, Nina Nesterova, Alexandra Runge, Alexandra Veremeeva, Melissa Ward Jones, Chandi Witharana and 2 more

Permafrost and Periglacial Processes · 2025-01 · 36 (1) · 83-92

DOI: 10.1002/ppp.2249 ↗

At a glance

Journal article in Permafrost and Periglacial Processes, published online on 2024-10-20. Written by Ingmar Nitze and 13 co-authors. Cited 21 times so far (Crossref); 43 references. The full text is freely available (open access, CC BY 4.0). Funded by: German Academic Exchange Service, National Aeronautics and Space Administration, Helmholtz Association.

AbstraktBibliografické údajeCiteRelated studies

Abstract (original language)

Deep‐learning (DL) models have become increasingly beneficial for the detection of retrogressive thaw slumps (RTS) in the permafrost domain. However, comparing accuracy metrics is challenging due to unstandardized labeling guidelines. To address this, we conducted an experiment with 12 international domain experts from a broad range of scientific backgrounds. Using 3 m PlanetScope multispectral imagery, they digitized RTS footprints in two sites. We evaluated label uncertainty by comparing manually outlined RTS labels using Intersection‐over‐Union (IoU) and F1 metrics. At the Canadian Peel Plateau site, we see good agreement, particularly in the active parts of RTS. Differences were observed in the interpretation of the debris tongue and the stable vegetated sections of RTS. At the Russian Bykovsky site, we observed a larger mismatch. Here, the same differences were documented, but several participants mistakenly identified non‐RTS features. This emphasizes the importance of site‐specific knowledge for reliable label creation. The experiment highlights the need for standardized labeling procedures and definition of their scientific purpose. The most similar expert labels outperformed the accuracy metrics reported in the literature, highlighting human labeling capabilities with proper training, site knowledge, and clear guidelines. These findings lay the groundwork for DL‐based RTS monitoring in the pan‐Arctic.

Funding

  • German Academic Exchange Service
  • National Aeronautics and Space Administration
  • Helmholtz Association
  • National Science Foundation
  • European Space Agency
  • International Permafrost Association

Cite

APA

Nitze, I., Sluijs, J. V. D., Barth, S., Bernhard, P., Huang, L., Kizyakov, A., Lara, M. J., Nesterova, N., Runge, A., Veremeeva, A., Jones, M. W., Witharana, C., Xia, Z., & Liljedahl, A. K. (2025). A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts. Permafrost and Periglacial Processes, 36(1), 83-92. https://doi.org/10.1002/ppp.2249

Vancouver

Nitze I, Sluijs JVD, Barth S, Bernhard P, Huang L, Kizyakov A, et al. A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts. Permafrost and Periglacial Processes. 2025;36(1):83-92. doi:10.1002/ppp.2249

BibTeX
@article{Nitze2025,
  author = {Nitze, Ingmar and Sluijs, Jurjen Van der and Barth, Sophia and Bernhard, Philipp and Huang, Lingcao and Kizyakov, Alexander and Lara, Mark J. and Nesterova, Nina and Runge, Alexandra and Veremeeva, Alexandra and Jones, Melissa Ward and Witharana, Chandi and Xia, Zhuoxuan and Liljedahl, Anna K.},
  title = {A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts},
  journal = {Permafrost and Periglacial Processes},
  year = {2025},
  volume = {36},
  number = {1},
  pages = {83--92},
  doi = {10.1002/ppp.2249}
}
Start for free →