Evaluating the predictive performance of presence–absence models: Why can the same model appear excellent or poor?
Ecology and Evolution · 2023-12 · 13 (12)
DOI: 10.1002/ece3.10784 ↗Kurz gefasst
Fachartikel in der Zeitschrift Ecology and Evolution, online erschienen am 18.12.2023. Verfasst von Nerea Abrego und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biological and Environmental Science University of Jyväskylä Jyväskylä Finland. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Academy of Finland, H2020 European Research Council, Jane ja Aatos Erkon Säätiö.
Abstract (Originalsprache)
When comparing multiple models of species distribution, models yielding higher predictive performance are clearly to be favored. A more difficult question is how to decide whether even the best model is “good enough”. Here, we clarify key choices and metrics related to evaluating the predictive performance of presence–absence models. We use a hierarchical case study to evaluate how four metrics of predictive performance (AUC, Tjur's R 2 , max‐Kappa, and max‐TSS) relate to each other, the random and fixed effects parts of the model, the spatial scale at which predictive performance is measured, and the cross‐validation strategy chosen. We demonstrate that the very same metric can achieve different values for the very same model, even when similar cross‐validation strategies are followed, depending on the spatial scale at which predictive performance is measured. Among metrics, Tjur's R 2 and max‐Kappa generally increase with species' prevalence, whereas AUC and max‐TSS are largely independent of prevalence. Thus, Tjur's R 2 and max‐Kappa often reach lower values when measured at the smallest scales considered in the study, while AUC and max‐TSS reaching similar values across the different spatial levels included in the study. However, they provide complementary insights on predictive performance. The very same model may appear excellent or poor not only due to the applied metric, but also how predictive performance is exactly calculated, calling for great caution on the interpretation of predictive performance. The most comprehensive evaluation of predictive performance can be obtained by evaluating predictive performance through the combination of measures providing complementary insights. Instead of following simple rules of thumb or focusing on absolute values, we recommend comparing the achieved predictive performance to the researcher's own a priori expectations on how easy it is to make predictions related to the same question that the model is used for.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biological and Environmental Science University of Jyväskylä Jyväskylä Finland.
- Department of Agricultural Sciences University of Helsinki Helsinki Finland.
- Department of Biology, Centre for Biodiversity Dynamics Norwegian University of Science and Technology Trondheim Norway.
- Organismal and Evolutionary Biology Research Programme, Faculty of Biological and Environmental Sciences University of Helsinki Helsinki Finland.
ORCID-Profile: Nerea Abrego, Otso Ovaskainen
Förderung
- Academy of Finland
- H2020 European Research Council
- Jane ja Aatos Erkon Säätiö
- Norges Forskningsråd
Zitieren
APA
Abrego, N., & Ovaskainen, O. (2023). Evaluating the predictive performance of presence–absence models: Why can the same model appear excellent or poor?. Ecology and Evolution, 13(12). https://doi.org/10.1002/ece3.10784
Vancouver
Abrego N, Ovaskainen O. Evaluating the predictive performance of presence–absence models: Why can the same model appear excellent or poor?. Ecology and Evolution. 2023;13(12). doi:10.1002/ece3.10784
BibTeX
@article{Abrego2023,
author = {Abrego, Nerea and Ovaskainen, Otso},
title = {Evaluating the predictive performance of presence–absence models: Why can the same model appear excellent or poor?},
journal = {Ecology and Evolution},
year = {2023},
volume = {13},
number = {12},
doi = {10.1002/ece3.10784}
}
EUVIMED