EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating the predictive performance of presence–absence models: Why can the same model appear excellent or poor?

Nerea Abrego iD, Otso Ovaskainen iD

Ecology and Evolution · 2023-12 · 13 (12)

DOI: 10.1002/ece3.10784 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Ecology and Evolution, online erschienen am 18.12.2023. Verfasst von Nerea Abrego und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biological and Environmental Science University of Jyväskylä Jyväskylä Finland. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Academy of Finland, H2020 European Research Council, Jane ja Aatos Erkon Säätiö.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

When comparing multiple models of species distribution, models yielding higher predictive performance are clearly to be favored. A more difficult question is how to decide whether even the best model is “good enough”. Here, we clarify key choices and metrics related to evaluating the predictive performance of presence–absence models. We use a hierarchical case study to evaluate how four metrics of predictive performance (AUC, Tjur's R 2 , max‐Kappa, and max‐TSS) relate to each other, the random and fixed effects parts of the model, the spatial scale at which predictive performance is measured, and the cross‐validation strategy chosen. We demonstrate that the very same metric can achieve different values for the very same model, even when similar cross‐validation strategies are followed, depending on the spatial scale at which predictive performance is measured. Among metrics, Tjur's R 2 and max‐Kappa generally increase with species' prevalence, whereas AUC and max‐TSS are largely independent of prevalence. Thus, Tjur's R 2 and max‐Kappa often reach lower values when measured at the smallest scales considered in the study, while AUC and max‐TSS reaching similar values across the different spatial levels included in the study. However, they provide complementary insights on predictive performance. The very same model may appear excellent or poor not only due to the applied metric, but also how predictive performance is exactly calculated, calling for great caution on the interpretation of predictive performance. The most comprehensive evaluation of predictive performance can be obtained by evaluating predictive performance through the combination of measures providing complementary insights. Instead of following simple rules of thumb or focusing on absolute values, we recommend comparing the achieved predictive performance to the researcher's own a priori expectations on how easy it is to make predictions related to the same question that the model is used for.

Schlagwörter

Stichwörter der Autor:innen

AUCTjur's R2accuracycross‐validationdiscriminationjoint species distribution modelmax‐TSSmax‐kappapresence–absence modelsensitivityspecificity

Autor:innen und Institutionen

  • Department of Biological and Environmental Science University of Jyväskylä Jyväskylä Finland.
  • Department of Agricultural Sciences University of Helsinki Helsinki Finland.
  • Department of Biology, Centre for Biodiversity Dynamics Norwegian University of Science and Technology Trondheim Norway.
  • Organismal and Evolutionary Biology Research Programme, Faculty of Biological and Environmental Sciences University of Helsinki Helsinki Finland.

ORCID-Profile: Nerea Abrego, Otso Ovaskainen

Förderung

  • Academy of Finland
  • H2020 European Research Council
  • Jane ja Aatos Erkon Säätiö
  • Norges Forskningsråd

Zitieren

APA

Abrego, N., & Ovaskainen, O. (2023). Evaluating the predictive performance of presence–absence models: Why can the same model appear excellent or poor?. Ecology and Evolution, 13(12). https://doi.org/10.1002/ece3.10784

Vancouver

Abrego N, Ovaskainen O. Evaluating the predictive performance of presence–absence models: Why can the same model appear excellent or poor?. Ecology and Evolution. 2023;13(12). doi:10.1002/ece3.10784

BibTeX
@article{Abrego2023,
  author = {Abrego, Nerea and Ovaskainen, Otso},
  title = {Evaluating the predictive performance of presence–absence models: Why can the same model appear excellent or poor?},
  journal = {Ecology and Evolution},
  year = {2023},
  volume = {13},
  number = {12},
  doi = {10.1002/ece3.10784}
}
Kostenlos starten →