How to evaluate community predictions without thresholding?

Show full item record



Scherrer , D , Mod , H K & Guisan , A 2020 , ' How to evaluate community predictions without thresholding? ' , Methods in Ecology and Evolution , vol. 11 , no. 1 , pp. 51-63 .

Title: How to evaluate community predictions without thresholding?
Author: Scherrer, Daniel; Mod, Heidi K.; Guisan, Antoine
Contributor organization: Department of Geosciences and Geography
Date: 2020-01
Language: eng
Number of pages: 13
Belongs to series: Methods in Ecology and Evolution
ISSN: 2041-210X
Abstract: Stacked species distribution models (S-SDM) provide a tool to make spatial predictions about communities by first modelling individual species and then stacking the modelled predictions to form assemblages. The evaluation of the predictive performance is usually based on a comparison of the observed and predicted community properties (e.g. species richness, composition). However, the most available and widely used evaluation metrics require the thresholding of single species' predicted probabilities of occurrence to obtain binary outcomes (i.e. presence/absence). This binarization can introduce unnecessary bias and error. Herein, we present and demonstrate the use of several groups of new or rarely used evaluation approaches and metrics for both species richness and community composition that do not require thresholding but instead directly compare the predicted probabilities of occurrences of species to the presence/absence observations in the assemblages. Community AUC, which is based on traditional AUC, measures the ability of a model to differentiate between species presences or absences at a given site according to their predicted probabilities of occurrence. Summing the probabilities gives the expected species richness and allows the estimation of the probability that the observed species richness is not different from the expected species richness based on the species' probabilities of occurrence. The traditional Sorensen and Jaccard similarity indices (which are based on presences/absences) were adapted to maxSorensen and maxJaccard and to probSorensen and probJaccard (which use probabilities directly). A further approach (improvement over null models) compares the predictions based on S-SDMs with the expectations from the null models to estimate the improvement in both species richness and composition predictions. Additionally, all metrics can be described against the environmental conditions of sites (e.g. elevation) to highlight the abilities of models to detect the variation in the strength of the community assembly processes in different environments. These metrics offer an unbiased view of the performance of community predictions compared to metrics that requiring thresholding. As such, they allow more straightforward comparisons of model performance among studies (i.e. they are not influenced by any subjective thresholding decisions).
Subject: community modelling
Jaccard index
null model
Sorensen index
stacked species distribution models
1181 Ecology, evolutionary biology
Peer reviewed: Yes
Usage restriction: openAccess
Self-archived version: acceptedVersion

Files in this item

Total number of downloads: Loading...

Files Size Format View
Scherrer_2020_H ... ions_accepted_version_.pdf 2.749Mb PDF View/Open

This item appears in the following Collection(s)

Show full item record