Figure 2. Proportion of publications within each year that conducted any of the three main methodological steps: (1) specifying the type of movement / connectivity being modelled, (2) conducting some form of sensitivity and / or uncertainty analysis, or (3) conducting some form of validation.
DISCUSSION
We reviewed 181 studies that generated novel landscape connectivity outputs using circuit theory and found that while such user-friendly programs are being widely used, researchers are typically not (i) clearly articulating the type of connectivity being modeled (ii) conducting some form of sensitivity or uncertainty analyses, and (iii) validating the connectivity model output.
Articulating the type of connectivity being modeled
Fewer than half of the studies we reviewed included a description of the type of connectivity being modeled. Examples of those that did include Xu et al. (2019) who clearly indicated that they were modeling connectivity for seasonal migrations of Tibetan antelope (Pantholops hodgsonii ); Alexander et al. (2019) who stated they were interested in connectivity for dispersal and gene flow of giant kangaroo rats (Dipodomys ingens ); and Di Febbraro et al. (2019) who predicted range shifts of alien squirrel species due to climate change and land use change.
Being clear about the type of connectivity is crucial because it determines the type of data that should be used to parameterize and validate the model, as well as the resulting conservation implications. Indeed, different types of data would be required to predict connectivity for within-home range movement versus range shifts, migrations, or dispersal (Wade et al. 2015; Aylward et al. 2020). For example, Keeley et al. (2017) found that connectivity models based on home-range habitat-use data were ineffective at predicting connectivity for dispersal and mating movements. Several studies, however, were careful to use location data from outside of home ranges (e.g., Merrick and Koprowski 2017; Popescu et al. 2021) or data only from dispersing individuals (Carroll et al. 2020).
Sensitivity and uncertainty analyses
Connectivity analysis workflows are characterized by numerous analytical decisions (often arbitrary), myriad assumptions, and a variety of widely-recognized sources of uncertainty (Bowman et al. 2020). For these reasons there have been repeated calls to include SUAs in connectivity studies (Sawyer et al. 2011; Zeller et al. 2012, 2017; Wade et al. 2015). Nevertheless, we found that only 19% of the 181 studies we reviewed conducted some form of sensitivity or uncertainty analysis.
Among studies that have implemented SUA, researchers found that important and substantial sensitivities are commonplace. For example, Churko et al. (2020) found that the current maps generated for amphibian species in Switzerland were highly sensitive to the choice of transformation applied to the input resistance layers. Marrec et al. (2020) used a factorial design to explore interactions among uncertainty sources in their large-scale, species-agnostic analysis of landscape connectivity in Alberta. They found that whether water was included as a barrier had the single greatest effect on predictions, followed by the scaling function used for resistance values.
Of primary importance is assessing the degree to which predictions about landscape connectivity vary in response to variation in the input parameter values. In practice, and tailored by one’s knowledge of the study system, the general approach is to: (i) identify the input parameters that most warrant scrutiny via SUA (e.g., the ranking of land cover types with respect to resistance) (ii) systematically vary parameter values (iii) use a factorial design to explore all unique parameter value (treatment) combinations (if more than one parameter is being examined) (iv) run the connectivity model using each unique parameter value combination, (v) quantify variation among the connectivity model outputs, and finally (vi) statistically analyze this variation in relation to the treatments. Exactly how variation among model outputs is quantified will depend on the goals of the study and will also govern the type of statistical method that is most suitable for the main SUA. For example, in their species-agnostic modeling of connectivity across Alberta, Canada, Marrec et al. (2020) used several complementary approaches, including one in which a correlation matrix was built using Pearson correlation coefficients from all pairwise comparisons of output layers from their factorial design. They then calculated a dissimilarity matrix from this correlation matrix, and used distance-based redundancy analysis (db-RDA) to evaluate the effects of their manipulated variables. One limitation was that they were unable to include interactions in their db-RDA due to overfitting concerns (Marrec et al. 2020).
Admittedly, computational demands may limit the number of parameters that can be explored within a fully factorial design. For instance, to have explored all possible landscape combinations for their study on the sensitivity of path selection function models and predicted road-crossing locations, Zeller et al. (2017) would have needed to create more than 5 million landscape definitions. Instead, the authors implemented a procedure that yielded 2500 combinations of inputs at random, and then used a multi-model inference and model averaging approach to infer the most important predictors of output variation (Zeller et al. 2017). This represents an effective alternative to fully factorial sensitivity analyses for which computational demands exceed capacity.
It is reasonable to ask whether SUAs are necessary if validation with independent data shows connectivity models to be suitably accurate. We believe that sensitivity analyses, in particular, could still be useful in this scenario because they can help identify which input variables have the greatest effect on the results, and should therefore be the focus of future research. For example, dispersal distances are fundamental to connectivity analyses (Liu et al. 2014) but are often estimated based on morphological traits (Tamme et al. 2014; Albert et al. 2017). Conversely, sensitivity analyses can also streamline analyses by identifying elements that can be simplified. For example, Bowman et al. (2020) found that current densities were generally insensitive to specific cost weights assigned to land cover categories so long as categories were ranked in the correct order.
Model validation
Although roughly half of the studies performed some form of validation, only 19% validated the output from the circuit analysis itself. Examples of studies that did conduct validation include Bond et al. (2017), who assessed the accuracy of the connectivity model output by determining whether the predicted wildebeest (Connochaetes taurinus ) corridors contained more or fewer occurrence records than expected by chance. Similarly, Xu et al. (2019) compared Tibetan antelope migratory routes predicted by connectivity models to those actually used according to GPS collar data. Several studies used genetics as a form of inferential validation (cf., Wade et al. 2015), including a study by Epps et al. (2013), which found that connectivity estimates based on genetic data were consistent with connectivity estimates developed using occupancy / habitat use data for African elephants (Loxodonta africana ). Carroll et al. (2020) used observations only from dispersing individuals and three different validation metrics to select the most accurate connectivity map from among six variations.
Other studies that included a validation step either validated the input habitat layer or the input resistance/permeability layer. Several used model selection/tuning tools (e.g., data resampling / cross validation, Akaike Information Criterion, etc.) to help assess or improve their habitat models (Pitman et al. 2017; Osipova et al. 2019; Zhang et al. 2019; Almasieh et al. 2019). While validating input layers adds a degree of rigor, it is not a substitute for post-modeling assessment of accuracy (Wade et al. 2015).
Only 20% of the studies that validated either their input or output layers did so using fully independent data (8 and 12 respectively). For example, Gantchoff et al. (2017) used an independent dataset of citizen-reported sightings of black bears (Ursus americanus ) to validate the output from their connectivity model, while Koen et al. (2014) used independent empirical datasets of herptile road kill sites and fisher (Martes pennanti ) telemetry to validate their current density map. Some studies split their data into training and validation datasets (McClure et al. 2016; Brennan et al. 2018). This approach is helpful for assessing model precision more so than accuracy (Warren et al. 2020), and in some instances could lead to model accuracy being overestimated when training and validation data are non-independent (Roberts et al. 2016).
A number of studies that either did not validate their model(s) or validated with non-independent data, cited a lack of independent data or the difficulty/cost associated with obtaining these data (e.g., Dutta et al. 2018). Advances in both genetic sampling and GPS tracking equipment are enabling the collection of more data at lower cost and the increasing availability of “open” movement data, such as those available in Movebank (Kranstauber et al. 2011; Wikelski et al. 2020), could provide a solution for some studies. Further, when input data are abundant, as is common with GPS telemetry, a portion of these data can be withheld to create pseudo-independent data for validation. However, the effectiveness of this approach hinges on the method by which these data are held back (random individual or random point), and accounting for inherent non-independence among data points that may bias validation results.
We acknowledge that there is no apparent consensus as to which is the ‘best’ method for validation (McClure et al. 2016; Zeller et al. 2018). As Goicolea et al, (2021) point out, it may be most prudent to use several methods to test different aspects of connectivity, as they did in their study on Iberian lynx.
Circuit-theory based models
While the purpose of this study was not to evaluate the accuracy of circuit theory models specifically, our assessment found that the output from those models were corroborated by validation in about 65% of the studies. However, that figure is not particularly reassuring considering (i) that only 19% of the studies attempted to validate their connectivity results, and (ii) the potential for bias against publishing studies with negative results (Fraser et al. 2018).
As Dickson et al. (2019) point out, current maps are an important component of The Nature Conservancy’s approach for identifying land protection priorities, which helped guide the use of $38 million (USD) of land protection funding. Nevertheless, without a well-designed validation procedure, there is clear potential for ineffective use of limited conservation resources and negative outcomes for the species of interest. LaPoint et al. (2013), for example, found that Circuitscape failed to predict movement corridors for fisher, which could have resulted in the wrong parcels of land being conserved. In an effort to identify potential wildlife crossing corridors and plan road mortality mitigation features for a road enlargement project, Laliberté et al. (2020) validated the results from both Circuitscape and LinkageMapper and found that Circuitscape predictions were less accurate for deer than for moose (Alces alces ) corridors.
Trends over time
Although we did not perform any statistical analyses, we found no indication of any systematic improvement in communicating the type of connectivity being modeled, conducting SUAs, nor in validating model outputs. Indeed, based on our assessment of 181 studies, there has been little improvement since earlier, similar reviews. For example, Sawyer et al. (2011) found that only 9 of 24 studies using least-cost path analysis performed some form of model validation. In 2012, Zeller et al. (2012) reviewed 96 studies that estimated landscape resistance and found that most studies were not clearly describing or assessing the uncertainty associated with the input parameters nor with the output surfaces. A United States Department of Agriculture report published a few years later (Wade et al. 2015) concluded that “validation of connectivity maps is virtually non-existent” and as such, resulting connectivity maps should “be considered to be a largely untested hypothesis rather than a tested solution to a problem.”
Potential issues with our assessments
We acknowledge several potential issues with our assessment. Firstly, we restricted our review to studies that used only circuit theory and as such our findings may not be representative of connectivity studies in general. However, user-friendly tools are also available for other approaches and we have no reason to believe that they would be applied in a systematically different way than for those that use circuit theory. Indeed, Sawyer et al. (2011) and Wade et al. (2015) found similar results for studies using least-cost paths.
Secondly, our classification scheme was devised to discretely categorize a wide variety of methods being used in connectivity modeling. Despite our attempt to ensure replicability of our review process, we acknowledge that our method relied on interpretation of (often brief) descriptions of validation procedure. We encourage researchers to make the details of their validation process unambiguous, as the nuances of these analyses may be important for ascertaining model reliability.
Finally, although there is an appreciable amount of model-based connectivity research on a diverse array of species (including insects, non-avian reptiles, birds, amphibians, plants, etc.), we chose to limit our scope to terrestrial mammals. We imposed this limitation so that our results would reflect studies that examined similar movement behavior (relative to aerial migration or seed dispersal, for example). For this reason, the proportions of validation data types and independence are not intended to reflect modeling efforts being done in non-mammalian ecology, although the importance of validation goes well beyond mammalian studies, as do the potential consequences of neglecting model validation.
Conclusion
Governments around the world agree that maintaining and restoring ecological connectivity is a key component of biodiversity conservation (e.g., Convention on Biological Diversity 2010; Convention on Migratory Species 2020). To target their conservation and restoration efforts appropriately, practitioners must be informed by rigorous and reliable connectivity modeling research. This requires, at a minimum, that researchers consistently articulate the type of connectivity being modeled, undertake sensitivity and uncertainty analyses, and validate the connectivity model outputs with independent data. We encourage connectivity modelers to be up-front and transparent about which, if any, of these steps have been completed, and reviewers of connectivity modeling manuscripts to request this information if it is lacking. These simple steps will enable consumers of the research, including practitioners, to more readily assess its reliability and utility. Without these measures, we risk mis-use of limited conservation resources and failing to meet key conservation targets.