Figure 2. Proportion of
publications within each year that conducted any of the three main
methodological steps: (1) specifying the type of movement / connectivity
being modelled, (2) conducting some form of sensitivity and / or
uncertainty analysis, or (3) conducting some form of validation.
DISCUSSION
We reviewed 181 studies that generated novel landscape connectivity
outputs using circuit theory and found that while such user-friendly
programs are being widely used, researchers are typically not (i)
clearly articulating the type of connectivity being modeled (ii)
conducting some form of sensitivity or uncertainty analyses, and (iii)
validating the connectivity model output.
Articulating the type of connectivity being modeled
Fewer than half of the studies we reviewed included a description of the
type of connectivity being modeled. Examples of those that did include
Xu et al. (2019) who clearly indicated that they were modeling
connectivity for seasonal migrations of Tibetan antelope
(Pantholops hodgsonii ); Alexander et al. (2019) who stated they
were interested in connectivity for dispersal and gene flow of giant
kangaroo rats (Dipodomys ingens ); and Di Febbraro et al. (2019)
who predicted range shifts of alien squirrel species due to climate
change and land use change.
Being clear about the type of connectivity is crucial because it
determines the type of data that should be used to parameterize and
validate the model, as well as the resulting conservation implications.
Indeed, different types of data would be required to predict
connectivity for within-home range movement versus range shifts,
migrations, or dispersal (Wade et al. 2015; Aylward et al. 2020). For
example, Keeley et al. (2017) found that connectivity models based on
home-range habitat-use data were ineffective at predicting connectivity
for dispersal and mating movements. Several studies, however, were
careful to use location data from outside of home ranges (e.g., Merrick
and Koprowski 2017; Popescu et al. 2021) or data only from dispersing
individuals (Carroll et al. 2020).
Sensitivity and uncertainty analyses
Connectivity analysis workflows are characterized by numerous analytical
decisions (often arbitrary), myriad assumptions, and a variety of
widely-recognized sources of uncertainty (Bowman et al. 2020). For these
reasons there have been repeated calls to include SUAs in connectivity
studies (Sawyer et al. 2011; Zeller et al. 2012, 2017; Wade et al.
2015). Nevertheless, we found that only 19% of the 181 studies we
reviewed conducted some form of sensitivity or uncertainty analysis.
Among studies that have implemented SUA, researchers found that
important and substantial sensitivities are commonplace. For example,
Churko et al. (2020) found that the current maps generated for amphibian
species in Switzerland were highly sensitive to the choice of
transformation applied to the input resistance layers. Marrec et al.
(2020) used a factorial design to explore interactions among uncertainty
sources in their large-scale, species-agnostic analysis of landscape
connectivity in Alberta. They found that whether water was included as a
barrier had the single greatest effect on predictions, followed by the
scaling function used for resistance values.
Of primary importance is assessing the degree to which predictions about
landscape connectivity vary in response to variation in the input
parameter values. In practice, and tailored by one’s knowledge of the
study system, the general approach is to: (i) identify the input
parameters that most warrant scrutiny via SUA (e.g., the ranking
of land cover types with respect to resistance) (ii) systematically vary
parameter values (iii) use a factorial design to explore all unique
parameter value (treatment) combinations (if more than one parameter is
being examined) (iv) run the connectivity model using each unique
parameter value combination, (v) quantify variation among the
connectivity model outputs, and finally (vi) statistically analyze this
variation in relation to the treatments. Exactly how variation among
model outputs is quantified will depend on the goals of the study and
will also govern the type of statistical method that is most suitable
for the main SUA. For example, in their species-agnostic modeling of
connectivity across Alberta, Canada, Marrec et al. (2020) used several
complementary approaches, including one in which a correlation matrix
was built using Pearson correlation coefficients from all pairwise
comparisons of output layers from their factorial design. They then
calculated a dissimilarity matrix from this correlation matrix, and used
distance-based redundancy analysis (db-RDA) to evaluate the effects of
their manipulated variables. One limitation was that they were unable to
include interactions in their db-RDA due to overfitting concerns (Marrec
et al. 2020).
Admittedly, computational demands may limit the number of parameters
that can be explored within a fully factorial design. For instance, to
have explored all possible landscape combinations for their study on the
sensitivity of path selection function models and predicted
road-crossing locations, Zeller et al. (2017) would have needed to
create more than 5 million landscape definitions. Instead, the authors
implemented a procedure that yielded 2500 combinations of inputs at
random, and then used a multi-model inference and model averaging
approach to infer the most important predictors of output variation
(Zeller et al. 2017). This represents an effective alternative to fully
factorial sensitivity analyses for which computational demands exceed
capacity.
It is reasonable to ask whether SUAs are necessary if validation with
independent data shows connectivity models to be suitably accurate. We
believe that sensitivity analyses, in particular, could still be useful
in this scenario because they can help identify which input variables
have the greatest effect on the results, and should therefore be the
focus of future research. For example, dispersal distances are
fundamental to connectivity analyses (Liu et al. 2014) but are often
estimated based on morphological traits (Tamme et al. 2014; Albert et
al. 2017). Conversely, sensitivity analyses can also streamline analyses
by identifying elements that can be simplified. For example, Bowman et
al. (2020) found that current densities were generally insensitive to
specific cost weights assigned to land cover categories so long as
categories were ranked in the correct order.
Model validation
Although roughly half of the studies performed some form of validation,
only 19% validated the output from the circuit analysis itself.
Examples of studies that did conduct validation include Bond et al.
(2017), who assessed the accuracy of the connectivity model output by
determining whether the predicted wildebeest (Connochaetes
taurinus ) corridors contained more or fewer occurrence records than
expected by chance. Similarly, Xu et al. (2019) compared Tibetan
antelope migratory routes predicted by connectivity models to those
actually used according to GPS collar data. Several studies used
genetics as a form of inferential validation (cf., Wade et al. 2015),
including a study by Epps et al. (2013), which found that connectivity
estimates based on genetic data were consistent with connectivity
estimates developed using occupancy / habitat use data for African
elephants (Loxodonta africana ). Carroll et al. (2020) used
observations only from dispersing individuals and three different
validation metrics to select the most accurate connectivity map from
among six variations.
Other studies that included a validation step either validated the input
habitat layer or the input resistance/permeability layer. Several used
model selection/tuning tools (e.g., data resampling / cross validation,
Akaike Information Criterion, etc.) to help assess or improve their
habitat models (Pitman et al. 2017; Osipova et al. 2019; Zhang et al.
2019; Almasieh et al. 2019). While validating input layers adds a degree
of rigor, it is not a substitute for post-modeling assessment of
accuracy (Wade et al. 2015).
Only 20% of the studies that validated either their input or output
layers did so using fully independent data (8 and 12 respectively). For
example, Gantchoff et al. (2017) used an independent dataset of
citizen-reported sightings of black bears (Ursus americanus ) to
validate the output from their connectivity model, while Koen et al.
(2014) used independent empirical datasets of herptile road kill sites
and fisher (Martes pennanti ) telemetry to validate their current
density map. Some studies split their data into training and validation
datasets (McClure et al. 2016; Brennan et al. 2018). This approach is
helpful for assessing model precision more so than accuracy (Warren et
al. 2020), and in some instances could lead to model accuracy being
overestimated when training and validation data are non-independent
(Roberts et al. 2016).
A number of studies that either did not validate their model(s) or
validated with non-independent data, cited a lack of independent data or
the difficulty/cost associated with obtaining these data (e.g., Dutta et
al. 2018). Advances in both genetic sampling and GPS tracking equipment
are enabling the collection of more data at lower cost and the
increasing availability of “open” movement data, such as those
available in Movebank (Kranstauber et al. 2011; Wikelski et al. 2020),
could provide a solution for some studies. Further, when input data are
abundant, as is common with GPS telemetry, a portion of these data can
be withheld to create pseudo-independent data for validation. However,
the effectiveness of this approach hinges on the method by which these
data are held back (random individual or random point), and accounting
for inherent non-independence among data points that may bias validation
results.
We acknowledge that there is no apparent consensus as to which is the
‘best’ method for validation (McClure et al. 2016; Zeller et al. 2018).
As Goicolea et al, (2021) point out, it may be most prudent to use
several methods to test different aspects of connectivity, as they did
in their study on Iberian lynx.
Circuit-theory based models
While the purpose of this study was not to evaluate the accuracy of
circuit theory models specifically, our assessment found that the output
from those models were corroborated by validation in about 65% of the
studies. However, that figure is not particularly reassuring considering
(i) that only 19% of the studies attempted to validate their
connectivity results, and (ii) the potential for bias against publishing
studies with negative results (Fraser et al. 2018).
As Dickson et al. (2019) point out, current maps are an important
component of The Nature Conservancy’s approach for identifying land
protection priorities, which helped guide the use of $38 million (USD)
of land protection funding. Nevertheless, without a well-designed
validation procedure, there is clear potential for ineffective use of
limited conservation resources and negative outcomes for the species of
interest. LaPoint et al. (2013), for example, found that Circuitscape
failed to predict movement corridors for fisher, which could have
resulted in the wrong parcels of land being conserved. In an effort to
identify potential wildlife crossing corridors and plan road mortality
mitigation features for a road enlargement project, Laliberté et al.
(2020) validated the results from both Circuitscape and LinkageMapper
and found that Circuitscape predictions were less accurate for deer than
for moose (Alces alces ) corridors.
Trends over time
Although we did not perform any statistical analyses, we found no
indication of any systematic improvement in communicating the type of
connectivity being modeled, conducting SUAs, nor in validating model
outputs. Indeed, based on our assessment of 181 studies, there has been
little improvement since earlier, similar reviews. For example, Sawyer
et al. (2011) found that only 9 of 24 studies using least-cost path
analysis performed some form of model validation. In 2012, Zeller et al.
(2012) reviewed 96 studies that estimated landscape resistance and found
that most studies were not clearly describing or assessing the
uncertainty associated with the input parameters nor with the output
surfaces. A United States Department of Agriculture report published a
few years later (Wade et al. 2015) concluded that “validation of
connectivity maps is virtually non-existent” and as such, resulting
connectivity maps should “be considered to be a largely untested
hypothesis rather than a tested solution to a problem.”
Potential issues with our assessments
We acknowledge several potential issues with our assessment. Firstly, we
restricted our review to studies that used only circuit theory and as
such our findings may not be representative of connectivity studies in
general. However, user-friendly tools are also available for other
approaches and we have no reason to believe that they would be applied
in a systematically different way than for those that use circuit
theory. Indeed, Sawyer et al. (2011) and Wade et al. (2015) found
similar results for studies using least-cost paths.
Secondly, our classification scheme was devised to discretely categorize
a wide variety of methods being used in connectivity modeling. Despite
our attempt to ensure replicability of our review process, we
acknowledge that our method relied on interpretation of (often brief)
descriptions of validation procedure. We encourage researchers to make
the details of their validation process unambiguous, as the nuances of
these analyses may be important for ascertaining model reliability.
Finally, although there is an appreciable amount of model-based
connectivity research on a diverse array of species (including insects,
non-avian reptiles, birds, amphibians, plants, etc.), we chose to limit
our scope to terrestrial mammals. We imposed this limitation so that our
results would reflect studies that examined similar movement behavior
(relative to aerial migration or seed dispersal, for example). For this
reason, the proportions of validation data types and independence are
not intended to reflect modeling efforts being done in non-mammalian
ecology, although the importance of validation goes well beyond
mammalian studies, as do the potential consequences of neglecting model
validation.
Conclusion
Governments around the world agree that maintaining and restoring
ecological connectivity is a key component of biodiversity conservation
(e.g., Convention on Biological Diversity 2010; Convention on Migratory
Species 2020). To target their conservation and restoration efforts
appropriately, practitioners must be informed by rigorous and reliable
connectivity modeling research. This requires, at a minimum, that
researchers consistently articulate the type of connectivity being
modeled, undertake sensitivity and uncertainty analyses, and validate
the connectivity model outputs with independent data. We encourage
connectivity modelers to be up-front and transparent about which, if
any, of these steps have been completed, and reviewers of connectivity
modeling manuscripts to request this information if it is lacking. These
simple steps will enable consumers of the research, including
practitioners, to more readily assess its reliability and utility.
Without these measures, we risk mis-use of limited conservation
resources and failing to meet key conservation targets.