3.3 Performance of the image-matching software packages
For both the Kenyan and Zimbabwean datasets, Hotspotter achieved the
highest image-matching accuracy (Figure 3). For the Kenyan dataset,
using Hotspotter with crops of the full individual from which the
background was removed, was most effective. This method detected 62% of
the matches in the 10 highest ranked crops (Figure 3B). This was
significantly higher than using manually cropped flanks of the
individual in both WildID (z = 5.0, p < 0.01) and Hotspotter
(z = 2.8, p = 0.046), as well as crops of the full individual in WildID
(z = 4.7, p < 0.01) and I3S-Pattern (z =
5.0, p < 0.01).For the Zimbabwean dataset, Hotspotter detected
88% of matches within the first 10 ranked images when the background
was removed from a crops of the full individual (Figure 3A). The
matching performance was significantly lower when crops of just the
flank were used (z = 2.7, p = 0.03). Hotspotter with background removal
performed significantly better than WildID with background removal, (z =
4.7, p < 0.01), as well as WildID with crops of the flanks (z
= 5.1, p < 0.01).
The probability of accurate image-matching occurring within the first 10
ranked images was significantly higher for wild dogs from Zimbabwe than
wild dogs from Kenya (OR = 9.64, 95% CI 3.65 - 15.63, Figure 4). The
proportion of matched individuals identified in this analysis was not
significantly associated with image size
(X21 = 0.16, p = 0.69) or image
quality (ORQuality Score 2 / Quality Score 1 = 0.89,
95% C.I. -2.26 – 4.04, ORQuality Score 3 / Quality
Score 1 = 1.82, 95% C.I. -2.20 – 5.83). In addition, the image
quality score did not differ between the populations (W = 15008, p =
0.33).
4. Discussion
This study presents a novel framework for automating the individual
recognition of species with distinct marks. The framework includes an
automated pre-processing method for identifying images suitable for
image-matching, and then using image-matching software for individual
recognition. The automated pre-processing method consists of five steps
that (1) crop all images containing animals from a large database, (2)
filter out a portion of the unsuitable images based on image aspect
ratio, (3) use convolutional neural nets to select images of standing
individuals (accuracy of 90%), (4) separate images into left and right
flanks (accuracy of 95%), and (5) remove image backgrounds. As a case
study, we applied the described methods to an image catalogue of African
wild dogs and found that Hotspotter (Crall et al., 2013) was the
most efficient software package for matching images. Image-matching
performance was also significantly improved by using the full image of
an individual from which the background was removed, as opposed to just
the cropped flank. Finally, we found that image-matching performance
differed between populations of wild dogs with different coat coloration
patterns. This work showed that image-matching software could become a
powerful method for monitoring populations of African wild dogs.
However, caution is needed as detection rates are likely to vary between
– and even within – populations. This could affect the certainty of
derived population-specific demographic parameters, such that careful
consideration is needed to account for individual heterogeneity in
detection when large variation in coat colouration occurs within a
population.
The automated pre-processing method presented in this study could
eliminate the need to manually select suitable images for image-matching
and crop individuals from original photographs. This method thus enables
processing of large image catalogues where selection using visual
inspection would be extremely time consuming. We found that the method
does discard a small number of suitable images, and therefore in
situations where it is important to include all suitable images, the
pre-processing method outlined here could also be used as a pre-sorting
approach. The user could then visually review images that were
classified as not suitable, to prevent usable images from being
discarded.
The described method of pre-processing is particularly useful for wild
dogs, since an individuals’ posture varies substantially between images.
Images taken by tourists provide an opportunity to bolster and spatially
extend image catalogues. However, these images are also likely to
contain many images unsuitable for identification, as they are not taken
for the purpose of identification. Accordingly, filtering unsuitable
images from these datasets using an automated approach could be
especially timesaving. The described pre-processing method is therefore
highly suitable to species targeted by wildlife watching excursions,
that have distinctive marks and where individual posture influences
image suitability, for example cheetahs, leopards Panthera
pardus , and tigers.
Hotspotter outperformed I3S-Pattern and Wild-ID at
matching images of individual wild dogs. This finding agrees with
studies on green toads that compared Hotspotter and
I3S-pattern (Burgstaller, Gollmann & Landler 2021),
as well as studies comparing Hotspotter and WildID (Nipko, Holcombe &
Kelly, 2020; Burgstaller, Gollmann & Landler, 2021; Chehrsimin et al.,
2018). Nevertheless, this result is not ubiquitous. Wild-ID was superior
to Hotspotter at matching images for a blotched amphibian species, the
Wyoming toad Anaxyrus baxteri (Morrison et al., 2016). This
indicates that the identification performance of different software
packages is dependent on species, even when two species’ patterns show
similarities. Consequently, we recommend that all three software
packages are tested on new species before deciding on which one to use.
Using crops of full individuals from which the background was removed
significantly increased the image-matching accuracy of Hotspotter,
compared to using crops of just individuals’ flanks. This method also
speeds up image pre-processing by eliminating the need to manually crop
the region of interest. The improved accuracy is likely caused by two
factors. Firstly, removing the background prevents images being matched
based on similar backgrounds, as the flanks are not perfect rectangles,
meaning that crops of the flank also contain some background (see Figure
S2). Secondly, using complete individuals allows images to be matched
based on unique features on the legs, in addition to the flanks. This
result is in line with studies on Saimaa ringed seals Pusa
hispida and Thornicroft’s giraffes Giraffa camelopardalis
thornicrofti , which found evidence that using a full individual from
which the background is removed, could result in a higher accuracy
(Chermin et al., 2018; Halloran, Murdoch & Becker 2015).
However, neither of these previous studies statistically tested whether
background removal increased identification accuracy. Our study
therefore provides the first statistical evidence that background
removal can increase the performance of image-matching software. This
also indicates that the common usage of Hotspotter, in which a
rectangular region of interest is manually cropped (e.g. Dunbar et
al. , 2021; Nipko, Holcombe & Kelly, 2020), could be improved by
removing the image background.
Hotspotter was significantly better at matching images from Zimbabwean
wild dogs, compared to Kenyan individuals. The higher image-matching
accuracy found for the Zimbabwean population is likely to reflect the
regional difference in wild dog coat colouration patterns. The Kenyan
population has darker, more uniform coats, consisting of large black
patches, often with few white or tan areas (McIntosh, Woodroffe &
Rabaiotti, 2016, Daniels, Woodroffe & Rabaiotti, 2022). By contrast,
the proportion of tan fur is ~1.5 times higher, and the
proportion of white fur is almost 7 times higher for the Zimbabwean
population (Figure 1, Daniels, Woodroffe & Rabaiotti, 2022). Therefore,
the higher contrast within the patterns of the Zimbabwean wild dogs
could make it easier for the software to match images of these
individuals. The identified relationship between image-matching
performance and software package remained unaltered when image quality
and image size were included in analyses, and there was no significant
difference between the image quality scores between the Zimbabwean and
Kenyan populations. The image quality score approach was modelled after
Nipko, Holcombe & Kelly (2020), who found that it significantly
affected the probability of matching ocelot and jaguar individuals. As a
result, we are confident that the differences in coat colouration
patterns between wild dogs from Zimbabwe and Kenya reflect variation in
identification performance between populations.
Inter-population variation in image-matching performance indicates that
detection probabilities derived from using this approach will not be
directly comparable between populations. Since the probability of
finding an accurate image-match depends on individual coat pattern, this
finding highlights that individual heterogeneity in detection may also
occur if large variation in coat colouration occurs within a population.
Capture-mark-recapture techniques assume individuals experience equal
detection probability across a population (White and Burnham, 2009).
Therefore, individual coat pattern may also need accounting for when
deriving survival estimates using such analysis. This also applies to
other species whose coat pattern varies regionally, such as Asian golden
cats Catopuma temminckii and ocelots (Allen et al., 2011; Khan,
Ali & Mohammed, 2017). Furthermore, the coat patterns of other wild dog
populations can differ considerably from the two populations included in
this study (McIntosh, Woodroffe & Rabaiotti, 2016, Daniels, Woodroffe
& Rabaiotti, 2022). Consequently, we advocate that estimating a
population-specific image-matching accuracy score becomes an essential
pre-requisite step for applying these techniques in different locations.
Automatically pre-processing wild dog image datasets and using
image-matching software facilitates the use of archived and citizen
science image catalogues where visually identifying all individuals
would be extremely time-consuming. Although the best performing
image-matching software did not detect all matches, it could be used to
identify a large proportion of the individuals in a dataset. Afterwards,
individuals that were not matched to any other images could be visually
identified, to prevent missing actual matches. Using image-matching
software in this way still saves time by rapidly identifying a large
portion of the matches, without compromising on accuracy. Furthermore,
it is plausible that the likelihood of correctly detecting matching
images increases if more than two images per individual are included,
for example if multiple viewpoints per individual are present in a
dataset, the probability of matching these is expected to increase
(Crall et al., 2013). Our accuracy values therefore represent a
conservative estimate of Hotspotter’s true accuracy.
Our study indicates that image-matching could provide a valuable new
approach for monitoring wild dogs. A combination of citizen science and
image-matching has already been successfully employed to monitor other
species, such as Blanding’s turtles Emydoidea blandingii and
whale sharks (Araujo et al. , 2017; Cross et al. , 2021).
Similarly, previous studies have used tourist images to estimate the
population size of wild dogs in Kruger National Park, South Africa
(Marnewick et al., 2014). Combining citizen science, image-matching
software, and capture-recapture methods therefore has the potential to
improve understanding of wild dog demography. However, more research is
needed to investigate whether photographic data could improve our
understanding of wild dog demography beyond population size, by
estimating parameters such as pack structure, dispersal rates, and death
and birth rates. This can be achieved by applying image-matching
software to existing image datasets, to assess whether they generate
enough data to estimate key demographic parameters, or whether more
intensive monitoring - for example using long term camera trap surveys -
would be necessary.
In conclusion, we have developed a new automated method for
pre-processing image datasets, by automatically cropping animals from
images, removing images in which the individuals’ posture hinders
identification, separating left and right flanks, and removing the image
background. This framework will enable large image datasets to be
analysed rapidly, thereby expanding monitoring efforts and expediting
conservation action. Furthermore, we have shown how well different
image-matching software packages perform on African wild dogs.
Hotspotter outperformed the other software packages, while its
performance differed between two populations which exhibit
intra-specific variation in their coat patterns. Our pre-processing
method, in combination with Hotspotter, has immediate application in
research and monitoring efforts for wild dogs and other species. Data
obtained in this way could provide cost-effective large-scale monitoring
for endangered species, therefore supporting the implementation of
effective conservation.