Changes in biodiversity reflect processes acting at multiple spatial scales, including globally, among habitats and within communities. This complexity makes it difficult to analyse the mechanisms that change biodiversity over time. To resolve this, we propose a novel approach to partition temporal changes in biodiversity into contributions from selection at multiple scales. We apply this approach to study changes in the biodiversity of invertebrate herbivores from a large-scale, plant community experiment. Though the experiment was designed to foster distinct insect communities due to differences in host plants, our approach shows that selection among these treatments was a negligible facet of diversity change. Instead, the dominant source of community dynamics was rapid changes in the relative abundances of individual species. These shifts produced surprisingly small changes in biodiversity. More broadly our work highlights how total change in biodiversity across a biogeographic region can be partitioned into logically distinct mechanisms.

Experiments suggest that increasing species’ diversity increases productivity over short observation periods. Theory predicts that diversity may increases productivity at equilibrium, such as in communities of coexisting competitors. In effect, experiment and theory are using different response variables. We illustrate how this gap can obscure some of the mechanisms through which diversity shapes productivity. To do this, we propose a formal link to dynamic models of population growth. Using this link, we show that selection (as it is formally defined in dynamic models) does not affect productivity. Instead, selection changes productivity over time. In contrast, niche complementarity can change productivity directly. We illustrate how to measure each of the terms in competition experiments. These results highlight how the effects of diversity on productivity can be obscured by our choice of response variable.

Biodiversity describes the variety of organisms on planet earth. Ecologists have long hoped for a synthesis between analyses of biodiversity and analyses of biotic interactions among species, such as predation, competition, and mutualism. However, it is often unclear how to connect details of these interactions with complex modern analyses of biodiversity. Using methods pioneered in studies of ecological-evolutionary dynamics, we link biotic interactions and changes in measures of biodiversity such as Hill numbers. We show that analyses of biodiversity obscure details about biotic interactions. For example, identical changes in biodiversity can arise from predation, competition or mutualism, locally or across a metacommunity. Our approach indicates that traditional models of community assembly miss key facets of diversity change. Instead, we suggest that analyses of diversity change should focus on partitions, which measure mechanisms that directly shape changes in diversity, notably relative fitness and immigration, rather than traditional analyses of biotic interactions.

Long-term ecological studies have consistently reported slower than expected changes in biodiversity over time. One explanation for this phenomenon is that commonly used diversity measurements such as species richness are too coarse to detect mechanisms shaping community assembly. It remains unclear whether abundance based diversity measurements are susceptible to the same problem. To test this, we study temporal changes in abundance based diversity indices across 3341 observations from 880 plots from 15 long-term vegetation plot studies. We then partition diversity change into mechanisms of interest to ecologists: selection, drift, and immigration. We show that these mechanisms are an imperfect predictor of temporal diversity change, creating a mismatch between changes in species abundances and changes in diversity, particularly when shifts in diversity are rapid. To resolve this mismatch, we quantify a less studied mechanism “rarity shifts”, which measure how an individual’s contributions to diversity changes over time. We found rarity shifts are an important component of diversity change across many studies. Furthermore, rarity shifts tend to oppose other mechanisms, particularly selection. Therefore, rarity shifts obscure changes in relative abundance from abundance based diversity measurements, revealing why diversity changes appear slower than expected. Ultimately, understanding rarity shifts can lead to a more accurate understanding of the rate and nature of temporal diversity change in ecology and conservation.

Changes in biodiversity reflect processes acting on the success of individual species at multiple spatial scales, including in communities, biogeographic regions, and globally. This complexity makes it difficult to analyse the mechanisms shaping diversity change using traditional approaches. To resolve this, we propose a novel approach to partition total biodiversity changes according to mechanisms reflecting species' success at multiple scales. We apply this approach to study changes in the diversity of invertebrate herbivores from a large-scale, plant community experiment. This partitioning showed that rapid changes in the relative abundances of individual species resulted in surprisingly small changes in diversity across scales. Our novel analytical method reveals how strong ecological effects at different hierarchical levels can counteract each other, resulting in weak effects on diversity across broad spatial scales.