Soil seed banks may offer great potential for restoring and maintaining desert ecosystems that have been degraded by climate change and anthropogenic disturbance. However, few studies have explored the annual dynamics in the composition and relative abundance of these soil seed banks. We conducted a long-term observational study to assess the effects of environmental factors (meteorology and microtopography) and aboveground vegetation on the soil seed bank of the Tengger Desert, China. The desert seed bank was dominated by annual herbs. We found that more rainfall in the growing season increased the number of seeds in the soil seed bank, and that quadrats at relatively higher elevations had fewer seeds. The species composition had more similarity in the seed bank than in the aboveground vegetation, though the seed bank and aboveground vegetation did change synchronously due to the rapid propagation of annuals. Together, our findings suggest that the combined effects of environmental factors and plant life forms determine the species composition and size of soil seed banks in deserts. Thus, if degraded desert ecosystems are left to regenerate naturally, the lack of shrub and perennial herb seeds could crucially limit their restoration. Human intervention and management may have to be applied to enhance the seed abundance of longer-lived lifeforms in degraded deserts.