The predicted temperature increase caused by climate change is a threat to biodiversity. Male reproduction is particularly sensitive to elevated temperatures resulting in sterility. Here we investigate temperature induced changes in reproductive tissues and the fertility reduction in male Drosophila melanogaster. We challenged males during development and either allowed them to recover or not in early adulthood, while measuring several determinants of male reproductive success. We found significant differences in recovery rate, organ sizes, sperm production and other key reproductive traits among males from our different temperature treatments. Spermatogenesis and hence sperm maturation was impaired before reaching the upper thermal sterility threshold. While some effects were reversible, this did not compensate the earlier damage imposed. Surprisingly, developmental heat stress was damaging to accessory gland growth and female post mating responses mediated by seminal fluid proteins were impaired regardless of the possibility of recovery. We suggest that sub-lethal thermal sterility and the subsequent fertility reduction is caused by a combination of malfunctioning reproductive traits: inefficient functionality of the accessory gland and alteration of spermatogenesis.