1. Many plants compensate for the damage caused by herbivorous insects through tolerance responses. Besides directly causing plant tissue loss and seed production reduction, herbivory causes phenological changes in the host plant. However, little is known about the fitness costs of phenological changes caused by tolerance responses to herbivorous attacks. 2. The girdling beetle Phytoecia rufiventris caused a short-term decrease in the number of flowers of the host plant Erigeron annuus. However, accelerated growth restored the number of flowers, but after a 2-week delay. With an objective to examine whether the tolerance response with such a delay fully compensates the fitness, we experimentally reproduced a 2-week delay in germination under greenhouse and field settings. Under both conditions, intraspecific competition resulted in serious defects in the growth and reproduction of E. annuus plants which of germination was delayed. However, delayed germination resulted in better growth when competition and herbivory were eliminated from the field. 3. Thus, we showed that the tolerance response to restore reproductive production does not fully compensate for the fitness loss caused by insect attack; rather, the delay in seed production in attacked plants leads to delayed germination and subsequent inferiority in intraspecific competition. 4. Synthesis. Our results imply that compensation for floral production after an herbivore attack does not fully restore offspring fitness in the presence of intraspecific competition and herbivory. Assessing the ecological consequences of defense traits in an appropriate layer of interaction is critical to interpreting adaptive values.