Whistling thorn acacia (Acacia drepanolobium) forms monodominant stands in black cotton soils in East Africa arid highlands. The tree defends itself against large mammal herbivore with spinescence and symbiotic ants. While these defences have been extensively studied, little is known about the extent to which A. drepanolobium defence may benefit other plants growing in close association. We examined variation in herbaceous vegetation height, biomass and composition between areas underneath A. drepanolobium canopies and the adjacent matrix in both fenced herbivore exclosures and unfenced areas. In unfenced areas, there was more tall herbaceous vegetation and biomass underneath tree canopies than away tree canopies, while these differences were not significant in fenced exclosures. Both height and biomass of understory vegetation were negatively correlated with A. drepanolobium canopy height. In the fenced herbivore exclosures, there was higher species diversity underneath trees than the surrounding matrix, but these differences was not apparent in the unfenced areas. The differences in herbaceous vegetation composition (Bray-Curtis dissimilarity index) between underneath tree and off tree locations were more pronounced in the unfenced areas than within the fenced herbivore exclosures. Our finding suggests that highly defended trees may moderate herbivore effects on herbaceous vegetation. To the extent that herbaceous vegetation underneath trees experiences protection from herbivory, such refugia microhabitats may serve as recolonization nucleus in attempts to restore chronically overgrazed systems.