Secondary and plantation forests are the main alternative forests remaining after the deforestation of primary forests. Understanding the conservation value of secondary and plantation forests is important for resource utilization. To explore the impact of forest conversion on biodiversity, we compared multiple diversity metrics (taxonomic, phylogenetic and functional diversity) and community structures of small mammals in the primary, secondary and plantation forests on Mt. Liangshan, Sichuan Province, China. Seven field surveys were conducted to survey local small mammal assemblages between 2016 and 2020. We found that the taxonomic, phylogenetic and functional diversity metrics of small mammals in the three forest types were similar at the landscape scale, while all diversity metrics were lowest in the plantation forest and highest in the primary forests at the site scale. The community structure analysis showed that random processes were dominant across the three forest types, and there was no difference in small mammal community structures among the three forest types. Our results indicated that secondary and plantation forests in the nature reserves, adjacent to the primary forest and exposed to little human disturbance, also can provide important habitats for small mammals.