Habitat fragmentation reduces gene flow, causing genetic differentiation and diversity loss in endangered species through genetic drift and inbreeding. However, the impact of habitat fragmentation on ectomycorrhizal (ECM) fungi remains unexplored, despite their critical roles in forest ecosystems. Here, we investigated the population genetic structure and the demographic history of Rhizopogon togasawarius, the ECM fungus specifically colonizing the host tree Pseudotsuga japonica, across its entire distribution range (>200 km). These two species are designated as endangered species on the IUCN Red List since they are found only in small, fragmented forests in Japan. We analyzed 236 R. togasawarius individuals from five remaining populations across the Kii Peninsula and the Shikoku Island, separated by a sea channel. Simple sequence repeat (SSR) analyses using 20 loci revealed strong genetic differentiation among populations (FST = 0.255), even significant in the nearest population pair separated by a distance of only 8 km (FST = 0.075), indicating extremely limited gene flow between populations. DIYABC-RF analyses implied that population divergence occurred approximately 6000 generations ago between the two regions, and nearly 1500 generations ago between the nearest populations within Shikoku Island, related to past climate events. Because of prolonged genetic isolation, significant inbreeding was confirmed in four of five populations, where effective population sizes became very small (Ne = 9.0–58.0). Although evaluation of extinction risks for microorganisms is challenging, our conservation genetic results indicated that habitat fragmentation increases extinction risk through population genetic mechanisms, and therefore should not be overlooked in biodiversity conservation efforts.