Climate change is increasingly impacting temperate forest ecosystems and many forest herbs might be unable to track the changing climate due to dispersal limitations. Forest herbs with a low adaptive capacity are prone to climate change effects and may benefit from conservation strategies mitigating dispersal limitations and evolutionary constraints, such as assisted migration. To assess the vulnerability of forest herbs to climate change it is key to evaluate their adaptive potential and to quantify the genetic offset. To this end, we quantified climate change vulnerability metrics using single nucleotide polymorphisms (SNPs) along a latitudinal gradient of the self-incompatible deciduous forest herb Primula elatior. Southern populations displayed a sharp genetic turnover and a considerable amount of local adaptation under diversifying selection was discovered. However, most of the outlier loci could not be linked to climate variables (71%) and were likely related to other local adaptation drivers, such as photoperiodism. Furthermore, specific adaptations to climate extremes, such as drought stress, could not be detected. Populations in the south of the distribution area had high sensitivity to climate change due to a low adaptive capacity and a moderate genetic offset, while central European populations were sensitive due to a high genetic offset. We conclude that assisted migration with southern source populations could bear significant risk due to local maladaptation and a low adaptive capacity. Local admixture and restoration of ecological connectivity to increase the adaptive capacity and assisted range expansion to suitable habitat in the north could be advised as potential mitigation strategies.