Permafrost peatlands play an important role in global carbon cycle. However, the initiation and development of the permafrost peatlands and their response to climate change remains unclear, hindering our understanding of the past and future of this region. Here we reconstructed the evolution process of permafrost peatland in the Greater Khingan mountains (GKM) from Northeast China, since 3500 cal. yr BP using palynological evidence from permafrost peat cores, as well as an AMS 14C dating. The results indicated that from 3500 to 2900 cal. yr BP, the vegetation mainly consisted of Pinus, thermophilic broadleaved trees, and Polypodiaceae. From 2900 to 2250 cal. yr BP, the vegetation mainly consisted of Pinus, thermophilic broadleaved trees, and Artemisia, with the peatland initiation period characterized by a warm and humid climate. From 2250 to 1650 cal. yr BP, the vegetation mainly consisted of Pinus, Betula and Polypodiaceae, with cold and wet climates lead to an initiation of peatland accumulation. From 1650 to 750 cal. yr BP, the vegetation principally consisted of Pinus and Artemisia, and the dry, cold climate led to a slowdown or stagnation in peatland development. Late in this period, the warmer, wetter climate allowed the peatland to develop again, thereby completing the transition from a eutrophic peatland to a mesotrophic peatland. Since 750 cal. yr BP, the vegetation has mainly consisted of Pinus, Alnus and Cyperaceae, indicating a colder and wetter climate, and the peatland shifted to an oligotrophic state. Our results showed that the evolution of the GKM’s permafrost peatlands mainly influenced by climate, and permafrost peatlands development in the future will depend upon global climate change trends.