Abstract Background Investigating the population structure and demographic history is crucial for elucidating the evolutionary imprints of a species and laying a foundation for effective utilization of its germplasm. Camellia oleifera Abel., a traditional woody oil-producing species, is widely distributed in south of the Yangtze River in China. However, the phylogeography and genetic diversity of wild population remain poorly understood. Results Data for 20 simple sequence repeat (SSR) markers and two chloroplast DNA (cpDNA) fragments was generated to assess the genetic variation and differentiation across 33 natural populations. The integrated Approximate Bayesian Computation (ABC) and ecological niche modeling (ENM) were utilized to analyze the demographic and evolutionary history. The analysis revealed high genetic diversity at the species level and significant genetic variation within populations for both SSR and cpDNA datasets. The genetic evidence based on cpDNA indicated no noteworthy genetic divergence and phylogeographical structure. Haplotype phylogenetic analyses suggest that the temporal pattern of species diversification is likely associated with the abrupt uplift of Hengduan Mountains and adjacent regions (HMR) during the late Miocene to late Pliocene. In contrast, the genetic structure in SSR data revealed a discernible geographic pattern: 33 populations were divided into two lineage regions, namely HMR and Central-East China. The identification of two potential refugia for C. oleifera during the LGM, namely the HMR and southeastern China, was achieved through a comprehensive analysis combining ENM and population structure analysis. Conclusions The presence of distinct genetic patterns observed in cpDNA and SSR makers may account for the divergent hereditary characteristics between the nuclear and chloroplast genomes, resulting in disparate phylogeographic patterns. The Nanling mountain range was regarded as a dispersal conduit that facilitated effective gene flow between the two refugia, leading to the extensive distribution and stable evolutionary trajectory of C. oleifera. These methods and findings can provide a reference for uncovering the demographic history of perennial woody oil crops and facilitate the utilization of wild C. oleifera genetic resources for the genetic breeding.