In order to investigate the genetic differentiation of Sinorhizobium strains nodulating Glycine max and related microevolutionary mechanisms, three housekeeping genes (SMc00019, truA, and thrA) and 16 symbiosis-related genes on the chromosome (7 genes), pSymA (6 genes), and pSymB (3 genes) were analyzed. Five distinct species were identified among the test strains by calculating the average nucleotide identity (ANI) of SMc00019-truA-thrA: Sinorhizobium fredii, Sinorhizobium sojae, Sinorhizobium sp. I, Sinorhizobium sp. II, and Sinorhizobium sp. III. These species assignments were also supported by population genetics and phylogenetic analyses of housekeeping genes and symbiosis-related genes on the chromosome and pSymB. Different levels of genetic differentiation were observed among these species or different replicons. S. sojae was the most divergent from the other test species and was characterized by its low intraspecies diversity and limited geographic distribution. Intergenic recombination dominated the evolution of 19 genes from different replicons. Intraspecies recombination happened frequently in housekeeping genes and symbiosis-related genes on the chromosome and pSymB, whereas pSymA genes showed a clear pattern of lateral-transfer events between different species. Moreover, pSymA genes were characterized by a lower level of polymorphism and recombination than those on the chromosome and pSymB. Taken together, genes from different replicons of rhizobia might be involved in the establishment of symbiosis with legumes, but these symbiosis-related genes might have evolved differently according to their corresponding replicons.