APOE ε4 allele is a major risk factor in Late-Onset Alzheimer's Disease (AD). Distinct phenotypes that depend on the APOE ε4 status have been demonstrated. The genetic etiology of APOE ε4 non-carriers is still elusive. Thus we investigated the genetic components of AD that is independent of APOE ε4 by combining genome association analysis with quantitative trait analyses in non-Hispanic Caucasian participants in the Alzheimer' s Disease Neuroimaging Initiative (ADNI) cohort. Five top susceptible single nucleotide polymorphisms (SNPs) in three loci in ZNF827, KDM2B and NANP were initially identified in APOE ε4 non-carriers and four of these SNPs were confirmed in mild cognitive impairment. These SNPs and one nominally significant SNP are located in three haplotype blocks. Quantitative trait analyses of these haplotype blocks demonstrated that the haplotype block in ZNF827 was associated with CSF Aβ₄₂ level, and the haplotype block in KDM2B with CSF p-tau₁₈₁p and p-tau₁₈₁p/Aβ₄₂ ratio. The haplotype block between NANP and NINL was associated with brain atrophy. Moreover, these SNPs took additive effects on AD incidence and demonstrated the interaction with APOE ε4 status. Therefore, we conclude that these novel loci are associated with AD in APOE ε4 non-carriers. This study indicates the distinct genetic risk genes for AD non-carrying APOE ε4 and provides new insight into the molecular mechanisms of AD.