Birth defects are the leading cause of infant death in the United States and a major cause of death and hospitalization among children of all ages. Despite the devastating impact of birth defects, we understand little of their etiology impeding progress towards treatment and prevention. Our inability to accurately predict, diagnose and treat the common sequelae of birth defects leaves the economic, social and public health burden of birth defects unacceptably high. Emerging genomic sequencing technologies provide an unprecedented opportunity to identify genetic variants, including those that cause birth defects. It is essential that we leverage these advances to enhance our understanding of birth defects. Such an effort will lead to new avenues for treatment and provide benefits to families, including improved understanding of the cause of a child’s condition and the risks to any future children. However, understanding the genetics of human birth defects presents many challenges. This White Paper outlines the need for a birth defect genomics initiative, the challenges to overcome, and suggested solutions. Ultimately, we conclude that understanding birth defects must be a trans-NIH effort and involve the development of a new type of interdisciplinary team working together in all aspects of the enterprise.