Based on the importance of the perforant pathway (PP) for normal hippocampal function, the vulnerability of temporal structures, and significant memory impairment in patients with traumatic brain injury (TBI), we investigated in vivo changes in the PP zone, hippocampus, and temporal lobe white and gray matter using diffusion tensor imaging (DTI) and volumetric analysis, and any specific relations with memory performance (Verbal Selective Reminding Test, Rey-Osterrieth Complex Figure Test), in 14 patients with severe TBI. Compared to a demographically-similar control group, our patients had significantly decreased fractional anisotropy (FA) and higher apparent diffusion coefficient (ADC) for the PP zone bilaterally, and higher ADC bilaterally in the hippocampus. Volumetric analysis revealed significantly decreased volumes in both hippocampi and temporal gray matter bilaterally. Consistent long-term retrieval (CLTR) and delayed recall were significantly related to (1) right and left PP zone ADC, (2) left hippocampus ADC, and (3) left hippocampal volume. Nonverbal memory (immediate and delayed recall) was significantly associated with (1) right and left PP zone ADC, (2) left hippocampal volume, and (3) gray (immediate recall) and white (immediate recall, bilaterally; delayed recall, left) matter temporal volumes. Advanced neuroimaging analysis can detect in vivo changes in the PP zone and temporal structures in patients with severe TBI, with these changes being highly associated with memory impairment.