The formation of stable radiation damage in crystalline solids often proceeds via complex dynamic annealing processes, involving migration and interaction of ballistically-generated point defects. The dominant dynamic annealing processes, however, remain unknown even for crystalline Si. Here, we use a pulsed ion beam method to study defect dynamics in Si bombarded in the temperature range from -20 to 140 °C with 500 keV Ar ions. Results reveal a defect relaxation time constant of ~10-0.2 ms, which decreases monotonically with increasing temperature. The dynamic annealing rate shows an Arrhenius dependence with two well-defined activation energies of 73 ± 5 meV and 420 ± 10 meV, below and above 60 °C, respectively. Rate theory modeling, bench-marked against this data, suggests a crucial role of both vacancy and interstitial diffusion, with the dynamic annealing rate limited by the migration and interaction of vacancies.