During antiretroviral therapy (ART) that suppresses HIV replication to below the limit-of-quantification, virions produced during ART can be detected at low frequencies in the plasma, termed residual viremia (RV). We hypothesized that a reservoir of HIV-infected cells actively produce and release virions during ART that are potentially infectious, and that following ART-interruption, these virions can complete full-cycles of replication and contribute to rebound viremia. Therefore, we studied the dynamics of RV sequence variants in 3 participants who initiated ART after ~3 years of infection and were ART-suppressed for >6 years prior to self-initiated ART-interruptions. Longitudinal RV C2V5env sequences were compared to sequences from pre-ART plasma, supernatants of quantitative viral outgrowth assays (QVOA) of cells collected during ART, post-ART-interruption plasma, and ART-re-suppression plasma. Identical, "putatively clonal," RV sequences comprised 8-84% of sequences from each timepoint. The majority of RV sequences were genetically similar to those from plasma collected just prior to ART-initiation, but as the duration of ART-suppression increased, an increasing proportion of RV variants were similar to sequences from earlier in infection. Identical sequences were detected in RV over a median of 3 years (range: 0.3-8.2) of ART-suppression. RV sequences were identical to pre-ART plasma viruses (5%), infectious viruses induced in QVOA (4%) and rebound viruses (5%) (total n = 21/154 (14%) across the 3 participants). RV sequences identical to ART-interruption "rebound" sequences were detected 0.1-7.4 years prior to ART-interruption. RV variant prevalence and persistence were not associated with detection of the variant among rebound sequences. Shortly after ART-re-suppression, variants that had been replicating during ART-interruptions were detected as RV (n = 5). These studies show a dynamic, virion-producing HIV reservoir that contributes to rekindling infection upon ART-interruption. The persistence of identical RV variants over years suggests that a subpopulation of HIV-infected clones frequently or continuously produce virions that may resist immune clearance; this suggests that cure strategies should target this active as well as latent reservoirs.