We developed an a priori analytical framework to guide the systematic review process (Appendix A). We systematically searched for randomized controlled trials (RCTs) that tested the efficacy or comparative effectiveness of treatments involving newer drugs in men with LUTS attributed to BPH. We defined these newer drugs as those that have been FDA approved for BPH since 2008 or which have been studied for treatment of BPH are not currently FDA approved for this indication. We searched Ovid Medline®, Ovid Embase®, and the Cochrane Central Register of Controlled Trials (CENTRAL) using subject headings and natural language for the concept of BPH and natural language terms for each included drug class and drug with filters for study design (Appendix B) to identify relevant RCTs through July 2015. We additionally searched for large (n≥100), longer-term (≥1 year duration) observational studies to assess long-term or rare treatment associated harms. We supplemented the bibliographic database search with forward and backward citation searching of relevant systematic reviews and other key references. We will update searches while the draft report is under public/peer review.

Titles and abstracts were screened by two independent investigators to identify studies meeting PICOTS framework. All studies identified as relevant by either investigator underwent full-text screening by two investigators to determine if inclusion criteria were met. We included trials published in English that studied the PICOTS described above. Inclusion criteria did not restrict RCTs by minimal sample size. Differences in screening decisions were uncommon and resolved by consultation between investigators. If necessary, consultation with a third investigator was used to make the final decision. We searched ClinicalTrials.gov and the Food and Drug Administration Web site to identify additional completed and ongoing studies for inclusion and assessment of reporting bias.

Data were extracted to evidence and outcomes tables by one investigator and reviewed and verified for accuracy by a second investigator. Data were extracted from crossover trials at time points before crossover. Postcrossover data were not used. Risk of bias of eligible studies were assessed using AHRQ guidance by one investigator and reviewed by a second.¹⁰ Relevant components included participant selection, method of randomization, allocation concealment, blinding, completeness of followup (attrition), and appropriateness of analytic methods. Investigators conferred with each other to reconcile discrepancies in overall risk of bias assessments. Overall summary risk of bias assessments for each study were classified as low, moderate, or high based upon the collective risk of bias and confidence that the study results were believable given the study's limitations.

We assessed clinical and methodological heterogeneity and variation in effect size to determine appropriateness of pooling data.¹¹ When three or more trials reported similar comparisons and outcomes, data were pooled using a Hartung, Knapp, Sidik, and Jonkman (HKSJ) method¹² random effects model for I-PSS responders or mean changes in I-PSS scores in Stata.¹³ We pooled other outcomes in RevMan¹⁴ and converted DerSimonian-Laird random effects confidence intervals to HKSJ confidence intervals using an excel spreadsheet provided in Inthout et al.¹² Risk ratios (RR) with corresponding 95 percent confidence intervals (CI) were estimated for binary outcomes and weighted mean differences (WMD) and/or standardized mean differences (SMD) with the corresponding 95 percent CIs were estimated for continuous outcomes. We assessed between study variance with Tau² and measured the magnitude of heterogeneity with the I² statistic. If substantial heterogeneity was present (i.e. I² ≥70%), we stratified the results to assess treatment effects based on patient or study characteristics and/or explored sensitivity analyses.^11,15

We interpreted efficacy and comparative effectiveness using established thresholds indicating clinical significance. Table 3 provides a list of these instruments, basic characteristics, and relevant thresholds for classifying improvement.¹⁶ Barry et al. conducted an anchor-based study to identify the minimal detectable difference (MDD) in I-PSS and BPH Impact Index (BII) scales.

Table 3

Symptom and quality of life scales measuring LUTS attributed to BPH.

When the established MDD or other valid threshold was used in the original research to classify individuals as responders and nonresponders, we pooled those results. When mean scale scores or mean change in scale scores for instruments with established MDDs, we used the MDD to interpret the WMD. Johnson et al. suggest an interpretation of the differences between groups in relation to the established minimal important difference.¹⁷ This approach suggests that when the WMD is equal to or larger than the MDD, many patients may have gained detectable benefits from treatment; when the WMD is at least half of the MDD but less than the MDD, an appreciable number of participants have likely achieved a clinically meaningful improvement; and when the WMD is less than one-half of the MDD, it is unlikely that an appreciable number of participants achieve detectable benefits. Following this guidance, we concluded that statistically significant differences were clinically meaningful when the WMD was at least 50 percent of the MDD. Therefore, the statistically significant WMD between treatment groups for post-treatment or change in I-PSS must be equal to or greater than -1.5 and the WMD between treatment groups for post-treatment or change in BII must be equal to or greater than -0.25. No threshold was established for the I-PSS QoL (quality of life) question. Responses to this question are ordinal and range from 0 to 6. We used an MDD of 1 to assess efficacy and comparative effectiveness. Therefore, if this question was analyzed as a continuous variable, we required statistical significance and a WMD of at least 0.5 to conclude a clinically meaningful difference.

The overall strength of evidence (SoE) for primary outcomes of KQ1 within each comparison was evaluated based on the number and size of trials, point estimate(s), relative difference or equivalence of the comparison-outcome, placement of the CI, and the assessed SoE domains (five required domains and three optional domains). The five required domains include: (1) study limitations (risk of bias); (2) directness (single, direct link between intervention and outcome); (3) consistency (similarity of effect direction and size among studies); (4) precision (degree of certainty around an estimate assessed in relationship to MDD); and (5) reporting bias.¹⁸ Optional domains of dose-response association, plausible confounding that would increase the observed effect, and strength of association were assessed to potentially upgrade strength of evidence assessments based upon required domains.¹⁸ Based on these elements, we assessed the overall SoE for each comparison and outcome as:

• High: We are very confident that estimate of effect lies close to the true effect for this outcome. The body of evidence has few or no deficiencies. We believe that the findings are stable, i.e., another study would not likely change the conclusion.
• Moderate: We are moderately confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has some deficiencies. We believe that the findings are likely to be stable, but some doubt remains.
• Low: We have limited confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has major or numerous deficiencies (or both). We believe that additional evidence is needed before concluding either that the findings are stable or that the estimate of effect is close to the true effect.
• Insufficient: We have no evidence, we are unable to estimate an effect, or we have no confidence in the estimate of effect for this outcome. No evidence is available or the body of evidence has unacceptable deficiencies, precluding reaching a conclusion.”¹⁸

Applicability of studies was determined according to the PICOTS framework. Study characteristics that may affect applicability include, but are not limited to, the population (age, race, and country from which the study participants were enrolled), narrow eligibility criteria, and patient and intervention characteristics potentially associated with treatment response different than those described by population studies.¹⁹