Criteria:

• Study limitations (low, medium, or high level of study limitations)

  • Rated according to the degree to which studies for a given outcome are likely to reduce bias based on study design and conduct across individual studies. Evidence was rated down for study limitations when higher-quality studies were not available or if there were few higher-quality trials and estimates differed in analyses stratified by study quality.
• Consistency (consistent, inconsistent, or unknown/not applicable)

  • Rated by degree to which studies find similar magnitude of effect (i.e., range sizes are similar) or same direction of effect (i.e., effect sizes have the same sign). When pooled estimates were available, evidence was rated inconsistent if the I² was greater than 40 percent, unless findings were consistent in subgroup analyses and there were sufficient trials (>20) for subgroup analyses to be informative.
• Directness (direct or indirect)

  • Rated by degree to which the outcome is directly or indirectly related to health outcomes of interest. Patient centered outcomes are considered direct
• Precision (precise or imprecise)

  • Describes the level of certainty of the estimate of effect for a particular outcome with a precise estimate being on that allows a clinically useful conclusion. This may be based on sufficiency of sample size and number of events, and if these are adequate, the interpretation of the confidence interval. Evidence was rated imprecise if the pooled estimate confidence interval (CI) crossed the null and the threshold for small magnitude of effect.
• Reporting bias (suspected or undetected)

  • Publication bias, selective outcome reporting, and selective analysis reporting are types of reporting bias. Reporting bias is difficult to assess as systematic identification of unpublished evidence is challenging. If sufficient numbers of RCTs (>10) are available, quantitative funnel plot analysis may be done.

An overall SOE grade of high, moderate, low, or insufficient was assigned, based on a four-level scale by evaluating and weighing the combined results of the above domains. Bodies of evidence consisting of RCTs are initially considered as high strength. The strength of the evidence may be downgraded based on limitations identified in the domains described above. The SOE levels were defined as:

• High—High confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has few or no deficiencies. We believe that the findings are stable, i.e., another study would not change the conclusions.
• Moderate—Moderate confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has some deficiencies. We believe that the findings are likely to be stable, but some doubt remains.
• Low—Limited confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has major or numerous deficiencies (or both). We believe that additional evidence is needed before concluding either that the findings are stable or that the estimate of effect is close to the true effect.
• Insufficient—No evidence, unable to estimate an effect, ore no confidence in the estimate of effect for this outcome. No evidence is available or the body of evidence has unacceptable deficiencies, precluding reaching a conclusion.