For full study citations in this appendix, please refer to the report’s main reference list.

Certainty of evidence

We assessed the certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach³⁶ for 4 domains:

View in own window

Domain	Rating	How Assessed
Risk of bias	Low Unclear High	Assessed primarily through study design and aggregate study quality
Consistency	Not serious inconsistency Serious inconsistency Very serious inconsistency	Assessed primarily through whether effect sizes are generally on the same side of “no effect,” the overall range of effect sizes, and statistical measures of heterogeneity
Directness	Not indirect Serious indirectness Very serious indirectness	Assessed by whether the evidence involves direct comparisons or indirect comparisons through use of surrogate outcomes or use of separate bodies of evidence
Precision	Not serious imprecision Serious imprecision Very serious imprecision	Based primarily on the size of the confidence intervals of effect estimates, the optimal information size and considerations of whether the confidence interval crossed a clinical decision threshold

Summary certainty of evidence ratings for a body of evidence:

• High—High confidence that the true effect lies close to that of the estimate of the effect.
• Moderate—Moderate confidence in the effect estimate. The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.
• Low—Limited confidence in the effect estimate. The true effect may be substantially different from the estimate of the effect.
• Very low—Very little confidence in the effect estimate. The true effect is likely to be substantially different from the estimate of effect.
• Insufficient—Impossible or imprudent to rate. In these situations, a rating of insufficient is assigned.

CERQual (Confidence in the Evidence from Reviews of Qualitative Research)

We assessed confidence in the evidence from using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CERQual approach^{36 ,37} for 4 domains:

View in own window

Domain	Rating	How Assessed
Methodologi cal Limitations	No limitations Minor limitations, Moderate limitations, Significant limitations	The extent to which there are problems in the design or conduct of the primary studies supporting a review finding
Relevance	No concerns, Minor concerns, Moderate concerns, Significant concerns	The extent to which the body of evidence from the primary studies supporting a review finding is applicable to the context specified in the review question
Coherence	No concerns, Minor concerns, Moderate concerns, Significant concerns	An assessment of how clear and cogent the fit is between the data from the primary studies and the review finding
Adequacy	No concerns, Minor concerns, Moderate concerns, Significant concerns	The degree of richness and quantity of data supporting a review finding

Summary confidence of evidence ratings:

• High confidence: It is highly likely that the review finding is a reasonable representation of the phenomenon of interest
• Moderate confidence: It is likely that the review finding is a reasonable representation of the phenomenon of interest
• Low confidence: It is possible that the review finding is a reasonable representation of the phenomenon of interest
• Very low confidence: It is not clear whether the review finding is a reasonable representation of the phenomenon of interest

Objective outcomes (ie, non–patient-reported outcomes)

Measures that are not subject to a large degree of individual interpretation and are likely to be reliably measured across patients in a study, by different health care providers, and over time.

Patient-reported outcomes

Outcomes that are directly reported by the patient without interpretation of the patient’s response by a clinician or anyone else and pertains to the patient’s health, quality of life, or functional status associated with health care or treatment.

Risk of bias (ROB)

An assessment of study quality. We used the following guidance in this report.

(1) For KQ 1, we used the Cochrane EPOC ROB tool,²³ which is applicable to randomized and nonrandomized studies:

• Randomization and allocation concealment
• Comparability of groups at baseline
• Blinded outcomes assessment
• Completeness of follow-up and differential loss to follow-up
• Whether incomplete data were addressed appropriately
• Protection against contamination
• Selective outcomes reporting
• Intervention independent from other changes (specific to interrupted time series)
• Intervention pre-specified (specific to interrupted time series)
• Intervention effect on data collection (specific to interrupted time series)

Summary ROB ratings for a study:

• Low ROB—Bias, if present, is unlikely to alter the results seriously
• Unclear ROB—Bias that raises some doubts about the results
• High ROB—Bias that may alter the results seriously

(2) For KQ 2, we used the Critical Appraisal Skills Programme (CASP) tool²⁶ :

• Validity of study results (clarity of aims, appropriate methodology/design/data collection)
• Nature of the results (ethical consideration, rigorous data analysis, clarity of findings),
• Helpfulness of the results (local value).

No summary ROB was possible for the CASP.