Literature Search

To identify relevant citations, we searched MEDLINE^®, the Cochrane Database of Systematic Reviews^®, the Cochrane Central Register of Controlled Trials^®, and the International Pharmaceutical Abstracts (through September 2010), using terms for included drugs, indications, and study designs (see Appendix E for complete search strategies). We limited the electronic searches to “human” and “English language.” We attempted to identify additional studies through hand searches of reference lists of included studies and reviews. In addition, we searched the FDA’s Center for Drug Evaluation and Research (CDER) and Center for Biologics Evaluation and Research (CBER), the Canadian Agency for Drugs and Technology in Health, and the National Institute for Health and Clinical Excellence web sites for medical and statistical reviews, and technology assessments. Finally, we searched dossiers submitted by pharmaceutical companies for the current review. All citations were imported into an electronic database (Endnote^® v. X.02).

Study Selection

All citations were reviewed for inclusion using the criteria shown in Table 5. Two reviewers independently assessed titles and abstracts, where available, of citations identified from literature searches. If both reviewers agreed that the trial did not meet eligibility criteria, it was excluded. Full-text articles of potentially relevant citations were retrieved and again were assessed for inclusion by two reviewers. Disagreements were resolved by consensus. Results published only in abstract form and unpublished data were not included unless adequate details were available for quality assessment.

Table 5

Study inclusion criteria.

We reviewed the literature using a hierarchy of evidence approach, where the best evidence is the focus of our synthesis for each question, population, intervention and outcome addressed. Results from well-conducted, systematic reviews and head-to-head trials provide the strongest evidence to compare drugs with respect to effectiveness, efficacy, and adverse events; head-to-head trials were defined as those comparing one included treatment of interest (those listed in Table 5) with another treatment of interest. If sufficient evidence was available from head-to-head trials we did not examine placebo-controlled trials for general efficacy/effectiveness. If no head-to-head evidence was published, as was the case for omalizumab, we reviewed placebo-controlled trials. We did not include studies that compare step-down therapy for people with stable asthma, different doses of the same medication, or different delivery devices with the same medication unless there was another eligible comparator arm. We did not include studies evaluating adjustable dosing strategies.

A review was considered to be systematic if it presented a systematic approach to reviewing the literature through a comprehensive search strategy, provided adequate data from included studies, and evaluated the methods of included studies (with quality review/critical appraisal).

Data Abstraction

We designed and used a structured data abstraction form to ensure consistency in appraisal for each study. Trained reviewers abstracted data from each study. A second reviewer read each abstracted article and evaluated the accuracy and completeness of the data abstraction. We abstracted the following data from included trials: study design, population characteristics (including age, sex, asthma severity, smoking status), inclusion and exclusion criteria, interventions (drugs, dose, delivery device, duration), comparisons, numbers enrolled, additional medications allowed, outcome assessments, attrition, withdrawals attributed to adverse events, results, and adverse events reported. We recorded intention-to-treat (ITT) results if available.

Validity Assessment (Quality Assessment)

Two independent reviewers assigned quality ratings; they resolved any disagreements by discussion or by consulting a third, senior reviewer. We assessed the internal validity (quality) of trials based on the predefined criteria (see www.ohsu.edu/drugeffectiveness). These criteria are based on the U.S. Preventive Services Task Force and the National Health Service Centre for Reviews and Dissemination (U.K.) criteria.^{14, 15}

Elements of internal validity assessment for trials included, among others, the methods used for randomization, allocation concealment, and blinding; the similarity of compared groups at baseline; maintenance of comparable groups; adequate reporting of dropouts, crossover, adherence, and contamination; overall and differential loss to follow-up; and the use of intention-to-treat analysis.

We assessed observational study designs based on the potential for selection bias (methods of selection of subjects and loss to follow-up), potential for measurement bias (equality, validity, and reliability of ascertainment of outcomes), and control for potential confounders.

Systematic reviews which fulfilled inclusion criteria were rated for quality using predefined criteria (www.ohsu.edu/drugeffectiveness): a clear statement of the questions and inclusion criteria; adequacy of the search strategy; quality assessment of individual trials; the adequacy of information provided; and appropriateness of the methods of synthesis.

Studies that had a fatal flaw were rated “poor quality” and were not included in the evidence report. Trials that met all criteria were rated “good quality”. The remainder received a quality rating of “fair”. This includes studies that presumably fulfilled all quality criteria but did not report their methodologies to an extent that answered all our questions. As the fair-quality category is broad, studies with this rating vary in their strengths and weaknesses: the results of some fair-quality studies are likely to be valid, while others are only probably valid. A poor-quality trial is not valid—the results are at least as likely to reflect flaws in the study design as the true difference between the compared drugs. A fatal flaw is reflected by failing to meet combinations of items of the quality assessment checklist.

Attrition, or loss to follow-up, was defined as the number of persons randomized who did not reach the endpoint of the study,¹⁶ independent of the reason and the use of intention-to-treat analysis. We adopted no formal cut-off point for loss to follow-up because many studies defined withdrawals due to acute worsening of the disease as an outcomes measure.

Data Synthesis

We constructed evidence tables showing the study characteristics, quality ratings, and results for all included studies. Trials that evaluated one included medication against another provided direct evidence of comparative effectiveness and adverse event rates. These data are the primary focus. In theory, trials that make comparisons with other drug classes or placebos can also provide evidence about effectiveness. This is known as an indirect comparison and can be difficult to interpret for a number of reasons, primarily issues of heterogeneity between trial populations, interventions, and assessment of outcomes. Data from indirect comparisons are used to support direct comparisons, where they exist, and are also used as the primary comparison where no direct comparisons exist. Such indirect comparisons should be interpreted with caution.

In addition to discussion of the findings of the studies overall, quantitative analyses were conducted using meta-analyses on outcomes for which a sufficient number of studies reported and for studies which they were homogeneous enough such that combining their results can be justified. Otherwise, the data are summarized qualitatively. Random effects models were used for the estimation of pooled effects.¹⁷ Forest plots are presented to graphically summarize the study results and the pooled results.¹⁸ The Q -statistic and the I² statistic (the proportion of variation in study estimates due to heterogeneity) were calculated to assess heterogeneity between the effects from the studies.^{19, 20} Potential sources of heterogeneity were examined with subgroup analysis by factors such as study design, study quality, variations in interventions, and patient population characteristics. Meta-analyses were conducted using Comprehensive Meta Analysis V2.exe.

Grading the Strength of Evidence

We graded strength of evidence using a modified GRADE approach that included assessment of the following domains: design, quality, consistency, directness, and magnitude of effect of the set of studies relevant to the question. We also considered other domains that may be relevant for some scenarios, such as equipotency (for inhaled corticosteroids), a dose-response association, strength of association (magnitude of effect), and publication bias.

Table 6 describes the grades of evidence that can be assigned. Grades reflect the strength of the body of evidence to answer key questions on the comparative effectiveness, efficacy, and harms of the drugs included in this review. Grades do not refer to the general efficacy or effectiveness of pharmaceuticals. Two reviewers assessed each domain for each comparison and differences were resolved by consensus.

Table 6

Definitions of the grades of overall strength of evidence.

We graded the strength of evidence for the outcomes deemed to be of greatest importance to decision makers and those most commonly reported in the literature. These included improvement in symptoms, exacerbations, rescue medication use, growth, overall adverse events, and asthma-related death. Because of time and resource constraints we did not grade the strength of evidence for every possible outcome reported everywhere in the included literature.