Criteria for Inclusion/Exclusion of Studies in the Review

The criteria for inclusion and exclusion of studies are designed to identify research that can answer the four Key Questions (KQs) concerning early RA specified in the introduction. The criteria are based on the population, intervention/exposure, comparator, outcomes, time frames, country and clinical settings, and study design (PICOTS) shown in Table 2.

Table 2

Eligibility criteria for review of treatments for early rheumatoid arthritis.

Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies To Answer the Key Questions

We systematically searched, reviewed, and analyzed the scientific evidence for each KQ. We included any study population defined as early RA by the authors if the diagnosis was no more than 1 year in the past. We included studies with mixed populations if more than 50 percent of the study populations had an early RA diagnosis. This definition was based on the context that the course of RA is highly variable; some researchers have suggested defining early RA as before development of bone erosion, but some patients never develop erosions. Given this variability, a recent task force of experts in RA and clinical trial methodology recommended defining early RA as no more than 1 year of diagnosed disease duration.²

Because no consensus on the definition of early RA exists, we also internally tracked studies with participants whose RA was between 1 to 2 years of diagnosis to describe the number of studies using this time frame. If studies did not clearly indicate how early RA was defined but met our other PICOTS criteria, we attempted to contact the corresponding author to request clarification of the definition (using a standard email request). We gave authors 2 weeks to respond; if we did not receive a response after a reminder, we did not include the studies in question.

A portion of our literature yield consisted of abstract-only references without full-text manuscripts (e.g., conference abstracts). If we could not locate associated full-text publications, we excluded them because of a lack of information needed to assess risk of bias (ROB).

To identify relevant published literature, we searched the following databases: MEDLINE® via PubMed, the Cochrane Library, Embase, and International Pharmaceutical Abstracts. The search strategies formatted for MEDLINE (Appendix A) comprise medical subject heading (MeSH) terms and natural language terms reflective of RA, drug interventions, and outcomes of interest. We adapted this search strategy for the other databases as needed. An experienced librarian familiar with SRs designed and conducted all searches in consultation with the review team.

The 2012 review had searched from June 2006 to January 2011. For the present update, our literature searches included articles published from July 2010 (to allow 1 year’s indexing time from the 2012 update) to October 5, 2017.

We manually searched the reference lists of included SRs to supplement the main database searches. At the outset, we ensured that our update adequately builds on the body of evidence of the 2012 update, including new drugs approved by the U.S. Food and Drug Administration (FDA) or undergoing FDA review during our review period.

Because the scope of this update is limited to patients with early RA, we carefully examined included studies in the prior review to identify those that focused exclusively on patients with early RA or that had mixed populations of patients in which 50 percent or more had a diagnosis of early RA.

We also searched the gray literature for unpublished studies relevant to this review. Gray literature sources included ClinicalTrials.gov, the World Health Organization International Clinical Trials Registry Platform, the New York Academy of Medicine’s Grey Literature Index, and Supplemental Evidence and Data information from targeted requests and from a Federal Register Notice (public invitation posted in the Federal Register to submit relevant study data to AHRQ on behalf of Evidence-based Practice Centers [EPCs]). From these, we included studies that met all the inclusion criteria and contained enough methodological information to assess ROB. When we updated our published literature search, we also updated the gray literature searches.

To answer the Contextual Questions, we identified relevant literature opportunistically from our literature searches for KQs and used targeted literature searches to address remaining gaps in information.

Literature Review, Data Abstraction, and Data Management

To ensure accuracy, two reviewers independently reviewed all titles and abstracts. We used Abstrackr, an online citation screening tool, to review title and abstract records and manage the results.⁶⁶ We then retrieved the full text for all citations deemed potentially appropriate for inclusion by at least one of the reviewers. Two team members independently reviewed each full-text article for eligibility. We resolved discrepancies by consensus or by involving a third, senior reviewer.

All results at both title/abstract and full-text review stages were tracked in an EndNote® bibliographic database (Thomson Reuters, New York, NY). Appendix B presents the list of studies excluded (with reasons) at the full-text level.

We designed, pilot-tested, and used a structured data abstraction form to ensure consistency of data abstraction. We abstracted data into categories that included (but were not limited to) the following: study design, eligibility criteria, intervention (drugs, dose, duration), additional medications allowed, methods of outcome assessment, population characteristics, sample size, attrition (overall and attributed to adverse events), results, and adverse event incidence. A second team member verified abstracted study data for accuracy and completeness.

Because studies often use more than one instrument to assess the same outcome, we established a hierarchy of outcome measures. We used this hierarchy to prioritize the information we abstracted. Table 3 documents this “priority” approach; preferred outcome measures are shown in bold. If study authors provided data for the preferred outcome measure, we did not abstract data from any other measure that assessed the same outcome. If no specific outcome measures are shown in bold in Table 3 within a category, we did not establish a hierarchy for that outcome.

Table 3

Outcomes and hierarchy of preferred measures for data abstraction.

For adverse events, we abstracted data on overall adverse events, overall study discontinuation, discontinuation because of adverse events or toxicity, patient adherence, and any serious adverse events as defined by FDA.⁶⁷ For head-to-head trials only, we abstracted data for the 11 specific adverse events (listed in Table 3) that are most commonly reported across all of our eligible drugs according to their FDA-approved labels.

Assessment of Methodological Risk of Bias of Individual Studies

To assess the ROB (i.e., internal validity) of studies, we used the Risk of Bias In Nonrandomised Studies of Interventions (ROBINS-I)⁶⁸ for nonrandomized controlled (nRCT) studies. We adapted the Cochrane ROB tool⁶⁹ for randomized controlled trials (RCTs) by adding an item about the adequacy of intention-to-treat analyses of RCTs. We used predefined criteria based on the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews.⁷⁰ These included questions to assess selection bias, confounding, performance bias, detection bias, and attrition bias; concepts covered include adequacy of randomization, similarity of groups at baseline, masking, attrition, whether intention-to-treat analysis was used, method of handling dropouts and missing data, validity and reliability of outcome measures, and outcome reporting bias.⁶³ To assess outcome reporting bias, we checked protocols for eligible studies in ClinicalTrials.gov (www.clinicaltrials.gov) when available, to determine which outcomes of a specific study had been registered a priori.

Two independent reviewers assessed ROB for each study. Disagreements between the two reviewers were resolved by discussion and consensus or by consulting a third member of the team.

Data Synthesis

We summarized all included studies in narrative form and in summary tables that tabulate the important features of the study populations, design, intervention, outcomes, setting, country, geographic location, and results. All new qualitative and quantitative analyses synthesized included relevant studies from the 2012 SR.

We considered performing pairwise meta-analyses for outcomes with information from at least three unique studies of low or medium ROB that we deemed to be sufficiently similar (in population, interventions, comparators, and outcomes). However, because of a lack of similar head-to-head trials, we were unable to conduct pairwise meta-analyses for any of the comparisons of interest. To address the dearth of studies directly comparing interventions of interest, we considered network meta-analyses. We assessed patient and study characteristics across studies that compared pairs of treatments to ensure the transitivity assumption (i.e., that potential effect modifiers are similar across studies) would hold. To be eligible for network meta-analyses, included studies had to fulfill the following four criteria: (1) patients with early RA had not attempted prior treatment with MTX; (2) doses of treatments were within FDA-approved ranges; (3) length of followup was similar; and (4) studies were double-blinded RCTs of low or medium ROB. Head-to-head and placebo-controlled RCTs were eligible for network meta-analyses; however, we did not find any eligible placebo-controlled trials in a population with early RA. We considered network meta-analyses for the following outcomes: American College of Rheumatology 50% improvement (ACR50), Disease Activity Score (DAS) remission, radiographic joint damage, all discontinuations from the study, and discontinuations attributed to adverse events.

Studies that we had rated high ROB were excluded from these analyses; we used them only in sensitivity analyses. We describe their findings briefly in the context of our main analyses.

We collected data on the number of participants and the number of events for each treatment group for dichotomous outcomes (ACR50, DAS, and discontinuations). For our sole continuous outcome analyzed (radiographic joint damage), we collected means and standard deviations (SDs) from the pre- and post-treatment time point for each study. Four studies did not have data for post-treatment SDs for radiographic joint damage; therefore, we imputed these data by pooling post-treatment SDs from four other studies. SDs for MTX were imputed by pooling SDs from the MTX arms of those studies (N=963 patients), while SDs for the other treatments were imputed by pooling SDs for the other treatment arms of those studies (N=1,730 patients).

We ran our network meta-analyses using a multivariate random effects meta-regression model with restricted maximum likelihood estimation.⁷¹ Models were fit using the Network package in Stata (StataCorp, College Station, TX).⁷² This approach accounts for multiarm trials. We provide diagrams outlining the structure of the network for each outcome, with the lines in the diagrams representing direct comparisons between treatments and the size of the nodes for each treatment being proportional to the sample size. For closed loops, we tested the transitivity assumption by comparing consistency and inconsistency models and network side splits. Because the global Wald test indicated significant differences between the consistency and inconsistency models, we presented the estimates from the consistency model.

We summarize results for dichotomous outcomes (ACR50, DAS, and discontinuations) in forest plots using relative risks. For the sole continuous outcome analyzed (radiographic joint damage), we report standardized mean differences (mean difference divided by standard deviation). We did not calculate ranking probabilities for treatments because such rankings may exaggerate small differences in relative effects.

We also carefully explored whether treatment strategies used for average patients with early RA can be used effectively or safely for patients with significant coexisting ailments such as hepatitis C, congestive heart failure, cancer, diabetes, and others. Because we lacked access to individual patient data, we used a qualitative approach to address this question.

Grading the Strength of Evidence for Major Comparisons and Outcomes

We graded the strength of evidence (SOE) based on the guidance established for the EPC Program.⁷³ Developed to grade the overall strength of a body of evidence, this approach incorporates five key domains: (1) study limitations (including study design and aggregate ROB), (2) consistency, (3) directness, (4) precision of the evidence, and (5) reporting bias. It also considers other optional domains that may be relevant for some scenarios. These included plausible confounding that would decrease the observed effect and strength of association (i.e., magnitude of effect) or factors that would increase the strength of association (i.e., dose-response effect). To grade the SOE of results from network meta-analysis, we used guidance from the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group.⁷⁴ The SOE for indirect estimates was downgraded for indirectness and imprecision in all cases. For comparisons that had both direct and indirect evidence, we commented on whether the indirect evidence was consistent with the direct evidence.

Table 4 describes the grades of evidence that can be assigned. Grades reflect the strength of the body of evidence to answer the KQs on the comparative effectiveness, efficacy, and harms of the interventions in this review. Two reviewers assessed each domain for each key outcome, and they resolved differences by consensus discussion.

Table 4

Definitions of the grades of overall strength of evidence.

We graded the SOE for the following outcomes, consistent with the prior report: disease activity, response, radiographic joint damage, functional capacity, discontinuation because of adverse events, and serious adverse events.¹

Assessing Applicability

We assessed the applicability of individual studies and the larger body of evidence, following guidance from the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.⁷⁵ We examined the following points: whether interventions were similar to those in routine use, whether comparators reflected best alternatives, whether measured outcomes reflected the most important clinical outcomes, whether followup was sufficient, and whether study settings were representative of most outpatient settings. For individual studies, we examined conditions that may limit applicability based on the PICOTS structure. In particular, we focused on factors such as race or ethnicity of populations in studies, clinical setting, geographic setting, and availability of health insurance.

Peer Review and Public Commentary

The AHRQ Task Order Officer and an AHRQ associate editor (a senior member of another EPC) reviewed the draft report before peer review and public comment. The draft report (revised as needed) was sent to invited peer reviewers and simultaneously uploaded to the AHRQ Web site where it was available for public comment for 52 days with a 1-week holiday-related extension.