Methods

Input From Stakeholders

The topic of hepatitis C virus (HCV) treatment was nominated for a comparative effectiveness review (CER) in a public process. The Key Questions were proposed in the public nomination process and developed by investigators from the Evidence-based Practice Center (EPC) with input from expert Key Informants (KI), who helped to refine Key Questions, identify important methodological and clinical issues, and define parameters for the review of evidence. The revised Key Questions were then posted to a public Web site for comment. The Agency for Healthcare Research and Quality (AHRQ) and the EPC agreed upon the final Key Questions after reviewing the public comments and receiving additional input from a Technical Expert Panel (TEP) convened for this report. We then drafted a protocol for this CER, which was reviewed by the TEP and is available on the AHRQ Web site where it was posted in November 2011: www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=855.

A multidisciplinary group of clinicians, researchers, and patient advocates with expertise in hepatitis C treatment and research were selected to serve as the TEP members to provide high-level content and methodological expertise throughout the development of the review. Prior to participation in this report, the TEP members disclosed all financial or other conflicts of interest. The AHRQ Task Order Officer and the authors reviewed all of these disclosures and determined the panel members had no significant conflicts of interest that precluded participation. KIs and TEP members had expertise in the areas of hepatology, epidemiology, screening, and primary care. TEP members and other experts were invited to provide external peer review of the draft report.

Search Strategy

To identify articles relevant to each Key Question, a research librarian searched the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, and Ovid MEDLINE^® (Appendix A) from 1947 to April 2011 (see Appendix A for the search strategies and a final updated search was conducted in August 2012 following the receipt of peer reviewer comments. The search strategies were peer reviewed by another research librarian and revised prior to finalization. Unpublished trials were sought by searching clinical trial registries (ClinicalTrials.gov, Current Controlled Trials, Clinical Trial Results, WHO Trial Registries) and grants databases (NIHRePORTER, HSRProj, and AHRQ GOLD). Scientific Information Packets on unpublished and published trials were solicited from manufacturers of included antiviral drugs through the Scientific Resource Center. We also hand-searched the reference lists of relevant studies. Searches were updated prior to finalization of the report to identify relevant new publications.

Study Selection

We developed criteria for inclusion and exclusion of studies based on the Key Questions and the populations, interventions, comparators, outcomes, timing, and setting (PICOTS) approach. Inclusion and exclusion criteria, summarized below, are described in more detail by Key Question in Appendix B. Papers were selected for full review if they were about chronic HCV infection, were relevant to Key Questions in the analytic framework, and met the predefined inclusion criteria. To evaluate potential effects of publication bias, we included trials published only as conference abstracts as sensitivity analyses. We restricted inclusion to English language articles. Studies of nonhuman subjects were also excluded, and studies had to include original data.

Abstracts and full-text articles were dual reviewed for inclusion and exclusion for each Key Question (Appendix B). Full-text articles were obtained for all studies that either investigator identified as potentially meeting inclusion criteria. Two investigators independently reviewed all full-text articles for final inclusion or exclusion (Appendix C). A list of excluded studies with primary reasons for exclusion can be found in Appendix D. Discrepancies were resolved through discussion and consensus, with a third investigator making the final decision if necessary.

Data Extraction

We extracted the following data from included studies into Excel spreadsheets: study design, setting, population characteristics, eligibility and exclusion criteria, the antiviral regimen (including duration and dose), and results for each outcome. Data abstraction for each study was completed by two investigators: the first abstracted the data, and the second reviewed the abstracted data for accuracy and completeness against the original articles.

For Key Question 4, some studies reported adjusted hazard ratios (HRs) for the association between achieving an SVR and clinical outcomes relative to untreated patients, and for no SVR and clinical outcomes relative to untreated patients, but did not report a risk estimate for SVR compared with no SVR. We calculated the HR for SVR compared with no SVR based on the two HRs and their reported confidence intervals, assuming zero correlation between the two reported HRs. Such HRs are usually positively correlated; an assumption of zero correlation results in the most conservative (widest) confidence interval for the HR for SVR compared with no SVR.

Assessing Quality

We assessed quality for each study based on the predefined criteria listed in Appendix E. We adapted criteria from methods proposed by Downs and Black⁴³ and the USPSTF.⁴⁴ The criteria used are consistent with the approach recommended in AHRQ’s Methods Guide for Effectiveness and Comparative Effectiveness Reviews (Methods Guide).⁴⁵ We used the term “quality” rather than the alternate term “risk of bias.” Although both refer to internal validity, “quality” may be more familiar to most users and has potential advantages in terms of readability.

We rated the quality of each randomized trial based on the methods used for randomization, allocation concealment, and blinding; the similarity of compared groups at baseline; maintenance of comparable groups; adequate reporting of dropouts, attrition, crossover, adherence, and contamination; loss to followup; the use of intent-to-treat analysis; and ascertainment of outcomes.⁴⁴

We rated the quality for each cohort study based on whether it used nonbiased selection methods to create an inception cohort; whether it evaluated comparable groups; whether rates of loss to followup were reported and acceptable; whether it used accurate methods for ascertaining exposures, potential confounders, and outcomes; and whether it performed appropriate statistical analyses of potential confounders.⁴⁴ For Key Question 4, we considered studies to have performed adequate statistical analyses of potential confounders if they adjusted at a minimum for age, sex, genotype, viral load, and hepatic fibrosis stage in a multivariate model including SVR or histological response; evaluated these factors and excluded them from the multivariate model because there was no association in either univariate or step-wise multivariate analyses; or accounted for these factors using other methods such as stratification or restriction.

Following assessment of individual quality criteria, individual studies were rated as good, fair, or poor quality, as defined below.^{44, 45}

Good quality studies are considered likely to be valid. Good quality studies clearly describe the population, setting, interventions, and comparison groups; use a valid method for allocating patients to interventions; clearly report dropouts and have low dropout rates; use appropriate methods for preventing bias; and appropriately measure outcomes and fully report results.

Fair quality studies have some methodological deficiencies, but no flaw or combination of flaws judged likely to cause major bias. The study may be missing information, making it difficult to assess its methods or assess limitations and potential problems. The fair quality category is broad, and studies with this rating vary in their strengths and weaknesses—the results of some fair quality studies are likely to be valid, while others are only probably valid.

Poor quality studies have significant flaws that may invalidate the results. They have a serious or fatal flaw in design, analysis, or reporting; large amounts of missing information; or discrepancies in reporting. The results of these studies are judged to be at least as likely to reflect flaws in the study design as true effects of the interventions under investigation. We did not exclude poor quality studies a priori, but they were considered the least reliable studies when synthesizing the evidence, particularly when discrepancies between studies were present.

Assessing Research Applicability

We recorded factors important for understanding the applicability of studies such as whether the publication adequately described the study population, the country in which the study was conducted (studies indicate that the rate of HCC in patients with chronic HCV infection is higher in Japan and other Asian countries compared with the United States),⁴⁶ how similar patients were to typical populations of those with chronic HCV infection, whether differences in outcomes were clinically (as well as statistically) significant, and whether the antiviral regimens and other aspects of care evaluated were reasonably representative of standard practice.⁴⁷ We also recorded the funding source and role of the sponsor. We did not assign a rating of applicability (such as high or low) because applicability may differ based on the user of this report.

Data Synthesis

For Key Questions 1 through 3, we performed meta-analysis of trials that evaluated similar populations, interventions, comparisons, and outcomes to estimate pooled relative risks using the DerSimonian-Laird method in a random effects model.⁴⁸ A random effects model results in estimates that are similar to a fixed effects model when there is little or no between-study statistical heterogeneity, but results in more conservative estimates (wider confidence intervals) when statistical heterogeneity is present. Heterogeneity was assessed by calculating the Q-statistic and the percentage of the total variance due to between study variability (I² statistic).⁴⁹ When present, statistical heterogeneity was explored through subgroup and sensitivity analyses, as well as qualitatively. Subgroup analyses were performed in groups stratified by HCV genotype as well as by race, age, body weight, viral load, stage/severity of disease, and IL-28b status when these data were available. We performed sensitivity analysis by excluding poor-quality studies, excluding outlier trials and including trials that used nonstandard doses of antiviral drugs, and adding results from trials published only as abstracts to evaluate the stability of estimates and conclusions. We did not formally assess for publication bias with funnel plots due to small numbers (<10) of studies for all comparisons. Small numbers of studies can make interpretation of funnel plots unreliable, and experts suggest 10 studies as the minimum number of studies to perform funnel plots.⁵⁰ All analyses were performed using Stata 11.0 (StataCorp, College Station, TX, 2009).

For Key Question 4, we did not perform meta-analysis, since all studies were cohort studies, and many had methodological shortcomings (including failure to adjust for important confounders) and varied in populations assessed, treatments received, and other factors. Rather, these studies were synthesized qualitatively.

Strength of the Body of Evidence

We assessed the overall strength of evidence for a body of literature about a particular Key Question in accordance with the AHRQ Methods Guide.⁴⁵ The strength of evidence was based on the overall quality of each body of evidence, based on the type and quality of studies (graded good, fair, or poor); the consistency of results within and between study designs (graded high, moderate, or low); the directness of the evidence linking the intervention and health outcomes (graded direct or indirect); and the precision of the estimate of effect, based on the number and size of studies and confidence intervals for the estimates (graded high, moderate, or low). We did not downgrade a body of evidence for directness that evaluated an intermediate outcome, if the intermediate outcome was the specific focus of the Key Question. We did not grade supplemental domains for cohort studies included in Key Question 4 because they were not relevant (dose-response relationship) or because important methodological shortcomings (in particular failure to adjust for critical confounders) limited their usefulness (magnitude of effect and direction of plausible confounding). We were not able to formally assess for publication bias due to small numbers of studies, methodological shortcomings, or differences across studies in designs, measured outcomes, and other factors.

We graded the strength of evidence for each Key Question using the four key categories recommended in the AHRQ Methods Guide.⁴⁵ A “high” grade indicates high confidence that the evidence reflects the true effect and that further research is very unlikely to change our confidence in the estimate of effect. A “moderate” grade indicates moderate confidence that the evidence reflects the true effect and further research may change our confidence in the estimate of effect and may change the estimate. A “low” grade indicates low confidence that the evidence reflects the true effect and further research is likely to change the confidence in the estimate of effect and is likely to change the estimate. An “insufficient” grade indicates evidence either is unavailable or is too limited to permit any conclusion.

Peer Review and Public Commentary

Experts in gastroenterology, hepatology, primary care, and prevention, and individuals representing stakeholder and user communities were invited to provide external peer review of a draft of this CER; AHRQ and an EPC associate editor also provided comments. The draft report was posted on the AHRQ Web site for 4 weeks to elicit public comment. All comments were reviewed and addressed as documented in a disposition of comments report that will be made available 3 months after the Agency posts the final CER on the AHRQ Web site www.effectivehealthcare.ahrq.gov.