NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
McCrory DC, Coeytaux RR, Yancy WS Jr, et al. Assessment and Management of Chronic Cough [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013 Jan. (Comparative Effectiveness Reviews, No. 100.)
The methods for this comparative effectiveness review (CER) follow those suggested in the Agency for Healthcare Research and Quality (AHRQ) Methods Guide for Effectiveness and Comparative Effectiveness Reviews (hereafter referred to as the Methods Guide)25 and Methods Guide for Medical Test Reviews (hereafter referred to as the Medical Test Guide).26 The main sections in this chapter reflect the elements of the protocol established for the CER; certain methods map to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist.27 All methods and analyses were determined a priori.
Topic Refinement and Review Protocol
During the topic refinement stage, we solicited input from Key Informants representing clinicians (adult and pediatric pulmonology, otolaryngology, school nursing, respiratory medicine, primary care), patients, scientific experts, and payers, to help define the Key Questions (KQs). The KQs were then posted for public comment in September 2011 for 4 weeks, and the comments received were considered in the development of the research protocol. We next convened a Technical Expert Panel (TEP) comprising clinical, content, and methodological experts to provide input in defining populations, interventions, comparisons, and outcomes, and in identifying particular studies or databases to search. The Key Informants and members of the TEP were required to disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts. Any potential conflicts of interest were balanced or mitigated. Neither Key Informants nor members of the TEP performed analysis of any kind, nor did any of them contribute to the writing of this report. We next drafted a protocol for the review applying the input received from both the Key Informants and the TEP panel.
Literature Search Strategy
Search Strategy
To identify the relevant published literature, we searched PubMed®, Embase®, and the Cochrane Database of Systematic Reviews (CDSR; last search date for all three sources June 4, 2012). Where possible, we used existing validated search filters (such as the Clinical Queries Filters in PubMed). An experienced search librarian guided all searches. Exact search strings are included in Appendix A. We supplemented the electronic searches with a manual search of references from a set of key primary and systematic review articles. All citations were imported into an electronic database (EndNote® X4; Thomson Reuters, Philadelphia, PA).
We used several approaches to identify relevant grey literature including a request for scientific information packets submitted to drug and device manufacturers and a search of U.S. Food and Drug Administration (FDA) device registration studies and new drug applications. We also searched study registries and conference abstracts for relevant articles from completed studies. Grey literature databases searched included ClinicalTrials.gov (July 18, 2012); the World Health Organization (WHO) International Clinical Trials Registry Platform Search Portal (July 18, 2012); and ProQuest COS Conference Papers Index (January 18, 2012). Search terms used for these sources are provided in Appendix A. We planned to search ClinicalStudyResults.org, but that Web site is no longer available.
Inclusion and Exclusion Criteria
The PICOTS (population, interventions, comparators, outcomes, timing, and settings) criteria used to screen articles for inclusion/exclusion at both the title-and-abstract and full-text screening stages are detailed in Table 2.
Table 2
Inclusion and exclusion criteria.
Study Selection
Using the prespecified inclusion and exclusion criteria described in Table 2, titles and abstracts were reviewed independently by two investigators for potential relevance to the KQs. Articles included by either reviewer underwent full-text screening. At the full-text review stage, paired researchers independently reviewed the articles and indicated a decision to “include” or “exclude” the article for data abstraction. When the two reviewers arrived at different decisions about whether to include or exclude an article, they reconciled the difference through review and discussion, or through a third-party arbitrator if needed. Full-text articles meeting our eligibility criteria were included for data abstraction. Relevant review articles, meta-analyses, and methods articles were flagged for manual searching of references and cross-referencing against the library of citations identified through electronic database searching.
For citations retrieved by searching the grey literature, the above-described procedures were modified such that a single screener initially reviewed all citations; final eligibility for data abstraction was determined by duplicate screening review. All screening decisions were made and tracked in a Distiller SR database (Evidence Partners Inc, Manotick, ON, Canada).
Data Extraction
The research team created data abstraction forms and evidence table templates for abstracting data for each KQ. Based on clinical and methodological expertise, a pair of investigators was assigned to abstract data from each eligible article. One investigator abstracted the data, and the second reviewed the completed abstraction form alongside the original article to check for accuracy and completeness. Disagreements were resolved by consensus, or by obtaining a third reviewer's opinion if consensus could not be reached. To aid in both reproducibility and standardization of data collection, researchers received data abstraction instructions directly on each form created specifically for this project with the DistillerSR database.
We designed the data abstraction forms to collect the data required to evaluate the specified eligibility criteria for inclusion in this review, as well as demographic and other data needed for determining outcomes (intermediate, final, and adverse events outcomes). We gave particular attention to describing the details of the treatment (e.g., pharmacotherapy dosing, methods of nonpharmacological therapies), patient characteristics (e.g., underlying etiology of chronic cough, age of patient), and study design (e.g., randomized controlled trial [RCT] versus observational) that were related to outcomes. In addition, we described comparators carefully, as treatment standards may have changed during the study period. The safety outcomes were framed to help identify adverse events, including those from drug therapies (sleep disturbance, allergic reaction, drowsiness, headache, chest pain, dizziness, and rash) and those associated with nonpharmacological therapies. Data necessary for assessing quality and applicability, as described in the Methods Guide,25 were abstracted. Before the data abstraction form templates were used, they were pilot-tested with a sample of included articles to ensure that all relevant data elements were captured and that there was consistency/reproducibility between abstractors. Forms were revised as necessary before full abstraction of all included articles. Appendix B provides a detailed listing of the elements included in the data abstraction forms.
Quality (Risk of Bias) Assessment of Individual Studies
We evaluated the quality of individual studies using the approach described in the Methods Guide.25 To assess quality, we used the strategy to (1) classify the study design, (2) apply predefined criteria for quality and critical appraisal, and (3) arrive at a summary judgment of the study's quality. We applied criteria for each study type derived from core elements described in the Methods Guide. Criteria of interest for all studies included similarity of groups at baseline, extent to which outcomes were described, blinding of subjects and providers, blinded assessment of the outcome(s), intention-to-treat analysis, differential loss to followup between the compared groups or overall high loss to followup, and conflicts of interest. Criteria specific to RCTs included methods of randomization and allocation concealment. For observational studies, additional elements such as methods for selection of participants, measurement of interventions/exposures, addressing any design-specific issues, and controlling confounding were considered.
To indicate the summary judgment of the quality of individual studies, we used the summary ratings of good, fair, or poor based on the study's adherence to well-accepted standard methodologies and adequate reporting (Table 3).
Table 3
Definitions of overall quality ratings.
For studies of diagnostic tests (KQ 1), we used the QUality Assessment tool for Diagnostic Accuracy Studies (QUADAS)-228 to assess quality. QUADAS-2 describes risk of bias in four key domains: patient selection, index test(s), reference standard, and flow and timing. The questions in each domain are rated in terms of risk of bias and concerns regarding applicability, with associated signaling questions to help with these bias and applicability judgments.
Study design was considered when grading quality. RCTs were graded as good, fair, or poor. Observational studies were graded separately, also as good, fair, or poor.
Data Synthesis
We began our data synthesis by summarizing key features of the included studies for each KQ. To the degree that data were available, we abstracted information on study design; patient characteristics; clinical settings; interventions; and intermediate, final, and adverse event outcomes.
KQ 1. Test Performance Measures
For KQ 1 we considered the three dimensions of (1) cough frequency, (2) cough severity (which might include quantity and characteristics of sputum, difficulty of expectoration, dyspnea, between cough sensations, or pain), and (3) cough-specific quality of life (QOL). While cough frequency is a unidimensional measure (although it is sometimes broken down into daytime and nighttime cough frequency), we considered cough severity and cough-specific QOL to be separate (and often multidimensional) dimensions of cough. Most of the standardized questionnaires included in this report measured aspects of both of these dimensions. Therefore, for the purpose of this report, we considered instruments that measured both severity and QOL together to be “severity/QOL” instruments. Within this report, we did not identify any validated instruments which focused purely on cough severity.
We sought to measure the validity, reliability, and responsiveness of various instruments used to assess each of these dimensions. For cough frequency, we evaluated validity by concurrence with measures of other constructs (e.g., cough severity, cough-specific QOL, tussigenic challenge (or cough reflex sensitivity), and exhaled nitrous oxide), and we assessed reliability using inter-method reliability (e.g., manual cough counts versus electronic recording device cough counts) and test-retest reliability. For severity/QOL instruments, we evaluated validity by looking at concurrence with measures of other constructs including cough frequency, quality of life, and tussigenic challenge findings. We assessed reliability by test-retest reliability, as well as internal consistency. We evaluated responsiveness of both frequency and severity/QOL measures by reporting data on changes in these measures over time associated with treatment (or no treatment) of cough symptoms or the underlying etiology of cough.
KQ 2. Overall Approaches and Meta-Analyses for Direct Comparisons
We determined the feasibility of completing a quantitative synthesis (i.e., meta-analysis). Feasibility depended on the volume of relevant literature, conceptual homogeneity of the studies, and completeness of the reporting of results. We considered meta-analysis for comparisons where at least three studies reported the same outcome. We considered measures of cough frequency, regardless of the scale used, to be similar enough to combine using effect sizes (standardized mean differences); similarly, measures of cough severity that used different measurement scales were considered similar enough to combine using effect sizes.
When a meta-analysis was appropriate, we used random-effects models to quantitatively synthesize the available evidence using Comprehensive Meta-Analysis software (Version 2; Biostat, Englewood, NJ). We tested for heterogeneity using graphical displays and test statistics (Q and I2 statistics), while recognizing that the ability of statistical methods to detect heterogeneity may be limited. We reported p-values for Q statistics as follows: 0.15 > p > 0.05 as some evidence of heterogeneity, 0.05 > p > 0.0001 as evidence of heterogeneity, and p<0.0001 as evidence of extreme heterogeneity. The degree of heterogeneity was reflected in our strength of evidence conclusions. For comparison, we also performed fixed-effect meta-analyses. We present summary estimates, standard errors, and confidence intervals in our data synthesis.
KQ 2. Indirect Comparisons With Mixed Treatment Comparisons Techniques
We supplemented the meta-analysis of direct comparisons with a mixed treatment meta-analysis that incorporated data from placebo comparisons and head-to-head comparisons, including multi-armed trials (i.e., trials that included more than one comparison). The general strategy for analysis was to construct a random-effects model that was comparable with the standard random-effects models used in the meta-analysis of effect sizes.
This model, which was fitted using SAS® PROC NLMIXED (2009; SAS Institute Inc., Cary, NC), estimated the effect sizes (relative to placebo) for each treatment. For some treatments that could not be included in the mixed treatment meta-analysis, we calculated effect sizes from data reported in the studies (raw data, means and variances, or test statistics) to present results in comparable terms.
Strength of the Body of Evidence
We rated the strength of evidence for each KQ and outcome using the general approach described in the Methods Guide25,29 and Medical Test Guide;26 we note, however, that the latter does not specifically address responsiveness or other psychometric properties of a test. In brief, the approach requires assessment of four domains: risk of bias, consistency, directness, and precision (Table 4).
Table 4
Strength of evidence—required domains.
Additional domains were used when appropriate: coherence, dose-response association, impact of plausible residual confounders, strength of association (magnitude of effect), and publication bias. These domains were considered qualitatively, and a summary rating of “high,” “moderate,” or “low” strength of evidence was assigned after discussion by two reviewers. In some cases, high, moderate, or low ratings were impossible or imprudent to make; for example, when no evidence was available or when evidence on the outcome was too weak, sparse, or inconsistent to permit any conclusion to be drawn. In these situations, a grade of “insufficient” was assigned. This four-level rating scale consists of the following definitions:
- High—High confidence that the evidence reflects the true effect. Further research is very unlikely to change our confidence in the estimate of effect.
- Moderate—Moderate confidence that the evidence reflects the true effect. Further research may change our confidence in the estimate of effect and may change the estimate.
- Low—Low confidence that the evidence reflects the true effect. Further research is likely to change the confidence in the estimate of effect and is likely to change the estimate.
- Insufficient—Evidence either is unavailable or does not permit estimation of an effect.
Test studies (KQ 1) are generally indirect, as the link between the test intervention and outcome is mitigated by prognosis, management, and the effectiveness of treatments. As a rule of thumb, we considered correlation coefficients > 0.7 as strong evidence of association, 0.40–0.69 as moderate evidence, and <0.40 as weak evidence. In our summary SOE assessments for KQ 1, lack of directness was weighed less heavily and risk of bias most heavily Thus, we allowed high SOE levels despite the lack of directness among these studies.
Applicability
We assessed applicability across our KQs using the method described in the Methods Guide.25,30 In brief, this method uses the PICOTS format as a way to organize information relevant to applicability. The most important issue with respect to applicability is whether the outcomes are different across studies that recruit different populations (e.g., age groups, exclusions for comorbidities) or use different methods to implement the interventions of interest; that is, important characteristics are those that affect baseline (control-group) rates of events, intervention-group rates of events, or both. We used checklists to guide the assessment of applicability (see Appendix B, sections IV and VIII). We used these data to evaluate the applicability to clinical practice, paying special attention to study eligibility criteria, demographic features of the enrolled population in comparison with the target population, characteristics of the intervention used in comparison with care models currently in use, and clinical relevance and timing of the outcome measures. We summarized issues of applicability qualitatively.
Peer Review and Public Commentary
The peer review process is our principal external quality-monitoring device. Nominations for peer reviewers were solicited from several sources, including the TEP and interested Federal agencies. Experts in adult and pediatric pulmonology, respiratory medicine, and primary care, along with individuals representing stakeholder and user communities, were invited to provide external peer review of the draft report; AHRQ and an associate editor also provided comments. The draft report was posted on AHRQ's Web site for public comment for 4 weeks, from June 12, 2012, to July 10, 2012. We have addressed all reviewer comments, revising the text as appropriate, and have documented everything in a disposition of comments report that will be made available 3 months after the Agency posts the final report on AHRQ's Web site. A list of peer reviewers submitting comments on the draft report is provided in the front matter of this report.
- Methods - Assessment and Management of Chronic CoughMethods - Assessment and Management of Chronic Cough
Your browsing activity is empty.
Activity recording is turned off.
See more...