Topic Refinement and Technical Expert Panel

The UAEPC was commissioned to conduct a preliminary literature review to gauge the availability of evidence and to draft the key research questions for a full comparative effectiveness review. In consultation with AHRQ and the Scientific Resource Center, a Technical Expert Panel (TEP) was invited to provide input in the development of the key questions and scope of the evidence report. The public was invited to comment on these questions over a period of 3 months. After reviewing the public commentary, the key questions were finalized and submitted to AHRQ for approval.

The TEP was subsequently invited to provide high-level content and methodological expertise throughout the development of the comparative effectiveness report. The names of technical experts are available in Appendix A.

Literature Search Strategy

Search strategies were designed and implemented to identify evidence relevant to the report. The following bibliographic databases were searched systematically for studies published from 1990 to 2009: Medline^®, Embase, EBM Reviews–The Cochrane Library, AMED, Cinahl^®, SPORTDiscus with Full Text, Academic Search Elite, Health Source, Science Citation Index Expanded (via Web of Science^®), Scopus^®, BIOSIS Previews^®, and PubMed. Additional searches of the Grey Literature were conducted in Conference Papers Index, Computer Retrieval of Information on Scientific Projects (CRISP), Scopus^®, as well as government websites by the U.S. Food and Drug Administration and Health Canada. Databases that yielded included studies (Medline^®, Embase, Central, and CINAHL) were searched again in September 2009 to identify recently published studies.

Search terms were selected by scanning search strategies of systematic reviews on similar topics and by examining index terms of potentially relevant studies. A combination of subject headings and textwords were adapted for each electronic resource which included terms for rotator cuff (‘rotator cuff*’ or ‘rotator interval*’ or ‘supraspin?tus’ or infraspin?tus or “teres minor” or ‘subscapularis’ or ‘anterosuperior’ or ‘posterosuperior’) and tear terms (‘tear’ or ‘tears’ or ‘tore’ or ‘torn’ or ‘lesion*’ or ‘rupture*’ or ‘avuls*’ or ‘injur*’ or ‘repair*’ or ‘debride*). Language restrictions were not applied. (See Appendix B for detailed search strategies)

Hand searches were conducted to identify literature from symposia proceedings from the following scientific meetings: Arthroscopy Association of North America (2007–2009), American Academy of Orthopaedic Surgeons (2007–2009), American Physical Therapy Association (2006–2008), American Shoulder and Elbow Surgeons (2005–2008), American Society of Shoulder and Elbow Therapists (2004–2008), European Congress of Physical and Rehabilitation Medicine 2008, Congress of the European Society for Surgery of the Shoulder and the Elbow (2009) and the Mid-America Orthopaedic Association (2006–2008). Ongoing studies were identified by searching clinical trials registers (See Appendix B) in addition to contacting experts in the field. Reference lists of relevant reviews were searched to identify additional studies.

The results from the literature searches were entered into a Reference Manager for Windows bibliographic database version 11.0 (© 2004–2005 Thomson ResearchSoft) for management.

Criteria for Study Selection

The study inclusion and exclusion criteria were developed in consultation with the TEP (Table 2). In consultation with the TEP, a post hoc decision was made to exclude uncontrolled studies that were either retrospective or unclear in their direction, as well as case series. For the literature update, only comparative studies were included. The decision was made to include only operative studies published in English due to lack of translation resources. English, German and French publications were considered for studies examining nonoperative treatments and postoperative rehabilitation, since the literature on these interventions was sparse (n=7). This resulted in the exclusion of 80 of the 1010 studies (7.9 percent) retrieved for selection.

Table 2

Eligibility criteria for the review.

Article screening was conducted in two phases. First, two reviewers (AM, DJ, LH, JS, NH) independently screened the titles, keywords and abstracts (when available) to determine if an article met the general inclusion criteria. Each article was rated as “include,” “exclude,” or “unclear.” The full text of all articles classified as “include” or “unclear” by one or both of the reviewers was retrieved for detailed review. Second, two reviewers independently assessed each study using a standard inclusion/exclusion form (Appendix C1). Disagreements were resolved by consensus or third-party adjudication. Non-English studies were assessed by only one reviewer.

Assessment of Methodological Quality

The internal validity of randomized controlled trials (RCTs) and controlled clinical trials (CCTs) was assessed using the Cochrane Collaboration Risk of Bias tool.⁵⁰ (Appendix C2) This tool consists of six domains (sequence generation, allocation concealment, blinding, incomplete outcome data, selective outcome reporting, and “other” sources of bias) and a categorization of the overall risk of bias. Each separate domain is rated “yes,” “unclear,” or “no.” Blinding and incomplete outcome data were assessed separately for subjective outcomes (e.g., quality of life or function scales) and objective clinical outcomes (e.g., range of motion). The overall assessment was based on the responses to individual domains. If one or more individual domains were assessed as having a high risk of bias, the overall score was rated as high risk of bias. The overall risk of bias was considered low only if all components were rated as having a low risk of bias. The risk of bias for all other studies was rated as unclear. In addition, information was collected for each study on the source of funding⁵¹ and whether an intention-to-treat analysis was performed.^52,53

Observational analytic studies were assessed using modified cohort and case-control Newcastle-Ottawa Quality Assessment Scales (NOQAS) (Appendix C2).⁵⁴ The NOQAS includes seven items assessing sample selection, comparability of cohorts, and the assessment of outcomes. One star was allotted for each item that was adequately addressed in the study, with the exception of the comparability of cohorts, for which a maximum of two stars could be given. The overall score was calculated by tallying the stars, with a total possible score of eight stars. In addition, information regarding the source of funding was collected.⁵¹

The methodological quality of uncontrolled studies was assessed using a quality checklist developed by the UAEPC (Appendix C2). The checklist assessed three components theoretically associated with bias in observational studies: consecutive enrollment, incomplete outcome data and standardized/independent approach to outcome assessment. In addition, the source of funding was documented for each study.⁵¹

Two reviewers (JS, JRS, KB, SM) independently assessed the methodological quality of the included studies. Non-English studies were assessed by only one reviewer (LH, JS) due to limited translation resources. Each assessment form was pilot tested on a sample of studies. Decision rules regarding application of the tools was developed a priori through discussions with content and methodology experts. Discrepancies in quality assessment were resolved through consensus or third-party adjudication.

Data Extraction

Data were extracted using a standardized form and entered into a Microsoft Excel™ database (Microsoft Corp., Redmond, WA) (Appendix C3). Data were extracted by one reviewer (AM, JS, JRS, KB, LH, SM) and checked for accuracy and completeness by a second (JS, JRS, KB, SM). Extracted data included study characteristics, inclusion/exclusion criteria, participant characteristics, interventions, and outcomes. Reviewers resolved discrepancies in data extraction by consensus or in consultation with a third party.

Operative studies were divided into three broad categories by type of intervention: approach, technique, and augmentation. Studies which focused on the use of an open, mini-open or arthroscopic approach to RC repair (RCR), debridement, acromioplasty or other procedure were categorized as “operative approach.” Studies that compared the effectiveness of different suture or anchor types or configurations were labelled as investigating an “operative technique.” “Operative augmentation” was reserved for studies that examined the use of a surgical augment, such as the use of grafts or patches in the repair of an RC tear.

Before-and-after (BA) studies were defined as single-arm studies that report both baseline and followup data scores. Cohort studies that compared the effectiveness of a single intervention across two patient populations (e.g., open repair in older vs. younger patients) were classified as “cohort studies with BA data.” For the purposes of examining the effectiveness of operative procedures (Key Question 2), the data across the patient groups was combined and analysed as for a BA study. BA studies and cohort studies with BA data are collectively referred to as uncontrolled studies. The effects of prognostic variables on treatment outcomes were explored separately in Key Question 6.

A post hoc decision was made to extract data on cuff integrity as an additional outcome of interest for all the included studies. For the uncontrolled studies, the decision was made to examine only four key outcomes considered to be the most clinically relevant by the clinical investigators (DS, CL): health-related quality of life, functional outcomes, time to return to work, and cuff integrity.

Applicability

The applicability of the body of evidence was assessed following the PICOTS (population, intervention, comparator, outcomes, timing of outcome measurement, setting) format used to assess study characteristics. Factors that may potentially weaken the applicability of individual studies were extracted and presented in the evidence tables (Appendix E).

Rating the Body of Evidence

We used the EPC GRADE approach, based on the standard GRADE approach,^55,56 to assess the quality of the body of evidence for each outcome. The strength of evidence was assessed for four key outcomes identified by the clinical investigators to be most clinically important: health-related quality of life, functional outcomes, time to return to work, and cuff integrity. The following four major domains were examined: risk of bias (low, medium, high), consistency (no inconsistency, inconsistency present, unknown or not applicable), directness (direct, indirect), and precision (precise, imprecise). When no studies were available for an outcome or comparison of interest, the evidence was simply graded as insufficient. Each key outcome on each comparison of interest was given an overall evidence grade based on the ratings for the individual domains. The overall strength of evidence was graded as high (further research is very unlikely to change our confidence in the estimate of effect), moderate (further research may change our confidence in the estimate of effect and may change the estimate), low (further research is likely to change the confidence in the estimate of effect and is likely to change the estimate) or insufficient (evidence either is unavailable or does not permit estimation of an effect). The body of evidence was graded by one reviewer (LH).

Data Analysis

The following data assumptions were made and imputations performed to transform reported data into the form required for analysis. Graphical data was extracted using CorelDRAW^® 9.0 (Corel Corp., Ottawa, Canada). If necessary, means were approximated by medians, and 95 percent confidence intervals (95% CI) were used to calculate approximate standard deviations (SD).

Evidence tables and qualitative description of results are presented for all included studies. When appropriate, meta-analyses were performed to support inferences on the effectiveness of nonoperative and operative interventions for treatment of RC tears. We reported outcomes only if numeric data were available in the study or could be derived from graphs. Outcomes that were only described qualitatively (e.g., “pain improved by 6 weeks”) or reported only as a p-value were not included in the evidence tables or data analysis.

Decision-making criteria regarding the instances in which pooled estimates should be derived from individual studies were established a priori. Comparative studies were considered appropriate to combine if the study design, study population, interventions being compared, and outcomes were sufficiently similar. Trials (RCTs and CCTs) and cohort studies were analysed separately. Study populations were considered similar if the type of tear (full-thickness or partial-thickness) and size of tear was common among eligible studies. More than two studies comparing the same intervention arms were necessary in order to conduct a meta-analysis. Finally, studies were only combined when they reported the use of similar outcome measures. Scales were classified as being either health-related quality of life measures or as functional outcome scales, and meta-analyses were only conducted within scales of the same classification.

Graphs were created to display the preoperative and postoperative scores of uncontrolled studies, cohort studies and trials, over the duration of the study followup period. Due to the low level of evidence represented by uncontrolled studies, these studies were not analyzed quantitatively.

Quantitative results were meta-analyzed in Review Manager version 5.0 (The Cochrane Collaboration, Copenhagen, Denmark). For continuous variables measured on the same scale (e.g., range of motion), mean differences were calculated for individual studies, and weighted mean differences (WMD) was calculated for the pooled estimates. For continuous variables measured on different scales (e.g., health-related quality of life or functional outcome scales), mean differences were calculated for separate studies and standardized mean differences (SMD) were calculated for the pooled estimates. All results are reported with 95% CI when possible. Statistically significant results were considered to be clinically relevant if they exceeded a minimal clinically important difference of ten percent on any given scale.⁵⁷

Results were combined using random effects models. Statistical heterogeneity was quantified using the I-squared (I²) statistic. A value greater than 50 percent was considered to be substantial heterogeneity.^58,59