Formulating questions and determining outcomes

In March 2009, experts invited by WHO drafted a list of 45 general questions about the effects of cryotherapy in women with CIN. These experts were then asked to rank the questions by priority.

Expert guideline panel

WHO selected a multidisciplinary expert guideline panel comprising clinicians with cryotherapy experience, researchers in cervical cancer prevention and treatment, programme directors, epidemiologists, public health officers and methodologists. The methodologists (evidence review team) were based at the McMaster University WHO Collaborating Center and had expertise in guideline development and evidence synthesis. A steering group of seven members was then created from the expert guideline panel, to guide the process.

Table 1

Interpretation of strong and conditional recommendations.

Following a review of the suitability of the initial 45 general questions, these questions were refined to 16 questions for which an evidence review was deemed necessary. The steering group also decided to assess the evidence for the effects of cryotherapy in women with histologically confirmed CIN, to provide the best estimate of the benefits and side-effects of cryotherapy without the potential for confounding the outcomes due to false-positive screening tests or diagnoses.

To determine the outcomes, a scoping review of cryotherapy studies was conducted by the evidence review team. The expert guideline panel was also consulted. A list of outcomes to be considered when making the recommendations was compiled. Nineteen members of the expert guideline panel independently and anonymously scored the outcomes by importance for decision-making, via an electronic survey (5). The mean and median importance of each outcome (scale: 1 – least important to 9 – critical) was calculated, and 16 outcomes were identified as important or critical (see Box 1).

Box 1

Outcomes identified as important or critical to making recommendations. Resource use (including cost, human resources and length of stay) CIN2-3

Preparation of the evidence profiles and grading of the evidence

The evidence review team conducted a series of systematic literature reviews following the methods of the Cochrane Collaboration, and prepared GRADE (Grading of Recommendations, Assessment, Development and Evaluation) evidence profiles for each question (6). During this process, the steering group held conference calls to discuss issues about the available evidence, the presentation of the results, and their impact on making recommendations.

MEDLINE, Embase, LILACS, The Cochrane Library and the WHO Clinical Trials Search Portal were searched up to July 2009, using key subject and text words for cryotherapy and cervical cancer, depending on the database (see appendix A for the MEDLINE search strategy). The search was not limited by language or by study type. The evidence review team screened titles, abstracts and full text of potentially relevant literature, in duplicate. The first screen was for controlled trials (randomized or non-randomized), but because only a few controlled trials were identified, observational studies without independent controls were also included as evidence. Authors in the field, and the expert guideline panel, were also contacted to identify missing studies, studies in progress or studies not yet published.

When possible, relative effects (such as relative risks and odds ratios of an event) were calculated from pooled data of controlled studies. When there were no data, indirect comparisons were made (e.g. randomized controlled studies of cryotherapy versus laser excision were compared to laser excision versus LEEP), and a network meta-analysis was conducted. In studies without independent controls, the risks of an event were pooled across studies (e.g. for cryotherapy and for LEEP), and a relative effect was then calculated to compare those pooled results. All results were normalized to effects over a period of one year, with the exception of adverse events, most of which would probably occur and be reported within one year. Cervical cancer rates in untreated CIN were obtained from McCredie et al. (2010), and annualized (7). It was assumed that these risks were constant over time.

Evidence summaries and profiles, which were based on the evidence of the systematic reviews, were prepared for each question using the GRADEprofiler software (8). GRADE evidence profiles present the effect of the intervention on each outcome (e.g. number of women with recurrent CIN), and the quality of the evidence for each outcome. The quality of a body of evidence is assessed based on the following criteria: risk of bias, imprecision, inconsistency, indirectness, publication bias, magnitude of effect, dose–effect relations and an assessment of the effect of residual confounding and bias. Quality is categorized into four levels, ranging from ⊕○○○, being the lowest quality, to ⊕⊕⊕⊕, being the highest quality. The GRADE evidence profiles allow the expert guideline panel to base its judgments on the same concisely summarized evidence when making recommendations. One week before the expert guideline panel met to develop the recommendations, panel members were able to review the evidence profiles for each question via a password-protected electronic SharePoint site.

Development of recommendations

The expert guideline panel met on 22 to 23 September 2010, to review the evidence and make recommendations. This meeting was chaired by a methodologist with experience in guideline development, and cochaired by a gynaecological oncologist. There were 32 panel experts, as well as WHO and International Agency for Research on Cancer (IARC) officers, who provided scientific input and guidance. The key objectives of the meeting were to formulate evidence-based recommendations for each of the priority questions, identify key research gaps and discuss a dissemination plan for the new guidelines.

During the September meeting, the panel developed recommendations based on the GRADE evidence profiles. For each recommendation, the panel considered and agreed on the following: the quality of the evidence; the balance of benefits and downsides; the assumptions about the values and preferences associated with the decision; and the extent of resource use. Recommendations were made by consensus. Before the meeting concluded, the panel used the evidence to classify each recommendation as “strong” or “conditional” and agreed on the wording and remarks for each recommendation.