3.2.1. Guideline Development Group meetings

Twenty-three eating disorders GDG meetings were held between March 2002 and October 2003. During the series of day-long meetings, clinical questions were written, clinical evidence was reviewed and assessed, statements developed and recommendations formulated. At each meeting, all GDG members declared any potential conflict of interests, and patient and carer concerns were routinely discussed as part of a standing agenda.

3.2.2. Topic groups

The GDG divided its workload along clinically relevant lines to simplify the guideline development process by forming smaller topic groups to undertake guideline work in specified areas of clinical practice. Topic groups covered physical management, service-level interventions, and psychological interventions. These groups were designed to efficiently manage the large volume of evidence appraisal prior to presenting it to the GDG as a whole. Each topic group was chaired by a GDG member with expert knowledge of the topic area (one of the healthcare professionals). Topic groups refined the clinical definitions of treatment interventions, identified relevant clinical questions, reviewed and prepared the evidence with the systematic reviewer before presenting it to the GDG as a whole, and helped the GDG to identify further expertise in the topic. Topic group leaders reported the status of the group’s work as part of the standing agenda. They also introduced and led the GDG discussion of the evidence review for that topic and assisted the GDG Chair in drafting that Section of the guideline relevant to the work of each topic group.

3.2.3. Patients and carers

The GDG included a patient and representatives of a national patient group. Given their direct experience of services, they gave an integral patient focus to the GDG and the guideline. They contributed as full GDG members to writing the clinical questions, helping to ensure that the evidence addressed their views and preferences, highlighting sensitive issues and terminology associated with eating disorders, and bringing service-user research to the attention of the GDG. They wrote the section on the patient perspective, organised, conducted and reported on the carer focus group contribution, and identified recommendations from the patient and carer perspective.

The carer perspective was provided through a focus group held with carers. The main objective of the group discussion was to discuss carers’ views about communication with clinicians and other professionals, including guidance on responsibility and expectations of clinicians, carers, and people with eating disorders; the role of carers in identifying eating disorders and subsequent treatment, and accessing appropriate services. Twelve people, currently caring for someone with anorexia nervosa as a parent or spouse, were recruited by the Eating Disorders Association and the Carers Volunteers Database at the Eating Disorders Research Unit of the Institute of Psychiatry. Geographic distribution of members included Cornwall, the Midlands, East Anglia, south east England and London. Participants were told the purpose of the session, and signed a participant consent form. Discussions were tape-recorded with the full knowledge of the participants, transcribed, and key themes identified.

3.2.4. Special advisors

Special advisors who had specific expertise in one or more aspects of the treatment and management of eating disorders were invited to assist the GDG, to comment on specific aspects of the developing guideline and to present evidence and observations on areas of their expertise to the GDG. Appendix 2 lists those who agreed to act as special advisors.

3.2.5. National and international experts

National and international experts in eating disorders research were identified through the literature search and through the experience of the GDG members. These experts were invited to recommend unpublished or soon-to-be published studies in order to ensure that the most recent evidence on the management of eating disorders was included in the development of the guideline. They informed the group about completed trials at the pre-publication stage, systematic reviews in the process of being published, studies relating to the cost-effectiveness of treatment, and trial data if the GDG could be provided with full access to the complete trial report. Appendix 5 lists researchers who were contacted.

3.4.1. Methodology

A stepwise, hierarchical approach was taken to locating and presenting evidence to the GDG. The NCCMH review team had developed this process based on advice from the National Guidelines Support and Research Unit (NICE) and after considering recommendations from a range of other sources. These included:

• Centre for Clinical Policy and Practice of the New South Wales Health Department (Australia)
• Clinical Evidence Online
• Cochrane Collaboration
• New Zealand Guideline Group
• NHS Centre for Reviews and Dissemination
• Oxford Centre for Evidence-Based Medicine
• Scottish Intercollegiate Guidelines Network (SIGN)
• United States Agency for Health Research and Quality
• Oxford Systematic Review Development Programme.

3.4.2. The review process

3.4.2.2. Questions of diagnosis and prognosis

For questions related to diagnosis and prognosis, the initial evidence base was formed from studies with the most appropriate and reliable design to answer the particular question (i.e, diagnosis – cross sectional studies; prognosis – cohort studies of representative patients). The review process was similar to that described for questions of efficacy. The main difference concerned the study search filter and the criteria used to assess methodological quality. In situations where it was not possible to identify high-quality systematic reviews or a substantial body of appropriately designed studies that directly addressed a clinical question, an informal consensus process was adopted (see Section 3.4.7).

Flowchart 1. Eating disorders Guideline Review Process (PDF, 28K)

3.4.3. Search strategies

In conducting the review, the team systematically searched the literature for all English-language systematic reviews relevant to the eating disorders scope that were published or updated after 1995.

Search filters developed by the review team consisted of a combination of subject heading and free-text phrases. A general filter was developed for eating disorders along with more specific filters for each clinical question. In addition, filters were developed for RCTs and for other appropriate research designs. (The search filters can be found in Appendix 8.)

Electronic searches were made of the major bibliographic databases (MEDLINE, EMBASE, PsycINFO, CINAHL), in addition to the Cochrane Database of Systematic Reviews, the NHS R&D Health Technology Assessment database, Evidence-Based Mental Health and Clinical Evidence (Issue 5).

Ineligible articles were excluded, and a second independent reviewer crosschecked these for relevance. The remaining references were acquired in full and re-evaluated for eligibility. The most recently published reviews that appropriately addressed a clinical question were selected. For each systematic review used, a search was made for new studies, and the papers for these and for existing studies, were retrieved.

The search for further evidence included research published after each review’s search date, in-press papers identified by experts, and reviewing reference lists and recent contents of selected journals. All reports that were retrieved but later excluded are listed with reasons for exclusion in the appropriate evidence table. Where no relevant systematic reviews were located, the review team asked the GDG to decide whether a fresh systematic review should be undertaken. Eligible reviews were critically appraised for methodological quality and the reliability of this procedure was confirmed by parallel independent assessment. The eligibility/quality assessment was tested on a representative sample of papers. (Appendix 10 provides the quality checklist.)

3.4.4. Synthesising the evidence

Where possible, outcome data were extracted directly from all eligible studies that met the quality criteria into Review Manager 4.2 (Cochrane Collaboration, 2003). Meta-analysis was then used to synthesise the evidence where appropriate using Review Manager. If necessary, reanalyses of the data or sensitivity analyses were used to answer clinical questions not addressed in the original studies or reviews. Where meta-analysis was not appropriate and/or possible, the reported results from each primary-level study were entered into the Access database. Evidence tables were used to summarise general information about each study. Consultation was used to overcome difficulties with coding. Data from studies included in existing systematic reviews were extracted independently by one reviewer directly into Review Manager and crosschecked with the existing data set. Two independent reviewers extracted data from new studies, and disagreements were resolved by discussion. Where consensus could not be reached, a third reviewer resolved the disagreement. Masked assessment (i.e. blind to the journal from which the article comes, the authors, the institution, and the magnitude of the effect) was not used since it is unclear that doing so reduces bias (Jadad et al., 1996; Berlin, 1997).

3.4.5. Presenting the data to the GDG

Where possible, the GDG was given a graphical presentation of the results using forest plots generated with the Review Manager software (see Figure 1 and Figure 2 for examples of forest plots). Each forest plot displayed the effect size and confidence interval (CI) for each study as well as the overall summary statistic. The graphs were organised so that the display of data in the area to the left of the ‘line of no effect’ indicated a ‘favourable’ outcome for the treatment in question. Dichotomous outcomes were presented as relative risks (RR) and the associated 95 per cent CI (see Figure 1). A relative risk (or risk ratio) is the ratio of the treatment event rate to the control event rate. A RR of 1 indicates no difference between treatment and control. In Figure 1, the overall RR of 0.73 indicates that the event rate (i.e. non-remission rate) associated with intervention A is about three-quarters of that with the control intervention, or in other words, intervention A reduces non-remission rates by 27 per cent. In addition, the CI around the RR does not cross the ‘line of no effect’ indicating that this is a statistically significant effect. The CI shows with 95 per cent certainty the range within which the true treatment effect should lie.

Figure 1

Example of a forest plot displaying dichotomous data.

Figure 2

Example of a forest plot displaying continuous data.

All dichotomous outcomes were calculated on an intention-to-treat basis (i.e. a ‘once-randomised-always-analyse’ basis). This assumes that those participants who ceased to engage in the study – from whatever group – had an unfavourable outcome (with the exception of the outcome of ‘death’). The Number Needed to Treat (NNT) or the Number Needed to Harm (NNH) was reported for each statistically significant outcome where the baseline risk (i.e., control group event rate) was similar across studies. In addition, NNTs calculated at follow-up were only reported where the length of follow-up was similar across studies. When length of follow-up or baseline risk varies (especially with low risk), the NNT is a poor summary of the treatment effect (Deeks, 2002).

Both the I² test of heterogeneity and the chi-squared test of heterogeneity (p <0.10) were used, as well as visual inspection of the forest plots, to look for the possibility of heterogeneity. I² describes the proportion of total variation in study estimates that is due to heterogeneity (Higgins & Thompson, 2002). An I² of less than 30 per cent was taken to indicate mild heterogeneity and a fixed effects model was used to synthesise the results. An I² of more than 50 per cent was taken as notable heterogeneity. In this case an attempt was made to explain the variation. If studies with heterogeneous results were found to be comparable, a random effects model was used to summarise the results (DerSimonian & Laird, 1986). In the random effects analysis, heterogeneity is accounted for both in the width of CIs and in the estimate of the treatment effect. With decreasing heterogeneity the random effects approach moves asymptotically towards a fixed effects model. An I² of 30 to 50 per cent was taken to indicate moderate heterogeneity. In this case, both the chi-squared test of heterogeneity and a visual inspection of the forest plot was used to decide between a fixed and random effects model.

To explore the possibility that the results entered into each meta-analysis suffered from publication bias, data from included studies were entered, where there was sufficient data, into a funnel plot. Asymmetry of the plot was taken to indicate possible publication bias and investigated further.

3.4.6. Forming and grading the statements and recommendations

The evidence tables and forest plots formed the basis for developing clinical statements and recommendations. For intervention studies, the statements were classified according to an accepted hierarchy of evidence. Recommendations were then graded A to C based on the level of associated evidence (see Table 1 overleaf). Key priorities for implementation are denoted by an asterisk following the grade.

Table 1

Hierarchy of evidence and recommendations grading scheme.

In order to facilitate consistency in generating and drafting the clinical statements the GDG utilised a statement decision tree (see Flowchart 2). The flowchart was designed to assist with, but not replace clinical judgement.

Where a statistically significant summary statistic (effect size; ES) was obtained (after controlling for heterogeneity), the GDG considered whether this finding was of clinical significance (i.e. likely to be of benefit to patients) taking into account the trial population, nature of the outcome and size of the effect. On the basis of this consideration the ES was characterised as ‘clinically significant’ or not. A further consideration was made about the strength of the evidence by examining the CI surrounding the ES. For level I evidence, where the ES was judged to be clinically significant and had a CI entirely within a clinical relevant range, the result was characterised as ‘strong evidence’ (S1, Flowchart 2). For non-level I evidence or in situations where the upper/lower bound of the CI was not clinically significant, the result was characterised as ‘limited evidence’ (S2). Where an ES was statistically significant, but not clinically significant and the CI excluded values judged to be clinically important, the result was characterised as ‘unlikely to be clinically significant’ (S3). Alternatively, if the CI included clinically important values, the result was characterised as ‘insufficient to determine clinical significance’ (S6).

Flowchart 2. Guideline Statement Decision Tree (PDF, 26K)

Where a non-statistically significant ES was obtained, the GDG reviewed the trial population, nature of the outcome, size of the effect and, in particular, the CI surrounding the result. If the CI was narrow and excluded a clinically significant ES, this was seen as indicating evidence of ‘no clinically significant difference’ (S4), but where the CI was wide this was seen as indicating ‘insufficient evidence’ to determine if there was a clinically significant difference or not (S5).

Once all evidence statements relating to a particular clinical question were finalised and agreed by the GDG, the associated recommendations were produced and graded. Grading the recommendations allowed the GDG to distinguish between the level of evidence and the strength of the associated recommendation. It is possible that a statement of evidence would cover only one part of an area in which a recommendation was to be made or would cover it in a way that would conflict with other evidence. In order to produce more comprehensive recommendations suitable for people in England and Wales, the GDG had to extrapolate from the available evidence. This led to a weaker level of recommendation (i.e. B, as data were based upon level I evidence). It is important to note that the grading of the recommendation is not a reflection of its clinical significance or relevance.

A number of issues relating to the study of eating disorders (see Section 3.4.8) meant that the outcomes available for analysis were classified as primary or secondary. When making recommendations, the primary outcomes were given more weight during the decision process.

The process also allowed the GDG to moderate recommendations based on factors other than the strength of evidence. Such considerations include the applicability of the evidence to people with eating disorders, economic considerations, values of the development group and society, or the group’s awareness of practical issues (Eccles et al., 1998).

3.4.7. Method used to answer a clinical question in the absence of appropriately designed, high-quality research

Where it was not possible to identify at least one appropriately designed study or high-quality systematic review, or where the GDG was of the opinion (on the basis of previous searches or their knowledge of the literature) that there was unlikely to be appropriately designed primary-level research that directly addressed the clinical question, an informal consensus process was adopted. This process focused on those questions that the GDG considered a priority.

The starting point for this process of informal consensus was that a member of the topic group identified, with help from the systematic reviewer, a narrative review that most directly addressed the clinical question. Where this was not possible a new review of the recent literature was initiated.

This existing narrative review or new review was used as a basis for identifying lower levels of evidence relevant to the clinical question. This was then presented for discussion to the GDG. On the basis of this, additional information was sought and added to the information collected. This may include studies that did not directly address the clinical question but were thought to contain relevant data. This led to the development of an initial draft report that addressed the following issues:

• A description of what is known about the issues concerning the clinical question.
• A brief review of the existing evidence, including RCTs, non-randomised controlled studies, cohort studies and other studies that help answer the question.
• The summary of the evidence so far obtained. This was then presented in narrative form to the GDG and further comments were sought about the evidence and its perceived relevance to the clinical question.
• If, during the course of preparing the report, a significant body of primary-level studies (of appropriate design to answer the question) were identified, a full systematic review was done.
• At this time, subject possibly to further reviews of the evidence, a series of statements that directly addressed the clinical question were developed.
• Following this, on occasions and as deemed appropriate by the development group, the report was then sent to appointed experts outside of the GDG for peer review and comment. The information from this process was then fed back to the GDG for further discussion of the statements.
• Recommendations were then developed and could also be sent for further external peer review.
• After this final stage of comment, the statements and recommendations were again reviewed and agreed upon by the GDG.

3.4.8. Issues concerning research on eating disorders

Studying the treatment of eating disorders presents certain particular challenges. First, since psychological treatments have a major role in their management, the methods needed to evaluate psychotherapeutic treatments have to be adopted. Some of these are mentioned in the national clinical practice guideline on schizophrenia (Kendall et al., 2003) including the need for some degree of standardisation of the psychological treatment provided (usually in the form of a treatment manual) and the importance of quality assessment and control. Additional refinements have been included in some eating disorder studies including the use of ‘dismantling’ research designs to identify the active components of psychological treatments (e.g. Fairburn et al., 1991, 1993); studies of the mediators of psychological treatments, a stage towards the identification of their mechanism of action (e.g. Wilson et al., 2002); fine-grain comparisons of different ways of implementing specific psychotherapeutic procedures (e.g. Bulik et al., 1998); and ‘effectiveness’ studies of simplified psychological treatments designed for general dissemination (e.g. Carter & Fairburn, 1998).

The second challenge is the measurement of outcome. Eating disorders affect many aspects of functioning including behaviour (e.g. eating habits), attitudes and values (e.g. the evaluation of shape and weight), psychosocial functioning (e.g. levels of depression and anxiety; interpersonal functioning) and physical health (e.g. body weight, serum electrolytes) (Fairburn & Harrison, 2003). It is also usual for there to be comorbidity with Axis I and II disorders co-existing with the eating disorder (Bulik, 2002). Therefore, most treatment studies employ a range of different outcome measures to determine whether the treatments studied affect all or part of the psychopathology present. Having said this, there are accepted primary outcome variables that treatments must influence if they can be said to have an impact on the core eating disorder. For anorexia nervosa, the primary outcome variable is body weight adjusted for height, usually represented as the BMI or the percentage of expected weight for the person’s age, height and sex. For bulimia nervosa the comparable variable is the frequency of binge eating and ‘purging’ (self-induced vomiting or the use of laxatives to influence body shape or weight); that is, the frequency of these forms of behaviour over a set period of time (usually one or four weeks). In addition, it is now established practice to report the proportion of participants who no longer practise the behaviour (sometime referred to as the ‘abstinence’ rates). For binge eating disorder the primary outcome variable is the frequency of binge eating, represented as for bulimia nervosa. An important point to note is that the frequency of binge eating and purging is skewed rather than being normally distributed so these data have to be transformed prior to analysis. However, because most reports give uncorrected means and standard deviations, the GDG gave less weight to frequency data than abstinence rates when making recommendations.

The third challenge is determining whether treatment effects persist over time. Anorexia nervosa and bulimia nervosa tend to run a chronic course so short-lived treatment effects are of limited clinical significance. Thus it has become established research practice to follow-up patients following the end of treatment to see whether any changes last, and with interventions that are known to be associated with a high rate of relapse (e.g. inpatient treatment of anorexia nervosa) there have been attempts to develop and evaluate subsequent ‘relapse prevention’ strategies (e.g. Kaye et al., 2001; Pike et al., 2003).

The fourth challenge is largely peculiar to research on the treatment of anorexia nervosa. It is exceptionally difficult recruiting cases for treatment studies. This is for a number of reasons. First, anorexia nervosa is uncommon. Second, it can be life-threatening with the result that it is not ethical to recruit certain patients. And third, patients with anorexia nervosa are notoriously reluctant to have treatment and this reluctance can be magnified if the clinician raises the possibility of taking part in a research trial. For these reasons the few studies of the treatment of anorexia nervosa have mostly involved small numbers of patients. As a result it is difficult to draw robust evidence-based conclusions about the treatment of the disorder.