3.3.1. Guideline Development Group meetings

Twelve GDG meetings were held between 14 March, 2013, and 2 September, 2014. During each day-long GDG meeting, in a plenary session, review questions and clinical and economic evidence were reviewed and assessed, and recommendations formulated. At each meeting, all GDG members declared any potential conflicts of interest (see Appendix 2), and service user concerns were routinely discussed as a standing agenda item.

3.3.2. Topic groups

The GDG divided its workload along clinically relevant lines to simplify the guideline development process, and GDG members formed smaller topic groups to undertake guideline work in that area of clinical practice. Topic Group 1 covered questions relating to case identification. Topic Group 2 covered psychological and psychosocial interventions, and Topic Group 3 covered pharmacological interventions. These groups were designed to efficiently manage the large volume of evidence appraisal prior to presenting it to the GDG as a whole. Each topic group was chaired by a GDG member with expert knowledge of the topic area (one of the healthcare professionals). Topic groups refined the review questions and the clinical definitions of treatment interventions, reviewed and prepared the evidence with the systematic reviewer before presenting it to the GDG as a whole, and helped the GDG to identify further expertise in the topic. Topic group leaders reported the status of the group's work as part of the standing agenda. They also introduced and led the GDG's discussion of the evidence review for that topic and assisted the GDG Chair in drafting the section of the guideline relevant to the work of each topic group.

3.3.3. Service users

Individuals with direct experience of services gave an integral service-user focus to the GDG and the guideline. The GDG included a service user and representatives of a national service user group. They contributed as full GDG members to writing the review questions, providing advice on outcomes most relevant to service users, helping to ensure that the evidence addressed their views and preferences, highlighting sensitive issues and terminology relevant to the guideline, and bringing service user research to the attention of the GDG. In drafting the guideline, they reviewed the chapter on experience of care and identified recommendations from the service user perspective.

3.3.4. Special advisors

Special advisors, who had specific expertise in one or more aspects of treatment and management relevant to the guideline, assisted the GDG, commenting on specific aspects of the developing guideline and making presentations to the GDG. Appendix 3 lists those who agreed to act as special advisors.

3.3.5. National and international experts

National and international experts in the area under review were identified through the literature search and through the experience of the GDG members. These experts were contacted to identify unpublished or soon-to-be published studies, to ensure that up-to-date evidence was included in the development of the guideline. They informed the GDG about completed trials at the pre-publication stage, systematic reviews in the process of being published, studies relating to the cost effectiveness of treatment and trial data if the GDG could be provided with full access to the complete trial report. Appendix 5 lists researchers who were contacted.

3.5.1. The search process

3.5.2. Data extraction

Qualitative analysis

After transcripts/reviews or primary studies of service user experience were identified (see 3.5.1), each was read and re-read and sections of the text were collected under different headings using an Excel-based form. Initially the text from the transcripts/reviews was organised using a matrix of service user experience (see Table 5).

Table 5

Matrix of service user experience.

The matrix was formed by creating a table with the eight dimensions of patient-centred care developed by the Picker Institute Europe², down the vertical axis, and the key points on a pathway of care (as specified by the GDG) across the horizontal axis. With regard to terminology, the GDG preferred the term ‘person-centred’ rather than ‘patient-centred’, therefore the former is used in the matrix. The Picker Institute's dimensions of patient-centred care were chosen because they are well established, comprehensive, and based on research. In addition, a variation of these dimensions has been adopted by the US Institute of Medicine (Institute of Medicine, 2001).

Under the broad headings in the matrix, specific emergent themes were identified and coded by two researchers working independently. Overlapping themes and themes with the highest frequency count across all testimonies were extracted and regrouped using the matrix. The findings from this qualitative analysis can be found in Chapter 8.

3.5.3. Evidence synthesis

The method used to synthesize evidence depended on the review question and availability and type of evidence (see Appendix 12 for full details). Briefly, for questions about test accuracy, bivariate test accuracy meta-analysis was conducted where appropriate. For questions about the effectiveness of interventions or harms associated with interventions, standard meta-analysis or network meta-analysis was used where appropriate, otherwise narrative methods were used with clinical advice from the GDG. In the absence of high-quality research, an informal consensus process was used (see Section 3.5.7).

3.5.4. Grading the quality of evidence

For questions about the effectiveness of interventions, the GRADE approach³ was used to grade the quality of evidence for each outcome (Guyatt et al., 2011). For questions about the experience of care, test accuracy, and harms associated with interventions (where case-control and cohort study designs were used) methodology checklists were used to assess the risk of bias, and this information was taken into account when interpreting the evidence. The technical team produced GRADE evidence profiles (see below) using GRADEprofiler (GRADEpro) software (Version 3.6), following advice set out in the GRADE handbook (Schünemann et al., 2009). All staff doing GRADE ratings were trained, and calibration exercises were used to improve reliability (Mustafa et al., 2013).

Evidence profiles

A GRADE evidence profile was used to summarise both the quality of the evidence and the results of the evidence synthesis for each ‘critical’ and ‘important’ outcome (see Table 6 for an example of an evidence profile). The GRADE approach is based on a sequential assessment of the quality of evidence, followed by judgment about the balance between desirable and undesirable effects, and subsequent decision about the strength of a recommendation.

Table 6

Example of GRADE evidence profile.

Within the GRADE approach to grading the quality of evidence, the following is used as a starting point:

• RCTs without important limitations provide high quality evidence
  • observational studies without special strengths or important limitations provide low quality evidence.

For each outcome, quality may be reduced depending on five factors: limitations, inconsistency, indirectness, imprecision and publication bias. For the purposes of the guideline, each factor was evaluated using criteria provided in Table 7.

Table 7

Factors that decrease quality of evidence.

For observational studies without any reasons for down-grading, the quality may be up-graded if there is a large effect, all plausible confounding would reduce the demonstrated effect (or increase the effect if no effect was observed), or there is evidence of a dose-response gradient (details would be provided under the ‘other’ column).

Each evidence profile includes a summary of findings: number of participants included in each group, an estimate of the magnitude of the effect, and the overall quality of the evidence for each outcome. Under the GRADE approach, the overall quality for each outcome is categorised into one of four groups (high, moderate, low, very low).

3.5.5. Presenting evidence to the Guideline Development Group

Study characteristics tables and, where appropriate, forest plots generated with Review Manager Version 5.2 and GRADE summary of findings tables (see below) were presented to the GDG.

Where meta-analysis was not appropriate and/or possible, the reported results from each primary-level study were reported in the study characteristics table and presented to the GDG. The range of effect estimates were included in the GRADE profile, and where appropriate, described narratively.

Summary of findings tables

Summary of findings tables generated from GRADEpro were used to summarise the evidence for each outcome and the quality of that evidence (Table 8). The tables provide illustrative comparative risks, especially useful when the baseline risk varies for different groups within the population.

Table 8

Example of a GRADE summary of findings table.

3.5.6. Extrapolation

When answering review questions, if there is no direct evidence from a primary dataset,⁴ based on the initial search for evidence, it may be appropriate to extrapolate from another data set. In this situation, the following principles were used to determine when to extrapolate:

• a primary dataset is absent, of low quality or is judged to be not relevant to the review question under consideration, and
• a review question is deemed by the GDG to be important, such that in the absence of direct evidence, other data sources should be considered, and
• non-primary data source(s) is in the view of the GDG available, which may inform the review question.

When the decision to extrapolate was made, the following principles were used to inform the choice of the non-primary dataset:

• the populations (usually in relation to the specified diagnosis or problem which characterises the population) under consideration share some common characteristic but differ in other ways, such as age, gender or in the nature of the disorder (for example, a common behavioural problem; acute versus chronic presentations of the same disorder) , and
• the interventions under consideration in the view of the GDG have one or more of the following characteristics:

  -

  share a common mode of action (for example, the pharmacodynamics of drug; a common psychological model of change - operant conditioning)

  -

  be feasible to deliver in both populations (for example, in terms of the required skills or the demands of the health care system)

  -

  share common side effects/harms in both populations, and

• the context or comparator involved in the evaluation of the different datasets shares some common elements which support extrapolation, and
• the outcomes involved in the evaluation of the different datasets shares some common elements which support extrapolation (for example, improved mood or a reduction in challenging behaviour).

When the choice of the non-primary dataset was made, the following principles were used to guide the application of extrapolation:

• the GDG should first consider the need for extrapolation through a review of the relevant primary dataset and be guided in these decisions by the principles for the use of extrapolation
• in all areas of extrapolation datasets should be assessed against the principles for determining the choice of datasets. In general the criteria in the four principles set out above for determining the choice should be met
  • in deciding on the use of extrapolation, the GDG will have to determine if the extrapolation can be held to be reasonable, including ensuring that:

    -

    the reasoning behind the decision can be justified by the clinical need for a recommendation to be made

    -

    the absence of other more direct evidence, and by the relevance of the potential dataset to the review question can be established

    -

    the reasoning and the method adopted is clearly set out in the relevant section of the guideline.

3.5.7. Method used to answer a review question in the absence of appropriately designed, high-quality research

In the absence of appropriately designed, high-quality research (including indirect evidence where it would be appropriate to use extrapolation), an informal consensus process was adopted.

The process involved a member of the GDG or review team drafting a statement about what is known about the issue based on expert opinion from existing narrative reviews. The statement was circulated to the GDG and used as the basis of a group discussion.

3.5.8. Key principles for recommendations

In reviewing the evidence for mental health problems in pregnancy and/or the postnatal period the GDG were guided by the principle that much of the assessment and treatment of mental health problems in pregnancy and the postnatal period is not different from that at other times of a woman's life, and so should be guided by relevant NICE guidelines for the specific mental health problem. However, new recommendations were developed where there was new evidence specifically for this guideline:

• for an intervention that was specific to pregnancy or the postnatal period;
• that an existing recommendation needed to be clarified or modified as a result of concerns about the health of the fetus or infant;
• that changes are necessary to the context in which interventions are delivered;
  • that specific variations are necessitated by changes in a woman's mental or physical health linked to pregnancy and the postnatal period.

3.6.1. Search strategy for economic evidence

3.6.2. Inclusion criteria for economic studies

The following inclusion criteria were applied to select studies identified by the economic searches for further consideration:

• Only studies from Organisation for Economic Co-operation and Development countries were included, as the aim of the review was to identify economic information transferable to the UK context.
• Only English language papers were considered.
• Studies published from 2006 onwards were included. This date restriction was imposed to obtain data relevant to current healthcare settings and costs.
• Selection criteria based on types of clinical conditions and service users as well as interventions assessed were identical to the clinical literature review.
• Studies were included provided that sufficient details regarding methods and results were available to enable the methodological quality of the study to be assessed, and provided that the study's data and results were extractable. Poster presentations, abstracts, dissertations, commentaries and discussion publications were excluded.
• Full economic evaluations that compared two or more relevant interventions and considered both costs and consequences, as well as costing analyses comparing only costs between two or more interventions, were included in the review.
• Economic studies were included if they used clinical effectiveness data from an RCT, a prospective cohort study, or a systematic review and meta-analysis of clinical studies. Studies that had a mirror-image or other retrospective design were excluded from the review. Also, studies that utilised clinical effectiveness parameters based mainly on expert opinion or assumptions were excluded from the review.
• Studies were included only if the examined interventions and populations under consideration were clearly described.

3.6.3. Applicability and quality criteria for economic studies

All economic papers eligible for inclusion were appraised for their applicability and quality using the methodology checklist for economic evaluations recommended by NICE (NICE, 2012a). The methodology checklist for economic evaluations was also applied to the economic models developed specifically for this guideline. All studies that fully or partially met the applicability and quality criteria described in the methodology checklist were considered during the guideline development process, along with the results of the economic modelling conducted specifically for this guideline. The completed methodology checklists for all economic evaluations considered in the guideline are provided in Appendix 20.

3.6.4. Presentation of economic evidence

The economic evidence considered in the guideline is provided in the respective evidence chapters, following presentation of the relevant clinical evidence. The references to included studies and the respective evidence tables with the study characteristics and results are provided in Appendix 21. Methods and results of economic modelling undertaken alongside the guideline development process are presented in the relevant evidence chapters. Characteristics and results of all economic studies considered during the guideline development process (including modelling studies conducted for this guideline) are summarised in economic evidence profiles accompanying respective GRADE clinical evidence profiles in Appendix 22.

3.6.5. Results of the systematic search of economic literature

The titles of all studies identified by the systematic search of the literature were screened for their relevance to the topic (that is, economic issues and information on HRQoL). References that were clearly not relevant were excluded first. The abstracts of all potentially relevant studies (15 references) were then assessed against the inclusion criteria for economic evaluations by the health economist. Full texts of the studies potentially meeting the inclusion criteria (including those for which eligibility was not clear from the abstract) were obtained. Studies that did not meet the inclusion criteria, were duplicates, were secondary publications of one study, or had been updated in more recent publications were subsequently excluded. Economic evaluations eligible for inclusion (9 studies in 12 publications) were then appraised for their applicability and quality using the methodology checklist for economic evaluations. Finally, 9 economic studies that fully or partially met the applicability and quality criteria were considered at formulation of the guideline recommendations.

3.7.1. Incorporation

In the current guideline, the following criteria were used to determine when a recommendation could be incorporated:

• a review question in the current guideline was addressed in another NICE guideline
• evidence for the review question and related recommendation(s) has not changed in important ways
• evidence for the previous question is judged by the GDG to support the existing recommendation(s), and be relevant to the current question
• the relevant recommendation can ‘stand alone’ and does not need other recommendations from the original guideline to be relevant or understood within the current guideline.

3.7.2. Adaptation

The following criteria were used to determine when a recommendation could be adapted:

• a review question in the current guideline is similar to a question addressed in another NICE guideline
• evidence for the review question and related recommendations has not changed in important ways
• evidence for the previous question is judged by the GDG to support the existing recommendation(s), and be relevant to the current question
• the relevant recommendation can ‘stand alone’ and does not need other recommendations from the original guideline to be relevant
• contextual evidence, such as background information about how an intervention is provided in the healthcare settings that are the focus of the guideline, informs the re-drafting or re-structuring of the recommendation but does not alter its meaning or intent (if meaning or intent were altered, a new recommendation should be developed).

In deciding whether to choose between incorporation or adaption of existing guideline recommendations, the GDG considered whether the direct evidence obtained from the current guideline dataset was of sufficient quality to allow development of recommendations. It was only where (a) such evidence was not available or insufficient to draw robust conclusions and (b) where methods used in other NICE guidelines were sufficiently robust that the ‘incorporate and adapt’ method could be used. Recommendations were only incorporated or adapted after the GDG had reviewed evidence supporting previous recommendations and confirmed that they agreed with the original recommendations.

When adaptation is used, the meaning and intent of the original recommendation is preserved but the wording and structure of the recommendation may change. Preservation of the original meaning (that is, that the recommendation faithfully represents the assessment and interpretation of the evidence contained in the original guideline evidence reviews) and intent (that is, the intended action[s] specified in the original recommendation will be achieved) is an essential element of the process of adaptation.

3.7.3. Roles and responsibilities

The guideline review team, in consultation with the guideline Facilitator and Chair, were responsible for identifying overlapping questions and deciding if it would be appropriate to incorporate or to adapt following the principles above. For adapted recommendations, at least two members of the GDG for the original guideline were consulted to ensure the meaning and intent of the original recommendation was preserved. The GDG confirmed the process had been followed, that there was insufficient evidence to make new recommendations, and agreed all adaptations to existing recommendations.

In evidence chapters where incorporation and adaptation have been used, the original review questions are listed with the rationale for the judgement on the similarity of questions. Tables are then provided that set out the original recommendation, a brief summary of the original evidence, the new recommendation, and the reasons for adaptation. For an adapted recommendation, details of any contextual information are provided, along with information about how the GDG ensured that the meaning and intent of the adapted recommendation was preserved.

3.7.4. Drafting of adapted recommendations

The drafting of adapted recommendations conformed to standard NICE procedures for the drafting of guideline recommendations, preserved the original meaning and intent, and aimed to minimise the degree or re-writing and re-structuring.