U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Young A, Rogers K, Davies L, et al. Evaluating the effectiveness and cost-effectiveness of British Sign Language Improving Access to Psychological Therapies: an exploratory study. Southampton (UK): NIHR Journals Library; 2017 Aug. (Health Services and Delivery Research, No. 5.24.)

Cover of Evaluating the effectiveness and cost-effectiveness of British Sign Language Improving Access to Psychological Therapies: an exploratory study

Evaluating the effectiveness and cost-effectiveness of British Sign Language Improving Access to Psychological Therapies: an exploratory study.

Show details

Chapter 7Exploratory economic evaluation (study 5)

Background

Economic evaluation compares the costs and health benefits (such as symptoms cured, life-years gained, improvements in overall health) of different care or treatments. The aim is to help patients, practitioners and commissioners choose care that offers value for money. A key concept is that care should be selected to give the most health benefit within the budget available. This is often translated to giving the most health benefit for the lowest cost. It is not known whether or not BSL-IAPT confers any benefit for Deaf people over and above accessing standard IAPT services. Similarly, the cost-effectiveness of BSL-IAPT has not been investigated. Rigorous examination of effectiveness and cost-effectiveness is needed to guide decision-making about longer-term sustainability and appropriate targeted primary care intervention for this hard-to-reach group.

Objectives

There were two key objectives of the economic evaluation:

  1. to explore the potential costs of health and social care, and the health benefit in terms of QALYs for Deaf users of BSL-IAPT and Deaf users accessing standard IAPT
  2. to estimate the net cost per QALY gained by BSL-IAPT.

The key research questions were:

  • The primary analysis addressed the question of whether or not BSL-IAPT was potentially cost-effective compared with standard IAPT, when service-specific PHQ-9 and GAD-7 tools and cut-off points were used to identify people with depression/anxiety and measure recovery.
  • Does the potential cost-effectiveness of BSL-IAPT vary if different assumptions are made about the probability, costs or QALYs of events?

Methods

Approach

The economic evaluation aimed to explore whether or not BSL-IAPT is cost-effective compared with standard IAPT services for the treatment of depression and/or anxiety in the Deaf BSL population in the UK. The overall perspective or decision-maker viewpoint used to determine the range of costs is that of health and social care providers. However, data constraints limited the perspective for the initial 6-month time horizon, when people are receiving IAPT treatment, to the providers of IAPT services. The costs of other health care and social care are not included. However, the costs of relapse/recurrence and recovery after the first 6 months do include a broader range of costs.

In the absence of a head-to-head trial of standard IAPT and BSL-IAPT for Deaf BSL users, it was recognised that a decision-analytic model to combine or synthesise available data was needed. The economic model focuses on Deaf BSL adults who are referred to IAPT for a low- (defined by NICE as step 2)142 or high-intensity (defined by NICE as step 3)142 intervention for the treatment of depression and/or anxiety. The key intervention is the BSL-IAPT specialist service, as provided by BSL Healthy Minds. This service was designed to meet the cultural and linguistic needs of the Deaf BSL population. IAPT interventions are delivered by health-care workers fluent in BSL, the majority of whom are Deaf. This is compared with the standard IAPT service, which the Deaf BSL population may access, usually through a sign language interpreter. Standard IAPT represents routine care for most Deaf BSL users.

The economic model estimates the costs and QALYs for 1 year after a person’s first contact with the service (i.e. the time horizon is 1 year), for the main or primary analysis. Depression and anxiety are long-term conditions, so the economic model also explored the costs and QALYs over longer time periods. However, these longer-term analyses are likely to be more uncertain, as the many of the data available for the model were collected over the short term.

The costs and outcomes for the long-term time horizons are discounted at a 3.5% rate per year, in line with UK recommendations. Discounting takes account of the fact that we prefer to take benefits now and delay costs to the future. Discounting is not needed for analyses using time frames of ≤ 1 year.133 The price year is 2015 and costs are presented in UK pounds sterling (£).

The population, intervention, control, outcome and time frame statement

  • The target population for the economic model was Deaf BSL users with depression and/or anxiety. Although service users may have only one of these conditions, it is common to suffer from both.143–146 This was demonstrated in the IAPT data sets, in which 75.9% (standard IAPT) and 81.6% (BSL-IAPT) of participants had mixed depression and anxiety at the point of entry into the IAPT service (see Table 14).
  • The intervention was the BSL-IAPT service, a specialist version of the IAPT service, with fluent BSL users delivering IAPT interventions.
  • The comparator was standard IAPT. In situations in which a Deaf BSL user is referred to a standard IAPT service, they are most likely to receive this intervention through the use of an interpreter.
  • The primary outcome of the economic analysis was the incremental cost-effectiveness ratio (ICER). This was estimated as the net cost of the intervention divided by the net health benefit. The measure of health benefit for the primary analysis was the QALY.
  • The primary time horizon for primary analysis was 1 year from entry into the decision model.

Model overview

In reality, depression and anxiety are complex conditions with varied and diverse treatment pathways. However, models are a simplification of real life. The economic model presents a simplification of the pathway of care for Deaf BSL users accessing IAPT services to capture the key costs and health benefits associated with IAPT treatment.

A review of the literature (see Appendix 8) indicated that there were no existing published economic evaluations of the BSL-IAPT service. Accordingly, a new decision-analytic model was developed to estimate the cost-effectiveness of BSL-IAPT. The model structure was agreed with the project team, including the PPI group, and the SSC, which included a health economics representative.

Model structure

The economic model structure includes two distinct parts: a decision tree and a Markov model.

The decision tree used the IAPT pathway18 and the stepped care pathway recommended by NICE142 to model the treatment events for depression and/or anxiety within the IAPT service. A diagram of the decision tree is shown in Figure 28. The decision tree illustrates the pathway of events that a person could experience during treatment. A number of events may happen at the same time in the decision tree; the pathway reflects the combination of events that a person could experience, but not necessarily the sequence they occur in.

FIGURE 28. Decision tree.

FIGURE 28

Decision tree.

The decision tree begins with a Deaf BSL user referred (or who self-refers) to an IAPT service and models the events that may occur in the first 6 months from referral. This is represented in Figure 28 by a square node. Standard IAPT and BSL-IAPT pathways follow the same decision tree structure, although the probabilities, costs and benefits of different events may vary between the two interventions.

It is possible that people will exit the pathway before treatment. This may be for various reasons (e.g. their symptoms changed during the wait, which means that they are now no longer suitable for IAPT treatment, or they received another intervention during the wait and now no longer need the IAPT service). For people starting treatment, there are two possible options: low- or high-intensity treatment (step 2 and step 3 in the NICE guidelines).142 The decision to refer to low- or high-intensity treatment will depend on patient characteristics/symptoms and treatment availability. Low- and high-intensity referrals have the same model structure in the decision tree, but have different probabilities and costs.

The model recognises that the intensity level may or may not be appropriate for the user’s needs. Service users may remain on the same treatment intensity throughout their IAPT therapy course (if appropriate), or they may need to change from low- to high-intensity or from high- to low-intensity services during treatment if the initial treatment is not appropriate. It is plausible that the BSL-IAPT service may be more able to appropriately refer users to low- or high-intensity intervention if it can assess symptoms more accurately (e.g. through more fluent and direct communication or using the BSL versions of the PHQ-9 and GAD-7 or more appropriate clinical cut-off points). Alternatively, services may be similar in terms of whether or not patients are appropriately referred, but BSL-IAPT may be better at identifying when a patient needs to change the intensity of treatment. Increasing the service’s ability to identify appropriate changes in treatment intensity effectively gives another chance of having a treatment benefit in the model. For the 6 months covered by the decision tree it is assumed that users will have multiple IAPT interventions only if the intensity of treatment is changed.

Service users may complete treatment or disengage and drop out, with these groups having different chances of benefiting from treatment. The benefit of treatment includes two potential outcomes: full recovery or partial recovery. It is assumed that those people who do not recover following treatment are either referred to other services or have no further treatment. The treatment pathways following a course of IAPT therapy are variable and complex and so further treatments are not explicitly included in the decision tree. However, they are included in the costs and outcomes of events in the Markov section of the economic model.

The Markov section of the model was used to capture longer time horizons and that depression and anxiety can be cyclical with recovery followed by relapse (return of symptoms before full recovery) or recurrence (recurrence of depression/anxiety symptoms following full recovery). It includes three possible states: partial or full recovery from depression/anxiety, relapse/recurrence and death. The Markov model is presented in Figure 29. Which health state people enter the Markov section of the model in is determined by the end points of the decision tree. For the decision tree pathways that end in recovery, the Markov model starts with a state of recovery (which is split into two substates for full recovery and partial recovery). Users can only move into the partial response state during the decision tree (i.e. following the modelled treatment). In the Markov model these users will either remain in this state or move out of it (into the recovery substate of full recovery, relapse or death). For the decision tree pathways that end in no recovery, the Markov model starts with a state of active depression/anxiety symptoms, which is termed as relapse/recurrence for subsequent cycles of the model. The Markov model allows transitions to occur between 3-month cycles, a half-cycle correction was applied.

FIGURE 29. Markov model.

FIGURE 29

Markov model. a, Recovery consists of two substates: full and partial recovery. Users can only move into the partial recovery substate on entry to the Markov model, in the Markov they can remain or transition (move) to another state. There are no data (more...)

The economic model was constructed in Microsoft Excel® 2010 (Microsoft Corporation, Redmond, WA, USA).

Users can move between health states (depicted by the arrows moving between states) and can remain in those states over a number of cycles (shown by the arrow looping back into the health state. Death is an absorbing state; people will remain in this state (shown by the single arrow looping back into ‘dead’).

Model parameters and data sources

Overview

The data for the model included the following categories: probabilities of events, outcomes, utilities, resource use and costs. Tables of the data estimates and sources used in the model are presented in Appendix 9.

The economic model synthesised data from a number of studies across the programme of work, as well as data from the wider literature. Data on outcomes and resource use (number of appointments and treatment intensity/step level) were estimated from the IAPT data sets analysed in study 2 (see Chapter 4). These data sets allowed us to make a comparison between the service options in terms of treatment effectiveness and the resource use and costs associated with IAPT intervention. The translation and validation of the BSL EQ-5D-5L (study 5; see Chapter 6), provided a sample of data to estimate utility values for Deaf BSL users.

These data were supplemented with data from the literature to estimate the probabilities, costs and outcomes over the longer term. This included the long-term rates of relapse/recurrence and recovery, costs of depression and/or anxiety, and mortality rates. Literature was identified through targeted searches of The Cochrane Library, NHS Evidence and MEDLINE database. Owing to a lack of evidence, which was specific to the Deaf population, data from the wider population had to be used to estimate some model parameters; this is discussed in Limitations. The choice of inputs in these cases was based on the availability of the best-quality (according to the hierarchy of evidence) and most relevant data. Given the probable mixed depression/anxiety diagnosis, the model used data relevant to the mixed diagnosis when possible. When these were not available, data that were either relevant to depression or anxiety were used, prioritising the best-quality sources of data.

Likelihood of treatment events

Probabilities of treatment events in the decision tree were estimated using the data from the BSL-IAPT and standard IAPT data sets described in Chapter 4. This assumes that the participants included in these two data sets are representative of the target population for the economic study. Parameter values used in the model are provided in Appendix 10, Tables 33 and 34.

After referral, the probability of whether a person continues to treatment or leaves the service before treatment was estimated from the total number of clients as the proportion of people with recorded appointments and the proportion of people with no recorded appointments, respectively. The probability a person had low- or high-intensity treatment was estimated as the proportion of service users with one or more attended appointments starting at each intensity level. Data about whether or not treatment was appropriate for a person’s needs were not available from the IAPT data sets. The probability that treatment was appropriate for a person’s needs was estimated by assuming that service users within the data sets were referred to an appropriate treatment if they did not experience a step change.

If the treatment given was appropriate for a person’s needs, then it was assumed that there would be no planned change in treatment. There is then the probability that a person completes treatment or not. The IAPT literature classifies service users with two or more attended appointments as having completed a treatment episode.147 Accordingly, the probability of completing treatment was calculated as the proportion of users attending two or more appointments.

If the treatment given was not appropriate for a person’s needs, then there is the chance that they continue with that treatment or change the intensity of treatment. Data about whether people would remain on treatment or experience a switch was not available from the IAPT databases. Therefore, it was assumed that, if treatment was not appropriate, treatment intensity would be changed (i.e. patients on low-intensity treatment would experience a ‘step up’ to high-intensity treatment and vice versa).

The likelihood of treatment effect (full recovery or partial recovery) for use in the decision tree was also estimated using the IAPT data sets (BSL-IAPT or standard IAPT) described above. For the economic model, ‘recovery rate’ was derived from the BSL study’s clinical effectiveness analysis and additionally broken down by treatment intensity. Therefore, unlike the main clinical effectiveness, it was based on complete case analysis within these intensity subgroups. Full recovery was defined as the IAPT measure of reliable recovery (treatment was completed and PHQ-9 and GAD-7 scores are below the clinical cut-off point for depression/anxiety106), and the probability was estimated as the proportion of users within a data set who experienced reliable recovery. Unlike the main clinical effectiveness analysis, the economic model also took account of ‘partial recovery rate’. Partial recovery was defined as the IAPT measure of reliable improvement (treatment was completed and PHQ-9 and GAD-7 scores are lower than at referral but not below the clinical cut-off point for depression/anxiety106), and the probability was calculated as the users experiencing a reliable improvement but without reliable recovery.

Treatment effect rates for users who leave the service before starting treatment, or who drop out of treatment, could not be calculated as there are no follow-up data available in the IAPT data sets. In the primary analysis, these users are assumed to have no chance of recovery or partial recovery. It is possible that these people left the service because their symptoms improved or resolved. Accordingly, this assumption was tested in the sensitivity analysis.

The longer-term probabilities of relapse/recurrence and recovery were sourced from the published literature. The probabilities and published sources used are shown in Appendix 10, Table 35.

Mortality was not included in the decision tree section of the model for two reasons:

  1. On referral to IAPT, initial assessment checks for warning signs for self-harm or suicide would be carried out and service users exhibiting these warning signs would be referred to services for complex and severe depression or alternative services and so leave the IAPT service.142
  2. Data from study 2 (see Chapter 4) identified no mortality cases during the IAPT intervention period.

Background all-cause mortality is included in the Markov model and is applied to the recovery health state, based on national UK life tables.148 Individuals who are in the relapse/recurrence health state of the Markov model are at an increased risk of death. The estimate used was from a large case–control observational study of anxiety and depression as predictors of mortality,149 which found a hazard ratio of 1.32 (95% CI1.14 to 1.54) associated with anxiety and depression.

Baseline user characteristics are included in the Markov section of the model as these have an impact on the mortality rates used; a mean age of 42 (SD 13.4) years is used, with 61% of the hypothetical cohort being female (see Chapter 4).

It is important to note that adverse events were not considered in the model for practical reasons. There are difficulties in defining what is an adverse event of psychological therapy and what is an unrelated deterioration in a patient’s health.150 Adverse events in psychological therapy are not well recorded or reported in clinical evaluations.150 In addition, adverse event data were not captured in the IAPT data sets used in this study or in the IAPT data sets that are required for reporting to the HSCIC.151

Utility values

The EQ-5D is the measure of health-related quality of life in adults preferred by NICE.133 The EQ-5D-5L was translated into BSL and validated (see Chapter 6); this study concluded that it is a reliable measure of health status in the Deaf signing population in the UK.130 The utility values used are reported in Chapter 6. The data from the translation and validation study were used to calculate EQ-5D utility values for the depression and remission health states included in the model. The values were calculated by separating the data into two groups based on the CORE-10 thresholds for anxiety and depression using the published English clinical cut-off points.138 The published CORE-10 cut-off points are not specific to the Deaf population but were the only cut-off points that could be applied with the measures collected in this data set.

The utility value for partial recovery could not be identified from the data collected as part of the translation and validation of the EQ-5D-5L BSL (see Chapter 6). To fill this gap, the value was identified from recent work undertaken for NICE guidelines.142,152 For one of the sensitivity analyses, utility values were estimated using the new value set for England136 rather than the crosswalk system. In this case the average of the utility estimates for full recovery and relapse/recurrence states is used as the utility rate for partial remission.

Resource use and costs

Costs include the following:

  • costs associated with IAPT appointments
  • costs associated with BSL-IAPT set up (BSL-IAPT arm only)
  • translation costs (standard IAPT arm only)
  • longer-term health and social care costs associated with depression/anxiety after the first 6 months.

The detailed service use and cost data are shown in Appendix 10, Tables 36 and 37.

The mean number of appointments for a person completing treatment was calculated from the relevant data set (BSL-IAPT or standard IAPT data). This included attended appointments and appointments where the user did not attend (i.e. the user did not cancel, but did not arrive for the appointment), as these appointments could not have been given to other users at short notice and hence the service will still incur the cost.

The unit costs for standard IAPT appointments and an interpreter cost were estimated from published sources.153,154 There is currently no unit cost of an appointment for a national BSL-IAPT service. Accordingly, the unit cost from a published BSL Healthy Minds report was used as the average unit cost.153,155 To inform the cost distribution and recognising that other service configurations may offer BSL-IAPT, the lower quartile cost of standard IAPT was used as the lower bound, with an upper bound set to the upper quartile cost of standard IAPT plus the cost of an interpreter.

The wider costs of relapse/recurrence were estimated from published sources.156,157 A systematic review of studies assessing the cost of illness of depression found that the excess costs of treating depression were about one-third of total health-care costs for people without depression.157 Data could not be found to inform the health-care costs of people recovered from depression; we assumed that costs would be two-thirds of the depression costs reported in the McCrone et al. paper.156 Service users experiencing a partial recovery were assumed to have an average of the cost of relapse/recurrence and full recovery states as data could not be found to inform this.

Costs that were taken from older sources were updated to 2014/15 prices using the Hospital and Community Health Service index.158

Primary analysis

The primary analysis addressed the question of whether or not BSL-IAPT was potentially cost-effective compared with standard IAPT when service-specific PHQ-9 and GAD-7 tools and cut-off points were used to identify people with depression/anxiety and measure recovery. The model estimated the net costs and QALYs of BSL-IAPT compared with standard IAPT. Monte Carlo simulation was used to take account of the fact that there is uncertainty and variation in the data used in the model. To do this, all the data inputs were assigned distributions. The Monte Carlo simulation samples from the distribution of possible values for each data input in the decision model. Probabilities and utilities were assigned beta distributions, and costs were assigned triangular distributions. Odds ratios and resource use were assigned normal distributions. Estimates of the mean and SD were used. This meant that mean costs and QALYs, as well as measures of variance (SD and 95% CI), could be estimated to assess the level of uncertainty in the results due to the data inputs.

The Monte Carlo simulation produced 10,000 pairs of net cost and net outcomes. These were then used to generate cost-effectiveness acceptability curves and the probability that the BSL-IAPT service is cost-effective compared with standard IAPT for Deaf BSL users, as recommended by NICE for health technology appraisals.133 This approach revalues benefits (e.g. QALYs gained) in monetary terms. However, there is no universally agreed monetary value to attach to QALYs. Therefore, the simulated net QALYs were revalued using a range of willingness-to-pay values that a decision-maker may be willing to pay to gain 1 QALY, ranging from £0 to £30,000. This was based on the range of willingness-to-pay values historically used in NICE decisions.159,160 This approach takes into account uncertainty about the amount that decision would be willing to pay to gain 1 additional QALY from BSL-IAPT.

Sensitivity analysis

Sensitivity analyses were conducted to assess whether or not variations in the model structure, time horizons, cost, utilities or outcome probabilities would alter the conclusions of the primary analysis. The costs, effects, ICERs and probability of cost-effectiveness at each threshold were re-estimated for each sensitivity analysis. An overview of the sensitivity analysis (assumptions tested) is provided in Appendix 11.

Results

The results of the primary and sensitivity analyses are presented below, along with estimates of uncertainty attributable to variation in the data used. The results are based on the limited data available. In particular, the model relied on service use and clinical data from the BSL-IAPT and standard IAPT data sets. Accordingly, it may well be that there is additional uncertainty due to the quality of the data and any differences between the demographic and clinical characteristics of the participants in the two types of IAPT service.

Primary analysis

The results of the primary analysis are shown in Table 29, when BSL-IAPT is compared with standard IAPT using the service-/language-specific PHQ-9 and GAD-7 scales and cut-off points for depression/anxiety. This indicates that BSL-IAPT is associated with a net saving of £240 per patient (SD £832 per patient; 95th percentiles –£2303 to £935).

TABLE 29

TABLE 29

Primary analysis cost-effectiveness model outputs, 2015 £

However, the 95th percentiles cross zero, indicating uncertainty about whether BSL-IAPT is associated with a net saving or net cost. The model predicts that BSL IAPT services are associated with a very small gain of 0.001 QALYs. Again, the 95th percentiles cross zero, indicating uncertainty about whether BSL-IAPT is associated with a QALY gain or loss. The cost-effectiveness plane in Figure 30 illustrates this uncertainty. Each dot represents a net cost–QALY pair from the Monte Carlo simulation. The figure shows that most of the points lie below zero, suggesting a net saving and that more of the points are towards the right, suggesting a net gain in QALYs.

FIGURE 30. Cost-effectiveness plane: distribution of net costs and QALY pairs.

FIGURE 30

Cost-effectiveness plane: distribution of net costs and QALY pairs. WTPT, willingness-to-pay threshold.

The incremental cost-effectiveness acceptability analysis suggests that there is a 57% probability that BSL-IAPT is cost-effective if decision-makers are willing to pay £20,000 to gain 1 QALY; this increases slightly to 58% if decision-makers are willing to pay £30,000 per QALY gain. The cost-effectiveness acceptability curve in Figure 31 demonstrates that the probability that BSL-IAPT is cost-effective changes as the amount that decision-makers are willing to pay to gain 1 QALY changes.

FIGURE 31. Cost-effectiveness acceptability plane.

FIGURE 31

Cost-effectiveness acceptability plane. WTPT, willingness-to-pay threshold.

Sensitivity analysis

The full results of the sensitivity analyses are reported in Appendix 12, Table 39. Most of the sensitivity analyses indicated that BSL-IAPT was likely to be cost-effective even if decision-makers were not willing to pay anything to gain 1 additional QALY (probability of being cost-effective was > 50%). There were some exceptions to this. First, if the costs of an interpreter were excluded from the costs of the standard IAPT service, there was a net cost associated with BSL-IAPT of £307 (95th percentiles –£751 to £1155). The net gain in QALYs remained the same as for the primary analysis. The net cost per QALY gained was £256,224, with a 22–26% likelihood that BSL-IAPT could be cost-effective.

Second, the BSL-IAPT service was constrained to providing primarily low-intensity services, whereas standard IAPT services had more capacity to provide high- and low-intensity service. We performed a sensitivity analysis to explore the extent to which the cost-effectiveness of BSL-IAPT may change if it was possible to offer low-intensity services to more people. This increases the costs of BSL-IAPT and reduces the probability that BSL-IAPT is cost-effective to < 50%. It is important to note that this is a very exploratory analysis. What both of the sensitivity analyses demonstrate is that the costs of interpreters for standard IAPT and whether or not BSL-IAPT is able to provide high-intensity care are potentially important factors to consider in future service development and research.

Discussion

Summary of findings

The primary analysis indicated that BSL-IAPT may be more cost-effective than standard IAPT. However, there is uncertainty within the estimates and so this result is not conclusive. As noted in the previous section, limited data and reliance on the IAPT data also mean that the robustness of the results is uncertain. Even if decision-makers are not willing to pay an additional cost to gain 1 QALY, BSL is > 50% likely to be cost-effective. This is when the BSL version of the PHQ-9 and GAD-7 and BSL cut-off points are applied to the BSL-IAPT service to define whether or not a person has depression, reliable recovery and reliable improvement, and the English versions are applied to the standard IAPT service. The sensitivity analysis indicated that applying the same language and cut-off points for these measures to each service did not substantially change the conclusion.

Most of the sensitivity analyses indicated that BSL-IAPT was likely to be cost-effective even if decision-makers were not willing to pay anything to gain 1 additional QALY, with a probability of being cost-effective of > 50%. There were some exceptions to this. First, if the costs of an interpreter were excluded from the costs of the standard IAPT service, the net cost per QALY gained by BSL-IAPT was £256,224, with a 22–26% likelihood that BSL-IAPT could be cost-effective. Second, if the BSL-IAPT was able to offer high-intensity services to more people, the costs of BSL-IAPT increase, reducing the probability that BSL-IAPT is cost-effective to < 50%.

Nevertheless, there were limited data available with which to estimate the different variables in the model, so that there is a high level of variance and uncertainty in the estimates of the costs and QALYs associated with the two services. In addition, although in principle a 50% probability of cost-effectiveness may be interpreted as BSL-IAPT being cost-effective, or similar to standard IAPT, the uncertainty in the data means that the results are best treated as equivocal.

Limitations

Scope and model structure

The objective of this study was to identify whether or not BSL-IAPT is potentially cost-effective compared with standard IAPT services accessed by Deaf people with reasonable adjustments. It does not consider a potential third option (a specialist service for Deaf people with the standard IAPT service on which there are limited data; see Chapter 5). In addition, it does not compare the cost-effectiveness of IAPT services against alternative service configurations and designs.

A number of assumptions were made, and constraints placed on the scope of the analysis, to deal with limited data availability. These increase uncertainty about the robustness of the results. The time horizon was limited to 12 months for the primary analysis, which is relatively short for evaluations of interventions for depression, which is often a long-term condition. The short time frame increases uncertainty about the potential cost-effectiveness of BSL-IAPT. However, the sensitivity analyses, using longer time horizons of up to 10 years, indicate broadly similar results.

The costs and consequences associated with adverse events linked with psychological therapy were not explicitly included in the decision tree part of the model. These data were not included in the IAPT data sets used to estimate variable values for the decision tree. A recent national survey of people who had received psychological therapy found that 5% of people reported lasting bad effects.161 These included the worsening of existing symptoms and the onset of new symptoms. It may be that the costs and consequences of these types of effects are included in the model indirectly through the probability that service users experience partial or full recovery, and the subsequent costs and QALYs associated with that.

Difficult experiences and therapist effects while receiving therapy may also contribute to services users stopping or dropping out of therapy.161,162 Again, these effects are not directly included in the model, which may overestimate the benefits of treatment and underestimate the costs. However, whether either the long-lasting or shorter-term bad effects differ between the BSL and standard IAPT services is unknown. The national survey data161 indicate that the rate of lasting bad effects was similar across all types of psychological therapy, including IAPT services.

The decision section of the model was restricted to treatment provided by IAPT and excludes the cost and QALY consequences of referral to other services in the first 6 months following referral. The BSL and standard IAPT data sets and the wider literature did not include sufficient data about the number and type of referrals to include these in the model. If the rate of referrals to other services differs between BSL and standard IAPT, then this will affect the relative cost-effectiveness of BSL-IAPT. The model also excluded the use of other primary care- and community-based health and social care services, as well as the use of secondary care services. This restricts the scope of the analysis and will underestimate the total costs of each service. However, whether or not excluding these costs from the model will affect the relative difference in the costs of the two services is unclear. There is evidence from evaluations of IAPT service that service users access other primary, community and hospital services.163,164 However, whether or not there are differences between standard and BSL IAPT in the use of these services by Deaf people with depression/anxiety is unknown.

The impact of any differences in waiting times between referral and the start of treatment was not included in the economic model. There were no statistically significant differences in waiting times between the BSL and standard IAPT in the data sets used for our analysis. In this instance the constraint is unlikely to affect the results and conclusions drawn from the analysis. However, waiting times may be important for future analyses if delays affect the probability of reliable recovery or improvement following treatment or the intensity and duration of treatment required. There is a limit to how many BSL-fluent health-care workers are available (whether Deaf or hearing). If tailoring services to meet the needs of Deaf people by employing BSL-fluent workers is cost-effective, then such workers will be in demand. It is plausible that demand will outstrip supply, especially if the services considered are not solely limited to IAPT. This may lengthen waiting times for treatment.

Additionally, in the long term, health-care workers may need to be educated in Deaf awareness and be trained to a fluent level in BSL and/or additional Deaf, BSL-fluent individuals will need to be trained as PWPs, to cope with demand. This would incur additional costs (both time and money), over and above those included in the economic model. Future work may need to consider including the longer-term costs associated with training.

It is possible that having a specialist service for Deaf BSL users will raise awareness of the IAPT service, especially in a group faced with communication barriers in standard services. This raised awareness of the service may increase demand for and attendance at the services, which would lead to greater costs but also the potential for increased health benefits at a population level. It was beyond the scope of this study to incorporate future changes in the demand for and supply of IAPT services in the economic model.

Data

The validity and robustness of the results of this economic evaluation are dependent on the quality and completeness of the data included in the two IAPT data sets. A number of limitations and issues about the data are discussed in Chapter 4. Additional limitations in the range of data used for the economic model are discussed in this section.

The search used to identify existing economic and clinical evidence to inform the economic model was focused and systematic, and pre-specified inclusion and exclusion criteria were applied to identified studies. Time and resource constraints meant that it was not feasible to conduct a comprehensive search of the literature or to have two independent reviewers screen and extract data, and quality appraise each of the articles selected. However, as the objective of this review was to inform the modelling approach and fill in gaps in the data available from the IAPT data sets, rather than to inform full quantitative or qualitative comparison of the literature, this method was judged to be sufficient for purpose.

There is a paucity of available literature on health-care use by the Deaf BSL population. In some instances, evidence for the hearing (or not specified) population had to be used. This increases uncertainty about the longer-term costs and QALYs estimated for both BSL and standard IAPT. Even when data from the general population were used, there were limitations in the data available to estimate some parameters. For example, only one study that reported total health-care costs for people with depression at a national level (not restricted to patients receiving certain treatments) was identified.

The economic model estimates the costs and QALYs for 1 year after a person’s first contact with the service (i.e. the time horizon is 1 year), for the primary analysis. This is because the majority of the data used to inform the model are from short-term studies (in particular the IAPT data which we rely on for key parameters), which means that the analyses of longer time horizons are associated with greater uncertainty. Trials for interventions for depression tend to have shorter time frames, with few studies with a follow-up of > 12 months.165 This may be why past economic evaluation modelling studies have focused on shorter time frames.166

The target population for the economic model was Deaf BSL users with depression and/or anxiety. Originally, it was planned to analyse the diagnosis groups separately (e.g. for people who had depression only). However, the majority of the participants in the data sets had both anxiety and depression (75.9% in standard IAPT and 81.6% in BSL-IAPT). Having both diagnoses is recognised as being common in the literature.143–146 Breaking down the data to estimate parameters for these subgroup analyses would mean relying on data from very small groups. Given that the data were already limited by sample size, we focused on the group as a whole.

There have been wide variations reported in the percentage of people recorded as having experienced a recovery following IAPT treatment across England (18.8–69.4%).147 Variation is likely to be even more marked in the standard IAPT data. Within standard IAPT, Deaf BSL users may see therapists who have a varying knowledge/awareness of the Deaf community and BSL. Our data come from a limited sample (in particular the data for Deaf BSL users accessing standard IAPT services). It is not clear whether this over- or understates the variability of treatment provided and recovery that would be found nationally.

The intensity or step level of service with which a Deaf client might engage could also be influenced by the availability of a specialist provider (e.g. in order to have a Deaf therapist a client might need to engage with the service at step 3 because that is where the Deaf therapist was located when in reality they might require only step 2). It is not possible to identify whether intensity level was determined by availability rather than need in the IAPT data sets used in this study.

The economic modelling work relied heavily on data from Chapter 4 (study 2); these are the only data available for Deaf BSL users accessing either IAPT service. Within the model we have compared the service options and we have tried to make best use of the data we had available to us, assuming that these groups are comparable. However, the results must be interpreted with caution because we cannot fully assess how similar the two populations were at baseline owing to inconsistent data reporting between services. Thus, the results may have been influenced by different population characteristics rather than the service interventions. It is important to note that this limits the robustness of the economic evaluation.

There are also some questions about how comparable the data on the two service options are. For instance, BSL-IAPT is reliant on low-intensity interventions, with few services offering high-intensity treatment. This means that service users for whom high-intensity treatment may be more suitable are restricted to low-intensity treatment. This affects the duration and costs of treatment in the short term and could affect the probability of recovery. The reliance of BSL-IAPT on low-intensity treatment also restricts the possibility that a patient can be moved to a higher-intensity treatment that is appropriate for their needs. In addition, we know little about which treatments were offered within each of the services, and some of the variation in outcomes may be explained by this.

What this study adds to the evidence base

This is the first study to use an economic model to synthesise data from different sources and explore the potential cost-effectiveness of a service that has been specifically adapted to meet the cultural and linguistic needs of the Deaf BSL population in the UK. Although there are several limitations to the structure of the economic model used and the available data, the analysis provides an initial indication of the costs and QALYs of BSL and standard IAPT. The primary analysis indicates that BSL-IAPT may be more cost-effective than standard IAPT. Much of the sensitivity analysis to explore the impact of assumptions and data limitations appears to support this conclusion. The sensitivity analyses identified two key variables that affected the costs of each service and probable cost-effectiveness of BSL-IAPT. The first was the costs of interpreter services to facilitate access to standard IAPT care for Deaf BSL users. The second factor was the intensity of services provided by BSL-IAPT. If BSL-IAPT services are not constrained to providing primarily low-intensity services, in the future this could increase the costs of treatment and may improve the recovery rates above what was found in this study.

Key findings

  • A new cost-effectiveness model, evaluating the potential cost-effectiveness of BSL-IAPT in comparison with standard IAPT for Deaf BSL users, has been developed.
  • The costs of BSL-IAPT over 1 year were estimated to be £2977 compared with £3218 for standard IAPT. This indicated a net saving for BSL-IAPT of £240 (95th percentiles –£2303 to £935). However, the 95th percentiles cross zero, suggesting that the difference in costs may have occurred by chance.
  • The QALYs associated with BSL and standard IAPT were very similar, at 0.719 and 0.717, respectively. Again, the 95th percentiles overlapped, indicating that this small difference may have occurred by chance. The simulation and cost-effectiveness acceptability analysis indicated that, overall, there was a > 50% chance that BSL was cost-effective. Key factors that could change the cost-effectiveness of BSL-IAPT are the costs of interpreter services for standard IAPT and the range of services provided by BSL-IAPT.
  • The model structure and variable estimates were limited by the range and quality of data available. This means that the results are preliminary and uncertain.
Copyright © Queen’s Printer and Controller of HMSO 2017. This work was produced by Young et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Included under terms of UK Non-commercial Government License.

Bookshelf ID: NBK448359
Contents

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (7.2M)

In this Page

Other titles in this collection

Recent Activity

ClearTurn OffTurn On

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...