Abstract
Objective
To estimate the cost-effectiveness of the SRT intervention, compared with ESC, a within-trial cost–utility analysis was undertaken.
Methods
A within-trial analysis was conducted, where costs and benefits were estimated over a 24-month period and discounted at 3.5% in the second year. Costs were estimated from the perspective of the NHS and Personal Social Services (PSS), at 2017/18 levels, and included intervention (training, therapy session and supervision) costs and other health professional and hospital admission costs. Total quality-adjusted life-year (QALY) scores were estimated from the EQ-5D-3L responses. Regression was undertaken to estimate the incremental cost-effectiveness ratio (ICER) (mean incremental cost/mean QALY gain), where value for money would correspond to an ICER below the cost-effectiveness threshold (λ) value of £20,000 per QALY. The level of uncertainty, according to the cost-effectiveness acceptability curve (CEAC), was also assessed at the same λ value. MI was used in the base-case analysis.
Results
Based on 270 patients, the mean incremental cost for SRT, compared with ESC, was estimated to be £3910.59 (95% CI £2708.32 to £5112.86), with a QALY gain of 0.001 (95% CI –0.099 to 0.10). Accordingly, the ICER was estimated to be > £5M, with a low level of uncertainty.
Conclusions
Social recovery therapy was estimated to be significantly more costly than ESC and there was no significant difference in outcome, and a low level of uncertainty. Consequently, SRT was not estimated to constitute value for money.
Background
Background
The protocol for this study, with associated rationale, inclusion criteria, setting and intervention descriptions, has been published.1 In this chapter, we report the methods and results for the economic evaluation component of the study.
Objectives
In order to estimate the cost-effectiveness of the SRT intervention, compared with ESC, a within-trial (24-month) cost–utility analysis was undertaken.
Methods
Estimating costs
Costs were estimated from the perspective of the NHS and PSS at 2017/18 levels. Below, the methods used to measure and value the intervention costs and other NHS and PSS costs are outlined.
Intervention (social recovery therapy) cost
Training
All therapists who delivered the intervention in the study received training. For costing purposes, informed by our knowledge of who delivered the intervention in the study, the therapists were assumed, on average, to be NHS band 7, and thereby have a cost per hour of employment of £53.2179 (). Those who delivered the training were assumed to have a cost per hour of employment of £63.00 (grade 8a).79 It was assumed that each therapist received 3 days of training (excluding research-related activities), with an average of four therapists receiving the training at one time, and the associated preparation/travel costs were assumed to be negligible. Total training costs (for both the trainer and therapists) were subsequently estimated, summed and equally apportioned across all intervention participants. The assumption to apportion costs to the trial participants only was in line with previous research,83 and was undertaken as we would rather be conservative and not underestimate such costs.
Estimated unit costs: most commonly reported (information sources)
Therapy sessions
To inform the assumptions about cost of intervention delivery, discussions took place with two therapists who delivered the intervention (one face-to-face meeting, followed by a number of e-mail discussions). In addition, therapists were asked to record in the database the duration of all face-to-face contacts. Details of other non-contact patient-related activities (e.g. e-mails and telephone calls) could also be recorded. After examining these data and following discussions with the therapists, it was concluded that these data were likely to be under-reported and that some therapists were better at recording non-contact activities than others. Previously, it had also been estimated that for every hour of face-to-face contact time there was an associated hour of non-contact time.80 Two therapists were asked whether or not they thought that this reflected their practice. They thought that the most appropriate assumption was that for every hour of face-to-face contact time there would be an associated hour of non-contact activity undertaken by the therapist for the same participant (example activities would include session reminders/bookings, liaison with other agencies and session preparation/planning). Travel time was not included in the 1 : 1 contact to non-contact time assumption and it was assumed that travel time would approximate to 30 minutes of therapist time for each face-to-face session. The associated travel costs were therefore assumed to be £6.00, on the assumption that the average associated mileage was 15 miles (the distance covered in 30 minutes of travelling at 30 miles per hour) at a cost of 40p per mile.
Supervision
Peer supervision was assumed to take place weekly (one-to-one with another therapist for 1 hour in total, including preparation) over the period when therapists were providing therapy. After estimating the total peer supervision cost for the study, this was equally apportioned across all intervention participants. Finally, the mean costs representing the cost of training, therapy provision and supervision per therapy participant were summed to estimate the mean total intervention cost (per intervention participant).
Other NHS and Personal Social Services costs
A self-reported modified version of the Client Services Receipt Inventory (CSRI84) was developed, which asked participants to report any contacts with health professionals (number and place) and hospital admissions (length of stay and type of ward/unit). These methods are in keeping with a previous study39 which used a modified self-report version of the CSRI in a similar population group. Participants were asked to complete the modified CSRI at baseline (recall: previous 6 months), 9 months (recall: previous 9 months), 15 months (recall: previous 6 months) and 24 months (recall: previous 6 months); post-baseline participants were asked not to include therapy received as part of the SRT intervention (this was to enable the researchers who helped with questionnaire completion to remain blind and because this information was routinely recorded by those who provided the intervention). Control participants did not receive any specific intervention and the modified CSRI was designed to capture the standard care that they received.
Costs were assigned to each reported item of NHS and PSS resource use, with these being estimated at 2017/18 financial year levels. Subsequently, the total intervention cost (see Supervision) was added to the health professional and hospital admission costs to estimate the overall NHS and PSS costs.
Measuring outcomes
To estimate levels of health-related quality of life, participants were asked to complete the EQ-5D-3L67 at baseline, and at 9, 15 and 24 months. Use of the EQ-5D was justified on the basis that this is in keeping with NICE’s Guide to the Methods of Technology Appraisal 2013,85 that it has been used before in similar population groups,39 and that such use is considered to be appropriate.86 Responses were then converted into a utility score (a scale where 0 is equal to death and 1 is full health) using the York A1 tariff.81 The total area under the curve method (based on the assumption of linear interpolation)82 was then used to estimate the total QALY score for each participant over the 24-month period.
Missing data assumptions
Across all time points, five participants reported that they had contact with a particular health professional, but they did not report the number of visits. Where this was the case, the average value for those who reported the number of visits was used. Similarly, four participants reported that they had had a hospital admission, but they did not report the associated length of stay, and, again, the average value for those who reported such data was used.
In the internal pilot (n = 100), the wrong resource use questionnaire was sent to participants at the 9-month follow-up point. The baseline questionnaire was sent rather than the 9-month questionnaire, and participants were, therefore, asked about resource use in the previous 6 months rather than in the previous 9 months. To estimate costs for the missing 3-month window post baseline, reported levels of resource use for these 100 participants were inflated by 50% in this period. All other missing data were left as such and the corresponding individuals were excluded from the complete-case analysis.
Analysis
Three analyses were undertaken, where the following approaches were adopted in all analyses. In each, a within-trial analysis was conducted in which costs and benefits were estimated over a 24-month period, with costs incurred in the second year discounted at 3.5%.85 A within-trial analysis, rather than a decision model, was undertaken as we are not aware of any previous studies that have compared SRT with ESC. The ITT approach was also followed, in which participants were analysed within the group to which they were allocated, regardless of, for example, the number of therapy sessions received. To estimate the mean incremental cost and incremental effect (QALY gain) associated with the intervention, compared with that for the control group, seemingly unrelated regression87 was undertaken to allow for any correlation between costs and effects. Baseline demographic variables [age (16–19 years, 20–25 years) and sex (male/female)] were included as covariates, along with the total baseline health professional and hospital admission cost for the overall NHS and PSS cost, and the baseline EQ-5D scores for the total QALY score. In the absence of dominance (where higher costs and lower benefits were associated with a particular intervention), the incremental cost and incremental effect would be used to estimate the ICER (mean incremental cost/mean QALY gain).85 It was also assumed that an estimated ICER below the cost-effectiveness threshold (λ) value of £20,000 per QALY85 would suggest that an intervention constituted value for money. In addition, to estimate the level of uncertainty associated with the decision about whether or not the intervention was cost-effective, the bootstrap technique88 (with 5000 replications) was used to estimate the probability that each intervention was cost-effective according to the CEAC.89 In particular, the probability of SRT being cost-effective was specifically estimated at the λ of £20,000 per QALY.
The first (base-case) analysis undertaken was based on overall NHS and PSS cost, and the total QALY score, where MI90 was used to estimate any missing data and enable all participants to be included. The MI model included costs (health professional and hospital admission costs at baseline, and at 9, 15 and 24 months, as well as total intervention costs) and outcomes (EQ-5D scores at baseline, and at 9, 15 and 24 months) and demographic variables [age, sex and social functioning (low functioning/very low functioning)]. EQ-5D scores were included, rather than individual dimension scores, as missing EQ-5D data were generally for the whole questionnaire. Health professional and hospital admission costs were, however, separated to allow for the possibility of different levels of missing data for these two variables. Two further complete-case91 sensitivity analyses92 were also conducted. In the first sensitivity analysis (SA1), participants were included only if they had complete cost and effect data at each time point. This enabled a comparison with the results of the base-case analysis, to assess whether or not results differed for participants who did not have missing data values imputed. A second sensitivity analysis (SA2) was also conducted, for a similar rationale, where only total intervention costs were included, as the level of missing data for this variable was anticipated to be lower.
Results
Costs
Social recovery therapy intervention cost
Training
A total of 19 therapists delivered the SRT intervention, where five group training sessions (each 3 days long) were assumed to be held. Total training costs were estimated to be £29,833.80, equating to a cost of £216.19 per participant (see ).
Social recovery therapy intervention costs
Therapy sessions
A total of 2470 face-to-face SRT sessions were recorded across the 138 intervention participants (mean number of sessions per participant = 17.9). The mean recorded contact time per session was 63 minutes [this value was assigned to the six sessions (< 1%) where no time was recorded]. The cost of 1 hour of face-to-face contact time was estimated to be £106.42 (this includes the cost of an associated 1 hour of non-contact time). Travel costs were estimated to be £32.60 per session. Together, this meant that the mean per participant total session cost was £2571.56.
Supervision
At any one time, it was assumed that, on average over the whole study period, two SRT therapists in each of the three sites would be providing therapy and that peer supervision would, therefore, take place weekly for 1 hour between these two individuals. This was assumed to have taken place over a 5-year period in two of the sites and over a 2-year period in the third site. Over the study period, a total of 624 peer supervision sessions were thereby assumed to have taken place. At a cost of £106.42 per session, this would equate to a total cost of £66,404.68 (£481.19 per participant).
After summing the aforementioned mean cost per intervention participant training, therapy session and supervision, it was estimated that the mean total intervention cost (per SRT intervention participant) was £3268.94 ().
Other NHS and Personal Social Services costs
Participant response rates for the modified CSRI at baseline, and at 9, 15 and 24 months, were 270 (100%), 231 (86%), 221 (82%) and 189 (70%). Data for the participants who completed the health professional visit/hospital admission questionnaires are presented for the two groups in . It can be seen that, in contrast to the aforementioned intervention costs, there was comparatively little difference in mean health professional resource use/costs between the two arms. Mean hospital admission costs were seemingly higher in the intervention arm, but this difference was sensitive to a small number of participants [e.g. if data for the two participants with the longest length of stay (59 and 98 days) were not included, then the mean costs would be higher in the control arm].
Levels of resource use and associated costs for most commonly reported items of resource use for the 2-year follow-up period
Finally, overall NHS and PSS costs are presented in , where it can be seen that the difference in costs between groups is comparable to the cost of the intervention itself.
Outcomes
The number of participants who completed the EQ-5D at baseline, and at 9, 15 and 24 months is shown in . It can be seen that, based on those who responded, the mean score improved for both groups over time and the total QALY score was similar in both groups.
Estimated EQ-5D scores at baseline, and at 9-, 15-, and 24-month follow-up points
Analysis
The results of the regression analyses are shown in . In the base case, it can be seen that the overall NHS and PSS costs were, on average, £3910.59 higher for SRT participants than for ESC participants (p < 0.01). The total QALY score was, however, not significantly different between groups. Therefore, the estimated ICER exceeded the λ value of £20,000 per QALY, at which level SRT would not be deemed cost-effective or to constitute value for money. Furthermore, according to the CEAC, it was estimated that there was a low level of uncertainty associated with that result. shows that similar results were obtained in both sensitivity analyses that were conducted.
Estimates of incremental cost, incremental effect and level of cost-effectiveness in the base-case and sensitivity analyses
Changes from protocol
The protocol1 stated that a cost-effectiveness analysis would be carried out using activity (time use) and symptoms (CAARMS); this was not conducted for the following reasons. The results of the primary analysis show that there was no evidence of any superiority of SRT over ESC for levels of structured activity at 15 months (see Table 15). Similarly, there was no evidence of any superiority of SRT over ESC in levels of psychopathology as assessed by CAARMS symptom severity scores (see Table 17). In addition to there being no statistically significant differences between groups, the numerical differences did not favour the SRT group. SRT is also more costly than ESC (see ). Given these results, which show that SRT is more costly and no more effective than ESC, we considered that the proposed cost-effectiveness analysis would be of limited value.
