Methods
Aim of economic analysis
The aim was to determine the cost-effectiveness of delivering a complex primary-care-based intervention, 3D, designed to improve the management of care for multimorbid patients, compared with usual care. Full details of the intervention and the trial population are given in Chapters 3 and 5.
Perspective
The primary economic analysis was from the NHS and PSS perspective. A secondary analysis was conducted from the perspective of patients and carers (including personal travel costs, expenditure on private health care, therapies and over-the-counter medication). The societal cost of time off work to attend health-care appointments was also considered in a separate analysis.
Time horizon
The economic analysis compared the costs and outcomes of each arm over 15 months of follow-up.
Identification of economic outcomes
The primary economic outcome measure was QALYs derived from utility scores, obtained using the EQ-5D-5L health-related quality-of-life instrument.111
Measurement of outcomes
Measurements were recorded at baseline and at 9 and 15 months post recruitment using questionnaires as described in Chapter 4. In the case of non-response, EQ-5D-5L data were also collected by telephone.
Valuation of outcomes
Utility scores were derived from responses to the EQ–5D-5L cross-mapped to valuations obtained for the EQ-5D-3L instrument from a UK population using the methods of van Hout et al.109 This was a change to the planned analysis (approved by the DMC), as NICE issued a position statement110 recommending this approach over the planned use of the English EQ-5D-5L value set prior to the commencement of the analysis. These values were used to form QALYs over the 15-month period by means of linear interpolation and an area under the curve calculation, adjusting for imbalance in baseline utility scores.112 Patients who died were treated as if their last-measured utility score was relevant until the date of death, and immediately set to zero at death.
Identification of relevant resource use
As the trial population had multiple conditions by definition, the scope of the economic evaluation was defined as resource use related to any health condition experienced by the participant. For the NHS and PSS perspective, data were collected on use of health services in primary care (consultations, investigations and prescribed medications) and community care, hospital admissions, outpatient attendances, emergency care, ambulance use, and social care. For the analysis from the patient/carer perspective, data were collected on travel costs to GP appointments, and expenditure on over-the-counter medication and private therapies and treatments. The value of productivity losses was estimated using data on time off work by both patients and carers to attend primary and secondary care appointments.
Practices in the trial were paid £30 for each complete 3D review (including both a GP and nurse consultation) to compensate them for the additional time spent on 3D reviews. This cost was not included within the cost of the intervention because of potential double counting, given that we were including the cost of the extra GP and nurse time for the longer consultations.
Measurement of resource use
Where possible, resource use was measured by programmatic downloads of medical records from GP systems, which was facilitated by the fact that all practices were using the same system (EMIS). These routine downloads were supplemented with data collected via patient-reported questionnaires administered on paper by post at 9 and 15 months’ follow-up and at baseline, and with data extracted from participants’ medical records by trained researchers. Individual data collection methods for each type of resource use are described in more detail below.
Set-up costs
Study records of the number/role of staff attending each training session were used to track resources used in the delivery of the training programmes for GPs, nurses and receptionists, including trainee and trainer time (and preparation time), travel costs and course materials to calculate the fixed cost of training.
Delivery of intervention
Delivery of the 3D GP and nurse appointments was recorded through manual data capture by researchers reviewing participants’ medical records at the end of the trial. Pharmacist reviews were captured through electronic practice downloads.
Health and social care utilisation
Details of the number and duration of primary care consultations were extracted from electronic downloads of routine GP records. These included face-to-face, telephone and home consultations with doctors, nurses or HCAs based in general practice. Duration details were not available for all consultations. Therefore, an average duration for each type of consultation by each staff type in each arm was derived using available data (practices in England only) and applied to all relevant consultations.
Data on medications prescribed and tests/investigations conducted in primary care were also extracted electronically from GP records. NHS secondary care data were collected from participants’ GP records by the research team. NHS community care, care from social services and patient personal resource use during the 15-month follow-up period were captured in the patient-reported questionnaires.
Transport
The patient’s normal transport method for GP appointments and the cost (for public transport) or mileage (for private transport) to use as a multiplier for calculating costs associated with each consultation, were collected in the patient-reported questionnaire at baseline.
Productivity
Time off work by patients and carers to attend hospital appointments was captured in the patient-reported questionnaires at 9 and 15 months. Participants were asked at baseline whether or not they usually took time off work for GP appointments.
Personal expenditure on health care
Expenditure on over-the-counter medication, and private use of treatments and therapies, was captured in the patient-reported questionnaires at 9 and 15 months. Details of whether or not the participant paid prescription charges were requested in the questionnaire at baseline.
Valuation of resource use
Unit costs for NHS staff time for training and delivery of the intervention were based on the most recently available national estimates.113 Actual expenses incurred for training materials, refreshments and staff travel were recorded. Based on the proportion of GPs trained in a practice, the training costs were inflated to estimate the cost of training a full practice, shared among the number of patients eligible for the intervention in that practice and annualised over an estimated 5-year period of relevance. The costs of medications were based on the cost recorded within the EMIS system at the time the medication was prescribed, supplemented by estimates from the British National Formulary where such costs were missing.114 When patients were responsible for paying prescription charges, these amounts were applied to each of the medications recorded in the practice download data and subtracted from the NHS perspective medication costs (leading to negative NHS medication costs for a small number of participants). Community and primary care costs were based on national estimates.113
Codes for Healthcare Resource Groups (HRGs), groups of events that have been judged to consume similar levels of resources, were assigned to secondary care contacts and costed based on the most recently published national reference costs available.115 Productivity costs were estimated based on average weekly earnings stratified by age group.116 Mileage costs were estimated using UK government allowances.117 Over-the-counter medication costs and costs arising from private therapies and treatments were all reported directly by patients. Unit costs used in the analyses are detailed in Appendix 19 (see Table 42).
All costs were reported in 2015/16 GB pounds, adjusted for inflation where necessary. Costs and outcomes occurring during the final 3 months of follow-up were discounted in line with NICE guidance (currently 3.5%).118 Dates were not available for all types of resource use measured in the trial; in these cases, 50% of the costs incurred in the final 6 months of follow-up were subjected to discounting.
The cost of each resource item was calculated by multiplying the number of resource units used by the unit cost. The total cost for each individual patient was then estimated as the sum of the cost of resource-use items consumed. These resource use data combined with unit costs were used to estimate the incremental cost or savings of the 3D approach. The results are reported in accordance with the specifications of the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.119 Changes from the health economic analysis plan drawn up in advance of the analysis are described in Appendix 20.
Economic analyses
All analyses were conducted by treatment allocated, comparing the two groups as randomised and including all patients in the primary analysis. A cost–utility analysis was conducted from the NHS and PSS perspective corresponding to the NICE reference case.118 The costs of each component of the intervention were estimated separately from each perspective and related to changes in a range of secondary outcomes in a cost–consequences analysis.120 Statistical analyses were conducted using Stata 14.2.96
Data cleaning and missing costs and outcomes
Data cleaning was undertaken prior to unblinding by the economic researcher. Data cleaning included the correction of obvious ‘free text’ response errors (e.g. misspelt health professional titles), group coding of similar resource items (e.g. ‘orthopaedics’ and ‘trauma & orthopaedics’ clinics) to enable unit costing, and simple imputation of data missing minor details (e.g. bus fares) based on reasonable assumptions (e.g. mean bus fare). Any areas of uncertainty were discussed between two health economists and, when necessary, referred for adjudication by a clinical expert. Questionnaires were not classed as ‘missing data’ for the cost analysis unless the questionnaire was not returned or the majority of responses were uninterpretable. Medication costs downloaded from GP practice notes were manually amended if they were clearly wrong (e.g. a prescription for a salbutamol inhaler with a recorded cost of > £1000).
The primary analysis included all participants using imputation to predict missing costs and outcomes.121 Data imputed using chained equation multiple imputation methods for the main statistical analysis were used (see Chapter 4 Methods, General considerations, Missing data).122,123 To facilitate convergence of the imputation model, costs were imputed using aggregated cost categories (medications, pharmacy reviews, secondary care, primary care, social care and other types of care) rather than at the level of individual resource-use items.
Analysis of costs and outcomes
The incremental mean difference in QALYs between the two arms of the trial and 95% CIs were derived. Overall mean NHS and PSS costs and standard errors for both arms of the trial were calculated. The incremental mean difference in total costs between the two arms of the trial and 95% CIs were estimated.
Relative costs and outcomes
Cost and QALY data were combined to calculate an incremental cost-effectiveness ratio (ICER) and net monetary benefit (NMB) statistic124 from the NHS and PSS perspective.
In the primary analysis it was estimated whether or not the 3D approach was cost-effective at the established NICE thresholds of £20,000 and £30,000 per QALY gained. The probability that the 3D approach was cost-effective at various societal ‘willingness to pay for a QALY’ thresholds was depicted using a cost-effectiveness acceptability curve (CEAC). All measures of cost-effectiveness (ICER, CEAC and NMB) and CIs were derived parametrically using the output of seemingly unrelated regression analysis to account for the correlation between costs and outcomes, and controlling for baseline imbalance in utility for the QALY equation. Clustering within GP practices was accounted for by including the randomisation variables in the regression.
Both costs and consequences were collated into a cost–consequences matrix presented from the NHS and PSS perspective, the patient/carer perspective and the societal productivity perspective for each arm. Consequences included QALYs accrued by both participants and carers, and deaths. The cost–consequences analysis was based on available cases, which differed in number for each type of health-care resource or outcome; an available case was defined as an individual having complete data for each relevant time point. Linear regression output was used to derive CIs parametrically, accounting for clustering within practices.
Sensitivity analyses
One-way sensitivity analyses were used to judge the potential impact of sources of uncertainty, including a complete case analysis to assess the impact of the imputation process, an analysis excluding participants who died to assess the impact of the imbalance in deaths between arms and an analysis without discounting either costs or outcomes to assess the impact of the discount rate. A complete case was defined as a participant for whom full resource-use data and full outcome data were available.
Results
Missing data
Of all participants, 797 were randomised to be offered the 3D approach, and 749 were randomised to receive usual care. Missing data occurred for a number of reasons, including withdrawal from the trial or leaving the participating practice. Twelve participants (1.5%) in the 3D arm and six (0.8%) in the usual-care arm had no information on secondary care use because it was not possible to locate their medical records (p = 0.2). Practice downloads of medication and investigation data failed for 18 participants (2.3%) in the intervention arm and eight (1.1%) in the usual-care arm (p = 0.07), and 19 (2.4%) and 10 (1.3%) participants were missing consultation data from practice downloads in the 3D and usual-care arms, respectively (p = 0.13). Inevitably, not all participants returned all questionnaires at all time points; 165 participants (20.7%) in the intervention arm and 125 (16.7%) in the usual-care arm did not return a questionnaire at one or more of the follow-up points (p = 0.04). Not all those who did return questionnaires completed the resource-use questions; in total, 181 (22.7%) in the intervention arm and 146 (19.5%) in the usual-care arm were missing resource-use data from questionnaires at one or more follow-up points (p = 0.12). Complete data sets were available for 1191 participants (599 (75.2%) in the 3D arm and 592 (79%) in the usual-care arm, p = 0.07). Participants with missing data were in a significantly poorer health state at baseline [mean EQ-5D-5L score: 0.453 (95% CI 0.422 to 0.485)] than participants with full data sets [mean EQ-5D-5L score: 0.589 (95% CI 0.574 to 0.605)].
Primary analysis
Outcomes and resource use
The primary analysis using imputed data showed that participants in the intervention arm gained a mean of 0.007 additional QALYs over the 15 months of the trial compared with participants in the usual-care arm (95% CI –0.009 to 0.023). Total costs from the NHS and PSS perspective were £126 (95% CI –£739 to £991) higher in the intervention arm than in the usual-care arm. Disaggregated resource-use data are presented in Appendix 21, Table 43.
Cost-effectiveness of 3D
Cost-effectiveness statistics from the NHS and PSS perspective are given in . The ICER was £18,499, and the NMB at a societal willingness-to-pay value of £20,000 was £10 (95% CI –£956 to £977). At this willingness-to-pay value, the probability that the 3D approach is cost-effective was 0.508, and at £30,000, the probability of cost-effectiveness was 0.558. A CEAC depicting the probability of cost-effectiveness at a range of willingness-to-pay values is shown in .
Cost-effectiveness of the 3D approach from a NHS and PSS perspective
Cost-effectiveness acceptability curve from the NHS and PSS perspective.
The CEAC is relatively flat, because of the similarity between the trial arms in estimates of both costs and effects, with considerable uncertainty around both parameters. Therefore, the probability that the intervention is more cost-effective than usual care is between 40% and 60% at any cost-effectiveness threshold between £10,000 and £40,000. We further consider the interpretation of the economic analysis below.
Sensitivity analyses
Results from an analysis restricted to complete cases only are given in . In contrast to the primary analysis, the complete-case analysis suggested that the 3D approach was dominant (i.e. the intervention was associated with both lower costs and better outcomes), with a probability of cost-effectiveness of 0.705 at a willingness-to-pay threshold of £20,000. A sensitivity analysis excluding participants who died suggested that the probability of cost-effectiveness of the 3D approach at £20,000 was 0.561. A further sensitivity analysis using undiscounted costs and outcomes did not suggest that the discount rate affected the conclusions.
Sensitivity analysis: cost-effectiveness of the 3D approach from a NHS and PSS perspective based on complete cases only
Cost–consequences analysis
Costs and selected outcomes (on an available case basis) are presented in from the primary perspective of the NHS and PSS and the secondary perspective of the patient/carer themselves alongside an estimate of the societal loss of productivity.
Costs and consequences of the 3D approach and usual care
Costs from all perspectives were very similar between arms and no cost group differed significantly (other than those associated with the intervention itself). Other than for day-case/outpatient care, emergency care and medications, costs to the NHS were higher in the intervention arm than in the usual-care arm, and social services usage was higher in the usual-care arm. Overall costs from the NHS and PSS perspective were slightly higher in the usual-care arm although, again, the difference was consistent with chance.
Costs borne by patients and carers were higher overall in the intervention arm, although the medication costs (both prescription charges and over-the-counter remedies) were slightly lower; no patient cost group exhibited a statistically significant difference. The societal cost of productivity losses was similar in the two arms (and statistically consistent with chance), although slightly higher in the 3D approach arm.
Quality-adjusted life-years (adjusted for baseline utility scores) were slightly higher for patients and lower for carers in the intervention arm than in the usual-care arm; however, the difference was consistent with chance. Although there was a higher number of deaths in the intervention arm, the difference was not statistically significant.
All costs and consequences are based on available data; the totals from each perspective are not, therefore, equal to the sum of the components. CI were calculated using standard errors from standard linear regressions adjusted for cluster at the level of the practice. QALYs were adjusted for baseline utility scores.
Discussion
No consequential difference was observed between arms for overall costs, resource use of any category or QALY outcomes. We concluded that the 3D intervention was unlikely to be either more or less cost-effective than usual care in the primary analysis from the NHS and PSS perspective. The NMB was very small, but positive, indicating that the costs associated with the intervention are less than society is willing to pay for the benefits that can be achieved.
From the NHS and PSS perspective, costs were slightly higher in the intervention arm in the primary analysis based on a full imputed data set, but were slightly lower in the intervention arm when complete cases were examined. This suggests that the participants with missing data were higher users of health and social care than responding participants, and this is consistent with the fact that complete cases had higher utility at baseline than those with missing data. Although the complete-case analysis suggested that the 3D approach was dominant (i.e. it provided higher gains at lower cost than usual care), the results should be treated with caution given the substantial uncertainty, and the likelihood that this represents a biased sample of healthier participants.
The 3D participants had a mean utility of 0.558 (SD 0.287) at entry to the study, which compares poorly to a UK population norm of 0.779 for ages 65–74 years.125 As a result of this, the participants were substantial users of health care, with inpatient hospital care and medications both high contributors to overall costs. Participants had a small positive increase in QALYs in the intervention arm, and carers for these participants had a small decrease in QALYs compared with those in the usual-care arm; it is possible that an analysis that took into account carer outcomes might reach an alternative conclusion. The small contribution to overall costs made by productivity losses is consistent with the predominantly retired study population; at baseline, > 65% of participants described themselves as ‘fully retired from work’.
The set-up costs for training the staff involved in delivering the intervention were small, varying from £1.70 per patient (in the least costly practice) to £8.71 per patient. It was estimated that the training received by 3D practitioners would be relevant for 5 years; however, it is possible that skill sharing might take the place of formal training if the intervention were rolled out. The number of patients to benefit from the training is also probably an underestimate, as new patients would join the practice and existing patients would become eligible for the intervention over the years. It is, therefore, likely that the set-up costs are slightly overestimated. The software template used to manage the 3D approach was developed using trial funding, and would not incur ongoing costs to the NHS as the supplier would incorporate it into the basic product. However, development costs would be incurred for the intervention to be implemented in other software systems. It was not possible to identify 3D appointments reliably through the practice record downloads; these appointments are, therefore, aggregated with all other practice-based appointments.
A crude estimate of the budget impact of implementing the intervention in England can be made using the trial results. England had a population of 43.5 million adults in 2016.126 A total of 3.5% of adults in the trial practices were both eligible for the 3D approach, and were considered suitable by their GP, suggesting an eligible population of 1.5 million people. At an incremental cost of £126 over 15 months, the intervention could be estimated to cost approximately £154M per year. However, given the uncertainty around the cost estimate, it is possible that the intervention could be associated with a saving of £900M per year, or a cost of as much as £1.2B per year.
Strengths
This economic evaluation was conducted alongside the largest RCT of its kind. Meticulous data collection practices allowed individual patient data to be measured for all the key cost drivers. The study contributes to the growing body of evidence supporting the care of patients with multiple long-term health conditions. Patterns of missing resource-use data were similar between arms, and high questionnaire return rates were achieved.127 Although imputed and complete-case analyses suggested different conclusions, the results were consistent with the minimal differences in costs and outcomes observed between arms, and the substantial uncertainty surrounding the results.
Limitations
Medication costs were based on scripts issued by the GP, and it is not certain that all scripts were filled by the participant; the medication costs may, therefore, be overestimated. In addition, a number of errors were identified in the medication costs downloaded from GP practice notes; although both high and low outliers were checked carefully and manually corrected as necessary, the volume of medications prescribed to the 3D population rendered it infeasible to check every entry, and it is possible that some errors persisted.
Use of care homes was not included in the economic evaluation. This can be a significant contributor to costs of social care; however, the funding of care homes within the UK is complex, with patients often paying considerable amounts themselves. The follow-up period of the 3D trial was only 15 months and longer-term outcomes are unknown. However, given the lack of any difference between arms in the utility values measured at 15 months post recruitment, it is unlikely that the conclusions would change substantially. Use of simple mean imputation methods for estimating missing information (such as the cost of a bus fare) in the questionnaire data will have reduced standard errors and underestimated the uncertainty around these costs.
Quality-adjusted life-year outcomes for participants who died were based on an immediate reduction from the previous known EQ-5D-5L value to zero at death. Although this will be accurate for some participants who died suddenly after living previously in a consistent health state, it is likely that some patients would have undergone a decline in health-related quality of life while approaching death. The QALYs may, therefore, be slightly overstated. Dates of completion of the EQ-5D-5L instrument were based on the recruitment date and expected follow-up dates and were not always the same as the date on which the patient actually filled in the questionnaire.
We used the NICE threshold of £20,000 to assess cost-effectiveness, as is conventional with most economic analyses of health-care interventions conducted in the UK. However, this threshold is largely arbitrary and has been controversial for many years.128 Some economists have argued that the threshold should be lower because implementing new interventions at a cost of £20,000 per QALY could displace other existing interventions that are more cost-effective.129 Conversely, other economists have argued that the threshold should be higher,130 or that there is insufficient consensus to justify a change from the £20,000 threshold.131 Studies of the social value of a QALY suggest a range between £18,000 and £40,000 (but with wide variations depending on the methods used and the population studied).128 These discussions about thresholds are less relevant to this particular case, because the costs and effects were so similar in both arms of the trial that it is unlikely that the intervention is either more or less cost-effective than usual care within a cost-effectiveness threshold in the range between £10,000 and £40,000.
