Economic analyses based on clinical trials

Economic analyses based on observational data

Patient population

As described in earlier chapters, the patient population in the REFLUX trial was patients with GORD whose symptoms required medication for reasonable control and for whom either surgery or continued medical management appeared to be an acceptable treatment option. A policy of offering relatively early laparoscopic fundoplication was compared with the alternative policy of continued medical management. The analysis used data only from the randomised trial component of the REFLUX trial (i.e. not from the preference groups). As described in Chapter 3, 357 patients were randomised to either surgical treatment (n = 178) or medical management (n = 179) and patients were followed for up to 5 years.

Health-care resource use

Health-care resource-use data were collected prospectively as part of the clinical report forms and patient questionnaires at 3 and 12 months and 2, 3, 4 and 5 years. Patient questionnaires at 3 and 12 months collected information for the previous 3 and 9 months respectively. In addition, a questionnaire at 12 months recorded resource use for the whole of the first year (see following section on costs). Patient questionnaires from the second year onwards collected information for the previous 12 months on hospital admissions (day and overnight admissions) and GP visits, and data on medication for the previous 2 weeks. Clinical report forms collected data on surgery and perioperative complications of surgery.

Costs

The cost for each individual patient in the trial was calculated by multiplying his or her use of health-care resources by the associated unit costs (Table 31). Discount was applied from year 2. Unit costs were all sourced from published data (see Table 31). Total costs include the costs of surgery, GP visits, hospital admissions and medication. Incremental costs (laparoscopic fundoplication vs medical management) for each year and per category of resource use, according to ITT allocation, were calculated using OLS regression.

TABLE 31

Unit costs employed to calculate the costs of reflux-related health-care use

The questionnaires asked for details of anti-reflux medication taken in the previous 2 weeks: name, dose and number of tablets/capsules. The cost of anti-reflux medication during these 2 weeks was calculated by multiplying the prices published in the Drug Tariff for December 2010⁶¹ for each medicine by the number of tablets taken. Yearly medication costs are calculated using the area under the curve method,⁶² which assumes linear interpolation between follow-up points. The costs of reflux-related inpatient, outpatient and day-case visits were derived from the NHS Reference Costs 2009–10,⁶³ in which the relevant codes were weighted by activity level.

For the base-case analysis, total costs included the costs of surgery, complications due to surgery, reoperations, reflux-related prescribed medication, reflux-related visits to and from the GP and reflux-related hospital inpatient, outpatient and day visits. For the sensitivity analysis, all GP visits and all hospital admissions are included in the calculation of total costs (see Incremental analysis for more details on sensitivity analysis). Costs of hospital admissions and GP visits were obtained by multiplying the relevant unit costs by the numbers of admissions and visits reported by the patients respectively. Patients themselves classified how many visits and admissions were reflux related in relation to the total number of visits. There is a possibility that patients may not have fully understood the clinical consequences of GORD; hence, they may misclassify the reason for a consultation. If such misclassification is different across treatment groups, estimates of incremental costs may be biased.

For the first year of the trial, data on resource use were collected at 3 months and 12 months, and for the whole year using an additional questionnaire. To make the most efficient use of the data available for the first year of the trial, resource use at 1 year was estimated as the greater of the area under the curve between the first and second questionnaire and the 12-month health-care survey. This is in line with the procedure employed for the earlier publication evaluating the REFLUX trial.¹

The cost of surgery included the costs of (1) presurgical procedures (endoscopy, pH monitoring and manometry), (2) the surgery team, (3) operative complications, (4) hospital stay, (5) capital costs and overheads and (6) consumables. The cost of reoperations was assumed to be equivalent to the mean cost of the first surgery. The cost of reflux-related visits to and from the GP was assumed to be equivalent to the average cost of visits to and from the GP.⁶⁴

Quality-adjusted life-years

Health outcomes were expressed in terms of QALYs. HRQoL was assessed in the REFLUX trial at baseline and 3 months and then yearly until 5 years using the EQ-5D.^65,66 The EQ-5D is a standardised and validated generic instrument for the measurement of HRQoL. It has five dimensions: mobility, ability to self-care, ability to undertake usual activities, pain and discomfort, and anxiety and depression. Each dimension has three possible responses (no problems, moderate problems or severe problems), creating 245 mutually exclusive health states. Each of these health states has been valued in a large UK population study using the time trade-off method, in which 1 corresponds to perfect health (thus the maximum value possible) and 0 corresponds to death.^65,66

QALYs for each patient were calculated as the area under the curve following the trapezium rule,⁶⁷ which assumes linear interpolation between follow-up points. Incremental mean QALYs between treatment groups were estimated with and without adjustment for baseline utility, using OLS regression.

Discounting

Costs and outcomes from year 2 were discounted using a 3.5% annual discount rate, in line with current guidelines.^65,68

Missing data and multiple imputation

Given the extent of missing data, the multiple imputed data set is presented as the base case. This was created using all available data and multiple imputation with chained equations.⁶⁹ Mean imputation was used to predict missing data at baseline,⁷⁰ as randomisation should ensure equal distribution of potentially confounding variables. Complete-case analysis refers to only those patients who returned all questionnaires and completed all EQ-5D profiles.

Missing or inconsistent answers to questions on resource use were dealt with as follows. For medication use, patients were asked at each follow-up questionnaire whether or not they were using prescribed medication for reflux and, if so, to indicate the name, strength and the number of tablets taken in the past 2 weeks. It was evident from preliminary analyses that the answers to the first question were not necessarily consistent with the answers to the second question. Therefore, the following rule was applied for the costing of drugs: (1) if the patient provided the name, strength and number of tablets taken, he/she was assumed to be taking medication; (2) if the patient did not specify either a drug or the number of tablets taken, he/she was considered not to be taking medication; (3) if the patient specified a particular drug but no dosage, the missing data were imputed as the median of all other patients on that medication. Similarly, missing answers to the questions regarding GP visits and hospital admissions were assumed to indicate that no visits or admissions occurred. Because of the nature of the questionnaire, it is reasonable to assume that absence of an answer indicates no use of services.

Multiple imputation⁷¹ was the statistical technique chosen to deal with missing cost and HRQoL data because of non-returned questionnaires and incomplete EQ-5D profiles, using the user-defined programme ‘ice’ in Stata 11.1. Multiple imputation presents three major advantages over standard ad hoc methods for dealing with missing data (such as mean imputation and last value carried forward): (1) it makes full use of all of the available data, (2) it incorporates uncertainty associated with the missing data and (3) it ensures unbiased estimates and standard errors as long as data are MAR.⁶⁹ [Little and Rubin⁷² defined three missing data mechanisms: (1) MCAR if the probability of data being unobserved is independent of both observed and unobserved values; (2) MAR if the probability of data being unobserved is dependent on the observed values but independent of unobserved ones and (3) missing not at random (MNAR) if the probability of data being unobserved is dependent on unobserved values.]

Multiple imputation follows three steps. First, regression models are used to predict plausible values for the missing observations from the observed values. A random component is included to reflect the uncertainty around the predictions. These values are then used to fill in the gaps in the data set. This process is repeated m number of times (m being the number of imputations), creating m number of imputed data sets. Second, each data set is analysed independently using complete-case methods. Third, the estimates obtained from each imputed data set are combined to generate mean estimates of costs and QALYs, variances and CIs using Rubin's rules,⁷³ in such a way that the uncertainty around the predicted values is fully taken into account.^69,74 Because the REFLUX trial has missing data for both costs and EQ-5D scores, multiple imputation using chained equations (MICE) was employed. For MICE, each variable is predicted with its own regression model. Each imputed data set is created by running the regression models over several cycles, in which each variable informs the prediction of the other variables.^69,74 To obtain overall estimates of mean and incremental costs and QALYs across all of the imputed data sets, the ‘mim’ command was used.⁷⁵ Semi-parametric bootstrapping in Stata 11.1 was employed to estimate the probability that surgery is cost-effective, while maintaining the correlation between costs and QALYs (see Incremental analysis for more details).⁷⁶

Plausible prediction of the missing data depends on the appropriate specification of the regression models used in MICE.⁷⁴ If a model is misspecified, the distribution of imputed values may not resemble that of the observed values, and thus the estimates of treatment effect may be biased.⁶⁹ The regression model specified will depend on the type and distribution of the variable to be predicted.⁷⁰ The variables required for the economic evaluation are costs for each year and EQ-5D scores at each time point. Both are continuous variables and neither is normally distributed; EQ-5D scores in the REFLUX trial are bounded between −0.594 and 1,⁶⁶ and costs are bounded at zero and tend to present a positive skew. Two approaches to deal with non-normality with MICE have been suggested in the literature:⁶⁹ (1) transformation towards normality and (2) predictive mean matching. [In predictive mean matching the missing observation is imputed with an observed value from an individual with a similar linear predictor.⁷⁰ Consequently, the distribution of imputed values tends to closely match the distribution of the observed values.⁶⁹] Using the REFLUX data set none of the transformation approaches (Box–Cox,⁷⁷ log-transformation and log-transformation of non-zero values with generation of an indicator variable⁷⁸) were successful in transforming the data distribution to normality. As a result, predictive mean matching was the strategy employed to ensure that the distribution of imputed values closely resembled the distribution of observed values. All known covariates thought to be associated with the missingness mechanism, costs and EQ-5D scores were included in the prediction equations: EQ-5D scores at each follow-up point, costs at each year, allocation, BMI, age and sex. A total of 100 imputations (m = 100) was used to ensure efficient and reproducible estimates.⁶⁹

Multiple imputation provides unbiased estimates of treatment effect if data are MAR. Whether or not data are MAR is an untestable assumption by definition, as unobserved values are unknown. Departure from the MAR assumption may have implications for decision-making if the results from the cost-effectiveness analysis differ from those of the base case. Sensitivity analysis was used to test the impact on the cost-effectiveness results if data were MNAR, that is, if patients with worst outcomes or greater costs were more likely to have missing data.^70,79 Four scenarios were tested. In scenario (1), all patients with missing data had their total QALYs reduced by 10%, 20%, 30%, 40% and 50%. Conversely, in scenario (2), for all patients with missing data costs were increased by the same proportions (10%, 20%, 30%, 40% and 50%). In scenario (3), only surgery patients with missing data had their QALYs reduced. In scenario (4), costs were increased only for patients undergoing surgery.

Incremental analysis

The cost-effectiveness of surgery was evaluated by comparing the costs and QALYs incurred in the surgery arm with the costs and QALYs in the medical management arm at 5 years of follow-up, using conventional decision rules and estimating ICERs as appropriate.⁸⁰ If one intervention is associated with greater mean QALYs and lower mean costs it is deemed cost-effective by dominance. The ICER is calculated if either treatment arm does not dominate. The ICER summarises the additional costs associated with one intervention over another and relates this to the additional benefits. This ICER is then compared with a threshold for the cost per QALY. The National Institute for Health and Care Excellence (NICE) uses a threshold cost per QALY of around £20,000–30,000 to determine whether or not an intervention represents good value for money in the NHS.⁶⁵ Consequently, if the ICER is < £20,000, laparoscopic fundoplication could be considered potentially cost-effective. ICERs between £20,000 and £30,000 per QALY are considered borderline and an ICER > £30,000 is not typically considered cost-effective.

The ICER can be re-expressed using the net monetary benefit (NMB). The NMB of an intervention is the value of the health benefits gained from a particular intervention compared with standard care in monetary terms, minus the incremental costs of the intervention. The translation of health benefits into the monetary scale was made using a cost-effectiveness threshold of £20,000. This is the threshold commonly use by NICE (this corresponds to 1 QALY being valued at £20,000). Therefore, the NMB provides a measure of the gain (or loss) in resources of investing in a particular intervention when those resources could have been used elsewhere.⁸¹ The NMB of laparoscopic fundoplication and medical management were calculated and used to demonstrate the influence of trial duration on the estimates of cost-effectiveness of surgery.

As discussed previously, the multiple imputed data set was used as the base case for the cost-effectiveness analysis because of the large proportion of data lost for the complete-case analysis. Because total costs and total QALYs are cumulative quantities, any missing data at any of the follow-up points will result in that patient being dropped from a complete-case analysis. The cost-effectiveness results using the complete case are presented for comparison. Complete-case analysis will provide unbiased estimates only if the data are MCAR, that is, the probability of data being unobserved is independent of both observed and unobserved values. Multiple imputation ensures unbiased estimates if the data are MAR (the probability of data being unobserved is dependent on the observed values but independent of unobserved ones). Because unobserved values are unknown, the missing data mechanism and hence the validity of either assumption is untestable. Nevertheless, multiple imputation presents two advantages. First, it requires a less stringent assumption for ensuring unbiased estimates. Second, if data are MCAR, both complete-case and multiple imputation estimates will be unbiased whereas, if data are MAR, complete-case analysis will be biased.

Analysis of uncertainty for incremental analysis

Sensitivity analysis is used to explore and quantify any uncertainty in the cost-effectiveness results. Three types of sensitivity analysis were undertaken: structural, scenario and probabilistic sensitivity analysis. Structural and scenario sensitivity analyses were carried out on the complete-case data set. Probabilistic sensitivity analysis was carried out in both the complete case and the multiple imputation data set.

Structural sensitivity analysis consisted of a PP analysis that classified patients according to treatment compliance at 1 year of follow-up, that is, whose management at 1 year was consistent with their original random allocation. Consequently, the PP data set consisted of the patients randomised to surgery who actually had surgery, and of the patients randomised to medical management who did not undergo surgery at 1 year. Patients randomised to medical management who had surgery might differ from those randomised to medical management who were managed medically without surgery, for several reasons. A patient's condition might have worsened, prompting surgery, or patients might have changed their preferences and wish to be taken off medication. The latter implies that, had they been screened for the study at the point in time when they had surgery, they would not have been eligible for the study. These patients would have had a preference and would not have accepted randomisation. The condition itself is complex because of its recurrent and cyclical nature (patients suffering from reflux have punctual exacerbations, which can lead them to change their preferences and request surgery). Therefore, the reasons for not complying with randomisation are likely to be a combination of the two motives (worsening of condition and change in preference). PP was chosen because it was thought to be more similar to clinical practice, where patients can experience a wait for surgery and change their preferences during this period. Any switching of treatment after 1 year is assumed to be because of a change in clinical status, which would preclude inclusion in the clinical trial.

The base-case analysis included only the costs of reflux-related GP visits and hospitalisations. Two alternative costing scenarios were tested in sensitivity analysis: including either all GP visits or all hospital use, regardless of whether they had been classified as reflux or non-reflux related.

Probabilistic sensitivity analysis attempts to quantify the joint effect of uncertainty around the costs and QALYs. Semiparametric bootstrapping was used to estimate the probability that each intervention is cost-effective for a range of cost-effectiveness threshold values. In bootstrapping, the original data are sampled with replacement to create a new data set, in order to calculate estimates of treatment effect. Repeating this process a large number of times results in a vector of replicated statistics, which ultimately provide an empirical estimate of the CIs around mean incremental costs and QALYs. The probability of an intervention being the most cost-effective is the conventional method of presenting the uncertainty around the cost-effectiveness results. The CIs around the ICER are not presented because they are difficult to interpret and are not easy to use: a negative ICER can indicate that an intervention dominates (because it is associated with more benefits and lower costs than its comparator) or it is dominated (because it is associated with fewer benefits and higher costs).⁷⁶

Validation

Several procedures were used to ensure the validity of the analysis. First, two statistical analysis codes (written in Stata) were developed in parallel and their results compared. Second, the code was developed by one analyst and checked independently by another. Third, the results were cross-checked in Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) for a sample of the data set. Lastly, selected results were represented graphically and examined for face validity. The validity of the imputation strategy was explored by (1) analysing the data for predictors of missingness,⁷⁰ (2) comparing the distributions of the observed and imputed values graphically⁷⁰ and (3) estimation of Monte Carlo errors.⁶⁹Appendix 8 describes the validation process in more detail.

Patient population

Complete-case analysis consisted of the patients who returned all questionnaires and completed all EQ-5D profiles. Overall, there are 172 patients in the complete-case analysis (88 randomised to medical management and 84 randomised to surgery). Table 32 shows the numbers of questionnaires returned (includes those with some missing data) and the numbers of completed questionnaires returned for each year. As expected, the number of questionnaires returned in each year of follow-up decreases with time. The return of questionnaires does not follow a monotonic pattern, that is, patients who did not return the questionnaire for one particular year may have returned a questionnaire in subsequent years. Therefore, the number of patients in the complete-case analysis is lower than the number of completed questionnaires in year 5. The large number of patients not included in the complete-case analysis because of missing data strengthens the rationale for using the multiple imputation data sets in the base case.

TABLE 32

Numbers of questionnaires returned and completed questionnaires returned and corresponding proportions per trial arm, according to ITT analysis

Health-care resource use

Table 33 summarises yearly health-care resource use in the two trial arms according to ITT analysis. During the first year of the trial, 111 patients randomised to surgery and 10 patients randomised to medical management underwent laparoscopic fundoplication. The 111 patients who were randomised to and received surgery constituted the surgery group in the PP analysis. The 169 patients who were randomised to medical management and did not undergo surgery during the first year of follow-up constituted the medical management group in the PP analysis. In the subsequent years of follow-up there were 15 patients who underwent surgery (one patient who had been randomised to surgery and 14 patients who had been randomised to medical management). These patients are included in the overnight hospital admissions category. Patients randomised to medical management reported more hospital and GP visits than the surgery patients over the 5 years of follow-up.

TABLE 33

Health-care resource use per year per trial arm, according to ITT analysis

Table 34 shows the costs associated with health-care use according to ITT analysis for all available cases (see Appendix 9 for corresponding table for PP). All available cases uses data from all questionnaires returned at each time point. Per annum costs and costs per category refer to all available data, that is, to all participants who returned the questionnaire for that particular year or for that particular category. Therefore, the sum of the costs per category is different from the sum of the costs per annum. Similarly, total costs for complete-case analysis do not correspond to the sum of the costs per category or to the sum of the costs per annum because complete case is a subset of all available data because of the non-monotone missing data pattern. Total costs for complete-case analysis refer to the patients who returned all questionnaires and completed all EQ-5D profiles (84 surgery patients and 88 medical management patients).

TABLE 34

Costs associated with resource use for all available cases, discounted from year 2 at 3.5%, according to ITT analysis

Patients randomised to medical management accumulate lower costs than patients randomised to surgery. Table 34 indicates that surgery patients accrued a large proportion of the total costs in the first year, and accumulated lower costs during the remaining 4-year follow-up than the medical management group. In contrast, the costs accrued by medical management patients are evenly distributed across the duration of the trial. These results suggest that the cost trend in medical management patients is steeper than in surgery patients; hence, that cumulative costs in medical management patients tend to increase at a greater rate than in surgery patients. Costs associated with surgery were the major cost driver for the surgery group. Costs associated with reflux-related medication were significantly greater for the medical management group than for the surgery group. Costs associated with admissions to hospital and GP visits were not statistically significantly different between the two groups. Surgery during years 2–5 is accounted for in the overnight hospital admissions. There were a few crossovers from medical management to surgery from year 2; hence, the difference in costs associated with overnight hospital admissions between the two treatment groups is small. These results suggest that patients undergoing surgery in subsequent years are not a major cost driver in determining the cost-effectiveness of surgery.

Quality-adjusted life-years

Table 35 summarises the EQ-5D scores reported at each follow-up point for all available cases (see Appendix 9 for the corresponding table for PP). All available cases uses data from all questionnaires returned at each time point. The surgery group appears to have better HRQoL than the medical management group, despite starting from a lower baseline EQ-5D on average (0.7201 in the medical management group and 0.7107 in the surgery group). The difference in HRQoL between the two treatment groups decreased with time. This may be due to patients randomised to medical management undergoing surgery throughout the follow-up period and/or to diminishing treatment effect over time.

TABLE 35

Health-related quality of life (EQ-5D) for all available cases according to ITT analysis

Comparison of costs and quality-adjusted life-years between multiple imputation and complete case

Table 36 shows the comparison of the total costs per year between the complete-case data set and the multiple imputation results. Complete case includes only those participants who returned all questionnaires and fully completed the EQ-5D questionnaires. The similarity of both the means and the CIs provides some reassurance of the validity of the multiple imputation model. The distribution of costs and EQ-5D scores in the imputed data sets matches reasonably well the distribution of the original data (see Appendix 8 for details). Furthermore, the Monte Carlo errors are < 15% of the coefficient and CI estimates, suggesting that 100 imputations are sufficient to ensure reproducibility and statistical efficiency.⁶⁹

TABLE 36

Comparison between the complete-case and multiple imputation data sets for costs and HRQoL, according to ITT allocation

For both the complete-case and multiple imputation data sets, the participants randomised to laparoscopic fundoplication accrued greater costs but also reported greater HRQoL than participants randomised to continued medical management. The 95% CI for mean incremental QALYs crosses zero for the unadjusted for baseline estimates, whereas it remains above zero for the adjusted values. This result reflects the baseline imbalance in mean utility between treatment groups. Therefore, these results strongly indicated that surgery is associated with a greater QALY improvement than medical management. The sum of the differences in EQ-5D for the ITT groups does not correspond to the incremental mean QALYs because of the effect of discounting.

Cost-effectiveness

The results of the incremental analysis suggest that laparoscopic fundoplication is a cost-effective strategy for GORD patients eligible for the REFLUX trial (Table 37). The results for the complete-case analysis concur with those for the multiple imputation data set; across adjusted and unadjusted ICER for baseline EQ-5D, ICERs range between £5468 and £8410, well below conventional cost-effectiveness thresholds of £20,000 and £30,000 per additional QALY. For both data sets (complete case and multiple imputation), the probability of surgery being the more cost-effective intervention is > 0.82 for incremental analysis unadjusted for baseline EQ-5D and > 0.93 once incremental QALYs are adjusted for baseline EQ-5D. In the ITT analysis the ICER is higher for the multiple imputed data than for the complete case if QALYs are adjusted for baseline EQ-5D, but lower if QALYs are unadjusted. This might reflect the effect of having baseline EQ-5D in the prediction model, which would preclude the need for adjustment.

TABLE 37

Incremental analysis for the ITT analysis at 5 years of follow-up for the complete-case and multiple imputation data sets

Figure 20 shows how the NMB associated with laparoscopic fundoplication increases with the duration of the trial. This reflects the increase in costs associated with the medical group, which offsets the initial investment made in laparoscopic fundoplication in the surgery group.

FIGURE 20

Net monetary benefit (incremental QALYs × £20,000 per QALY – incremental costs) over the duration of the REFLUX trial for the multiple imputation and complete-case data sets (QALYs adjusted by baseline EQ-5D).

Structural sensitivity analysis: per-protocol status for the complete case

Structural sensitivity analysis consisted of PP status at 1 year for the complete case. In the PP analysis patients are classified according to the treatment actually received at 1 year of follow-up. The PP group consists of 111 patients who were randomised to surgery and who actually had surgery during the first year of the trial and 169 patients who were randomised to medical management and who did not undergo surgery during this time period. However, complete-case data exist only for 84 medical management patients and 66 laparoscopic fundoplication patients. Appendix 9 presents detailed results for costs and HRQoL according to PP analysis. As expected, patients who actually had surgery have higher costs than patients who did not undergo surgery, regardless of their randomisation. Table 38 summarises the incremental results of the PP analysis. Similar to the ITT analysis, the surgical policy is likely to be cost-effective at conventional (NICE) thresholds for cost-effectiveness. The incremental costs are higher and the incremental QALYs lower for the PP analysis (for surgery compared with medical management) than for the ITT analysis if no adjustment is made for baseline imbalances in EQ-5D. Therefore, the ICER is also greater (surgery is less cost-effective than suggested by the ITT analysis). Once total QALYs are adjusted for baseline EQ-5D, however, the incremental mean QALYs increase substantially and the ICER is reduced. Nevertheless, the adjusted ICER in the ITT analysis is lower than that in the PP analysis by around £2000.

TABLE 38

Incremental analysis for the PP analysis at 5 years of follow-up for the complete-case data set

Scenario sensitivity analysis: all general practitioner and all hospital costs for complete case

The results of the scenario analyses strengthen the case for the surgical policy (Table 39). For scenario 1, replacing reflux-related GP costs by all GP costs, the ICER increased slightly in relation to the base case. Nevertheless, the ICER remains well below conventional thresholds and the probability of surgery being cost-effective is > 0.83, for both adjusted and unadjusted analyses. In scenario 2, replacing reflux-related hospital costs by all hospital costs, medical management was ‘dominated’ by the surgical policy because of this intervention being associated with greater benefits in terms of QALYs and lower costs. For this scenario the probability of surgery being cost-effective was > 0.93.

TABLE 39

Incremental analysis for the scenario sensitivity analysis at 5 years of follow-up for the complete-case data set

Sensitivity analysis for the multiple imputation model: departure from missing at random assumption

The multiple imputation procedure assumes that the individuals who completed and returned all questionnaires are similar to the individuals who did not, conditional on their observed characteristics (MAR assumption).^69,79 However, this may not be the case: patients who did not return a questionnaire may have experienced worse HRQoL and accrued higher health service costs, or vice versa. Sensitivity analysis on the multiple imputation model tested how sensitive the cost-effectiveness results are to the MAR assumption. Figure 21 represents the change in NMB adjusted for baseline EQ-5D as costs and QALYs are varied in patients with missing data. The origin, marked as ‘base case’, refers to the incremental results from the multiple imputed data set (ICER = £7028 per additional QALY). The right quadrant plots NMB after increasing the total costs in steps of 10% for patients for whom there was missing data, for both treatment groups and for surgery-allocated patients. The left quadrant plots NMB after decreasing total QALYs in similar fashion. Positive values for NMB indicate that surgery is cost-effective; negative values indicate that surgery is not cost-effective for a threshold of £20,000 per additional QALY.

FIGURE 21

Net monetary benefit (incremental QALYs adjusted for baseline EQ-5D × £20,000 per QALY – incremental costs) over variation in total costs and total QALYs in the multiple imputed data set.

The cost-effectiveness of surgery is relatively insensitive to any increase in costs; the NMB changes little if costs are increased for patients with missing data in both treatment groups and if costs are increased just for surgery-allocated patients with missing data. A similar result is observed for the reduction in total QALYs for all patients with missing data. In contrast, the cost-effectiveness of surgery is highly sensitive if it is assumed that surgery-allocated patients with missing data experience lower HRQoL than patients with complete data. A 10% decrease in QALYs for patients randomised to surgery with missing data results in NMB decreasing to negative values. This scenario shows that missing data can have an impact on the results under certain conditions. It is impossible to empirically confirm or refute the scenario from the data in the trial, but it could be considered an extreme case. It seems improbable in practice that surgical patients with poor quality of life are less likely to respond to follow-up questionnaires than similar participants undergoing medical management.

Overview

The aim of this analysis was to estimate the difference in costs and the difference in HRQoL (measured with the EQ-5D) between the surgical and medical management randomised groups and describe how this difference evolves over time. A simple way of doing this would be to estimate the difference in costs and outcomes at each time point independently. The results of this analysis were shown in Table 34 (for costs) and Table 35 (for EQ-5D). These showed that costs were greater in the surgical group in the first year but greater in the medical group thereafter. EQ-5D tended to be higher in the surgical group in years 4 and 5 but the CIs crossed zero. There are two main limitations of this simple analysis:

1. The outcomes at each time point are unlikely to be independent. If the outcomes at one time point are correlated with those at other time points this analysis may lead to biased estimates of standard errors.
2. The analysis does not take account of missing data. If missing data are not MCAR then this analysis may lead to biased estimates of the mean of the coefficients.

The multiple imputation accounts for the correlation of responses from the same individuals and for the missing data (see Table 36). However, the validity of this analysis depends on the correct specification of the equations used to impute the missing data. Moreover, other regression-based methods are available for handling missing data in longitudinal studies, principally mixed models, and results may be sensitive to the methods used. This section uses a mixed model to handle the missing data and compares predicted outcomes with those using multiple imputation.

Mixed models

A mixed model is a regression-based method for handling continuous data that is measured at more than one time point during follow-up. It allows estimation of treatment effects under the assumption that the data are MAR, that is, dropout may depend on intermediate values. Analysing each time period separately assumes that dropouts are MCAR, a stronger assumption. A mixed model uses all of the observed data. Individuals who dropped out after providing intermediate data contribute to the estimation of the final outcomes. This analysis has the same aims as multiple imputation but uses a different method and with different assumptions. Therefore, it can also be viewed as a sensitivity analysis to test the robustness of the multiple imputation.

The mixed model can be written as:

Where for an individual i,

Y_i = α +βR_i+X_i+ e_i, e_i ∼ MVN(0,Σ)

R_i = randomised group

Y_i = vector of all outcomes (at times 1…T)

X_i = vector of covariates

The variance-covariance matrix Σ is unstructured, that is, no prior assumptions are made about the values of the correlations. Separate models are fitted for costs and for EQ-5D. Baseline values of the EQ-5D are included as an ‘outcome’ (i.e. at t = 1). Dummies representing time points 1 to T were included as covariates X_i. Treatment effects are included as time^* randomised group interactions although no treatment effect at baseline is allowed. No other covariates are included in the model.

Costs

Figure 22 shows the difference in costs (excluding initial surgery) in years 1–5. Mean costs are greater in the medical management arm of the trial after the first year and the CIs only just cross zero. These results are very similar to those of Table 34.

FIGURE 22

Difference in costs (£) excluding initial surgery (mean, 95% CI).

European Quality of Life-5 Dimensions

Figure 23 shows the difference in EQ-5D at 3 months and in years 1–5. Mean HRQoL tends to be greater in the surgical group during the trial, although the CIs cross zero in some periods. These results are very similar to those of Table 35.

FIGURE 23

Difference in EQ-5D, adjusted for baseline (mean, 95% CI).