Quality of life

Quality of life was assessed using the EuroQol-5 Dimensions, three-level version (EQ-5D-3L).⁶⁸ This was collected at baseline and at 1 and 3 months. In some cases, the EQ-5D-3L was completed with the help of (or by) a proxy. At baseline and at 1 month, the EQ-5D-3L was completed face to face with a researcher or health-care professional present; at 3 months it was completed by means of a postal survey.

Health-care resource use

Health-care resource use was captured using a specially designed questionnaire, which was completed by patients (and/or proxies) at 3 months only. The questionnaire asked the respondent to record any primary care (e.g. GP visit or nurse visit) or secondary care (e.g. hospital stay) resource use in the previous 3 months. Unit costs from national sources (e.g. NHS reference costs and the PSSRU report)¹²⁴ were used to cost the resource use (in Great British pounds at 2015 prices). The questionnaire was completed by means of a postal survey. We had information from the case report forms on the initial hospital stay to provide a cross-check with the patient-recalled information. In 336 cases, the total length of inpatient hospital stay reported by patients was shorter than that captured by hospital records for the initial event alone. For the main analysis, we used patient-reported stay, except when this was shorter than the hospital record, in which case we used the latter. In a sensitivity analysis, we calculated outcomes solely using patient-reported stay.

The cost of the POD intervention was also estimated. This included material costs (e.g. printing of manuals), the time to deliver and receive the training and also time to provide support during POD delivery. This information was provided by the POD research team members, who kept a contemporaneous diary of visits and travel.

Feasibility

The feasibility of data collection in this group was determined by establishing the number of missing data (missing questionnaires and missing items in returned questionnaires) and the validity of the assessments used.

For both the EQ-5D-3L and resource use questionnaires, counts (percentages) were produced for the number of missing questionnaires and missing items. Regression analyses were used to determine whether or not individual and clinical characteristics predicted missing data. Questionnaires with a high response rate and low numbers of missing items could be considered acceptable to patients and useful and practicable in a larger trial. Specifying what is an acceptable return and completion rate for questionnaires is difficult as it is likely to be population, time point and completion-mode specific. However, we might expect, at 3 months, the return rate to be around 60–70% and the percentage of missing data to be no more than 5–10% on each item of the completed questionnaires.

The criterion validity of the EQ-5D-3L was explored by correlating values with those from the NEADL⁶² and the discriminant validity was explored by calculating mean values by the groups of interest (i.e. delirium vs. no delirium). If valid in the population, we might expect the EQ-5D-3L to correlate significantly with the NEADL and to distinguish between people who did and people who did not experience delirium (with those experiencing delirium obtaining lower EQ-5D-3L values).

Missing data and baseline imbalance

In the event, a non-trivial number of missing data was observed. We adopted a number of approaches to deal with this and present the cost-effectiveness results for each. We present results based on:

• Complete cases only (only those who completed all questionnaires and items) (n = 138).
• Multiple imputation [multiple imputation was conducted in Stata^® (StataCorp LP, College Station, TX, USA) using information on age, sex, trial arm, cognitive impairment, ward type, delirium at 10 days and existence of comorbidities] of missing item data for respondents with < 50% of the health economics questionnaire items missing (n = 314).
• Multiple imputation (multiple imputation was conducted in Stata using information on age, sex, trial arm, cognitive impairment, ward type, delirium at 10 days and existence of comorbidities) of total NHS costs and/or EQ-5D-3L (only those who had completed at least one EQ-5D-3L had total EQ-5D-3L values imputed) values for respondents with > 50% of the health economics questionnaire responses missing or who had missed the questionnaire entirely (n = 616).

The trial data descriptives suggested that there was some baseline imbalance between trial arms; hence, adjustment was required. QALYs were adjusted using treatment arm, baseline EQ-5D-3L, age, ward type (orthopaedic vs. general), sex and cognitive impairment status (yes vs. no) as controls.

Cost-effectiveness

The primary economic evaluation adopted the NICE-preferred approach of a cost–utility analysis comparing the costs and benefits of POD with those of usual care.⁸ The costs were those relating to health-care use and (for the POD arm only) those relating to the POD intervention. The benefits were measured in terms of survival, which was quality-adjusted using the EQ-5D. The analysis time horizon was 3 months based on the trial follow-up. The main analysis result was the ICER per QALY. ICERs below the range of £20,000–30,000 indicate that POD would be considered cost-effective. Non-parametric bootstrapping was employed to determine the level of sampling uncertainty. Results were presented on cost-effectiveness planes and cost-effectiveness acceptability curves.

It was stated in the economic analysis plan that a multilevel model would be used to analyse the predictors of net monetary benefit and how these vary between hospital sites and wards. An initial multilevel model analysis found sites to be insignificant predictors with extremely small coefficients. Instead, a simple linear regression was employed whereby individual variables and clinical variables, and treatment arm were entered in a regression model to predict net monetary benefit. This approach also permitted the control of baseline differences between arms. An additional cost-effectiveness analysis was conducted that presented cost per case of delirium prevented.

Costs were calculated from the perspective of the health provider and Personal Social Services. A wider cost perspective was also planned. For the trial-based cost-effectiveness analysis, no discounting of costs or effects was conducted.

Early in the research programme grant work (see Appendix 4) and in the absence of observed data, a decision-analytic model was developed to determine the potential for POD to be cost-effective. The model made assumptions about delirium prevalence, POD effectiveness and costs, survival and quality of life. It concluded with a high degree of certainty that POD would be cost-effective. This model was updated to include information from the trial, including POD costs, delirium rates and POD effectiveness, and the estimates of cost-effectiveness were updated. This analysis allowed us to test our previous assumptions, but also to estimate the cost-effectiveness, taking into account a longer time horizon.

Feasibility

Missing data

Table 34 presents completion and missing rates overall and by relevant subgroups. Table 35 presents results of a regression predicting questionnaire completion. Table 36 includes missingness by questionnaire item at 3 months.

TABLE 34

Number of received and complete questionnaires

TABLE 35

Regression predicting missed patient resource questionnaires

TABLE 36

Number missing for each item in questionnaires

The key findings are as follows:

• The EQ-5D-3L return rates were 98.6%, 77.5% and 65.3% at baseline and at 1 and 3 months, respectively; on each occasion, 94–98% of these were fully complete.
• The resource use questionnaire was in the same survey pack as the EQ-5D-3L, so return rates were the same at 3 months. However, completion rates were lower, with only 48.7% (n = 190) fully complete.
• There did not appear to be significant differences in return rates according to treatment arm or delirium status, although a trend for higher return rates in the control arm and among those who did develop delirium was observed.
• A regression analysis predicting survey return at 3 months found no factors that significantly explained return rate. However, individuals with cognitive impairment at baseline were less likely to return the questionnaire than individuals with no cognitive impairment, with the p-value approaching significance (see Table 35).
• Although the returned EQ-5D-3L measures suffered from minimal missing data, some of the resource use items had high rates of missingness (see Table 36). For example, 13.6% of patients missed the question asking whether or not they had had an inpatient stay in the previous 3 months. This is important as inpatient stays represent a high proportion of total costs.
• Twenty-three per cent of individuals did not include the number of hours per week for which they received help from friends or family. Difficulty quantifying average number of hours is common and not necessarily an indicator of poor acceptability of the question.
• It should be noted that mode of completion was not the same across time points. At baseline and 1 month, it was face to face, whereas, at 3 months, it was via postal survey. This is likely to have contributed to the higher number of missing data at 3 months.
• Results suggest that there was a large difference in missing resource use measure rates between the best- and worst-performing centres (results not shown).

Validity of patient outcome assessments

The scatterplot showing the correlation between baseline EQ-5D-3L and NEADL is given in Figure 18. Mean EQ-5D-3L values are provided in Table 37 and scores by delirium status are included in Table 38.

FIGURE 18

Correlation of EQ-5D-3L and NEADL scores at 3 months.

TABLE 37

Mean EQ-5D-3L values of sample at baseline and population norm

TABLE 38

Mean EQ-5D-3L and dimension scores by time point and delirium status

The key findings are as follows:

• A significant, positive correlation existed between EQ-5D-3L and NEADL scores at 3 months (r = 0.66), indicating that they measure similar constructs in this patient group.
• The trial sample had significantly lower EQ-5D-3L scores at baseline than the UK age-matched population norms averages (reported in Kind et al.⁷⁷) (see Table 37).
• Patients who experienced delirium had much lower average baseline EQ-5D-3L scores than those who did not, and this difference was maintained across the three time points (see Table 38).
• This difference appears mainly to have been driven by worse status in terms of self-care and usual activities and, to a lesser extent, mobility (see Table 38). Dimensions relating to pain and mental health were less important.

Responsiveness of patient outcome assessments

The EQ-5D-3L scores across time points are included in Table 38 and Figure 19:

FIGURE 19

The EQ-5D-3L scores over time by delirium status.

• Both those who did not have delirium and those who did at 10 days experienced a similar improvement in health status from baseline to 30 days, followed by a worsening between 30 days and 3 months (although this was more marked in the delirium group).
• Those who had persistent delirium at 30 days did not experience an improvement in health status. It declined from baseline to 30 days and remained almost at the same level up to 3 months. However, it should be noted that the sample was small for this group.

Validity of proxy outcomes assessments

The EQ-5D-3L scores according to whether or not a patient had help from a proxy are included in Table 39:

TABLE 39

Mean EQ-5D-3L scores by proxy status

• At baseline, proxy-completed EQ-5D-3L questionnaire values were similar to the values of those completed by participants themselves.
• At 1 and 3 months, proxy-completed (or aided) EQ-5D-3L completion underestimated quality of life.

Cost-effectiveness

Costs of Prevention of Delirium intervention

The resources used in the delivery of the POD intervention are presented in Table 40:

TABLE 40

Cost of POD intervention

• A total of 2115 patients were screened in POD wards. However, this underestimates the number of patients to whom POD applied, as those aged < 65 years were not screened.
• Given this, a POD cost denominator sample of 3563 was agreed to be the most suitable. This was calculated using the number of beds in POD wards multiplied by the assumed number of admissions over 6 months. The number of admissions was calculated using the number of days in 6 months divided by the average length of stay for a POD participant (10.7 days).
• The total final cost of the POD intervention was estimated to be £39,120. This included printing of the manuals, staff time (for researcher and nurses) to attend introductory meetings, POD facilitators, POD-related team meetings and contact between researchers and the POD staff. Therefore, the per-patient cost was £10.98.

Resource use

The average resource use per completed question is presented in Table 41:

TABLE 41

Mean response to resource use items

• The POD arm participants had higher average resource use for every health-care resource except GP surgery visits and psychiatrists, psychologist or counsellor visits.
• Participants in the POD arm had an average of 2.2 more overnight days in hospital and 1 more day in nursing/residential homes.
• Hospital inpatient stay appears to be driving costs. The mean cost was £4965 in the POD arm and £4365 in the control arm. The mean costs per resource use item are presented in Table 42.
• As data quality on items relating to carer time/costs was poor, and because POD no longer relied on volunteer time, it was decided not to conduct the wider-perspective cost-effectiveness analysis.

TABLE 42

Mean cost of resource use items

Trial-based cost-effectiveness

The EQ-5D-3L score at baseline was slightly higher in the POD arm than in the control arm. To control for this, QALYs were adjusted using age, ward type, sex and cognitive impairment.

The key findings are as follows:

• The ICER resulted in the POD intervention being dominated by standard care. That is, POD resulted in higher costs and lower QALYs. However, the QALY differential was negligible.
• The difference in cost varied from £920 in the complete-case group to £1127 for complete-case and imputed items group. The difference in QALY varied from –0.01 in both imputation groups to –0.02 in the complete-case analysis. Mean cost, mean QALYs and ICER calculations are in Table 43.
• NHS total cost and QALYs were replicated 10,000 times in a Monte Carlo simulation; the simulation is presented in Figures 20–22.
• Using a £20,000 per QALY threshold, the probability that POD was cost-effective was 0.01 (1% chance) in a simulation using adjusted QALYs and complete-case and imputed items. This chance increased to 10% when using unadjusted QALYs and complete-case data only.
• A sensitivity analysis was conducted using hospital inpatient length of stay from the 3-month patient questionnaire solely. The results from this analysis can be found in Table 44. The difference in cost between both arms increases to between £1148 and £1414 depending on sample group used.
• An analysis was conducted using incremental cost and percentage of patients in each arm who experienced delirium to produce the cost per percentage of patients who avoided delirium. The cost percentage reduction in delirium ranges from £657 to £805 depending on the sample used. Detailed results are in Table 45.

TABLE 43

Incremental cost-effectiveness ratio

FIGURE 20

Cost-effectiveness acceptability curve.

FIGURE 22

Complete-case (unadjusted QALY) cost-effectiveness plane.

TABLE 44

Sensitivity analysis: patient questionnaire hospital inpatient days

TABLE 45

Cost per percentage of delirium avoided

FIGURE 21

Complete case and imputed items cost-effectiveness plane.

Net-benefit regression

• Data were analysed in the net benefit regression framework. As we had employed imputation, and this may confound the model results, we opted to run the analysis on complete cases only (n = 138).
• A multilevel model was run on the data predicting net monetary benefit with site and ward entered as levels. However, it was clear that site was not a significant factor, and the ward influence, although important, was related only to whether the ward was general or orthopaedic. For this reason, multilevel modelling was not deemed appropriate for the data and a simple linear regression was employed.
• In the linear regression, treatment arm was not a significant predictor of net monetary benefit. However, sex and delirium status were significant predictors. Female participants experienced significantly higher benefit, whereas participants who experienced delirium had significantly lower net benefit. Results from this analysis are presented in Table 46.

TABLE 46

Regression predicting net monetary benefit

Model-based cost-effectiveness

The original model was updated using information from the trial. There were noticeable differences between some parameter values used in the 2010 modelling exercise and those observed in the feasibility trial data. For example, the model assumed a delirium incidence of 15%, versus the observed incidence of 9.4% in the trial control arm. The effectiveness of POD was also initially overestimated as it was assumed that the intervention would reduce the delirium rate by 33%. In the event, the delirium incidence between arms was 9.4% (control) and 8% (POD). There were also significant differences in quality of life. For example, the modelling assumed that hospitalised patients who went on to experience delirium had a utility of 0.598, but the trial revealed that this was much lower (0.1169). As POD appeared to result in additional resource use (a difference of £419), a sensitivity analysis was run in which this was added to the POD cost. The updated model parameters and assumptions are in Table 47.

TABLE 47

Updated model parameters

The key findings are as follows:

• The lifetime time horizon cost-effectiveness results from the updated model are included in Table 48.
• These show that POD has an incremental cost and QALY of £1775 and 0.11, respectively. This results in an ICER of £16,133, which indicates that POD is cost-effective.
• A sensitivity analysis adding in additional resource for POD (Table 49) yields an ICER of £19,942.
• The probabilistic sensitivity analyses yielded mean incremental costs and QALYs of £1774 and 0.11, respectively, and an ICER of £15,454. The mean incremental net monetary benefit (at λ = £20,000) was £521.90.
• Figure 23 shows the cost-effectiveness acceptability curve and indicates (where λ =£ 20,000) that POD has a 100% chance of being cost-effective.
• Figure 24 is the EVPI across different levels of λ. As uncertainty is low when λ = £20,000, there is a low per-person EVPI.
• However, given the contrasting trial and model results and data quality issues, the results are, in fact, highly uncertain.

TABLE 48

Lifetime modelled cost-effectiveness

TABLE 49

Lifetime modelled cost-effectiveness (sensitivity analysis – additional POD cost)

FIGURE 23

Cost-effectiveness acceptability curve.

FIGURE 24

Expected value of perfect information vs. willingness to pay.

Feasibility: missing data

• The return rate of questionnaires was in the range of what could be expected from an elderly group of people who had been hospitalised.
• The completion rate of the EQ-5D-3L appeared to be acceptable.
• The return rate did not appear to be influenced by trial arm or delirium status, which is encouraging for future studies.
• The completion rate of the resource use measure was much lower and some items were missed by one in five people. The resource use questionnaire would benefit from further refinement to improve response rates and data quality.
• Indeed, it is debatable whether or not self-report measures of resource use based on recall are suitable in this group. Even when items were complete, the accuracy of responses is uncertain. For example, there was a significant mismatch between self-reported hospital length of stay and that captured by hospital records (for the initial stay). Given this, it is recommended that data requests and linkage from the NHS Digital and primary care sources should be pursued in future studies (possibly alongside self-report measures).
• Response rates typically drop off in later trial follow-ups. However, in the current study, it is also likely that the mode of completion played a part, with the 3-month measure being completed by postal survey. This was a factor in the high numbers of missing data. Consequently, greater reliance on imputation is needed, which increases uncertainty in the analysis. Because uncertainty has a cost in economic evaluations, future studies should consider the trade-off between this and research costs. On this basis, it is arguable that greater investment is warranted in data collection (i.e. face-to-face interviews) or alternative strategies are needed (e.g. routine data capture).
• The return rates for the resource use measure also varied significantly across centres, suggesting that return rates could have been improved if best practices were followed.

Feasibility: validity and responsiveness

• There was evidence that the EQ-5D-3L was a valid assessment in this group. It was highly correlated with the NEADL and indicated much lower health status in these patients than in age-specific general population estimates, as we might expect.
• It was notable, however, that those patients who went on to develop delirium had poorer health status to begin with, suggesting that health status was a significant predictor of delirium onset.
• This difference appeared to be driven by functional status, rather than mental health or pain.
• There was evidence that the EQ-5D-3L was responsive to change in health in this group over time, as we observed an increase in status over time. However, there was a suggestion that those who develop (especially persistent) delirium do not recover to the same extent.

Feasibility: proxy completion

• Aside from baseline assessments, there was evidence that proxy-aided completion of health status may diverge from patient reports.
• Patient completion should be sought when possible; when not possible, a systematic approach to proxy data collection should be employed.
• When using proxy reports, some method of calibrating these values with patient values may be needed.

Costs and effects

• The total costs for the delivery of POD were estimated to be £39,120. Clearly, the per-patient cost depends on the number receiving the intervention and will fall over time. In this study, we defined the number in receipt as 3563, which led to a per-patient cost of £10.98.
• We also tested other assumptions, but these had little impact on the overall total costs, as intervention costs were dwarfed by those relating to health-care resource use.
• Health-care use appeared to be greater in the POD arm. It is unclear why this might be. It may relate to greater levels of observation or more intensive care encouraged by POD.
• Among the group of patients who experienced delirium, 27.4% died during the trial period, compared with 6.9% of patients who did not experience delirium. In the control arm, 13% of patients died during the trial period, compared with 16.3% in the POD arm.
• There was a slightly higher EQ-5D-3L mean value in the POD arm at baseline, which meant that adjustment was necessary. There were negligible QALY differences between arms, although, in all analyses, these were in favour of the control arm. This is despite the fact that fewer cases of delirium were detected in the POD arm. It is unclear why. This finding may be a chance occurrence or an artefact of the missing data and imputation, the influence of proxy-aided completion or the fact that lower health status predisposed patients to delirium onset.

Cost-effectiveness: trial analysis

• The POD intervention appeared to lead to fewer cases of delirium, but this did not appear to translate to lower costs or higher QALYs, regardless of the data adjustment, imputation method and Monte Carlo simulation used.
• In the net monetary benefit analysis, sex and experiencing delirium appeared to be significant predictors of benefit, whereas the POD intervention was not.
• Hence, the POD intervention did not appear to represent value for money in the cost–utility framework over a 3-month period.
• However, in the cost-effectiveness framework, the cost per percentage of delirium avoided appeared to be quite low (£657–805), although interpreting this value is difficult.

Cost-effectiveness: regression and decision modelling

• The net benefit regression did not find treatment to be a significant predictor of net monetary benefit over the trial period. In fact, the POD intervention appeared to be associated with less monetary benefit.
• The presence of delirium was associated with a substantial drop in net benefit (of £5969).
• The updated decision model yielded expected costs and benefits, both of which were higher for the POD arm than for the usual care arm. The ICER for the analysis (deterministic and probabilistic) indicated that the POD intervention was, in fact, cost-effective.
• When we are willing to pay £20,000 per QALY gained, the POD intervention was cost-effective in 100% of the Monte Carlo simulations.
• Ordinarily, we can interpret this as meaning that, with certainty, the POD intervention would represent a cost-effective strategy and the benefit of further research (measured here as EVPI) is low.
• However, there are a number of results that lead to doubts over the cost-effectiveness estimates, including the number of missing data and the contrast between trial- and model-based conclusions.
• In the light of this, and all results considered, it is recommended that additional research relating to the POD intervention is conducted.

Contrasting trial and model results

The results of the trial- and model-based analyses were in conflict. It is unclear why this might have occurred, but possible explanations are as follows:

• Different time horizons – the model has a lifetime time horizon and thus captures cost savings and benefits of delirium prevention over a much longer period.
• The modelling assumes a robust and deterministic relationship between delirium and the outcomes of interest to the cost-effectiveness analysis (e.g. length of stay, health-related quality of life and mortality), that is that delirium avoidance has, with certainty, positive effects on these outcomes. This may perhaps give an unrealistically clean result in favour of the POD intervention in the light of positive point estimates for delirium prevention.
• The trial analysis uses these outcome data directly and the relationship between delirium and health outcomes/costs may be weaker than presumed, or not as expected.
• The trial data are also subject to potential bias that results from imbalance, missing data and type (potentially not at random), reliance on proxy reports and noise.
• The incidence in delirium was small and the differential between arms was smaller still (< 2%). It is quite possible that any benefit of the POD intervention, if there was any, was subsumed by variation from the much larger proportion (< 90%) of the sample that did not experience delirium (who would potentially have used significant health-care resource unrelated to delirium).