Design

A multicentre, pragmatic, single-blind, two-treatment arm (iCST vs. TAU), randomised, controlled, clinical trial was conducted over 26 weeks. Participants were randomised to the two groups using a dynamic adaptive randomisation stratified for site and whether or not the person with dementia was taking acetylcholinesterase inhibitors at baseline. Data collection was at baseline, 13 weeks and 26 weeks after completion of the intervention. The primary outcomes were assessed at both time points, with the primary hypothesis examining outcomes at 26 weeks.

Ethics approval

A protocol was submitted for ethical review to the East London 3 Research Ethics Committee (REC) (reference number 10/H0701/71) in January 2010, with provisional approval being granted in July 2010. The following information was identified by the Committee as needing further clarification and amendment:

• Participant Information Sheet to be modified to cover video-recordings and information about the control/usual care group and minor editing changes (i.e. language use)
• consideration of a cross-over or add-on scheme and whether or not participants will be given a copy of a DVD and manual
• provision of additional information about the interviews in both consent and participant forms and further clarification over whether or not recruitment posters and leaflets would be used.

Final approval was granted in September 2010. Participating centres obtained approval from the appropriate local REC and relevant NHS research and development departments.

Intervention and control conditions

Participants randomised to the intervention group received iCST at their own home. The control condition was TAU, with participants in this group receiving no additional intervention. The services and interventions available to people with dementia and family carers randomised to receive TAU varied between and within the iCST centres and may have changed over time. We recorded the use of drugs and services across the two groups and any changes that occurred. In general, services offered to the TAU group were also available to those in the active treatment group condition.

It is very unlikely that any comparable (or even any other) individual cognitive stimulation intervention for the person with dementia would have been available, as these types of therapies are generally unavailable in the UK. We followed standard best-practice methods around pragmatic trials involving an intervention group compared with usual care. Outside the iCST intervention, both groups, in general, would have access to the same kinds of mentally stimulating activities. It is possible that some participants in the TAU group may have engaged in some form of mentally stimulating activities in day-centres; however, this is unlikely to have been as structured as iCST. We asked sites to note instances in which the person with dementia may have been engaged in cognitive stimulation groups by their local services. Those participants who have engaged in such activities during the 3 months prior to recruitment were considered to be ineligible.

Study population

Eight centres in England and Wales were involved in the study: London, Bangor, Hull, Manchester, Norfolk and Suffolk, Dorset, Lincolnshire and Devon (covering Devon North and Devon South), which comprised 12 recruitment sites in total. Researchers in three centres (London, Bangor and Manchester) were based in universities, whereas those in Hull, Norfolk and Suffolk, Dorset, Lincolnshire and Devon were based in NHS mental health services (Table 2). Recruitment commenced in April 2012 and was completed in July 2013.

TABLE 2

Centres in the iCST trial

Eligibility criteria

Sample size

The main analysis was based on intention to treat for the primary outcome of cognition [Alzheimer’s Disease Assessment Scale – Cognitive Subscale (ADAS-Cog)]. Our group CST study⁶ had an effect size of 0.32. Our Cochrane review of RO²⁶ found a standardised mean difference (SMD) of 0.58, whereas the individual RO/CST study¹² found a SMD of 0.41. Taking a conservative approach, we estimated the SMD relative to TAU to be 0.35. A sample size of 260 will have to yield 80% power to detect a SMD of 0.35 using a two-group t-test with a 0.05 (two-sided) significance level comparing the iCST and the TAU groups. Assuming 15% attrition, we originally proposed to recruit 306 people with dementia. During the course of the trial the observed attrition rate was nearer to 25% than the 15% accounted for in the sample size calculation and, therefore, the target recruitment was revised to account for this. The final recruited sample size was 356.

Recruitment procedures

In each iCST centre, people with dementia and their family carers were recruited through mental health services for older people, such as Memory Clinics and Community Mental Health Teams, through dementia care professionals, including psychiatrists, and through local voluntary sector organisations, such as the Alzheimer’s Society (see www.alzheimers.org.uk). The centres in London, Bangor and Manchester were supported by clinical studies officers accessed through the National Institute for Social Care and Health Research Clinical Research Centre in Wales and the Dementias and Neurodegenerative Disease Research Network (DeNDRoN) in England. In Hull and the East Riding of Yorkshire, all patients and carers referred with dementia (and their general practitioners who currently have additional DeNDRoN support to assist with recruitment to dementia trials) were automatically provided with ‘opt-in information’ on current NHS portfolio studies in dementia care, via a centralised clinical academic unit, The Hull Memory Clinical Resource Centre.

The aim of the project was briefly described to potential participants by members of the research and clinical team, and permission for them to be contacted by local researchers was obtained prior to further contact. Research assistants discussed the project and provided full details to participants, answered any questions related to the project and, if participants agreed, undertook informed consent.

Informed consent

Participants enrolled to the study only after providing informed consent in line with guidelines set by the Mental Capacity Act 2005.²⁷ Participants were in the mild to moderate stages of dementia and were therefore expected to be competent to give informed consent for participation, provided that appropriate care was taken in explaining the research and sufficient time was allowed for them to reach a decision. It was helpful for a family member to be involved, and we aimed to ensure that this was done wherever possible. Both people with dementia and family carers were informed that no disadvantage would accrue if they chose not to participate, and all participants were provided with at least 24 hours to review information about the study prior to making a decision. In seeking consent, we followed current guidance from the British Psychological Society²⁸ on the evaluation of capacity. In this context, consent is regarded as a continual process rather than a one-off decision, and willingness to continue participating was continually checked through discussion with participants during the assessments. If, at any point, the person with dementia or family carer became uncomfortable with the assessments, these were discontinued.

Ethical arrangements

The study was approved through the appropriate REC. All researchers received training in Good Clinical Practice guidelines.²⁹ There appear to be no documented harmful side effects from participating in CST interventions or other types of cognitive-based interventions. Regular monitoring by, and support from, the key local unblinded researchers in each centre was undertaken during the intervention to ensure that people with dementia participating in the iCST sessions did not feel deskilled or undervalued.

Prospective participants were fully informed of the potential risks and benefits of the project. A reporting procedure was put in place to ensure that serious adverse events (SAEs) were reported to the Chief Investigator (see Appendix 8). On becoming aware of an adverse event involving people with dementia or their carers, a member of the research team assessed whether or not it was ‘serious’. A SAE was defined as any untoward occurrence experienced by either a person with dementia or carer that:

• resulted in death
• was life-threatening
• required hospitalisation or prolongation of existing hospitalisation
• resulted in persistent or significant disability or incapacity
• was otherwise considered medically significant by the investigator
• came within the scope of the Protection of Vulnerable Adults protocol,³⁰ which was in place to ensure that suspected cases of abuse or neglect were followed up in an appropriate manner.

A reporting form was submitted to the Chief Investigator who assessed whether or not the SAE reported was:

• related to the conduct of the trial
• unexpected.

Serious adverse events that were judged to be related and unexpected were to be reported to the REC and the trial Data Monitoring and Ethics Committee (DMEC) within 15 days of occurrence.

Randomisation

Remote randomisation of participant allocation treatment was undertaken via a web-based randomisation service managed by the North Wales Organisation for Randomised Trials in Health (NWORTH) clinical trials unit, after baseline assessment and informed consent. Randomisation was completed using a dynamic adapative allocation method,³¹ with an overall allocation ratio of 1 : 1. Random allocation was stratified by site and receipt of acetylcholinesterase inhibitors (AChEIs). For each participant randomised, the likelihood of their allocation to each treatment group is recalculated based on the participants already recruited and allocated. This recalculation is done at the overall allocation level, within stratification variables and within stratum level (the relevant combination of stratification levels). By undertaking this recalculation, the algorithm ensures that balance is maintained within acceptable limits of the assigned allocation ratio while maintaining unpredictability.

Allocation concealment

The randomisation database was held at NWORTH, and the analysts involved in the trial did not have access to the database. The dynamic adaptive algorithm is tuned using weighting parameters. These parameters are chosen by simulation modelling to ensure that the balance is maintained at an acceptable level, while ensuring that the sequence of allocations does not become predictable. Strong parameters would make the randomisation algorithm behave in a deterministic way, thus making allocation concealment difficult. Unblinded researchers were the only staff who were informed at each of the iCST centres of participants’ allocation.

Implementation

A web-based randomisation system was set up at NWORTH. Unblinded researchers could log into the system, enter participants’ details and receive randomisation results on screen and by confirmation e-mail. The system ensured that each entry had a unique trial identification number.

Blinding

As with all psychosocial interventions, participants cannot be blind to the allocation they receive. Within each iCST centre, there were nominated blinded and unblinded researchers; both were able to conduct baseline assessments and request randomisation. However, once participants were randomised, follow-up data were collected by the team of blinded researchers only, whereas the training and carer support in delivering iCST was run by unblinded researchers. Given that participants may occasionally and inadvertently inform researchers of the treatment they are receiving, we aimed to reduce this bias by use of self-report measures wherever feasible and brief reminders to participants. We asked all blinded researchers to record their impression of the group to which each participant was allocated and their confidence in that prediction. Statisticians remained blind to allocation for the main analysis, whereas compliance analysis incorporating compliance to the intervention was conducted after the main analyses only.

Data collection

Primary and secondary measures were completed at baseline, 13 weeks after baseline (week 13) and 26 weeks after baseline (week 26). Researchers were instructed to conduct all week-13 assessments within the 13-week period, but no later than 2 months from the scheduled first follow-up appointment (starting at date of baseline assessment), and to conduct all week-26 assessments by 26 weeks, but no later than 2 months from the scheduled second follow-up (starting at date of baseline assessment).

Most interviews were conducted in dyads’ homes. All questionnaire instruments were arranged in the form of booklets, with additional show cards of responses supporting the person with dementia during the assessment. If, at any point, the person with dementia felt uncomfortable with the assessment this was discontinued and was only rescheduled to take place during a second visit where appropriate.

Measures

Data checking

A full data-management plan was written, encompassing data storage and processing, data filing, data sharing, data freezing and data archiving. Data were collected in questionnaire packs and entered into a data-management system (MACRO version 4.1.2.3750, InferMed, London), which was audited for data entry accuracy, before being exported to Statistical Package for the Social Sciences (SPSS) files. SPSS Predicative Analytics SoftWare version 20 (IBM Corporation, Armonk, NY, USA) was used for all further data manipulations and analysis. In all SPSS files, cleaning processes were undertaken, including checks for consistency and out-of-range data. If applicable, questionnaire data were cross-checked with the SPSS data to explore any issues of inconsistency. CSRI data were cleaned and analysed in Stata 13 (StataCorp LP, College Station, TX, USA), again checking for consistency. Adherence data were entered by unblinded researchers into the MACRO system and used in both outcomes and economic analyses.

Data analysis

Economic analyses

The economic evaluation was a cost-effectiveness analysis, conducted first from a health and social care perspective and, second, from a societal perspective. The primary outcome measures for the person with dementia were the incremental cost of achieving:

1. one SMD (taken to be 2.4 points on the scale) in the ADAS-Cog
2. one SMD (taken to be 1.7 points on the scale) in the QoL-AD.

The primary outcome measure for the carer was the incremental cost per quality-adjusted life-year (QALY) (derived using the EQ-5D-3L with societal weights).

In the analysis plan, the secondary economic measures for the person with dementia were set out as: QALYs derived from the DEMQOL-U and DEMQOL-Proxy-U, the MMSE, BADLS, GDS-15 and QCPR. Secondary economic measures for the carer were the HADS, MCS-12, PCS-12 and QCPR.

Costs

Perspective

Costs from the health and social care perspective covered services used by the person with dementia, including care/nursing home care, hospital care (inpatient, day, outpatient and accident and emergency services), primary and community health and social care. Costs from a societal perspective covered the aforesaid services, plus the costs of care and support provided by unpaid carers.

Time horizon

The economic analysis used the mean end point (week 26) outcome measure (as the primary outcome for people with dementia and secondary outcomes) and costs over the 26-week period. In the case of the ICER for the primary outcome for caregivers (cost per QALY), we calculated QALYs over the year from the baseline assessment; we likewise assumed that mean costs remained unchanged since the end of the intervention period and calculated annual equivalent costs by doubling the costs estimated for the 26-week period.

Cost data collections

Costs were calculated based on several collections:

1. data on services used by the person with dementia, as observed and reported by carers using the CSRI.⁴³ Service use items were collected and aggregated into cost categories
2. data on time spent by professionals (unblinded researchers) in supporting carers to deliver the training package, using one of a suite of ‘treatment adherence’ forms
3. data on professionals’ labour costs, using a pro-forma distributed to the unblinded researchers
4. data on time spent by carers to deliver the training package:
   1. average time spent in preparing and in delivering the session: these data were collected by unblinded researchers during their telephone contacts, using one of a suite of ‘treatment adherence’ forms
   2. number of sessions completed: this information was collected from carers who were asked to complete a workbook feedback form after every session. A manual count of sessions completed was conducted by unblinded researchers at each follow-up monitoring visit (one per follow-up period)
5. data on carer time spent on care and support activities, and lost employment and out-of-pocket costs, using the CSRI
6. costs of training materials (excluding costs of the initial development and testing of the package), supplied by the project management team.

Unit costs/valuation of health and social services and unpaid care

Unit costs were applied to units of resources in the estimation of per-participant costs. The base year for unit costs was 2012–13. Unit costs employed are summarised in Table 3 (unit costs and sources are given in further detail in Appendix 4). Costs of health and social care were calculated from service-use data by applying relevant, nationally generalisable unit costs [e.g. NHS reference costs⁵⁴ and the Personal Social Services Research Unit (PSSRU) costs compendium⁵⁵]. Costs of carers’ inputs were calculated using two methods: replacement costs and opportunity costs.^56–58 The primary analysis used opportunity costs, attaching a value equal to the national minimum wage to each hour of unpaid carer time spent on care and the cost of lost production; a replacement costs approach was employed in the sensitivity analyses (valuing time spent on care at the unit cost of a home care worker) (see Table 3). Certain out-of-pocket payments were also considered to be a cost to the dyad rather than to health and social care: the travel costs of accompanying the person with dementia to dementia-related appointments by car, public transport or taxi, and the private purchasing of equipment and adaptations.

TABLE 3

Unit costs summary

Valuation strategy for intervention costs

The iCST intervention was produced from both professional and carer inputs. Unblinded researchers from the nursing and psychology disciplines worked to set up and train carers to deliver the sessions and then provided ongoing face-to-face and telephone support to carers throughout the study period.

In order to value the costs of professional time taken to deliver the intervention, we collated information on each researcher’s Agenda for Change band,⁵⁹ on costs and full-time equivalents on the project. We estimated workers’ indirect and direct overheads using PSSRU unit costing methods;⁵⁶ in estimating capital costs we assumed that workers were based in premises with a shared treatment space. A weighted hourly cost of professional support was then calculated based on the full-time equivalent contribution per Agenda for Change band. In addition, we calculated site-level average travel costs per visit, including professionals’ travel time and costs of mileage (unblinded researchers were asked to estimate their average travel time and miles driven in visiting participants on their iCST case load in each site). The costs of iCST training, including professional time and travel expenses and venue costs, were provided by the University College London project team. The project team also provided an estimate of the cost of the iCST training manual and materials (excluding the costs of developing the manual). We calculated an average training cost per participant and the average cost of manual materials per participant. The unit costs of professional support time and travel, training and materials costs of the intervention are summarised in Table 4. These unit costs were used to calculate a total cost of the package of materials and professional support in the cost-effectiveness analysis. We attached the weighted hourly cost of professional time to the reported telephone and face-to-face contact time for each participant, and a site average cost of mileage and travel time to reported face-to-face visits. We spread the per-participant manual and training costs, and the costs of professionals’ time spent in providing the set-up visit, across the two follow-up periods, allocating half the cost to each period.

TABLE 4

Individual cognitive stimulation therapy intervention unit costs: summary of components of professional support to carer

Carers provided data on the number of iCST sessions completed over each follow-up period (see also Cost data collections); they were asked by the unblinded researchers to estimate the time spent in preparing for and delivering the sessions during scheduled telephone support calls. The average time spent in preparing and delivering sessions in each follow-up period was calculated and this estimate was used in turn to calculate the total hours spent in these activities in each period. This time was valued at the national minimum wage in the primary analysis and at the unit cost of a home care worker in the sensitivity analysis.

Missing data for cost-effectiveness

For missing service-use data (collected from the CSRI), the following rules were applied: when service use was indicated but frequency was missing, a suitable nationally applicable unit cost was used if available (e.g. cost per visit). If no suitable unit cost was available, the cost was calculated as follows: (1) establish the mean duration of use of those with frequency information; (2) assign that mean value to cases with missing duration data; (3) estimate the average cost by multiplying frequency by duration by unit cost and estimate the mean cost of those where any use of the service has been indicated; (4) assign the mean cost to cases where both frequency and duration of use information are missing. For each case, items in each cost category (see Trial results) were summed to give a total cost per category. Category-level costs were summed to give a total overall cost per case. If all costs in the category were missing, the category total (per case) was also calculated as missing; if some items were missing, these were treated as zeros and the case was assigned the cost of the sum of available costs in the category. Missing outcomes and costs data were multiply imputed separately for the person with dementia and the carer. We used the MI impute chained command in Stata 13⁶⁰ to build a regression model, including demographic variables (site, whether or not acetylcholinesterase inhibitors were being taken, sex, relationship with the other member of the dyad, ethnicity, who the carer/person lived with, level of education), treatment allocation, cost categories and scale-level non-missing primary and secondary outcome measure variables as predictors. For the adherence data entered by unblinded researchers, the same procedure was followed; professional support costs were imputed within the model for imputing carers’ costs. The cost of carers’ time in delivering the intervention was imputed in cases where unblinded researchers had not recorded the time taken to prepare and deliver sessions within the adherence forms (so that the cost of the session time could not be estimated). Again, these costs were imputed within the model for imputing carers’ costs. The multiple imputation procedure was used to generate five complete data sets, to be combined according to Rubin’s rules.^60,61 Service use and intervention contact counts were not multiply imputed, only the costs. If the CSRI had not been completed (e.g. no questions or just one or two initial questions had been answered), these cases were considered wholly missing and were not included in the cost-effectiveness analyses.

Cost-effectiveness analyses

The iCST intervention was to be defined as cost-effective compared with TAU if it was:

1. less costly and more effective
2. more costly and more effective, and society is willing to pay the additional cost in order to achieve the gain in outcome
3. less costly and less effective, and society is willing to sacrifice some of the outcome difference in order to make a saving.

The iCST intervention was to be defined as not cost-effective if it was both significantly more costly and less effective compared with TAU.

The criteria for this decision was based on the following rule:

$$\text{ΔC/ΔE<λ},$$

(1)

where ΔC represents the additional cost, ΔE represents the gain in outcome associated with the treatment and λ represents the willingness to pay (WTP) for that outcome gain.⁶² The incremental cost-effectiveness ratio (ICER) (ΔC : ΔE) must be below λ to be considered cost-effective.

The ICER was defined as the difference in the mean costs of the iCST and TAU groups over the period of follow-up, divided by the difference in the mean end point outcome measure (primary outcome for people with dementia and secondary outcomes) between groups. In the case of the ratio of incremental costs and QALY, the denominator was the difference in the mean QALY over the year from the baseline assessment. The numerator was the difference between annualised costs, calculated by doubling the costs estimated over the full follow-up period. The decision rule can be rearranged to be expressed in terms of the net monetary benefit as λ × ΔE – ΔC > 0,⁶² the monetary value of gains in outcome associated with the treatment at a given WTP, net of the additional cost of the treatment.⁶² CEACs were produced to represent graphically the uncertainty around the point estimate of the ICER.

Summary of changes to the protocol

Approval was obtained from the REC for one substantial amendment to the protocol during the trial. This was related to the inclusion of additional questionnaires for both the person with dementia and their family carer. There was one non-substantial amendment notified to the study’s sponsor (University College London) representative, which was related to minor changes to the study protocol and inclusion of new sites and investigators. There was one protocol violation (this is described in detail in Appendix 9).