Key findings
Stroke patients can become housebound, miserable and in poorer health because they cannot participate in outdoor mobility.14 This report presents the findings of a multicentre RCT to evaluate an outdoor mobility rehabilitation intervention for people with stroke. The intervention under evaluation was a complex mix of goal setting, practising outdoor mobility (which included walking outside), psychological support, and provision of information and self-monitoring of outdoor mobility through daily travel calendars. These calendars were used as an outcome measure. It was delivered by NHS therapists based in the primary care setting. The aim of the study was to replicate a single-centre study that found significant improvements in outdoor mobility participation when stroke patients were given the intervention by one highly skilled therapist. It aimed to evaluate whether the results were generalisable to NHS therapists in different UK locations. In addition, health-related quality of life and cost-effectiveness were measured.
The multicentre study replicated the methodology used in the single-site study. It reached its recruitment target on time. There was a high rate of follow-up at both the 6-month follow-up (to measure for an immediate effect of the intervention) and the 12-month follow-up (to assess whether it had a longer-lasting effect), and 70% of all travel diaries were returned. Stroke survivors and therapists were keen to take part in the study and although no formal qualitative study was completed to assess processes, it would appear from the proportion of patients who completed the travel diaries that getting out of the house is a major issue, even many years after stroke. A total of 568 participants from 15 UK-based sites were recruited over 18 months. The intervention was delivered, according to the protocol, to 287 participants, by 29 therapists, and was similar to that delivered in the single-centre study. Outcomes were collected by postal questionnaire and participants were supported, if needed, in their completion by RAs blind to allocation. The results were analysed using an intention-to-treat analysis, with the effect of different therapists at different sites being taken into consideration. A small qualitative study explored how a reduction in confidence might have affected people in the intervention group. The primary outcome measure used to calculate the sample size was health-related quality of life, as assessed by the Social Function domain of the SF-36v2. In addition, outcome measures used in the single-centre study – SWOM, travel journeys, activities of daily living ability and psychological well-being – were completed. A full economic evaluation was undertaken. This chapter will discuss the intervention, results, strengths and limitations of the study.
Intervention delivery
The intervention in this trial was developed over a number of years (2000–8), a treatment manual was produced and the CI taught therapists how to deliver it by a 2-hour training session. The intervention was a mixture of physical, psychological and planned preparation to achieve getting out of the house. Therapists were instructed to provide treatments as needed up to a maximum of 12 sessions over a 4-month period. They reviewed and planned their treatments following local protocols.
We consider that the intervention was delivered as per protocol to the majority of participants. We make this assertion because the intervention was delivered by occupational therapists and physiotherapists or by supervised rehabilitation assistants 100% of the time. These therapists had been trained to use the outdoor mobility manual by the study CI. It may have been that the manual was too prescriptive and did not allow therapists the possibility of providing more treatment sessions, but, owing to research methodology and financial restraints, therapists had to stop treatment after 12 sessions. The fidelity of treatment checking recorded that 100% of sample indicated that the therapists were delivering the correct treatment. We have to acknowledge that the fidelity of treatment checklist was not a standardised assessment owing to a lack of published measurements and that the last question which gave us this 100% rate had not undergone psychometric testing. The participants received a median of seven rehabilitation sessions with a duration of 3.5 hours. This is similar to that delivered in the single-centre study (six sessions, duration 4 hours). However, when the range of the intervention sessions was explored, it was found that approximately equal numbers (20 had one session, 20 had two sessions . . . , 20 had 11 sessions) but 60 participants received 12 sessions. It would appear that some patients may have needed > 12 sessions to achieve their goal. This could be the reason that nearly one-third did not complete their rehabilitation to the satisfaction of the therapist. Studies of upper limb rehabilitation suggest that > 3 hours per day for 30 days is needed to measure an improvement.69
Although the number of intervention sessions was restricted to replicate what might be provided by the NHS, the content of the intervention had been developed to be person centred and therapists could be flexible in how it was delivered. Therapists who provided the intervention and participants provided informal feedback and said they felt it was an appropriate and clinically relevant treatment technique. The research team have received over 50 requests for the intervention manual and training over the last 3 years. This indicates a real need for an evidence-based outdoor mobility programme. A more recent study developed a person-centred rehabilitation intervention by asking 132 participants to prioritise their main goals. The goals stated were mainly related to active recreation, household and community management, mobility and socialisation, information on stroke and prevention of new strokes, outdoor mobility and transportation.70
Effect of the intervention on health-related quality of life
The primary outcome measure of health-related quality of life found no significant differences between the control and intervention group at either the short- or long-term follow-up. The Social Function domain of the SF-36v2 was used to assess health-related quality of life in this population because the SF-36v2, as a whole, has been subjected to psychometric testing, used in numerous descriptive and intervention studies, and can be completed in < 10 minutes by self, proxy, interviewer or telephone.71 This allows comparisons to be made with other health-care treatments. It includes a range of domains, such as loss of role, participation away from the home and social inclusion, which are areas that therapists hope to improve through the type of rehabilitation under evaluation. The Social Function domain of the SF-36v2 consists of two questions and was completed by most (500) participants out of the original sample (568 participants). The two questions were:
During the past 4 weeks, to what extent has your physical health or emotional problems interfered with your normal social activities with family, friends, neighbours or groups?
(Please tick one.) Not at all/Slightly/Moderately/Quite a bit/Extremely
During the past 4 weeks how much of the time have your physical health or emotional problems interfered with your social activities (like visiting friends, relatives, etc.)?
(Please tick one.) All of the time/Most of the time/Some of the time/A little of the time/None of the time
This measure was not used in the single-centre study and other trials of rehabilitation using this type of measure have found similar neutral findings.72
–
74
Therapists and participants were concerned about the use of this measure, as they felt that the intervention aimed to improve outdoor mobility participation and the expectation that it would impact on social participation was overambitious, especially as the intervention was limited to 12 sessions over 4 months. In addition, they stated that participants wanted to undertake mobility activities for a number of reasons: getting to the doctors, dentists, shopping and work, which might not be classified as an improvement in quality of life as measured by these two questions. Improvement in outdoor mobility therefore might not have an impact on health-related quality of life as measured or it might take longer for the benefits to be realised. We conclude that the results were reliably collected and that the outdoor mobility intervention delivered in this study did not effectively improve stroke patient’s quality of life as measured by the Social Function domain of the SF-36v2. We suggest, however, that this is not a reason to stop outdoor mobility rehabilitation beginning offered to stroke patients. We believe that this targeted intervention has potential to increase outdoor mobility participation, which we consider an important part of activities of daily living. In addition, we have to consider whether the control group received a treatment that may have a role to play with patients for whom there have been many years since their stroke. We present our reasoning below.
Effect of the intervention on outdoor mobility participation
The main finding of the single-site study24 was that the intervention group were significantly more likely to participate in outdoor mobility. This positive outcome was considered an important clinical benefit. Stroke patients have complained about becoming housebound, isolated and miserable. This single-centre study demonstrated that, with a relatively short period of rehabilitation, patients could be taught how to increase their outdoor mobility. This included walking, using buses and electric pavement scooters. The outcome was assessed using two measures: SWOM participation and number of journeys. In the multicentre study, the intervention participants were no more satisfied with their outdoor mobility participation than the control participants at both 6 and 12 months but they were more likely to make journeys than those participants in the control group. However, this difference in number of journeys was apparent only when the data were adjusted to take the therapist effect into consideration. It appears that some therapists are able to produce these significant results, whereas others are not. We do not know whether it was the way the intervention was delivered, the characteristics of the participant or the skills of the therapist that produced this significant result. This outcome was measured using participant-reported journeys, which included outdoor walking, by travel diaries collected each month by post. Participants were taught how to use the calendars at the baseline visit by the RA and were encouraged to send them back using pre-paid envelopes. We feel that this was an objective measure as participants had to record each time they left the house and went somewhere. All participants completed the diaries and, as 70% of all travel diaries were returned, we feel this is a reliable outcome.
The results replicate those found in the single-site study, for which intervention participants took significantly more journeys at both 4 and 10 months. We need further research to explore the therapist and site effects in more detail. We consider that an increase in the number of outdoor journeys is an outcome that patients and health-care providers will feel is worthy of investment. It may be that an increase in outdoor mobility is helping people get to the doctors or health centre, they maybe going to work, or helping family with childcare. We know that people wish to get out of the house ‘just for the sake of it’ or ‘to get fresh air’ or to ‘enjoy just moving around’.23 In addition, there is evidence from a systematic review of exercise after stroke, which found improvements in health-related quality of life in the short term.75 It is possible that the outdoor mobility intervention provided in our study, which was mostly mobility training with the aim of walking > 100 m, was providing an exercise programme.
Satisfaction with outdoor mobility was measured using a single question: ‘Do you get out of the house as much as you would like?’ Although this measure was not designed as a patient-reported outcome measure (PROM) to assess health-related quality of life, it has many of the elements. It measures how people perceive the impact of a health intervention, is pertinent to the intervention being delivered and is easy to complete. Prior to this multicentre study, the measure had been subjected to some reliability testing76 and we have no reason to believe that the participants in the multicentre study found the question ambiguous. When we compared the groups we found no significant difference in outcome for this measure.
Effect of the control on outdoor mobility
There was very strong evidence that the control group improved markedly over time. At baseline, 259 out of 281 (92.2%) participants were dissatisfied with outdoor mobility but at the 6-month assessment this had reduced to 78% (160/205), a 15% reduction. The corresponding reduction in the intervention group was slightly greater (18%) with 268 out of 287 (93.4%) expressing dissatisfaction with outdoor mobility at baseline and 171 out of 227 (75.5%) expressing this at the 6-month assessment.
Participants in the control group must have been provided with a treatment that affected a change. It would be expected that people 3.5 years after stroke would remain stable over the next 6 months. This was a pragmatic trial and the therapists were told to deliver the baseline control as they would in the NHS. This was to replicate that given in the single-centre study. However, the combination of a face-to-face baseline visit presenting tailored (and participant focused) local transport and mobility information and use of daily travel diaries led to an effect in both the control and intervention groups. The baseline visit included a discussion about potential goals, which could have focused the participant’s thoughts towards improving outdoor mobility. We state previously that passive provision of leaflets and information is ineffective. However, we feel, with the motivated people who took part, that the control (targeted provision of leaflets and information) has acted in a different way. In addition, it is established that providing patients with daily diaries (e.g. food diaries) can lead to behavioural changes and an effect on the outcome being recorded. They provide a reference point for reflection on past and current behaviours, and they allow people to track changes over time and bring their behaviour to the front of their mind. In essence, they are a self-monitoring tool. Diaries have been used in behaviour modification programmes for a range of problems from weight loss and exercise programmes through to managing incontinence, tinnitus and migraines/headaches. Greenhalgh77 who investigated the feasibility of using diaries to measure the daily impact of multiple sclerosis on participants found that by completing a dairy participants became more aware of their symptoms, and this resulted in increased reporting. This impact may be further improved by the participants being aware that each month we would request the information they had recorded.
Effect of the outcome on functional ability and psychological distress
The intervention did not lead to greater levels of activity or reduce participant or carer psychological distress. The assessments used to collect the outcomes were well known and standardised measures used in many stroke studies.78
–
81 We have to conclude that the intervention, as delivered, does not impact on these areas. This result is in line with previous studies, which has found that a targeted rehabilitation intervention delivered over approximately six sessions can improve the specific domain being addressed but does not affect other domains.82
Was the intervention cost-effective?
In the base-case analysis the mean incremental cost (total NHS and PSS cost) was estimated to be £3413.75 (95% CI –£448.43 to £7121.00) with an incremental QALY gain of –0.027 (95% CI –0.060 to 0.007). This would suggest that the intervention was dominated compared with the control. The sensitivity analyses were broadly in line with the results of the base case.
Effect of the intervention on falls
Falls occurred in all age groups of stroke patients, with a median of three falls per year compared with one in four people from an aged-matched, non-stroke population falling each year.83 It may be that we need to explore specific falls-prevention interventions for people with stroke. There were no differences in falls between the intervention group and the control group, providing evidence that the intervention did not cause people to fall over more often.
What did we learn about confidence after stroke?
Strongly emerging themes indicated that barriers to high levels of confidence included loss of former self, negative reinforcement from others including health professionals, loss of confidence in roles and fear of everyday activities. Confidence levels were found to change over time, with participants having ‘good and bad days’, but, generally, the participants spoke of improved confidence as time since stroke increased. The factors that helped stroke patients regain their confidence were positive reinforcement from others, successful skill mastery and self-defined positive changes in roles. The study demonstrated that stroke survivors in this study were experts of the confidence phenomenon by identifying a variety of factors that have both enabled them to achieve and succeed, resulting in increased confidence levels, and also identifying barriers that prevent them from regaining confidence. These findings suggest there is a need for therapists to understand the impact that low confidence might have on outdoor mobility and to aim to provide interventions to overcome this barrier.
Adverse events
We had a small number of adverse events [20 from 17 (17/287; 5.9%) participants] within the intervention group during the delivery of the intervention. There was no evidence to suggest that the intervention led to an increase in falls that required the assistance of a health-care professional.
Methodological issues
Comparison with other studies
This Getting out of the House Study evaluated a complex, but targeted, outdoor mobility intervention for community-based people who had experienced a stroke many years previously. Owing to the number of unique methodological features there is only one study to which we can compare the results. This was the single-centre study on which this multicentre study was based.24 We aimed to replicate the recruitment and intervention delivery but a major difference between the studies was the time from stroke. In the single-centre study, participants were recruited 10–11 months after stroke, whereas in the multicentre study they were recruited 37–40 months after stroke. By completing a pragmatic multicentre trial and recruiting participants many years after stroke, we may have found a population more adapted to their situation and less likely to respond to the intervention. This could have contributed to the neutral effects found in the multicentre study compared with the positive effects found in the single-centre study. This highlights the difficulty of completing a pragmatic study over an explanatory study where inclusion criteria are more rigid. It would appear by comparing the two studies that the population most likely to benefit from the outdoor mobility training are those 1 year after their stroke. The other major difference between the studies is that in the single centre only one therapist delivered the intervention, whereas in the multicentre 29 therapists provided the intervention. The results from both studies provide some evidence that outdoor mobility rehabilitation is more effective when provided by specific therapists. However, we are not sure what characteristics these therapists need to have to elicit the benefit. Again, owing to the multicentre study being pragmatic and using routine NHS therapists, we consider that the results are more likely to be those seen if the intervention was provided to all stroke patients.
Generalisability of findings
The intervention delivered in this study was developed through university academics working with NHS and international partners. It was found to be feasible for delivery in the NHS by both occupational therapists and physiotherapists in all parts of the UK and suitable for patients, both men and women, many years post stroke. The age of the population was similar to the stroke population generally, so the intervention could be delivered to routine patients. The results cannot be generalised to patients early after stroke, who are most likely receiving Early Supported Discharge services or Community Rehabilitation. The evidence from the single-centre study is most applicable to this population and provides evidence that outdoor mobility participation can be increased by receiving a targeted intervention.
Strengths and limitations
The main strengths of the study are that it was conducted and reported following the CONSORT recommendations.84 The Nottingham CTU managed the data collection, keeping the group allocation locked until the analysis had been completed. The study was overseen by an independent TSDC who provided rigorous governance checks. The patient and carer outcomes were collected by post to reduce bias and three statisticians independently checked the analysis. This study was designed to be as pragmatic as possible by recruiting participants from a variety of settings and using NHS staff to deliver the intervention in 15 different sites across Scotland, Wales and England. The participants were randomised using a secure and independent service and the baseline characteristics of the two groups are balanced, providing evidence that the randomisation process worked. Key areas of strengths and weakness are discussed below.
Economic evaluation
The study was strengthened by having the economic data collection and evaluation being nested within the trial. The analysis was predefined and completed by a research team at a separate university. A weakness was that a number of assumptions were required in order to estimate some of the cost variables, for example it was unclear (for a few participants) whether they were reporting average times per carer/home help visit or total times for the week. That said, the results are broadly consistent, for example for the different cost perspectives, so it does not seem that this has any great effect on the results of the cost-effectiveness analysis.
Qualitative study
The inclusion of a small qualitative interview study that explored confidence levels in participants who received the intervention provided some indications that confident people with stroke are very keen to get out of their houses and are often restricted by a reduction in confidence as much as a physical impairments. A major limitation of this study was that a full qualitative study to explore processes was not undertaken. This may have reinforced the main findings, provided valuable implementation information and let us understand how the therapist effect was influencing the travel journeys. In future studies of this kind it is recommended that participants, staff, commissioners and managers are interviewed.
Choice of outcome measures
The study was strengthened by measuring outcomes at the functional and social participation level, using standardised patient-reported outcomes but weakened by not having available measures that have been used in people late after stroke. The main outcome measure has been discussed above. The rest of the measures have been used successfully with people in other stroke rehabilitation trials8 up to a year after stroke. There are very few outcome measures designed for changes in people with long-term neurological conditions. The participants would most likely have a number of comorbidities that may have affected their outcomes, such as arthritis. Therapists who provided the intervention were concerned that the clinical changes they observed were not measured. They felt that a goal attainment scale might have been a better primary outcome measure as it would have reflected the diverse needs of the patients. Some were aiming to walk 20 m and some were aiming to use the bus.
Participant identification and randomisation
Originally, we calculated that seven sites would be needed to reach the sample size over the period of the study. It was obvious that recruitment was slower than anticipated and the sites were doubled, and then the recruitment period was lengthened by 4 months. This improved the generalisability, as the sites were more representative of different locations – rural, cities, suburban – and participants were recruited in all seasons. Invitations from patients identified from stroke registers led to a higher proportion of randomised participants than identification from the GP database, with anecdotal evidence of involvement of the stroke survivor’s original rehabilitation team leading to enhanced response rates. This also indicated that more use of stroke rehabilitation services in stroke research would be beneficial for participant identification and recruitment purposes, as well as these services acting as hosts to future research. Whether there is a correlation between participant perception of stroke care from the sender of the invitation and their willingness to participate is unclear and deserves further investigation. GP approach yielded a poor response rate; however, this is not unusual among other research studies from a range of disciplines.
The majority of participants who were not randomised did not meet eligibility criteria, with a small number wishing to enter the study only if they could receive the intervention. Overall, there was a high level of acceptance and enthusiasm for the study and the intervention in particular. Whether the acceptance and enthusiasm was due to existing services not meeting the participants needs and the opportunity to receive any sort of therapy was appealing is not clear, but again deserves further investigation. Anecdotal evidence from participants showed that they were very appreciative of receiving the intervention and found it of great benefit; however, conversely, there were high levels of dissatisfaction and frustration from participants in the control group, indicating that maybe there is a strong desire from participants to receive ongoing therapy to address ongoing issues with their recovery from stroke. The eligibility criteria was very broad, with an aim to include as many potential people as possible, so people living in care homes, including those in wheelchairs who needed hoisting to transfer. Although this makes the findings very generalisable it often caused concerns to the treating therapists, as they felt that 12 sessions was not enough to achieve the goals in severely impaired patients.
An additional concern from the sites, but again with no real evidence, was that delivery of the intervention to the quality expected by NHS staff was sometimes difficult despite the great deal of enthusiasm and support from research networks to recruit participants to the study.
Collection of follow-up data
The dual approach of collection of follow-up questionnaire by either postal approach or RA approach proved very effective and logistically was easily managed. Response rates for travel diaries were surprisingly strong considering that only a single postal attempt per month was implemented. These diaries were the source of a vast number of data and, potentially, for future trials, could be adapted to collect additional information, for example social activity and resource use (GP appointments, carers).
Issues of unblinding
Although there was a high level of unblinding either prior to or during 6-month visits by the RA, it was tempered by the fact that the outcomes were still participant reported and the RAs were not assessing a subjective outcome measure for either primary or secondary outcomes. However, there was evidence from personal communication with RAs that they clarified certain questions that participants (in the opinion of the RA) had a tendency to misunderstand or misinterpret. Any impact of this bias will be minimal in the overall interpretation of the data, as this incidence is likely to be equally distributed across both treatment groups and occurred independently of participant allocation or RA unblinding status.
There was a clear risk of unblinding when collecting outcome data in a complex intervention in stroke patients via RA-assisted questionnaires, so there is a need for prevention methods and alternative approaches to be evaluated, while balancing that with the potential bias to outcome data from unblinding.
Differential follow-up
A common issue with studies that have a participant-perceived differential benefit, as apparent as in the Getting out of the House Study, is that a proportion of participants will be inclined to feel that continued participation in the control group is of no overall benefit. This benefit is related to the participant themselves, whereas others feel that it is of no benefit to study data overall. Although the former may be true, of course, at the outset of the study there was only preliminary evidence that the intervention was effective and the latter is definitely not the case. Despite efforts to maintain engagement with participants wishing to withdraw, of which the majority were in the control group, there was still differential follow-up between the two groups. However, this does not affect the overall study results, as we have 6-month follow-up data from both groups within predefined attrition rates.
Attention control
One of the biggest limitations of this study was the input given to the control participants. The exploratory data imply that control-group participants were given a treatment that caused an improvement, which was more than that expected from routine care. This is a methodological weakness and one that is difficult to overcome in rehabilitation research. We considered that the information given to all participants was similar to that provided routinely but collaborating therapists disagreed. They would have preferred in retrospect for control participants to receive nothing, as they believe this is what routine care is at present for people late after stroke. The addition of the monthly travel diaries seems to have changed the behaviour of participants whom therapists would have expected to be stable. As with other rehabilitation research projects it is impossible to complete a double-blind trial and control participants may have been eager to use the travel information and the monthly diaries to change their lives.
Therapist and site effect
A strength of the analysis undertaken in this study was that the statistical analysis plan was predefined and locked prior to the start of the analysis (see
Appendix 1
).
The two patient groups were compared using an adjusted modelling method, which took into consideration the fact that many therapists were providing the intervention and that some patients would be treated by a number of different therapists. This technique has been recommended for studies of this kind,85
,
86 as there is the potential for clustering of outcomes (owing to which site a patient is at and which therapist treated them). However, this adjusted modelling method has not been readily used in rehabilitation studies and is therefore a point of discussion.
Before the adjusted modelling method could be used, a weighting for each therapist providing the intervention needed to be calculated using an approach that considered the number of therapists treating each patient and the number of times that each patient was treated. This weighting was then applied to the outcome results while the groups were compared using the adjusted method. The adjusted results and the unadjusted results can be seen in
Tables 8–10
. Only one of the outcomes was affected by the therapist and site adjustments, and that was the number of journeys made. The result went from a neutral one to a significant positive one. A concern, though, is that the descriptive data in the 6- and 12-month tables suggests that the control group are perhaps slightly more likely to do more journeys. Another concern for the number of journeys outcome is that it was not possible to calculate the sizes of the obviously influential site and therapy effects (owing to the modelling process used). This meant that it was therefore impossible to determine which of these effects had the most weight on the adjusted result. However, even if it was possible to calculate the site and therapy effects for the number of journeys outcome, as was done for the other outcomes of this study, it would be very difficult to determine how these effects specifically altered the outcome. For instance, it would be hard to discover if any of the individual sites or therapists had a strong (or weak) influence on an outcome.
These results have been checked by three medical statisticians. This therapist effect can be explained by understanding that the intervention is a combination of the therapist and the techniques prescribed in the manual. It is very reliant on the skills of the therapist and the willingness of the participant. This type of analysis and understanding of the therapist effect needs further research.
