Chapter 11Post-trial assessment

Publication Details

Introduction

The final evaluation of the study involved telephone interviews with 20 of the men who had participated in the feasibility trial. This was designed to clarify which components of the recruitment process and intervention and control packages were acceptable to the men and whether or not the intervention package influenced drinking patterns. Participants were also asked to identify ways in which the study could be improved. This evaluation is recommended for trials of complex interventions.160

Aims

To identify:

  • participants' motivation for taking part in the study
  • reasons for their engagement with the components of the intervention
  • areas where improvements to study design could be made.

Methods

Semi-structured interviews were conducted by telephone with 20 men. Participants were selected so that 10 men from the intervention group and 10 from the control group were included. Five men from each group had been recruited by RDS and five from GPs. Interviews were conducted after the follow-up questionnaire had been completed. The interviews were designed to explore the participants' reasons for taking part in the study and the acceptability of all stages of the recruitment process. The men were asked to identify ways in which the delivery of the intervention could be improved, whether or not aspects of the intervention had caused them to review their drinking patterns or consider moderating their drinking. Participants were also asked to reflect on and report factors which might limit their ability to drink less (see Appendix 6).

Men in the intervention group received 36 messages which used social cognition models, MI techniques and components of behaviour change strategies to encourage a reduction in binge drinking. The control group received 34 messages on four health topics: healthy diet, physical activity, sexual health and mental health. These messages did not incorporate any behaviour change techniques. However, the control messages were comparable to the intervention package in the duration and number of texts, the use of questions to elicit responses, the provision of health information and the frequent use of humour.

Results

Characteristics of participants

The Phase 3 participants were representative of the whole study population. Five men participated from each of the four age groupings from 25 to 44 years. Half of the men were married and 9 of the 20 men were in employment. Thirteen men had received no education after leaving school and half of them lived in the most socioeconomically disadvantaged areas of the city. Twelve men were in the pre-contemplation stage on the Readiness to Change Questionnaire,123 although 13 indicated at baseline that they had at least thought about cutting down on their drinking. Almost all (90%) of the men were drinking hazardously at baseline (i.e. had a positive FAST).121

Factors influencing the decision to take part in the study

Curiosity and self-interest were most commonly given as the main reasons for taking part in the study (e.g. ‘Just to see what it was. I was curious’). Others were interested because the study was about alcohol. One man said that he thought the study would help to ‘try to take care of my alcohol problem’, one reported that he took part because he was in the process of trying to stop drinking, and another said he ‘was thinking about it anyway’. However, it appeared that alcohol as a topic had appeal even when the participant was not considering cutting down: ‘I thought it would be interesting topic. It [alcohol] was medium to a big part of my life’.

Several men took an altruistic approach. One man felt that it was ‘Important that people give their views and if it helps you [the researcher]’, whereas another said it would be ‘Good experience to help out’. Some men who were recruited through RDS appeared to want to please the person who had nominated them: ‘Mentioned by a friend and thought I would see what it was about’ or simply ‘My friend asked me’.

One man, recruited by RDS, appreciated being asked, although he clearly had apprehensions. He said that he wanted to take part ‘because nobody offered me these type of questions before’. He was encouraged because the study was conducted by telephone: ‘I didn't have to face the person’. He confessed he would be concerned about ‘a clash of personality’.

Six of the 20 men stated that they had doubts and concerns about taking part when asked directly. These were often general uncertainties: ‘Didn't know what was letting myself into, what was in store’, although these apprehensions were short-lived with one man saying they were ‘not borne out by the study’. Two men were concerned that they were being judged as alcoholics. One was ‘Afraid about what you were going be told. Would I get classed as an alcoholic?’, whereas another said ‘I was worried it was like the AA’. The participant information pack helped to alleviate worries: ‘Got the letter and read it and that helped a lot’. Four participants who were recruited by friends (RDS) thought that the decision to take part could have been easier if they were given more information by text message, e-mail or by a trustworthy person.

Nineteen of the 20 men said that at the end of the study they would be willing to nominate a friend to take part. The most common reason given was that the men felt a benefit from taking part and that the study raised awareness on health issues: ‘Good idea to improve the health of the community’. Although men in the control group did not receive any messages on alcohol, similar numbers from the intervention and control groups said that they would nominate friends to raise awareness on alcohol and help them with alcohol-related problems:

make them thinks about their drinking and what it's doing to them.

Cause I have got a few friends in same situation as me. And they could do with the help as well.

One man from each group (intervention and control) said they would nominate someone for entertainment value: ‘Such a good study, and you learn a lot from it! The text messages always fun to receive as well’. Credibility of the source of the messages was important. One man said he would nominate a friend ‘because you are from the university and he will know you are trying to help’. One man was dubious about whether or not his friends would take part if nominated, although he believed the study would benefit them: ‘a few friends that I think would benefit but they wouldn't take part’. Only one man said that he would nominate a friend for financial benefit.

Likes and dislikes

When asked specifically what they liked about the study, most participants commented on the appeal and acceptability of the overall approach. The comments indicated that both the method of delivery and the content of the messages were appropriate for the target group. One man summarised this succinctly: ‘Liked the style and the way it was delivered. Up to date, not letters. Felt direct’. Another commented that the study ‘Made you think about it in a non-serious way. More relaxed about it. If it was too serious, I would be less willing to think about’. Another man concurred saying that it ‘Didn't feel like it was pointing the finger’. The humour also appealed to the men: ‘I thought it was a good laugh. It cheered me up when I got it’.

Two men from the intervention group commented on the timing of the delivery of the text messages: ‘The randomness of the texts was good. They came in at times that stopped me from drinking. Seven out of 10 texts made me change my mind from drinking’.

Men from the control group also reported on the usefulness of the text messages, although they were given general health promotion messages only: ‘Couple of messages that opened your eyes to things that you didn't know before’ and ‘No pressure. Some of them were interesting. The questions were good, gave me something to do’.

The participants were hesitant in divulging details of anything they didn't like about the study. One participant said ‘The initial telephone call. I felt you were maybe trying to accuse me of being an alcoholic. Maybe trying to trip me up’. However, his apprehensions were fully resolved during the initial telephone call.

Opportunities for improvement

Participants were asked to consider how the intervention and control messages could be improved. The majority of men were satisfied, with some commenting that they enjoyed the texts. One man commented that ‘Anybody could have understood the messages even if you were dyslexic’. Only one man commented on the timing of the message delivery. He reported that he did not like receiving messages when he was out drinking. However, on reflection he decided ‘in long run [the intervention] has had a good effect, but at the time not good, but now think maybe it was good I got them’.

Most comments were suggestions on how to improve the images. The images were clearly important in reinforcing the text messages. One participant noted that the ‘ones with the pictures made me think more’, whereas another man said that the ‘ones with pictures – remembered better’ and suggested it maybe better to ‘put them all with pictures’. One man suggested introducing animation to the images and another wanted the ‘Cartoon pictures more life like. . . Real life’. He thought that he did not associate the images with himself if they were in cartoon format. One participant suggested the images could contain adult humour or be ‘a bit cheekier’.

Influences on drinking patterns

The majority of men from both groups reported that taking part in this part of the study made them think about changing their drinking. When asked if they could remember what made them want to change, the most common response was for family and health reasons:

My daughter is only 4½. Want to be with her as long as possible.

To save my health for my family.

One of the texts came through and I showed it to my daughter, and she said ‘you are going to have to do something about your drinking Dad’.

Men in the intervention group tended to give general answers about the effect of the intervention package and reasons for wanting to change:

The whole survey altogether.

Thinking about it, it was making me actively think about it. Text messages were making me think about it. It has been a good thing.

Reading the texts and make you think.

However, men in the control group who had not received the intervention text messages also reported that the study made them think about changing their drinking. They also recalled information given in the baseline questionnaire:

It was the binge drinking units in the questionnaire at the start.

Not to binge drink. Realising 4 pints is binge drinking.

Something I was doing anyway, but binge drinking of 4 made me think again.

Reading about facts. Did not think 4 pints is binge drinking.

Moving on from desire to change, the men were asked ‘Do you think you did change in any way?’

Several men from the intervention group reported that they had changed:

Drunk less, only one binge a month.

Yes, thought more about it and have cut back some weeks certainly.

Yes. Doing more stuff. When I went out drank juice, went to gym more, took my daughter out more.

A bit, but in the process of changing.

Yes, definitely, drink a lot less.

However, men in the control group also reported actual changes:

Took more interest in walking more. Don’t drink as much. Don’t down 3 to 4 pints each time.

Yes, bit more aware of drinking.

Yes – during my time of study had more alcohol in the fridge but was not going to the fridge. Was gradually drinking slower.

Yeah, more conscious of what I am drinking.

Two men, both from the control group, realised that they had not changed, but reported that this was due to family problems:

Not sure, as had a blow out as had problems with girlfriend.

No because a bad time with family, broke with missus.

Barriers to reducing drinking

Following discussions around the impact of the study, the men were asked to consider what the barriers would be to anyone who wanted to reduce their drinking. A few men from each group were unable to identify any barriers: ‘If they are going to do it they are going to do it’. One man thought it was about ‘Being able to control themselves, once you go past the limit its difficult to control yourself’. Overcoming addiction was highlighted by several men, some of whom felt lack of willpower was the barrier:

If you’re addicted, it’s about willpower.

Too hard, not got the will-power. It’s psychological.

Some men realised that drinking was a habit and the barrier is ‘Changing their habit, it's a bad habit. If you are in a rut, it is trying to find something to do. Fill in your time better’. Others identified peer pressure, unemployment, boredom and potential barriers. One man thought the main barriers were ‘Availability and price, supermarket cheap vodka, cheap beer’.

In an attempt to identify ways of improving and enhancing the impact of the intervention, participants were then asked if they could think of anything that would help people to cut down.

Two men felt it was entirely up to the individual and the answer was about self-control: ‘You have to make your mind up yourself’.

Six men felt that having support, both from family members and from a doctor would help. Others had thought of ways to reduce the amount consumed in sessions. One man identified two strategies: ‘Count up how many you would normally have and then reduce 1 or 2 pints a time, or keep the bank card at home’. Others proposed strategies to avoid drinking situations: ‘Getting out and doing more, and not sitting in the house. Going out for walks, to clear your head’. Another was aware that being employed was one of the best ways to reduce: ‘Something that doesn't stress people out. Having a job helps people cut down’.

One man proposed a complicated strategy to reduce gradually in order to avoid withdrawal problems: ‘Try to get them to drink half as much! Even ¾ as much, to stop them rattling. Then ½ for 4 days. Then ¼ lager for 2 weeks. Then few tins of lager. If they come straight down they can have a heart attack or die on their own puke’.

Some men reported that this study had helped them and had the potential to help other people:

The initial survey. I thought when I came off the telephone – I have a bit of a problem. The text made you think twice about drinking. Not all but some of them.

Show them the messages that might help them. It helped me and it helped my friend as well because I showed him the messages and pictures and he used to drink a lot, now he has calmed down. I saved the pictures and went round and showed him.

The final question asked of participants was whether or not the gift vouchers (£10 on completion of the baseline questionnaire, £20 during the intervention period and £10 each for the follow-up questionnaire and the in-depth interview) encouraged them to stay in the study. Two-thirds of the men said that it did. Some reported how they used them, for example ‘bought kids toys’ or ‘saved them up and bought some aftershave’. For some the vouchers were seen as a ‘a little reward, little bonus’. The remainder of the men said they were not influenced by the vouchers: ‘I enjoyed the texts. The vouchers was just a bonus’ and ‘Would have done it anyway’.

Discussion

The post-trial assessment with 20 participants provided insight into all aspects of the study. Some men were apprehensive when they were first approached and were unclear about the nature of the study, or whether or not they would be judged to be an alcoholic. These apprehensions were allayed, in some cases, by the participant information leaflet, and also by the first telephone conversation with the research fellow. An informal and friendly approach in the initial telephone contact was essential for facilitating recruitment. The main reason for taking part was curiosity about what would be involved and a general interest in the topic. Some men appreciated being asked to take part and thus felt valued. The men also demonstrated altruism in their decision to take part. Future recruitment should emphasise the importance of each individual's contribution to the study.

The attractive and informal style of the text messages was popular with the men as they did not come across as didactic or threatening. By the end of the study almost all the men said they had found the experience sufficiently worthwhile that they would recommend participation to their friends. This suggests that interventions should be designed to be enjoyable to promote engagement with the behaviour change strategies. Variety of content and timing of the messages was also important. As was expected,94,95 images were thought to be more memorable and to convey information and ideas more clearly than text. However, realistic pictures were thought likely to be more effective than the cartoon style that was used. The images will be modified for the full trial.

The modest financial incentives given at recruitment, at regular intervals during the intervention period and at the follow-up interview were important. The majority of men stated that these incentives encouraged them to stay in the study. Although many also said that they would have taken part without them, this may be a view of hindsight; having experienced the intervention, and enjoyed it, the men may have concluded that participation would have been worthwhile without the incentives. If so, it is likely that the incentives would have encouraged participation at recruitment and the early stages of intervention delivery. The literature shows that incentives are effective45,107 so it would seem wise to use them for the full trial.

The men confirmed that the intervention had made them think about changing their drinking habits. However, this was generally not just a simple wish to drink less. Instead it involved more engagement with close family and adopting new behaviours as an alternative to drinking. This suggests that text messages which ask the men about what activities they could do more of if they drank less could be added to the intervention. The comments that drinking was a habit support the idea of promoting new activities as a means of breaking the habit.

The finding that men in the control group felt that participation in the study increased their awareness of alcohol and willingness to reduce consumption has important implications. It is possible that the control messages, although intended to be neutral, elicited behaviour change. This could have been because the control messages were comparable to the intervention package in the duration and number of texts, the use of questions to elicit responses, the provision of health information and the frequent use of humour. The resulting engagement with the health messages could have led to a reappraisal of many aspects of life including drinking. The aim of the control messages was to promote retention in the study. As retention of the controls was 100% and the levels of satisfaction with the study were very high, a less active control package would still promote retention. Thus, for the full trial the number of control texts should be reduced, as should the amount of health information.

Several men in the control group reported that the baseline questionnaire prompted a reassessment of current drinking. It has been suggested that a screening questionnaire used to assess that participants meet the alcohol consumption entry criterion may be sufficient to prompt behaviour changes.9 The baseline questionnaire stated the definition of binge drinking and asked questions on current alcohol consumption, the harms of alcohol, the benefits of moderated drinking, drinking refusal skills and readiness to change. These topics are also components of the behaviour change strategy for the intervention group. They were included in the baseline questionnaire to measure whether or not, at outcome, these components had been changed to a greater extent in the intervention group than the control. To prevent the baseline questionnaire from acting as an intervention, fewer questions should be asked in the full trial. The complete set of questions could still be asked at follow-up and, combined with the extensive process measures described in Chapter 10, should still identify which components of the intervention were associated with reduced binge drinking.

In summary, the post-trial assessment provided insight into the recruitment process, the reasons for high levels of engagement with the intervention and ways in which the study could be improved. In particular, it suggested ways to improve recruitment and identified new behaviour change opportunities which could be added to the intervention. It also identified modifications to be made to the baseline questionnaire and the control group messages to ensure that these did not prompt a re-evaluation of drinking intentions.