Eleven process evaluation interviews were conducted, involving five trial participants and six members of staff involved in supporting the study.
Participant interview results
Method for inviting to research
Participants described how they found out about the study. Responses included an e-mail (that was not provided by the research team) sent around the workplace from management, which the participants assumed was part of a ‘work health kick’ (804CS, lines 18–20). Text messaging was also mentioned (846GP, line 10) and was viewed as acceptable (846GP, line 32) and preferable to a letter:
When it’s a text you automatically look at it and read it and it’s saved on your phone so you can’t lose it. With a letter you just throw it in the bin.
846GP, lines 37–39
A SM participant was introduced to the study by friends (1003SM, line 7), and they later shared it (1003SM, lines 19–21). One participant reported a method the study team had not anticipated, in which their GP asked for the participant’s e-mail address during a consultation and sent something to them following this interaction (867GP, lines 2 and 3).
Reasons for taking part
Participants gave several reasons for choosing to take part. These reasons included the fact that the information was interesting (804CS, lines 26 and 27), they were happy to help and provide an opinion (867GP, lines 16 and 17) and they felt that the research might ‘help improve services’ (867GP, line 117):
You could gather more data of what things tend to be working for people, and what wasn’t.
867GP, lines 130–133
Participants noted how participation might aid smoking cessation attempts and be ‘worth a go’ (804CS, line 9; 743CS, line 6):
When I hear things about smoking it just puts me off and makes me think about quitting more.
846GP, lines 17 and 18
The provision of vouchers was also deemed a motivator for participation (846GP, line 14):
It pushes people who want to do it though, doesn’t it?
1003SM, line 284
Secondary reason was free money from Amazon [Amazon.com, Inc., Seattle, WA, USA].
743CS, line 8
Ease of participation, comprehension and time to complete
Ease of participation was mentioned, with some expressing that the process was not problematic (804CS, line 35):
I’d just say it was easy, easy to follow, straightforward.
846GP, lines 47–49
No part in it was difficult that I came across anyway.
1003SM, line 54
The log-in process within eNgage was described as ‘familiar’ and therefore something participants were accustomed to (804CS, line 55). Equally, one participant expressed that the process made it clear that they did not have to take part:
It was my choice to give my e-mail address. I don’t think in a way it forced anybody to do it.
867GP, lines 42–44
However, a lack of clarity about how the study would be conducted was conveyed (743CS, line 52), and concerns were expressed about the usability of the study research platform eNgage:
[The] website is not good so stopped using it, not user friendly and could not log on so gave up.
743CS, lines 87 and 88
Poor website access, poor communication, poor follow-up, incorrect platform.
743CS, lines 113–115
The online process was clunky, and it was difficult to participate in.
743CS, line 120
[Study hosted on] ineffective platform . . .
743CS, line 173
It leads to people disengaging.
743CS, line 190
Participants felt that the survey questions were relatively quick to complete:
. . . was just right – if it was too short you wouldn’t have taken the information in. If it had been long, you’d have probably given up before you deleted it.
804CS, lines 49–51
Literally takes you 5 minutes, not even that.
1003SM, lines 50, 51 and 67
Relatively quick and simple.
867GP, line 93
The number of questions was also deemed appropriate by participants:
There wasn’t too many questions, they were very easy to go through.
1003SM, lines 69–71
[It was] quite simple, quite understanding, and there was a lot of information on there about what it was on about and what it was for.
867GP, lines 29 and 30
Just the right amount.
867GP, line 71
Participants reported that they found the questions easy to comprehend (846GP, line 83) and were clear about what was being asked:
They were brilliant they were.
1003SM, line 105
Really explanatory, really simple, the questions were put in simple form. There was nothing too complicated about it.
867GP, lines 53 and 54
Participants all agreed that the level of personal detail collected was appropriate, relevant and not excessive:
I thought that it was easy and it was just enough to target what you were asking.
867GP, lines 75–77
Reminders and amount of contact from researchers
Reminders to complete the survey were seen as beneficial rather than obtrusive:
You get reminders which is good. Yeah I felt like since I did it there’s been a lot of contact after.
846GP, lines 47–49
They kept in touch, but you weren’t getting 100 e-mails a day.
846GP, line 65
You don’t feel like you’re being hounded but you don’t feel like you’re being ignored either.
846GP, lines 72 and 73
It was keeping me informed at times of what was happening, if I needed to do more, if I was to have telephone calls, so it kept me in the loop the whole time.
867GP, lines 180 and 181
There was an expectation about follow-up contact as participants all knew to expect further questionnaires, so this was deemed ‘fine’ (804CS, line 187). One participant felt it effectively nudged them to complete the survey again, which was good (1003SM, lines 221–223).
The role of participation on stopping smoking
A number of participants described how the process of taking part had spurred thoughts about stopping:
Made you think as you were completing it . . . I liked that.
804CS, lines 43–46
I think there should be more things out there like this that’ll encourage people to think about changing their life or giving up smoking . . . but yeah it was really good. Excellent [taking part in the study].
804CS, lines 273–281
It’s kind of made me urge to stop smoking a bit more. I booked as soon as I did them, its actually pushed me into going to the doctors. Taking part prompts you to stop – it sort of triggers me really to want to stop again.
846GP, line 210
It’s nice to have little reminders about smoking, it puts you off.
846GP, line 212
Its [the study] brought to light more in a way of how much I want to do it obviously for my children . . . I have cut down.
867GP, lines 195–198 and 200
One participant described how participation led to them booking a SSS GP appointment and nudged them closer to their goal with increased motivation:
It did prompt me to go to the SSS which actually, to be fair, has put it in my mind that even though I haven’t maintained that that I will be quitting smoking by the end of the year, because I don’t want to be a smoker . . . so it helped me in that way to resolve in my mind psychologically that when I hit that point of having I’m ready, that I will actually go and quit smoking.
743CS, lines 297–299
Acceptability of randomisation and self-report measures
When randomisation was explained to process evaluation participants, all participants reported an understanding of why randomisation was necessary and were comfortable with the process (e.g. 846GP, line 191), although one participant expressed disappointment to have not been assigned to the intervention arm (867GP, line 166). Receiving intervention content after the study ended for those in the control group was also deemed to be advisable (e.g. 846GP, line 195). The demographic questions were reasonable and expected for the study (804CS, lines 70 and 71; 743CS, line 196) and were seen to increase equality (867GP, lines 67 and 68). One participant suggested that data on sexual orientation and other protected characteristics (within the Equality Act49) to ensure ‘well-founded demographic data’ would have been beneficial additions (743CS, lines 196–207).
Smoking questions appeared to make sense and were relevant and thought-provoking (804CS, lines 75–77) (846GP, line 97):
It’s what the doctor would ask anyway and you need to give this information in order to stop.
1003SM, lines 109–112
Quality-of-life questions were also viewed in a particularly favourable light to demonstrate the impact of smoking and stopping on health in general:
I think they are just as relevant . . . makes you think about not just your smoking but your other health issues you have.
804CS, lines 84–86
Questions about SSS use were also considered to be acceptable (846GP, line 125):
I just thought it was brilliant that part.
1003SM, line 162
Participants themselves did not feel it was intrusive (804CS, line 114); however, they recognised that other people might find it to be (804CS, lines 116–118).
Acceptability of StopApp (for intervention arm participants only)
Participants randomised to the intervention arm were generally very positive about StopApp. Specifically, one participant liked the structure of the prompts to think about when to stop, how to stop and then receiving encouragement (804CS, lines 29 and 30). Participants appeared to like using StopApp (804CS, line 31) and how it provided a choice (804CS, line 126). They commented that it was useful to provide information about what services are out there:
. . . because they’re not advertised.
804CS, lines 124 and 125
Participants did not know that there were so many SSS out there:
I didn’t realise how many SSS there were around. I thought there was only one lot of SSS, but actually that surprised me. So that’s why, just think it encourages you to, you could try different ones as well. If you’re not happy with one, you can try a different one, so yeah.
804CS, lines 131–136
StopApp had prompted one participant to look at other websites and information about smoking:
It encouraged me to check out other sites and other research that was done.
804CS, line 170
Participants suggested that an improvement could be made by including more information on the SSS, as this might encourage people to go:
You have a bit of an idea yourself then before you get there.
804CS, lines 140–143
For instance, this information could comprise how long the session would last, what happens at the session and provision of information on vaping (804CS, line 153). More information was also needed up front about the choices available (804CS, lines 178–183), and the inclusion of more smoking cessation content and options for methods to try. This could include information about how to stop smoking and the different sources of help for smoking cessation (804CS, lines 58–60 and 62–64).
Another participant expressed a preference for an online intervention:
I don’t like talking to doctors about my problems, so I’d prefer to do it online because they gave you the answer back . . . with doctors I feel quite awkward talking to them . . . I felt a lot more comfortable doing it at home online.
1003SM, lines 99–102
Provision of incentives (vouchers)
It was argued that providing vouchers encourages people who ‘just need a nudge’ (804CS, line 258). However, concerns were also expressed, with one participant arguing that the data would be of a higher quality and have more integrity if no monetary incentive was provided (743CS, lines 379–390). Another participant argued that the vouchers should be offered after people complete the surveys:
Because I think a lot of people just do it for the vouchers and that’s not right.
804CS, lines 258–270
You shouldn’t always get paid to do stuff like that.
1003SM, lines 279 and 280
I’m not entirely sure whether you’re necessarily going to attract legitimate data.
743CS, line 375
They want to please the payment source.
743CS, line 377
Thoughts about why people may not want to take part in the study or complete follow-up measures
It was naturally recognised that ‘some [people] don’t want to be told about smoking’ (846GP, lines 228–230) and, therefore, were reluctant to participate:
People think of I’m not doing that, if I do, I might feel forced into quitting, not actually understand it’s for research.
867GP, lines 211–213
Additional thoughts about why people might choose not to take part in MyWay included time (804CS, line 250), privacy concerns (804CS, line 250) and capacity to use smart phones:
People are nervous about data.
743CS, lines 359–361
[If] not good at working phones . . . but it’s that straightforward you can’t really get it wrong.
846GP, line 224
One participant stated that there was a need to make it clearer who the follow-up e-mail was from so it was not accidentally deleted as spam (743CS, lines 288–291):
I think that realistically I had not made the cognitive connection to the interest and the desire to participate with any of the names or the titles included within the e-mail body or the e-mail address.
Another explained that their partner did not want to take part in the process interview because of anxiety; however, it was not clear whether or not the participant incorrectly thought that the telephone call was necessary to take part in the main study rather than just the process interviews:
[He’s] got very bad anxiety and he don’t like to talk on the phone, that’s the thing that put him off.
1003SM, lines 252 and 253
A further participant suggested that the study was not targeting the right age group and should instead focus more on younger smokers, whose habits are less entrenched (743CS, lines 24–28).
Other suggestions for additions and improvements to the study
Better advertising and marketing
More advertising (internet, press) was recommended (804CS, line 13), with a more strategic marketing plan necessary. Participants disclosed that ‘the leaflet was dreadful’ (743CS, line 34) and to instead ‘use a marketing model like AIDA [attention, interest, desire, action]’ (743CS, line 49). A need to create something to attract the younger demographic was also proposed. Participants felt that the current recruitment advertising was not memorable (743CS, lines 39–44), and needed to hold interest with marketing tools (743CS, lines 49–51) by offering something unusual or ‘interesting’ to generate a reaction’ (743CS, lines 56–58) and is ‘relevant to today’ (743CS, line 61). It was suggested that we needed to make it more fun and improve branding to ensure that the message ‘sticks in people’s minds’, use ‘psychological anchoring’ to get them ‘involved and invested’ (743CS, lines 19–325) and be memorable in terms of an experience rather than monetary reward (743CS, lines 337–339; 743CS, lines 19–325).
A suggestion was made to employ a marketer to design advertising that was non-public health focused (743CS, lines 72 and 73). Stickers on leaflets that emphasised free vouchers were appraised positively, as these highlighted the benefits of taking part (743CS, line 33).
Greater focus on social media recruitment
Participants suggested that recruitment methods should include SM channels such as Facebook and Instagram (846GP, line 21). Participants clarified that we should ‘use more social media that people actually go to’ (1003SM, lines 31 and 32) because the study did not have ‘a good social media presence’ (743CS, lines 11 and 12).
Alternative mode of study delivery
The need to include an app to go alongside participation was suggested (743CS, lines 53–55); however, it seemed that this participant thought that the trial’s objective was to get participants to stop smoking which was clearly not made clear enough to them. Concerns about poor communication and updates were also raised; for example, one participant said ‘it was a cool study and I thought it was a good idea . . . but the execution of communication was entirely wrong’ (743CS, lines 102–105), and another participant disclosing ‘the actual study but was like what’s going on, how are you studying me, you’re not interacting with me?’ (743CS, lines 109 and 110).
Additional questionnaires
One participant suggested that she would like to have received more surveys as it ‘helps you’ to think about stopping smoking (804CS, line 58). More regular updates were also requested as 2 months was deemed a long time between contacts from the research team (804CS, lines 212–214) and the provision of weekly updates and regular encouragement was suggested (804CS, line 217).
Follow-up timing
One participant felt that follow-up data collection should take place at a later date (e.g. 12 months post first survey) to ‘see if they’ve gone through with it [stopping smoking]’ (846GP, lines 215–217). Follow-up e-mails were described as being problematic as they ‘may look like spam when you get many’ (743CS, lines 284 and 285).
Consent
Participants also expressed the need to ensure that participants still gave consent for their data to be used at the point of attending SSS (743CS, lines 235–240).
General comments about stopping smoking
General thoughts about stopping smoking were also discussed in the interviews, with comments made about personal quit attempts, different methods tried, the psychological mindset needed to quit and reducing smoking behaviours to increase the likelihood of success at SSS (804CS, lines 199–209):
People will say they want to quit smoking whereas actually they probably don’t.
743CS, lines 353 and 354
Staff interview results
A total of 23 members of staff supporting recruitment to the study were invited to take part in the staff process interviews. This included professionals from a variety of CSs including public health services (PH), charities, GP practices and pharmacies. Six staff took part in the interviews. This included one GP (GPA), one GP administrator (GPB), one practice manager (GPC), one staff member from a charity providing PH and two pharmacy staff members (pharmacy32 and pharmacy14). The remaining 17 individuals did not respond to the contact or explained that they were unable to commit owing to time constraints. Interviews lasted between 5 and 14 minutes and were conducted by telephone.
Method by which staff found out about the study
Staff were introduced to the process evaluation study via a range of different means. GP-based staff were approached via the CRN contact (GPA, lines 2 and 3; GPC, lines 2 and 3) or through their managers or lead doctor (GPB, lines 2 and 3). The PH staff member received a flyer from their team manager (PH, line 2), whereas the two pharmacists were either contacted by the study team (pharmacy32, line 2) or through discussion with their manager (pharmacy14, lines 3–9). One pharmacy-based participant was a pharmacist with stop smoking advisor training (pharmacy32); the other was a pharmacy technician who had received the stop smoking advisor training.
Motivation to be involved
Staff stated that they were keen to be involved in research (GPA, lines 8–10) and were enjoying their involvement in the study recruitment process (pharmacy32, line 53):
They help people out in the long-run as well.
Pharmacy32, line 56
If not too onerous we just sign up to it.
GPC, lines 6 and 7
If nobody does it then you’re not going to get anywhere.
GPC, line 8
[The value is] we’re always learning about new research and things and what the outcomes going to be and it’s also interesting and it’s good to get involved.
GPC, lines 22 and 23
I think it’s quite interesting really and I think it can probably help you sort of like deliver a more focused service, I suppose, if you’re kind of seeing what people are leaning more towards.
Pharmacy14, lines 109–111
Positive experiences and thoughts about the study
Overall, staff reported that the study had minimal impact on their place of work and their time and their thoughts were largely positive:
It was fine, it was organised quite smoothly.
GPA, line 10
[that’s] the ones we like [minimal involvement]
GPC, line 10
For GP staff the study was largely managed by the CRN contact, whom they found very helpful and supportive:
Support fantastic [from CRN].
GPB, lines 21–24
[no involvement from research team but this was] absolutely fine.
GPB, lines 27 and 31
Well everything just worked, I mean like I was saying we only had one referral and they didn’t turn up.
Pharmacy32, lines 61 and 62
Pharmacy staff also described support, this time from the study team, which meant that the impact on them was minimal:
[Set-up support] was great.
Pharmacy14, line 75
They actually came to the pharmacy and they went through it, showed us everything that went on . . . they took a lot of time showing us exactly what to do [study team].
Pharmacy14, lines 78–83
I think its fine the way you conducted yourselves and the way the system was set up. There’s no problem at the pharmacy side, it’s all to do with the patients isn’t it.
Pharmacy32, lines 105 and 106
I’m quite happy with the way you ran it, and like I said it’s just a shame we didn’t have more patients. But that’s always going to be the case with stopping smoking.
Pharmacy32, lines 131 and 132
Pharmacy staff also described the ease of using the system (PharmOutcomes) (pharmacy14, line 113):
Not at all time consuming, seeing how the calendar works, fairly easy.
Pharmacy14, lines 69–72
The calendar system and intervention itself were also mentioned by pharmacy staff, including the provision of text reminders for appointments in the intervention arm, and the usefulness of booking appointments online for local SSS (GPC, lines 40 and 41):
I think some patients were quite grateful for it.
GPB, lines 39 and 40
One pharmacy staff member felt that participation in the study had an impact on the care she offered:
I [thought] . . . I’ve got to make sure I see this patient because he or she has been referred . . . I’ve got to make sure they have a good experience with us.
Pharmacy32, lines 114–116
Negative experiences of taking part
Negative experiences were described in terms of a lack of communication between the CRN and the study team, which resulted in sending out letters on a set day the contact reported being unaware of (GPA, lines 19–29); and a lack of funding available to send letters as expected. Pharmacy staff described feeling disappointed about the lack of bookings received (pharmacy14, lines 113–115):
It would have worked well if anything happened.
Pharmacy14, line 119
I would probably just be more than happy to carry on with this stage [awaiting referrals from StopApp].
Pharmacy14, line 150
The PH staff member felt that the study was not explained well by their manager, but did feel that the flyers that accompanied this had ‘a fair bit of information’ (PH, lines 5 and 6).
Feedback from the study team
One GP staff member was unhappy with the lack of contact and feedback from the study team (GPA, lines 39 and 40). Feedback about how the study had gone, how many patients were recruited and the demographic characteristics of who took part (GPA, lines 68–70) was judged to be missing:
Obviously you don’t want to be hounded or anything like that, but it was just silence, kind of forgotten about it really.
GPA, lines 45 and 46
We always like to know, especially from a mail-out point of view because you send these things out because the patients aren’t responding to you directly.
GPA, lines 49–51
It was suggested that updates were needed (GPA, line 52), as would be received for ‘commercial studies’, which could be delivered via a newsletter (GPA, line 53):
Did it work or did it not work or just different ways to maybe improve things.
GPA, lines 57 and 58
Suggestions for improvement
Staff suggested improvements that could be made to the study.
Feedback from the study team
Better feedback from researchers was requested (GPA, lines 89–92). This was echoed by the PH staff member, who felt that she was not as well informed as she hoped and did not have enough information to pass on; she suggested that a briefing would have been helpful (PH, lines 66–72). This had been offered to all CSs and, in fact, this organisation had withdrawn from the training after a training date was agreed.
Advertising and marketing
More advertising was proposed (GPA, line 95), including posters for the surgery (GPA, lines 97–105) and pharmacy (pharmacy14, lines 12–16), and more advertising in public areas ‘not just GP and children’s centres, etc.’ (pharmacy14, lines 28 and 29). Bus advertising was also mentioned by a pharmacy staff member who was not aware that the study had already utilised this method (pharmacy14, lines 33–35).
Training/face-to-face support for community settings
Training was suggested, however, concerns were raised that staff would forget the training as they do not use the system regularly enough (GPB, lines 59–62). Face-to-face support was also recommended:
I think it possibly would have been good if someone did come in and deliver it maybe to staff . . . that might have helped just a little bit.
PH, lines 8 and 9
Vouchers
Vouchers were thought to have both positive and negative implications:
. . . good thing but don’t want ‘to encourage people to do that just for the voucher’.
PH, lines 74 and 75
Intervention mode of delivery
The PH staff member queried whether the study could be more paper based (PH, line 100) or, alternatively, the provision of iPads (Apple Inc., Cupertino, CA, USA) ‘may have helped’ (PH, line 103), as potential participants might need computer support (PH, lines 107 and 108). She felt that PH staff could deliver such training (PH, line 114) but only with the necessary technology to present it (PH, lines 115 and 116).
Recruitment method
Both pharmacy staff members queried whether it is best to recruit smokers when these individuals enter the pharmacies directly (pharmacy32, lines 15 and 16). These comments emphasised the need for other ways in which people can access SSS as not everyone uses the internet (pharmacy14, lines 129, 130 and 143–145).
Adding technology to the pharmacy system
One staff member mentioned combining the existing PharmOutcomes calendar system that is linked to the intervention with a ‘PharmAlarm’, which indicates to pharmacists when appointments have been made/messages have been received, and is currently under development by Pinnacle Healthcare Ltd (pharmacy32, lines 32–38). This feature would be beneficial and would streamline how StopApp communicates with SSS advisors about appointments booked.
