A total of 17 (77%) parents provided resource use information for the 2 weeks following recruitment. There was little resource use in any arm () and, because of the small numbers, one patient in the immediate topical antibiotics arm contributed nearly all the 14-day resource use and costs for that arm (). Two out of the six patients who had a delayed prescription did not use antibiotics during the 14 days’ follow-up. Excluding these costs from the analysis reduced the trial medicine costs for this by one-third.
A case note review was conducted for 21 patients. As with the 14-day follow-up, there was little resource use between week 3 and month 3, and the small sample size means it is not meaningful to compare the costs between trial arms ().
Qualitative
Sixteen primary care staff were interviewed: nine GPs and seven non-clinical staff from recruiting and non-recruiting practices, including one practice that withdrew from the study (). Some of the GPs had experience of recruiting to the trial and some had experience of getting the TRANSFoRm platform software to work. All of the interviewed GPs were partners and research leads for their practice, with time in practice ranging from 4 to 33 years. The non-clinical staff included practice managers, practice IT leads, a research co-ordinator and a research nurse (with no clinical role), who had experience of installing the TRANSFoRm platform software and the processes involved in getting it to work.
Primary care staff qualitative interview sample
Nine parents (all mothers) were interviewed: seven mothers participated in the trial and two mothers declined to take part. The parent sample captures some diversity with respect to treatment arm, and child age and gender, but most parent interviewees were from more affluent areas rather than deprived areas ().
Parent qualitative interview sample
Eight key themes were developed from the analysis:
whether or not the trial addressed an important clinical question
clinicians’ views of training materials
frustrations of trying to get the TRANSFoRm platform to function
effects of the study on primary care practices
primary care IT context – key challenges
barriers to trial recruitment
experiences of trial recruitment
reasons for parents declining.
Findings are illustrated below using anonymised verbatim quotations; P denotes quotations from parents.
The trial addressed an important clinical question
General practitioners felt that the trial addressed an important clinical question that was relevant to their practice and to their patients. GPs described being unsure about when antibiotics were needed and (in different practices) facing pressure from parents to both prescribe and not prescribe antibiotics. GPs felt that evidence about the effectiveness of the treatment options would support better treatment and communication with parents:
I liked the fact that it’s something that we see a lot and, like I say, there’s always this kind of unsureness whether we should be giving these kids antibiotics or not.
GP08
Parents don’t want to give their children antibiotics unless they absolutely have to, so they are quite willing to try some alternatives.
GP01
[W]e do see a lot of children with ear problems and there is always quite a lot of pressure to provide antibiotics from parents for them . . . having either data that says ‘You don’t need to do it’, [or] giving topical drops worked . . . would have the advantage . . . But it’s currently an off-licence use. So if we could get evidence that that was more supportive of its use, then that would be really helpful for ongoing care for patients.
GP09
Normal practice varied between GPs, reflecting a lack of clarity in relation to optimal treatment. Different GPs described normal practice as watch and wait, immediate oral antibiotics and, in some cases, delayed antibiotics:
I think I’ve probably been more likely not to give anything actually, just watch and waiting.
GP02
. . . so, normally, if they’ve got ear discharge present then we would be starting, erm, immediate oral antibiotics.
GP06
. . . if they were completely systemically well and they’ve got a runny ear then I think there might be a bit more of a negotiation about ‘well, is it getting better’, you know, ‘do you want to hang on for a day or two and see what happens, do you want a delayed prescription’.
GP07
Views of treatment options also varied between GPs, and not all were in equipoise, as not all necessarily viewed the different treatment arms as equally valid. Some GPs liked the ciprofloxacin drops because they are applied topically and, therefore, the GPs believed, less likely to cause systemic side effects. Some were familiar with the use of ciprofloxacin drops with adult patients. One GP raised concerns about the risk of toxicity and another about how quickly patients would be able to access the ear drops. Those GPs whose normal practice was to prescribe immediate antibiotics were concerned about delaying antibiotic treatment. Several GPs expressed a desire for clear evidence about the best treatment, to both guide practice and ‘validate’ their practice ‘to the powers that be’ (GP02):
. . . personally, I think [ciprofloxacin drops] are good . . . I have seen patients with recurrent [infections] who have had treatment with antibiotic drops, on occasions recommended by ENT [ear, nose and throat] . . . so I am sort of aware of it as a practice and I’m sort of comfortable with that . . .
GP06
[I]f there was data that said actually giving topical drops worked, then that would have the advantage that you don’t have the systemic side effects.
GP09
I was always a little bit hesitant to use [ciprofloxacin drops] to be honest, just because of the risk of toxicity . . . generally, if I thought there was perforation I’d stay away from drops, generally.
GP03
[S]ometimes it’s a problem actually getting them, so sometimes we’ve sent them to the chemist and they’ve commented that the chemist hasn’t had them in stock and, therefore, we’ve been rung up to say, you know, we want an alternative and I think then we’ve gone to oral antibiotics again.
GP02
I think it’s risky because . . . my fear is, is that what if a patient can’t then get in and if they can’t re-present or if their situation’s changed and . . . so I just feel uncomfortable with this delayed concept.
GP04
. . . it validates to the sort of powers that be that actually antibiotics, topically, are indicated so we are treating appropriately . . . and we wouldn’t get the pressure to, you know, from the kind of prescribing committee, we’d be able to validate the fact that, actually, we are better treating this with something rather than nothing.
GP02
The parents who participated in the trial were happy with the treatment options. The topical ear drops were seen as a good option by both parents who did want and parents who did not want antibiotic treatment:
I’m quite intrigued, I quite [like] the idea of the ear drop antibiotics for ear infections, I think they [. . .] might work.
P02
I was happy to do it if it helps, but also because he’s had so many antibiotics and [I] was happy for him to not have them . . . to be involved in something which would almost limit the use of them. Because I have been worried about the amount that he’s had to have for his ears over the years. So, yeah, the idea that there might be another option eventually . . . I think that would be really good.
P05
I thought it was good. I did want her to have some kind of treatment, I didn’t want just to leave it because I think it had been like 5 days or something at this point . . . I thought . . . there was a two out of three chance that she get a medicine. So I thought that was probably a good idea and yeah it was really easy to do and I was really pleased. And I got the drops . . . it was really simple.
P04
Clinician views of training materials
Clinicians described the training videos as detailed and comprehensive, if quite long. The length of the training videos may have meant that fewer clinicians were willing to complete the training to become recruiters, particularly as the expected number of recruits per practice, and, therefore, the recruitment-based payment to the practice, was quite small. Because of delays due to difficulties in getting the TRANSFoRm platform software to work, some clinicians reported a significant period between completing the training and actually being in a position to recruit:
. . . they were very detailed and comprehensive, I seem to remember at least one of them was quite long . . . but they were good in terms of guiding me through both the study and the sort of use of the app[lication], so what I was doing was watching little bits of it and doing little bits . . . and then pausing the video rather than watching it in one whole lot, so I watched it in piecemeal fashion as I was going through, sort of thing . . . so it did work quite well.
GP06
[T]here were quite a few . . . anything like that is going to put off some people who don’t have protected time for research . . . in a practice, you’re expecting maybe five or six doctors . . . to be recruiting, then that is going to limit because, you know, we just don’t have the time to do that . . . the problem as well is that it’s not a high recruiting study so if, for example, the practice was going to be recruiting 20 or more patients, then it would be worth watching those videos because you would get, you know, more money for doing that sort of number, but I think that you’re only expected to get quite a small number per practice so you might watch videos and do one or two patients and then that . . . you know forget really.
GP01
. . . certainly between the time that I think the training for the online software, so the integrated software rather, and then the, erm, the time the study actually went live, it seemed to be a sort of significant gap.
GP07
Frustrations of trying to get the TRANSFoRm platform to function
General practitioners and non-clinical staff involved in installing the TRANSFoRm platform and getting it to work described a long and frustrating process of troubleshooting and multiple reinstallations. Many were initially keen on the idea of the TRANSFoRm platform, as they hoped it would make the recruitment process quicker and easier. However, participants described encountering multiple problems and spending a considerable amount of time troubleshooting, seeking support from the trial team and reinstalling the programme:
I think as an idea it’s brilliant . . . it means you haven’t got piles and piles of paperwork and . . . paper forms that you’ve then got to somehow get scanned to e-mail through . . . it self-populates . . . it puts the details in, which is time saving. ‘Cause time is one of the big things in general practice.
GP09
I thought it was really interesting, I was really excited and then it’s made me a bit despondent really.
GP02
[T]he software didn’t work the first time for us, so I had quite a lot of communication with [the trial team] . . . they’d telephone and speak on the phone about working through different ways of doing it . . . to get that to actually work was quite hard.
Practice data manager, ITA01
[T]o be brutally honest [. . .] it was quite a nightmare . . . I’ve probably spent about 10 hours in total trying to install the piece of software on one computer. Um, quite often there would be loads of errors with it installing, with it not working. Um, I’d then have to send e-mails to the people that were dealing with it . . . it’s definitely taken so much longer than what we thought it was going to take.
Practice operations manager, ITA02
[I]t’s been very frustrating . . . communication [with the trial team] has been very good, but the problem has been with the REST TRANSFoRm platform . . . upset’s not the word, but it has taken an awful lot of time, to the point where the practices are saying ‘we don’t want to do this’.
Research nurse, ITA03
The software had to be installed individually on each recruiting clinician’s computer, and limited access to these computers led to delays. It was difficult to find time with a clinical computer, particularly for the early installations, which took several hours. If troubleshooting was also needed, then finding a time when the clinical computer and someone from the trial team were available led to delays in addressing problems:
[T]he major problem, um, of the installation is actually getting time to get into the GP’s room . . . we’re really limited on space so if that GP isn’t in, there’ll be a locum in their room . . . from 8.00 in the morning ‘til 6.30 or 7 o’clock at night. So actually trying to get in to have . . . 2 hours . . . is virtually impossible.
Research nurse, ITA03
[T]he software was only being installed on one particular PC [personal computer] at our practice . . . The time we had when his room was available and getting IT involved was always quite tricky. . . . we’d have to come back to the office, e-mail, wait for a time when we could speak to somebody . . . then you’re reliant on having to go back into a clinician’s room to go in and . . . use their PC and . . . they are seeing patients all day, every day so you’re trying to combine your REST study person with me . . . and then the clinician that’s in the room, to try and combine the three . . . how long these things take.
ITA04
Those who were involved early in the study and who were, therefore, installing an early version of the software felt that it was not sufficiently developed for use in practices. Participants were concerned about the time it was taking away from normal duties and the potential risk posed to practice computers:
The problem is it was rolled out far . . . too early with far too many problems and hadn’t been tested widely enough.
GP09
It basically felt like it was a prototype; it weren’t like the finished product.
Practice operations manager, ITA02
. . . it has got certainly the admin staff and the practice managers to the point of saying ‘do we want this on our practice computers? . . . is it going to cause more problems?’ . . . we have not got the time to be spending installing, take hours out installing this platform . . . it’s a very difficult situation and I’m getting everyone whingeing at me.
Research nurse, ITA03
In addition, during the trial, there was a widespread transition to Windows 10 in primary care practices and this caused some problems. Practices that had installed the early version of the TRANSFoRm platform software reported that it did not work properly after the transition to Windows 10. It also made troubleshooting more difficult as the person in the practice was using a different version of Windows to that used by the trial staff providing technical support:
. . . halfway through signing up for the REST study all our computers were then transferred over to Windows 10, so that caused a bit of an issue apparently . . . [trial staff] said that . . . [the trial had] programmed all the software for Windows 7 . . . I think the knowledge on Windows 10 wasn’t there because their systems are Windows 7 . . . so, therefore, it was very difficult for me to tell them what wasn’t working on our system because of the difference in the systems.
Practice data manager, ITA01
. . . when we went onto Windows 10, which was after the first recruitment, it’s not really settled down . . .
GP02
Some participants reported that later versions of the software worked well. The IT support manager in a practice that installed the final version of the software described the installation process as quick and easy. A GP who had not been involved in the installation process felt that the final version of the TRANSFoRm platform software worked well and had found it straightforward to recruit patients:
And now it nearly works. As in as long as you do this trick to get it to load properly, it works.
GP09
I installed [it on GP and nurse] PCs and it went absolutely fine, there was no problems with it and it just, yeah, installed . . . literally minutes . . . it was absolutely fine. You know it was just pressing next, next, next and finish and, yeah, it was no problem at all.
IT support manager, ITA07
I didn’t really do the installing. That was done by our research nurse and my practice manager . . . I do think the actual computer recruiting system, I think, works really, really well.
GP08
In most practices, however, participants described the final version of the software as unreliable and unpredictable. Participants reported that it worked for dummy patients but not real ones, or for some patients or staff but not others, or in some sites but not others. Participants felt that they did not understand why the software appeared to work in only some circumstances and this sense of unpredictability contributed to a lack of confidence in the software:
I think one would follow all the instructions and it seemed to say, yes, it was working, erm, but then it wasn’t when you went into it the next time and lots of shutting down; you kept having to shut down Windows in order to try it again.
Research nurse, ITA06
[W]e tried doing a dummy patient this morning again and it seemed to open up, but only randomly . . . even if we have it set up, it’s going to disappear again and my admin person comes down and sets it all up again and she can’t get it to work intermittently either.
GP02
[I]t just wasn’t working on [eligible patient] . . . but it was working on most [patients] that we tried. I have no idea why.
Research nurse, ITA06
[E]ventually . . . we were able to use it, but the functionality of what the REST study app[lication] was meant to be doing wasn’t happening . . . when [GP] was trying to, like, put in the code on SystmOne, the pop-up was meant to appear . . . and apparently that pop-up was never coming up, even though it was running in the background . . . On some days, it would be behind; other days it wouldn’t come up at all . . . no apparent reason.
Practice operations manager, ITA02
I still have this background lack of confidence that its going to if I needed to.
Research Nurse, ITA06
Effects of the study on primary care practices
The biggest impact of the study was the time taken to get the TRANSFoRm platform working in the already very time-pressured context of primary care. This had an impact on the other work taking place at the general practices, including preparing data for the Quality and Outcomes Framework (QOF) and, in one case, seeing patients. When the time commitment started to impede essential work, then practices started to consider withdrawing from the study:
I mean, all of the surgeries, all of the, all of the admin staff, practice managers, the secretaries . . . GPs, there’s not a spare minute in primary care at the moment . . . Research in primary care, I think we’re, I think we’re struggling a bit . . . There’s such a burden on, on GP time for major problems that research . . . the GPs would look at the studies and say, ‘yeah, I don’t have time to do this’.
Research nurse, ITA03
I’ve spent literally hours on this trying to install the software, hours. I think our clinicians have said that, you know, enough’s enough. They don’t want me to spend any more time on it . . . they [GP partners] just [got] cross ‘cause I wasn’t doing other things . . . I do all the QOF stuff so all the quality registers and things and all the statistics, all the claims . . . all that sort of was a bit on hold really.
Data manager, IT01
At one point, I started to refuse to install stuff because we were having so many problems with what it was doing to our computers . . . We wanted to wait until they’d got it more sorted because it was just eating so much time . . . it was probably towards the end of the QOF year last year . . . that I said I wasn’t prepared to put it back on until after we finished the QOF year because I couldn’t risk the machines not working.
GP09
When practices had staff with protected research time, participation in the project was more possible. In most of the practices, the recruiting clinician was a research lead with protected time for research. Some of the practices had other research support staff, including research nurses and, in a highly research-active practice cluster, a research co-ordinator:
. . . had I not been doing [a role with protected time for research] I would not have had the time to persevere and so the trial might not have been able to recruit . . . from my practice . . . it was . . . 4, maybe 5 hours that I spent . . . I do have protected time to do it so it does enable me to do things in a slightly different way, which I don’t think might be sort of rolled out for other practices . . . if you think they’ve just got to fit it in in their lunch breaks.
GP01
. . . one of them that tried a lot was our research lead . . . the other [clinicians] have basically really given up. I can’t get their computers to work now.
Data manager, IT01
Some participants reported financial costs to the practice as a result of participating in the study. At least one had paid their IT support person for extra hours to try to get the TRANSFoRm platform to work. Several participants felt that financial support to practices did not compensate for the time spent on research activities for this study:
I’ve worked extra time to do it as well. I’ve, they’ve actually paid me extra to come in and do the REST software, so I think that’s sort of annoyed them a little bit.
Data manager, IT01
[W]e’ve asked about, erm, resources for the amount of time that it spent us to install this software and provide the feedback that they’ve asked for and answered the questions they’ve asked for . . . the practice has actually lost quite a lot of money in terms of their time by trying to engage with this and that just puts people off . . . we do expect our costs to be covered. We do research ’cos we’re interested in research. If we were interested in making money, we would do pharma, but we don’t. But we are interested in not making a loss on it and not making a significant loss on it, which is what we’ve done with this.
GP09
Primary care information technology context: key challenges
Participants identified challenges with practice, CCG and NHS IT that contributed to the difficulties in getting the TRANSFoRm platform to function.
At the practice level, there was limited IT capacity and expertise. There were issues with outdated hardware and software and with the way in which individual GPs had adjusted SystmOne settings. There was varied IT expertise and capacity in individual practices, with many reliant on a GP, manager or administrator with only modest knowledge of IT. Computer administrator rights (which are needed to install software) were usually restricted to a small number of staff and not necessarily those with the time or responsibility for setting up research studies. This meant that, in many practices, the person tasked with undertaking the work to get the TRANSFoRm platform to function often struggled with the tasks and understanding the various problems encountered:
[The TRANSFoRm platform is] designed for really up-to-date computer systems and primary care is running on . . . old equipment and it just couldn’t cope with it . . . not only is it old, but it’s very old [and] not all on the same Windows.
Research nurse, ITA06
[T]he user guide was relatively comprehensive, but I would say not written for a user . . . that had basic IT skills. I think what they were asking for [was] somebody who was fairly IT literate to install the software, which I certainly wasn’t . . . We don’t have a particular IT person that can just go along and install . . . this sort of software.
Assistant practice manager, ITA04
[W]e are a fairly small practice . . . it was pretty much . . . me on my own . . . just trying to go through the installation step by step to work out where it wasn’t working and then trying to work out why, so trial and error.
GP06
[W]e’re not all IT proficient and that’s the problem . . . large practices, they do [have IT expertise] . . . but they don’t necessarily have the admin rights . . . these [small] practices don’t [have IT expertise]. So it’s the practice manager or . . . the secretary or someone who does it.
Research nurse, ITA03
. . . we didn’t locally have full admin rights. Well, the practice manager did, but, you know, to get her to sit down for a couple of hours and set it all up was very difficult; she didn’t have a couple of hours.
Research co-ordinator, ITA05
There were issues with obtaining help from outsourced CCG-funded IT support. All practices had some IT support provided by an external body, sometimes a private provider and sometimes a CCG or NHS provider. Five different providers were mentioned by our small number of participants, and IT support arrangements varied with respect to whether the external body held exclusive administrator rights for practice computers or supported research IT. Several practices reported that their external IT support provider would not assist because the software was not on the CCG’s approved list. When they were asked to provide support, these external bodies often raised concerns about the unknown TRANSFoRm platform software, were usually unfamiliar with software for research projects and were slow in providing support because of limited capacity. Only a research co-ordinator at a very research-active practice reported getting help easily, which she attributed to a good relationship with a particular person:
[W]e’re not sort of in charge of our own IT; the IT goes out to another company . . . our IT people who are [company name A], they’re not really supposed to give admin rights to anybody in a practice . . . [company name A] will not get involved with other people’s software . . . they have a list of software that’s allowed on the system and if we’re going to put some other software onto it they will not support us installing that software.
Practice data manager, ITA01
[W]e do have [company name C], but I didn’t get them involved in it . . . ‘cause . . . [company name C] wouldn’t help with it anyway . . . because it hadn’t been signed off by our CCG so we shouldn’t be installing it on our computers.
Operations manager, ITA02
[T]here’s been problems and we’ve had to go to our CCG IT team. And I’m just looking at an e-mail that’s come in this morning and their IT team are saying ‘what is this application? Is it a trusted . . .?’
Research nurse, ITA03
. . . [company name D] so they’re IT, they’re all NHS staff, but they’re a helpline, so you ring them, erm, with an IT queries . . . they were very dubious, actually, [about the TRANSFoRm platform] . . . I’m on, I think, my third or fourth call now with our IT about it. [It] wasn’t always easy to try and co-ordinate them ringing with then trying to install the software.
Assistant practice manager, ITA04
Some practices reported that they had to obtain permission from their CCG before installing software on their practice computers. The transition to Windows 10 during the trial was linked to the loss of practice-level administrator rights over computers in some CCG areas. Whether or not practices retained some administrator rights over their computers (and, therefore, the ability to install software) varied across recruited practices. When practices had to obtain permission from their CCG to install software, this could be a lengthy process. CCGs had questions about the risk that this unknown software could corrupt NHS software or practice computers and concerns about patient data crossing the NHS firewall. The centralisation of management to the CCG was seen as supporting initiatives such as the single domain, which allows better sharing of patient notes between different types of practitioners in primary care. However, it also had the unintended consequence of restricting the installation of study-specific software. Although some practices appeared to be able to install software freely, most reported having to defer to the CCG and wait for their approval. One practice reported that their CCG had required modification of the software to increase data security before permission was given:
[W]e had a big change at our practice, erm, something called single domain, which basically means that they’ve taken a lot of admin rights away from a lot of the users, including me . . . ‘cause I think it was becoming problematic across the practices that, you know, we had free rein really. And that’s going to cause a problem with things like REST because we can’t install it; so you give us a set of instructions and we won’t be able to do it because it has to go to our localised IT who has to verify they’re OK with it first.
Assistant practice manager, ITA04
[T]he CCG took it upon themselves to be responsible for all of our hardware and software, so when Windows 10 came for the whole of the CCG, they then took charge of everything really, which in a way makes sense because they paid for it and therefore they should control it and the flow of information that’s available and try to link it all up with other bits of the NHS, but as a result things . . . fell by the wayside, unfortunately.
GP05
. . . because of the way that the NHS is set-up we had to get firewalls opened, which wasn’t something we were told about in the beginning, to enable the software to contact [trial database] and then also for them to contact back through to our software; so, basically, you had to go through the firewall through a different port. And because we, we’re not sort of in charge of our own IT, the IT goes out to another company, so that was quite complicated at the beginning, having to go through these firewalls by logging it with our IT and then our IT doing it and that took a while. So that was the first sort of big issue we came across.
Data manager, ITA01
I think [the CCG] are quite – lax might be the wrong word – but we can install software and we do install software. So we’ve installed software for other research studies with no problems.
GP09
. . . we were all set to go and then we . . . needed to change our operating system . . . [to] Windows 10 so the original . . . [the TRANSFoRm platform] downloads would no longer work . . . to get them to rework . . . we couldn’t do it ourselves any more, we had to get the CCG computer boffins in to do it for us, they didn’t want to do it because they said its software may corrupt the NHS software and they wanted more assurance from higher levels than me that it was all safe to go, so . . . I got cross the told them it was all, it had been approved at high level . . . co-ordinated at committee level and approved and was being used elsewhere and they shouldn’t be so silly . . . so they then did come and put it on for me, so it’s now up and running.
GP05 – with role in CCG
[I] installed the software once I had permission from the CCG and that took [from] July/August . . . until December . . . the CCG felt they hadn’t got enough information from the installation guys . . . to be satisfied about the security aspect of the data that’s being transferred . . . the [study] IT engineers [tested] an encryption patch . . . [and] in December 2019, it was approved for installation . . . it’s just checking the security side of things, just make sure we’re not going to get any viruses . . . it’s about data protection; you know, they want to make sure that no patient identifiable data is going to be sent over for the studies.
IT support manager, IT07
Barriers to trial recruitment
Trial recruitment was reduced or slowed by the problems with the TRANSFoRm platform software. Several practices discussed identifying eligible patients but not being able to recruit them to the study because the software did not work properly. In some practices, there was a long delay while staff struggled to get the TRANSFoRm platform to work before clinicians even started trying to recruit patients. Some practices decided to withdraw from the study before any patients had been recruited because of the time taken to try to make the software work:
. . . last month there’s been four we’ve missed and all because the software just does not open up . . . I had someone in front of me on Thursday whose mom was really interested in doing it. I couldn’t even do a decline, and we just couldn’t get it to work and I tried, honestly, I must have spent about 20 minutes, I think, trying to sort it out and I had to move on really.
GP02
We’ve missed recruiting patients to the study because we couldn’t get the software working. We’ve had them in the consulting room agreeing to do the study, but we just couldn’t get the software to work so, therefore . . . we’ve had to abandon it. So there was a possibility of at least seven or eight that we couldn’t get because I’ve gone in and sat with the clinicians as well to try and get the software to work . . . it’s not that we’re not wanting to; it’s not that we’re not trying. It’s the software that’s not working . . .
Data manager, ITA01
I think that’s probably why there’s no recruitment, because the actual software itself wasn’t doing what they needed to do and then if they’re in an appointment they obviously don’t have the time to figure it out so they just move on to the next patient. . . . the doctors . . . they don’t have the knowledge to quickly figure things like that out . . . especially if it’s software that they don’t usually use.
Operations manager, ITA02
[W]e’ve taken a massively long time to recruit four patients . . . we should have done eight in a third of the time – probably would have done if the software worked.
Research co-ordinator, ITA05
Eligible patients were sometimes missed because they were not seen by a clinician who was able to recruit to REST. The processes for channelling eligible patients towards a recruiting clinician differed across practices. This was partly due to differing processes for dealing with same-day appointment requests for acute illness. Some practices had minor illness nurses who would usually see all children with suspected ear infections. Some practices triaged all patients requesting same-day appointments, and those patients might be seen by one or two duty clinicians or any clinician with a free slot. Study recruitment problems arose when these normal processes channelled a potentially eligible patient towards a clinician who was not able to recruit to the study. Administration staff and triaging clinicians responsible for booking patients into appointment slots did not always remember to book potentially eligible patients with the few recruiting clinicians. Where an eligible patient was seen by a non-recruiting clinician, there were various ad hoc arrangements to redirect patients to recruiting clinicians, but these were probably impracticable, and there were no accounts of this happening in practice:
[W]e’ve got minor illness nurses and the minor illness nurses don’t really get involved in research as much . . . the problem is that if someone rings up with a cough or cold, all that kind of stuff, they are being put into the minor illness slot . . . we tried to actually put REST study slots into my clinic so that if anybody rings up with an earache . . . put them in here, but . . . I’ve had those slots up and running and I think it’s not even been used once . . . but we know that these guys are coming through because they have gone and seen the nursing team, yeah?
GP03
. . . reception are made aware that if they have a child with a runny ear to try and book them in with myself or the other GP in question, where possible; where not though, and this has happened . . . a couple of times, that someone else [sees an eligible patient], they send me an instant screen message to say . . . ‘do you want to see them’ . . . and then depending on . . . how my appointments are looking, I might say ‘yes that’s fine’, or I might say . . . ‘could you see patient Y for me instead?’ and then we sort of switch patients, if you see what I mean . . .
GP06
. . . whoever’s doing surgeries will have some appointments added on at the end for same day so we don’t do triage or anything else . . . the nurses will do the same . . . they might see . . . children with coughs and colds . . . the receptionists have an aide memoire to say any child with . . . earache of the right age and they’ve had any discharge at any time that they should be booked in with me.
GP05
We can’t do too much opportunistically, I mean people coming to urgent care, they struggle to get an appointment in the first place . . . our advanced practitioner might not be on the delegation log, but one of my GPs is, but they’re seeing somebody else.
ITA05
Some eligible patients were missed because recruiting clinicians were not available when the patient needed to be seen. Some recruiting clinicians worked part time and were able to protect certain time slots only, which did not necessarily coincide with when patients needed to be seen. A participant from a large, recently merged practice described the challenge of keeping triaging clinicians primed to recruit to the study when they faced long lists and multiple study reminders, and were not even sure of being able to book a patient in with a recruiting clinician:
I think there may have been a couple of kids with discharge who might have seen other GPs. You know, ‘cause I only work part time.
GP08
. . . nothing came in in the morning, and then I had a full-booked surgery in the afternoon . . . somebody came in in the afternoon . . . they came in at 4.30 which is . . . right in the middle of my surgery that was running late anyway . . . nothing’s come in this morning; I had protected time this morning . . .
GP01
. . . now we’ve merged, we’ve got 25 GPs, six partners, but . . . lots are part time . . . so trying to keep it at the forefront of their mind, those images on the computer screen that they have got them every morning, in a way you just get used to them being there, you don’t actually look at them . . . they come and face an absolutely hideous list, phone calls every morning . . . there were 2 days when there wouldn’t have been a slot, if they had remembered, they couldn’t have found a slot, which [. . .] probably make them think ‘oh, well I actually remembered and then I couldn’t book them in’.
ITA06
The number of clinicians in each practice who could recruit to the study was limited, and this contributed to the number of missed recruits. There seemed to be several reasons for the limited number of clinician recruiters per practice. In many practices, only the research lead and perhaps one other clinician completed the training to become a recruiter to minimise the training burden on practice staff. It is likely that the time it took to install the TRANSFoRm platform also contributed to practices installing it on a limited number of computers only:
I could recruit; none of the others did the training on the basis that it was most likely that the [trained] nurses would do it . . . when you’re not recruiting massive numbers, if you have too many people able to recruit, nobody does it very often, so it just takes everybody ages.
GP09
What we can’t do is install it on everybody’s system . . . we’ve got about 20 studies we might be doing at one time so you have to pick and choose your staff of who’s going to be involved.
ITA05
[W]e have it installed on mine and the other GP . . . who has the research experience . . . and the administrative machine; it’s not running on the other clinician’s machine . . . so not everyone’s recruiting, just the sort of people who know about the study.
GP06
The nature of the target patients also presented some barriers to recruitment. Acute infections are seasonal and by the time the TRANSFoRm platform software was working in most practices, the winter season, when these patients are more common, was over. Cases that fitted the REST inclusion criteria were relatively rare. One research nurse described studies evaluating a new medication for acutely unwell children who attend with a parent (and often other siblings) as one of the hardest studies to recruit to:
[I]n the summer, we don’t see that many; in the winter, probably one to two a week . . . as a surgery . . . I’m trying to think, probably about two a week, something like that.
GP02
. . . it started at a time in summer where there wasn’t very much happening [. . .] and I think that’s taken a lot of momentum away . . .
GP03
. . . anything that involves a child that’s unwell, the parents have got other children with them, you know, younger children . . . children’s studies are harder straight off ’cause the parent and the child are involved rather than just an adult and if it’s a child who’s already a bit anxious then that makes it harder; so I think the children’s ones, where its opportunistic recruitment are probably the harder ones we do . . . and if it involves medication, its more complicated straight off. I suppose it is one of the hardest scale of ones that we do in general practice. I’m afraid our current issue is time.
ITA06
Experiences of trial recruitment
When clinicians had successfully recruited, this seemed to be associated with parent interest in the study and consultations with ‘relaxed’ mothers of relatively well children who had time for recruitment. Clinicians reported positive responses to the novel treatment (ciprofloxacin ear drops) and to the study processes:
. . . well, parents were happy that something was being looked at . . . those parents that we’ve seen . . . it’s not . . . the first time they’ve been in; they’re coming in regularly with the same child saying ear pain and discharge . . . And some of them are very reluctant to have antibiotics, but all of them want some help. You know, they all seemed to like the idea of trying topical drops.
Research co-ordinator, ITA05
. . . both of [the recruited children] were . . . it made it easier . . . both children were actually incredibly well, but there was loads of gunk coming out of their ears, erm, and they both had very laid-back mums . . . I said to both of them, you know, ‘there’s this trial going on looking at the different treatments, obviously I will examine your child and if I disagree with what you’re randomised to, you know, we can decide not to go along with it’ . . . I think [the first mum] really liked the follow-up that she had with the trial team . . . all the leaflets with the advice and things like that . . . I think it was helped that they were relaxed mums and they obviously weren’t in a massive hurry and – yeah. It made it easier.
GP08
. . . mum was pretty excited to be part of it so I think that was really quite nice . . . She was randomised to cipro . . . the ear drops . . . She was fine with it, I think the patient information leaflet was very good, you know, so that was excellent.
GP03
Recruited parents described study processes as straightforward. They had a clear understanding of the study purpose and found it easy to provide follow-up data:
I think everything was really well explained and the actual medicine that she got was really easy to use; the questionnaire was really simple; it was really easy to send stuff back. No, it was absolutely fine.
P04
. . . the nurse did explain to me as much as she could but [child] was quite . . . unsettled . . . it was a bit, erm, difficult to kind of grab all the information that the nurse had to give me, but the pack was quite comprehensive . . . it did have quite a lot of information and leaflets about what the study was about so I had the chance to read a bit about it when I got home so, and I had a call from one of the ladies involved in the study as well pretty much the next day . . . which was good because then you kind of get their reassurance of the study itself.
P07
I would have been happy with . . . any of [the treatment options], because they said if you’d have had nothing then if he started to get a . . . higher temperature or got really unwell then they would . . . look at him again . . . if he became really unwell. So they said that, as it was at the moment, he was happy for him to either have antibiotics or to not have it . . . They said at any time I could sort of take him back in and have him checked.
P05
Reasons for parents declining
Of the two parent decliners who participated in an interview, one declined because she wanted her child to have antibiotic ear drop treatment and one because she did not want her child to have antibiotics. In the former case, the child had a history of ear problems and, as a result of this experience, the parent perceived ear drop antibiotics as more effective than oral antibiotics for her son’s ear infections. In the latter case, the parent was aware of the drive to reduce unnecessary antibiotics, was told by the recruiting clinician that he would not normally give antibiotics for her child’s symptoms and she did not want her child to take something that was not needed. Recruiting clinicians described parents declining to participate because they did not want to have to:
[H]e’s had some ongoing problems with his ears . . . about a year ago, he had like a runny ear for quite a long time and he went on an oral antibiotic and it didn’t clear it up. And it carried on for probably about 2 months where . . . when I went back to the doctors and they found in his records, I think from the ENT, that he should have the drops in his ear. And when he did have the drops, it cleared it up within about a week, so when they asked me to do the, you know, be part of the study they said it was a randomised, where they just leave it or have the drops or have the antibiotics. And because I knew that that had been really effective with him last time and I didn’t want it to drag on for a few months like it did last time, I just asked if they would give the drops . . . I think if it had been my daughter, who’s not had any ear problems, you know, or recommendations about what she needed, I definitely would have done the whole study.
PD01
. . . the reason that I didn’t want to participate in the study because if there was no study she wouldn’t have got any medication for it at all. So it did seem a bit odd that . . . if she participated in the study then she’d be given antibiotics. I thought the idea was that you only get antibiotics when you really, really need them . . . The doctor had said that, at this moment, she didn’t need anything . . . I didn’t want her taking something.
PD02
. . . we did start to use the software and go through some of it, sort of, and then I showed them the leaflets about it and mom just started to look more and more doubtful about the whole thing as we went on, so she showed some initial interest and sort of said ‘yeah, OK, maybe I’d be interested in that’ and then as we talked about it and what she might have to do in terms of having a phone call, doing a sort of diary and things like that, she just looked more and more doubtful about it, to the point where you thought ‘no she really doesn’t want to do this’ and I sort of said ‘look, . . . it is up to you, you don’t have to’ and she said ‘no, I think probably not’, so it was a more slow sort of build to her saying ‘no I’m not interested in it’.
GP07
