Trial

To our knowledge, this is the first time a clinical trial of an investigational medicinal product has been conducted using an electronic trial platform in the UK. We planned to establish 175 sites to recruit 399 children. By study closure, 122 general practices from 12 CRNs had expressed an interest in supporting the study, of which 71 confirmed participation; 61 received sponsorship; 44 opened to recruitment, with the TRANSFoRm platform installed on 72 clinical computers; and seven sites randomised 22 children.

Although the main reason for poor recruitment was the delayed and intermittent functioning of the electronic platform, the number of AOMd presentations was also smaller than usual seasonally adjusted averages, contributing an estimated 25% to under-recruitment.

Despite the ‘hands-off’ nature of recruitment, randomisation and the use of standard NHS FP10 prescriptions for treatment, baseline data were available for 21 (95%) children, all children were given treatment as randomised, 38% of children fully adhered to the treatment and the symptom-based primary outcome was available at the 14-day follow-up for 17 (77%) children. The number recruited was too small to perform comparative analyses and draw definitive conclusions regarding clinical effectiveness, cost-effectiveness or safety, but we observed that all symptoms (including the primary outcome) except one were of shorter duration in the immediate topical arm than in the immediate oral antibiotic arm, and that parent satisfaction with treatment was higher in the immediate topical antibiotics arm than in the immediate oral antibiotic arm.

Electronic trial platform

Delays in the set-up and functionality of the TRANSFoRm platform function led to a cycle of increasing challenges resulting in critically low recruitment and early trial closure. Key performance challenges included (1) underestimating the technical challenge of integrating platform and EHR software; (2) underestimating the resources required to troubleshoot the resulting problems; (3) the need for repeated platform reinstallations at the sites, sometimes due to unannounced changes to EHR software; (4) multiple and complex site IT security arrangements, often involving third parties without contracts covering research; (5) failure to foresee the need for a platform ‘dashboard’ function, resulting in the TMG being unaware when the platform was/was not functional; and (6) site staff’s progressively reduced motivation to reinstall and use the software. When the electronic trial platform was operational, clinicians reported strongly liking its features and also reported that it assisted recruitment as intended; however, the function of the platform was acknowledged as ‘too little, too late’.

Qualitative

Qualitative clinician interviews found that the trial addressed a question of importance to clinicians and parents, and that, when the platform functioned as intended, it was liked. However, site staff reported the software not working properly for long periods, resulting in potentially eligible patients being missed. Moreover, getting the software to work placed significant burdens on general practices, diverting staff time from core activities.

The IT arrangements in primary care practices were varied and changing, with limited capacity, expertise and support. Although some practices had employed an IT expert, many relied on non-experts who had limited knowledge or training to provide internal IT support. This limited IT expertise within practices made it difficult for them to work with the software development team to diagnose and solve problems with the software. External IT support, which was provided by experts, was often not available for research. Administrator rights over practice computers, which were needed to install the TRANSFoRm platform, were not always held by staff within practices. Practice IT arrangements changed during the trial so that practices that did have administrator rights for the first installation of the TRANSFoRm platform did not have administration rights for subsequent installations and had to apply to their CCG for permission and support. Some of the changes to the practice IT arrangements were driven by new legislation, such as the General Data Protection Regulation,⁸¹ and initiatives to allow better integration of primary care medical record systems with others, such as out-of-hours services. The changes made in service of those broader NHS objectives restricted the use of specialist project software on practice computers.

Trial

To our knowledge, this is the first RCT to investigate the clinical effectiveness and cost-effectiveness of immediate topical antibiotics or delayed oral antibiotics compared with immediate oral antibiotics (standard care) for children with AOMd without grommets, and one of the first to attempt this using an integrated electronic trial platform. The qualitative evaluation provided rich contextual evidence regarding the advantages of and problems with the TRANSFoRm platform.

The over-riding weakness was the failure to recruit enough children to address the research question. That said, when functioning, the electronic trial platform did assist with within-consultation recruitment, demonstrating that it (1) can work and (2) has multiple advantages.

First, the system did support the identification, baseline assessment and consenting of children. Clinicians provided treatment as randomised for all children and most parents of recruited children provided symptom duration data. One child with a visible tympanic membrane was recruited, but should have been excluded. This cannot be attributed to the trial platform, but could have been prevented if the platform alerted the clinician to the presence of an exclusion criterion. The recruitment reminder ‘pop-ups’ were triggered by Read diagnostic codes, but would have been more sensitive had we also used Read symptom codes.

Second, as previously discussed, an electronic platform could provide unprecedented evidence regarding the generalisability of the final sample, support the recruitment of a very large number of ‘real-world’ pragmatic studies, improve baseline and follow-up data entry efficiency and accuracy, and ensure that the studies meet the many regulatory requirements.

Although the main outcome data attrition rates were acceptable, we note the low rate of stool collection at day 14 and the poor recording of analgesic use. We are also aware that one child who was recruited had otorrhoea that was possibly related to a foreign body. This should have been an exclusion criterion. Regarding treatment fidelity, there were high levels of agreement with allocated arms and no crossover between arms. Although the level of adherence appears low, this is of less concern when participants are randomised to a treatment strategy in an open trial because adherence (i.e. antibiotic use) is likely to mirror actual management in real-world settings. It is, for instance, recognised that the levels of antibiotic use are higher when the prescription is supplied during a consultation and when following a delayed strategy outside the trial context.^78,82

Qualitative

The small number of recruited parents and the early shut-down of the study because of the COVID-19 (coronavirus disease 2019) lockdown constrained recruitment to the qualitative study. However, sufficient ‘information power’ was achieved for the core themes presented.⁷⁵ The qualitative interviews captured a range of views from clinicians and other primary care staff involved in the study, which could be undertaken in a relatively small sample because of the specificity of experiences.⁷⁵ It was possible to purposively sample clinicians and non-clinicians from recruiting and non-recruiting practices, which captured a good range of views and experiences with respect to the trial. The primary care staff also described considerable variety in terms of the IT arrangements for practices, although it seems likely that this study does not capture the full range of variation in UK general practice IT arrangements.

The parent sample captured experiences from all three arms of the trial and a range of experiences of children’s ear infections. It also captured a range of views from parents from medium-to-affluent neighbourhoods, but did not capture views of parents from more deprived areas. Although the experience of being involved in the trial is quite specific, which means that a small sample can still provide information power, we recognise that the experiences of parents from deprived areas may be different and we do not know how well the experiences of our sample represent the experiences of parents from these areas. If larger numbers of parents had been recruited to the trial, it could have been possible to conduct purposive sampling of parents and capture a greater range of experiences, particularly those from more deprived areas.

Results in the context of other research

There is strong evidence that children with AOMd benefit from immediate oral antibiotics,² but, to our knowledge, there is an absence of evidence regarding clinical effectiveness and economic implications of immediate topical antibiotics and delayed oral antibiotics for AOMd in children without grommets tubes. REST has a sister trial, called PLOTS (Pijnlijk LoopOor Therapie Studie), which is still running in the Netherlands, that is investigating the role of a combined topical antibiotic and steroid preparation for children with AOMd without grommets.⁶⁸ This study has yet to publish a report. Although the non-inferiority design and eligibility criteria are similar for PLOTS and REST, the studies are complementary with regard to primary outcomes, offering the prospect of meta-analysis using the REST results.

The number of children recruited was too small for definitive comment, but we note that our sample was older [median age 5 years (IQR 2–7 years)] than the median age of 3 years (IQR 1–5 years) reported by Smith et al.³ in an observational study investigating the natural history of AOMd in children presenting to UK primary care. We also note that there was a predominance of boys (62%) in our sample.

Thirty-eight per cent of the parents fully adhered to the treatment as prescribed. Sixty-six per cent of children who were prescribed immediate (oral or topical) antibiotics started them within 24 hours of randomisation, compared with 50% in the delayed oral antibiotic arm, which suggests that delaying advice had only marginal effects on parental behaviour.

The RCGP Research & Surveillance Centre’s Weekly Returns Service⁸³ provides weekly notifications for communicable and respiratory disease for England graphically. For AOM, markedly smaller numbers were evident compared with the 5-year average. Over the period when REST was actively recruiting, numbers were reduced by between one-quarter and half compared with other years. We, therefore, estimate that at least one-quarter of the shortfall of recruited patients (compared with those projected) could, accordingly, be explained by this unusual drop in relevant infections.

Acute otitis media with discharge research

The clinical and research communities should wait for PLOTS to publish its report, and for any REST–PLOTS data syntheses to be completed before deciding whether or not sufficient evidence is available to change the management of children with AOMd, and whether or not further research investigating the clinical effectiveness and cost-effectiveness of immediate topical antibiotics and delayed oral antibiotics for children with AOMd is necessary.

If this question remains unanswered, the NIHR and research community will need to consider whether or not a further REST-type study is feasible. We remain convinced that the most efficient way to conduct this study would rely on a functioning electronic trial platform. Efforts should, therefore, be focused on establishing this facility, not only for this research question, but for the wider research community.

Recommendations arising from lessons learned

These are grouped by those responsible for the following activities: site identification, site set-up, site training, platform development, platform installation, troubleshooting, platform function monitoring and data management. Finally, there are two recommendations for national stakeholders, including DHSC and NIHR.