This report has been prepared using CHEERS (Consolidated Health Economic Evaluation Reporting Standards) (see Report Supplementary Material 4) and CONSORT (Consolidated Standards of Reporting Trials) (see Report Supplementary Material 5) guidance. The completed checklists are provided as supplementary material.
Internal pilot statement
The internal pilot (NIHR reference 10/104/501) was funded in March 2012 and began recruitment in January 2013. The aims of the pilot were to (1) assess recruitment rate, quality of data collection and follow-up, (2) provide a final check on procedures in the protocol and (3) conduct a qualitative substudy to inform the objectives, using a qualitative, service user perspective. The following stop–go criteria were used to determine if it was appropriate for the pilot to be considered an internal pilot and progress to the definitive trial: (1) no necessity for substantive changes to the protocol, (2) recruitment of 80% of the planned total in the first 12 months, (3) retention of participants within the study with baseline and outcome assessments completed for > 80% of participants for the secondary and other outcomes and mediators, and for 90% of participants for the primary outcome, and (4) satisfactory delivery of competent and adherent therapy to > 80% of the treatment group.
The stop–go criteria were all satisfied. No changes to the trial protocol were necessary. The pilot recruitment was on time and to target. Primary and secondary outcome retention was excellent, achieving 92% at both 9- and 15-month assessment points. Primary and secondary outcome retention at the later-added 24-month assessment point necessitated a new consent procedure and a completion rate of 69% was achieved. Of those randomised to SRT (n = 47), 41 (87%) were deemed to have received competent and adherent therapy. The pilot DMEC and Trial Steering Committee (TSC) reviewed trial data in 2014 and supported confirmation of the pilot as an internal pilot. These two committees and the PPI representatives (the PAT) reached consensus that no changes to the trial protocol were required. The sponsor also performed a full Good Clinical Practice audit of the trial and concluded that no changes to trial procedures were needed. Additional funding was sought (NIHR reference 10/104/51) for the extension and was awarded in May 2015, with recruitment beginning in September 2015.
Process evaluation
The PRODIGY process evaluation was an evaluation of both the research and the intervention processes undertaken as three substudies, two involving patient participants and one involving SRT therapists, during the initial internal pilot phase. The patient process evaluation was conducted by a subresearch team led by a user researcher (RB) and an independent qualitative researcher (CN), neither of whom were involved in the development of SRT. Both parts of the patient process evaluation (i.e. research and intervention process substudies) were investigated using a qualitative methodology with qualitative interview data collection (see Report Supplementary Material 3) and thematic analytic methods.69 These substudies were published as project outputs.70,71 The final part of the process evaluation was conducted by a separate subresearch team during the non-pilot extension phase and focused exclusively on SRT therapist experiences of therapy delivery.
Research process evaluation
The research process evaluation was a qualitative exploratory substudy. All participants who had been randomised were invited to participate between April 2013 and mid-June 2013, resulting in a convenience sample of 13 participants.71 An attempt was made to recruit substudy participants from both trial sites (East Anglia and Manchester) and both trial arms. Participant characteristics are shown in . Interviews were conducted by a user researcher (RB) and a RA between 1 month and 3 months after study randomisation. The interview schedule was semistructured and designed to probe experiences relating to study participation. Data were analysed using an inductive thematic approach72,73 with a critical realist epistemic stance.74 Themes are shown in .
Process evaluation substudy participant characteristics
Qualitative themes derived from process evaluation substudies
The themes focused around key reflections on noteworthy aspects of participating in the PRODIGY trial.71 First, ‘practicalities’ reflected the importance of RAs offering flexible research appointments. Flexibility on behalf of the researchers was seen as a manifestation of empathy and a person-centred approach. Second, ‘acceptance’ emphasised the position of openness from which participants approached their trial participation. Participants appeared to support the process of randomisation as a ‘fair’ way to allocate the intervention; they also appreciated the scientific importance of the ‘control’ group or comparator. The assessment process was described as acceptable even if questions were sensitive, again, within the context of a positive rapport with the RA. Third, ‘disclosure’ was positioned as a useful and therapeutic process within the context of assessment completion with the RA. The extent of their own disclosure, and the positive experience thereof, seemed to be a surprise for many participants. Fourth, ‘altruism’ appeared to be a key reason for participants wanting to be involved in the trial. This related to trial involvement in general and, more specifically, to participants being in support of the randomised controlled nature of the trial. A minority of participants appeared not to have a good understanding of randomisation. Finally, ‘engagement’ was a key theme that reflected the sense of being involved in the trial as a positive experience. Despite the main focus of the substudy being the research process, participants spoke of their thoughts and experiences in relation to the SRT intervention. Engagement in SRT was seen to be helpful; participants highlighted the benefits of understanding their experiences and how things could change for the better. Nevertheless, participants also described SRT as challenging.
Intervention process evaluation
The focus of the intervention process evaluation was on both arms of the trial (i.e. SRT plus ESC, and ESC alone). Participants were purposively sampled from those involved in the trial (pilot 100 participants only) aiming to balance across sex, study site (East Anglia and Manchester), allocation and baseline ARMS status. Attempts were also made to involve participants reflecting maximum variation in age and prior service use, and to include looked-after children.70 Nineteen people participated, and their characteristics are shown in .
Participants were interviewed by one of two researchers using a semistructured interview schedule. Questions focused on experience of psychological difficulties and previous service use, experience of trial participation and interventions received, and perceived outcomes and thoughts about future well-being. Data were analysed with an inductive thematic analysis approach69,72 taking a critical realist epistemic stance. Themes are shown in .
There were three themes common to the accounts of all participants.70 First, “‘It’s just the speaking to someone’: the value of talking” reflected participants in both arms perceiving there to be a therapeutic value in talking to someone (RA or therapist) during their trial involvement. Many participants felt that they had avoided or not had the opportunity to do so before their trial involvement. Subthemes reflected that talking to someone helped in two main ways: ‘it’s not boiled up in me no more’ and ‘it helped me recognise the things that I wanted to change’. Second, “‘Just do it’: the importance of activity” reflected the sense of importance placed on meaningful activity by participants in both trial arms. For SRT plus ESC participants, doing these activities reflected an important part of the intervention. Yet for ESC-alone participants, increasing their occupational activity also seemed important, and several participants described making clear efforts to increase their activity levels. Finally, ‘motivation to change’ reflected a sense that determination to make positive changes in their lives was a key feature across the accounts of all participants.
There were four themes corresponding to participant experiences of the SRT intervention.70 First, “‘She understood me on a personal level’: the therapeutic relationship” emphasised the centrality of the relationship with the SRT therapist to the participant’s experience of SRT. Participants emphasised the informal yet boundaried relationships developed with SRT therapists, which produced a sense of collaborative engagement. Second, the ‘flexibility’ theme reflected participant appreciation for the flexible way in which the intervention was delivered, for example with respect to timing and location. Third, in “‘It’s given me tools’: the CBT toolkit” participants spoke of how SRT equipped them with tools for managing their own distress and increasing their activity levels. Commonly described ‘tools’ included behavioural activation and behavioural experiments. Most participants felt that they could use the tools they had gained to their benefit beyond the intervention, although one participant felt that the intervention period was too short to allow them to use the same techniques independently. Finally, ‘No pain, no gain: SRT as difficult’ reflected that participants experienced the SRT intervention as challenging and even at times painful or overwhelming. Participants tended to emphasise that the ‘pain’ was worth it as they needed to push themselves to complete challenging exercises in order to improve their situations.
There were three themes corresponding to the experience of treatment as usual (i.e. ESC) only.70 First, most participants expressed ‘allocation ambivalence’ and did not seem to hold negative views about having been randomised to ESC. Some expressed relief at not having to attend therapy, which they appeared to feel would have been anxiety-provoking. However, two participants did express negative views. Second, the theme ‘No treatment, as usual’ spoke to the experience of young people in the ESC-alone arm of the trial as reflecting an absence of care and support provision. Only two participants reported having received specialist mental health care since their involvement in the trial. Some participants did report support from their general practitioner (GP), although they were not particularly satisfied with the nature of this support. Finally, ‘I was the one who had to do everything to help overcome it’ reflected ESC-alone participants’ sense that they had to manage their mental health independently, in the context of not being offered SRT and the perceived lack of treatment offered by standard mental health services. Some participants did emphasise that there had been considerable improvement in their mental health despite feeling unsupported by any services, conveying a sense of pride and achievement in improving without this additional support.
Therapist experience process evaluation
The focus of the therapist experience process evaluation was to explore therapist experiences of delivering SRT, with a particular focus on therapist hopefulness in the face of engaging with patients with complex presentations. All SRT therapists involved in therapy delivery in the extension phase of the trial, including two therapists who had also been involved in therapy delivery in the internal pilot, were invited to participate. Information about participation was shared verbally and using a PIS (see Report Supplementary Material 1). Ten SRT therapists participated in a semistructured individual interview following the provision of consent using an informed consent form (see Report Supplementary Material 2). The interview guide focused on asking therapists to reflect on their experiences of delivering SRT, including experiences where therapy had gone ‘well’ and experiences where it had not gone ‘well’. Therapists were also asked about their experiences of informal and formal support and supervision. Data were analysed using interpretative phenomenological analysis.75 Coding and analysis were performed by a SRT therapist experienced in SRT delivery but not involved in the inception or design of SRT and by a non-therapist researcher.
The finding of the therapy process evaluation was that SRT therapists emphasised the importance of working with this complex client group. In standard services, therapists reported that these young people would typically not be seen, because the typically protracted period needed to engage these young people is usually not possible within the services’ policies and practices. SRT therapists reported that, with this complex group, understanding and formulating treatment ambivalence and disengagement was essential, as was the pursuit of meaningful connection with these young people. SRT therapists also spoke of the difficulty of finding and sustaining their own hopefulness in the face of participants’ sense of ambivalence, hopelessness and ‘stuckness’. Sources of hope and scaffolds for its maintenance included access to expert and hopeful supervision, and opportunities to engage in peer supervision with other SRT therapists.
Process evaluation limitations and conclusions
The key limitations of the process evaluation substudies are that, because of the nature of the convenience sampling methods, the conclusions cannot be said to be generalisable to the whole trial sample or population beyond the trial. Nevertheless, the epistemic stance and qualitative methodologies taken were such that we sought not generalisability, but rather the unique experiences of individual participants of interest. Despite this the findings do show consistency with the wider literature, for example a previous study involving young people with ARMS.76 Further limitations of the process evaluation substudies include the apparent weighting of the intervention arm participants towards those who had engaged with the therapy. Only one participant was included who had not received a ‘dose’ of the intervention. Although the majority of participants in the trial did receive a dose, those who did not are an under-represented group and their experiences are particularly important in understanding both research and therapy processes.
Key conclusions of the process evaluation centred around the apparent value of having someone with whom to talk and reflect on mental health and well-being. This value was identified for both the research assessments and SRT. With respect to contact with RAs, the perceived value of the research assessment process seems to reflect the notion of therapeutic assessment77 (i.e. that assessment alone can have a therapeutic effect on patients). In the first substudy, all but two participants indicated at least some degree of past mental health service involvement, ranging from CAMHS or youth mental health services to school, university, private or third-sector counselling. Nevertheless, these participants still appeared to experience their trial involvement as reflecting a novel sense of ‘opening up’. This appears to reflect the particularly in-depth nature of the research assessments and also the flexible, engaging and skilled nature of RAs’ interactions with trial participants. It is also notable that in both substudies70,71 SRT participants emphasised the difficult and challenging nature of the intervention.
