Aims and objectives

The CaFI feasibility study (phase 3) addressed the second of four research questions:

Is it feasible for culturally adapted FI to be delivered in hospital and community settings to African-Caribbean service users and their families?

Design

A feasibility cohort study incorporating a qualitative component.

Sample size

Over 9 months, we planned to recruit and consent a convenience sample of 30 participants via MHSCT and community referral. An audit of MHSCT data via the trust clinical information system (AMIGOS 3.0; Usable Systems, Dyfed, UK) at one time point (18 February 2015), indicated that there were 290 service users meeting our inclusion criteria. We estimated around 200 potential participants would be well enough to participate, of whom 150 would be eligible and 20% (n = 30) would opt in to the study. Thirty service users are sufficient to examine the feasibility of delivering the intervention across a range of service user and family types.

Participant inclusion/exclusion criteria

Participants and recruitment procedures

Service users

Service users were recruited via their care teams or self-referral. Recruitment at MHSCT was slower than anticipated and, therefore, we expanded recruitment to a second site, GMW (see Second wave of recruitment).

African-Caribbean people remain in inpatient services for longer than their White British counterparts,^13,15 so implementation of CaFI in acute as well as community settings might improve engagement and outcome. Therefore, we recruited participants from acute wards, rehabilitation units and community settings to include service users at differing levels of acuity and chronicity, and assess the feasibility of delivering CaFI across different clinical environments.

Advertisement posters and flyers (see Appendix 24) were placed in MHSCT sites that were accessible to service users and in community locations. NIHR Clinical Research Network (CRN) clinical studies officers (CSOs) supported recruitment by helping to identify and recruit suitable participants. CSOs and RAs visited teams at CMHTs and inpatient services to present the study and inform clinical staff about the inclusion criteria. Recruitment packs, including the study PIS (see Appendix 21), were posted to service users who were well enough [following risk assessment from care co-ordinator (CC)/clinical team], had the capacity to consent and gave permission for their contact details to be handed to the research team. The CSO/RA followed this up with a telephone call (at least 24 hours later). Interested service users were invited to meet with the RA to receive further information about the study and ask any questions. During the meeting, service users were asked to self-ascribe their ethnicity (as this is often recorded incorrectly in case notes) and provide written informed consent. Consenting participants were invited to complete baseline assessments in the initial meeting or an additional meeting. Figure 8 shows the Consolidated Standards of Reporting Trials (CONSORT) flow diagram¹⁶³ that details CaFI service user participant flow¹⁶³ and the progress of service user participants through the study.

FIGURE 8

The CONSORT flow diagram: CaFI service user participant flow. Reproduced from Edge et al. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits (more...)

There were four ways in which service users were identified for referral into the CaFI study:

1. CRN face to face – CSOs were located at different NHS settings and spoke to service users and clinical staff about the CaFI study, alongside other studies they were recruiting to. They recorded names and contact details of service users who met the criteria and were willing to be approached by the research team for further information. This information was then passed to the RPM/RA.
2. CaFI research visit teams – RAs/CSOs regularly presented the CaFI study to clinical and management teams in various NHS settings. CCs approached the service users about the study and passed on the contact details of eligible service users who were willing to be approached to the RPM/RA.
3. E-mails to CCs based on electronic case notes – the trust recruitment facilitator/information technology manager screened the clinical information system (AMIGOS) and generated a list of CCs of those service users who met the study eligibility criteria. RAs/CSOs sent e-mails to CCs asking if they would approach the service users on their caseload to see if they would be happy to be contacted by the research team.
4. Direct contact from referrer – CCs heard about the study through adverts (via NHS staff intranet, daily mail, posters/flyers at trust sites), presentations at team meetings or ‘word of mouth’; and contacted the research team directly with names of interested service users.

Intervention

In phase 1, we designed the CaFI intervention comprising 10 sessions as follows (see Appendix 16 for description of manual components):

• sessions 1 and 2 – engagement and assessment
• sessions 3 and 4 – shared learning
• sessions 5 and 6 – communication
• sessions 7 and 8 – stress management, coping and problem-solving
• sessions 9 and 10 – staying well and maintaining gains.

Quantitative data collection

Service user and family member/family support member outcome measures

Tables 4 and 5 show the service user assessment schedule and the family member/FSM assessment schedule used in the study respectively.

TABLE 4

Service user assessment schedule

TABLE 5

Family member/FSM assessment schedule

Psychosis symptom severity (service users)

The Positive and Negative Syndrome Scale (PANSS)¹⁶⁴ is a widely used 30-item semistructured interview designed to assess positive, negative and general symptoms in service users with schizophrenic spectrum diagnoses. The PANSS has good psychometric properties of reliability and validity and is sensitive to change.¹⁶⁵ Trained RAs rated the PANSS. High levels of inter-rater reliability were found between the four RAs’ PANSS scores. The average measure intraclass correlation coefficient (ICC) was excellent for PANSS positive (ICC 0.986, 95% CI 0.961 to 0.996), negative (ICC 0.976, 95% CI 0.934 to 0.994), general (ICC 0.975, 95% CI 0.931 to 0.994) and total (ICC 0.985, 95% CI 0.958 to 0.996).

Social functioning (service users)

The Personal and Social Performance Scale (PSP)¹⁶⁶ is a 100-point, observer-rated, single-item scale. The scale measures social functioning across the past month in four areas: socially useful activities (including work and study), personal and social relationships, self-care, and disturbing and aggressive behaviours. It is reliable, valid, sensitive to change and correlates with PANSS scores.¹⁶⁷ Ratings were made by trained RAs on the basis of service users’ reports of symptoms, service users’ behaviour during PANSS interviews and reports from care staff and significant others. PSP data were collected at baseline and at the 3-month follow-up only.

Perceived criticism (service users)

The Perceived Criticism Scale (PCS)¹⁶⁸ is a four-item self-report measure of service user perceptions of relatives’ criticism. It provides an efficient way of assessing negative aspects of the psychosocial environment. The PCS is a reliable and valid measure¹⁶⁹ and perceived criticism has been shown to predict symptom course, treatment outcome and relapse in schizophrenia.^169,170 Service users were asked to complete this measure in relation to their family member/FSM. If there was more than one family member/FSM taking part in the intervention, the service user was asked to rate the person who was currently the most important to them and with whom they shared the closest relationship.

Knowledge About Psychosis (family members/nominated family support members)

The KAPI¹³³ is a revised version of the KASI.³⁴ The KAPI was not conducted with recruited FSMs at baseline as they had limited knowledge of the service user’s problems before the intervention.

Family stress/burden (family members/family support members)

The 12-item General Health Questionnaire (GHQ)¹⁷¹ is one of the most widely used and valid measures of emotional distress and is frequently used to detect the risk of psychiatric morbidity. It was used as a measure of burden and general stress among family members and FSMs.

Working alliance (service users, family members/family support members)

The Working Alliance Inventory (WAI) short form¹⁷² is a 12-item self-report measure of the quality of staff–service user relationships and comprises three subscales: agreement on goals; agreement on tasks and emotional bond. The WAI short form has good psychometric properties.¹⁷³ Working alliance has also been shown to influence outcome in therapy.^174–176 Service users completed the WAI in relation to their key worker at the three assessment time points. Service users and family members/FSMs also completed the WAI in relation to the therapist dyad at the end of session 3.

Healthy utility index (service users, family members/family support members)

The EuroQol-5 Dimensions, five-level version (EQ-5D-5L)¹⁷⁷ is a generic preference-based self-report measure of health-related quality of life which covers five domains: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. Individuals’ responses to the EQ-5D-5L can be used to calculate a single index utility value from a tariff derived from UK population-based valuation studies. These utility values are used to calculate quality-adjusted life-years within economic evaluations. The EQ-5D-5L has been validated in diverse populations¹⁷⁸ and is recommended by NICE.¹⁷⁹ We assessed the feasibility of using the EQ-5D-5L as a utility measure in a formal cost–utility analysis as part of a future definitive RCT.

Relapse measure

Reduction in relapse was analysed using two recognised methods:¹⁸⁰

1. number and duration of inpatient admissions identified from hospital notes
2. number and duration of exacerbations of symptoms lasting longer than 2 weeks and leading to a change in service user management, such as increased observation and/or medication change by clinical team as assessed by hospital case notes (when symptom exacerbation preceded hospitalisation, only one relapse was recorded).

Discharge rates and hospital admissions were recorded, as was whether service users were discharged to higher- or lower-intensity services. For each participant, we used a ‘mirror image protocol’ to identify and compare relapse events for CaFI participants during two time periods:

1. The time from the individual’s first CaFI session until the date when case notes were accessed [time point 1 (TP1)].
2. A pre-intervention period, matched to the same length [time point 2 (TP2)]. For those participants who did not start therapy, the date of consent was used rather than the first therapy session. Relapse was rated retrospectively by two independent raters (JM and AS).

Hospital admissions

Using the aforementioned ‘mirror image protocol’ design, an independent rater recorded admissions data from electronic case notes on the number of hospital admissions, their dates, and the length of inpatient stay (days) during two time periods (TP1 and TP2) for the MHSCT service users (n = 29). Data were also gathered for two matched samples of Black and White British participants to draw comparison with controls receiving usual care versus CaFI. Participants were matched on diagnosis, gender, age and NHS setting.

Staff outcome measures

Working alliance (key workers, therapists)

The key worker assessment schedule and the therapist assessment schedule are presented in Tables 6 and 7 respectively. The staff version of the WAI short form¹⁷³ was completed by key workers in relation to the service user at the three assessment time points. Items were identical to those of the service user measure but were reworded to reflect staff members’ perspectives. The lead therapist and co-therapists also completed the WAI in relation to the service user and each family member/FSM at the end of session 3. The WAI has previously been used with key workers and therapists.^186–188

TABLE 6

Key worker assessment schedule

TABLE 7

Therapist assessment schedule

Service engagement (key workers)

The Service Engagement Scale (SES)¹⁸⁹ is a 14-item self-report measure assessing participants’ engagement with services from a key worker perspective. The measure has four subscales: availability, collaboration, help-seeking and treatment adherence. The SES has been validated in a psychosis sample and has evidence of good psychometric properties.¹⁸⁹

Data analysis

Quantitative data were analysed using Stata^® 14 (StataCorp LP, College Station, TX, USA). Data analysis followed a predetermined statistical analysis plan (see Appendix 25) drawn up by the trial statistician and agreed with the research team.

We present descriptive statistics on our primary outcome of recruitment, consent, demographics, adherence, missing or incomplete scales and attrition of participants, including an exploratory analysis of factors affecting adherence and attrition (using univariate associations with failure to complete).

We examined the characteristics of the various outcome measures to consider which might be appropriate in a future trial and estimated variability to inform sample size calculation.

The EQ-5D-5L scores were used to calculate a health utility index,¹⁹⁰ commonly used in cost–utility analyses of definitive RCTs. This made its distribution of interest in understanding the scale’s suitability for a larger trial recruiting from this population. As a secondary analysis, the hypothesis that the intervention would improve health utility index over time in all participants was tested by comparing scores before and after the intervention. Given that the data were skewed and that there were theoretical, prima facie reasons that CaFI might work differently when service users worked with family members rather than with FSMs, the hypothesis was tested using t-tests, bootstrapped with 200 repetitions from separate strata for dyads with family members and those with FSMs. This analysis was performed for both service users and participants, without correction for multiple testing given its exploratory nature.¹⁹¹

Relapse data were unavailable for definitive analysis, but readmission data from the trial participants and the two groups of matched controls were summarised to examine the proportions readmitted over follow-up; the number and length of readmissions; and the difference in these data between groups and within individuals between TP1 (before the start of intervention) and TP2 (after the start of admission). Each time period was defined as a ‘mirror image’ that is the duration of TP2 was the period from recruitment to data collection at the end of the trial and TP1 was defined as an identical duration before recruitment. As a series of secondary analyses, the number and duration of admissions during TP1 and TP2 were compared between groups and a survival analysis comparing time to first admission during TP2 between groups was conducted. These analyses tested the hypotheses that (1) CaFI would lead to a reduction in the number, total duration and hazard of admissions; and (2) these changes would be greater than any changes in the populations from which the matched samples were drawn. The comparison allowed for identification of trends in admission that were common to the intervention and control groups. Again, for such exploratory analyses, there was no correction for multiple testing.¹⁹¹

Recruitment

Eligibility

Of the 88 service users to whom recruitment packs were sent, 74 (84%) were eligible, nine were ineligible (they did not meet the inclusion criteria), and for five there were insufficient data to determine their eligibility. Of those who were ineligible, three were assessed as lacking capacity, one was deemed too high risk, and one had a diagnosis that did not meet the study’s inclusion criteria. Two participants were not of African-Caribbean ethnicity and for two there was no reason for ineligibility recorded.

Compared with those who were not eligible to join the study, the eligible service users were slightly younger [35 (SD 16.54) years vs. 41 (SD 13.49) years] and were just as likely to be male (74.32% were male vs. 71.43% in the ineligible group). Service users from rehabilitation units were more likely to be ineligible than those from other settings as can be seen in Table 8.

TABLE 8

Characteristics by eligibility

Baseline characteristics

Service users

We report the baseline characteristics of service users in Tables 12–14.

TABLE 12

Variability of outcome measures

TABLE 14

The EuroQol-5 Dimensions item scores during the study for family members (n = 15) and FSMs (n = 21)

Missing data

Data on gender, age and ethnicity were provided by all 31 participants. The remaining variables were available for 26–28 participants.

Personal characteristics

The patient sample (n = 31) was predominantly male (n = 21, 67.74%) with a mean age of 43 (range 17–81, SD 13.77) years. All were Black and 17 (54.84%) were African-Caribbean, 6 (19.35%) were mixed White and African-Caribbean, 4 (12.90%) were Black British, three (9.68%) were of other mixed background and one (3.23%) was of other Black background. The majority (n = 22, 78.57%) were born in England and 6 (21.43%) were born in the Caribbean. Most had some form of religious affiliation (n = 22, 81.48%), principally Christian (n = 17, 77%).

Employment and education

More than half (n = 14, 51.85%) of the service users were either long-term sick or registered disabled. Seven were unemployed, two were volunteers, two were students and one was retired. None was in work. The majority (n = 23, 85.19%) held qualifications at GCSE level or higher, including three (14.8%) who had Advanced levels (A levels) and one (3.7%) who had a degree-level qualification.

Family and living arrangements

Most of the service users were single (n = 21, 80.77%), one was married and two were cohabiting. Just over half of service users (n = 15, 53.57%) had children. Over half (n = 15, 55.56%) lived alone; the rest lived with partners or spouses (n = 3, 11.1%), parents (n = 3, 11.1%) and other family members (n = 2, 7.4%).

Mental health status: service users

All service users had schizophrenia spectrum diagnoses, bar three participants with first episode psychosis recruited from EIS (psychiatrists confirmed that these service users had non-affective psychosis and were on the schizophrenia spectrum). The age at which they first received treatment ranged between 11 and > 30 years.

Family members/family support members

We report the baseline characteristics of 36 out of the 37 family members (n = 16) and FSMs (n = 21) who took part in the study in Tables 16–18.

TABLE 16

Number of admissions before and after intervention and lengths of these periods by group

TABLE 18

Frequency count of items covered part II: CaFI and FIPAS subscales

Missing data

Data on gender, religion, employment status and children were provided by all 36 participants. The remaining variables were available for 18–35 participants.

Personal characteristics

Twenty-nine of the 36 family member/FSM participants were female (80.56%) and seven (19.44%) were male. Most participants (58.33%) self-identified as being of African-Caribbean background and 14 (38.89%) as White British. The majority of relatives/FSMs described themselves as having a religion, including 21 (58.33%) Christian, three Muslim and one Buddhist. Eleven (30.56%) participants were not religious (Table 9).

TABLE 9

Personal characteristics of the family members/FSMs who took part in the intervention (n = 36)

Employment and education

Almost half (47.23%) of the relatives/FSMs were employed full- or part-time. Seven were unemployed. Five were students, four were unemployed, and the rest were retired (n = 3), long-term sick (n = 3), volunteers (n = 2) and looking after the family/home (n = 2). The majority (n = 33, 91.67%) held qualifications at GCSE level or higher, including six (16.67%) who had A levels and 12 (33.33%) who had a degree-level qualification.

Family and living arrangements

Most of the relatives/FSMs were single (n = 18, 51.43%), seven (20%) were married, five (14.29%) were cohabiting, four (11.43%) were separated and one was widowed. The majority (69.44%) had children. One-quarter (n = 9, 25.71%) lived alone; the rest lived with partners or spouses (n = 13, 37.14%), parents (n = 2, 5.71%) or other family members (n = 10, 28.57%).

Feasibility of CaFI delivery

Of the 31 service users recruited into the study, 27 completed the baseline outcome measures and 26 started CaFI by attending at least one therapy session. Twenty-four family units completed all 10 therapy sessions.

Completion of treatment

CaFI comprised 10 sessions. We define ‘completion’ of treatment in three ways.

First, either the participant and/or a family member/FSM attend all 10 sessions. This flexibility was acceptable because some sessions can be beneficial if only service user or family member attends. By this definition, 24 out of 31 (77.42%) family units completed treatment. This is the definition we have adopted for the CONSORT diagram. Completion rates did not vary by clinical setting (Table 10).

TABLE 10

Number of family units recruited and completed by setting

Second, completion by the service user. Twenty service users (64.52% of those recruited) completed all 10 sessions. Twenty-three (74.19%) service users attended at least eight sessions.

Third, attendance by a family member/FSM at all 10 sessions although not necessarily the same individual family member/FSM at every session. Thirteen family members/FSMs (41.94%) completed treatment by this definition.

Service users attended a mean of 7.74 (SD 3.81) sessions out of 10 and family members/FSMs attended a mean of 6.87 (SD 3.85) sessions. The mean number of sessions attended by family units (i.e. the service user, relative/FSM or both) was 7.90 (SD 3.96) out of 10.

The number of family units, service users and FSMs attending by session did not vary greatly between sessions, other than a slight tailing off over time. Of the 26 family units starting therapy, only two did not complete the full 10 sessions (one missed the final session and another missed session 8 and the final session) (see Changes in family members/family support members during therapy for details of withdrawals/attrition). We compared completion for family units who were treated by psychologists with those who were not and found no difference.

Outcome measures for a future randomised controlled trial

We report on (1) data completeness, (2) variability, (3) distribution and (4) change in outcomes.

Data completeness (missing data)

The responses to each measure are summarised in Table 11. As the questionnaires were administered by the RAs, there were few missing responses and little variation in response rates.

TABLE 11

Response rate to outcome measures: service users

Service users

Service users were asked to complete all measures, but those working with recruited FSMs did not complete the PCS if they had not yet met the recruited FSM. Of the 31 service users recruited to the CaFI study, 27 completed baseline assessments, 22 completed assessments post intervention and nine completed assessments at the 3-month follow-up (see Table 11). We were unable to approach most service users at 3 months owing to delays in therapy delivery and data collection assessments (see Therapy delivery and Chapter 5, Therapist recruitment).

We achieved very high rates of data collection on all measures. Only one person missed PANSS and PSP measures at baseline. Two people did not complete the PCS at baseline as they had not yet met the FSMs. Four people did not complete the WAI post intervention (one did not complete the full measure and three had missing items). The common missing item was ‘I believe she/he likes me’ (service users often reported feeling uncomfortable responding to this item).

Family members/family support members

All family members/FSMs were asked to complete the quantitative assessments at baseline, post intervention and at the 3-month follow-up. Of the 37 family members/FSMs recruited to the CaFI study, 36 completed baseline measurements. Twenty-five completed post-intervention assessments (of whom 21 completed EuroQol-5 Dimensions), and nine completed assessments at the 3-month follow-up.

The recruited FSMs completed the GHQ but did not complete the KAPI and the BIPQ at baseline, as they did not know the service user. Furthermore, the medication subscale on the KAPI was not relevant for two service users who were currently not taking medication and, therefore, this subscale score was replaced with the mean subscale score. This meant that only 24 family members/nominated FSMs completed the KAPI and the BIPQ at baseline. We achieved very high rates of data collection on all measures.

Key workers

Of the 27 service users who completed the baseline assessments, all of their corresponding key workers agreed to complete the outcome measures. Twenty-eight key workers completed assessments at baseline, 23 completed assessments post intervention and 10 completed assessments at the 3-month follow-up.

The completion rates for individual measures were higher for the WAI than for the SES. This was because key workers reported being unable to answer some WAI items because they were not relevant to their relationship with their clients. Three service users’ key workers changed during the therapy and, therefore, the post intervention and follow-up assessments were completed by a different key worker.

Healthy utility index

We assessed the feasibility of administering the EQ-5D-5L¹⁷⁷ utility measure to service users and FSMs/family members for a formal cost–utility analysis in a future definitive RCT.

Service users

Descriptive statistics for the EQ-5D-5L responses are shown in Table 13. Because data were available for only nine service users at the 3-month follow-up (see Outcome measures for a future randomised controlled trial), only the first two time points’ data are presented here. The reasons for the low proportion followed up at the final assessment are discussed in Withdrawals/attrition from therapy. EQ-5D-5L scale completion was 100% for those who were seen.

TABLE 13

The EuroQol-5 Dimensions item scores for service users during the study

At each time point, the EQ-5D-5L health utility score¹⁹⁰ was substantially positively skewed: the most frequently reported level was ‘no problems’ for all five of the domains at baseline and at post intervention. Perhaps unsurprisingly, mobility and self-care were the two items for which most individuals reported having ‘no problems’ at baseline. Pain/discomfort and anxiety/depression were the only items on which any individuals reported having ‘extreme problems’. Post intervention, self-care was the item for which most individuals reported having ‘no problems’, followed by mobility and usual activities. Between baseline and post intervention, the most substantial change appears to have occurred in the usual activities item.

The range of scores at baseline was –0.20 to 1.00 and at the end of the intervention was 0.11 to 1. The mean EQ-5D-5L utility index was 0.74 at baseline, rising to 0.82 post intervention. Health utility index scores improved significantly after intervention (mean difference 0.09, 95% CI 0.02 to 0.17; p < 0.05, bootstrapped t-test). Although those with family members involved already reported their health state as better at baseline, the differences in health utility index between the 16 service users entering the intervention with family members (median 0.88; quartiles 0.71, 0.98; mean 0.71; SD 0.31) and the 11 with FSMs (median 0.67; quartiles 0.58, 0.83; mean 0.67; SD 0.23) were non-significant (mean difference –0.12, 95% CI –0.32 to 0.08; p > 0.05). In any case, both of these groups showed a trend to improve their utility score, although those with FSMs improved more (median score 0.89; quartiles 0.54, 1.00; mean 0.88; SD 0.26) than those with family members (median 0.94; quartiles 0.79, 1.00; mean 0.84; SD 0.25), who had less scope to improve. There was no significant difference between the two groups in the final score (mean difference 0.06, 95% CI –0.16 to 0.28; p > 0.05, bootstrapped t-test).

Family members/family support members

The range of health utility index scores at baseline was 0.31 to 1.00 and after intervention was 0.47 to 1.00. At both baseline and post intervention, health utility scores for family members/FSMs were negatively skewed, even more than for service users (see Table 13). Health utility index did not improve significantly after the intervention (mean difference 0.05, 95% CI –0.02 to 0.12; p > 0.05). This was the case both for those who were family members and for those who were FSMs (Tables 14 and 15). Given the high scores at baseline (e.g. > 25% scoring a maximum of 1.00, most family members scoring > 0.96), it was difficult for the intervention to produce any meaningful improvement.

TABLE 15

Utility index during the study for family members and FSMs

Relapse rates

Relapse measure

At the time of writing, data for the comprehensive assessment of relapse¹⁸⁰ remain to be finalised. As not all relapse data were collected owing to the hospital merger, this work is ongoing. Data regarding relapse will be available and presented in future trial publications. For the report, relapse data will therefore be presented for the audit at MHSCT only. Not all relapse data were collected owing to the hospital merger. This work is ongoing. Data will be included in the main trial outcome paper (in preparation).

Hospital admissions

For all three groups (CaFI service users, matched Black and White British controls), the mean period of assessment before and after the intervention was 313 (SD 142) days. The number of inpatient days was skewed. Before intervention, the overall median stay across all groups was 0 (quartiles 0, 0; range 0–733) days. Afterwards, the overall median was also 0 (quartiles 0, 0; range 0–168) days; this did not differ significantly over time across groups (p > 0.05, Wilcoxon matched-pairs signed-rank test), despite a noticeable shortening of maximum admission length (Table 16).

There were no significant differences between the groups in the number of admissions in the control period before the intervention started (p > 0.05, Fisher’s exact test) or in the length of inpatient stay (p > 0.05, Kruskal–Wallis test). Nor did time to relapse differ between the groups (p > 0.05, Cox regression model stratified by gender and controlling for age; Figure 9).

FIGURE 9

Kaplan–Meier survival estimates. Survival time (days) without relapse before intervention across groups. BC, Black Caribbean; WB, White British.

There were no significant differences between the groups in the number of admissions in the period after intervention started (p > 0.05, Fisher’s exact test; see Table 16). This did not represent a signficant decrease in the number of admissions for the CaFI group (p > 0.05, Wilcoxon matched pairs signed-rank test) or the two control groups (p > 0.05 and p > 0.05). Nor did days in hospital differ signficantly across all groups (p > 0.05, Wilcoxon), between groups (p > 0.05, Kruskal–Wallis test) or over time for any of the groups (p > 0.05, p > 0.05, p > 0.05 for the CaFI group, White and Black controls, respectively, Wilcoxon test).

Time to admission after the intervention did not differ signficantly between groups (p > 0.05, Cox regression model stratified by gender and controlling for age). Controlling for length of survival in the previous period by adding this to the model (adjusting for differences between individuals, and hence groups, in propensity to be admitted) made almost no difference (p > 0.05 between groups).

Withdrawals/attrition from therapy

Of the 31 service users who consented to the study, five dropped out pre intervention, four pre baseline and the remaining one post baseline. Out of these, the research team lost contact with three, one reported not feeling well enough to take part and one did not feel able to commit to the study due to new work commitments.

Of the 26 service users and respective family members/FSMs who started the therapy, three service users dropped out during therapy and did not re-engage. Out of these, one service user disengaged after session 3 (but their two family members continued with the therapy) and two service users dropped out after session 9; one service user relapsed and one agreed but did not attend the final session. In addition, one family member dropped out after session 1 due to a family bereavement and two FSMs (one nominated and one recruited) dropped out at session 4 because they could not be contacted after session 3.

Post intervention, four service users dropped out. Two did so following relapse, one refused to complete assessments and one could not be contacted. One family member and one nominated FSM were lost post intervention, one did not complete the assessments because their relative had relapsed shortly after the intervention and the research team were unable to contact the nominated FSM after the service user had completed therapy. All seven recruited FSMs who finished therapy completed post-intervention assessments. Two staff members did not complete post-intervention assessment: one did not want to because the client had relapsed and they had not been in contact with them for a while; and the other staff member was unavailable because of illness.

Thirteen family units were not approached to complete 3-month follow-up assessments as a result of delays in therapy delivery (see Chapter 5, Therapist recruitment). Of the participants approached, one service user dropped out as they no longer wished to be contacted, one refused to complete assessments and two did not complete assessments owing to relapse. In addition, one nominated FSM did not complete follow-up assessments as they could not be contacted. All staff members and recruited FSMs who were approached completed the 3-month follow-up assessments.

Changes in family members/family support members during therapy

Six service users changed family members/FSMs during the study, but these changes were addressed early on in therapy. Two service users nominated their key workers to participate with them, but these key workers dropped out post baseline as they changed their place of work before the therapy started. One of these service users subsequently took part with a recruited FSM and the other took part with their CC. Two service users initially opted to take part with family members but then changed to take part with FSMs post baseline. One experienced a breakdown in their relationship with their family member before therapy started, so a FSM was recruited, who joined from session 3. The other family member withdrew owing to a family bereavement after session 1; there was then a suspension of therapy and the service user started therapy again from session 1 with a recruited FSM. Of the remaining two participants, one changed to a different recruited FSM from session 7 following a period of disengagement resulting from the service user’s alcohol misuse and the other service user changed from two recruited FSMs to a nominated FSM following a breakdown in therapy after session 3 (this service user used subsequently resumed therapy from session 1 with two new therapists).

Two other service users experienced a suspension in therapy and did not change family member/FSMs but changed therapists. One service user disengaged at session 5 and re-engaged at session 7 following a pause in therapy. The other disengaged after session 1 because he did not feel comfortable working with male therapists, and the family restarted therapy after being allocated female therapists.

Method

The CaFI fidelity measure is a modified version of the subscale of the Cognitive Therapy Scale for Psychosis,⁴⁶ which has been adapted for this study to account for the presence of two therapists and the relatives. Seven items were used: agenda setting, feedback, understanding, interpersonal effectiveness, collaboration, homework, and quality of CBT techniques. The second subscale includes components that map directly on to the CaFI therapy manual, along with two additional items from the FIPAS:⁴⁷ reducing criticism and conflict; and reducing overinvolvement. The third and final subscale comprised six items that rated the degree of cultural awareness or competencies of the therapists during the session. This subscale was based on the manual and existing cultural competency questionnaires and literature^192–196 (see Appendix 26 for the CaFI fidelity measure).

Fidelity was assessed by independent ratings of 10% of semi-randomly selected sessions, ensuring equal spread across therapist pairs and session number. This was achieved by initially randomly choosing a recording for each therapist pair from the sessions remaining unsampled and randomly selecting a recording. We randomly sampled from all remaining recordings until we had reached the desired 10%. Treatment fidelity and quality were also monitored via discussion of audio-recordings of sessions in supervision. Every 6 weeks therapists also submitted a session to the clinical supervisor who rated the recording using the fidelity scale and gave feedback to the therapists.

Results

Overall, therapists scored well on part I of our scale, which reflected skills in CBT for psychosis (Table 17). Scores on the understanding, collaboration and interpersonal effectiveness items were particularly high, whereas feedback and homework items were lower.

TABLE 17

Descriptive statistics for part I: general subscales

Part II of the fidelity scale reflects the nature of topics covered within sessions, as opposed to the quality of delivery. All of the components of the manual were represented at least once, with the exception of three subcomponents (plans are put in place to reward and evaluate goals; goals are evaluated and rewarded when achieved; and help family to tolerate slow change within sessions). The independent rater also noted that it was not possible to rate the expressed emotion items on the content fidelity scale when family members were not present. Subcomponents related to communication and to helping the family formulate their own role in maintaining stress and the service user’s problems in a non-judgemental way seemed to be particularly well represented at least on the basis of the random selection of recordings rated (Table 18). Although, the independent rater noted that when family members were not present during sessions, the therapy was more akin to individual CBT than FI, and some aspects of family work during the sessions were not listed in the content part of the fidelity scale (Box 11).

BOX 11

Examples of additional family work that could not be rated for fidelity

Part III of the fidelity scale reflects cultural competence. At least half the sessions rated demonstrated evidence of aspects of cultural competence assessed. In fact, four out of six competencies assessed were evident in all, or nearly all, of the sessions rated (Table 19). The independent rater did, however, note that part III of the fidelity subscale was particularly difficult to rate. Without detailed information about the families, it was difficult to know whether cultural awareness or a cultural narrative should have been present but was missed, or if it was appropriately omitted.

TABLE 19

Descriptive count of part III: cultural awareness items

The Barrowclough and Tarrier¹⁹⁷ FI model clearly influenced the therapists’ practice. Non-psychologists had limited experience implementing psychological therapies with complex client groups, and so struggled in formulating this complexity and intervening in a less circumscribed idiosyncratic way. The clinical psychologists were more experienced in formulating and adapting to complexity. Although they clearly drew on the Barrowclough and Tarrier FI model,¹⁹⁷ they also drew on other areas of expertise. For example, one lead therapist specialised in child and family work outside the study and another therapist was carrying out a more specialist course in cognitive analytic therapy. The independent rater also noted that therapists from non-psychological backgrounds tended to conceptualise psychosis in medical terms and used illness model language which did not necessarily facilitate discussion about other ways of thinking about mental health, such as including cultural appraisals and psychological factors.

Discussion of fidelity findings

Overall, fidelity assessment demonstrated that therapists were able to deliver therapy to protocol with good evidence of core skills in CBT for psychosis, as well as cultural competency and coverage of core components of the manual. A few exceptions may require increased emphasis in future training and supervision protocols, including opportunities for feedback, setting and/or reviewing homework, and reviewing and rewarding goals. The independent rater also highlighted some aspects of therapy that were evident in the recordings but were not listed in the second part of the fidelity scale, such as addressing problems not related to psychosis, managing communication outside the family and services, and understanding voices and ways of coping with these (see Box 11), which suggests that iterations of the measure may need to be more comprehensive. Expressed emotion items were not rated when the family were absent, which may also suggest that future iterations of the measure should include the clause ‘rate only if family member present’ prior to the expressed emotion items. Independent raters in future trials may benefit from more detailed information about family background and context to facilitate rating cultural competency subscales. Further consideration should be given to the role of FSMs and how to better utilise this resource within sessions.

Although we did not systematically assess therapists’ effects on the fidelity ratings because of the small number of therapist pairs, observations in supervision did suggest that therapists who had completed a doctorate in clinical psychology and were routinely providing therapy within other roles within the NHS, were more skilled in formulating and working with complex issues that presented in therapy. Observations by the independent rater also noted that non-psychologists presented a more medicalised concept of psychosis, which may partly reflect the predominance of this model within mental health services, but is nonetheless inconsistent with the ethos of the family therapy we aimed to deliver in the trial. Taken together, these findings highlight the benefits of the therapy being delivered by individuals with more specialised training in therapy or providing more intensive training and supervision to less-experienced staff.

Summary of findings

We successfully recruited our target of service users to CaFI, slightly exceeding our recruitment goal of 30. We recruited an additional participant because one had withdrawn before we ended the recruitment period and we had 30 spaces for therapy.

We also successfully allocated at least one family member or FSM to each service user. Half the service users who started therapy nominated a member of their family to join them in the CaFI intervention; around one-quarter (23%) nominated someone else known to them and one-quarter selected someone from the pool of recruited FSMs. One (4%) attended with both a family member and a nominated FSM. In a few cases, service users selected a relative or CC (nominated FSM) to take part with them, but they could not work with the service user because of a reported lack of interest or work commitments. There were a couple of cases where service users started the sessions with their chosen relative/FSM but certain circumstances (e.g. bereavement) meant that the relative/FSM dropped out of therapy. In both situations, service users were given the opportunity to work with a recruited FSM, which highlights the benefits of this approach as it enabled service users to receive CaFI who otherwise would not have access to the intervention.

As might be expected, the most successful method of identifying potential participants was face to face, with either the CRN CSO staff approaching service users or the CaFI researchers visiting clinical staff in their work setting; three-quarters of recruitment was done face to face with service users or clinical staff. Recruiting NHS service users can be a convoluted process as researchers must go through the service users’ CCs who have limited capacity to support research studies and refer service users on their caseload. In the present trial, this proved to be a lengthy process and led to delays in reaching those service users who were interested in the research or receiving complimentary therapy. Often RAs were required to wait for a couple of months before they could attend clinical team meetings and CCs would not respond to e-mails or return telephone calls. The CRN CSOs working in inpatient and rehab wards were, however, able to access service users face to face, which highlights the need for CRN support in mental health trials.

Most people (84%) who were identified proved to be eligible for the study. Reasons for ineligibility included lacking capacity, assessed as being ‘high risk’ and not being of African-Caribbean origin. NHS staff often incorrectly referred people of Black African ethnicity, which may highlight problems with the recruitment materials and language use (i.e. anecdotal feedback suggests that ‘Black Caribbean’ is less confusing than ‘African-Caribbean’, which could be mistaken for both Black African and Black Caribbean). It may also point to a lack of cultural understanding among clinical professionals about the diversity of Black cultures and the ethnic background of their clients. The research team have previously conducted audits at the host NHS trust (MHSCT) and found that ethnicity is often recorded incorrectly. The conversion rate from eligibility to consent was 42%, and the most common reasons for not providing consent were lack of interest, being too busy, or it being the wrong time. Compared with non-consenters, those who consented to the study were slightly older and more likely to be women.

In our study protocol,¹⁵⁸ we planned to recruit from three types of setting: community settings, rehabilitation units and acute inpatient settings, with 10 services users in each setting. We successfully recruited from all three settings, but the spread across them was unequal: 21 were living in community settings (17 CMHTs, three EIS and one third sector), seven in rehabilitation units (including 24-hour staffed supported housing and hospital units) and three in acute inpatient wards. This may be an accurate reflection of the distribution of people meeting the criteria in these different settings. However, there were high numbers of referrals via the CRN from acute inpatient settings, presumably because the CSOs were located in these settings and could approach service users face to face and monitor new referrals. We found these acute settings more difficult to recruit from, because people tended to go in and out of them quite quickly, and some service users had moved on by the time we contacted them for the consent visit. Others were too ill to participate or lacked capacity to provide informed consent. In such cases, RAs/CSOs would approach the service user once the CC confirmed that they were well enough. It may therefore be helpful to gauge interest from service users receiving acute care, but to engage them in therapy only when deemed appropriate and when they have the capacity to participate.

Of the service users who entered the study, two-thirds were male and all were of African-Caribbean origin; the mean age was 43 years. Reflecting national census data,¹⁴ most were born in the UK (79%), with around one-third being of mixed ethnic background, one of the fastest growing ethnic categories. Most were single (81%) and over half lived alone (56%), reflecting previous reports of the impact of schizophrenia on social networks.¹⁹⁸ This finding also perhaps underscores the need to identify FSMs to work alongside service users who may be socially isolated. All but three service users had schizophrenia spectrum diagnoses; these people were recruited from EIS and, therefore, had not yet received a diagnosis (but this was confirmed by a psychiatrist to be non-affective psychosis).

Of the 31 service users who consented to take part in the study, 27 completed the baseline measures and 26 started CaFI. Among the 26 family units who started therapy, there was a high completion rate: 24 (92.31%) completed treatment, with completion defined as attendance at all 10 sessions by the service user, the family member or both. The mean number of sessions attended by family units was 7.74 (SD 3.81) out of 10 sessions. The number of family units who completed CaFI within the expected therapy window of 20 weeks was 12 out of 24 (50%) and the mean time to complete therapy was 23.27 weeks.

The therapy was delivered by therapist pairs, of which three were led by psychologists and four were led by CCs. Completion rates were the same for psychologists (12/13) and non-psychologists (12/13). The time taken to complete the therapy was longer for the non-psychologists (29.62 weeks) than the psychologists (16.92 weeks). These findings are supported by literature on the implementation of FI, which suggests that therapists need to have ring-fenced time rather than taking on therapy as part of their care co-ordination role.^33,161

The outcome measures are suitable for use in a future clinical trial, but they may need some adjustment for skewness. We were successful in collecting outcome measures from therapists and for most of the service users and family members who completed each stage of the trial. We report the variability of the measures for use in estimating sample size for a future trial.

Recommendations for the design of a future trial

• The identification of service users to approach for the trial took more time than we had anticipated. Although face-to-face approaches are more labour intensive, a future trial should concentrate on these methods of recruitment, as they were the most successful in this feasibility trial. This is perhaps because of previous adverse relationships and limited experience or knowledge of therapy in this group.
• Consent rates were fairly high in community settings, but gaining consent was much more difficult in acute settings as a result of high turnover, acute mental health state and lack of capacity. A future trial involving acute and rehabilitation settings should identify greater numbers of service users to allow for this drop-off. Recruitment efforts at acute sites should involve gauging interest of eligible participants and following them up when they are deemed well enough to engage in therapy.
• We were able to recruit families, nominated FSMs and a pool of recruited FSMs. All three types of families were taken up by service users, demonstrating the feasibility of delivering CaFI with FSMs. The findings from the fidelity study suggest that further work is needed to determine whether or not this represents a different intervention. Nevertheless, it is important to note that half of the service users who started therapy would not have been able to participate without FSMs, suggesting a role that FSMs may have for other socially isolated groups beyond CaFI (e.g. the forensic population).
• Most family units completed CaFI, which suggests that it was acceptable, and this is reinforced by the acceptability findings that will be discussed in Chapter 7. However, only half completed treatment within the therapy window of 20 weeks and, in particular, therapy delivered by non-psychologists took a longer time, suggesting that in a future trial we may need to consider additional training and supervision if non-psychologists are to deliver the intervention. We need to take greater measures to ensure that staff members have ring-fenced time to provide therapy, which involves them taking a reduced caseload of care co-ordination clients. Although this ring-fenced time was agreed by managers at the outset as part of the present study, these boundaries were not always adhered to. This was partly because therapy was only a small part of each of the initial lead therapists’ role, and all therapists were still trying to engage a relatively large caseload of other clients.
• Although reasonable levels of fidelity were achieved, qualitative feedback from supervision suggested that, compared with less experienced therapists, those who were more experienced in delivering therapy as part of their routine job roles demonstrated greater skill in case formulation and working with the complexity presented by some of the families in the study. This suggests that future studies need to recruit experienced therapists or provide more intensive training and supervision to less experienced staff. Although these training requirements have implications for delivery in the NHS, it is noteworthy that the psychologists in the study were all relatively newly qualified (< 3 years post qualification) and had worked at no higher than band 7 in their other posts.
• The methods of data collection adopted in this feasibility trial achieved high rates of completion at baseline and post intervention. We also achieved a high proportion of responses at 3 months post intervention, although the number of participants who reached that point before the trial ended was small (due to delays in therapy delivery) and, as such, not all participants could be approached for 3-month follow-up assessments. This suggests that the outcome measures were acceptable and feasible to collect from service users, families, FSMs and key workers.
• One of the anticipated outcomes of CaFI is an improvement in the relationship between the service user and their key worker, and this should be measured as one of the outcomes of a future RCT. However, we identified that 3 out of the 31 services users in this feasibility study changed their key worker during therapy and, therefore, the post-intervention and follow-up assessments were completed by a different key worker. Further thought is needed about how this discrepancy can be assessed in future trials. Additionally, some items on the WAI were not deemed acceptable by service users and key workers, which led to missing total scores. Future trials should identify whether or not these items should be included and how to manage missing data for single items.