Introduction
We held a 1-day consensus conference, inviting experts by experience and profession to synthesise data from phases 1A and 1B (see Chapters 2 and 3). Through discussion,137 ‘experts’ resolved areas of disagreement considered key to modifying the intervention.138,139 Consensus was reached on:
contents of the therapy manual and materials to support the delivery of CaFI
training needs of therapists and ‘proxy families’ (phase 2; see
Chapter 5)
key outcomes measures for the feasibility trial (phase 3; see
Chapter 6).
Methods
Consensus methods are used in health service research when there is complexity and little previous work.43 Historically, decisions in health care were made informally, an approach criticised for lack of scientific credibility, rationality and authority.140 Consensus methods provide a mechanism for improving informal group decisions.141 There are a number of approaches to building consensus, the most common being (1) Delphi studies,142 (2) nominal group technique143 and (3) consensus development conferences or panels, commonly referred to as ‘consensus conferences’ or ‘expert consensus conferences’.43
Participant recruitment
Despite ongoing debate about the expertise of experts,144 defining participants by expertise remains an essential feature of consensus methods.43 A cohort of local, regional and national experts (n = 24) was identified using a range of approaches.
Academics were recruited based on their track record of conducting and publishing national and international research pertaining to African-Caribbean people with schizophrenia and cultural adaptation.
Current and former service users who regarded themselves as being from African-Caribbean backgrounds (‘Black British’ and ‘mixed’ heritage) were recruited from the University of Manchester’s Service User Research Group, ACMHS, MHSCT and service user forums.
Carers and advocates were recruited from a number of Manchester carer forums and third-sector organisations. To ensure as broad a range of consensus as possible, we also recruited members of voice hearing networks and the ‘antipsychiatry’ movement.
Health professionals were identified by their expertise in working with BME groups in general and African-Caribbean people in particular. We specifically involved individuals with expertise in FI.
Police representation was provided by contacting Greater Manchester Police.
Procedures and materials
Interested individuals were provided with the PIS (see Appendix 14) and met with the RPM to determine eligibility, discuss their involvement and address any queries. Individuals were given at least 48 hours to make a decision about their participation before meeting with the RPM to sign a consent form (see Appendix 8 for service user example).
A full-day consensus conference was co-facilitated by the PI, RPM and co-applicant (JB) at the University of Manchester.
The day was divided into three sessions reflecting the emergent themes from focus groups (phase 1B). Accompanying discussions focused on the cultural specificity of the proposed intervention to agree:
content
outcome
delivery.
Findings from phases 1A and 1B were presented to the consensus panel using PowerPoint presentations (see Appendix 15), which also outlined:
the rationale for the study
the overview of the work conducted to date (phases 1A and 1B)
the purpose and structure of the consensus conference
an explanation of how ‘consensus’ would be determined.
Turning Point (version 5.3.1., Turning Technologies, Belfast, UK), an interactive audience response system, was used to facilitate data collection. Embedded within the PowerPoint slides, the easy-to-use polling software enabled participants to respond to questions using their handsets (‘clickers’), generating ‘real-time’ data. ‘Real-time’ voting and analysis was chosen in preference to other consensus methods such as the Delphi, which can lack transparency and is comparably slower.145 As votes were instantly translated into on-screen graphs, they were viewed by participants and used to generate further discussion and resolution when needed (e.g. when ranked items received the same score).
Reflecting our co-production approach, the views of ‘experts by experience’ (service users, carers and the community) were considered as important as expertise by professional standing.144 Each expert had one vote per item. All votes were considered equal and no individual had a casting vote. Voting ceased when all experts had voted.
Data collection
The findings from the focus groups were presented to the experts for consideration of the extent to which the proposed adaptations would improve the cultural specificity of FI for African-Caribbean people. For each aspect of the intervention, experts were shown the current content of FI alongside the additional elements focus group members identified as key to improving its cultural specificity (see Chapter 3). Consensus was reached via two methods: (1) discussion and (2) subsequent voting.
Discussion
Discussion, occurring in small groups of four to six and two larger groups (each comprising half the participants), preceded each round of voting. Small groups were a means of ensuring that dissenting and less dominant voices were enabled to contribute and to identify important issues that might have not previously been considered. Group discussions were recorded, transcribed and later revisited by the research teams to ensure that stakeholders’ disagreements and the process of resolution were documented. During discussions, participants were asked to record important items about cultural adaptation that were missing from those identified by focus groups via a series of Post-it® notes (3M, Berkshire, UK). Thirty-four Post-it notes were collected and discussed to determine whether or not they should be added to those generated from phase 1B prior to consensus being sought on agreed items.
Voting
Voting followed each round of discussion. Participants rated their responses to items that emerged from the focus groups and literature by ranking, ‘yes/no’ responses, or on seven-point Likert scales. In relation to Likert scaling, we decided to include a ‘neutral’ mid-point to enable voting by participants who did not feel strongly about a topic.146 Votes were presented back to the experts via graphical representation (bar charts), displaying numbers and percentages embedded within the PowerPoint. The presentation (see Appendix 15) was saved to capture votes for further analysis.
To familiarise participants with the voting system and stimulate discussion, we commenced the day by asking participants to vote ‘true/false’ in relation to the statement, ‘[T]here’s no such thing as African-Caribbean culture’.
Data analysis
As consensus does not necessarily denote full agreement,43 we agreed that consensus would constitute ‘near-unanimous agreement’ achieved by:
Results
Participants
The final sample comprised 22 experts (24 were recruited but two were unable to attend on the day). Participants included ‘experts by experience’ [services users and carers (n = 5); carers and advocates (n = 9), including the third sector and church] and ‘experts by profession’ [(n = 7) including academics, police, health professionals and service managers with expertise in schizophrenia in the African-Caribbean population].
Cultural specificity
shows that experts achieved near-unanimous consensus (between 81% and 95%) that all items generated by the focus groups and the literature would improve the cultural specificity of CaFI for African-Caribbean people: specifically, its content, outcome and delivery.
Summary of responses on the relevance of proposed changes to FI for African-Caribbean people
Content
Service user assessment
shows that consensus of between 85% and 95% was reached that all proposed additional items would improve the model’s cultural appropriateness. The highest level of agreement was in relation to ‘strengths and resources’, on which 67% of experts ‘strongly agreed’ and a further 28% ‘agreed’ that the items presented would make CaFI culturally specific for African-Caribbean people. There was no disagreement, with one individual (5%) recorded as ‘neutral’.
Family assessment
This item yielded the greatest dispersion of views. One expert ‘agreed somewhat’, two were ‘neutral’ and one ‘disagreed’ that the proposed adaptation would make this aspect of the intervention more culturally appropriate. This reflects experts’ discussions in which participants stated that issues such as family structure and hierarchy, causes of family tension (blame, criticism, rivalry) and previous experiences of services were common to all ethnic groups. However, consensus was reached that the suggested items were sufficiently specific to African-Caribbean families to warrant their inclusion (81% agreement).
Psychoeducation
This aspect of FI generated the greatest number of recommended additional items from the focus groups (eight in total). Items considered especially relevant for African-Caribbean people included:
better understanding of how the mental health ‘system’ works
information about rights in relation to police involvement
effects of stigma (within families, communities and wider society)
challenging stereotypes and misconceptions
understanding psychotropic medication – how it works, side effects
understanding the relationship with illicit drug use and medication/illness progression
culturally specific models of mental illness
better awareness of available resources (community and NHS).
Experts ranked these items numerically so that those ranked more important were assigned higher numbers. The cumulative value of rankings for each of the content areas is represented in by the weighted count. shows that items were ranked fairly evenly, with ‘how the “system” works’ (15%), ‘models of illness’ (15%) and ‘medication’ (14%) rated the top three most important aspects of psychoeducation in CaFI. Illicit drug use (8%) was ranked lowest.
Ranked importance of additional ‘psychoeducation’ content.
Focus group members indicated a strong dislike of the term ‘psychoeducation’. Experts chose ‘shared learning’ () from the available options (6 out of 21 voters, 29%). As ‘mental (ill) health awareness’ was voted a close second (5 out of 21 voters, 24%), further discussion ensued from which shared learning received the most votes by a show of hands (n = 15).
Votes for preferred alternatives to psychoeducation.
Stress management and coping
Additional items identified in phase 1B included managing conflict between service users and relatives (specifically in relation to ‘sectioning’) and between families and health-care professionals. Carers’ rights and responsibilities, as well as acknowledging carer burden and the fact that some people are simultaneously service users and carers, were regarded as important additions. Relapse prevention, crisis planning and developing more positive ways of coping were also suggested. There was 87% agreement among experts that these items would improve cultural relevance for African-Caribbean people.
Problem-solving and goal-planning
Experts were shown the diagram used to illustrate the 10 steps that service users, families and therapists work through to identify problems, goals and rewards (). As with focus group members, experts agreed that this was a good model. They also agreed that additional items, such as contingency planning, highlighting resources and focusing on progress versus failure, were important. In relation to the latter, celebrating success along the ‘journey’ to acknowledge even small steps was suggested as an important means of maintaining focus and momentum.
Culturally adapted FI content by session
Although focus group members endorsed Specific, Measurable, Achievable, Realistic and Timely (SMART) goals, experts recommended they should be made ‘SMARTER’ by adding ‘Evaluated’ and ‘Rewarded’. It was agreed that evaluating and rewarding goals should occur on a session-by-session basis, accompanied by evaluation of goal achievement against between-session tasks. End-of-therapy evaluation was regarded as important for reviewing and agreeing how to maintain gains.
At the end of this process, agreement was reached through discussion that the content of the 10 CaFI sessions would be as outlined in . However, in line with good clinical practice, experts highlighted that this should not be regarded as a linear process. Rather, the order and pace of delivery should reflect the needs of individual service users and their families.
Outcomes
Focus group members identified 10 desired outcomes. For CaFI, the top five outcomes were ranked as ‘better familial relationships’, ‘health and well-being’, ‘social functioning’, ‘reduction in relapse and readmission’ and ‘relatives’ positive attributions and attitudes towards illness’. As shown in , several of the outcomes had similar weighted counts, indicating that most were deemed important by experts. This is reflected in this quote from one expert about the difficulty of ranking inter-related items:
Ranked importance of potential outcomes from the CaFI.
I’m a bit torn between ‘social functioning’ . . . and also ‘coping with stress’ because I think with these topics some of them go hand-in-hand. But I think coping with stress is fundamental to staying well and hopefully not becoming unwell again, not being readmitted and also with activity and how you’re doing on a day-to-day basis so I don’t know which one’s [to rank] one and two really.
‘Knowledge about schizophrenia’ received the lowest ranking. Although some experts felt that this was an important outcome to measure, others disagreed:
The problem that I have with this is that it implies that if you rank something 10 or whatever it implies that you have fairly neutral feelings about it. I would actually positively object to knowledge about schizophrenia being an outcome measure in a study like this . . .
Following further discussion, experts agreed that in future studies, to examine CaFI’s effectiveness, ‘reduction in relapse and readmission’ should be a primary outcome and ‘relatives’ attributions and attitudes’ a secondary outcome. Interestingly, although originally ranked last, ‘knowledge about schizophrenia’ was agreed to be an important outcome that should be considered, with the proviso that a culturally valid version would be developed.
Delivery
In the final consensus conference session, experts were tasked with agreeing:
how best to adapt the delivery (vs. content) of FI to ensure that it suited the needs of African-Caribbean people
key approaches to supporting delivery of the intervention
training needs of therapists and ‘proxy family’ members.
Experts agreed that these three objectives were inextricably linked. Consequently, much of their discussion focused on CaFI’s ‘ethos of delivery’ and therapists’ ability to establish and maintain therapeutic relationships with members of this community in the context of a history of fear and mistrust of services.
‘Ethos of delivery’
Revisiting earlier discussion about ‘psychoeducation’, experts endorsed the overarching ethos of delivery as one of ‘shared learning’. Building on focus group discussions, they agreed that ‘shared learning’ must be a ‘three-way process’, involving service users, carers and therapists in mutual exchange of information. Experts agreed that the main aims of CaFI should be:
facilitating positive family relationships/interactions that enhance mental well-being for relatives and service users
enabling families to optimise interactions with services to foster engagement, thereby improving access, experiences and outcomes.
To achieve these aims, experts concluded that therapists must work collaboratively with families to realise the following related objectives, as detailed in the therapy manual (see Appendix 16):
develop more helpful explanations for symptoms (e.g. viewing service users’ experiences as ‘symptoms’ of mental illness vs. ‘deviance’)
facilitate better communication in the family (e.g. improve active listening)
develop more effective coping and stress management strategies, including self-monitoring
improve problem-solving skills using appropriate tools and resources (e.g. case studies and vignettes, exemplars from a weekly report from family)
promote recovery and relapse prevention, including identifying ‘relapse signature’
improve understanding of how ‘the system’ works and strategies for engaging and communicating with mental health services and partner agencies
promote strategies for engaging and communicating with mental health services and partner agencies.
Therapists’ cultural awareness, knowledge and skills
Although all items in Box 8 were regarded as important for delivering CaFI, experts agreed that a crucial aspect of delivery would be therapists’ cultural awareness and insights into the potential impact on engagement and alliance of their:
Key issues in delivering CaFI identified by focus groups
background (e.g. ethnicity, faith, social class, sexual orientation)
beliefs (potential misconceptions about ‘African-Caribbean culture’)
experiences (e.g. working transculturally).
This led some experts to suggest that therapists should have personal experience of African-Caribbean communities either as members themselves or through close personal contacts:
The first thing is to recruit therapists from the community, that’s the first thing. The second thing is if you’re not recruiting therapists from the community, to recruit therapists who have personal experience in the community or they have friends . . .
Academic professor
There were, however, dissenting voices. Some experts felt strongly that all therapists should be equipped to work with any individual, irrespective of their background versus ‘training up’ individuals to work with particular groups:
I find it offensive that we’re in this day and age and you think that only certain groups of people need to be aware of the groups of people that they’re working with. Large numbers of Black and minority ethnic groups are in mental health services so therefore it’s everyone’s responsibility.
Health-care professional (emphasis in bold)
Others suggested that an important aspect of a ‘shared learning’ approach would be enabling African-Caribbean people to raise White professionals’ awareness of the impact of additional dimensions of power (rooted in the historic legacy of colonialism and slavery) on the therapeutic relationship when working with African-Caribbean clients:
I think in terms of experiential work people from Black communities [we] have a very important role in changing the sensitivities of White professionals who may well not be aware of the power implications and the power differentials and how those are sedimented historically in our shared histories of colonialism and all the rest of it.
Advocate
Therapist training
As with focus group members, experts felt that a prerequisite for working with this community was having high levels of skill and expertise in family work, as the issues being brought to therapy were likely to be complex and potentially deeply ingrained because of long delays in receiving care and negative experiences.
Cultural competence training would be required to ensure therapists develop the confidence to explore issues that affect members of this community, such as racism and discrimination, which some therapists might find personally challenging. Training should address both contemporary and historical aspects that affect this community. These include the history and heterogeneity of the Caribbean islands as well as patterns of migration. In addition, knowledge and awareness of disproportionately high rates of schizophrenia diagnosis (and related theories), together with reports of negative experiences and outcomes for African-Caribbean people, were regarded as vital to enable therapists to understand the high levels of mistrust within this community and to work effectively to develop trusting relationships with families.
When asked how therapists should be trained, experts suggested that bespoke training needed to be commissioned as few mainstream NHS therapists would have the cultural competency required. They advocated the involvement of service users and carers in designing and delivering the training. It was acknowledged that it would be difficult to evaluate the effectiveness of such training in the absence of a cultural competence framework. They endorsed our proposed fidelity study (see Chapter 6, Fidelity study) and the collection of qualitative data from the perspectives of service users, family members and ‘proxy’ family members and therapists.
Family support member/’proxy family’ training
Experts concluded that the purpose of training (see Chapter 5, Family support members) for ‘proxy family’ members was not to develop expert knowledge in psychosis and schizophrenia. Instead, it was agreed that a basic introduction to serious mental illness in general and to schizophrenia in particular was appropriate because many family members had at least some knowledge about mental illness. Furthermore, it was felt that this level of knowledge was required for interested individuals to make informed choices about becoming FSMs (‘proxy family’ members).
In addition, experts suggested that FSM training should focus on key aspects of governance (e.g. confidentiality, health and safety) and measures to safeguard their own mental health and well-being, alongside clarifying the scope and nature of the role and boundary-setting.
Evaluation of consensus conference
Amorphous processes such as consensus conferences may achieve high degrees of statistical consensus without reaching ‘genuine agreement’.149 Understanding the differing positions and degrees of polarisation/agreement between experts by experience versus profession may be more significant than statistical consensus.150 We were unable to explore such differences on the day. Providing a period of discussion prior to voting enabled ‘outliers’ to voice their views and ensured that dissenters did not withdraw from the process.151,152 Nevertheless, it is possible that items achieving statistical consensus represented the safer ‘middle ground’ of opinion. Using predetermined questions/items based on findings from the literature review and focus groups might have made disagreement difficult for some members of the conference.
Feedback and evaluation indicate that this was not the case here. At the end of the consensus conference, we asked participants if they had been able to contribute to consensus-making. Of the 16 evaluations returned, 94% (n = 15) agreed/strongly agreed that they had ‘been able to contribute views and ideas’, that ‘their views had been heard and valued’ and that the ‘people here today were the right group to develop talking therapy for African-Caribbean people’. When asked whether or not any other voices should have been included, 50% agreed/strongly agreed and 43.8% disagreed. Participants suggested that there should have been greater involvement of the criminal justice system and psychiatry. Both groups were represented on the day. This may reflect our decision to use only first names (vs. professional titles or labels) to reduce social distance between conference members and minimise the influence of social and professional hierarchies.
Discussion and conclusions
Our expert consensus conference marked the culmination of a process involving three inter-related studies to determine how to adapt FI to increase its cultural specificity for African-Caribbean service users diagnosed with schizophrenia and their families. This is important because, although previous studies had identified components of the cultural adaptation process, none had identified how these should be applied.
Having identified essential components of cultural adaptation, we applied them systematically to the content of an existing FI model, seeking key stakeholders’ views about what was needed to improve its cultural specificity for African-Caribbean people. Findings from current literature (phase 1A) and focus groups (phase 1B) were synthesised and presented to a range of experts in a specially convened consensus conference in which they agreed the content, outcome measures and mode of delivery of our CaFI. These experts by experience and profession achieved robust consensus (≥ 80% agreement) on all aspects on the intervention.
A central feature of this co-production approach was achieving genuine service user and community engagement and ‘buy-in’ to the process of developing CaFI. This was a significant achievement given the endemic barriers to fear, mistrust and lack of engagement153 within this community. It was important that the panel included African-Caribbean ‘experts by profession’ such as psychologists, counsellors, psychotherapists, RMNs and voluntary sector organisations. We also invited those with ‘antipsychiatry’ views and members of voice-hearing networks to ensure that consensus reflected the widest possible range of stakeholder perspectives.
Participants were indeed very robust in sharing their views, particularly in relation to the validity of concepts like ‘schizophrenia’. Other areas of contention concerned questions about the pros and cons of ‘ethnic matching’ of therapists and families and whether or not training therapists in ‘cultural competency’ is achievable. Participants debated the value of endorsing such concepts in the absence of frameworks to evaluate them. These discussions and the fact that we were able to resolve them through debate, demonstrate our success in managing difference to achieve genuine consensus on key CaFI deliverables. Specifically, we:
agreed content of CaFI and the basis of the therapy manual to support its delivery (see
Appendix 16 for summary of therapy components)
identified primary, secondary and other desired outcomes
identified training needs of therapists and ‘family support’ members as the basis for co-producing bespoke training with service users and carers.
The cultural adaptation process we have described was undertaken with African-Caribbean service users, families and community members in collaboration with health-care professionals. Our approach has significance for developing interventions with other marginalised communities. Co-producing interventions with, versus for, marginalised groups has the potential to reduce barriers to mistrust and improve uptake and engagement of evidence-based care, thus reducing inequalities.
