The focus of this research was KIT agents (NHS and university-based staff in knowledge brokering roles) and the work they do in the knowledge mobilisation process with the health-care practitioner and management community. The study aimed to provide insight into the outcomes and impact of KIT agents’ knowledge brokering with NHS managers and clinicians.
The research used an in-depth qualitative case study approach, focused on a sample of KIT agents drawn from across England and Wales. Data were gathered from interviews, audio-diaries and observations. By examining cases (KIT agents) from a number of discrete initiatives designed to facilitate knowledge mobilisation, the study aimed to reveal how these endeavours worked in practice and how benefits can be maximised. To address one of the research questions, we used a consensus method in a meeting of experts (nominal group technique).
Study design
Theoretical frameworks
Within the case study design, data gathering was shaped by Kirkpatrick’s framework,198 which provides a useful structure for ensuring data collection beyond immediate reactions to an initiative and has been widely used in business and education.199 The original Kirkpatrick framework comprises four levels. Level 1 is concerned with assessing the participants’ reactions to the activity; for example, did they think it was relevant to their needs? Level 2 relates to learning gains (knowledge acquired by the practitioners). Level 3 focuses on behaviour change as a consequence of participation (application of knowledge to practice). Level 4 is about impact (what difference changed behaviour makes). This framework fits well with social marketing theory and its focus on behaviour.
In earlier work,62 we modified the framework to address known limitations199,200 and to maximise its relevance to innovation and knowledge transfer in the NHS. These adaptations recognise that reactions, learning gains and behaviour change all contribute to outcomes and are themselves ‘impacts’; that these ‘impacts’ are linked to each other and are not arranged hierarchically; and that the processes and dynamics of initiatives (including motivations) and the wider context are important.62
Our approach to data analysis is described later. In this, we drew on other classifications or frameworks in the literature, notably Walker et al.’s128 work on factors affecting organisational change and the PARiHS framework.201
To inform our understanding of the consequences and implications of our findings, we used social marketing theory. Social marketing is defined as ‘the systematic application of commercial marketing concepts and techniques to achieve specific behavioural goals relevant to the social good’.202 It is focused on understanding (or having ‘insight’ into) why people (e.g. NHS managers) do what they do now (in relation to innovation and knowledge) and what ‘competition’ the new behaviour faces. Social marketing theory recognises that this may vary by subgroups and may require different kinds of support (‘segmentation and targeting’). These insights can be used for ‘creating attractive exchanges’,203 which can encourage the effective uptake of the new behaviour. Part of this is taking account of, and most likely modifying or adding an ‘offer’.203 The strength of such an approach is that it starts with investigating KIT from the perspective of people who are being asked to do it. It seeks to understand the meaning they attach to it and the barriers to, and enablers of, doing it, without attempting to prejudge issues. The approach allows factors that challenge or enable KIT to be identified and context-specific solutions to be developed. Do different managers, clinicians, innovations, questions and health-care organisations require different activities or transfer pathways? What are the implications for practice? For example, managers with limited access to information technology and statistical support might find it impossible to apply modelling techniques, even if they know about them. The solution lies in improving hardware and support, not in communicating research findings differently. To give another example, to enhance knowledge mobilisation in a setting in which a KIT agent’s line manager expects them to ‘deliver’ without senior support may need a solution that focuses on the line manager and not the agent.
Advisory group
A project advisory group was established that comprised NHS managers, chief executives, representatives of the HSDR programme, academics with expertise in knowledge mobilisation (including from overseas), patient representatives and other partners. These are listed in Appendix 2 along with the main focus of each meeting.
Primary communication with the advisory group was by teleconference, preceded by an e-mailed agenda and supporting papers. Nine meetings were held over the course of the study. The advisory group helped to resolve practical issues and acted as a sounding board for sampling, data collection and analysis decisions. Most notable was its contribution to the discussions of early findings. Throughout the project, we presented drafts and questions to the advisory group members who used their knowledge and experience to inform our way forward. As a group, they provided a strategic voice for service users and service leaders. By facilitating the ongoing validation of research findings, the advisory group enhanced the credibility and relevance of the research project and contributed to early feedback of findings to stakeholder communities.
Patient and public involvement
As this project arose in response to a specific HSDR programme call for ‘research to improve knowledge transfer and innovation in health-care delivery and organisation’, patients and the public were not actively involved in identifying the research topic.204 Public involvement was defined broadly in the call to include ‘local communities . . . members of the public, users of services, carers and minority ethnic groups as well as healthcare practitioners and managers’.204 Our proposal was developed with the direct input of the research networks lead (de Pury), the director of an AHSP (Denman) and a health board manager (Howell). They made a significant contribution to prioritising and adding detail and contextual understanding to the objectives. Their views and opinion of their needs regarding innovation and knowledge transfer shaped the detail of the questions we asked.
Users are motivated to take part in research management when they feel they can make a contribution given their own expertise, experience and interest. For this reason, we focused our inclusion on patient representatives in the advisory group and the nominal group, in which discussion could be informed by their first-hand experience of services.
Data collection and sampling
We describe data collection methods in relation to three parts of the study: the initial mapping and typology development; the case studies; and the nominal group.
Mapping and typology development
For the national mapping of the KIT intentions of the networks (AHSNs and SEWAHSP) and the development of a typology of KIT agents we collected data from desk research and telephone interviews.
Desk research (June–August 2013)
The 15 AHSN prospectuses and business plan documents (versions submitted to NHS England for licensing in 2012) together with the SEWAHSP’s Five Year Strategy31 (July 2012) were reviewed to understand their interpretation of knowledge mobilisation and their planned approach and, in particular, their intentions to engage knowledge broker roles. The documents were scrutinised and searched for the terms ‘fellow’; ‘associate’; ‘lead’; ‘broker’; ‘agent’; ‘manager’; ‘boundary’; ‘transfer’; ‘exchange’; ‘span’; ‘connect’; ‘linkage’; ‘mobilise’; and ‘gap’.
Telephone interviews (November 2013–February 2014)
In order to supplement the initial mapping of knowledge mobilisation intentions, as set out in the documentation of the respective networks, the managing directors of the networks were approached by e-mail. The initial contact described the project’s aims and requested a 30-minute conversation, either in person or over the telephone, to understand their approaches to knowledge mobilisation. This was followed up with support staff to schedule the discussion and provide further background on the study’s aim and objectives. Interviews were conducted using a semistructured interview schedule (see Appendix 3).
Fourteen of the 16 network managing directors (including SEWAHSP) agreed to share their regional approach to knowledge mobilisation. Most managing directors stated that they were still in the planning and set-up stages of establishing their organisation and, therefore, could discuss their intentions around the use of KIT agents but not specific individuals. Eleven telephone and three face-to-face meetings were held between November 2013 and February 2014, ranging in length from 20 minutes to 1 hour. Twelve of the 14 discussions were held with the managing director and the other three were held with a named lead in the region. Two managing directors out of the 16 did not respond to multiple attempts to secure a discussion.
Appendix 4 summarises the main findings of the document review stage and the results are described in Chapter 4.
Case studies
Sampling
Using data from the intentions of the AHSNs and SEWAHSP, we purposively sampled five diverse sites, from which we selected case study KIT agents. The five networks were selected on the basis of:
stage of network development (e.g. ranging from de novo to well established)
diversity in regional research infrastructure (e.g. established links with CLAHRCs/no CLAHRCs; AHSCs/no AHSCs)
planned KIT roles in the region (e.g. part of the core team, secondments, fellowships)
geographical diversity (largely urban/rural and north/south representation)
willingness to engage with the project.
Our target population was KIT agents across England and Wales, that is, individuals who facilitate the mobilisation of knowledge to practice, knowledge brokers who link NHS managers/clinicians and the developers of knowledge and innovation. In the national mapping exercise we delineated definitions and expectations of the KIT agent role. From this, we drafted a typology of KIT agent roles, which we used to inform the identification of our individual case studies. In making our purposive selection of 13 KIT agents we actively sought variation of role, sampling within 4 of 15 AHSNs in place in 2013 and the first AHSP in Wales. We selected a number small enough to enable in-depth study of the work and impact of these KIT agents over time and yet provide good geographical coverage and system contrast. As a qualitative study, the sample size was necessarily small to enable richness and depth of data gathering and analysis. The specific innovations and knowledge that the agents helped to mobilise was diverse. This diversity enabled the detection of content-specific knowledge transfer challenges, as well as generic mechanisms that support transfer and mobilisation.
Data collection
Detailed case studies of KIT agents and their work were the source material for describing the roles of KIT agents, their linkages, relationships and engagement activities, and reporting the outcomes and perceived impact of their work on practice and management. They also provided data that enabled us to investigate how KIT agents can be best supported. We collected data from observation of KIT events/meetings; recurrent semistructured interviews with agents and interviews with their key Links (NHS managers/practitioners); and, when possible, audio-diaries kept by agents over 4 months.
Observations
We aimed to observe two or three activities of each case study agent. These were at the invitation of the KIT agent and included showcase events, reports to workplace colleagues, presentations of research and meetings to discuss progress on initiatives. These diverse, non-participant observations varied in length from relatively short meetings (< 2 hours) to 1-day events. Larger showcase events lasting 1 day were attended by two members of the research team; others were attended by one. Field notes were made, recording who was involved, what happened, reactions and reflections. These events also provided opportunities for informal discussion with people with whom the agent linked. We collected copies of any documentation used during the event.
Interviews
Knowledge and innovation transfer agents
All interviews were audio-recorded and transcribed. An initial face-to-face lengthy interview was held with each of the case study agents. The schedule is presented in Appendix 5. In most cases, further interviews were conducted. These provided an opportunity to gain updates on specific activities and further explore outcomes and perceptions of impact. One agent withdrew from the study after the initial interview.
Line managers
We had contact with agents’ line managers as part of arranging access and most agreed to a one-to-one interview, face to face or by telephone, depending on their preference (see Appendix 6 for the interview schedule).
Links
We also conducted semistructured interviews with some of the key people with whom each agent linked. These Links were identified by the agents. Further details regarding the background of the Links are presented in , and it should be noted that these individuals do not represent the entire range of an agent’s network of contacts. The content of the interview varied depending on the nature and reason for the connection to the agent.
Information on line managers and Links
Audio-diaries
Knowledge and innovation transfer agents were asked to keep diaries of activities and audio-recordings of their reflections on events and brokerage meetings. We explained the purpose of the audio-diaries and how to make recordings at our first face-to-face meeting. All those who agreed to take part were requested to record an audio-diary of their experiences (via a personal digital audio-recording device provided for participants or using their own digital recording equipment if they preferred). These solicited audio-diaries captured participants’ sense-making in action as they told their stories both to the researchers and to themselves205,206 and commented on outcomes and the potential impact that the role had on others. In order to enable participants to set the agenda for the content of their narratives, we gave them one principal request: ‘Please tell us about something that has happened in your role as a KIT agent since the last time you left a message’. However, we also included specific questions in subsequent requests to follow-up on earlier recordings. Participants were requested to record at least one short (2–3 minutes) diary entry per week, over a period of 4 months. However, despite regular prompts via text messaging or e-mail, contributions were highly variable.
provides a summary of data collected from the case studies. Excluding set-up meetings, we conducted 50 interviews (23 with KIT agents, 22 with Links and five with line managers), 20 observations, and received 72 audio-diary entries (6 hours 20 minutes). This provided us with 46 hours of recorded data.
Overview of data collection for case studies
Nominal group
Sampling
A nominal group is a highly structured meeting of a panel of suitably experienced ‘experts’.207 The size of the panel is typically 8–12 experts. Our group comprised a purposive sample of eight people from England, Scotland and Wales. Through consultation with our advisory group, we identified the roles we wanted represented (knowledge brokers, researchers/academics, network/policy leads, information scientists, patients/the public) and drew up a long list of potential members, informed by our knowledge of the field and the literature. To secure this group, we approached 28 individuals in two main waves. First, we approached 12 individuals and gained agreement from four. One of the 12 did not respond, but all others replied and suggested alternatives we might contact if they could not themselves attend. Second, we invited a further 16 people: five agreed, two did not respond and the remainder gave apologies. All those who replied were interested but were unable to make the date either because of other commitments or workload. In the event, one of the nine expected attendees could not attend because of a hospital appointment and another was ill on the day. As this was an academic working in the field, we felt justified in including a member of the research team in the group, although this was not our original plan.
Most members of the final panel of eight had dual roles. The group included three knowledge brokers, three researchers/academics, two network/policy leads, a senior information scientist and a patient and public involvement (PPI) specialist.
Data collection
The panel took place on 10 June 2015 (10.00 to 15.30), in Cardiff, and was facilitated by the project lead. In advance of the meeting, the panel members were given background information. It was explained that what counts as knowledge or evidence can be broadly interpreted, including not only the results of research but also the analysis and use of an organisation’s data as well as knowledge gained from experience and practice. ‘Knowledge mobilisation’ was described as a term used to refer to the use of knowledge to develop practice and services and inform decision-making. It was explained that the process of mobilising knowledge is not simple and linear; rather it is contingent on relationships and interactions. Knowledge brokers were described as having an important role to play in this process. While they may include a diverse group of individuals (clinical and non-clinical based in various locations including universities, libraries, public health settings, primary and secondary care settings, health boards and trusts), what they have in common is that they provide a two-way link between those who have or develop knowledge and those who use knowledge (practitioners).
The steps of the nominal group process were:
Silent generation of ideas in writing – the leader read aloud the question that the participants were to answer. This was clarified or modified as required following discussion. Each participant was then given a worksheet with the question written at the top and asked to take a few minutes to write down his or her ideas. Discussion was not permitted at this point.
‘Round-robin’ feedback of ideas – the leader asked each member to contribute one of his or her ideas summarised in a few words. These ideas were numbered and written on a flip chart so they were visible to all members. The process continued until no further ideas were forthcoming. Discussion was not permitted during this stage.
Serial discussion of ideas – each of the ideas on the board was discussed in turn. The objective of this discussion was to obtain clarity and to air points of view, but not to resolve differences of opinion.
Preliminary vote – the participants identified their top six items using a weighted voting system. The participants were given six voting cards (two 3-point cards; two 2-point cards; and two 1-point cards). They selected the six ‘most important’ items from the total list and wrote these on the 3-, 2- and 1-point cards, according to the level of importance that they assigned them (with 3 being the most important). Thus, each participant submitted six cards; on each card was written an item number and a ‘score’ (3 points; 2 points or 1 point). The cards were then collected and shuffled to maintain anonymity, and the votes read out and recorded on a tally chart that showed all the items and the votes allocated to each.
Discussion of preliminary vote – a brief discussion of the voting pattern was permitted. The purpose of the discussion was additional clarification, but pressure to change others’ votes was not permitted.
Final vote – step 4 was repeated.
Ethics and approvals
After seeking advice from a number of local, regional and national quarters [Cardiff University Research and Commercial Division and the Research Governance Officer, the NISCHR Clinical Research Centre, Aneurin Bevan R&D office and Queries National Research Ethics Service (NRES) Health Research Authority], it was confirmed that the project was deemed a service evaluation and, therefore, NHS research ethical approval was not required. Research ethics approval was then obtained from a Research Committee of Cardiff University (reference number 20/08/13) and the project registered on the Welsh portfolio (16/10/13; number 15479). Prior to fieldwork, we obtained any required approvals from the AHSNs and from the health board for each of the agents in Wales.
We prepared participant information sheets and consent forms (see Appendix 7) and sent these to all participants in advance of data gathering. Data were gathered in confidence and all individuals and places associated with the case studies were anonymised for analysis and presentation.
Data analysis
The data from the mapping were summarised and developed into a typology of KIT agents, which is reported in Chapter 4 and Appendix 4. The results of the nominal group are detailed in Chapter 7. In this section we explain the process of data analysis from the case studies. The analysis of the field notes and documents, the interview transcripts and audio-diary data were both deductive (based on a priori coding themes) and inductive (emerging from the data). Data from all these sources were managed in NVivo10 (QSR International, Melbourne, VIC, Australia).
The process entailed a thematic analysis using an adapted framework analysis approach.208 This entailed developing a coding frame based on a priori themes derived from the research questions and wider literature supplemented by emergent themes identified through discussion with the research team and advisory group. We used Walker et al.’s128 classification of factors influencing change as the basis of our analytic framework. This groups factors into four broad categories that we applied to our agents: context (factors in the external and internal environment); content (what knowledge or change was being transferred or implemented); processes (actions taken by the change agents); and individual dispositions (attitudes and behaviours of agents and reactions to change). It would have been appropriate to adopt an alternative model such as the PARiHS framework.201 The overlap between these classifications has been shown (see Chapter 2, Enabling knowledge and innovation transfer). As the starting point for our analysis framework, Walker et al.’s128 classification had utility. However, we extended this to include outcomes and impacts, conceptual definitions and descriptive codes for background and role. We used Kirkpatrick’s framework198 to report outcomes and impact in relation to reactions, learning gains and behaviour change. The coding frame is presented in Appendix 8.
The process of data coding was undertaken by three members of the research team. To ensure consistent application of the coding frame, on four separate occasions we independently coded several pages of transcript and then compared the coding. We debated and agreed any variation. This led to the addition of subthemes and further clarification or elaboration of codes.
The way we distributed transcripts between the coders facilitated greater sharing of the data such that it was rare for a researcher to code the interviews they had conducted. Team and advisory group discussion of the analysis helped to integrate themes and further summarise findings.
In reporting, we sought to analyse the work of individual agents, broadly within Walker et al.’s128 classification, presenting a narrative of the agent and their work. These narratives drew directly from the agent and their Links’ accounts. Alongside this, we took a broader look across agents in order to draw out general findings and implications for how models of knowledge mobilisation can be helped to work in future.
Validation processes
We detail some of the strengths of the study in Chapter 8. Here we briefly detail the validation processes. Data, methodological and investigator triangulation209–211 were achieved through the use of multiple data sources, conducting both interviews and observations of the agent’s work, and having at least two members of the research team working with each case study. These steps provided a broader understanding of the agents and their work. Particularly important was the voice of others who linked with the agents. Representing a variety of relationships and interactions with the agents, these key informants provided their own perspectives on the value of the work of the agents. The observations were an additional means of corroborating and expanding on how the agents saw their work. Multiple team members’ involvement in the data gathering, analysis and reporting of each case study also allowed the work to be viewed from more than one standpoint.
Throughout the course of the project, a number of mechanisms helped us to discuss, validate and disseminate our emergent findings. We utilised a range of opportunities to discuss with stakeholders including health service managers and leaders (two of whom were included in our research team). Data were regularly presented to, and discussed with, the advisory group.
Another part of the validation process was through presentations to research workshops and conferences, notably the UK Knowledge Mobilisation Forum 2014 and 2015, the Health Services Research Network Symposium 2014 and 2015, a series of workshops and meetings to inform the knowledge mobilisation strategy for Wales, University Health Board Research and Development Conferences 2014 and 2015, and Fellows Connect meetings.
At the end of the fieldwork, we offered feedback to the agents and their network sites. These were taken up by two of the AHSNs (covering six agents) and the three sites in Wales. These meetings were discursive in nature and served to test out the validity or ‘truthfulness’ of our findings.
A final part of the verification process was achieved by drawing parallels with published studies in the field. Taken together, these validation mechanisms strengthen the transferability of our findings.
Caveats
No study is perfectly designed and no research runs exactly as planned. We briefly review each part of the study here; a more detailed discussion of study limitations is provided in Chapter 8.
We achieved our objective in the first mapping exercise and typology development on the basis of document review and discussion with 14 out of 16 leads. Ideally we would have liked to have spoken to all 16.
Although we recruited more than the expected 10–12 case study agents for this study, not all participated as fully as we hoped or in the way that we anticipated, and the number of data we have on each agent varies. Despite this, the number and quality of data we amassed made the job of systematic and thorough coding a time-consuming process. However, the variability means that we know more about the work of some agents and its effects than others, and we cannot make claims about the wider population of knowledge brokers. This is addressed to some extent by our process of validation, which increases our confidence in the trustworthiness and transferability of our findings. However, it is necessary to accept that analysis of interview data represents an account rather than observation of behaviour. We did not witness change in practitioners’ behaviour as a direct result of KIT agents’ activity. However, their accounts were largely corroborated from triangulating what agents claimed with the reflections of their Link interviewees and managers, and our observations of events.
From the nominal group we distilled important messages. However, the questions remain challenging, and there would be value in developing and extending the exercise to build on these initial findings.
Despite enormous effort and persistence, we were unable to complete one aspect of our study. We planned to run action learning sets with groups of knowledge brokers. These were designed partly as a means to ‘give back’ to our case study’s organisations, partly as a mechanism to disseminate early findings to stakeholders, and partly to learn more about the support needs of knowledge brokers. Case study agents, and those who participated in Link interviews, were invited to take part and invite others in their networks as appropriate. Invitations to attend the sessions were later made to a group of nurses engaged with knowledge mobilisation, public health staff with an interest in knowledge mobilisation, and R&D leads across Wales, without success. Although in most respects this omission does not distract from our main findings, we remain disappointed that despite considerable effort and interest from potential participants, commitments and other pressures prevented them from engaging with the activity.
Concluding remarks
This chapter has presented a detailed account of our research methods in studying the KIT role within the context of the AHSNs in England and the regional network in Wales. With several caveats noted, we next turn to our findings.
