Chapter 2Methodology and data sources

Literature review

An extensive search and review of the literature on existing scales used to measure relational aspects of care was conducted. Literature was accessed using the Web of Science online database, as well as the online search platforms PubMed and Google Scholar (Google Inc., Mountain View, CA, USA). The following search terms were used to identify relevant literature:

• relational aspects of care
• emotional care
• patient–doctor relationship
• patient–doctor communication
• patient emotional care
• relational communication.

In addition to databases and search engines, the research team used a snowball strategy to identify seminal literature based on the citations provided in the previously accessed literature.

Focus group and interviews

To understand what patients thought constituted relational aspects of care, one focus group with individuals who had visited an A&E department in the last 3 months was conducted on 24 September 2014. Participants were recruited through adverts in local newspapers. Although seven participants confirmed their participation on the telephone, only four attended the meeting. Of the four participants, three (75%) were females and all (100%) described themselves as white British. Focus group participants ranged in age from 28 to 65 years. The focus group was conducted over the course of 2 hours and included a discussion on what constitutes relational aspects of care and a ranking of their importance using a sort card exercise. Questions asked in the focus group were designed to explore what different aspects of care mean to patients in terms of their recent experiences. As little detail on what matters most to patients is provided in existing literature it was also important to explore this. Questions asked were:

• What comes to your mind when you think about emotional care or relational care?
• Sometimes it seems that one or two hospital or A&E department staff members are especially good with patients all around. Did you get this sense during your recent A&E department visit? If so, what made the care these staff provided stand out?
• During your recent A&E department visit, were these relational or emotional aspects of care present or addressed by staff?
• Why are these relational or emotional aspects important when you receive treatment?
• Which components are most important to patients’ overall experiences of care?
• What relational or emotional aspects cannot be missing from a good care experience?

Eight face-to-face interviews with older adults who had recently stayed overnight in hospital were conducted in lieu of a second focus group during the week commencing 13 October 2015. Three interview participants were male (37.5%) and aged 75 or 76 years. Five interviewees were female (62.5%) and between the ages of 75 and 85 years. All participants identified themselves as white British. Interviewers utilised the same discussion questions and prompts as in the focus group. Each interviewee also completed the sort card exercise. Interviews lasted between 40 and 50 minutes each.

For the sort card exercise, 19 cards were prepared prior to the focus group and six statements were prepared during the focus group, based on aspects of relational care that arose during discussions. The previously prepared statements were based on relational aspects of care identified in the following existing survey instruments:

• the PEECH measure³²
• the 2013 NHS Adult Inpatient Survey³⁶
• the 2012 NHS Emergency Department Survey³⁵
• the CARE measure³¹
• the 27-item Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) questionnaire
• the General Practice Patient Survey (GPPS).³³

The exercise was completed by the eight interviewees and by two groups of four focus group participants. Therefore, data were available from a total of 10 card sort activities. During the exercise participants grouped the 25 statements into three categories, namely ‘most important’, ‘quite important’ and ‘least important’. The 25 statements and their importance rankings were entered into a Microsoft Excel^® 2013 file (Microsoft Corporation, Redmond, WA, USA). The following values were assigned to each category: 2 = most important; 1 = quite important; and 0 = least important.

The focus group discussions and interviews were transcribed verbatim. Three researchers reviewed the transcriptions independently and created an initial coding framework. Comparison of frameworks highlighted extensive similarities among themes and categories. The framework with the least specific categories and which utilised participants’ own words for themes to the greatest extent was chosen for analysis purposes. It was anticipated that the framework would evolve during the coding process; however, only three additional categories were added for two themes during the coding process.

Data from the nine transcripts were then coded by assigning or coding each section or newly shared thought to the best-fitting categories or themes. Where sections of the transcripts appeared to fit under multiple themes or categories, they were coded for each. The coding process was conducted independently by two researchers, each using the software package Nvivo, version 10 (QSR International, Warrington, UK). Inter-rater reliability was computed and found to be 97%.

Factor analyses

To build on the understanding of relational aspects of care gained through the literature review, focus group and interviews with recent patients, factor analyses were conducted on patient experience data collected for the 2012 NHS Emergency Department Survey³⁵ and the 2013 NHS Adult Inpatient Survey.³⁶ The purpose of these secondary data analyses was to identify a cohesive set of existing questions that measure relational aspects of care in the national surveys. At present, national inpatient and A&E department surveys do not specifically set out to measure relational aspects of care; however, they do include items which may be considered to tap into many of the aspects of relational care identified in our preliminary literature review and qualitative analyses. Survey items included in the analyses were selected based on themes of relational aspects of care identified in the literature review. Themes included patient-perceived level of security, knowledge and personal value during their hospital stay or visit.³²

Prior to conducting factor analyses, multiple imputation⁴⁰ was conducted to replace missing and unscored responses to all items. For all survey questions, missing data ranged from 2% to 54%. However, the majority of survey questions had missing responses of ≤ 5%. Five imputations were completed for each survey data set. Next, factor analyses were conducted utilising data from 15 questions on the NHS Emergency Department Survey and from 18 questions on the NHS Adult Inpatient Survey. Data for both surveys were collected from separate patients and could not be linked across the two surveys. A polychoric correlation matrix for each survey was generated from the imputed data sets.

Exploratory factor analysis was then conducted on each data set using Factor v9.3 (Rovira i Virgili University, Tarragona, Spain). Each matrix was subject to two tests to determine the likely number of factors appropriate in each case.⁴¹ The first test was parallel analysis, in which data eigenvalues are compared with those that would be obtained from random data. The second was Velicer’s minimum average partial (MAP) test, which seeks to minimise the amount of residual variance after different numbers of factors are extracted. The indicated numbers of factors were extracted using unweighted least-squares factoring and rotated (where appropriate) using Promin oblique rotation. The resulting loadings, communalities, residuals and fit statistics were examined to determine the suitability of the solution. Items were then allocated to scales on the basis of their loading patterns.

Scale analysis was conducted on the item sets for both surveys by fitting a single-factor model to each proposed scale. The resulting loadings were then used to analyse the overall reliability of the scales and the contribution of individual items. Reliability was evaluated using Cronbach’s alpha and McDonald’s omega (a more general coefficient of which alpha is a special case).⁴² Item contribution was evaluated using McDonald’s item information index (the ratio of communality to uniqueness).⁴²

Composite scores

To select six case study sites, we first sought to identify high- and low-performing trusts on the 2012 NHS Emergency Department Survey³⁵ and the 2013 NHS Adult Inpatient Survey.³⁶ Both were the most recent relevant national survey administrations for which complete data sets were available. Composite scores were computed based on data collected using existing survey questions that capture relational aspects of care.

Those survey questions that were identified to most closely measure relational aspect of care in previous factor analyses were utilised to compute composite scores. Composites were computed for all respondents who had answered at least 60% of the selected questions. For the 2012 NHS Emergency Department Survey,³⁵ only one composite score was computed for each trust as the previous factor analysis indicated one underlying factor. For the 2013 NHS Adult Inpatient Survey,³⁶ factor analyses indicated two underlying factors of relational aspects of care. Therefore, two composite scores were computed for each trust on the inpatient survey data, based on question loadings for each factor. To allow for comparison across trusts, an average score was calculated for each trust based on the three composite scores. Trusts were then sorted by the average score to identify the top and bottom one-third of the distribution.

Recruit six case study sites

All trusts falling within the top or bottom one-third of the composite score distribution described in Composite scores were initially contacted and informed about the opportunity for participation as case study sites; however, sufficient and timely interest was not generated using this approach. Therefore, trusts falling within the middle of the ranked distribution were also contacted. At the same time, reminder messages were sent to the trusts that were initially approached.

All trusts were sent an information sheet describing the project and the case study participation opportunity. Individual telephone conference calls were held with 26 trusts to further inform them about the purpose and benefits of the study and time and resource requirements for their participation.

Following the telephone discussions, interested trusts were asked to confirm in writing their interest in participating in the study as a case study site. The first three trusts falling below and above the median of the composite score distribution from which written confirmation was obtained were selected to participate in this study.

The six case study sites participating in the research were Hinchingbrooke Healthcare NHS Trust, Milton Keynes University Hospital NHS Foundation Trust, North Cumbria University Hospitals NHS Trust, Northern Lincolnshire and Goole NHS Foundation Trust, Poole Hospital NHS Foundation Trust and Salisbury NHS Foundation Trust. More details are presented about the case study sites in Chapter 3.

Develop draft survey instrument

The survey items identified as most closely related in the factor analyses were mapped to the themes of relational aspects of care identified in the literature, focus group and interviews. Additional survey questions were subsequently developed to ensure adequate coverage of each theme and that a pool of at least five survey items was available for each theme. This resulted in a pool of 62 potential survey items.

An expert advisory group, comprising nine members, was consulted at various stages of the project. Of these nine members, seven were female and two were male. The group included two patient and public representatives, two academic experts and two voluntary organisation representatives with a special focus on ageing, as well as three members of NHS organisations (specifically, one deputy chief nurse and two heads of patient experience).

During the meeting, members reviewed the themes and associated draft survey items. Under each theme, the two most representative and easy to understand questions were selected for face validity based on the following criteria:

• Do the survey questions make sense?
• How well do the survey questions represent their theme?
• Which survey questions were easiest to understand?
• Do any survey questions appear to be duplicates for this theme?

This selection process ensured that each theme was addressed with at least two survey items, while also reducing the burden placed on survey respondents by a lengthy survey instrument. Suggestions were offered for making the survey easier to understand and operate on a tablet. The researchers incorporated all suggestions, focusing on the tablet-based survey process and question wording, into the draft survey instrument.

Apply for ethics approval

The East of Scotland Research Ethics Service (EoSRES) was assigned through the NHS Integrated Research Application System to review this research, including the instrument development process. After the draft survey instrument was developed and prior to cognitive testing, the survey instrument was submitted to the same Research Ethics Service for review as a substantial amendment.

Cognitively test survey instrument

The draft patient survey was cognitively tested in three rounds with a total of 30 patients. This cognitive testing provided the basis for finalising the survey instrument. The testing took place at the following three case study sites at the beginning of May 2015:

1. Poole Hospital NHS Foundation Trust on 6 May 2015
2. Salisbury NHS Foundation Trust on 7 May 2015
3. Milton Keynes University Hospital NHS Foundation Trust on 11 May 2015.

A specific probing approach, including preprepared and spontaneous probes, was used to test each survey item with patients.^43,44 The cognitive process of responding in terms of the model described by Tourangeau⁴⁵ was considered, seeking to establish consistency in:

• comprehension – people understand what the question is asking in a consistent way that matches the intended RQ
• retrieval – people are able to retrieve from memory the information necessary to evaluate their response to the question
• evaluation – people are able to use retrieved information to evaluate the question meaningfully, and do this in an unbiased manner (e.g. not simply acquiescing or providing socially desirable responses)
• response – people are able to match their evaluation to one of the available responses in a meaningful and appropriate way; the response selected adequately reflects the person’s experience.

Patients were approached on two wards and within the A&E department at each of the three trusts. Senior staff members directed the researchers to all patients who had the capacity to consent and were well enough to participate in a survey. Consenting patients then completed all survey questions, if necessary with help from a researcher. Assistance provided to respondents included reading questions and response options, entering responses or physically holding the tablet up for the patient to read. After each survey question was answered, a researcher asked two or three prepared follow-up prompts for each question. This question-by-question approach was selected as participants, who were primarily elderly patients, did not want to complete the survey and then return to discuss individual survey questions once they had already provided a response.

Specific probes were used to gauge respondents’ understanding of more complex words, such as condition, treatment and conversation.^43,44 In addition to these specific probes, if needed, spontaneous probes were also used to further assess patients’ understanding of survey questions and answer options.

Two examples of survey questions and corresponding specific probes are listed below:

1. If you have needed attention, have you been able to get a member of medical or nursing staff to help you?

   1. Yes, always.
   2. Yes, sometimes.
   3. No, never.
   4. I have not needed attention.
   What do you think is meant by ‘attention’?
   Who do you consider to be medical staff? Do you think they are different from nursing staff?
2. Has a member of staff told you about what danger signals regarding your condition or treatment to watch for?

   1. Yes, completely.
   2. Yes, to some extent.
   3. No.
   4. I have not needed this type of information.
   What does ‘watch for’ mean to you?
   What do you think is meant by ‘danger signals’?

A total of 38 patients were approached, of whom 30 completed the cognitive testing of the survey. Four elderly patients began the process but decided that they did not want to continue after a few questions and another four patients did not have the capacity to consent. Of the 30 participants, 13 were male and 17 were female. Twenty-eight patients identified themselves as white British. English was not a native language for two patients. One participant had dementia and a family member provided answers from the patient’s perspective for most survey questions. Twenty-one patients were currently inpatients in wards that provide care primarily to the elderly. The remaining nine patients were currently visiting A&E departments.

Volunteer training

Volunteer training sessions were conducted at each of the six case study sites to prepare volunteers thoroughly to collect patient feedback using the near real-time survey. The sessions lasted for 2.5–3.5 hours. As it was not possible for all volunteers involved in the data collection to attend the training, NHS trust staff, such as the project co-ordinators, patient experience leads and volunteer co-ordinators, also participated in the volunteer training. This way, additional volunteers could be trained by trust staff throughout the project.

Two members of the research team co-led the volunteer training at each site. The volunteer training was designed to address the following learning objectives:

• understand the overarching goals of the After Francis research project
• understand how the current RTF approach differs from existing approaches, such as the FFT
• assist patients in reading the patient information sheet and understanding the purpose of the survey, including risks and benefits
• obtain informed consent from patients
• understand what constitutes capacity to consent and how to judge it based on patient’s communications
• follow up on patient concerns, if necessary
• provide assistance to patients by reading the survey questions and response options
• follow standard infection control procedures for hands, tablet and patient information sheet
• use computer tablets to administer the survey
• understand the purpose of a weekly volunteer diary and discuss frequency and mode of administration
• understand the patient collaborator role, descriptively termed peer researcher, and have an opportunity to ask questions or register interest.

In total, 61 volunteers were involved in the training sessions led by the research team, with further volunteers subjected to cascade training by the trained volunteers. We are unable to determine the number of volunteers who were subject to cascade training; however, we know that at least 35 individuals were trained by staff members or other volunteers at the trusts.

During the training, volunteers also practised administering the tablet-based survey to a colleague, followed by administering the survey to patients on study wards. The research team members were available to provide directions and answer any questions.

Following the initial training, the patient data collection process for volunteers was as follows:

1. check in with volunteer co-ordinator to receive ward/department assignment and collect tablet
2. report to ward manager or senior sister on the ward/department and get list of patients who should not be approached
3. approach remaining patients and enjoy interacting with them.

Volunteer diary and interviews

Patient experience data collection

To understand patients’ experiences of the relational aspects of care, a 10-month-long survey data collection phase was implemented using trained volunteers to administer the survey. Data collection began during June 2015; one case study site started to collect near real-time data on 1 June 2015 and two other trusts began their data collection at a later date during the month. Two of the remaining case study sites began their data collection during the month of July and one case study site began its patient data collection in early August. Discrepancies in the dates on which case study sites began their data collection were a result of the varying times of volunteer availabilities. Moreover, most volunteer training was conducted at the beginning of June, prior to which trusts could not begin their NRTF survey data collection.

Patient experience data were collected on one to five wards/departments selected by staff at each trust. These included wards providing care primarily to patients aged ≥ 75 years and A&E departments. The recruitment targets were a minimum of 20 completed surveys per month for each ward and 50 completed surveys per month for each A&E department.

Initially, the research team planned to provide a stationary kiosk to each trust to facilitate data collection in the A&E department and two protected tablets for data collection on study wards. Four of the six sites requested a third tablet for their data collection in lieu of a stationary kiosk placed in the A&E department. These requests were made for a number of reasons, including lack of space, previous unsuccessful experiences with a kiosk in A&E and a strong desire to avoid placing a burden on staff who would be directing patients towards the kiosks. During the data collection, the remaining two trusts switched to using a tablet in the A&E department as a result of the low response rate obtained with the kiosk methodology.

Over 43 weeks of data collection, the recruitment targets were met 50 times out of the 643 potential times across the 15 participating wards. In A&E departments, recruitment targets were met 40 times out of a potential 258 times across the six sites.

An interactive reporting system using Microsoft Excel was designed by the research team to showcase weekly results from the patient experience survey. Feedback from case study sites, shared through staff interviews, the online staff survey and e-mail communications and at the participatory workshops, was used to guide the continued refinement of the reporting system to best meet the informational needs of the trusts. The reporting system continued to evolve throughout the data collection phase of the research. Specifically, weekly and monthly summaries were added to the reporting system, along with printable dashboard reports. Case study sites also requested the option to remove non-specific responses from results displayed for each survey question.

Following the 10-month data collection period, the quantitative patient experience data from all six sites were analysed using factor analyses. These are described in detail in Chapter 4. Free-text comments from all sites were combined and underwent a thematic analysis. Themes and example comments are presented in the results section of this report.

Participatory workshop

Two participatory workshops were conducted at each case study site for staff involved with the project implementation. Frontline staff, such as nurses, matrons and ward managers, were especially encouraged to attend the events. Workshops were held 3 months into patient data collection and again on completion of the data collection phase.

Prior to the workshops, project advisory group members were asked to share their thoughts and suggestions for increasing the usefulness of participatory workshops to staff involved with the project at the case study sites.

First, advisory group members discussed the usefulness of giving staff time off from their regular duties to consider and plan for improvements, as they considered it a task unable to be undertaken by administrators.

Another advisory group member stressed the importance of having a team of nurses or staff from each ward or department attending the workshops. This way, a ‘shared group perspective’ forms the basis of discussions and one person does not ‘feel overwhelmed’ by the perception of bearing the sole responsibility for improvements to patient care.

One advisory group member explained that some language used by the research team, such as ‘improvements’ or ‘changes to patient care’, may be perceived negatively by staff. The wording ‘working together for the future’ was suggested for use instead of or in addition to the current wording. Additionally, it was suggested to make the workshop content and patient data meaningful to staff by continuously relating discussions back to the patient. One way to make the data meaningful is to relate them to activities associated with the professional revalidation process for nurses. As part of this process, staff must engage in, and demonstrate, reflection and self-learning. Feedback from patients collected using the tablet-based survey can be used for this purpose, especially as responses provided in free-text comments may provide ‘personal feedback’ needed for the professional revalidation.

Advisory group members also proposed asking staff to focus on small and manageable changes for the action-planning activity. All changes must be easy to implement. In addition, a short- and long-term focus of the improvements should be apparent to staff. It was also suggested that ‘staff not shy away from understanding underlying reasons for improvements, as it may be necessary so staff can truly understand and improve patient care’.

Based on staff suggestions, participatory workshop activities were designed to provide a collaborative space for staff to interpret patient experience feedback collected as part of the project, and to maximise engagement. Staff identified and prioritised areas of patient experience that needed improvement and developed concrete action plans for each ward/department. In addition to engaging staff in a guided action-planning process, the research team sought to understand the levels of staff engagement with the near real-time data and discuss factors that encourage patients to provide feedback and staff to use the NRTF for improvements and decision-making. Finally, as part of the workshops, any barriers identified as hindering the use of NRTF could be explored as a group and mitigating strategies shared with the site leadership.

Staff survey

An online staff survey was designed to understand the types and methods of patient experience data collected and methods of data collection on the wards/departments involved in the research. In addition, the survey instrument explored the following contextual factors associated with patient experience data collection at each case study site:

• how results from patient experience data collection are communicated to staff, including by whom and how frequently
• preferences for communication modes and their desired frequency
• preferred reporting formats
• usefulness of patient experience feedback
• perceived barriers to, and enablers of, using patient feedback
• experiences with RTF data collection and using volunteers to collect patient feedback.

To capture changes in perceptions of and experiences with near real-time data collection, the staff survey was administered before (pre) and after (post) the patient data collection. This survey was made available to all staff working on wards and in A&E departments involved in the patient experience survey. At each case study site, an additional two wards that were not involved with the project were selected as control groups. All staff from these wards were eligible to take the online survey.

In the first administration, the survey was made available to staff in an online format. To notify staff of the upcoming survey opportunity, project leads at each trust prenotified all staff or senior staff members via e-mail, as well as in person if possible. If senior staff were notified, they were asked to pass the information on to their team members.

The survey was administered using the Snap survey platform (version 11; Snap Surveys, Tidestone Technologies, Overland Park, KS, USA) and hosted on Snap WebHost. Invitations were sent to trust contacts on 21 May 2015. Five project leads at the trusts forwarded the invitations on to their teams shortly thereafter. One trust was still awaiting the site-wide NHS permission letter, which delayed the survey invitation mailings by 2 weeks. Reminder messages were sent to project leads at the case study sites 1 week following the initial survey invitation mailings.

Owing to the low number of survey responses (see Table 3), project leads at the case study sites were asked to share any suggestions or ideas they had for increasing the number of responses for the second survey administration. The main suggestions received focused on the substitution of the online survey for of a paper-based version, which could be distributed to all staff working on the wards.

Advisory group members were also asked for suggestions to improve the response rates. Similar to staff, they suggested the use of an alternative survey format, as staff on wards are generally not provided with access to e-mail during their work hours. A paper- or a tablet-based survey version that volunteers could take to staff was suggested for use instead.

Advisory group members recommended shortening the survey, as frontline staff have limited time. It was also considered beneficial to make it clear to staff why it was important to complete the survey.

For the second administration, the research team reduced the number of questions on the survey and made it available in a paper-based format or on the tablets previously used for patient data collection. In addition, the survey prenotification, invitation and reminder messages were replaced by in-person communication from the project team with the ward/department leaders and senior staff.

The paper-based survey was printed as an A5 booklet and handed out to all staff members working on wards/departments by the senior staff members. Where possible, time was made available in ward or department meetings for staff to complete the survey. Staff placed their completed surveys into envelopes, which were collected by a trust research and development (R&D) team member and returned to the research team. The survey was available for completion over a 2-week period.

Three trusts chose to administer the survey as a tablet-based version, which was also available over a 2-week period. Using this method, R&D team members took the tablets to the wards during handover times and encouraged staff to complete the survey at that time. Table 3 presents the number of responses and selected survey administration modes for each of the two survey time points at each case study site, along with the change in number of completed staff surveys between the first and second time point.

Networking event for case study staff

A networking event was held on Tuesday 24 November 2016, at Harris Manchester College, Oxford, to enable key staff from each of the six case study sites to meet each other and share their experiences regarding the project. Specifically, staff provided feedback on experiences of data collection; barriers and enablers that staff had come across; impacts the data have had within the trust; and lessons learnt.

Meetings with key staff at case study sites

Five meetings with key staff from each study site were held on 24 and 25 November 2016 to coincide with the networking event. As one of the sites was unable to send a representative to the networking event, a conference telephone call was held on 2 December 2016 with four staff members involved in the project.

Support calls

To support volunteers with their data collection, the research team invited all volunteers to join them in monthly conference telephone calls. Support telephone calls were scheduled separately for each trust. During these telephone calls, volunteers could discuss any questions, comments or concerns with the research team. Conference call opportunities were available on the following dates:

1. 23 October 2015
2. 13 November 2015
3. 4 December 2015.

Invitations to participate in calls were sent to the key contacts at the case study sites, who confirmed that they forwarded the messages and reminders on to all their volunteers currently involved in data collection for the project.

Telephone interviews with staff

Prior to implementing patient data collection, the project leads at each case study site were asked to assist the research team in scheduling 10 telephone interviews with staff who would be involved with the research project. A total of 52 interviews were conducted via telephone during the months of May and June 2015. This constituted the first round of staff interviews. Interview duration ranged between 14 and 35 minutes.

Through the interviews, we aimed to understand, in detail, the types of patient experience data currently collected at the trusts, how results are communicated, and the factors or structures that facilitate or hinder the use of patient experience data for improvement purposes. In addition, staff shared examples of improvements or changes to services that have been made based on patient experience feedback. Finally, staff shared their expectations for the upcoming patient data collection, the benefits and challenges associated with real-time data collection and the use of volunteers to collect patient experience feedback.

Some example questions and follow-up probes are as follows:

• What types of patient experience data are your trust currently collecting? How frequently? Are any volunteers involved?
• What procedures or structures are in place to make improvements or changes based on patient feedback?
• What are your experiences with NRTF of patient experiences? What benefits do you anticipate? What drawbacks do you anticipate?
• How could your trust benefit from volunteer involvement to collect NRTF? What should be considered when working with volunteers?

The following number of interviews were conducted at each case study site:

• Hinchingbrooke Health Care Trust (n = 10)
• Milton Keynes University Hospital NHS Foundation Trust (n = 10)
• North Cumbria University Hospitals NHS Trust (n = 7)
• Northern Lincolnshire and Goole NHS Foundation Trust (n = 5)
• Poole Hospital NHS Foundation Trust (n = 10)
• Salisbury NHS Foundation Trust (n = 10).

The roles of the interviewed staff working on wards and departments included the director of nursing, head of nursing, senior sister, senior staff nurse, deputy sister, matron, ward sister, staff nurse, lead consultant, unit lead, Foundation (year 2) doctor, health-care assistant, ward clerk and receptionist.

In addition, a lead governor and staff working in R&D departments, voluntary services and customer care were interviewed. Staff whose roles do not typically involve contact with patients were selected to participate in interviews as they were involved in collecting patient experience feedback from patients. For example, staff administered the FFT or other short surveys designed specifically for the trust to collect data from patients while they are still in hospital. Interviewees’ length of employment at the sites ranged from 6 months to 41 years.

Specific survey for interviewees

On completion of the 10-month patient data collection period, the 52 staff members across the six case study sites who participated in the initial telephone interviews were asked to complete an online survey consisting of closed and open questions. The purpose of the survey was to understand their involvement and experiences with the project and to determine how their previously shared expectations of near real-time data collection and experiences working with volunteers to collect patient experience feedback have changed. Staff were also asked about improvements to care made as a result of the NRTF collection. Similar questions were asked to those during the initial telephone interview, with answer options derived from the responses to the telephone interviews. Additional space for other answers was available.

This survey was available for completion online. It was set up in Snap and hosted on Snap WebHost. Staff received a prenotification message from their colleagues who were responsible for the implementation of the research at each case study site. Following this initial communication, the research team invited staff by e-mail to participate in the survey. The survey was available for a 2-week period in April 2016. Non-respondents received two reminder e-mails. Of the 52 staff members who participated in the initial telephone interviews, seven had left the trusts or were currently on maternity leave. Therefore, a total of 45 staff members were invited to take the online survey and 31 completed the survey, yielding a response rate of 68.9%.

In-person interviews with staff

A second round of staff interviews was conducted following the end of the patient experience data collection. Five staff members from each trust were interviewed in person or via telephone by the research team. Questions asked were similar to those asked during the first round of interviews. Instead of expectations for the data collection, we asked about their experiences with the data collection as part of this research. Interviews allowed us to explore any changes in opinions about NRTF and working with volunteers to collect patient experience feedback that resulted from their involvement in the current research. Interviews lasted between 14 and 35 minutes and staff members with the following roles participated:

• matrons
• ward sisters
• deputy sister
• director of nursing.

Develop a survey toolkit and evidence-based recommendations

Based on the patient experience data collection and the evaluation data collected from staff and volunteers through surveys and interviews, a toolkit was developed to share the survey instrument, including guidelines and recommendations, with other NHS hospitals. These data sources evidenced the success of the NRTF approach for improving relational aspects of care. With the help of the advisory group, the toolkit contents were developed and refined to make them more relevant and meaningful to the target audience at NHS hospitals. In addition, the advisory group selected the branding, including colours, font, title and pictures for the toolkit. Four guides and three case studies were also included alongside the survey instrument. These documents showcase the challenges, lessons learnt and impacts of the work at the case study sites. Volunteer training materials were made available to assist trusts in implementing the approach.

Disseminate toolkit and practice recommendations to NHS trusts

To share the toolkit, three regional toolkit launch and networking events were held at the following locations in February and early March 2017:

• Novotel Paddington, London, 20 February 2017
• Novotel Leeds Centre, Leeds, 27 February 2017
• Harris Manchester College, Oxford, 2 March 2017.

The launch events commenced with a presentation providing an overview of the research, followed by a detailed description of the purpose and components of the toolkit. This was followed by time for questions about the content and functionality of the toolkit. Time for networking and further small group discussions were provided.

In February 2017, the toolkit was made freely available on the Picker website (URL: www.picker.org/compassionatecare, accessed 22 November 2017) and publicised through a published policy commentary.⁴⁶ In addition, the toolkit was disseminated through Twitter (Twitter, Inc., San Francisco, CA, USA) and e-mails sent to our contacts at NHS trusts nationally.