Question 1: speaking with ward managers for patient selection

Of the 258 volunteers who completed the diary, 34.9% found it very easy to speak with the ward managers to determine which patients were suitable for participation in the patient survey. The use of volunteer diaries was considered easy by 30.2% and neither difficult nor easy by 22.5%. Only 7.8% of respondents found it difficult to speak with ward managers and only 0.8% found it very difficult to do so.

Question 2: were patients interested in the survey?

Volunteers were asked to feed back on how interested the patients appeared to be in taking part in the survey. The majority (60.7%) of volunteers reported that patients seemed somewhat interested in taking part in the survey, followed by patients seeming very interested in taking part in the survey (25.6%). Only 4.7% of volunteers reported that patients seemed not interested in taking part in the survey.

Question 3: reasons why patients declined to participate

The most frequent reason for declining to take part in the survey was the patient feeling unwell (40.7%), followed by the volunteer approaching the patient at an inconvenient time (18.6%), lack of interest (17.8%), no reason given (8.5%) and language barriers (3.1%). Other reasons why patients declined to participate were also given (16.0%):

Felt that it would be a bad report as she is very dissatisfied.

Volunteer

Banana more important!

Volunteer

Patient extremely anxious and awaiting crisis team.

Volunteer

Never take part in surveys!

Volunteer

Question 4: patients answering the survey questions

Volunteers also reported that, for the most part, patients found it easy to answer the survey questions (43.9%), with approximately one-third (33.7%) of patients finding it neither difficult nor easy to do so. Thirty-one responses suggested that patients found it very easy to answer the survey questions (12.0%). Only 15 responses suggested that it was difficult for the patients to answer the questions (5.8%) and only two volunteer responses stated that patients found the survey very difficult to answer (0.8%).

Patients’ privacy was also a concern when completing the survey. Often the wards lacked privacy for patients to answer survey questions, causing some patients to feel uneasy when giving negative feedback:

Yeah, you would often see them following the member of staff with their eyes. And they didn’t want to answer the question . . .

Volunteer

Question 5: engaging hard-to-reach patients

Volunteers were asked how easy or difficult they found it to engage hard-to-reach patients to take part in the survey. The most common response was that it was neither difficult nor easy (37.2%) to engage hard-to-reach patients. The proportions of hard-to-reach patients that were reported to be easy or difficult to engage were similar (20.9% and 20.5%, respectively). Fourteen responses reported that it was very difficult (5.4%) to engage hard-to-reach patients, whereas only 11 responses (4.6%) indicated that the volunteers found this aspect very easy.

Question 6: forming relationships with patients

When asked if they were able to form relationships with patients during data collection, the majority of volunteers said that they could form relationships with most patients (46.9%), all patients (24.0%) or some patients (23.6%). In only six cases were volunteers unable to form a relationship with the patient (2.3%).

Question 7: free-text comment

Respondents were asked ‘Please use the box below to tell us about your experiences with data collection this week’. Of the 258 responses to the volunteer diary, 160 responses to this question were given. These experiences provided additional detail to illustrate the answer options selected in response to the previous closed questions in the survey, as well as restating some of the complaints already shared by patients.

Patient experience and feedback

Comments from volunteers suggest that some patients found their experiences in hospital to be a positive experience and that they reported being happy with their care:

It was interesting to hear from one of the patients who had been in a few years ago say how much better the care they have received this time has been.

Volunteer

[Patients] have shared positive and beneficial [feedback] about the hospital, as well as their health, statements about their care and the outcomes of it.

Volunteer

However, some patients explained to volunteers that their experiences and care while in hospital required much improvement. Some discussions focused on the lack of staff availability, low staff levels and the fact that patients have been unhappy regarding the lack of explanations surrounding their diagnoses, which have impacted negatively on their emotional well-being:

One lady had very negative experiences on the ward. She has no relatives and is anxious about her care. She has asked to speak to the Sister three times but still has not seen her. She has praised all of the foreign nurses, and said that they are very kind, but that some of the English nurses are very hard and not kind.

Volunteer

Concern on [department/ward name redacted] with staffing levels, referred patient to PALS [Patient Advice and Liaison Service].

Volunteer

One patient stated that the professionals had not taken the time to explain the anatomy associated with the diagnosis made as well as treatment. The lack of this knowledge impacted the emotional aspect of the health status of the patient.

Volunteer

Some comments recorded in volunteers’ diaries were positive and some were negative. Negative comments related to concerns about staffing levels and, in one case, the fact that the patient, although happy with the care received, was unhappy with the ward environment:

Feedback was mostly positive, however a recurring comment was how understaffed patients felt the ward is.

Volunteer

Most people are very happy with their treatment and care. But we keep hearing about the noise in the wards at night with patients crying out and shouting while the rest of the patients are trying to sleep!!

Volunteer

Patient recruitment

Volunteers also described their experiences with the recruitment of patients to participate in the survey. Some volunteers reported having no problems in engaging the patients to participate in the survey, often commenting that they seemed ‘keen’ to take part. During an interview, one volunteer explained that, generally, 99.9% of the people were willing to give their opinion:

A very willing sample of patients today!

Volunteer

No problems – all the patients were keen to take part and generally very positive about the ward. Patients were easy to talk to – interviews took longer because of very interesting conversations with patients.

Volunteer

However, some responses suggested that the volunteers also often found it difficult to get patients to participate in the survey. Volunteers learnt that they only had short windows of opportunity to survey the patients, because of protected mealtimes and patient care, causing challenges with recruiting patients. However, in the A&E department there was a different challenge – the faster turnover of patients:

Difficulty with a deaf lady. I had to sit really near to her and enunciate the words clearly. Three patients did not want to complete the survey, looked ill, hot and bothered in the heat of the ward.

Volunteer

Still a problem getting on to some wards before 10.00 a.m. due to patient care time and have to get out for mealtimes which causes a bit of a panic.

Volunteer

Some mixed feedback was also provided in the diary, suggesting that, although volunteers experienced some problems with data collection, in general, it was easy to recruit and engage with patients:

The patients were approachable and unless they felt unwell were happy to participate, although one patient fell asleep part-way through the questionnaire so I had to end that session.

Volunteer

As I am assigned to a ward for hip, knee replacements, etc. There is a quick turnover of patients so there is usually no problem in finding participants. Sometimes there have been very few patients and when some are asleep it can be a wasted visit!

Volunteer

Staff interest and buy-in

Volunteer comments within the diary suggested that levels of staff buy-in of the project can differ (e.g. depending on the ward/department the volunteer is working in). The levels of staff interest or buy-in directly affect the experiences of volunteers.

Substantial staff buy-in is important, as volunteers see staff as fundamental to ensuring that patient recruitment goes smoothly, by directing volunteers to approach the correct patients:

The [department/ward] staff was really friendly and was able to be approached. They guided me to people I can talk to and others I couldn’t which was helpful for me.

Volunteer

Staff very helpful in assisting me with advising me of which patients I could approach with the survey on the [department/ward]. A member of staff on the [department/ward] was extremely helpful and showed me respect.

Volunteer

Despite it being the day of the strike the staff were very accommodating and helpful.

Volunteer

On the other hand, volunteers also explained that some staff acted as barriers towards data collection. Some volunteers reported that staff reacted negatively towards them when entering the ward or department. Partly, this was as a result of staff being suspicious of the data collection or because staff remained unaware of the project and its purpose despite frequent explanations from the volunteers:

One consultant questioned me about the survey and wanted to know why we chose his department, which is one of the top three departments in the country. He felt that relationships with patients are very good in [department/ward], and although it is not perfect, it would be more use to use another ward.

Volunteer

Nursing staff did not seem to think that patients were appropriate for approach. I am not sure that the nursing staff understood my brief.

Volunteer

These barriers were experienced during the first month of data collection only, and were brought to the attention of the site lead, who explained the purpose of the research in more detail to the staff.

Comments on the survey

Feedback in the diary suggested that the majority of volunteers, and some patients, found some survey questions more challenging to answer than other questions. These issues were mainly associated with question repetition, or questions confusing patients because of a lack of understanding or language barriers. Volunteers also reported that patients found the survey to be lengthy:

Question 17 seems to always confuse patients. But other than that it’s very good and easy to communicate and get answers.

Volunteer

Question, ‘have family or carers been informed’ patients found difficult to answer as some didn’t know whether their family had been or not – no option for this answer.

Volunteer

A lady whose first language was not English found some of the vocabulary hard to understand so had to simplify some questions. One person thought the questions needed reviewing as some were repetitive.

Volunteer

However, despite an opinion that the survey requires further review, feedback also suggested that the survey was well constructed. Yet some patients gave vague, non-direct answers that required follow-up from the volunteer:

On the whole well-constructed survey although one or two patients perceived some of the questions to be repetitive.

Volunteer

The questions are straightforward for the patients but, sometimes the patients are too vague with their answers.

Volunteer

Tablet and technology

The majority of volunteers experienced some technological issues with their tablets, which were resolved during the initial 2 months of the data collection. Some volunteers also found their tablets heavy to work with, especially when standing next to patients to hold the tablet on their behalf. However, these issues applied to only two case sites:

I find the tablet heavy to work with. No one seems interested in using it themselves on elderly care – it isn’t familiar or easy for patients who are not lying very comfortably in bed.

Volunteer

Technical difficulties: tablet froze on several occasions or went to friends and family or patient experience demo and unable to get it back to Picker had to wait a while so missed a couple of patients.

Volunteer

Having volunteers to aid patients with the survey meant that patients could still complete the survey if they were hesitant to use the tablets. Volunteers often reported that few patients used tablets. When given a choice between completing the survey themselves and getting aid from the volunteer, patients would often request that the volunteer fill in the survey on their behalf.

Although the tablets provided initial challenges, they were not problematic for the duration of data collection. Generally, tablets were found to be easy to use. Sometimes, volunteers were unsure whether or not survey responses were received by Picker:

Most people found the tablet easy to use with my assistance.

Volunteer

I am a little concerned as to whether all the surveys are sent off properly when we give the iPad [Apple Inc., Cupertino, CA, USA] to the patients to fill in by themselves. And they hand it back to us completed? Otherwise all good.

Volunteer

It was also suggested that the tablets were beneficial to the approach. The electronic devices were generally held in higher regard than paper surveys.

Volunteer benefits

Comments suggested that volunteers generally had a positive experience with data collection and found that they received many benefits from spending time with the patients. These benefits include feeling useful in carrying out an important role, forming relationships with patients, enjoying the opportunity to talk to patients and finding it rewarding and interesting:

I was talking so some older patients and was told some amazing stories while filling out the questions. I helped fill it in as they were not great with the tablets but we built a relationship and had a great time.

Volunteer

Had end-of-life patient in [department/ward]. Had a great conversation with them about their past and they said I made a difference to their day. I felt good and happy.

Volunteer

Plain that many patients on [ward/department name redacted] relish opportunity to talk to someone. The process of the survey often opens memory of long and interesting life experiences. This provides a gratifying sense of usefulness, likely to encourage volunteers to undertake this kind of work.

Volunteer

Other

Other comments that did not align with the previously discussed themes have been grouped into general positive and negative comments. Positive comments mainly described pleasant and generally positive experiences with data collection on the wards and how volunteers can help improve the patient experience by feeding back to staff after data collection. For example, they informed staff of immediate patient needs, such as needing food or water:

Very pleasant experience.

Volunteer

In my opinion and experience I have seen patients have been treated with respect, dignity and compassion.

Volunteer

Relatives helped with two patients especially with one more confused.

Volunteer

Other comments were considered less positive. They detailed that some staff were unaware of how the weekly reports related back to their wards, issues related to data collection, such as ward closures, and that there were generally very few volunteers available to collect patient feedback. For example, one trust relied only on one volunteer to collect data. This was considered challenging, as volunteers felt more pressure to survey patients:

The senior Sister told me that in the weekly report from Picker, she does not know which results are attributed to [department/ward name redacted].

Volunteer

[Department/ward name redacted] became [department/ward name redacted] which was then closed and is now temporarily a discharge lounge, long-term future is unknown.

Volunteer

When asked if they would like to be involved in future NRTF within the trusts, volunteers stated that they would like to see a continued use of the approach. However, there needs to be comprehensive volunteer training and a support system in place for volunteers to use. In addition, volunteers stated that they would like to have the results of the feedback frequently shared with them on a regular basis.

After Francis preliminary analyses

Prior to the post-data collection analyses, the following preliminary analyses were conducted:

• item frequencies
• analysis of interview length
• mode of administration
• measurement properties of the questionnaire
• assessment of data dimensionality
• scale analysis
• reducing the item pool
• handling missing item responses in the scale
• unit-level reliability
• score variation by demographic groups
• differential item functioning (DIF).

The complete analyses and corresponding results are detailed as follows.

Item frequencies

Item frequencies provide basic information about questionnaire functioning, including completion or ‘skip’ rates, and potential differentiation issues through ‘floor’ or ‘ceiling’ effects.

A ‘point reached’ analysis (Table 14) indicated that just under 95% of respondents (n = 3713) proceeded to the end of the scored survey questions, also known as closed survey questions. Thirty-two respondents did not complete question 20 (Overall, do you feel you have been treated with respect and dignity while in hospital?). Other than that, there appeared to be no particular point at which responses ceased. Refer to Appendix 1 for the full list of survey questions.

TABLE 14

Last question completed (point reached)

All further analyses were restricted to the 3889 responses deemed ‘useable’ (i.e. having at least four valid responses to scored items).

Basic item frequencies for all scored items are presented in Appendix 2. The highest proportion of ‘skipped’ or missing responses was for question 17 (Has a member of staff told you about what danger signals regarding your condition or treatment to watch for?): 6.2% of cases. For all other questions, at least 95% of respondents recorded an answer.

For two questions, > 90% of respondents gave the same answer:

• question 18 (During your time in hospital, have staff made you feel safe?) – 90.9% of respondents selected ‘yes, completely’
• question 20 (Overall, do you feel you have been treated with respect and dignity while in hospital?) – 92.8% of respondents selected ‘yes, all of the time’.

These questions provided limited differentiation in the current sample.

The proportion giving a response of ‘do not know’ was generally low (< 2% of responses). However, for a number of questions, the proportion of respondents indicating that the question was not applicable to them (e.g. ‘I have not asked for help’) was high: these responses are not scored and, therefore, the contribution of these questions to evaluating provider performance is reduced.

Descriptive statistics for the scored items are presented in Table 15. Refer to Appendix 1 for the full list of question and answer options. Five items had very high mean scores (> 90): questions 3, 4, 10, 18 and 20. For all items, the mean score was < 1 standard deviation (SD) from the scale maximum (range 0.26–0.88 SDs).

TABLE 15

Descriptive statistics for scored items

Analysis of interview length

Interview length is an aspect of usability. Statistics for the time taken to complete the questionnaire and the mean time per question are shown in Table 16, both overall and split by mode of administration. These times are broken down in Table 17 by who completed the questionnaire. For these analyses, data were restricted to questionnaire sessions that started and finished on the same day, to avoid problems caused by data not being submitted directly on completion.

TABLE 16

Time to complete questionnaire with mean time per question by mode of administration

TABLE 17

Time (minutes) to complete questionnaire with mean time per question by respondent type

Scale analysis

Reliability coefficients indicate how well a score differentiates between cases that differ in whatever construct the score is measuring. Reliability can be measured at the respondent level (how well a score differentiates levels of experience of individual patients) or at the group level (how well a score differentiates between wards or other units in terms of the experience of their patients).

The loadings from the factor analysis were used to analyse the overall respondent-level reliability of a single scale derived from all the items and the contribution of the individual items to reliability. Reliability was evaluated using Cronbach’s alpha and McDonald’s omega (a more general coefficient, of which alpha is a special case when certain restrictive conditions apply).⁴² Item contribution to reliability was evaluated using McDonald’s ⁴² item information index (the ratio of communality to uniqueness) (Table 18). The ratio increases as the item’s contribution increases. Refer to Appendix 1 for a full list of survey questions.

TABLE 18

Item factor loadings and information

The overall reliability was very high, indicating that different levels of individual experience could be distinguished with high precision. Two items with relatively low loadings (questions 12 and 17) did not contribute to reliability, and overall reliability could be improved by a very small increment if these were excluded from the scale. Question 17 was subsequently removed from the scale; however, question 12 was kept.

Reducing the item pool

Taking into account the extent of missing or non-evaluative item data, and qualitative information from cognitive interviews and other feedback on use of the questions, it was decided to remove question 9 (Have staff responded quickly when you ask for help?), question 16 (Has a member of staff answered your questions in a way that you could easily understand?) and question 17 (If you have had any anxiety or fears about your condition or treatment, has a member of staff discussed them with you?) from contributing to the overall score.

The exploratory factor analysis was rerun using the reduced item set (see Appendix 4). The key output is given in Appendix 5 and factor loadings in Table 19. The results from the analysis were virtually identical to those from the earlier analysis incorporating all items. Refer to Appendix 1 for the full list of survey questions.

TABLE 19

Item factor loadings and information (reduced item set)

The retained items included some with a relatively large number of missing data, which presented potential problems in the computation of an overall score; only 1561 cases had complete data on the reduced item set. It was therefore decided to test a short form of the scale using only items with at least a 95% response and to compare this with the full scale.

Scale statistics are reported in Table 20. Figure 3 shows the relationship between the two forms for cases with complete data on both the long and short forms.

TABLE 20

Scale descriptive statistics

FIGURE 3

Scatterplot of long- and short-form scores.

The differences between the two scales were assessed using a paired-samples t-test. The difference was statistically significant in raw score form [t = –8.017, degrees of freedom (df) = 1560; p < 0.0005], but not when both scales were standardised (t = 1.304, df = 1560; p = 0.192). Agreement between the scales was measured using the intraclass correlation coefficients for consistency (0.976) and exact agreement (0.975) and for ordinal agreement using Spearman’s ρ (0.945).⁴⁹

The level of consistency between the two forms was high and considered sufficient for overall scores to be based on the short form of the questionnaire. Respondent-level reliability for the short form was estimated from the factor loadings (see Table 19) as Cronbach’s α was 0.944 and McDonald’s ω was 0.945. Incorporating the scale data from Table 20, this gives a standard error (SE) of measurement of 3.74 and a 95% confidence band between 81.4 and 96.1 for a score at the sample mean. Refer to Appendix 1 for the full list of survey questions.

The item–total correlations for this scale are reported in Table 21.

TABLE 21

Item–total correlations for short form

Unit-level reliability

Reliability of scores at the unit (ward) level was estimated using the generalisability theory approach.⁵⁰ This relates to the capability of distinguishing differing levels of performance at the unit level, using scores on the short form calculated at the patient level and then aggregated to the ward level. Data for the 12 short-form items were entered in a three-level variance components model, with item responses nested within respondents within wards.⁵¹ Variance estimates at each level were obtained using MLwiN v2.36 (MLwiN, Centre for Multilevel Modelling, Bristol, UK).

Generalisability theory follows the same classical ‘true score’ measurement model as for respondent-level reliability and seeks to estimate the proportion of overall variance that is ‘true’ variance between (in this case) wards. This affords both an estimate of unit-level reliability for the study (based on the harmonic mean numbers of respondents per ward and items per respondent) and also a minimum number of respondents required to achieve a threshold reliability of 0.80. The results of this analysis are reported in Table 22.

TABLE 22

Generalisability analysis

Score variation by demographic groups

Variation in the overall score was investigated using a multilevel regression model with ward as a random effect and with indicators for the various demographic factors entered as main effects. This approach enabled all demographic variables to be taken into account when assessing the effects of each.

The variable groups entered were who completed the questionnaire; patient sex; patient age group; presence of long-term health condition(s); and ethnic group. Within each factor, there were groups whose score differed significantly from the reference group. In some groups – particularly those for ethnicity – numbers were very small and the coefficient estimates must therefore be regarded as tentative. Estimates for the fixed effects were obtained using MLwiN and are reported in full in Appendix 7. A chart of the score differences associated with each demographic factor is shown in Figure 4.

FIGURE 4

Score differences associated with demographic factors.

Differential item functioning

Differential item functioning is concerned with score differences at the question level: whether or not certain groups respond differently to a particular question. This is examined by conditioning score differences between a ‘focal’ group and a reference group on their overall scale score.⁵² Two types of DIF are possible: uniform DIF shows a consistent difference between focal and reference group responses; and non-uniform DIF describes a difference in one direction for high scores and a difference in the opposite direction for low scores.

The analysis was applied to two grouping variables that were of particular interest in use of the overall score. These were (1) the use of a proxy respondent and (2) white British compared with other ethnicities. All else being equal, and taking overall experience into account, we would hope to see no difference in the score allocated to a given item between the patient themselves and a friend or relative responding on their behalf, or between a patient of white British ethnicity and one of another ethnic group. For the purposes of this analysis, joint responses completed by a friend/relative or the volunteer in conjunction with the patient were excluded.

The method used followed closely that recommended by Zumbo.⁵² This involves a sequence of three ordinal logistic regression models in which the response to a selected question is conditioned (1) on the overall scale score, (2) on the scale score and a group membership indicator and (3) on the scale score, group indicator and the interaction between these. An item is considered to show DIF if (a) there is a significant improvement in model fit between models 1 and 3 (p < 0.01) and (b) the magnitude of the effect is greater than a recommended threshold (> 0.13 increase in the pseudo-R² index). The analyses were conducted using the PLUM procedure in IBM SPSS Statistics v23 (IBM Corporation, Armonk, NY, USA). Two pseudo-R² indices were examined: the Nagelkerke and McFadden indices.

None of the questions showed evidence of substantive DIF according to these criteria, for either grouping variable. One question showed a statistically significant, but small, effect for both comparisons: question 2 (Have staff taken the opportunity to learn about you as a patient?). Estimated adjusted mean scores for this question, controlling for overall score, are given in Tables 23 and 24. Similarly, question 6 (During your time in hospital, have you had enough contact with staff?) showed a small but statistically significant effect for respondent type. The estimated mean scores are shown in Table 25. Additional results are presented in Appendices 8 and 9.

TABLE 23

Survey question 2 score by respondent ethnicity

TABLE 24

Survey question 2 score by who filled in the questionnaire

TABLE 25

Survey question 6 score by who filled in the questionnaire

After Francis results analysis, following the validation study

Following the preliminary analyses, an analysis of the impact of the intervention was conducted first at the whole-study level, then at the individual ward level. The main focus of the analysis was the overall score and the difference between scores at the baseline ‘pre-intervention’ period (weeks 1–18) and the ‘post-intervention’ period (weeks 21–43).

Results at the study level

Figure 5 shows the trajectory of scores for the survey as a whole, calculated as the mean of the ward-level means. The source data are in Appendix 10. The chart also shows a 95% confidence interval (CI) for the null hypothesis of no overall change in scores for the 11 months. The general trend for the score is upwards, although mostly it remains within the CI.

FIGURE 5

Trend in overall score for whole survey. LCL, lower confidence limit; UCL, upper confidence limit.

Results for the whole study were analysed using a multilevel regression model of the overall score, with ward as level 3, respondent as level 2 and item score as level 1 in the model. Modelling was done using MLwiN v2.36. Fixed effects were entered for items and for the results period (pre or post intervention). The relevant model terms are presented in Table 26 and show a small but statistically significant effect for the intervention term.

TABLE 26

Multilevel model of pre- and post-intervention scores

Quality of care

Generally, patients who provided a comment reported that they experienced good-quality care. Specifically, feedback on care was related to the care provided by staff, rather than their overall experiences of care. Staff were mostly found to be helpful, compassionate, kind and caring. Many patients left a thank-you comment at the end of the survey, which reflected their positive experience of care:

I am very happy, we had a laugh last night.

Patient

The nursing staff have been amazing, kind, caring and thoughtful.

Patient

However, some patients fed back that the quality of care received was lower than expected. In particular, some patients felt that staff appeared unfriendly. Agency, night and reception staff were often perceived to provide a lower standard of care:

Reception staff were unfriendly, rude and dismissive.

Patient

There is a nurse who never smiles.

Patient

X-ray staff member very rude. 7-year-old scared to go in on her own as mum had a newborn and she huffed and puffed and told her to hurry as others waiting!

Patient

Poor care. Left in your urine and left with no dignity . . . Nurses not caring enough and don’t check on you enough.

Patient

One member of staff did not like me and I was scared of her.

Patient

Some patients also highlighted that their care staff seemed to lack knowledge, confidence and ability to perform their duties:

One nurse did not know what coeliac meant.

Patient

Staff did not really understand mental health. [I was] made to feel uncomfortable.

Patient

Not enough contact with doctors or nurses

At times, patients felt that they had too little contact with staff, especially with the doctor in charge of their care. However, patients also recognised that the nurses were often overworked and so could not always attend to their needs immediately:

I have not met the consultant named on the board by my bed and I have been here 2 months.

Patient

I haven’t seen a doctor for 3 days to discuss my health.

Patient

Not seen much of the doctors. Staff are very busy but come over as quick as they can.

Patient

The nurse who dealt with me was clearly not very interested in me as a person, seem to give me minimum contact.

Patient

Involved in decisions about care

Some patients reported that they did not feel as involved as they would like to be, if at all, in decisions regarding their care. In one case, a patient stated that family and doctors had seemingly made decisions without their involvement:

Don’t always feel involved as much as I would like.

Patient

Cross that family have apparently taken control of decisions about where she will live on discharge.

Patient, as reported by volunteer assisting with data collections

On the other hand, one patient did report that doctors had considered them to be an expert in their condition and complex health needs, so always consulted the patient in care decisions:

The doctors have been really good at treating me as an expert patient due to my complex health needs and have consulted me in regards to my treatment and what I would like to do.

Patient

Communication

Patients referencing communication varied in their feedback. Some frequently shared that the communication they experienced was not acceptable. Patients often felt that the information they were given regarding their health was not clear, or was seriously lacking in content, leaving them confused and without the opportunity to ask questions to gain a better understanding of their situation.

Some patients also reported witnessing discussions between staff members, which came across as unprofessional and made them uneasy:

Staff complain about being short-handed which is alarming for a patient.

Patient

Last night I was very upset by a female member of staff after my other [half] asked her at reception for a bottle for urine. She came to my bedside and said that I should be using the call button . . . She then told the nurse in charge of our care to look after her patients better in front of us. Very unprofessional.

Patient

Communication between departments was also erratic at times, causing problems and uncertainty for patients about their treatment, medication, dietary needs or care after discharge:

Mealtimes are quite erratic which is not good for a diabetic. There seemed to be a mix up and they did not know I was diabetic.

Patient

Contact with social services to enable timely discharge has been impossible.

Patient

The medicine I take at home wasn’t given to me correctly at the hospital for the duration of my stay.

Patient

On the other hand, some patients responded that levels of communication between members of staff, and between the staff and patient, were excellent. They reported being well informed about their condition and care, that family members were kept updated and that staff answered any questions they had.

Isolation

In their feedback, some patients spoke of feeling isolated during their stay in hospital. Patients described being left alone, frequently commenting that the nurses were too busy to spend time with them:

They don’t listen. No one talks to you. You sit all day. No one comes around.

Patient

However, this was not the case for all patients, as some fed back that their care staff were always happy to spend time with them and get to know them and took an interest in them as people, not patients:

We chat, we laugh. In a roundabout way you get to know them very well. I feel very relaxed about talking about ‘personal’ things without any blushing.

Patient

They were there for me morning, noon and night . . . never felt alone always someone to talk to.

Patient

Hygiene

Hygiene was also briefly mentioned in some feedback provided by patients. In particular, a few patients noted that student nurses did not always adhere to hygiene and infection control practices, and one patient stated that they had not received the personal hygiene care that they would have liked:

Had feet washed [today] for first time since admission.

Patient

Staffing levels

In their comments, several patients reported on the visible lack of staff during their hospital stay, stating that staff often look ‘busy’ or ‘rushed off their feet’. In particular, patients perceived that there were often not enough staff at night. Patients mostly recognised that staff were unable to spend time with them because they were overworked and understaffed:

Sometimes short staffed this is no problem to me but it does sometimes effect how much time they can spend with you.

Patient

Short staffed on here but they do their best.

Patient

Language barriers

Difficulties in understanding care staff as a result of language barriers were also cited by patients. Specifically, patients reported having difficulty understanding what was being said to them during conversations with foreign staff with unfamiliar accents:

Difficulty at times understanding some of the overseas staff.

Patient

Discharge

Discharge was also identified as an area of care with which some patients were dissatisfied. Reasons included being discharged with little to no notice or too much advance notice:

I was told I could go home 2 days ago.

Patient

Discharged with little notice which made the patient very anxious.

Patient

Call bells and waiting times

Comments received in the patient experience survey also described longer than expected waiting times in A&E, as well as on wards, once the call bell had been used. The wait on wards, at times, left patients in an undignified situation because they were unable to attend the bathroom. The wait in A&E – while understandably long – was found to cause some patients distress:

Having broken foot to the extent where my foot was black, in pain and crying, I was left for 7 hours before having my cast on, my treatment was unpleasant and cost me almost losing my foot.

Patient

Patient next to me wet the bed because staff took too long to help her at night.

Patient

Similarly, one patient highlighted that the check-in process at reception can be lengthy:

[On] one occasion, after pushing the buzzer for 15 minutes and getting no assistance I resorted to picking my mobile phone and calling the main hospital, asking to be connected to [ward].

Patient

However, although waiting times were frequently commented on, most patients understood that the long waiting time to be seen on wards was related to staff being busy and acknowledged that they would usually be seen when staff were available.

Food

Generally, feedback regarding the food provided during the hospital stay was positive, with statements suggesting that food was always served warm and that choices which catered to different dietary needs were offered.

On the other hand, some patients reported that the food should be of a better quality. For instance, it was sometimes described as tasteless, cold and often ‘stuck to the plate’. Similarly, patients described that they often did not receive regular meals or were completely missed on food rounds because they were in a side room:

Only meal I got offered was breakfast this morning. I arrived via ambulance at 14.00 yesterday.

Patient

Environment

Feedback describing the hospital environment was predominantly negative, with many patients stating that it was unclean and noisy. Similarly, some patients requested that equipment be installed (e.g. handrails in bathrooms) or stated that their beds were uncomfortable. In addition, some patients were not comfortable with being on a mixed-sex ward; some who wanted to be moved had their request denied because of a lack of beds on single-sex wards. Other concerns raised related to the lack of entertainment, such as radios, or the fact that the TVs were broken or too expensive to use. Other requests were simple, such as requests for air-fresheners, more wheelchairs and access to more pillows.

Comments regarding the survey

Feedback on the questionnaire was also given by some patients. Comments referred to some repetition within survey questions and that many questions should be split between nursing staff and doctors to make it easier to give an accurate response. One comment also suggested that the tablet was a little awkward to use. Positive feedback was also given on the survey, with patients praising it as an excellent tool to improve health care.

Data collection challenges encountered

Staff at some sites explained that weekly results from patient experience data collection were not shared with nurses on the participating wards or departments. Some matrons found the weekly reports ‘too time-consuming’ to read or ‘too difficult to understand’ and did not cascade them down to their staff. In addition, ward staff were not given sufficient time to review the weekly reports as part of their regular duties. Hence, the case study sites requested a printable dashboard report page that could provide them with an overview of progress and allow them to quickly identify any areas that received lower scores.

At this time, staff were unaware if any changes or improvements had already been made based on the collected patient experience feedback. With the exception of staff from one case study site, all workshop participants found that the research project at their trust needed more management and leadership support in order to drive improvements. Patient experience boards, if available, should also be informed of the research and involved where possible.

The two case study sites using stationary kiosks in the A&E departments found that these did not generate a comparable number of data to those generated by the tablet-based approach, despite staff directing patients to use the equipment.

Some sites found that they did not have sufficient volunteers to continue with the data collection, as some of the initially trained volunteers stopped collecting data for the project. Volunteers stopped data collection because of personal preferences or being asked to move on to other tasks by the trust. Some volunteers also found it difficult to access the wards as a result of ‘protected times’, such as mealtimes.

Areas for improvement

Communication between staff and patients was a common theme for improvement identified by case study sites. Specifically, not all patients understood their conditions, treatment plans and medications, although these had been communicated to them by their doctors. Strategies or action points developed to improve communication and understanding included spot checks of consultants to gauge patients’ understanding, nurses restating or explaining again what consultants had previously described to a patient and consultants providing information to patients in writing. Staff introductions were also reintroduced as an area of focus to help patients feel more informed. Some sites planned a fortnightly coffee hour with the matron when families and patients could discuss any questions in a safe and supportive environment.

Management of patient expectations was also seen as an area for improvement for the case study sites. Specifically, staff wanted to make patients more aware of what they could expect from their hospital stay and how long it generally takes before call bells are answered.

In addition to improving staff communication with patients, staff across the sites identified a need for greater engagement of the participating ward/department staff with the project. Staff suggested that involvement of the trust leadership and the inclusion of key stakeholders, such as patient experience board members, could encourage staff to actively engage with the project and utilise the collected patient experience data. By including the project on a trust’s quality agenda, continued engagement and sustainable changes could be fostered.

Staff also planned to share the patient feedback with each other as part of regularly scheduled monthly ward meetings. Wards recognised the need to integrate the results from this research with areas for improvement identified through other feedback collected at the trust, such as the FFT, informal feedback and call bell response time audits.

Challenges faced

Addressing challenges

To address the challenges described above, staff derived strategies to ensure the success of the NRTF approach.

Factors that make the near real-time feedback approach a success

Staff noted several factors that contribute towards the success of a NRTF approach to improving compassionate care. To reduce the burden of data collection for volunteers and patients, staff suggested the use of a shorter survey instrument. In addition, some suggested that each survey question should only have two instead of three answer options to facilitate recall.

To maintain volunteer engagement, recruitment efforts should target volunteers who wish to use technology and approach patients to collect patient feedback. This purposive recruitment is anticipated to contribute towards volunteer retention and provide greater benefits to volunteers.

The weekly reports should have an ‘at a glance’ section with graphs. Staff would like to see a single page with graphs and pictures that clearly highlight where improvements are needed.

Although the project was predominantly administered by R&D teams, retrospectively staff recognised that it should have been in the domain of the quality matrons and operational matrons to drive trust engagement. There should be involvement from everyone, at all levels of the trust. One staff member said, ‘We bought into the research as a trust but there should have been more done on a trust level, more board interaction, as opposed to having it purely in R&D’s domain’. Senior management also need to provide support, as the majority of staff did not have supporting professional activity time to review data, action plan and implement changes.

Improvement efforts based on weekly reports

Types of patient experience data collected

The first questions in the staff survey focused on patient experience data collected at the trust level (Table 28). Staff were asked to indicate the types of patient experience data that had been collected, at their trust, during the current year.

TABLE 28

Staff awareness of patient experience data collection (pre data collection only)

Pre data collection, staff most frequently reported the collection of patient experience data through national surveys (84.5%), followed by informal patient feedback (60.6%) and bespoke surveys developed exclusively for their ward, department or hospital (56.3%). Other patient experience feedback collected by the trust was highlighted:

Formal anonymised 360-degree feedback from a random sample of my patients for my revalidation.

Staff member

Methods used to collect patient experience data

Staff were asked about the methods used in their trusts to collect patient experience feedback (Table 29). The most common method was comment cards, reported by 88.7% of respondents, followed by conversations, reported by 50.7%. Only 4.2% of staff said that tablet-based surveys were used to collect patient experience feedback. Other methods used to collect patient feedback reported by staff were the use of tokens in boxes and a wipe board on which to record comments.

TABLE 29

Staff awareness of patient experience data collection methods (pre data collection only)

Focus of patient experience data

Staff also specified that most patient experience data collected focused on recommendations to family and friends (87.3%), relationships with staff, such as being treated with kindness, dignity and respect (76.1%), and interactions with staff (69.0%) (Table 30).

TABLE 30

Focus of patient experience data collection (pre data collection only)

Additional aspects of the patient experience, captured by data collection, were specified:

All aspects of stroke – patients are invited to comment on everything and anything.

Staff member

Meals and nutrition.

Staff member

General comments, such as ‘what we can improve and what we did well’.

Staff member

Timing of data collection

Staff were asked at what points within the patient journey feedback is collected (Table 31). For the pre-patient data collection survey, one-third of staff reported that data are collected during the patient’s stay (33.8%) and 25.4% said that data are collected at all points of the patients’ journey. Only 2.8% of staff said that data are collected on a patient’s admission to hospital.

TABLE 31

Time point of patient experience data collection

The post-data collection survey showed similar results, with again around one-third of respondents saying that most patient experience data collection is carried out during the patient’s stay (35.4%). This was followed by 23.6% saying that collection are carried out at all points of the patient’s journey. Again, admission to the hospital was the least reported time point for data collection (7.9%).

Other data collection points reported by staff were at the point of discharge or shortly before discharge.

Types of patient experience data collected

Staff were asked to indicate the types of patient experience data collected on their ward or department during the past year (Table 32). Similar results were reported in both the pre- and post-data collection surveys.

TABLE 32

Types of patient experience data collection employed on wards

The majority of staff specified that national surveys were used (81.7%, pre; 62.4%, post), followed by informal patient feedback (52.1%, pre; 60.1%, post). Other types of patient data collection at the ward/department level were stated as:

Personal letters/cards.

Staff member

Suggestions box.

Staff member

Methods used to collect patient experience data

Staff were asked to describe the methods used to collect patient feedback during the current year. Similar responses were given pre and post data collection (Table 33). Comment cards were mentioned by the majority of staff (83.1%, pre; 88.2%, post), followed by conversations (49.3%, pre; 58.4%, post) and paper surveys (26.8%, pre; 32.6%, post). Pre data collection, the method reported by staff to be used the least was telephone surveys (5.6%). Post data collection, the method used the least was focus groups (3.4%).

TABLE 33

Methods used to collect patient experience feedback

Visits by a dedicated patient experience team and the use of tokens were other methods identified by staff.

Focus of patient experience data

Pre data collection, staff reported that data collection most often focused on recommendations to family and friends (81.7%), followed by relationships with staff (76.1%) and interactions with staff (70.4%). Waiting times (19.7%) were mentioned the least. Post data collection, the majority of staff reported that data collection focused on interactions with staff (71.9%), followed by recommendations to family and friends (69.7%). Just over one-quarter of respondents said that medications were the main focus of surveys (29.2%) (Table 34).

TABLE 34

Parts of the patient experience covered in data collection

Other aspects of patient experiences captured by data collection at the ward or department level were specified as:

All aspects important to patients or generic experience.

Staff member

Meals and nutrition.

Staff member

Pre data collection, some differences were observed in the percentage of staff reporting a focus on waiting times between those working on the study (23.8%) and those working on the control (13.8%) wards/departments. Differences were also observed post data collection between the study and control wards/departments for waiting times (41.5% and 30.5%, respectively), recommendations to family and friends (72.6% and 61.0%, respectively) and relationships with staff (63.7% and 71.2%, respectively).

Communication of results

Staff were asked how results from the patient experience data collection were communicated in their ward or department (Table 35). Here, communication on staff noticeboards was mentioned most frequently (62.0%, pre; 59.0%, post), followed by communication during staff meetings (59.2%, pre; 53.4%, post). A complete lack of communication of results within the ward or department was reported least frequently (1.4%, pre; 4.5%, post).

TABLE 35

Communication of results to staff

The following other methods of communication were specified by staff:

Results are sent via e-mails to every staff on the ward.

Staff member

Communication folder.

Staff member

Newsletter.

Staff member

Staff were asked who communicates results from the patient experience data collection to them (Table 36). Staff most frequently reported communications from ward managers (77.5%, pre; 87.1%, post), followed by patient experience leads (21.1%, pre; 9.0%, post). Board members were reported as the staff group least likely to communicate patient experience data to staff (2.8% pre/post data collection). Other individuals communicating patient experience results were specified as:

Consultant.

Staff member

Department administrator or communications team.

Staff member

Matron or ward sister.

Staff member

Clinical director or operational lead.

Staff member

TABLE 36

Staff members communicating results to colleagues

Similar responses were provided by staff working on study and control wards.

Staff were then asked to indicate how they would like to have the results of patient experience data collection communicated to them (Table 37). Most staff selected communications during staff meetings (57.7%, pre; 57.9%, post) and on staff notice boards (57.7%, pre; 51.7%, post). Only 1.4% (pre) and 2.2% (post) of staff reported that patient experience data were not communicated to them.

TABLE 37

Preferred method of communicating patient experience results

Other preferred reporting formats included:

Via e-mail.

Staff member

[I] think this needs to be a more formal process where themes are pulled out, prioritised and actions derived.

Staff member

Pre data collection, some differences were noted for preferences for online portal or trust intranet communications (38.1% and 17.2%, respectively) by staff working on study and control wards. No differences were identified post data collection.

When asked about the frequency of communication, most staff indicated that results of patient experience data collection are communicated to them monthly (60.6%, pre; 23.6%, post) or weekly (22.5%, pre; 9.0%, post). Daily communication of results was not reported by any staff members in the pre data collection survey and by only 5.1% in the post data collection survey. Less than 2% of staff reported results being communicated annually (1.4%, pre; 1.7%, post) (Table 38).

TABLE 38

Frequency of communication of results

The following other frequencies of communication were specified:

As they come in.

Staff member

It varies when they are communicated.

Staff member

Pre data collection, some differences in frequencies of communication can be observed between the study and control ward staff for weekly communications (26.2% and 17.2%, respectively). Post data collection, differences were identified between selections for monthly (21.2% and 30.5%, respectively) and weekly (11.5% and 5.1%, respectively) communication.

Staff were then asked to indicate their preferred frequency for the communication of patient experience results (Table 39). Similar to current frequencies of communication, as reported in Table 38, staff preferred monthly communication (63.4%, pre; 53.9%, post), followed by weekly communication (25.4%, pre; 25.3%, post). Annual communication of results was preferred by the fewest respondents (1.4%, pre; 1.1%, post). Some differences between respondents from study and control wards can be observed for weekly data collection, which was preferred by 31.0% of staff from study wards pre data collection and by 17.2% of staff from the control wards pre data collection. Similarly, differences were observed in preferences for monthly data collection, which was desired by 52.2% of staff from study wards post data collection and 61.0 % of staff on the comparison wards post data collection. These differences may be attributable to the priming of staff on study wards. Specifically, staff on these wards may expect weekly reporting of results, as this frequency is provided by the current project.

TABLE 39

Preferred frequency of communication of results

In addition, respondents indicated their preferred reporting format to facilitate understanding and use of the results presented (Table 40). Staff indicated that they would like to see reports that include example quotes detailing patients’ feedback in their own words (64.8%, pre; 47.8%, post). This was followed by preferences for written narrative (40.8%, pre; 42.1%, post) and charts and graphs (40.8%, pre; 29.2%, post).

TABLE 40

Preferred reporting format

Usefulness of patient experience data

Many staff indicated that they found patient experience data to be extremely useful (50.7%, pre; 40.4%, post) or very useful (25.4%, pre; 30.9%, post) for their work with patients. Only 1.4% (pre) and 1.1% (post) indicated that they found patient experience data to be not at all useful for their work (Table 41).

TABLE 41

Usefulness of patient experience data for staff’s work

To follow up on the perceived usefulness of patient experience data, staff were asked to explain their response selection provided for the previous question. Staff who considered the patient experience data to be extremely useful or very useful to their work provided explanations that can be grouped into three themes. Specifically, staff explained that patient experience data help them understand how patients and their families view their care:

Because it shows areas that need improvement to ensure that patients are getting a high level of care. Also it shows what patients feel is important to them.

Staff member

The patient perceives their care very differently and the impact we have as carers. It is good to get that perspective to change the way we think and nurse.

Staff member

Staff also explained that patient experience data help them identify areas for improvement and make changes to their services. Under this theme, improving care to meet higher standards was mentioned by several staff:

Informs us of areas for improvement and patient wishes – helps us identify areas of good practice.

Staff member

The only way we will improve the care given is to listen to our patients. This data will enable us to make changes where needed to keep our hospital at the top for great patient care.

Staff member

Staff who considered patient experience data to be extremely or very useful for their work also highlighted that patient feedback provided a source of motivation:

It can help us to improve standards as well as giving encouragement to staff who have been mentioned for doing a good job.

Staff member

Few staff members who considered patient experience data to be somewhat useful to their work explained their opinion. Some staff explained that patient feedback can help to make the patient’s stay more pleasant. However, one staff member felt that patient feedback was not necessary for their role, as they do not have a clinical-based patient interaction.

Other staff members highlighted challenges in collecting or interpreting patient feedback, such as awkwardness in handing out surveys:

I find it awkward . . . giving out feedback cards to patients. I feel that they will think that I only delivered good care because I want good feedback. I deliver good care regardless.

Staff member

Another member of staff questioned the content validity of the patient feedback data collection instrumentation, such as the FFT:

Patients tend to come to the nearest hospital in an emergency or perceived emergency. So asking if they would recommend the trust to family, etc., doesn’t really make sense because no one would travel miles to an emergency department in an emergency to attend one recommended by a family member or friend.

Staff member

Only one respondent, who considered patient experience data to be not very useful to their work, explained that data collection highlighted logistical issues beyond their power to change:

It mainly says that patients think there should be more staff which is not something we can change on the ward.

Staff member

The staff member who thought patient experience data to be not at all useful to their work explained that feedback may not include a sufficient level of detail to provide a meaningful basis for improvements:

[The] token system doesn’t give [an] opportunity to say what is good or bad about the dept [department]. Numbers are also very small.

Staff member

Using patient experience data to drive service improvements

In the first administration, staff were asked about their awareness of changes that have been implemented at their trust as a result of patient experience data collection (Table 42). In response to this question, staff were most likely to say that they were aware of changes implemented by themselves and colleagues (31.0%), with almost as many being aware of others who had implemented changes or not aware of any implemented changes (21.1% and 21.1%, respectively).

TABLE 42

Awareness of changes based on patient experience data (pre patient data collection)

To follow up on the previous question, staff were asked to identify the types of changes made at their trust as a result of patient experience data collection (Table 43). Again, this was only asked pre patient experience survey. Here, staff most frequently listed changes to the way care is provided to patients (73.9%) and changes to the way staff interact with patients (69.6%). Both categories can encompass relational aspects of care. Staff least frequently mentioned changes to the patient survey instrument (4.3%) as a result of patient experience feedback. Other changes mentioned by staff were:

Changes to how the ward runs.

Staff member

Signage to remind staff of care essentials, such as care bells.

Staff member

Waiting times for medications.

Staff member

TABLE 43

Types of changes implemented based on patient feedback (pre patient data collection)

Some differences can be noted between responses from staff on study and responses from those on control wards for changes to the way staff interact with colleagues (39.3% and 27.8%, respectively) and physical changes to the layout of the hospital, ward or department (28.6% and 38.9%, respectively).

Staff were also prompted to specify whether or not any changes have been made on their ward or department as a result of patient experience feedback (Table 44). Staff most frequently selected that they themselves and their colleagues have implemented changes at the ward level (32.4%, pre; 25.8%, post), followed by responses stating that the changes were made by other staff (23.9%, pre; 19.1%, post). Only 14.1% of staff were not aware of any changes made pre patient data collection, compared with 41.0% of staff post data collection.

TABLE 44

Staff implementing changes

Next, staff described the types of changes made on wards/departments based on results of patient experience data collection (Table 45). The majority of staff listed changes to the way staff interact with patients (75.0%, pre; 61.0%, post), followed by responses listing changes to the way care is provided to patients (72.9%, pre; 58.1%, post). Changes to the patient survey instrument were mentioned least frequently (4.2%, pre; 3.7%, post). Other changes listed by staff included enforcing staff introductions, reducing noise at night, discussing medication changes with the patient and allowing for added, visible information to increase positive communication and reduce anxiety:

TABLE 45

Types of changes made based on patient feedback

Folders at the end of beds to provide extra stroke info, contact numbers, etc.

Staff member

We now routinely leave a sign by the bed when a patient leaves the ward to reduce anxiety if a relative visits.

Staff member

Differences in the responses provided by staff on study and control wards (pre survey) were observed for changes to the way staff interact with colleagues (35.7% and 15.0%, respectively). Post data collection differences for study and control wards were also observed in staff interactions with patients (64.2% and 51.5%, respectively) and changes to the layout of the ward/department (25.4% and 18.2%, respectively).

Barriers that affect the collection and use of patient experience data

When asked to give examples of the barriers encountered when collecting and using patient experience feedback, staff reported that the most frequent barrier was the lack of staff time to administer surveys (43.7%, pre; 49.4%, post) (Table 46).

TABLE 46

Types of barriers which affect collection and use of patient feedback

Differences between study and control wards/departments were identified pre data collection. These differences were highlighted in cost (14.3% and 3.4%, respectively) and lack of staff interest (23.8% and 13.8%, respectively). Post data collection differences were also seen in lack of staff interest (20.4% and 11.9%, respectively), language barriers (13.3% and 6.8%, respectively) and patients receiving multiple surveys (8.0% and 1.7%, respectively).

Other barriers encountered include patients not wanting to complete forms and staff sometimes being so busy they forget to hand patients a survey card.

Using volunteers to collect data

Pre data collection, most staff indicated that they have used volunteers to collect patient experience feedback on their ward or department (39.4%). However, post data collection, most staff were not sure if they had used volunteers to collect data previously (34.3%).

Pre data collection, differences in responses for staff from study and control wards can be noted for those using volunteers (45.2% and 31.0%, respectively) and those unsure about the use of volunteers (21.4% and 41.4%, respectively). Post data collection differences can also be found for those using volunteers (38.9% and 20.3%, respectively) (Table 48).

TABLE 48

Use of volunteers for patient experience data collection

Next, staff were asked about their reasons for using or not using volunteers. Those who had used volunteers explained that they were useful in that they can provide time to patients, which is something that nursing staff are unable to do on many occasions as a result of time constraints. The volunteers also alleviate staff pressures by being able to spend time with the patients, which in turn allows staff to carry out more tasks. Staff also highlighted that volunteers are, generally, more available to and flexible with the patients. Other staff explained that patients and their families feel they can be more honest with volunteers about their experiences, as they do not provide care to patients. Volunteers were seen as ‘neutral’.

Another member of staff explained that volunteers can relate to patients if they have experienced similar health issues and so can build rapport with the patients:

[We] used stroke survivors as I thought they would understand how patients with stroke feel.

Staff member

Finally, some staff explained that they did not have a specific reason for using volunteers to collect data; instead, the volunteers ‘just appear on the wards and do work’.

Some staff who had not used volunteers for data collection explained that they were a ‘limited or non-existent resource’ in the hospital.

Staff who had used volunteers were asked to suggest any lessons learnt or new ideas that should be considered when working with volunteers. Suggestions focused on different ways of using volunteers in order to collect patient experience data:

Volunteers can be used to collect and give out data in the future. This saves staff a job.

Staff member

Organised hours that staff will be expecting them.

Staff member

Finally, staff who worked on the wards/departments involved with the After Francis project were asked to describe any concerns or thoughts about the introduction of volunteers as part of this project. The majority of comments reflected that staff had no concerns over using volunteers to collect patient experience data and that it was a good idea to use them. Volunteer safety on wards and the A&E departments, confidentiality of responses and volunteers’ knowledge or skills to interact with patients were raised by a few staff members.

Factors that promote improvements based on patient experience feedback

Factors that help improvement as discussed by staff are listed below.

Factors that hinder trusts in making improvements

Factors that hindered improvement within the trust as discussed by staff are as follows.

Benefits of near real-time feedback

The benefits perceived by staff regrading NRTF are outlined as follows.

Drawbacks of near real-time feedback

Drawbacks of NRTF which staff identified are as follows.

Benefits of volunteer use

Staff identified many benefits to using volunteers within trusts. These benefits are outlined as follows.

Considerations around volunteer use

The considerations around volunteer use within trusts, as identified by staff, are as follows.

Communication

Communication was the main challenge that staff experienced throughout the project, which caused delays with the weekly reports getting back to the ward staff on time. In addition, staff at one trust reported that, initially, volunteers were not approaching staff to check whom they could and could not interview.

Staff engagement

It was reported that some of the ward staff could be resistant to change, particularly when the focus was on implementing changes to target areas of improvements. Staff found that, with senior staff buy-in, overall engagement of other staff with the research improved. However, there was still resistance to improvements or changes, especially from the junior members.

Recruiting patients

Recruiting patients to participate in the survey was often a challenge because of the hospital settings. Staff found that it was harder to recruit on the wards because of the low patient turnover, resulting in lower patient recruitment numbers.

All trusts continued with other feedback methods, such as the FFT, after which some patients refused to complete the tablet-based survey, stating ‘no, we’ve already done that’. After combining the After Francis survey results with other patient experience data within the trust (e.g. the FFT scores), staff saw recruitment numbers improve.

Recruiting volunteers

Staff perceived that a large number of patients did not complete the survey. They suspected that not all volunteers were confident or comfortable enough to approach all patients to gather their feedback. It was highlighted that it was important to recruit volunteers specifically suited to data collection in the future.

Reporting format

It was difficult for the staff to interpret the weekly reports in the format that the research team provided. Some trusts tackled this challenge by creating their own shorter reports to distribute to wards if there were the resources to do so.

Reporting format

Staff reported that they found it difficult to understand the weekly feedback using the reporting format that the research team sent to them. After some discussion, staff explained that, although they found the free-text comments to be extremely valuable, the quantitative data needed to be displayed visually, using colours and graphs, rather than plain line graphs or bar charts. This would allow them to quickly view the data and identify areas of improvement.

Value of volunteers

Volunteers were seen to be a valuable resource throughout the project, as they have more time to spend talking to patients and can encourage patient participation in the survey. Volunteer interactions with patients also provided an opportunity for patient concerns to be resolved quickly, at times while the patients were still on the wards.

Implementing little changes often

It was found that, when addressing challenges in the feedback, ward staff were often more receptive to making small changes often, as opposed to making numerous large changes in one go. This ensured that staff were not overwhelmed by any sudden changes and gave them the time to incorporate changes within their daily routine before the next improvement was made.

Call bells

Call bell response times have repeatedly been a challenge for staff to address. One case study trust conducted an audit of response times and found that patients were often not waiting as long as they thought. As a result, the site’s wards began to display posters with the average wait time to help manage patients’ expectations of how long it may take to be seen by a member of staff once called.

Communication

Several challenges relating to communication were identified throughout the project: the way staff communicated with patients about their treatments and conditions, as well as the way staff communicated with their colleagues about staffing levels. In order to improve the delivery of information, trusts used boards to highlight patient feedback results and the number of staff available (and their roles) and to provide up-to-date information about waiting times.

Similarly, staff encouraged each other to reintroduce the ‘Hello, my name is . . .’ campaign and provide more information to patients regarding discharges. One staff member said:

We introduced the Ticket Home, which is a patient’s discharge summary . . . it was completely de-medicalised, everything was simple . . . people that were more forgetful or where the family needed something in writing, you were able to give them something to read and know what was going on.

Privacy

Privacy was a concern of patients, specifically in A&E when checking in. As a result, screens were put up around reception to give patients more privacy when speaking to the reception about their conditions.

Other changes

Some changes, although they did not specifically target relational aspects of care, were implemented to help improve the overall patient experience. At one site, these included employing a housekeeper whose role was to ensure that the patients on the ward had enough drinks, thus relieving the nurses of some pressure. Matrons and senior sisters worked with ward staff to make them more aware of how their verbal and behavioural communications could be perceived by patients. At another site, improvements were made to the overall environment, such as repainting to brighten up areas for the patients.