Introduction

Digital health is fast becoming a new determinant of health. Access to and use of digital services will soon influence both the care options available to individuals and the outcomes they gain from them.¹ In this context, a new challenge for the NHS is to know how to interpret online patient feedback in relation to other sources of data on patient experience, and if and how to act on this content to improve services. Online feedback may have advantages, such as timeliness and transparency, but anecdotally it is sometimes seen as providing unrepresentative information from just a few users and, at the extreme ends of feedback, from overly negative and very positive experiences. The overarching aim of this study was therefore to provide the NHS with the evidence required to make best use of online patient feedback to improve health-care delivery in combination with other local qualitative and quantitative information on patients’ experiences.

Background

Person-centredness is a fundamental pillar of health-care quality,^2,3 and patient experience is associated with patient safety and self-rated and objectively measured health outcomes for a wide range of disease and service areas.^4–6 Despite the importance placed on creating a patient-centred, responsive health system, a series of high-profile investigations, including those by Sir Robert Francis into the Mid Staffordshire NHS Foundation Trust,⁷ Sir Bruce Keogh’s investigation into struggling trusts⁸ and Don Berwick’s national review of patient safety,⁹ noted a failure at both the team and organisational level within the NHS in recognising and responding to feedback from patients and their families and carers.

At the same time, as most ‘traditional’ feedback mechanisms, such as surveys and complaints systems, are struggling to elicit good response rates and be used to make a difference, health-care providers are receiving large amounts of (often unacknowledged) commentary from patients and carers via the internet.^10–17 Gathering, interpreting and responding to solicited and unsolicited online consumer feedback is now established practice and fundamental to success in industries, such as retail, travel and hospitality industries.^18,19 In 2015, the UK Competition and Markets Authority estimated that online reviews influence £23B of consumer spending each year.²⁰ The digital consumer has become accustomed to leaving such feedback on products and services, and these industries harness crowdsourced evaluations to drive consumer choice and to inform service improvements, albeit this has not been without challenges, including the potential gaming and manipulation of feedback.

The internet is having a major impact on people’s relationships within health care and people are already commentating on their health experiences online.^10,21–25 UK and US data show that online feedback on health care is increasing and likely to continue to grow fast.^26,27 This includes comments on structured patient rating sites [e.g. NHS Choices (URL: www.nhs.uk), iWantGreatCare (URL: www.iwantgreatcare.org) and Care Opinion (URL: www.careopinion.org.uk)], and also unstructured and unsolicited commentary about treatment, health services and illness in online settings, such as blogs, forums and social media. (Note: in this document we use terms such as feedback or comments to refer to all of this solicited and unsolicited content.)

When we started this project, NHS England had just committed to using internet feedback as part of its vision for a digital NHS founded on the concepts of participation, transparency and transaction. NHS managers and health-care practitioners will therefore need to understand how to interpret, respond to and harness online content from patients. Patients, carers and the public need to understand how they can provide useful feedback to the NHS and what influence this can have. Yet, there is no consensus or clear policy about how and who should use online feedback to deliver NHS and patient benefit, and there is a very limited evidence base. Little is known about the people who provide online content on their experience of care, why they do this, whether or not there are issues of inequality and what influence this feedback has on other patients, practitioners and organisations. We need to understand the strengths, weaknesses and uses of the data. There is some limited work on this from outside the UK^14,28,29 (e.g. from surveys conducted by the Pew Research Center).¹⁴ However, research exploring motivation to provide feedback is sparse and the focus is on administrative procedures for handling complaints, rather than patients themselves.³⁰ In the USA, 40% of a nationally representative sample reported that online ratings were ‘very important’ in choosing a physician.¹³ In Germany, online raters were more likely to be younger, female and more educated.¹⁴ A small UK study suggested that the views of certain groups may be disproportionately represented in ratings.¹⁵

We need better data to provide a robust understanding of online feedback from a user’s perspective andthe role of online feedback in improving health-care services, and more information about the authors and receivers of feedback. We also need to understand individual, professional and organisational issues influencing the use of online feedback in health care. Many clinicians appear resistant to using online feedback, worrying about selection bias, vulnerability to gaming or malice, and have the concern that subjective patient experience and objective care quality may be only tangentially related.³¹ To the best of our knowledge, before this study, there were no representative data on health professionals’ attitudes to and experiences of online feedback and no in-depth analysis of the organisational issues to guide its use in NHS organisations.

Objectives

We therefore had three research objectives, each of which addressed gaps in the current evidence base:

1.

to identify the current practice and future challenges, for online patient feedback, and to determine the implications for the NHS

2.

to understand what online feedback from patients represents and who is excluded, with what consequences

3.

to understand the potential barriers to and facilitators of the use of online patient feedback by NHS staff and organisations, and the organisational capacity required to combine, interpret and act on patient experience data.

We also had a fourth ‘knowledge translation’ objective:

4.

to use the study findings to develop a toolkit and training resources for NHS organisations, to encourage appropriate use of online feedback in combination with other patient experience data.

Methods

The study comprised five projects, listed here and aligned with our three research objectives:

1. stakeholder consultation and evidence synthesis (scoping review) regarding use of online feedback in health care (to address objective 1)
2. questionnaire survey of the public on the use of online comment on health services (to address objectives 1 and 2)
3. qualitative study of patients’ and carers’ experiences of creating and using online comment (to address objectives 1 and 2)
4. survey and focus groups of health-care professionals (to address objectives 1 and 3)
5. ethnographic organisational case studies with four NHS secondary care provider organisations (to address objectives 1 and 3).

There was one minor change to the protocol during the course of the study. Our questionnaire survey of the public was originally going to form one part of the Oxford Internet Surveys (OxIS) for 2015, but OxIS did not take place and so we used exactly the same method but as a standalone survey.

Research team and advisers

The research team had quantitative and qualitative expertise in the areas of digital health and patient experience research, and included people with disciplinary backgrounds in health services research (especially in primary care and public health), sociology, science and technology studies, psychology, epidemiology and nursing and statistics, as well as a lay co-investigator with experience as an expert patient and blogger. This range of perspectives has been a particular strength throughout the programme, enabling us to examine findings through several different lenses. We also participated in the learning set established to bring together the other projects funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research (HSDR) programme under its themed call for patient experience projects. Our project was not originally submitted to this call, but given the obvious synergy of our topic area, the funders subsequently included it with these other studies. The learning set meetings were very helpful in building a community of collaborative researchers interested in this area, and in sharing our emerging findings and receiving constructive feedback to inform our methods, analyses and discussion.

The study was overseen by a Study Steering Committee (SSC) (see Appendix 1 for membership), which met approximately every 6 months. The full research team also met every 6 months, with a smaller core team meeting monthly. Our public and patient involvement (PPI) activity was led by the lay co-investigator, who was a full member of the project team and we were advised by a Patients, Carers and Public Reference Group (PCPRG) chaired by an independent lay representative, which met as needed and which also provided feedback via e-mail (full details of our PPI activity is in Chapter 7).

Structure of monograph

Each of our five projects is described in a separate chapter (see Chapters 2–6), with full details of methods and findings. Chapter 7 describes our PPI activity, and an overarching synthesis of findings and their implications is provided in Chapter 8. Research ethics considerations are covered in each project chapter.

The next chapter describes the first of our projects: the stakeholder consultation and evidence synthesis work.