Background

The NHS provides emergency care for a diverse population of patients who need medical care for a wide range of conditions and have different levels of urgency. For many patients, the first point of contact with the NHS is when they request help from the ambulance service.

Demand for ambulance services has been steadily increasing for many years. In 1974, ambulance services in England responded to 1.5 million emergency (‘999’) calls per year. By 2016/17 this had increased to > 6 million responses to almost 10 million calls.¹ In the past, the main purpose of the ambulance service was to respond to serious emergencies and transport patients to the nearest hospital emergency department (ED). As the number of calls has grown, so has the type of health problems people call 999 for.

Ambulance services now provide care for conditions that range from life-threatening emergencies, such as heart attacks, stroke and serious injury, a wide range of illnesses and problems associated with chronic disease and long-term conditions, to relatively minor illness or injury. In response to these changes, ambulance services have also adapted and developed.

In 2005, the Department of Health and Social Care policy document Taking Healthcare to the Patient: Transforming NHS Ambulance Services² recognised that patients who call 999 should receive care that not only is timely but also best meets their clinical needs. Patients with serious medical emergencies and injury still need a fast response, early treatment and good clinical care at the scene, followed by transport to hospital. However, research evidence also showed that, for some patients with less urgent problems, a trip to the ED was not always necessary and their needs could be better met by providing the care at home or in a community service.³

As a result of the 2005 report, and subsequent broader policy initiatives focusing on delivering care closer to home for urgent problems and developing expert centres for specialist care such as stroke units and major trauma centres for emergencies,⁴ there have been a variety of innovations in types of response provided by the ambulance service. These types of response now fall into three main categories:

1. More detailed clinical telephone assessment of some 999 calls by nurses or paramedics so that patients who do not need an emergency ambulance can be provided with self-care advice or referred to the right service – ‘hear and treat’.
2. Development of paramedic skills including advanced paramedic practitioners so they can treat minor illness and injury at home and, when needed, refer to other services for follow-up care, for example community falls services – ‘see and treat’.
3. Improving prehospital assessment and care pathways so that patients who need specialist care can be taken straight to the best facility (e.g. a stroke or heart attack centre) and other patients taken to the nearest ED – ‘see and convey’.

The changes in the way that ambulance services provide care have contributed to broader efforts to improve emergency and urgent care by developing a more ‘joined up’ system approach in which ambulance, accident and emergency (A&E), and community services work in partnership to ensure that patients can access and receive care in the right place and at the right time.⁴

These changes also need to reflect the NHS principles set out in the Next Stage Review,⁵ which are to provide high-quality services through improving effectiveness, safety and patient experience. If these aims are to be achieved, then the assessment and monitoring of the quality of services and the wider system in which they operate become important tasks.⁶ This means that we need ways of measuring how well ambulance services are performing, in terms of both the services they are delivering and the impact that these services have on the patients that they care for.

Quality assessment and quality improvement can be achieved only if we can find ways of routinely and consistently measuring these important aspects of patient care so that we can identify where care is good and where it needs to be improved. Measuring performance and quality of care is never easy but is particularly difficult for ambulance services, as they provide care to such a diverse group of patients and those patients will be in their care for a relatively short period of time only.

For many years, the quality of ambulance service care has been mainly assessed by measuring how quickly they respond to 999 calls – response time performance. This is not unique to the UK and has been the predominant performance or quality measure for Emergency Medical Services (EMS) internationally.⁷ Response time has been used as a proxy measure because of the relationship between speed of response and survival following out-of-hospital cardiac arrest – the faster the response, the more likely a patient is to survive.⁸ However, cardiac arrest accounts for a very small proportion of 999 calls only, currently 0.6% in England,⁹ and research evidence shows that, for other calls, the speed of response has little impact on survival or outcomes.^10–12 Response time also tells us nothing about what clinical care was given or the impact of that care on patient outcomes so, overall, it is a poor measure of service quality for the vast majority of patients who request an ambulance.

The need to develop better ways of measuring ambulance service performance and quality of care, particularly the effect on patient outcomes that are relevant to all people who use the service, not just a few, has been recognised for some time. This need was a key recommendation of Taking Healthcare to the Patient: Transforming NHS Ambulance Services² and also a high priority in a UK Delphi study assessing priorities for prehospital care research.¹³ Despite this recognition, a review of outcome measurement in prehospital care found few research studies investigating this area and found that most published literature focused on discussion of the need to develop measures rather than any solution to the problem.¹⁴

In England, some progress has been made with the introduction of a set of 11 ambulance service quality indicators adopted in 2011.¹⁵ Having a set of indicators, rather than just a single measure of response time performance, that had relevance to all calls was a big step forward, as was the inclusion of a small number of clinical indicators designed to measure the care provided for some key conditions like cardiac arrest, stroke and heart attack.¹⁶ However, the focus remained on processes [what the ambulance service did in terms of response times, type of service (e.g. how many calls are managed using ‘hear and treat’) and providing treatment for a small number of conditions] rather than how care affected patients. It is also fair to say that these indicators or measures have mainly been developed by health professionals and academics so may not include aspects of care that are important to patients.

There has been a major problem holding back the development of better ways of measuring how well ambulance services provide care to the population who call 999 and that was the lack of information available to them about what happens to patients and their outcome once they had left ambulance care. If the only information they have is their own data about how they respond and what they do then, it is not surprising that these are the only things that were measured. These data are also incident rather than patient based, which makes it difficult to monitor related calls for the same patient. However, if ambulance service information about patients could be linked to, for example, hospital information, so that they have a more complete picture of what happens to patients and their outcomes, then a better assessment could be made of the impact of care and the benefits it produced. Creating linked data sets can also potentially add value by creating comprehensive information sources that can be used to provide feedback on outcomes and improve performance and practice, but this was not something that was being done routinely.

This programme was designed to try and resolve these problems by bringing together the two key themes of:

1. developing more meaningful ambulance service performance and quality measures that reflect the key principles of quality, safety and effectiveness of care
2. creating linked data sets that can combine ambulance service, hospital and mortality data to provide the information needed to support better performance and quality measurement.

Even with a linked data set that includes patient outcomes, measuring impact is problematic. Outcomes can be influenced by a whole range of factors in addition to ambulance care received, such as a patient’s age, illness severity or hospital care. One way of overcoming this is to develop performance measures that can take account of these intrinsic and extrinsic factors by using risk or case-mix adjustment models. These have been successfully developed and used for research and audit in settings such as trauma¹⁷ and intensive care¹⁸ and have shown they can make an important difference to improving processes and outcomes of care.

Aims and objectives

The aim of this programme was to develop new ways of measuring the impact of care provided by ambulance services. This could support quality improvement by providing information to monitor and audit service performance and assess the impact of change through service evaluation. The programme was called PhOEBE (Prehospital Outcomes for Evidence Based Evaluation).

Its objectives were to:

1. review and synthesise the research literature on prehospital care outcome measures and use consensus methods to identify a small set of measures relevant to the NHS and patients for further development
2. create a data set linking routinely collected prehospital (ambulance service) data, hospital data and mortality data to provide outcome information
3. use the linked data to develop the measures identified in objective 1 by building case-mix adjustment models that could potentially be used to assess ambulance service performance and detect change over time with repeated measurement
4. explore the practical use of the linked data set and the case-mix adjustment models to measure the effectiveness and quality of ambulance service care and assess how they can be best used to support quality improvement strategies.

Programme design

The programme was designed to be conducted in four linked stages or workstreams (Figure 1).

FIGURE 1

The PhOEBE programme overview. APC, admitted patient care; CAD, computer-aided dispatch; ePRF, electronic patient report form; ONS, Office for National Statistics.

• Workstream 1: a review and synthesis of the evidence on ambulance service-related performance and quality measures to produce a list of potential measures. Then use consensus methods to assess and prioritise these measures and to identify a small number that are relevant to different stakeholders, including patients and NHS staff, research academics and patient representatives for further development.
• Workstream 2: linking ambulance service call information and patient information from the care record completed by ambulance clinicians with routine ED and hospital information [Hospital Episode Statistics (HES)] and national records of patient deaths. This creates a single data set that follows what happens to each patient who makes a 999 call.
• Workstream 3: explore and develop case-mix-adjusted models for processes and outcomes in patients attended by the ambulance service using the linked data. This allows us to assess whether or not case-mix adjustment is needed to improve the usefulness of process or outcome measures, and their potential as indicators to measure quality and performance between services or within services over time.
• Workstream 4: testing the risk adjustment models to assess if they can be used to measure effectiveness and quality. This looks at how they might be used in practice and explores with users (patients and staff) how useful they are and how they might be best implemented in the NHS. In addition, the linked data can be used to estimate the costs of different types of ambulance response.

Figures 1 and 2 provide a summary of the four workstreams and how they are linked together.

FIGURE 2

Workstream 1. PPI, patient and public involvement.

The programme was overseen and supported by a project management group that included all of the programme collaborators, the research team, representatives from the two ambulance services taking part and a public involvement member. We also had a project steering committee comprising key members of the management group and external advisors representing ambulance services, the College of Paramedics, NHS commissioners and emergency care research. The programme was carried out over the 6-year period of 2011–17.

Public involvement

Public involvement made an important contribution to the PhOEBE programme. At the outset we formed a public involvement reference group of three members who provided substantial support and input to each stage of the programme, both individually and by providing links to other relevant external groups. Our public involvement reference group not only contributed advice but also co-produced some of the programme work and outputs. We describe the patient involvement work in more detail under Patient and public involvement below.

Changes from the original proposal

During the course of the programme we encountered a major problem during workstream 2. The data linkage component was contracted to the then Health and Social Care Information Centre (HSCIC) (now NHS Digital). NHS Digital held the central HES data and also provided a trusted data linkage service. We planned to use this service to link the ambulance service, hospital and Office for National Statistics (ONS) mortality data. The plan was to obtain the first set of linked data by the end of year 2 and a second set in year 3 (2013–14).

During 2013, major data security issues arose at NHS Digital and, as a result, there was a major review and restructuring of the organisation. This meant that no data were released for (any) research use for almost 2 years. These issues are described in more detail in Creating a linked data set but the overall impact was that, because of these external delays, we did not receive a linked data set until October 2016 – 4 months after the original expected end date of the programme. We were fortunate to obtain a 1-year extension to the programme, which enabled us to complete the workstream 3 work to develop the risk-adjusted measures, albeit within a much reduced time frame.

The delays meant that we were unable to conduct much of the planned work for workstream 4, principally because the second linked data set that we had intended to use to test and validate the measures developed in workstream 3 arrived too late and we had no time left to complete this. The implications of these delays and the difficulties in obtaining and managing the linked data are discussed in more detail in Discussion and conclusions.