NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Headline
This research programme identified and prioritised a set of potential ambulance quality measures that reflected preferences of services and users.
Abstract
Background:
Ambulance service quality measures have focused on response times and a small number of emergency conditions, such as cardiac arrest. These quality measures do not reflect the care for the wide range of problems that ambulance services respond to and the Prehospital Outcomes for Evidence Based Evaluation (PhOEBE) programme sought to address this.
Objectives:
The aim was to develop new ways of measuring the impact of ambulance service care by reviewing and synthesising literature on prehospital ambulance outcome measures and using consensus methods to identify measures for further development; creating a data set linking routinely collected ambulance service, hospital and mortality data; and using the linked data to explore the development of case-mix adjustment models to assess differences or changes in processes and outcomes resulting from ambulance service care.
Design:
A mixed-methods study using a systematic review and synthesis of performance and outcome measures reported in policy and research literature; qualitative interviews with ambulance service users; a three-stage consensus process to identify candidate indicators; the creation of a data set linking ambulance, hospital and mortality data; and statistical modelling of the linked data set to produce novel case-mix adjustment measures of ambulance service quality.
Setting:
East Midlands and Yorkshire, England.
Participants:
Ambulance services, patients, public, emergency care clinical academics, commissioners and policy-makers between 2011 and 2015.
Interventions:
None.
Main outcome measures:
Ambulance performance and quality measures.
Data sources:
Ambulance call-and-dispatch and electronic patient report forms, Hospital Episode Statistics, accident and emergency and inpatient data, and Office for National Statistics mortality data.
Results:
Seventy-two candidate measures were generated from systematic reviews in four categories: (1) ambulance service operations (n = 14), (2) clinical management of patients (n = 20), (3) impact of care on patients (n = 9) and (4) time measures (n = 29). The most common operations measures were call triage accuracy; clinical management was adherence to care protocols, and for patient outcome it was survival measures. Excluding time measures, nine measures were highly prioritised by participants taking part in the consensus event, including measures relating to pain, patient experience, accuracy of dispatch decisions and patient safety. Twenty experts participated in two Delphi rounds to refine and prioritise measures and 20 measures scored ≥ 8/9 points, which indicated good consensus. Eighteen patient and public representatives attending a consensus workshop identified six measures as important: time to definitive care, response time, reduction in pain score, calls correctly prioritised to appropriate levels of response, proportion of patients with a specific condition who are treated in accordance with established guidelines, and survival to hospital discharge for treatable emergency conditions. From this we developed six new potential indicators using the linked data set, of which five were constructed using case-mix-adjusted predictive models: (1) mean change in pain score; (2) proportion of serious emergency conditions correctly identified at the time of the 999 call; (3) response time (unadjusted); (4) proportion of decisions to leave a patient at scene that were potentially inappropriate; (5) proportion of patients transported to the emergency department by 999 emergency ambulance who did not require treatment or investigation(s); and (6) proportion of ambulance patients with a serious emergency condition who survive to admission, and to 7 days post admission. Two indicators (pain score and response times) did not need case-mix adjustment. Among the four adjusted indicators, we found that accuracy of call triage was 61%, rate of potentially inappropriate decisions to leave at home was 5–10%, unnecessary transport to hospital was 1.7–19.2% and survival to hospital admission was 89.5–96.4% depending on Clinical Commissioning Group area. We were unable to complete a fourth objective to test the indicators in use because of delays in obtaining data. An economic analysis using indicators (4) and (5) showed that incorrect decisions resulted in higher costs.
Limitations:
Creation of a linked data set was complex and time-consuming and data quality was variable. Construction of the indicators was also complex and revealed the effects of other services on outcome, which limits comparisons between services.
Conclusions:
We identified and prioritised, through consensus processes, a set of potential ambulance service quality measures that reflected preferences of services and users. Together, these encompass a broad range of domains relevant to the population using the emergency ambulance service. The quality measures can be used to compare ambulance services or regions or measure performance over time if there are improvements in mechanisms for linking data across services.
Future work:
The new measures can be used to assess different dimensions of ambulance service delivery but current data challenges prohibit routine use. There are opportunities to improve data linkage processes and to further develop, validate and simplify these measures.
Funding:
The National Institute for Health Research Programme Grants for Applied Research programme.
Contents
- Plain English summary
- Scientific summary
- SYNOPSIS
- Identifying potential measures to assess ambulance service performance and quality of care
- Creating a linked data set
- Developing case-mix-adjusted performance indicators
- Introduction
- Indicator development methods
- Summary of indicators developed
- Indicator 1: mean change in pain score
- Indicator 2: proportion of serious emergency conditions correctly identified at the time of the 999 call
- Indicator 3: response time
- Indicator 4: proportion of decisions to leave a patient at scene (‘hear and treat’ and ‘see and treat’), which were potentially inappropriate
- Indicator 5: proportion of patients transported to emergency department by 999 emergency ambulance, but who were discharged to usual place of residence or care of general practitioner without treatment or investigation(s) that needed hospital facilities
- Indicator 6: proportion of ambulance patients with a serious emergency condition who survive to admission, and to 7 days post-admission
- Summary of the indicator development
- Patient safety in prehospital ambulance care: exploratory structured judgement case record review study
- Patient and public involvement
- How useful could the indicators be in the future and what are the costs associated with different clinical management options?
- Discussion and conclusions
- Acknowledgements
- References
- Appendix 1. Systematic searches and summary of identified measures
- Appendix 2. Assessment criteria for identifying indicator set
- List of abbreviations
About the Series
Article history
The research reported in this issue of the journal was funded by PGfAR as project number RP-PG-0609-10195. The contractual start date was in June 2011. The final report began editorial review in November 2017 and was accepted for publication in November 2018. As the funder, the PGfAR programme agreed the research questions and study designs in advance with the investigators. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PGfAR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
Anne Spaight reports grants from East Midlands Ambulance Service NHS Trust during the conduct of the study. Steve Goodacre is a member of the Health Technology Assessment (HTA) Clinical Trials Board, HTA Funding Boards Policy Group and HTA IP Methods Group. Helen Snooks is a member of National Institute for Health Research HTA and Efficacy and Mechanism Evaluation Editorial Board.
Last reviewed: November 2017; Accepted: November 2018.
- NLM CatalogRelated NLM Catalog Entries
- Prioritizing novel and existing ambulance performance measures through expert and lay consensus: A three-stage multimethod consensus study.[Health Expect. 2018]Prioritizing novel and existing ambulance performance measures through expert and lay consensus: A three-stage multimethod consensus study.Coster JE, Irving AD, Turner JK, Phung VH, Siriwardena AN. Health Expect. 2018 Feb; 21(1):249-260. Epub 2017 Aug 25.
- Prehospital early warning scores for adults with suspected sepsis: the PHEWS observational cohort and decision-analytic modelling study.[Health Technol Assess. 2024]Prehospital early warning scores for adults with suspected sepsis: the PHEWS observational cohort and decision-analytic modelling study.Goodacre S, Sutton L, Ennis K, Thomas B, Hawksworth O, Iftikhar K, Croft SJ, Fuller G, Waterhouse S, Hind D, et al. Health Technol Assess. 2024 Mar; 28(16):1-93.
- Review Patient safety in ambulance services: a scoping review[ 2015]Review Patient safety in ambulance services: a scoping reviewFisher JD, Freeman K, Clarke A, Spurgeon P, Smyth M, Perkins GD, Sujan MA, Cooke MW. 2015 May
- Review Configuration of vascular services: a multiple methods research programme[ 2021]Review Configuration of vascular services: a multiple methods research programmeMichaels J, Wilson E, Maheswaran R, Radley S, Jones G, Tong TS, Kaltenthaler E, Aber A, Booth A, Buckley Woods H, et al. 2021 Apr
- Review Identifying models of care to improve outcomes for older people with urgent care needs: a mixed methods approach to develop a system dynamics model.[Health Soc Care Deliv Res. 2023]Review Identifying models of care to improve outcomes for older people with urgent care needs: a mixed methods approach to develop a system dynamics model.Conroy S, Brailsford S, Burton C, England T, Lalseta J, Martin G, Mason S, Maynou-Pujolras L, Phelps K, Preston L, et al. Health Soc Care Deliv Res. 2023 Sep; 11(14):1-183.
- Developing new ways of measuring the quality and impact of ambulance service car...Developing new ways of measuring the quality and impact of ambulance service care: the PhOEBE mixed-methods research programme
Your browsing activity is empty.
Activity recording is turned off.
See more...