Background:

There is a need to identify approaches to reduce medication errors. Interest has converged on ePrescribing systems that incorporate computerised provider order entry and clinical decision support functionality.

Objectives:

We sought to describe the procurement, implementation and adoption of basic and advanced ePrescribing systems; to estimate their effectiveness and cost-effectiveness; and to develop a toolkit for system integration into hospitals incorporating implications for practice from our research.

Design:

We undertook a theoretically informed, mixed-methods, context-rich, naturalistic evaluation.

Setting:

We undertook six longitudinal case studies in four hospitals (sites C, E, J and K) that did not have ePrescribing systems at the start of the programme (three of which went live and one that never went live) and two hospitals (sites A and D) with embedded systems. In the three hospitals that implemented systems, we conducted interviews pre implementation, shortly after roll-out and at 1 year post implementation. In the hospitals that had embedded systems, we conducted two rounds of interviews, 18 months apart. We undertook a three-round eDelphi exercise involving 20 experts to identify 80 clinically important prescribing errors, which were developed into the Investigate Medication Prescribing Accuracy for Critical error Types (IMPACT) tool. We elicited the cost of an ePrescribing system at one (non-study) site and compared this with the calculated ‘headroom’ (the upper limit that the decision-maker should pay) for the systems (sites J, K and S) for which effectiveness estimates were available. We organised four national conferences and five expert round-table discussions to contextualise and disseminate our findings.

Intervention:

The implementation of ePrescribing systems with either computerised provider order entry or clinical decision support functionality.

Main outcome measures:

Error rates were calculated using the IMPACT tool, with changes over time represented as ratios of error rates (as a proportion of opportunities for errors) using Poisson regression analyses.

Results:

We conducted 242 interviews and 32.5 hours of observations and collected 55 documents across six case studies. Implementation was difficult, particularly in relation to integration and interfacing between systems. Much of the clinical decision support functionality in embedded sites remained switched off because of concerns about over alerting. Getting systems operational meant that little attention was devoted to system optimisation or secondary uses of data. The prescriptions of 1244 patients were audited pre computerised provider order entry and 1178 post computerised provider order entry implementation of system A at sites J and K, and system B at site S. A total of 21,138 opportunities for error were identified from 28,526 prescriptions. Across the three sites, for those prescriptions for which opportunities for error were identified, the error rate was found to reduce significantly post computerised provider order entry implementation, from 5.0% to 4.0% (p < 0.001). Post implementation, the overall proportion of errors (per opportunity) decreased significantly in sites J and S, but remained similar in site K, as follows: 4.3% to 2.8%, 7.4% to 4.4% and 4.0% to 4.4%, respectively. Clinical decision support implementation by error type was found to differ significantly between sites, ranging from 0% to 88% across clinical contraindication, dose/frequency, drug interactions and other error types (p < 0.001). Overall, 43 out of 78 (55%) of the errors had some degree of clinical decision support implemented in at least one of the hospitals. For the site in which no improvement was detected in prescribing errors (i.e. site K), the ePrescribing system represented a cost to the service for no countervailing benefit. Cost-effectiveness rose in proportion to reductions in error rates observed in the other sites (i.e. sites J and S). When a threshold value of £20,000 was used to define the opportunity cost, the system would need to cost less than £4.31 per patient per year, even in site S, where effectiveness was greatest. We produced an ePrescribing toolkit (now recommended for use by NHS England) that spans the ePrescribing life cycle from conception to system optimisation.

Limitations:

Implementation delays meant that we were unable to employ the planned stepped-wedge design and that the assessment of longer-term consequences of ePrescribing systems was impaired. We planned to identify the complexity of ePrescribing implementation in a number of contrasting environments, but the small number of sites means that we have to infer findings from this programme with considerable care. The lack of transparency regarding system costs is a limitation of our method. As with all health economic analyses, our analysis is subject to modelling assumptions. The research was undertaken in a modest number of early adopters, concentrated on high-risk prescribing errors and may not be generalisable to other hospitals.

Conclusions:

The implementation of ePrescribing systems was challenging. However, when fully implemented the ePrescribing systems were associated with a reduction in clinically important prescribing errors and our model suggests that such an effect is likely to be more cost-effective when clinical decision support is available. Careful system configuration considering clinical processes and workflows is important to achieving these potential benefits and, therefore, our findings may not be generalisable to all system implementations.

Future work:

Formative and summative evaluations of efforts will be central to promote learning across settings. Other priorities emerging from this work include the possibility of learning from international experiences and the commercial sector.

Funding:

This project was funded by the National Institute for Health and Care Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 10, No. 7. See the NIHR Journals Library website for further project information.

Article history

The research reported in this issue of the journal was funded by PGfAR as project number RP-PG-1209-10099. The contractual start date was in November 2011. The final report began editorial review in February 2018 and was accepted for publication in September 2020. As the funder, the PGfAR programme agreed the research questions and study designs in advance with the investigators. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PGfAR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.