Introduction

A decision to implement health information technology should carefully weigh the costs and benefits of incorporating it into the clinical environment. This is especially true in settings involved in the healthcare of infants and children, where patterns of practice and the needs of clinicians are unique. A recent report issued by the medical informatics taskforce of the American Academy of Pediatrics (AAP) cited a number of special requirements for the effective use of electronic medical record (EMR) systems in pediatrics. ¹⁵ The practice of primary care and subspecialty pediatrics requires specialized collection of growth data, immunization history, longitudinal developmental inventories, parent education, age- and weight-based norms and dosing of therapeutics, specialized terminologies, and unique school-based forms and reports.

In the area of pediatric patient-safety, a growing number of studies have described the frequency of medication errors and adverse drug events (ADEs) in both the inpatient and ambulatory settings. ^{16 – 19} For a number of reasons—including weight- and age-based medication dosing, medication unit-doses designed for adult patients, and the limited ability of children to communicate or self-check medications before they are administered, ^{20, 21} —infants and children are at higher risk for serious medication errors and resultant ADEs than are adults. HIT is believed to be a vital component in the quest to improve medication safety in pediatrics.

These special requirements, combined with a small commercial market for pediatric HIT systems relative to the adult population, make the implementation of HIT in the pediatric setting challenging and perhaps costly. Clearly, more must be known about the relative costs and benefits of HIT implementation and use in pediatrics and evidence of its impact on the six quality aims identified in the IOM report, Crossing the Quality Chasm, ²² to deliver safe, effective, efficient, patient-centered, timely, and equitable healthcare.

Literature

Of the 256 articles included in the database, 14 articles were determined to contain quantitative data on the costs and/or benefits of HIT use in the pediatric healthcare setting. Because of a paucity of evidence, we also included descriptive quantitative studies in this section.

Summary of Evidence

Medication Use and Patient Safety. Given recent insight into the prevalence of medication errors in the pediatric population, health information technology is believed by most to be an important tool in reducing the rate of medication errors that occur in the care of infants and children.

Mullett et al. ²³ enhanced an existing adult antiinfective computerized decision-support system for use in an academic pediatric intensive care unit (PICU) and measured its impact on medication-related outcomes. The study reported a 59-percent decrease in pharmacist interventions for erroneous drug doses and a decreased number of patient days of subtherapeutic (p<0.001) or excessive (p<0.001) antiinfective doses. In addition, the surveyed physicians reported that the use of the system improved their antiinfective choices and perhaps reduced the likelihood of ADEs. The authors also reported a decreased number of orders per patient-antiinfective course as well as decreased robust estimated costs of antiinfective use by 9 percent in the intervention group vs. control ($86.60 vs. 78.43).

A study by Fortescue and colleagues ²⁴ examined and characterized 616 medication errors occurring in the pediatric inpatient units of two academic tertiary referral medical centers. In a hypothetical experiment, physician experts determined what percentage of these errors could potentially have been prevented by the implementation of safety systems. Specifically, this hypothetical experiment determined that basic CPOE would avert 60 percent of potentially harmful errors, while CPOE with clinical decision-support systems (CPOE +CDSS) would increase the prevention of harmful errors to 75.8 percent. Other HIT systems identified by the report as being important for averting medication errors in pediatrics settings included computerized/electronic medication administration record (e-MAR) (19.2 percent of potentially harmful errors), robots in pharmacy (2.5 percent), smart intravenous infusion devices (4.2 percent), medication and patient and staff bar-coding (4.2 percent), and an automated bedside medication dispensing device (5.8 percent).

A number of studies have directly measured the benefit of CPOE using a variety of error-capture methodologies and study designs in different pediatric clinical environments. In a prospective cohort study, the authors documented medication prescribing errors (MPEs) and potential adverse drug events (PADEs) in a pediatric intensive care unit before and after implementation of a “home-grown” CPOE system. ²⁵ The data showed a significant reduction of both MPEs (30.1 to 90.2 percent, p< 0.001) and PADEs (2.2 to 1.3 percent, p<0.001). A study by Cordero and colleagues in the neonatal intensive care setting (NICU) showed that CPOE could eliminate gentamicin prescribing errors as well. ²⁶ The sum of this early evidence indicates that CPOE +CDSS has significant potential to reduce harmful medication errors, but the relative costs and complexities of achieving these beneficial outcomes need to be examined further.

Immunizations. Although a growing body of literature suggests that the use of HIT in pediatrics may be an important ingredient in reducing medication errors, a key challenge for pediatric providers lies in the area of maximizing adherence to vaccination recommendations. Paper-based immunization records do not allow for rigorous population-based monitoring or quality control. Therefore, computerized immunization registries, as separate or integrated systems and with clinical decision-support or reporting capabilities, offer tremendous potential in tracking and improving the rates of adherence to recommended immunization guidelines.

Ornstein et al. ²⁷ evaluated a computer-based preventive services alerting system integrated into an electronic medical record system in an academically affiliated family practice clinic. In addition to surveying patient and physicians regarding their perceptions of the reminder system, the researchers performed before-and-after audits of adherence to recommended preventative services including childhood immunizations. Of the five immunization services tracked, only the administration of diphtheria and tetanus booster showed a small but significant improvement (48.8 to 50.6 percent, p=0.02). Adherence to the other recommended vaccinations did not show a significant improvement.

Szilagyi, and colleagues, ²⁸ in an academically affiliated pediatric urban clinic, used a computerized database system to generate reminder letter for influenza vaccination to patients identified with moderate to severe asthma. Eligible patients were randomized into an intervention group, which received the reminders, and a control group. After four months, a review of the medical chart revealed a significant difference in influenza vaccination rates (30 percent intervention vs. 7 percent control, p<0.01). This study demonstrated that computerized disease registry systems could serve as an important tool in improving vaccination rates in pediatrics.

Effective Disease Management. In addition to providing a potential means to influence prescribing and immunization practices in pediatrics, HIT systems also hold tremendous promise in improving clinical decisionmaking and disease management.

Medication Dosage and Delivery. Chiarelli et al. ²⁹ evaluated a microprocessor device with computerized algorithms for insulin dose adjustment for pediatric patients with insulin-dependent diabetes mellitus, based on self-monitored blood glucose (SMBG) levels. This prospective randomized on-off-on study revealed that although the mean glycosylated hemoglobin levels and pre-meal SMBG levels did not improve, control patients were more likely to experience episodes of hypoglycemia than were patients using the device, and patients using the device used less insulin than during their corresponding baseline phase (p<0.0001) and less insulin than the control group.

Disease-Based Clinical Decision Support. Schriger and colleagues ³⁰ implemented an electronic medical record in a university hospital emergency department that provided documentation advice and recommendations for laboratory testing and treatment. Using an on-off-on interrupted study design, the authors measured appropriateness of care for febrile children less than three years of age, when measured against an evidence-based guideline. No evidence was found for improvements (or worsening) in appropriateness of care during the intervention phase compared to the baseline phase. However, use of the system was found to increase documentation of essential elements of the history and physical examination by 13 percent (95% confidence interval, 10 to 15 percent) as well as documentation of after-care instructions by 33 percent (95% confidence interval, 28 to 38 percent).

The appropriate course of antibiotic treatment for acute otitis media (AOM) is an area of concern in pediatrics. A study by Christakis et al. ³¹ measure the impact of HIT on the antibiotic prescribing behavior of pediatric providers in an academic pediatric residency training clinic and compared cohorts during the pre-intervention and post-intervention phases. During the post-intervention phase, providers were randomized to receive point-of-care advice recommending a course of antibiotics of less than 10 days duration (primary outcome) or delayed initiation of antibiotics (secondary outcome) for the treatment of AOM. Measurement of adherence to this computerized alert showed that providers in the intervention group had a 34-percent increase compared to the control group in the proportion of antibiotic prescriptions that were for less than 10 days (p<0.01). However, during the intervention period, both the intervention and control groups became more likely to prescribe antibiotics, with the intervention group deteriorating less than the control group (p<0.095). The results demonstrate that the prescribing practices of pediatricians for treatment of a common pediatric illness can be affected by a computerized reminder system.

Using a similar study design, Margolis et al. ³² developed a computerized algorithm system that mandated structured input of data by providers for common pediatric problems. In return, the system provided recommendations for disease management and correct use of antibiotics. The investigators demonstrated decreased use of antibiotics for OM (p<0.001) and pharyngitis (p<0.01) as well as increased adherence to protocol recommendations for these two disease processes in the intervention group compared with the control group. However, the use of antibiotics for upper respiratory infections (URIs) did not change. The authors noted that the structured algorithms in the HIT system did improve the documentation of clinical elements important to ideal clinical care of pharyngitis, otitis media, and upper respiratory infections. It must be noted however, that this system's rigid requirements for physician documentation also made the HIT system unusable, and the physicians refused to use the system after five weeks.

Improved Documentation. Because many studies have reported an impact of HIT use on the quality and completeness of medical documentation, a study by Carroll and colleagues focused on the impact of a personal digital assistant (PDA) on documentation discrepancies in a NICU. ³³ In this before-and-after study, all the NICU resident physicians used a PDA-based charting system during the intervention phase, comparing their progress notes against a predefined reference standard during both phases. The authors demonstrated that after adjustment for covariates, PDA-based charting did reduce discrepancies in patient weights in the charts but did not affect the number of medication or vascular line discrepancies.

Timeliness, Efficiency and Cost-Effectiveness of Care. Quattlebaum et al. ³⁴ studied a scheduling/practice management system that automatically generated reminder postcards for appointment the following week. In this randomized controlled trial, the authors demonstrated a reduction of the no-show rate in their pediatric ambulatory practice from 19 to 10 percent. A cost-benefit analysis of the HIT system and its impact on missed appointments revealed that for each $1 spent on reminders, an additional $7.50 of revenue was captured.

In the inpatient setting, the previously discussed study of CPOE in the NICU by Cordero and colleagues ²⁶ measured not only CPOE's effect on gentamicin dosing errors but also the time from medication prescription to administration for initial doses of a single medication and radiology tests during the pre- and post-CPOE phases. The authors documented significant reductions in the average turnaround time for both medications (10.5 to 2.8 hours, p<0.01) and radiology tests (42 to 32 minutes, p<0.001).

Summary

Early evidence shows that stand-alone CDSS can reduce medication dosing errors, and CPOE + CDSS can reduce the incidence of harmful medication errors in the inpatient pediatric and neonatal intensive care settings. However, other HIT systems, such as electronic MAR, pharmacy-based robots, smart infusion pumps/devices, and medication bar coding, are predicted to reduce medication errors but need to be studied further.

HIT also has tremendous potential to improve vaccination rates and disease management in pediatric outpatients. CDSS and registries have been shown to be effective in increasing vaccination rates in targeted populations, but only a limited HIT impact on general pediatric immunization rates has been demonstrated. Similarly, a patient clinical decision-support device that assists insulin dosing in children with diabetes reduces episodes of hypoglycemia and overall insulin requirements, but does not affect traditional measurements of glycemic control. And the use of computerized documentation systems with integrated CDSS has been demonstrated, in separate studies, to 1) reduce the frequency or duration of antibiotic use for common pediatric illness such as pharyngitis and otitis media, and 2) improve completeness and somewhat reduce variation in clinical documentation.

In the ambulatory setting, a single study showed that an appointment reminder system is cost-effective and significantly reduces missed appointments. Another study showed that CPOE in the NICU can reduce medication and radiology turnaround times. Therefore, the evidence for HIT cost-savings in pediatrics is limited but deserving of optimism.

Conclusion

A small body of literature supports the assertion that HIT use in pediatrics is beneficial in the areas of medication safety, adherence to immunization and disease-based guidelines, patient decision-support in diabetes management, clinical documentation, patient appointments, and in-hospital order processing. No data on the costs or cost-effectiveness of implementing these systems were found, except in one case. In addition, because many of these HIT systems were tested and/or developed in academic settings, the ability to generalize these findings to other organizations is uncertain.

Introduction

Despite rapid advances in the biomedical sciences, a growing body of evidence shows serious shortfalls in the quality of care Americans receive, ^{35, 36} and significant longstanding shortfalls in performance have persisted despite recent increases in attention to quality. ^{37 – 40} International comparisons have demonstrated similar problems in quality. ^{41 – 43} If the United States is to realize the full value of biomedical knowledge and of financial investments made in healthcare, the mechanisms through which that knowledge is operationalized and care is delivered must be radically redesigned.

Although the content of healthcare continues to change dramatically, the methods of health care delivery have not. In particular, a vast majority of the healthcare industry continues to deliver care, manage information, and conduct clinical transactions through the use of paper records.

Although the use of electronic health records (EHRs) is limited in healthcare, there is a renewed conviction by the government, provider groups, and healthcare purchasers that widespread adoption is critical to the delivery of consistent, high-quality care. However, EHR implementation, without other important changes in the way healthcare services are provided, is unlikely to improve quality. Such process redesign and reengineering is difficult and resource-intensive and is also hampered by the complexity and fragmentation of our current healthcare system. Therefore, despite the potential benefits of widespread EHR use, better empirical evidence is needed to confirm that EHR use does in fact improve quality and—perhaps more fundamentally—to understand what capabilities EHRs need to have for quality to be improved. At present, the depth and breadth of the empirical evidence regarding EHR use and its attributable impact on the quality of care remains unclear.

The purpose of this review is to examine and synthesize the available research evidence for the impact of EHR on quality of care in the outpatient setting. The review will also attempt to differentiate the direct impact of EHRs as point-of-care and workflow tools from how EHRs have been used to indirectly achieve those results, by measuring clinical and process outcomes. We elected to focus on ambulatory care because of the large volume of health services delivered in this arena. In addition, because the vast majority of outpatient practices comprise fewer than ten providers—many of whom lack technical infrastructure and resources—it is unclear whether widespread implementation of EHRs will be feasible in this environment.

Research Study Inclusion Criteria

From our database of 256 articles, we selected all 84 papers that related to EHRs. We then screened these articles against the following inclusion criteria: (1) the study reported quality-of-care data as study outcomes, (2) the EHR was documented to have the following minimal functionality—electronic documentation (viewing, entry, or both), results management, CPOE, and some form of decision support, (3) the study was conducted in the ambulatory setting. The criteria for functionality were chosen based on the IOM's “White Paper on Key Capabilities of an Electronic Health Record.” Given the rapid technical advances in EHR systems, we reviewed additional functional criteria to provide decisionmakers with the most relevant and forward-looking information available.

Analytic Framework

The Donabedian “Structure—Process—Outcome” model for quality was used as a framework for this review. ⁴⁴ In this model, structure is defined as the resources and factors involved in producing care and the manner in which those resources and factors are organized. Examples of structural quality include the number of beds in a hospital, the number of physicians in an emergency room per shift, the budget for a clinic, and the presence of disease management program for diabetes. Process of care is defined as the activities that constitute health care. Examples include screening for breast cancer, ordering laboratory tests, and prescribing a medication. Outcomes are the end results of healthcare delivery processes. They are the consequences of health services or can be logically attributed to the act of providing those services. Whereas structure relates to the environment in which healthcare is delivered and process relates to the provisions of care, outcomes are events that occur with patients and consumers—as individuals, groups, or populations.

Two aspects of this model are particularly relevant to EHRs. First, to fully assess quality of care, there need to be links from structure to process to outcomes. The technical and functional capabilities of EHRs form a structure for care. In order to derive value from the EHR structure, new clinical processes need to be designed to utilize the EHR functional structure. These EHR-mediated processes should in turn lead to a specific set of better outcomes. Second, the distinctions among structure, process, and outcome are somewhat arbitrary in the model. Health care delivery is viewed as an interconnected series of structure—process—outcome relationships. For example, in a primary care clinic of three physicians (structure 1), a patient may have an electrocardiogram performed (process 1), which shows an abnormality (outcome 1). This abnormal result necessitates a referral to a cardiologist (structure 1) who orders a stress test (process 2), which comes back suggestive of coronary artery disease (outcome 2) and so forth. This structure—process—outcome chain is central to the role of EHRs in quality, because an EHR is a tool that explicitly links the three. An EHR with decision support for diabetes management (structure) allows a physician to order a hemoglobin A1C (process) and check the results (outcome). Because this outcome is stored in the EHR database, it in turn becomes part of the structure of care. The EHR can allow or even remind the physicians to act on that result (process 2), e.g., modify the patient's insulin dose, which will, in turn, lead to a lowering of the patient's blood sugar (outcome 3) or a reduction in the likelihood of a long-term diabetic complication.

In addition to imposing the Donabedian Structure—Process—Outcome model, we organized deficits in quality by means of a conceptual framework that divides quality problems into three types: (1) the underuse of appropriate health services, (2) the misuse or inappropriate use of health services, and (3) the overuse of health services. ³⁶

Analysis

Seven research studies were identified using the search criteria outlined earlier. ^{45 – 51} Four of the seven were conducted at academic medical centers, ⁴⁵ – ^{47, 51} and three of those four were conducted at a single institution, Regenstrief Institute (the fourth was conducted at Beth Israel Hospital in Boston). ^{45 – 47} Two studies were conducted at a large, integrated healthcare delivery network, Kaiser, ^{48, 49} and one was conducted in the Netherlands. ⁵⁰ All four studies from the academic medical centers assessed internally developed HIT systems, rather than a commercially available system. One study ⁴⁸ assessed two different systems at two sites, one of which was internally developed by the organization and the other a commercially developed product.

All studies included data on structural quality. These varied highly and were largely qualitative in nature. In particular, reporting on the organizational and workflow changes needed to implement an EHR or a new EHR functionality was limited. All seven studies analyzed quality with respect to process of care. Six of the seven ⁴⁵ – ^{49, 51} assessed quality with respect to some type of outcome.

In terms of the types of problems the interventions were trying to address, six of the seven ^{45 – 50} included data on the effects of EHRs on decreasing overused or redundantly used healthcare services. Two ^{48, 51} included measures of the effects of EHRs on appropriate but underused care. None used explicit methods to evaluate the impact of EHRs on inappropriate use of care.

Conclusions

The studies reviewed in this analysis illustrate a range of ways in which ambulatory EHRs can serve to improve quality of care. In particular, they demonstrate how provider performance can be improved when the clinical information management and decision support tools are available within an EHR system. A recurrent theme in these studies was the capacity of EHRs to store data with high fidelity, to make those data readily accessible, and to help translate them into context-specific information that can empower providers in their work.

This analysis is limited by a number of factors. The small number of studies included in the sample was largely a function of the search criteria. In particular, few systems with the core EHR functionalities of documentation, results management, provider order entry, and decision support have been examined, particularly for commercially developed products. These functional criteria were chosen to make the analysis more pertinent to decisionmakers currently considering EHR adoption. Because of the rapid technical advances in EHR, many of the studies of EHR systems are out of date. This review has focused on EHRs with these core functionalities in order to provide decisionmakers with an overview of the evidence that is most likely to be pertinent to the choices they are making now. Another major limitation is the lack of description (and data) pertaining to the workflow reengineering and organizational change that were required for EHR use. As discussed earlier, the “intervention” in these studies is not only the EHR system but also the manner in which these systems change the way healthcare professionals work, organizations function, and consumers receive care. This information is highly context-specific, and for the findings of research on EHRs to be more widely generalizable, this part of the “intervention” needs to be characterized, described, and measured more accurately and comprehensibly. Without such process implementation data, the applicability of findings from one context to another will be a barrier to informed decisionmaking.

Summary of Key Findings from Non-Financially Focused Studies

Among its other functions, an EHR system can facilitate automatic generation of patient reminders for preventive services, screening, and disease management. Five Canadian studies used the same EHR system to generate patient reminders and compared the effectiveness and cost-effectiveness of three strategies—physician reminders, telephone reminders, or letter reminders—to remind patients to get preventive services. ^{56 – 60} All forms of reminders were effective, with reminders delivered directly to patients being somewhat more effective than reminders to physicians. Another study used computerized pharmacy records to generate patient feedback and compared the effectiveness and cost-effectiveness of two depression care programs. ⁶¹ Feedback with care management was significantly better than feedback alone.

Electronic charting is a feature of EHR that has been reported to affect provider productivity. ^{62 – 65} These studies found that the time needed for development of care plans and documentation initially increased, but preparation time decreased subsequently. The initial loss of productivity was associated with the baseline computer skills of the users (clinicians). Two studies assessed computerized documentation systems used for the ICU. ^{66, 67} The authors of the first study asserted that addition of their computer-based nursing documentation required no specific ICU software or bedside workstations because it was implemented in a well-networked information technology environment. ⁶⁶ Thus, they found that compared with paper charting, their electronic charting system was relatively inexpensive (although the figures were not provided). In addition, the documentation was more complete because of the presence of reminders for missing entries, and data quality remarkably improved. The other study used work sampling and cost analysis methodologies to show net savings of a vendor-developed bedside documentation system specifically designed for an ICU. ⁶⁷

Another potential benefit of EHR systems is avoidance of morbidity because of improved patient safety. One study of ADEs found that building ADE detection and reporting capability into EHR can improve detection and potential reduction of ADE in hospital settings because of the ability of the EHR system to easily identify and confirm patients experiencing ADEs and thus enabled early intervention. ⁶⁸ Several studies have shown that severe ADEs were associated with longer hospitalizations and higher hospitalization costs (over $2,000 in 2005 dollars). ^{68 – 70}

Several studies investigated the impact of point-of-care alerts and reminders imbedded within an EHR system during the process of documentation or entry of orders into the system. ^{45, 47, 71 – 76} These decision-support functions within an EHR system, when accompanied by the required changes in process and communication, altered physicians' ordering behaviors by facilitating appropriate resource utilization and reducing unnecessary charges. ^{45, 47, 71 – 76} For example, one study estimated that reducing the ordering of redundant clinical laboratory tests could produce an annual savings of $35,000 in laboratory charges. ⁷² A randomized controlled trial that tested the effect of immediately printed summaries of a computerized medical record on physician test ordering rates in an emergency room setting ⁷⁷ showed significant improvement in the cost-effectiveness of internists' ordering behaviors. The impact on surgeons' ordering patterns was positive but not significant.

Some EHR systems offer sophisticated CPOE and decision-support functionality. A randomized controlled trial found positive effects of an EHR with integrated CPOE on resource utilization, provider productivity, and care efficiency. ⁷⁸ Two additional studies showed that an EHR with integrated decision support helped providers improve the quality of documentation, clinical decisionmaking, and guideline compliance, and resulted in reduced utilization of services and costs of care. ^{76, 79} However, another study found that implementation of clinical guidelines via an EHR had no significant effect on clinical outcomes or healthcare costs. ³⁰ The benefits of information technology seem to depend greatly on the quality of the implementation and the level and type of decision-support technology.

Analytic Methods to Assess the Economic Value of an EHR System

This section provides a more detailed review of the nine articles that quantitatively assessed the economic value of an EHR system, including summaries of the analytic methods used in the studies. Two articles by the same author described the same ambulatory EHR system, although the economic estimates differ slightly between articles. ^{53, 54} Therefore, we refer to them as one study described in two articles. Of the remaining seven articles, four report on evaluations of the economics of an EHR system: two in the ambulatory care setting ^{52, 80} and two in an integrated delivery network (IDN) (an IDN comprises providers—both inpatient and outpatient—as well as payers and purchasers, in one connected managed care organization). ^{81, 55} Another study concerns health care information exchange and interoperability, ⁸² and the remaining two are methodological papers. ^{83, 84} A brief summary of the methods and findings of these studies is presented in Table 1.

To assess the value of EHR, one methodological paper described a return-on-investment framework to evaluate the costs and benefits of implementing an ambulatory EHR system, and another presented a spreadsheet tool to help family physicians estimate the costs of implementing an EHR system. Both reports provided illustrative examples. Of the remaining six studies, five used cost-benefit analysis ⁵² – ^{55, 81, 82} and one used cost-consequence analysis. ⁸⁰ All the cost-benefit analyses adopted the ROI framework, which assesses the difference between the costs of an EHR investment and the benefits reaped from it. In these studies, both costs and benefits are quantified in monetary terms to the extent the authors determined was feasible. For example, one study aimed to justify the cost of EHR by first identifying the goals of the system in order to determine benefits. ⁵⁵ To quantify benefits, the authors then performed an extensive literature review, surveyed other institutions that were implementing EHR, interviewed EHR vendors, and conducted process-mapping sessions to identify potential cost savings on work processes affected by EHR. However, cost estimation was based largely on vendor response to a request for information or on current information system costs in the healthcare organization. ^{52, 80} The cost-consequence study showed costs in monetary terms but did not quantify the benefits of EHR except for time saved from chart pulling. ⁸⁰ All studies except two used the perspective of an organization, either outpatient settings ⁵² – ^{54, 83, 84} or IDN. ^{55, 81} Of the other two studies, one adopted a societal perspective, ⁸⁰ and the other used multiple perspectives, from organization level to national level. ⁸²

Six studies reported their data sources. ^{52, 55, 80 – 83} All used multiple data sources, including primary data collected from an existing EHR system, ⁵² published data, ^{52, 55, 82, 83} workplace and demonstration site observations, ^{80, 81} and surveys or interviews of key informants or EHR users. ^{55, 80} Experts were a primary source of data for some studies, ^{52, 81, 82} as were vendors. ^{55, 81} One study also used process-mapping sessions. ⁵⁵

Cost and benefit variables used in each study are listed in Table 1. The cost variables included in most studies included hardware and network acquisition, software licensing, ongoing technical support and maintenance, and training and other implementation costs. Costs associated with temporary productivity loss due to the EHR implementation were captured by two studies, ^{52, 80} but a third study assumed no cost associated with loss of productivity, given a long-term EHR implementation strategy. ⁸¹ Other cost variables include installation (which was not quantified in greater detail), ⁸³ data entry, ⁸⁴ printing, ^{53, 54} system integration in IDN setting, ⁵⁵ personnel, ^{53, 54} and institutional and project management. ^{55, 80} The health care information exchange and interoperability study also included interface development cost.

The benefit variables included the following: savings from chart pull and transcription; ^{52, 80 – 83} time saved to document diagnostic codes; ^{81, 83} prevention of ADEs; ⁵² reduction in drug ^{52, 81} laboratory, ^{52, 81} or radiology costs; ^{52, 81} improvement of charge capture; ^{52, 81} decreased billing errors; ⁵² personnel and space savings from reduced existing and future medical record storage requirements; ^{53, 54} as well as automated generation of clinical forms, ^{53, 54} pharmacy information, ⁸¹ and billing data generation. ^{53, 54} One study provided a comprehensive list of potential benefits grouped into four categories: data capture and access, decision support, business management, and streamlining patient flow. ⁵⁵ The health care information exchange and interoperability study reported quantifiable benefits from health information connectivity for providers, payers, and other stakeholders, including radiology centers, laboratories, pharmacies, and public health departments. ⁸² Three studies showed the net ROI. ^{55, 82}

Various analytic designs were used, although the reference strategy was always the traditional paper-based medical record. One study assessed the economic value of an EHR system concurrently with its implementation and reported the first-year economic consequences of the system. ⁸⁰ Another study predicted a fixed annual savings and expenses based on seven years of EHR implementation experiences in a HMO. ^{53, 54} Yet another study constructed a hypothetical primary care provider patient panel using average statistics from one of the nation's leading organizations in EHR implementation. ⁵² Both studies of EHR systems in IDN settings are analytic predictions because EHR had not been implemented in the studied organizations. ^{55, 81} The health care information exchange and interoperability study for a fully standardized nationwide system is also an analytic prediction based on a conceptual framework describing how health care entities share information and a functional taxonomy reflecting the amount of human involvement required, the sophistication of IT, and the level of standardization. ⁸² Sensitivity analyses were performed in three studies. ^{52, 80, 82}

Evidence of Economic Costs and Benefits of an EHR System

Our interactive evidence database (http://healthit.ahrq.gov/tools/rand) provides a structured abstract for each of the nine identified studies regarding the costs and benefits of EHR. Main findings are highlighted in Table 1 and summarized below.

Costs of Implementing an EHR System. Five studies quantitatively assessed the costs of implementing an EHR system. ⁵² – ^{55, 80, 81} The costs can be divided into two categories: (1) cost of the system itself (hardware, software, license, maintenance, and support) and (2) implementation cost (training, temporary loss of productivity, etc.). The costs vary significantly by the scale of the healthcare organization and the functionality of the EHR system.

(1) Cost of an EHR System. One study estimated the system costs for an ambulatory EHR to be $9,700 per provider (in 2002 dollars), which included $1,600 for the annual software license, $1,500 for annual support and maintenance, and $6,600 for hardware (three computers and network, refreshed every three years). ⁵² The estimate was for a hypothetical primary care provider office and was modeled after a well-developed and widely used EHR system at a leading IDN health care system. The component parts of the EHR system include online patient charts, electronic prescribing, laboratory order entry, radiology order entry, and electronic charge capture.

In a Swedish primary health care setting with 50 staff, the system cost of a vendor-developed EHR system was estimated to be $240,000 in the first year (in 1995 U.S. dollars). ⁸⁰ The EHR system supported full-text patient records and included a controlled medical terminology, a structured patient database, and tools for the analysis and reporting of patient data. The hardware at the sites comprised one server supporting approximately 40 workstations and 20 printers.

In a large HMO with 13 outpatient care locations in Ohio, a homegrown ambulatory EHR was estimated to have had a system development cost of $10 million (in 1996 dollars) and additional annual expenses of $630,000 (in 1996 dollars) for printing, network expenses, memory, and license renewals. ^{53, 54} The EHR system was used routinely by 220 physicians and 110 allied health professionals. It implemented an encounter system that collected and presented such medical data elements as diagnoses, allergies, medications prescribed, immunizations, vital signs, and smoking status at the time of an encounter. The system also generated physician reminders for guideline compliance and patient reminders for preventive services and was linked to centralized clinical data on the mainframe, such as laboratory results, radiology reports, emergency department notes, and hospital discharge summaries.

An academic cancer center with a staff of about 8000 and facilities that included a hospital, outpatient clinics, and remote patient-care sites was interested in implementing an EHR for its IDN as both a clinical and financial information management tool in 1994. The cost estimates for vendor-developed EHR systems to meet the center's organizational needs ranged from $15.8 million to $21 million (in 1994 dollars), which included costs of hardware, software, interface development, network cost, data conversion, training, and annual maintenance. ⁵⁵ Additionally, the annual support costs were estimated to range from $3.8 million to $5.3 million.

A 2002 study estimated the costs of implementing an EHR ⁸¹ for an IDN that included a medical center with a 280-bed acute care hospital, 16 hospital-based and satellite outpatient clinics, a research institute, and a network of about 400 employed physicians. A vendor-developed EHR was estimated to cost approximately $19 million (in 2001 dollars) for the seven-year implementation period. This included costs for the various software products, server hardware, professional services related to installation and training, as well as desktop devices, monitors, biometric security devices, imaging hardware and software, additional technical-support staff, and other associated costs.

(2) Cost of the Implementation Process. Only two studies provided an estimate of costs associated with the EHR implementation process. Both were for ambulatory settings. ^{52, 80} One estimated an implementation cost of $3,400 per provider (in 2002 dollars) in the first year associated with workflow process redesign, training, and historical paper chart abstracting. ⁵² It also estimated a revenue loss of $11,200 in the first year due to temporary loss of productivity. The total implementation process cost, $14,600 per provider, is 1.5 times the estimated EHR system cost.

The Swedish study used a societal perspective and included costs of training and unexpected costs pertained to self-training during working hours, loss of normal activities in leisure hours, increase in administrative work load, extra service, and medical records summary. ⁸⁰ These costs were estimated at $75,000 (in 1995 U.S. dollars), approximately 30 percent of the EHR system cost.

A third study projected the costs of implementing health care information exchange and interoperability where EHR is a requirement for Levels 3 and 4 implementation, ⁸² which, according to the authors' taxonomy, refers to the ability to handle machine-organizable data and machine-interpretable data, respectively. The authors projected costs for multiple stakeholders of the healthcare system for Levels 3 and 4 health care information exchange and interoperability. The national ten-year rollout cost of Levels 3 and 4 were estimated to be $320 billion and $276 billion, respectively. Additionally, the national ten-year annual costs were estimated at $20.2 billion and $16.5 billion, respectively.

Quantified Benefits from an EHR System. Benefits of an EHR system or health care information exchange and interoperability were also quantified in the six studies that assessed the cost of implementing such a system.

One study was based on the Partners HealthCare ambulatory EHR, which not only provided health information and data storage capability but also possessed results management, order entry management, point-of-care decision support, and administrative information management functionalities. Therefore, many benefits were expected. The study divided the benefits into three categories: (1) payer-independent benefits, including savings from chart pulls and transcription; (2) benefits under capitated reimbursement, including averted costs from ADEs, drug utilization, laboratory utilization, and radiology utilization; and (3) benefits under fee-for-service reimbursement, including improvement in charge capture and decreased billing errors. ⁵² The authors predicted that savings from chart pulls and transcription would be seen immediately after the EHR implementation, and costs associated with ADEs and drug utilization could be averted from second year on, but other potential savings would not be realized until the fourth year. Five-year total benefits of an EHR implementation were estimated to be $129,300 per provider (in 2002 dollars), or a net savings of $86,400 per provider (in 2002 dollars). Sensitivity analyses showed that the estimates were sensitive to the assumption of the proportion of patients whose care was capitated. The net financial value could range from a $2,300 net cost to a $330,900 net benefit per provider.

The Swedish study examined an EHR system with functionality limited only to health information and data storage. Therefore, the expected benefits were limited and included increase in knowledge capital for the primary health care team, easier and quicker communication for general practitioners during telephone consultations, clearer information to patients, and time saved in retrieving paper-based medical records. ⁸⁰ Only the value of the last item was estimated, at a total of approximately $10,500 (in U.S. 1995 dollars) for the first year of EHR implementation.

Despite potential savings from ADE prevention and reduced resource utilization under capitated reimbursement, the study of the ambulatory EHR system in a HMO (13 outpatient clinics) did not quantify this aspect of benefits. Instead, it quantified only the averted costs associated with improved efficiency. ^{53, 54} The study estimated an annual savings of $3,700,000 (in 1996 dollars) from reduced medical record room and support staff, elimination of clinical forms, and automatic collection of billing data.

The cancer center study projected the benefits of an EHR over ten years. ⁵⁵ This projection made several key assumptions, including no benefit until the third year after implementation, benefits to phase-in as the EHR system became functional, physician acceptance and use of the system, a link between business management benefits and managed care, and productivity changes. The authors divided the benefits into capture and access, decision support, optimization of clinical practice, business management, and streamlining of patient flow. The estimated total quantified benefits were $129.69 million over ten years. Adjusted by the total implementation and system costs, the authors' assigned confidence factor, and 9.5-percent discount rate, the net value was predicted to be $24.9 million (in 1994 dollars).

The other IDN expected even greater benefits from an EHR implementation. The authors predicted approximately $68.5 million in gross quantifiable benefits over a seven-year period; subtracting the cost of the EHR system implementation, the net benefit would be $31.4 million (in 2001 dollars), using a 10-percent discount rate. ⁸¹ The authors predicted and quantified benefits from savings in laboratory and radiology order entry, pharmacy order entry, documentation availability of information, and charge capture.

Another study estimated that implementation of a standardized interoperable EHR system by all healthcare organizations in the United States would yield substantial financial benefits. The health care information exchange and interoperability study predicted that investment on a fully standardized, Level-4 nationwide system will have the most financial return, a net value of $77.8 billion per year once fully implemented. ⁸² Non-standardized health care information exchange and interoperability also can have positive financial returns, but the returns are smaller compared to the Level-4 implementation.

In summary, despite the heterogeneity in the analytic methods used, all five cost-benefit analyses predicted substantial savings from EHR (and health care information exchange and interoperability) implementation. ⁵² – ^{55, 81, 82} In other words, the quantifiable benefits are projected to outweigh the investment costs. However, the predicted time needed to break even varied from three ^{52, 81} to six ⁵⁵ to perhaps as long as 13 years. ⁵⁴

Conclusion. Our evidence review found consistent predictions from five cost-benefit studies that implementation of an EHR system can be financially viable at the individual organization level or through a nationwide implementation with high levels of health care information exchange and interoperability. ⁵² – ^{55, 81, 82} However, there are several caveats.

1. All studies are predictive analyses that are based on many analytical assumptions and limited empirical data. The strength of evidence is considered weak.
2. In all studies, the EHR system was assumed to have multiple functionalities that include, at minimum, health information and data storage, administrative processes, decision support, and results management, as well as information exchange capabilities. The functional capability of an EHR system is critical to the benefit accrued.
3. The individual organizations that were the subjects of four studies were all large organizations. Large organizations involve many people, units, and subsystems and have complicated processes and interactions. They can benefit greatly from automated, transparent information processing through HIT, and substantial economies of scale. The literature review did not identify cost-benefit studies for EHR implementation in small organizations.
4. The costs of implementing an EHR system may be underestimated. Only one of the five cost-benefit analyses included the cost of the implementation process, ⁵² and it found that this cost was 1.5 times the cost of the EHR system. Implementing an EHR system requires extensive changes in the organizational processes, individual behaviors, and the interactions between the two. These resulting costs are often omitted or not reported from studies but can be substantial.
5. The financial benefits depend on the financing system. As shown in the sensitivity analysis of one study, ⁵² the benefit estimates are most sensitive to the assumption of the proportion of capitated patients. Realizing all quantifiable benefits of EHR implementation would require changes to the current health care financing system.
6. Both the cost and the benefit of attaining interoperability among EHR systems are directly proportional to the level of data exchange achieved. For example, the cost of achieving machine-organizable (Level 3) or machine-interpretable (Level 4) interoperability is greatest, but it offers the most potential for increased efficiency, improved healthcare utilization, and reduced costs.

In conclusion, there is some empirical evidence to support the positive economic value of an EHR system and the component parts of EHRs. However, realizing the projected benefits will require proper alignment of the healthcare financing system, strong leadership, effective implementation strategies, and focused efforts to successfully adapt the EHR system.