Definition of a High-Reliability Organization

In order to consistently achieve high standards for both patient safety and quality, Trinity Health searched for successful models from other industries to emulate. This led to the study of high-reliability organizations (HROs), including the work of Weick and Sutcliffe.¹ HROs are highly complex and have the potential for catastrophic failures, including many deaths, yet they operate with few such defects. An HRO has an exceptional safety record, not merely above average. Examples of HROs include nuclear aircraft carriers, nuclear power plants, and air traffic control systems. Although unique in many ways, health care is also a highly complex industry with many catastrophic defects² that could benefit from employing HRO principles. HRO reliability results from a capability to:

• Discover unexpected events.
• Manage unexpected events.
• Reduce the occurrence of unexpected events.

An HRO has five important attributes that determine an organization’s mindfulness of potential errors and defects. These include:

1. A preoccupation with failure. An organization must expect error and train staff to recognize and recover.
2. A reluctance to simplify. The organization rejects the first impression of the cause of defects and abnormal values. Instead, all staff are trained to investigate the root causes of potential defects.
3. Sensitivity to operations. Sensitivity to operations stresses the importance of the frontline employee. All staff are encouraged to address anomalies while still tractable and able to be isolated. This allows defects to be corrected at a point of low intensity.
4. A commitment to resiliency. A commitment to resiliency places a high value on staff expertise. Resilient staff have the ability to detect, contain, and mitigate defects and errors.
5. Deference to expertise. Deference to expertise requires concentrating decisionmaking with frontline staff, who are authorized to make critical decisions. This HRO attribute is epitomized by the Joint Commission’s patient safety standards, which allow any employee involved in a surgical procedure to speak up during the timeout to avoid mistakes that might occur related to wrong patient or wrong site in the operating room suite.

Taken together, these five attributes are the key components of “mindfulness.” It is mindfulness that induces the capability to discover and manage unexpected events, which then leads to reliability. The information gleaned from these events is used proactively to reduce future occurrences and to improve the design of suspect processes.

Need for an Event-Reporting System

It is no longer necessary to emphasize the importance of error reduction or the cost of medical errors, both financially and in terms of human suffering.^{3, 4} Everyone is well aware of the need to provide safe and effective health care services. The challenge lies in finding a method to achieve this goal.

Having studied HROs, Trinity Health recognized the need for a better method to discover defects, mitigate harm, and prevent future occurrences. To facilitate the work involved in decreasing error frequency, Trinity Health needed first to collect as much data as possible, examine mistakes, and learn from them to redesign systems and avoid future errors.

The literature indicates that medical errors are severely underreported.⁵ It was determined that traditional paper-based incident-reporting systems, although originally designed to provide information about events, do not meet the needs of an HRO. Too few incidents make it through the paper-based reporting process, and it is nearly impossible to categorize events. Those events that do not result in serious harm frequently are ignored while the focus remains on high-impact events.

We were unable to understand the frequency and cause of events across all of Trinity Health. Nor did our paper-based system foster a preoccupation with errors or encourage mindfulness, one of the key principles of an HRO. Information about “near events” was being lost. Also, incidents tended to be viewed as a means of determining fault, rather than as a way to uncover systemic problems.

Trinity Health needed a tool and a culture that would encourage reporting of all events, so that data could be methodically studied and analyzed. In a culture of blame, the focus is on human error rather than on root causes. In a system that identifies fault and dispenses punishment, events tend to go underreported because the incentive is to hide information. It was Trinity Health’s desire to learn from past events in order to prevent future occurrences.

The Potential Error and Event Reporting System

Trinity Health embarked on a journey to improve reporting of errors and “near events” through the use of an automated reporting system. After much consideration, a decision was made to develop a new tool in-house based on the National Aeronautics and Space Administration’s voluntary Aviation Safety Reporting System. In 2000, this was an innovative approach to event reporting.

Building, rather than buying, such a tool would allow for customization of the tool and provide more control over its content, cost, and configuration. At the time, available commercial tools were expensive and did not meet our need for ease of use. In response to mandatory reporting requirements, the Veteran’s Health Administration has taken a similar approach.⁶

The PEERs questions were developed based on survey responses from physicians, nurses, an attorney, and risk and quality management professionals. After nearly a year of research and development, the Potential Error and Event Reporting System (PEERs) was born. PEERs was intended to:

• Reduce fear of reporting and thus, increase reporting.
• Formulate a database to standardize the information.
• Reduce errors through data tracking, trending, analyses, and targeted improvement projects.
• Reduce errors through prevention.
• Improve quality along with patient and employee satisfaction.
• Reduce costs associated with errors.
• Increase mindfulness and begin the transition toward Trinity Health becoming an HRO.

Design

Development of PEERs began shortly after the birth of Trinity Health, and the prototype was ready for a pilot study in April 2001. The PEERs system consists of three interrelated components:

1. PEERs Survey Tool, which allows the original reporter to enter an event into the system through a Web-based interface.
2. Report Management Tool, which allows the PEERs coordinator/risk manager to review an event, enter additional information, and follow up on the report.
3. Database of events, which has reporting capabilities.

The tool was designed to gather input from any staff member with access to a computer within a Trinity Health Ministry Organization. Trinity Health now has 6 years of historical data and over 200,000 records in its PEERs database.

A reporter could enter a PEERs report using a Web-based interface in approximately 5 minutes. The survey tool uses “pick lists” to facilitate report entry and improve data consistency, but it also offers ample opportunity to describe events. PEERs was developed with back-end reporting in mind, so that data could be used for reporting and analysis. The objective was to create information from gathered PEERs data to implement safer care processes.

The PEERs survey tool is divided into several sections with a series of followup questions that are linked directly to the event category that has been identified as the source of the problem. For example, if an incident is related to a patient fall, followup questions ask about the patient’s risk of falls and when the last fall-risk assessment was performed. If an event is related to medications, the followup questions ask about the type of order and the method of administration. Additional questions are applicable to any event category. Data are gathered about the type of event, the location of the event, contributing factors, injuries, and other factors. The reporter has the opportunity to describe the event and to provide feedback about cause and prevention of the event. These sections of the report can provide the most valuable information as processes are redesigned.

Technical and Cultural Preparation

In order to implement PEERs, a hospital had to prepare for both the tool’s technical and cultural impacts, the technical aspects being the easier of the two. PEERs is Web-based and resides on all hospital computers. Reporters wishing to create a PEERs report access the survey tool via an icon on the desktop of all hospital computers. In the first implementation, computer skills tended to be a factor for some clinical staff. However, that has been less of a problem over time, as more and more clinical functions have become computerized, and as the computer has become a tool routinely used by all hospital staff. Originally, inconsistencies in technical capabilities across hospitals and departments within a hospital were an issue, but these too have been mitigated over time, as Trinity Health has implemented standardized technical tools and processes across the system.

The cultural preparation posed, and continues to pose, the greatest challenges. Along with changes in policy, it was necessary to terminate the use of paper forms in order to implement the PEERs system. Abandoning the process associated with the old paper reports was frequently met with ambivalence.

Since no passwords were required to enter a report, anonymity was an option. The anonymous reporting feature was deemed critical to encourage the use of the PEERs system, since many employees still did not entirely trust the “just culture.”⁷ Despite the perceived need for anonymous reporting, 70 percent of reporters identify themselves.

The tool is designed to capture identifying information, should the reporter choose to provide it. This allows for discrete followup by the risk manager/PEERs coordinator and helps provide additional information to study events.

At the outset, each site was required to implement a nonpunitive reporting policy and field a readiness survey. However, culture is not static and must be nurtured throughout the organization. An additional part of the preparation involved education. Although the survey tool is intuitive and easy to use, at the very least, staff had to be made aware of its existence. It has taken a major communication effort in larger hospitals to ensure that all staff in all departments know about the importance of completing PEERs reports.

The volume of reports is monitored monthly for each site, and an increase in the number of reports is celebrated and rewarded. A PEERs training site is available for use by each hospital to demonstrate the survey tool. Most of our PEERs hospitals introduce the tool to new employees as part of their orientation programs.

The Report Management Tool is used to manage event reports, analyze event classes and the cause of events, and identify mitigation strategies. It allows for more in-depth information to be added to a report subsequent to its creation as the report is further investigated. The Report Management Tool is more complex and requires advanced training. Risk managers and PEERs coordinators lead these efforts in our Ministry Organizations.

Legal Issues/Peer Review Protection

As PEERs was developed, Trinity Health worked to maintain the peer review protection afforded by State law. PEERs is designed to be used entirely for process improvement, and all staff involved in the PEERs report management process must be under the peer review umbrella of their particular hospital. Trinity Health legal staff guided the deployment of PEERs so that wherever PEERs is used, it complies with local and State law peer review protection requirements.

Building the Base of PEERs Users

After using the PEERs tool for several months at the pilot site, it was ready for a broader audience. Using lessons learned from the pilot site, a deployment plan was developed that allowed for the rapid replication of the tool across the system. The implementation plan included both technical requirements to ensure the proper use of the tool and the more critical cultural issues.

The success of the PEERs system depends on reporters’ trust. In part, trust is gained through communication. Staff need to feel that their contributions are valued and acted upon. They need to observe first-hand that reports are not being used to exact punishment. During the design phase and design upgrades, it remains a critical function to ensure that the questions in the survey tool, along with the possible answers available in the pick lists, are blame-neutral.

Naturally, dealing with many different hospitals, there are many different stages of readiness and many different cultures already in place. Trinity Health developed a nonpunitive error-reporting policy that was the keystone of the PEERs system. The goals of the policy are to:

• Support the values of respect, social justice, compassion, and care of the poor and underserved.
• Foster excellence, with direct error-prevention efforts aimed at the root cause of system and process weaknesses, not at individuals.
• Ensure there are no reprisals for reporting of errors and injuries, both actual occurrences and potential conditions (“near events”).
• Develop a working culture in which communication flows freely, regardless of authority or position. The policy states the philosophy behind error reporting and outlines the exceptions, including criminal activity. It is assumed that employees are doing their best, and that errors are not the result of incompetence or misconduct.

One major lesson learned from the PEERs implementations is that culture does not remain static. It is not enough to merely have the policies in place at the outset. Work must continue daily to maintain and support a healthy culture. New employees regularly join the team, and seasoned employees need to be reminded that their input is valued and that senior leadership supports them. It is all too easy to slip back into old patterns. The PEERs Users Group must remain vigilant to keep blame and punishment out of the PEERs process.

PEERs is currently used by 32 different hospitals and four Trinity Health Home Care Agencies. Trinity Health has used the Agency for Healthcare Research and Quality (AHRQ) Culture of Safety Survey⁸ as an assessment of the culture across Trinity Health hospitals. Results show a strong correlation between the perception of a blame-free work environment and the number of PEERs reports received. The survey also found that there was work to be done, and that we had not completely achieved our goals.

Overcoming Acceptance Issues

One interesting aspect of an anonymous reporting system is the scope of issues that are reported. Staff see PEERs as an opportunity to share issues not related to patient safety. For example, some hospitals found that a greater volume of employee health issues was reported through PEERs than through normal channels.

Even though a number of reports have not been considered helpful from a risk-management perspective, they nevertheless add value in that they provide employees with a place to air concerns they might not feel comfortable doing otherwise. This contributes to the “culture of trust” required to keep PEERs functioning at its highest level.

As the number of hospitals using PEERs increased, a user support group was convened. This group meets monthly to discuss issues related to the tool, data issues, and information identified as a result of PEERs reports. This group has been instrumental in helping guide the growth of the PEERs tool through several iterations. Members of the group work together to develop enhancements to the tool. Formation of the group has led to increased acceptance of the tool and greater understanding of errors and their causes.

The use of PEERs by several Trinity Health home health care agencies and by a behavioral health hospital increases its complexity, since these entities require additional event categories and questions in order to meet their reporting needs.

Trinity Health is currently using PEERs version 5.1, so there have been five new and improved versions of PEERs since its original 2001 deployment. Since the original survey tool, report entry has evolved over time. A “facelift” has updated its appearance. Some new fields and event categories have been added to meet the needs of the PEERs Users’ Group. The biggest changes have been in streamlining the entry process, so a reporter could provide the maximum amount of information in a limited timeframe. It has been a struggle to balance the need for complete and accurate information while respecting a reporter’s time constraints.

Because a major difficulty with patient safety research into events and errors is the lack of an adequate denominator, one of the purposes of the PEERs system is to increase reporting. It is difficult to assess the scope of many problems, and the errors that are eventually reported represent only the “tip of the iceberg.”

Development of the Report Management Capability

To accommodate the needs of the risk manager/PEERs coordinator for more information, the Report Management Tool was developed for use in conjunction with the PEERs survey tool. When the reporter enters information into the survey tool and creates a report, the PEERs coordinator is notified by e-mail that a report has been generated. To assist with prioritization of reports, the e-mail notification indicates whether the event resulted in any harm to a person. The report appears immediately in the Report Management Tool.

This tool allows the coordinator to assign the report to others involved in the peer review process for comments. Information can be corrected or clarified, although the original content of the report is saved for future reference. Recent enhancements allow the risk manager/PEERs coordinator to assign one report as the “parent” if multiple reports are filed on the same incident. This allows all versions of the event to be archived, but it prevents the system from double reporting when PEERs data are used to quantify events.

Using the Report Management Tool, the PEERs coordinator can “clarify” reports. The original report is never altered. Clarified reports are used for analytic purposes. Reports are clarified to correct information that might have been entered erroneously and add additional information that was not available at the time the report was created. The more thorough the clarification, the more useful the data generated by the reports. PEERs 5.1 allows the PEERs coordinator to upload pertinent documents so that they can be attached to the original PEERs report. This keeps all pertinent information electronic, negating the need for file drawers full of information.

PEERs reports are used for quality improvement within the Trinity Health facilities, and as such, they are protected under State laws that govern peer review protection. It is critical that all staff who use the PEERs system and are privy to the reports be under the umbrella of the hospitals’ peer review protection committee. As many reports contain sensitive information, the Report Management Tool has different levels of user access and requires authorization through user identities and passwords. Although communication about events drives increased reporting, there is a fine line that separates the types of information shared.

As highly complex organizations, HROs place a high value on expertise. For this reason, the role of “category administrator” was designed for the PEERs system. In larger hospitals, the risk manager/PEERs coordinator does not always have the time or the expertise to manage all varieties of PEERs reports. This is particularly true of medication events. In order to have the most useful information, the pharmacy safety officer has been brought into the loop of PEERs reports and has been given the role of “medication administrator,” providing support to the peer review committee as a medication therapy expert. This individual is able to view and manage all medication or adverse drug events reported in PEERs without action from the PEERs coordinator. This ensures that the information goes directly to the people who need it most and have the most knowledge of the event to complete the back-end information required for analyses. The pharmacist is able to assign the drug class for all medication events, so that potential problems can be uncovered related to a given type of drug. Combined with the severity of the event and the phase of the drug cycle implicated, these data have been extremely helpful to the pharmacy safety committee in prioritizing their improvement efforts. Other category administrators have been designed for events related to security and to employee health.

PEERs Current Status

Trinity Health currently has over 200,000 reports in its PEERs database. Approximately 4,000 reports are logged each month, nearly half of which are filed by nursing staff: 60 percent are attributed to care problems/care processes, medications, or slip/falls. Upon review of each report, the risk manager/PEERs coordinator assigns it a severity index. This assists with prioritization by allowing data to be stratified based on levels of harm. However, because 90 percent of the reports were filed in the absence of significant harm to patients, the bulk of the data provide an opportunity to identify weaknesses proactively. These near events are the reports that really help achieve the tenets of the HRO by providing information on system weaknesses.

A “preoccupation with failure” requires analyses of these events in order to reduce complacency and increase attention. The preoccupation with failure is both about preventing an isolated failure and making the system as robust as possible so that it can respond quickly to operational hazards. The data are available at both the local level and the system level. Tools to use the data are provided with the system to allow data mining at the local hospital. These tools are easy to learn and to use. In addition, many standard reports are created monthly to assist with the patient safety efforts across the system.

PEERs Users Group

One of the key components of the PEERs system is the PEERs Users Group. The group consists of the risk manager/PEERs coordinator from each site and local administrators who are responsible for specific types of safety issues. The group meets monthly to discuss PEERs issues and to share success stories. Although these representatives are responsible for driving the technical aspects of the tool, this is not their most critical function. The main purpose for the group is to provide a forum that uses PEERs reports as the basis for discussing safety and process improvements.

Even though each hospital is unique, there is much common ground. This is especially true when it comes to faulty equipment. One hospital might experience only one or two instances of any given equipment failure. In HRO research, these are called “weak signals,” and they are frequently overlooked. Previously, one of the hospitals using the PEERs system noticed they had several reports that were attributed to the failure of a certain pump. They brought this issue to the group, and it became apparent that several other hospitals also had one or two events attributed to that same pump. This provided enough information to generate a warning to all Trinity Health staff using that particular pump. This alert allowed for a proactive response that prevented additional errors related to the discovered defect. It was only when data were studied across all hospitals that certain trends, that otherwise might have gone unnoticed, became observable.

Using PEERs Data for Process Improvements

De-identified PEERs data are also used by the pharmacy council to look for opportunities for improvement in medication administration across the system. This group routinely analyzes the types of medication errors that have occurred, the phases in the administration process where there are vulnerabilities, and the drug class most commonly associated with medication events. It is not surprising that a high percentage of reports that document harm to patients are frequently attributed to high-risk medications, such as heparin, opiates, and electrolytes.⁹ This work has led to the development of a “heparin policy” that was recently shared with all hospitals at the Trinity Health Fall Conference.

PEERs data were also used to link the use of Computer Provider Order Entry (CPOE) to the decrease in medication events attributed to transcription error. This work provided the pharmacy council with hard data to convert reluctant technology adopters to make the leap to the available computer tools. Other medication-related improvements implemented because of PEERs data include barcoding technologies, smart pumps, and other improvements in the medication cycle.

In addition to the work related to medication errors, PEERs data also have been used to add urgency to the implementation of rapid response teams. The deployment of rapid response teams, one of the key safety recommendations of the Institute for Healthcare Improvement,¹⁰ was fully endorsed by Trinity Health. Data from PEERs made it easier to get a buy-in across the organization and to motivate teams.

PEERs data have also been instrumental in the development of programs for falls risk assessment and mitigation, pressure ulcer risk assessment and mitigation, and a suicide risk assessment, all of which have been hard-wired into our information system. The assessments, implemented in 14 organizations, require every nurse to assess each patient’s risk for falls, pressure ulcers, and suicide consistently across all 14 organizations. If a patient is found to be at risk, nurses are well-positioned to initiate order sets to mitigate that risk. These changes increase the reliability of the organization and prevent patients from falling through the cracks.

PEERs data reveal that many inpatient falls occur when patients get out of bed to use the bathroom. As a result, Trinity Health has implemented the practice of hourly rounding to meet these patient needs. This action has had the added advantage of increasing patient satisfaction with care because nursing staff are more visible. Using PEERs data is more about increasing resolve to deliver high-quality and defect-free care. The solutions implemented frequently have been thoroughly researched and well documented by experts. Making improvements requires resolve and follow-through, attributes that are encouraged in an HRO.

PEERs reports also have driven patient safety in the obstetrics department. Events related to obstetrics can have tragic consequences for patients, and this has resulted in high liability for a facility. A review of PEERs reports revealed that approximately 25 percent of all events could be attributed to failures in communication. This information has been used to develop simulation protocols and to encourage the use of such protocols by obstetric teams to improve teamwork and ensure that all staff can respond appropriately to emergent situations in the delivery room. Simulations for shoulder dystocia have been of particular importance because all team members must be prepared to move quickly to prevent such injury to the newborn.

It is very helpful to have data related to events, but to have real value, the data must result in actions and improvements. In addition to tracking events, the PEERs system also documents the results attributed to PEERs reports. During the past fiscal year, July 1, 2006 through June 30, 2007, eight policies were changed, 22 root cause analyses were completed, and an additional 22 policies have been initiated; 45 processes were changed, and 35 more are currently being revised, all as a result of PEERs reports. In addition, nine changes to computer software, nine changes to the facility and 127 teachable moments for staff were implemented.

PEERs reports have been instrumental in increasing the amount of hospital-related safety communication. Reports are referred to applicable departments and leaders within the facility, ensuring that information is shared and that required action is taken. Many senior leaders are initially shocked at the volume of reports. In the previous paper world, many reports were filtered out as they rose up the chain of command until only those that resulted in harm actually reached the senior leaders within the organization.

Since Trinity Health engages in many different improvement efforts at any given time, it is difficult to attribute improvements solely to PEERs. However, Trinity Health facilities have demonstrated a 26 percent decrease in severity-adjusted mortality rates since January 2005. The current rate is 74 percent of the expected rate. Performance on Joint Commission core measures exceeds the norm for all measures and is above the top decile in five measures. There has been a 36 percent reduction in the pressure ulcer rate since July 2006. In addition, hospital professional liability costs have decreased by 46 percent, along with a 45 percent decline in the number of claims. PEERs has increased risk managers’ awareness of a greater number of potentially compensable events in real time that could have been addressed while there was still a potential to improve or avoid serious, unintended consequences.

Striving for Zero Defects

Trinity Health’s goal is to achieve zero defects. In most HROs, the cost of failure is extremely high; this is particularly true in health care. However, too many processes in health care still rely on human perfection to prevent error. As humans are imperfect, these systems are doomed to failure. The PEERs system attempts to measure how events are identified. Unfortunately, the majority of reported events are identified by individuals involved in the event rather than through these safety systems. Twenty-three percent of the reports were identified by human diligence, but only 3 percent were identified due to a built-in safety system; 4 percent were identified by a quality assurance process. The good news is that only 2 percent can be attributed to “good luck.”

As we continue to strive for zero defects and work to more closely resemble an HRO, we look for an increase in the percentage of events identified by our improved processes. One complication of using PEERs reports for interventional analyses is the danger of having incomplete information. It is difficult to determine how many events should have been reported. Because potential events identified by built-in safety systems could be interpreted as successes rather than defects, they might have gone underreported.

Also, incident reports are not the best way to determine the success of a safety intervention. The number of reports could actually go up because of increased interest in reporting. As staff see their reports are having a positive effect, they might increase their vigilance. One major goal of the system is to increase reporting. That viewpoint makes increased reports a system success rather than a failure. However, it is not an effective way to judge the effectiveness of an intervention.

PEERs Has Made a Difference

The PEERs system has become part of the culture within Trinity Health and has been instrumental in helping Trinity Health achieve its quality and safety goals. It has helped drive common processes to deal with similar problems and events across the system and decrease variability. This leads to a common understanding and helps to foster a consistent culture within Trinity Health. PEERs data have been used by many different groups for their focused improvement work.

Data from falls are routinely collected and used in an ongoing effort to reduce the number of patient falls. The number of patient falls with injury per 1,000 inpatient days is one of the core clinical indicators used across Trinity Health to assess patient safety for the system. Because PEERs is an in-house tool, it has grown and developed to meet the needs of many different groups in hospitals across the system. In a number of cases, it has grown as the users determine new safety endeavors.

The newest version of PEERs includes an updated section to capture events specifically related to the laboratory. Several hospitals within the system plan to accelerate their focused improvement work around laboratory events, necessitating that specific questions be added to target key laboratory safety issues. PEERs is helping Trinity Health achieve its goal of becoming a high-reliability organization. It has been instrumental in both discovering and managing unexpected events and also in reducing unexpected events.

Next Steps

Trinity Health’s experience with incident reporting will position the PEERs hospitals to respond quickly when patient safety organization reporting becomes a Federal requirement. Also, as Trinity Health moves forward with Genesis, our systemwide computer implementation, there are plans to link other clinical data to the data in the PEERs system. This will allow us to review incidents by clinical condition and to pull in full patient demographic information to add new depth to our data mining and reporting. The more information we can gather and use about our safety issues, the more opportunities we will have to improve our systems and to offer a safer hospital for our staff, our patients, and their families.