Setting

We conducted 10 in situ simulations in the labor and delivery unit of a full service, 390-bed Midwestern community hospital. This level II birthplace performs approximately 3,400 deliveries per year, with a C-section rate of 32 percent of total deliveries. The unit has 15 labor rooms, two operating rooms, a postpartum unit, and a level II neonatal nursery.

Participants

The in situ simulation trials involved two classes of personnel drawn from hospital staff: direct team members and indirect team members. Direct team members were informed about the in situ simulation ahead of time and agreed to participate in a particular simulation trial. We recruited direct team members, including obstetricians, labor and delivery and special-care nursery nurses, neonatal nurse practitioners, anesthesiologists, certified registered nurse anesthetists (CRNA), and operating room staff from the hospital for every simulation trial. Two people were recruited to play the roles of a mother and a significant other for each trial (confederates). During the briefing preceding each trial—direct team members were instructed to call upon any indirect team members, such as backup surgeons, blood bank, laboratory, and central supply personnel, as well as extra personnel, code teams, language interpreters, respiratory therapists, and others—to treat the patient, just as they would during a true obstetrics emergency. Indirect team members were included on an as-needed basis, only if drawn into the simulation by the direct team members. Each simulation trial included an average of 20 staff members.

In Situ Simulation Setup

Production of the in situ simulation required the use of a labor and delivery room, a fetal heart tone simulator (connected to our usual fetal heart tone monitor), a cervical dilatation box, a confederate playing the mother, other confederates as significant others, an operating room, and two manikins (SimMan^® and SimBaby^® by Laerdal). An artificial gravid uterus was made by enveloping a rubber baby toy in a plastic bag with water, sometimes colored red by gelatin (to mimic blood) or green by pea soup (to mimic meconium). This was then wrapped in fabric foam and taped to mimic the uterus. Finally, the “uterus” was placed on the manikin and covered with thin dark fabric to mimic skin. The normal paperwork from labor and delivery was used for documentation. Video cameras were placed in the labor and delivery room and also in the operating room to capture all interactions of the surgical and pediatric teams. A handheld video camera captured all events as the team traveled through the hallways from the labor and delivery room to the operating room.

The stationary video cameras in the delivery room and the operating room were wired to an observation room where nonparticipants—such as our debriefers, the FMEA team (core leadership team), and administrators—could monitor the simulation in real time. This served several purposes, including allowing debriefers to observe and identify active and latent failures in the simulation. It also allowed a simulation director to communicate wirelessly to the obstetrician and describe the operative field during the C-section as dictated by the scenario.

In Situ Simulation Scenarios

Three scenarios based on actual sentinel events created collaboratively by an obstetrician and a clinical nurse specialist served as the basis for our in situ simulation trials. Each scenario was designed to prompt nontechnical team behaviors, such as leadership, shared mental model, situational awareness, and closed-loop communication.¹⁹ The scenarios included typical distractions—such as an overly inquisitive or rude significant other, a language barrier, talkative mother, lack of a prenatal record, and other factors that could interrupt team flow—so that the simulation team would be stressed by both the clinical and social aspects of the care.

Each simulation started with a briefing on labor and delivery, including a discussion of the simulation process, its limitations, and the importance of performing as one would normally perform during actual clinical care. Participants were told that observers were looking for teamwork and communication skills, not for technical skills. In situ simulations started with the nurse’s first encounter with the patient, often walking into the room with the patient. Simulations averaged 45 minutes and typically included such factors as one nurse and the patient; two nurses and the patient; the addition of an obstetrician; taking the patient to the operating room for an emergency cesarean; entering the operating room; delivering the baby; a need for blood products; and neonatal resuscitation. Simulation typically ended after 10 to 15 minutes of neonatal resuscitation or until blood products reached the operating room.

Immediately after each simulation, the interdisciplinary team was debriefed on the following topics related to team performance:

• What went well during the trial?
• What did not go well?
• What could have been better?

Debriefing was facilitated by two experienced debriefers (one obstetrician and one clinical nurse specialist) and the video playback of the simulation trial. The debriefers stopped the playback at any juncture deemed important, such as after a communication lapse or loss of situational awareness, and the issues observed were discussed in detail with the participants. All participants were given an open opportunity to add comments before and after viewing the video playback. Debriefing typically lasted approximately 2 hours.

Conducting the FMEA Based on In situ Simulations

To conduct an FMEA on emergency C-sections, we used the six-step model developed by the Joint Commission:⁵ (1) identification of the process, (2) team selection, (3) description of the process, (4) listing and ranking of failure modes, (5) identification of root causes, and (6) determination of an action plan.

Step 1: Identification of the process. We focused on the emergency C-section process because it is a high-risk process with a large amount of variability in the teams that provide this emergent care. This presents a significant opportunity to improve patient safety for laboring mothers and to improve team performance during emergent, high-risk procedures.

Step 2: Team selection. We formed two separate types of teams for the FMEA analysis: a simulation trial team and an FMEA team. There were 10 interdisciplinary simulation trial teams, each consisting of an obstetrician, labor and delivery nurses, neonatal registered nurses, clinical nurse leader, birthplace manager, neonatal nurse practitioner, anesthesiologist, blood bank director, pharmacy manager, certified registered nurse anesthetist, operating room technicians, and numerous personnel drawn into the simulation from various support departments, including the laboratory, rapid response teams, and code teams. During the debriefing stage of the in situ simulation, these teams identified failure modes and their reflection of past failure mode experiences triggered by the simulation trial. The interdisciplinary FMEA team consisted of an obstetrician, clinical nurse specialist, birthplace director, neonatal nurse practitioner, and clinical quality consultant. This core team conducted the FMEA analysis based on the data from the simulation trials.

Step 3: Description of the process. The in situ simulation (briefing, simulation, debriefing, and debriefing of the debriefing) provided an intense framework for process mapping. The FMEA team studied the process by flow-charting the key process steps, identifying process weaknesses, and analyzing the data gathered from the debriefing phase of the simulation trial. While the flow diagram is an important tool to understand the process steps, it typically is created by a team that is removed from the core process in time and space.

Our process mapping was developed by the FMEA team based on three data sources: (1) the descriptions and analysis of the data obtained from the in situ teams at the end of each trial, (2) the detailed and in-depth studies of the video recordings of trials by the FMEA team, and (3) the data obtained through a focus group of the FMEA team. In other words, the process diagram was prepared not only from team members’ prior experiences, but also from data collected during in situ simulations (recorded trials) and participating team members’ recall immediately after each in situ trial.

The process mapping identified six major stages of the emergency C-section process: (1) admission of the mother to the unit and initial assessment by the primary nurse, (2) unfavorable changes in the clinical condition of the mother and fetus and arrival of the second nurse for help, (3) assessment of the mother by the obstetrician and the decision for an emergency C-section, (4) transfer of the mother to the operating unit, (5) operation, and (6) infant resuscitation.

Step 4: Listing and ranking of failure modes. Unlike the traditional FMEA, our failure mode identification was conducted in three stages. First, the FMEA team prepared a list of potential failure modes based on the common method of brainstorming. Next, this team reviewed the video recordings of the 10 in situ simulations, identified additional potential failure modes using a structured observation form (consisting of a list of known failure modes with room to add new failure modes), and revised the list of failure modes accordingly. Lastly, this revised list was supplemented with the failures identified by the interdisciplinary simulation trial teams during the debriefings. The participant debriefings were facilitated by experts and conducted using a structured method to elicit team observations. This included reducing the authoritarian gradient between team members and use of the nominal group process techniques.²⁰

All observations were listed on flip charts for team discussion and later categorized according to specific types of active failures and latent conditions. This resulted in comments and feedback from approximately 200 individuals. The participant’s identification of failure modes originated from the in situ simulation experience. In addition, we learned that the simulation triggered participants’ reflections of past failure mode experiences, which they frequently identified during the debriefing. The FMEA team categorized the data from these 10 in situ simulation trials and then used this information as the basis for identifying potential failure modes and for developing risk priority numbers. The FMEA team also categorized each failure mode as either an active failure or a latent condition.¹ Next, based on the revised list of failure modes and the process flow chart developed in Step 3, the FMEA team assessed the potential effect of each failure mode on the emergency C-section process.

After developing a list of potential failure modes, the next step was to calculate a risk priority number (RPN) for each potential failure mode. An RPN is the quantitative estimate of the risk associated with each failure mode.⁵ FMEA teams assigned an RPN to each failure mode based on three factors: (1) its likelihood of occurrence (L), (2) its severity if it occurred (S), and (3) the detectability of the occurrence (D). The RPN was calculated using the formula: L x S x D, where high numbers indicated a high priority for intervention and action. Table 1 defines each of the three factors included in the RPN calculation and the rating scales associated with each factor.

Table 1

The factors included in the risk priority number calculation and their rating scale.

Step 5: Identification of root causes. Based on the RPN scores, the FMEA team prioritized and decided which failure modes to focus on for further investigation. Next, an RCA was conducted for each of these selected failure modes. The RCA was conducted based on the information obtained during debriefings, the FMEA team’s review of recorded in situ simulations, and brainstorming sessions based on their personal experiences. Each failure mode, with its root cause, was listed in a chart as a summary from the debriefing data (Table 3). This allowed the FMEA team to visualize inter-relationships between process failures and group them into similar categories for action plans that were developed subsequently.

Table 3

Results of root cause analysis.

Step 6: Action plan. In this stage, action items were developed for the causes of failure modes.⁵ The RCA indicated two types of interventions: those that could be completed reasonably soon through rapid cycle improvement, and those that required extensive planning and interdepartmental collaboration. Immediately after each simulation trial, to prevent or mitigate failure modes, the FMEA team implemented changes that could be done easily with few resources and without delay using a rapid cycle improvement approach. In addition to these types of interventions, the FMEA team also developed an action plan that required extensive planning based on the traditional FMEA model. An action plan was developed for each failure mode that was identified as needing further action based on RPN scores. A single individual or a group of individuals at the institution responsible for completing or facilitating each action plan was identified and required to periodically report back on set due dates.

A unique feature of our action plan stage was that the effectiveness of the actions taken was reevaluated in subsequent simulation trials. During followup simulations, we observed in real time and also gathered data from the in situ trial participants regarding the influence of any actions taken on the failure modes.

Failure Modes

Results of the FMEA with in situ simulation are presented in Table 2. Ten failure modes were identified with RPN values ranging from 40 to 720 points. Five of the failure modes were categorized as latent conditions, and five emerged from active failures.

Table 2

Failure modes and effects analysis with risk priority number.

We distinguished between latent conditions that were created due to decisions at higher organizational levels (where unintended consequences can lie dormant for a long time until triggered by local conditions) and active failures, which are unsafe acts committed by those at the patient/provider interface.²¹

The highest ranked failure mode, with an RPN score of 720, was the “lack of identified and clear roles for team members in an emergency C-section.” The potential effect of this failure mode was identified as “confusion in task assignments” along with “uncoordinated and fragmented care.” This failure mode occurred in every simulation trial, and in situ participants repeatedly identified it as a source of poor team performance and recurrent potential for patient harm.

The next highest ranked failure mode, with an RPN value of 700, was the “inconsistent process of ordering and receiving blood products and lab results.” Again, this failure mode occurred in every simulation and resulted in delays in receiving blood and/or critical lab results with subsequent mismanagement of the clinical situation.

The third highest failure mode ranking, with an RPN of 600, was the lack of closed-loop communication between the operating room and the blood bank. The participants of every simulation identified this failure mode. Although the second and third failure modes both deal with delays in receiving blood for transfusions in a timely manner, an important distinction was recognized regarding the type of failure by the participants. Specifically, lack of closed-loop communication was identified as an active failure, but an inconsistent process to order blood was considered a latent condition. These three failure modes reflected 59 percent of the total RPN values calculated for the entire FMEA (2,020/3,426 points). They required extensive action plans, while the remaining seven failure modes were remediable with more immediate rapid-cycle action plans.

Root Cause Analysis

The RCA of the failure modes by the FMEA team identified three common causes of error: (1) staff misunderstanding of policies/procedures and roles during emergency C-sections, (2) interdepartmental or intrateam communication issues, such as not having a standardized, common language and other human factors, and (3) institutional process failures, such as poor logistics, equipment failures, and poor policies/procedures (Table 3). This categorization allowed the FMEA team to better understand how an action plan could be developed for each failure mode.

The delay in blood getting to the operating room during a maternal hemorrhage deserves further mention and serves as a more specific example of our results. The simulation provided invaluable information regarding this important process that might have been missed with routine FMEA.

The simulation participants’ comments revealed the process to order and draw labs and receive results depended on (1) the obstetrician remembering five different necessary labs (type and screen), hemoglobin, platelets, fibrinogen, and International Normalized Ratio/Partial Thromboplastin Time; (2) the circulator taking the order and calling the health unit coordinator at a desk remote from the operating room to put the order into the computer; (3) the lab technician responding to the operating room to draw the labs and return them for analysis; and (4) the paperwork being completed with correct instructions, or the results could be called in to the main labor and delivery desk and not into the operating room. Routine FMEA and process mapping could determine all of the above.

However, our simulation revealed that successful completion of this process also required closed-loop communication at many different critical junctures (communication). The process of ordering blood was inconsistent between simulations, and confusion existed around the policy and procedure for ordering labs and blood while in the operating room (staff misunderstanding). This process was complicated by the poor placement of the operating room phone, which was distant from the anesthesiologist (poor logistics). Thus, this one process had all three causes of error present and required simulation to develop a more complete understanding of the failure modes. This understanding would not have been possible with routine FMEA.

The debriefings also exposed staff misunderstanding of the time requirements for blood products to become available. Many staff members did not realize that it would take 40 minutes to obtain type-specific blood, much longer than they thought. Impatient, the operating room teams repeatedly asked, “Where is the blood?” often making frequent calls to the blood bank to repeat that they needed “2 units” of blood. The blood bank personnel often wondered, “Do they need 2 units or is it now 4 units?” One lab technician succinctly summed up the problem in the debriefing, “You called four different times for 2 units of blood, so did you want 2, 4, 6, or 8 units?” Finally, once a blood product was ordered correctly and the blood bank was ready to release the product, no person was assigned (lack of role definition) to retrieve the product. Often, the blood bank assumed that the operating room would be sending someone, and the team in the operating room assumed that the blood bank would be sending the blood, neither of which occurred. The need for clear training, role clarification, and consistent communication regarding the timing of blood products and retrieval of blood products would not have been apparent from routine FMEA carried out remotely from the clinical site.

Rapid-Cycle Improvement

Immediately following each simulation, the hospital instituted numerous interventions to the labor and delivery unit that could be achieved easily with few resources and without delay. Some examples include moving the telephone to the head of the operating room table for direct access by the anesthesiologist; renaming the operating rooms to avoid confusion during an emergency C-section; initiating immediate point-of-care education for staff about how to call the interpreter on the phone, rather than wait for them to arrive on site; and having engineers do a site review to find out where dead zones existed in the hospital for the overhead paging system. Most of these changes were completed within a week.

Extensive Planning

Action plans that required extensive planning involved failure modes that had high RPN numbers. A multidisciplinary team composed of obstetric and other hospital staff met regularly and was held accountable with timelines. The second highest ranked failure mode, “inconsistent process of ordering and receiving blood products and lab results,” had an action plan that required extensive planning with personnel from other departments.

A multidisciplinary team including a pathologist, lead lab technician, clinical nurse specialist, obstetrician, and anesthesiologist developed an “OB hemorrhage panel.” A pre-formatted form was designed on bright lime green paper so that RNs or MDs could order a standard set of labs and blood products for the mother or the baby during an emergency. This paper form is now kept in a zip-lock bag with three blood tubes on the wall in the operating room near the anesthesia table. The paper form has a checklist format and includes the standard orders, instructions regarding timing for all blood products (when they can be expected), a place to circle a call-back phone number for the operating room, and instructions to send a runner to the lab to retrieve the product. Since no other form is lime green, it communicates the message that this particular order is “STAT” and therefore should have the highest priority in the lab. When experiencing a maternal hemorrhage, the RN or MD needs only to order an “OB hemorrhage panel,” and there is no need for the RN or MD to remember which labs to order. The blood bank is put on notice immediately as to the clinical situation, and the OR team has a reminder on how long blood products will take, and that a runner needs to be sent for blood.

Advantages of In situ Simulation in Combination with FMEA

The application of in situ simulation data provided a more objective, comprehensive, and systematic way to identify potential system risks in emergency C-section processes and resulted in a more realistic list of potential active and latent failures. The in situ teams identified six failure modes categorized as latent conditions, some of which resided dormant on our labor and delivery desk for many years (for example, the operating room phone being situated out of the anesthesiologist’s reach was a work-around for 40 years).

Four of the failure modes were active failures. Whenever possible, the action step plans developed by the FMEA team were implemented immediately to provide prompt feedback to the teams and microsystem that identified the process failures. Unlike typical FMEA approaches, the in situ simulation allowed us to evaluate the effectiveness of the rapid-cycle action steps in subsequent in situ simulations.

This new methodology allowed for more open discussion of failure than a typical RCA environment because the guilt, shame, and embarrassment of a recent failure resulting in patient harm was not a factor. Compared to other risk assessment methodologies, a unique advantage of the in situ simulation is the safe environment it creates in which health care scenarios—developed based on sentinel events—can be replicated and videotaped. Undetectable failure modes present a higher risk to patient safety than others modes.⁷ In situ simulation facilitates the RCA of high RPN failure modes because it allows for more open discussion and encompasses more staff input for understanding failure and its causes.

In situ simulation gives observers a real-time visualization in which to observe both failure modes and the effects of failure at a moment in time. These critical junctures in time, which are important to the patient care process, are much more vivid and analyzable in in situ simulations. Both the failure mode and its effects can be immediately analyzed to appreciate how they influence team performance and possibly result in patient injury. In situ simulation views the failure mode in its normal context of place and time.

Using in situ simulation, the process diagram was not flow-charted from memory; it was experienced and recalled by the participants immediately after a simulation trial. Through the use of simulation, we were able to systematically interrogate the process to undercover process failures that otherwise would continue to be unknown and undiscovered, remaining dormant until involved in a patient injury.

FMEA with in situ simulation permits the evaluation of teamwork and communication skills and provides a concurrent internal audit from staff as to the fidelity of the experience. It links both latent conditions and active failures that are typically not identified in FMEA. One of the prevailing themes of these 10 failure modes was poor team performance.

The foremost action plans included advanced team training using the AHRQ TeamSTEPPS™ training curriculum. Such action plans are not typically found in FMEA-only approaches. Because of this ability to see human factors at important times in patient care, we have found more failure modes than usual. The in situ simulation helps us understand team training and performance at a moment in time. Time-dependent communication is appreciated by staff and observers, and the debriefings make its relationship to potential harm apparent. By identifying communication issues, in situ simulation helps address the concerns of the Joint Commission’s Sentinel Event Alert #30, “Preventing death and injury during delivery.”¹⁸ RCA conducted by the Joint Commission indicated that communication issues topped the list of causes for sentinel events at 72 percent.

The advantages of using in situ simulation for detecting risk as discussed in this study can be summarized as follows. It is a prospective method used by an interdisciplinary team to uncover and analyze process failures on a care unit and thereby identify and rank failure modes in a way that realistic actions can be taken to create countermeasures for patient safety. Unlike conventional FMEA analysis, the failure modes are isolated by stressing the process in a way that can only be done during an actual emergency, when process failures usually result in creative workarounds to address the problem of the moment rather than deliberate system improvement.

Limitations of In situ Simulation in Combination with FMEA

There are several disadvantages of applying in situ simulation for FMEA. First, administrative support is required for supplies, equipment, and human resources. Second, in situ simulation is time-intensive for both participants and facilitators. Third, it can create confusion for other departments that are drawn into the simulation. Fourth, it can cause disruption in the patient care unit.

Cost is certainly a factor, but as simulation becomes more commonplace, efficiencies will occur. A Hawthorne effect is certainly possible among the participants; they were aware of being watched and filmed. However, despite this knowledge, many process flaws and team failures were identified. Health care workers, even on their best behavior, are not perfect.

Finally, this methodology has not been compared with results from the traditional FMEA technique. The next step in validating the FMEA using in situ simulation is to prepare a process map and RPN of the same process with both techniques. This comparison would help to determine which features are identified by the in situ simulation in contrast to FMEA without this technique.