NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Hsu EB, Jenckes MW, Catlett CL, et al. Training of Hospital Staff to Respond to a Mass Casualty Incident. Rockville (MD): Agency for Healthcare Research and Quality (US); 2004 Jul. (Evidence Reports/Technology Assessments, No. 95.)
This publication is provided for historical reference only and the information may be out of date.
Principal Findings
The extensive literature search identified 21 articles that described and evaluated an educational intervention designed to train hospital staff to respond to an MCI. The majority of these studies addressed the use of hospital disaster drills as a training tool (key question 1).
The studies represent a heterogeneous body of literature, ranging from descriptions of local drills to sophisticated telecommunications exercises. Studies also varied in terms of targeted staff, learning objectives, identified outcomes, and evaluation methods. Because of the wide range of foci for the studies, it is difficult to make definitive recommendations on the most effective approaches for training clinicians to respond to an MCI. However, some potentially valuable points can be identified in the literature.
Question 1. What is the effectiveness of hospital disaster drills in training hospital staff to respond to an MCI?
- Sixteen studies were identified that evaluated disaster drills as a training tool for hospital disaster procedures. The studies focused on drills for responding to conventional disasters such as transportation incidents, fires, and chemical spills. None of these studies used disaster drills to provide training in how to respond to a biological MCI.
- Disaster drills appeared to be an effective way to improve clinicians' knowledge of hospital disaster procedures.
- Drawing lessons from planning and outcome evaluation in the published disaster drill literature may strengthen future disaster response planning, especially in the areas of incident management and communications.
- Lessons learned from one type of disaster response must be applied with some caution to other types of drills.
- Disaster drills have the potential to identify problems with incident command, communications, triage, patient flow, materials and resources, security, and decontamination.
- Disaster drills usually were not designed to evaluate the effectiveness of patient care.
- It is difficult to draw firm conclusions about the effectiveness of specific types of hospital disaster drills for different types of disasters because of marked heterogeneity of training methods and weaknesses in study design and evaluation.
Question 2. What is the effectiveness of computer simulations in training hospital staff to respond to an MCI?
- Only one study described and evaluated the use of computer simulation as a training tool for educating clinicians about their hospital's disaster plan.
- Computer simulation is an economical method to educate key hospital decision makers about disaster preparedness. This approach can be used to improve hospital disaster preparedness prior to implementation of a full scale drill.
- Computer simulation was able to identify bottlenecks in patient care, electromechanical failures, crowd control issues and other security problems, and resource deficiencies.
- The evidence was insufficient to make definitive conclusions regarding the effectiveness of computer simulation as a training tool.
Question 3. What is the effectiveness of tabletop or other exercises in training hospital staff to respond to an MCI?
- One study described and evaluated a tabletop exercise as a training tool.
- Tabletop exercises can be used to teach disaster-related patient care in a way that simulates the practice setting.
- A tabletop exercise can provide an evaluation that yields immediate feedback and reinforces learning.
- Evidence is insufficient to reach definitive conclusions regarding the effectiveness of tabletop exercises as training tools for educating clinicians about hospital disaster response.
- One report described a regional exercise testing the readiness of top government officials to response to terrorist attacks. This exercise increased the awareness of the need for better disaster response planning.
- One study used audio-graphic teleconferencing as a means to educate emergency department staff in six countries about radiation incidents. This may be an effective way to educate hospital employees over a geographically diverse area.
- One study evaluated the use of a video simulation to educate hospital employees about disaster response. Video demonstrations may be an inexpensive, convenient way to educate a large number of staff about disaster procedures and equipment use in a short time, especially when staff work in different locations.
- Evidence is insufficient to make definitive recommendations on the use of tabletop and other exercises as training tools for educating clinicians about hospital disaster response.
Question 4. What methods or tools have been used to evaluate the effectiveness of hospital disaster drills, computer simulations, tabletop or other exercises in training hospital staff to respond to an MCI?
- Nineteen studies described the methods that were used to evaluate the educational intervention used to train clinicians in disaster response procedures.
- Thirteen studies used more than one evaluation method.
- Twelve studies used group interviews or debriefings.
- Six studies included “smart” observers (those with medical training).
- Three studies included “smart” casualties.
- Four studies included a written exam or questionnaire.
- Other methods to evaluate the educational intervention included observer checklists, victim tracking cards, self-assessment forms, video tapes, and a computer-generated picture of the situation.
Lessons Learned Based on Outcome Categories
Incident Command System
- The presence of a well-defined incident command system reduces confusion during exercises.
Internal Communications
- Overhead intercom systems may be unreliable during an MCI.
- Important telephone numbers and staff contact information must be updated on a regular basis and readily available in the event of an MCI.
- Staff must be trained to use various modes of communication (e.g., radio communications, telephones).
External Communications
- Effective communication during an MCI is key to the disaster response.
- Drills and tabletop exercises may be an effective method of improving interfaces between hospitals and federal, state, and local response agencies.
- The process of decisionmaking by conference calls can be inefficient and may lead to delays in taking action.
- Radio communication is an effective backup to land lines but may experience technical difficulties.
- Phone numbers of Emergency Operations Centers must be updated regularly and checked for accuracy.
Patient Triage
- Effective patient triage requires emergency department staff experienced in triage procedures.
- Triage zones should be easily identifiable.
Patient Care
- Simulated casualties are not always examined thoroughly.
- Documentation requirements may detract from patient care.
- Patients must be continually reassessed.
- Adequate care for victims with serious injury must begin in the field.
Patient Flow
- Corridors, exits, and routes for transportation should be clear of extra equipment that could block patient transport and delay care.
- Bottlenecks to patient flow (e.g., radiology, operating rooms) should be identified and addressed.
Patient Tracking
- Patients should be clearly identified with a bracelet, tag or some other method.
Security
- Drills and exercises may identify security and crowd control issues.
- Adequate security must be provided.
Materials and Resources
- Drills and exercises can identify deficiencies in supplies, equipment, personnel, and pharmaceuticals.
- Central storage and the emergency department must communicate supply and demand.
- Emergency department staff must be familiar with location of critical supplies.
Decontamination
- A significant amount of time is required to set up decontamination equipment and don personal protective equipment.
- Appropriate personal protective equipment must be worn.
- Decontamination of deceased must be addressed.
Other
- Each drill provides learning opportunities. In any given drill, these groups of participants, but currently there is no standard by which whole as a complete success.
- Disaster response personnel must be clearly identified.
- Adequate pre-drill training is important for the drill's success.
Limitations
This evidence report has a number of limitations, of which the most obvious is the small number of studies that were directed to the training of hospital staff in how to respond to MCIs. In addition, the search was limited to published English language articles. There may be classified, unpublished material or studies in press that were not included in this report. An example would be material from the U.S. Department of Defense, which undoubtedly has experience at testing different scenarios, but these materials are not available in the published literature. There may be aspects of military disaster drills that have potential applications for hospitals. Another example is unpublished results of drills and exercises associated with JCAHO requirements taking place at state and local levels. It is not known if this unpublished material includes evaluation data.
Another major limitation relates to the fact that different types of disasters raise different issues for training of hospital staff. For example, the issues differ for drills that simulate a transportation incident or fire and those that simulate a biological incident. The latter most likely would evolve over an extended time period while the former would introduce a sudden influx of cases to hospitals. Since nearly all of the published studies focused only on training in how to respond to conventional types of disasters, little direct evidence exists on the effectiveness of training hospital staff in how to respond to a biological MCI.
Although many experts believe that tabletop exercises have an important role to play in disaster preparedness,37–40 the literature search identified only one study that evaluated use of a tabletop exercise for training of hospital staff in disaster preparedness.36 One other study reported on the use of a tabletop exercise to provide disaster training to emergency medical technicians (EMTs).40 However, this study did not meet our eligibility criteria because it did not involve hospital staff. In this study, Chi assessed the attitudes of EMTs toward tabletop exercises. Survey results showed that EMTs believed that tabletops performed better than field exercises in linking the results of disaster exercises to appropriate changes in terms of training, equipment, and supplies. Other tabletop exercises have been described in the literature,38, 39 but none of these reported any evaluation data. Without evaluation data, one cannot draw conclusions about the effectiveness of such exercises. Although numerous tabletop exercises have been conducted as a less expensive alternative to operationalizing drill training, most of the identified studies on tabletop exercises were not focused on hospital-based activity, and among those that were, no results, i.e., no data, were given.
The quality and methodological limitations of the studies make it difficult to judge external validity of results. Furthermore, marked differences in educational interventions, objectives, targeted audience, and evaluation methods present challenges in drawing generalized conclusions relevant to bioterrorism preparedness. Another specific limitation is that the search identified only one evaluation of a tabletop exercise, one of a computer simulation, one of video training, and one of teleconferencing. Although each of these educational techniques may have distinct advantages, the evidence is insufficient to draw definitive conclusions about their effectiveness.
In addition, the financial burden of the educational interventions generally was not reported. Full-scale disaster drills are expensive.27 The large-scale three-day drill described by Inglesby et al. cost $3 million (U.S.). None of the other studies identified cost figures, thereby leaving a gap in this important aspect of hospital disaster preparedness. Finally, very few studies identified the organizing or sponsoring entity (e.g., federal, state, or local agency, or hospital) for the drill or exercise, thereby precluding any conclusions about who most effectively plans and conducts drills.
Future Research
Part of the challenge in reviewing the existing literature regarding training of hospital staff to respond to an MCI arose from the numerous formats of studies and differences in evaluation methods. Creating a template for future training reports (e.g., Utstein-style guidelines)41 may facilitate the accessibility, synthesis, and interpretation of collected data. Authors of future reports should consider the merits of adopting a common nomenclature and explore establishing the incident command system among hospitals as a standard.
One major issue is the cost of conducting drills and the need for a steady stream of funding to support these activities. Although governmental funding has increased recently, many hospitals are short of flexible funding and are unable to assign high priority to disaster preparedness. Given the evidence on the potential value of drills, the recent international events that indicate an increasing likelihood of future MCIs, and the heightened focus of the government in providing funding, it has become a priority to explore the uses of drills. Evidence is needed. One approach might be to provide funding for hospital exercises that are designed to overcome the limitations identified here.
The purposes of drills are important and in general underexamined. Drills may have many different purposes and it is imperative to conduct different types of exercises to test the different operational elements involved. Valid purposes include testing communications, triage planning, evacuation or decontamination procedures, and focusing on improving familiarity with emergency protocols. Related issues in drilling include the necessity to be efficient and to incorporate continuous training to meet the needs related to turnover in the hospital workforce. To develop a drill, major preparedness issues should be identified and then tested in different types of drills; however, hospitals should remain open to learning from the unexpected that occurs during the course of a drill as well. Overall, it is important to follow the principle of learning from the experience without judging the drill as a success or a failure.
The current evidence is not definitive on the effectiveness of hospital disaster drills in training staff to respond to an MCI. Although hospital disaster drills arguably may provide the most realistic training, they also represent a resource-intensive training format for MCI preparation. To date, no evidence supports the cost effectiveness of any particular type of training intervention. Future studies addressing the costs of educational interventions will facilitate recommendations regarding training strategies. For example, the strength of a video is that it is a relatively inexpensive way to standardize training for a large group (hospital employees or others) who have different schedules and operate remotely from each other. A good tabletop exercise allows observers to see the action develop, to gain increased awareness, to build teamwork, and to test strategic scenarios. Tabletops are economical and more efficient for some purposes. Given these attributes of these different types of exercises, a logical progression to familiarize employees with a hospital disaster drill plan may be to use videos in a group setting, then demonstrate key points with a tabletop exercise, and later graduate to a fully operationalized partial or complete drill.
Another major issue that needs to be addressed is the lack of evaluation of completed drills, and an equally important issue is the dearth of published reports from individual hospital- or health department-supported drills. Very few reports of hospital disaster drills have been ublished or made available in electronic databases of relevant material. Therefore, hospitals and other agencies are denied the benefit of others' experience. In the future, agencies or institutions funding drills may chose to prioritize both evaluation and more rigorous written reports post-drill. Federal agencies might direct grantees to document their findings and submit them for publication. As drills of some nature are now mandated by JCAHO, hospitals should be able to generate an increased number of reports about what does and does not work well. JCAHO may also want to encourage hospitals to meet the drill requirements through other than standard (i.e., non-disaster-related) training exercises.
Translating the reports of drills into future activity may help to promote orderly development of capability in the field. Disaster drills might be designed to test specific elements of response as identified in this report, including incident command system, communications, triage, patient flow, tracking, security, materials and resources, and decontamination.
More attention should be given to evaluating the effectiveness of relevant training programs in a scientifically rigorous manner. The weak study designs led to insufficient strength of the evidence to draw firm conclusions. This is typical of the present literature in disaster medicine and points to a need for better_designed studies.
This synthesis of the existing evidence on the implementation and evaluation of hospital disaster drills, computer simulations, and tabletop or other exercises may help to establish criteria for assessing the effectiveness of future training exercises. Because the current evidence on tools or methods used to evaluate effectiveness of training hospital staff to respond to an MCI is insufficient, development of a modular evaluation tool to address the effectiveness of different educational interventions will be of significant importance. These modules could follow the elements of response identified above (e.g., incident command system, triage, treatment, communication, and security).
Finally, the published evidence includes very little information that directly pertains to the training of hospital staff in how to respond to a biological MCI. Although bioterrorism is a current federal priority, only one study described the response to a mock bioterrorist attack.27 The majority of studies focused on more common disasters (e.g., a fire/explosion or transportation accident).18–20, 22, 23, 25, 26, 30, 32, 33, 35, 36 It seems reasonable to postulate that preparedness for a conventional MCI would enhance readiness for unconventional MCIs. For example, an effective response to a bioterrorist event will depend on the general training and preparedness necessary for other MCIs, including training on communications, triage, and treatment during an MCI. However, a biological event would differ from a conventional MCI in important ways, such as evolving presentation of cases over days or weeks with a potentially high casualty toll, coupled with a large number of concerned and potentially exposed victims. Future training should therefore also address biological as well as chemical, nuclear, or radiation events.
Disaster training activities at the local, state, and national level could improve our knowledge pertaining to drill training, but they have not yet been reported consistently in the peer-reviewed literature. This lack of information suggests a need for creating improved ways of sharing such training experiences. Future disaster preparedness would be facilitated by a systematic method for collecting this information and making it readily available for review and synthesis.