Abstract

Objective: This study is intended to advance patient safety by demonstrating the effectiveness of coupling surveillance report performance feedback with evidence-based educational materials and other communications that help hospitals identify priorities and methods for medical injury prevention. Methods: Medical injury surveillance findings are reported semiannually to hospitals to support their quality improvement efforts. We developed educational manuals for five priority medical injury topics that covered indicator definitions, problem scope, interventions to prevent injuries, and evaluation methods. Bimonthly newsletters provide updates. Evaluation of 123 hospitals in a randomized controlled trial with structured surveys and medical injury surveillance is ongoing. Results: Preliminary results at 29 intervention hospitals suggest our approach resonates with stakeholders, has the general support of participating hospitals, addresses barriers to using information, and builds on existing competencies. Conclusions: While the impact of this medical injury prevention educational strategy requires further evaluation, the results may guide construction of performance reports, educational manuals, and newsletters to maximize the usefulness of patient safety information.

Introduction and background

Medical injuries with serious clinical and public health consequences occur regularly in the course of diagnostic or therapeutic interventions, as shown in studies based on medical record reviews. ^{1 – 4} Improving patient safety in hospitals is a complex process that requires receiving accurate, usable data to understand where the problems lie; having the evidence and other information to inform changes; and establishing institutional buy-in to make those changes.

Evidence of effective interventions for the prevention of some medical injuries has been growing steadily. ⁵ However, knowledge of injuries and effective interventions has not been easily translated into clinical practice to prevent or mitigate the occurrence of injuries within the health care system. ⁶

The importance of data

A recent Institute of Medicine (IOM) patient safety report stated that “better management of health information is a prerequisite to achieving patient safety as a standard of care.” ⁷ Efficient and representative patient safety reporting systems using valid data are an essential foundation for development, implementation, and evaluation of effective practices in patient safety.

Ideally, the data also should be relatively easy to obtain and interpret. Individual hospitals already routinely gather and report administrative discharge data. While publicly available hospital performance data have limitations, their usefulness has not been adequately explored. Administrative data could be valuable for identifying patient safety problems and monitoring the effectiveness of interventions to prevent the problems' occurrence and reduce their impact, especially when the data are appropriately refined, risk-adjusted, and confidentially delivered in periodic surveillance reports.

Information to educate and create buy-in

Even when data have these desirable properties, a single data source is insufficient to gain a complete understanding of medical injuries. Additionally, these data alone will not lead to improvement in clinical care. The data must be used in a context of other information, such as clinical guidelines and best practices, and communicated throughout a system characterized by complex operations, established organizational culture, and competing priorities. Having relevant tools and products closely accompany the injury reports is a useful strategy because it provides evidence regarding the problem and “immediately” offers ready-to-use solutions, helping to reduce resistance due to competing priorities.

Objective and program overview

We developed a multifaceted demonstration approach using publicly available hospital discharge data to report medical injury performance directly and confidentially to hospitals. We also examined the effects of confidential patient safety surveillance reports combined with evidence and better practice-based educational intervention tools on quality improvement efforts and patient safety in 29 Wisconsin acute care hospitals.

The objective of this paper is to describe the initiative, the implementation process, and observations that are important for patient safety efforts. This descriptive paper serves two important functions in patient safety:

(1)

It shows how surveillance reports can help hospitals to identify priority areas for medical injury prevention and control.

(2)

It demonstrates the development of education manuals that can provide relevant information to staff overwhelmed with data. We provide scientific evidence and best practices for five specific priority medical injury topics in a usable and accessible format for clinicians and hospital personnel.

Over a 2-year period, 29 intervention and 60 feedback-only hospitals in Wisconsin have been receiving periodic surveillance reports containing medical injury data aggregated for the State, as well as specific data to their individual facilities. During the second year of the project, intervention hospitals also received educational manuals developed by the Wisconsin Medical Injury Reporting System (WMIRS) Research Group. Each manual contains evidence-based guidelines, intervention recommendations, and review articles specific to the priority injury area, coupled with related discharge data for their own hospital. The intervention director, a registered nurse, clarifies information in the reports and manuals to assist in the interpretation of the data and application of the manuals to guide patient safety efforts of quality improvement personnel in hospitals. Hospitals continue to receive semiannual surveillance reports plus bimonthly newsletters to provide reinforcement of key guidelines, and intervention recommendations and updates of new information specific to priority areas.

We expect that pertinent, timely education for hospital personnel concerned about protecting their patients will have a positive effect on health outcomes. The impact of this educational strategy for medical injury prevention and control is being evaluated. Results of longitudinal analysis of administrative data and surveys of hospital personnel regarding the usefulness and impact of the reports and intervention manuals will provide data to inform insight about the effectiveness of this approach.

Useful tools to prevent medical injuries

The conceptual model for the potential influence of WMIRS patient safety reports and educational manuals on medical injury prevention at hospitals is depicted in Figure 1

Figure 1

Framework for the influence of WMIRS patient safety reports and educational manuals on medical injury prevention at hospitals. Adapted from Ginsberg.

Effective education is a classic component of injury prevention and control. Educational strategies, including persuasion, behavior modification, and social influence, play an important role in laying the foundation for all injury prevention approaches. ⁹ A focus on preventing medical injuries (rather than identifying errors) identifies more reliable data, avoids defensiveness, and encourages an emphasis on patient outcomes. ¹⁰ Another IOM report, Health Professions Education, contended that “all health professionals should be educated to deliver patient-centered care as members of an interdisciplinary team, emphasizing evidence-based practice, quality improvement approaches, and informatics.” ¹¹ These methods have generally been found to be effective in changing physician behavior. ¹²

Good evidence supports the use of specific interventions to reduce the risk of priority medical injuries, yet these strategies are not always implemented. ¹¹ For example, although several high quality, randomized controlled trials have shown that perioperative beta-blockers reduce cardiac events during surgery, they are used only 36 percent of the time. ¹³ We also know that traditional continuing medical education strategies often do not result in practice change if they take place out of the context of a specific patient care event. Therefore, we couple the provision of hospital-specific patient safety data with complementary educational manuals targeted to the five priority injury areas.

Employing an injury control model, we use surveillance techniques to identify patient safety problems and confidentially report them back to each hospital, supplementing the data with educational materials that provide the best evidence available for effective interventions for change. These evidence-based, quality improvement strategies include the following:

• Audit of electronic hospital discharge records and confidential feedback summarizing clinical indicators of hospital performance of medical injury rates over a 6-month period, with subsequent recommendations for clinical action.
• Recognition of needs and outcomes assessments as precursors to change and tools for evaluating results.
• Distribution of educational materials and printed recommendations for clinical care, including practice guidelines and reminders to hospital personnel.

These actions are supported by educational outreach visits by a trained expert who meets with quality improvement personnel to inform the use and interpretation of feedback reports and educational manuals, to identify patient safety issues of high priority in their settings and “best practices” from their own work, and to solicit feedback on materials we have provided.

Methods

For this demonstration, the WMIRS group at the Medical College of Wisconsin randomized the State's 123 nonfederal, general, acute care hospitals into three groups. For the 3-year study period, the control group of 34 hospitals receives no reports or interventions. The feedback-only group of 60 hospitals receives just the semiannual surveillance reports. The intervention group of 29 hospitals receives semiannual surveillance reports plus educational interventions of five priority-topic manuals, professional staff guidance and support, and bimonthly newsletters. All reporting is confidential. Feedback-only and intervention groups receive aggregate data for all State hospitals plus their own information.

Selection of priority medical injuries and preventive interventions

We selected key medical injury topics for hospitalwide intervention based on the Agency for Healthcare Research and Quality's priority areas for national action, ⁵ the frequency and severity of medical injury occurrences based upon WMIRS surveillance reports, input from intervention hospitals, and expert panel assessment of the relative feasibility of potential interventions. The priority topics, definitions, problem scope, and key interventions to prevent medical injuries are listed in Table 1.

Table 1

Priority topics, definitions, problem scope, and interventions for preventing medical injuries.

Educational manuals and newsletters

Using extensive literature review and examples of hospital best practices, WMIRS physician experts developed a set of condition-specific manuals that summarize research evidence and suggest practical strategies to prevent medical injuries. The overwhelming and often fragmented amount of information regarding medical injury prevention is condensed to a focused, organized product the hospitals can easily understand and use.

The educational manuals are presented in a cycle framework that is familiar to quality improvement personnel. ²⁶ In this way, the data and suggestions for interventions are integrated into a tool designed to assist hospital personnel as they compile and interpret their patient safety data; determine their own needs and priorities; establish goals; and plan, implement, and evaluate efforts to prevent medical injuries. In addition to intervention recommendations, key references are included to inform steps in the quality improvement framework. For example, we reinforce the 1996 Joint Commission on the Accreditation of Healthcare Organizations (JCAHO) sentinel event policy that requires investigation of the causes of adverse events such as medical injuries using root-cause analysis and monitoring the effectiveness of strategies. The key elements and contents of manuals are noted in Table 2.

Table 2

Key contents of the educational manuals to prevent priority medical injuries.

Follow-up newsletters, sent bimonthly, are designed to reinforce the educational manuals and provide practical, evidence-based resources for patient safety interventions. Each newsletter contains a timely lead article, a question-and-answer section, and three additional features. The “Toolbox” section lists Web sites and resources to promote patient safety, such as the National Patient Safety Foundation and the National Guideline Clearinghouse. The “In the Spotlight” column highlights hospitals that share their successes in implementing new safety standards within their organizations. The “Coders Corner” provides answers to coding concerns, such as the overuse of unspecified or nonspecific codes.

Evaluation

Our evaluation is designed to ascertain whether demonstration project hospitals that receive patient safety reports and intervention tools have a greater reduction of medical injuries over time, compared to those hospitals that do not receive the reports or tools. Further, the evaluation strives to determine whether the patient safety information and intervention tools are perceived as useful and, thus, are disseminated and used to reduce medical injuries. Evaluation data include extensive field notes by the intervention director during her encounters with hospitals and collection of data in structured surveys.

Preliminary results of hospital interventions

Based on feedback from hospitals to our intervention director, the perceived usefulness of the WMIRS patient safety reports has been positive, overall. Throughout the project, a number of issues have been identified that either foster or impede use of WMIRS data. The first advantage of these data is that they place no additional reporting burden on the hospital or health care providers and, therefore, personnel were more open to their use. Second, research project personnel reviewed the initial reports in detail with hospital personnel until they were comfortable with the interpretation of the data. Third, the data were analyzed with a sophisticated risk-adjustment model. This influenced the perceived quality of the report because, although some hospitals examine their own discharge data, they typically lack the expertise to sufficiently adjust them for risk. Based on preliminary data, being able to direct the report to a patient safety committee appears to facilitate report use. Additionally, community-based hospitals that are large enough to have a substantial number of injuries but are too small to have adequate patient safety and quality improvement resources in particular seem to place high value on the reports.

Of the factors that appear to impede the use of the reports, first is an excessive volume of information. Most quality improvement personnel voiced concern over the sheer number of performance reports passing across their desks. Some hospitals elected not to participate in the project because they felt “overwhelmed” with the number of data initiatives with which they were involved. Although most hospitals have fairly sophisticated and well-integrated data systems for fiscal outcomes, most do not have equally functional clinical information systems. Moreover, additional resources are often required to “drill down” data in the report; and the cost-driven reduction of quality improvement personnel in some hospitals prevents this from occurring. A second factor is the lag of approximately 1 year from the time the data are collected to the time they are released by the State, then analyzed and reported to hospitals by WMIRS. This presents challenges in linking hospital interventions with the injury report. Finally, quality of discharge data is heavily dependent on coding quality, thereby preventing strict comparisons among hospitals.

Rates of medical injuries and formal survey results by study groups are pending.

Discussion

Academic institutions may provide valuable resources for hospitals by reporting discharge data in a usable format while maintaining rigor, validity, and confidentiality. Data alone may be insufficient to drive hospital safety efforts. Combining reports of severity-adjusted medical injury data with complementary manuals, including evidence-based strategies and frameworks for guiding systematic quality improvement, may have synergistic results. Screening criteria applied to hospital discharge diagnoses can identify frequently occurring medical injuries that have substantial impact. They also ultimately may serve to target efforts aimed at identifying the causes and reducing the frequency of medical injury.

In 2001, JCAHO patient safety standards required hospital leadership to create a nonpunitive culture of safety with routinely conducted, organization-wide assessment of risk and regular reports to the hospital board. According to the IOM's Crossing the Quality Chasm, “Health care organizations [should] design and implement more effective organizational support processes to make change in the delivery of care possible.” ⁶ Furthermore, the IOM's To Err Is Human report recommended that “health care organizations and the professionals affiliated with them should make continually improved patient safety a declared and specific aim by establishing patient safety programs.” ²⁷

The WMIRS interdisciplinary intervention model aims to assist hospitals with performance feedback and educational tools to advance their patient safety programs. Our WMIRS research team comprises physicians and a nurse. Experience spans family and community medicine, pediatrics, emergency medicine, and surgery. Researchers have expertise in epidemiology, biostatistics, sociology, health services research, management, and public health. The wide variety of clinical experience, as well as familiarity with multiple research and intervention methods, allow a broad spectrum of injuries to be addressed using many different strategies. On the hospital side, key personnel involved with the project include quality improvement managers and directors, top-level administrators, and nursing and medical directors. Similar clinical and management education, experiences, and perspectives foster communication between the researchers and hospital personnel. However, many hospital personnel lack extensive scientific training and consequently may have difficulty understanding methods such as risk adjustment. The role of the intervention director includes facilitating such understanding.

Past research has identified many factors that influence perceived usefulness of performance reports. These include dissemination intensity, past experience with data, performance achieved, and organizational culture, ^{8, 28} as well as the characteristics of the data: quality, source credibility, report complexity, timeliness, appropriate unit of analysis, and ease of use. Similarly, the effectiveness of intervention tools is thought to be contingent upon such factors as quality and complexity of guidelines, characteristics of the health care professionals, and social norms. ^{29, 30} Observations made in our intervention director's field notes suggest that factors such as hospital size, information exclusivity, availability of human resources, and risk adjustment of the data may also play a role in perceived usefulness and, in turn, in the likelihood of dissemination and the translation of information into meaningful changes in patient safety.

This surveillance and reporting system using administrative patient discharge data avoids any additional reporting burden for health care providers and may be more comprehensive and feasible than systems relying on voluntary reports by health care professionals. We provide hospitals with important measures of medical injuries that are scientifically sound and robust. Coupling this performance feedback with evidence-based educational resources is a new model to promote medical injury prevention.

Limitations

Because we have analyzed field data only from a small number of hospitals, we have little objective information about the limitations of this approach that couples performance feedback with evidence-based educational resources to reduce medical injuries. Many of these limitations stem from the inability to test the assumptions underlying the interventions. We assumed that hospital personnel—

• Have knowledge deficits about medical injuries and preventive interventions and that the deficits can be remedied with our educational manuals.
• Would prefer sharp and concise materials that achieve a balance between easy-to-understand references and broad information.
• Are overwhelmed with data and may lack organizational systems to integrate medical injury information with other sources of data and tools for prioritizing improvements.
• Would act on evidence-based information provided directly to responsible individuals to influence complex processes of hospital care.

However, we are external change agents who are not privy to the politics and complexities of the hospital environment, and thus our ability to influence improvements is limited. The ideal alternative might be employing an internal change agent in each hospital, such as a quality improvement director who has established credibility and influence with a standing patient safety committee and medical staff leaders.

Conclusion

By demonstrating the effectiveness of coupling performance feedback with evidence-based education resources, we intend to advance patient safety from research to implementation. Our approach resonates with stakeholders, has the general support of participating hospitals, recognizes and addresses barriers to using information, and builds on existing competencies in data reporting. The results of the ongoing study may serve to guide construction of performance reports, educational manuals, and newsletters to maximize the usefulness of patient safety information to ultimately prevent and reduce medical injuries. The impact and value of the described medical injury prevention educational strategies will be evaluated further.

Acknowledgments

The Agency for Healthcare Research and Quality grant U18 HS11893 provided funding to the Medical College of Wisconsin for the Patient Safety Demonstration Project: Wisconsin Medical Injury Reporting System. 3M Health Information Systems provided a no-cost software license to the Medical College of Wisconsin Injury Research Center to use the All Patient Refined-Diagnosis Related Groups. The WMIRS Research Group includes surveillance specialists Clare E. Guse, M.S.; Hongyan Yang, M.S.; Prakash Laud, Ph.D.; and Evelyn M. Kuhn, Ph.D., who contribute to the epidemiological and statistical design, analysis, and reporting for the project; and support specialists Janice B. Babcock, M.A., and Michele Leininger.

References

1.

Brennan T, Leape L, Laird N. et al. Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324:370–6. [PubMed: 1987460]

2.

Leape LL, Brennan TA, Laird N. et al. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324:377–84. [PubMed: 1824793]

3.

Thomas EJ, Studdert DM, Newhouse JP. et al. Costs of medical injuries in Utah and Colorado. Inquiry. 1999;36(3):255–64. [PubMed: 10570659]

4.

Thomas EJ, Studdert DM, Burstin HR. et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care. 2000;38:261–71. [PubMed: 10718351]

5.

Making health care safer: a critical analysis of patient safety practices. Evidence Report/Technology Assessment Number 43. AHRQ Publication 01-E058. Rockville, MD: Agency for Healthcare Research and Quality; 2001 Jul. http://www​.ahrq.gov/clinic/ptsafety/. [PMC free article: PMC4781305] [PubMed: 11510252]

6.

Crossing the quality chasm: a new health system for the 21st century. A report of the Committee on Quality in Health Care in America. Institute of Medicine. Washington, DC: National Academy Press; 2001.

7.

Institute of Medicine. Patient safety: achieving a new standard of care. Washington, DC: National Academy Press; 2003.

8.

Ginsberg LS. Factors that influence line managers' perceptions of hospital performance data. Health Serv Res. 2003;38(1):261–86. [PMC free article: PMC1360884] [PubMed: 12650391]

9.

Christoffel T, Gallagher SS. Injury prevention: educational strategies. Injury prevention and public health: practical knowledge, skills, and strategies. Aspen, CO: Aspen Publishers; 1999. pp. 139–60.

10.

Layde PM, Cortes LM, Teret SP. et al. Patient safety efforts should focus on medical injuries. JAMA. 2002;287(15):1993–7. [PubMed: 11960544]

11.

Institute of Medicine. Health professions education: a bridge to quality. Washington, DC: National Academy Press; 2003.

12.

Mazmanian PE, Davis DA. Continuing medical education and the physician as learner: guide to the evidence. JAMA. 2002;288(9):1057–60. [PubMed: 12204068]

13.

Siddiqui AK, Ahmed S, Delbeau H. et al. Lack of physician concordance with guidelines on perioperative use of beta-blockers. Arch Intern Med. 2004;164:664–7. [PubMed: 15037496]

14.

Layde PM, Meurer LN, Guse C, et al. Medical injury identification using hospital discharge data. In: Henriksen K, Battles JB, Marks ES, Lewin DI, editors. Advances in patient safety: from research to implementation. Vol. 2, Concepts and methodology. AHRQ Publication No. 05-0021-2. Rockville, MD: Agency for Healthcare Research and Quality; 2005 Feb. [PubMed: 21249827]

15.

Institute for Clinical Systems Improvement (ICSI). Health care guideline: venous thromboembolism. Bloomington, MN: ICSI. 2002 Jan.

16.

Dunn S, Pretty L, Reid H, et al. Management of short term indwelling urethral catheters to prevent urinary tract infections. South Australia: Joanna Briggs Institute; 2000.

17.

Guideline for prevention of catheter-associated urinary tract infection. Atlanta, GA: Centers for Disease Control and Prevention; 2002 Nov 9. http://www​.cdc.gov/ncidod​/hip/guide/uritract.htm.

18.

McGee DC, Gould MK. Preventing complications of central venous catheterization. N Engl J Med. 2003;348:1123–33. [PubMed: 12646670]

19.

O'Grady NP, Alexander M. et al. Guidelines for the prevention of intravascular catheter-related infections. Morb Mortal Wkly Rep. 2002;51(RR10):1–26.

20.

Mermel LA, Farr BM. et al. Guidelines for the management of catheter-related infections. Clin Infect Dis. 2001;32:1249–72. [PubMed: 11303260]

21.

American College of Cardiology/American Heart Association Task Force on Practice Guidelines. ACC/AHA guideline update on perioperative cardiovascular evaluation for noncardiac surgery: a report of the Committee to Update the 1996 Guidelines on Perioperative Cardiovascular Evaluation for Noncardiac Surgery. Bethesda, MD: ACC/AHA; 1996 Mar 15 (revised 2002).

22.

Auerbach AD: Beta-blockers and reduction of perioperative cardiac events. In: Making health care safer: a critical analysis of patient safety practices. Evidence Report/Technology Assessment Number 43. Rockville, MD: Agency for Healthcare Research and Quality; 2001 Jul. [PMC free article: PMC4781305] [PubMed: 11510252]

23.

American Society of Health-system Pharmacists therapeutic guidelines on antimicrobial prophylaxis in surgery. Am J Health-System Pharmacist 1999;56: 1839–88. [PubMed: 10511234]

24.

Van den Berghe G, Wouters P, Weekers F. et al. Intensive insulin therapy in critically ill patients. N Engl J Med. 2001;245:1359–67. [PubMed: 11794168]

25.

Mangram AJ, Horan TC, Pearson ML. et al. Guideline for prevention of surgical site infection, 1999. Infect Control Hosp Epidemiol. 1999;20(4):247–78. [PubMed: 10219875]

26.

Byers JF, Beaudin CL. The relationship between continuous quality improvement and research. J Healthcare Quality. 2002 Jan/Feb;24(1):4–8. [PubMed: 11813537]

27.

Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000. [PubMed: 25077248]

28.

Jiang J, Ciccone K, Urlaub C. et al. Adapting the Healthcare Cost Utilization Project quality indicators for hospital use: the experience in New York State. Joint Commis J Qual Improve. 2001;27:200–15. [PubMed: 11293837]

29.

Cabana JD, Rand CS, Powe NR. et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;22(15):145–65. [PubMed: 10535437]

30.

Grimshaw JM, Shirran L, Thomas R. et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001 Aug;39(8 Suppl 2):II2–45. [PubMed: 11583120]