Abstract

Background: Diagnosis errors are frequent and important, but represent an underemphasized and understudied area of patient safety. Diagnosis errors are challenging to detect and dissect. It is often difficult to agree whether an error has occurred, and even harder to determine with certainty its causes and consequence. The authors applied four safety paradigms: (1) diagnosis as part of a system, (2) less reliance on human memory, (3) need to open “breathing space” to reflect and discuss, (4) multidisciplinary perspectives and collaboration. Methods: The authors reviewed literature on diagnosis errors and developed a taxonomy delineating stages in the diagnostic process: (1) access and presentation, (2) history taking/collection, (3) the physical exam, (4) testing, (5) assessment, (6) referral, and (7) followup. The taxonomy identifies where in the diagnostic process the failures occur. The authors used this approach to analyze diagnosis errors collected over a 3-year period of weekly case conferences and by a survey of physicians. Results: The authors summarize challenges encountered from their review of diagnosis error cases, presenting lessons learned using four prototypical cases. A recurring issue is the sorting-out of relationships among errors in the diagnostic process, delay and misdiagnosis, and adverse patient outcomes. To help understand these relationships, the authors present a model that identifies four key challenges in assessing potential diagnosis error cases: (1) uncertainties about diagnosis and findings, (2) the relationship between diagnosis failure and adverse outcomes, (3) challenges in reconstructing clinician assessment of the patient and clinician actions, and (4) global assessment of improvement opportunities. Conclusions and recommendations: Finally the authors catalogue a series of ideas for change. These include: reengineering followup of abnormal test results; standardizing protocols for reading x-rays/lab tests, particularly in training programs and after hours; identifying “red flag” and “don't miss” diagnoses and situations and use of manual and automated check-lists; engaging patients on multiple levels to become “coproducers” of safer medical diagnosis practices; and weaving “safety nets” to mitigate harm from uncertainties and errors in diagnosis. These change ideas need to be tested and implemented for more timely and error-free diagnoses.

Introduction

Diagnosis errors are frequent and important, but represent an underemphasized and understudied area of patient-safety. ^{1 – 8} This belief led us to embark on a 3-year project, funded by the Agency for Healthcare Research and Quality (AHRQ), to better understand where and how diagnosis fails and explore ways to target interventions that might prevent such failures. It is known that diagnosis errors are common and underemphasized, but they are also challenging to detect and dissect. It is often difficult even to agree whether or not a diagnosis error has occurred.

In this article we describe how we have applied patient safety paradigms (blame-free reporting/reviewing/learning, attention to process and systems, an emphasis on communication and information technology) to better understand diagnosis error. ^{2, 7 – 9}

We review evidence about the types and importance of diagnosis errors and summarize challenges we have encountered in our review of more than 300 cases of diagnosis error. In the second half of the article, we present lessons learned through analysis of four prototypical cases. We conclude with suggested “change ideas”—interventions for improvement, testing, and future research.

Although much of the patient safety spotlight has focused on medication errors, two recent studies of malpractice claims revealed that diagnosis errors far outnumber medication errors as a cause of claims lodged (26 percent versus 12 percent in one study; ¹⁰ 32 percent versus 8 percent in another study ¹¹ ). A Harris poll commissioned by the National Patient Safety Foundation found that one in six people had personally experienced a medical error related to misdiagnosis. ¹² Most medical error studies find that 10–30 percent (range = 0.6–56.8 percent) of errors are errors in diagnosis (Table 1). ¹ – ^{3, 5, 11, 13 – 21} A recent review of 53 autopsy studies found an average rate of 23.5 percent major missed diagnoses (range = 4.1–49.8 percent). Selected disease-specific studies (Table 2), ^{6, 22 – 32} also show that substantial percentages of patients (range = 2.1 – 61 percent) experienced missed or delayed diagnoses. Thus, while these studies view the problem from varying vantage points using heterogeneous methodologies (some nonsystematic and lacking in standardized definitions), what emerges is compelling evidence for the frequency and impact of diagnosis error and delay.

Table 1

General medical error studies that reported errors in diagnosis.

Table 2

Illustrative disease-specific studies of diagnosis errors.

Of the 93 safety projects funded by AHRQ, only 1 is focused on diagnosis error, and none of the 20 evidence-based AHRQ Patient Safety Indicators directly measures failure to diagnose. ³³ Nonetheless, for each of AHRQ's 26 “sentinel complications” (e.g., decubitus ulcer, iatrogenic pneumothorax, postoperative septicemia, accidental puncture/laceration), timely diagnosis can be decisive in determining whether patients experience major adverse outcomes. Hence, while diagnosis error remains more in the shadows than in the spotlight of patient safety, this aspect of clinical medicine is clearly vulnerable to well-documented failures and warrants an examination through the lens of modern patient safety and quality improvement principles.

Traditional and innovative approaches to learning from diagnosis error

Traditionally, studying missed diagnoses or incorrect diagnoses had a central role in medical education, research, and quality assurance in the form of autopsies. ^{34 – 36} Other traditional methods of learning about misdiagnosed cases include malpractice litigation, morbidity and mortality (M&M) conferences, unsystematic feedback from patients, other providers, or simply from patients' illnesses as they evolved over time. ^{3, 14, 15, 32} Beyond the negative aspects of being grounded in patients' deaths or malpractice accusations, there are other limitations of these historical approaches, including

• Lack of systematic approaches to surveillance, reporting, and learning from errors, with nonrandom sample of cases subjected to such review ^{37, 39}
• Lack of timeliness, with cases often reviewed months or years after the event ³⁸
• Examinations that rarely dig to the root of problems: not focused on the “Five Whys” ⁴⁰
• Postmortems that seldom go beyond the case-at-hand, with minimal linkages to formal quality improvement activities ⁴¹
• Atrophy of the value of even these suboptimal approaches, with autopsy rates in the single digits (in many hospitals, zero), many malpractice experiences sealed by nondisclosure agreements, and shorter hospitalizations limiting opportunities for followup to ultimate diagnosis ^{34, 41, 42}

What is needed to overcome these limitations is not only a more systematic method for examining cases of diagnosis failure, but also a fresh approach. Therefore, our team approached diagnosis error with the following perspectives:

Diagnosis as part of a system. Diagnostic accuracy should be viewed as a system property rather than simply what happens between the doctor's two ears. ^{2, 43 – 45} While cognitive issues figure heavily in the diagnostic process, a quite from Don Berwick summarizes ⁴⁶ a much lacking and needed perspective: “Genius diagnosticians make great stories, but they don't make great health care. The idea is to make accuracy reliable, not heroic.”

Less reliance on human memory. Relying on clinicians' memory—to trigger consideration of a particular diagnosis, recall a disease's signs/symptoms/pattern from a textbook or experience—or simply to remember to check on a patient's lab result—is an invitation to variations and failures. This lesson from other error research resonates powerfully with clinicians, who are losing the battle to keep up to date. ^{9, 45, 47}

Need for “space” to allow open reflection and discussion. Transforming an adversarial atmosphere into one conducive to honest reflection is an essential first step. ^{48, 49} However, an equally important and difficult challenge is creating venues that allow clinicians (and patients) to discuss concerns in an efficient and productive manner. ³⁷ Cases need to be reviewed in sufficient detail to make them “real.” Firsthand clinical information often radically changes our understanding from what the more superficial “first story” suggested. As complex clinical circumstances are better understood, new light is often shed on what at first appeared to be indefensible diagnostic decisions and actions. Unsuspected additional errors also emerge. Equally important is not to get mired in details or making judgments (whether to label a case as a diagnosis error). Instead, it is more valuable to focus on generalizable lessons of how to ensure better treatment of similar future patients. ¹⁶

Adopting multidisciplinary perspectives and collaboration. A broad range of skills and vantage points are valuable in understanding the complex diagnostic problems that we encountered. We considered input from specialists and primary care physicians to be essential. In addition, specialists in emergency medicine (where many patients first present) offered a vital perspective, both for their diagnostic expertise and their pivotal interface with system constraints (resource limits mean that not every patient with a confusing diagnosis can be hospitalized). Even more valuable has been the role of non-MDs, including nursing quality specialists, information scientists, and social scientists (cognitive psychologist, decision theory specialist) in forging a team to broadly examine diagnosis errors.

Innovative screening approaches. Developing new ways to uncover errors is a priority. We cannot afford to wait for a death, lawsuit, or manual review. Approaches we have been exploring include electronic screening that links pharmacy and lab data (e.g., to screen for abnormal results, such as elevated thyroid stimulating hormone [ TSH], unaddressed by thyroxin therapy), trajectory studies (retrospectively probing delays in a series of cases with a particular diagnosis), and screening for discrepancies between admitting and discharge diagnoses. A related approach is to survey specialists (who are poised to see diagnoses missed in referred patients), primary care physicians (about their own missed diagnoses), or patients themselves (who frequently have stories to share about incorrect diagnoses), in addition to various ad hoc queries and self-reports.

Where does the diagnostic process fail?

One of the most powerful heuristics in medication safety has been delineation of the steps in the medication-use process (prescribing, transcribing, dispensing, administering, and monitoring) to help localize where an error has occurred. Diagnosis, while more difficult to neatly classify (because compared to medications, stages are more concurrent, recurrent, and complex), nonetheless can be divided into seven stages: (1) access/presentation, (2) history taking/collection, (3) the physical exam, (4) testing, (5) assessment, (6) referral, and (7) followup. We have found this framework helpful for organizing discussions, aggregating cases, and targeting areas for improvement and research. It identifies what went wrong, and situates where in the diagnostic process the failure occurred (Table 3). We have used it for a preliminary analysis of several hundred diagnosis error cases we collected by surveying physicians.

Table 3

Taxonomy of where and what errors occurred.

This taxonomy for categorizing diagnostic “assessment” draws on work of Kassirer and others, ⁵³ highlighting the two key steps of (a) hypothesis generation, and (b) differential diagnosis or hypothesis weighing/prioritization. We add another aspect of diagnostic assessment, one that connects to other medical and iatrogenic error work—the need to recognize the urgency of diagnoses and complications. This addition underscores the fact that failure to make the exact diagnosis is often less important than correctly assessing the urgency of the patient's illness. We divide the “testing” stage into three components—ordering, performing, and clinician processing (similar but not identical to the laboratory literature classification of the phases of lab testing as preanalytic, analytic, and

postanalytic). ^{50, 51} For each broad category, we specified the types of problems we observed.

A recurring theme running through our reviews of potential diagnosis error cases pertains to the relationship between errors in the diagnostic process, delay and misdiagnosis, and adverse patient outcomes. Bates ⁵² has promulgated a useful model for depicting the relationships between medication errors and outcomes. Similarly, we find that most errors in the diagnostic process do not adversely impact patient outcomes. And, many adverse outcomes associated with misdiagnosis or delay do not necessarily result from any error in the diagnostic process—the cancer may simply be undiagnosable at that stage, the illness presentation too atypical, rare, or unlikely even for the best of clinicians to diagnose early. These situations are often referred to as “no fault” or “forgivable” errors—terms best avoided because they imply fault or blame for preventable errors (Figure 1). ^{7, 53}

Figure 1

Relationships between diagnostic process errors, misdiagnosis, and adverse events.

While deceptively simple, the model raises a series of extremely challenging questions—questions we found ourselves repeatedly returning to in our weekly discussions. We hope these questions can provide insights into recurring themes and challenges we faced, and perhaps even serve as a checklist for others to structure their own patient care reviews. While humbled by our own inability to provide more conclusive answers to these questions, we believe researchers and practitioners will be forced to grapple with them before we can make significant progress.

Diagnosis error case vignettes

Lessons and issues raised by the diagnosis error cases

Conclusion

Because of their complexity, there are no quick fixes for diagnosis errors. As we review what we learned from a variety of approaches and cases, certain areas stood out as ripe for improvement—both small-scale improvements that can be tested locally, as well as larger improvements that need more rigorous, formal research. Table 4 summarizes these change ideas, which harvest the lessons of our 3-year project.

Table 4

Change ideas for preventing and minimizing diagnostic error.

As outlined in the table, there needs to be a commitment to build learning organizations, in which feedback to earlier providers who may have failed to make a correct diagnosis becomes routine, so that institutions can learn from this aggregated feedback data. To better protect patients, we will need to conceptualize and construct safety nets to mitigate harm from uncertainties and errors in diagnosis. The followup of abnormal test results is a prime candidate for reengineering, to ensure low “defect” rates that are comparable to those achieved in other fields. More standardized and reliable protocols for reading x-rays and laboratory tests (such as pathology specimens), particularly in residency training programs and “after hours,” could minimize the errors we observed. In addition, we need to better delineate “red flag” and “don't miss” diagnoses and situations, based on better understanding and data regarding pitfalls in diagnosis and ways to avoid them.

To achieve many of these advances, automated (and manual) checklists and reminders will be needed to overcome current reliance on human memory. But information technology must also be deployed and reengineered to overcome growing problems associated with information overload. Finally, and most importantly, patients will have to be engaged on multiple levels to become “coproducers” in a safer practice of medical diagnosis. ⁸⁵ It is our hope that these change ideas can be tested and implemented to ensure safer treatment based on better diagnoses—diagnosis with fewer delays, mistakes, and process errors.

Acknowledgments

This work was supported by AHRQ Patient Safety Grant #11552, the Cook County-Rush Developmental Center for Research in Patient Safety (DCERPS) Diagnostic Error Evaluation and Research (DEER) Project.

References

1.

Baker GR, Norton P. Patient safety and healthcare error in the Canadian healthcare system. Ottawa, Canada: Health Canada; 2002. pp. 1–167. (http://www​.hc-sc.gc.ca​/english/care/report. Accessed 12/24/04.)

2.

Leape L, Brennan T, Laird N. et al. The nature of adverse events in hospitalized patients: Results of the Harvard medical practice study II. NEJM. 1991;324:377–84. [PubMed: 1824793]

3.

Medical Malpractice Lawyers and Attorneys Online. Failure to diagnose. http://www​.medical-malpractice-attorneys-lawsuits​.com/pages/failure-to-diagnose​.html. 2004.

4.

Ramsay A. Errors in histopathology reporting: detection and avoidance. Histopathology. 1999;34:481–90. [PubMed: 10383691]

5.

JCAHO Sentinel event alert advisory group. Sentinel event alert. http://www​.jcaho.org​/about+us/news+letters​/sentinel+event+alert/sea_26.htm. Jun 17, 2002. [PubMed: 12092445]

6.

Edelman D. Outpatient diagnostic errors: unrecognized hyperglycemia. Eff Clin Pract. 2002;5:11–6. [PubMed: 11874191]

7.

Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what's the goal? Acad Med. 2002;77:981–92. [PubMed: 12377672]

8.

Kuhn G. Diagnostic errors. Acad Emerg Med. 2002;9:740–50. [PubMed: 12093717]

9.

Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000. [PubMed: 25077248]

10.

Sato L. Evidence-based patient safety and risk management technology. J Qual Improv. 2001;27:435.

11.

Phillips R, Bartholomew L, Dovey S. et al. Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care. 2004;13:121–6. [PMC free article: PMC1743812] [PubMed: 15069219]

12.

Golodner L. How the public perceives patient safety. Newsletter of the National Patient Safety Foundation. 2004;1997:1–6.

13.

Bhasale A, Miller G, Reid S. et al. Analyzing potential harm in Australian general practice: an incident-monitoring study. Med J Aust. 1998;169:73–9. [PubMed: 9700340]

14.

Kravitz R, Rolph J, McGuigan K. Malpractice claims data as a quality improvement tool, I: Epidemiology of error in four specialties. JAMA. 1991;266:2087–92. [PubMed: 1920696]

15.

Flannery F. Utilizing malpractice claims data for quality improvement. Legal Medicine. 1992;92:1–3.

16.

Neale G, Woloshynowych M, Vincent C. Exploring the causes of adverse events in NHS hospital practice. J Royal Soc Med. 2001;94:322–30. [PMC free article: PMC1281594] [PubMed: 11418700]

17.

Bogner MS, editor. Human error in medicine. Hillsdale, NJ: Lawrence Erlbaum Assoc.; 1994.

18.

Makeham M, Dovey S, County M. et al. An international taxonomy for errors in general practice: a pilot study. Med J Aust. 2002;177:68–72. [PubMed: 12098341]

19.

Wilson R, Harrison B, Gibberd R. et al. An analysis of the causes of adverse events from the quality in Australian health care study. Med J Aust. 1999;170:411–5. [PubMed: 10341771]

20.

Weingart S, Ship A, Aronson M. Confidential clinician-reported surveillance of adverse events among medical inpatients. J Gen Intern Med. 2000;15:470–7. [PMC free article: PMC1495482] [PubMed: 10940133]

21.

Chaudhry S, Olofinboda K, Krumholz H. Detection of errors by attending physicians on a general medicine service. J Gen Intern Med. 2003;18:595–600. [PMC free article: PMC1494901] [PubMed: 12911640]

22.

Kowalski R, Claassen J, Kreiter K. et al. Initial misdiagnosis and outcome after subarachnoid hemorrhage. JAMA. 2004;291:866–9. [PubMed: 14970066]

23.

Mayer P, Awad I, Todor R. et al. Misdiagnosis of symptomatic cerebral aneurysm. Prevalence and correlation with outcome at four institutions. Stroke. 1996;27:1558–63. [PubMed: 8784130]

24.

Craven ER. Risk management issues in glaucoma diagnosis and treatment. Surv Opthalmol. 1996 May-Jun;40(6):459–62. [PubMed: 8724638]

25.

Lederle F, Parenti C, Chute E. Ruptured abdominal aortic aneurysm: the internist as diagnostician. Am J Med. 1994;96:163–7. [PubMed: 8109601]

26.

Goodson W, Moore D. Causes of physician delay in the diagnosis of breast cancer. Arch Intern Med. 2002;162:1343–8. [PubMed: 12076232]

27.

Clark S. Spinal infections go undetected. Lancet. 1998;351:1108.

28.

Williams VJ. Second-opinion consultations assist community physicians in diagnosing brain and spinal cord biopsies. http://www3​.mdanderson​.org/~oncolog/brunder.html. Oncol 1997. 11-29-0001.

29.

Arbiser Z, Folpe A, Weiss S. Consultative (expert) second opinions in soft tissue pathology. Amer J Clin Pathol. 2001;116:473–6. [PubMed: 11601130]

30.

Pope J, Aufderheide T, Ruthazer R. et al. Missed diagnoses of acute cardiac ischemia in the emergency department. NEJM. 2000;342:1163–70. [PubMed: 10770981]

31.

Steere A, Taylor E, McHugh G. et al. The over-diagnosis of lyme disease. JAMA. 1993;269:1812–6. [PubMed: 8459513]

32.

American Society for Gastrointestinal Endoscopy. Medical malpractice claims and risk management in gastroenterology and gastrointestinal endoscopy. http://www​.asge.org/gui​/resources/manual​/gea_risk_update_on_endoscopic.asp, Jan 8, 2002.

33.

Zhan C, Miller MR. Administrative data based patient safety research: a critical review. Qual Saf Health Care. 2003;12(Suppl 2):ii58–ii63. [PMC free article: PMC1765777] [PubMed: 14645897]

34.

Shojania K, Burton E, McDonald K. et al. Changes in rates of autopsy-detected diagnostic errors over time: a systematic review. JAMA. 2003;289:2849–56. [PubMed: 12783916]

35.

Gruver R, Fries E. A study of diagnostic errors. Ann Intern Med. 1957;47:108–20. [PubMed: 13435683]

36.

Kirch W, Schafii C. Misdiagnosis at a university hospital in 4 medical eras. Medicine. 1996;75:29–40. [PubMed: 8569468]

37.

Thomas E, Petersen L. Measuring errors and adverse events in health care. J Gen Intern Med. 2003;18:61–7. [PMC free article: PMC1494808] [PubMed: 12534766]

38.

Orlander J, Barber T, Fincke B. The morbidity and mortality conference: the delicate nature of learning from error. Acad Med. 2002;77:1001–6. [PubMed: 12377674]

39.

Bagian J, Gosbee J, Lee C. et al. The Veterans Affairs root cause analysis system in action. J Qual Improv. 2002;28:531–45. [PubMed: 12369156]

40.

Spear S, Bowen H. Decoding the DNA of the Toyota production system. Harvard Bus Rev 1999;106.

41.

Anderson R, Hill R. The current status of the autopsy in academic medical centers in the United States. Am J Clin Pathol. 1989;92:S31–S37. [PubMed: 2801621]

42.

The Royal College of Pathologists of Australasia Autopsy Working Party. The decline of the hospital autopsy: a safety and quality issue for health care in Australia. Med J Aust. 2004;180:281–5. [PubMed: 15012566]

43.

Plsek P. Section 1: evidence-based quality improvement, principles, and perspectives. Pediatrics. 1999;103:203–14. [PubMed: 9917464]

44.

Leape L. Error in medicine. JAMA. 1994;272:1851–7. [PubMed: 7503827]

45.

Reason J. Human error. New York, NY: Cambridge University Press; 1990.

46.

Gaither C. What your doctor doesn't know could kill you. The Boston Globe. Jul 14, 2002.

47.

Lambert B, Chang K, Lin S. Effect of orthographic and phonological similarity on false recognition of drug names. Soc Sci Med. 2001;52:1843–57. [PubMed: 11352410]

48.

Woloshynowych M, Neale G, Vincent C. Care record review of adverse events: a new approach. Qual Saf Health Care. 2003;12:411–5. [PMC free article: PMC1758034] [PubMed: 14645755]

49.

Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;78:775–80. [PubMed: 12915363]

50.

Pansini N, Di Serio F, Tampoia M. Total testing process: appropriateness in laboratory medicine. Clin Chim Acta. 2003;333:141–5. [PubMed: 12849897]

51.

Stroobants A, Goldschmidt H, Plebani M. Error budget calculations in laboratory medicine: linking the concepts of biological variation and allowable medical errors. Clin Chim Acta. 2003;333:169–76. [PubMed: 12849900]

52.

Bates D. Medication errors. How common are they and what can be done to prevent them? Drug Saf. 1996;15:303–10. [PubMed: 8941492]

53.

Kassirer J, Kopelman R. Learning clinical research. Baltimore, MD: Williams & Wilkins; 1991.

54.

Pisarska M, Carson S, Buster J. Ectopic pregnancy. Lancet. 1998;351:1115–20. [PubMed: 9660597]

55.

Soderstrom, RM. Obstetrics/gynecology: ectopic pregnancy remains a malpractice dilemma. http://www​.thedoctors​.com/risk/specialty/obgyn/J4209.asp. 1999. Apr 26, 2004.

56.

Tay J, Moore J, Walker J. Ectopic pregnancy. BMJ. 2000;320:916–9. [PMC free article: PMC1117838] [PubMed: 10742003]

57.

Gracia C, Barnhart K. Diagnosing ectopic pregnancy: decision analysis comparing six strategies. Obstet Gynecol. 2001;97:464–70. [PubMed: 11239658]

58.

Helliker K, Burton TM. Medical ignorance contributes to toll from aortic illness. Wall Street Journal; Nov 4, 2003.

59.

Hagan P, Nienaber C, Isselbacher E. et al. The international registry of acute aortic dissection (IRAD): new insights into an old disease. JAMA. 2000;283:897–903. [PubMed: 10685714]

60.

Kodolitsch Y, Schwartz A, Nienaber C. Clinical prediction of acute aortic dissection. Arch Intern Med. 2000;160:2977–82. [PubMed: 11041906]

61.

Spittell P, Spittell J, Joyce J. et al. Clinical features and differential diagnosis of aortic dissection: experience with 236 cases (1980 through 1990). Mayo Clin Proc. 1993;68:642–51. [PubMed: 8350637]

62.

Loubeyre P, Angelie E, Grozel F. et al. Spiral CT artifact that simulates aortic dissection: image reconstruction with use of 180 degrees and 360 degrees linear-interpolation algorithms. Radiology. 1997;205:153–7. [PubMed: 9314977]

63.

Cabot R. Diagnostic pitfalls identified during a study of 3000 autopsies. JAMA. 1912;59:2295–8.

64.

Conveney E, Geraghty J, O'Laoide R. et al. Reasons underlying negative mammography in patients with palpable breast cancer. Clin Radiol. 1994;49:123–5. [PubMed: 8124890]

65.

Burstein, HJ. Highlights of the 2nd European breast cancer conference. http://www​.medscape.com​/viewarticle/408459. 2000.

66.

Berlin L. Malpractice issues in radiology: defending the “missed” radiographic diagnosis. Amer J Roent. 2001;176:317–22. [PubMed: 11159064]

67.

Cigarroa J, Isselbacher E, DeSanctis R. et al. Diagnostic imaging in the evaluation of suspected aortic dissection—old standards and new directions. NEJM. 1993;328:35–43. [PubMed: 8416269]

68.

McCarthy B, Yood M, Boohaker E. et al. Inadequate follow-up of abnormal mammograms. Am J Prev Med. 1996;12:282–8. [PubMed: 8874693]

69.

McCarthy B, Yood M, Janz N. et al. Evaluation of factors potentially associated with inadequate follow-up of mammographic abnormalities. Cancer. 1996;77:2070–6. [PubMed: 8640672]

70.

Boohaker E, Ward R, Uman J. et al. Patient notification and follow-up of abnormal test results: a physician survey. Arch Intern Med. 1996;156:327–31. [PubMed: 8572844]

71.

Murff H, Gandhi T, Karson A. et al. Primary care physician attitudes concerning follow-up of abnormal test results and ambulatory decision support systems. Int J Med Inf. 2003;71:137–49. [PubMed: 14519406]

72.

Iordache S, Orso D, Zelingher J. A comprehensive computerized critical laboratory results alerting system for ambulatory and hospitalized patients. Medinfo. 2001;10:469–73. [PubMed: 11604784]

73.

Kuperman G, Boyle D, Jha A. et al. How promptly are inpatients treated for critical laboratory results? JAMIA. 1998;5:112–9. [PMC free article: PMC61280] [PubMed: 9452990]

74.

Schiff G, Kim S, Wisniewski M. et al. Every system is perfectly designed to: missed diagnosis of hypothyroidism uncovered by linking lab and pharmacy data. J Gen Intern Med. 2003;18(suppl 1):295.

75.

Kuperman G, Teich J, Tanasijevic M. et al. Improving response to critical laboratory results with automation: results of a randomized controlled trial. JAMIA. 1999;6:512–22. [PMC free article: PMC61393] [PubMed: 10579608]

76.

Poon E, Kuperman G, Fiskio J. et al. Real-time notification of laboratory data requested by users through alphanumeric pagers. J Am Med Inform Assoc. 2002;9:217–22. [PMC free article: PMC344581] [PubMed: 11971882]

77.

Schiff GD, Klass D, Peterson J. et al. Linking laboratory and pharmacy: opportunities for reducing errors and improving care. Arch Intern Med. 2003;163:893–900. [PubMed: 12719197]

78.

Massachusetts Coalition for the Prevention of Medical Error. Communicating critical test results safe practice. JCAHO J Qual Saf 2005;31(2) [In press] [PubMed: 15791766]

79.

Berwick D. Eleven worthy aims for clinical leadership of health system reform. JAMA. 1994;272:797–802. [PubMed: 8078145]

80.

Dawson N, Arkes H. Systematic errors in medical decision making: judgment limitations. J Gen Intern Med. 1987;2:183–7. [PubMed: 3295150]

81.

Bartlett E. Physicians' cognitive errors and their liability consequences. J Health Care Risk Manag. 1998;18:62–9. [PubMed: 10537844]

82.

McDonald J. Computer reminders, the quality of care and the nonperfectability of man. NEJM. 1976;295:1351–5. [PubMed: 988482]

83.

Smetzer J, Cohen M. Lesson from the Denver medication error/criminal negligence case: look beyond blaming individuals. Hosp Pharm. 1998;33:640–57.

84.

Fischhoff B. Hindsight [does not equal] foresight: the effect of outcome knowledge on judgment under uncertainty. J Exp Psychol: Hum Perc Perform. 1975;1:288–99. [PMC free article: PMC1743746] [PubMed: 12897366]

85.

Hart JT. Cochrane Lecture 1997. What evidence do we need for evidence based medicine? J Epidemiol Community Health. 1997 Dec;51(6):623–9. [PMC free article: PMC1060558] [PubMed: 9519124]