1. Introduction

The purpose of this document is to serve as a guide for the design and use of patient registries for scientific, clinical, and health policy purposes. Properly designed and executed, patient registries can provide a real-world view of clinical practice, patient outcomes, safety, and comparative effectiveness. This user's guide primarily focuses on practical design and operational issues, evaluation principles, and best practices. Where topics are well covered in other materials, references and/or links are provided. The goal of this document is to provide stakeholders in both the public and private sectors with information they can use to guide the design and implementation of patient registries, the analysis and interpretation of data from patient registries, and the evaluation of the quality of a registry or one of its components. Where useful, case examples have been incorporated to illustrate particular points or challenges.

The term registry¹ is defined both as the act of recording or registering and as the record or entry itself. Therefore, “registries” can refer to both programs that collect and store data and the records that are so created.

The term patient registry is generally used to distinguish registries focused on health information from other record sets, but there is no consistent definition in current use. E.M. Brooke, in a 1974 publication of the World Health Organization, further delineated registries in health information systems as “a file of documents containing uniform information about individual persons, collected in a systematic and comprehensive way, in order to serve a predetermined purpose.” ²

The National Committee on Vital and Health Statistics³ describes registries used for a broad range of purposes in public health and medicine as “an organized system for the collection, storage, retrieval, analysis, and dissemination of information on individual persons who have either a particular disease, a condition (e.g., a risk factor) that predisposes [them] to the occurrence of a health-related event, or prior exposure to substances (or circumstances) known or suspected to cause adverse health effects.”

Other terms also used to refer to patient registries include clinical registries, clinical data registries, disease registries, and outcomes registries.^{4, 5}

This user's guide focuses on patient registries that are used for evaluating patient outcomes. It is not intended to address several other types of or uses for registries (although many of the principles may be applicable), such as geographically based population registries (not based on a disease, condition, or exposure); registries created for public health reporting without tracking outcomes (e.g., vaccine registries); or listing registries that are used solely to identify patients with particular diseases in clinical practices but are not used for evaluating outcomes. This user's guide is also not intended to address the wide range of studies that use secondary analyses of data collected for other purposes.

Many of these other types of registries are included in the Registry of Patient Registries (RoPR) effort. RoPR is a central listing of patient registries established in 2012 by the Agency for Healthcare Research and Quality (AHRQ) in collaboration with the National Library of Medicine.⁶ It is designed to improve transparency and reduce redundancy in registry-based research. Inclusion of all types of patient registries is important to achieve RoPR's goals, and the system therefore defines patient registries broadly.

In contrast to RoPR, this user's guide focuses on the subset of patient registries used for evaluating patient outcomes, defined as follows:

• A patient registry is an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes.
• The patient registry database describes a file (or files) derived from the registry.

Based on these definitions, the user's guide focuses on patient registries in which the following are true (although exceptions may apply):

• The data are collected in a naturalistic manner, such that the management of patients is determined by the caregiver and patient together and not by the registry protocol.
• The registry is designed to fulfill specific purposes, and these purposes are defined before collecting and analyzing the data. In other words, the data collection is purpose driven rather than the purpose being data driven (meaning limited to or derived from what is already available in an existing data set).
• The registry captures data elements with specific and consistent data definitions.
• The data are collected in a uniform manner for every patient. This consideration refers to both the types of data and the frequency of their collection.
• The data collected include data derived from and reflective of the clinical status of the patient (e.g., history, examination, laboratory test, or patient-reported data). Registries include the types of data that clinicians would use for the diagnosis and management of patients.
• At least one element of registry data collection is active, meaning that some data are collected specifically for the purpose of the registry (usually collected from the patient or clinician) rather than inferred from sources that are collected for another purpose (administrative, billing, pharmacy databases, etc.). This definition does not exclude situations where registry data collection is a specific, but not the exclusive, reason data are being collected, such as might be envisioned with future uses of electronic health records, as described in Chapter 15. This definition also does not exclude the incorporation of other data sources. Registries can be enriched by linkage with extant databases (e.g., to determine deaths and other outcomes or to assess pharmacy use or resource utilization), as discussed in Chapter 6.

Data from patient registries are generally used for studies that address the purpose for which the registry was created. In some respects, such as the collection of detailed clinical and longitudinal followup data, studies derived from the patient registries described in this user's guide resemble traditional observational cohort studies. Beyond traditional cohort studies, however, some registry-based studies may be more flexible in that the scope and focus of the data collection activity of the registry may be adapted over time to address additional needs. For example, new studies, such as cluster-randomized studies or case-control studies, may be nested within an ongoing registry, and the database derived from the registry may be used to support secondary studies, such as studies that link the registry database with other data sources to explore new questions.

2. Current Uses for Patient Registries

A patient registry can be a powerful tool to observe the course of disease; to understand variations in treatment and outcomes; to examine factors that influence prognosis and quality of life; to describe care patterns, including appropriateness of care and disparities in the delivery of care; to assess effectiveness; to monitor safety and harm; and to measure quality of care. Through functionalities such as feedback of data, registries are also being used to study quality improvement.⁷

Different stakeholders perceive and may benefit from the value of registries in different ways. For example, for a clinician, registries can collect data about disease presentation and outcomes on large numbers of patients rapidly, thereby producing a real-world picture of disease, current treatment practices, and outcomes. For a physician organization, a registry might provide data that can be used to assess the degree to which clinicians are managing a disease in accordance with evidence-based guidelines, to focus attention on specific aspects of a particular disease that might otherwise be overlooked, or to provide data for clinicians to compare themselves with their peers.⁸ For patients and patient advocacy organizations, a registry may increase understanding of the natural history of a disease, contribute to the development of treatment guidelines, or facilitate research on treatment.^{9, 10} From a payer's perspective, registries can provide detailed information from large numbers of patients on how procedures, devices, or pharmaceuticals are actually used and on their effectiveness in different populations. This information may be useful for determining coverage policies.¹¹ For a drug or device manufacturer, a registry-based study might demonstrate the performance of a product in the real world, meet a postmarketing commitment or requirement,¹² develop hypotheses, or identify patient populations that will be useful for product development, clinical trials design, and patient recruitment. The U.S. Food and Drug Administration (FDA) has noted that “through the creation of registries, a sponsor can evaluate safety signals identified from spontaneous case reports, literature reports, or other sources, and evaluate the factors that affect the risk of adverse outcomes such as dose, timing of exposure, or patient characteristics.”¹³

The use of patient registries varies by priority condition, with cancer and cardiovascular disease having a large number of registries and areas such as developmental delays or dementia, far fewer. Overall, the use of patient registries appears to be active and growing. For example, a review of ClinicalTrials.gov in the area of cancer reveals over 270 large (more than 2,000 patients) observational studies that would meet the criteria for a patient registry. Of these studies, 4 have more than 100,000 patients, and 27 have more than 10,000. In some cases, the drivers for these registries have been Federal stakeholders. For example, since 2005, the FDA Center for Devices and Radiological Health has called for some 160 postapproval studies, many of which use new or existing registries to study the real-world effectiveness of specific devices in community practice.¹⁴ The establishment of RoPR provides a new resource for tracking registry development and use by condition, purpose, type, and multiple other factors.⁶

3. Taxonomy for Patient Registries

Even limited to the definitions described above, the breadth of studies that might be included as patient registries is large. Patients in a registry are typically selected based on a particular disease, condition (e.g., a risk factor), or exposure. This user's guide uses these common selection criteria to develop a taxonomy or classification based on how the populations for registries are defined. Three general categories with multiple subcategories and combinations account for the majority of registries that are developed for evaluating patient outcomes. These categories include observational studies in which the patient has had an exposure to a product or service, or has a particular disease or condition, or various combinations thereof.

4. Patient Registries and Policy Purposes

In addition to the growth of patient registries for scientific and clinical purposes, registries are receiving increased attention for their potential role in policymaking or decisionmaking.⁵⁶ As stated earlier, registries may offer a view of real-world health care that is typically inaccessible from clinical trials or other data sources and may provide information on the generalizability of the data from clinical trials to populations not studied in those trials.

The utility of registry data for decisionmaking is related to three factors: the stakeholders, the primary scientific question, and the context. The stakeholders are those associated with the disease or procedure that may be affected from a patient, provider, payer, regulator, or other perspective. The primary scientific question for a registry may relate to effectiveness, safety, or practice patterns. The context includes the scientific context (e.g., previous randomized trials and modeling efforts that help to more precisely define the primary scientific question), as well as the political, regulatory, funding, and other issues that provide the practical parameters around which the registry is developed. In identifying the value of information from registries, it is essential to look at the data with specific reference to the purpose and focus of the registry.

From a policy perspective, there are several scenarios in which the decision to develop a registry may arise. One possible scenario is as follows. An item or service is considered for use. Stakeholders in the decision collaboratively define “adequate data in support of the decision at hand.” Here, “adequate data” refers to information of sufficient relevance and quality to permit an informed decision. An evidence development strategy is selected from one of many potential strategies (RCT, practical clinical trial, registry, etc.) based on the quality of the evidence provided by each design, as well as the burden of data collection and the cost that is imposed. This tradeoff of the quality of evidence versus cost of data collection for each possible design is termed the “value of information” exercise (Figure 1–1). Registries should be preferred in those circumstances where they provide sufficiently high-quality information for decisionmaking at a sufficiently low cost (relative to other “acceptable” designs).

Figure 1–1

Deciding when to develop a registry: The “value of information” exercise.

One set of policy determinations that may be informed by a patient registry centers on the area of payment for items or services. For example, the Centers for Medicare & Medicaid Services (CMS) issued Guidance on National Coverage Determinations With Data Collection as a Condition of Coverage in 2006. That original guidance document (which has undergone subsequent revisions, including an additional draft guidance published in 2012⁵⁷) provided several examples of how data collected in a registry might be used in the context of coverage determinations. As described in the Guidance:

[T]he purpose of CED [Coverage with Evidence Development] is to generate data on the utilization and impact of the item or service evaluated in the NCD [National Coverage Determination], so that Medicare can (a) document the appropriateness of use of that item or service in Medicare beneficiaries under current coverage; (b) consider future changes in coverage for the item or service; (c) generate clinical information that will improve the evidence base on which providers base their recommendations to Medicare beneficiaries regarding the item or service.⁵⁶

The Guidance provided insight into when registry data may be useful to policymakers. These purposes range from demonstrating that a particular item or service was provided appropriately to patients meeting specific characteristics, to collecting new information that is not available from existing clinical trials. CED based on registries may be especially relevant when current data do not address relevant outcomes for beneficiaries, off-label or unanticipated uses, important patient subgroups, or operator experience or other qualifications. Registry-based studies may also be important when an existing treatment is being reconsidered. (An RCT may not be possible under such circumstances.) Registry-based studies are also being used increasingly in fulfillment of postmarketing commitments and requirements.

In many countries, policy determinations on payment rely on cost-effectiveness and cost-utility data and therefore can be informed by registries as well as clinical trials.⁵⁸ These data are used and reviewed in a variety of ways. In some countries, there may be a threshold above which a payer is willing to pay for an improvement in patient outcomes.⁵⁹ In these scenarios—particularly for rare diseases, when it can be difficult to gather clinical effectiveness data together with quality-of-life data in a utility format—the establishment of disease-specific data registries has been recommended to facilitate the process of technology assessment and improving patient care.⁶⁰ In fact, the use of new or existing registries to assess health technology or risk-sharing arrangements is growing in such countries as the United Kingdom, France, Germany, and Australia, and in conditions ranging from bariatric surgery to stroke care.^61-66

Consider the clinical question of carotid endarterectomy surgery for patients with a high degree of stenosis of the carotid artery. Randomized trials, using highly selected patients and surgeons, indicate a benefit of surgery over medical management in the prevention of stroke. However, that benefit may be exquisitely sensitive to the surgical complication rates; a relatively small increase in the rate of surgical complications is enough to make medical management the preferred strategy instead. In addition, the studies of surgical performance in a variety of hospitals may suggest substantial variation in surgical mortality and morbidity for this procedure. In such a case, a registry to evaluate treatment outcomes, adjusted by hospital and surgeon, might be considered to support a policy decision as to when the procedure should be reimbursed (e.g., only when performed in medical centers resembling those in the various randomized trials, or only by surgeons or facilities with an acceptably low rate of complications).⁶⁷

5. Global Registries

As many stakeholders have international interests in diseases, conditions, and health care products and services, it is not surprising that interest in global patient registries is growing. While some of the specific legal and regulatory discussions in this user's guide are intended for and limited to the United States, most of the concepts and specifics are more broadly applicable to similar activities worldwide. Chapter 7 (ethics, data ownership, and privacy), Chapter 9 (protection of registry data), and Chapter 12 (adverse event detection, processing, and reporting) are perhaps the most limited in their applicability outside the United States. There may be additional considerations in data element selection and patient-reported outcome measure selection (Chapters 4 and 5) stemming from differences ranging from medical training to use of local remedies; the types of data sources that are available outside the United States (Chapter 6); the requirements for informed consent (Chapter 8); the issues surrounding clinician and patient recruitment and retention in different health systems and cultures (Chapter 10); specific data collection and management options and complexities (Chapter 11), ranging from available technologies to languages; and specific requirements for mandated pregnancy registries (Chapter 21).

6. Summary

A patient registry is an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure and that serves predetermined scientific, clinical, or policy purpose(s). Studies derived from well-designed and well-performed patient registries can provide a real-world view of clinical practice, patient outcomes, safety, and clinical, comparative, and cost-effectiveness, and can serve a number of evidence development and decisionmaking purposes. In the chapters that follow, this user's guide presents practical design and operational issues, evaluation principles, and good registry practices.

References for Chapter 1

1.

Webster's English Dictionary. [August 12, 2012]. http://www​.m-w.com.

2.

Brooke EM. The current and future use of registers in health information systems. Geneva: World Health Organization; 1974. Publication No. 8.

3.

National Committee on Vital and Health Statistics. Frequently Asked Questions About Medical and Public Health Registries. [August 14, 2012]. http://ncvhs​.hhs.gov/9701138b.htm.

4.

Dokholyan RS, Muhlbaier LH, Falletta JM, et al. Regulatory and ethical considerations for linking clinical and administrative databases. Am Heart J. 2009 Jun;157(6):971–82. [PubMed: 19464406]

5.

Hammill BG, Hernandez AF, Peterson ED, et al. Linking inpatient clinical registry data to Medicare claims data using indirect identifiers. Am Heart J. 2009 Jun;157(6):995–1000. [PMC free article: PMC2732025] [PubMed: 19464409]

6.

Developing a Registry of Patient Registries (RoPR). Project Abstract. Agency for Healthcare Research and Quality; [August 14, 2012]. http://www​.effectivehealthcare​.ahrq.gov/index​.cfm/search-for-guides-reviews-and-reports​/?productid=690&pageaction=displayproduct.

7.

LaBresh KA, Gliklich R, Liljestrand J, et al. Using “get with the guidelines” to improve cardiovascular secondary prevention. Jt Comm J Qual Saf. 2003 Oct;29(10):539–50. [PubMed: 14567263]

8.

Kennedy L, Craig AM. Global registries for measuring pharmacoeconomic and quality-of-life outcomes: focus on design and data collection, analysis and interpretation. Pharmacoeconomics. 2004;22(9):551–68. [PubMed: 15209525]

9.

Charrow J, Esplin JA, Gribble TJ, et al. Gaucher disease: recommendations on diagnosis, evaluation, and monitoring. Arch Intern Med. 1998 Sep 14;158(16):1754–60. [PubMed: 9738604]

10.

Bushby K, Lynn S, Straub T, et al. Collaborating to bring new therapies to the patient--the TREAT-NMD model. Acta Myol. 2009 Jul;28(1):12–5. [PMC free article: PMC2859629] [PubMed: 19772190]

11.

Dhruva SS, Phurrough SE, Salive ME, et al. CMS's landmark decision on CT colonography--examining the relevant data. N Engl J Med. 2009 Jun 25;360(26):2699–701. [PubMed: 19474419]

12.

Postmarketing studies and clinical trials – implementation of Section 505(o) of the Federal Food, Drug and Cosmetic Act. FDA Guidance for Industry. Draft guidance. Jul, 2009. [August 14, 2012]. http://www​.fda.gov/downloads​/Drugs/GuidanceComplianceRegulatoryInformation​/Guidances/UCM172001.pdf.

13.

U.S. Food and Drug Administration. Guidance for Industry. Good pharmacovigilance practices and pharmacoepidemiologic assessment. Mar, 2005. [August 14, 2012]. http://www​.fda.gov/downloads​/RegulatoryInformation​/Guidances/UCM126834.pdf.

14.

U.S. Food and Drug Administration. FDA Post-Approval Studies. [August 14, 2012]. http://www​.accessdata​.fda.gov/scripts/cdrh​/cfdocs/cfPMA/pma_pas​.cfm?utm_campaign​=Google2&utm_source​=fdaSearch&utm_medium​=website&utm​_term=post-approval​%20studies&utm​_content=1.

15.

Dreyer NA, Garner S. Registries for robust evidence. JAMA. 2009 Aug 19;302(7):790–1. [PubMed: 19690313]

16.

Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000 Jun 22;342(25):1887–92. [PMC free article: PMC1557642] [PubMed: 10861325]

17.

Guyatt GH, Sackett DL, Sinclair JC, et al. Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA. 1995 Dec 13;274(22):1800–4. [PubMed: 7500513]

18.

Agency for Healthcare Research and Quality. Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews. Version 1.0. Rockville, MD: [August 14, 2012]. [Draft posted Oct. 2007] http:​//effectivehealthcare​.ahrq.gov/repFiles​/2007_10DraftMethodsGuide.pdf.

19.

Vandenbroucke JP. Observational research, randomised trials, and two views of medical science. PLoS Med. 2008 Mar 11;5(3):e67. [PMC free article: PMC2265762] [PubMed: 18336067]

20.

Atkins D, Eccles M, Flottorp S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004 Dec 22;4(1):38. [PMC free article: PMC545647] [PubMed: 15615589]

21.

Rawlins M. De Testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med. 2008 Dec;8(6):579–88. [PMC free article: PMC4954394] [PubMed: 19149278]

22.

Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004 Jun 19;328(7454):1490. [PMC free article: PMC428525] [PubMed: 15205295]

23.

Grading the Strength of a Body of Evidence When Assessing Health Care Interventions – AHRQ and the Effective Health Care Program: An Update. [October 12, 2012]. Draft released for public comment. http://www​.effectivehealthcare​.ahrq.gov/ehc​/products/457/1163​/GradingTheStrengthofEvidence​_DraftMethodsChapter_20120625​.pdf.

24.

Viswanathan M, Ansari MT, Berkman ND, Chang S, Hartling L, McPheeters LM, Santaguida PL, Shamliyan T, Singh K, Tsertsvadze A, Treadwell JR. Agency for Healthcare Research and Quality Methods Guide for Comparative Effectiveness Reviews. Rockville, MD: Agency for Healthcare Research and Quality; Mar, 2012. [October 12, 2012]. Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. AHRQ Publication No. 12-EHC047-EF. http://www​.effectivehealthcare​.ahrq.gov/ehc​/products/322/998​/MethodsGuideforCERs​_Viswanathan_IndividualStudies.pdf.

25.

Clancy CM, Eisenberg JM. Outcomes research: measuring the end results of health care. Science. 1998 Oct 9;282(5387):245–6. [PubMed: 9841388]

26.

Lipscomb J, Snyder CF. The Outcomes of Cancer Outcomes Research: focusing on the National Cancer Institute's quality-of-care initiative. Med Care. 2002 Jun;40(6 Suppl):III3–10. [PubMed: 12064755]

27.

National Cancer Institute. The Emerging Field of Outcomes Research. [August 14, 2012]. http://outcomes​.cancer​.gov/aboutresearch/index.html.

28.

Lipscomb J, Donaldson MS, Hiatt RA. Cancer outcomes research and the arenas of application. J Natl Cancer Inst Monogr. 2004;(33):1–7. [PubMed: 15504917]

29.

Institute of Medicine; Committee on Quality of Heallth Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington DC: National Academy Press, Institute of Medicine; 2001. [PubMed: 25057539]

30.

Schwamm LH, LaBresh KA, Pan WQ, et al. Get With The Guidelines – Stroke produces sustainable improvements in hospital-based acute stroke care. Stroke. 2006;37(2):624.

31.

Schwamm LH, LaBresh KA, Pan WQ, et al. Get With The Guidelines – Stroke improves the rate of “defect-free” acute stroke care. Stroke. 2006;37(2):659.

32.

Barranger JA, O'Rourke E. Lessons learned from the development of enzyme therapy for Gaucher disease. J Inherit Metab Dis. 2001;24 Suppl 2:89–96. discussion 87-8. [PubMed: 11758684]

33.

Wennberg DE, Lucas FL, Birkmeyer JD, et al. Variation in carotid endarterectomy mortality in the Medicare population: trial hospitals, volume, and patient characteristics. JAMA. 1998 Apr 22-29;279(16):1278–81. [PubMed: 9565008]

34.

MacIntyre K, Capewell S, Stewart S, et al. Evidence of improving prognosis in heart failure: trends in case fatality in 66 547 patients hospitalized between 1986 and 1995. Circulation. 2000 Sep 5;102(10):1126–31. [PubMed: 10973841]

35.

Konstam MA. Progress in heart failure Management? Lessons from the real world. Circulation. 2000 Sep 5;102(10):1076–8. [PubMed: 10973832]

36.

Hutchins LF, Unger JM, Crowley JJ, et al. Underrepresentation of patients 65 years of age or older in cancer-treatment trials. N Engl J Med. 1999 Dec 30;341(27):2061–7. [PubMed: 10615079]

37.

Brown SL, Bright RA, Tavris DR, editors. Medical device epidemiology and surveillance. John Wiley & Sons, Ltd.; 2007. Opthalmic devices and clinical epidemiology. chapter 28.

38.

Lipscomb J, Yabroff KR, Brown ML, et al. Health care costing: data, methods, current applications. Med Care. 2009 Jul;47(7 Suppl 1):S1–6. [PubMed: 19536004]

39.

Eichler HG, Kong SX, Gerth WC, et al. Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge? Value Health. 2004 Sep-Oct;7(5):518–28. [PubMed: 15367247]

40.

Palmer AJ. Health economics--what the nephrologist should know. Nephrol Dial Transplant. 2005 Jun;20(6):1038–41. [PubMed: 15840678]

41.

Good ReseArch for Comparative Effectiveness. [August 14, 2012]. http://www​.graceprinciples.org.

42.

Institute of Medicine. Learning what works best: the Nation's need for evidence on comparative effectiveness in health care. Washington DC: National Academy Press; 2007. pp. 1–80.

43.

Congressional Budget Office. Research on the Comparative Effectiveness of Medical Treatments: Issues and Options for an Expanded Federal Role. 2007. [August 14, 2012]. http://www​.cbo.gov/sites​/default/files/cbofiles​/ftpdocs/88xx​/doc8891/12-18-comparativeeffectiveness​.pdf.

44.

Federal Coordinating Council for Comparative Effectiveness Research. Report to the President and the Congress. U.S. Department of Health and Human Services; Jun 30, 2009. [August 14, 2012]. http://www​.hhs.gov/recovery​/programs/cer/cerannualrpt.pdf.

45.

Patient-Centered Outcomes Research Institute. Patient-Centered Outcomes Research. [August 28, 2012]. http://www​.pcori.org/what-we-do/pcor/

46.

Public comment draft report of the Patient-Centered Outcomes Research Institute (PCORI) Methodology Committee presented on July 23, 2012, and revised thereafter. [August 28, 2012]. http://pcori​.org/assets​/MethodologyReport-Comment.pdf.

47.

Agency for Healthcare Research and Quality. Bridging Knowledge Gaps in the Comparative Effectiveness of ACE Inhibitors and ARBs. Draft abstract. 2008. [August 14, 2012]. http://www​.effectivehealthcare​.ahrq.gov/index​.cfm/search-for-guides-reviews-and-reports​/?productid=296&pageaction=displayproduct.

48.

Hodgson DC, Fuchs CS, Ayanian JZ. Impact of patient and provider characteristics on the treatment and outcomes of colorectal cancer. J Natl Cancer Inst. 2001 Apr 4;93(7):501–15. [PubMed: 11287444]

49.

Reeves MJ, Fonarow GC, Zhao X, et al. Quality of care in women with ischemic stroke in the GWTG program. Stroke. 2009 Apr;40(4):1127–33. [PubMed: 19211482]

50.

Fonarow GC, Abraham WT, Albert NM, et al. Influence of a performance-improvement initiative on quality of care for patients hospitalized with heart failure: results of the Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients With Heart Failure (OPTIMIZE-HF). Arch Intern Med. 2007 Jul 23;167(14):1493–502. [PubMed: 17646603]

51.

Greene FL, Gilkerson S, Tedder P, et al. The role of the hospital registry in achieving outcome benchmarks in cancer care. J Surg Oncol. 2009 Jun 15;99(8):497–9. [PubMed: 19466739]

52.

Schweikert B, Hunger M, Meisinger C, et al. Quality of life several years after myocardial infarction: comparing the MONICA/KORA registry to the general population. Eur Heart J. 2009 Feb;30(4):436–43. [PubMed: 19019995]

53.

Lane K, Kempf A, Magno C, et al. Regional differences in the use of sentinel lymph node biopsy for melanoma: a potential quality measure. Am Surg. 2008 Oct;74(10):981–4. [PubMed: 18942627]

54.

Griffiths I, Silman A, Symmons D, et al. BSR Biologics Registry. Rheumatology (Oxford). 2004 Dec;43(12):1463–4. [PubMed: 15328420]

55.

U.S. Food and Drug Administration. Guidance for Industry: Establishing Pregnancy Exposure Registries. [August 15, 2012]. http://www​.fda.gov/downloads​/Drugs/GuidanceComplianceRegulatoryInformation​/Guidances/ucm071639.pdf.

56.

Centers for Medicare & Medicaid Services. Guidance for the Public, Industry, and CMS Staff: National Coverage Determinations with Data Collection as a Condition of Coverage: Coverage with Evidence Development. Jul 12, 2006.

57.

Centers for Medicare & Medicaid Services. Draft Guidance for the Public, Industry, and CMS Staff: Coverage with Evidence Development in the context of coverage decisions. [February 20, 2013]. http://www​.cms.gov/medicare-coverage-database​/details/medicare-coverage-document-details​.aspx?MCDId=23.

58.

Connock M, Burls A, Frew E, et al. The clinical effectiveness and cost-effectiveness of enzyme replacement therapy for Gaucher's disease: a systematic review. Health Technol Assess. 2006 Jul;10(24):iii–iv. ix–136. [PubMed: 16796930]

59.

Devlin N, Parkin D. Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Econ. 2004 May;13(5):437–52. [PubMed: 15127424]

60.

Connock M, Juarez-Garcia A, Frew E, et al. A systematic review of the clinical effectiveness and cost-effectiveness of enzyme replacement therapies for Fabry's disease and mucopolysaccharidosis type 1. Health Technol Assess. 2006 Jun;10(20):iii–iv. ix–113. [PubMed: 16729919]

61.

Chalkidou K, Tunis S, Lopert R, et al. Comparative effectiveness research and evidence-based health policy: experience from four countries. Milbank Q. 2009 Jun;87(2):339–67. [PMC free article: PMC2881450] [PubMed: 19523121]

62.

National Institute for Health and Clinical Excellence (UK). Final appraisal determination: alteplase for the treatment of acute ischaemic stroke. 2007. [August 14, 2012]. http://www​.nice.org.uk​/nicemedia/live/11617/33971/33971.pdf.

63.

Graves SE, Davidson D, Ingerson L, et al. The Australian Orthopaedic Association National Joint Replacement Registry. Med J Aust. 2004 Mar 1;180(5 Suppl):S31–4. [PubMed: 14984361]

64.

Owen A, Spinks J, Meehan A, et al. A new model to evaluate the long-term cost effectiveness of orphan and highly specialised drugs following listing on the Australian Pharmaceutical Benefits Scheme: the Bosentan Patient Registry. J Med Econ. 2008;11(2):235–43. [PubMed: 19450082]

65.

Haute Autorité de Santé Heart surgery with or without extracorporeal circulation: role of the second surgeon. 2008. [August 14, 2012]. http://www​.has-sante​.fr/portail/upload/docs​/application/pdf/abstract​_heart_surgery.pdf.

66.

Haute Autorité de Santé Interview du docteur Jean-François Thébaut, cardiologue libéral et Président du Conseil National Professionnel de Cardiologie. [August 14, 2012]. [French.]. http://www​.has-sante​.fr/portail/jcms/c_777504​/interview-du-docteur-jean-francois-thebautcardiologue-liberal-et-president-du-conseil-nationalprofessionnel-de-cardiologie?portal​=c_63456.

67.

Matchar DB, Oddone EZ, McCrory DC, et al. Influence of projected complication rates on estimated appropriate use rates for carotid endarterectomy. Appropriateness Project Investigators. Academic Medical Center Consortium. Health Serv Res. 1997 Aug;32(3):325–42. [PMC free article: PMC1070194] [PubMed: 9240284]