NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Comparative Effectiveness Review Methods: Clinical Heterogeneity
Methods Research Reports
Investigators: Suzanne L West, MPH, PhD, Gerald Gartlehner, MD, MPH, Alyssa J Mansfield, MPH, PhD, Charles Poole, MPH, ScD, Elizabeth Tant, BS, Nancy Lenfestey, MHA, Linda J Lux, MPA, Jacqueline Amoozegar, BA, Sally C Morton, PhD, Timothy C Carey, MD, MPH, Meera Viswanathan, PhD, and Kathleen N Lohr, PhD.
Author Information and AffiliationsStructured Abstract
Objectives:
The Agency for Healthcare Research and Quality (AHRQ) funded the RTI International—University of North Carolina at Chapel Hill Evidence-based Practice Center to determine best practices for addressing clinical heterogeneity in systematic reviews (SRs) and comparative effectiveness reviews (CERs). These best practices address critiques from patients, clinicians, policymakers, and others who assert that SRs typically focus on broad populations and, as a result, often lack information relevant to individual patients or patient subgroups.
Data sources and methods:
We used numerous data sources. We abstracted information from guidance documents prepared by U.S. and international organizations engaged in preparing reviews. We searched MEDLINE® to identify studies on how to handle clinical heterogeneity and subgroup analyses. We reviewed more than 120 SRs conducted by AHRQ’s Evidence-based Practice Centers (EPCs), the Cochrane Collaboration, the Drug Effectiveness Review Project, the United Kingdom’s National Institute for Health and Clinical Excellence and others that we identified from the Centre for Reviews and Dissemination Database of Abstracts of Reviews of Effects and Health Technology Assessment. We reviewed peer and public review comments from AHRQ’s Scientific Review Center for three CERs, and we conducted key informant interviews with authors of six SRs prepared by AHRQ’s EPCs or international organizations.
Results:
Clinical heterogeneity has been defined as the variation in study population characteristics, coexisting conditions, cointerventions, and outcomes evaluated across studies included in an SR or CER that may influence or modify the magnitude of the intervention measure of effect (e.g., odds ratio, risk ratio, risk difference). Statistical heterogeneity is defined as variability in the observed treatment effects beyond what would be expected by random error. The review organizations we studied varied in their inclusion of factors, in terms of the key questions and analysis that may modify the treatment-outcome association. They tended to give more consideration to demographic factors than to disease factors (e.g., disease severity, risk factors, coexisting disease, or cointerventions). Individual systematic reviewers whom we interviewed preferred a priori identification of effect modifiers to post hoc determination because of the latter’s data-dredging nature and the possibility of type 1 error when many subgroups are evaluated. Many publications that we identified through our literature searches did indicate that analysis of individual patient-level data in meta-analyses does allow better assessment of clinical heterogeneity, but the time, cost, and difficulty in obtaining these data are often prohibitive.
Conclusions:
Identifying factors that may influence the treatment-outcome association is important to clinicians and patients because it helps them understand which patients will benefit most, who is least likely to benefit, and who is at greatest risk of experiencing adverse outcomes. Clear evidence-based guidance on addressing clinical heterogeneity in SRs and CERs is not available currently but would be valuable to AHRQ’s EPCs and to others conducting SRs internationally.
Contents
- Preface
- Executive Summary
- 1. Introduction
- 2. Methods
- 3. Results
- KQ 1 What Is Clinical Heterogeneity?
- KQ 2 How Have Systematic Reviews Dealt with Clinical Heterogeneity in the Key Questions?
- KQ 3 How Have Systematic Reviews Dealt With Clinical Heterogeneity in the Review Process?
- KQ 4 What Are Critiques in How Systematic Reviews Handle Clinical Heterogeneity?
- KQ 5 What Evidence Is There To Support How Best To Address Clinical Heterogeneity in a Systematic Review?
- 4. Discussion
- The Concept of Clinical Heterogeneity
- Clinical Heterogeneity in Systematic Review Key Questions
- How Systematic Reviews Dealt With Clinical Heterogeneity in the Review Process
- Critiques of How Systematic Reviews Handle Clinical Heterogeneity
- Evidence for How Best To Address Clinical Heterogeneity in a Systematic Review
- AHRQ EPC Work Group
- References
- Appendixes
- Abbreviations
Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2007-10056-I. Prepared by: RTI International–University of North Carolina Evidence-based Practice Center, Research Triangle Park, North Carolina.
Suggested citation:
West SL, Gartlehner G, Mansfield AJ, Poole C, Tant E, Lenfestey N, Lux LJ, Amoozegar J, Morton SC, Carey TC, Viswanathan M, Lohr KN. Comparative Effectiveness Review Methods: Clinical Heterogeneity. Agency for Healthcare Research and Quality; September 2010. Methods Research Paper. AHRQ Publication No. 10-EHC070-EF. Available at http://effectivehealthcare.ahrq.gov/.
This report is based on research conducted by the RTI International–University of North Carolina Evidence-based Practice Center(EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10056-I). The findings and conclusions in this document are those of the author(s), who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
The information in this report is intended to help clinicians, employers, policymakers, and others make informed decisions about the provision of health care services. This report is intended as a reference and not as a substitute for clinical judgment.
This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.
This document was written with support from the Effective Health Care Program at AHRQ. None of the authors has a financial interest in any of the products discussed in this document. This document is in the public domain and may be used and reprinted without permission except those copyrighted materials noted, for which further reproduction is prohibited without the specific permission of copyright holders.
The investigators have no relevant financial interests in the report. The investigators have no employment, consultancies, honoraria, or stock ownership or options, or royalties from any organization or entity with a financial interest or financial conflict with the subject matter discussed in the report.
- 1
540 Gaither Road, Rockville, MD 20850; www
​.ahrq.gov
- NLM CatalogRelated NLM Catalog Entries
- Review Low-Dose Aspirin for the Prevention of Morbidity and Mortality From Preeclampsia: A Systematic Evidence Review for the U.S. Preventive Services Task Force[ 2014]Review Low-Dose Aspirin for the Prevention of Morbidity and Mortality From Preeclampsia: A Systematic Evidence Review for the U.S. Preventive Services Task ForceHenderson JT, Whitlock EP, O'Conner E, Senger CA, Thompson JH, Rowland MG. 2014 Apr
- Review Updating Systematic Reviews[ 2007]Review Updating Systematic ReviewsShojania KG, Sampson M, Ansari MT, Ji J, Garritty C, Rader T, Moher D. 2007 Sep
- Review Clinical heterogeneity in systematic reviews and health technology assessments: synthesis of guidance documents and the literature.[Int J Technol Assess Health Ca...]Review Clinical heterogeneity in systematic reviews and health technology assessments: synthesis of guidance documents and the literature.Gartlehner G, West SL, Mansfield AJ, Poole C, Tant E, Lux LJ, Lohr KN. Int J Technol Assess Health Care. 2012 Jan; 28(1):36-43. Epub 2012 Jan 23.
- Review Screening for Chronic Obstructive Pulmonary Disease: A Systematic Evidence Review for the U.S. Preventive Services Task Force[ 2016]Review Screening for Chronic Obstructive Pulmonary Disease: A Systematic Evidence Review for the U.S. Preventive Services Task ForceGuirguis-Blake JM, Senger CA, Webber EM, Mularski R, Whitlock EP. 2016 Apr
- Review Screening for Cognitive Impairment in Older Adults: An Evidence Update for the U.S. Preventive Services Task Force[ 2013]Review Screening for Cognitive Impairment in Older Adults: An Evidence Update for the U.S. Preventive Services Task ForceLin JS, O'Connor E, Rossom RC, Perdue LA, Burda BU, Thompson M, Eckstrom E. 2013 Nov
- Comparative Effectiveness Review Methods: Clinical HeterogeneityComparative Effectiveness Review Methods: Clinical Heterogeneity
Your browsing activity is empty.
Activity recording is turned off.
See more...