The Observational CER User's Guide serves as a resource for investigators and stakeholders when designing observational comparative effectiveness research (CER) studies, particularly those with findings that are intended to translate into decisions or actions. The User's Guide provides principles for designing research that will inform health care decisions of patients and other stakeholders. Furthermore, it serves as a reference for increasing the transparency of the methods used in a study and standardizing the review of protocols through checklists provided in every chapter.

Contents

• Acknowledgments
• Introduction to Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide
  Scott R Smith, PhD.
  • Background
  • Aims of the User's Guide Related to the Design of Observational CER Protocols
  • Summary and Conclusion
  • References
• 1. Study Objectives and Questions
  Scott R Smith, PhD.
  • Overview
  • Identifying Decisions, Decisionmakers, Actions, and Context
  • Synthesizing the Current Knowledge Base
  • Conceptualizing the Research Problem
  • Determining the Stage of Knowledge Development for the Study Design
  • Defining and Refining Study Questions Using PICOTS Framework
  • Discussing Evidentiary Need and Uncertainty
  • Specifying Magnitude of Effect
  • Challenges to Developing Study Questions and Initial Solutions
  • Summary and Conclusion
  • References
• 2. Study Design Considerations
  Til Stürmer, MD, MPH, PhD and M. Alan Brookhart, PhD.
  • Introduction
  • Issues of Bias in Observational CER
  • Basic Epidemiologic Study Designs
  • Other Epidemiological Study Designs Relevant to CER
  • Study Design Features
  • Other Study Design Considerations
  • Conclusion
  • References
• 3. Estimation and Reporting of Heterogeneity of Treatment Effects
  Ravi Varadhan, PhD and John D Seeger, PharmD, DrPH.
  • Introduction
  • Subgroup Analyses: Special Considerations for Observational Studies
  • Conclusion
  • References
• 4. Exposure Definition and Measurement
  Todd A Lee, PharmD, PhD and A Simon Pickard, PhD.
  • Introduction
  • Conceptual Considerations for Exposure Measurement
  • Creating an Exposure Definition
  • Issues of Bias
  • Conclusion
  • References
• 5. Comparator Selection
  Soko Setoguchi, MD, DrPH and Tobias Gerhard, PhD.
  • Introduction
  • Choosing the Comparison Group in CER
  • Operationalizing the Comparison Group in CER
  • Conclusion
  • References
• 6. Outcome Definition and Measurement
  Priscilla Velentgas, PhD, Nancy A Dreyer, MPH, PhD, and Albert W Wu, MD, MPH.
  • Introduction
  • Conceptual Models of Health Outcomes
  • Outcome Measurement Properties
  • Clinical Outcomes
  • Humanistic Outcomes
  • Study Design and Analysis Considerations
  • Conclusion
  • References
• 7. Covariate Selection
  Brian Sauer, PhD, M. Alan Brookhart, PhD, Jason A Roy, PhD, and Tyler J VanderWeele, PhD.
  • Introduction
  • Causal Models and the Structural Relationship of Variables
  • Proxy, Mismeasured, and Unmeasured Confounders
  • Selection of Variables To Control Confounding
  • Conclusion
  • References
• 8. Selection of Data Sources
  Cynthia Kornegay, PhD and Jodi B Segal, MD, MPH.
  • Introduction
  • Data Options
  • Considerations for Selecting Data
  • Ensuring Quality Data
  • Data Privacy Issues
  • Emerging Issues and Opportunities
  • Conclusion
  • References
• 9. Study Size Planning
  Eric S Johnson, PhD, MPH, M Alan Brookhart, PhD, and Jessica A Myers, PhD.
  • Introduction
  • Study Size and Power Calculations in RCTs
  • Considerations for Observational CER Study Size Planning
  • Conclusion
  • References
• 10. Considerations for Statistical Analysis
  Patrick G Arbogast, PhD and Tyler J VanderWeele, PhD.
  • Introduction
  • Descriptive Statistics/Unadjusted Analyses
  • Adjusted Analyses
  • Missing Data Considerations
  • Conclusion
  • References
• 11. Sensitivity Analysis
  Joseph AC Delaney, PhD and John D Seeger, PharmD, DrPH.
  • Introduction
  • Unmeasured Confounding and Study Definition Assumptions
  • Data Source, Subpopulations, and Analytic Methods
  • Statistical Assumptions
  • Implementation Approaches
  • Presentation
  • Conclusion
  • References
• Supplement 1. Improving Characterization of Study Populations: The Identification Problem
  John M Brooks, PhD.
  • Introduction
  • Background
  • Properties of the Study Population
  • Relationship of Estimation Methods to Patient Subsets
  • Assumptions Required To Yield Unbiased Estimates
  • Identification of Research Objectives Other Than ATT or LATE
  • Appendix Treatment Choice/Outcome Model Specifications, Estimators, and Identification
• Supplement 2. Use of Directed Acyclic Graphs
  Brian Sauer, PhD and Tyler J VanderWeele, PhD.
  • Introduction
  • Estimating Causal Effects
  • DAG Terminology
  • Using DAGs To Select Covariates and Diagnose Bias
  • Using DAGs To Diagnose Selection Bias
  • Conclusion
  • References
• Authors
• Reviewers
• Suggested Citations

Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services¹, Contract No. 290-2005-0016-I and 290-2005-0035-I. Prepared by: Quintiles Outcome, Cambridge, MA

Suggested citation:

Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM, eds. Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. AHRQ Publication No. 12(13)-EHC099. Rockville, MD: Agency for Healthcare Research and Quality; January 2013. www.effectivehealthcare.ahrq.gov/Methods-OCER.cfm.

The Agency for Healthcare Research and Quality’s (AHRQ) Effective Health Care Program conducts and supports research focused on the outcomes, effectiveness, comparative clinical effectiveness, and appropriateness of pharmaceuticals, devices, and health care services. More information on the Effective Health Care Program and electronic copies of this report can be found at www.effectivehealthcare.ahrq.gov.

This report was produced under contract to AHRQ by the Brigham and Women's Hospital DEcIDE (Developing Evidence to Inform Decisions about Effectiveness) Methods Center and Quintiles Outcomes under Contract No. 290-2005-0016-I and 290-2005-0035-1. The AHRQ Task Order Officer for this project was Parivash Nourjah, Ph.D. The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ or the U.S. Department of Health and Human Services. Therefore, no statement in this report should be construed as an official position of AHRQ or the U.S. Department of Health and Human Services.

None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report.

1

540 Gaither Road, Rockville, MD 20850; www​.ahrq.gov