Randomized controlled trials (RCTs) have traditionally served as the gold standard for generating evidence about medical interventions. However, RCTs have inherent limitations and may not reflect the use of medical products in the real world. Additionally, RCTs are expensive, time consuming, and cannot answer all questions about a product or intervention. Evidence generated from real-world use, such as real-world evidence (RWE) may provide valuable information, alongside RCTs, to inform medical product decision making. To explore the potential for using RWE in medical product decision making, the National Academies of Sciences, Engineering, and Medicine planned a three-part workshop series. The series was designed to examine the current system of evidence generation and its limitations, to identify when and why RWE may be an appropriate type of evidence on which to base decisions, to learn from successful initiatives that have incorporated RWE, and to describe barriers that prevent RWE from being used to its full potential. This publication summarizes the discussions from the entire workshop series.

Contents

• THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
• The National Academies of SCIENCES • ENGINEERING • MEDICINE
• The National Academies of SCIENCES • ENGINEERING • MEDICINE
• PLANNING COMMITTEE ON EXAMINING THE IMPACT OF REAL-WORLD EVIDENCE ON MEDICAL PRODUCT DEVELOPMENT
• FORUM ON DRUG DISCOVERY, DEVELOPMENT, AND TRANSLATION
• Reviewers
• Acronyms and Abbreviations
• 1. Introduction
  • DEFINING THE TERMS
  • THE U.S. FOOD AND DRUG ADMINISTRATION'S VISION FOR THE FUTURE
  • FIT-FOR-PURPOSE EVIDENCE
• 2. Perspectives on Real-World Evidence
  • PAYER PERSPECTIVE
  • DELIVERY SYSTEMS PERSPECTIVE
  • PATIENT PERSPECTIVE
  • DISCUSSION
• 3. Learning from Success
  • SALFORD LUNG STUDIES
  • SENTINEL
  • DEVICE REGISTRIES
• 4. Barriers and Disincentives to the Use of Real-World Evidence and Real-World Data
  • THE USE OF REAL-WORLD EVIDENCE AND REAL-WORLD DATA IN PRODUCT DEVELOPMENT
  • MAKING CHOICES ABOUT RESEARCH DESIGN
  • ACCELERATING EVIDENCE GENERATION THROUGH DEFRAGMENTATION
  • EVIDENCE HIERARCHIES
  • OPPORTUNITIES TO INTEGRATE REAL-WORLD DATA AND REAL-WORLD EVIDENCE IN RESEARCH
  • DISCUSSION
• 5. Getting Unstuck: Mythbusting the Current System
  • FROM PRECISION TO RELIABILITY
  • INTEGRATING THE NEW WITH THE OLD
  • REGULATORY PERSPECTIVE
• 6. When Is a Real-World Data Element Fit for Assessment of Eligibility, Treatment Exposure, or Outcomes?
  • REAL-WORLD DATA ELEMENTS
  • ILLUSTRATIVE EXAMPLES
  • DISCUSSION: CHARACTERIZING REAL-WORLD DATA AND REAL-WORLD EVIDENCE
  • DECISION AID
  • “FIT FOR PURPOSE” AND RELEVANCE OF DATA
  • PATIENT-GENERATED DATA
  • DISCUSSION: REAL-WORLD DATA CONCERNS FOR FUTURE RESEARCH
• 7. How Tightly Should Investigators Attempt to Control or Restrict Treatment Quality in a Pragmatic or Real-World Trial?
  • ILLUSTRATIVE EXAMPLES
  • DECISION AID
  • PATIENT-CENTERED RESEARCH
  • CONTEXT OF THE DECISION
  • OBLIGATION TO PATIENTS
• 8. Obscuring Intervention Allocation in Trials to Generate Real-World Evidence: Why, Who, and How?
  • ILLUSTRATIVE EXAMPLES
  • DECISION AID
  • DISCUSSION
• 9. Gaining Confidence in Observational Comparisons
  • ILLUSTRATIVE EXAMPLES
  • DISCUSSION: OBSERVATIONAL STUDIES AND RANDOMIZATION
  • DECISION AID
  • PRESENTATIONS: OBSERVATIONAL STUDIES AND BIAS
  • DISCUSSION: THE FUTURE OF OBSERVATIONAL STUDIES
• 10. Looking Ahead
  • REAL-WORLD EVIDENCE TO IMPROVE HEALTH TECHNOLOGY ASSESSMENT
  • REAL-WORLD EVIDENCE TO TURN PATIENTS INTO PARTNERS
  • REAL-WORLD EVIDENCE TO TRANSFORM RESEARCH AND DEVELOPMENT
  • REAL-WORLD EVIDENCE TO INFORM REGULATORY DECISIONS
  • FINAL THOUGHTS
• References
• Appendix A. Related Resources
• Appendix B. Workshop One Agenda
• Appendix C. Workshop Two Agenda
• Appendix D. Workshop Three Agenda