The concept of utilizing big data to enable scientific discovery has generated tremendous excitement and investment from both private and public sectors over the past decade, and expectations continue to grow. Using big data analytics to identify complex patterns hidden inside volumes of data that have never been combined could accelerate the rate of scientific discovery and lead to the development of beneficial technologies and products. However, producing actionable scientific knowledge from such large, complex data sets requires statistical models that produce reliable inferences (NRC, 2013). Without careful consideration of the suitability of both available data and the statistical models applied, analysis of big data may result in misleading correlations and false discoveries, which can potentially undermine confidence in scientific research if the results are not reproducible. In June 2016 the National Academies of Sciences, Engineering, and Medicine convened a workshop to examine critical challenges and opportunities in performing scientific inference reliably when working with big data. Participants explored new methodologic developments that hold significant promise and potential research program areas for the future. This publication summarizes the presentations and discussions from the workshop.

Contents

• The National Academies of SCIENCES · ENGINEERING · MEDICINE
• The National Academies of SCIENCES · ENGINEERING · MEDICINE
• PLANNING COMMITTEE ON REFINING THE CONCEPT OF SCIENTIFIC INFERENCE WHEN WORKING WITH BIG DATA
• COMMITTEE ON APPLIED AND THEORETICAL STATISTICS
• BOARD ON MATHEMATICAL SCIENCES AND THEIR APPLICATIONS
• Acknowledgment of Reviewers
• 1. Introduction
  • WORKSHOP OVERVIEW
  • WORKSHOP THEMES
  • ORGANIZATION OF THIS WORKSHOP PROCEEDINGS
• 2. Framing the Workshop
  • PERSPECTIVES FROM STAKEHOLDERS
    Michelle Dunn, Nandini Kannan, and Chaitan Baru.
  • INTRODUCTION TO THE SCIENTIFIC CONTENT OF THE WORKSHOP
    Michael Daniels.
• 3. Inference About Discoveries Based on Integration of Diverse Data Sets
  • DATA INTEGRATION WITH DIVERSE DATA SETS
    Alfred Hero, III.
  • DATA INTEGRATION AND ITERATIVE TESTING
    Andrew Nobel.
  • PANEL DISCUSSION
  • STATISTICAL DATA INTEGRATION FOR LARGE-SCALE MULTIMODAL MEDICAL STUDIES
    Genevera Allen.
  • DISCUSSION OF STATISTICAL INTEGRATION FOR MEDICAL AND HEALTH STUDIES
    Jeffrey S. Morris.
  • PANEL DISCUSSION
• 4. Inference About Causal Discoveries Driven by Large Observational Data
  • USING ELECTRONIC HEALTH RECORDS DATA FOR CAUSAL INFERENCES ABOUT THE HUMAN IMMUNODEFICIENCY VIRUS CARE CASCADE
    Joseph Hogan.
  • DISCUSSION OF CAUSAL INFERENCES ON THE HUMAN IMMUNODEFICIENCY VIRUS CARE CASCADE FROM ELECTRONIC HEALTH RECORDS DATA
    Elizabeth Stuart.
  • PANEL DISCUSSION
  • A GENERAL FRAMEWORK FOR SELECTION BIAS DUE TO MISSING DATA IN ELECTRONIC HEALTH RECORDS-BASED RESEARCH
    Sebastien Haneuse.
  • DISCUSSION OF COMPARATIVE EFFECTIVENESS RESEARCH USING ELECTRONIC HEALTH RECORDS
    Dylan Small.
  • PANEL DISCUSSION
• 5. Inference When Regularization Is Used to Simplify Fitting of High-Dimensional Models
  • LEARNING FROM TIME
    Daniela Witten.
  • DISCUSSION OF LEARNING FROM TIME
    Michael Kosorok.
  • PANEL DISCUSSION
  • SELECTIVE INFERENCE IN LINEAR REGRESSION
    Jonathan Taylor.
  • STATISTICS AND BIG DATA CHALLENGES IN NEUROSCIENCE
    Emery N. Brown.
  • DISCUSSION OF STATISTICS AND BIG DATA CHALLENGES IN NEUROSCIENCE
    Xihong Lin.
  • PANEL DISCUSSION
• 6. Panel Discussion
  • RESEARCH PRIORITIES FOR IMPROVING INFERENCES FROM BIG DATA
  • INFERENCE WITHIN COMPLEXITY AND COMPUTATIONAL CONSTRAINTS
  • EDUCATION AND CROSS-DISCIPLINARY COLLABORATION
  • IDENTIFICATION OF QUESTIONS AND APPROPRIATE USES FOR AVAILABLE DATA
  • FACILITATION OF DATA SHARING AND LINKAGE
  • THE BOUNDARY BETWEEN BIOSTATISTICS AND BIOINFORMATICS
• References
• Appendixes
  • A. Registered Workshop Participants
  • B. Workshop Agenda
  • C. Acronyms

Rapporteur: Ben A. Wender.

Suggested citation:

National Academies of Sciences, Engineering, and Medicine. 2017. Refining the Concept of Scientific Inference When Working with Big Data: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24654.