NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Structured Abstract
Research Objective:
Systematic reviews, in addition to summarizing the evidence, generally also discuss needs for future research. However, in contrast to the methods of the systematic review, future needs are not identified systematically. There is limited literature describing organizing principles or frameworks for determining research gaps. We developed and pilot-tested a framework for the identification of research gaps from systematic reviews.
Study Design:
We reviewed the research gaps identification practices of organizations involved with evidence synthesis. We contacted: (i) evidence-based practice centers (EPCs) (n=12) associated with the Agency for Healthcare Research and Quality (AHRQ) in the US and Canada, and (ii) other organizations around the world (n=64) that conduct systematic reviews, cost-effectiveness analyses, or technology assessments. Based on the responses, we developed a framework for identifying research gaps. We obtained feedback from two technical experts at our institution and pilot-tested this framework on two randomly selected EPC evidence reports. We also developed a simple, user-friendly worksheet with instructions to facilitate the use of the framework by investigators during or after a systematic review.
Population Studied:
Not Applicable.
Principal Findings:
Four (33.3%) EPCs and 3 (8.1%) of the other organizations reported currently using an explicit framework to determine research gaps. We did not identify one framework that captured all elements needed to determine and characterize research gaps. Variations of the PICO (population, intervention, comparison, outcomes) framework were most common. It is also important to classify the reason(s) for the gap to help determine how to address the gap. Therefore, we propose a framework that includes both the characterization of the gap using PICOS elements (also including setting) and the identification of the reason(s) why the gap exists. The framework allows investigators to classify reasons for the existence of a research gap as: (a) insufficient or imprecise information, (b) biased information; (c) inconsistency or unknown consistency, and (d) not the right information. We mapped each of these reasons to concepts from three commonly used evidence grading systems: the Grading of Recommendations Assessment, Development and Evaluation (GRADE); the United States Preventive Services Task Force (USPSTF); and the Strength of Evidence (SOE) used by EPCs. This allows leveraging of work already being completed during evidence grading. During pilot-testing, we identified challenges including difficulty in applying the framework for completed systematic reviews and differences in the specificity of research gaps abstracted by different users. These could be tackled with a priori discussions amongst investigators. Further testing should determine if these challenges are ameliorated if the framework is used during a systematic review.
Conclusions:
We developed a framework to identify and characterize research gaps from systematic reviews. The framework provides for the classification of where and why the current evidence falls short.
Implications for Policy, Delivery, or Practice:
In synthesizing evidence, systematic reviews inform health-care decisions for patients, policymakers, and clinicians. Systematic reviews can also be invaluable for identifying research gaps, thus helping develop research agendas. This potential impact of systematic reviews has not been realized. Our framework provides for systematically identifying and characterizing research gaps from systematic reviews. This explicit identification of research gaps will help determine the type of research needed to address the goals of comparative effectiveness research.
Contents
- Preface
- Acknowledgments
- Technical Expert Panel
- Peer Reviewers
- Executive Summary
- Background
- Methods
- Step 1. Focused Literature Review
- Step 2. Review of Current Practices of Evidence-based Practice Centers (EPCs)
- Step 3. Review of Current Practices of Organizations Involved With Evidence Synthesis
- Step 4. Development of Framework
- Step 5. Pilot Test of Framework
- Step 6. Refinement and Finalization of Framework
- Results
- Step 1. Focused Literature Review
- Step 2. Review of Current Practices of Evidence-based Practice Centers (EPCs)
- Step 3. Review of Current Practices of Organizations Involved With Evidence Synthesis
- Step 4. Development of Framework
- Step 5. Pilot Test of Framework
- Step 6. Refinement and Finalization of Framework
- Discussion
- Conclusions
- References
- Abbreviations
- Appendixes
- Appendix A. AHRQ’s Seven Questions About the Development, Prioritization, and Presentation of Research Needs
- Appendix B. Step 2: Review of Current Practices of Evidence-based Practice Centers (EPCs)—Data Abstraction Form for Audit of Evidence Reports
- Appendix C. Step 2: Review of Current Practices of Evidence-based Practice Centers (EPCs)—Responses Obtained From EPCs
- Appendix D. Step 3: Review of Current Practices of Organizations Involved With Evidence Synthesis—List of Contacted Organizations
- Appendix E. Step 3: Review of Current Practices of Organizations Involved With Evidence Synthesis—Responses Obtained and Final Determinations of Formal Processes
- Appendix F. Step 4: Development of Framework—Instructions for Research Gaps Abstraction Worksheet
Suggested citation:
Robinson KA, Saldanha IJ, Mckoy NA. Frameworks for determining research gaps during systematic reviews. Methods Future Research Needs Report No. 2. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. HHSA 290-2007-10061-I.) AHRQ Publication No. 11-EHC043-EF. Rockville, MD: Agency for Healthcare Research and Quality. June 2011. Available at: www.effectivehealthcare.ahrq.gov/reports/final.cfm.
This report is based on research conducted by the Johns Hopkins University Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10061-I). The findings and conclusions in this document are those of the author(s), who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
The information in this report is intended to help health care researchers and funders of research make well-informed decisions in designing and funding research and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of scientific judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical research and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances.
This report may be used, in whole or in part, as the basis for research design or funding opportunity announcements. AHRQ or the U.S. Department of Health and Human Services endorsement of such derivative products or actions may not be stated or implied.
None of the investigators has any affiliations or financial involvement that conflicts with the material presented in this report.
- 1
540 Gaither Road, Rockville, MD 20850; www
.ahrq.gov
- NLM CatalogRelated NLM Catalog Entries
- Review Development of a framework to identify research gaps from systematic reviews.[J Clin Epidemiol. 2011]Review Development of a framework to identify research gaps from systematic reviews.Robinson KA, Saldanha IJ, McKoy NA. J Clin Epidemiol. 2011 Dec; 64(12):1325-30. Epub 2011 Sep 19.
- The future of Cochrane Neonatal.[Early Hum Dev. 2020]The future of Cochrane Neonatal.Soll RF, Ovelman C, McGuire W. Early Hum Dev. 2020 Nov; 150:105191. Epub 2020 Sep 12.
- Review Framework for Determining Research Gaps During Systematic Review: Evaluation[ 2013]Review Framework for Determining Research Gaps During Systematic Review: EvaluationRobinson KA, Akinyede O, Dutta T, Sawin VI, Li T, Spencer MR, Turkelson CM, Weston C. 2013 Feb
- Review Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update.[Methods Guide for Effectivenes...]Review Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update.Berkman ND, Lohr KN, Ansari M, McDonagh M, Balk E, Whitlock E, Reston J, Bass E, Butler M, Gartlehner G, et al. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. 2008
- Review Understanding Health-Systems' Use of and Need for Evidence To Inform Decisionmaking[ 2017]Review Understanding Health-Systems' Use of and Need for Evidence To Inform DecisionmakingWhite CM, Sanders Schmidler GD, Butler M, Wang Z, Robinson K, Mitchell MD, Berkman N, Henderson J, Fiordalisi C, Hartling L, et al. 2017 Oct
- Frameworks for Determining Research Gaps During Systematic ReviewsFrameworks for Determining Research Gaps During Systematic Reviews
Your browsing activity is empty.
Activity recording is turned off.
See more...