Framework for Determining Research Gaps During Systematic Review: Evaluation
Methods Research Reports
Authors
Investigators: Karen A Robinson, PhD, Oluwaseun Akinyede, MPH, Tania Dutta, MS, MPP, Veronica Ivey Sawin, BA, Tianjing Li, MD, PhD, Merianne Rose Spencer, BS, Charles M Turkelson, PhD, and Christine Weston, PhD.Affiliations
Structured Abstract
Background:
Research gaps prevent systematic reviewers from making conclusions and, ultimately, limit our ability to make informed health care decisions. While there are well-defined methods for conducting a systematic review, there has been no explicit process for the identification of research gaps from systematic reviews. In a prior project we developed a framework to facilitate the systematic identification and characterization of research gaps from systematic reviews. This framework uses elements of PICOS (Population, Intervention, Comparison, Outcomes, Setting) to describe the gaps and categorizes the reasons for the gaps as (A) insufficient or imprecise information, (B) biased information, (C) inconsistent or unknown consistency results, and/or (D) not the right information.
Objective:
To further develop and evaluate a framework for the identification and characterization of research gaps from systematic reviews.
Methods:
We conducted two types of evaluation: (1) We applied the framework to existing systematic reviews, and (2) Evidence-based Practice Centers (EPCs) applied the framework either during a systematic review or during a future research needs project (FRN). EPCs provided feedback on the framework using an evaluation form.
Results:
Our application of the framework to 50 systematic reviews identified about 600 unique research gaps. Key issues emerging from this evaluation included the need to clarify instructions for dealing with multiple comparisons (lumping vs. splitting) and need for guidance on applying the framework retrospectively. We received evaluation forms from seven EPCs. EPCs applied the framework in 8 projects, five of which were FRNs. Challenges identified by the EPCs led to revisions in the instructions including guidance for teams to decide a priori whether to limit the use of the framework to questions for which strength of evidence has been assessed, and the level of detail needed for the characterization of the gaps.
Conclusions:
Our team evaluated a revised framework, and developed guidance for its application. A final version is provided that incorporates revisions based on use of the framework across existing systematic reviews and feedback from other EPCs on their use of the framework. Future research is needed to evaluate the relative costs and benefits of using the framework, for review authors and for users of the systematic reviews.
Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2007-10061-I. Prepared by: Johns Hopkins University Evidence-based Practice Center, Baltimore, MD
Suggested citation:
Robinson KA, Akinyede O, Dutta T, Sawin VI, Li T, Spencer MR, Turkelson CM, Weston C. Framework for Determining Research Gaps During Systematic Review: Evaluation. Methods Research Report. (Prepared by Johns Hopkins University Evidence-based Practice Center under Contract No. 290-2007-10061-I.) AHRQ Publication No. 13-EHC019-EF. Rockville, MD: Agency for Health care Research and Quality. February 2013. www.effectivehealthcare.ahrq.gov/reports/final.cfm.
This report is based on research conducted by the Johns Hopkins University Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10061-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients.
This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.
None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report.
- 1
540 Gaither Road, Rockville, MD 20850; www
.ahrq.gov