NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Structured Abstract
Objectives:
Evidence presented in systematic reviews informs the development of healthcare practice, guidelines, and policy. The inherent complexity and quantity of data in systemic reviews may impede understanding and use in decision processes, but little evidence exists on transforming large volumes of these data into accessible formats for end users. The objectives of this Evidence-based Practice Center (EPC) pilot project were (1) to identify the information needs of health systems guideline/protocol developers; (2) to assess existing, off-the-shelf software or Web platforms that would allow creation of interactive presentations of systematic review data in formats that would address the identified needs, and (3) to test the ability of selected software/platforms to make the large amount of data included in a recent systematic review of chronic pain management more accessible for decision makers at Oregon Health & Science University.
Methods:
To develop and test alternative formats for dissemination, we assessed stakeholder needs through qualitative interviews with a department director and four health system content experts. We reviewed interview notes and identified the key themes in team discussion, and arrived at consensus. We then conducted a literature search regarding core functionalities desired in evidence summaries and systematic reviews, as described by the content experts. Next, we compared recommendations from the content experts and the literature search to several existing software tools in order to select two tools for the pilot test. We imported data from a recent systematic review on chronic pain into the selected tools to mock up example outputs. Finally, we solicited reactions from the department director and six health system content experts (four of whom were interviewed initially) on the mocked-up report examples in terms of accessibility and utility, and we based recommendations for next steps on these assessments and our experience.
Results:
The key theme that emerged from the initial interviews with content experts was the need for two core functionalities: the ability to drill down from a general overview to specific more information and the ability to select subsets of evidence from a larger review. We identified two tools that provided these functions and that met our other criteria: MAGICapp is a platform for evidence summaries; Tableau is a data management and visualization tool. MAGICapp required less time and skill to mock up, as the data were entered manually into the Web-based platform, while Tableau required more time and a staff member with knowledge of informatics such as the ability to set up the relational databases for the dashboard. MAGICapp parameters required the data output to follow the structure of the pain review and allowed users to drill down to granular detail; Tableau allowed users to explore evidence without adhering to the organization of the review, but could not provide the granularity found in MAGICapp. Neither of the two tools we tested were able to fulfill both core functionalities, drilling down to specific study data and reviewing subsets of evidence outside the confines of the organization of the pain review. The second round of health system content expert interviews provided positive feedback on the products, aesthetically as well as for their potential functionality. Respondents perceived Tableau as ideal for content experts reviewing data, as the functionality allows users to query the data in multiple ways. Respondents perceived MAGICapp as the better choice for multidisciplinary groups or decision makers less familiar with the data, given the tool’s organized structure and capacity for explanatory text.
The two key themes from the second-round interviews and our evaluation were (1) the need for the learning health system administrators to consider the level of expertise of the end users, as those with more or less familiarity with a set of data may require the granularity of MAGICapp or the freedom of Tableau and (2) the need for EPCs to test one or both prototype in an actual review from the beginning in order to accurately estimate what additional staff time and expertise is needed to prepare, import, and manage data beyond the traditional EPC report formats.
Conclusions:
The results of this “proof-of-concept” prototype development demonstrate that existing tools could be used to make large systematic reviews more accessible and usable. However, an individual tool may not have the capacity to provide all desired functionalities, and each tool has differing requirements for time, data management, and staff expertise. To better understand the actual time required, the data storage needs, implications for EPCs and learning health systems, and issues related to Section 508 accessibility standards and government data rights, we recommend a follow-on pilot be conducted to allow systematic review teams to test these tools as integrated components of one or a small number of future reviews. This follow-on research would provide realistic data on the resources needed to generate systematic reviews in alternative formats and allow further assessment of whether these formats can increase uptake of EPC reports within learning health systems.
Contents
Suggested citation:
Totten AM, Smith C, Dunham K, Jungbauer RM, Graham E. Improving Access to and Usability of Systematic Review Data for Health Systems Guidelines Development. Methods Research Report. (Prepared by the Pacific Northwest Evidence-based Practice Center under Contract No. 290-2015-00009-I.) AHRQ Publication No. 19-EHC010-EF. Rockville, MD: Agency for Healthcare Research and Quality. February 2019. Posted final reports are located on the Effective Health Care Program search page. DOI: https://doi.org/10.23970/AHRQEPCMETHENGAGEIMPROVING.
This report is based on research conducted by the Pacific Northwest Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2015-00009-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report.
The information in this report is intended to help healthcare decision makers—patients and clinicians, health system leaders, and policymakers, among others—make well-informed decisions and thereby improve the quality of healthcare services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients.
This report is made available to the public under the terms of a licensing agreement between the author and the Agency for Healthcare Research and Quality. This report may be used and reprinted without permission except those copyrighted materials that are clearly noted in the report. Further reproduction of those copyrighted materials is prohibited without the express permission of copyright holders.
AHRQ or U.S. Department of Health and Human Services endorsement of any derivative products that may be developed from this report, such as clinical practice guidelines, other quality enhancement tools, or reimbursement or coverage policies may not be stated or implied.
Persons using assistive technology may not be able to fully access information in this report. For assistance contact vog.shh.qrha@eraChtlaeHevitceffE.
- NLM CatalogRelated NLM Catalog Entries
- Review Improving Health Systems’ Access to High-Quality Evidence: AHRQ EPC 2018 Pilot Projects Summary[ 2019]Review Improving Health Systems’ Access to High-Quality Evidence: AHRQ EPC 2018 Pilot Projects SummaryFiordalisi CV, Borsky A, Chang S, Gerrity M, Schmidt AM, Guise JM. 2019 Aug
- Review Understanding Health-Systems' Use of and Need for Evidence To Inform Decisionmaking[ 2017]Review Understanding Health-Systems' Use of and Need for Evidence To Inform DecisionmakingWhite CM, Sanders Schmidler GD, Butler M, Wang Z, Robinson K, Mitchell MD, Berkman N, Henderson J, Fiordalisi C, Hartling L, et al. 2017 Oct
- The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol.[JBI Database System Rev Implem...]The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol.Sinclair P, Kable A, Levett-Jones T. JBI Database System Rev Implement Rep. 2015 Jan; 13(1):52-64.
- Student and educator experiences of maternal-child simulation-based learning: a systematic review of qualitative evidence protocol.[JBI Database System Rev Implem...]Student and educator experiences of maternal-child simulation-based learning: a systematic review of qualitative evidence protocol.MacKinnon K, Marcellus L, Rivers J, Gordon C, Ryan M, Butcher D. JBI Database System Rev Implement Rep. 2015 Jan; 13(1):14-26.
- Review Use of a Clinical Pathway To Facilitate the Translation and Utilization of AHRQ EPC Report Findings: FY2018 Dissemination Pilot: Early Diagnosis, Prevention, and Treatment of Clostridium difficile[ 2018]Review Use of a Clinical Pathway To Facilitate the Translation and Utilization of AHRQ EPC Report Findings: FY2018 Dissemination Pilot: Early Diagnosis, Prevention, and Treatment of Clostridium difficileFlores E, Jue JJ, Giradi G, Schoelles K, Umscheid CA. 2018 Dec
- Improving Access to and Usability of Systematic Review Data for Health Systems G...Improving Access to and Usability of Systematic Review Data for Health Systems Guidelines Development
- LOC114384428 [Glycine soja]LOC114384428 [Glycine soja]Gene ID:114384428Gene
Your browsing activity is empty.
Activity recording is turned off.
See more...