3.1. Strengths and limitations of the review

This review has several limitations. First, systematic literature review methodology was unable to identify all the latest innovations and practices from peer-reviewed literature because not all available tools and cutting-edge approaches are formally published. Second, there was a bias towards English language literature because of the greater representation of such publications in PubMed and Scopus. Third, it was not possible to explicitly appraise methodologies or qualities across the studies included in this review as studies were primarily of a descriptive nature. Finally, there was a gap in evidence regarding tools specifically used for incorporating health information into policy-making as well as for achieving behaviour change among policy-makers and health information producers. This may be because most authors do not make a distinction between tools or mechanisms for evidence in general and those for health information. In addition, information on how policy-makers actually employ knowledge translation tools and mechanisms to generate the intended outcomes was also limited. Hence, it was difficult to comprehensively list the support (e.g. technical, organizational, administrative, legislative, infrastructural) required to implement the mechanisms and tools.

Nevertheless, this review should prove useful for health policy stakeholders, including national policy-makers, knowledge brokers and health information producers, in understanding the variety of mechanisms and tools that may be used to incorporate health information into policy-making and, thus, improve the relevance and effectiveness of health policy. The taxonomy of tools and mechanisms, although not exhaustive or comprehensive, provides a good starting point for policy-makers to explore ways in which they may use health information, consider various options based on their policy requirements and build fit-for-purpose health information systems.

3.2. Additional considerations: health information quality

The quality of the health information is an important consideration when deciding whether to use it for policy-making. Health information tools and mechanisms will only have added value when the underlying health information and analysis is relevant, accurate, timely and interpretable. It is also important to understand its strengths and limitations. This will depend on the type of data collected and the availability of guidelines for policy-makers to assess the quality of health information in manuals and toolkits as well as in the literature (105). For example, Walker et al. suggest five questions that decision-makers should ask about disease burden estimates (106).

A common problem with health information is that it is typically aggregated from different sources and different geographical regions, and it often lacks harmonization and, therefore, requires careful interpretation. Further, data collection processes may not be uniform or complete. The key to interoperability is to establish standards for core datasets and data for interchange, quality and use (102). Hammond et al. propose some ways to accomplish this (107), as do Michelsen et al. (108).

These include development of robust and comparable health indicators, which requires innovation in tools and instruments; enhanced national capacity to collect, process, archive and share data; norms and standards for indicator definitions and computation; multiple independent analyses of data; and effective translation of results into policy (109). Crucially, countries need to take a comprehensive strategic approach to build systems that not only allow the generation and use of high-quality health information for the present but can also accommodate greater capacity in the future (105). Moreover, these systems should cover a full variety of data sources, ranging from routine service data and population-based surveys to civil registration systems with detailed data on births and deaths.

The WHO Regional Office for Europe is actively supporting its Member States in strengthening their health information systems through the European Health Information Initiative and associated support tools (67,104). Progress towards increased use of health information in national policy requires technical, legal and coordination strategies. Technical issues include standard data collection and harmonization; data integration from multiple sources; analysis, interpretation and reporting; quality assurance; creation of information platforms and products that are fit for purpose; maintenance of data confidentiality; and provision of a mechanism for information to feed into policy-making. Increasingly, some of these problems are circumvented by providing interactive access to a database whereby a user can obtain harmonized and packaged information on demand without having to access the raw data itself. Another important consideration is building capacity for leadership, oversight and coordination among personnel responsible for a health information system and provision of a strong knowledge translation mechanism in a country.

Capacity-building for evidence-informed policy-making is best conducted in a safe environment where experimentation and practice are allowed. Connecting traditional tools and mechanisms with, for example, the recent theory of gamification can lead to new behaviours in government and public policy (110). New approaches such as experimental learning through tools such as Knowledge Brokers, an innovative game-based workshop to support decision-making for public professionals (111), and open policy-making (112) through “policy labs” will undoubtedly bring about changes in health policies and in the delivery of health care.

The emergence of what is termed big data is currently the subject of a major ongoing debate in the field of evidence-informed policy-making, highlighting the need for linking medical, administrative, consumer, economic and personal activity data in an ethically (113) and legally sound manner.

In a recent briefing on big data for health and well-being (114), the WHO Regional Office for Europe considered it to include data sources and approaches that:

• enable better and/or new use of existing data sources either by innovations in methods of analysis or through integration/linking across data sources;
• contribute to the collection of information and enable data sharing for synthesis of data sources in health and feed into advanced analysis methodologies; and
• allow identification of new data sources and analysis methods that can support existing knowledge with new evidence and contexts.

Social media data for public policy (115) also gain traction where creating traditional datasets is expensive, slow or cumbersome, for example for surveillance (116), or for detecting areas of poor health service delivery (116). However, it will be essential to validate and refine such datasets to ensure their relevance and utility for policy-makers.

3.3. Tools and mechanisms to support the use of health information

The availability and use of health information for policy- and decision-making by public health authorities is fundamental to improving population health and reducing health inequality (104). Therefore, countries need to strengthen efforts to gather information that can contribute to health policy formulation and implementation. For the same reason, collection of, access to and incorporation of health information for policy-making are promoted by the WHO Regional Office for Europe through its European Health Information Initiative.

Both data held in national clinical databases and research based on such data have been used by policy-makers to inform policies, but this review has indicated that these databases are not currently used to their full potential, partly because many data custodians are clinicians, who may not be fully aware of the usefulness of databases for policy or are simply not interested in applications of the data outside clinical research. In addition, data privacy laws may also restrict the use of data outside the clinical setting. Use can be improved if there is greater awareness among custodians and policy-makers of this potential, through the signing of data-sharing agreements that maximize opportunities for using the data, and if necessary, by improving the coverage of data that are more useful for policy-making purposes.

This report has examined the available literature describing tools and mechanisms for using health information in policy-making. The relative strengths and applicability of these tools and mechanisms depend on the contexts and the stakeholders in question, which range from health information producers, brokers and networks through to the intended end-users.

The tools and mechanisms discussed are expected to be largely transferable even in countries of low and middle incomes because of their applicability in low-resource settings. However, context and environment will impact on the choice of tools to incorporate health information into policy-making. One or more tools should be selected to fit the local context and the purpose of the intended policy intervention. In the first place, the availability and quality of health information will depend on the underlying health information systems. Since most countries are implementing electronic systems to handle health information, potential problems may arise during transitioning from paper-based governance to electronic governance. In such cases, historical and current health information may not be effectively integrated, thus making it difficult to access and/or use relevant longitudinal health information. Importantly, the future success and effectiveness of evidence-informed policy-making will depend on an engaged community of information producers, brokers and users. Policy-makers’ attitudes towards health information will depend on whether they are surrounded by a culture of evidence-informed or ideology-based policy-making.

Other considerations include whether to institutionalize mechanisms (e.g. advisory committees, statistics task forces or health information system coordinating groups) or to use complementary tools and approaches. For example, several tools may be combined to achieve optimal benefit from existing health information. As a first step, health information can be presented in a form that is easily understood by stakeholders using packaging and application tools. Further, using dissemination and communication tools, this packaged health information can be brought to the attention of policy-makers, increasing the likelihood of them using the information. In addition, linkage and exchange tools could be applied to promote understanding of the health information among users and create mutual trust between stakeholders. Well-coordinated utilization of the tools described in this review, therefore, can be very effective for increasing the use of health information in evidence-informed policy-making. An example is provided in Box 10. This describes the packaging of health information into a synthesis report and its communication through discussions between stakeholders, which took place within steering committee and working group structures and, thus, created opportunities for linkage and exchange.

As a recent review of knowledge translation platforms in low- and middle-income country settings showed, evidence-informed health policy-making is enhanced by tools that strengthen exchange efforts (16). In particular, deliberative policy dialogues informed by evidence briefs created meaningful partnerships (linkages) between information producers and users. Exchange efforts were suggested as a way to encourage the involvement of local community members, who are often unwilling to share local data (16). Deliberative dialogues were also key to closing the gap between implementation research and health policy, using conversations about methodological rigour and feasibility constraints of evidence and its effective uptake by policy-makers (64).

While health information is increasingly available in an accessible format, its actual use by policy-makers is still largely undocumented. Nevertheless, health information has the potential to contribute to evidence-informed policy-making if packaged and communicated appropriately. This also requires “educating” decision-makers about the value of health information and strengthening assessments of health impacts of policies and actions on population health (117). Encouragingly, researchers and health information producers are starting to recognize the value of influencing policy priorities and the decision-making process and hence are making efforts to get the relevant data to policy-makers (118).

3.4. Policy options and implications

This synthesis report examined the use of health information for evidence-informed policy-making and found examples of how the use of tools and mechanism enabled better-quality information to be provided and disseminated for policy-making. Some examples showed how multiple tools could be used together or in sequence to derive the maximum benefit (16,53). Based on these findings, a number of options are set out for three stakeholder groups: health information producers, knowledge brokers and potential end-users of health information (e.g. policy-makers).

Health information producers might consider:

• ensuring that the health information produced meets the needs of, and is relevant to, the end-users by engaging with them on a continuous basis;
• establishing personal contact with brokers and end-users to build trust because trusted partnerships increase the prospects of health information being considered reliable and then used;
• fostering the use of packaging tools in conjunction with dissemination and communication tools to achieve the highest benefit from health information; importantly, the better the coordination between the tools, the greater the benefit is likely to be; and
• adding value to health information packaging by using application tools such as models and simulations to fill gaps and present scenarios.

Knowledge brokers might consider:

• establishing relationships with and acting as an active link between producers and users of health information to build a value chain and bring health information into immediate practical use;
• presenting suitably packaged health information, both on their own initiative (push) as well as when prompted by end-users (pull); and
• advocating a key role for knowledge brokers in increasing the capacity of policy-makers to use health information for decision-making.

Health information users might consider:

• ensuring effective strategic oversight over information integration and production;
• establishing personal contacts with health information providers to learn about available health information and its potential uses, and deepen personal understanding;
• informing health information providers about health information needs and working with them to identify and fill remaining information gaps; and
• institutionalizing links with health information providers to ensure an adequate supply of health information in terms of coverage and timeliness.

The most effective use of health information is when all stakeholders interact and communicate to ensure that the best available evidence is used to support the choice of policy options.