KEY POINTS
Health systems are sensitive to political, economic and social factors that occur locally, nationally and internationally.
Context is determined by situational factors, structural factors, cultural factors and international or exogenous factors.
Implementation and context are highly interconnected, with implementation of evidence-based interventions always taking place within a given context, which in turn influences how the implementation takes place.
Contextual information is fundamental for policy planning and development, but is often stripped away in systematic reviews.
Reporting rich contextual information in primary studies helps to enrich the usefulness of subsequent systematic reviews.
4.1. INTRODUCTION
This chapter introduces the notion of context and its relationship to health policies and systems. It then considers how context may be taken into account in systematic reviews and evidence syntheses of health policy and systems research (HPSR) questions. The issue of context is closely related to the methods described in both Chapter 3 and the methods commentary on realist reviews, elsewhere in this volume; it is also fundamental to reviews of complex health interventions, as described and discussed in Chapter 5.
4.2. THE IMPACT OF CONTEXT ON HEALTH POLICY AND SYSTEMS INTERVENTIONS
Health systems exist within, and are sensitive to, particular contexts (1–4). In other words, health systems are sensitive to political, economic and social factors that occur locally, nationally and internationally (5). Health systems interact “with the population and with the specific contexts in which they are embedded” (2). They will react and adapt to changes in the immediate environment (the local context), such as an epidemic or a major environmental incident, but they are also sensitive to changes in the national and global environments, such as major policy shifts, changes in technology or politics, and fiscal changes (6). These systemic factors have been described as falling into four main categories (5):
Situational factors, transient or impermanent occurrences, such as a flood, which may lead to an increase in waterborne disease and thus bring attention to issues of sanitation.
Structural factors, elements that are relatively unchanging or slow to change, such as a country’s political system or its level of inequity, which can determine the extent to which residents depend on public health. High levels of wealth or poverty are likely to also affect a country’s disease profile and other characteristics.
Cultural factors, including pervasive belief systems, such as beliefs about women, hierarchy and ethnicity, which can shape local policies. For example, in some countries where the dominant religion does not favour abortion, this is mirrored in health policies that do not favour women’s right to access abortion services either.
International or exogenous factors, whereby nation states, although independent, may also be subject to interdependencies. For example, if one country has managed to control a particular infectious disease (such as malaria), it may be concerned that a neighbouring country has not done the same, and therefore may seek to influence how its neighbour manages the disease.
One of the major contemporary changes facing health systems is the shift in focus from the Millennium Development Goals to the Sustainable Development Goals. Although the decision to adopt the Sustainable Development Goals was made at the United Nations, within the global context, and national governments have agreed to these goals, it is health systems managers at the front line of health care delivery, operating within their local contexts, who will have to adapt their services to reflect this shift within the global context (7). To take another example, the World Health Organization has responsibility for population health around the globe, that is, the global health system. As such, one of its functions is to set policies and develop guidelines to be used by countries across the globe, in an effort to ensure the highest attainable level of health for all people (8). These policies and guidelines, which are agreed to as being of global good, or good for the global health system, are then taken up by national governments, which ensure that they are implemented by health workers at the front line of delivery of care.
This context sensitivity contributes to the diversity of varied health systems, and also to their complexity (6, 9). Another contributor to diversity is the fact that health systems are path dependent, that is, they are products of historical processes that have shaped them. For example, if a country has a colonial past, then its health system will be shaped by that past, and it may continue to organize services in the same manner as during the colonial period (10). Health systems are also social institutions, in that they are both a product of the society in which they occur and an influence on that society (6, 11, 12). The values and principles held by the society are likely to be reflected in the health system; for example, if a society values social equity, it may have a health system that favours universal access to health care (2, 6, 11).
Health systems activity takes place at different levels: the macro level (global and national health systems), the meso level (local or district health system) and the micro level (individual health facilities up to the patient–provider interface) () (13–15).
The different levels of health systems.
In Europe in particular, reference is also made to a supranational level, because the legal and policy environment created by the European Union affects the health policy environment of European member states (16, 17). The specificities, roles and functions of each of these levels are also sensitive to context (6) and may “vary quite substantially between countries, depending on the type and level of decentralization and autonomy of regions, provinces, or districts” (2). At the micro level, there are further sublevel divisions, in that health service delivery can take place within communities and at primary health care facilities, as well as at secondary and tertiary care hospitals. Each of these sublevels of front-line service delivery can be seen as a context, with its own systems and contextual interactions. The immediate environment of health services delivery is also sometimes referred to as the setting, although the term “setting” is often used interchangeably with the word “context”. The difference between setting and context is described in Box 4.1.
Reviews of health policy and systems research questions need to take context into account to enhance the relevance and usability of the research outputs.
BOX 4.1DISTINGUISHING CONTEXT AND SETTING
The literature is not always clear on the difference between context and setting, with some authors using the terms interchangeably, and others distinguishing the two. The following definitions by Pfadenhauer and colleagues (14, 18) are appealing:
Context is ‘‘conceptualized as a set of characteristics and circumstances that consist of active and unique factors that surround the implementation. As such it is not a backdrop for implementation but interacts, influences, modifies and facilitates or constrains the intervention and its implementation. Context is usually considered in relation to an intervention or object, with which it actively interacts. A boundary between the concepts of context and setting is discernible: setting refers to the physical, specific location in which the intervention is put into practice. Context is much more versatile, embracing not only the setting but also roles, interactions and relationships’’ (18).
Setting ‘‘usually has a narrower focus. It often refers to the place where an intervention is delivered (e.g. primary care setting) or the circumstances of an intervention (e.g. low-income setting)’’ (14).
4.3. IMPLEMENTING HPSR EVIDENCE IN CONTEXT
Implementation of evidence-based interventions is the process of bringing into use practices that have been proven effective through research evaluation. This process occurs within health systems, and thus is also sensitive to the health policy and systems context in which the implementation occurs (14, 18, 19). Pfadenhauer and colleagues have explored the relationship between implementation and context, arguing that these are highly interconnected, with implementation of evidence-based interventions always taking place within a given context, which influences how the implementation takes place (14). Tomoaia-Cotisel and colleagues support this argument (15), further claiming that understanding context is important to the replication of research, because contextual knowledge is important to interpreting and applying the findings. In addition, before the findings can be applied, the evidence-based intervention needs to be adapted for the context in which it will be implemented. Therefore, the adapters need information about the original context or contexts in which the intervention was research-tested, so as to determine what changes might be needed for the intervention to work in the new context. To think about this more practically, imagine an intervention that works when tested in a research study in rural northern Sweden, where, although people are living remotely, they have good infrastructural access. Now imagine trying to implement that same intervention in rural Sudan, where access to resources and infrastructure is really poor. Or, to use a less extreme example, consider implementing the same intervention in rural northern Sweden and rural Alaska, where the environmental conditions may be similar, but the health systems (of Sweden and the United States, respectively) have vast differences. Thus, interventions that work in one context cannot simply be transported to another context, without some consideration and potential adaptation. This consideration is made easier when researchers offer details about the original context and setting in which the intervention was tested.
4.4. ADDRESSING THE CHALLENGE OF CONTEXT TO SYSTEMATIC REVIEWS OF HEALTH POLICY AND SYSTEMS INTERVENTIONS
The field of HPSR seeks to inform policy and implementation through evidence (hence the need for policy relevance), as outlined in Chapter 1. However, traditional systematic reviews examining the effectiveness of interventions have been criticized for being too reductionist and for not taking context into account (15, 20–22). These limitations present a challenge in performing systematic reviews of HPSR questions because contextual information is fundamental for policy planning and development, for instance, in assisting decision-makers to decide whether certain policy options are applicable to their context and setting (23–25). Greenhalgh has argued that in these traditional systematic reviews, the “technical process of stripping away all but the bare bones of a focused experimental question removes what practitioners and policymakers most need to engage with: the messy context in which people get ill, seek health care (or not), receive and take treatment (or not), and change their behaviour (or not)” (20). Key to these criticisms is that after stripping away the context, the researchers can only say whether an intervention works or not; they cannot explain why this is, why the intervention works or not (18, 26, 27). Systematic reviews that strip away context may be perceived as lacking relevance to policy- and decision-makers seeking information that will help them adapt the interventions reviewed to their local context (15, 21, 28). The lack of contextual relevance is, in turn, offered as a potential explanation for why policy- and decision-makers may not routinely use systematic reviews as part of their decision-making process (28, 29).
A closer examination of context is recommended, because the context in which an intervention takes place will act as a mediator in the success or failure of the intervention; therefore, policy- and decision-makers need to know why a given intervention works in one place yet may fail in another (15, 18, 20, 22, 26, 27). Thus, reviews of HPSR questions need to take context into account to enhance the relevance and usability of the research outputs. But review authors cannot do this on their own. A clearer description of context in reviews is reliant on a clearer description of context in primary studies (see Box 4.2).
4.5. METHODS PAPER
Although systematic reviewers may be keen to focus on context in their reviews, the authors of the primary studies upon which systematic reviews and other types of evidence synthesis are based may not have paid attention to the value of reporting on contextual factors that might affect the intervention
BOX 4.2A PLEA FOR CONTEXT-RICH PRIMARY STUDIES TO SUPPORT CONTEXT-RICH SYSTEMATIC REVIEWS AND EVIDENCE SYNTHESIS
The challenge of including context starts at the level of primary studies. If systematic reviewers are to take context into account, they need primary studies that do the same. Evidence synthesis becomes difficult when the primary studies included in the syntheses do not offer sufficient contextual information (15). Reporting rich contextual information in primary studies helps to enrich the usefulness of systematic reviews (15). This can enable these reviews to include an understanding of contextual factors that will, in turn, allow decision-makers to transfer knowledge gained from interventions implemented and evaluated in one context to the implementation of such interventions in other contexts (15).
Glenton, Lewin & Scheel struggled to find qualitative studies conducted alongside experimental trials, despite actively searching for them (30). This team had conducted an effectiveness review in which the results were promising but heterogeneous. Thus, they sought qualitative studies that had been conducted alongside the included trials, with the aim of using this contextual information to help explain the heterogeneity. Yet they found that 83% of the included trials either had no linked qualitative studies, or the qualitative studies that did exist were not accessible. Ultimately, they were able to access corresponding qualitative studies for only 17% of the included trials, and even then they found that the descriptions of the methods and the qualitative results were often sparse. They therefore concluded that qualitative studies conducted alongside trials hold some promise for explaining heterogeneity, by offering insight into the trial intervention context, but that too few of these studies are being conducted for their full promise to be realized.
In response to the poor reporting of context in primary studies, Tomoaia-Cotisel and colleagues developed a tool for the researchers with whom they were working, to be used in collecting contextual information in primary studies using quantitative, qualitative and mixed-methods designs (15). This tool is based on their experience of collecting contextual information across 14 research teams. When publishing their findings, each of the 14 teams added the contextual information that they had collected, as appendices to the main article. Investigators performing primary studies would be well advised to consider using a tool such as that developed by Tomoaia-Cotisel and colleagues, or developing their own tool. The use of such tools could improve the richness of their recording of the context in which their study took place, and in which the intervention that they are evaluating was implemented. Furthermore, reporting guidelines for journal articles have been extended to encourage authors of primary quantitative studies to report context in more detail. Enhanced reporting on context in evidence syntheses is therefore contingent on the authors of primary studies expanding on their context reporting.
(
15,
21). Pfadenhauer and colleagues have some advice to offer in this regard (
14,
18,
19). These authors developed the Context and Implementation of Complex Interventions (CICI) framework, as a means to guide the inclusion of context and implementation in health technology assessments and systematic reviews of complex interventions (
18,
19). The guidance offered is intended for use by review authors across the range of quantitative, qualitative and mixed-methods reviews. Rather than presenting a new method, these authors offer a checklist that systematic reviewers may use to ensure that their reviews become more sensitive to the role that context plays in the implementation of the intervention or phenomenon being studied. They offer clear descriptions of all the context domains listed, as well as examples of context-sensitive data extraction forms that can be used in reviews of quantitative and qualitative studies. The authors also turn their attention to how an intervention may be affected by the way in which it is implemented. The checklist presented in their most recent article (
19) details aspects of the implementation process, including implementation theory, process, strategy, agents and outcomes. This information is supported by seven appendices, which include lists of the articles that contributed to their framework, further elaboration on the framework, data extraction forms, a guide to expert consultation and a worked example of the use of the framework. This article is available on an open-access basis:
Pfadenhauer
LM, Gerhardus
A, Mozygemba
K, Lysdahl
KB, Booth
A, Hofmann
B, et al. Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implement Sci. 2017;12:21. doi: 10.1186/s13012-017-0552-5 [PMC free article: PMC5312531] [PubMed: 28202031] [CrossRef] (19).
Attention to context in systematic reviews is further discussed in Chapter 3, concerning methods; Chapter 5, concerning reviews of complex interventions; and the methods commentary on realist reviews. In particular, Chapter 3 introduces the concepts of quantitative, qualitative and mixed-methods reviews. The CICI framework could be considered a companion to any of these review types, including qualitative reviews, although the latter tend to be inherently more context-focused than quantitative reviews. In quantitative reviews in particular, reviewers could consider using the CICI framework to inform a narrative reporting of the information on context that has been extracted from primary studies, so as to shed further light on the synthesis of quantitative outcomes.
4.6. SYSTEMATIC REVIEWS TAKING CONTEXT INTO ACCOUNT: EXAMPLES
Review example
Liu and colleagues (31) explicitly set out to take context into account in their systematic review exploring interventions to attract and retain health workers in underserved rural areas. In exploring the literature, they found that studies and reviews of interventions presented contradictory evidence, yielding a complex picture of the effectiveness of the interventions. In response, the review authors recognized that these interventions had been developed and implemented in various contexts through different processes, and they felt that this heterogeneity might explain the variation in intervention effectiveness. Thus, they set out to conduct a review that would take this variation into account, with the objective of identifying contextual factors that policy-makers should consider when they design and implement interventions. Context was therefore taken into account in the design of the search terms, the study selection, and the collection and analysis of data. Using this approach, the authors were able to offer review findings that addressed contextual factors at the macro, meso and micro levels of the health systems. These factors included the fiscal capacity of a country or organization, decentralization of the health system and legislative processes. In making these contextual factors explicit, the review authors enhanced policy-makers’ ability to consider how the intervention might work in their own contexts.
Liu
X, Dou
L, Zhang
H, Sun
Y, Yuan
B. Analysis of context factors in compulsory and incentive strategies for improving attraction and retention of health workers in rural and remote areas: a systematic review. Hum Resour Health. 2015;13:61. doi: 10.1186/s12960-015-0059-6 [PMC free article: PMC4508764] [PubMed: 26194003] [CrossRef] (31).
Review protocol example
Like Liu and colleagues (31), Belrhiti and colleagues (32) have embarked upon an HPSR systematic review that explicitly takes context into account. These authors very specifically focus on the meso level of the health system, by exploring interventions to improve district health systems management and leadership. In their protocol (32), they explain how exploring the effectiveness of such interventions is a primary objective of the review, whereas exploring contextual factors that enable or constrain the interventions is a secondary objective. This secondary objective is justified by the authors’ understanding of the interventions as multifaceted and complex in nature, and their observation that the interventions are “implemented in social systems characterized by human agency, uncertainty, and unpredictability” (32). The authors therefore begin by using a logic model (see Chapter 5 in this Methods Guide, on performing reviews of complex interventions, in particular section 5.5) to illuminate key contextual issues that may affect the interventions. The authors also describe how they will use “best fit” framework synthesis (described in more detail in Box 4.3) to analyse organizational policies and procedures, to allow them to interpret what is happening in specific contexts. They propose using this framework for the qualitative studies included in the review. These authors therefore acknowledge context in the overall focus of their review, in their inclusion criteria and in the analytical process of the review.
Belrhiti
Z, Booth
A, Marchal
B, Verstraeten
R. To what extent do site-based training, mentoring, and operational research improve district health system management and leadership in low- and middle-income countries: a systematic review protocol. Syst Rev. 2016;5:70. doi: 10.1186/s13643-016-0239-z [PMC free article: PMC4847191] [PubMed: 27116915] [CrossRef] (32).
The “best fit” framework mentioned above may be unfamiliar to some readers. Although Belrhiti and colleagues use the framework, they do not offer a detailed explanation of it. Box 4.3 includes a description of this framework, along with the antecedent thematic and framework synthesis approaches for comparison. These approaches are best considered in relation to the literature on qualitative methods, introduced in Chapter 3.
BOX 4.3THREE APPROACHES TO ANALYSIS IN SYSTEMATIC REVIEWS OF QUALITATIVE STUDIES
Analysis within reviews of qualitative studies (also known as qualitative evidence synthesis) follows the same principles as analysis of data in primary studies. Such analysis can be either inductive (whereby themes, codes and categories emerge from the data) or deductive (whereby themes, codes and categories are chosen a priori, before the analysis starts).
Thematic synthesis (33, 34)
Thematic synthesis involves analysing data from primary qualitative studies in an inductive manner, the approach commonly used for many primary studies. Using this approach, reviewers code the primary studies (sometimes just the results section, but often the discussion and conclusions too), line by line, as if coding a transcript of an interview or field notes from a qualitative observation. This coding can then lead to the development of descriptive themes (analysis at the manifest or superficial level) and analytic themes (when the reviewers go deeper, trying to identify patterns, relations and explanations in the data, thus analysing at the latent level). This approach is appropriate when the reviewers are doing an exploratory study, wanting to see what emerges from the data, and when they hold no prior assumptions about what they might find in relation to the review question. Although this approach can lead to a rich and nuanced analysis, the downside is that exploring the data in depth can take a very long time. However, in HPSR, when working with policy-makers who are seeking quick answers to prespecified questions and challenges, reviewers may not have the luxury of the time required by such an approach. Another challenge with this approach, as with thematic analysis of qualitative primary studies, is that the process of arriving at codes, themes and categories is often intuitive, with many of the links and explanations being made in the researcher’s or the reviewer’s mind, rendering transparency of the process hard to achieve.
Framework synthesis (34, 35)
Framework synthesis of qualitative studies follows the same principles as framework analysis of primary studies, whereby a deductive approach is used to analyse data from primary studies included in systematic reviews. With this approach, a tentative framework of themes or concepts is identified in advance. This up-front framework could be developed through the reviewers’ own understanding of the issue being reviewed, it could be developed from the literature on the subject, or it could be developed in conjunction with the requesters of the review (such as policy-makers, health systems managers or health policy lobbyists). As is often the case in reviews of HPSR questions, the requesters of the review are likely to have a predefined set of questions and issues that they would like to have addressed. Using the predefined framework, the reviewers can ensure that they actively seek out data to answer those questions. Having a predefined framework that is developed in collaboration with the review requesters can also be more transparent than trying to explain how themes have emerged from the data. The predefined framework is also useful for combining data from multiple study types, because data about the same issue can be grouped under the same predefined theme and then compared from there. One of the dangers of this approach is that reviewers may become attached to their predefined themes or categories and may be unwilling to consider data that do not fit within this framework. Those data could easily be lost, with the attendant risk that contradictory data or new insights become “buried”, even if there was no attempt to hide the data.
“Best fit” framework synthesis (35–37)
“Best fit” framework synthesis combines thematic and framework synthesis, using both a deductive and an inductive approach to analyses. With this approach, the authors begin by systematically searching the literature for a theory or framework that would best align with their research question. In their search, they explicitly take context into account. For example, as described in section 4.6 of this chapter, Belrhiti and colleagues (32) are using the “best fit” approach to answer the question “To what extent do site-based training, mentoring and operational research improve district health system management and leadership in low- and middle-income countries (LMICs)?” Their question contains both interventions – site-based training, mentoring and operational research – and two levels of settings – district health systems and LMICs. Therefore, in their approach, they will look for explanatory theories and frameworks related both to the interventions and to how these interventions operate in the identified settings (this concept could also be incorporated into a logic model; see Chapter 5 on performing reviews of complex interventions, in particular section 5.5). The authors will also simultaneously search for primary studies that meet the intervention and setting criteria. They will then develop a tentative framework, based on the “best fit” of what they find in the literature, and as they analyse the context-sensitive primary studies, they will fit the data from these studies into the predefined framework. However, the originators of the “best fit” approach recognized that it would be unlikely for all of the data from the primary studies to fit within such a predefined framework; they furthermore recognized that the “best fit” theories that reviewers find are likely to be generic and not context specific (36, 37). Thus, using this approach, reviewers will also code data from the context-specific primary studies inductively, looking at what new themes and categories emerge. These new themes and categories will then be compared and translated into the predefined framework, bringing context-specific data and insights to what might originally have been a generic framework. From there, the reviewers can develop a new, higher-level framework that brings together the predefined theory and framework with the intervention and context-specific data. Thus, Belrhiti and colleagues (32), as well as the originators of the approach, suggest that this approach is context sensitive. However, the originators argue that this approach can only be used where predefined theories or frameworks exist. In instances where the reviewers have little advance knowledge about the topic, a more inductive approach remains preferable.
Thomas
J, Harden
A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45. doi: 10.1186/1471-2288-8-45 [PMC free article: PMC2478656] [PubMed: 18616818] [CrossRef] (33).
Gough
D, Oliver
S, Thomas
J. An introduction to systematic reviews. Los Angeles (CA): Sage Publications; 2012 (34).
Dixon-Woods
M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011;9:39. doi: 10.1186/1741-7015-9-39 [PMC free article: PMC3095548] [PubMed: 21492447] [CrossRef] (35).
Carroll
C, Booth
A, Leaviss
J, Rick
J. “Best fit” framework synthesis: refining the method. BMC Med Res Methodol. 2013;13:37. doi: 10.1186/1471-2288-13-37 [PMC free article: PMC3618126] [PubMed: 23497061] [CrossRef] (36).
Carroll
C, Booth
A, Cooper
K. A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11:29. doi: 10.1186/1471-2288-11-29 [PMC free article: PMC3068987] [PubMed: 21410933] [CrossRef] (37).
4.7. CONCLUSION
Contextually rich systematic reviews and evidence synthesis may better support health policy and systems decision-makers, as they consider how to apply the evidence for implementation in their settings. Contemporary developments in evidence synthesis methods can enable reviewers to produce such contextually rich reports. Production of such reviews is supported even further when reviewers are able to extract contextually rich data from the primary studies included the final systematic reviews.
REFERENCES
- 1.
Bosch-Capblanch
X, Lavis
JN, Lewin
S, Atun
R, Røttingen
JA, Dröschel
D, et al. Guidance for evidence-informed policies about health systems: rationale for and challenges of guidance development. PLoS Med. 2012;9(3):e1001185. doi: 10.1371/journal.pmed.1001185. [
PMC free article: PMC3295823] [
PubMed: 22412356] [
CrossRef]
- 2.
Damme
WV, Pirard
M, Assefa
Y, Van Olmen
J. How can disease control program contribute to health system strengthening in sub-Saharan Africa: a concept note for engaging in the global debates on health systems strengthening. Washington (DC): World Bank; 2010 (Health, Nutrition and Population Discussion Paper,
http://hdl.handle.net/10986/13611, accessed 19 June 2018).
- 3.
- 4.
- 5.
Buse
K, Mays
N, Walt
G. Making health policy. Berkshire (England): Oxford University Press; 2005.
- 6.
Balabanova
D, Mills
A, Conteh
L, Akkazieva
B, Banteyerga
H, Dash
U, et al. Good Health at Low Cost 25 years on: lessons for the future of health systems strengthening. Lancet. 2013;381(9883):2118–33. doi: 10.1016/S0140-6736(12)62000-5. [
PubMed: 23574803] [
CrossRef]
- 7.
- 8.
- 9.
Atun
R. Health systems, systems thinking and innovation. Health Policy Plan. 2012;27 Suppl 4:iv4–8. doi: 10.1093/heapol/czs088. [
PubMed: 23014152] [
CrossRef]
- 10.
Wilsford
D. Path dependency, or why history makes it difficult but not impossible to reform health care systems in a big way. J Public Policy. 2008;14(3):251–83. doi: 10.1017/S0143814X00007285. [
CrossRef]
- 11.
Gilson
L. Trust and the development of health care as a social institution. Soc Sci Med. 2003;56(7):1453–68. doi: 10.1016/S0277-9536(02)00142-9. [
PubMed: 12614697] [
CrossRef]
- 12.
Freedman
LP. Achieving the MDGs: health systems as core social institutions. Development. 2005;48(1):19–24. doi: 10.1057/palgrave.development.1100107. [
CrossRef]
- 13.
Gilson
L. Part 1: Introduction to health policy and systems research. In: Gilson
L, editor. Health policy and systems research: a methodology reader. Geneva: World Health Organization, Alliance for Health Policy and Systems Research; 2012:19–40 (
http://www.who.int/alliance-hpsr/alliancehpsr_reader.pdf, accessed 8 June 2018).
- 14.
Pfadenhauer
LM, Mozygemba
K, Gerhardus
A, Hofmann
B, Booth
A, Lysdahl
KB, et al. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes. 2015;109(2):103–14. doi: 10.1016/j.zefq.2015.01.004. [
PubMed: 26028447] [
CrossRef]
- 15.
- 16.
Greer
SL. Uninvited Europeanization: neofunctionalism and the EU in health policy. J Eur Public Policy. 2006;13(1):134–52. doi: 10.1080/13501760500380783. [
CrossRef]
- 17.
Waldschmidt
A. Disability policy of the European Union: the supranational level. ALTER Eur J Disabil Res / Rev Eur Rech Handicap. 2009;3(1):8–23. doi: 10.1016/j.alter.2008.12.002. [
CrossRef]
- 18.
Pfadenhauer
L, Rohwer
A, Burns
J, Booth
A, Lysdahl
KB, Hofmann
B, et al. Guidance for the assessment of context and implementation in health technology assessments (HTA) and systematic reviews of complex interventions: the Context and Implementation of Complex Interventions (CICI) framework. INTEGRATE-HTA Consortium; 2016 (
http://www.integrate-hta.eu/downloads/, accessed 19 June 2018).
- 19.
Pfadenhauer
LM, Gerhardus
A, Mozygemba
K, Lysdahl
KB, Booth
A, Hofmann
B, et al. Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implement Sci. 2017;12:21. doi: 10.1186/s13012-017-0552-5. [
PMC free article: PMC5312531] [
PubMed: 28202031] [
CrossRef]
- 20.
- 21.
- 22.
Pawson
R, Greenhalgh
T, Harvey
G, Walshe
K. Realist review - a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10 Suppl 1:21–34. doi: 10.1258/1355819054308530. [
PubMed: 16053581] [
CrossRef]
- 23.
Lewin
S, Oxman
AD, Lavis
JN, Fretheim
A, Garcia Marti
S, Munabi-Babigumira
S. SUPPORT Tools for evidence-informed Policymaking in health 11: Finding and using evidence about local conditions. Health Res Policy Syst. 2009;7 Suppl 1:S11. doi: 10.1186/1478-4505-7-S1-S11. [
PMC free article: PMC3271822] [
PubMed: 20018101] [
CrossRef]
- 24.
Lavis
JN, Wilson
MG, Oxman
AD, Grimshaw
J, Lewin
S, Fretheim
A. SUPPORT Tools for evidence-informed health Policymaking (STP) 5: Using research evidence to frame options to address a problem. Health Res Policy Syst. 2009;7 Suppl 1:S5. doi: 10.1186/1478-4505-7-S1-S5. [
PMC free article: PMC3271832] [
PubMed: 20018112] [
CrossRef]
- 25.
Fretheim
A, Munabi-Babigumira
S, Oxman
AD, Lavis
JN, Lewin
S. SUPPORT Tools for evidence-informed Policymaking in health 6: Using research evidence to address how an option will be implemented. Health Res Policy Syst. 2009;7 Suppl 1:S6. doi: 10.1186/1478-4505-7-S1-S6. [
PMC free article: PMC3271833] [
PubMed: 20018113] [
CrossRef]
- 26.
- 27.
McDonald
KM. Considering context in quality improvement interventions and implementation: concepts, frameworks, and application. Acad Pediatr. 2013;13:(6 Suppl):S45–53. doi: 10.1016/j.acap.2013.04.013. [
PubMed: 24268084] [
CrossRef]
- 28.
Tricco
AC, Cardoso
R, Thomas
SM, Motiwala
S, Sullivan
S, Kealey
MR, et al. Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review. Implement Sci. 2016;11:4. doi: 10.1186/s13012-016-0370-1. [
PMC free article: PMC4709874] [
PubMed: 26753923] [
CrossRef]
- 29.
- 30.
Glenton
C, Lewin
S, Scheel
IB. Still too little qualitative research to shed light on results from reviews of effectiveness trials: a case study of a Cochrane review on the use of lay health workers. Implement Sci. 2011;6:53. doi: 10.1186/1748-5908-6-53. [
PMC free article: PMC3117743] [
PubMed: 21619645] [
CrossRef]
- 31.
Liu
X, Dou
L, Zhang
H, Sun
Y, Yuan
B. Analysis of context factors in compulsory and incentive strategies for improving attraction and retention of health workers in rural and remote areas: a systematic review. Hum Resour Health. 2015;13:61. doi: 10.1186/s12960-015-0059-6. [
PMC free article: PMC4508764] [
PubMed: 26194003] [
CrossRef]
- 32.
Belrhiti
Z, Booth
A, Marchal
B, Verstraeten
R. To what extent do site-based training, mentoring, and operational research improve district health system management and leadership in low- and middle-income countries: a systematic review protocol. Syst Rev. 2016;5:70. doi: 10.1186/s13643-016-0239-z. [
PMC free article: PMC4847191] [
PubMed: 27116915] [
CrossRef]
- 33.
- 34.
Gough
D, Oliver
S, Thomas
J, editors. An introduction to systematic reviews. Los Angeles (CA): Sage Publications; 2012.
- 35.
- 36.
- 37.
Carroll
C, Booth
A, Cooper
K. A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11:29. doi: 10.1186/1471-2288-11-29. [
PMC free article: PMC3068987] [
PubMed: 21410933] [
CrossRef]
