Methods
A methodological systematic review of the literature, including ‘grey’ literature such as reports, doctoral theses and book chapters, was conducted to identify existing guidance and recommended practice in conducting and reporting meta-ethnography from any academic discipline. This review has been registered on PROSPERO (the International Prospective Register of Systematic Reviews) as registration number CRD42015024709. A key focus of the review was on the meta-ethnography analytic synthesis phases, which are complex and currently very poorly reported.
Search strategy
We first conducted comprehensive database searches that were followed by expansive searches to identify published and unpublished research in any language. These searches were iterative and evolved as the review progressed because their purpose was to build our knowledge of recommendations and guidance in conducting and reporting meta-ethnography rather than to answer a tightly defined research question.43
To identify relevant literature, we started with seminal methodological and technical publications known to our expert academic advisors and the project team including Noblit and Hare’s book,25 detailed worked examples of meta-ethnographies and publications relating to qualitative evidence synthesis more generally (e.g. reporting guidelines for qualitative evidence synthesis approaches and reviews of qualitative syntheses including meta-ethnographies). Relevant texts were included from other disciplines that use meta-ethnography, such as education and social work. We performed citation searching, reference list checking (also known as backward and forward ‘chaining’) of the seminal texts and searched key websites (e.g. The Cochrane Library). Comprehensive database searches were also conducted to identify other methodological publications.43 Details of databases and other sources that were searched, as well as the search terms that were used, can be found in Appendix 1 (Tables 5 and 6, respectively).
Comprehensive database searches to identify methodological publications
Searches were performed in 16 databases, in July and August 2015. The search strategy was designed in MEDLINE (Medical Literature Analysis and Retrieval System Online, or MEDLARS Online) and refined by testing against a set of key papers already known to the team. The search terms were developed and piloted in collaboration with a researcher highly experienced in the conduct of systematic reviews (RT). The terms related to meta-ethnography and qualitative synthesis, and to methodological guidance, were tailored to each bibliographic database. Reviewers also hand-searched reference lists in included texts (those meeting inclusion criteria for the review) for other relevant studies not already identified. A systematic approach was used to record and manage references, which were stored in the bibliographic software EndNote version X7.4 (Thomson Reuters, CA, USA). The list of included publications from database searches and expansive searches was shared with our academic expert advisors who suggested potential additional publications.
Screening and selection of texts to include
We originally intended to independently double-screen all references by title and abstract; however, we reviewed this decision because the highly sensitive search strategy resulted in a very large number of retrieved references from the comprehensive database searches. We reviewed our screening strategy after independent screening had started; we decided not to double-screen references from the database searches published prior to 2006 to enable us to meet our aims and project timelines. The references published pre 2006 that referred to qualitative evidence synthesis had generally been superseded and the majority of relevant references about meta-ethnography were already known to the project team and expert advisors. We were confident that any relevant publications published prior to 2006 would be identified through expansive searches and expert recommendations. However, as a precaution, titles and abstracts of references from 2005 and older (n = 1204) were electronically searched for key terms (‘ethnograph’, ‘Noblit’) to identify any that referred to meta-ethnography; references containing these terms were then screened by title and abstract by one reviewer (EFF).
Overall, titles and abstracts of 6271 references retrieved through database searches were independently double-screened and a further 1204 were screened by one reviewer. All additional references identified from other sources were double-screened. A total of seven reviewers (IU, EFF, Derek Jones, NR, JN, EASD and MM) were involved in screening retrieved publications, using the inclusion and exclusion criteria presented in .
Stage 1 inclusion and exclusion criteria
Disagreements over inclusion/exclusion were resolved through discussion. A third reviewer also screened publications if the first two reviewers could not reach agreement. A final check of the full text of the articles was conducted for inclusion/exclusion before the data extraction was conducted.
Data extraction
Data were extracted in the qualitative analysis software NVivo 10.0 (QSR International, Warrington, UK), using a coders’ guidance document, which was shared by all coders. The guidance was developed by Emma F France and piloted against five key methodological publications and then discussed with the team for refinement. Four reviewers performed the data extraction (IU, EFF, Karen Semple and Joanne Coyle), working from the same guidance. Data were extracted from each included publication by only one reviewer because this was a qualitative review in which the key principles are transparency and consensus, rather than independence and inter-rater reliability. However, the completeness of the data extraction was double-checked by a second reviewer for 13 of the publications to ensure accuracy. In order to maximise the resources and time available, data were extracted from the richest and seminal publications first, as assessed by Emma F France and Isabelle Uny, and then from the other publications until all were coded and analysed. NVivo 10.0 was used to facilitate management of, and data extraction from, the publications. Guidance and recommendations from the 57 methodological texts were coded into the ‘nodes’ or data extraction categories described below, which are primarily based on Noblit and Hare’s25 seven phases of meta-ethnography conduct, with some additional categories for the data (e.g. ‘Definition or nature of meta-ethnography’) that were not specifically about the seven received meta-ethnography phases. The reason for creating these nodes was their fitness to providing an answer for the research question. The nodes at which data were extracted were:
definition or nature of meta-ethnography and how it differs from other qualitative evidence synthesis approaches
selection of a qualitative evidence synthesis approach: how to select a suitable qualitative evidence synthesis approach for one’s aim or research question (this new phase was labelled ‘phase 0’ and added by the eMERGe team).
Phase 1: getting started – deciding the focus of the review (e.g. guidance or recommendation on choosing a topic).
Phase 2: deciding what is relevant – which encompassed three subcategories –
quality appraisal of studies – recommendations on ways to appraise the qualities of primary studies to be included.
search strategies for meta-ethnography – recommendations on searching for primary texts or studies.
selection of studies – guidance or recommendations on the manner in which studies to be synthesised were selected.
Phase 3: reading studies – where advice or recommendation is given on how to read the studies and identify and record the concepts and metaphors contained in each study.
Phase 4: determining how the studies are related – determining how the concepts and metaphors used in each study relate to others and how they can be synthesised. This phase was also divided into three subcategories –
definition of reciprocal translation – when concepts in one study can incorporate those of another; the coding entailed defining this type of translation and identifying advice and recommendations on how this could be undertaken.
definition of refutational translation – where concepts in different studies contradict one another; the coding entailed defining this type of translation and identifying advice and recommendations on how this could be undertaken.
definition of line of argument (LOA) – when the studies identify different aspects of the topic under study that can be drawn together in a new interpretation; the coding entailed defining this type of translation and identifying advice and recommendations on how this could be undertaken.
Phase 5: translating studies into one another – the way in which metaphors and/or concepts from each study and their inter-relatedness are compared and translated into each other.
Phase 6: synthesising translations – how to synthesise the translations to make them into a whole that is greater than its parts.
Phase 7: expressing the synthesis – how the synthesis is presented, the message conveyed and for which audience.
Some other categories were also included in data extraction, which are reported on in this document:
issues of context in meta-ethnography
number of reviewers required to undertake a meta-ethnography
validity, credibility and transferability issues in meta-ethnography.
Data analysis
Publications were read repeatedly and compared using processes of constant comparison. Extracted data were analysed qualitatively mainly by two members of the team (IU and EFF). To support the analysis, memos were written for each category in which each reviewer could record their analysis of the data extracted at the particular node. As the analysis progressed, areas of agreement and uncertainties were noted in the memos and Isabelle Uny and Emma F France drew on each other’s understanding of the data from each node. For complex nodes (e.g. regarding conduct of phases 4, 5 or 6), each reviewer individually identified the key themes and issues in a NVivo memo and then the two coders compared what they had written to check their different interpretations. Following this, one of the coders wrote a detailed analytic memo, to which the other subsequently added more details, or which they could question in light of what they had read. For less complex or less contentious nodes (e.g. regarding conduct of phases 1, 2 or 3), one reviewer conducted the analysis, also using memos that were then checked by the other reviewer for accuracy and transparency. Throughout, each reviewer maintained an analysis journal in NVivo and any analysis decisions made at project meetings or internal meetings were logged in a folder on our shared electronic drive (all meetings were also audio-recorded for easy reference). Once completed, the initial analysis was collated and shared with the wider team and then discussed and revised to add rigour to the process and gain further perspectives on wider interpretation and analysis of the data contained in each node.
The guidance and advice provided in the included publications around each node/category varied in richness and detail. Nonetheless, the full range of practice was documented, regardless of the richness of the text. However, each reviewer also noted whether or not they felt that the texts they extracted data from were ‘rich in details’ (i.e. whether or not they were a detailed account related to meta-ethnography with in-depth explanation and rationales that went beyond description). As the analysis drew to its final stage, we detailed definitions for each of the phases as understood and described in the included publication. We summarised and analysed advice and recommendations given on the conduct of each and every phase, and noted the pitfalls and criticisms in the conduct and reporting of meta-ethnography raised by each author. The findings that emerged from the analysis are, thus, presented below.
Findings
As per the PRISMA flow diagram (), 9285 references were retrieved from the databases searches and 47 from expansive search techniques, resulting in a total of 7522 records after deduplication. A total of 7417 clearly irrelevant texts were excluded by screening title and abstract. A total of 105 papers were screened in full text. There were 48 papers excluded at full-text screening because either they were found to be clearly irrelevant once the full text was retrieved or they did not report on methodological issues of meta-ethnography, were not a reporting guideline or did not provide guidance on the reporting of meta-ethnography. This resulted in a final total of 57 included publications.
The PRISMA flow diagram for stage 1. Adapted from Moher et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided (more...)
Those 57 publications were included for data extraction. Five were from the field of education, 46 from health and six from other disciplines. There were 19 worked examples and 30 were considered to be rich in detail. The authors of 28 publications were solely from the UK, nine solely from the USA/Canada, four solely from Scandinavia and 16 had international (multicountry) teams. The full list of the included publications and their characteristics is provided in Appendix 2.
The following subsections will cover the findings from this review for every phase of a meta-ethnography, as described in Chapter 1. The analysis of the 57 publications showed that the aspects of meta-ethnography on which most methodological uncertainties remain are those regarding the nature and definition of meta-ethnography, the methods for selecting a qualitative evidence synthesis approach (a new initial phase labelled ‘phase 0’ by the eMERGe team) and those regarding phases 4–6 of the meta-ethnography conduct (because they are complex and usually the most poorly reported in meta-ethnographies). Therefore, more space has been devoted in this report to the findings related to those particular phases. During the analysis, it became clear that some of the methodological texts were more rich in details than others and, therefore, contributed more heavily to the analysis. A table is provided in Appendix 3 that shows clearly the contribution of the major methodological publications to the categories and findings presented below. This is so that the reader is able to trace the contribution of each publication to the analysis of findings. Some of the publications are also directly referenced in the text of this report, where they made a particularly pertinent point or offered a particularly useful example. There were few publications relating to meta-ethnography reporting; most were about its conduct. On the whole, the review identified very little in the way of advice or recommendations about meta-ethnography conduct and reporting based on empirical evidence, such as from methodological research, and, rather, more evidence based on the opinion or reasoned argument of the publication authors. In the findings presented below, we have, therefore, stated whether or not the uncertainties and issues raised with regard to meta-ethnography conduct and reporting were those of the authors of the methodological text analysed, or issued from the reflections and analysis of the eMERGe team.
Definition or nature of meta-ethnography and how it differs from other qualitative evidence synthesis approaches
The analysis of the methodological texts determined that meta-ethnography is an interpretive method of synthesis rather than simply an aggregative one. It was described by Noblit and Hare25 as: ‘the comparative textual analysis of published field [qualitative] studies’, the aim of which is to create new interpretations.
A meta-ethnography analyses qualitative data in an inductive way to develop concepts, theories and models. Meta-ethnography attempts to preserve the contexts of the studies synthesised and uses a process of translation, which will be described at length in Phase 5: translating studies into one another.
Although it was conceived solely as a method of synthesis by Noblit and Hare,25 other authors have, over time, also used meta-ethnography as a systematic literature review methodology.6 Moreover, although meta-ethnography was designed to synthesise interpretive qualitative studies, one text44 in this review argued that meta-ethnography could be used to synthesise qualitative and quantitative studies together; however, in order to do so, those authors drew on meta-ethnography to develop a new approach called ‘critical interpretive synthesis’.
There was some discussion within the methodological texts regarding what constitutes the ‘unique’ characteristics of meta-ethnography as a qualitative evidence synthesis method. According to Noblit and Hare,25 the processes that they presented in seven phases were not necessarily unique to meta-ethnography. However, they argued that the underpinning use of Turner’s theory of social explanation45 embedded in the process of translation differentiated meta-ethnography from other qualitative evidence synthesis methods. After a meeting with the eMERGe team in June 2016, Professor George Noblit provided further reflections on the process of translation as follows:
In Noblit and Hare’s text, synthesis is seen as a form of translation of accounts into one another. The nature of such translation is based on S. Turner’s Sociological Explanation as Translation (Cambridge: Cambridge University Press, 1980) in which he examines comparative explanations—the essential form of meta-ethnography. Turner notes that practices, and the concepts used to describe such practices, may vary from those in another society. In doing comparisons, then one may use the concept from one society, or create a new concept, in making the comparisons of the societies. In this, explanation is a form of translation and that ‘an adequate translation would yield us claims that had the same implications in both languages’ (p. 53). Accounts can be substituted for language in this quote, for the purposes of meta-ethnography. Synthesis as translation starts with a puzzle that is of the form where one study says x, what is another study saying? Addressing this puzzle requires formulating an analogy between the studies. As we add studies, we may find that the translation/analogy offered with the initial studies does not hold up.
Professor George W. Noblit, University of North Carolina at Chapel Hill, 2016, personal communication
The analysis of the methodological texts in this review indeed showed that what is seen to distinguish meta-ethnography from other qualitative evidence synthesis methodologies is the translation process.
One of the other key characteristics of meta-ethnography, as seen by some authors, is that it aims to arrive at new interpretations greater than those of individual studies.6,25
The main uncertainties surrounding the nature of meta-ethnography are around whether or not it could be used to synthesise both qualitative and quantitative studies, and whether or not purely descriptive studies (of which there tend to be many in health research) or deductive studies should be excluded from the synthesis.
‘Phase 0’: selecting a qualitative evidence synthesis approach
This review identified a new stage before ‘phase 1: getting started’, which was labelled ‘phase 0: selecting a qualitative evidence synthesis approach’. It relates to the rationale for choosing meta-ethnography as the qualitative evidence synthesis approach for the topic in question. This review demonstrated that better guidance is needed here to avoid reviewers choosing the wrong method of qualitative evidence synthesis or having to amend a method to suit their needs when a more suitable one might already exist.
Through stages 1 and 2 of this project, it became clear to the eMERGe team that authors of meta-ethnographies often cite Noblit and Hare25 and state that their method is meta-ethnography when, in fact, they are not conducting a meta-ethnography. A number of strategies to avoid this were identified in the review of methodological texts, including:
investigating other qualitative evidence synthesis approaches before choosing meta-ethnography (e.g. ensuring that an interpretive qualitative evidence synthesis is required and that the type and quantity of studies to be synthesised fit with the method selected)
ensuring that the synthesis research question and aim drive the choice of qualitative evidence synthesis approach (e.g. whether it aims to generate a model or theories of behaviour or experiences, or aims at conceptual and theoretical development)
making sure that the qualitative evidence synthesis chosen fits with the epistemological stance of the team of reviewers/the reviewer, their skills and experience of the methods used (meta-ethnography may not be best suited to novices in qualitative research)
ensuring that the time and resources available fit with the conduct of a meta-ethnography.
Ultimately, the review revealed that meta-ethnography needs a high level of qualitative expertise. It is a time-consuming enterprise and this must also be taken into account in phase 0. Clear guidance is required on the conduct of meta-ethnography (particularly phases 5 and 6) to help researchers choose the most suitable qualitative evidence synthesis approach.
Phase 1: getting started
Noblit and Hare25 describe this phase as:
identifying an intellectual interest that qualitative research might inform . . . In this phase, the investigator is asking, How can I inform my intellectual interest by examining some set of studies?
pp. 26–725
Ideally, a meta-ethnography aims to address a gap in knowledge, for instance by asking whether or not a qualitative evidence synthesis has previously been conducted on a particular topic or by asking whether or not it can offer new explanations of the topic. The methodological review found that authors recommended that an aim and research objectives be defined, at least in broad terms, at the start of undertaking the meta-ethnography, even if they are refined later in the process. An example of how this may be reported can be found in a worked example by Britten et al.46 about lay experiences of diabetes mellitus and diabetes mellitus care.
Although the issues identified regarding phase 1 were not contentious, there were some uncertainties around the best way to define, or refine, the research question in a meta-ethnography as there is a link between the research question and the selection of studies to be included in the thesis (e.g. the final research question will determine which studies are included).28,47
Phase 2: deciding what is relevant
With regard to the meta-ethnographic conduct, it bears remembering that Noblit and Hare’s25 book was published at a time when online bibliographic databases were unavailable. They created meta-ethnography as a method of synthesis but did not provide detailed guidance on selecting studies for inclusion in the synthesis. Subsequently, other researchers have applied systematic search and selection procedures to the identification and selection of studies for inclusion in a meta-ethnography.
The analysis of the methodological texts confirmed the view that published meta-ethnographies mostly use comprehensive systematic review style searches traditionally used for quantitative reviews of intervention effectiveness. However, some authors in this review stressed that this was not the only method available for meta-ethnography. For instance, some suggested that exhaustive searching may be suitable for making generalisable claims or to provide a comprehensive picture of research on a topic, whereas non-linear or purposeful searching might be more appropriate in other cases, such as meta-ethnography, where the intention is to generate a theory. Whatever the case may be, authors stressed that the search strategy ought to match the intended purpose of the meta-ethnography. One of the difficulties raised by authors, however, is that qualitative research reports are sometimes challenging to identify through electronic database searches. They therefore urged reviewers to supplement database searches with alternative methods, such as searching grey literature.
Quality appraisal and sampling for meta-ethnography
Noblit and Hare25 did not advocate a formal appraisal of studies prior to synthesis, rather, they argued that each study’s quality would become apparent by how much it eventually contributed to the synthesis. However, recent reviews of meta-ethnographies, including that carried out in the stage 2 audit of this project, indicate that most meta-ethnography reviewers conduct some form of quality appraisal of studies.17,29
There is a wide variety of quality appraisal approaches that can be used, some judging conceptual richness (which is more rarely done) and some judging the methodological quality of primary studies (which is most commonly done). This review found that a number of meta-ethnographies use the Critical Appraisal Skills Programme,48 or adapted forms of it, to assess the quality of the primary studies. Malpass et al.49 offer an exemplar of how the quality appraisal of the studies can be used in their synthesis of patients’ experiences of antidepressants.
This review of methodological texts showed that there is debate over whether or not formal quality appraisal is necessary or even useful in meta-ethnography. There were uncertainties around whether or not papers appraised as being of lower methodological quality should be included in the synthesis (as the findings may still be credible) and how difficult it may be to assess the quality of papers from radically different contexts. A number of authors suggested that a possible benefit of quality appraisal is the close reading of papers that it encourages, which is useful for meta-ethnography.
In terms of sampling, the review of methodological texts shows that what is seen as the optimum number of studies to synthesise in a meta-ethnography is also controversial. For instance, some argue that too many studies (n > 40) could make the translation process difficult and result in a more superficial synthesis, whereas others argue that too few studies might result in an underdeveloped conceptualisation. However, the real issue may be the number of data to be synthesised relative to the capacity of the review team rather than the number of studies per se. Some authors in this review found that there is a relationship between the research question and sampling (e.g. a narrow question can lead to a smaller sample, and starting with a wider sample and applying quality appraisal may help refine the question). The review showed that there are perceived benefits and problems with applying purposive and theoretical sampling to meta-ethnography, and that theoretical sampling in meta-ethnography has rarely been tested empirically.
Phase 3: reading the studies
The review identified various reading strategies for phase 3, such as reading while recording themes and identifying concepts (including refutational ones) and their context within the framework of the research question. Some read the papers or accounts chronologically and some started with the most conceptually rich, although there was no evidence to indicate how reading papers in different orders may affect the synthesis output. Authors of methodological texts often used tables (or mind maps) to display concepts, sometimes distinguishing concepts of the research participants from those of the primary study authors (referred to by some authors as first- and second-order concepts, respectively). Some also used phase 3 to appraise the quality of the studies. Some authors further specified the importance of reading being carried out by two or more reviewers. This review concludes, however, that one of the key uncertainties in this phase was around how to preserve the meaning of, and relationships between, concepts within and across studies when reading.
Phase 4: determining how the studies relate
Noblit and Hare25 expressed that the studies may relate in three main ways:
reciprocally (because they are about similar things)
refutationally (because they are about different things)
as a LOA (because they offer part of a higher meaning).
Some methodological texts in this review ventured that a well-conducted phase 3 will help to determine how studies relate to one another; however, most authors show how they related the studies in a grid or table (some detailed descriptions of how this has been reported can be found49–51). Some texts analysed in this review suggested that ‘relating studies’ is best done collaboratively by a team who interpret the concepts separately first and then come to a decision together.
We concluded that the main uncertainties about the conduct of phase 4 are:
whether or not it is possible to relate studies that are profoundly epistemologically different, and what is the best way to preserve the semantics and context of the metaphors or concepts contained in each study through the ‘relating’ process
how the order in which studies are appraised and synthesised may affect the outcome of the synthesis (e.g. use of index paper)
whether or not reciprocal findings in studies may tend to be given more weight than refutational ones.
Phase 5: translating studies into one another
As expressed earlier, the process of translation is key to meta-ethnography conduct. It was defined by Noblit and Hare25 as idiomatic rather than literal. From this methodological review, we can conclude that the process of translation is not a linear process but an iterative one, which aims to translate concepts from one study into another study and, thus, arrive at concepts or metaphors that embody more than one study.52
This review found that there is no single way of conducting the translation in a meta-ethnography and the various methods have not been formally compared in methodological research. However, it is the eMERGe team’s contention that whichever method of translation is used should be made explicit and transparent by the authors. The diagrams in were designed by the eMERGe team to represent three well-defined methods of translation described by some of the authors of the methodological texts included in this review.
Three possible methods for the conduct of phase 5 as interpreted by the eMERGe project.
Reciprocal translation
Reciprocal translation, according to Noblit and Hare25 and to a number of other authors reviewed in this study, takes place when studies are roughly about the same thing and their meaning can be interpreted into one another. The conduct of reciprocal translation was described in detail by Campbell et al.,24,52 and their approach has been used by other authors in this review. We have summarised the approach of Campbell et al.24,52 in .
Reciprocal translation process as interpreted by the eMERGe project.
A number of authors in the review recommended using tables or grids to represent the reciprocal translation analysis (a particularly detailed example of this can be seen in Malpass et al.49).
This review found that one of the issues regarding the conduct of reciprocal translation is that it can be done in such a way as to result in a simple recoding or recategorising of themes from the primary studies rather than being interpretive. A meta-ethnography should strive to lead to new interpretations and theories of the topic under study.
Refutational translation
Noblit and Hare describe refutation as ‘an interpretation designed to defeat another interpretation’ (p. 48).25 According to authors whose accounts are contained in this review, the purpose of a refutational translation is to explain differences and exceptions in the studies. Meta-ethnography is one of the few qualitative evidence synthesis methods that requires the researcher ‘to give explicit attention to identification of incongruities and inconsistencies’ (p. 128).56 Some authors state that refutational translation can take place at the level of the overall studies, accounts or reports, whereas others state that it occurs at the level of themes, concepts or even findings across study accounts. It is our understanding from the review literature that it is likely that both types of refutation exist and are possible.
However, in the review, it was clear that there are uncertainties as regards how to conduct refutational translation and the review questions whether or not undertaking refutational translation makes it more difficult to develop an overarching LOA synthesis (LOA synthesis is described in Phase 6: synthesising translations, Line of argument synthesis).
Phase 6: synthesising translations
Noblit and Hare25 define this phase as follows:
Synthesis refers to making a whole into something more than the parts alone imply. The translations as a set are one level of meta-ethnographic synthesis. However, when the number of studies is large and the resultant translations numerous, the various translations can be compared with one another to determine if there are types of translations or if some metaphors and/or concepts are able to encompass those of other accounts. In these cases, a second level of synthesis is possible, analyzing types of competing interpretations and translating them into each other.
p. 2925
The manner in which the translation is synthesised depends mainly on the way phase 5 was conducted. Some authors express that, to a certain extent, translation and synthesis happen together, in an iterative manner. There is also no single way in which to carry out the synthesis. Some of the methods used by authors of worked examples of meta-ethnographies are described in two diagrams in .
Two possible methods for the conduct of phase 6 as interpreted by the eMERGe project.
The review indicates that there is potential to develop a theory from the synthesis in phase 6, but that very few authors describe whether or not, or how, they did this. One notable exception is Britten and Pope,51 whose worked example produced middle-range theories in the form of hypotheses that could be tested by other researchers. Part of the issue is that theory is understood differently by different authors.
Line of argument synthesis
Another type of synthesis is LOA synthesis. Noblit and Hare25 define LOA as being about inference:
What can we say of the whole (organization, culture, etc.), based on selective studies of the parts?
p. 6325
Other authors also conceive of it as a process that produces new interpretations based on the analysis and translation of the primary studies. Several authors state that you can further develop translations into a LOA synthesis, which was how Noblit and Hare25 described it. LOA is described as a synthesis that links translations and the reviewer’s interpretation. Some clear and detailed examples of how LOA synthesis has been conducted can be found in Britten and Pope.51
It is this project team’s understanding that a LOA synthesis is distinct from the translation process and follows on from it. However, depending on the nature of the data, it may or may not be possible to achieve a LOA synthesis in a meta-ethnography. One of the main uncertainties around LOA synthesis is whether or not it may constitute a model in itself or whether or not developing a model is a further analytic step.
The uncertainties with regard to phase 6, as revealed in this review, are:
How strong or valid is the evidence produced by a meta-ethnography (e.g. when the interpretation in the synthesis is three times removed interpretation from the lived experience of the participants in the original studies)?
How does the process of translation and synthesis work in a team?
How transparent is the creative and interpretive synthesis process?
Phase 7: expressing synthesis
Noblit and Hare25 expressed in their book that the meta-ethnography synthesis output must be intelligible to the audience at whom it is aimed. Because of this, it could take the form of a written statement, but also could be conveyed by video or other art forms, although this has been rare. However, Noblit and Hare state:
The intention here is not to pander to the audience. Having our syntheses readily intelligible does not mean reducing the lessons of ethnographic research to an everyday or naive understanding of a culture. The focus on translations is for the purpose of enabling an audience to stretch and see the phenomena in terms of others’ interpretations and perspectives.
p. 2925
Authors in this review noted that the expression of the synthesis has to suit the audience and be clear so that policy makers and intervention planners can make use of it. However, some warned about the difficulties in remaining independent in the expression of the synthesis; Booth57 for instance expressed concerns that pressures from funders to publish new findings may influence the final product.
Issues of context in meta-ethnography
The context in a meta-ethnography concerns not only the characteristics of primary studies (e.g. the socioeconomic status of participants, their location, study setting, research designs, methodological details, and political and historical contexts), but also the context of the reviewers themselves (funding, political climate, respective expertise and world views).
The review found that authors deemed context to be important to meta-ethnography. Indeed, from their initial work on meta-ethnography, which was designed specifically to preserve the contextual aspects of studies to be synthesised, Noblit and Hare25 contended that other aggregative qualitative evidence syntheses, by contrast, were:
context-stripping [and] impeded explanation and thus negated a true interpretive synthesis.
p. 2325
For the authors in this review, taking into account the context of the studies to be synthesised was seen to bring credibility to a meta-ethnography. Authors in this review recommended laying out the context from each primary study in a grid or table for readers to see. Unfortunately, context is often a problem for meta-ethnography as it tends to be poorly reported in primary studies in health-care research. The uncertainty with regard to the issue of context is how to synthesise large numbers of studies with different contexts.
Number of reviewers required to undertake a meta-ethnography
The review showed that authors believed that there are benefits in a meta-ethnography, as with qualitative research in general, being undertaken by more than one reviewer. The reasons given were that it:
aids the translation process
leads to richer and more nuanced interpretations, as reviewers have alternative viewpoints and perspectives
encourages explorations of dissonance
brings more rigour to the process and increases the credibility of the research process.
Although an optimum number of reviewers for a meta-ethnography cannot be stated here as it has not been the subject of empirical research comparisons, the review certainly expressed that there were weaknesses in undertaking a meta-ethnography with only one reviewer, for example a lack of exploration of alternative interpretations. A review of meta-ethnographies published between 2012 and 201329 showed that the actual number of reviewers in recently published meta-ethnographies varied from one to seven, with the most common number being two or three. The composition and experience of the team of reviewers were seen as important. Findings suggest that the team must fit the aim of the synthesis and represent a range of perspectives, genders and skills (e.g. translators, data retrievers, user representatives and reviewers from different disciplines). Some authors suggested that qualitative evidence synthesis expertise is needed in the team to undertake a meta-ethnography. Other authors addressed the issue of power dynamics within the team of reviewers (e.g. different levels of seniority).
Validity, credibility and transferability issues in meta-ethnography
Within this review, the debate around validity or credibility and around generalisability or transferability in meta-ethnographies revolved around how useful or credible the findings from this type of synthesis are, as well as on whether or not they can be generalised or transferred to other settings.
Validity and credibility
Depending on the publication, the authors talked about either the validity or credibility, or sometimes the trustworthiness, of the findings; credibility and trustworthiness, rather than validity, are the terms usually used for qualitative research. Bondas and Hall58 offered some clear and concrete advice for ensuring validity in meta-ethnography:
questions such as Does the report clarify and resolve rather than observe inconsistencies or tensions between material synthesized? Does a progressive problem shift result? Is the synthesis consistent, parsimonious, elegant, fruitful, and useful (Noblit & Hare, 1988)? Is the purpose of the meta-analysis explicit?
Bondas and Hall58
For other authors, the search for disconfirming cases or studies (and the use of refutation) can enhance validity, as could the use of a multidisciplinary team, as it can improve rigour and quality. For some authors, the trustworthiness of the output of a meta-ethnography is related to how rigorously phase 2 has been conducted, that is, how data were collected (e.g. having a large number of heterogeneous studies or too few studies that were rich in details could compromise trustworthiness).
Although a few authors suggested returning to the authors of primary studies to check the validity of the metaphors used and the interpretations formed, this was seen by most as impractical and tricky. Furthermore, as Noblit and Hare25 stated, the interpretation formed by the meta-ethnography synthesis could be construed as simply one possible interpretation, not as ‘truth’. Campbell et al.24 offered another view on validity by suggesting that ‘One possibility would be to test the relevance of the synthesis findings by presenting them to pertinent patient groups, health professions, academics and policy makers’.
Garside54 suggested that trustworthiness may be easier to establish in a qualitative evidence synthesis because the study reports are in the public domain, unlike the raw data of most qualitative primary studies, and so can be accessed by readers. To increase credibility, most authors suggested that the choice of meta-ethnography must be justified, the conduct of the synthesis clearly laid out and the place of the reviewer reflexively assessed.
Generalisability and transferability
Here, generalisability is understood to mean the degree to which findings from a particular meta-ethnography can be generalised to another sample of studies or another context. This term is most often used in more positivist-type research, and some authors in this review were doubtful of its usefulness to qualitative evidence synthesis. However, some of the authors believed that ‘generalisation across studies adds to the findings of individual studies’.24 This was with the caveat that the heterogeneity of studies and their potential competing interpretations should still be taken into account within the synthesis. An alternative term more often used for qualitative research is ‘transferability’, meaning the ability to transfer findings to other settings and contexts.
Criticisms of meta-ethnography
Some of the main criticisms of meta-ethnography identified in the review of methodological papers included (1) the fact that, although there was some good practice in the conduct of meta-ethnography, a lot of those reviews that are labelled as such are actually critical literature reviews rather than interpretive meta-ethnographies; (2) the large number of studies selected for inclusion in some meta-ethnographies; and (3) the fact that some reviewers of meta-ethnography have used aggregative approaches in the attempt to conduct meta-ethnographies, and, in others, it is unclear what process was used to arrive at the final synthesis. Another criticism included that meta-ethnography reviewers sometimes failed to make clear how they selected their studies, whereas others offer incomplete analyses, in which first-, second- and third-order constructs are not always distinguished. Another main critique was that few meta-ethnographies actually conduct any refutational translations and few offer proper LOA syntheses, instead conducting only reciprocal translations.
