U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Cunningham M, France EF, Ring N, et al. Developing a reporting guideline to improve meta-ethnography in health research: the eMERGe mixed-methods study. Southampton (UK): NIHR Journals Library; 2019 Feb. (Health Services and Delivery Research, No. 7.4.)

Cover of Developing a reporting guideline to improve meta-ethnography in health research: the eMERGe mixed-methods study

Developing a reporting guideline to improve meta-ethnography in health research: the eMERGe mixed-methods study.

Show details

Appendix 7Standards met for phases 0–7 in audit sample of meta-ethnographies

TABLE 8

Audit standards with percentages for phase 0

Meta-ethnography phase 0Standard numberReporting criteriaMeta-ethnography publications, n (%)

Number of standards applicable to any meta-ethnography: 7

Number of standards applicable only to some meta-ethnographies: 0

Standard met in full or in partStandard not met or not reported
Phase 0: choosing meta-ethnography1Studies report why meta-ethnography was considered most appropriate QES methodology14 (73.6)5 (26.3)
2Reviewers state their initial intention was to produce a new theory, interpretation or model (even if this was not ultimately possible)12 (63.1)7 (36.8)
3Reviewers state the type of social explanation(s) review findings are expected to produce17 (89.4)2 (10.5)
4Reviewers state type of social explanation(s) review is expected to produce in line with Turner’s theory452 (10.5)17 (89.4)
5The qualitative expertise of reviewers is stated0 (0)19 (100)
6Review context is stated (e.g. any funding sources, time scales for meta-ethnography), findings to inform guideline development, Health Technology Assessment or promote evidence implementation15 (78.9)4 (21.0)
7Reviewer(s) perspectives contributing to this interpretive process is/are stated [e.g. epistemological position(s), positions held, academic disciplines, organisation(s) or health bodies represented, cultural diversity]7 (36.8)12 (63.1)

QES, qualitative evidence synthesis.

TABLE 9

Audit standards with percentages for phase 1

Meta-ethnography phase 1Standard numberReporting criteriaMeta-ethnography publications, n (%)

Number of standards applicable to any meta-ethnography: 5

Number of standards applicable only to some meta-ethnographies: 4 (shaded green)

Standard met in full or in partStandard not met or not reported
Phase 1: getting started with meta-ethnography8Information (e.g. in a literature review) on the availability of qualitative data that potentially could be synthesised is provided10 (52.6)9 (47.3)
9A statement on the research/knowledge gap to be filled by meta-ethnography (or an updated meta-ethnography) is given18 (94.7)1 (5.2)
10Explicitly stated review aim(s)19 (100)0 (0)
11Explicitly stated review questions or objectives6 (31.5)13 (68.4)
12Review aim(s) and/or questions congruent with meta-ethnography (e.g. reviewers intend to produce new interpretation, model or theory)12 (63.1)7 (36.8)
13If the meta-ethnography approach was reported as adapted/modified: Adaptations/modifications should be clearly described1
14If the meta-ethnography approach was reported as adapted/modified, a rationale for any adaptations or modifications is given1
15If the meta-ethnography approach was reported as adapted/modified, reviewers state whether or not they considered using another QES approach rather than adapting/modifying meta-ethnography0
16If reviewers reported changing/refining their initial aims and/or questions following literature review, details of changes or refinements to the initial aims and/or research questions are given1

QES, qualitative evidence synthesis.

TABLE 10

Audit standards with percentages for phase 2

Meta-ethnography phase 2Standard numberReporting criteriaMeta-ethnography publications, n (%)

Number of standards applicable to any meta-ethnography: 17

Number of standards applicable only to some meta-ethnographies: 12 (shaded green)

Standard met in full or in partStandard not met or not reported
Phase 2: deciding what is relevant17A statement(s) regarding the choice of overall search strategy is given (e.g. how this was informed by review purpose and intended audience)10 (52.6)9 (47.3)
18Details on the electronic database(s) search strategies used (e.g. thesaurus, free text and broad-based terms)13 (68.4)6 (31.5)
19Details on the electronic databases searched19 (100)0 (0)
20Details on the searching approach(es) used (e.g. comprehensive, purposive or combined)19 (100)0 (0)
21Details on the alternative searching methods (e.g. if e-databases were not used)5 (26.3)14 (73.6)
22Details of all the data search processes and procedures including number of reviewer(s) involved in literature searching6 (31.5)13 (68.4)
23Details of all the data search processes and procedures including which reviewer(s) were involved in literature searching5 (26.3)14 (73.6)
24Details of all the data search processes and procedures including whether or not reviewers worked independently and then collaboratively to review searching decisions6 (31.5)13 (68.4)
25Details of all the data search processes and procedures including if complementary searching conducted (e.g. hand and/or internet searches and/or original authors were contacted)17 (89.4)2 (10.5)
26Details of all the data search processes and procedures including years data search covered16 (84.2)3 (15.7)
27Details of all the data search processes and procedures including rationale for years data search covered6 (31.5)13 (68.4)
28Details of all the data search processes and procedures including time period over which searches were conducted (e.g. weeks/months that reviewers took to search for studies)4 (21.0)15 (78.9)
29Details of all the data search processes and procedures including whether or not potential studies were screened by titles and abstracts prior to reading full texts15 (78.9)4 (21.0)
30Details of all the data search processes and procedures including rationale for stopping searching is provided0 (0)19 (100)
31Detailed study inclusion/exclusion criteria, for example whether or not:
  • only peer-reviewed data or grey literature also used
  • only traditional qualitative data (e.g. focus groups/interviews) or whether or not free-text survey data used too
  • studies from different traditions/approaches/methods of inquiry included/excluded
  • purely descriptive studies were excluded (i.e. those reporting only first order constructs)
  • specific data/publication time periods were used
  • studies were excluded on the basis of a specific context
  • study inclusion/exclusion were based solely on study narrative or whether or not original authors were contacted for more information/data
19 (100)0 (0)
32Explicit information on the number of qualitative studies found for inclusion in meta-ethnography18 (94.7)1 (5.2)
33Explicit information on the number of studies actually synthesised19 (100)0 (0)
34If initial searches were updated later, details are provided0
35Appropriate literature searching reporting formats (e.g. PRISMA, STARLITE89) if the meta-ethnography used comprehensive literature searches in the style of quantitative systematic reviews11
36If reviewers used a sample rather than all studies meeting inclusion/exclusion criteria: details of the type of sample (e.g. exhaustive or purposive) are provided3
37If reviewers used a sample rather than all studies meeting inclusion/exclusion criteria, a rationale for the type of sample used is given (e.g. only heterogeneous studies were included)3
38If the review limited included studies to a maximum (e.g. 50), the maximum number is clearly stated0
39If the review limited included studies to a maximum (e.g. 50), the rationale for this maximum number is stated0
40If the review limited included studies to a maximum (e.g. 50), studies excluded because maximum number was exceeded are identified0
41If included studies were quality appraised, the type of quality processes/tools/methods used are specified15
42If included studies were quality appraised, a rationale is given for the choice of quality assessment processes8
43If included studies were quality appraised, it is clear which reviewer(s) conducted the quality appraisal6
44If included studies were quality appraised, the outcome of any quality appraisal processes are provided13
45If included studies were quality appraised, any studies excluded following quality appraisal are clearly identified5

TABLE 11

Audit standards with percentages for phase 3

Meta-ethnography phase 3Standard numberReporting criteriaMeta-ethnography publications, n (%)

Number of standards applicable to any meta-ethnography: 13

Number of standards applicable only to some meta-ethnographies: 5 (shaded green)

Standard met in full or in partStandard not met or not reported
Phase 3: reading included studies46How many reviewers read full papers/reports is stated10 (52.6)9 (47.3)
47Whether or not papers were read in full is stated14 (73.6)5 (26.3)
48What order papers were read in (e.g. starting point for reading was an index paper or a particular year)3 (15.7)16 (84.2)
49Why that reading order was chosen is specified0 (0)19 (100)
50Data extraction processes: which reviewer(s) extracted data for participant and context details is stated5 (26.3)14 (73.6)
51Which reviewer(s) extracted data for first- and second-order constructs is stated7 (36.8)12 (63.1)
52Where data were extracted from is stated (e.g. if used findings in original studies or findings and discussion sections, etc.)8 (42.1)11 (57.8)
53Whether or not data were extracted independently by reviewers is stated9 (47.3)10 (52.6)
54Whether or not extracted data were checked for accuracy is stated6 (31.5)13 (68.4)
55What order data were extracted is stated in (e.g. chronological or started with index paper)4 (21.0)15 (78.9)
56Reason why data extracted in that order is stated0 (0)19 (100)
57Where data were extracted to is stated (e.g. into Microsoft Word documents, diagrams or qualitative data analysis software)11 (57.8)8 (42.1)
58The context (characteristics or summaries) of included studies is provided for readers in narrative and/or tabular form and includes key information [e.g. original study aim(s) re: study country/countries, health setting, any funding; data collection methods (e.g. focus groups); details of participants (e.g. number, age, gender, socioeconomic status); any significant contextual developments impacting on the included papers (e.g. launch of a new health strategy or an international public health outbreak)]18 (94.7)1 (5.2)
59Studies excluded on detailed reading because their contexts are clearly identified2
60Studies excluded on detailed reading because their contexts have an explanation provided for their exclusion (e.g. studies not homogeneous)3
61Studies excluded on full-text reading because of the lack of rich conceptually deep data are clearly identified2
62Studies excluded on full-text reading because of the lack of rich conceptually deep data have a rationale provided for these exclusions (e.g. survey data only)3
63Studies excluded on full text reading because of the lack of rich conceptually deep data: reviewers state whether or not authors of these studies were contacted for additional data (e.g. full study reports)1

TABLE 12

Audit standards with percentages for phase 4

Meta-ethnography phase 4Standard numberReporting criteriaMeta-ethnography publications, n (%)

Number of standards applicable to any meta-ethnography: 6

Number of standards applicable only to some meta-ethnographies: 0

Standard met in full or in partStandard not met or not reported
Phase 4: determining how studies are related64Details on how multiple perspectives were introduced into the translation and synthesis processes (e.g. if there was a single reviewer, was their interpretation presented to a wider group?)7 (36.8)12 (63.1)
65How reviewers decided how studies were related: the basis on which they determined how studies were related is given (e.g. by theoretical approach and/or in metaphors, aims, focus, context)15 (78.9)4 (21.0)
66How reviewers decided how studies were related: whether or not studies were excluded during phase 4 and, if so, why (e.g. concepts or metaphors could not be deciphered or identified, theoretical approach or meta-ethnography focus)2 (10.5)17 (89.4)
67How reviewers decided how studies were related: report states how studies were compared/juxtaposed to decide how they relate14 (73.6)5 (26.3)
68How reviewers decided how studies were related: report states how studies relate to each other [e.g. are the studies commensurable (about roughly similar things)]15 (78.9)4 (21.0)
69How reviewers decided how studies were related: authors’ concepts/themes/metaphors (second-order constructs) (i.e. raw data) from original studies are clearly reported (e.g. in grids/tables, visual diagrams/maps)16 (84.2)3 (15.7)

TABLE 13

Audit standards with percentages for phase 5

Meta-ethnography phase 5Standard numberReporting criteriaMeta-ethnography publications, n (%)

Number of standards applicable to any meta-ethnography: 20

Number of standards applicable only to some meta-ethnographies: 0

Standard met in full or in partStandard not met or not reported
Phase 5: translating studies70Translation processes (reciprocal and/or refutational): the different levels of interpretation (e.g. first-, second- and third-order constructs) within the translation/synthesis process are clearly differentiated for readers16 (84.2)3 (15.7)
71Translation processes (reciprocal and/or refutational): reviewers report steps taken to preserve context and meaning of the relationships between concepts within and across studies8 (42.1)11 (57.8)
72Translation processes (reciprocal and/or refutational): the order in which studies were translated/synthesised8 (42.1)11 (57.8)
73Translation processes (reciprocal and/or refutational): the reason for the order in which studies were translated/synthesised is given4 (21.0)15 (78.9)
74Translation processes (reciprocal and/or refutational): state which kind(s) of translation or synthesis was done – reciprocal, refutational, and/or line of argument17 (89.4)2 (10.5)
75Translation processes (reciprocal and/or refutational): methods used to translate concepts from one study into another are specific and clearly stated12 (63.1)7 (36.8)
76Translation processes (reciprocal and/or refutational): reviewers involved in translation are identified5 (26.3)14 (76.3)
77Translation processes (reciprocal and/or refutational): studies included within translation are clearly identified19 (100)0 (0)
78Translation processes (reciprocal and/or refutational): a rationale is provided for studies excluded from translation2 (10.5)17 (89.4)
79Translation processes (reciprocal and/or refutational): the outcome of the translation is given – this could be in table, grid format or narrative18 (94.7)1 (5.2)
80Refutational analysis: reviewers state in which phase(s)/at which point refutational translation was considered11 (57.8)8 (42.1)
81Refutational analysis: reviewers state whether or not social and cultural factors were considered during refutational translation (e.g. whether or not age/gender of participants, settings/contexts may have contributed to disconfirming cases or whether or not reviewers considered how findings might be interpreted from different cultural or social perspectives)10 (52.6)9 (47.7)
82Refutational analysis: reviewers state if refutational translation was not possible and why2 (10.5)17 (89.4)
83LOA synthesis: reviewers state what they mean by LOA10 (52.6)9 (47.7)
84LOA synthesis: reviewers state which reviewer(s) were involved in the LOA7 (36.8)12 (63.1)
85LOA synthesis: reviewers state which studies were included in the LOA14 (73.6)5 (26.3)
86LOA synthesis: reviewers state explicitly and transparently steps taken in LOA9 (47.7)10 (52.6)
87LOA synthesis: reviewers state whether or not social and cultural factors were considered within the LOA6 (31.5)13 (68.4)
88LOA synthesis: reviewers state clearly their LOA findings; this could be in text or grid or table format15 (78.9)4 (21.0)
89LOA synthesis: reviewers state if LOA was not possible and why not0 (0)19 (100)

TABLE 14

Audit standards with percentages for phase 6

Meta-ethnography phase 6Standard numberReporting criteriaMeta-ethnography publications, n (%)

Number of standards applicable to any meta-ethnography: 3

Number of standards applicable only to some meta-ethnographies: 2

Standard met in full or in partStandard not met or not reported
Phase 6: synthesising translations90Synthesising translations: reviewers state the methods used to develop overarching concepts (‘synthesised translations’)13 (68.4)6 (31.5)
91Synthesising translations: reviewers state their new (third order) interpretation(s) in text and/or visually (e.g. as a model, theory or film)19 (100)0 (0)
92Synthesising translations: reviewers state which reviewer(s) were involved in this process8 (42.1)11 (57.8)
93Synthesising translations: reviewers state if development of a new theory, interpretation or model was not possible and why not1
94If the meta-ethnography included lots of studies (≥ 50), reviewers state if they adapted their methods to remain grounded with original data/avoid losing conceptual richness (e.g. if they translated and synthesised original studies in clusters?)1

TABLE 15

Audit standards with percentages for phase 7

Meta-ethnography phase 7Standard numberReporting criteriaMeta-ethnography publications, n (%)

Number of standards applicable to any meta-ethnography: 15

Number of standards applicable only to some meta-ethnographies: 0

Standard met in full or in partStandard not met or not reported
Phase 7: expressing the synthesis95Include the term meta ethnography (or meta-ethnographic approach) in the title, abstract and/or keywords17 (89.4)2 (10.5)
96Provide clear abstracts for readers: number of included studies stated17 (89.4)2 (10.5)
97Provide clear abstracts for readers: number of studies synthesised stated16 (84.2)3 (15.7)
98Provide clear abstracts for readers: differentiate reporting of primary study findings from new interpretation16 (84.2)3 (15.7)
99Provide clear abstracts for readers: connect key findings to policy or practice17 (89.4)2 (10.5)
100State target audience(s) for findings11 (57.8)8 (42.1)
101Present interpretive findings19 (100)0 (0)
102When quotations are used, state where they originate from (e.g. original study participants, original study authors, reviewer’s own field notes)16 (84.2)3 (15.7)
103Present to readers translations and syntheses clearly related to the original data17 (89.4)2 (10.5)
104State how reviewers encouraged reflexivity in the development of their new interpretation, for example deliberated their findings from different perspectives (e.g. their target audience, epistemology, academic discipline, health background, culturally, etc.)5 (26.3)14 (76.3)
105State how reviewers took steps to keep their interpretations grounded with original data6 (31.5)13 (68.4)
106Highlight limitations of the review to readers16 (84.2)3 (15.7)
107Discuss how limitations of the review may have affected validity and reliability, for example:
  • the order of studies reviewed and synthesised
  • impact of any sampling (e.g. if only used studies with similar methods or epistemology)
  • influence of team member backgrounds
  • context of original studies
  • context of review (e.g. sole reviewer or funding)
  • number of included studies affected translation and/or synthesis
  • limitations of the primary studies
13 (68.4)6 (31.5)
108Possible limitations of the new theory, interpretation or model (e.g. if findings apply only to certain groups, countries)13 (68.4)6 (31.5)
109Clearly indicate how findings relate to potential end-users (e.g. application of findings to policy and/or practice)17 (89.4)2 (10.5)
Copyright © Queen’s Printer and Controller of HMSO 2019. This work was produced by Cunningham et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Bookshelf ID: NBK537419

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (2.4M)

Other titles in this collection

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...