Consensus ratings from the final round (round 3) of the eDelphi studies

Maggie Cunningham; Emma F France; Nicola Ring; Isabelle Uny; Edward AS Duncan; Rachel J Roberts; Ruth G Jepson; Margaret Maxwell; Ruth L Turley; Jane Noyes

Appendix 8Consensus ratings from the final round (round 3) of the eDelphi studies

Publication Details

TABLE 16

Consensus ratings of the final round (round 3) from both meta-ethnography expert and key stakeholder expert groups

Publication Details

Copyright

Copyright © Queen’s Printer and Controller of HMSO 2019. This work was produced by Cunningham et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Publisher

NIHR Journals Library, Southampton (UK)

NLM Citation

Cunningham M, France EF, Ring N, et al. Developing a reporting guideline to improve meta-ethnography in health research: the eMERGe mixed-methods study. Southampton (UK): NIHR Journals Library; 2019 Feb. (Health Services and Delivery Research, No. 7.4.) Appendix 8, Consensus ratings from the final round (round 3) of the eDelphi studies.

TABLE 16

Consensus ratings of the final round (round 3) from both meta-ethnography expert and key stakeholder expert groups

Item	Group	Final round consensus^a
		Meta-ethnography expert group		Key stakeholder expert group
		Number of important/very important responses	%	Number of important/very important responses	%
Include the term ‘meta-ethnography’ in the title, abstract and/or keywords	Abstract	38/39	97	20/21	95
While acknowledging publication requirements and house style, the abstract should ideally contain brief details of the study’s background; aim and research question or objectives; search strategy; methods of selection, appraisal, analysis and synthesis of primary study accounts	Abstract	38/39	97	22/23	96
While acknowledging publication requirements and house style, the abstract should ideally contain: main findings including a description of the model, conceptual framework, theory and the number of studies synthesised	Abstract	39/39	100	23/23	100
While acknowledging publication requirements and house style, the abstract should ideally differentiate between reported findings of the primary studies and of the synthesis	Abstract	25/39	64	18/23	78
While acknowledging publication requirements and house style, the abstract should ideally contain implications for policy, practice and/or theory	Abstract	33/39	85	21/22	95
State the research or knowledge gap to be filled by the synthesis	Introduction	39/39	100	23/23	100
Describe the availability of qualitative data that potentially could be synthesised (e.g. from an exploratory scoping of literature, if done)	Introduction	29/39	74	21/22	95
Explicitly state review aim(s) compatible with the intention to produce a new theory, new conceptual framework, configuration (interpretation) of data or new model and give details of any refinements to the initial aim(s)	Introduction	37/39	97	23/23	100
Explicitly state review question(s) (or objectives) and give details of any changes or refinements to the initial question(s)/objectives	Introduction	36/39	92	23/23	100
State the context of the synthesis (e.g. any funding sources for the synthesis; time scales for the synthesis conduct; political, cultural, social, policy or other relevant contexts). Refer to existing frameworks for guidance on how to specify the review context	Introduction	22/39	56	23/23	100
State why meta-ethnography was considered the most appropriate qualitative synthesis approach and whether or not use of other approaches was considered	Method	31/39	79	23/23	100
Approach to searching. Indicate whether or not the search(es) was (were) pre-planned (comprehensive search strategies to seek all available studies) or iterative (to seek all available concepts until theoretical saturation is achieved)	Method	39/39	100	23/23	100
State the rationale for the literature search strategy (e.g. how this was informed by purpose of the synthesis). Refer to existing frameworks for guidance on how to determine if the context in primary study accounts is sufficiently relevant to the context specified in the review question	Method	35/39	90	22/22	100
Searching processes. While considering specific requirements of the journal or other publication outlet, state and provide a rationale for how the literature searching was done. Provide details on all the sources accessed for information in the review (e.g. use of any electronic databases, grey literature databases, relevant organisational websites, experts, information specialists, generic web searches, hand searching, reference lists). Where searching in electronic databases has taken place, the details should include, for example, name of database, search terms, dates of coverage and date last searched. Provide the rationale for selection of the data sources	Method	39/39	100	23/23	100
If iterative or expansive searches were used, provide a rationale for deciding when to stop searching	Method	39/39	100	23/23	100
Rationale for years covered by data searches	Method	37/39	95	22/23	96
Study screening methods. Describe the process of study screening (e.g. by title, abstract and full-text review, number of reviewers who screened studies)	Method	38/39	97	22/22	100
Study selection. Specify the inclusion/exclusion criteria (e.g. in terms of population, language, year limits, type of publication, study type, methodology, epistemology, country, setting, type of qualitative data, methods, conceptual richness of data, etc.). Refer to existing frameworks for guidance on how to determine if the context in primary study accounts is sufficiently relevant to the context specified in the review question	Method	38/39	97	23/23	100
State if and how quality appraisal of primary study accounts was conducted and give a rationale for this decision	Method	38/39	97	23/23	100
State whether papers were read in full or in part and specify the reading process or strategy used	Method	37/39	95	22/23	96
Data extraction methods and process. Indicate which sections of the primary study accounts were extracted and analysed (e.g. if used data from anywhere in the publication or just findings and discussion sections, etc.)	Method	35/38	92	23/23	100
Data extraction methods and process. State how the extracted data from the primary studies were recorded (e.g. how was a computer software program or other method used). If publication requirements prevent full reporting, state where readers can access these data in full (e.g. a project website, online files)	Method	38/39	97	21/22	95
Data extraction methods and process. State in which order primary study accounts had data extracted from them (e.g. chronological or starting with an ‘index’ paper, and rationale for that order)	Method	30/39	77	14/21	67
Contributions of reviewers. Identify who was involved in literature searching and screening, reading of studies, data extraction, translation and synthesis. State whether or not processes were conducted independently by reviewers and whether or not data were checked for accuracy (e.g. for screening/data extraction). (Depending on publication requirements, this information could be provided in the ‘Methods’ or the ‘Author contributions’ section)	Method	35/38	92	22/23	96
Reviewers should state what they understand by the synthesis terminology they have used (whichever terms are used) (e.g. metaphor, concept, theme, first-, second- and third-order constructs, LOA synthesis, refutational translation, reciprocal translation)	Method	38/39	97	21/22	95
Determining how studies are related. State which aspect(s) of the studies was (were) compared in order to determine how they are related (e.g. the theoretical approach and/or concepts/metaphors, aims, focus, contexts, overarching explanations for the phenomenon). State how the studies were compared (i.e. the methods and process of comparison). State how studies relate to each other (e.g. reciprocally, refutationally, and/or are about different aspects of the topic)	Method	39/39	100	22/22	100
Translation and synthesis processes. Clearly differentiate between the different levels of interpretation in the translation and synthesis process by: listing the data from primary studies to be synthesised (concepts, themes, metaphors, second-order constructs, explanations); stating the translated and synthesised concepts developed by reviewers (this could be in a table, grid and/or narrative format); showing the inter-relationships between the data from primary studies and the reviewers’ concepts (e.g. in grids, tables, visual diagrams). Depending on publication requirements, this information could be provided across the methods and findings sections and elsewhere (e.g. project website, online files)	Method	39/39	100	20/20	100
Translation and synthesis processes. Report steps taken to preserve the context and meaning of the relationships between concepts within and across studies. Refer to existing frameworks for guidance on how to determine the context of primary study accounts	Method	38/39	97	22/22	100
Translation and synthesis processes. State the order in which studies were translated/synthesised (e.g. chronologically from the earliest or most recent) and the rationale for this	Method	28/39	72	11/21	52
Translation and synthesis processes. State whether the translation conducted was reciprocal or refutational, or both (depending on how reviewers have conceptualised reciprocal and refutational translation). State if refutational synthesis was not conducted and say why not	Method	34/39	87	17/18	94
Translation and synthesis processes. Translation methods used (for reciprocal and/or refutational translation) to translate meaning from one study into another are specific and clearly stated (e.g. give one or more examples of how this was done)	Method	38/39	97	22/22	100
Translation and synthesis processes. State whether or not and how the contexts of the primary study accounts were considered throughout the analysis and synthesis process. Refer to existing frameworks for guidance on how to determine the context of primary study accounts	Method	32/39	82	22/22	100
Translation and synthesis processes (synthesising translations). State the methods used to develop overarching concepts (‘synthesised translations’)	Method	39/39	100	22/22	100
Translation and synthesis processes. State if a LOA synthesis was conducted and if not, say why not	Method	33/39	85	21/21	100
Translation and synthesis processes. State explicitly how the LOA synthesis was conducted	Method	37/39	95	20/20	100
If a single reviewer conducted the synthesis, give details of how potential alternative interpretations or explanations were considered in the translation and synthesis processes	Method	38/39	97	23/23	100
Clearly describe and give a rationale for any adaptations or modifications to Noblit and Hare’s²⁵ approach	Method	32/39	82	20/20	100
Translation and synthesis processes (synthesising translations). Describe the new theory, conceptual framework, model, configuration or interpretation of data developed from the synthesis. If development of a new theory, conceptual framework or model was not possible, state why not	Method	38/39	97	23/23	100
Provide details on the number of primary study accounts assessed for eligibility and included in the review with reasons for exclusion at each stage as well as an indication of their source of origin (for example, from searching databases, reference lists and so on). You may consider using the example templates (which are likely to need modification to suit the data) that are provided. If publication requirements prevent full reporting, state where readers can access these data in full (e.g. a project website, online files)	Findings	37/38	97	22/23	96
State how many and which studies were synthesised	Findings	39/39	100	23/23	100
Study characteristics. Present the characteristics of the included studies (e.g. year of publication, country, population, number of participants, data collection, methodology, analysis, research questions, setting, study funder, participant characteristics relevant to the aim such as, but not limited to, gender, age, socioeconomic status). If publication requirements prevent full reporting, state where readers can access these data in full (e.g. a project website, online files)	Findings	37/39	95	23/23	100
Study characteristics. Describe the context of included studies (depending on which contexts are relevant to the aim). Refer to existing frameworks for guidance on how to specify the context of primary study accounts	Findings	38/39	97	22/22	100
Study selection results. Identify the number of studies screened and provide reasons for study exclusion (e.g. for comprehensive searching provide numbers of studies screened and reasons for exclusion indicated in a figure/flow chart; for iterative searching describe reasons for study exclusion and inclusion based on modifications to the research question and/or contribution to theory development)	Findings	37/39	95	22/22	100
Translation and synthesis processes (synthesising translations). State the interpretive findings of the translation, the synthesis of translations, the LOA synthesis, and any new model, conceptual framework or theory developed in a narrative, grid, table and/or visually (e.g. as an illustration, diagram or film)	Findings	39/39	100	20/20	100
When quotations are used, state where they originate from (e.g. primary study participants, primary study authors, reviewer’s own field notes)	Findings	38/39	97	23/23	100
Summarise the main interpretive findings of the translation and synthesis, taking into account the synthesis objective(s), review question(s), focus and intended audience(s)	Discussion	38/38	100	22/23	96
State the qualitative research expertise of reviewers. (Depending on publication requirements, this information could be provided in a different section, e.g. the ‘Author contributions’ section)	Discussion	20/39	51	15/23	65
State reviewer(s)’ background(s) or perspectives that may have influenced the interpretive process such as, but not limited to, epistemological position(s), professional position(s) held, academic discipline, organisation(s) or professional bodies represented. [Depending on publication requirements, this information could be provided in a different section (e.g. the ‘Author contributions’ section)]	Discussion	33/39	85	21/23	91
Discuss the strengths and limitations of the synthesis and its findings. These should include (but need not be restricted to) (a) consideration of all the processes in conduct of the synthesis and (b) comment on the characteristics and content of the primary studies supporting the synthesis findings and how these may have affected the synthesis findings	Discussion	39/39	100	23/23	100
Identify any areas where further research is needed	Discussion	38/39	97	23/23	100
Where applicable, compare and contrast the synthesis findings (concept, model, theory) with the existing literature (e.g. other syntheses on the same topic)	Discussion	38/39	97	23/23	100
State the implications of the synthesis findings for policy, practice and/or theory	Discussion	37/39	95	23/23	100
Provide details of funding source (if any) for the synthesis, the role played by the funder (if any) and any conflicts of interests of the reviewers	Discussion	37/39	95	23/23	100
Introduction: rationale for the synthesis	Headings	36/39	92	22/22	100
Introduction: objectives, focus and context of the synthesis	Headings	38/39	97	22/22	100
Methods: rationale for using meta-ethnography	Headings	35/39	90	22/22	100
Methods: searching processes and rationale for these	Headings	38/39	97	22/22	100
Methods: selection and appraisal of primary study accounts	Headings	37/39	95	22/22	100
Methods: reading of primary study accounts and data extraction Methods: analysis and synthesis processes: determining how studies are related; translating studies; synthesising translations; LOA synthesis; model, conceptual framework or theory generation	Headings	37/39	95	22/22	100
Findings: primary study flow diagram	Headings	38/39	97	21/21	100
Findings: primary study characteristics	Headings	35/39	90	20/21	95
Findings: main findings	Headings	38/39	97	22/22	100
Discussion: summary of findings	Headings	38/39	97	22/22	100
Discussion: reflexivity	Headings	35/39	90	22/22	100
Discussion: strengths, limitations and future research directions	Headings	38/39	97	20/20	100
Discussion: comparison with existing literature	Headings	37/39	95	22/22	100
Discussion: conclusion, recommendations and implications for policy and practice	Headings	38/39	97	22/22	100
Discussion: funding and conflicts of interest	Headings	33/38	86	22/22	100

a: The levels of consensus were calculated on the number of actual responses to each item. Participants had the opportunity to indicate that they had no expertise on specific items.