Risk Stratification

To be included for research question 1, SRs must have the term “systematic review” or “meta-analysis” in the title or elsewhere in the text; include a detailed description of comprehensive selection criteria and search methods (i.e., as described in Assessment of Multiple Systematic Reviews [AMSTAR] checklist item 3, search at least two electronic sources, adequately report years searched and databases used, key words and/or MeSH terms, and where feasible, the search strategy provided); assess the quality (or risk of bias) of included studies; and synthesize the findings quantitatively and/or qualitatively. For studies that meet the other criteria but do not perform a quality assessment of the primary included studies, the studies will be included as SRs if they have relevant outcomes or subgroups that are not present in any of the other included SRs, or if they include unique primary studies not reviewed by another SR. In this case, quality assessment of the primary studies will be conducted de novo in duplicate. SRs will be excluded if they do not meet the selection criteria outlined in Table 1, if they are duplicate publications, or if they were published prior to 2006. Multiple publications of the same SR will be excluded unless they provide additional outcomes of interest. Older SRs (based on publication year) identified in the literature search results will be excluded if they are superseded by an updated SR, or if all the included studies in the older SRs are included in newer SRs. However, two SRs that include identical primary studies will be included if they report different outcomes or identical outcomes but present different subgroup analyses. The degree of overlap between SRs with overlapping primary studies will be judged by building a matrix of included studies in the SRs, which will be reported within the results section of a final report. If an SR has unique primary studies, but they are limited to case reports, they will be excluded in the interest of appraising higher-quality evidence and in acknowledgement of design limitations resulting in high risk of bias. Similarly, SRs with unique primary studies that do not meet the inclusion criteria will be excluded if all other studies are included within another SR. A list of excluded studies, with reasons for exclusion after full-text review, will be provided.

Diagnostic Pathways and Diagnostic Imaging Studies

For questions being addressed by a review of primary studies (research questions 2 and 3), studies will be excluded if they do not meet the selection criteria outlined in Table 1, if they are case reports, or if they are duplicate publications. If there are multiple publications of the same study, the less recent will be excluded unless it provides additional information on the outcomes of interest. There is no restriction regarding the duration of time between symptom presentation and assessment, or length of follow-up. Studies will be excluded if they are not published in English or French. Further, conference abstracts, published thesis documents, and evidence that have not been peer-reviewed will not be included.

Risk Stratification

A summary of SR characteristics — including the total number of SRs by population, intervention, comparator, outcomes, and study design (PICOS) elements, countries and years of publication, and findings — will be provided. Additionally, information from SRs — including characteristics, numbers, types, and years of publication of primary studies included in the SRs — will be summarized. In the case that more than one SR is included for a given intervention, the comparator(s), and outcome(s) of interest, and any overlap of included studies among SRs will be described and presented (e.g., by preparing a matrix of included studies in the SRs).

Diagnostic Pathways and Diagnostic Imaging Studies

A summary of primary study characteristics — including the total number of studies by PICOS elements, and countries and years of publication — will be provided in the form of tables and a narrative summary.

Risk Stratification

A narrative synthesis of the results of included SRs will be conducted for research question 1. The findings will be grouped by outcome, with diagnostic test accuracy, clinical utility, and safety outcomes grouped separately. No re-synthesis of the findings from primary studies will be conducted. Results will be represented as reported in SR study reports, including a summary estimate and confidence interval (CI), measure of heterogeneity, and number of studies and participants contributing to each estimate, as available. Tables will be developed to present results by outcome. This is intended to accompany the narrative summary, to ensure consistency of presented information across all included SRs and to facilitate comparisons by the reader. Results will be synthesized by outcome for the overall study population, and also for each subgroup listed in Table 1.

Diagnostic Imaging Studies

In addition to the SR of primary studies described above, a narrative synthesis of the results of the scoping review of SRs on diagnostic imaging studies will be presented to contextualize the findings of the primary study review conducted for questions 2 and 3. The findings will be grouped by outcome, with diagnostic test accuracy, clinical utility, and safety outcomes grouped separately. No re-synthesis of the findings from primary studies will be conducted. Results will be represented as reported in SR study reports, including a summary estimate and CI, measure of heterogeneity, and number of studies and participants contributing to each estimate, as available. Tables will be developed to present results by outcome. This is intended to accompany the narrative summary, to ensure consistency of presented information across all included SRs and to facilitate comparisons by the reader. Results will be narratively synthesized by outcome for the overall study population, and also for each subgroup listed in Table 1.

Meta-Analysis of Diagnostic Test Accuracy Studies

Based on the scoping review, studies that assessed diagnostic accuracy varied in terms of their applied reference standards, diagnostic pathways, settings, and patient populations. Thus, it is unclear whether pooling will be appropriate. If MA is deemed inappropriate, studies that report on diagnostic accuracy will be reviewed and results will be reported narratively.

Between-study heterogeneity within groups of studies being considered for pooling will be assessed, using graphical presentations including forest plots and plots of sensitivity and specificity in ROC-space, and calculation of between-study variance t², summary and predictive CIs.⁵³

Reasons for observed heterogeneity will be explored by subgroup or multivariate regression analyses, given the availability of covariate data. Individual comparisons will be summarized separately, and the consistency assessed. Additional sensitivity analyses dealing with study outliers, study size, study quality, study design, and other study- or design-related factors will also be considered to establish the robustness of findings. As some variation in patient population and associated prevalence of PE is anticipated, risk of verification bias as determined during critical appraisal will be assessed in sensitivity analysis. If substantial verification bias is detected, models will be adjusted using the method of de Groot et al.⁵⁴

There are no established thresholds to determine the appropriateness of pooling of diagnostic testing studies,⁵³ so the findings of the above will be appraised in terms of usefulness in answering our clinical and policy questions. Should we decide that MA is appropriate, we will use the model developed by Rutter and Gatsonis to generate hierarchical summary receiver operating characteristic (HSROC) curves,⁵⁵ as well as pooled sensitivity, specificity, diagnostic odds ratio (DOR) values, and their 95% CIs. The area under the curve (AUC) will be used as a quantitative measure of the diagnostic accuracy of imaging studies for PE diagnosis, with values closer to 1 indicating better diagnostic performance, and a value closer to 0.5 indicating poor performance.⁵⁶ Positive and negative likelihood ratios above 10 and below 0.1, respectively, will indicate low misdiagnosis rates.

In the event that we observe substantial variation in reference standards between studies put forth for pooling, particularly involving composites of multiple tests, we will use an extension of the Rutter and Gatsonis model that allows imperfect and composite reference standards.^57,58 Unknown parameters will be estimated using a Bayesian approach with non-informative prior distributions, which allow the observed data to dominate the final estimates of sensitivity and specificity. Our models will assume independence of the combined tests acknowledging evidence that suggests the results may be affected if tests show dependency.^59,60

Exploration of heterogeneity, plotting, and MA will be conducted using the statistical software R,⁶¹ with packages mada⁶² and HSROC.⁶³

If pooling is not appropriate, a narrative synthesis will include the presentation of findings within summary tables, alongside study and clinical characteristics believed to contribute to heterogeneity, as determined during the exploration of the data. A narrative description will aim to synthesize observed test performance in the absence of an MA.

Meta-Analysis of Primary Clinical Utility and Safety Studies

The clinical utility of risk stratification strategies and imaging studies for PE diagnosis will be based on findings about the benefits (e.g., diagnostic efficiency, influence on choice of treatment and subsequent reduced exposure to imaging harms or harms of unnecessary intervention, and the indirect effect on clinical outcomes), and harms (e.g., failure rate).

Dichotomous outcomes (e.g., mortality) will be summarized, using relative risks and 95% CIs (or odds ratios and 95% CIs, if case-control studies are included). Continuous outcomes will be summarized, using differences in means and 95% CIs, if appropriate. If indicated (e.g., for quality-of-life scales), standard methods for converting between units of measurement will be used, and we will calculate standardized mean differences if possible. For outcomes reported as time-to-event and given available individual patient data in the form of a survival curve or table of events per patients at risk, analyses will be performed, using Kaplan–Meier curves and Cox regression. If studies report adjusted effects measures, the adjusted results in the primary analysis will be used, with the unadjusted result in exploratory analyses presented and comments on any differences between the two. If required measures of variance are not available, variances will be imputed if possible.⁶⁴ Forest plots will be shown for all individual summary estimates. Findings will be reported as “not statistically significant” if the 95% CI of the overall estimate includes unity for dichotomous data or includes 0 for continuous data.

Between-study heterogeneity within groups of studies being considered for pooling will be assessed, using graphical presentations (including forest plots and plots of outcomes against covariates), and calculations of the I² and Cochran’s Q test statistic. An I² ≥ 75% will be interpreted to indicate considerable heterogeneity across studies, as suggested by the Cochrane Handbook for Systematic Reviews of Interventions. Cochran’s Q test statistic — based on chi-squared, where I² = (Q-degrees of freedom)/Q — will be based on a level of significance of 10%. Clinical and methodological heterogeneity will be assessed in consultation with the clinical experts.

Reasons for observed heterogeneity will be explored by subgroup or multivariate regression analyses, given the availability of the data. Individual contrasts will be summarized separately, and the consistency assessed. Additional sensitivity analyses dealing with study outliers, study size, study quality, study design, and other study- or design-related factors will also be considered to establish the robustness of findings.

If pooling of outcome data is appropriate, summary measures and CIs for the reported outcomes will be reported. Random-effects models will be used. In the event that both randomized and non-randomized studies report on the same outcome, RCTs will be considered separately from non-randomized studies. The influence of study design will be explored in sensitivity analyses. MAs would be carried out using the Cochrane Review Manager software, version 5.3, or using R with package metafor.⁶⁵

If pooling is not appropriate, a narrative synthesis will include the presentation of findings within summary tables, alongside study and clinical characteristics believed to contribute to heterogeneity, as determined during the exploration of the data. A narrative description will aim to synthesize the direction and size of any observed effects across studies in the absence of a MA and will include an assessment of the likelihood of clinical benefit or harm.

Publication bias will be assessed using visual funnel plots, and tested using Egger’s regression test and Begg’s rank correlation test.⁶⁶

Network Meta-Analysis

The results of studies assessing the diagnostic accuracy of diagnostic imaging studies for PE diagnosis (questions 2 and 3) will be pooled, using network meta-analysis (NMA), if appropriate. Methods will be further developed in consultation with an expert if it is deemed possible to conduct the NMA. The scope of this analysis is presented in Appendix 6. All possible comparisons between diagnostic imaging studies (interventions) of interest will be evaluated.