NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Observational evidence on the effectiveness of endovascular aneurysm repair compared with open surgical repair of unruptured abdominal aortic aneurysms: Abdominal aortic aneurysm: diagnosis and management: Evidence review K2. London: National Institute for Health and Care Excellence (NICE); 2020 Mar. (NICE Guideline, No. 156.)
Observational evidence on the effectiveness of endovascular aneurysm repair compared with open surgical repair of unruptured abdominal aortic aneurysms: Abdominal aortic aneurysm: diagnosis and management: Evidence review K2.
Show details3.1. Identification of evidence
From an initial database of 2,964 references, we could confidently exclude 2,696 on the basis of title and abstract, leaving 268 articles to be reviewed in full. Of these, we excluded 217 (for a list with reasons, see Appendix C), leaving 51 that meet the eligibility criteria for this review. We identified 3 additional studies by consulting the reference lists of included publications, leading to a final total of 54 included studies.
3.2. Description and critical appraisal of included studies
3.2.1. Exclusively or predominantly infrarenal AAAs
We included 42 studies that explore AAAs that are likely to be infrarenal. Their characteristics are summarised in Table 2. Between them, these studies reflect cohorts recruited from 1999 to 2017. There are 18 studies that report exclusively infrarenal AAAs and 24 that either explicitly include all AAAs regardless of anatomy or adopt methods that would have led to the inclusion of some complex AAAs among a mainly infrarenal cohort. Two of the studies report UK practice; however, the majority (31/42) originate from the USA, with 3 from Canada, 1 from Japan, and the remainder from mainland Europe.
Table 2
Characteristics of included studies – exclusively or predominantly infrarenal AAAs.
Before adjustment for duplication of participants, these studies comprise a total of 914,062 participants (532,816 undergoing EVAR and 381,246 undergoing OSR). Once we deduplicate, to ensure that there are no studies with overlapping recruitment periods from the same datasource, we are left with a maximum possible sample size of 417,032 participants (238,873 EVAR and 178,159 OSR).
In studies that report unadjusted baseline characteristics, the EVAR cohort is older and less likely to be female than the OSR group. In the relatively few studies that report baseline AAA diameter, cases undergoing OSR tend to have somewhat larger AAAs than those receiving EVAR.
On evaluation according to our bespoke appraisal instrument (see Appendix D), we identified meaningful threats to the internal validity of almost all studies. Among studies adopting recommended methods of casemix adjustment, we considered only 1 study at low risk of bias (unfortunately, this study only provided data for 1 safety outcome – incidence of acute kidney injury); 2 were judged at moderate risk of bias and the remaining 11 had a high risk of bias. Common issues include a failure to account for AAA anatomy among adjustment variables, limited consideration of missing data, and a failure to examine the overlap (or ‘common support’) of propensity-matched cohorts, a step that is critical to assess the validity of those methods (see Faria et al., 2015). All-bar-1 of the studies using matched cohorts trimmed 25–50% of the unmatched population in the matching process (for Sugimoto et al., 2017, the figure was 57%). None of these studies reports details of the process by which trimming was accomplished; however, it is evident that – as would be expected – the process resulted in a concentration on the area of overlap between the cohorts. For example, matched EVAR candidates are younger than unmatched ones, so disproportionately many older cases must have been discarded, and matched OSR candidates are older than unmatched ones, so disproportionately many younger cases must have been trimmed.
Similarly, of the 28 included studies relying on naïve multivariable regression, all-bar-2 are at high risk of bias. The quality of regression analyses is generally poor: only 2 studies take any account of AAA anatomy (Locham et al., 2017, and Locham et al., 2018b), and only 2 explore interactions between treatment assignment and any other covariates (Locham et al., 2018b, and Wald et al., 2006). The majority of studies attempt to consider more covariates than is plausible, given the number of events available in their datasets; an extreme example is Gupta et al. (2012), which reports a multivariable logistic regression based on 5 deaths with 4 covariates (that had been selected from a larger pool of unreported size by stepwise elimination).
We had fewer concerns about the external validity of the evidence: 5 studies were judged partially applicable to our whole target population, as they focus on restricted cohorts defined by age and/or comorbidity.
3.2.2. Complex AAAs
We included 12 studies; 6 use a recommended method to adjust for factors that may confound treatment effect and a further 6 studies use naive multivariable regression alone. Their characteristics are summarised in Table 3. Eight focus explicitly on fenestrated or fenestrated/branched EVAR; the other 4 look at all complex AAA. Between them, they reflect cohorts recruited from 2001 to 2016. Four studies report experience from European centres, while 9 include participants from the USA (1 has both). All 8 of the USA-only publications were derived from the same datasource, the American College of Surgeons’ National Surgical Quality Improvement Program.
Table 3
Characteristics of included studies – complex AAAs.
These studies comprise a total of 11,321 participants (3,974 undergoing EVAR and 7,347 undergoing OSR). Once we deduplicate, to ensure that there are no studies with overlapping recruitment periods from the same datasource, we are left with a maximum sample size of 4,363 participants (980 EVAR and 3,383 OSR).
Two of the studies are at moderate risk of bias and the remaining 10 have a high risk of bias. As for infrarenal AAAs, common issues include a failure to account for AAA anatomy among adjustment variables, limited consideration of missing data, and a failure to examine the overlap (or ‘common support’) of propensity-matched cohorts. In the studies using matched cohorts, 25–50% of the unmatched populations were trimmed in the matching process. None of these studies reports details of the process by which trimming was accomplished.
The regression methods are suboptimal in all 6 cases: each study attempts to include too many covariates, relative to the number of events observed, and no interactions with the treatment effect are considered.
Ten of the 12 studies are directly applicable to our decision problem; 1 study only includes octogenarians and another is only partially applicable because it includes some supradiaphragmatic thoracoabdominal aortic aneurysms, which are beyond the scope of this guideline (with a greater proportion of these among the EVAR cohort).
3.3. Evidence synthesis – exclusively or predominantly infrarenal
3.3.1. Perioperative mortality
Figure 1 shows a random-effects meta-analysis of casemix-adjusted observational evidence reporting perioperative mortality with EVAR and OSR. It suggests that people undergoing OSR have approximately 3 times higher odds of death than those receiving EVAR.
The results of this analysis are conspicuously similar to those found in RCTs: the data are strongly consistent with a null hypothesis of no difference between RCTs and observational studies (p=0.847). If we restrict the dataset to the 8 casemix-adjusted observational studies that explicitly limit their datasets to infrarenal cases, the pooled OR is 0.41 (0.27 to 0.62) – marginally less favourable to RCTs for EVAR.
Figure 2 plots the same data as Figure 1, but also illustrates the period over which each study recruited its participants. It is commonly asserted that the balance of benefits between EVAR and OSR is likely to have shifted in EVAR’s favour in the time since the RCTs were undertaken, owing to improvements in devices and/or operator technique. If this were true in the domain of perioperative mortality, we would expect to see a secular trend with relative effects becoming increasingly favourable for EVAR, and the most recent evidence would show distinctly greater benefit than was observed in the RCTs. Neither of these things appears true in Figure 2: there is no appearance of a trend (as confirmed in formal meta-regression, which finds the data to be consistent with a null hypothesis of no change over time; p=0.318), and the pooled estimate from the RCTs appears to be a valid estimate of effect over the entire period covered.
There is no evidence of publication bias in funnel plots (not shown) or on formal hypothesis testing (Egger’s p=0.240).
In addition to this between-study evidence, Schermerhorn et al. (2015) provide an analysis of within-study time-trends. This shows that, for people undergoing repair in the period 2001–2008, perioperative mortality declined with both EVAR (p=0.001) and OSR (p=0.013), with no evidence that the year-on-year change was different between the 2 approaches (interaction p=0.129).
3.3.2. Duration of procedure
Only 1 observational study reports duration of procedure. Figure 3 summarises the evidence.
Figure 3
Duration of procedure – meta-analysis of casemix-adjusted observational data, with comparison with RCTs.
The data are not consistent with between-design homogeneity (p<0.001). It is difficult to ascribe a single reason to this apparent discrepancy: Sugimoto et al.’s study (2017) is more recent that the RCTs, but it also took place in a very different setting (Japan).
3.3.3. Perioperative complications
3.3.3.1. Perioperative complications – all
Figure 4 presents relative differences in proportions of participants experiencing at least 1 perioperative complication (according to authors’ definitions).
Statistically, there is evidence of heterogeneity, especially in the substratum relating to naive multivariable regressions in studies including all types of AAA. However, this is almost entirely confined to numerical variation between large studies that all agree that EVAR is associated with substantially fewer perioperative complications. The observational data appear consistent with those reported in RCTs.
Figure 5 examines evidence for an interaction between treatment effect and recruitment period. There is no suggestion of a secular trend.
3.3.3.2. Perioperative complications (cardiovascular)
Figure 6 summarises data relating to perioperative cardiovascular complications. Again, the quantitative heterogeneity in observational results reflects varying findings regarding the magnitude, rather than the direction, of effects: all included studies agree that EVAR is associated with at least half the odds of cardiovascular complication seen with OSR. The presence of statistically detectable heterogeneity is unsurprising, in a dataset including 2 very large observational studies with tight confidence intervals. It would be less likely in a synthesis of randomised trials, where trials with greater precision can be expected to converge on a ‘true’ mean; we can have no such expectation here, because increasing sample size will reduce the variance, but not the accuracy, of observational estimates that are subject to varying degrees of selection bias.
The casemix-adjusted observational studies estimate a benefit for EVAR that is meaningfully different from the RCT results, in which the data are consistent with no difference.
There is no evidence of any time-trend at a study level (Figure 7), though it can be seen that relatively little of the evidence comes from more recent years.
3.3.3.3. Perioperative complications (respiratory)
Results for perioperative respiratory complications (Figure 8) are closely comparable to those for cardiovascular events. All included studies agree that EVAR is associated with at least half the odds of respiratory complication seen with OSR, with a pooled estimate of around one-fifth the odds. This time, the estimate is effectively identical to the RCTs.
As before, we do not consider the presence of statistically detectable heterogeneity among the observational studies to be an important finding, in this context.
There is no evidence of any time-trend at a study level (Figure 9); again, the evidence is heavily concentrated in an era that does not extend much beyond the RCTs.
3.3.3.4. Perioperative complications (renal)
Findings for renal complications (Figure 10) are consistent with the picture seen for other perioperative morbidity. As before, the apparent heterogeneity of effect in the observational evidence is almost entirely driven by 2 very large studies – Schwarze et al. (2009) and Schermerhorn et al. (2015) – that have qualitatively similar findings.
There is evidence that RCTs find a meaningfully smaller benefit for EVAR compared with OSR; however, on this occasion, the pooled summary estimates at least agree that EVAR is superior, at a 95% confidence level, in this domain.
There is no evidence of any time-trend at a study level (Figure 11).
3.3.4. Length of critical care stay
Only 2 observational studies report length of stay in critical care. Figure 12 summarises the evidence, showing somewhere between 1.5 and 2 days’ benefit for EVAR. This agrees with randomised evidence; both designs together produce a pooled effect estimate of −1.72 (−1.93 to –1.51), with no evidence of between-design heterogeneity (p=0.155).
Figure 12
Length of critical care stay – meta-analysis of casemix-adjusted observational data, with comparison with RCTs.
3.3.5. Length of hospital stay
Figure 13 shows a random-effects meta-analysis of casemix-adjusted observational evidence on total length of hospital stay with EVAR and OSR. It shows a clear benefit for EVAR, of the order of 5–6 fewer days’ hospitalisation.
There is, on the face of it, clear evidence of heterogeneity between the observational estimates. However, this finding predominantly reflects statistical uncertainty about the precise extent to which length of stay is shorter with EVAR, rather than meaningful uncertainty about whether a large effect exists.
There is no evidence of between-design differences in effect (p=0.471).
Plotting the effect estimates against recruitment period (Figure 14) makes it clear that all the observational evidence is approximately contemporaneous with the RCTs, and we do not have any more recent data.
3.3.6. Discharge to location other than home
We identified 3 non-duplicated, casemix-adjusted observational studies that report data on whither patients were discharged at the end of their operative admission. In particular, they allow us to estimate the relative odds of participants being discharged to somewhere other than their home. This outcome is not available in any of the RCTs.
As depicted in Figure 15, the studies agree that EVAR is associated with a substantially lower chance of discharge to somewhere other than home, although there is obvious heterogeneity between the estimates.
3.3.7. Post-perioperative survival
As depicted in Figure 16, a meta-analysis of casemix-adjusted observational studies shows that EVAR is associated with a 24% (13% to 35%) increase in the hazard of post-perioperative death compared with OSR. The observational studies reflect follow-up of between 4 and 12 years, whereas the RCTs are 12–15 years.
The RCTs estimate that EVAR raises this hazard to a lesser degree; at a 95% confidence level, we would reject a null hypothesis of no difference between RCTs and observational studies (p=0.017). A random-effects meta-analysis combining the design types estimates a pooled HR of 1.19 (1.10 to 1.28).
Figure 17 shows that there is no apparent trend in relative effect over time (p=0.544).
3.3.8. Reinterventions
Individual studies adopt heterogeneous criteria to define reinterventions – some include only ‘AAA-related’ events; some include any surgical procedure; some do not report what definition they used. For this reason, we analysed the data in 3 groups: overall (using whatever definition the authors adopted), vascular only (e.g. graft revisions, embolisations, thrombectomies) and non-vascular only (predominantly laparotomy-related procedures, e.g., incisional hernias and bowel resections).
3.3.8.1. Reinterventions (all or unspecified)
Given the clinical heterogeneity in what constitutes reintervention, it is unsurprising that there is conspicuous statistical heterogeneity between estimates (Figure 18). Nevertheless, there is a consistent finding that EVAR is associated with higher rates of reinterventions. The average effect is closely comparable to that observed in RCTs.
There is no evidence of a between-study time-trend in meta-regression (p=0.387; see Figure 19). However, 1 included publication provides evidence of a within-study trend. Schermerhorn et al. (2015) show that, over the 8 years covered by their recruitment period (2001–2008), there was a year-on-year decrease in the number of reinterventions within 2 years of repair for people undergoing EVAR (p<0.001), but no such phenomenon in their OSR cohort (p=0.650). A test for interaction between reintervention trend and repair approach confirms the difference, suggesting the observed data are inconsistent with the null hypothesis that event-rates changed at a similar rate (p=0.001).
3.3.8.2. Reinterventions (vascular only)
Findings are similar in the subgroup of reinterventions that relate to vascular procedures: the included studies estimate a benefit for OSR that is heterogeneous in magnitude but consistent in direction, with a pooled effect that is closely comparable to that observed in RCTs (Figure 20). There is no evidence of a secular trend at study level (Figure 21). However, one included study (Schermerhorn et al., 2015) explores the evidence for a within-study time-trend in 2-year reintervention rates analysed by year of recruitment. These analyses show that the year-on-year reduction in all EVAR reinterventions (see 3.3.8.1, above) is primarily driven by a decrease in vascular procedures (p<0.001) which, in turn, is driven by a decrease in minor procedures, primarily coil embolization (p<0.001). In contrast, there is no evidence of a decrease in major vascular reinterventions (p=0.424). The authors summarise these data by saying the trend ‘probably represents a more conservative attitude toward the management of type 2 (side branch) endoleak’. There are no trends in the rate of vascular reinterventions following OSR over the same period (p=0.112), and a test for interaction confirms the difference between the 2 approaches, in this regard (p=0.002).
3.3.8.3. Reinterventions (non-vascular only)
The 1 included nonrandomised study shows that OSR is associated with a hazard of non-vascular reintervention that is approximately 2.5 times higher than that for EVAR.
Once again, we cannot explore time-trends between studies, but Schermerhorn et al.’s (2015) publication provides details of within-study trends; in this instance, it appears there are none (p=0.550 for EVAR; p=0.845 for OSR; p=0.533 for interaction).
3.4. Evidence synthesis – complex AAAs
3.4.1. Perioperative mortality
Of the 5 studies comparing perioperative mortality with EVAR and OSR using a recommended method to adjust for confounders, none estimates a benefit for EVAR, in marked contrast to the findings in the analogous analysis for infrarenal AAAs (see 3.3.1). The results of this meta-analysis (Figure 23) show that the data are consistent with no difference between surgical approaches.
Figure 24 shows that there is no evidence of a time-trend in effects across these studies.
3.4.2. Duration of procedure
Two casemix-adjusted studies report duration of procedure. In an echo of the infrarenal analysis (see 3.3.2), the 2 studies share the conclusion that EVAR procedures are significantly shorter than analogous OSRs, but they are at odds with each other as regards the magnitude of benefit. As shown in Figure 25, Orr et al. (2017) report a difference of just over 1 hour, whereas Tinelli et al.’s (2018) estimate is 2.5 hours. It may be relevant to note that the fEVAR and OSR cohorts in the latter study underwent their repairs at different hospitals in different countries. This makes it likely that the difference in theatre time reflects factors that go beyond the requirements imposed by the repair itself. While the study adjusted for factors related to the patients and their aneurysms, it cannot adjust for any structural and/or cultural modifiers that are exclusively associated with 1 form of repair or the other.
Figure 25
Duration of procedure – meta-analysis of casemix-adjusted observational data.
3.4.3. Perioperative complications
The same 4 studies (Fiorucci et al., 2019; Orr et al., 2017, Raux et al., 2014, Tinelli et al., 2018) provide unique data for all 4 datasets relating to perioperative complications. The pattern is similar in 3 of the 4 syntheses – all complications (Figure 26), respiratory events (Figure 28) and renal morbidity (Figure 29). There is conspicuous heterogeneity in these analyses, with Orr et al. and Fiorucci et al. estimating a substantial advantage for EVAR, while Raux et al. and Tinelli et al. find no such difference – indeed, Raux et al.’s overall estimate suggests that fEVAR is associated with more perioperative complications than OSR. This mirrors these studies’ results for perioperative mortality (see 3.4.1).
In the case of perioperative cardiovascular complications, in contrast, there is homogeneous evidence of fewer events with EVAR than OSR across all 4 studies.
3.4.3.1. Perioperative complications – all
3.4.3.2. Perioperative complications (cardiovascular)
3.4.3.3. Perioperative complications (respiratory)
3.4.3.4. Perioperative complications (renal)
3.4.4. Length of critical care stay
No studies report mean duration of critical care. However, 2 studies report medians and interquartile ranges from propensity-score-matched cohorts. In the absence of other data, we used published methods to estimate mean and variance from these quantiles (Wan et al., 2014; Luo et al., 2018). Results are shown in Figure 30. There is very clear disagreement between the 2 datapoints: Orr et al. (2017) find that people undergoing EVAR require over 2.5 fewer days’ critical care than people who have had OSR. Tinelli et al. (2018), on the other hand, report no difference between the 2 groups (in fact, it appears that the mean duration of stay is likely to be somewhat longer in the EVAR group, as the maximum observed value in that group was 286 days, compared with 11 days for EVAR; even if the very high observation in the EVAR group represents a single, extreme outlying estimate, the mean expectation of critical care time would be substantially affected in a sample of 102 participants).
Figure 30
Length of critical care stay – meta-analysis of casemix-adjusted observational data. (a)Mean difference and its variance approximated from arm-level medians and IQRs using the methods of Wan et al. (2014) and Luo et al. (2018)
3.4.5. Length of hospital stay
The 1 casemix-adjusted observational study that reports length of hospital stay (again, as median and IQR, from which we have approximated mean and SD) has findings that are extremely similar to those in the infrarenal dataset (see Figure 31 and compare with 3.3.5). Both datasets identify a substantial benefit of the order of around 5.5 fewer days’ hospitalisation for people receiving EVAR (the infrarenal RCTs reach a closely comparable conclusion, too).
Figure 31
Length of hospital stay – meta-analysis of casemix-adjusted observational data. (a)Mean difference and its variance approximated from arm-level medians and IQRs using the methods of Wan et al. (2014) and Luo et al. (2018)
3.4.6. Discharge to location other than home
There is only 1 estimate of discharge probability available for complex EVAR – see Figure 32. The estimate that EVAR is associated with around a fourfold reduction in the odds of discharge to somewhere other than home is closely comparable to the evidence found in infrarenal cases (see 3.3.6).
Figure 32
Discharge to location other than home – meta-analysis of casemix-adjusted observational data.
3.4.7. Post-perioperative survival
Two studies report long-term survival following complex EVAR or OSR. They show that, for people who survive the perioperative period, fEVAR is associated with approximately double the hazard of mortality.
Figure 33
Post-perioperative survival (long-term survival conditional on surviving the perioperative period) – meta-analysis of casemix-adjusted observational data.
3.4.8. Reinterventions
3.4.8.1. Reinterventions (all or unspecified)
One study (Tinelli et al., 2018) reports time to reintervention with fEVAR and OSR. The data suggest that people experience 3 times the rate of reintervention with fEVAR as with OSR. The authors report 2 hypothesis tests for their dataset, one of which is adjudged significant and one of which is not; our analysis suggests that the data are consistent with no difference at a 95% confidence level.
Figure 34
Reinterventions (all or unspecified) – meta-analysis of casemix-adjusted observational data.
- Results - Observational evidence on the effectiveness of endovascular aneurysm r...Results - Observational evidence on the effectiveness of endovascular aneurysm repair compared with open surgical repair of unruptured abdominal aortic aneurysms
Your browsing activity is empty.
Activity recording is turned off.
See more...