Discussion

Key Findings and Strength of Evidence

In this Comparative Effectiveness Review (CER), we reviewed 185 unique studies represented by 320 publications that evaluated stroke and bleeding prediction tools and stroke prevention strategies in patients with nonvalvular atrial fibrillation (AF).

KQ 1. Predicting Thromboembolic Risk

Our review included 61 studies comparing the diagnostic accuracy and impact on clinical decisionmaking of available clinical and imaging tools for predicting thromboembolic risk. The clinical tools assessed for this question included the CHADS₂ score (Congestive heart failure, Hypertension, Age ≥75, Diabetes mellitus, prior Stroke/transient ischemic attack [2 points]), CHA₂DS₂-VASc score (Congestive heart failure/left ventricular ejection fraction ≤ 40%, Hypertension, Age ≥75 [2 points], Diabetes mellitus, prior Stroke/transient ischemic attack/thromboembolism [2 points], Vascular disease, Age 65–74, Sex category female), Framingham risk score, ABC (age, biomarkers, clinical history), imaging tools, as well as individual patient risk factors not included in the existing tools. Current guidelines recommend that oral anticoagulation be considered in patients with CHADS₂ or CHA₂DS₂-VASc score ≥2.

The reviewed studies had varying categorical arrangements of risk scores with patients receiving antiplatelet therapy and/or anticoagulant therapy or not, making direct comparisons across studies examining these tools difficult. The CHADS₂, CHA₂DS₂-VASc, and ABC scores had the best prediction abilities given available evidence, but this advantage was incremental on an absolute basis. Imaging risk tools found conflicting results when the presence of left atrial thrombus was assessed, and there was insufficient evidence to support conclusions.

Our conclusions may be limited by the limitations in the development and validation of risk scores. Specifically, although many of the studies use clinical data sources to derive or validate these risk scores, some studies relied on billing data and institutional electronic medical records to identify patients with AF and comorbidity information. Since few of these administrative studies used a formal clinical adjudication process to validate the occurrence of a clinical event and may suffer from insufficient coding, the risk scores could underestimate stroke risk, particularly in patients incorrectly identified as having few or no comorbidities. Likewise, lack of validated results or common event definitions for the endpoints of thromboembolism and bleeding could have underestimated the performance of these risk scores. Additionally, lack of standard definitions for comorbidities such as heart failure, diabetes mellitus, hypertension, etc. could also lead to discrepancies across studies validating the various risk scores. Moreover, our review included both ambulatory and hospitalized patients, which inherently introduces bias in comparing studies and results give the heterogeneity with regard to stability of covariates, concomitant medications, stroke inducing procedures, etc.

Table 75 summarizes the strength of evidence (SOE) for the thromboembolic risk prediction abilities of the included tools. This summary table represents only those studies that evaluated the risk prediction abilities of the tools using a c-statistic. Details about the specific components of these ratings (risk of bias, consistency, directness, and precision) are available in the Results section.

Table 75

Summary of strength of evidence and c-statistic estimate for Key Question 1 (prediction of thromboembolic risk).

KQ 2. Predicting Bleeding Events

Thirty-eight studies were included in our analyses comparing the diagnostic accuracy and impact on clinical decisionmaking of clinical tools and associated risk factors for predicting bleeding events. Five different bleeding risk scores were evaluated in these studies, including ATRIA, ABC, Bleeding Risk Index, HAS-BLED, and HEMORR₂HAGES.

Of note, many included studies used administrative data sources to identify patients with AF, as well as comorbidity information. As a result, many of the included studies used different approaches to calculating the risk scores of interest due to unavailable data, particularly for the HEMORR₂HAGES and HAS-BLED scores. For example, in HEMORR₂HAGES, due to unavailability of information on genetic factors, multiple database studies left out the “genetic factors” component of the score. To further complicate this issue, not all studies described in detail whether certain factors were omitted from their calculations of these scores. Inter-study differences in approach to calculating some of the bleeding risk scores limited comparison of bleeding risk scores across populations and precluded meta-analysis. Similarly, use of administrative data in some cases prevented validation of clinical bleeding events, and this could have affected studies’ estimates of the performance of these risk scores.

Among the tools for predicting risk of major bleeding and ICH, there was a suggestion that HAS-BLED is the most accurate for predicting major bleeds in patients on warfarin although only has modest prediction abilities; but the majority of studies for other patient scenarios showed no statistically significant differences in predictive accuracy among tools. Evaluating these bleeding risk prediction scores was complicated by the fact that, though studies consistently reported event rates and c-statistics, measures of calibration, strength of association, and diagnostic accuracy were inconsistently reported.

Table 76 summarizes the SOE for the bleeding risk prediction abilities of the included tools. This summary table represents only those studies that evaluated the risk prediction abilities of the tools using a c-statistic. Details about the specific components of these ratings (risk of bias, consistency, directness, and precision) are available in the Results section.

Table 76

Summary of strength of evidence and c-statistic estimate for Key Question 2 (prediction of bleeding risk).

KQ 3. Interventions for Preventing Thromboembolic Events

Our review included 117 studies comparing the safety and effectiveness of specific anticoagulation therapies, antiplatelet therapies, and procedural interventions for preventing thromboembolic events. Among these studies, several direct oral anticoagulant agents were evaluated including thrombin inhibitors (dabigatran) and Xa inhibitors (apixaban, edoxaban, rivaroxaban, idraparinux). The included RCTs were often very large, of good quality, and considered definitive in the field. These trials were, however, limited to comparing direct oral anticoagulant therapies with warfarin or aspirin and have not involved head-to-head comparison among the newer agents. Based on these trials though, clinical leaders and professional societies have determined that these newer agents are better than the prior lone treatment of warfarin in terms of stroke prevention, side effects, and risk of bleeding.

In comparative effectiveness analyses, warfarin was found to be superior to aspirin for stroke prevention, and the combination of aspirin and clopidogrel was found to be superior to aspirin alone in patients with warfarin contraindications. Triple therapy with aspirin, clopidogrel, and warfarin did not provide any additional stroke protection beyond warfarin alone, but increased bleeding events significantly. Percutaneous left atrial appendage (LAA) closure is non-inferior to warfarin, while direct oral antithrombotics (apixaban, rivaroxaban, dabigatran) were non-inferior or superior to warfarin for stroke prevention.

Table 77 summarizes the SOE for the various comparisons and outcomes of interest. Details about the specific components of these ratings (risk of bias, consistency, directness, and precision) are available in the Results section.

Table 77

Summary of strength of evidence and effect estimate for Key Question 3 (interventions for preventing thromboembolic events).

Findings in Relation to What Is Already Known

Several scores have been developed to risk stratify patients with AF for stroke and other thromboembolic events. Given the known bleeding risks of oral anticoagulants that are used to reduce the risk of thromboembolism in patients with AF, risk scores for bleeding have also been developed to help inform therapeutic decisions. Risk scores for prediction of these events have been touted as a way of guiding antithrombotic therapy in patients with AF. In the current CER, we found that of the available risk scores, the CHADS₂ and CHA₂DS₂VASc scores are the most commonly studied. Several factors limited our ability to compare the different risk scores. Such factors included the heterogeneous patient populations and the variability in treating patients with antiplatelets and oral anticoagulants. Also, few studies used clinical validation in reporting main outcomes especially stroke, and although event rates were consistently reported, measures of predictability, calibration, and strength of association were inconsistently reported. Despite these limitations, the CHADS₂, CHA₂DS₂-VASc, and ABC risk scores appeared to be similar and to have the most prediction ability of stroke events. While some studies have explored the inclusion of biomarkers in stroke risk scores (i.e. the ABC stroke risk score), and preliminary evidence supports this score being comparable to CHADS₂ and CHA₂DS₂-VASc, the experience with ABC is limited and more data are needed on the contribution of these biomarkers to the overall risk assessment. Note that this differs from the current 2014 AHA/ACC/HRS Guidelines for the Management of Patients with Atrial Fibrillation which concludes that CHA₂DS₂-VASc was superior.

Similar to comparisons of stroke risk scores, comparisons of bleeding risk scores in our CER were hard to interpret. The difficulty in interpreting comparisons of bleeding risk scores stemmed from the different approaches to calculating bleeding risk scores, the inability to validate clinical bleeding events, and the inconsistency in reporting measures of calibration, strength of association, and diagnostic accuracy. Limited evidence favored the HAS-BLED risk score based on two studies demonstrating that it has significantly higher prediction ability for major bleeding events than other scores among patients on warfarin, but the majority of studies showed no statistically significant differences in prediction, reducing the SOE. Bleeding risk scores are currently not included in the American Heart Association/American College of Cardiology guideline recommendations on AF, and they are generally not used to decide whether to prescribe an oral anticoagulant to individual patients. However, bleeding risk scores may inform shared decision making discussions of the risks of stroke and bleeding incorporating patients’ values and preferences. As more data on stroke and bleeding risk scores emerge, it is possible that improvement in the tools and methods for risk stratification of both stroke and bleeding will be important to better individualize treatment using different oral anticoagulants in patients with AF.

With more available treatments, our review found that not only do risk algorithms need to be updated, but physician decisionmaking about when to use which agent does as well. Until recently, there was only one established oral anticoagulant available for stroke prevention in patients with AF. This single agent—warfarin—while effective when compared with placebo or antiplatelet agents such as aspirin, is associated with significant limitations from both the health system and patient perspectives. Limitations of warfarin and other vitamin K antagonists (VKAs) led to the development of several direct oral anticoagulants for stroke prevention in nonvalvular AF. It is important to note that for warfarin to be effective, time in the therapeutic range has to be high; patients in whom this is hard to achieve should be considered for other types of oral anticoagulants. Trials of dabigatran, rivaroxaban, apixaban, and edoxaban have demonstrated favorable efficacy and safety results compared with warfarin, but direct comparisons of their efficacy and safety have not been done. In addition, the trials used different dosing strategies, were performed in different health systems, used varying event definitions, and recruited populations at varying risk for stroke and bleeding. Thus, it is not possible to affirm here which medication is better, and cross-trial comparisons may not be reliable. The direct oral anticoagulants do, however, have different attributes and important advantages over warfarin and offer, after many years without options, new alternatives for the treatment of patients with nonvalvular AF who are at risk for stroke. Notably, approved doses of these medications for stroke prevention in patients with AF are: 150 and 75 mgs twice a day for dabigatran, 20 and 15 mgs once a day for rivaroxaban, 5 and 2.5 mgs twice a day for apixaban and 60 and 30 mgs once a day for edoxaban. Lower doses are generally recommended in patients with moderate to severe kidney disease.

Specifically, our review provides evidence of the following within the field of stroke prevention for patients with AF, as follows.

Applicability

Efficacy of interventions as determined in RCTs does not always translate to usual practice, where patient characteristics, provider clinical training, and available resources may differ from trial conditions. Additionally, the availability and/or specific features of interventions studied in our review may differ from those available to patients within the United States. Table 78 illustrates the specific issues with the applicability of our included evidence base by KQ.

Table 78

Potential issues with applicability of included studies.

In general, concerns about study applicability were not a major factor for this project’s body of evidence. The main issues related to applicability were concerns about short-term outcomes; concerns about large differences between demographics of study populations and community patients in terms of age, renal function, and comorbidities; and concerns about inadequate comparison therapies.

Implications for Clinical and Policy Decisionmaking

Although stroke prevention in patients with nonvalvular AF in contemporary clinical practice is complex and challenging, it is critically important given the morbidity and mortality associated with stroke events. It is noteworthy that aspirin is not an effective treatment for stroke prevention in patients with AF. The European Society of Cardiology guideline on AF confirms that evidence supporting antiplatelet monotherapy for stroke prevention in AF is very limited. It also clarifies that the bleeding risk on aspirin is not different from the bleeding risk on apixaban (AVERROES trial) while VKA and DOACs, but not aspirin, effectively prevent strokes in AF patients.²⁷ Although traditional anticoagulants like warfarin can significantly reduce the risk of stroke in patients with AF, the bleeding risk is increased with these agents, potentially attenuating their effects. In addition, use of warfarin is further challenged by numerous interactions with food items and other medications, inability to predict the best dose in an individual patient, and the need for regular monitoring of INR. The direct oral anticoagulants promise improved efficacy with reduction in bleeding events, especially intracranial bleeding, and more predictable pharmacokinetics. However, the long-term effects of these agents in broad populations have not been established. Therefore, clinicians are constantly struggling to find the right balance between efficacy and risk in the use of these therapies in this patient population. Also while bleeding risk scores are generally not used to decide whether or not to use an oral anticoagulant in a given patient, high scores may help guide intensity of patient follow-up and monitoring.

Despite the availability and validation of numerous tools for both stroke and bleeding risk assessment in patients with nonvalvular AF, meaningful comparisons of the tools could not be performed in this CER due to the heterogeneous patient populations, the variability in treating patients with antiplatelets and oral anticoagulants, the lack of clinical validation of endpoints, and the underreporting of measures of predictability, calibration, and strength of association. In their most recent update in 2014, the AHA/ACC published guidelines that acknowledge the limitation of current risk tools to identify patients at high risk for thromboembolic risk. The 2014 guideline recommends all patients with a CHA₂DS₂-VASc score of ≥ 2 be considered for oral anticoagulant therapy. This guideline, along with other professional guidelines, recommends use of the CHA₂DS₂-VASc score for assessment of stroke risk in AF patients. Our review highlights the similar evidence supporting the prediction abilities of CHA₂DS₂-VASc, CHADS₂, and the ABC stroke risk scores. Whether biomarkers such as brain natriuretic peptide, C-reactive protein or troponin can enhance clinically-based scores and as a result be incorporated in guideline recommendations remains to be seen. Also, the current ACC/AHA guidelines¹⁷ do not recommend use of bleeding risk scores. Whether biomarkers (e.g., brain natriuretic peptide, C-reactive protein or troponin) can enhance these scores is uncertain. Another gap in the evidence is the absence of randomized controlled trials comparing the direct oral anticoagulants head-to-head. For effective stroke prevention in patients with AF, clinicians will have to understand the risk and benefits, indications, side effects, and monitoring patients taking direct oral anticoagulants (e.g. renal function as dose may need to be adjusted), further complicating treatment decisions in patients with AF.

With the growing prevalence of digitized medical records, there is an opportunity to monitor the real world uptake of the direct oral anticoagulants. Additionally, with these electronic records, there will be the opportunity to continue to evaluate and modify risk prediction tools to improve their prediction for stroke and bleeding risk, particularly with these newer anticoagulants diffusing into clinical practice. Also, newer clinical markers (e.g. MRI to assess scar), comorbidities (i.e., renal failure, etc.) and biomarkers should be tested and validated with or alongside current risk tools to improve their prediction of both stroke and bleeding risks. Additionally, more prescriptive guidelines on how to use risk scores and apply necessary therapies, possibly in the form of physician decision support tools, will be important for clinical decisionmaking. Data on efficacy, effectiveness and safety of direct oral anticoagulants are needed on important patient groups such as patients with severe kidney disease including end stage renal disease on dialysis and patients older than 75 years of age. Also, although not part of this review, the best strategies should be defined for patients undergoing procedures including cardioversion and catheter ablation of AF, switching among anticoagulant therapies, and starting or restarting anticoagulant therapy in patients with previous major bleeding events.

As new interventions are introduced, determining their relative risks and benefits in the overall scheme for stroke prevention in AF is critically important in order to minimize the use of less efficacious, less safe, and more expensive therapies. Although the results of the current review are largely consistent with existing guidelines, they do help identify gaps in the evidence base and areas of needed future research, particularly as agents are rapidly entering into broader clinical practice.

We also explored relevant ongoing studies within clincialtrials.gov to determine whether any of these studies could impact our findings. One such study targeted KQ1 and KQ2. The “Thromboembolic and Bleeding Risk Stratification in Patients With Nonvalvular Atrial Fibrillation” or FASTRHAC study is currently listed as recruiting (target enrollment of 825 patients) and is looking to be complete at the end of 2020. Twenty additional studies evaluated the safety and effectiveness of different treatment strategies. These studies are summarized in Table 79 and represent 13 ongoing RCTs and 7 ongoing observational studies. Of note are the four ongoing studies of devices representing over 4000 patients—three of these studies however will not be completed until 2020. Also note that there are two RCTs which directly compare direct oral anticoagulants although the first of these studies will not be finished until December 2018.

Table 79

Ongoing studies potentially relevant to Key Questions.

Limitations of the Evidence Base and the Comparative Effectiveness Review Process

Our findings have limitations related to the literature and our approach. Important limitations of the literature across the KQs include inconsistency across studies that assess prediction tools for thromboembolic or bleeding risk in terms of the methods used and findings reported; and the lack of RCTs which directly compare specific stroke prevention therapies.

Our review methods also had limitations. Our study was limited to English-language publications and excluded studies conducted exclusively in Asia, Africa, or the Middle East and observational studies with less than 1000 patients which studied only pharmacological interventions. It was the opinion of the investigators that the resources required to translate non-English articles, to include areas of the world where clinical practice differs significantly from standards in the United States, or findings from small pharmacologic observational studies would not be justified by the low potential likelihood of identifying relevant data which would change decisionmaking. Note this exclusion does not restrict observational studies that target nonpharmacologic interventions where evidence is more sparse and smaller studies may have a larger impact on the review findings. We also limited our analysis to studies published since 2000. Given the rapidly changing treatment alternatives for stroke prevention for patients with AF this recent literature was considered the most relevant to today’s clinical and policy uncertainties.

Research Recommendations

In our analyses, we have identified several areas for recommended future research. Specifically, many of the available studies for KQ 1 and KQ 2 had methodological issues that point to limitations of the current evidence base. Many studies’ utilization of administrative data sources led to different approaches to calculating the risk scores of interest due to unavailable data (notably for the HEMORR₂HAGES and HAS-BLED scores). Similarly, use of administrative data in some cases prevented validation of clinical stroke/bleeding events, which could have affected studies’ estimates of the performance of these risk scores. Finally, though studies consistently reported c-statistic as a measure of model prediction, other relevant statistics (including measures of calibration, strength of association and diagnostic accuracy) were inconsistently reported. Further studies are needed that: (1) utilize complete data; (2) use validated clinical outcomes; and (3) compare all available risk scores using consistent and appropriate statistical evaluations.

We can identify well patients at risk for stroke, who usually are the same patients at high risk for bleeding. Thus, there is a need for a score that could be used for decisionmaking about antithrombotic therapy in AF patients taking into account both thromboembolic and bleeding risks. Scores that identify only patients at risk for stroke or only those at risk for bleeding are not so helpful since the clinical factors in these scores are usually similar and treatments which reduce one or the other risk may increase the other for the same patient. Another challenge is that both stroke events and bleeding events are on a spectrum of severity and therefore predicting overall stroke might not align with outcomes that matter most to patients. For example, some strokes may have symptoms lasting <24 hours with complete resolution, whereas others can cause death. Additional studies utilizing prospectively constructed databases with longer-term outcomes data that compare all available risk prediction scores would be of great use in better clarifying which risk score system is superior in predicting major bleeding or thromboembolic risk. Specific to bleeding risk, additional prospective comparisons of the standard deviation of transformed international normalized ratio (SDT_INR) and time in therapeutic range (TTR) are needed to establish which variable has better predictive accuracy for major bleeding.

Additionally, even assuming an optimal risk prediction score can be identified, further work is needed to clarify how scores should be used prospectively in clinical practice.

Specific to treatment strategies, although recent years have been exciting in stroke prevention and development of new agents as alternatives to warfarin, there are several evidence gaps that remain and should inform future research. It is important to have new studies with head-to-head comparisons of available prevention strategies. Given variability in patient populations, concomitant therapies, and underlying patient care, cross-trial comparisons in this field is of limited use. Patients with AF usually have other comorbidities that also require the use of other antithrombotic agents. There are many antithrombotic agents available at different doses for different clinical indications. There is a need for further study of these agents, particularly focusing on methods of monitoring adequacy of anticoagulation, as well as the development of antidotes for severe bleeding events. There is a need for studies assessing the safety and effectiveness of different combinations of antithrombotics (anticoagulants and antiplatelet agents) at different doses, as well as their duration. In frail older patients, there may be concerns about using anticoagulation in the presence of multimorbidity due to a higher prevalence of pre-existing conditions that predispose to bleeding, concomitant interacting medications (antiplatelet therapy, nonsteroidal anti-inflammatory drugs), and additional complicating conditions such as risk of falls. Such a patient population needs further study.

There are also many novel invasive treatments for AF but the evidence remains sparse about these interventions. Studies need to be conducted in patients who receive these procedures to determine if and how anticoagulation strategies should be modified in patients receiving these procedures.

Finally, despite all the potential advantages of the direct oral anticoagulants demonstrated in the clinical trials when compared with warfarin, except for dabigatran, these drugs still do not have an approved immediate antidote. Similarly, for warfarin-treated patients, although there are data showing that fresh frozen plasma or vitamin K can help in normalizing INRs, there are not good data on actually stopping or reversing bleeding events for such warfarin-treated patients. Once a bleed occurs, the event has happened, and regardless of the original treatment strategy, it is not clear that any reversal or antidote will alter patient outcomes. Therefore, a focus should be on preventing bleeds—in particular, fatal bleeds. The shorter half-life of the direct oral anticoagulants may help in the management of bleeding episodes in patients receiving these drugs and should provide comfort that bleeding can be controlled without an antidote. Other areas worthy of further study relate to the use of the direct oral anticoagulants in patients with severe kidney disease.

Conclusions

Overall, we found that CHADS_2, CHA₂DS₂-VASc, and ABC scores have similar evidence regarding their ability to predict stroke risk in patients with AF, whereas HAS-BLED has the best evidence to predict bleeding risk. Imaging tools require further evidence in regard to their appropriate use in clinical decisionmaking. Additionally, simple clinical decision tools are needed that incorporate both stroke risk and bleeding risk to assist providers choosing agents in patients with AF. Additional work will be required to develop risk tools for patients to discriminate those individuals with AF where the bleeding risk may be high enough to warrant more intensive follow-up and monitoring. These tools could be embedded into electronic medical record systems for point-of-care decisionmaking, developed into applications for smartphones and tablets, or be delivered via web-based interfaces. Additional evidence of the use of these stroke and bleeding risk scores (and clinical decision tools which balance these risks) among patients on therapy is also required.

DOACs (specifically apixaban and dabigatran) demonstrate reductions in stroke events and reductions (apixaban) or similar (dabigatran) rates in bleeding events when compared with warfarin while rivaroxaban was similar in both benefits and harms with warfarin. Comparative effectiveness of these direct oral anticoagulants as compared to one another however is limited by the lack of randomized studies directly comparing their safety and effectiveness.