1.1.4.1. Included studies

A systematic search was carried out to identify prognostic observational studies and systematic reviews of prognostic observational studies, which found 4,462 references (see Appendix C for the literature search strategy). Based on title and abstract screening, 4,417 references were excluded, and 45 references were ordered for full text screening. No new observational evidence was found to update the combination of measures of kidney function and markers of kidney damage to identify increased risk of progression. The three studies included in the 2014 CG182 guideline (Peralta 2011a; Peralta 2011b; Waheed 2013) were included. Six validation studies were included for kidney failure prediction equations. Therefore, 9 studies were included in total.

A second set of searches was conducted at the end of the guideline development process for all updated review questions using the original search strategies, to capture papers published whilst the guideline was being developed. This search returned 102 references for this review question, these were screened on title and abstract. One reference was ordered for full text screening and it was excluded based on its relevance to the review protocol (Appendix A). An additional validation study was highlighted during stakeholder consultation (already found by the systematic search) which was included based on their relevance to the review protocol (Appendix A).

See section 1.1.10 References – included studies for a list of references for included studies.

1.1.4.2. Excluded studies

See Appendix K for a list of excluded studies with reasons for exclusion.

Combination of measures to predict outcomes

See Appendix G for full GRADE tables.

Prognostic equations

Table 4

Validity of end-stage renal disease (ESRD) risk prediction – c-statistics.

See Appendix G for full GRADE tables.

Table 5

Validity of end-stage renal disease (ESRD) risk prediction – Calibration (Brier score).

See Appendix G for full GRADE tables.

Table 6

Validity of end-stage renal disease (ESRD) risk prediction – Calibration (R² statistic).

See Appendix G for full GRADE tables.

Table 7

Validity of end-stage renal disease (ESRD) risk prediction – sensitivity and specificity to start RRT.

See Appendix G for full GRADE tables.

Summary of included economic evidence

1.1.8.1. The outcomes that matter most

The committee agreed that the key outcomes to identify increased risk of progression in adults, children and young people with CKD were all-cause mortality and end stage renal disease using the four and eight variable kidney failure risk equations (KFRE). The committee noted that c-statistics for KFRE were high, especially for the four variable KFRE in adults, which meant that it was a useful tool to predict the risk of progression to end stage renal disease. These studies were also very large, with one cohort being almost three-quarters of a million people. The committee also noted that there was a single large validation study of the KFRE in the UK and that a health economic model would be useful to assess data on the predictive accuracy of the different referral rules for predicting progression to end stage renal disease (need for dialysis or a renal transplant). The committee was aware that using the KFRE would represent a significant change in practice and discussed this in depth, however it agreed that the clinical evidence and economic modelling justified this.

The committee discussed the evidence for combinations of measures that had not changed since publication of the previous guideline. Overall, it agreed the KFRE data were more current and more important and so focussed on these data.

1.1.8.2. The quality of the evidence

The quality of the evidence ranged from moderate to high quality evidence. The committee noted the high heterogeneity identified in the meta-analysis of KFRE c-statistics. It agreed that because the included studies were very large, and therefore the confidence intervals were very narrow, the heterogeneity was not as great as it appeared (all the studies had a calibration estimate of between 88 and 93%). Because the c-statistics were high and confidence intervals narrow, the committee was confident that the tests had high accuracy because even the worst-case estimate was 88%.

The evidence for combinations of measures was moderate to high, but the committee noted that all of the outcomes relied on one study each and none of them were poolable in a meta-analysis. It additionally noted that the 3 studies included in the combination of measures analysis were the same studies that had been included at the last update, and that even though they demonstrated that combinations of measures are better than single measures at predicting progression of disease, this was no surprise.

1.1.8.3. Benefits and harms

Since no new evidence was identified and included which examined combinations of measures to predict progression of chronic kidney disease, the committee discussed the previous evidence on combined measures (different combinations of eGFRcreatinine, eGFRcystatin C, and albumin:creatinine ratio). It showed that these combinations predicted a higher risk of all-cause mortality, end stage renal disease and acute kidney injury, but not all evidence reported on the same combinations of measures and the reference groups also varied. The committee agreed that the evidence was not as conclusive as the evidence on the KFRE.

New evidence was found on the 4 variable and 8 variable KFRE. Most evidence was found for the 4 variable KFRE in adults, though there was also evidence for older people. This evidence had not been considered by the previous committee because it was newer evidence. The committee discussed the KFRE equations together with the evidence from the health economic model and amended the criteria for referral from using GFR less than 30 ml/min/1.73 m² to using the KFRE with 5-year risk of end-stage renal disease greater than 5% or an ACR >70. Given the size and quality of the included studies, the economic modelling and the high discrimination of the equation, the committee were able to make a strong recommendation. This recommendation was based on the 2014 NICE guideline which had the same list of referral rule criteria apart from the referral rule of the KFRE ≥5%.

The committee agreed that the 4 variable KFRE could provide helpful information about adults risk of disease progression over time. Having this information might help people to be more proactive in terms of managing their own risk and be used as part of the management plan with the nephrologist.

The committee highlighted that it is important to discuss risk with people. The committee agreed that education to explain what risk means and how to manage risk is essential when discussing the risk of severe kidney disease with people. Therefore, the committee made additional recommendations about allowing enough time for the provision of information, using jargon free language and documenting the discussion to allow people to make an informed decision with their health practitioner.

The committee discussed the practicalities of using the 4 variable KFRE in daily practice (see section 1.1.8.4 Cost effectiveness and resource use which includes a discussion about implementation issues).

The use of the 4 variable KFRE might identify people at risk of progression to end stage renal disease (ESRD) earlier, which could help to optimise their care before referring to secondary care.

The 4 variable KFRE has not been validated in the UK for children and young people with CKD. The committee noted a US validation study, but agreed that since the validation of the equation in UK adults involved a different mathematical multiplier than in the US it was not possible to extrapolate from US children and young people to those in the UK. Therefore, the committee agreed that children and young people should not be added to the recommendation including the KFRE. Instead, the committee recommended to have an agreement between primary, secondary and tertiary care services for the referral of children and young people being followed-up in primary care and made a research recommendation to validate the equation in the UK for children and young people.

The committee highlighted that black, some Asian and other minority ethnic groups were underrepresented in the UK study validating the KFRE. Therefore, it was agreed to make a research recommendation to validate the risk equation in this population.

1.1.8.4. Cost effectiveness and resource use

The committee noted that there was evidence from the clinical review to suggest the kidney failure risk equations (KFRE) may be a useful tool to predict the need for dialysis or a kidney transplant, and therefore may have value as a tool to guide decisions on referral to secondary care. They felt that a health economic model would be useful for this review question as a UK validation study for the KFRE had recently been published, with data available to model different referral rules. The committee noted that paper only looked at the predictive accuracy of the referral rules, and that there is a trade-off between the sensitivity and specificity of different referral rules. High specificity is important as it prevents referral of patients who will not go into ESRD, where referral would increase the cost of monitoring, without providing clear benefits to the patient. Conversely, high sensitivity is important as it is also crucial not to miss patients who will go into ESRD, as they need to be identified early enough to provide sufficient time to be prepared before starting renal replacement therapy. The NICE guideline on renal replacement therapy suggests that assessment should take place at least 1 year before therapy is likely to be needed.

There has been a single validation study of the KFRE in the UK (Major 2019). This found that the equations had high predictive accuracy within the UK population, once the baseline risk was adjusted to account for the lower baseline risks of kidney disease in the UK compared to the US, where the equations were originally derived. The committee noted as a limitation that these new baseline risks had not been externally validated in a separate study, but agreed that as the equation itself was not changed from the US version, they were still confident in the results of this validation. This study supplied data from which the cost-effectiveness of different referral rules could be assessed; specifically, data on the predictive accuracy of the different referral rules for predicting progression to end stage renal disease.

The committee agreed that the model should be restricted to the use of the four-variable KFRE equations in adults, as there was no evidence from the clinical review that the eight-variable equation performed better, and also that version had not been validated in the UK. Similarly, they agreed that since there was no UK validation for the KFRE equations in children, they should be excluded from the analysis. The committee agreed that children should usually be referred to secondary care much earlier, due to chronic kidney disease being rarer and therefore there being less experience of how to manage the condition in primary care. They agreed that children with CKD would almost always be under secondary care management, and that for the small number discharged back to primary care, rules for re-referral would need to be agreed for the individual.

The KFRE equations are designed to identify the need for dialysis or a kidney transplant, and therefore the committee agreed it was appropriate to design a model based around the costs and benefits of identifying people and referring them to secondary care earlier. Thus, the model should estimate the additional secondary care monitoring costs for people referred earlier, and the benefits of better outcomes for renal replacement therapy (RRT) when people are identified sufficiently in advance. The key clinical data parameters used to populate the model was an update of a Cochrane review looking at the impact of early referral on post-dialysis outcomes. This found that people referred at least 6 months in advance had considerably lower hospitalisation costs and post-dialysis mortality than those only referred within 6 months of needing to start dialysis.

The cost-effectiveness model found that the two best referral rules were the “KFRE ≥5% or ACR ≥70” and the existing 2014 NICE referral rule (eGFR <30 or ACR ≥70). More specific referral rules, such as using the KFRE at a threshold of 15%, saved money on monitoring costs, but at the expense of considerably worse outcomes for people who were missed and therefore not appropriately prepared for needing dialysis. Similarly, more sensitive referral rules, such as those using the KFRE at a threshold of 3%, did identify more people who will enter ESRD earlier, but at the expense of too high an increase in monitoring costs as to be cost-effective.

In the base-case analysis, the “KFRE ≥5% or ACR ≥70” referral rule came out as the most cost-effective option, though the magnitude of the benefit over the current NICE criteria was small. Most of the sensitivity and scenario analyses agreed that “KFRE ≥5% or ACR ≥70” was the preferred rule, as did the results of the probabilistic sensitivity analysis, and therefore the committee were confident in this ordering, despite the small magnitude of the differences. They also noted this finding was consistent with the results of the Major study, which found this rule had both higher sensitivity and specificity than the current NICE criteria. They also noted this rule tends to, on average, refer younger people than the current criteria. The committee recognised that kidney function naturally reduces with age, meaning that a referral rule based on a simple eGFR cut-off will identify people who have normal age-related kidney function decline but are unlikely to reach ESRD within their lifetime. Referral to secondary care is only expected to improve outcomes for people who progress to ESRD and require RRT or conservative management. Because of this, the committee considered that identifying younger people who are more likely to require RRT is a benefit of the ‘KFRE>5% or ACR 70’ referral rule.’

The committee noted there was more uncertainty in the part of the model relating to pre-emptive transplants than the part on dialysis. For example, one scenario where the current NICE criteria came out as better than the “KFRE ≥5% or ACR ≥70” rule was when the hazard ratio for mortality from living pre-emptive kidney donors versus deceased pre-emptive kidney donors was set to the upper limit of the 95% confidence interval. However, the upper limit of the confidence interval implied that clinical outcomes for transplants from deceased donors are better than transplants from living donors, which the committee did not believe to be true, therefore the committee were not concerned about this result. They also noted that when the outcomes for people with a pre-emptive transplant were excluded from the analysis (so benefits were measures solely in people who go on to dialysis) the model once again showed the “KFRE ≥5% or ACR ≥70” rule to be the most cost-effective.

The committee discussed the probabilistic sensitivity analysis and noted that the “KFRE ≥5% or ACR ≥70” rule had the highest probability of being cost-effective. However, in some cases other referral rules may be preferred. The committee felt that the “KFRE ≥5% or ACR ≥70” rule was still a better referral rule than the 2014 NICE criteria and therefore should be adopted into practice. The committee also felt that the sensitivity analysis around the costs did not change the preferred referral rule. Specifically, they noted that the two referral rules involving the KFRE≥5% had a combined probability of over 60% of being the optimal choice, compared to around 20%for the current NICE criteria, and therefore it was three times more likely that a change in practice would be optimal, rather than remaining with the current criteria.

The committee discussed whether the small benefits identified for the alternative referral rule were sufficient to justify changing the referral recommendations and agreed that they were. In particular, they noted that the model only captured clinical benefits for people based on whether they were referred more or less than 6 months before needing dialysis, as this is what the source of the key clinical data. They noted the finding that at the start of the Major study considerably (close to 10%) more people who will progress to ESRD were correctly identified using the “KFRE ≥5% or ACR ≥70” rule, but by 6 months before dialysis both rules were predicted to be picking up approximately the same number. Therefore, the alternative rule, whilst not being much better at identifying people 6 months in advance, was meaningfully better at identifying people at earlier time points (such as 1 year in advance). The committee agreed that, whilst there was no data to capture this in the model, they were confident there would be additional benefits from identifying people 1 year in advance compared to 6 months in advance, and therefore the real world benefits of switching from the current criteria would be larger than those estimated in the model. Additional benefits include more time to test family members to find a suitable match for a kidney transplant, and more time for the patient to choose and prepare for the right type of dialysis for them (there was evidence that a higher proportion of people referred earlier will choose peritoneal rather than haemodialysis). They also noted the model currently only captures the post-dialysis benefits of early referral, not the benefits that may result if early referral either enables progression to ESRD to be delayed, or enables someone to obtain a pre-emptive transplant rather than needing to go on to dialysis at all. Once again therefore, they agreed that the earlier identification of people using the “KFRE ≥5% or ACR ≥70” rule was likely to provide additional clinical benefits to those captured in the model.

The committee also felt that the analysis was not able to capture the full benefits of using the KFRE equations in other ways. They agreed many patients would find it useful to be given information on their risk of going into ESRD in the next 5 years. The committee felt that this is a large gain for the patient as sometimes they do not understand eGFR and ACR, and what changes in these measurements may mean for their condition, but are likely to understand the meaning of risk over the next 5 years. They agreed it may also be a useful way of motivating patients into making healthier choices i.e. giving up smoking or losing weight. However, they also noted that if explained poorly, being given a risk may be worrisome for some patients. The committee felt that this worry could be mitigated by supporting the patient, ensuring the information is delivered in a clear way and providing opportunities for discussion. To capture this, the committee adapted recommendations from other NICE guidelines where risk scores are used, and noted the upcoming guideline on shared decision making, which is also expected to contain recommendations on this topic.

The committee noted that the model had used a hard cut-off of an ACR of 70 for referral decisions, as this was the data available, whilst in reality the criteria is more complex - “ACR ≥70, unless known to be caused by diabetes and already appropriately treated.” The committee noted this as a limitation but agreed that, since this modification applied to both the 2014 NICE criteria and the new KFRE based criteria, this change was unlikely to lead to changes in differential effectiveness between the two different referral rules.

The committee noted there will be implementation difficulties in moving to the new referral rule. Changing to the new referral rule requires the entire country to adopt the new practice as the combination of some areas using the “KFRE ≥5% or ACR ≥70” referral rule and for some the current NICE guidance may cause confusion and so be worse than either rule on its own. The committee acknowledged this difficulty but felt that there should be ways these implementation difficulties could be managed. They noted there were two ways the KFRE could be implemented in practice – either the calculations could be performed by laboratories and returned to primary care alongside eGFR and ACR results (and then stored as a separate field in GP computed systems), or the individual results could be returned to primary care and automated calculations built into GP computer systems. They felt that either one of these methods would in principle be appropriate and were confident if one of these methods were available GPs would adopt the KFRE quickly. They also noted that no other measurements need to be taken to use the KFRE, as it is based on the same variables already measured for monitoring CKD. The committee also noted that other risks associated with using the equation, for example that the US numbers could be used by accident instead of the UK numbers, would also be mitigated by the KFRE being routinely built into IT systems. It was also felt that some GPs would be unwilling to input data into a website, or use some other method to manually calculate the results. In practice, the committee agreed that laboratories performing calculations was likely to be an easier system to introduce, due to the smaller number of laboratories that would need to introduce these calculations, compared to the number of primary care providers.

The committee were aware there was likely to be an implementation period before KFRE results were available to all GPs, and therefore for a time some GPs were likely to have to continue to base referral decisions on eGFR and ACR values independently, as is currently done, and would be unable to provide patients with a quantitative assessment of their risk of needing renal replacement therapy.

The committee noted the potential for the KFRE (or other similar equations) to be improved in the future; noting that equations can be improved, but exact limits such as the current NICE criteria cannot. For example, there is also an eight-parameter equation which includes co-morbidities. Whilst this has not been shown to have any benefit over the four-parameter equation at present, the committee agreed there was potential for comorbidities or other factors to be built into the equations in the future.

There are other sections to the recommendations that have not been changed by the committee. The four parameter KFRE does not consider comorbidities and therefore the committee felt that the other criteria for referring patients (such as those with haematuria or hypertension) were important to keep. Referral back to the GP was included as a recommendation as not all patients require secondary monitoring, and this re-referral was included in the economic model. The committee agreed that a discussion with secondary care on whether the patient needs to be referred or can still be monitored in primary care could be considered as the equivalent to a referral.

Downstream costs of dialysis were excluded from the analysis, in keeping with other NICE guidelines that contain dialysis. This is because dialysis is not a cost-effective treatment by standard NICE criteria, and therefore including the costs of it can lead to nonsensical results (such as an intervention with high mortality coming out as better, since it saves costs of later dialysis). The committee noted this and agreed it was appropriate, since society has indicated that it is willing to pay for dialysis even though it is not a cost-effective treatment by the standard criteria, and therefore the analysis should reflect this choice. One sensitivity analysis was done including downstream dialysis costs which showed that the preferred option was KFRE ≥15%. KFRE ≥15% has the highest specificity; it does not refer many patients unnecessarily; however, it also does not find many patients who will need RRT. This is why when the committee looked at the information, they disregarded this option as patients ‘crash landing’ onto dialysis is very detrimental to patient costs and quality of life.

1.1.8.5. Other factors the committee took into account

An issue within the data from the Major study was that the black population was under-represented, both compared to the UK population overall and compared to the population in the UK Renal Registry. The committee noted this was an issue that had run throughout the guideline, with many studies not containing sufficient representation of the black population to enable good recommendation to be made for that group, or to be confident the same recommendations were appropriate. In the long-term the committee agreed this could only be addressed by further research studies appropriately sampling from this population. However, the committee noted that in this case, these limitations applied to both the current NICE criteria and the KFRE based criteria, and therefore there was no reason to suspect that changing would have a detrimental effect on any given population. They also noted again that an advantage of moving to an equation-based method was the potential for those equations to be improved with the inclusion of extra factors, be those ethnicity itself or other factors (such as muscle mass) that may be correlated with ethnicity at a population level, but be better predictors of individual outcomes than ethnicity itself.

COVID-19 has changed the way healthcare has been provided and it is unknown how much of this change will persist. Monitoring appointments have been moved to telephone consultation; this is less expensive than in person appointments. This change was caught in the sensitivity analyses with reducing monitoring costs, this showed that the KFRE ≥5% or ACR ≥70 referral rule was still the most cost-effective rule, and therefore the committee were confident their recommendations were robust to any potential future changes in the configuration of CKD services.

The rules on organ donation changed on 20 May 2020 to an opt out system, which means that there are likely to be more available kidneys for transplantations. Therefore, more patients are likely to be able to receive a kidney transplant, rather than need to go on to dialysis. However, early referral is then even more important in making sure the necessary tests are completed, otherwise the patient may have to go on dialysis before the transplant, which leads to worse outcomes.

J.4.1. Principal findings

The model consistently found that the best two referral rules were “KFRE ≥5% or ACR ≥70” and the 2014 NICE criteria. All other referral rules tested were less cost-effective than these two rules. This was because they either had too low a sensitivity, meaning that more patients who will need RRT are not identified and therefore not referred which means the opportunity to prepare for RRT is missed, or too low a specificity, leading to increase monitoring costs from unnecessary referrals to secondary care.

The “KFRE ≥5% or ACR ≥70” referral rule came out as more cost-effective than the 2014 NICE criteria, but only by a small margin in the base-case analysis. The earlier it is deemed important to identify people in advance of RRT, the more advantage the KFRE based rule has over the 2014 NICE criteria. For example, the margin is larger if 1 year is used as the cut-off for a sufficiently early referral, rather than 6 months.

J.4.2. Strengths and limitations of the analysis

The majority of the sensitivity analyses found that the best referral rule was “KFRE ≥5% or ACR ≥70”. The model was relatively robust to both parameter uncertainty and many of the scenario analyses tested, meaning that we are confident the “KFRE ≥5% or ACR ≥70” provides benefits over the 2014 NICE criteria, even if the magnitude of those benefits is small.

The analysis is based predominantly around the Major 2019 UK validation study of the KFRE equations. This is the only UK validations study for the equations, and therefore there is no second set of data on predictive accuracy to which these results can be compared. Therefore, if the results in the Major study are in some way unrepresentative of England as a whole, that will not be appropriately captured in the analysis.

A second issue with the analysis is the Major stuudy was taken as a cross-section of data at a single point of time, which does not represent what happens in the real world. In the real world a patient would be assessed by the referral rule at each monitoring appointments; meaning that if a patient is not referred at a single appointment then they might get referred at the next one. Whilst the model does simulate eGFR and ACR progression over time in an attempt to address this issue, these simulated data will necessarily be at higher risk of error than if longitudinal data had been available for the individuals in the Major study.

The impact of early referral on post-dialysis outcomes was based on a meta-analysis of cohort studies. Whilst considered effort was made in the Cochrane review used as this basis for this analysis to assess the potential for selection bias, and no evidence for it was found, there is still necessarily more uncertainty in these data than would be the case for equivalent results from randomised controlled trials. Additionally, since data were only available on the impact of referral on post-dialysis outcomes, this means the model is not able to account for differences that occur before renal replacement is necessary. For example, it is possible that earlier referral may either delay the onset of kidney failure or enable more people to receive pre-emptive transplants rather than go on to dialysis at all. These potential additional benefits of earlier referral are not currently possible to capture in the analysis.

In May 2020, the organ donation rules in England changed to be an ‘Opt out’ system rather than an ‘Opt in’ system (Organ Donation 2020). This means that patients who have not stated a preference on organ donation are assumed to be willing to donate their organs. This was introduced to increase the number of available organs for patients who require a new organ. However, as this new system has been introduced recently there is no data on the real-world changes to the number of available organs. This meant that the data used in the model was based on the previous organ donation rules. However, with the new rules the number of available organs will only increase not decrease and therefore the values in the model are a lower bound of available organs. With more kidneys available more patients are likely to be able to have a transplant and not have to go onto dialysis. It will be a few years before sufficient data is available to estimate the impact these changes may have on practice, and therefore how they may affect the most appropriate referral rules to use.

J.4.3. Comparison with other CUAs

The KFRE are new equations that have not existed long enough for there to be other cost-effectiveness analyses that have been conducted using them. Therefore, it is not possible to compare the results obtained with any similar published work. Other studies that have attempted to assess the place for the KFRE in practice (such as Hingwala 2017 in Canada) have been restricted to looking at clinical outcomes and predictions and did not consider cost-effectiveness.

However, the approach this analysis has taken is similar as the NICE guideline on renal replacement therapy and conservative management (NG107) when assessing the comparative cost-effectiveness of different forms of haemodialysis. That analysis also used data from the UK Renal Registry to build a natural history model, to which relative effects were then applied.

J.4.4. Conclusions

Using the “KFRE ≥5% or ACR ≥70” is the most cost-effective referral rule for identifying patients at risk of end stage renal disease over the next 5 years, and who will benefit from referral to secondary care. However, the benefits of using this referral rule are only modest, compared to using the criteria from the 2014 NICE guideline.