Introduction
We present methods for assessment of the cost-effectiveness of several novel tests as rule-out and rule-in tests for active TB. That is, in the case of a rule-out test, we consider using novel tests as an initial test, with a negative result indicating that a patient does not have TB, thus accelerating diagnosis of the actual cause of disease in such patients. In the case of a rule-in test, we consider using novel tests as an initial test, with a positive result indicating that a patient does have TB, averting the need for other tests for TB. The use of several tests was compared against current practice, as determined by analysis of patient records. We considered which diagnostic tests were performed, their costs and the time taken between decision points involving each test. The time taken to diagnose, rule in or rule out, TB is a key consideration. Our report adheres to the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.72
Methods
Decision tree model
We developed a decision tree model to calculate the incremental costs and incremental QALYs of changing from current practice to using a novel test as an initial rule-in or rule-out test. Current practice was determined by analysis of patient records. Adding a rule-out test to the diagnostic pathway introduces additional delays in the diagnosis of active TB in those patients who have the disease, as it introduces an additional step in the pathway. Patients who were not initially diagnosed with active TB have a follow-up consultation after approximately 2 months; those who had a false-negative rule-out test result (i.e. they had TB incorrectly ruled out) can have TB identified at this point. The final diagnostic outcomes were the four categories described in Dosanjh et al.,11 herein referred to as ‘Dosanjh categories’.
The health economic analysis was undertaken from an NHS perspective. No discounting was required as the diagnostic process occurs over a relatively short time period. The time horizon was up to final diagnosis of TB, which was assumed to be < 1 year.
The model contained two levels of uncertainty:
individual-level uncertainty – patient records revealed variation in the number and type of tests used for TB diagnosis and time to diagnosis
parameter uncertainty – uncertainty in the costs of tests and procedures, and the sensitivity and specificity of the novel tests.
Uncertainty distributions for each are estimated for either uniform or gamma distributions, depending on the form of the data available.
A (balanced) bootstrap sample of TB and non-TB patients was created for each simulation of the decision tree, which retained the TB and non-TB subsample sizes. Individual patient costs and time to diagnosis were jointly sampled to preserve the dependency between them. We sampled across the model parameter distributions to obtain a set of realisations. This was then repeated in a Monte Carlo framework to obtain a sample of 10,000 model runs, which encapsulated the first-order (patient variability) and second-order (parameter) uncertainty in the model outputs. summarises the key model parameters.
The model was implemented in the statistical programming language R. The key model parameters are presented in .
An example decision tree representation is given in . Equivalent figures for alternative scenarios are given in Appendix 16, and .
Decision tree comparing current practice (‘no rule-out test’) with a diagnostic pathway incorporating an initial rule-out test (‘rule-out test’). The ‘standard pathway’ branch represents the range of variation (more...)
Estimation of costs and health impact used in the model
The costs and distributions used in the probabilistic sensitivity analyses are summarised in Tables 63–65 in Appendix 16, using 2014/15 prices. When necessary, costs were inflated from previous years using the Hospital and Community Health Service pay and price index.74
If uncertainty bounds were not available in the recent sources used, uncertainty ranges were informed by previous studies. Uniform distributions were used when upper and lower limits were available and gamma distributions were used when the standard error of the average cost was available.
When modifying costs, the uncertainty around the central value was transformed maintaining the proportions from the original value rather than an absolute difference. For example, a lower bound of half and an upper bound of twice the point estimate values were used.10,17,75 Skewed distributions were represented using a gamma distribution.
As the end point of the diagnosis is diagnosis of TB or ruling out TB, treatment costs after final diagnosis are out of scope. However, when a patient was started on TB treatment and then a lack of response to that treatment informed a decision that the patient did not in fact have TB, the cost of this treatment was included, as it is part of the cost of ruling out TB in those patients.
The treatment costs are given in Table 66. Following the NICE guidelines for active TB management,8 it was assumed that such patients are on treatment until their 2-month follow-up appointment, when they are reassessed for response to treatment. The regimen in this period is daily treatment with rifampicin, isoniazid, pyrazinamide (Rifater, sanofi) and ethambutol. From the NICE British National Formulary,76 adult dosages are fixed except for ethambutol, which is determined by patient weight. The mean weight at time of first presentation, of 67.98 kg, was used in the model.
Selection of tests to evaluate in health economic analysis
In the VANTDET study, we have evaluated several signatures using each new technology, but we wished to evaluate the health economics in only the most promising signatures. As none of the proteomic signatures performed with high enough test accuracy to be clinically useful, these signatures were not used in these analyses. For the transcriptomic signature, the test accuracy data for the Sweeney et al.39 four-gene signatures, calculated using the TBS method, were used. For the flow cytometry data, the test accuracy for the %HLA-DR signature26 in the HIV– cohort was used. For the molecular rule-out test, the performance of the CXCL9 qRT-PCR in Mtb antigen-stimulated PBMC was used. gives the test sensitivity and specificity values used for the probabilistic sensitivity analyses.
Diagnostic test performance and distributions for different tests
The QALY loss as a result of TB treatment is given in . The sum total was used in the analysis:
Cost-effectiveness outcomes
The current (‘status quo’) cost and QALY loss are the weighted sums of the TB and non-TB pathways observed in the clinical data:
Results
For decision tree diagrams, please see Appendix 16. The results of the cost-effectiveness analysis for the Sweeney et al.39 transcriptomic signatures used as a rule-out test are given in and , and the transcriptomic signatures used as a rule-in test are given in and .
Cost-effectiveness planes for status quo against transcriptomic rule-out test scenario, with contour lines.
Cost-effectiveness statistics for transcriptomic rule-out scenarios
Cost-effectiveness planes for status quo against transcriptomic rule-in test scenario, with contour lines.
Cost-effectiveness statistics for transcriptomic rule-in scenarios
Flow cytometry rule-in test
The results of the cost-effectiveness analysis for the %HLA-DR flow cymometry signatures26 to be used as a rule-in test are presented in and .
Cost-effectiveness planes for status quo against flow cytometry %HLA-DR rule-in test scenario, with contour lines.
Cost-effectiveness statistics for flow cytometry rule-in scenario
Molecular rule-out test
The results of the cost-effectiveness analysis for the molecular rule-out test are presented in and .
Cost-effectiveness planes for status quo against molecular rule-out test scenario, with contour lines.
Cost-effectiveness statistics for molecular rule-out scenarios
Discussion
This health economic analysis has demonstrated that the use of those tests that performed best in classifying all TB cases in our VANTDET substudies (i.e. the Sweeney et al.39 four-gene transcriptomic signature, the %HLA-DR cellular immune signature26 and the CXCL9 qRT-PCR test for ruling out active TB) cannot be considered cost-effective, given the assumption of the modelling. This is due in part to the significant cost of these tests, which, in their current proposed format, are very expensive. Furthermore, the sensitivity and specificity provided by these tests were suboptimal, meaning that they did not improve effectiveness when introduced into the current diagnostic pathway. All tests incurred extra costs and there was a minor reduction in QALYs, probably because of the increased delay in diagnosis resulting from the introduction of new tests into the standard pathway. This was the case even when tests were considered as either rule-in or rule-out tests.
It should be noted that, as all of the technologies and signatures are being investigated in research laboratory settings, and have not begun to be commercialised in any way, the cost estimates had a high degree of uncertainty, which affected the cost-effectiveness analysis. The effectiveness for the %HLA-DR cellular immune assay, for which we propose using in the context of IGRA-positive TB suspects only, the cost-effectiveness plane demonstrated reasonable effectiveness, but with a very high cost differential. If the test accuracy of this signature could improve slightly, alongside a significant reduction in cost, it might be that this approach could be cost-effective for diagnosing TB in the IGRA-positive population. It is feasible that the cost of some of these technologies will become more economical in the coming years. Furthermore, it is likely that the costs of a commercial assay using simpler but comparable technologies (i.e. a simple flow cytometry method rather than complex multicolour analysis) would be lower.
This health economic analysis has some limitations. Given the lack of accurate data on novel test costs, we have relied on estimates based on current laboratory research costs and timings, which may not reflect the likely costs of these tests after they have been developed commercially. Therefore, there is a high level of uncertainly in our analyses. The uncertainty about the test sensitivities and specificities is represented as marginal distributions when there is a correlation between the two values that may be represented in a joint distribution if the appropriate information is available. Furthermore, we highlight that one of the main assumptions of the analysis is that we consider a relatively short time frame up to TB diagnosis and, therefore, cannot extend the conclusions beyond this.
Recommendations
In these analyses, we have considered the introduction of new tests into the pathway of a wide range of TB suspects, that is in complex populations with a diverse mix of TB and OD patients, with wide-ranging symptoms, disease severities and clinical suspicion of TB. It is likely that a blood-based test for a specific subset of TB, which could be used to assist in diagnosis against a specific subset of OD patients, would be a more cost-effective approach for these blood-based novel tests. In addition, such a test may have improved performance when used in combination with other tests. We have not yet evaluated such scenarios in this study, but it is a direction for future research on novel tests for diagnosing TB.
For rule-out tests, a test with a combination of high sensitivity and specificity, low cost and rapid speed of result is required to be clinically effective or cost-effective as a screening tool, or as a test to be used at the beginning of a diagnostic pathway for a TB suspect.16
Future research on the cost-effectiveness of tests would benefit from looking at long-term impacts beyond initial diagnosis and ideally would consider the infectious nature of the condition.
