Overview
This chapter reports the results of the triage analyses, which aimed to assess the diagnostic performance of the four imaging tests in a triage setting. The specific diagnostic performance analyses covered in this chapter are the default triage analysis (, Default triage analysis) along with eight sensitivity analyses (see , Triage sensitivity analyses 1–8) for a list with definitions. A further set of three analyses specifically to inform the economic model are described in Appendix 6. The default triage analysis was defined as one in which the reference standard was the person-level clinical decision (‘not discharged’ or ‘discharged’). The test was defined as categorising a patient as requiring to be referred on (‘for referral’) if any of the elements of the composite triage test (imaging, IOP and/or VA) were themselves ‘abnormal’: imaging outside normal limits on the overall classification of the imaging test (see Chapter 2), IOP > 21 mmHg or VA of 6/12 or poorer under the default triage analysis.
If the imaging test did not produce an overall classification or its quality was poor, the imaging test result was again defined as abnormal and, therefore, the patient was classified as ‘for referral’. The eight sensitivity analyses assessed the impact of varying assumptions made in the default triage analysis relating to the definition of a positive test result, modifying or removing the IOP and/or VA components of the triage test, and how cases where the test did not produce an overall classification were handled in the analysis.
The analyses in this chapter pertain to the 943 participants remaining in the study (see Chapter 4). The reference standard was available for 933 cases. For all analyses, a STARD diagram shows the flow of participants. The subset of participants who received all four tests and were considered in the statistical analyses are separated out into three groups according to whether each triage test result was ‘abnormal’, ‘normal’ or ‘no result’ (the triage test result was not available because either the test was inconclusive or the result was missing). For each of these three groups the group status according to the reference standard (‘discharged’ or ‘not discharged’) for each participant is given or alternatively the reference standard was stated to be missing or inconclusive. The final categorisations of the triage test result by reference standard status provides the four possible combinations (true and false positive, false and true negative) from which the diagnostic performance was assessed. Sensitivity, specificity, likelihood ratios and DOR are provided with associated 95% CIs for each analysis.
Default triage analysis
The results for the default triage analysis are presented in two sections:
Diagnostic performance of the triage tests
For the default triage analysis, the triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Imaging test results that did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to the default triage analysis is shown in , with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 576 were not discharged and 357 were discharged and the discharge status was missing for 10 participants. The diagnostic performance of the four tests is given in . The results showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (86.0%, 95% CI 82.8% to 88.7%) but lowest specificity (39.1%, 95% CI 34.0% to 44.5%), GDx had the lowest sensitivity (64.7%, 95% CI 60.7% to 68.7%) but the highest specificity (53.6%, 95% CI 48.2% to 58.9%), and the other two tests provided intermediate results [HRT-MRA values were very similar to the HRT-GPS results, as might be expected given that they use the same machine, and OCT had lower sensitivity (75.4%, 95% CI 71.9 to 78.9) but higher specificity (41%, 95% CI 35.8 to 46.3) values than HRT-GPS and HRT-MRA]. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.12 for GDx and OCT to 3.94 for HRT-GPS.
Flow diagram: default triage analysis. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Diagnostic performance: default triage analysis
Paired comparisons of imaging tests
shows the paired difference (with 95% CI) and corresponding McNemar’s tests p-value for comparisons between pairs of tests. There was evidence that the sensitivity of all tests differed from each other, except for HRT-GPS versus HRT-MRA.
Paired comparisons of sensitivity and specificity between the triage tests
The highest sensitivity was found in HRT-GPS and HRT-MRA, and HRT-MRA and GDx had the lowest sensitivity. Differences varied from 0.2% (HRT-GPS vs. HRT-MRA) to 21.3%. (HRT-GPS vs. GDx). Similarly, there was evidence that specificities for all the tests varied from each other (according to McNemar’s test), except for HRT-GPS versus OCT.
Triage sensitivity analysis 1
Triage sensitivity analysis 1 differed from the default triage analysis in that a borderline finding on the imaging test was also classified as an abnormal result.
For triage sensitivity analysis 1, the triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’ or ‘borderline’, (2) IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to triage sensitivity analysis 1 is shown in , with the corresponding numbers of referral, not for referral and no result cases by triage test, and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 576 were not discharged and 357 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in . The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (94.0%, 95% CI 91.8% to 95.8%) but second lowest specificity (24.9%, 95% CI 20.4% to 29.7%), GDx had the lowest sensitivity (74.9%, 95% CI 71.1% to 78.4%) but the highest specificity (45%, 39.7% CI 39% to 50.4%), and the other two tests provided intermediate results (HRT-MRA values were very similar though marginally inferior to the HRT-GPS results, and OCT had lower sensitivity (84.2%, 95% CI 80.9 to 87.1) but slightly higher specificity than HRT-GPS and HRT-MRA). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.04 for OCT to 5.21 for HRT-GPS.
Flow diagram: triage sensitivity analysis 1. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Diagnostic performance: triage sensitivity analysis 1
Triage sensitivity analysis 2
Triage sensitivity analysis 2 has the same reference standard and definition of abnormal test result as the default analysis but did not include all no result cases (see ).
Diagnostic performance: triage sensitivity analysis 2
For triage sensitivity analysis 2, the triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Poor-quality imaging test results were included, and those where an image was acquired but no classification generated were included as abnormal. All other missing imaging results were excluded. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to triage sensitivity analysis 2 is shown in , with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in . The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (84.6%, 95% CI 81.4% to 87.5%) but the second lowest specificity (39.7%, 95% CI 34.6% to 45.1%), GDx had the lowest sensitivity (61.1%, 95% CI 56.9% to 65.1%) but the highest specificity (59.0%, 95% CI 53.7% to 64.2%) and the other two tests provided intermediate results [HRT-MRA values were very similar, although slightly inferior to the HRT-GPS results, and OCT had the second lowest sensitivity (75.0%, 95% CI 71.3% to 78.5%) but the second highest specificity (42.1%, 95% CI 36.9% to 47.4%) values]. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.19 for GDx to 3.61 for OCT.
Flow diagram: triage sensitivity analysis 2. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Triage sensitivity analysis 3
Triage sensitivity analysis 3 was the same as triage sensitivity analysis 2 except that ‘borderline’ test results were also classified as abnormal.
For triage sensitivity analysis 3, the triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’ or ‘borderline’, (2) IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Poor-quality imaging test results were included, and those where an image was acquired but no classification generated were included as abnormal. All other missing imaging results were excluded. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to triage sensitivity analysis 3 is shown in , with corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in . The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (93.3%, 95% CI 91.0% to 95.2%) but second lowest specificity (24.9%, 95% CI 20.4% to 29.7%), GDx had the lowest sensitivity (72.3%, 95% CI 68.4% to 75.9%) but the highest specificity (49.0%, 95% CI 43.6% to 54.4%) and the other two tests provided intermediate results [HRT-MRA values were very similar to the HRT-GPS results, although slightly inferior, and OCT had the second lowest sensitivity (84.2%, 95% CI 80.9% to 87.1%) but the second highest specificity]. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.12 for OCT to 4.63 for HRT-GPS.
Flow diagram: triage sensitivity analysis 3. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Diagnostic performance: triage sensitivity analysis 3
Triage sensitivity analysis 4
Triage sensitivity analysis 4 differed from the default triage analysis in that referral IOP > 21 mmHg rather than clinician IOP > 21 mmHg was used to identify abnormal tests. The triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) referral IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to triage sensitivity analysis 4 is shown in , with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in . The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (86.5%, 95% CI 83.4% to 89.2%) but second lowest specificity (24.0%, 95% CI 19.6% to 28.8%), GDx had the lowest sensitivity (67.2%, 95% CI 63.2% to 71.0%) but the highest specificity (35.8%, 95% CI 30.8% to 41.1%) and the other two tests provided intermediate results (HRT-MRA values were very similar to the HRT-GPS results, although slightly inferior, and OCT had the second lowest sensitivity (76.8%, 95% CI 73.1% to 80.2%) but the second highest specificity (27.7%, 95% CI 23.1% to 32.7%). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 1.14 for GDx to 2.02 for HRT-GPS.
Flow diagram: triage sensitivity analysis 4. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Diagnostic performance: triage sensitivity analysis 4
Triage sensitivity analysis 5
Triage sensitivity analysis 5 differed from the default triage analysis in that the IOP component was removed from the composite triage test. The triage test is classified as abnormal if the imaging test result is classified as (1) ‘outside normal limits’ or (2) VA is 6/12 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to triage sensitivity analysis 5 is shown in , with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 933 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged. The discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in . The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (68.9%, 95% CI 64.9% to 72.7%) but the lowest specificity (52.3%, 95% CI 46.9% to 57.7%), GDx had the lowest sensitivity (32.8%, 95% CI 29.0% to 36.8%) but the highest specificity (81.1%, 95% CI 76.6% to 85.1%) and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had the second lowest sensitivity but the second highest specificity). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 1.80 for OCT to 2.91 for HRT-GPS.
Flow diagram: triage sensitivity analysis 5. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Diagnostic performance: triage sensitivity analysis 5
Triage sensitivity analysis 6
Triage sensitivity analysis 6 differed from the default triage analysis in that the VA component was removed from the composite triage test. The triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’ or (2) IOP is > 21 mmHg. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to triage sensitivity analysis 6 is shown in , with corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in . The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (84.9%, 95% CI 81.9% to 87.7%) but second lowest specificity (37.4%, 95% CI 32.3% to 42.8%), GDx had the lowest sensitivity (60.5%, 95% CI 56.4% to 64.6%) but the highest specificity (57.6%, 95% CI 52.2% to 62.8%), and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had the second lowest sensitivity but the second highest specificity). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.03 for OCT to 3.97 for HRT-GPS.
Flow diagram: triage sensitivity analysis 6. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Diagnostic performance: triage sensitivity analysis 6
Triage sensitivity analysis 7
Triage sensitivity analysis 7 differed from the default triage analysis in that a higher IOP threshold of 26 mmHg rather than 21 mmHg was used to identify abnormal tests. The triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) IOP is > 26 mmHg or (3) VA is 6/12 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to triage sensitivity analysis 7 is shown in , with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in . The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (77.2%, 95% CI 73.5% to 80.6%) but second lowest specificity (51.8%, 95% CI 46.3% to 57.2%), GDx had the lowest sensitivity (47.9%, 95% CI 43.7% to 52.1%) but the highest specificity (79.1%, 95% CI 74.4% to 81.2%), and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had very similar sensitivity and specificity). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.61 for OCT to 4.03 for HRT-GPS.
Flow diagram: triage sensitivity analysis 7. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Diagnostic performance: triage sensitivity analysis 7
Triage sensitivity analysis 8
Triage sensitivity analysis 8 differed from the default triage analysis in that a higher VA threshold of VA 6/18 or poorer was used to identify abnormal tests. The triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) IOP is > 21 mmHg or (3) VA is 6/18 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.
The flow of study participants according to triage sensitivity analysis 8 is shown in , with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in . The results showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (85.1%, 95% CI 81.8% to 87.9%) but lowest specificity (35.1%, 95% CI 30.0% to 40.4%), GDx had the lowest sensitivity (61.9%, 95% CI 57.8% to 65.9%) but the highest specificity (55.6%, 95% CI 50.2% to 60.9%) and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results, and OCT had the second lowest sensitivity (72.9%, 95% CI 69.1% to 76.5%) but the second highest specificity (42.9%, 95% CI 37.7% to 48.3%). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.03 for OCT to 3.80 for HRT-GPS.
Flow diagram: triage sensitivity analysis 8. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Diagnostic performance: triage sensitivity analysis 8
Discussion
Four composite triage (imaging, IOP measurement and VA assessment) tests were compared with regard to their diagnostic performance for determining who should be referred for further assessment or discharged using the GATE population of referrals to a glaucoma clinic in secondary care.
The sensitivity and specificity of the four triage tests incorporating each of the imaging technologies along with IOP and VA for the default triage analysis and sensitivity analyses (see for details) are summarised in and , respectively.
Summary of the sensitivity of the composite test across all triage analyses. SA, sensitivity analysis.
Summary of the specificity of the composite test across all triage analyses. SA, sensitivity analysis.
All four triage tests had value in terms of ruling in and ruling out the need for referral on to a consultant ophthalmologist. The diagnostic performance of the triage tests differed with substantial differences in the ability to correctly detect those who need to be referred and those who do not. HRT-GPS and HRT-MRA consistently had the highest sensitivities across analyses but at a cost of lower specificity than other tests. HRT-GPS had the slightly higher specificity. In contrast, GDx consistently had the best specificity, although the lowest sensitivity. HRT-GPS results were typically similar to HRT-MRA. OCT generally had similar levels of sensitivity and specificity. The choice of which triage test is to be preferred reflects the inherent trade-off regarding diagnostic testing, where the desire to refer onwards when referral is needed must be balanced again the desire to discharge those who do not need a further assessment. A formal assessment of this trade-off and the consequences in terms of health outcome and costs is covered in Chapters 6 and 7.
The triage was formed from three components, an imaging test as evaluated in Chapter 4, a measurement of IOP and VA measurement. The elements were combined in an additive manner where an individual was referred if any one of the three components met the relevant referral criteria. A number of sensitivity analyses were carried out to assess the robustness of the findings of this default triage analysis. Varying the imaging test definition of a positive result by including the borderline category of imaging test result was carried out; this had the expected impact of improving the detection of glaucoma, although at the cost of more non-glaucoma cases being falsely identified as having glaucoma. This resulted in very high detection of glaucoma for HRT-MRA, HRT-GPS and high sensitivities for GDx and OCT but the consequence of lower specificities (GDx had a higher specificity value than the other three triage tests). Additionally, the impact of using the classification from the imaging test when the quality criterion was not met was assessed. The impact was at most a small reduction in sensitivity with an increase in specificity (only GDx had more than a nominal change in values). The added value of the IOP and VA components was assessed by dropping one of the components, varying the cut-off point used to define abnormality, and, for the IOP component, using the referral IOP measurement in place of the ophthalmologist’s. Removal of the IOP component had a noticeable impact on the diagnostic performance with exclusion leading to a reduction in sensitivity, although a gain in specificity. Modifying the IOP cut-off value changed the balance in terms of sensitivity and specificity as expected. When the referral IOP was used in place of the ophthalmologist’s IOP the specificity was reduced. Such an impact is unsurprising given the known variability in IOP measurements42 and the use of an absolute cut-off will lead to a regression to the mean effect when another measurement is taken (in this case by a different observer). Removing the VA component had very little impact on the diagnostic accuracy with a slight reduction in the sensitivity and corresponding increase in specificity. This impact may have been limited by the method of data collection (referral letter quotation) as opposed to complete data capture of a new VA measurement.
A number of assumptions underpinned the analyses and interpretation of these results in addition to those highlighted previously for diagnoses analyses. The reference standard here was the clinical decision to discharge or not, which will vary to some degree between individual clinicians and centres according to policies and practices (perhaps most noteworthy for individuals with glaucoma suspect). Components of the triage test were combined in an additive manner which reflects an implicit desire to favour sensitivity over specificity. No other options were assessed, although arguably this approach reflects clinical practice. The use of the ophthalmologist’s measurement does not reflect the reality of how a triage system would be implemented where, if a measurement was taken in hospital eye services, it would be by another individual (e.g. a technician). Using the referral IOP did have a substantial impact, although most if not all of this impact might be attributed to the inevitable variability between measurements taken at different times by different observers and the impact of regression to the mean. The finding does suggest there is value in taking a measurement upon referral to hospital eye services.
