Overall approach

Our approach was designed to estimate the cost-effectiveness of SES primarily from a health-care perspective but to consider other costs where data were available. In this report this was limited to consideration of transport costs associated with families attending diagnostic evaluation; thus the perspective adopted in this report was that of the NHS and the family. The study was designed to capture the costs and benefits of the two different methods of SES in order to inform the policy decision regarding the appropriate use of SES. Since SES is likely to have an impact on the costs and outcomes of all types of hearing impairment, the analysis is concerned with the identification of children with sensorineural or permanent conductive (SNPC) hearing impairment or transitory hearing impairment not diagnosed during the newborn hearing screen or the first 3 years of life (the final pre-school year which is the starting point for the economic model).

There are three key considerations that inform whether or not screening is cost-effective relative to no screening. First, there is the issue of whether or not screening at school entry improves the timeliness with which children are referred to diagnostic evaluation and, if indicated, to management, thus generating a more prolonged improvement in quality of life than would otherwise occur. Second, cost-effectiveness will depend on whether or not the diagnostic accuracy of the test makes a difference in the time at which children are identified, referred and managed. If a test has a high level of false negatives, children who should have been picked up by the screening test will experience a delay in their diagnosis and management. Third, to the extent that there are false positives (either from screening or other modes of identification), there is a potential associated negative impact on the child and family of stress and lost time in school or work, as well as additional unnecessary costs to the health system.

The following sections outline the key components of our modelling approach. This includes the rationale and important assumptions required to estimate the cost-effectiveness of the PTS and HC tests and provides some background to the evolution of the modelling approach since the 2007 HTA report.¹²

Decision-analytic structure

The screening question was addressed using a decision-analytical approach to evaluate the cost-effectiveness of school entry hearing screening programmes. The model estimates the costs and consequences of a hypothetical cohort of 10,000 children with a given prevalence of hearing impairment receiving the PTS, HC testing or no screening. The basic structure is presented in Figure 18. Figure 19 shows the route followed from screening to diagnosis of hearing impairment while Figures 20 and 21 illustrate the management follow-up for transitory and SNPC hearing impairment, respectively. As the decision tree representation does not explicitly capture the time element in the model, this aspect is detailed in the following sections.

FIGURE 18

Basic decision tree structure.

FIGURE 19

Decision tree: screening arm.

FIGURE 20

Transitory hearing impairment.

FIGURE 21

Sensorineural or permanent conductive hearing impairment.

Initial model development

Although a version of the original model from the 2007 HTA report¹² [built in TreeAge Pro 2005 (TreeAge Software Inc., Williamstown, MA, USA)] was provided for the current research, the updated model was developed in Microsoft Excel to provide flexibility in the modelling approach. Part of the initial work on this project was to simplify the decision tree model developed for the 2007 HTA report.¹² Based on consultations with the Project Steering Group, a flexible modelling approach was adopted which would retain the logic of the original model and could be represented by a decision tree format, but also capture the way in which hearing impairment is identified over a period of time. This is relevant not only when SES is unavailable and hearing impairment is identified exclusively by other means, but also in the presence of screening, when these other routes of identification may also be important. This chapter describes the initial development of the model in anticipation of the results of the clinical studies described elsewhere in this report (see Chapters 3, 5–7) and subsequent to the data becoming available, at which point we were able to generate final estimates of cost-effectiveness for the new technology (HC screener) and the existing technology (PTS) compared with no screening. For the two screening tests, the structure of the evaluation is identical.

Based on information in the 2007 HTA report¹² and correspondence with the lead modeller (Professor Linda Davies), we reconstructed the model used to generate the cost-effectiveness results presented in that report. Before updating the model, as specified in the original protocol, the modelling approach was reviewed to ensure that it was still fit for purpose. The alternative of a more detailed simulation model was considered but it was felt that it would add minimal benefit in terms of answering the research question. It was concluded that a Markov model or discrete-event simulation would not greatly improve the representation of screening while adding to the complexity of the modelling by increasing its data requirements (e.g. the use of transition probabilities). At the same time, this would have presented a considerable challenge in terms of data availability and would have introduced additional parameter uncertainty. As part of the validation of the initial model we developed, we were able to reproduce, within reasonable limits, the original base-case results from the 2007 HTA report.

This initial model was populated with the parameter values used in the original modelling exercise conducted for the 2007 HTA report¹² and was driven by the diagnostic accuracy of the screening tests. For the screening arm of the model, this generated a number of referrals for suspected hearing impairment in the first year (true positives and false positives). As Figure 19 indicates, children who are referred by the screening test (positive result) are sent for a DEA for a definitive assessment (DEA being assumed to be 100% accurate). It was then assumed that all remaining cases of hearing impairment (false-negative results of the screen) would be identified over the following 2 years (i.e. up to age 7) by referral for a DEA as a result of concern by parents, teachers or other professionals. An assumption was required about the timing of these additional cases. In the absence of other evidence, they were evenly spread over the subsequent 2 years of the model. Similarly, an assumption was required about the rate of identification in the no screening arm. Under no screening, it is assumed that those with a suspected hearing impairment (on the basis of concern by parents, teachers or other professionals) are referred for a DEA. The background rate of identification was set so as to generate an even flow of diagnosed cases over a 3-year period.

Those referred by the screen or referred for a DEA on the basis of the concerns of parents, health professionals, teachers and others (in the presence or absence of screening) will either be found not to have a hearing impairment (false positives) or will be confirmed as having transitory or SNPC hearing impairment in either one or both ears (true positives). Management strategies received by children with hearing impairment include active observation, non-surgical and surgical interventions, with the type of intervention offered varying depending on the severity of hearing impairment.

Model development incorporating new clinical data

Although the basic model structure illustrated in Figure 18 has remained essentially unchanged in the current version of the model, there have been substantial revisions compared with the model described above to take account of the clinical observations reported in previous chapters. The present model has been informed by the two-gate (‘case–control’) study investigating the diagnostic accuracy of the two screening methods (see Chapter 3), a comparison of a site with a SES programme (Nottingham) and a site without a SES programme (Cambridge) (see Chapter 5), a study exploring the impact of referral for diagnostic evaluation on parents and children (see Chapter 6) and data on practical implementation in schools (see Chapter 7).

For both transitory and SNPC hearing impairment, the model takes account of the benefits of management for those found to have mild, moderate or severe hearing impairment. The analysis runs for a 4-year time period, starting in the year before school entry. By the end of the 4-year time period, evidence on referrals from the studies undertaken in Nottingham and Cambridge indicates that all cases of hearing impairment are likely to have been diagnosed regardless of whether children at age 4 years are screened or not screened. In the absence of screening, identification of hearing problems occurs as a result of the concerns of parents, teachers or other professionals, sources which can also generate referrals when a screening programme is in place. As discussed in the following sections, the results of the model are primarily driven by the total numbers of referrals with and without screening, and the numbers referred for a DEA in each year.

Given the key influence of referrals on the results of the analysis, the analysis has explored the effect of varying the number and pattern of referrals over time. This analysis has been reported as a threshold analysis following the presentation of the base-case results. A probabilistic sensitivity analysis (PSA) was not carried out, as it was not considered that probability distributions could usefully be attributed to the numbers of referrals or their timing based on referrals data drawn from two areas. It is argued here that PSA may not always be the most informative technique for exploring parameter uncertainty, as it can result in attention being focused on those variables for which probability distributions are most easily attributed rather than those that have the greatest influence on the results. In this case, while it is recognised that there will be variability in the numbers and rate of referrals with and without hearing screening, this uncertainty is particularly difficult to quantify. Threshold analysis was therefore performed on these variables.

Total referrals and cases of hearing impairment

Table 28 presents the key items of data obtained from the clinical studies that were used in obtaining estimates of the prevalence of hearing impairment, numbers of referrals and results of the screening tests.

TABLE 28

Key data points derived from clinical studies

The total number of cases of hearing impairment to be identified (with or without screening) is based on the prevalence of hearing impairment using the number of confirmed cases of hearing impairment in Nottingham (n = 195) as a proportion of the base population (N = 42,553). This gives a prevalence of approximately 46 cases in a population of 10,000. These cases were divided into SNPC and transitory cases of hearing impairment in accordance with the assumption used in the 2007 HTA report.¹² In the base case, the number of referrals has been assumed to be the same in both screening and non-screening arms of the model and is given by the number of children referred in Cambridge (n = 1108) as a proportion of the base population (N = 17,624), or around 6.3%. This is equivalent to 629 referrals in a population of 10,000. Using the rate of referrals observed in Nottingham does not have a material impact on the cost-effectiveness results. The estimates of test sensitivity are the child-level estimates reported in the diagnostic accuracy study. The implications of these data in terms of screening results (true positive, false positives, true negatives and false negatives) are presented in Table 29.

TABLE 29

Screening results: persons per hypothetical cohort of 10,000 children screened

The three groups of most interest are the true positives, false positives and false negatives since no further intervention is required in the true-negative group. True positives and false negatives are generated by the observed diagnostic sensitivity of the two tests, given the prevalence of hearing impairment. Total referrals are made up of true positives, false positives and false negatives, the last of these groups being referred by means other than the screening test so that all cases of hearing impairment are ultimately identified.

Owing to the constraints imposed by the numbers of referrals and prevalence of hearing impairment, the resulting number of false positives implies a substantially higher test specificity (the probability of a child without hearing impairment testing negative) than that found in the new diagnostic accuracy data from Nottingham (Table 30). Moreover, these constraints mean that the implied specificity is the same under either screening method in the base case and cannot be varied between the PTS and HC screener.

TABLE 30

Diagnostic accuracy

However, it was possible to explore the impact of varying the numbers of false-positive referrals between the screening and no screening options by varying the total number of referrals. This is a particularly relevant area of uncertainty to consider as, in the base case, total referrals are assumed to be the same with and without screening, whereas evidence from the service comparison study suggests that the rate of referrals may be higher in the absence of screening than when a screening programme is in place.

Although the significantly higher referral rate in Cambridge than Nottingham (34.4 vs. 21.9 per 1000 children per year) does not lead to a significantly higher yield of confirmed cases in Cambridge (3.04 vs. 2.51 per 1000 children per year), the possibility that a screening programme will reduce the number of referrals needs to be considered. Given the differences between the characteristics of the populations in the two areas, it is unclear what the difference in referral rates might be in any given area in the presence or absence of screening. Nevertheless, a higher rate of referrals in the no screening arm could result in a substantial increase in costs relative to the screening option (given the unit cost of diagnostic evaluation relative to screening) and thus have an important influence on the cost-effectiveness of screening. We therefore vary the referral rate between screening and no screening arms of the model in sensitivity analysis.

Distribution of referrals over time

In the current model, the previous assumptions about the identification of hearing impairment over time have been superseded by data on referrals obtained from the screening area (Nottingham) and the no screening area (Cambridge). Table 31 gives the distribution of referrals by age at last birthday in Cambridge and Nottingham. Referrals at age 3 years are taken to apply to the pre-school entry year (year 1 of the model) while referrals at ages 4–6 years are taken to apply to the first 3 years of school (years 2–4 of the model). In addition to the distribution over time, the model also takes account of the different sources of referral in the presence and absence of screening, based on the available data (Table 32). It is worth noting the importance in the screening area of sources of referral other than screening.

TABLE 31

Percentage of children being referred to diagnostic evaluation by year: base case

TABLE 32

Distribution of referrals by type of referral

Management of hearing impairment

Once cases of hearing impairment are confirmed by a DEA, their management will depend on the type of hearing impairment and its severity. The available management options are essentially unchanged from the 2007 HTA report.¹² We simply note here that management of hearing impairment results in an improvement in quality of life, which translates into QALYs. While the total QALY difference between screening and no screening will include the QALYs experienced by those with no hearing impairment, we exclude these from our results, as we assume that the characteristics of the modelled populations are the same in respects other than their exposure to type of screening (or no screening). The reported QALYs in screening and no screening groups represent the QALYs accruing only to those with managed hearing impairment who thereby experience an improvement in quality of life. The assumptions underlying the QALY calculations and the other parameters in the model are reported in the following sections.

Sources of parameter inputs

Values of input parameters for the model are based on the original HTA model, the literature, the observational studies described in greater detail elsewhere in this report (see Chapters 3, 5–7) and standard reference sources. A limited literature review was carried out to update health state utilities and costs. For probabilities of different severities of hearing impairment and utilisation of management options, the model used the data on which the original HTA report¹² was based as the steering group felt that these were still relevant.

An overview of sources for the main broad categories of data input is given in Table 33.

TABLE 33

Data sources used for the current model in comparison with the 2007 HTA report

Reviews of the literature

A short literature review was undertaken to search for any updates to the data required by the model that were not expected to be included in the results of the clinical studies. These were utility values of hearing impairment, the prevalence of hearing impairment (using data from the NHSP) and costs of management.

The scope for searches was publications since 2007, to capture new research published since the 2007 HTA report.¹² Search ranges were from January 2007 to July 2014 and were carried out on 10 July 2014.

The following databases were searched:

• MEDLINE
• EconLit
• SocINDEX
• PsycINFO.

Searches were also run through Google Scholar.

The following charity and support group websites were checked for useful publications (grey literature) that may not show up in searches of conventional databases:

• Action on Hearing Loss
• National Children’s Bureau
• National Deaf Children’s Society.

Referrals

The clinical studies have, to date, reported on samples of 1108 children referred to audiology services in the no screening area (Cambridge) and 1702 children referred in the screening area (Nottingham).

Number of children with hearing impairment

Determining the number of children who need to be identified is a critical assumption within the model and the majority of the analysis relies on this calculation. The original HTA report¹² on which this analysis draws assumed that the prevalence of hearing impairment was 78 of 1000 children. This was split into approximately 3.5 of 1000 children with SNPC hearing impairment and 74.5 of 1000 children with transitory hearing impairment through the probabilities that populate the model. The SNPC number is supported by research used, and subsequently published, for the 2007 HTA report,¹² which gives a figure of 3.65 of 1000 children with SNPC hearing impairment.^8,12

However, although this figure is likely to be an accurate representation of the total prevalence of SNPC hearing impairment in the child population, it is also likely to be a sizeable overestimate of the numbers of children who have SNPC hearing impairment yet to be diagnosed at the age of 3 years (prior to the start of the model). This is because cases of hearing impairment in children are generally identified either through the NHSP or by parents before school age. Moreover, while our main interest is in children with SNPC hearing impairment, the vast majority of the children who are detected as a result of screening will have transitory hearing impairment.

The prevalence of both transitory and SNPC hearing impairment in the modelled population has been estimated on the basis of the numbers with a confirmed diagnosis of hearing impairment as a proportion of the base population in the two areas studied. Based on last appointment, we obtain estimates of 46 per 10,000 children in Nottingham (195/42,553) or 56 per 10,000 children in Cambridge (98/17,624). The former has been used for the purposes of the model and divided between transitory and SNPC hearing impairment in the ratio 96% : 4% as applied in the 2007 HTA report.¹² This gives approximately 2 children per 10,000 with undiagnosed SNPC hearing impairment and 44 with undiagnosed transitory hearing impairment.

Diagnostic accuracy

The sensitivity of the two screening tests used in the model was based on the child-level analysis of the case–control study using all children whether nominally recruited as cases (HI) or controls (NHI), where hearing impairment was defined as a PTA score of ≥ 30 dB on at least one of the four frequencies. Estimates of diagnostic accuracy have been reported (see Table 30).

Both screening methods generate a small number of false-negative cases, and, in these instances, the children should be picked up via other referral methods such as through GPs or speech therapists. Differences in sensitivity between the two screening methods will give rise to different numbers of false negatives which have been distributed over time in the model in the manner illustrated in Table 34. The distribution draws on the distribution of total referrals but adjusts for false negatives of the screening tests occurring only in year 2 onwards (rather than all 4 years of the model).

TABLE 34

False negatives under the PTS and HC tests

Probabilities of hearing impairment by severity

Table 35 reports the proportions of cases of transitory and SNPC hearing impairment that are unilateral or bilateral and the distributions of unilateral and bilateral SNPC hearing impairment by severity. The source of these probabilities is the 2007 HTA report.¹²

TABLE 35

Distribution of hearing impairment states

Management probabilities

Probabilities of each management type for unilateral and bilateral transitory hearing impairment and the different severities of unilateral and bilateral SNPC hearing impairment were sourced from the 2007 HTA report¹² and are listed in Table 36.

TABLE 36

Distributions of management types

Resource use and costs

The following sections report the sources underlying the calculation of costs over the 4-year modelling period. The cost base year is 2012–13. Costs in the first year are undiscounted and, in subsequent years, are discounted at an annual rate of 3.5%. This is the rate recommended by the UK Treasury Green Book,⁵⁷ based on the rate at which individuals discount future consumption over present consumption.

Screening costs

All children in the screening arm incur the costs of screening. Children who are referred by the test also incur the cost of a DEA and the subsequent cost of management is incurred by children who are referred by the screening test and are diagnosed with hearing impairment by a DEA.

The length of time to perform the test was collected (see Chapter 7). A mean duration of 1.4 minutes (measured to be the same for both methods) and an hourly rate of £73 per hour from the relevant 2012–13 Reference Costs⁶⁰ (N05OGS – School-Based Children’s Health Other Services – Group Single Professional) give a staff cost associated with the screening tests of £1.69. This is reported in Table 37 while unit costs for other sources by which hearing impairment can be identified are presented in Table 38. The cost per child of screening was calculated by dividing the total costs (total costs of screening tests and total capital costs) by the total cohort of children screened over 5 years (10,000). The cost of each diagnostic evaluation was £150 (NHS Reference Costs 2012–13⁶⁰).

TABLE 37

Unit costs of the PTS and HC tests

TABLE 38

Unit costs of all non-screening forms of identification

Cost of travelling to appointments

Sixty respondents to a questionnaire asking about travel costs to attend screening appointments in Nottingham reported a total of 129 appointments and expenditure of £788.64, or £13.14 per child.

Unit costs of the pure-tone screen and HearCheck screener

Unit costs of each method of screening are presented in Table 37. Where costs extend beyond 1 year, these have been discounted accordingly in the model. Equipment suppliers have provided the costs of devices and consumables. Although the device costs for the PTS are higher than those for the HC screener (£898.80 vs. £139.20), the total costs for the HC screener are higher, primarily owing to the costs of ear cups. Table 38 reports the unit costs associated with other means by which hearing impairment is identified.

Management costs

Management costs were compiled from NHS Reference Costs for 2012–13,⁶⁰ representing national unit costs, and are reported in Table 39. Where cost estimates were available for unilateral impairment only, these were multiplied by two to give the corresponding costs associated with bilateral impairment. This is a conservative assumption to account for the costs of providing and maintaining hearing aids for two ears. In the case of surgery, expert opinion suggests that this approach may overestimate costs. No follow-up costs or postoperation observation (active observation) were taken into account with surgical interventions. As with NHS reference costs generally, figures relating to paediatric services or the under 18 years age group were used where available. However, for hearing aids, the reference costs are not broken down for children and adults separately.

TABLE 39

NHS management-related reference costs used in the model

Utilities

A search to update the utilities from the 2007 HTA model¹² was undertaken and the results were used to inform the updated parameter sheet (Table 40). Where possible, a distinction was made between unilateral and bilateral hearing impairment; otherwise, the utility associated with hearing impairment was determined primarily by its severity. In the first year (the pre-screening year), we assume that children entering the model do so evenly over the course of the year. Total utilities in the first year are therefore reduced by 50% (in undiscounted terms) compared with subsequent years. QALYs have been calculated only for children with a managed hearing impairment who consequently achieve a quality-of-life improvement. As QALYs accruing to children without hearing impairment do not affect the incremental cost-effectiveness ratio (ICER), they have been excluded from the calculations. Limited updating of the parameter values used in the 2007 HTA report was possible. For example, utilities for cochlear implants came from Summerfield et al.,⁶² and utilities for grommet surgery came from Bisonni et al.⁶³ These studies provided intervention-specific utility data to supplement the evidence on utility by severity of hearing impairment.

TABLE 40

Utilities used in the model

Modelling the potential costs and consequences of false-positive results

Chapter 6 presents the outcomes of a questionnaire given to parents whose children were referred from SES for a DEA appointment. Survey responses were obtained in respect of 60 children over 129 appointments.

The questionnaire asked a range of questions concerning parents’ views of the screening programme and the impact on themselves and their children. The travel costs associated with screening and diagnostic visits were based on this survey. One impact that is commonly discussed in the context of screening programmes, although not straightforward to value in monetary terms, is the anxiety associated with false-positive results. This was an issue addressed by the questionnaire.

Anxiety

Parents were asked to rate their anxiety level on a scale from 0 to 10 as a result of finding out that their child needed further testing. Approximately 60% of respondents listed their anxiety as ≤ 5 out of 10, with only one parent rating their anxiety > 8 out of 10. Between finding out that their child needed further testing and attending the clinic visit, mean parental anxiety score fell slightly, from a score of 5.3 to a score of 4.7. The survey was felt to provide insufficiently compelling evidence to make an adjustment to the model, either by incorporating anxiety into the QALY or as a monetary disbenefit. On the basis of current evidence, it is unknown whether or not there is a health impact on parents of further hearing tests that would outweigh the benefits gained by children receiving a correct diagnosis; this is an issue on which further research may shed some light.

Number of referrals to diagnostic evaluation and numbers diagnosed with hearing impairment

Tables 41 and 42 show the number of children referred for a DEA and the numbers diagnosed with hearing impairment in each model arm over the 4 years of the model. Under the base case, a hypothetical population of 10,000 children has been used, of which the number referred for diagnostic evaluation in the counterfactual and the two intervention arms is 629, and the number with hearing impairment is approximately 46. The numbers referred in each year are determined by the data on referrals at different ages in the screening and no screening sites and have an important bearing on the cost-effectiveness of screening compared with no screening.

TABLE 41

Rate at which children are referred to diagnostic evaluation with audiologist in each model arm: base case

TABLE 42

Rate at which children are diagnosed in each model arm (based on a hypothetical 10,000 population)

School entry hearing screening versus no screening: costs and quality-adjusted life-years

Tables 43 and 44 present the undiscounted and discounted costs broken down into the costs of identification (whether by screening or other means), diagnostic evaluation and management over 4 years for each arm of the model.

TABLE 43

Undiscounted costs

TABLE 44

Discounted costs

Diagnosis and management of hearing impairment has a positive impact on children’s quality of life as measured using the QALY. The figure for QALYs gained outlined in Table 45 is an estimate of the QALYs gained from children being diagnosed and managed (children moving from hearing impairment to no hearing impairment/managed hearing impairment) over the 4 years of the model. The QALYs accruing to children with no hearing impairment are not included here. The results show that, in the base case, not having a screening programme results in more QALYs than either the PTS or HC screen, which is associated with the lowest QALY gain of the three options.

TABLE 45

Quality-adjusted life-years for children with hearing impairment

In order to determine the cost-effectiveness of each method of screening compared with no screening, the ICER needs to be calculated. The ICER presents the ratio of the marginal gain of the intervention over the counterfactual in terms of both costs and benefits. It is calculated as:

Table 46 presents incremental costs and QALYs for the two screening approaches relative to no screening and for the PTS compared with the HC. In the base case, it is not appropriate to report an ICER, as no screening dominates (is more effective and less costly than) either screening approach. In the context of our research question, the results indicate that SES is not cost-effective compared with no screening.

TABLE 46

Incremental costs and QALYs (discounted)

Sensitivity analysis

The key data from the clinical studies that influence the magnitude and direction of the cost-effectiveness results primarily relate to the number and timing of referrals with and without a SES programme. While there is uncertainty about the applicability of the findings from Nottingham and Cambridge to a more general assessment of the costs and benefits of a screening programme as opposed to no screening programme, it is difficult to put boundaries on this uncertainty. This is in part because of the sociodemographic differences between the populations in Cambridge (the counterfactual) and Nottingham (screening site), and to the differences in service configuration between the two sites. A factor of particular significance is the difference in the rates of referrals between the two areas (the question of whether a screening site has more or fewer referrals than a non-screening site was an explicit objective of the study). In the light of clinical data suggesting that the referral rate in Cambridge is higher than that in Nottingham, the impact of a higher referral rate in the absence of screening was felt worthy of exploration. Increasing the referral rate when no screening programme exists increases the costs of this option while leaving QALY benefits unchanged, as these depend on the timing rather than the number of referrals. Increasing referrals sufficiently will render no screening more costly than screening and enable the trade-off between costs and benefits to be investigated.

In order to determine whether an intervention is cost-effective, the National Institutes of Health and Care Excellence (NICE) recommends comparing the ICER with a benchmark value of between £20,000 and £30,000 per QALY gained.⁵⁸ Technologies with an ICER of < £20,000 are generally considered to be cost-effective while, for those with an ICER > £20,000, reference needs to be made to other factors when considering value for money (and the case needs to be made increasingly strongly in relation to these factors when the ICER is > £30,000). Using £30,000 per QALY as the cut-off point for cost-effectiveness, the referral rate in the absence of screening would need to increase by ≥ 36% for no screening to cease being cost-effective relative to the PTS. This gives an upper limit on the extent to which referrals can be increased in the absence of screening without this option becoming excessively costly relative to its QALY benefits.

We are also interested in the circumstances under which screening becomes more effective than no screening. Leaving the baseline level of referrals unchanged, we investigated the extent to which referrals would need to be brought forward with screening compared with no screening. As we lacked clear bounds to place around the proportions of children referred at different ages, we again conducted a threshold analysis. It was found that the benefits of the PTS test would be increased sufficiently for the ICER to fall below £30,000 per QALY gained compared with no screening if the proportion of children referred in the first year of school (the screening year) increased by 5.9 percentage points or more.

Compared with the distribution and total numbers of referrals, other variables had relatively little impact on the conclusions of the analysis. For example, raising the sensitivity of the screening tests to 100% increased the QALY benefits under the PTS and HC but not sufficiently to make screening more effective than no screening. Altering the prevalence of hearing impairment had no impact on the results.

Discussion of key assumptions

Several key modelling assumptions require further discussion. Good use of assumptions is crucial to economic modelling. Done correctly, assumptions simplify the modelling approach, allowing for targeted models to be built using the best quality of data. A model with large parameter demands often has to make compromises and assumptions that can end up weakening the model.

One of the main assumptions in the model concerns the rate of referral to diagnostic evaluation. For the base-case scenario, it has been assumed that the number of referrals in the screening and non-screening areas is the same. From the comparative data presented in Chapter 5, the rate of referral in Cambridge (area without SES) was higher than that of Nottingham (area with SES). However, after careful consideration of the issue, these comparative data were not used for the base case, as Cambridge may not be reflective of all non-screening areas. Assuming that the rates of referral would be higher when no screening programme is in place makes an implicit assumption that other methods of referral, such as referral via GP, health visitors and speech therapists have low specificity. That is, it assumes that these other methods require a higher number of referrals than are required with a screening programme to identify the same number of cases of hearing impairment when, in practice, it is not known what the true increase in rate of referrals for non-SES areas relative to SES areas would be.

In the base case, screening is more costly than no screening. However, if referrals are increased in the no screening option, there is a point when no screening becomes more costly than screening. If referrals are increased further, the no screening option will eventually become too costly to justify its additional QALY benefits over screening relative to conventional cost-effectiveness benchmarks such as the £30,000 figure used by NICE. Sensitivity analysis considered the ‘tipping point’ in terms of referral numbers under the no screening option at which no screening would no longer be cost-effective. It was found that the cost per QALY ratio of no screening relative to the PTS test increased to £30,000 if the rate of referrals was 36% higher in the absence of screening compared with the PTS. In this case, screening is the cost-effective (if less effective) option.

An alternative way in which screening could become cost-effective is if, despite being more costly than no screening (as in the base case), it was also more effective. This could come about if screening was associated with more rapid detection of cases of hearing impairment than no screening. A threshold analysis on the pattern of referrals suggested that, under screening, referrals would need to increase by around 5.9 percentage points in the screening year (increasing the proportion of referrals taking place either in the pre-screening year or the screening year from 59% to 64.9%) in order to reduce the cost per QALY of the PTS relative to no screening to £30,000. While there are grounds for believing that the number of referrals is likely to be higher without screening, the potential for screening to achieve timelier referral and management of HI children is less clear.

Strengths and weaknesses

The model developed has a number of features that support the validity of its results.

It builds on an existing model (2007 HTA report¹²), which allowed a systematic consideration of areas where the original model could be improved and incorporates these into the updated model. This was greatly assisted through the involvement of the architect of the original model who contributed to the project as a consultant (Professor Linda Davies).

The model was conducted by an experienced multidisciplinary team of researchers who had been involved in the development of economic models, and models concerning hearing impairment in particular, prior to this project.

The model also drew on the experience of the project steering group, both in terms of content knowledge to advise on the design of the model and the input of a health economics specialist who fed back on the model design and early results.

The model was underpinned by a protocol outlining the key features and limiting the opportunity for results to be data driven. In the event, changes were made to the structure of the model so that it could capture aspects of the impact of SES that were not originally anticipated. These changes have been fully documented relative to the original model plan, so reducing the opportunity for bias.

The model was conducted in parallel with a series of clinical studies that were designed to improve information on key parameters where high levels of uncertainty had been identified in the 2007 HTA report model (see Chapters 3, 5–7).¹²

There were also some limitations. The greatest limitation was that, despite attempts to reduce uncertainty, the data collection in the accompanying clinical studies was unable to overcome this uncertainty completely. For instance, the study designs for accuracy were chosen on the basis of feasibility and used a diagnostic case–control study, which is known to exaggerate accuracy, particularly where the controls are healthy subjects. Similarly, a randomised comparison between SES and non-SES areas would have been desirable to assess the impact of SES on referrals and yield. Instead we had to employ an observational two-centre comparative study design subject to major potential limitations, including confounding and lack of generalisability. In retrospect we could have invested research in quantifying the accuracy of processes used to refer children for a DEA where SES was not in place, but the importance of this emerged only when the results of the clinical studies were reported.

The inevitable lack of availability of the results of the clinical studies until late in the research programme limited the time available to perform sensitivity analyses in the economic model. These were thus prioritised in consultation with the research and steering group and we remain confident that all key aspects have been covered and that the cost-effectiveness findings remain robust.

Findings in comparison with other health economic evaluations

The 2007 HTA report¹² identified virtually no health economics literature on SES, and the update search for this report identified no new health economics literature since the 2007 report. Thus the main point of comparison for the new economic model is the economic model from the 2007 HTA report.¹²

The most important difference between the two reports is a change in view about the likely cost-effectiveness of SES from possibly cost-effective (albeit with considerable uncertainty in 2007) to probably not cost-effective in this report. This change is primarily because of the use of observations on the timing of referrals as the basis for the updated model. The 1-year results from the 2007 report, showing a favourable cost-effectiveness ratio for screening, are consistent with a substantial advantage for screening in terms of the timeliness of referrals compared with no screening. This is also implied by the assumptions used in initial attempts to replicate the 2007 results. In comparison, the observational data incorporated into the model on which the findings presented here are based suggest that screening does not result in a more rapid rate of referral and that other methods used in the absence of screening may be more effective in this regard.