Methods
Data collection
Data on NHS 111 calls and online contacts were collected between October 2010 and December 2019 and between January 2018 and December 2019, respectively. The NHS 111 calls data were obtained from the NHS 111 Minimum Data Set, Time Series, to December 2019 and accessed online (URL: www.england.nhs.uk/statistics/statistical-work-areas/nhs-111-minimum-data-set/nhs-111-minimum-data-set-2019-20/) in January 2020.49 The NHS 111 Online data were provided by NHS Digital. Call data provided monthly counts at the NHS 111 area code level, whereas online provided daily, individual-level data. As call data were at a summary level only, analyses of both data sets were conducted at this level. Both data sets provided variables for all calls and contacts; triaged calls and contacts; calls and contacts passed for clinical advice; and the final disposition for the call/contact as the advice about which service to contact or attend, or where no service was needed.
The call data were made up of 71 NHS 111 area codes and the online data were classified by both Sustainability and Transformation Partnership (STP) and CCG areas, which were mapped to 38 of the NHS 111 area codes. For historical call data, some of the older codes have been merged into newer area codes. These 38 area codes were the potential sites for our analysis. However, because of the way the NHS 111 telephone service is commissioned, geographical coverage of different service providers means that either the area codes can be made up of multiple STPs or some STPs can be split over two area codes. Because the NHS 111 Online service was introduced at STP level, this meant that not all online area codes could be included, as some STPs had different start dates. In addition, for the interrupted time series analysis, 1 full year of NHS 111 Online data were required; therefore, any area codes where the online service had not been operating for at least 1 year were removed. For consistency, the call data were capped at 2 years prior to the introduction of the NHS 111 Online service, so each area code had a minimum of 36 months of data. This meant that there were 18 area codes remaining for the analysis; however, these areas are spread across the country and represent a range of geographical, service size and provider types, so we are confident that they are a reasonable representation. It was unfortunate that the area where the NHS 111 Online service was first introduced (Yorkshire and the Humber) and, therefore, had the longest post-implementation time could not be used. This is because the NHS 111 telephone call data are reported for Yorkshire and the Humber as a single area, but there are 22 CCGs and the NHS 111 Online service became live at different times in these CCGs, meaning that we could not account for the time point of change in the interrupted time series models. A list of the 18 sites included is provided in Appendix 3, Table 28.
Outcomes measured
The primary outcome was the impact of introducing the NHS 111 Online service on the number of triaged calls to the original NHS 111 call service. Triaged calls are those where a call advisor assesses the call using NHS Pathways and excludes calls that, for example, just provide health information with no assessment.
The secondary outcomes were to explore the impact of NHS 111 Online on the total number of calls, the number of clinical call backs and the effect on the relative proportions of final dispositions and hence on other related services.
These primary and secondary outcomes can be split into those which affect the NHS 111 telephone service only and those that have an impact on the wider NHS system.
NHS 111 telephone service only:
triaged calls
total calls answered – including non-triaged calls for health information, and those where the caller terminates before triage
clinical call backs – calls referred to clinical advisors for further assessment.
Wider NHS system:
emergency ambulance referrals or advice to contact 999
advice to attend an emergency department or other urgent care treatment facility
advice to contact primary care
advice to contact with community service
advice to attend another service
no recommendations to attend any service and self-care advice.
For calls to the telephone service, there is a specific data variable recording the number of calls passed for and receiving a clinical assessment. This is less clear for the NHS 111 Online data, as clinical call-backs are managed outside the online platform. The online data provided two data variables, a clinical call back variable (offered and sent or offered and failed to send) and the final disposition (advice) code (DX code) at the end of the questioning. There were 11 DX codes for ‘speak to a clinician from our service immediately’ as the final disposition. If a patient had both a clinical call back variable and a relevant DX code, they were assigned as having a clinical call back. The online clinical call backs were added to the telephone call clinical call backs as the outcome for this analysis.
The secondary outcomes, which look at the impact on the wider NHS system, were created in a similar way to the clinical call back outcome. The outcome was made up of a combination of the dispositions (DX codes) from both the call and the online data. For example, the outcome of all ambulance referrals is made up of all the ambulance dispositions in the call data plus all the ambulance dispositions to call 999 in the online data.
Statistical analysis
All analysis was conducted using R, version 3.6.3 (R Core Team, 2020; The R Foundation for Statistical Computing, Vienna, Austria).
Interrupted time series
To model whether or not the introduction of the NHS 111 Online service has had an impact on the monthly number of calls, an interrupted time series was used. However, unlike with conventional interrupted time series, a dose–response model was used. This means that instead of the number of calls being modelled as a function of the time after the launch of the online service, it was modelled as a function of the number of online contacts that month. The dose–response model provides an estimate of the reduction or increase in the number of telephone calls per online contact. The model also included some systematic components: an underlying time trend, a step change for when NHS 111 Online was introduced and ‘fixed’ seasonal effects (four levels: December–February, March–May, June–August and September–November).
As each area code had different start dates for the introduction of NHS 111 Online, each was modelled separately and a meta-analysis was used to determine the overall effect (see Meta-analysis). Given that there were 18 different area codes and a range of outcomes to model, we decided to use the same model for each site and outcome, but different models were used as a sensitivity analysis.
To determine which model to use, four area codes were chosen independently by two statisticians (RS and RJ) to test the models. Both statisticians chose the same four sites (Hertfordshire, Milton Keynes, North East and Nottinghamshire) as these represented areas with large to small numbers of calls. The models tested were Poisson or negative binomial, and whether or not the model was autoregressive (AR) was also examined.
To determine whether an AR model was appropriate, the primary outcome was differenced to remove the general upwards trend (see Appendix 3, ) and the autocorrelation function and partial autocorrelation function plots were investigated. It was agreed for all four models that an AR model was not needed but there may be some seasonality. Seasonality would be accounted for with the season variable which was prespecified to be included in the model.
As the primary outcome variable was the number of triaged calls to 111 each month, these are count data, so either a Poisson or a negative binomial model was considered. After fitting both the Poisson and the negative binomial models, the model output for the Poisson model showed that the data were overdispersed. Given this, the negative binomial model was chosen over the Poisson model. The autocorrelation function and partial autocorrelation function plots were investigated again and it was confirmed that an AR model was not needed (see Appendix 3, and ).
The final model used for the analysis was:
where the outcome is the number of calls to the NHS 111 telephone service each month, time is a linear variable 0, 1, 2, . . ., dose is the number of NHS 111 Online contacts for each month, step is a binary variable that is coded 0 before the introduction of NHS 111 Online and 1 afterwards, and season is a fixed variable that represents the four seasons in the year.
For sensitivity analyses, an AR(1) model and a non-linear model with a non-linear term for time were used.
Both of these sensitivity models were applied to all sites and outcomes. The Isle of Wight was one of the 18 area codes included in the analysis. Owing to the size of the Isle of Wight, with small call volumes and an atypical urgent care service configuration, we have included a further sensitivity analysis from which we have excluded the Isle of Wight.
All three of these models included a log-link function. For the linear and non-linear negative binomial model, the glm.nb function of the Modern Applied Statistics with S (MASS) package was used.50 For the AR model the tsglm function of the tscount package was used.51
Summary
The introduction of the NHS 111 Online service has added another access point for urgent and emergency care in the NHS. The online service operates as well as, not instead of, the existing telephone service, creating two sources of activity. Both services direct users to services in the emergency and urgent care system, unless the health problem is suitable for self-care. We have conducted an interrupted time series analysis to assess changes in activity following the introduction of NHS 111 Online using a dose–response model, where the ‘dose’ is the number of contacts with the NHS 111 Online service. The results for the primary and secondary outcomes assessed are summarised in .
Average percentage increase in calls per 1000 online contacts
The primary outcome was the impact on activity to the NHS 111 telephone service as the two are directly related. We found that the online service has little impact on the number of triaged and offered calls, so the workload for NHS 111 has neither increased nor decreased as a result of the introduction of NHS 111 Online, suggesting that there has been no substantial shift to using the online service instead of the telephone service. However, this was not a consistent finding, and four sites showed a reduction in triaged calls, indicating that it is possible that there has been a shift away from the telephone service to the online service.
The secondary outcome most directly related to the NHS 111 service is assessment by an NHS 111 clinician, as the online service has a facility for requesting a clinical call back from an NHS 111 clinician. We found that the number of clinical calls decreased, which was unexpected, as there is potential for additional activity to be generated by NHS 111 Online ‘speak to our clinician’ dispositions. This may be a genuine finding, that more people with less urgent problems may be using the online service and having their problem resolved without the need for a clinician assessment, diverting these people from the telephone service. However, it could also be a consequence of the way we have identified clinician call-backs in the NHS 111 Online data. We took a cautious approach and included contacts recorded as clinical call back offered and sent and with a final disposition (advice) code of ‘speak to our clinician’. It is possible that there were some relevant contacts where only one of these conditions was fulfilled and, as a consequence, the total number of clinical call backs has been undercounted.
For all other secondary outcomes, and hence for indirectly related services, the combined activity from the NHS 111 telephone service and the online service increased the overall number of recommendations to contact or attend those services following the introduction of the NHS 111 Online service. The biggest increase is for ‘other services’, which account for a relatively small proportion of NHS 111 telephone dispositions (6–7%).49 This finding may reflect a lower urgency for online contacts that can be directed to services such as an optician. At face value, the results would suggest a net increase in the demand for emergency and urgent care services, and this would not be surprising. For the 18 sites in our analyses, there were almost 600,000 contacts with the NHS 111 Online service, with no apparent shift away from the telephone service, and, nationally, there were over 2 million contacts during 2019. Previous research has shown that the introduction of new services and access points for emergency and urgent care, including NHS Direct, NHS 111 and walk-in centres, has created an increase in, and therefore new demand for, services.56–58 It is entirely plausible that this new online service could produce the same effect. However, the findings of the previous research were based on data of actual utilisation of other services in the emergency and urgent care system. For this analysis, we have been able to demonstrate only the recommendations about services to contact or attend and so potential increases in service utilisation. The potential service use increases estimated by our analyses would hold true only if every recommendation was acted on and if, by using NHS 111 Online, users subsequently accessed a service that they would not have used without a recommendation from the online service. The actual change in service utilisation is dependent on user behaviour: whether they choose to contact the recommended service, another service or no service or if they use the online service to confirm an action they would have carried out anyway (e.g. attend the ED or call their GP). We explore these factors in Chapters 6 and 7, in which we report the findings of the user survey and interviews, and we comment further in the discussion in Chapter 10.
It is also likely that any changes in demand may be influenced by other external factors. We have used a meta-analysis to produce an overall summary measure of effect from the 18 sites included in the analyses. However, the forest plot figures presented show considerable variation between different area codes, indicating that there may be local differences, for example in service availability and the amount of integration between services, and hence that the effect of introducing NHS 111 Online is unlikely to be consistent across different health economies.
Limitations
There are some limitations to these analyses. First, because of the rapid roll-out of NHS 111 Online as a national service, we were not able to use an experimental design with a control arm. This means that we cannot establish if the effects we have found are the direct result of the introduction of the new service or if they would have happened anyway because of other factors influencing both the NHS 111 telephone service and the wider emergency and urgent care system.
Second, the compromise of using the NHS 111 minimum data set aggregated data rather than patient-level data for the telephone service meant that we were able to successfully match only 18 of the 38 potential area codes to NHS 111 Online data, so we have not been able to establish a national estimate of impact. However, we are confident that the 18 sites included are representative of different geographical areas, activity volume and provider types across England to allow reasonable inferences.
Third, we evaluated the impact of the NHS 111 Online service during the early stage of its implementation, and at the sites we used it had been in operation for only 12–18 months. We have therefore estimated system impact based on the ‘dose’, in terms of contacts with the new service, present at that time. As the service becomes more widely understood by the public and as contacts increase, it is possible the impact may change and any subsequent assessment of impact would be more robust.
Finally, as discussed above, we have considered only recommendations for care, not actual care accessed, which might be quite different and will depend on how people use the service and their subsequent utilisation of other services: both those in the emergency and urgent care system and those in routine primary care. This makes it difficult to estimate how much new demand there may be. This is discussed further in Chapter 10.
