Research question

How effective and cost-effective is implementing an IPB for one exemplar injury prevention intervention?

Methods

Work stream 6 consisted of a review of reviews on implementation and facilitation of health promotion interventions (study L) and a RCT evaluating the effectiveness and cost-effectiveness of an IPB for the prevention of fire-related injuries (study M) with a nested economic analysis and qualitative study. The findings were incorporated into the development of a second IPB covering fire-related injuries, falls, poisonings and scalds.

Study M was a three-arm multicentre cluster RCT set in 36 children’s centres in Nottingham, Bristol, Norwich and Newcastle. Families with a child aged < 3 years were eligible to participate. Children’s centres were randomly allocated to one of three arms: IPB plus support (training and facilitation) (IPB+ arm), IPB without support (IPB-only arm) and control (usual care). IPB+ arm children’s centres were provided with training as well as facilitation contacts at 1, 3, 5 and 8 months. The intervention period was 12 months. The primary outcome measure was the proportion of families with a fire escape plan. Secondary outcomes included other fire safety behaviours and measures of IPB implementation, resource use and expenditure. Random-effects modelling was used to compare outcomes between treatment arms and for the economic analysis. Qualitative data were subject to thematic analysis.

Results

In study L, 10 reviews were identified. A number of common themes emerged about factors affecting the implementation of community prevention programmes. The review identified the Promoting Action on Research in Health Services (PARIHS) framework and Carroll et al.’s fidelity framework, which informed intervention design and the measurement of fidelity and implementation.

Thirty-six children’s centres and 1112 families participated in study M. Follow-up data were obtained from all children’s centres and from 751 (68%) families.

The IPB was implemented by children’s centres in both intervention arms, with greater implementation in the IPB+ arm. Compared with control arm families, more IPB+ arm families received advice on key safety messages and more families in each intervention arm attended fire safety sessions. The intervention did not increase fire escape plan prevalence (AOR IPB only vs. control 0.93, 95% CI 0.58 to 1.49; AOR IPB+ vs. control 1.41, 95% CI 0.91 to 2.20) but did increase the proportion of families reporting more fire escape behaviours (AOR IPB only vs. control 2.56, 95% CI 1.38 to 4.76; AOR IPB+ vs. control 1.78, 95% CI 1.01 to 3.15). IPB-only arm families were less likely to report children playing with matches or lighters (AOR 0.27, 95% CI 0.08 to 0.94) and reported more bedtime fire safety routines (AOR for a 1-unit increase in the number of routines 1.59, 95% CI 1.09 to 2.31) than control arm families. The IPB-only intervention was less costly and marginally more effective than usual care, whereas the IPB+ intervention was both more costly and marginally more effective than usual care.

Conclusions

Neither intervention was effective at increasing the proportion of families with a fire escape plan, but both IPB+ and IPB increased the delivery of fire safety messages by children’s centres and improved some fire prevention behaviours by families.

Objective

The objective was to evaluate the effectiveness and cost-effectiveness of an educationally based intervention (IPB) with or without facilitation as a means of changing the behaviours of families and children’s centre staff to improve fire safety in the home.

Design

We undertook a three-arm multicentre cluster RCT with an economic analysis and a nested qualitative study set in children’s centres in four trial sites in England (Nottingham, Bristol, Norwich and Newcastle). A cluster RCT was appropriate because the intervention was delivered at the level of the children’s centres and individual allocation of families living in close proximity could lead to contamination.

Participants

Allocation to the intervention and delivery of the intervention

Once the required number of families for each children’s centre had been recruited, children’s centres were stratified by trial site (four strata: Nottingham, Bristol, Norwich and Newcastle) and randomly allocated within strata to one of three arms using permuted block randomisation with a block size of 3. The allocation schedule was produced by an independent statistician using the randomisation algorithm in Stata/SE version 11 and was provided to an independent administrator who prepared sequentially numbered sealed opaque envelopes (one set for each of the four trial co-ordinating centres) containing allocations. Children’s centres were randomised in trios; once each stratum contained three children’s centres, the administrator opened envelopes for each block of three children’s centres.

The three trial arms consisted of:

1. IPB plus facilitation (IPB+ arm)
2. IPB without facilitation (IPB-only arm)
3. usual care (control arm).

The intervention was delivered over a period of 12 months. Children’s centres in the IPB+ arm were provided with the IPB, a training session (immediately before the start of the 12-month intervention period) and a facilitation package, described later in this section.

The IPB contained information:

• for managers on why preventing fire-related injuries is important, who the target group is, what effective interventions can be provided, creative ways of reaching the target group and how to evaluate use of the IPB
• for staff on why preventing fire-related injuries is important, who is at greatest risk, the main causes of house fires, what staff can do to help, what works to prevent house fires, where to get specialist advice and help and activities for use with parents, including session plans and resources covering five key messages (use of smoke alarms, fire escape plans, causes of house fires, children’s development and risk of house fires and bedtime safety routines).

Children’s centres were asked to deliver the fire prevention messages to participating families using the format that they considered most appropriate to their target audience. If they were unable to deliver all five messages, they were asked to focus on use of smoke alarms and fire escape plans, as these have the strongest evidence base. The IPB is provided in Appendix 6, Injury prevention briefing 1.

Children’s centres were provided with a training session prior to commencing delivery of the intervention. The aims of the training were to ensure that key staff:

• were familiar with the IPB and confident about its authority, reflecting how it had been developed
• understood the information that it contained
• felt confident about delivering the key safety messages to parents
• were aware of the support that the local fire and rescue service was able to offer to staff and parents
• understood their obligations as part of the trial and the support that the local research teams would be providing.

The training was participative in nature and started with injury epidemiology and why children are at risk of injuries and progressed to the content of the IPB and how to use the IPB. The training allowed people to try out an exercise from the IPB and to ask for further information. Training was led by the same person from the Child Accident Prevention Trust in conjunction with local research teams and the local fire and rescue services. When the detailed content of the IPB was introduced to participants, it was stressed that it was the key messages that they needed to present to parents. The exercises in the IPB were seen as one means of doing this, but centres were given the freedom to choose how best to deliver the key messages, having regard for the way that they interact with parents (e.g. group sessions, outreach, one-to-one opportunities, displays). The final part of the training programme (‘Using the IPB as effectively as possible’) was key to ensuring that children’s centres developed and implemented a plan for delivering the IPB. A draft implementation plan was provided, and this was incorporated into the facilitation package to enable researchers to assess progress with implementing the IPB.

The training session was piloted with nine staff covering a range of roles and seniorities from two children’s centres that were not participating in the trial. Piloting led to several small changes in the programme. Training was provided at venues away from children’s centres. To ensure consistency of training, the content of the presentations and discussions was monitored and recorded by the research teams. The training was evaluated by a questionnaire (shown in Appendix 6, Injury prevention briefing training evaluation questionnaire) completed by delegates at the end of the session.

The facilitation package consisted of the training plus contacts at months 1, 3 and 8. These used a two-stage approach with a postal or electronic questionnaire followed by a face-to-face or telephone interview, depending on progress with implementing the IPB. A fourth contact was made at months 4–5 if there was no progress with implementation of the IPB at month 3. The facilitation contacts collected information on progress with implementing the IPB, addressed children’s centre staff questions and discussed barriers to implementation, gave advice and examples of good practice from other centres and provided a resource list and list of contacts for other organisations. The facilitation package was designed to be similar to the advice and support that might be provided by an injury prevention co-ordinator (as recommended in the NICE guidance on injury prevention²⁵).

Children’s centres in the IPB-only arm were mailed the IPB and covering letter encouraging them to use the IPB. They were not provided with any training or facilitation. Children’s centres in the usual-care arm were asked to continue to provide their usual information on home safety. The IPB was posted to usual care arm children’s centres after collection of post-intervention data.

Outcome measures

Outcome measures were ascertained at 12 months’ follow-up, defined as 12 months post commencement of the intervention in the IPB+ and IPB-only arms and 12 months post randomisation in the usual-care arm.

Ascertainment of outcomes

Outcomes were ascertained using a range of tools as summarised in Table 104 and described in the following sections.

TABLE 104

Tools for measuring parent and children’s centre outcomes by treatment arm

Sample size

Eleven children’s centres per arm were required to detect an absolute difference in the percentage of families with a fire escape plan of 20% in either of the two intervention arms compared with the usual-care arm. This was based on 80% power and a 5% significance level (two sided), and assumed a usual-care arm prevalence of 42% and an ICC of 0.05 (ascertained from structured interviews with families attending children’s centres in the four trial sites⁴⁸⁰) and that outcomes were assessed on 20 families per children’s centre. In total, 33 children’s centres were required, which was increased to 36 to allow for potential dropouts. Allowing for 33% loss to follow-up among families, 30 families per children’s centre (total 1080) were required.

Blinding

It was not possible to blind children’s centre managers and staff, researchers providing the facilitation package or families to treatment arm allocation. When parents required support from a researcher to complete the follow-up questionnaire, or when questionnaires were completed by telephone, researchers were not blinded to treatment arm allocation. Quantitative analyses were undertaken blind to treatment arm allocation for the primary and secondary outcomes but not for the economic analysis.

Withdrawals

Participants were free to withdraw from the trial at any stage, but their data were included up to the date of withdrawal.

Analysis

This section describes the quantitative analysis for the primary and secondary outcomes followed by the health economic analysis and the qualitative analysis.

Baseline characteristics are described informally by treatment arm. All analyses of primary and secondary outcomes were conducted on an intention-to-treat basis in that families and children’s centres were analysed in the arm to which they were randomised. Quantitative analyses for the primary and secondary outcomes were undertaken using Stata versions 11 and 13.

Missing data

The main analysis was a complete-case analysis. Sensitivity analyses were undertaken for the primary outcome and included multiple imputation assuming that data were missing at random and analyses assuming no change in the primary outcome compared with the baseline value in families lost to follow-up.⁴⁸¹ Fifty data sets were imputed and combined using Rubin’s rules.⁷⁷ The imputation model included all variables in the model for the main analysis plus baseline variables, which were age of youngest child, number of children in the family, maternal age, accommodation type, housing tenure, ethnic group, number of adults in the household, number of smokers and whether or not there is a heavy drinker in the household. English as a first language was not included as a variable in the multiple imputation model because of problems with perfect prediction. Cluster number (as a categorical variable) was not included as a variable in the imputation model as the imputation model would not converge when it was included, but the multilevel logistic model run on the imputed data set took clustering into account.

Health economic analysis

The cost-effectiveness analysis of the trial utilised the primary effectiveness end point – whether a family reported having a fire escape plan – and the economic end point of total cost of the intervention. A summary of the base-case analysis is provided in Table 105 and includes the items recommended by the Consolidated Health Economic Evaluation Reporting Standards (CHEERS)⁴⁸² when reporting economic evaluations of health interventions.

TABLE 105

Summary of the base-case analysis

Data on the effectiveness end point – whether a family reported having a fire escape plan – were extracted from the parent 12-month follow-up questionnaires.

Resource use and cost data were obtained from three different sources:

1. trial site (i.e. Nottingham, Norwich, Newcastle and Bristol) researchers’ logbooks, which were used to record all of the activities relating to IPB implementation and, when applicable, facilitation
2. children’s centre follow-up questionnaires detailing their fire safety activities
3. parent 12-month follow-up questionnaires, which provided information about the resources and costs related to fire safety sessions that they attended and home safety inspections undertaken at their homes.

The cost-effectiveness analysis was conducted from a societal perspective and included costs incurred by children’s centres, fire and rescue services and other agencies that provide home safety advice/inspections such as local councils and family costs. Costs were analysed at the family level. To achieve this, the trial site and children’s centre-level costs were averaged equally across families randomised within each trial site and cluster, respectively, and then combined with the family-level costs to give a total cost per family.

As the inverse of the difference in probabilities of having a fire escape plan, say between IPB only and usual care, is equal to the number needed to treat (NNT), the cost-effectiveness ratio can be interpreted as the cost per additional fire escape plan under the intervention. The cost per additional fire escape plan (i.e. the primary outcome) was estimated for the IPB-only and IPB+ facilitation arms of the trial compared with the usual-care arm. In addition, the cost of developing the IPB was estimated from developers’ logbooks, but was not incorporated into the cost-effectiveness analysis as this is a fixed one-off cost that would not be encountered again if this intervention was implemented in practice.

The economic analysis was carried out with the family as the unit of analysis, and had to take account of the clustered nature of the trial design (i.e. randomisation was at the children’s centre level),⁴⁸³ adjusting for the baseline covariates included in the primary effectiveness analysis. Additionally, the clinical and economic end points may themselves be correlated within families as well as within children’s centre clusters, thus the analysis needed to simultaneously allow for this. Methods for such an analysis have recently been reviewed and compared⁴⁸⁴ (although they have rarely been used in practice), and guidance for good practice has been written. We adopted a random-effects modelling approach⁴⁸⁵ that extended the random-effects model used to analyse the effectiveness data on their own, as described above. A further complicating factor was that our effectiveness outcome (proportion of families reporting having a fire escape plan) was dichotomous, unlike many cost-effectiveness analyses for which both effectiveness and cost outcomes are continuous and often assumed to be multivariate normal.⁴⁸⁵ Multivariate methods for analysing a mixture of continuous and dichotomous outcomes have been developed⁴⁸⁶ including approaches that allow for clustering.⁴⁸⁷ We approached this complication by a factorisation of the model likelihood for costs and effects into the product of a marginal and conditional likelihood (continuous for costs and logistic for effects).^486,488 Such a factorisation also allowed the specification of distributions other than normal for the costs and, following a preliminary examination of the cost data, which were heavily (right) skewed, it was decided to model total costs at the family level using a gamma distribution (which has been advocated in the literature).⁴⁸⁹ This required adding £1 to each family cost for the analysis and then subtracting £1 from the results.

To construct an appropriate but somewhat ‘non-standard’ model we used WinBUGS⁴⁸⁸ software, which allows great flexibility in model specification and estimates model parameters using MCMC methods.⁴⁸⁸ An algebraic outline of the model is provided in Appendix 6, Statistical appendix and the associated WinBUGS code is available from the authors on request.

For the estimated model parameters, we report means, SEs and 95% CrIs, taken as the medians, SDs and 2.5–97.5% centiles from the samples of the posterior distributions. Cost-effectiveness acceptability curves were calculated by estimating the probability that the intervention was cost-effective for each value of the ceiling ratio (this is the value of the willingness to pay per additional fire escape plan) from the posterior distributions. The results tables present IPB only and IPB+ compared with usual care (but not compared with each other) to be consistent with the effectiveness analysis presented. However, the cost-effectiveness acceptability curves give the probability of each comparator being the most cost-effective for a range of willingness-to-pay thresholds and intrinsically compare all comparators simultaneously, which facilitates identification of the optimal intervention.

As a sensitivity analysis to check the robustness of the findings to missing data, a multiple imputation analysis was conducted. Following the same method as for the effectiveness imputation analysis described earlier, we extended the imputation model to include the four cost components (listed in Table 127), which sum together with the intervention costs (excluded from the imputation model because of no missing data) to produce the total overall cost per family. Because of non-normality of the cost component variables, predictive mean matching was used for imputation as well as for IMD score (as opposed to regression-based estimates for the other variables). As data analysis was carried out using MCMC in WinBUGS, it was not practical to perform 50 imputations (as performed for effectiveness); instead, 10 imputations were conducted and combined using Rubin’s rules⁷⁷ as before.

Qualitative analysis

Data from the facilitation contacts were analysed manually using content analysis after categorisation into main subheadings²⁵⁶ followed by a thematic analysis. Data from the implementation fidelity interviews were subject to framework analysis⁴⁹⁰ using the NVivo 10 software package (QSR International, Warrington, UK). A priori themes were identified that reflected the structure of questions within the 12-month follow-up facilitation and implementation fidelity interview schedules.

Initial analysis was carried out by two researchers who identified levels of implementation and emergent major and minor themes through cycles of coding. Levels of implementation were described based on recommendations for use contained in the IPB and the four elements of ‘adherence’ in the ‘implementation fidelity framework’ (content, coverage, frequency and duration of delivery). The initial coding framework was reviewed by the principal investigator and two senior researchers. Further cycles of coding enabled researchers to identify, develop and refine more detailed themes within the data and to classify them within the framework.⁴⁷² Discrepancies and disagreements were identified and addressed. The final classification was reviewed by researchers from all four trial sites based on their more detailed local knowledge of their children’s centres, IPB implementation and the 12-month interview data. When necessary, the categorisation was also verified against facilitation interviews at earlier time points and study activity logs. This was particularly important in cases in which there had been staff changes between RCT inception and completion. Adjustments were made in three cases following this process.

Incorporating findings from the trial into the development of a second injury prevention briefing

Following completion of the trial, we incorporated evidence from the trial into the development of a second IPB. This covered the prevention of fire-related injuries, falls, poisonings and scalds, based on findings from studies A and D–M in the KCS programme of research. We undertook four workshops, one in each trial site, with users of the fire prevention IPB in the trial and with potential future users of a future IPB to inform decisions about the content of the second IPB, the preferred structure and how to make the IPB more user-friendly.

Ethics and organisational review

Ethics approval was provided by Derbyshire Research Ethics Committee (reference number 11/EM/0011) and the University of the West of England Bristol Research Ethics Committee (reference number HSC/11/06/61). The trial received NHS organisational approval from PCTs when staff who worked in children’s centres were employed by PCTs.

Trial registration

This trial was registered as ClinicalTrials.gov NCT01452191 (13 October 2011) and ISRCTN65067450 (6 December 2012).

Developing the injury prevention briefing

Developing the facilitation package

The facilitation package consisted of a combination of face-to-face and telephone contacts at 1, 3, 4/5 and 8 months using structured electronic questionnaires and interviews. The questionnaires and interview schedules are provided in Appendix 6. They were intended to address the barriers to undertaking injury prevention identified in earlier work in the KCS programme (studies D–F) and the key issues that emerged from the workshops. In addition, they were also designed to raise the profile of fire prevention within children’s centres and ensure that it was kept on their agenda throughout the intervention period, allow assessment of progress with IPB implementation, identify difficulties and solutions to these, identify examples of good practice to share with other centres, provide information (e.g. a resources list, contact details for other agencies who could contribute to delivering fire prevention safety messages) and provide support to help implement the IPB.

The injury prevention briefing training sessions

A total of 31 children’s centre staff from the IPB+ arm attended training sessions at four locations. Their roles are presented in Table 106. Twenty-eight attenders completed the evaluation questionnaire wholly or in part. The responses to the questions are shown in Figures 58 and 59. Attenders were generally very positive about the training and stated that the training achieved its aims. One possible exception to this was that only 71% of attenders agreed or strongly agreed that they felt confident about presenting the key fire safety messages to parents at their children’s centre.

TABLE 106

Roles of children’s centre staff attending training sessions

FIGURE 58

Responses to positively worded statements about the IPB training in IPB+ arm training session attenders.

FIGURE 59

Responses to negatively worded statements about the IPB training in IPB+ arm training session attenders.

Structured interviews with parents to ascertain fire safety practices and fire escape behaviours

Twenty-one children’s centres were invited to participate and all agreed to participate. This included five centres each from Nottingham, Newcastle and Norwich and six from Bristol. Interviews were conducted with a total of 200 parents across the four centres, representing an 84% response rate. The characteristics of participants are shown in Table 107. Most respondents (92%) were mothers and described themselves as being white British (83%), 50% lived in rented accommodation and 19% lived in single parent households, with 45% of households having only one child.

TABLE 107

Characteristics of participants

Table 108 shows that smoke alarms were reported in the vast majority of homes (96%), of which virtually all (95% of those reporting a smoke alarm) were reported to be functional. Just over two-thirds (71%) of parents reported having functional smoke alarms on at least two levels of their home. Just over half of the respondents (54%) reported having a bedtime safety routine, but most described only one element of this routine (median 1, IQR 0–1, range 0–3). Eighty-one parents (42% of the 191 respondents answering that question) reported having a fire escape plan. The ICC for having a fire escape plan was 0.049 (95% CI 0.004 to 0.259). Only nine parents (11%) had practised their plan.

TABLE 108

Fire safety practices reported by participants

When asked to describe their fire escape plan, most respondents described one element (median 1, IQR 0–1, range 0–4). The elements described by families are shown in Table 109. The descriptions given were insufficiently detailed to allow for assessment of the comprehensiveness or adequacy of the plan.

TABLE 109

Elements of fire escape plans described by participants who reported having a plan

Findings from these interviews illustrated that it was not feasible to use functional smoke alarms as a trial outcome measure because of its high prevalence. The lower prevalence of having a fire escape plan would enable this to be used as an outcome measure but the open-ended question did not produce sufficiently detailed responses to enable understanding of what parents meant when they reported having a fire escape plan. It was therefore decided that separate closed questions about component elements of a fire escape plan would be used to assess the contents of fire escape plans. These questions covered five key components of fire escape plans that were included in the IPB, namely having a smoke alarm and knowing what it sounded like, having door keys accessible, having window keys accessible, keeping exits clear and having a torch next to the bed. To reduce type 1 error arising from multiple significance testing, it was decided that a composite measure would be developed describing behaviour across these five components for use as a secondary outcome measure. Latent variable analysis was undertaken for this purpose and the results of this are outlined in the following section. Similarly, an open question about bedtime safety routines produced a limited number and type of responses, with most parents describing only one element. Closed questions covering the elements of a bedtime safety routine used in the IPB were therefore included in the trial questionnaires.

Developing a composite fire escape behaviour variable

Data from the baseline trial questionnaire completed by 1112 parents were used to develop the composite fire escape behaviour variable. The frequency of reporting of each of the five component elements is shown in Table 110.

TABLE 110

Frequency of reporting of the five component elements of a fire escape plan

A two-class model provided the best fit to the data, categorising participants into ‘more fire escape behaviours’ (87% of participants) and ‘fewer fire escape behaviours’ (13% of participants). Table 111 presents the posterior probabilities for each component element for the two groups.

TABLE 111

Posterior probabilities derived from the categorical latent variable model

A typical member of the ‘more fire escape behaviours group’ had a torch, was aware of how their smoke alarm sounded, had door and window keys accessible for > 2–3 days per week and had exits clear for > 4–5 days per week. A typical member of the ‘fewer fire escape behaviours’ group did not have a torch, was not aware of how their smoke alarm sounded, had door and window keys accessible on ≤ 1 day per week and had exits clear on ≤ 1 day per week. The question about whether participants had a fire escape plan was used as an external validation criterion for the new composite measure. A multivariable logistic regression model estimated the association between parents reporting that they had a fire escape plan and the binary composite measure. After adjusting for potential confounders, participants allocated to the ‘more fire escape behaviours’ group had a 2.5 times higher odds of reporting having an escape plan (OR 2.48, 95% CI 1.59 to 3.86).

Trial results

Recruitment and retention

Thirty-eight children’s centres were recruited to the trial. This included two pairs of children’s centres that shared the same management team. Each pair was therefore treated as one children’s centre for trial purposes, giving a total of 36 children’s centres. A total of 1112 parents were recruited to the trial from the 36 children’s centres. Recruitment commenced in June 2011 and was completed in May 2012. The flow of children’s centres and parents through the trial is shown in Figures 60 and 61, respectively.

FIGURE 60

Recruitment of children’s centres and flow of children’s centres through the trial. a, When there were more children’s centres than were required we randomly sampled those to participate; b, sample size fulfilled in study centre (more...)

FIGURE 61

Recruitment of parents and flow of parents through the trial. CC, children’s centre. From Hindmarch et al. under the Creative Commons Attribution License 4.0 (see https://creativecommons.org/licenses/by/4.0/legalcode).

Outcome data were collected from all 36 children’s centres and from 751 (68%) parents. Follow-up rates did not differ significantly between treatment arms (IPB+ 65%, IPB only 68%, usual care 70%; OR for IPB+ vs. usual care 0.79, 95% CI 0.49 to 1.27; OR for IPB only vs. usual care 0.96, 95% CI 0.59 to 1.55). Follow-up rates did differ by baseline characteristics as shown in Table 112. Families with mothers aged 16–20 years (AOR 0.44, 95% CI 0.24 to 0.81 vs. families with older mothers), those in non-owner-occupied accommodation (AOR 0.48, 95% CI 0.35 to 0.66 vs. those in owner-occupied accommodation) and those living in more disadvantaged areas (AOR 0.47, 95% CI 0.29 to 0.78 comparing the most disadvantaged quintile of the IMD with the least disadvantaged quintile⁶⁵) were significantly less likely to be retained in the trial.

TABLE 112

Univariate and multivariable analysis of baseline factors associated with retention in the trial

Characteristics of participants

The characteristics of the children’s centres that participated in the trial are shown in Table 113, and the fire safety advice that they provided at baseline is shown in Table 114. The characteristics of parents at baseline are shown in Table 115 and the fire safety practices reported by parents at baseline are shown in Table 116. The characteristics of children’s centres and parents appeared well balanced between treatment arms. Most children’s centres (72%) were managed by the local authority and were phase 1 centres (89%), one-quarter (26%) were rated by Ofsted as outstanding for overall effectiveness and the median catchment population was 811 (IQR 574–998). Most children’s centres reported providing advice on each of the key messages from the IPB. More than 75% provided advice on fire escape planning, > 80% provided advice on smoke alarms and keeping cigarettes/matches/lighters out of reach and > 90% provided advice on other causes of house fires and bedtime safety routines.

TABLE 113

Characteristics of the children’s centres at baseline

TABLE 114

Fire safety advice provided by children’s centres at baseline

TABLE 115

Sociodemographic characteristics of participating families at baseline

TABLE 116

Fire safety practices reported by parents at baseline

One-fifth of parents (18%) lived in single adult households, 51% had only one child, 57% did not own their accommodation, 95% classed themselves as white British and 92% spoke English as their first language. The mean IMD score was 31.7 (SD 16.6).

Two-fifths (42%) of parents had a fire escape plan, of whom 89% had discussed the plan with other adults in the house and 28% had practised the plan. Four of the five component elements of the fire escape plan were reported by > 75% of parents, with only 32% reporting having a torch next to the bed. Most parents (87%) were in the ‘more fire escape behaviours’ group for the composite fire escape behaviours variable. Most parents (75%) reported that they had a smoke alarm fitted and working on every level of their home. Only 4% of parents had found their children playing with matches or lighters. Most parents undertook most bedtime safety practices on at least 4 days per week, with 41% of parents not closing all internal doors and 44% not turning electrical appliances off at the sockets on at least 4 days per week.

Implementation of the injury prevention briefing

Analysis of data from the implementation fidelity interviews suggested that there were four levels of IPB implementation, which were associated with different levels of delivery. A description of the criteria for each level of implementation is given in Table 117. Most children’s centres achieved extended implementation, followed by essential implementation. Only six (25%) children’s centres achieved minimal or non-implementation. It appeared that more IPB+ children’s centres than IPB-only children’s centres achieved extended levels of implementation.

TABLE 117

Classification criteria for levels of implementation of the IPB and numbers of children’s centres achieving each level by treatment arm

Quotations from the implementation fidelity interviews for children’s centres in each of the levels of implementation are provided in Boxes 5–8 to illustrate the varying degrees of implementation.

BOX 5

Examples of fire safety promotion at a children’s centre classified as achieving extended implementation (children’s centre B3; IPB+)

BOX 6

Examples of fire safety promotion at a children’s centre classified as achieving essential implementation (children’s centre D1; IPB+)

BOX 7

Examples of fire safety promotion at a children’s centre classified as achieving minimal implementation (children’s centre D2; IPB+)

BOX 8

Example of fire safety promotion at a children’s centre classified as achieving non-implementation (children’s centre C7; IPB only)

All (n = 12, 100%) of the IPB+ arm children’s centres and 58% (n = 7) of the IPB-only arm children’s centres developed a plan for implementing the IPB. Two-thirds (n = 7/11, 64%) of the IPB+ arm children’s centres had developed their plan by the 3-month facilitation contact. Figures 62 and 63 show the percentage of children’s centres that reported giving advice on each of the five key IPB messages and reported the use of the IPB exercises, respectively. The numbers were too small to compare these quantitatively. Fire-safety promotion activities reported by children’s centres on the follow-up questionnaire, by treatment arm, are shown in Table 118. From Table 118 and Figures 61 and 62 it appears that more of the IPB+ arm children’s centres than IPB-only arm children’s centres gave advice on each of the key safety messages and used each of the exercises. In addition, 92% (n = 11) of the IPB+ arm children’s centres and 50% (n = 6) of the IPB-only arm children’s centres reported using methods other than the IPB to deliver fire safety messages.

FIGURE 62

Provision of advice on each of the five key IPB messages by children’s centres in the IPB-only and IPB+ arms reported in the implementation fidelity interviews at follow-up.

FIGURE 63

Use of the IPB exercises by children’s centres in the IPB-only and IPB+ arms reported in the implementation fidelity interviews at follow-up.

TABLE 118

Fire safety promotion activities reported by children’s centres on the follow-up questionnaire by treatment arm

Table 119 shows parent-reported receipt of advice and other fire prevention promotion. These data are consistent with the findings above regarding implementation of the IPB and suggest that the IPB+ arm achieved significantly greater implementation of the IPB than the usual-care arm. The IPB-only arm achieved a lesser degree of implementation, with significant differences between the IPB-only arm and the usual-care arm only in terms of parents attending fire safety sessions. Significantly more parents in the IPB+ arm than in the usual-care arm reported receiving advice on each of the five key IPB messages, with ORs ranging from 2.21 (95% CI 1.18 to 4.12) (bedtime safety routines) to 3.35 (95% CI 1.98 to 5.68) (causes of fires). There were no significant differences in the proportion of parents who reported receiving advice for each of the five key IPB messages between the IPB-only arm and the usual-care arm. The proportion of parents who received advice ranged from 10% to 24% in the usual-care arm, from 12% to 34% in the IPB-only arm and from 21% to 43% in the IPB+ arm. In total, 28% of the IPB+ arm, 13% of the IPB-only arm and 14% of the usual-care arm parents received advice about fire escape planning. Significantly more IPB+ arm parents than usual-care arm parents received advice on more than two safety messages (30% vs. 12%; AOR 3.06, 95% CI 1.72 to 5.43), but there was no significant difference in this item between IPB-only and usual-care arm parents (15% vs. 12%; AOR 1.09, 95% CI 0.57 to 2.10).

TABLE 119

Reported receipt of fire safety advice and other fire safety promotion at follow-up by treatment arm

Significantly more families in the IPB+ and IPB-only arms than usual-care arm parents had attended one or more fire safety sessions in the last year (19%, 12% and 4%, respectively; AOR IPB+ vs. usual care 7.07, 95% CI 3.05 to 16.38; AOR IPB only vs. usual care 3.20, 95% CI 1.27 to 8.06) and significantly more IPB+ and IPB-only parents than usual-care arm parents had attended a fire safety session at a children’s centre (15%, 8% and 4%, respectively; AOR IPB+ vs. usual care 5.14, 95% CI 2.20 to 12.03; AOR IPB only vs. usual care 2.18, 95% CI 0.85 to 5.63). Significantly more parents in the IPB+ arm than usual-care arm parents had attended a fire safety session on each of the key IPB messages, with AORs ranging from 5.52 (95% CI 2.29 to 13.30) (session on causes of fires) to 9.88 (95% CI 3.31 to 29.43) (session on fire escape planning). Significantly more parents in the IPB-only arm than usual-care arm parents had attended a fire safety session on three of the key IPB messages [session on smoke alarms: AOR 3.34, 95% CI 1.30 to 8.58; session on fire escape planning: AOR 3.48, 95% CI 1.06 to 11.44; session on bedtime safety routines: AOR 3.93, 95% CI 1.04 to 14.93).

Primary and secondary outcome measures

Table 120 shows the primary and secondary outcome measures by treatment arm. There was no significant difference between treatment arms in the proportion of families who reported having a fire escape plan (AOR IPB only vs. usual care 0.93, 95% CI 0.58 to 1.49; AOR IPB+ vs. usual care 1.41, 95% CI 0.91 to 2.20) and this did not vary by family-level deprivation measured using the IMD 2010 score (p-value for interaction 0.86). Significantly more IPB-only families (AOR 2.56, 95% CI 1.38 to 4.76) and IPB+ families (AOR 1.78, 95% CI 1.01 to 3.15) were in the ‘more fire escape planning behaviours’ group than usual-care arm families. Parents in the IPB-only arm were significantly less likely to have found their children playing with matches or lighters than usual-care arm parents (AOR 0.27, 95% CI 0.08 to 0.94) and they also reported significantly more bedtime fire safety routines than usual-care arm parents (AOR for a 1-unit increase in the number of bedtime fire safety routines 1.59, 95% CI 1.09 to 2.31). There were no other significant differences in other secondary outcome measures. The ICC for the primary outcome measure was 0.00261 (95% CI 0.00000 to 0.02737).

TABLE 120

Primary and secondary outcome measures at follow-up, by treatment arm

Adjusted ORs for the primary outcome measure from the complete-case analysis (AOR IPB only vs. usual care 0.93, 95% CI 0.58 to 1.49; AOR IPB+ vs. usual care 1.41, 95% CI 0.91 to 2.20) differed from those in the analysis using multiply imputed data by only 1% for the IPB only arm compared with the usual-care arm (AOR 0.92, 95% CI 0.58 to 1.46) and by < 1% for the IPB+ arm compared with the usual-care arm (AOR 1.40, 95% CI 0.89 to 2.21).

Assuming that participants with missing data at follow-up had a baseline value for having a fire escape plan, there were similar results for the IPB-only arm compared with the usual-care arm (AOR 0.95, 95% CI 0.60 to 1.51; 2% difference in AORs between the complete-case analysis and the analysis assuming no change from baseline) and for the IPB+ arm compared with the usual-care arm (AOR 1.39, 95% CI 0.91 to 2.12; 1% difference in AORs between the complete-case analysis and the analysis assuming no change from baseline).

Cost-effectiveness

The cost of developing the IPB was estimated from researchers’ logbooks to be £15,860. Table 121 presents the unit costs applied to the resource use data that were obtained from the questionnaires and logbooks to obtain the overall costs. The cost of implementing the IPB with or without facilitation is presented in Table 122. The costs are reported as both a cost per children’s centre (i.e. cluster) and a cost per family randomised; the latter is used in the cost-effectiveness analysis. It can be observed that the costs associated with facilitation of the IPB varied across study centres (range £84.74–327.84).

TABLE 121

Unit costs (UK£, 2012)

TABLE 122

Costs of providing the IPB, training and facilitation (UK£, 2012)

Tables 123 and 124 present details about the fire safety activities, including home safety inspections, undertaken by the different agencies (i.e. children’s centres, fire and rescue service, parents, etc.). This information was combined with the unit costs presented in Table 121 to obtain the cost estimates incurred by the different agencies by treatment arm (Table 125). When these ‘other intervention costs’ were aggregated across agencies, costs in the usual-care arm were estimated to be highest.

TABLE 123

Summary of the fire safety activities at children’s centres

TABLE 124

Summary of fire safety activities attended by parents and home safety inspections

TABLE 125

Other intervention costs expressed per cluster (i.e. children’s centre) and per family

One children’s centre was identified as a potential outlier with maximum costs incurred by the centre estimated at £1800. However, when this centre was removed, costs in the usual-care arm still remained higher than in the IPB-only or IPB+ arms (mean £254.25 with a maximum per cluster of £792.00) because of the fire safety sessions run by children’s centres being, on average, longer in duration and more staff intensive Table 119. The distribution of costs by cluster within a trial site is presented separately in Figure 64 for each of the intervention arms.

FIGURE 64

Plot of total costs per family (2010 UK£) by cluster (i.e. children’s centre): (a) usual care; (b) IPB only; and (c) IPB+. Boxes represent the lower and upper quartiles around the median, lines represent the lower and higher extremes, (more...)

Table 126 presents the results of the cost-effectiveness analysis comparing the IPB-only and IPB+ arms with usual care. As stated in the methods section, the inverse of the difference in the probabilities of having a fire escape plan, say between IPB only and usual care, is equal to the NNT and therefore the cost-effectiveness ratio can be interpreted as the cost per additional fire escape plan under the intervention. The results of the analysis ignoring the effect of clustering, covariates and correlation between costs and effects show that IPB only is less costly and only marginally more effective than usual care, resulting in an ICER of –£1260 per additional fire escape plan, whereas IPB+ is more costly and only marginally more effective than usual care, resulting in an ICER of £616.13 per additional fire escape plan. It can be observed that, when allowing for the effect of clustering and correlation between costs and effects, the uncertainty is reduced.

TABLE 126

Cost-effectiveness analysis results for the complete-case data set

In a sensitivity analysis, the children’s centre with the potentially outlying cost (as noted above) was removed from the analysis (see Table 126). The resulting ICERs were –£53.01 for the IPB-only group compared with usual care and £3778.55 for the IPB+ group compared with usual care. Cost-effectiveness acceptability curves for the base-case analysis and the sensitivity analysis removing the outlying children’s centre are presented in Figure 65.

FIGURE 65

Cost-effectiveness acceptability curves: base-case (complete-case) analysis and complete-case analysis omitting the outlier.

Table 127 shows the extent of the missing data within the cost components, which ranged from just under 50% (parental costs) to nearly 60% (children’s centre costs). Because of the extent of the missing values, the results of the imputation analysis should be interpreted with caution.

TABLE 127

Cost component missing data description

Table 128 displays the results of the cost-effectiveness analysis based on the imputed data set. It can be observed that the average total costs per family and thus the differences between the arms are broadly similar to those in the primary (non-imputed) analysis reported in Table 126. For example, in the imputed data set the mean cost difference between the IPB-only arm and the usual-care arm is –£8.60, whereas in the complete-case data set it is –£8.49; similarly, in the imputed data set the mean cost difference between the IPB+ arm and the usual-care arm is £23.80, whereas in the complete-case data set it is £20.26. The probability of a fire escape plan was similar to that in the primary analysis for the usual-care arm, but it decreased from 0.49 to 0.44 for the IPB-only arm and increased from 0.48 to 0.58 for the IPB+ arm. These changes mean that the point estimate for the difference in effectiveness between the IPB-only arm and the usual-care arm is now fractionally negative, which makes the ICER positive. Note that these results should be interpreted with caution because of the large proportion of missing data imputed, ranging from just under 50% for parental costs to nearly 60% for children’s centre costs.

TABLE 128

Results of the cost-effectiveness analysis for the imputed data set

Incorporating findings from the trial into the development of a second injury prevention briefing

Four workshops were held with users of the fire prevention IPB and potential users of a future IPB. The numbers that were invited and who attended each workshop are shown in Table 129.

TABLE 129

Numbers of delegates attending IPB workshops by location

The key findings from the workshops were:

• activities need to be designed in ways that help parents think about their situations and possible solutions, rather than just telling them what to do
• activities need to be flexible and adaptable by users of the briefing to accommodate their opportunities and the capabilities of their client groups
• separate information for managers is not needed
• advice on how to obtain local data is needed
• information on the non-financial consequences of injury and, when known, the cost-effectiveness of interventions should be included
• a short section highlighting the needs of parents and children with disabilities would be helpful
• all injury topics should be covered in one document, with a front section on child development, anticipatory guidance, common risk factors, etc., followed by self-contained activities and, finally, detailed information on each type of injury.

These findings were incorporated into the design of the IPB covering the prevention of fire-related injuries, falls, poisonings and scalds. The IPB is shown in Appendix 6, Injury prevention briefing 2 and is available to download from www.nottingham.ac.uk/research/groups/injuryresearch/projects/kcs/index.aspx (accessed 1 November 2016).

Main findings

A complex intervention to change behaviours to improve fire safety in the home was developed using the MRC complex interventions framework.⁴⁶⁹ Data from a series of studies undertaken earlier in the KCS programme, interviews with stakeholders and workshops with practitioners were used to develop an IPB for the prevention of fire-related injuries and a training and facilitation package to support its implementation in children’s centres. The implementation of the IPB was tested using a RCT (study M) with an economic evaluation and nested qualitative study. A further IPB covering the prevention of fire-related injuries, scalds, falls and poisonings was subsequently developed using data from all component elements of the KCS programme.

The three-arm trial, which compared IPB+, IPB only and usual care, found that families in either intervention arm were not significantly more likely to report having a fire escape plan than usual-care arm families (AOR IPB vs. usual care 0.93, 95% CI 0.58 to 1.49; AOR IPB+ vs. usual care 1.41, 95% CI 0.91 to 2.20). However, families in both intervention arms reported significantly more behaviours that were component elements of fire escape planning (AOR IPB vs. usual care 2.56, 95% CI 1.38 to 4.76; AOR IPB+ vs. usual-care 1.78, 95% CI 1.01 to 3.15). Families in the IPB-only arm reported significantly more bedtime fire safety practices (AOR for a 1-unit increase in the number of bedtime fire safety routines 1.59, 95% CI 1.09 to 2.31) and were significantly less likely to report that their children had been found playing with matches or lighters (AOR 0.27, 95% CI 0.08 to 0.94) than families in the usual-care arm.

Families in the IPB+ arm were significantly more likely than usual-care arm families to report receiving advice on each of the five key safety messages in the IPB, whereas, although the proportion who reported receiving advice was higher in the IPB-only arm than in the usual-care arm for most messages, it was not significantly higher. The proportion receiving advice on the five key messages ranged from 21% in the IPB+ arm, 12% in the IPB-only arm and 10% in the usual-care arm for advice on bedtime safety routines to 43%, 34% and 24% for advice on smoke alarms, respectively. Families in both intervention arms were significantly more likely to report attending a fire safety session than usual-care arm families; most of these sessions were delivered at children’s centres.

All children’s centres in the IPB+ arm and 58% of children’s centres in the IPB-only arm developed a plan for implementing the IPB. More children’s centres in the IPB+ arm provided advice on each of the five key messages and ran sessions on each of the five key messages than IPB-only children’s centres, but the numbers were too small to allow for statistical analysis. Data from the qualitative study supported the finding that IPB+ children’s centres showed greater implementation of the IPB than IPB-only children’s centres and that children’s centres found the IPB and the facilitation package relevant and useful.

The economic analysis, which was conducted from a societal perspective including costs incurred by the family, children’s centre, fire and rescue service and other agencies that provided home safety inspections, showed that the IPB-only intervention dominates usual care as it is both less costly and marginally more effective, whereas the IPB+ intervention is more costly but only marginally more effective than usual care.

Strengths and limitations

The intervention that we developed was theoretically based and used the MRC complex intervention framework and evidence generated from numerous studies within the KCS programme. Our evaluation included a thorough assessment of the implementation of the intervention, used mixed methods and incorporated an economic evaluation. The treatment arms appeared well balanced at baseline, recruitment exceeded our required sample size, losses to follow-up were as estimated in our sample size calculation and the retention rate was similar across treatment arms. Findings from the multiply imputed data set were very similar to those in the complete-case analysis for the main analysis and the economic analysis.

Although we found evidence of implementation of the IPB, it is clear that the five key safety messages did not reach all families in either intervention arm (67%, 59% and 47% of families did not receive information on any of the messages in the usual-care arm, IPB-only arm and IPB+ arm, respectively). It is possible that greater implementation of the intervention may have achieved greater behavioural change. The qualitative study provided insight into possible explanations for the limited implementation. All children’s centres described major current, imminent or recent restructuring, which made it hard to deliver services and implement health promotion messages, including the IPB. Staff changes, budget constraints and staff capacity to take on additional tasks also limited implementation. Some centres found it difficult to prioritise fire safety over other health promotion topics because of a lack of local statistics to demonstrate local need. Centres had difficulty in delivering specific fire safety ‘sessions’, with poor parental attendance because of competing or more urgent issues or life changes for parents, frequent moves or children’s illnesses. Some centres found engaging parents in fire safety education difficult because of a perceived lack of relevance, perceptions that the information was already known, fear of being patronised or peer pressure. However, once parents were ‘through the door’, staff were frequently surprised by the depth of their engagement. Implementation was more effective when integrated into existing sessions. Trial procedures introduced additional demands for the children’s centre staff around data collection.

The finding that a significantly higher proportion of families in both intervention arms than in the usual care arm belonged to the ‘more fire escape behaviours’ group, without a significant difference in the proportions reporting having a fire escape plan, requires further exploration. The study questionnaire defined a fire escape plan as ‘. . .  a plan of what you would do to escape from the house if a fire broke out or the smoke alarm went off’. As a result of structured parent interviews earlier in the programme we added questions covering some of the elements of a fire escape plan, which the IPB provided advice on (knowing the sound of a smoke alarm, having a torch beside the bed, having door and window keys accessible, having clear exit routes). These were combined into a composite secondary outcome measure categorised as ‘more fire escape behaviours’ or ‘fewer fire escape behaviours’. Our trial findings suggest that it may not be useful to use a single-item question to measure possession of a fire escape plan. Further work will explore responses to open questions about actions that families would take if they could smell smoke and/or the smoke alarm was sounding and how these relate to the single-item question on having a fire escape plan. Our primary and most of our secondary outcome measures were self-reported by parents or children’s centre staff. Participants (parents or children’s centre staff) could not be blinded to treatment arm and hence there was the potential for outcome detection bias.

Our finding that providing the IPB without facilitation cost less than usual care appears counterintuitive. A potential explanation is that session plans provided in the IPB allowed children’s centres to deliver more focused sessions that required fewer staff. We found that the intervention arms provided sessions of a similar length using a similar number of staff for delivery, and that both intervention arms provided shorter sessions using fewer staff than the usual-care arm. It is acknowledged that the intervention cost component of the cost-effectiveness analysis is subject to the size of the target population (i.e. the number of children’s centres). Although we excluded IPB developmental costs from the cost-effectiveness analysis, we did include printing and distribution costs for the IPB and facilitator training costs. However, we would expect the latter costs to decrease as the target population increased because of economies of scale for printing and training, etc., which would potentially make the IPB+ intervention more cost-effective.

Our trial had three arms and multiple secondary outcome measures, leading to multiple significance testing and the potential for type 1 errors. Many of the effect estimates for the secondary outcome measures, especially at the level of children’s centres, were imprecise because of small numbers. We were unable to fit the imputation model without getting error messages if we included cluster as a categorical variable, and so clustering has not been taken into account in the imputation model, although cluster was accounted for in the analysis. There is some evidence that multiple imputation with a classical logistic regression (not accounting for clustering) can provide unbiased estimates of the intervention effect.⁴⁹⁴ In addition, our use of multiple imputation assumes that data are missing at random, which may not be the case, in particular for participants who did not respond at 12 months, as discussed in the following paragraph. In addition, there were a large number of missing cost data in the economic analysis and, although the results using multiple imputation were similar to those from the complete-case analysis, caution must be taken in interpreting these findings.

We used strategies to minimise losses to follow-up and retained sufficient participants to meet our sample size requirements, but there was some evidence that families retained in the trial were less disadvantaged than those lost to follow-up. Following up participants was challenging and resource intensive because of household moves, families no longer using the children’s centre and changes in mobile phone numbers. The ongoing national evaluation of children’s centres has found that most children’s centre services were used by families for < 1 year.⁴⁹⁵ Under these conditions, it is difficult to achieve high levels of penetration of the intervention, to deliver multiple safety messages or to reinforce the same message and to achieve high follow-up rates. Higher follow-up rates may have been achieved through the use of a repeated outcome measurement, but this has to be weighed against the burden that this places on participants, particularly those from disadvantaged communities. Despite differential loss to follow-up, those retained in our trial still represented a population experiencing substantial disadvantage.

All children’s centres were retained in the trial. Most (89%) children’s centres participating in the trial were phase 1 centres in the most disadvantaged areas, so our findings should be generalisable to other children’s centres in similar areas. Participants (children’s centres or parents) may differ from non-participants in terms of interest in fire prevention and this may limit generalisability. Attenders at children’s centres are likely to differ from non-attenders and few participants came from a black or ethnic minority group or had English as a second language. Our findings are therefore likely to be generalisable to the predominantly white British, English-speaking population of children’s centre attenders.

Comparisons with existing research

We were unable to find any published evaluations of injury prevention interventions delivered by children’s centres. Sure Start Local Programmes (SSLPs) were the forerunners to children’s centres, aiming to improve health and well-being of families and young children. They provided integrated early education, child care, health care and family support services in disadvantaged areas. The National Evaluation of Sure Start (NESS)²¹⁴ followed up > 9000 families and children in 150 SSLPs, comparing outcomes with those in families and children in the Millennium Cohort Study (MCS) living in similarly disadvantaged areas without SSLPs. At the age of 3 years, children in SSLP areas had a significantly lower accidental injury rate than those in the non-SSLP areas. SSLP area families used more child and family-related services than those in non-SSLP areas. The authors note that differences in injury rates may reflect temporal trends because of non-equivalent data collection periods for SSLP and MCS families. A further evaluation when children reached the age of 5 years included > 7000 randomly selected families (from the 9000 used in the 3-year evaluation) and found no significant difference in the injury rate between families in SSLP areas and MCS families in non-SSLP areas.⁴⁹⁶

Children’s centres are currently being evaluated nationally in a multicomponent 6-year study [Evaluation of Children’s Centres in England (ECCE)], which has yet to report its main findings. The first strand of the ECCE surveyed children’s centre leaders from approximately 500 centres to characterise children’s centres and the services that they provide.²¹² The proportion of children’s centres in the most deprived areas led by local authority staff was similar to that found in our trial (72% vs. 81%, respectively). Findings from the most deprived areas in the ECCE were similar to those from our trial in terms of lone parent families (19% vs. 18%, respectively), families with only one child (47% vs. 51%, respectively) and families living in rented accommodation (48% vs. 54%, respectively). The proportion of white British families in the ECCE was lower than in the trial (70% vs. 95%, respectively). Fewer parents in the ECCE reported having received home safety advice than at baseline in our trial (15% vs. 69%, respectively), but this may reflect differences in the questions asked in the two studies. The proportion of families reporting having a smoke alarm on every floor of their home was similar in the ECCE and in the trial at baseline (79% vs. 74%, respectively), but more ECCE families had tested their smoke alarms in the last 6 months than families in the trial (69% vs. 40%, respectively).⁴⁹⁵ Our trial population was therefore broadly similar to families using children’s centres in the most disadvantaged areas of the country with the exception of the trial population being more likely to be white British, more likely to have received home safety advice at baseline and possibly displaying fewer fire safety behaviours.

The ECCE survey²¹² also found that children’s centres were operating in a changing environment, with 40% experiencing recent cuts in services or staffing and many leaders managing two or more centres. This echoes the findings from our interviews with children’s centre staff, who frequently reported difficulties with implementing the IPB because of reorganisations, staff changes and loss of staff members. The second strand of the ECCE, a survey of 5700 parents,⁴⁹⁵ showed varied patterns of children’s centre use by parents. Some families were only limited users of services (19%), some used many children’s centre services and activities (38%) and some showed no clear pattern of service use (43%). Our finding that children’s centre staff reported low levels of parental attendance at fire safety sessions may reflect the ECCE findings that most parents are not frequent users of children’s centre services. In addition, the ECCE found that only 8% of parents had used home safety advice or courses provided by children’s centres. This may indicate a lack of prioritisation and provision of injury prevention by children’s centres [as found in our national surveys (study D) in work stream 3; see Chapter 4], coupled with less parental interest in home safety compared with other children’s centre services.⁴⁹⁵ Consistent with these findings, children’s centres in our trial reported difficulties in prioritising injury prevention and a lack of parental interest in the subject. We found in work stream 4 (study G; see Chapter 5) that some parents failed to anticipate injury-producing events or the injuries that they could result in. Previous research suggests that perceived susceptibility to injury is important for safety behaviour change.⁴⁷⁹ Although the activities in the IPB were aimed at raising parents’ perceptions of susceptibility, this will have required parents to perceive their families as sufficiently susceptible to have participated in fire safety activities provided by children’s centres. Future studies and injury prevention programmes may achieve greater changes in safety behaviour if they incorporate activities aimed at enhancing parents’ perceived susceptibility to injury prior to commencement of the study or programme, as well as during the programme.

The challenges of delivering evidence-based programmes within children’s centres were explored in questionnaires and interviews with staff in 121 children’s centres in the ECCE evaluation. The evaluation found widespread use of evidence-based programmes, particularly parenting programmes, but children’s centre staff ‘appeared to struggle with the concept of evidence-based practice. Some gave equal weight to research evidence and personal experience’ (p. 56).²¹¹ Tension was also reported between maintaining programme fidelity and offering potentially less demanding programmes for families.²¹¹ In addition, only a small number of families were reached by the best-evidenced programmes.²¹¹ These findings share some similarities with those from our trial. Fewer than 50% of families received each of the key safety messages in the IPB and < 20% attended fire safety sessions. Children’s centres reported that it was easier to incorporate fire safety messages into existing activities than run specific fire safety sessions. As in the ECCE, this may have resulted in a reduction in intervention fidelity, as activities are likely to have been adapted and shortened. Our implementation fidelity interviews showed that most children’s centres, particularly those receiving training and facilitation, undertook a range of activities to implement the IPB. This suggests that children’s centres can provide evidence-based injury prevention if provided with the resources and support to do so. However, greater behavioural changes may be achieved if intervention penetration could be increased and if intervention fidelity could be enhanced.