Included studies
shows the included studies by treatment comparison. Fifteen studies compared some form of ABA-based early intensive intervention against a comparator treatment (typically characterised as ‘eclectic’ or TAU).82–97,104–110 Three studies26,90,91,98,103,111 compared ABA-based early intensive interventions of different intensity, two studies99,100 compared clinic-based with parent-managed ABA-based early intensive interventions, and one study101 compared two different forms of ABA-based early intensive behavioural therapy. summarises the characteristics of the included studies. Twenty studies (reported in 33 publications26,82–101,103–112) published between 1987 and 2017 were identified. The majority of these were published in academic journals, although one was a Doctor of Philosophy (PhD) thesis112 and another was a conference abstract that met the review inclusion criteria but did not report any results.88 Full details of all studies are provided in Table 30 in Appendix 6.
Studies and comparisons. a, Included multiple arms, so included in two comparisons; and b, unpublished study.
Baseline characteristics of included studies
Children in the included studies were aged on average from 2483,106,107,112 to 6685,95 months at intake (median 37.4 months) and had a diagnosis of autism, ASD or (in some earlier studies) pervasive developmental disorder, as confirmed by the Diagnostic and Statistical Manual of Mental Disorders, Third or Fourth Edition,26,82,105,108 the ICD-10,84 the ADI-R,85,86,89,92,95,96,104,109,110 the ADOS93 or some combination of these.83,88,94,97,99,101,106,107,112 As would be expected, the majority of participants in the included studies (from 71% to 100%) were boys. Many included studies did not record the severity of baseline autism symptoms and those that did used a range of different measures (see Chapter 6, Individual participant data received from included studies). Across all included treatment arms, mean baseline VABS composite score ranged from 55.8 to 71.6 and mean baseline standard cognitive ability scores ranged from 51 to 83 (see ).
Studies recruited participants from the USA,26,83,98,100,103–108,111,112 the UK,87,89–92,96,99,109,110 Norway,84–86,88,95 Australia,82,93 Israel94,97 and Canada.101
One additional study102 comparing an ABA-based early intensive intervention against eclectic intervention was published during the writing of the current report, after the completion of the systematic review and meta-analyses. Although there was insufficient time available to integrate this study into the body of the report, a separate description of this study and meta-analyses incorporating its aggregate outcome data can be found in Appendix 12.
Applied behaviour analysis-based early intensive intervention versus treatment as usual and eclectic comparators
Fifteen studies (14 published26,82–87,89–101,103–112 and one unpublished88) compared some form of EIBI with a TAU or eclectic intervention.
Applied behaviour analysis-based early intensive interventions of different intensity
Three studies compared EIBI with a lower-intensity variation of the same approach, requiring fewer one-to-one contact hours between child and therapist.26,90,91,98,103,111 The original UCLA study by Lovaas26 compared 40 hours per week of EIBI against the same kind of treatment for < 10 hours per week (this study also included a retrospective cohort of children not receiving any ABA-based treatment, but insufficient data were available in either publications or IPD to inform the current meta-analysis).26,98 A later US study compared 30 hours per week of planned intensive EIBI (plus 5 hours/week of parental treatment) against parental training in EIBI techniques alone.103 One UK study compared 20–40 planned hours of ABA-based intervention with around 10–20 hours per week of the same approach.90
Clinic- versus parent-directed applied behaviour analysis-based early intensive interventions
Two studies compared clinic-directed EIBI against some form of parent-directed EIBI.99,100 In these studies, children in both treatment arms received similarly intensive intervention with therapists (30–40 hours/week), but the parent-directed groups either received less frequent supervision by senior therapists and clinical supervisors,100 or required parents to recruit and manage therapists.99
Different forms of applied behaviour analysis-based early intensive behavioural therapy
One Canadian study, including 28 children, compared two forms of ABA-based intervention in young autistic children.101 One treatment arm consisted of ‘Nova Scotia EIBI’, in which children received 15 hours per week of one-to-one instruction based on the PRT approach (a NDBI approach that targets ‘pivotal’ areas of a child’s development and emphasises natural reinforcement). The comparator arm was a group ABA preschool programme, based on the verbal behaviour method (a structured approach focused on teaching communication and language). Children in the group ABA group received 15–25 hours per week of training, of which 3–5 hours consisted of one-to-one discrete trial training.
Characteristics of early intensive applied behaviour analysis-based interventions
All included studies evaluated some form of early intensive ABA-based intervention. All such interventions were rooted in ABA and incorporated replications, extensions, adaptations or variations of teaching techniques originally described by Lovaas et al. at the UCLA during the 1970s and 1980s.26,98 Early studies closely resembled the original UCLA method, although without physical aversives.82,103 Subsequent studies have incorporated additional manualised ABA procedures into the original UCLA EIBI intervention model.85,86,90,91,94,95,97
Several studies incorporated some or all the aspects of NDBI approach into the EIBI model.83,84,88,92,93,99,100,104,106,107,110,112 This included approaches such as the ESDM.83,93,106,107,112
Children in the included studies received these early intensive ABA-based interventions for a period of 9–36 months, at a planned intensity of 15–40 hours per week of mostly one-to-one teaching (when recorded). Comparator treatments were delivered for a similar duration, although treatment intensity was more variable, ranging from 2 to ≥ 30 hours per week (when recorded), with considerably less one-to-one contact.
Characteristics of comparator interventions
As stated in Chapter 4, Individual participant data meta-analysis methods, comparator treatments could be broadly classified as ‘eclectic’ or TAU. Comparators were classified as ‘eclectic’ when individual children were known to have received a mix of teaching approaches, such as TEACCH, PECS, other behavioural or development programmes, speech and language therapy, music therapy or occupational therapy. Ten studies included eclectic comparator intervention arms, eight of which were delivered in a school or nursery classroom setting84–86,89–93,95,96,105,108,110 and two of which were delivered in a university or specialist centre setting.94,97 As well as an eclectic arm, one study included a portage treatment arm [a home-visiting educational service for preschool children with special educational needs (SEN) and disability].90,91
Other comparators were classified as TAU when children received non-autism-specific special education or other forms of standard local provision. Six studies included TAU arms, of which one was delivered in a school/nursery,105,108 three were delivered in a range of settings83,92,104,106,107,110,112 and two did not provide clear information about setting.82,88 Study investigators were not typically involved in the provision of TAU comparator treatments, so often did not have detailed information on the exact interventions received by individual children (see Study quality and risk of bias).
Study quality and risk of bias
The following section illustrates some of the core issues concerning risk of bias among studies included in the review, and primarily references studies comparing early intensive ABA-based with comparator interventions as examples. A full detailed assessment of the risk of bias for each meta-analytic comparison is presented in Appendix 9.
Three included studies were RCTs83,100,103,106,107,112 and were assessed using the Cochrane Risk of Bias 2.0 tool.57 Risk of bias in the remaining non-randomised studies was assessed using the ROBINS-I tool.58 All non-randomised studies were rated as being at ‘serious’ or ‘high’ risk of bias for at least one outcome on one domain on the relevant assessment tool. All three randomised studies83,100,103,106,107,112 were rated as having ‘some concerns’ about risk of bias.
Bias due to a range of confounding factors
All of the non-randomised studies were at serious risk of bias due to confounding. In five studies, the type of treatment received by children was explicitly based on parental preference,82,87,92,104,105,108,110 with parents actively seeking or lobbying for early intensive ABA-based treatment and, in some cases, paying for it themselves.87,92,110
In other studies, the type of treatment received was primarily based on location84,86,90,91 or staff availability,26,85,95,98 for which the influence of parental preference was unclear.
In some studies, baseline differences in parental education, family composition or socioeconomic status were observed between treatment groups.89,92,96,104,105,108–110
Bias due to deviation from intended interventions
Some studies described the methods used to assess treatment fidelity in the early intensive ABA-based intervention arm. These included monitoring, observation and feedback to tutors,83,93,104–108,112 or obtaining congruent descriptions of the intervention from parents and supervisors.90,91 In other studies, early intensive ABA-based interventions were supervised, but without explicit monitoring for treatment fidelity. Studies noted difficulties, such as high tutor turnover resulting in the intervention being delivered for fewer hours per week than intended,92,110 high proportions of children not completing the intervention,104 unreliable recording of weekly hours of EIBI,84 and families changing between different EIBI organisations or supervisors and consultants during the study period.89,96,109
The delivery and content of comparator arms was not closely monitored in the available studies, although three studies84–86,95 did report a high proportion of children receiving ABA techniques as part of ‘eclectic’ therapy or TAU comparators.
Studies rarely recorded whether or not children received any co-interventions alongside those being evaluated. One study appeared to compare groups in terms of independently procured co-interventions, finding that children receiving EIBI received more dietary and other biological interventions, extracurricular educational interventions and alternative treatments, than children receiving TAU.89,96,109
Bias in measurement of outcomes
Truly independent and blinded measurement of outcomes were rarely achieved in the evaluation of early intensive ABA-based intervention studies. In some cases, the participants in the ABA-based intervention and the comparison intervention arm were assessed by treatment supervisors84 or study investigators,86 sometimes with an independent second evaluator. Although some studies described employing outcome assessors who were independent of direct intervention delivery,82,85,89,92,95,96,104,105,108–110 the assessments typically involved interaction with children and parents who were not blinded to intervention. Assessors who were blinded to allocation could potentially have been unblinded by the assessment location (if this differed between intervention arms or – when delivered in the family home – provided contextual information about likely treatment allocation). Consequently, all of the included studies were considered to be at moderate or serious risk of bias for this domain.
Summary
Although randomisation is clearly feasible, most studies used convenience samples, with the allocation to early intensive ABA-based interventions being based on location or parental preference. Although some attempts were made to avoid bias in the measurement of outcomes, the nature of the intervention can make true blinded assessment difficult to achieve. There is evidence from some studies to suggest differences between the two intervention groups in terms of socioeconomic status and use of co-interventions, but this information was not consistently recorded across studies. In some cases, outcome data were missing or available for only one treatment group. It is also important to note that despite requesting them, we did not receive any protocols for the included studies. The original Lovaas study,26 in particular, was at risk of several forms of bias, including the comparator groups differing on the few available baseline variables.26,98 Taken together, these concerns increase our uncertainty about the results observed in several included studies, making it possible that the effects observed in the meta-analysis may overestimate the true effects of early intensive ABA-based interventions.
Study variability and relevance to the current UK context
The data in the included studies were collected over a period of > 40 years (from around 1968 to 2011), during which time the understanding, diagnosis and management of autism has evolved significantly. Consequently, there is noticeable variation between individual studies in terms of the delivery of interventions and comparators, the conceptualisation of autism and the outcomes of interest. As well as differences between the studies, there may be important differences between this body of evidence and the context in which early intensive ABA-based interventions and other treatment alternatives may be delivered in the UK in the future.
Interventions
Content, delivery and expertise
The original EIBI study by Lovaas at UCLA26,98 mainly employed discrete trial training, with some generalisation activities and community outings. Contingent physical aversives (the delivery of a loud ‘no’ or slap on the thigh) were employed as a last resort. As noted in Chapter 9, later variations in early intensive ABA-based interventions have incorporated more naturalistic components in their delivery and discontinued the aversive contingencies used in the original UCLA EIBI approach. Corporal punishment has been illegal in the UK for > 30 years and physical aversives have long been abandoned as part of ABA-based intervention delivery. Although all interventions included in this review had a theoretical basis in ABA, their content and delivery has evolved over time.
The UCLA EIBI intervention was delivered by trained student therapists, overseen by the study authors who were based at a specialised university centre, ensuring a high level of resource and expertise being made available to children and their families.26 Smith et al.103 reported 10 years’ experience working at the UCLA Young Autism Project. Similar expertise was available in the randomised study of the ESDM, which also involved the authors who developed the intervention method.83,106,107,112
The original narrow definition of what once constituted ‘true’ EIBI may not resemble very closely what is delivered in the current UK context, in which the intervention is more likely to be informed by more naturalistic approaches and delivered in a home or community setting, without the resources of an expert university research centre to train and supervise treatment staff.
Intensity and duration
The original UCLA EIBI was highly intensive. In addition to children receiving an average of 40 hours per week of one-to-one contact with therapists in their home, school and community for at least 2 years, parents were asked to take a year off from their current employment103 and were trained so that ‘treatment could take place for almost all of the subjects’ waking hours, 365 days a year’.26 Although actual treatment intensity and duration data were not recorded for individuals, it would appear that children had around 4 years of treatment on average (based on reported age at recruitment and follow-up), with children who had not ‘recovered’ continuing to receive > 40 hours per week of one-to-one teaching with therapists for > 6 years.26
As stated in Included studies, subsequent studies of early intensive ABA-based interventions were more varied in intensity and duration. Interventions rarely exceeded Lovaas’ minimum requirement of 40 weekly hours of one-to-one teaching.26 For example, authors who had been involved in the UCLA programme have delivered forms of EIBI that are deliberately less intensive than originally proposed (30 hours/week rather than 40 hours/week, with treatment phased out after 18 months if progress was slow).103 Other studies have shown intervention hours to substantially reduce after the first 2 years of treatment83,106,107,112 or when children started school.85,95 Although IPD on actual intensity were largely unavailable, the average weekly hours of teaching reported in UK studies ranged from 25.692,110 to 37.499 hours per week, and ranged from 13.6 to 38.6 hours per week in other studies.
As well as a possible impact on effectiveness, the duration and intensity of ABA-based treatment has implications for resource use (e.g. staffing costs) and possibly setting. Children in the UK typically start school at 5 years of age, whereas much of the evidence included in the current meta-analysis is from countries in which the primary or elementary school starting age is 6 years (USA, Norway, Canada, Israel). So in other countries, EIBI has been delivered in home and/or nursery or preschool settings, but in the UK it may be given in a primary education environment.
Another issue is that of the intervention supervision and management model. Several EIBI studies described the intervention being delivered by tutors who received ongoing training and feedback from supervisors, who in turn were overseen by a consultant or clinic director.26,103 However, this form of management is not always applied in the delivery of early intensive ABA-based interventions in UK practice.
Comparators
Comparator interventions for autistic children have also evolved over time, with the emergence of autism-specific rather than generic special needs care. More recently, ‘eclectic’ comparators have explicitly incorporated some ABA techniques.83,84,86,89,93,94,96,97,105,108,109 In most studies, children in the eclectic or TAU comparator arms received fewer hours of intervention and/or less one-to-one contact. However, this was not always the case: Howard et al.105,108 compared EIBI against an eclectic autism-specific classroom, with ≈30 hours per week of one-to-one or one-to-two intensive intervention; and Zachor et al.94,97 reported both intervention groups receiving similar levels of funding per child, hours in preschool setting, support for parents and staff, and individual one-to-one treatments.
Although we did not restrict inclusion by comparator, all of the identified comparators were eclectic intervention or TAU; no studies compared early intensive ABA-based interventions with discrete medical or educational interventions.
It is difficult to map the comparators in the available evidence to the current UK standard provision for two reasons. First, most studies had very limited available information on the content of eclectic interventions or TAU, as the study investigators were rarely involved in their delivery. Second, there is evidence that standard provision in the UK differs substantially between local authorities, although information obtained from York local authority suggests a mix of therapies not dissimilar to those cited in the studies (Ruth Horner, City of York Council, York, 2018, personal communication).
Participants
The studies included in this review cover a period when a large increase in the annual incidence of autism has been observed (more than fivefold from 1988 to 1995).113,114 Part of that increase has been attributed to changing and broadening diagnostic criteria,115,116 as well as increased medical and public awareness.117 This raises concerns about whether or not children receiving early intervention ABA-based interventions in included studies are similar to those currently eligible for intervention in the UK.
The overall population of children for whom IPD were provided were young (mean age 38 months) with mild-to-moderate intellectual disability (mean IQ 57) (see Chapter 6, Individual participant data received from included studies). As all children had to have an established diagnosis of autism or related condition to participate in the included studies, the study populations ought to be comparable to those who would be eligible for treatment in the current UK context. However, some of the more highly controlled studies excluded children with comorbidities, so it is plausible that typical UK treatment populations are more heterogeneous than those in the available evidence.
Outcomes
Although we accepted any outcomes for the IPD meta-analysis, only a small number of outcome domains were consistently collected across the included studies (most commonly, verbal and non-verbal IQ, adaptive behaviour and language measures; less commonly, autism symptom severity, behaviours that challenge and school placement).
The original Lovaas study26,98 was almost entirely focused on IQ and mainstream schooling placement as measures of treatment success. Children who achieved IQ in the average range were considered ‘recovered’ and the authors made ‘considerable effort’ to keep these children in mainstream preschool. In some cases, this involved withholding the child's diagnosis of autism. The authors stated, ‘If the child became known as autistic (or as “a very difficult child”) during the first year in pre school, the child was encouraged to enrol in another, unfamiliar school (to start fresh)’.26 Apart from any ethics and bias concerns it may raise, this excerpt shows how different the goals of early autism interventions were 30–40 years ago.
Subsequent studies incorporated behavioural measures, such as adaptive behaviour, while retaining IQ or cognitive development measures. Schooling as an outcome was only collected in a minority of studies and did not use consistent classifications.26,89,92,96,98,103,104,109,110
Outcomes relating to social participation, well-being and quality of life were not measured among the included studies. Although measures of cognitive development and adaptive behaviour can be used to track progress and development, no studies investigated how these measures correlate with measures of well-being, either during treatment or in the long term. In fact, any measurement of outcome beyond the end of the early intervention treatment period was rare.
With the exception of one study stating that ‘no serious adverse effects related to the intervention were reported during the 2-year period’,83,106,107,112 adverse or unintended effects of intervention were not addressed in the available evidence, with no study providing IPD on adverse effects.
The selection of measurement tools used for the collected outcomes varied within and between studies. Different measures of IQ and cognitive development were used at baseline and follow-up, based on the relevant normative populations [e.g. the WPPSI-R67 or BSID scales at baseline and Wechsler Intelligence Scale for Children (WISC)65 at follow-up]. At baseline, in particular, decisions about which measure to use are also informed by children’s developmental skills and ability to meaningfully be assessed in particular tests. The IPD meta-analyses separate measures of non-verbal skills (such as the MPSMT)70 from standard intelligence tests, which include verbal and non-verbal scales, as the former are known to yield higher scores.100
Several studies reported difficulties when using standardised measures in the evaluation of interventions in young autistic children, including floor and ceiling effects on different tests at different ages. Authors dealt with these difficulties by recording minimum or maximum scores,103 reporting age-equivalent scores,101 raw scores89,92,96,109,110 or the number of children capable of achieving a score.92,110 Although the provision of IPD facilitated the harmonisation and synthesis of scores across some of these studies, this was not always possible or appropriate.
Most studies provided standard scores when these were available and these scores informed the IPD meta-analyses. However, although standard scores allow comparisons with typically developing populations, it has been argued that they may miss information about changes that are relevant within the autism population specifically.118 This problem is not limited to early intensive ABA-based interventions, but is one of several barriers to valid and reliable outcome measurement in young autistic children in general.119
