Moderate Effects of Low-Intensity Behavioral Intervention

Abstract

We compared clinical outcomes in a treatment group of 98 individuals who received between 8 and 15 weekly hours (M = 10.6; SD = 1.7) of applied behavior analysis (ABA) intervention with a comparison group of 73 individuals who received another provision, including some ABA, (between 1.4-8 weekly hours, M = 5.7; SD = 1.6). After 2 years, the treatment group made greater gains than the comparison group on language and social skills, and other areas assessed by the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP). We evaluated the outcome on adaptive skills for a smaller sample of participants using the Vineland Adaptive Behavior Scales II (VABS), but found no significant differences between the treatment (n = 17) and comparison groups (n = 11). Although the treatment group made important and clinically meaningful gains, the gains were moderate. These findings underline the importance of intervention intensity and provide further support for a dose–response relationship between ABA intervention hours and outcomes.

Keywords

low-intensity behavioral intervention applied behavior analysis outcomes autism

Autism spectrum disorder (ASD) is a pervasive developmental disorder that is characterized by deficits in social and communication skills, as well as restricted and repetitive behaviors (American Psychiatric Association, 2013). According to the Centers for Disease Control and Prevention’s Autism and Developmental Disabilities Monitoring (ADDM) Network, one in every 59 children is diagnosed with autism (Baio et al., 2018). The cost of supporting an individual with ASD throughout the lifespan in the United States ranges between $1.4 and $2.4 million (Buescher, Cidav, Knapp, & Mandell, 2014). Currently, there is limited evidence to support pharmaceutical or other medically based treatments as the primary intervention for ASD (McPheeters et al., 2011; Williamson et al., 2017). However, there is evidence that early intervention based on applied behavior analysis (ABA) leads to clinically meaningful gains in adaptive skills, intellectual functioning, and a reduction in autism severity (e.g., Cohen, Amerine-Dickens, & Smith, 2006; Eikeseth, Klintwall, Jahr, & Karlsson, 2012; Howard, Stanislaw, Green, Sparkman, & Cohen, 2014; Sallows & Graupner, 2005).

ABA is the scientific discipline concerned with the application of fundamental principles of learning to effectively change socially significant observable behaviors (Baer, Wolf, & Risley, 1968; Cooper, Heron, & Heward, 2007). ABA interventions can range from low-intensity interventions that focus on a particular skill domain for a few hours per week to comprehensive intensive interventions that focus on many skill domains for many hours per week (Behavior Analyst Certification Board, 2014). In a review of five meta-analyses, Reichow (2012) found that Early Intensive Behavioral Intervention (EIBI) has the most empirical support and should be given strong consideration when choosing a treatment. The term intensive was initially used by Lovaas (1987) to describe an ABA intervention consisting of 40 weekly intervention hours for at least 2 years. Although there is no consensus on how many therapy hours per week constitute intensive interventions, the majority of intensive ABA intervention outcomes studies have examined over 15 weekly hours of intervention (e.g., Smith, Klorman, & Mruzek, 2015). We will refer to interventions that are based on less than 15 weekly hours as low-intensity interventions.

Several studies have evaluated clinical outcomes for low-intensity interventions, below 15 weekly hours (Eldevik, Eikeseth, Jahr, & Smith, 2006; Eldevik, Hastings, Jahr, & Hughes, 2012; Kamio, Haraguchi, Miyake, & Hiraiwa, 2015; Landa & Kalb, 2012; Lovaas, 1987; Nair et al., 2014; Peters-Scheffer, Didden, Mulders, & Korzilius, 2010). Three of these studies utilized a pre–post design without a comparison group or the intervention was not described in enough detail to verify their procedures were in line with principles of ABA (e.g., Kamio et al., 2015; Landa & Kalb, 2012; Nair et al., 2014). Three of the remaining four studies compared between 6.5 and 13.6 weekly ABA intervention hours with a treatment as usual comparison group (Eldevik et al., 2006; Eldevik et al., 2012; Peters-Scheffer et al., 2010), the fourth study compared less than 10 weekly hours (no additional details provided) with a no treatment control group and a group of children that received 40 hr of intervention per week (Lovaas, 1987). Although most of these studies report clinically meaningful gains following low-intensity behavioral interventions, the gains were more moderate than studies reporting outcomes of intensive behavioral interventions.

Low-intensity ABA interventions may be provided for a host of reasons including cost limitations, scheduling difficulties, availability of competent professionals, and limited family availability because of other treatments or school. Johnson and Hastings (2002) reported survey results for a group of U.K. parents who had at least one child receiving ABA services. The respondents identified recruiting staff and team sustainability as the biggest barriers for intensive ABA services. In addition, the annual costs for intensive ABA interventions range from $40,000 to $60,000 per person (Amendah, Grosse, Peacock, & Mandell, 2011), which may cause some families to opt for low-intensity ABA interventions.

The Lovaas (1987) study was the first experimental study that evaluated the outcome of both low- and high-intensity ABA interventions. The participants were younger than 46 months at intake, and they were assigned to either a high- or low-intensity group based on therapist availability and location. The intensive group consisted of 19 children who received 40-hr of ABA intervention each week and the low-intensity group consisted of 19 children who received 10-hr or less each week. The third group was a control group of 21 children recruited from a nearby hospital, who did not receive any specific intervention. The groups were similar with respect to prorated mental age and across eight other measures taken at intake (e.g., chronological age, measures of pathology, etc.). After 2 years, the high-intensity group had statistically significantly higher IQ scores and a higher educational placement than the low-intensity and no-treatment control groups. However, the low-intensity ABA group did not differ from the no-treatment control group and both had similar IQ and school placement scores after 2 years of intervention.

Peters-Scheffer et al. (2010) utilized a non-randomized experimental design to evaluate outcomes for a low-intensity ABA intervention in individuals younger than 7 years at intake. Twelve participants who received on average 6.3-hr of ABA intervention each week in a school setting were compared with 22 participants who did not receive any ABA following 8 months of intervention. All participants also received an average of 28.4-hr per week of eclectic classroom-based interventions throughout the study. At the start of the study, the two groups did not differ on developmental age. By the end of the study, however, the low-intensity ABA intervention group had made greater gains on all domain scores (communication, socialization and daily living) on the Vineland Adaptive Behavior Scale I (VABS I; Sparrow, Balla, & Cicchetti, 1984) as well as on the adaptive behavior composite score.

Two other studies have evaluated low-intensity ABA intervention outcomes (Eldevik et al., 2006; Eldevik et al., 2012). Eldevik et al. (2012) compared low-intensity ABA intervention with a relatively less intensive eclectic intervention comparison and Eldevik et al. (2006) compared low-intensity ABA intervention with an eclectic treatment of the same intensity. Eldevik et al. (2012) evaluated 2-year outcomes for a group of 43 participants between the ages of 2 and 6 years. Thirty-one participants received 13.6 weekly ABA intervention hours and 12 participants received at least five weekly hours of an eclectic intervention. They provided the intervention in a preschool setting and by trained school staff. After 2 years of intervention, the low-intensity ABA intervention group made greater gains than the relatively less intensive eclectic group on measures of intelligence, VABS I or II (Sparrow, Cichhetti, & Balla, 2005) adaptive composite scores, and the communication and socialization domains, but not on the daily living skills domain. In contrast, Eldevik et al. (2006) compared low-intensity ABA intervention with an eclectic intervention of equal intensity and demonstrated greater gains for the ABA intervention group on intelligence but only in the communication domain of the VABS I. These results provide some additional evidence that, even at a low-intensity, ABA interventions can be beneficial and more effective than other treatment options. A limitation with the Eldevik et al. (2012) study is that they used a retrospective nonexperimental design with a small sample of participants and there were some discrepancies in the VABS I or II results across these studies.

It is possible that the VABS I or II and other global measures may not be sensitive enough to consistently detect moderate intervention gains. Some researchers have suggested that moderate gains may be detected with more focused and fine-grained measures (Smith, Klevstrand, & Lovaas, 1995). In addition, there is a need for additional replications of the findings that low-intensity ABA intervention can result in gains in adaptive behavior, social skills, and language skills. Given that low-intensity interventions are now commonly delivered, particularly for individuals who have school or other obligations, it is important to further evaluate the effects of low-intensity ABA intervention across different demographics. The purpose of the present study was to evaluate the effects of low-intensity ABA interventions by replicating prior studies that used a global outcome measure, namely the VABS II, and by using a more fine-grained outcome measure that is based on direct observations of behavior, the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP; Sundberg, 2008).

Method

Participants and Design

Participants

We obtained a list of approximately 700 individuals who had started ABA services between June 2012 and June 2014 at the organization where the first and third authors were affiliated. We then filtered the list to only include those who had received 2 years of ABA interventions, which reduced the list to 274 individuals. Subsequently, we included the individuals who met the following criteria: (a) an independent ASD diagnosis prior to the referral; (b) a VB-MAPP assessment conducted at intake and at 24 (±1) months (this excluded 35 participants); (3) less than an average of 15 weekly intervention hours across the 2 years (this excluded 14 additional participants); and in an attempt to avoid ceiling effects (4) mastery of up to all Level 1 and Level 2 but not Level 3 VB-MAPP milestones, which is equivalent to mastery of less than 62% of all the VB-MAPP milestones at intake (this excluded 54 additional participants). A total of 171 individuals met the selection criteria (see Figure 1). The average participant age was 5 years 10 months (SD = 4 years 4 months) and ranged from 1 year 7 months to 30 years 4 months at intake. All but five participants were younger than 18 years at intake. See Table 1 for additional information on the participant characteristics.

Figure 1.

Flowchart of inclusionary criteria and the number of participants excluded during each phase of the participant selection process.

Table 1.

Participant Demographic Information and Other Characteristics (n = 171).

	Comparison group	Treatment group
	% (n)	% (n)
Boys	74 (54)	86.7 (85)
Girls	26 (19)	13.3 (13)
Diagnosis
ASD	98.6 (72)	100 (98)
PDD-NOS	1.4 (1)	0 (0)
Other services received
Speech only	17.8 (13)	12.2 (12)
OT only	4.1 (3)	4.1 (4)
Both speech and OT	6.8 (5)	9.2 (9)
Two parent household	89 (65)	95.9 (94)
Other caretakers present in home	21 (15)	24.5 (24)
Primary language spoken at home
English	57.5 (42)	67.3 (66)
Spanish	10.1 (8)	1 (1)
Other	2.7 (2)	4.1 (4)
English & Spanish	16.4 (12)	13.3 (13)
English & other (non-Spanish) language	11 (8)	13.3 (13)
Non-English bilingual	0 (0)	1 (1)
Unknown	1.4 (1)	0 (0)
Other siblings
No sibling	20.6 (15)	26.1 (29)
One sibling	53.4 (39)	47.7 (53)
Two siblings	12.3 (9)	20.7 (23)
Three siblings	9.6 (7)	1.8 (2)
Four or more siblings	2.7 (2)	2.7 (3)
Not reported	1.4 (1)	0.9 (1)
Ethnicity
Caucasian	9.6 (7)	13.3 (13)
Hispanic/Latino	27.4 (20)	25.5 (25)
Asian	5.5 (4)	4.1 (4)
African America	2.7 (2)	4.1 (4)
Native American	0 (0)	1.0 (1)
Pacific Islander	1.4 (1)	6.1 (6)
Middle Eastern	0 (0)	5.1 (5)
Multiethnic	5.5 (4)	7.1 (7)
Other	4.1 (3)	1.0 (1)
Unknown	43.8 (32)	32.7 (32)

Note. ASD = autism spectrum disorder; PDD-NOS = pervasive developmental disorder not otherwise specified; OT = occupational therapy.

Design

We used a retrospective group design. The independent variable was ABA intervention, which consisted of a combination of direct 1:1 ABA therapy and direct on-site supervision. The participants were divided into two groups; 98 participants in the treatment group received between 8 and 14.5 weekly ABA intervention hours (M = 10.6, SD = 1.7) and 73 participants in the comparison group received between 1.4 and 8 weekly ABA intervention hours (M = 5.7, SD = 1.6). The VB-MAPP served as the outcome measure for this sample. Eight hours was selected as the cutoff because it served as a rounded median value. We also compared outcome on the VABS II for a subsample of the participants who had VABS II data. We divided the participants in this subsample into two groups. Seventeen participants in the treatment group had VABS II data and received between 8.2 and 13.7 weekly intervention hours (M = 10.7, SD = 1.8) and 11 participants in the comparison group had VABS II data and received between 4.2 and 7.7 weekly intervention hours (M = 5.9, SD = 1.3).

Intervention

Intervention setting

All participants received the majority of their ABA intervention in their homes or the setting where they spent most of their free time (e.g., daycare, community, or settings where behavioral support was needed). The intervention team provided individualized behavioral interventions designed to meet the specific needs of each participant as determined best fit by the Board Certified Behavior Analyst® (BCBA®) who was overseeing the case. The teaching methods consisted of both discrete trial training (DTT; Smith, 2001) and naturalistic teaching approaches, such as incidental teaching (Hart & Risley, 1975), natural environment training (NET; Sundberg & Partington, 1998), and pivotal response training (PRT; Koegel & Frea, 1993; Koegel, Koegel, & Brookman, 2003).

Intervention team

A multitier service delivery model was used for the majority of cases. A BCBA® supervised master’s level Program Managers, who in turn supervised a team of ABA therapists who implemented the prescribed intervention directly with each client. A team of at least two trained behavior therapists served each participant and provided therapy during separate sessions. The ABA therapists initially received 5 days of classroom- and computer-based instructions from experienced trainers. At least one BCBA®-level and another training team member conducted the classroom training sessions. The training consisted of modeling skills, role-playing, and providing feedback. After the 5-day initial training was completed, the therapist-in-training overlapped an experienced therapist when working with clients in the field for a minimum of 20 hr. The trainees received additional feedback in the field and were subsequently assigned to their individual cases after most of them reached the competency criterion of 80% or higher accuracy in implementing the target clinical skills with an adult trainer. Finally, the therapists received an additional 10 days of computer-based instructions after the overlaps, focusing on various behavioral principles and procedures. All ABA therapists had a bachelor’s degree in psychology or in a related field and most had some prior experience working with individuals diagnosed with ASD.

The Program Managers provided approximately 1 hr of direct on-site supervision during therapy sessions for approximately every 4 hr of direct 1:1 ABA therapy. The Program Managers had a master’s degree in behavior analysis, psychology, or in a related field. The BCBA®, in turn, supervised the Program Manager and provided ongoing support and attended at least one therapy session each month. The therapists provided the majority of ABA intervention hours, but in some cases the Program Manager or BCBA® provided direct therapy as well. The authors in this study were not directly involved in the design or implementation of the individual interventions. Most of the participants also attended school with some support for the remainder of the week. The support varied among the participants, depending on their needs and the resources available. Generally, it was a school-based intervention as usual or special education with autism specific programming.

Outcome Measures

Verbal Behavior Milestones Assessment and Placement Program

The VB-MAPP is an assessment tool based on Skinner’s (1957) analysis of verbal behavior. The VB-MAPP takes a comprehensive functional and topographical approach to assessing language and other skills (e.g., social skills, play skills, math skills) that occur up to 4 years of age in typically developing children. The VB-MAPP consists of 16 domains that are broken down into 5 to 15 milestones each. Each milestone assesses specific observable skills. The milestones are sequenced into three developmental age levels of 0 to 18 months (Level 1), 18 to 30 months (Level 2), and 30 to 48 months (Level 3). The VB-MAPP milestone scoring involves summing up the total number of milestones mastered, which yields a minimum value of zero and a maximum of 170. For the purposes of this study, the VB-MAPP measure of interest was the VB-MAPP percent score. This measure was obtained by dividing the VB-MAPP milestones score by 170 and multiplying by 100. Given that there is a maximum on the number of skills that the VB-MAPP assesses, we excluded clients who had an intake VB-MAPP percent score of 62% or higher. This cutoff was selected in an attempt to avoid a ceiling effect. It corresponds with someone who has up to all Level 1 and 2 milestones in their skill repertoire but none of the Level 3 milestone skills.

The VB-MAPP has been used in studies to report participant characteristics or as a dependent variable (e.g., Kisamore, Karsten, & Mann, 2016; Mason & Andrews, 2014). The scores on the VB-MAPP have also been shown to correlate with other behavioral measures (Dixon et al., 2015). In addition, preliminary data indicate good to excellent VB-MAPP milestones scoring reliability (Montallana, 2018). The VB-MAPP has received considerable attention and promise as a clinical assessment tool and a curriculum for guiding comprehensive ABA interventions (Gould, Dixon, Najdowski, Smith, & Tarbox, 2011). The VB-MAPP assesses a number of specific observable behaviors through direct observations by trained assessors rather than parent interviews and emphasizes language assessment. As a result, it may provide different information that may be helpful for understanding intervention outcomes.

Vineland Adaptive Behavior Scale II

The VABS (I and II) is one of the most widely used and validated assessment tools in ASD research (Stolte, Hodgetts, & Smith, 2016). We utilized the VABS II, which contains four domains (communication, socialization, daily living skills, and motor skills) with standard scores and an adaptive behavior composite score. Each domain contains subdomains that provide age-equivalent scores. The communication domain consists of receptive, expressive, and written subdomains. The daily living skills domain consists of personal, domestic, and community subdomains. The socialization domain consists of interpersonal relationships, play and leisure time, and coping skills. The motor skills domain consists of fine and gross motor subdomains. In addition, an adaptive behavior composite score is obtained based on the domain scores. In the present study, we only reported the adaptive behavior composite score gains.

Assessors

The BCBA® or the Program Manager on each case conducted the assessments at intake. They conducted the VB-MAPP once every 6 months and the VABS II annually thereafter. In some cases, an independent contracted assessor conducted the VABS II assessment at intake, if a BCBA® and Program Manager had not already been assigned to the case, or before the referral. In the latter case, a subsequent VABS II was not conducted at intake, and the VABS II data from the original pre-referral assessment were utilized. The analysis consisted of data from the intake assessment and the assessment conducted after 24 (±1) months of treatment.

All qualified assessors were trained to administer the assessment tools during an initial 6-hr training using a slide presentation with guided notes. A trainer reviewed and discussed the guided notes with the assessors in training and again about 2 months later in a follow-up training. In some instances, if the trainee requested additional training or if the training team identified skill deficits, the trainee conducted a practice assessment on a confederate adult, the trainer provided immediate feedback, and the trainees rehearsed the skill until they conducted at least one of the VABS II domain and five of the VB-MAPP domains satisfactorily. The assessors conducted additional self-paced computer-based trainings annually and each lasted 45 to 75 min. Some assessors who were employed solely to conduct assessments also attended additional classroom-based trainings and in-field overlaps prior to conducting their assessments.

Data Analysis

We used independent samples t tests to compare the two groups at intake across the following measures: VB-MAPP percentage score, VABS II adaptive behavior composite score, communication standard score, daily living skills standard score, socialization standard scores, intake age, and average hours of ABA intervention received (Table 2). Since the individuals who had VABS II data consisted of only a small subsample of the total participants, we analyzed outcomes for the sample with VB-MAPP and VABS II data separately.

Table 2.

Comparison of Participant Characteristics at Intake Using Independent Samples t Test Results (α = 0.05).

	Entire sample with VB-MAPP score change								Subsample with VABS ABC score change
	Comparison group		Treatment group		Comparison of groups				Comparison group		Treatment group		Comparison of groups
	n	M (SD)	n	M (SD)	df	t	p	$η_{p}^{2}$	n	M (SD)	n	M (SD)	df	t	p	$η_{p}^{2}$
VB-MAPP Percent Score	73	25.62 (16.21)	98	20.87 (13.92)	169	2.055	.041	2.4	11	27.76 (14.67)	17	20.31 (13.12)	26	1.4	.173	7.0
VABS ABC	34	59.32 (11.54)	53	60.13 (13.89)	85	−0.282	.778	0.1	11	60.36 (12.66)	17	62.53 (9.87)	26	−0.508	.616	1.0
VABS Communication	70	60.93 (15.59)	96	63.2 (17.57)	164	−0.861	.39	0.5	11	55.82 (18.0)	17	63.47 (12.8)	26	−1.317	.199	6.3
VABS Socialization	70	61.54 (13.39)	96	63.38 (11.23)	164	−0.956	.34	0.6	11	61.0 (13.87)	17	62.59 (10.97)	26	−0.337	.739	0.4
VABS Daily Living Skills	70	66.54 (15.2)	96	67.26 (12.97)	164	−0.327	.744	0.1	11	64.55 (14.44)	17	65.29 (14.36)	26	−0.134	.894	0.1
Intake age	73	6.56 (4.9)	98	5.32 (3.81)	169	1.868	.064	2.0	11	9.38 (8.71)	17	5.48 (4.81)	13.99	1.359	.196	8.3
ABA hours/week	73	5.68 (1.61)	98	10.6 (1.7)	169	−19.14	<.001	68.4	11	5.90 (1.34)	17	10.74 (1.77)	26	−7.747	<.001	69.8

Note. VB-MAPP = Verbal Behavior Milestones Assessment and Placement Program; VABS = Vineland Adaptive Behavior Scales; ABC Score = Adaptive Behavior Composite Score.

Change on the VB-MAPP

The primary dependent variable was gains on the VB-MAPP. The groups were similar on all measured variables at intake except intake VB-MAPP scores. As expected, the groups received different intervention intensities. We conducted a one-way analysis of covariance (ANCOVA) to compare VB-MAPP gains (score change) for the treatment group (n = 98) with the comparison group (n = 73), controlling for intake VB-MAPP score.

Change on the VABS II

For the subsample with VABS II scores, the primary dependent variable was gains in the VABS II adaptive behavior composite score. The comparison (n = 11) and treatment (n = 17) groups did not differ from one another at intake across the measured areas but the groups received different intervention intensities. We conducted an independent samples t test to determine whether the treatment and comparison groups differed from one another on VABS II adaptive behavior composite score gains after 2 years of intervention.

Interobserver agreement (IOA)

A team of four research assistants reviewed the participants’ records on file and inputted the relevant data in a separate confidential datasheet. A second independent observer did the same for 36% of the coded participants and recorded the data on a separate confidential datasheet. Point-by-point IOA was calculated by dividing the number of corresponding data points that were scored the same across the two observers by the total number of data points scored, and the result was multiplied by 100. The IOA coefficient was 95%.

Results

When comparing the treatment and comparison groups at intake, the treatment group had a lower VB-MAPP score. As a result, an ANCOVA was conducted to control for intake VB-MAPP scores. The treatment group made greater gains (M = 39.7; SD = 16.4) than the comparison group (M = 31.1; SD = 16.9) (Figure 2). There was a statistically significant effect of intervention intensity on VB-MAPP score change after controlling for intake VB-MAPP score, F(1, 168) = 9.788, p = .002, $η_{p}^{2}$ = 5.5. The covariate, intake VB-MAPP score, was not significantly related to VB-MAPP gains, F(1, 168) = 1.175, p = .28, $η_{p}^{2}$ = 7.0.

Figure 2.

Mean VB-MAPP percent score change and mean VABS ABC score change after 2 years of ABA intervention for the treatment group and the control group.

We compared VABS II adaptive behavior composite score gains across the treatment groups using an independent-samples t test. The treatment group made greater gains (M = 7.0; SD = 10.5) than the comparison group (M = −1.45; SD = 11.8; Figure 2). This group difference was not statistically significant, t(26) = −1.988, p = .057, $η_{p}^{2}$ = 13.2, confidence interval (CI) 95% [−17.2, .29].

Visual inspection of score change for individual cases (Figures 3 and 4) indicates that more individuals in the treatment group made greater gains than the comparison group across both the VB-MAPP and VABS II. For example, 13 of the 17 (76%) participants in the treatment group made positive gains on the VABS II adaptive behavior composite score but only 5 of 11 (45%) in the comparison group made positive gains.

Figure 3.

Individual participants’ VB-MAPP percentage score change after 2 years of ABA intervention for the treatment group and control group.

Figure 4.

Individual participants’ VABS ABC score change after 2 years of ABA intervention for the treatment group and control group.

Discussion

We compared 2-year outcomes in a group that received low-intensity ABA intervention with a comparison group. Two analyses were conducted, one using a relatively larger sample with VB-MAPP data and one using a smaller subsample with VABS II adaptive behavior composite scores as the outcome measure. For the VB-MAPP sample, the treatment group made greater gains than the comparison group. For the subsample of participants who had VABS data, the treatment group made larger gains. There was a 9.6 point difference in percent gains across the groups on the VB-MAPP, which was statistically significant, and an 8.5 point difference on the VABS II, which was not statistically significant.

There are a number of possible explanations for why only one of the two measures resulted in a statistically significant difference in outcomes. The most likely one is sample size. We had 171 participants in the VB-MAPP analysis and 28 in the VABS II analysis. In the other studies that used the VABS I or II as an outcome measure, the study with the largest sample size (n = 43) reported a significant effect on all but one of the measured VABS I or II domains (Eldevik et al., 2012). However, the study with the smallest sample size (n = 28) reported a significant effect on only one of the VABS I or II domains, and not the adaptive behavior composite score (Eldevik et al., 2006). The results of the present study seem to be similar to what has been reported in the literature earlier.

The outcome measures that we used differed from one another in many regards and that could have also accounted for why only one of them resulted in a statistically significant difference. The VABS I or II is a commonly used assessment tool in clinical outcomes research (e.g., Stolte et al., 2016). However, the VB-MAPP has not been used in any longitudinal ABA intervention outcomes studies that we are aware of. In the clinical setting, the VB-MAPP has received considerable attention as a promising tool for designing comprehensive early intervention programs (Gould et al., 2011). Unlike the VABS I and II, the VB-MAPP has limited psychometric data, it extends to skills exhibited up to age 4 years only, it is not as comprehensive, and the scores are not equally weighted within or across domains.

The VB-MAPP has some advantages over the VABS I and II that may be particularly useful for detecting gains following treatment. First, the VB-MAPP milestones assessment emphasizes language skills, a primary focus area in ABA interventions. For example, 10 out of the 16 VB-MAPP domains are related to language development, whereas 1 out of 4 VABS II domains is specifically related to language (the communication domain). In addition, the VB-MAPP recommends direct observations during assessment rather than relying on caretaker information. As a result, the VB-MAPP provides different and more fine-grained information than the VABS I or II, which may be why it detected gains in this study. Conversely, because the VB-MAPP can be used as a curriculum guide, it is possible that it was used in such a manner and the participants learned the specific skills that were assessed on the VB-MAPP, but not the skills that were measured with the VABS, resulting in larger gains on the VB-MAPP. It should be noted that the different results that we obtained on the two assessment tools for the different samples was not due to differing intervention intensities being compared in each of the two samples. For the VB-MAPP and the VABS II sample, the treatment and comparison groups differed by 4.9 hr and by 4.8 hr, respectively.

Some participants in our study received speech and/or occupational therapy in addition to ABA interventions at some point, which resulted in eclectic interventions for them. Several studies have suggested that eclectic intervention produce fewer gains than ABA interventions of similar intensity (e.g., Eldevik et al., 2006; Howard et al., 2014), and if the time and money spent on other interventions was spent on ABA interventions instead, it could have resulted in larger gains. This may have been the case for those who received eclectic interventions in our study as well but we did not control for this variable to determine whether this was in fact the case. Conversely, it is also possible that some of the interventions comprising an eclectic approach could have resulted in greater gains under some circumstances and for certain individuals. For example, speech therapy may have been more beneficial for individuals with a higher VB-MAPP echoic domain score. Additional research is warranted to determine the parameters that result in greater gains when providing eclectic interventions.

The findings in this study are consistent with two prior low-intensity ABA intervention outcomes studies (Eldevik et al., 2012; Peters-Scheffer et al., 2010), but there are also some noteworthy inconsistencies in outcomes. In the other studies that compared low-intensity ABA interventions with relatively less intensive interventions using the VABS I or II, Eldevik et al. (2012) and Peters-Scheffer et al. (2010) found an effect on the adaptive behavior composite score but we did not. It is important to point out that the two groups in the Eldevik et al. study differed from one another in intervention intensity by approximately 8.6 hr, which is much greater than the 4.8 hr difference between the groups in our study. Our group difference was also less than the 6.3 hr difference in the Peters-Scheffer et al. study, which found an effect on all VABS II measures. It is possible that if the intervention dosage difference was greater between our groups, we would have obtained similar results.

Like previous studies, our findings lend support to the notion of a dose–response relationship between intervention intensity and outcomes (see Virués-Ortega, 2010), and for a wide age range. Many individuals from the population that we recruited our sample from received low-intensity ABA interventions. In fact, during the participant selection process, only 14 out of the 239 individuals (6%) who had received ABA interventions for 2 years had received intensive interventions consisting of more than 15 weekly hours the entire 2 years. As the present study suggested, receiving 10.6 as opposed to 5.7 weekly ABA intervention hours led to greater gains. One of the implications of this is that when there is limited resources, families should consider slightly more ABA intervention hours over no ABA interventions or over other interventions. As our results indicated, the relationship between ABA intervention intensity and outcomes may hold up for a wider age range, not just early start interventions. The average participant age in this study was 5 years 10 months, but there were 57 participants who were older than that at intake and five of them were older than 18 years. Unfortunately, the present study did not evaluate intervention outcomes based on participant age because we were not able to compare subgroups of different ages with large representative samples.

To better evaluate the global gains obtained with low-intensity ABA interventions, an important question to ask is how much the improvements associated with the interventions offset the costs of various health care services. For example, if the mild gains in adaptive skills result in better hygiene and preventive health behaviors, how much does that offset health care costs? Cidav et al. (2017) compared total intervention costs for a group that received EIBIs (the Early Start Denver Model) with a group that received community-based interventions and found that within a few years after the intervention, the costs of early intervention were offset due to a reduction in costs in many services. In the present study, we did not have intervention cost data or data on related health care costs, but future outcomes studies should conduct a cost-benefit analysis if possible.

There are notable limitations in the present study that highlight the need for clinical trials in ABA intervention outcomes research. The most notable limitation is that we did not utilize a randomized experimental design. Our findings should therefore be looked at with caution. It is possible that the two groups differed along traits that we did not measure or control for. For example, some of the participants in both groups received speech or occupational therapy but a higher percentage of individuals in the comparison group received speech therapy. We did not have sufficient data on the dosage of those interventions to control for those variables statistically. Another noteworthy limitation is that the VB-MAPP was the only assessment tool we had for the larger sample but the psychometric properties of the VB-MAPP are unknown. Before adopting any measurement tool for research or clinical purposes, it is necessary to examine its psychometric properties. Prior studies have indicated that clinicians can conduct the VB-MAPP with high fidelity if they receive sufficient training (Barnes, Mellor, & Rehfeldt, 2014), and presumably with high reliably. Preliminary data indicate that it has good to excellent milestones scoring reliability (Montallana, 2018), but to our knowledge there are no published peer-reviewed studies that have examined this. A further limitation in this study is that because we excluded participants with more advanced skills to prevent a ceiling effect, the results may be generalizable only to individuals who have a VB-MAPP percentage score of less than 62% at intake.

Future studies could directly compare VABS I or II and VB-MAPP gains by using VABS I or II raw scores across the same sample to evaluate whether or not both measures yield similar findings. In the present investigation, we did not have access to VABS II raw scores to perform this analysis. Future studies should also replicate the low-intensity ABA intervention outcomes studies using the VABS I or II but with a larger sample size.

In conclusion, our findings further support the notion of a dose–response relationship between intervention intensity and clinical outcomes (see Virués-Ortega, 2010), as the outcomes were more moderate than other studies that reported the outcomes of higher intensity ABA interventions. While as a field we should determine what minimum gains are considered socially beneficial and acceptable, in the present study we were able to detect moderate gains that were probably clinically meaningful as they represent important socialization and communication skills, such as the ability to express wants and needs. The VB-MAPP may have some promise for measuring clinical outcomes in research, but additional psychometric data are needed.

Footnotes

Acknowledgements

Special thanks to Sarah Long and Darin Hogen for assisting with data collection and assessment of interobserver agreement throughout the study. Also, thanks to Jose Sanchez, Shiesha Toomer, and Timiza Farley for assisting with data entry.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Amin D. Lotfizadeh

Author Biographies

Amin D. Lotfizadeh is the director of Research at Easterseals Southern California and a part-time lecturer at California State University, Los Angeles. His primary area of interest is the experimental analysis of behavior. His scholarly publications are in the areas of behavioral pharmacology, behavior analysis, organizational behavior management, and autism.

Ellie Kazemi is a professor at California State University, Northridge (CSUN) in the department of psychology, where she teaches undergraduate and graduate coursework in behavior analysis and research methodology. Her current research interests involve identification of efficient, effective strategies for practical training and supervision in human care services.

Paula Pompa-Craven, PsyD, is a licensed psychologist and the chief clinical officer at Easterseals Southern California. She specializes in the area of Autism Spectrum Disorder, assessments, diagnosis, and provision of services for individuals diagnosed with autism.

Sigmund Eldevik is an associate professor at Oslo Metropolitan University, Department of Behavioral Science. He is a psychologist from the University of Oslo, and has a PhD from the University of Bangor, Wales. His research interests are on early intensive behavioral interventions for children with autism and other developmental disabilities.

References

Amendah

Grosse

S. D.

Peacock

Mandell

D. S.

(2011). The economic costs of autism: A review. In Amaral

Geschwind

Dawson

(Eds.), Autism spectrum disorders (pp. 1347-1360). Oxford, UK: Oxford University Press.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Baer

D. M.

Wolf

M. M.

Risley

T. R.

(1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91-97.

Baio

Wiggins

Christensen

D. L.

Maenner

M. J.

Daniels

Warren

. . . Durkin

M. S.

(2018). Prevalence of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 Sites, United States, 2014. Morbidity and Mortality Weekly Report, 67(6), 1-23.

Barnes

C. S.

Mellor

J. R.

Rehfeldt

R. A.

(2014). Implementing the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP): Teaching assessment techniques. The Analysis of Verbal Behavior, 30, 36-47. doi:10.1007/s40616-013-0004-5

Behavior Analyst Certification Board. (2014). Applied behavior analysis treatment of autism spectrum disorder: Practice guidelines for healthcare funders and managers (2nd ed.). Retrieved from https://www.bacb.com/wp-content/uploads/2017/09/ABA_Guidelines_for_ASD.pdf

Buescher

A. V.

Cidav

Knapp

Mandell

D. S.

(2014). Costs of autism spectrum disorders in the United Kingdom and the United States. Journal of the American Medical Association Pediatrics, 168, 721-728. doi:10.1001/jamapediatrics.2014.210

Cidav

Munson

Estes

Dawson

Rogers

Mandell

(2017). Cost offset associated with Early Start Denver Model for children with autism. Journal of the American Academy of Child & Adolescent Psychiatry, 56, 777-783.

Cohen

Amerine-Dickens

Smith

(2006). Early intensive behavioral treatment: Replication of the UCLA model in a community setting. Journal of Developmental & Behavioral Pediatrics, 27, S145-S155.

10.

Cooper

J. O.

Heron

T. E.

Heward

W. L.

(2007). Applied behavior analysis (2nd ed.). Upper Saddle River, NJ: Pearson Books.

11.

Dixon

M. R.

Belisle

Stanley

Rowsey

Daar

J. H.

Szekely

(2015). Toward a behavior analysis of complex language for children with autism: Evaluating the relationship between PEAK and the VB-MAPP. Journal of Developmental and Physical Disabilities, 27, 223-233. doi:10.1007/s10882-014-9410-4

12.

Eikeseth

Klintwall

Jahr

Karlsson

(2012). Outcome for children with autism receiving early and intensive behavioral intervention in mainstream preschool and kindergarten settings. Research in Autism Spectrum Disorders, 6, 829-835. doi:10.1016/j.rasd.2011.09.002

13.

Eldevik

Eikeseth

Jahr

Smith

(2006). Effects of low-intensity behavioral treatment for children with autism and mental retardation. Journal of Autism and Developmental Disorders, 36, 211-224. doi:10.1007/s10803-005-0058-x

14.

Eldevik

Hastings

R. P.

Jahr

Hughes

J. C.

(2012). Outcomes of behavioral intervention for children with autism in mainstream pre-school settings. Journal of Autism and Developmental Disorders, 42, 210-220. doi:10.1007/s10803-011-1234-9

15.

Gould

Dixon

D. R.

Najdowski

A. C.

Smith

M. N.

Tarbox

(2011). A review of assessments for determining the content of early intensive behavioral intervention programs for autism spectrum disorders. Research in Autism Spectrum Disorders, 5, 990-1002. doi:10.1016/j.rasd.2011.01.012

16.

Hart

Risley

T. R.

(1975). Incidental teaching of language in the preschool. Journal of Applied Behavior Analysis, 8, 411-420.

17.

Howard

J. S.

Stanislaw

Green

Sparkman

C. R.

Cohen

H. G.

(2014). Comparison of behavior analytic and eclectic early interventions for young children with autism after three years. Research in Developmental Disabilities, 35, 3326-3344. doi:10.1016/j.ridd.2014.08.021

18.

Johnson

Hastings

R. P.

(2002). Facilitating factors and barriers to the implementation of intensive home-based behavioural intervention for young children with autism. Child: Care, Health and Development, 28, 123-129. doi:10.1046/j.1365-2214.2002.00251.x

19.

Kamio

Haraguchi

Miyake

Hiraiwa

(2015). Brief report: Large individual variation in outcomes of autistic children receiving low-intensity behavioral interventions in community settings. Child and Adolescent Psychiatry and Mental Health, 9, Article 6. doi:10.1186/s13034-015-0039-6

20.

Kisamore

A. N.

Karsten

A. M.

Mann

C. C.

(2016). Teaching multiply controlled intraverbals to children and adolescents with autism spectrum disorders. Journal of Applied Behavior Analysis, 49, 826-847. doi:10.1002/jaba.344

21.

Koegel

R. L.

Frea

W. D.

(1993). Treatment of social behavior in autism through the modification of pivotal social skills. Journal of Applied Behavior Analysis, 26, 369-377.

22.

Koegel

R. L.

Koegel

L. K.

Brookman

L. I.

(2003). Empirically supported pivotal response interventions for children with autism. In Kazdin

A. E.

(Ed.), Evidence-based psychotherapies for children and adolescents (pp. 341-357). New York, NY: Guilford Press.

23.

Landa

R. J.

Kalb

L. G.

(2012). Long-term outcomes of toddlers with autism spectrum disorders exposed to short-term intervention. Pediatrics, 130(Suppl. 2), S186-S190.

24.

Lovaas

O. I.

(1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55(1), 3-9.

25.

Mason

L. L.

Andrews

(2014). Referent-based verbal behavior instruction for children with autism. Behavior Analysis in Practice, 7, 107-111. doi:10.1007/s40617-014-0018-z

26.

McPheeters

M. L.

Warren

Sathe

Bruzek

J. L.

Krishnaswami

Jerome

R. N.

Veenstra-VanderWeele

(2011). A systematic review of medical treatments for children with autism spectrum disorders. Pediatrics, 127(5), e1312-e1321.

27.

Montallana

K. L.

(2018, May). Evaluating the inter-rater reliability of the VB-MAPP. In Pompa-Craven

(Chair), Evaluating the outcomes of low-intensity behavior interventions. Symposium conducted at the 44th annual conference of the Association for Behavior Analysis International, San Diego, CA.

28.

Nair

M. K. C.

Russell

P. S. S.

George

Prasanna

G. L.

Mini

A. O.

Leena

M. L.

. . . Minju

K. A.

(2014). CDC Kerala 8: Effectiveness of a clinic based, low intensity, early intervention for children with autism spectrum disorder in India: A naturalistic observational study. The Indian Journal of Pediatrics, 81, 110-114. doi:10.1007/s12098-014-1601-6

29.

Peters-Scheffer

Didden

Mulders

Korzilius

(2010). Low intensity behavioral treatment supplementing preschool services for young children with autism spectrum disorders and severe to mild intellectual disability. Research in Developmental Disabilities, 31, 1678-1684. doi:10.1016/j.ridd.2010.04.008

30.

Reichow

(2012). Overview of meta-analyses on early intensive behavioral intervention for young children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42, 512-520. doi:10.1007/s10803-011-1218-9

31.

Sallows

G. O.

Graupner

T. D.

(2005). Intensive behavioral treatment for children with autism: Four-year outcome and predictors. American Journal on Mental Retardation, 110, 417-438.

32.

Skinner

B. F.

(1957). Verbal behavior. Englewood Cliffs, NJ: Prentice Hall.

33.

Smith

(2001). Discrete trial training in the treatment of autism. Focus on Autism and Other Developmental Disabilities, 16, 86-92.

34.

Smith

Klevstrand

Lovaas

(1995). Behavioral treatment of Rett’s disorder: Ineffectiveness in three cases. American Journal on Mental Retardation, 100, 317-322.

35.

Smith

Klorman

Mruzek

D. W.

(2015). Predicting outcome of community-based early intensive behavioral intervention for children with autism. Journal of Abnormal Child Psychology, 43, 1271-1282. doi:10.1007/s10802-015-0002-2

36.

Sparrow

S. S.

Balla

D. A.

Cicchetti

D. V.

(1984). Vineland Adaptive Behavior Scales. Circle Pines, MN: American Guidance Service.

37.

Sparrow

S. S.

Cichhetti

D. V.

Balla

D. A.

(2005). Vineland Adaptive Behavior Scales, survey forms manual (2nd ed.). Bloomington, MN: Pearson Books.

38.

Stolte

Hodgetts

Smith

(2016). A critical review of outcome measures used to evaluate the effectiveness of comprehensive, community based treatment for young children with ASD. Research in Autism Spectrum Disorders, 23, 221-234. doi:10.1016/j.rasd.2015.12.009

39.

Sundberg

M. L.

(2008). Verbal behavior milestones assessment and placement program: The VB-MAPP. Concord, CA: AVB Press.

40.

Sundberg

M. L.

Partington

J. W.

(1998). Teaching language to children with autism and other developmental disabilities. Pleasant Hill, CA: Behavior Analysts.

41.

Virués-Ortega

(2010). Applied behavior analytic intervention for autism in early childhood: Meta-analysis, meta-regression and dose–response meta-analysis of multiple outcomes. Clinical Psychology Review, 30, 387-399.

42.

Williamson

Sathe

N. A.

Andrews

J. C.

Krishnaswami

McPheeters

M. L.

Fonnesbeck

. . . Warren

(2017, May). Medical therapies for children with autism spectrum disorder—An update (Comparative Effectiveness Review, No. 189). Rockville, MD: Agency for Healthcare Research and Quality.