Intensity and Learning Outcomes in the Treatment of Children With Autism Spectrum Disorder

Abstract

Ample research has shown that intensive applied behavior analysis (ABA) treatment produces robust outcomes for individuals with autism spectrum disorder (ASD); however, little is known about the relationship between treatment intensity and treatment outcomes. The current study was designed to evaluate this relationship. Participants included 726 children, ages 1.5 to 12 years old, receiving community-based behavioral intervention services. Results indicated a strong relationship between treatment intensity and mastery of learning objectives, where higher treatment intensity predicted greater progress. Specifically, 35% of the variance in mastery of learning objectives was accounted for by treatment hours using standard linear regression, and 60% of variance was accounted for using artificial neural networks. These results add to the existing support for higher intensity treatment for children with ASD.

Keywords

autism spectrum disorder treatment intensity applied behavior analysis treatment outcomes

Applied behavior analysis (ABA) is a well-established treatment for the symptoms and behaviors commonly associated with autism spectrum disorder (ASD; Eldevik, Eikeseth, Jahr, & Smith, 2006; Matson & Goldin, 2014; Myers & Johnson, 2007; Reichow, 2012; Reichow, Barton, Boyd, & Hume, 2012). This intervention is typically initiated in early development and provided for multiple years, generally at 20 to 40 hr per week (Eldevik et al., 2009; Granpeesheh, Dixon, Tarbox, Kaplan, & Wilke, 2009; Reichow et al., 2012). Despite the overall consensus that ABA is the preeminent treatment for ASD, there is still debate surrounding the most effective “dosage,” meaning the ideal quantity of treatment provided in a specific interval of time (e.g., hours per week). Some researchers speculate that there may be a point where treatment is too intense and the child “burns-out” (i.e., Matson & Smith, 2008) or that there may be a point of diminishing returns at which significant improvements are no longer made (reviewed by Fava & Strauss, 2014). However, others argue that as treatment hours increase, improvements likewise increase (e.g., Granpeesheh et al., 2009; Virués-Ortega, 2010).

Apart from the seminal study by Lovaas (1987) and a much later study by Reed, Osborne, and Corness (2007), few or no other studies have directly compared outcomes for groups receiving high- versus low-intensity ABA. Lovaas (1987) contrasted high intensity (40 hr) to low intensity (10 hr) and found that the high-intensity group achieved robust treatment effects, whereas the low-intensity group improved little. Likewise, Reed and colleagues (2007) contrasted high intensity (30 hr) to low intensity (12 hr) and found the high-intensity group performed much better than the low-intensity group. More information regarding the impact of treatment intensity may be gleaned from examining the study by Eldevik, Eikeseth, Jahr, and Smith (2006), who compared groups of participants with ASD and intellectual disability in two low-intensity groups. Participants received either 12.5 hr per week of treatment based almost exclusively on ABA principles or 12 hr per week of eclectic treatment (including alternative communication, ABA, sensory-motor therapies, programs based on principles from Division TEACCH®, etc.). While the ABA group outperformed the eclectic group, the gains made by the ABA group were significantly lower than those reported in studies in which ABA was implemented at an intensive level.

Several reviews and meta-analyses have been published that provide additional support for early intensive behavioral intervention (EIBI) while highlighting the role of treatment intensity. Some reviews have indicated overall improvement among groups but discrepant results among individual participants, reportedly affected by various child-specific factors, such as pretreatment IQ, adaptive, and language skills (Fava & Strauss, 2014). These variables are critical to identify to maximize the outcome of individualized treatment. In their 2009 meta-analysis of EIBI based on the Lovaas model, Reichow and Wolery found only two studies that compared different levels of treatment intensity (Lovaas, 1987; Smith, Eikeseth, Klevstrand, & Lovaas, 1997). They concluded that the greatest changes in IQ occurred among those children treated at a high level of treatment hours (30-40) for a long duration of time. Virués-Ortega (2010) found a variety of treatment dose–response relationships, wherein IQ did not show a clear improvement from increased intensity, but language and adaptive skills did. Virués-Ortega, Rodríguez, and Yu (2013) later conducted a study investigating intervention time in terms of both intensity (i.e., hours per week) and duration (i.e., total number of weeks). Their results indicated that increased intervention time, lower age at the beginning of treatment, and higher preintervention functioning are important variables in determining outcomes for children in programs that are up to 4 years long.

One particular challenge in drawing conclusions regarding the role of treatment intensity is due to a lack of studies with consistent experimental methodology or similar study samples that can be appropriately contrasted and compiled as evidence. For instance, Howlin, Magiati, and Charman (2009) reviewed 11 studies and noted that the researchers found that EIBI was effective at the group level, primarily in terms of increasing IQ. However, hours of intervention were difficult to estimate because few studies reported these parameters in sufficient detail. If hours were reported, they were provided by parents or therapists, rather than systematically monitored by the research teams. For most studies, only approximate average hours per week were provided. Additionally, at the individual participant level, varying degrees of improvement were found. Eldevik and colleagues (2010) also argued for a need to evaluate outcomes, not just at the group level but by looking for meaningful changes in individual children. To perform an individual participant data meta-analysis, they obtained individual participant records from 16 published studies on intensive behavioral intervention. They found that pretreatment IQ and adaptive behavior skills were predictive of gains in adaptive behavior. They also noted the importance of treatment intensity as a variable affecting treatment outcomes. These results further support the need for individualization in terms of treatment components, including intensity.

From their meta-analysis, Strauss, Mancini, SPC Group, and Fava (2013) concluded that most of the studies they reviewed from 2009 to 2011 provided insufficient reports of treatment hours. These inadequate reporting practices included only providing an approximate weekly range of intervention hours, not reporting control group hours, or not reporting details about treatment hours at all. The authors also found that caregiver involvement improved treatment results, with more intensive programs with parental inclusion (i.e., parents applying teaching strategies at home) resulting in better treatment outcomes.

As noted in numerous reviews and meta-analyses, the methodology (including outcome measures) chosen to evaluate these treatments has varied so significantly as to make contrasts difficult. In their recent article, Matson and Goldin (2014) reviewed targeted behaviors and outcome measures of EIBI. They concluded that there is not a current standard for outcome measures of studies of EIBI, which is problematic in that this prevents comparisons of studies and conclusions about appropriate dosage of intervention. In particular, they reported that standardized scales are the most frequently used method of measuring outcomes, with the most common being the Vineland Adaptive Behavior Scales, standardized tests of IQ, and the Bayley Scales of Infant Development. This is problematic in that many standardized measures have not been normed on children with ASD (Reichow & Wolery, 2009). Furthermore, use of IQ as an outcome measure of program efficacy is questionable, given that intelligence is not a diagnostic marker of ASD (Reichow & Wolery, 2009). Although some of the studies they included used measures of socialization, communication, repetitive behavior, and restricted interests to monitor outcomes, use of these measures was much less common. As such, the more common methods may be helpful in determining if more global improvements have taken place, but they do not allow monitoring of effects on core symptoms of ASD. As previously discussed by Granpeesheh and colleagues (2009), one alternative is to monitor the number of behavioral objectives a participant masters in a certain time period (e.g., mastered objectives per month). These data are readily available from ongoing ABA service delivery, as ABA service providers rely on such data on a daily basis to track treatment progress and make decisions regarding treatment planning.

Despite the difficulty in contrasting treatment intensity among studies to identify the ideal dosage of treatment, the emerging consensus among researchers is that treatment outcomes are significantly better when the dosage is high (over 30 hr per week). Nonetheless, this has not readily translated into clinical practice. Indeed, there is a high degree of variability among what clinicians provide. In a survey of 211 program supervisors, Love, Carr, Almason, and Petursdottir (2009) found an alarming degree of variability among the average hours of treatment reported, with roughly 25% of their sample falling into each of their four response options: 1 to 10, 11 to 20, 21 to 30, or 31 to 40 hr per week. Clearly, there is a disparity between what is reported in treatment literature as the optimal dosage of ABA and what is practiced in clinical settings. It would be easy to suggest that this disparity is simply due to mistranslation of research to practice. However, many factors impact the number of hours of treatment that each child receives in addition to the clinician’s treatment recommendations, including determinations by funding agencies to authorize fewer hours than those recommended by the clinician, arbitrary financial caps placed on treatment, and caregiver availability, among many others. Bridging the gap between research and practice will need to take into consideration all of these factors to be successful.

One organization that has helped to set standards for the field of ABA is the Behavior Analyst Certification Board (BACB), which began offering a national certification in behavior analysis in 1998. Recently, the BACB released updated treatment guidelines for health plans addressing the treatment of ASD (BACB, 2014). In its document, the BACB defines comprehensive ABA as consisting of 30 hr to 40 hr of treatment per week. While these clinical guidelines are a welcome addition, it is too soon to tell if they will improve standards of care. There is a need for further studies that focus on treatment outcomes within clinical settings.

The purpose of the present study was to further examine the relationship between ABA treatment hours and mastery of learning objectives within a large archival data set collected from a community-based provider of ABA services, which implements the CARD Model of ASD service delivery (Granpeesheh, Tarbox, Najdowski, & Kornack, 2014).

Methods

Data Collection

Treatment data were collected retrospectively from a large archival database. Clinical records were selected from a pool of 1,258 children receiving behavioral intervention services from a large community-based behavioral health agency. The Skills™ Assessment is an instrument that evaluates skills across eight areas of child development (Dixon, Tarbox, Najdowski, Wilke, & Granpeesheh, 2011). A study by Persicke and colleagues (2014) evaluated the validity of the Skills™ Assessment by contrasting parent response to the Skills™ items with direct observation. Pearson product-moment correlation coefficients ranged from moderate (r = .65) to high (r = .95). Through the course of normal service delivery, clinicians used the Skills™ system to identify treatment targets, plan interventions, and track treatment response. These data were integrated with operational information (such as treatment hours) collected by the participating treatment centers. These sources of information constituted the child’s clinical record and were queried for the information included in the present study.

Clinical records were selected if they met the following criteria: a diagnosis of ASD (American Psychiatric Association, 2013), autistic disorder (American Psychiatric Association, 2000), pervasive developmental disorder–not otherwise specified (PDD-NOS; American Psychiatric Association, 2000), or Asperger’s disorder (American Psychiatric Association, 2000); age between 18 months and 12 years old; and receiving a minimum of 20 hr of ABA treatment per month. Further, any individuals who were in their first month of treatment were excluded from the data set. These criteria resulted in a sample size of 726 individual records. The age (at end of study period), diagnosis, and gender profiles of the individuals whose clinical records were used in the study were as follows: 598 males (age range = 2.08-11.92 years, mean age = 7.46 years, 347 autistic disorder, 201 ASD, 46 PDD-NOS, four Asperger’s disorder) and 128 females (age range = 3.17-11.83 years, mean age = 7.59 years, 82 autistic disorder, 39 ASD, six PDD-NOS, one Asperger’s disorder). The average number of hours received per month was 72.81 (SD = 36.31) with a range from 20.02 to 197.25 (treatment hours per month did not significantly differ between gender groups). The vast majority of participants (N = 716) began treatment services prior to the study period (January 1, 2014-December 31, 2014). These participants on average had received 1.48 years of treatment (SD = 1.35, range = 0-4.67 months) prior to the start of the study period. For all participants, the average age at the start of treatment services was 5.15 years (SD = 2.04) with a range of 0.9 to 11.0 years. Participants in this study resided and received services in the states of Arizona, California, Colorado, Illinois, Louisiana, New York, Texas, and Virginia.

Treatment

Each child’s treatment program was customized to build upon his/her individual strengths and to address his/her individual deficits in proportion to individual need. In addition, local and regional variables, such as funding agency requirements, influenced whether treatment was provided in home, school, clinic, or a combination of settings. Despite the individualization of each child’s program, the following elements were common to all: (a) treatment was delivered on a one-to-one basis by trained behavioral therapists; (b) treatment included both more-structured (discrete trial training) and less-structured (natural environment training) behavioral teaching strategies; (c) language intervention took a verbal behavior approach; (d) both errorless and least-to-most prompting strategies were used; (e) all major empirically validated behavioral principles and procedures were used (i.e., reinforcement, extinction, stimulus control, generalization training, chaining, and shaping), as appropriate; (f) assessment and treatment of challenging behaviors followed a function-based approach; (g) parents were included in all treatment decisions and received training on a regular basis; (h) direct supervision was provided frequently (e.g., biweekly) by an expert in behavioral intervention for children with ASD; and (i) treatment content was based upon the CARD curriculum (Granpeesheh et al., 2014). Training for behavioral practitioners was multifaceted and included a combination of an eLearning program (www.ibehavioraltraining.com), classroom-style training, field-experience training, and evaluation. Practitioners received supervision by a Board Certified Behavior Analyst (BCBA) and attended monthly staff meetings that review treatment procedures.

Billing records were reviewed to determine the number of treatment hours received. All direct treatment service hours provided to the participant were included. Activities that were not direct treatment services, such as traveling to a participant’s home, were excluded. Further, any activities that were not client-specific would not have been a billed activity and thus were not included in the analyses.

Mastery of learning objectives was used as the dependent variable for all analyses within this study. The definition of mastery of a learning objective was set on an individual basis by the treatment supervisor, but was required to be within the bounds of the following criteria: greater than 70% accuracy of responding to the learning objective for a minimum of two treatment sessions across two different days. Typically, a more stringent mastery criterion of 80% accuracy is required, but supervisors have the discretion to deviate if they feel it is clinically appropriate to do so.

Data Analysis

To gain insight into the relationship between mastery of learning objectives and treatment intensity, an exploratory data analysis was conducted on the number of therapy hours and treatment duration received by the 726 participants included in the data set, as well as the number of learning objectives mastered during the course of a 12 months period (January 1, 2014 through December 31, 2014). The number of treatment hours an individual received during the 12-month period was matched with the total number of learning objectives mastered during that same time period. Not all participants received the same duration of treatment during this time period, with data on some spanning as little as 2 months of treatment and others having data through the entire 12-month period (range = 2-12, mean = 6.87, SD = 2.72). Further, the initial months of treatment data do not imply that these were the individuals’ first months of treatment. For some participants, the 12-month period may have captured the start of treatment whereas for others, they may have received treatment for a number of months prior to the 12-month period from which data were queried. Figure 1 and Figure 2 provide a visualization of the distributions for therapy hours and mastered learning objectives. From the histograms, it becomes apparent that the distributions for both treatment hours and mastered learning objectives are positively skewed. Furthermore, the value distribution for both variables spans several orders of magnitude, with the range of total therapy hours being 40 to 1,973 and the range of mastered learning objectives being 2 to 1,973. To ensure data integrity, a manual inspection of the database was undertaken, with the audit showing that data points representing extreme values were recorded correctly based on historical records.

Figure 1.

Histogram of total patient treatment hours.

Figure 2.

Histogram of total number of mastered learning objectives.

Based on exploratory analysis of the raw data, a log transform was applied to the data to account for values spanning several orders of magnitude, which is a standard practice in the statistics community when working with non-negative data. This transform was chosen because it is both order-preserving and easy to interpret when it forms the basis for a regression model. Figures 3 and 4 show the log-transformed distributions for these variables, which result in normal distributions and form the basis of the regression analysis detailed in the next section.

Figure 3.

Histogram of log-transformed therapy hours.

Figure 4.

Histogram of log-transformed mastered learning objectives.

Results

After transforming the data, a regression analysis was undertaken, with total treatment hours being used as the sole predictor variable for the number of total learning objectives mastered. A log transform was used for each variable. The scatter plot in Figure 5 depicts the relationship between these variables, as well as the line fit by a simple least-squares linear regression model. The linear relationship between treatment hours and mastery of learning objectives is apparent, with the R² statistic indicating that 35% of the variance in number of learning objectives mastered is explained by this relationship (see Table 1). For completeness, a linear regression model was fit to the untransformed data, yielding an R² of .18. This model is depicted in Figure 6.

Figure 5.

Relationship of treatment hours and mastered learning objectives based on linear regression.

Table 1.

Linear Regression Parameters for Total Treatment Hours.

Regression parameters
	Estimate	p value
Intercept	−0.65	.000
Hours	1.03	.000
R ²	.35
F-test	394.2	.000

Figure 6.

Relationship of the untransformed variables for treatment hours and mastered learning objectives based on linear regression.

Table 1 provides a listing of the pertinent parameters for the regression model. This is a substantial improvement over the results reported by Granpeesheh and colleagues (2009), who reported an R² of .147 for a sample size of 245 children. The previous study also leveraged age as a predictor variable in addition to treatment hours.

The use of total treatment hours as the independent variable in the regression analysis brought up a question of whether the source of the correlation was from the intensity of the treatment or the duration of the treatment. A secondary regression analysis was run using average monthly treatment hours and months of treatment as the predictor variables for the number of total learning objectives mastered. The results showed that both average monthly treatment hours and months of treatment significantly contributed to the number of total mastered learning objectives. This model resulted in an improved fit with an R² of .453. The relevant regression parameters can be seen in Table 2.

Table 2.

Linear Regression Parameters for Average Intensity and Duration.

Regression parameters
	Estimate	p value
Intercept	0.03	.839
Intensity	0.34	.000
Duration	1.76	.000
R ²	.45
F-test	300.0	.000

To compare more closely with Granpeesheh and colleagues (2009), the previous regression analyses were repeated with the addition of age as a predictor variable. In both cases, the age of the child was negatively correlated with the number of total mastered learning objectives. Although the effect of age was highly variable, the effect size was large enough to have a significant influence on the number of mastered learning objectives. The results of these regressions can be seen in Table 3 and Table 4. As age and average monthly treatment hours were used as predictor variables in the same regression model, it is important to check their collinearity. A regression model using age as the predictor variable for average monthly treatment hours shows that the average number of monthly treatment hours was reduced by 3.13 for each year. Again, this relationship was highly variable resulting in an R² of .04 which shows that the correlations of these two variables is not a cause for concern in the previous model. These results are shown in Table 5.

Table 3.

Linear Regression Parameters for Total Treatment Hours With Age.

Regression parameters
	Estimate	p value
Intercept	−0.25	.171
Hours	1.01	.000
Age	−0.38	.002
R ²	.36
F-test	204.5	.000

Table 4.

Linear Regression Parameters for Average Intensity and Duration With Age.

Regression parameters
	Estimate	p value
Intercept	0.62	.001
Intensity	0.27	.001
Duration	1.75	.000
Age	−0.53	.000
R ²	.47
F-test	213.9	.000

Table 5.

Age Influence on Average Intensity.

Regression parameters
	Estimate	p value
Intercept	96.26	.000
Age	−3.13	.000
R ²	.04
F-test	30.26	.000

With a baseline established using linear regression, it becomes possible to explore more sophisticated machine learning techniques to predict mastery of learning objectives. A hurdle in standard regression techniques is that the form of the function to be fit to the data must be picked a priori, despite the fact that in many cases the relationship between predictor and response variables is not well understood beforehand. To this end, a simple feed-forward neural network was applied, consisting of only 1 hidden layer, to the task of modeling the relationship of therapy hours to mastery of learning objectives. Artificial neural networks (ANNs) are a widely studied and applied subset of data mining algorithms (Mitchell, 1997), and even small networks with simple topologies have the power to learn any continuous function (Hornik, 1991). In particular, the learning of the function is unsupervised, and a human need not specify the shape of the curve to be learned. This substantial benefit is the primary motivation for considering an ANN-based approach as a separate, but related, analysis to understand the relationship between treatment and learning outcomes.

Figure 7 shows a generic diagram of a feed-forward ANN with a single hidden layer. The independent variables (therapy hours in this case) are fed to the network as an input, and the weights of the network connections (initialized randomly at first) are used to produce a predicted output (mastered learning objectives). The data are then used to adjust the weights of the network until the predicted output is as close as possible to the desired output specified by the data, at which point the training of the network is complete. Mathematically, training the weights of the network corresponds to minimizing the error of the network predictions at the output layer. Because this error function is chosen to be continuous and differentiable, the internal weights can be adjusted incrementally by solving a system of partial derivatives. In computer science this algorithm is known as backpropagation (Rumelhart, Hinton, & Williams, 1986), one of the fundamental algorithms in ANN research. To ensure that the model learned by the network is generalizable, the network is trained on a random subset of the available data. Remaining data are used as an unseen test data set, which is used to measure the accuracy of predictions after training.

Figure 7.

Topology of a feed-forward artificial neural network with one hidden layer.

To apply neural networks to the data presented here, the data were randomly partitioned into training (65%), testing (30%), and validation (5%) subsets. The validation data were used in the training process to increase the efficiency of the algorithm and were not used to test the final fit of the learned model. The network was trained via backpropagation for 1000 iterations. Bayesian regularization (Foresee & Hagan, 1997) was applied as part of the training process to improve the robustness of the learned target function to noise, as well as improve generalization. While a treatment of Bayesian regularization is beyond the scope of this article, it effectively works by adding an additional term to the error function being optimized, which has an overall smoothing effect.

For the research question considered here, an ANN was trained consisting only of therapy hours as the input and mastered learning objectives as the target. To begin, we trained an ANN on untransformed data, which yielded an R² of .469, an immediate improvement over linear regression due to the model’s ability to adapt to non-linearity in the data. We followed this with a model trained on log-transformed data, to parallel the analysis carried out using linear regression. Figure 8 shows the resulting fit, which demonstrates a non-linear trend to the line fit by the model. Using therapy hours alone, the neural network achieves an R² of .60 on the entire data set, explaining a substantially higher amount of variance than the more simple linear regression model. Finally, for completeness, we trained a final model which incorporated patient age as an input in addition to therapy hours. This resulted in a trivial increase of the R² to .61.

Figure 8.

Relationship of treatment hours and mastered learning objectives learned by artificial neural network.

While the artificial neural network outperforms linear regression, it is important to note that the nature of neural networks make them black boxes, meaning that the internal parameters used by the neural network to construct the fit function are not easily interpretable by humans. This parameters learned by the network have no direct probabilistic or geometric interpretation. Thus, researchers must make a tradeoff when determining whether to select neural networks to model the relationship between independent and dependent variables. The ANN offers the advantages of increased goodness of fit without having to constrain the form of the fit function a priori. Traditional models, linear regression in this case, may sacrifice some of this flexibility in exchange for interpretability of model parameters. Nevertheless, the properties of the backpropagation algorithm are well understood and mathematically sound, and so an artificial neural network approach to this regression problem still provides an attractive alternative to traditional techniques. In particular, the trained neural network model can be used to estimate the expected mastery of learning objectives for a given number of therapy hours, allowing for the same interpolation and extrapolation as provided by standard least-squares linear regression.

Discussion

These results show a clear relationship between treatment intensity and mastery of learning objectives in the context of behavioral intervention for children with ASD in a community-based clinical setting, regardless of the age of the child receiving the service. This study builds upon the findings of Granpeesheh and colleagues (2009) in several important directions. One of the limitations noted by Granpeesheh and colleagues (2009) was the non-standardized nature of using mastered learning objectives. A standardized assessment and treatment-tracking tool (Skills™), which has been shown to have strong reliability (Dixon et al., 2011) and validity (Persicke et al., 2014), was used to ensure that all participants were measured according to the same criteria in a valid and reliable manner. While there is still inherent variability in difficulty to master one objective from another, the impact of this is likely mitigated by the large sample size.

It is also worth noting that the current study found a clear relationship between treatment hours and mastery of learning objectives across a sample that included a substantial portion of older children (mean age of 7.1 years). As discussed in the introduction, previous research on treatment intensity has focused on young children with ASD. This study is among the first to evaluate the effects of treatment intensity on mastery of learning objectives in older children with ASD. Although further research on treatment intensity in older children with ASD is still needed, the current results suggest that the common assumption that intensive treatment is only appropriate for young children may not be true.

Multiple factors are involved in a child’s response to treatment, and one consistent finding across EIBI outcome studies is a high degree of variability among participants in treatment response (Fava & Strauss, 2014). Therefore, while a complicated relationship among factors influencing treatment response is assumed, it is worthwhile to note that, across a large number of children receiving behavioral intervention services in a community-based clinical setting, a strong relationship was found that accounted for 35% of the variance in a child’s mastery of learning objectives using a standard linear regression and 60% of the variance using ANN. That is to say, without taking into consideration any child-specific variables, such as age (Granpeesheh et al., 2009) or parent involvement (Strauss et al., 2013), this single treatment-specific variable of intensity accounts for a large portion of how much a child will progress during treatment. Further, these data were not limited to children receiving only intensive treatment (e.g., 25-40 hr). This relationship was found across all levels of treatment intensity, most notably those who were also receiving relatively low treatment hours.

Given the nature of the present study, that is, a retrospective analysis of archival data, we are able to describe what occurred but are left to only speculate as to why. However, based upon the improvement in the model by moving from a simple linear relationship to a non-linear relationship developed by the ANN, one may conclude that while increased treatment hours was strongly related to more learning occurring within a given period of time, there are also bands within the intensity spectrum wherein an individual receiving ABA-based treatment for ASD will learn more per hour. The relationship between treatment hours and learning objectives found in Figure 8 shows that the shape is slightly sigmoidal. That is, at the lowest and highest levels of intensity, learning per hour was not as great as in the middle of the distribution. It may be the case that as treatment intensity moves from low to high, there is a base level of exposure needed, that once received increases the rate of learning in subsequent presentation of other stimuli. Further, at the highest levels of treatment intensity, the learning objectives mastered per hour were slightly less. This is contrary to the results found by Granpeesheh and colleagues (2009) who found that as treatment hours increased, significantly more learning objectives were mastered for every hour of treatment. Future research is needed to further explore the relationship between treatment hours and mastery of learning objectives within both the high and low levels of intensity. It should be noted that the simple relationship observed between treatment hours and mastered learning objectives far outweighs the differential rate of learning at higher or lower levels of treatment intensity.

Response to treatment is multifaceted, and dose–response relationships are likely stronger for some domains than for others. For example, Virués-Ortega (2010) found that language skills benefited from increased treatment duration, whereas adaptive skills benefited from treatment intensity, and intellectual functioning appeared to not show a relationship to intensity nor duration, as discussed previously by Matson and Smith (2008). Further research looking at treatment response within particular curricula domains would allow for a more fine-grained analysis and could provide insight into which specific treatment manipulations would result in the best outcomes.

Per their 2014 review, Matson and Goldin noted that, although it is the most common practice, use of standardized scales as outcome measures might not be the best option. These authors argued that, although such measures evaluate a broad range of behaviors, they are not tailored to the individual and are not as sensitive as progress monitoring of target behaviors. Furthermore, most standardized measures utilized thus far for outcomes in studies on dosage are not necessarily representative of improvement in symptoms of ASD (i.e., socialization, communication, and repetitive behaviors and restricted interests; Reichow & Wolery, 2009). As such, using mastery of objectives to monitor progress provides a manner by which to measure individualized gains in target behaviors and also allows comparison at the group level.

Nevertheless, as was noted by Granpeesheh and colleagues (2009) and recently discussed by Fava and Strauss (2014), mastery of learning objectives may or may not directly translate to making a change in the core deficits of ASD. This remains a limitation of the present methods of using mastery of discrete learning objectives as a primary outcome. Future research could consider including only mastery of particular behavioral domains that correspond directly to diagnostic criteria, such as language, social skills, and play, and decreasing repetitive behavior. Regardless, using mastery of behavioral objectives as a measure of treatment response is arguably more representative of what is commonly practiced in EIBI programs. In our experience, some service providers may administer standardized assessments when required by funding sources; however, this is the exception and not the norm.

Another limitation of the current study is that treatment hours were not randomly assigned. There may be a number of reasons that one individual received more treatment hours than another. The authors can only speculate as to the reasoning that each clinician used in making treatment recommendations, as well as each funding source’s decision process either to fund or deny treatment at a particular intensity or duration. Nevertheless, the current study included a relatively large sample dispersed over a relatively large and heterogeneous geographical area, so it seems unlikely that any of these variables were systematically associated with individuals who would have been higher or lower treatment responders for other reasons.

The strong relationship between treatment intensity and mastery of learning objectives is an important finding and has implications for setting clinical standards and guiding public policy decisions. As reported by Love and colleagues (2009), there is a high degree of variability in the number of treatment hours that clients receive in clinical settings. This is likely due to multiple causes, one of which is the current role that funding sources play in determining treatment intensity and duration. Unfortunately, clinical practice until now has been shaped as much by financial constraints, such as the cost borne by families and arbitrary caps on treatment hours imposed by funding agencies, as it has by the establishment of best practice standards. Multi-pronged efforts, however, have begun to increase access to ABA at the proper dosage and intensity, shifting treatment decisions from the funding source to the clinician where best practices have greater influence. The momentum of autism insurance reform laws (commonly known as “autism mandates”) has made ABA-based autism treatment a covered benefit of insurance policies in 43 states (as of the date of writing). Additionally, litigation arising from treatment denial by state agencies has clarified that ABA-based autism treatment is medically necessary and must be included in Early Periodic Screening, Diagnosis, and Treatment (EPSDT), the child health component of Medicaid that is required in every state. Underpinning both of these efforts and representing a primary factor in this shift toward best practices is the large body of research documenting the effectiveness of ABA in treating the behaviors and deficits associated with ASD, which has disarmed funding agencies that relied on a characterization of ABA as “experimental” to deny authorizations for treatment. Collectively, these efforts have given weight to treatment guidelines that can safeguard critical decisions about treatment intensity by taking them out of inexpert hands and leaving them to the discretion of highly trained clinicians. The authors are hopeful that clinical practices will continue to evolve to ensure that treatment intensity reflects best practices, such as those described in the ASD treatment guidelines issued by the BACB (2014).

The current results suggest several potentially fruitful areas for future research. First, little previous research has evaluated the effects of the intensity of supervision included in behavioral intervention programs (Eikeseth, Hayward, Gale, Gitlesen, & Eldevik, 2009). The treatment intensity data included in the current study only comprised the number of direct therapy hours delivered by therapists, not the number of hours that such therapy was supervised by master’s or doctorate-level clinicians and/or Board Certified Behavior Analysts. Future research should evaluate whether the amount of supervision impacts learning rate. Second, there is currently little consensus regarding the amount of training or experience required for line therapists or supervisors and whether or how much such training and experience impact learning rate in children with ASD. Future research could include a measure of clinician experience as a covariate in analyses of treatment intensity and learning rate. Finally, much more research is needed on the impact of parent training and parent involvement on learning rate. Future research should include some measure of parent training and/or parent involvement in ongoing intervention when analyzing the effects of treatment intensity on learning rate.

Perhaps, the most exciting potential direction for future research based on the current study is the possibility of using big data analytics to predict probable future learning rates based on child and other variables to ascertain reasonable expectations for dose–response at the outset of treatment. While it is unlikely that any other single variable would account for as high an effect as treatment intensity (e.g., 60%), numerous other variables must be targeted to account for the remaining unexplained variance in treatment outcome. These factors may include the child’s medical conditions and other interventions (such as speech, diet, and medications). Based on such predictions, clinicians might someday be able to identify individuals who are likely to be lower responders and target them for treatment enhancements, so they may be helped to respond to treatment at a higher rate. Possible treatment enhancements might include additional parent training, greater focus on visual supports, greater focus on establishing social interaction as a source of conditioned positive reinforcement, and/or early intervention for comorbid behavioral challenges, such as feeding or sleep disorders.

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Erik Linstead is an assistant professor of computer science and software engineering at Chapman University, where he leads the Machine Learning and Assistive Technology Laboratory (MLAT). He holds a BS from Chapman University, an MS from Stanford University, and a PhD from UC Irvine, all in computer science.

Dennis R. Dixon is the chief strategy officer at the Center for Autism and Related Disorders. He holds his PhD in psychology from Louisiana State University and completed a postdoctoral fellowship at Johns Hopkins University, School of Medicine.

Ryan French is a doctoral student of computational science at Chapman University, Schmid College of Science and Technology; a software engineer at the Economic Science Institute; and an adjunct professor of math and computer science. His research specializes in machine learning and computational economics.

Doreen Granpeesheh is the founder and executive director of the Center for Autism and Related Disorders. She received her PhD in psychology from the University of California, Los Angeles. She is a board certified behavior analyst and licensed psychologist with decades of experience treating individuals with autism spectrum disorder.

Hilary Adams is a predoctoral intern in clinical psychology at Indiana University School of Medicine, where she is fulfilling the final requirement to earn her PhD from Louisiana State University. She is a former therapist and current research assistant at CARD.

Rene German is an instructor of computer science and software engineering at Chapman University and a member of the MLAT. He holds an undergraduate degree in computer science, as well as an MS degree in computational and data science, both from Chapman University.

Alva Powell currently serves as chief technology officer at the Center for Autism and Related Disorders. Previously, he pioneered work with GPS mapping software, International Space Station software simulations, and developed large-scale websites. He received his BS degree in computer science from Michigan Technology University.

Elizabeth Stevens is a doctoral student in the Computational and Data Science program at Chapman University, where she previously earned her undergraduate degree in mathematics. Her research focuses on machine learning and data mining applied to autism spectrum disorder.

Jonathan Tarbox is director of research and regional clinic director at FirstSteps for Kids. He has published two books and over 60 peer-reviewed articles and chapters, is an associate editor of Behavior Analysis in Practice, and is on the editorial boards of The Analysis of Verbal Behavior and Behavior Modification.

Julie Kornack is the Director of Public Policy at the Center for Autism and Related Disorders. Her work includes identifying, developing, and supporting state and federal initiatives that increase access to autism treatment. She works directly with stakeholders to increase access to evidence-based treatment and serves on several advisory committees.

References

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Behavior Analyst Certification Board. (2014). Applied behavior analysis treatment of autism spectrum disorder: Practice guidelines for healthcare funders and managers (2nd ed.). Retrieved from http://bacb.com/wp-content/uploads/2015/07/ABA_Guidelines_for_ASD.pdf

Dixon

D. R.

Tarbox

Najdowski

A. C.

Wilke

A. E.

Granpeesheh

(2011). A comprehensive evaluation of language for early behavioral intervention programs: The reliability of the SKILLS language index. Research in Autism Spectrum Disorders, 5, 506-511.

Eikeseth

Hayward

Gale

Gitlesen

Eldevik

(2009). Intensity of supervision and outcome for preschool aged children receiving early and intensive behavioral interventions: A preliminary study. Research in Autism Spectrum Disorders, 3, 67-73.

Eldevik

Eikeseth

Jahr

Smith

(2006). Effects of low-intensity behavioral treatment for children with autism and mental retardation. Journal of Autism and Developmental Disorders, 36, 211-224.

Eldevik

Hastings

R. P.

Hughes

J. C.

Jahr

Eikeseth

Cross

(2009). Meta-analysis of early intensive behavioral intervention for children with autism. Journal of Clinical Child & Adolescent Psychology, 38, 439-450.

Eldevik

Hastings

R. P.

Hughes

J. C.

Jahr

Eikeseth

Cross

(2010). Using participant data to extend the evidence base for intensive behavioral intervention for children with autism. American Journal on Intellectual and Developmental Disabilities, 115, 381-405.

Fava

Strauss

(2014). Response to early intensive behavioral intervention for autism—An umbrella approach to issues critical to treatment individualization. International Journal of Developmental Neuroscience, 39, 49-58.

10.

Foresee

F. D.

Hagan

M. T.

(1997). Gauss-Newton approximation to Bayesian regularization. Proceedings of the 1997 International Joint Conference on Neural Networks, 3, 1930-1935.

11.

Granpeesheh

Dixon

D. R.

Tarbox

Kaplan

A. M.

Wilke

A. E.

(2009). The effects of age and treatment intensity on behavioral intervention outcomes for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 3, 1014-1022.

12.

Granpeesheh

Tarbox

Najdowski

A. C.

Kornack

(Eds.). (2014). Evidence-based treatment for children with autism: The CARD model. Waltham, MA: Elsevier.

13.

Hornik

(1991). Approximation capabilities of multilayer feedforward networks. Neural Networks, 4, 251-257.

14.

Howlin

Magiati

Charman

(2009). Systematic review of early intensive behavioral interventions for children with autism. American Journal on Intellectual and Developmental Disabilities, 114, 23-41.

15.

Lovaas

O. I.

(1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3-9.

16.

Love

J. R.

Carr

J. E.

Almason

S. M.

Petursdottir

A. I.

(2009). Early and intensive behavioral intervention for autism: A survey of clinical practices. Research in Autism Spectrum Disorders, 3, 421-428.

17.

Matson

J. L.

Goldin

R. L.

(2014). Early intensive behavioral interventions: Selecting behaviors for treatment and assessing treatment effectiveness. Research in Autism Spectrum Disorders, 8, 138-142.

18.

Matson

J. L.

Smith

K. R. M.

(2008). Current status of intensive behavioral interventions for young children with autism and PDD-NOS. Research in Autism Spectrum Disorders, 2, 60-74.

19.

Mitchell

(1997). Machine learning (1st ed.). New York, NY: McGraw-Hill.

20.

Myers

S. M.

Johnson

C. P.

(2007). Management of children with autism spectrum disorders. Pediatrics, 120, 1162-1182.

21.

Persicke

Bishop

M. R.

Coffman

C. M.

Najdowski

A. C.

Tarbox

Chi

Deering

(2014). Evaluation of the concurrent validity of a skills assessment for autism treatment. Research in Autism Spectrum Disorders, 8, 281-285.

22.

Reed

Osborne

L. A.

Corness

(2007). Brief report: Relative effectiveness of different home-based behavioral approaches to early teaching intervention. Journal of Autism and Developmental Disorders, 37, 1815-1821.

23.

Reichow

(2012). Overview of meta-analyses on early intensive behavioral intervention for young children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42, 512-520.

24.

Reichow

Barton

E. E.

Boyd

B. A.

Hume

(2012). Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD). Cochrane Database of Systematic Reviews, 10, CD009260.

25.

Reichow

Wolery

(2009). Comprehensive synthesis of early intensive behavioral interventions for young children with autism based on the UCLA young autism project model. Journal of Autism and Developmental Disorders, 39, 23-41.

26.

Rumelhart

D. E.

Hinton

G. E.

Williams

R. J.

(1986). Learning representations by back-propagating errors. Nature, 323, 533-536.

27.

Smith

Eikeseth

Klevstrand

Lovaas

O. I.

(1997). Intensive behavioral treatment for preschoolers with severe mental retardation and pervasive developmental disorder. American Journal of Mental Retardation, 102, 238-249.

28.

Strauss

Mancini

, SPC Group, & Fava

(2013). Parent inclusion in early intensive behavioral interventions for young children with ASD: A synthesis of meta-analyses from 2009 to 2011. Research in Developmental Disabilities, 34, 2967-2985.

29.

Virués-Ortega

(2010). Applied behavior analytic intervention for autism in early childhood: Meta-analysis, meta-regression and dose-response meta-analysis of multiple outcomes. Clinical Psychology Review, 30, 387-399.

30.

Virués-Ortega

Rodríguez

C. T.

(2013). Prediction of treatment outcomes and longitudinal analysis in children with autism undergoing intensive behavioral intervention. International Journal of Clinical and Health Psychology, 13, 91-100.