Interest level in 2-year-olds with autism spectrum disorder predicts rate of verbal,nonverbal,and adaptive skill acquisition

Abstract

Recent studies have suggested that skill acquisition rates for children with autism spectrum disorders receiving early interventions can be predicted by child motivation. We examined whether level of interest during an Autism Diagnostic Observation Schedule assessment at 2 years predicts subsequent rates of verbal, nonverbal, and adaptive skill acquisition to the age of 3 years. A total of 70 toddlers with autism spectrum disorder, mean age of 21.9 months, were scored using Interest Level Scoring for Autism, quantifying toddlers’ interest in toys, social routines, and activities that could serve as reinforcers in an intervention. Adaptive level and mental age were measured concurrently (Time 1) and again after a mean of 16.3 months of treatment (Time 2). Interest Level Scoring for Autism score, Autism Diagnostic Observation Schedule score, adaptive age equivalent, verbal and nonverbal mental age, and intensity of intervention were entered into regression models to predict rates of skill acquisition. Interest level at Time 1 predicted subsequent acquisition rate of adaptive skills (R² = 0.36) and verbal mental age (R² = 0.30), above and beyond the effects of Time 1 verbal and nonverbal mental ages and Autism Diagnostic Observation Schedule scores. Interest level at Time 1 also contributed (R² = 0.30), with treatment intensity, to variance in development of nonverbal mental age.

Keywords

autism spectrum disorders behavioral measurement interventions—psychosocial/behavioral development

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by impairments in social interaction and atypical repetitive body movements and interests (American Psychiatric Association, 2013). Although the etiology of ASD is widely assumed to be genetic, the pathological mechanism between specific genes and the behavioral deficits defining the disorder remains unknown. The social motivation hypothesis of autism stresses the importance of social motivation and learning in the development of deficits in social interaction (Chevallier et al., 2012). It is argued that ASD is not primarily a disorder in social cognitive ability, but rather the result of a lack of social motivation (Dawson, 2008). The deficits in social cognition exhibited by children with autism (e.g. deficits in verbal and nonverbal communication, deficits in emotion and face recognition, and deficits in theory of mind) are hypothesized to be a downstream effect of a more basic deficit in social interest, which would normally motivate children to acquire social skills and abilities. Studies have shown that deficits in attention to people and their faces can be seen as early as at 6 months, at a group level, in infants later diagnosed with ASD (Shic et al., 2014).

Several behavioral interventions for children with autism have been developed, such as Early and Intensive Behavioral Intervention (EIBI), Pivotal Response Training (PRT), or the Early Start Denver Model (ESDM). In EIBI, children are taught whichever adaptive skills they lack, using highly structured training, typically in one-to-one settings (Klintwall and Eikeseth, 2014). PRT, by contrast, puts more emphasis on child initiations and focus training on hypothesized core, or pivotal, skills: responding to multiple cues, self-management, and initiating communication (Koegel et al., 2008). The ESDM aims to establish social motivation in children, and most training takes place in natural settings. Compared to EIBI, teaching is less structured (Smith et al., 2008). Outcome studies have shown that children who receive behavioral interventions such as EIBI, PRT, or ESDM can acquire adaptive behaviors, language, and other social skills (Reichow et al., 2012). These intervention programs all include elements of applied behavior analysis (ABA), specifically positive reinforcement of target behaviors. A positive reinforcer is defined as a stimulus presented after a behavior which increases the probability for that behavior to be repeated in similar situations (Cooper et al., 2007). In behavioral interventions, reinforcers are used to increase target behaviors (social and nonsocial behaviors, such as language and motor skills). Novel skills are taught by prompting the novel response and then reinforcing either scaffolded responses or approximations. Subsequently, naturally occurring reinforcers are assumed to maintain the acquired behaviors in environments outside of intervention. Reinforcers typically include edibles or toys, social routines, or preferred activities. Reinforcers in treatment are typically chosen based on child interest (approach behaviors, exhibited positive emotion, curiosity about, etc.), although several techniques for formal reinforcer assessment have been developed (Kang et al., 2013). The three intervention types described above tend to rely on different types of reinforcers. For instance, EIBI might use reinforcers that are arbitrary to the behavior that is being taught (e.g. access to YouTube might be used as a reinforcer when teaching a child to dress). PRT typically teaches behaviors related to whatever reinforcer seems of momentary interest to the child (e.g. if the child shows interest in a swing, saying “swing” or “I want to swing” is taught). In ESDM, more emphasis is put on the use of social reinforcement and social routines, although contrived reinforcers are used for children who are hard to motivate. In all these behavioral interventions, however, it seems likely that children for whom it is easy to find motivating stimuli would be easier to engage in treatment.

Although behavioral interventions have been shown to be effective in controlled group studies (Reichow et al., 2012; Tonge et al., 2014), individual outcomes vary considerably. Some children exhibit substantial rates of skill acquisition in treatment, sometimes catching up with typically developing peers, whereas others show little or no progress (Klintwall et al., 2013). Although several factors predict which children will benefit most, such as higher IQ and younger age (Harris and Handleman, 2000), a recent meta-study combining data from 16 studies failed to replicate these findings (Eldevik et al., 2010). Young children who receive a range of interventions exhibit considerable variation in outcomes later in life. Although attempts have been made to predict developmental trajectories (Macari et al., 2012), diagnostic stability (Chawarska et al., 2009), and language development (Charman et al., 2003), sources of variability are largely unknown. Although the majority of children diagnosed with ASD go on to receive different kinds of treatment—and this might explain some of the variation—some of this variability is potentially explainable by child characteristics, such as initial levels of receptive language, IQ, adaptive skills (Flanagan et al., 2012), rate of communicative bids (Charman et al., 2005), functional play (Vivanti et al., 2012), and joint attention skills (Kasari et al., 2012). By identifying child prerequisites for successful treatments, children lacking those prerequisites can be identified, and interventions can be developed to prepare children to enter into already-existing treatments.

Given that autism has been conceptualized as a social motivation disorder (Dawson, 2008), and that effective treatments rely on the child’s ability to respond to positive reinforcement, it has been suggested that the concept of reinforcement should be considered as a key factor in explaining outcome variability (Klintwall and Eikeseth, 2012). A lack of social motivation would then lead to autistic behaviors to develop, but this could then potentially be reversed by the use of nonsocial motivation in the form of nonsocial positive reinforcement (e.g. edibles) to establish social behaviors in interventions. Clinical practice suggests that it is easier to identify reinforcers for the behavior of some children, and it is known that varying reinforcers makes training more effective (Egel, 1981). Thus, one possibility is that children who exhibit interest in a wide range of stimuli would acquire skills more rapidly in intervention programs.

Sherer and Schreibman (2005) compared three children who were considered good-outcome participants in a PRT program to three children whose outcome was less favorable. One difference between the groups was “toy interest” at intake. In another study, Schreibman et al. (2009) used “toy contact” frequency at intake as a predictor of outcome in children receiving both Discrete Trial Training (DTT) and PRT. Toy contact predicted outcome in PRT but not in DTT. Vivanti et al. (2012) measured “functional object use” at intake in an ESDM program, which was found to predict treatment outcome as measured by Mullen IQ tests. Finally, Yoder and Stone (2006) found that “object exploration” predicted response to a Picture Exchange Communication System (PECS) intervention. These different measures might possibly reflect one single construct: object interest. However, to varying degrees, the definitions also measure child level of functioning (e.g. the cognitive requirements of getting a high score in “functional object use”).

Klintwall and Eikeseth (2012) investigated predictors of outcome in EIBI. Among the factors considered in the study was the number of stimuli that parents and teachers reported as being reinforcers (such as toys, social routines, and activities such as watching TV). This factor was found to account for a quarter of the variance in outcome, measured as rate of adaptive skill acquisition. However, the study had limitations: assessment of the number and type of reinforcers was based on retrospective parental or teacher report, in a small sample. Note that the measure of “potential reinforcers” used by Klintwall and Eikeseth is wider than the “object interest” used in the studies cited above (e.g. including videos, social routines, and edibles). Note also that although some of these studies describe predictors of outcome in specific interventions, it is quite possible that the same factors might be general maturation predictors, irrespective of the type of intervention that the child receives.

The aim of this study was to investigate whether interest in objects, social routines, and activities during a social communication assessment predicts outcomes in terms of change in adaptive behavior and verbal and nonverbal skills from Time 1 (around 22 months of age) to Time 2 (around 38 months of age). In addition to this motivational factor, predictors included adaptive behavior, verbal and nonverbal skills, and Autism Diagnostic Observation Schedule–Toddler (ADOS-T) scores at intake, and treatment intensity. We developed a new coding system: the Interest Level Scoring for Autism (ILSA). The purpose of ILSA is to quantify level of interest in toddlers with ASD. We applied this scoring system to video recordings of the ADOS-T (Luyster et al., 2009).

Method

Participants

Sample characteristics

Participants were 70 toddlers enrolled in large study of social cognition. Participants were included if they received a diagnosis of ASD around their second birthday and returned for a confirmatory assessment (including psychiatric diagnosis) 1 year later. Clinical best estimate diagnosis was assigned by two expert clinicians at first visit (Time 1: mean age = 21.9 months, standard deviation (SD) = 3.1 months) and again at the second visit (Time 2: mean age = 38.2 months, SD = 4.4 months) (Table 1). Although all toddlers were diagnosed with ASD at Time 1, five no longer met criteria for ASD at Time 2 but were instead diagnosed with other developmental problems. Diagnosis was based on the ADOS-T (Luyster et al., 2009) at the first visit and Autism Diagnostic Observation Schedule–Generic (ADOS-G), Modules 1 or 2 (Lord et al., 2000), at second visit and on the Mullen Scales of Early Development (MSEL; Mullen, 1995), the Communication and Symbolic Behaviors Scales (CSBS; Wetherby and Prizant, 1993), and the Vineland Adaptive Behavior Scales-II (VABS; Sparrow et al., 2005) as well as a review of medical and developmental history (see Chawarska et al., 2009, for details). In all, 62 of the children were boys (89%); 74% of the parents identified their child’s race as Caucasian; of the remainder, 4% were African-American, 4% Asian, and 18% of parents reported mixed race or did not report racial background.

Table 1.

Sample characterization.

	Time 1	Time 2
	Mean (SD)	Mean (SD)
N	70	70
Age (months)	21.9 (3.1)	38.2 (4.4)
VABS ABC SS	78.9 (8.6)	80.5 (12.7)
MSEL verbal DQ	46.7 (26.7)	72.6 (38.1)
MSEL nonverbal DQ	76.8 (18.7)	79.2 (25.5)
VABS ABC AE	17.3 (3.4)	30.7 (5.7)
MSEL verbal AE	10.3 (6.2)	27.8 (14.5)
MSEL nonverbal AE	16.7 (4.5)	30.1 (9.8)
ADOS-T total algorithm score	18.6 (5.0)	15.3 (6.2)

VABS ABC: Vineland Adaptive Behavior Scales, Adaptive Behavior Composite; SS: Standard Score, MSEL: Mullen Scales of Early Development; DQ: developmental quotient; AE: age equivalent, ADOS-T: Autism Diagnostic Observation Schedule–Toddler.

Another 10 toddlers were considered, but 8 were excluded due to technical errors in video recording, and 2 were excluded because of incomplete assessments. The excluded participants did not differ from the final sample in terms of clinical characteristics. Informed consent was obtained from parents, and the study was conducted in accordance with the University Human Investigation Committee.

The participants were recruited from states in the northeastern US, where toddlers diagnosed with ASD typically receive intensive, home-based intervention services provided by state-run intervention agencies. Information regarding the type and intensity of intervention was collected through a parent interview at Time 2. Between Time 1 and Time 2 visits (average time between being 16.3 months; SD = 4.6 months), all children for whom data were available (69 out of 70) received some kind of community-based treatment. The average weekly treatment intensity was 13.9 h/week (SD = 9.4). A total of 94% of children received speech-therapy (average 2 h/week), 77% received ABA (e.g. EIBI; average 10 h/week), and 36% received developmental interventions (e.g. ESDM; average 7 h/week). A total of 84% of children received more than one type of intervention. Again, although the therapeutic approaches differ, they share employing positive reinforcers to increase frequency of desirable behaviors (Koegel et al., 2008).

Assessments

Rates of skill acquisition

The dependent variables were skill acquisition rates (Bagnato and Neisworth, 1980; see below) for verbal and nonverbal domains as measured by the MSEL and VABS adaptive behavior composite (ABC; Sparrow et al., 2005). Rate of skill acquisition was computed as change in age equivalent (AE) between Time 1 and Time 2, divided by the time elapsed between the two assessments. By definition, a child who gains as many months in mental age as those elapsing between measurements would have a skill acquisition rate of 1, a child who exhibits no increase in AE over time has a skill acquisition rate of 0, whereas a child who exhibits absolute regression would have a negative rate. The main benefit of this approach is that it controls for differences in time between assessments. This approach was first suggested by Bagnato and Neisworth (1980) and was recently utilized by Klintwall and Eikeseth (2012). For an expanded discussion on the use of rate of skill acquisition (or learn rate) as an outcome measure, see Klintwall et al. (2013).

Interest level

To evaluate the level of interest in objects, social routines, and activities, we designed a novel scoring system, ILSA. The scoring system rates interest and curiosity in objects and activities presented during assessments such as the ADOS or CSBS, whose semistructured probes aim to elicit social and communicative behaviors. The probes are designed such that to “tempt” the child to initiate social and communicative bids to share interesting experiences or to request toys and activities that elicited their interests. Clinical observations suggest that while some children are interested in the presented objects and engage readily with the examiner, others exhibit a limited range of interests and consequently require marked effort on the examiner part to find motivating activities. In this study, we coded the child’s response during the standard administration of the ADOS-T, a play-based instrument used to assess autism symptoms (Luyster et al., 2009). During the ADOS-T administration, the examiner creates multiple opportunities for the child to initiate social behaviors such as joint attention, requesting, or shared enjoyment. The session starts with a free play probe, in which the examiner remains passive, while the child explores toys (vehicles, balls, cause/effect toys, etc.). This is followed by a set of structured probes, with the examiner offering tangibles (snack), activities (bubbles, a balloon), and social routines (playing peekaboo).

To quantify the child’s level of interest during the structured play session, ILSA includes 10 items and activities from the ADOS-T, selected based on whether they could be expected to function as reinforcers in an intervention and whether interest could be assessed irrespective of the child’s communication skills (e.g. ability to request verbally). Table 2 lists the items included in ILSA and the criteria for scoring the children’s interest in them. Every item was scored on a 5-point Likert scale, ranging from “no interest” (scored as 0) to “high interest” (scored as 4), with the total ILSA score representing the average across the 10 items.

Table 2.

Items and scoring criteria for ILSA during an ADOS-T assessment. Note that ILSA total score is the mean across all 10 items.

Stimulus	0. No interest	1. Little interest	2. Some interest	3. Interest	4. High interest
Pop-up toy	Ignores	Activates once or twice	Some activations (≥3)	Frequent activations (≥10) OR emotion	Frequent activations (≥10) AND never bores
Toy telephone	Ignores	A few activations (<3)	Some activations (≥3)	Frequent activations (≥10) OR emotion	Frequent activations (≥10) AND never bores
Vehicles	Ignores	Touches	Interacts by exploring, driving, repairing, or flying	Seeks out AND interacts extensively more than once	Seeks out, interacts extensively, AND never bores
Shape-sorter	Ignores	Less than two sorting attempts	Sorts or tries to sort (<10)	Sorts (≥10 blocks)	Sorts (≥10 blocks) AND never bores
Bubbles^a	Ignores	Watches briefly	Requests (<5) OR some attending OR touches	Frequent requests, (≥5), always attends when presented, OR emotion	Frequent requests (≥5) AND never bores of bubbles
Balloon^a	Ignores	Watches briefly	Some requests (<5) OR some attending OR touches	Frequent requests (≥5), always attends when presented, OR emotion	Frequent requests (≥5) AND never bores of the balloon
Edibles^a	Never tries	Tries one, may spit out	Eats two pieces or more	Requests OR eats all that are offered	Eats all offered AND never forgets about the snack before eating 10 pieces
Doll^a	Ignores	Brief interaction, such touches (<3)	Some interaction (e.g. carrying or pretend play)	Extended interaction OR emotion	Extended interaction AND never bores
Remote control toy	Ignores	Watches or touches only briefly	At least one request/activation OR touches (≥5) OR some attending	Requests/activations (≥5), attends when activated, OR emotion	Frequent requests or activations (≥5) AND never bores
Peekaboo	Ignores	Eye contact but no emotion	Emotion OR attempts to repeat	Emotion AND attempts to repeat (pulling blanket over own head)	Emotion AND attempts to repeat AND never bores

Also used to evaluate test–retest reliability using the CSBS.

The Time 1 ADOS-T video-recorded sessions were scored using ILSA by the first author (L.K.), blind to the Time 2 diagnosis and test score data. In total, 10 examiners conducted the ADOS assessments, and there were no significant differences among the mean interest level scores obtained from assessments across the different examiners (F(9, 60) = 1.03, p = 0.429). This suggests that the ADOS-T is sufficiently structured to enable comparisons between scores obtained with different examiners. To evaluate interrater reliability of ILSA, 25% of the videos were re-coded by a research assistant with experience working with toddlers with ASD. Interrater reliability was calculated using Cohen’s kappa with quadratic weighting, pooling together all item scores. This approach is particularly suitable for ordinal data (Ben-David, 2008). This analysis yielded an adequate level of interrater reliability: 0.86 (McHugh, 2012).

To evaluate whether the interest levels of individual children were stable over time, we also analyzed test–retest reliability for ILSA. This was achieved by comparing child responses on four stimuli that were also presented during a CSBS assessment for a random selection of 25% of the participants. The CSBS was conducted with participants the day after the ADOS assessment and with another examiner. The four items that were presented as part of both assessments were soap-bubbles, balloons, edibles, and dolls. Test–retest reliability was analyzed identically to the interrater reliability described above, by pooling together all item scores and computing a Cohen’s kappa with quadratic weighting (Ben-David, 2008). This yielded an adequate level of reliability: 0.66 (McHugh, 2012), meaning that child score on the ILSA reflects a somewhat stable trait.

Data analysis

To investigate whether the ILSA score predicted the rate of verbal, nonverbal, and adaptive skill acquisition beyond the levels of social, verbal, nonverbal, and adaptive functioning at Time 1 as well as treatment intensity between Time 1 and Time 2, we conducted three standard multiple linear regression analyses with rate of skill acquisition for VABS ABC and two composite measures for MSEL: verbal and nonverbal, as predicted variables. MSEL verbal domain was defined as the mean of the two subscales: Perceptual Language and Expressive Language. MSEL nonverbal domain was defined as the mean of the subscales: Gross Motor and Fine Motor. The following predictors were considered in the regression analyses: Time 1 ILSA score, ADOS-T total algorithm score, intensity of intervention between Time 1 and Time 2, and AE at Time 1 for VABS ABC and verbal and nonverbal domains in MSEL. Data were analyzed using SPSS 20.0.

Results

Mean rates of skill acquisition varied across the three dependent measures. For VABS ABC, it was found to be 0.83 (range: 0.36–1.42; SD) =) 0.23). For MSEL verbal, the mean rate of skill acquisition was 1.09 (range: −0.10 to 2.47; SD = 0.72). For MSEL nonverbal, the mean rate of skill acquisition was 0.84 (range: −0.08 to 1.88; SD = 0.45). Note the large individual variation in the rate of skill acquisition, evident in all three metrics. The interest level scores ranged from 1.0 to 3.3, with a mean of 2.2 (SD = 0.59). For distribution of interest level scores, see Figure 1.

Figure 1.

Distribution of ILSA scores across the 70 participants.

Results of the regression analyses indicate that ILSA score was a significant predictor of all three dependent variables (Table 3). With VABS ABC as the dependent variable, the regression model accounted for 41% of the variance, F(6, 62) = 7.227, p < 0.001. With MSEL verbal domain as the dependent variable, the regression model accounted for 42% of the variance, F(6, 62) = 7.425, p < 0.001. With MSEL nonverbal domain as the dependent variable, the regression model accounted for 33% of the variance, F(6, 62) = 6.643, p < 0.001.

Table 3.

Linear regressions predicting rate of skill acquisition between Time 1 and Time 2 in VABS and verbal and nonverbal MSEL (N = 70).

Predicted variable	Predictor (Time 1)	B	95% CI		Standard β	p	R²
Predicted variable	Predictor (Time 1)	B	Lower	Upper	Standard β	p	R²
VABS ABC	ILSA score	0.184	0.071	0.296	0.472	0.002	0.360(p < 0.001)
	VABS ABC AE	0.007	−0.005	0.019	0.153	0.257
	MSEL verbal AE	0.006	−0.005	0.017	0.158	0.283
	MSEL nonverbal AE	0.004	−0.014	0.021	0.076	0.664
	ADOS-T total score	0.008	−0.011	0.027	0.119	0.414
	Treatment intensity	−0.001	−0.007	0.004	−0.057	0.596
Verbal MSEL	ILSA score	0.433	0.082	0.784	0.354	0.016	0.304(p < 0.001)
	VABS ABC AE	−0.012	−0.050	0.027	−0.080	0.548
	MSEL verbal AE	−0.018	−0.052	0.016	−0.154	0.292
	MSEL nonverbal AE	0.054	−0.001	0.109	0.338	0.054
	ADOS-T total score	0.004	−0.056	0.065	0.021	0.885
	Treatment intensity	−0.008	−0.025	0.008	−0.108	0.311
Nonverbal MSEL	ILSA score	0.233	0.011	0.456	0.308	0.040	0.298(p < 0.001)
	VABS ABC AE	−0.009	−0.034	0.015	−0.104	0.446
	MSEL verbal AE	0.013	−0.009	0.034	0.173	0.250
	MSEL nonverbal AE	−0.009	−0.044	0.025	−0.096	0.587
	ADOS-T total score	0.015	−0.023	0.054	0.117	0.430
	Treatment intensity	−0.011	−0.021	0.000	−0.224	0.043	0.166(p = 0.001)

ILSA: Interest Level Scoring for Autism; VABS ABC: Vineland Adaptive Behavior Scales, Adaptive Behavior Composite; MSEL: Mullen Scales of Early Development; AE: age equivalent; ADOS-T: Autism Diagnostic Observation Schedule–Toddler.

CI = 95% confidence interval for B.

R² = explained variance when only this variable is included in the model.

ILSA score was the only significant predictor of rate of adaptive and verbal skill acquisition as measured by the VABS and the MSEL. For rate of nonverbal skill acquisition measured by the MSEL, interest level score was a significant predictor together with treatment intensity between Time 1 and Time 2. As a post hoc analysis, the proportion of explained variance for these significant predictors was analyzed for each significant predictor separately, as shown in the last column of Table 3. Identical analyses were conducted using change in standardized scores (developmental quotient (DQ) for MSEL, standard scores for VABS) as the dependent variable and Time 1 standard scores as predictors, with highly similar results.

Discussion

Longitudinal studies of children with ASD have documented considerable variability in outcomes. We found that child interest in toys, activities, and social routines during a semistructured social communication assessment is a strong predictor of the rate of skill acquisition in the domains of adaptive behavior and both verbal and nonverbal abilities from the second to the fourth year of life. These results extend the findings of Klintwall and Eikeseth (2012), employing a prospective research design, larger sample of children, direct observation, and additional measures of outcome. This study adds to the small body of literature documenting the role of object interest and width of interest both as an important child factor in responding to specific treatments (Sherer and Schreibman, 2005; Vivanti et al., 2012; Yoder and Stone, 2006) and for outcome in general (Bornstein et al., 2013). Notably, our results were similar to the findings by Klintwall and Eikeseth (2012), even though the children received a wider range of treatments. Note that positive reinforcement played an important role in all treatments that the children received. These results can be interpreted to suggest that positive reinforcement may be a critical treatment component, or active ingredient (Kasari, 2002) of interventions for children with autism, although the lack of control of what type of interventions children received limits the conclusions that can be drawn about ILSA a predictor of treatment outcome, as opposed to a predictor of outcome regardless of intervention.

In other words, it is quite possible that the findings indicate a general maturation predictor and not a predictor of treatment response. In fact, object exploration has been shown to be a predictor of subsequent development for typically developing infants (Bornstein et al., 2013). Also, because children received a mix of interventions of varying intensities, we do not know whether the ILSA predicts outcome in some types of interventions and not others, as were demonstrated in the study by Schreibman et al. (2011) predicting outcomes in PRT and DTT. For instance, it can be hypothesized that ILSA score would predict outcomes in EIBI (which relies heavily on contrived tangible reinforcers) but not in ESDM (which relies on social routines as reinforcers, which is only represented in the ILSA on one item: the peekaboo).

The ILSA score by itself, as shown in the post hoc analysis, explained a quite large proportion of the variation in developmental progress, ranging between 30% and 36% over the three outcome variables. For comparison, consider the predictors reported in the meta-analysis by Eldevik et al. (2010) who computed a regression model including intake VABS, IQ, and treatment intensity for 309 children who had received behavioral interventions. For change in VABS ABC, all predictors together predicted 20% of variance in outcome.

It has been hypothesized that behavioral interventions might be effective with children with autism partly because the treatment establishes more socially mediated reinforcers (Dawson et al., 2012). The lack of social interest children with autism exhibit (Chevallier et al., 2012) is, in behavioral interventions, “substituted” by motivation from contrived consequences (until social reinforcers are established). Children with deficits in social motivation may acquire or improve their language when communication is made contingent on other, nonsocial reinforcers. If stimuli that motivate a particular child are difficult to find, attempts could be made to help the child expand the repertoire of motivating toys or activities.

If indeed interest width is causally related to outcome for children with autism, it might be interesting to speculate about the possible clinical implications this might point to. To date, only a few techniques for expanding reinforcer repertoire has been described in the literature. One example is play-based interventions (Kasari et al., 2006; McDuffie et al., 2012). Another example is techniques for teaching children to eat disliked food. Preferred foods are presented as reinforcers for eating disliked food, and once these disliked food items become accepted, they can often be used as reinforcers for eating other disliked foods (Seiverling et al., 2011; Williams and Foxx, 2007). A final example is encouraging sampling of novel stimuli (Ayllon and Azrin, 1968), that is, encouraging the child to test new things and thus making the expansion of the reinforcer repertoire a treatment goal. Again, these clinical implications are preliminary.

This study indicates that our quick and easily mastered assessment of child interest in potential reinforcers, the ILSA, might be used to identify children with ASD who will not benefit as much from standard, community-based interventions as might be expected. Future research should investigate how interventions could be made more effective for these children. Interventions can perhaps be made more effective for children who make less improvement (“nonresponders”; Sherer and Schreibman, 2005) if, first, more emphasis is placed early in the intervention on finding stimuli that are interesting and motivating for the child. Simply spending some time exploring different stimuli and conducting extensive reinforcer assessments might be sufficient to expand the array of reinforcers to an acceptable size. Second, it might be possible to change what functions as reinforcers for the child’s behavior. In other words, future studies could attempt to increase the number of stimuli that can be used as reinforcers in treatment. As noted above, techniques for changing a nonreinforcing stimulus into an actual reinforcer already exist (Williams and Foxx, 2007). Whether such an expansion of the reinforcer repertoire leads to larger treatment effects could also be a topic of further study.

There are several strengths of our novel scoring system. First, codes in ILSA are clearly operationalized, and therefore, the system requires minimal training to implement. Indeed, our experience suggests that reliable coders can be recruited from a population of students or clinical staff with some background in child development and developmental disabilities but without advanced training. Furthermore, ILSA can be utilized in conjunction with semistructured play assessments including the ADOS and CSBS. This study made use of previously recorded sessions, but given its brevity and ease of mastery, it is likely that the scoring system could be employed on a live basis. It could also be easily adapted for use with other assessments.

Yet, the construct targeted by the scoring system appears broad. Future research will have to clarify directly to key issues. The first issue regards whether child interest in or enjoyment of the stimuli is a valid approximation of the reinforcing properties of these stimuli. A reinforcer is not defined by expressed emotion but by effect on the probability of the operant that consistently precedes it (Skinner, 1953). Yet rather unsurprisingly, studies have indicated that child preferences, even in low-functioning children, predict reinforcing properties of the selected objects (Green et al., 1988). Second, there is the possibility that ILSA quantifies something else other than interest or the reinforcing properties of the stimuli, such as social motivation (the child interacting with the objects as a means to gain adult attention). Finally, a low ILSA score might reflect more general deficits in regulation of attention and, therefore, more time spent in an inattentive state or time spent engaged in stereotypic behaviors.

Furthermore, considering that in a majority of cases, children are diagnosed later than in the second year of life, it will be necessary to examine whether the potential for motivation in older children also constitute predictor of gain in cognitive and adaptive skills. This study has two major limitations. First, the ADOS play probes differed in duration across assessments, so that the opportunity to obtain a high interest score on a given probe differed across children. This might have led examiners to spend more time trying to engage uninterested children, thus inflating their scores. Second, even though the ILSA scores appeared to be reasonably stable (as test–retest reliability was measured based on the CSBS administered on a separate day), child interest in the items likely varies across days. To address these issues, a standardized presentation procedure would have to be used, with assessments on different days. One possibility would be to present the stimuli contingent on an arbitrary response (such as a key-press) and measure response frequencies (Ewing et al., 2013). An interesting alternative would involve progressive ratio schedules (i.e. requiring more and more key-presses to obtain the reinforcer) and identifying response breakpoints (i.e. the amount of effort each reinforcer is “worth”; Jerome and Sturmey, 2008; Johnson and Bickel, 2006; Roane, 2008). This would likely be more time-consuming than the ILSA and could not be used as an add-on to the already widely used ADOS assessment.

To conclude, we have shown that within a group of toddlers with ASD, interest level for activities, toys, and social routines during a standardized assessment predicted developmental rate of verbal and nonverbal ability and adaptive functioning over the subsequent year. Our scoring system was created for use with semistructured social communication assessments. Also, it is brief and can be reliably scored with minimal training. Given the proposed role of interest in toys and activities as a moderator of treatment effectiveness, future studies could develop and evaluate techniques for expanding the repertoire of stimuli functioning as reinforcers for children who receive a low interest level score, thus possibly improving their outcomes.

Footnotes

Acknowledgements

Thanks are due to Eugenia Gisin, Judah Koller, Elizabeth Simmons, Sophy Kim, Daniel Campbell, Prakriti Gupta, and Suzanne Horwitz for essential assistance with the study.

Funding

This research was funded by the NIMH P50 MH081756 (Project 2, PI: KC), NIMH R01 MH087554 (PI: KC), and NICHD P01 HD003008 (Project 1, PI: KC).

References

American Psychiatric Association (2013) Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Publishing.

Ayllon

Azrin

(1968) Reinforcer sampling: a technique for increasing the behavior of mental patients. Journal of Applied Behavior Analysis 1(1): 13.

Bagnato

Jr Neisworth

(1980) The Intervention Efficiency Index: an approach to preschool program accountability. Exceptional Children 46(4): 264–269.

Ben-David

(2008) Comparison of classification accuracy using Cohen’s Weighted Kappa. Expert Systems with Applications 34(2): 825–832.

Bornstein

Hahn

C-S

Suwalsky

(2013) Physically developed and exploratory young infants contribute to their own long-term academic achievement. Psychological Science. Epub ahead of print 20 August. DOI: 10.1177/0956797613479974.

Charman

Baron-Cohen

Swettenham

. (2003) Predicting language outcome in infants with autism and pervasive developmental disorder. International Journal of Language & Communication Disorders 38(3): 265–285.

Charman

Taylor

Drew

. (2005) Outcome at 7 years of children diagnosed with autism at age 2: predictive validity of assessments conducted at 2 and 3 years of age and pattern of symptom change over time. Journal of Child Psychology and Psychiatry 46(5): 500–513.

Chawarska

Klin

Paul

. (2009) A prospective study of toddlers with ASD: short-term diagnostic and cognitive outcomes. Journal of Child Psychology and Psychiatry 50(10): 1235–1245.

Chevallier

Kohls

Troiani

. (2012) The social motivation theory of autism. Trends in Cognitive Sciences 16: 231–239.

10.

Cooper

Heron

Heward

(2007) Applied Behavior Analysis. Columbus, OH: Pearson/Merrill-Prentice Hall.

11.

Dawson

(2008) Early behavioral intervention, brain plasticity, and the prevention of autism spectrum disorder. Development and Psychopathology 20(3): 775–803.

12.

Dawson

Bernier

Ring

(2012) Social attention: a possible early indicator of efficacy in autism clinical trials. Journal of Neurodevelopmental Disorders 4(1): 11.

13.

Egel

(1981) Reinforcer variation: implications for motivating developmentally disabled children. Journal of Applied Behavior Analysis 14(3): 345–350.

14.

Eldevik

Hastings

Hughes

. (2010) Using participant data to extend the evidence base for intensive behavioral intervention for children with autism. American Journal on Intellectual and Developmental Disabilities 115(5): 381–405.

15.

Ewing

Pellicano

Rhodes

(2013) Using effort to measure reward value of faces in children with autism. PloS One 8(11): e79493.

16.

Flanagan

Perry

Freeman

(2012) Effectiveness of large-scale community-based intensive behavioral intervention: a waitlist comparison study exploring outcomes and predictors. Research in Autism Spectrum Disorders 6(2): 673–682.

17.

Green

Reid

White

. (1988) Identifying reinforcers for persons with profound handicaps: staff opinion versus systematic assessment of preferences. Journal of Applied Behavior Analysis 21(1): 31–43.

18.

Harris

Handleman

(2000) Age and IQ at intake as predictors of placement for young children with autism. Journal of Autism and Developmental Disorders 30(2): 137–142.

19.

Jerome

Sturmey

(2008) Reinforcing efficacy of interactions with preferred and nonpreferred staff under progressive-ratio schedules. Journal of Applied Behavior Analysis 41(2): 221–225.

20.

Johnson

Bickel

(2006) Replacing relative reinforcing efficacy with behavioral economic demand curves. Journal of the Experimental Analysis of Behavior 85(1): 73–93.

21.

Kang

O’Reilly

Lancioni

. (2013) Comparison of the predictive validity and consistency among preference assessment procedures: a review of the literature. Research in Developmental Disabilities 34(4): 1125–1133.

22.

Kasari

(2002) Assessing change in early intervention programs for children with autism. Journal of Autism and Developmental Disorders 32(5): 447–461.

23.

Kasari

Freeman

Paparella

(2006) Joint attention and symbolic play in young children with autism: a randomized controlled intervention study. Journal of Child Psychology and Psychiatry 47(6): 611–620.

24.

Kasari

Gulsrud

Freeman

. (2012) Longitudinal follow-up of children with autism receiving targeted interventions on joint attention and play. Journal of the American Academy of Child & Adolescent Psychiatry 51(5): 487–495.

25.

Klintwall

Eikeseth

(2012) Number and controllability of reinforcers as predictors of individual outcome for children with autism receiving early and intensive behavioral intervention. Research in Autism Spectrum Disorders 6(1): 493–499.

26.

Klintwall

Eikeseth

(2014) Early and Intensive Behavioral Intervention (EIBI) in autism. In: Patel

Preedy

Martin

(eds) The Comprehensive Guide to Autism. New York: Springer, pp. 117–137.

27.

Klintwall

Eldevik

Eikeseth

(2013) Narrowing the gap: effects of intervention on developmental trajectories in autism. Autism. Epub ahead of print 8 November. DOI: 10.1177/1362361313510067.

28.

Koegel

Fredeen

. (2008) Naturalistic behavioral approaches to treatment. In: Chawarska

Klin

Volkmar

(eds) Autism Spectrum Disorders in Infants and Toddlers. New York: Guilford Press, pp. 207–242.

29.

Lord

Risi

Lambrecht

. (2000) The Autism Diagnostic Observation Schedule–Generic. Journal of Autism and Developmental Disorders 30(3): 205–223.

30.

Luyster

Gotham

Guthrie

. (2009) The Autism Diagnostic Observation Schedule–Toddler module. Journal of Autism and Developmental Disorders 39(9): 1305–1320.

31.

Macari

Campbell

Gengoux

. (2012) Predicting developmental status from 12 to 24 months in infants at risk for autism spectrum disorder. Journal of Autism and Developmental Disorders 42(12): 2636–2647.

32.

McDuffie

Lieberman

Yoder

(2012) Object interest in autism spectrum disorder: a treatment comparison. Autism 16(4): 398–405.

33.

McHugh

(2012) Interrater reliability: the kappa statistic. Biochemia Medica 22(3): 276–282.

34.

Mullen

(1995) Mullen Scales of Early Learning. San Antonio, TX: Pearson.

35.

Reichow

Barton

Assouline

. (2012) Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD). Cochrane Database Systematic Review 10: CD009260.

36.

Roane

(2008) On the applied use of progressive-ratio schedules of reinforcement. Journal of Applied Behavior Analysis 41(2): 155–161.

37.

Schreibman

Dufek

Cunningham

(2011) Identifying moderators of treatment outcome for children with autism. In: Matson

Sturmey

(eds) International Handbook of Autism and Pervasive Developmental Disorders. New York, NY: Springer, pp. 295–305.

38.

Schreibman

Stahmer

Barlett

. (2009) Brief report: toward refinement of a predictive behavioral profile for treatment outcome in children with autism. Research in Autism Spectrum Disorders 3(1): 163–172.

39.

Seiverling

Williams

Ward-Horner

. (2011) Interventions to treat feeding problems in children with Autism Spectrum Disorders: a comprehensive review. In: Matson

Sturmey

(eds) International Handbook of Autism and Pervasive Developmental Disorders. New York: Springer, pp. 491–508.

40.

Sherer

Schreibman

(2005) Individual behavioral profiles and predictors of treatment effectiveness for children with autism. Journal of Consulting and Clinical Psychology 73(3): 525–538.

41.

Shic

Macari

Chawarska

(2014) Speech disturbs face scanning in 6-month-old infants who develop autism spectrum disorder. Biological Psychiatry 75(3): 231–237.

42.

Skinner

(1953) Science and Human Behavior. New York: The Free Press.

43.

Smith

Rogers

Dawson

(2008) The Early Start Denver Model: A Comprehensive Early Intervention Approach for Toddlers with Autism. Austin, TX: Pro-Ed.

44.

Sparrow

Cicchetti

Balla

(2005) Vineland Adaptive Behavior Scales (II). Livonia, MI: Pearson.

45.

Tonge

Bull

Brereton

. (2014) A review of evidence-based early intervention for behavioural problems in children with autism spectrum disorder: the core components of effective programs, child-focused interventions and comprehensive treatment models. Current Opinion in Psychiatry 27(2): 158–165.

46.

Vivanti

Dissanayake

Zierhut

. (2012) Brief report: predictors of outcomes in the Early Start Denver Model delivered in a group setting. Journal of Autism and Developmental Disorders 43: 1717–1724.

47.

Wetherby

Prizant

(1993) Profiling communication and symbolic abilities in young children. Communication Disorders Quarterly 15(1): 23–32.

48.

Williams

Foxx

(2007) Treating Eating Problems of Children with Autism Spectrum Disorders and Developmental Disabilities. 1st ed. Austin, TX: Pro-Ed.

49.

Yoder

Stone

(2006) Randomized comparison of two communication interventions for preschoolers with autism spectrum disorders. Journal of Consulting and Clinical Psychology 74(3): 426–435.