Effectiveness of Medication Combined with Intensive Behavioral Intervention for Reducing Aggression in Youth with Autism Spectrum Disorder

Abstract

Background:

The use of antipsychotic medications to treat aggression in youths with autism spectrum disorders (ASD) is based on open-label trials and efficacy studies. There are no studies examining the combined effectiveness of antipsychotic medications and intensive behavioral intervention (IBI) to treat aggression in ASD.

Methods:

Youths with ASD and aggressive behavior received IBI. Medication use remained stable during the study period and was coded into antipsychotic, mood-stabilizing, and nonstimulant attention-deficit/hyperactivity disorder (ADHD)/sleep medication classes. Analysis of covariance (ANCOVA) and survival analyses examined the effects of medication classes on the average number of aggressive behaviors and time to behavior plan success.

Results:

Thirty-two youths (mean age = 11.16, standard deviation [SD] = 3.31, range = 4–16 years, 75% male) with ASD received aggression reduction plans. Of these, 25 youths were taking at least one psychiatric medication (antipsychotic n = 18, mood stabilizing n = 10, and nonstimulant ADHD/sleep n = 12). Aggression dropped substantially following implementation of IBI (p < 0.001; d = 1.70). Antipsychotic medication use predicted significantly fewer sessions to achieve behavior plan success (χ²(1) = 5.67, p = 0.017; d = 0.93). No other medication classes influenced aggressive behavior (largest χ²(1) = 0.16, p = 0.694).

Conclusions:

Behavioral treatment combined with antipsychotic medication was the most effective approach to reducing aggressive behaviors in youths with ASD. Mood-stabilizing and nonstimulant ADHD/sleep medications did not contribute to aggression reduction.

Introduction

A ggression is a frequent and impairing symptom in individuals with autism spectrum disorder (ASD) (Matson et al. 2005; Dominick et al. 2007). There are a large number of treatment options available for reducing aggression in children with autism, but many have little to no evidence base (Goin-Kochel et al. 2007). Growing evidence supports the use of intensive behavioral intervention (IBI) (Matson et al. 1996; Howlin et al. 2009) or medications (McDougle 2002; Chez et al. 2004; Malone et al. 2005) to treat difficult behavior in children with ASD. To the authors' knowledge, no studies have examined the simultaneous application of these treatments toward reducing aggression in youths with autism.

Two decades of studies have supported the use of typical and atypical antipsychotic medications in the treatment of aggressive behavior in ASD (Anderson et al. 1984; Horrigan and Barnhill 1997; Posey et al. 1999; Vercellino et al. 2001; McCracken et al. 2002; Accardo 2003; Kratochvil et al. 2005; McDougle et al. 2005; Canitano 2006). However, risperidone was the first medication approved specifically for treatment of irritability and aggression in autism (Food and Drug Administration [FDA] approval, October 6, 2006). Findings supporting the use of risperidone, and antipsychotics more generally, include a combination of case reports (Posey et al. 1999; Sanchez et al. 1995; Gobbi and Pulvirenti 2007), open-label safety trials (Horrigan and Barnhill 1997; Vercellino et al. 2001; Luby et al. 2006), and efficacy studies (Anderson et al. 1984; McCracken et al. 2002; Accardo 2003; Kratochvil et al. 2005; McDougle et al. 2005). There are as yet no effectiveness studies that examine the benefit of antipsychotic medications relative to other medication classes or in conjunction with behavioral intervention in decreasing aggressive behavior in ASD.

Mood-stabilizing medications are not specifically indicated for the treatment of aggressive behavior in ASD (McDougle 2002). However, these medications are frequently prescribed, likely due to the dual efficacy of these medications in reducing emotional lability and maintaining seizure control, which are both frequently observed in ASD (Tuchman et al. 2009). Preliminary evidence has suggested possible phenotypic overlap of bipolar disorder (BD) and ASD (Kerbeshian et al. 1990; Munesue et al. 2008). This preliminary evidence coupled with the presence of significant aggression in youths with these disorders (Matson et al. 2005; Youngstrom et al. 2008) and the effectiveness of mood stabilizers in treating symptoms of pediatric BD (McClellan and Werry 1997; Kowatch et al. 2005) suggests that mood-stabilizing medications may have efficacy in treating aggressive behavior in ASD.

Nonstimulant medications, such as clonidine, have been frequently used to reduce attention-deficit/hyperactivity disorder (ADHD) symptoms, particularly hyperactive/impulsive behavior, and improve sleep difficulties in youths with ASD. However, these medications have not typically been considered a first-line treatment for aggressive behavior in ASD. The high prevalence of ADHD symptoms (Lee and Ousley 2006) and sleep difficulties (Allik et al. 2006; Malow et al. 2009) in ASD and the interfering nature of these symptoms for attending and learning may indicate a role for these medications in enhancing behavioral treatment of aggressive behavior. The present study used an effectiveness design to concurrently examine antipsychotic, mood-stabilizing, and nonstimulant ADHD/sleep medication classes in conjunction with behavioral intervention in the treatment of aggressive behavior in ASD.

IBI has a long history in the treatment of autism (Lovaas 1987). However, no group studies have examined the effect of IBI on reducing aggressive behavior in ASD. The majority of data comes from well-controlled, single-case designs demonstrating strong efficacy in reducing the frequency and intensity of aggressive behavior in individuals with ASD (Belcher 1995; Foxx and Meindl 2007; Matson et al. 2008). To the authors' knowledge, no previous studies have concurrently examined the effectiveness of IBI and the combination of IBI with medication in the treatment of aggression in individuals with autism. The combination of medication and psychosocial interventions has produced the best outcomes in some studies and conditions (e.g., March et al. 2007). For example, cognitive behavioral therapy plus medication was superior to either alone in the treatment of depression and obsessive compulsive disorder (OCD) in youths (POTS 2004; Wilhelm et al. 2008). In other cases, psychosocial interventions have not provided incremental effects in acute treatment when an appropriately structured medication regimen is already in place (e.g., Jensen et al. 2001). Combination therapy is recommended in several practice parameters for other conditions affecting youths when severity is extreme or when there is a threat of harm to self or others (e.g., McClellan et al. 2007). However, if the combination is not more efficacious, then it substantially increases the burden on the family, and potentially exposes the youths to unnecessary risks. Previous studies have focused on comparing efficacy using less intensive cognitive behavioral therapies. It would be valuable to determine whether the combination of IBI and medication produces better acute treatment response than either intervention in isolation.

The present study also provided a preliminary examination of moderators of aggression treatment response. Increasingly, ASDs are recognized as a complex set of etiologically heterogeneous phenotypes that will likely show substantial individual differences in treatment outcome (Geschwind 2007). Identifying modifiers of treatment response is useful for selecting subgroups most likely to benefit from treatment as well as individuals that may require shorter/longer, more/less intensive, or different forms of behavioral treatment. Determining moderators of treatment response may also facilitate the identification of subgroups of ASD cases for further genetic or biological pathway examination (e.g., pharmacogenomics).

The primary purpose of the present study was to examine the effects of adding IBI to medication in the treatment of aggressive behavior in youths with ASD. The study used an effectiveness design, which is important for demonstrating the generalizability of efficacy data, to determine the likely outcomes of various treatment approaches in real-world settings (Ernst and Pittler 2006; Gartlehner et al. 2006; MacDonald et al. 2006; Depp and Lebowitz 2007; Prochaska et al. 2007). Initiating IBI was predicted to substantially decrease aggressive behavior in youths with ASD (hypothesis 1). The time required to achieve behavior plan success was predicted to be lower in individuals taking antipsychotic, mood-stabilizing, or nonstimulant medications used to treat sleep difficulties and/or ADHD symptoms relative to those not taking medication (hypothesis 2).

An exploratory aim of this study was to determine whether other factors—demographics, baseline aggression, symptom levels, or behavior plan characteristics—moderated the treatment effect. Younger, verbal males with lower levels of baseline aggression and lower levels of stereotypy, irritability, and hyperactivity who received a differential reinforcement of other behavior (DRO) plan were predicted to show shorter times to behavior plan success (hypothesis 3). This hypothesis was based on previous moderator findings in the ASD and IBI treatment literatures (Lord et al. 1982; Lord and Schopler 1985; Eikeseth et al. 2002; Eldevik et al. 2006; Carter et al. 2007; Eikeseth et al. 2007; Magiati et al. 2007; Howlin et al. 2009).

Method

Participants

All youths were attending a center-based intensive behavioral intervention program in the midwestern United States that implemented the methods of applied behavioral analysis and served as the youths' primary education and treatment setting. Inclusion criteria were intentionally broad to enhance the generalizabilty of findings, consistent with recommendations for effectiveness studies (Ernst and Pittler 2006; Gartlehner et al. 2006). Individuals with a Diagnostic and Statistical Manual of Mental Disorders, 4^th edition (DSM-IV) (American Psyschiatric Association 1994) diagnosis of autistic disorder or pervasive developmental disorder not otherwise specified (PDD-NOS) were eligible for inclusion if they received an IBI plan targeting aggressive behavior. Clinical diagnoses were based on DSM-IV criteria, and all diagnosing clinicians had extensive experience evaluating ASD. Clinical diagnoses were also confirmed by the staff psychologist of the treatment center. Only 2 individuals met criteria for PDD-NOS, with the remaining individuals diagnosed with autistic disorder. All individuals had moderate to severe symptom levels and moderate to severely impaired adaptive skill levels based on the results of clinical evaluation and the observations of the staff psychologist.

Procedures

The present study was a retrospective review of the effectiveness of comprehensive medical and behavioral care for youths with ASD and aggression. Consecutive youths meeting criteria for an aggression behavior plan from 2000 to 2007 were included in this effectiveness study. Gartlehner and colleagues (2006) identified the following characteristics as accurately distinguishing efficacy and effectiveness designs: (1) The study sample is derived from specialized care facilities if a low prevalence disease or disorder is examined, (2) less stringent eligibility criteria, (3) focus on key long-term health-related outcomes, (4) longer study duration with clinically relevant treatment parameters, (5) assessment focused on only critical adverse events, (6) study powered to detect clinically relevant effects, and (7) intent-to-treat analyses. Study procedures met six of the seven characteristics. The only characteristic not present was assessment of adverse events. Although not directly probed, no caregivers reported adverse effects of the behavioral treatment and no participants discontinued medication due to adverse events during the study.

Each participant received IBI for ≥30 hours/week, comprised of at least 6 hours per day, 5 days per week year round. No participants were receiving any additional home-based or outside intensive behavioral interventions. Any participant with at least four aggressive behaviors per day and at least 1 day per week of multiple aggressive behaviors was eligible to be placed on an aggression behavior reduction plan. Aggressive behavior was defined as any behavior that either harms or attempts to harm another person or causes destruction of property. Aggressive behavior met at least one of two additional criteria: (1) Interfere with the ability to successfully participate in academic, socialization, daily living, and leisure activities; and/or (2) restrict access to community involvement. The average number of aggressive behaviors per session (equivalent to each school day) was recorded for 1 week prior to plan implementation (baseline) and for 3 weeks following plan implementation (weeks 1–3). To address hypothesis 1, average weekly aggressive behavior frequency counts for the first 4 weeks of treatment (baseline, weeks 1–3) were recorded. Interrater reliability was examined using frequency counts from two independent classroom behavior therapists supervised by a licensed clinical psychologist with expertise in functional behavioral assessment. Classroom therapists were blinded to other therapists' ratings. Interrater reliability for average weekly aggressive behavior frequency counts was excellent across all 4 initial weeks of the study (all intraclass correlations [ICCs] >0.85; range 0.86–0.95). Ongoing reliability checks were performed throughout the remaining treatment period and raters were required to continue to show high within-subjects reliability (ICCs >0.70) for all additional treatment weeks.

To address hypothesis 2, the total number of treatment sessions (corresponding to school days) required to achieve success was also recorded. The maximum number of recorded sessions was 500 or approximately 2–2.5 years of treatment. Behavior plan success was operationally defined as less than one aggressive behavior on average per session for five consecutive sessions and five consecutive sessions without a moderate to severely impairing aggressive episode. One participant exhibited only one moderate to severely impairing aggressive behavior per day, on average, during the baseline period. Exclusion of this participant would decrease the generalizability of findings and therefore the participant was included and the criterion for this participant was adjusted downward to no aggressive episodes per day for 5 consecutive days. Treatment sessions corresponded to school days, with the exception of school absences and weekends when behavior plans were not enforced.

Youths were prescribed medication by their treating physician, who was a pediatric neurologist, psychiatrist, or developmental pediatrician. The rating physicians had extensive clinical experience treating individuals with ASD for associated behavior or medical problems. No specific algorithm was used to select medications beyond existing practice guidelines for medication use in youths to be consistent with typical clinical practice. All participants were taking medication at least 2 weeks (>14 days) prior to initiation of a behavior plan. No participants changed medication type or other treatment during the behavior plan implementation period. Dose changes were permitted as recommended by the treating physician, and no youths with an IBI plan were excluded to maintain an effectiveness design. Medication use, but not dosing, was recorded for all participants. Seven participants were not taking any medication during the course of treatment.

Aggression was one of the primary reasons for medication use in the majority of youths (88%). In 3 children, medication was prescribed for ADHD symptoms, sleep-onset difficulties, and impulsive behavior, but not specifically aggression. In these 3 children, aggressive behavior emerged following medication treatment, but was determined by the treating physician to be unrelated to medication use. Additional reasons for medication prescription, beyond aggressive behavior, were common (72% of youths taking any medication) and included sleep-onset and maintenance difficulties, mood lability/BD symptoms, self-injurious behavior, seizure disorder, ADHD symptoms, and/or other impulsive behavior.

Three medication classes were coded based upon the pattern of specific medication usage observed in this sample: (1) Antipsychotics, (2) mood stabilizers, and (3) nonstimulant medications used to treat ADHD and/or sleep symptoms. No participants were receiving stimulant medications. Antipsychotics included risperidone (n = 12, 38%), aripiprazole (n = 5, 16%), ziprasidone (n = 1, 3%), clozapine (n = 1, 3%), molindone (n = 1, 3%), and olanzapine (n = 1, 3%). Three individuals were taking two antipsychotic medications (2, abilify + risperidone; 1, clozapine + molindone). Mood stabilizers were divalproex sodium (n = 6, 19%), lithium (n = 3, 9%), and lamotrigine (n = 1, 3%). Nonstimulant medications used to treat ADHD and/or sleep symptoms included clonidine (n = 11, 34.4%) and atomoxetine (n = 1, 3%). Post hoc analyses examined the individual effects of the most frequently prescribed medications, risperidone and clonidine, on aggressive behavior. Antidepressant medications were not included, as only a few participants were taking these medications, and some findings have suggested lack of efficacy or poor tolerability in the treatment of aggressive behavior in autism (McDougle 2002). Three youths received medications from all three classes. Six youths were taking both antipsychotics and mood stabilizers and 4 youths were taking both antipsychotics and nonstimulant medications. Power was inadequate for examining the effects of multiple medications in primary analyses. To maintain the effectiveness design, individuals taking multiple medicines were not excluded.

The Aberrant Behavior Checklist (ABC)–Community was completed by the youth's classroom teacher who had 30–35 hours per week of direct contact with the youth. Raw scores for the Stereotypy, Irritability, and Hyperactivity subscales were computed (Aman et al. 1985; Marshburn and Aman 1992). Scores for Lethargy and Inappropriate Speech were also computed but were not included, as no a priori hypotheses were made regarding these variables. The Vineland Adaptive Behavior Scales–Second Edition (VABS-II)–Teacher Rating Form was completed by the classroom teacher with whom the child spent >30 hours per week. Adaptive behavior composite scores were provided to evaluate functional level of the participants. The Institutional Review Board of the Cleveland Clinic approved this study.

Statistical analyses

Recent empirical work (Schafer and Graham 2002; Kang and Schafer 2007; Schafer and Kang 2008), in conjunction with longstanding, well-validated methods (Rubin 1974; Rosenbaum and Rubin 1983), has provided a theoretical and statistical framework for making inferences concerning treatment effects in observational treatment studies. This work has been crucial for analyzing effectiveness designs and has indicated that inferences from these designs may be tenable under conditions where the individuals receiving and not receiving treatment show sufficient overlap in the distributions of covariates potentially relevant to treatment assignment (e.g., age, gender, baseline frequency of aggressive behavior, etc.).

Several approaches were available to examine the level of overlap of covariate distributions. A succinct approach that has performed well in simulations is to use propensity scores (Schafer and Kang 2008). On the basis of this recommendation, propensity scores were computed by predicting the presence or absence of medication treatment for each medication class using logistic regression and saving predicted values. Predictors in these analyses included age, sex, verbal status (nonverbal or verbal, defined as ≥20 words used in meaningful communication), ABC subscales (Stereotypy, Irritability, and Hyperactivity), behavior plan type, and baseline frequency of aggressive behavior. Recent Monte Carlo simulations have shown that propensity scores alone typically do not adequately estimate causal effects (Kang and Schafer 2007; Schafer and Kang 2008). For this reason, when examining treatment effects, analyses included both propensity scores and any covariate that showed significant or marginally significant differences. A liberal significance level (p < 0.15) was used to determine whether group differences on potential covariates were sufficient to include the variable in the model with propensity scores. This represents a conservative approach to ensuring that variables showing any potential differences between individuals taking or not taking medications were included and is likely to substantially increase the accuracy of estimation of treatment effects (Schafer and Kang 2008). Independent samples t-tests (for polyvalued ordinal variables) and the Fisher exact test (for categorical variables) were computed to examine medication group differences in potential covariates.

Hypothesis 1 predicted that initiating IBI would decrease aggressive behavior in youths. To examine hypothesis 1, two approaches were used. First, a repeated-measures analysis of covariance (ANCOVA) model was computed with average number of aggressive behaviors per day for each week as the dependent variable. The three between-subjects independent variables in this model were the presence or absence of antipsychotic, mood-stabilizing, and nonstimulant ADHD/sleep medications. Each medication class was separately coded 0 = not present, 1 = present. The within-subjects variable was week with baseline, week 1, week 2, and week 3 as the four repeated measurements. Covariates included the propensity scores for each medication class and any covariates that differed between groups taking or not taking each medication class. Hypothesis 1 predicts a main effect of week, with post-IBI initiation weeks 1–3 showing a strong downward trend relative to baseline in the frequency of aggressive behaviors. To examine this downward trend, the contrast comparing baseline to weeks 1–3 was computed. In addition, due to the possibility of an extinction burst, a transient increase in aggression frequency with the implementation of the behavior plan, linear, quadratic, cubic, and repeated contrasts were also computed to determine the exact shape of any trend. The multivariate interactions between medication Class IV and week were also examined to determine whether individuals using any of the medication classes showed decreased aggressive behavior frequency relative to individuals not using that medication class.

The second approach was to compute a piecewise growth curve model with baseline and weeks 1–2 as the first estimated growth (slope1) parameter, weeks 1–3 without baseline as the second estimated growth (slope 2) parameter, and medication classes as separate exogenous variables predicting the intercept (baseline value) and linear growth (slope 1 and 2) parameters. This model extended the previous ANCOVA analysis by examining whether medications primarily exert their effect on aggression from baseline to week 1 or from week 1 to week 3. The piecewise growth model took advantage of newer trends in statistical analytic strategies and had greater power to detect change over time, whereas the first approach was more likely to be familiar to a broad audience.

Hypothesis 2 predicted that the time to aggression behavior plan success would be shorter in individuals taking antipsychotic, mood-stabilizing, or nonstimulant ADHD/sleep medications. To examine this assertion, Kaplan–Meier curves and Cox regression analyses were computed with medication classes (antipsychotics, mood stabilizers, and nonstimulant ADHD/sleep medications) as predictors, number of sessions until discontinuation of the behavior plan as the time variable, and success of the behavior plan as the end point/status variable. Separate Kaplan–Meier curves were computed for each medication class and Tarone–Ware chi-squared statistics were used to determine significance (p < 0.05). A Cox regression was also computed with the presence or absence of all three medication classes included as separate predictors to determine independent effects of each medication class controlling for the effects of other medication classes.

Exploratory hypothesis 3 predicted that younger age, verbal capacity, male gender, receiving a DRO behavior plan, and having lower levels of aggressive behavior and lower levels of stereotypy, irritability, and hyperactivity at baseline would show reduced time to IBI success. Cox regression analysis using each of these variables as simultaneous predictors was used to examine the impact of these potential moderators.

Results

Sample characteristics and power

Thirty-two youths with ASD (mean age = 11.16, standard deviation [SD] = 3.31; range = 4–16, 75% male) received a behavior reduction plan targeting aggression. Hypothesis 1: Based upon n = 32, with four repeated-measures observations and an average correlation among repeated measures of r = 0.50, post hoc estimates indicated excellent power (0.92; α = 0.05, two-tailed) for detecting changes in aggression behavior frequency across weeks and for detecting week by medication class interactions. Hypotheses 2 and 3: Based upon a total n = 32, statistical power was >80% to detect hazard ratios of 3.0 or larger, equivalent to participants in the medication groups remaining on the behavior plan for a median of 41.2 sessions versus 121.5 sessions for individuals not taking medication (Borenstein et al. 2001).

IBI plan characteristics

Table 1 provides behavior plan characteristics by presence or absence of antipsychotic, mood-stabilizing, and ADHD symptom medication classes. The majority of children received differential reinforcement of alternative (DRA) behavior plans (DRA, n = 22, 69%; DRO, n = 10; 31%). The time schedule for DRA/DRO plans was implemented with an initial constant interval of reinforcement delivery, with an increase to longer intervals and a shift from fixed to variable interval schedules to increase stability of behavior acquisition. The most common consequences resulting from the display of an aggressive behavior were a physical hold (n = 12, 38%) and physical redirection (n = 10, 31%). These were used to reduce potential harm to the youth and/or therapists.

Table 1.

Number (%) of Participants with a Specific Behavior Plan Type by Presence or Absence of Medication Classes

	Any antipsychotic		Any mood stabilizer		Any ADHD medication
	No	Yes	No	Yes	No	Yes
Behavior plan type	n = 14	n = 18	n = 22	n = 10	n = 20	n = 12
Successful plan	10 (71)	17 (94)	19 (86)	8 (80)	16 (80)	11 (92)
Reinforcement type
DRA	10 (71)	12 (67)	17 (77)	5 (50)	13 (65)	9 (75)
DRO	4 (29)	6 (33)	5 (23)	5 (50)	7 (35)	3 (25)
Consequences
Physical hold	4 (13)	8 (25)	9 (41)	3 (30)	6 (30)	6 (50)
Physical redirection	4 (13)	6 (18)	5 (23)	5 (50)	5 (25)	5 (42)
Token economy	1 (3)	5 (16)	3 (14)	3 (30)	4 (20)	2 (17)
Time out	3 (9)	2 (6)	4 (18)	1 (10)	4 (20)	1 (8)
Compliance drills	3 (21)	1 (6)	3 (14)	1 (10)	4 (20)	0 (0)
Extinction	2 (6)	1 (3)	2 (9)	1 (10)	2 (10)	1 (8)
Visual screen	1 (3)	0 (0)	1 (5)	0 (0)	1 (5)	0 (0)

Nine participants had two consequences used jointly. Time out was nonexclusionary. All Fisher exact tests nonsignificant p values >0.100.

Abbreviations: ADHD = Attention-deficit/hyperactivity disorder; DRA, differential reinforcement of alternative; DRO, differential reinforcement of other.

Medication class covariate overlap

Table 2 presents demographics; mean raw scores on the Stereotypy, Irritability, and Hyperactivity scales from the ABC; and natural log-transformed median baseline aggression frequency; separately by the presence or absence of medication classes. The natural log was used to normalize the distribution of aggression frequencies, because observed scores were moderately skewed and leptokurtotic. The only medication group differences meeting the significance level p < 0.15 were for age (p = 0.027), verbal status (p = 0.098), and irritability (p = 0.025) between individuals taking and not taking ADHD medications. These variables were used as covariates with propensity scores in subsequent models.

Table 2.

Sample Characteristics Including Demographics, Autism Symptoms, and Pre- and Postbehavior Plan Initiation Aggression Frequencies, Separately by the Presence or Absence of Medication Classes

	Any antipsychotic		Any mood stabilizer		Any ADHD medication
	No M (SD)	Yes M (SD)	No M (SD)	Yes M (SD)	No M (SD)	Yes M (SD)
N (%)	14 (43.8)	18 (56.2)	22 (68.8)	10 (31.2)	20 (62.5)	12 (37.5)
Age	9.7 (3.4)	12.3 (2.9)^a	10.7 (3.6)	12.1 (2.6)	11.4 (3.6)	10.8 (2.9)
Sex (N, % male)	9 (64)	15 (83)	15 (68)	9 (90)	16 (80)	8 (67)
Race (N, % white)	12 (85.7)	13 (72.2)	17 (77.3)	8 (80.0)	17 (85.0)	8 (66.7)
Nonverbal (N, %)	3 (21)	9 (50)^b	7 (32)	5 (50)	6 (30)	6 (50)
Adaptive Functioning–VABS (SS)	50.8 (10.1)	41.1 (13.0)	48.3 (12.0)	40.8 (11.9)	47.2 (12.7)	41.7 (12.2)
Baseline aggression^d	4.1 (1.9)	4.0 (1.6)	3.7 (1.7)	4.2 (1.9)	4.0 (1.7)	3.5 (1.8)
ABC–Irritability	24.5 (10.7)	27.1 (7.8)	25.0 (8.7)	28.0 (10.2)	29.1 (6.4)	20.7 (10.7)^c
ABC–Stereotypy	12.3 (5.8)	10.6 (5.8)	12.1 (5.7)	9.6 (5.9)	12.3 (5.9)	9.8 (5.4)
ABC–Hyperactivity	25.2 (12.6)	23.1 (8.0)	24.9 (11.0)	22.0 (8.0)	26.3 (10.5)	20.3 (8.7)

p = 0.027, ^b p = 0.098, ^c p = 0.025, all other p values >0.10.

Baseline aggression is reported as the median of natural log transformed number of aggressive behaviors per day at baseline.

Abbreviations: ADHD = Attention-deficit/hyperactivity disorder; SD = standard deviation; VABS = Vineland Adaptive Behavior Scale (teacher report form) Adaptive Behavior Composite; SS = Standard Score (M = 100, SD = 15).

Logistic regression analyses used to compute propensity scores indicated only marginally significant prediction of antipsychotic usage (Nagelkerke R ² = 0.47, χ²[7] = 13.71, p = 0.057). Logistic regression analyses predicting mood stabilizer and ADHD medication usage were not significant (largest Nagelkerke R ² = 0.37, χ²[7] = 9.98, p = 0.189). This demonstrates minimal group differences on potential covariates. As would be expected with minimal group differences on covariates, propensity score distributions showed substantial overlap, making investigation of group differences tenable.

Initiating IBI

Inspection of Fig.1, A–C, indicates that average aggressive behavior rates were high in the 5 days preceding behavior plan implementation (baseline period), with a strong downward trend following behavior plan initiation. Comparison of baseline levels across medication classes indicated a slightly lower level of aggression frequency for individuals taking antipsychotics, a roughly equivalent level for individuals taking mood stabilizers, and slightly higher levels for individuals taking nonstimulant ADHD/sleep medications relative to individuals not taking these medications. Aggressive behavior dropped significantly in all medication groups following implementation of the behavior plan (linear contrast F[1,25] = 18.05, p < 0.001). This decrease was substantial (Cohen d = 1.70). Repeated contrasts indicated successive decreases across all remaining time points (p values <0.001), and the cubic contrast (p = 0.006) indicated a flattening of these decreases over time with the largest decrease from baseline to week 1 and the smallest from week 2 to week 3. The interaction between antipsychotic usage and treatment week was marginally significant (F[3,18] = 2.66, p = 0.079); the presence of antipsychotic medication and behavioral treatment resulted in larger decreases in aggressive behavior than IBI alone. No other medication class by week interactions were significant (largest F[3,18] = 1.00, p = 0.414).

FIG. 1.

Changes in the frequency of aggressive behavior from baseline to week 3 of IBI treatment, separately for antipsychotic (A), mood-stabilizing (B), and nonstimulant ADHD/sleep (C) medication groups.

To investigate further the marginally significant trend of antipsychotic medication decreasing aggressive behavior in the presence of behavioral intervention, a piecewise growth curve model was computed. This model indicated a significant effect of antipsychotic medication on the second slope parameter (standardized estimate = −0.51, p = 0.034), indicating that the effect of antipsychotic medication in reducing aggression occurred later in the treatment period after the large initial reduction due to IBI initiation. No other significant relationships between medication classes and baseline (intercept) or linear growth (slopes) were observed (smallest p = 0.167).

Medication classes as predictors of time to behavior plan success

Figure 2 presents the survival curves for the presence versus absence of antipsychotic (Fig. 2A), mood stabilizing (Fig. 2B), and nonstimulant ADHD/sleep (Fig. 2C) medications. The presence of antipsychotic medication significantly decreased the number of sessions required to achieve behavior plan success (χ²[1] = 5.67, p = 0.017; no antipsychotic, mean = 228.1, standard error [SE] = 52.9, median = 149.0, SE = 60.8, 95% confidence interval [CI] = 29.8–268.2; antipsychotic, mean = 83.2, SE = 29.6, median = 30, SE = 4.2, 95% CI = 21.7–38.3). The presence of mood-stabilizing medication or nonstimulant ADHD/sleep medication did not significantly influence time to behavior plan success (χ²[1] = 0.16, p = 0.694 and χ²[1] = 0.08, p = 0.783, respectively). Cox regression including all three medication classes as predictors confirmed the above pattern (antipsychotic β = −2.01, SE = 0.67, χ²[1] = 9.04, p = 0.003), even when covariates (propensity scores, age, verbal status, irritability, and stereotypy) were excluded from the model (p = 0.007). This pattern also held when only individuals using antipsychotic monotherapy were compared to individuals not taking an antipsychotic (β = −1.38, SE = 0.62, χ²[1] = 4.95, p = 0.026).

FIG. 2.

Number of sessions to achieve success for aggression behavior plans by antipsychotic (A), mood stabilizer (B), and nonstimulant ADHD/sleep (C) medication classes.

Post hoc Cox regression analyses separately examined the presence versus absence of risperidone and clonidine, which are the most frequently used medications within the antipsychotic and nonstimulant ADHD/sleep medication classes. Consistent with the above findings, risperidone showed a significant prediction of time until behavior plan success (β = −1.33, SE = 0.56, χ²[1] = 5.74, p = 0.017), but clonidine did not (β = −0.11, SE = 0.45, χ²[1] = 0.06, p = 0.808). Post hoc Cox regression also indicated that aggression treatment was less effective for individuals not taking any medication (n = 7) versus individuals taking one or more medications (n = 25) (Fig. 3; β = −1.15, SE = 0.55, χ²[1] = 4.39, p = 0.036).

FIG. 3.

Number of sessions to achieve behavior plan discontinuation for youths using versus not using medicine.

Exploration of additional predictors of behavior plan success

Figure 4 presents the number of sessions needed to achieve behavior plan success by sex (Fig. 4A) and verbal status (Fig. 4B). Younger, verbal males with higher baseline levels of aggression and irritability showed significantly decreased time to IBI plan success (sex β = 1.74, SE = 0.67, χ²[1] = 6.75, p = 0.009; age β = 0.37, SE = 0.11, χ²[1] = 12.27, p < 0.001; verbal status β = −2.11, SE = 0.63, χ²[1] = 11.29, p = 0.001; ln average baseline aggression β = −0.59, SE = 0.18, χ²[1] = 11.15, p = 0.001; ABC–Irritability β = −0.08, SE = 0.04, χ²[1] = 3.90, p = 0.048). Behavior plan type (DRA vs. DRO) and levels of stereotypy and hyperactivity did not predict time to behavior plan success (largest χ²[1] = 0.40, p = 0.528).

FIG. 4.

Number of sessions to achieve behavior plan discontinuation by sex (A) and verbal/nonverbal (B) status.

Discussion

The present study was unique in that no previous studies have examined the combined effectiveness of medication and IBI in treating aggression in youths with ASD. Furthermore, no previous studies have concurrently evaluated the relative effectiveness of different medication classes in treating aggression in youths with ASD.

Antipsychotic medication combined with IBI substantially decreased time to behavior plan success, and this effect extended to individuals receiving monotherapy. Aggression frequency remained high during the baseline period in individuals taking and not taking antipsychotic medication, but then dropped dramatically during week 1. Antipsychotic medication alone did not produce large, clinically meaningful reductions during the baseline period in these impaired youths, although effects may have been present prior to study initiation. However, even in youths who did not show a strong initial effect of antipsychotic medication in reducing aggression, the combination of IBI and antipsychotic medication was crucial to promoting the most positive outcomes. The effect of antipsychotic medication appears to be most prominent after the large initial reductions in aggression resulting from IBI initiation, suggesting an ongoing effect of antipsychotic medications in promoting aggression reduction.

IBI alone, while showing substantial decreases in aggressive behavior, was more effective when paired with antipsychotic medication. These findings extend previous efficacy data for antipsychotics (McCracken et al. 2002; Kratochvil et al. 2005; McDougle et al. 2005) by demonstrating effectiveness in augmenting behavior therapy in the treatment of aggression. Furthermore, results of the present study suggest that a combined antipsychotic–IBI treatment approach warrants further exploration in larger-scale efficacy and effectiveness studies.

Mood-stabilizing medications and nonstimulant medications used to treat ADHD symptoms and/or sleep difficulties (primarily clonidine) did not improve the effectiveness of IBI during the course of study. Although these medications may have had some effect in reducing aggression prior to initiation of the study and may play a role in treating other symptoms in youth with ASD, the present results indicate limited effectiveness for augmenting behavior therapy in the management of aggression. It should be noted that medication classes included a diverse array of compounds, so it is possible that an individual compound's effectiveness was diminished through aggregation. Future studies using even larger sample sizes will be needed to examine individual medications and combinations of medications. At minimum, the present results suggest greater synergy between antipsychotic treatment and IBI than for the other two medication classes.

Initiating IBI resulted in rapid decreases in the frequency of aggressive behavior, achieving statistical and clinical significance in the first week of treatment. The magnitude of this effect was consistent with previous case studies (Belcher 1995; Foxx and Meindl 2007). Together, these findings indicate that IBI should be an important component of an aggression treatment package. Practitioners in specialty care settings who implement IBI plans to treat aggression in youths with autism are likely to observe substantial effectiveness for the majority of youths. Interestingly, there was no evidence of an extinction burst at the group level, indicating that behavioral treatment did not result in initial resistance to behavior change in the majority of youths. This is consistent with the notion that aggression in many youths with ASD is a reaction to frustration and difficulty with appropriately communicating affect state or identifying alternative reactions to frustrating experiences (Foxx and Meindl 2007). Future efficacy and effectiveness studies will be helpful for identifying the most efficient and effective combinations of behavioral methods. In future research, it will be useful to examine the frequency, severity, and function of aggressive behavior, as well as alternative teaching approaches for replacing aggression with functionally equivalent behaviors.

The present study also provided novel data concerning the average and median session numbers required to reduce aggressive behaviors to minimal levels. Results indicated that meaningful reduction often takes considerable time, with median session numbers >30. Sessions required for meeting aggression reduction criterion ranged from 5 to 411 for the 84% of individuals achieving success. This tremendous range substantiates the need for persistence and patience in implementing and sustaining a behavior treatment plan for many children.

Exploratory moderator analyses found that the specific form of the plan (DRA vs. DRO) does not appear to be an important variable in determining IBI plan success. However, the present study did not randomly assign youths to different plan types; it is possible that one type of plan is more efficacious. Additionally, results suggested that older, nonverbal youths, particularly females, may require longer intervention periods to maintain sustained reductions in aggression. A subset of younger males with functional language and more severe baseline levels of aggression and irritability showed the largest decreases and shortest time to IBI plan success. The latter findings are counterintuitive but may suggest a unique subgroup of youths with autism who quickly learn response–reinforcement contingencies when structured and presented in an intensive fashion. Although very preliminary, qualitative review of these cases indicates that these individuals benefited from a behavior plan that taught them to use verbal communication in response to antecedent stimuli that would otherwise trigger aggressive behavior.

Limitations and Future Directions

The present study was limited by lack of random assignment to medication classes or behavior plan types. Blinded, random assignment studies examining medication and IBI plan types will be useful for determining whether individual or combined medication/IBI treatment packages are most efficacious. Randomized assignment to medication treatment groups will be helpful for determining the differential causal effects of antipsychotics or other medication classes in augmenting intensive behavior therapy. However, conducting efficacy studies in youths with moderate to severe ASD may result in samples that are poorly representative of the larger population, because many individuals with ASD are taking multiple medications and would be excluded from efficacy studies. The present effectiveness study had the advantage of examining multiple medication classes in a realistic clinical scenario where some individuals will not be taking medicine and others will be taking multiple medications to treat different symptoms or co-morbidities. The observation that antipsychotic medications showed substantial effectiveness in augmenting the effects of IBI in an unselected clinical sample suggests that these results are likely to generalize to other clinical settings and samples.

The present study was also limited by alterations in dosing during the course of treatment and the possibility of confounding medication class effects for individuals on multiple medications. The latter is likely a problem for future effectiveness studies where typical clinical practice is modeled. These possibilities could not be disentangled in the present study because the sample size was too small to a priori examine monotherapy or combinations of medications. Larger samples, diagnosed using gold-standard measures such as the Autism Diagnostic Interview–Revised or the Autism Diagnostic Observation Scale, will be needed to conduct finer-grained examinations of medication and IBI plan effectiveness in youths with ASD.

Carefully examining the effectiveness of combinations of medications is a promising future research avenue. Individuals with ASD often show multiple behavioral difficulties that may contribute to aggressive behavior—inattention, impulsivity, sleep loss, irritability/mood dysregulation, etc. Given the finding that the presence of functional communication/language was a predictor of behavior plan success, future studies may also wish to investigate the use of augmentative communication devices or techniques for improving the outcome of IBI and medication treatment of aggression. However, subgroup sample sizes and the fact that subjects did not receive a gold-standard diagnostic assessment instrument limit conclusions regarding moderators of treatment response. For example, only 8 female and 12 nonverbal participants were included in moderator analyses. Conclusions regarding moderators should be viewed as tentative and will require replication in larger samples prior to influencing clinical practice.

Overall, results indicate that IBI produces large treatment effects by itself, and augmentation with atypical antipsychotics, but not mood stabilizers or nonstimulant ADHD medications, significantly improves the average treatment response in youths with ASD. Future effectiveness studies are needed to examine the generalizability of efficacy findings for medication treatments to larger and broader clinical samples and settings.

Footnotes

Disclosures

Dr. Frazier served as a statistical consultant to Shire Development Inc. for a post hoc analysis of quality-of-life data for a Daytrana classroom treatment of ADHD study. Dr. Frazier was not involved in the design or conduct of the study and maintained scientific control during the drafting and revision of results of these secondary data analyses. Drs. Youngstrom, Sinoff, and Knapp and Mr. Haycook, Ms.Dimitriou, and Ms. Sinclair have no conflicts of interest or financial ties to disclose.

Acknowledgments

We gratefully acknowledge the youths who participated in this study.

Location of Work: Center for Autism, The Cleveland Clinic.

References

Accardo

. Risperidone in children with autism and serious behavioral problems. J Pediatr, 142:86–87. 2003.

Allik

, Larsson

, Smedje

. Sleep patterns of school-age children with Asperger syndrome or high-functioning autism. J Autism Dev Disord, 36:585–595. 2006.

Aman

, Singh

, Stewart

, Field

. The Aberrant Behavior Checklist: A behavior rating scale for the assessment of treatment effects. Am J Ment Defic, 89:485–491. 1985.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4 ^th (DMS-IV) Washington (DC): American Psychiatric Association, 1994.

Anderson

, Campbell

, Grega

, Perry

, Small

, Green

. Haloperidol in the treatment of infantile autism: Effects on learning and behavioral symptoms. Am J Psychiatry, 141:1195–1202. 1984.

Belcher

. Behavioral treatment vs. behavioral control: A case study. J Devel Phys Disabil, 7:235–241. 1995.

Borenstein

, Rothstein

, Cohen

, Schoenfeld

, Berlin

. Power, Precision, 2.1. New York: Biostat, Inc., 2001.

Canitano

. Self injurious behavior in autism: Clinical aspects and treatment with risperidone. J Neural Transm, 113:425–431. 2006.

Carter

, Black

, Tewani

, Connolly

, Kadlec

, Tager-Flusberg

. Sex differences in toddlers with autism spectrum disorders. J Autism Dev Disord, 37:86–97. 2007.

10.

Chez

, Memon

, Hung

. Neurologic treatment strategies in autism: An overview of medical intervention strategies. Semin Pediatr Neurol, 11:229–235. 2004.

11.

Depp

, Lebowitz

. Clinical trials: Bridging the gap between efficacy and effectiveness. Int Rev Psychiatry, 19:531–539. 2007.

12.

Dominick

, Davis

, Lainhart

, Tager-Flusberg

, Folstein

. Atypical behaviors in children with autism and children with a history of language impairment. Res Dev Disabil, 28:145–162. 2007.

13.

Eikeseth

, Smith

, Jahr

, Eldevik

. Intensive behavioral treatment at school for 4- to 7-year-old children with autism. A 1-year comparison controlled study. Behav Modif, 26:49–68. 2002.

14.

Eikeseth

, Smith

, Jahr

, Eldevik

. Outcome for children with autism who began intensive behavioral treatment between ages 4 and 7: A comparison controlled study. Behav Modif, 32:264–278. 2007.

15.

Eldevik

, Eikeseth

, Jahr

, Smith

. Effects of low intensity behavioral treatment for children with autism and mental retardation. J Autism Dev Disord, 36:211–224. 2006.

16.

Ernst

, Pittler

. Efficacy or effectiveness? J Intern Med, 260:488–490. 2006.

17.

Foxx

, Meindl

. The long term successful treatment of the aggressive/destructive behaviors of a preadolescent with autism. Behav Intervent, 22:83–97. 2007.

18.

Gartlehner

, Hansen

, Nissman

, Lohr

, Carey

. A simple and valid tool distinguished efficacy from effectiveness studies. J Clin Epidemiol, 59:1040–1048. 2006.

19.

Geschwind

. Autism: Searching for coherence. Biol Psychiatry, 62:949–950. 2007.

20.

Gobbi

, Pulvirenti

. Long-term treatment with clozapine in an adult with autistic disorder accompanied by aggressive behavior. J Psychiatry Neurosci, 26:340–341. 2001.

21.

Goin-Kochel

, Myers

, MacKintosh

. Parental reports on the use of treatments and therapies for children with autism spectrum disorders. Res Autism Spectr Disord, 1:195–209. 2007.

22.

Horrigan

, Barnhill

. Risperidone and explosive aggressive autism. J Autism Dev Disord, 27:313–323. 1997.

23.

Howlin

, Magiati

, Charman

. Systematic review of early intensive behavior interventions for children with autism. Am J Intellect Develop Disabilities, 114:23–41. 2009.

24.

Jensen

, Hinshaw

, Swanson

, Greenhill

, Conners

, Arnold

, Abikoff

, Elliott

, Hechtman

, Hoza

, March

, Newcorn

, Severe

, Vitiello

, Wells

, Wigal

. Findings from the NIMH Multimodal Treatment Study of ADHD (MTA): Implications and applications for primary care providers. J Dev Behav Pediatr, 22:60–73. 2001.

25.

Kang

, Schafer

. Demystifying double robustness: A comparison of alternative strategies for estimating population means from incomplete data. Statist Sci, 26:523–539. 2007.

26.

Kerbeshian

, Burd

, Randall

, Martsolf

, Jalal

. Autism, profound mental retardation and atypical bipolar disorder in a 33-year-old female with a deletion of 15q12. J Ment Defic Res, 34,Pt 2:205–210. 1990.

27.

Kowatch

, Fristad

, Birmaher

, Wagner

, Findling

, Hellander

. Treatment guidelines for children and adolescents with bipolar disorder: Child Psychiatric Workgroup on Bipolar Disorder. J Am Acad Child Adolesc Psychiatry, 44:213–235. 2005.

28.

Kratochvil

, Findling

, McDougle

, Scahill

, Hamarman

. Pharmacological management of agitation and aggression in an adolescent with autism. J Am Acad Child Adolesc Psychiatry, 44:829–832. 2005.

29.

Lee

, Ousley

. Attention-deficit hyperactivity disorder symptoms in a clinic sample of children and adolescents with pervasive developmental disorders. J Child Adolesc Psychopharmacol, 16:737–746. 2006.

30.

Lord

, Schopler

. Differences in sex ratios in autism as a function of measured intelligence. J Autism Dev Disord, 15:185–193. 1985.

31.

Lord

, Schopler

, Revicki

. Sex differences in autism. J Autism Dev Disord, 12:317–330. 1982.

32.

Lovaas

. Behavioral treatment and normal educational and intellectual functioning in young autistic children. J Consult Clin Psychol, 55:3–9. 1987.

33.

Luby

, Mrakotsky

, Meade-Stalets

, Belden

, Heffelfinger

, Williams

, Spitznagel

. Risperidone in preschool children with autistic spectrum disorders: An investigation of safety and efficacy. J Child Adolesc Psychopharmacol, 16:575–587. 2006.

34.

MacDonald

, Vancayzeele

, Deblander

, Abraham

. Longitudinal observational studies to study the efficacy-effectiveness gap in drug therapy: Application to mild and moderate dementia. Nurs Clin North Am, 41:105–117 vii 2006.

35.

Magiati

, Charman

, Howlin

. A two-year prospective follow-up stduy of community-based early intensive behavioral intervention and specialist nursery provision for children with autism spectrum disorders. J Child Psychol Psychiatry, 48:803–812. 2007.

36.

Malone

, Silvia

, Delaney

, Hyman

. Advances in drug treatments for children and adolescents with autism and other pervasive developmental disorders. CNS Drugs, 19:923–934. 2005.

37.

Malow

, Crowe

, Henderson

, McGrew

, Wang

, Song

, Stone

. A sleep habits questionnaire for children with autism spectrum disorders. J Child Neurol, 24:19–24. 2009.

38.

March

, Silva

, Petrycki

, Curry

, Wells

, Fairbank

, Burns

, Domino

, McNulty

, Vitiello

, Severe

. The treatment for adolescents with depression study (TADS): Long-term effectiveness and safety outcomes. Arch Gen Psychiatry, 64:1132–1143. 2007.

39.

Marshburn

, Aman

. Factorial validity and norms for the Aberrant Behavior Checklist in a community sample of children with mental retardation. J Autism Dev Disord, 22:357–373. 1992.

40.

Matson

, Benavidez

, Compton

, Paclawskyj

, Baglio

. Behavioral treatment of autistic persons: A review of research from 1980 to the present. Res Dev Disabil, 17:433–465. 1996.

41.

Matson

, Dixon

, Matson

. Assessing and treating aggression in children and adolescents with developmental disabilities: A 20-year overview. Educ Psychol, 25:151–181. 2005.

42.

Matson

, LoVullo

, Boisjoli

, Gonzalez

. The behavioral treatment of an 11-year old girl with autism and aggressive behaviors. Clin Case Stud, 7:313–326. 2008.

43.

McClellan

, Werry

. Practice parameters for the assessment and treatment of children and adolescents with bipolar disorder. J Am Acad Child Adolesc Psychiatry, 36:157–176. 1997.

44.

McClellan

, Kowatch

, Findling

. Practice parameter for the assessment and treatment of children and adolescents with bipolar disorder. J Am Acad Child Adolesc Psychiatry, 46:107–125. 2007.

45.

McCracken

, McGough

, Shah

, Cronin

, Hong

, Aman

, Arnold

, Lindsay

, Nash

, Hollway

, McDougle

, Posey

, Swiezy

, Kohn

, Scahill

, Martin

, Koenig

, Volkmar

, Carroll

, Lancor

, Tierney

, Ghuman

, Gonzalez

, Grados

, Vitiello

, Ritz

, Davies

, Robinson

, McMahon

. Risperidone in children with autism and serious behavioral problems. N Engl J Med, 347:314–321. 2002.

46.

McDougle

. Current, emerging therapeutics of autistic disorder, related pervasive developmental disorders. Neuropsychopharmacology: The Fifth Generation of Progress. Davis

, Charney

, Coyle

, Nemeroff

. Philadelphia: Lippincott Williams and Wilkins, 2002; 565–576.

47.

McDougle

, Scahill

, Aman

, McCracken

, Tierney

, Davies

, Arnold

, Posey

, Martin

, Ghuman

, Shah

, Chuang

, Swiezy

, Gonzalez

, Hollway

, Koenig

, McGough

, Ritz

, Vitiello

. Risperidone for the core symptom domains of autism: Results from the study by the autism network of the research units on pediatric psychopharmacology. Am J Psychiatry, 162:1142–1148. 2005.

48.

Munesue

, Ono

, Mutoh

, Shimoda

, Nakatani

, Kikuchi

. High prevalence of bipolar disorder comorbidity in adolescents and young adults with high-functioning autism spectrum disorder: A preliminary study of 44 outpatients. J Affect Disord, 111:170–175. 2008.

49.

Posey

, Walsh

, Wilson

, McDougle

. Risperidone in the treatment of two very young children with autism. J Child Adolesc Psychopharmacol, 9:273–276. 1999.

50.

POTS: Cognitive-behavior therapy, sertraline, and their combination for children and adolescents with obsessive-compulsive disorder: The Pediatric OCD Treatment Study (POTS) randomized controlled trial. JAMA, 292:1969–1976. 2004.

51.

Prochaska

, Evers

, Prochaska

, Van Marter

, Johnson

. Efficacy and effectiveness trials: examples from smoking cessation and bullying prevention. J Health Psychol, 12:170–178. 2007.

52.

Rosenbaum

, Rubin

. The central role of the propensity score in observational studies for causal effects. Biometrika, 70:41–55. 1983.

53.

Rubin

. Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol, 66:688–701. 1974.

54.

Sanchez

, Adams

, Uysal

, Hallin

, Campbell

, Small

. A comparison of live and videotape ratings: Clomipramine and haloperidol in autism. Psychopharmacol Bull, 31:371–378. 1995.

55.

Schafer

, Graham

. Missing data: Our view of the state of the art. Psycholog Methods, 7:147–177. 2002.

56.

Schafer

, Kang

. Average causal effects from nonrandomized studies: A practical guide and simulated example. Psycholog Methods, 13:279–313. 2008.

57.

Tuchman

, Moshe

, Rapin

. Convulsing toward the pathophysiology of autism. Brain Dev, 31:95–103. 2009.

58.

Vercellino

, Zanotto

, Ravera

, Veneselli

. Open-label risperidone treatment of 6 children and adolescents with autism. Can J Psychiatry, 46:559–560. 2001.

59.

Wilhelm

, Buhlmann

, Tolin

, Meunier

, Pearlson

, Reese

, Cannistraro

, Jenike

, Rauch

. Augmentation of behavior therapy with D-cycloserine for obsessive-compulsive disorder. Am J Psychiatry, 165:335–341 quiz 409 2008.

60.

Youngstrom

, Birmaher

, Findling

. Pediatric bipolar disorder: Validity, phenomenology, and recommendations for diagnosis Bipolar Disord, 10:194–214. 2008.