Comorbid Anxiety and Social Avoidance in Treatment of Severe Childhood Aggression: Response to Adding Risperidone to Stimulant and Parent Training;Mediation of Disruptive Symptom Response

Abstract

Objective:

In the four-site Treatment of Severe Childhood Aggression (TOSCA) study, addition of risperidone to stimulant and parent training moderately improved parent-rated disruptive behavior disorder (DBD) symptoms. This secondary study explores outcomes other than DBD and attention-deficit/hyperactivity disorder (ADHD) as measured by the Child and Adolescent Symptom Inventory-4R (CASI-4R).

Methods:

A total of 168 children ages 6–12 with severe aggression (physical harm), DBD, and ADHD were randomized to parent training plus stimulant plus placebo (basic treatment) or parent training plus stimulant plus risperidone (augmented treatment) for 9 weeks. All received only parent training plus stimulant for the first 3 weeks, then those with room for improvement received a second drug (placebo or risperidone) for 6 weeks. CASI-4R category item means at baseline and week 9 were entered into linear mixed-effects models for repeated measures to evaluate group differences in changes. Mediation of the primary DBD outcome was explored.

Results:

Parent ratings were nonsignificant with small/negligible effects, but teacher ratings (n=46 with complete data) showed significant augmented treatment advantage for symptoms of anxiety (p=0.013, d=0.71), schizophrenia spectrum (p=0.017, d=0.45), and impairment in these domains (p=0.02, d=0.26), all remaining significant after false discovery rate correction for multiple tests. Improvement in teacher-rated anxiety significantly (p=0.001) mediated the effect of risperidone augmentation on the primary outcome, the Disruptive-total of the parent-rated Nisonger Child Behavior Rating Form.

Conclusions:

Addition of risperidone to parent training plus stimulant improves not only parent-rated DBD as previously reported, but also teacher-rated anxiety–social avoidance. Improvement in anxiety mediates improvement in DBD, suggesting anxiety-driven fight-or-flight disruptive behavior with aggression, with implications for potential treatment strategies. Clinicians should attend to possible anxiety in children presenting with aggression and DBD.

Clinical Trial Registry:

Treatment of Severe Childhood Aggression (The TOSCA Study). NCT00796302. clinicaltrials.gov.

Introduction

Samples of children selected for attention-deficit/hyperactivity disorder (ADHD) and/or disruptive behavior disorders (DBD, including oppositional-defiant disorder [ODD] and conduct disorder [CD]), are highly comorbid for a range of psychiatric syndromes, including anxiety and mood disorders. For example, 34% of the children in the Multimodal Treatment Study of ADHD met diagnostic criteria for at least one anxiety disorder (MTA Cooperative Group 1999a). Less well appreciated is the fact that subdiagnostic symptoms of even relatively rare diagnoses such as schizophrenia spectrum disorder (SSD) and autism spectrum disorder (ASD) are widely distributed (Starling and Dossetor 2009; Kelleher et al. 2010; Padgett et al. 2010; King and Lord 2011; Gadow 2012; Gadow and Drabick 2012). The behavioral, cognitive, and affective characteristics of SSD are moderately correlated, and occur independently, or in some combination in the general population, often without apparent mental health implications (Kelleher et al 2010; Gadow 2012), and in even higher levels in youth referred for psychiatric evaluation (Starling and Dossetor 2009; Padgett et al. 2010; King and Lord 2011; Gadow and Drabick 2012). The pervasiveness of these co-occurring symptoms contrasts with the absence of studies demonstrating their responsiveness to first-line treatments for ADHD and DBD. These co-occurring symptoms may require additional treatment, improving the child's (and family's) quality of life, and, perhaps, enhancing response for the primary target symptoms.

The four-site Treatment of Severe Childhood Aggression (TOSCA) study selected children ages 6–12 years with DBD, ADHD, and severe physical aggression to study risperidone augmentation for those who did not respond well to parent training and stimulant. The primary outcome, the disruptive total (D-total) of the Nisonger Child Behavior Rating Form, typical intelligence quotient (IQ) version (NCBRF-TIQ), showed a significant (p=0.0016) advantage of augmented treatment (parent training+stimulant+risperidone) over basic treatment (parent training+stimulant+placebo), with a medium effect size (Aman et al. 2014). In a related report, augmented treatment was superior to basic in reducing the symptoms of ODD at home, ADHD in the classroom, and peer aggression in both settings (Gadow et al. 2014).

This secondary outcome analysis examines the effect of augmented versus basic treatment on the severity of anxiety, mood, ASD, and SSD symptoms in the same children. These explorations are indicated because of the following. 1) ADHD sometimes has a high rate of comorbid anxiety (MTA Cooperative Group 1999a); ASD, an increasingly common diagnosis, has a high rate of comorbid ADHD, ODD, and anxiety (Kaat et al. 2013) and in some reports, ADHD and ODD have a high rate of comorbid mood disorders (Biederman et al. 1996, 2004) and SSD symptoms. 2) Anxiety has been reported to moderate treatment response in ADHD (MTA Cooperative Group 1999b). 3) Children with DBD are reported to attribute hostile intent inappropriately to benign social interactions (Dodge 1980; Dodge and Coie 1987; Dodge 1991; Dodge et al. 1997).

Methods

TOSCA was a two-stage 9-week parallel group placebo-controlled randomized clinical trial. The first 3 weeks combined parent training in behavior management with open optimized stimulant titrated by telephone and weekly visits (basic treatment) for everyone. In stage 2, a second drug, risperidone or matched placebo, could be added. Risperidone plus parent training plus stimulant (augmented treatment) was compared with placebo plus parent training plus stimulant (basic treatment) for those who did not have an excellent response by Week 3. Written Informed consent was obtained using procedures approved by the Institutional Review Board at each of the four clinical sites. Further design details can be found in Farmer et al. (2011) and Aman et al. (2014).

Participants

Participants were 168 children of average intelligence, ages 6–12 years inclusive, recruited at four sites (Ohio State University, Case-Western Reserve University, University of Pittsburgh, and Stony Brook University). Each child met Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV) diagnostic criteria for a diagnosis of CD (n=44) or ODD (n=124) and a DSM-IV diagnosis of ADHD (any subtype), and had serious physical aggression, disruptive behavior ≥27 (90th percentile on the NCBRF-TIQ D-total score), and a Clinical Global Impressions (CGI) Severity score of ≥4 (moderately ill or worse) for aggression (American Psychiatric Association 1994).

Exclusion criteria included full-scale IQ <70, seizures, abnormal liver function, pervasive developmental disorder, psychotic disorder, eating disorders, major depressive disorder, bipolar disorder, substance use disorder, evidence of current child abuse, suicide attempt, or first-degree family history of type 2 diabetes. They were screened with the Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS) for both inclusionary and exclusionary diagnoses by a trained clinician, confirmed on interview by a child psychiatrist.

Mean NCBRF-TIQ D-total score for the sample was 42.3 (SD=10.4). There were 129 boys (77%) and 39 girls. Mean age was 8.89 (SD=2.01) years (range 6–12). Family income was low (mode <$20,000), although middle-class families also participated (Aman et al., 2014). Children were rated by parents as more reactive than proactive: The Antisocial Behavior Scale reactive item mean was 2.51+0.35, compared with a proactive item mean of 1.99+0.45 (scale of 0–3; Cohen's d=1.3). Also, 157 of the 168 children had a higher reactive than proactive score. More detailed sample characteristics can be found in Aman et al. (2014).

Procedure

The intervention period comprised two stages. First, all participants received parent training in behavior management and stimulant medication (usually osmotic release oral system methylphenidate) for at least 3 weeks. This was followed, if needed, by up to 6 weeks of additional randomized risperidone (augmented) or placebo (basic). One-to-one randomization to risperidone or placebo occurred at baseline, stratified by site and balanced by ODD versus CD diagnosis. The addition of the second medication occurred at end of Week 3 or later for children who did not experience optimal clinical response (CGI-Improvement=1 and NCBRF-TIQ D-total <15). This second, double-blind phase lasted though Week 9.

Of the 168 randomized participants, 14 children (3 from basic and 11 from augmented) dropped out before Week 3, when the second medication would have been added. (Note that this differential dropout rate had nothing to do with randomized treatment, which had not begun yet.) Eight children were classified as excellent clinical responders by the end of Week 3, and were, therefore, not given the second medication. Therefore, 22 children either dropped out before they had an opportunity for possible benefit from augmentation or were deemed not to need it, leaving 78 in basic treatment and 68 in augmented treatment who actually took the second drug.

Measures

Among the secondary outcome measures was the Child and Adolescent Symptom Inventory-4R (CASI-4R) (Gadow and Sprafkin 2002), a DSM-IV-referenced symptom severity scale. Individual items are rated on a scale from 0 (never) to 3 (very often), with separate forms for parent and teacher. The Symptom Severity score is dimensional, and it is the sum or mean of all item scores for a specific subscale. The last item in each symptom subscale addresses impairment: “How often do the behaviors in [this category] interfere with youth's ability to do schoolwork or get along with others.” Impairment severity is rated on the same four point scale. Numerous studies indicate that CASI-4R scales demonstrate satisfactory psychometric properties (Gadow and Sprafkin 2013).The ADHD, ODD, and CD subscales were considered primary diagnostic treatment targets and are reported separately. Here we focus on the following subscales: Generalized anxiety, other anxiety (specific phobia, obsessive-compulsive disorder [OCD], and posttraumatic stress), social phobia, schizoid personality disorder, schizophrenia symptoms, depression, manic symptoms, and autism spectrum symptoms. The symptoms of schizoid personality and schizophrenia are pooled to make an SSD score (Gadow 2012). Similarly, scores for all the anxiety items are pooled to generate a global composite anxiety score. The parent form also had subscales for separation anxiety, eating disorders, and enuresis/encopresis. The impairment items for all categories listed are averaged for a global impairment rating. The CASI-4R was collected at screen, baseline, and the end of stage 2 of the trial (Week 9 or at early termination).

Medication adherence

Dosage adherence was monitored by pill counts by returned pill minders. Adherence was calculated in two ways (Table 1). “Overall” average medication adherence was calculated by summing the percent adherence from all visits by all subjects, then dividing by the total number of visits. “By subject” adherence was computed by first calculating the mean adherence for individual subjects over their entire time in the study, then averaging the subjects. For each method, adherence was >94% for stimulant and risperidone and >91% for placebo. The mean final methylphenidate dose was 44.8±14.6 mg/day for the basic group and 46.1±16.8 mg/day for the augmented group (p=0.88)). For the second drug, the final placebo dose was 1.9±0.72 mg/day, and the final risperidone dose was 1.7±0.75 mg/day (p=0.07).

Table 1.

Medication Adherence

	Stimulant a		Second drug b
Treatment	Overall mean % pills taken	Mean of % taken by each subject	Overall mean % pills taken	Mean of % taken by each subject
Basic	96.03%	95.50%	93.40%	91.72%
Augmented	95.84%	94.54%	94.85%	94.19%

Adherence for stimulant over all subjects and visits=95.94%.

For basic treamtent, second drug was placebo, whereas for augmented treatment it was risperidone.

Statistical analyses

To save power by reducing the number of tests, we collapsed the three anxiety categories into one outcome (anxiety composite) and used the established composite SSD instead of the two separate schizoid outcomes. This made 6 outcomes for teacher ratings (global impairment, anxiety composite, SSD, autism symptoms, manic symptoms, and depression) and 10 outcomes for parent ratings (the same 6 as for teachers plus separation anxiety, enuresis/encopresis, anorexia, and bulimia). These were corrected for multiple tests by the Benjamini–Hochberg (1995, 2000) false discovery rate method. The descriptive components of the composite measures are also shown as exploratory dismantling of the composites.

A constrained longitudinal data analysis (cLDA) model (Lu 2010), in which both baseline and postbaseline values for parent-rated mean symptom severity scores are treated as dependent variables, was used in the intention-to-treat population, including all 168 randomized subjects. Fixed effects included those for time, treatment, site, and disorder type (CD vs. ODD). An unstructured variance covariance matrix was assumed for the repeated measures within each subject. Empirical-based sandwich estimators were obtained to assess the treatment differences at Week 9 given their robustness against the deviations from model assumptions (Gurka et al. 2011). Because of concerns about missing data, teachers' ratings were modeled including fixed effects for time, treatment, treatment-by-time interaction, site, and disorder type. For variables with a nonnormal distribution, a square root transformation was performed before analysis, but raw scores are shown in results tables. Last observation carried forward was used for subjects with data available at any visit after baseline. Effect sizes were calculated as Cohen's d, the difference in change score divided by pooled SD. All analyses were completed in SAS version 9.2 (SAS Institute 2009).

In cases in which augmented treatment was superior to basic treatment in improving a CASI-4R composite score, we explored change in that score as a possible mediator of the demonstrated effect of augmented treatment on the primary outcome. To do so, the median-centered change score of the relevant CASI symptom score was entered into the primary analysis model (dependent variable=NCBRF-TIQ D-total). A significant triple interaction of time by treatment by the CASI change score was considered evidence of mediation (Kraemer et al. 2002). According to the MacArthur guidelines, criteria for mediation include temporal precedence of treatment, a demonstrated effect of treatment on the mediator, and a main effect or interaction with treatment in predicting the outcome. Demonstration of mediation does not establish mechanism or cause in itself, but is the first step (Kraemer et al. 2002).

Results

Parent ratings were available for 150 children and teacher ratings were available for 46, at both baseline and end-point. There was no significant advantage of augmented over basic treatment on any parent-rated secondary outcome (Table 2).

Table 2.

CASI-4R Parent-Rated; Change Score Summaries and Mixed Models P Values

	Treatment	Baseline (BL) a	Change from BL to Week 9 b	p value	Adjusted change score difference (95% CI)	Effect size
Anxiety composite	Basic	0.65±0.43	−0.37±0.45	0.26	0.04 (−0.03, 0.12)	0.04
	Augmented	0.61±0.46	−0.37±0.35
Separation anxiety	Basic	0.53±0.74	−0.28±0.54	0.40	−0.04 (−0.13, 0.05)	0.28
	Augmented	0.34±0.51	−0.12±0.42
Enuresis, encopresis	Basic	0.33±0.57	−0.16±0.43	0.14	−0.07 (−0.16, 0.02)	0.05
	Augmented	0.46±0.75	−0.19±0.44
Schizophrenia spectrum	Basic	0.39±0.4	−0.21±0.43	0.09	0.07 (−0.01, 0.16)	0.10
	Augmented	0.38±0.46	−0.26±0.38
Depression	Basic	0.34±0.29	−0.20±0.3	0.98	−0.001 (−0.08, 0.08)	0.10
	Augmented	0.32±0.31	−0.15±0.23
Manic symptoms	Basic	1.06±0.77	−0.77±0.78	0.43	0.05 (−0.07, 0.16)	0.31
	Augmented	0.81±0.77	−0.54±0.7
Autism spectrum	Basic	0.73±0.55	−0.46±0.55	0.36	0.04 (−0.05, 0.14)	0.05
	Augmented	0.69±0.52	−0.46±0.42
Anorexia	Basic	0.26±0.39	−0.12±0.42	0.67	0.02 (−0.07, 0.11)	0.24
	Augmented	0.15±0.32	−0.02±0.36
Bulimia nervosa	Basic	0.27±0.43	−0.2±0.43	0.15	−0.07 (−0.15, 0.02)	0.19
	Augmented	0.25±0.4	−0.12±0.44
Global Impairment	Basic	0.67±0.53	−0.39±0.53	>0.05	0.10 (−0.0001, 0.20)	0.06
(Mean Impairment)	Augmented	0.57±0.44	−0.37±0.39
Untested exploratory outcomes dismantling composites
Generalized anxiety	Basic	1.13±0.65	−0.69±0.77		0.12 (−0.02, 0.25)	0.01
	Augmented	1.04±0.62	−0.70±0.6
Other anxiety	Basic	0.41±0.42	−0.24±0.38		0.03 (−0.05, 0.11)	0.04
	Augmented	0.43±0.5	−0.24±0.36
Social phobia	Basic	0.5±0.75	−0.2±0.68		0.05 (−0.07, 0.17)	0.06
	Augmented	0.37±0.54	−0.11±0.49
Schizoid personality disorder	Basic	0.49±0.62	−0.16±0.59		0.09 (−0.03, 0.21)	0.25
	Augmented	0.61±0.74	−0.38±0.62
Schizophrenia	Basic	0.35±0.41	−0.23±0.46		−0.01 (−0.14, 0.12)	0.06
	Augmented	0.27±0.44	−0.21±0.38

The p values are for the interaction between time and treatment, adjusted for site and disorder. The adjusted group differences in change scores and confidence intervals are adjusted for site and disorder. All subscales except for generalized anxiety and manic symptoms were square-root transformed prior to analysis.

n=84 for both treatment groups except for anorexia and bulimia subscales, where augmented n=83.

Basic n=77 and augmented n=73.

Other anxiety=specific phobia, obsessive-compulsive disorder, posttraumatic stress disorder. Anxiety composite=mean of generalized, phobic, and other anxiety. Schizophenia spectrum=mean of schizoid personality and schizophrenic symptoms. Global impairment=impairment from the listed symptoms.

CASI-4R, Child and Adolescent Symptom Inventory-4R.

However, teacher ratings showed significant differential improvement in several symptom clusters (Table 3). Augmented treatment showed significantly more improvement than basic for the anxiety composite (p=0.013, d=0.71), SSD composite (p=0.017, d=0.45), and overall impairment from the symptoms examined (p=0.020, d=0.26).

Table 3.

CASI-4R Teacher-Rated; Change Score Summaries and Mixed Models P Values

	Treatment	Mean±SD	Mean±SD	p value	Adjusted change score difference (95% CI)	Effect size d
Anxiety composite	Basic	0.37±0.39	−0.15±0.27	0.013*	0.17 (0.04, 0.31)	0.71
	Augmented	0.50±0.51	−0.29±0.29
Schizophrenia spectrum	Basic	0.29±0.47	−0.05±0.34	0.017*	0.19 (0.04, 0.35)	0.45
	Augmented	0.38±0.55	−0.18±0.34
Depression	Basic	0.31±0.32	−0.09±0.27	0.18	0.10 (−0.05, 0.25)	0.21
	Augmented	0.30±0.35	−0.11±0.19
Manic symptoms	Basic	0.69±0.63	−0.56±0.6	0.80	0.04 (−0.26, 0.34)	0.25
	Augmented	0.68±0.6	−0.41±0.58
Autism spectrum	Basic	0.40±0.44	−0.14±0.28	0.07	0.13 (−0.01, 0.27)	0.31
	Augmented	0.52±0.59	−0.18±0.3
Global impairment	Basic	0.65±0.54	−0.22±0.42	0.020*	0.25 (0.04, 0.46)	0.26
(mean impairment)	Augmented	0.77±0.75	−0.46±0.64
Untested exploratory outcomes dismantling composites
Generalized anxiety	Basic	0.66±0.65	−0.29±0.5		0.34 (0.04, 0.63)	0.49
	Augmented	0.94±0.93	−0.57±0.67
Other anxiety	Basic	0.18±0.27	−0.08±0.21		0.15 (0.01, 0.30)	0.62
	Augmented	0.31±0.35	−0.20±0.23
Social phobia	Basic	0.26±0.52	0.07±0.42		0.17 (−0.02, 0.36)	0.44
	Augmented	0.33±0.59	−0.09±0.45
Schizoid personality	Basic	0.55±0.81	−0.01±0.68		0.26 (0.02, 0.50)	0.35
	Augmented	0.61±0.8	−0.22±0.59
Schizophrenia	Basic	0.10±0.16	−0.01±0.23		0.17 (0.04, 0.30)	0.60
	Augmented	0.27±0.47	−0.16±0.27

All subscales except for manic symptoms were square-root transformed prior to analysis. The p values are for the interaction between time and treatment, adjusted for site and disorder. The adjusted group differences in change scores and confidence intervals are adjusted for site and disorder.

n for all analyses is 28 for basic, 18 for augmented, except for other anxiety and schizophrenia which is n=27 for basic.

Significant at p<0.04 after correction for multiple tests by Benjamini–Hochberg false discovery rate.

CASI-4R, Child and Adolescent Symptom Inventory-4R.

Mediator analyses

When the improvement (change score) in global anxiety composite and SSD were added into the primary analysis model of the parent-rated D-total, there was a significant interaction of time by treatment by change score for the anxiety composite (p=0.001) and SSD (p=0.047). However, the interaction for SSD was not significant after correction for multiple tests. Thus, improvement in teacher-rated anxiety mediated the effect of augmented treatment on parent-rated D-total scores. With risperidone augmentation, but not with basic treatment, children with more improvement in anxiety also improved more on the D-total primary outcome than those with less improvement in anxiety (Fig. 1). As illustrated in Figure 1, panel B, ∼20% of the observed augmentation effect on D-total (1.34/6.77 scale points) was mediated by improvement in teacher-rated anxiety symptoms.

FIG. 1.

Mediation of improvement in parent-rated D-total by improvement (change score) in teacher-rated anxiety composite. p=0.001 (square root transformation). (A) Although the mediator test of significance was based on dimensional analysis, a median split into high and low anxiety improvement is used for illustration. (B) As anxiety improvement increases (horizontal axis), disruptive Improvement (vertical axis) decreases in basic treatment and increases in augmented treatment. Horizontal faint dotted lines are the mean D-total improvement of augmented (upper) and basic (lower) treatments. The open section of the vertical bar represents the amount of D-total improvement caused by mediation at the mean of anxiety improvement, ∼20% of the observed improvement in D-total caused by risperidone augmentation. Basic treatment=parent training plus stimulant plus placebo; augmented treatment=parent training plus stimulant plus risperidone. NCBRF D-total, Nisonger Child Behavior Rating Form–Typical IQ Disruptive Behavior total score.

Informant discrepancy

The fact that the significant findings were confined to teacher ratings, for which only 46 of the 168 children had data at both baseline and end-point (in contrast to 150 with parent data at both times), raised a question about possible bias in the subsample with both parent and teacher data. Baseline parent ratings showed lower baseline severity of NCBRF ADHD symptoms for the 46 children with complete teacher ratings than for the 122 without (26.9 vs. 24.3, p=0.014), reflecting possible bias in the subsample with teacher data. However, there were only negligible and nonsignificant differences in parent-rated baseline D-total, age, sex, IQ, parents' education, and family income between the 46 with complete teacher ratings and the rest of the sample. Similarly, no CASI parent ratings in these analyses showed any significant difference between the 46 with teacher ratings and the other 104. As a sensitivity analysis to check the effect of missing-data bias, we reran the parent-rating analysis on the subsample (n=46) with complete teacher data, and these subsample parent results were similar to those from the whole sample (not significant).

Discussion

This report addressed the effects of risperidone augmentation on psychopathology other than disruptive behavior, aggression, and ADHD outcomes previously reported (Aman et al. 2014; Gadow et al. 2014). Parent ratings showed no significant effect on anxiety, depression, manic symptoms, SSD, eating disorders, autism symptoms, or enuresis/encopresis, However, teacher ratings showed significant improvement in anxiety, SSD, and global impairment that withstood the false discovery rate correction for multiple tests. Teacher ratings also showed significant mediation by anxiety improvement of the risperidone-augmentation improvement in disruptive behavior.

The finding that teachers observed a decrease in anxiety and SSD symptom severity in children receiving augmented versus basic treatment, with effect sizes in the medium/moderate range, is important, because this could be a clinically significant additional benefit for many children. Considering the significant teacher-rated improvement in CASI-4R subscales, we find an overall picture of improvement in anxiety and social avoidance from the addition of risperidone. The scale items accounting for the significant improvement included “difficulty controlling worries,” “irritable,” “extremely tense,” “overly fearful of objects or situations,” “can't get distressing thoughts out of mind,” “feels compelled to perform unusual habits,” “continues to be bothered by an extremely upsetting event,” “prefers to be alone,” “little interest in having close relationships,” “emotionally cold or indifferent,” “extremely strange and illogical thoughts,” “laughs or cries at inappropriate times or shows no emotion when others would,” and “lost interest in doing things or talking to people.” Hereafter, we use the hybrid term anxiety–social avoidance to describe this constellation of symptoms responsive by teacher ratings to risperidone augmentation.

Risperidone improvement of anxiety

This finding adds fresh information to the literature (mostly on adults) about effects of risperidone on anxiety, which varies from a few case reports of worsening anxiety, through no effect in randomized clinical trials (RCTs), to positive results in RCTs (Ravindran et al. 2007). The only report we found for children was of three cases of boys ages 10–18 who developed separation anxiety disorder when their medication for OCD or disruptive behavior was augmented with risperidone (Hanna et al. 1999). The following clinical trials were in adults.

An 8 week open trial in 30 patients with anxiety refractory to antidepressants and benzodiazepines showed a significant (p=0.0005) decrease in anxiety from risperidone augmentation (Simon et al. 2006). The 8 weeks may have been important because in a 4 week 417 subject RCT after 8 weeks of anxiolytic treatment for generalized anxiety disorder (GAD), adjunctive placebo showed almost as much improvement as 4 weeks of adjunctive risperidone, in other words, there was no significant difference (Pandina et al. 2007). In a 5 week RCT, 40 patients with GAD who were unresponsive to anxiolytic treatment, responded better to 5 weeks of added low-dose risperidone than to placebo (Brawman-Mintzer et al. 2005). In a 6 week RCT, 392 patients only partially responsive to anxiolytic treatment showed greater reduction of residual anxiety with 6 weeks of risperidone augmentation than with placebo augmentation (Pandina et al. 2006, 2007). In another 6 week RCT, 36 patients with OCD refractory to a serotonin reuptake inhibitor (SSRI) were randomized to augmentation with risperidone or placebo; half of the risperidone group but none of the placebo group responded. Summarizing this and three other RCTs in OCD, Ravindran et al. (2007) concluded that there was “reasonable empirical support” (p. 1696) for low-dose risperidone's value as augmentation to SSRIs for OCD. The five cited studies taken together suggest a possible duration threshold requiring ≥5 weeks for risperidone to ameliorate anxiety more than placebo. In this study, we used 6 weeks of risperidone augmentation.

RCTs addressing comorbid anxiety also show mixed results. In 111 patients with bipolar disorder and comorbid panic or GAD, risperidone was not significantly better than placebo (Sheehan et al. 2009). Subgroup analyses of that study sample showed that nonresponders to risperidone were more likely to have panic disorder than nonresponders to placebo (Seo et al. 2013). Upon reviewing five RCTs in posttraumatic stress disorder (PTSD), Ravindran et al. (2007) found “substantial evidence” (p.1698) for risperidone benefit. They thought that risperidone augmented by resensitizing serotonin autoreceptors in the orbitofrontal cortex, and reported the term “anxiothymic regulator” for risperidone. Seventeen psychotic patients in an open safety trial showed significant (p=0.001) improvement in the anxiety-depression factor of the Brief Psychiatric Rating Scale (Mesottent et al. 1989). In two randomized clinical trials, 513 patients with schizophrenia showed greater improvement in anxiety-depression with risperidone, especially at 6 mg/day, than with placebo (p=0.001, d=0.36) (Marder et al. 1997).

In summary, the literature, mainly in adults, presents a mixed but generally positive picture regarding risperidone effects on anxiety. Despite the child/adolescent case reports of separation anxiety side effects and some failed trials, the bulk of evidence shows risperidone benefiting anxiety in adults, especially in OCD and PTSD. There may possibly be a duration threshold at 5 weeks of treatment. The study presented here is unusual in showing placebo-controlled risperidone benefit for subdiagnostic anxiety in preadolescent children. Notably, this study treated for 6 weeks.

Speculative hypothesis: Anxiety-driven fight or flight

The anxiety improvement that we found may be particularly relevant to a fight-or-flight reaction in these children selected for aggression, who appeared more reactively than instrumentally aggressive. The reduction of teacher-rated anxiety actually mediated (p=0.001) the reduction of disruptive behavior, including aggression, as measured by the primary outcome, the parent-rated D-total (Fig. 1), supporting the speculation of anxiety-driven fight-or-flight aggression. Anxiety induced by the stress of real or imagined peer threat or other sources of anxiety may prompt a response before the opportunity for cognitive/higher cortical processing and reevaluation (Kashani et al. 1991). Imagined peer threat is plausible in view of the work showing misinterpretation of hostile intent by children with disruptive behavior disorder (Dodge 1980; Dodge and Coie 1987; Dodge 1991; Dodge et al. 1997). This hypothesis is supported by two reports in children and adolescents ages 8–17. In a sample of 274 with ADHD, conduct disorder symptoms (but not oppositional-defiant symptoms) predicted anxiety. Kashani et al. (1991) reported a highly significant association of anxiety with both verbal and physical aggression in 210 children and adolescents ages 8–17. They concluded that although anxiety does not directly cause aggression, aggression is one way of coping with anxiety and fear. They hypothesized that anxiety interferes with assessing situations and alternative responses rationally, thus restricting behavioral options and predisposing to falling back on such primitive responses as aggression.

Further support for the hypothesis comes from neuroscience, including brain imaging. The amygdala is involved in assessing threat and deciding between fight and flight, and the frontal cortex is involved in modulating the response (Coccaro et al. 2007, 2011); patients with high-aggression intermittent explosive disorder showed abnormally high activation of the amygdala and abnormally low activation of the orbitofrontal cortex in response to threatening angry faces (Coccaro et al. 2007).

The speculation that anxiety–social avoidance drives some of the aggression in these children is also supported by animal research. Neumann et al. (2010), reviewing rat data, discussed the overlap of brain pathways between aggression and anxiety, and reported that both rats bred for high trait anxiety and rats bred for low trait anxiety showed greater aggression than normal rats (Beiderbeck et al. 2012). Therefore, it is possible that these two pathways to aggression in rats (low vs. high anxiety) reflect factors similar to the callous-unemotional versus the reactive-emotional distinction in humans. Interestingly, the low-anxiety rats' aggression appeared related to activation of the nucleus accumbens, the reward center (suggesting that aggression was rewarding), and their aggression could be reduced by dopamine D2 receptor blockers such as haloperidol, a first-generation antipsychotic (Beiderbeck et al. 2012).

This speculation (anxiety-driven fight-or-flight aggression and social withdrawal) is supported not only by the literature and by the mediator analyses showing that the improvement from risperidone augmentation was mediated by improvement in anxiety, but also by other evidence from this study. On the parent-rated NCBRF-TIQ, prosocial behavior increased with augmentation (p=0.01, d=0.46) (Aman et al. 2014), and baseline parent-rated generalized anxiety correlated with the parent-rated antisocial behavior scale (r=0.37, p=0.001). Therefore, we may need to consider subtyping aggression/DBD into anxious and nonanxious types. These considerations point to a possible testable strategy for augmenting stimulant plus parent training in a safer way than with an antipsychotic agent; namely, making possible interventions (both psychosocial and pharmacological) to allay anxiety. For example, one augmentation strategy could be use of anxiety-targeted cognitive-behavioral therapy, or even inclusion of anxiety-allaying techniques in parent training. It would also have been interesting to see what impact adding an anxiolytic drug (instead of risperidone) to parent training plus simulant would have had on the NCBRF D-total. However, diazepam (Zrull et al. 1964), lorazepam (Walters et al. 1977), meprobamate (Craft 1958), and phenobarbital have been reported to cause behavioral worsening in children with intellectual disability or brain damage who were selected for disruptive, emotional, and hyperactive/inattentive behaviors or who were being treated for epilepsy (Aman 1983). On the other hand, benzodiazepines have reduced aggression in some other disorders (Neumann et al. 2010). The possibility that other drugs (e.g., β blockers (Arnold and Aman 1991), α-2 agonists, or antidepressants, especially SSRIs or buspirone) would effectively augment remains largely uninvestigated. Therefore, RCTs are needed to investigate the augmentive value of anxiolytic agents (and psychosocial anxiety treatments) as safer replacements for the antipsychotic augmentation tested in this TOSCA study.

Schizophrenia-spectrum changes

To the best of our knowledge, this may be the first report of treatment-responsive reduction in SSD symptoms in a sample of nonpsychotic children, possibly, in part, because of the absence of psychotic symptoms from most rating scales used in pediatric clinical trials (Leary et al. 2011; Gadow 2012). Our sample was carefully screened with the K-SADS and child psychiatrist interviews to exclude psychosis, schizophrenia, and bipolar disorder. Therefore, mean severity ratings for schizophrenia symptoms were low at baseline, but nevertheless fell even further with augmented versus basic treatment. This should not be surprising for an antipsychotic drug, but we might wonder why these nonpsychotic children showed any schizophrenia symptoms at all.

Actually, with the exception of hallucinations and delusions, SSD symptoms as rated on the CASI-4R are widely distributed in clinical samples, particularly in children with ADHD, and even in the general population (Gadow 2012). It may be reasonable to speculate that the social and communication deficits of ADHD and DBD share similar presentations and adaptive functioning deficits with SSDs. Inspection of the item ratings in this sample disclosed no mention of hallucinations, and extremely low mention of thinking that could be interpreted as delusional. The significant treatment effect was carried mainly by items indicating social withdrawal and disorganized impulsiveness: “prefers to be alone,” “little interest in having close relationships,” “emotionally cold or indifferent,” “extremely strange and illogical thoughts,” “laughs or cries at inappropriate times or shows no emotion when others would,” and “lost interest in doing things or talking to people.” Many of these same symptoms characterize people with antisocial personality disorder, and are no surprise to clinicians who work with severely aggressive children. Many children with ADHD who are aggressive grow up to have antisocial personalities (Klein et al. 2012). Therefore, one possible speculation from these findings is that they provide some of the first evidence for an effect on callous-unemotional traits. Additionally, in light of the significant anxiety effect, perhaps these children were anxious about what they mistakenly perceived as a hostile environment.

Collectively, these results underscore the potential value of dimensional strategies for understanding interrelations among clinical phenotypes. The results are also compatible with the National Institute of Mental Health emphasis on dimensional constructs that cut across diagnoses (Research Domain Criteria) (Lende 2014).

Possible role of anxiety and avoidance in aggression

Conceptually, it appears that a construct of anxious avoidance of social interaction may best explain the confluence of improvement in anxiety and schizophrenia spectrum symptoms in the school setting. This speculation is compatible with the literature, suggesting that aggressive children have deficits in social skills and misinterpret cues from others, excessively ascribing hostile intent to neutral words and acts of others (Dodge 1980; Dodge and Coie 1987; Dodge 1991; Dodge et al. 1997). Because the behavioral problems of children with aggression, ADHD, and DBD are so prominent, it is easy to overlook these children's internal emotional problems, which may be an important contributor to disruptive behavior in some cases.

An alternative explanation might be that the child's aggression antagonized others and socially isolated that child (the reverse causation). However, the relevant items showing significant differential improvement – “prefers to be alone,” “little interest in having close relationships,” and “lost interest in doing things or talking to people” – suggest that they reflect the child's preference rather than resulting from peer rejection. In addition, this alternative explanation is not supported by the mediator analyses.

Discussion of informant discrepancy

One wonders why the improvement in anxiety and social withdrawal appears only on teacher ratings and not on the more numerous parent ratings. There may be an elicitation difference in the two settings. Exposure to greater numbers of peers in an inescapable setting may be stressful for some children, eliciting anxiety and social avoidance. However, this speculation is not supported by the relatively lower baseline anxiety ratings by teachers compared with those of parents. There may also be an observation opportunity difference. The presence of many people in a more structured setting may allow observation of changes in the child's anxiety and social avoidance in a way not possible at home. Finally, the child may not act in the same way at home as at school; familiar surroundings may stabilize anxiety at a consistent level.

Limitations

These secondary exploratory analyses have several limitations. The augmented group started with more severe anxiety symptoms, so at least some of the significant difference in improvement could be caused by rate dependency or a base rate phenomenon (Sahakian and Robbins 1977). However, these were randomized groups, and the sample was not selected for high anxiety or schizoid symptoms, making base rate dependency a less attractive explanation. The fact that teacher ratings for both baseline and Week 9 were available on little more than a quarter of the sample raised a question about biased missing data. The missing teacher data were partly a result of treating children throughout the year, not just while school was in session; and unavailability of teachers in the summer to provide ratings was not likely related to any child's characteristics Nevertheless, the baseline parent NCBRF ADHD ratings for the 46 children with teacher ratings were higher than for the 122 children without complete teacher ratings. A sensitivity analysis found that the parent-rated treatment comparisons for the subsample with complete teacher data were similar to parent-rated results from the whole sample (not significant). Therefore, the sensitivity analysis failed to support the bias explanation of the difference between parent and teacher results.

Finally, although the mediator analysis, according to MacArthur guidelines (Kraemer et al. 2002), showed mediation of improvement in the disruptive behavior primary outcome by teacher-rated anxiety improvement, we cannot be sure of the mechanism or causal connection, because we had only pre-post scores, with no data to show anxiety improvement preceding improvement of the primary outcome. It is even possible that improvement in disruptive behavior preceded improvement in anxiety, and that the direction of causation was actually the reverse of what the literature suggested. Alternatively, it is also possible that the improvement in anxiety and in the primary outcome were simply parallel effects of risperidone; however, the significant (p=0.001) interaction argues against that explanation. Further research addressing this question is obviously indicated.

Conclusions and Clinical Significance

In addition to improving disruptive/aggressive behavior as previously reported (Aman et al. 2014; Gadow et al. 2014), risperidone augmentation of parent training plus stimulant improves school-setting anxiety and social avoidance in aggressive children with DBD and ADHD. According to teacher ratings, the improvement in anxiety significantly (p=0.001) mediated the improvement in parent-rated disruptive behavior. Clinicians need to attend to both internal emotional and external behavioral symptoms in children presenting with blatant behavioral symptoms. Although additional empirical study is needed, aggression accompanied by anxiety and social avoidance may partially reflect an anxiety-driven fight-or-flight reaction, which we speculate may respond to specific anxiety treatment. This possibility deserves further exploration.

Footnotes

Disclosures

Dr. Arnold has received research funding from CureMark, Forest, Lilly, and Shire; advisory board honoraria from Biomarin, Novartis, Noven, Otsuka, Roche, Seaside Therapeutics, and Shire; consulting fees from Gowlings, Pfizer, and Tris Pharma; and travel support from Noven. Dr. Gadow is a shareholder in Checkmate Plus, publisher of the Child and Adolescent Symptom Inventory-4R. Dr. Aman has received research contracts from, consulted with, or served on advisory boards of Biomarin Pharmaceuticals, Bristol-Myers Squibb, CogState, Confluence Pharmaceutica, Coronado Bioscience, Forest Research, Hoffman LaRoche, Johnson & Johnson, Neuren Pharmaceuticals, Novartis, ProPhase LLC, and Supernus Pharmaceutica. Dr. Hurt received research support from Bristol-Myers Squibb. Dr. Bukstein has received royalties from Routledge Press and acted as a consultant for Ezra Innovations and PRIME CME. Dr. Bangalore received research support from Supernus Pharmaceutica. Dr. Findling receives or has received research support from, acted as a consultant and/or served on a speaker's bureau for Alexza Pharmaceuticals, American Academy of Child & Adolescent Psychiatry, American Physician Institute, American Psychiatric Press, AstraZeneca, Bracket, Bristol-Myers Squibb, Clinsys, CogCubed, Cognition Group, Coronado Biosciences, Dana Foundation, Forest, GlaxoSmithKline, Guilford Press, Johns Hopkins University Press, Johnson & Johnson, KemPharm, Lilly, Lundbeck, Merck, NIH, Novartis, Noven, Otsuka, Oxford University Press, Pfizer, Physicians Postgraduate Press, Rhodes Pharmaceuticals, Roche, Sage, Seaside Pharmaceuticals, Shire, Stanley Medical Research Institute, Sunovion, Supernus Pharmaceuticals, Transcept Pharmaceuticals, Validus, and WebMD. Dr. Rundberg-Rivera received research support from GlaxoSmithKline, Merck/Schering Plough, National Institute of Mental Health, Covance/Otsuka, and Pfizer. Dr. McNamara receives research support from FORUM, Lundbeck, Merck, Otsuka, Novartis, Pfizer, Roche, Sunovion, and SyneuRx. Ms. Brown, Ms. Page, and Drs. Farmer, Li, Molina, and Rice have nothing to declare.

References

Aman

: Psychoactive drugs in mental retardation. In: Treatment Issues and Innovations in Mental Retardation, edited by Matson

J.L.

and Andrasik

. New York: Plenum Press, 455–505, 1983.

Aman

, Bukstein

, Gadow

, Arnold

, Molina

, McNamara

, Rundberg-Rivera

, Li

, Kipp

, Schneider

, Butter

, Baker

, Sprafkin

, Rice

, Bangalore

, Farmer

, Austin

, Buchan-Page

, Brown

, Hurt

, Grondhuis

, Findling

: What does risperidone add to parent training and stimulant for severe aggression in child attention-deficit/hyperactivity disorder: J Am Acad Child Adolesc Psychiatry, 53:47–60, 2014.

Aman

, Leone

, Lecavalier

, Park

, Buican

, Coury

: The Nisonger child behavior rating form: Typical IQ version, for children with typical IQ. Int Clin Psychopharmacol, 23:232–242, 2008.

American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Washington, DC: American Psychiatric Association; 1994.

Arnold

, Aman

: Beta blockers in mental retardation and developmental disorders. J Child Adolesc Psychopharmacol, 1:361–373, 1991.

Beiderbeck

, Reber

, Havasi

, Bredewold

, Veenema

, Neumann

: High and abnormal forms of aggression in rats with extremes in trait anxiety–involvement of the dopamine system in the nucleus accumbens. Psychoneuroendocrinology, 37:1969–1980, 2012.

Benjamini

, Hochberg

: Controlling the false discovery rate: A practical and powerful approach to multiple testing. J R Stat Soc, 57:289–300, 1995.

Benjamini

, Hochberg

: On the adaptive control of the false discovery rate in multiple testing with independent statistics. J Educ Behav Stat, 25:60–83, 2000.

Biederman

, Faraone

, Mick

. Attention-deficit and hyperactivity disorder and juvenile mania: An over-looked comorbidity?. J Am Acad Child Adolesc Psychiatry, 35:997–1008, 1996.

10.

Biederman

, Mick

, Faraone

, Van Patten

, Burback

, Wozniak

: A prospective follow-up study of pediatric bipolar disorder in boys with attention-deficit/hyperactivity disorder. J Affect Disord, 82:S17–S23, 2004.

11.

Brawman–Mintzer

, Knapp

, Neitert

: Adjunctive risperidone in generalized anxiety disorder: A double-blind, placebo-controlled study. J Clin Psychiatry, 10:1321–1325, 2005.

12.

Coccaro

, McCloskey

, Fitzgerald

, Phan

: Amygdala and orbitofrontal reactivity to social threat in individuals with impulsive aggression: Biol Psychiatry, 62:168–178, 2007.

13.

Coccaro

, Sripada

, Yanowitch

, Phan

: Corticolimbic function in impulsive aggression behavior. Biol Psychiatry, 69:1153–1159, 2011.

14.

Craft

: Tranquilizers in mental deficiency: Meprobamate. J Ment Defic Res, 2:17–20, 1958.

15.

Dodge

: Emotion and social information processing. In: The Development of Emotion Regulation and Dysregulation, edited by Garber

, and Dodge

K.A.

New York: Cambridge University Press, 159–181, 1991.

16.

Dodge

: Social cognition and children's aggressive behavior. Child Dev, 51:162–170, 1980.

17.

Dodge

, Coie

: Social–information-processing factors in reactive and proactive aggression in children's peer groups. J Pers Soc Psychol, 53:1146–1158, 1987.

18.

Dodge

, Lochman

, Harnish

, Bates

, Pettit

: Reactive and proactive aggression in school children and psychiatrically impaired chronically assaultive youth. J Abnorm Psychol, 106:37–51, 1997.

19.

Farmer

, Arnold

, Bukstein

, Findling

, Gadow

, Li

, Butter

, Aman

: The treatment of severe child aggression (TOSCA) study: Design challenges. Child Adolesc Psychiatry Ment Health, 5:1–11, 2011.

20.

Gadow

: Schizophrenia spectrum and attention-deficit/hyperactivity disorder symptoms in autism spectrum disorder and controls. J Am Acad Child Adolesc Psychiatry, 51:1076–1084, 2012.

21.

Gadow

, Arnold

, Molina

, Findling

, Bukstein

, Brown

, McNamara

, Rundberg–Rivera

, Li

, Kipp

, Schneider

, Farmer

, Baker

, Sprafkin

, Rice

, Bangalore

, Butter

, Buchan–Page

, Hurt

, Austin

, Grondhuis

. Risperidone added to parent training and stimulant medication: Effects on attention–deficit/hyperactivity disorder, oppositional defiant disorder, conduct disorder, and peer aggression. J Am Acad Child Adolesc Psychiatry, 53:948–959, 2014.

22.

Gadow

, Drabick

DAG

: Anger and irritability symptoms among youth with ODD :Cross-informant vs. source-exclusive syndromes. J Abnorm Child Psychol, 40:107–1085, 2012.

23.

Gadow

, Sprafkin

. Child Symptom Inventory–4 Screening and Norms Manual. Stony Brook, NY: Checkmate Plus; 2002.

24.

Gadow

, Sprafkin

: The Symptom Inventories: An Annotated Bibliography. Stony Brook, NY: Checkmate Plus, 2013. Available at www.checkmateplus.com.

25.

Gurka

, Edwards

, Muller

: Avoiding bias in mixed model interference for fixed effects. Stat Med, 30:2696–2707, 2011.

26.

Hanna

, Fluent

, Fischer

: Separation anxiety in children and adolescents treated with risperidone. J Child Adolesc Psychopharmacology, 9:277–283, 1999.

27.

Kaat

, Gadow

, Lecavalier

: Psychiatric symptom impairment in children with autism spectrum disorders. J Abnorm Child Psychol, 41:959–969, 2013.

28.

Kashani

, Deuser

, Reid

: Aggression and anxiety: A new look at an old notion. J Am Acad Child Adolesc Psychiatry, 30:218–223, 1991.

29.

Kelleher

, Jenner

, Cannon

: Psychotic symptoms in the general population: An evolutionary perspective. Br J Psychiatry, 197:167–169, 2010.

30.

King

, Lord

: Is schizophrenia on the autism spectrum?. Brain Res, 1380:34–41, 2011.

31.

Klein

, Manuzza

, Olazagasti

, Roizen

, Hutchinson

, Lashua

, Castellanos

: Clinical and functional outcome of childhood attention-deficit/hyperactivity disorder 33 years later: Arch Gen Psychiatry, 69:1295–303, 2012.

32.

Kraemer

, Wilson

, Fairburn

, Agras

: Mediators and moderators of treatment effects in randomized clinical trials: Arch Gen Psychiatry, 59:877–883, 2002.

33.

Leary

, Collett

, Myers

: Rating Scales. In: Dulcan's textbook of child and adolescent psychiatry, edited by Dulcan

M.K.

. Washington, DC: American Psychiatric Publishing; 2011. Pp. 89–110.

34.

Lende

: The Research Domain Criteria of the NIMH and the RDoC Vision for Mental Health Research and Diagnosis. Plos Blogs February 9, 2014. Available at http://blogs.plos.org/neuroanthropology/2014/02/09/research-domain-criteria-nimh-vision-mental-health-research-diagnosis/

35.

: On efficiency of constrained longitudinal data analysis versus longitudinal analysis of covariance: Biometrics, 66:891–896, 2010.

36.

Marder

, Davis

, Chavinard

: The effects of risperidone on the five dimensions of schizophrenia derived by factor analysis: Combined results of the N. American trials. J Clin Psychiatry, 58:538–546, 1997.

37.

MTA Cooperative Group: A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder: A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder: Arch Gen Psychiatry, 56:1073–1086, 1999a.

38.

MTA Cooperative Group: Moderators and mediators of treatment response for children with attention-deficit/hyperactivity disorder: Arch Gen Psychiatry, 56:1088–1096, 1999b.

39.

Neumann

, Veenema

, Beiderbeck

: Aggression and anxiety: social context and neurobiological links Front. Behav Neurosci, 4:1–16, 2010.

40.

Padgett

, Miltsiou

, Tiffin

: The co-occurrence of nonaffective psychosis and the pervasive developmental disorders: A systematic review. J Intellect Dev Disabil, 35:187–198, 2010.

41.

Pandina

, Canuso

, Turkoz

, Kujawa

, Mahmoud

: Adjunctive risperidone in the treatment of generalized anxiety disorder: a double-blind, prospective, placebo-controlled, randomized trial. Psychopharmacol Bull, 40:41–57, 2007.

42.

Pandina

, Kujawa

, Canusco

, Kosik–Gonzalez

, Turkoz

, Gharabawi

: Adjunctive risperidone in the treatment of generalized anxiety disorder: A double- blind, placebo-controlled, randomized study. Int J Neuropsychopharmacol, 9: S119–S119, 2006.

43.

Ravindran

, Bradbury

, McKay

, daSilva

: Novel uses for risperidone: Focus on depression, anxiety, and behavioral disorders. Expert Opin Pharmacother, 8:1693–1710, 2007.

44.

Sahakian

, Robbins

: Are the effects of psychomotor stimulant drugs on hyperactive children really paradoxical?. Med Hypotheses, 3:154–158, 1977.

45.

SAS Institute: SAS/GIS 9.2: Spatial Data and Procedure Guide. Cary, NC: SAS Institute, Inc.; 2009.

46.

Seo

, Jamieson

, Cosgrove

, Gwizdowski

, Yang

, Sheehan

, McElroy

, Suppes

: Characteristics of responders and non-responders to risperidone monotherapy or placebo in co-occurring bipolar disorder and anxiety disorder. Eur Psychiatry, 28:190–196, 2013.

47.

Sheehan

, McElroy

, Harnett–Sheehan

, Keck

, Janavs

, Rogers

, Gonzalez

, Shivakumar

, Suppes

: Randomized, placebo-controlled trial of risperidone for acute treatment of bipolar anxiety. J Affect Disord, 115:376-385, 2009.

48.

Simon

, Hoge

, Fischmann

, Worthington

, Christian

, Kinrys

, Pollack

: An open-label trial of risperidone augmentation for refractory anxiety disorders: J Clin Psychiatry, 67:381–385, 2006.

49.

Starling

, Dossetor

: Pervasive developmental disorders and psychosis. Curr Psychiatr Rep, 11:190–196, 2009.

50.

Walters

, Singh

, Beale

: Effects of lorazepam on hyperactivity in retarded children: N Z Med J, 86:473–475, 1977.

51.

Zrull

, Westman

, Arthur

, Rice

: A comparison of diazepam, d-amphetamine, and placebo in the treatment of hyperkinetic syndrome in children: Am J Psych, 121:388–389, 1964.