Changes in Positive and Negative Syndrome Scale–Derived Hostility Factor in Adolescents with Schizophrenia Treated with Aripiprazole: Post Hoc Analysis of Randomized Clinical Trial Data

Abstract

Introduction:

This post hoc analysis evaluated the effects of aripiprazole on Positive and Negative Syndrome Scale (PANSS) Hostility factor scores in adolescents with schizophrenia.

Methods:

In total, 302 adolescents (13–17 years) with schizophrenia were enrolled in a 6-week, multicenter, double-blind, randomized, placebo-controlled trial comparing aripiprazole (10 or 30 mg/day) with placebo. The PANSS was the primary outcome measure. To determine the effect of aripiprazole on hostility, a post hoc analysis of the PANSS Hostility factor and individual items was performed.

Results:

Aripiprazole was superior to placebo in reducing PANSS Hostility factor scores in adolescents with schizophrenia. After 6 weeks, aripiprazole 10 mg/day and aripiprazole 30 mg/day showed a statistically significant improvement versus placebo (−3.0, −3.7, versus −2.1; p < 0.05; last observation carried forward [LOCF]) in the PANSS Hostility factor. For aripiprazole 30 mg/day, statistically significant separation from placebo was evident from week 3 through week 6 and at week 6 for aripiprazole 10 mg/day. Individual PANSS Hostility, Uncooperativeness, and Poor Impulse Control Items showed statistically significant improvement with aripiprazole 30 mg/day over placebo at end point.

Conclusions:

This post hoc analysis shows that aripiprazole (10 and 30 mg/day) is an effective treatment for hostility symptoms in adolescents with schizophrenia.

Clinical trials information:

ClinicalTrials.gov identifier: NCT00102063

Introduction

A lthough childhood-onset schizophrenia (before 13 years of age) is rare, one third of adults with a diagnosis of schizophrenia developed the disorder during adolescence (Loranger 1984; Hafner et al. 1993). As adolescent-onset schizophrenia is generally more severe, with a worse prognosis than adult-onset illness (Frazier et al. 2007; Sikich et al. 2008); effective, well-tolerated treatments that address the range of symptom domains in schizophrenia are needed.

Aggressive, violent, and threatening behavior is a frequent reason for admission to a psychiatric facility for individuals with schizophrenia; it can prolong hospitalization and is seen as a major burden for caregivers (Volavka 2002). Hostility and aggression also contribute to the need for seclusion and restraint, which are both restrictive and unpleasant for patients and caregivers (Kranzler et al. 2005). Aggressive and hostile behavior is also one of the most common reasons for referrals to child psychiatrists (Steiner et al. 2003) and common symptoms of early-onset schizophrenia. In the Treatment of Early-Onset Schizophrenia Spectrum Disorders (TEOSS) study, 25% of youths had a history aggression or legal problems (Frazier et al. 2007). Furthermore, aggressive behavior is one of the most common reasons for psychotropic medication use in residential treatment facilities (Connor et al. 1997).

Aripiprazole is an atypical antipsychotic with a novel mechanism of action—partial-agonist activity at dopamine D₂ D₃ and 5-hydroxytryptamine 1A (5-HT_1A) receptors and antagonist activity at 5-HT_2A receptors (Burris et al. 2002; Shapiro et al. 2003; Tadori et al. 2008). The antipsychotic efficacy and safety of aripiprazole has been demonstrated in a double-blind, placebo-controlled study in adolescents with schizophrenia who were randomized to aripiprazole (10 mg/day or 30 mg/day) or placebo for up to 6 weeks (Findling et al. 2008). Aripiprazole was approved by the U.S. Food and Drug Administration for the acute treatment of schizophrenia in adolescents (13–17 years of age) based on this data and is also indicated for maintenance treatment of schizophrenia in adolescents and bipolar disorder in pediatric patients (10–17 years of age) based on extrapolation of adult data. The efficacy of aripiprazole as adjunctive therapy to either lithium or valproate for the acute treatment of manic and mixed episodes associated with bipolar I disorder in pediatric patients (10–17 years of age) can also be extrapolated from adult data. Aripiprazole has also recently been approved for the treatment of pediatric patients (6–17 years of age) with irritability associated with autistic disorder.

Positive and Negative Syndrome Scale (PANSS) factor analyses represents an established methodology for investigating the effects of treatment on five psychopathological symptom domains of schizophrenia—positive, negative, depression/anxiety, cognitive, and hostility symptoms (Lindenmayer et al. 1994; Lindenmayer et al. 1995) and can be used for assessing the effects of therapy on different areas of schizophrenic symptomatology. Aripiprazole has been shown to be an effective treatment for hostility in adult patients with schizophrenia using pooled data from five short-term, double-blind studies comparing aripiprazole with placebo (Volavka et al. 2005; Janicak et al. 2009). In one study, aripiprazole was superior to placebo in reducing hostility measured on the PANSS Hostility item over 4 weeks (Volavka et al. 2005), and in a second evaluation, aripiprazole was significantly better than placebo in improving PANSS Hostility factor scores over 4 weeks (Janicak et al. 2009). However, the effects of aripiprazole on hostility in children/adolescents are unknown. The objective of this post hoc analysis was to determine the effects of aripiprazole on hostility using a PANSS Hostility factor analysis of data from the previously conducted double-blind, placebo-controlled study in adolescents with schizophrenia (Findling et al. 2008). Hostility factor scores were low in this population and contributed approximately 12% to the PANSS Total score. This analysis was conducted as part of a post hoc analysis program for data from this study.

Methods

Study design and subjects

This was a post hoc analysis of data from a 6-week, multicenter, double-blind, randomized, placebo-controlled trial evaluating the efficacy and safety of aripiprazole in adolescents with schizophrenia (Findling et al. 2008). Adolescents, male or female aged 13–17 years of age, with a Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) (American Psychiatric Association 1994) diagnosis of schizophrenia (confirmed using the Schedule for Affective Disorders and Schizophrenia for School-Age Children–Present and Lifetime Version diagnostic interview [K-SADS-PL]) and a PANSS Total score ≥70 were enrolled. All subjects had a K-SADS-PL that excluded other Axis I disorders. Subjects were excluded if they had any current psychiatric co-morbidity requiring pharmacotherapy, any evidence of suicide risk (assessed using the Children's Depression Rating Scale, Revised suicidal ideation item), or a history or current diagnosis of schizoaffective disorder, substance abuse/dependence, mental retardation, major depressive episodes, neuroleptic malignant syndrome, any neurologic disorder other than Tourette's syndrome, severe head trauma, or any unstable medical condition.

Subjects whose illness was resistant to antipsychotic treatment with prior trials of two different antipsychotics of adequate dose and duration were also excluded, as were adolescent girls who were pregnant or breast-feeding and sexually active adolescent boys or adolescent girls who did not agree to abstinence or birth control, to prevent exposure of a fetus to the study drug. Subjects with positive screens for illegal drugs within 3 months of baseline or during the study were either excluded or removed from the study, respectively. Additionally, subjects who had been hospitalized for acute schizophrenia within 2 weeks or enrolled in a clinical trial within 4 weeks of baseline testing were excluded. Subjects were permitted to participate in this study on an outpatient, partial hospitalization or full inpatient basis at any given time of the study. Details of prohibited and concomitant medications have been reported previously (Findling et al. 2008).

Following a ≥3-day antipsychotic washout period, individuals were randomized 1:1:1 to receive placebo or a fixed dose of 10 mg/day or 30 mg/day aripiprazole titrated from 2 mg/day to the target dose over 5 or 11 days, respectively. Target doses (10 and 30 mg/day) were maintained for a minimum of 2 weeks. Individuals experiencing a dose-related tolerability issue were discontinued from the study prior to day 25; after day 25, individuals were allowed one down-titration (5 mg/day for the 10 mg/day arm or 15 mg/day for the 30 mg/day arm).

The study was conducted at 101 global sites, including the United States, Europe, South America, Asia, the Caribbean, and South Africa, and was approved by the local institutional review boards. All patients/caregivers gave written informed consent prior to participation. Adolescents gave written or verbal assent to be in the study.

Efficacy assessments and analysis

The primary outcome measure was the mean change from baseline to end point (last observation carried forward [LOCF] dataset) for the PANSS Total score (Findling et al. 2008). At end point, both aripiprazole doses (10 mg/day and 30 mg/day) showed statistically significant differences from placebo in reduction in PANSS Total scores (10 mg/day−26.7, 30 mg/day−28.6, placebo−21.2). Full details of secondary outcomes are reported in the primary paper (Findling et al. 2008). To maximize interrater reliability between sites, the number of raters at each site was minimized; they were all provided with identical instructions for administering the PANSS and efforts were to be made to ensure that the same rater administered the PANSS for a given patient. In addition, each site was required to demonstrate interrater reliability on an ongoing basis throughout the period in which patients participated in the study. Those raters who failed to achieve the requisite level of interrater reliability were not allowed to rate subjects in the trial. These steps have subsequently been shown to have produced an excellent degree of interrater reliability in PANSS ratings; raters achieved a 75–83% interrater consistency (i.e., weighted kappa value) at the time of training, and at 6-month and 12 month posttraining (Veroff et al. 2008).

For this post hoc analysis conducted subsequent to the primary publication, hostility was measured using the Hostility factor score of the PANSS. The Hostility factor score was derived from the sum of the following four PANSS line item scores (Lindenmayer et al. 1994; Lindenmayer et al. 1995; Marder et al. 1997; Janicak et al. 2009): Positive Subscale items Excitement (P4) and Hostility (P7), and General Psychopathology Subscale items Uncooperativeness (G8) and Poor Impulse Control (G14). The range of scores is from 4 to 24 on the Hostility index.

The mean change from baseline for the PANSS Hostility factor score and individual item scores were analyzed using analysis of covariance (ANCOVA) with treatment and regional strata as the main effects and baseline score as a quantitative covariate. Treatment effects versus placebo were derived as differences between least-squares means (aripiprazole group − placebo group) and the corresponding p values were obtained by the respective t-tests. A significance level of 0.05 (two-tailed) was used in testing the statistical significance of these two comparisons. Analyses were performed on the LOCF and observed cases (OC) datasets.

Results

The study enrolled 302 subjects, of which 294 subjects (efficacy sample) were included in this analysis (placebo, n = 98; aripiprazole 10 mg/day, n = 99; aripiprazole 30 mg/day, n = 97). Disposition of enrolled subjects is shown in Table 1. Baseline demographics were well matched between treatment groups, details of which have been reported previously (Findling et al. 2008). In brief, the patient population was mostly (56.6%) male, with an average age of 15.5 years and average weight of 63.8 kg. The average baseline PANSS Total score of this population was 94.1. Figure 1 shows the mean contribution of the PANSS Hostility factor score to the mean PANSS Total score at study baseline and end point; across treatment arm, the PANSS Hostility factor score contributed 11.8–12.4% to the PANSS Total score. The contribution of the individual Hostility factor items to the PANSS Total score at baseline and end point ranged from 2.7% to 3.3%.

FIG. 1.

Mean contribution of the PANSS Hostility Factor score to PANSS Total score at baseline and end point. PANSS = Positive and Negative Syndrome Scale.

Table 1.

Disposition of Enrolled Subjects

n (%)	Placebo	Aripiprazole 10 mg/day	Aripiprazole 30 mg/day
Randomized	100	100	102
Discontinued	10 (10)	16 (16)	18 (18)
Adverse event	2 (2)	7 (7)	4 (4)
Lack of efficacy	1 (1)	5 (5)	1 (1)
Withdrew consent	5 (5)	4 (4)	12 (12)
Lost to follow-up	1 (1)	0 (0)	0 (0)
Other	1 (1)	0 (0)	1 (1)
Completed	90 (90)	84 (84)	84 (82)
Efficacy sample	98	99	97

Figure 2 shows the mean change from baseline to week 6 for PANSS Hostility factor score. At end point (week 6), both 10 and 30 mg/day aripiprazole doses produced statistically significantly greater improvements than placebo in reducing hostility in adolescents with schizophrenia, as measured by the PANSS Hostility factor scores (10 mg/day, −3.0; 30 mg/day, −3.7; placebo, −2.1; t = 1.97, degress of freedom [df] = 288, p < 0.05 and t = 2.68, df = 288, p < 0.05 vs. placebo, respectively [LOCF]). Aripiprazole 30 mg/day produced statistically significantly greater improvements than placebo on the Hostility factor scores from week 3 onward. OC analysis produced similar results (data not shown), although aripiprazole 10 mg/day was statistically superior to placebo from week 4 onward. The contribution of the mean change in PANSS Hostility factor score at end point toward the overall improvement in PANSS Total score was 9.9% for placebo, 11.1% for aripiprazole 10 mg/day, and 12.8% for aripiprazole 30 mg/day.

FIG. 2.

Mean change from baseline in PANSS Hostility Factor score. Baseline PANSS Hostility factor score placebo = 11.22; 10 mg/day = 11.03; 30 mg/day = 11.74. p < 0.05. PANSS = Positive and Negative Syndrome Scale.

In addition, mean change from baseline to week 6 for each of the individual PANSS Hostility line items was analyzed (Fig. 3). Aripiprazole 30 mg/day produced statistically significantly greater improvements than placebo in the PANSS Hostility, Uncooperativeness, and Poor Impulse Control line items (all p ≤ 0.05) at end point, with a trend toward significance for the excitement line item at end point (p = 0.07). Aripiprazole 10 mg/day produced statistically significantly greater improvements than placebo in the PANSS Poor Impulse Control line item at end point (p < 0.05), although improvements with Hostility, Uncooperativeness, and Excitement line items failed to reach significance at end point. The contribution of the individual PANSS Hostility factor items to mean change in PANSS Total at end point was 1.8–3.2% for placebo, 2.2–3.3% for aripiprazole 10 mg/day, and 2.7–3.6 for aripiprazole 30 mg/day.

FIG. 3.

Mean change from baseline in PANSS Hostility factor line items: (A) Excitement (item P4); (B) Hostility (item P7); (C) Uncooperativeness (item G8); (D) Poor Impulse Control (item G14). Baseline line item scores: Excitement line item placebo = 2.84, aripiprazole 10 mg/day = 2.89, aripiprazole 30 mg/day = 2.92; Hostility line item placebo = 2.89, aripiprazole 10 mg/day = 2.68, aripiprazole 30 mg/day = 2.96; Uncooperativeness line item placebo = 2.61, aripiprazole 10 mg/day = 2.69, aripiprazole 30 mg/day = 2.74; Poor Impulse Control line placebo = 2.89, aripiprazole 10 mg/day = 2.78; aripiprazole 30 mg/day = 3.12. (*) p ≤ 0.05 versus placebo; (#) p = 0.08 versus placebo; (†) p = 0.07 versus placebo. PANSS = Positive and Negative Syndrome Scale.

Discussion

This post hoc analysis of data from a 6-week, double-blind, placebo-controlled trial has demonstrated that aripiprazole, at doses of 10 and 30 mg/day, is significantly more effective than placebo at improving Hostility factor scores in adolescents with schizophrenia. Analysis of individual PANSS Hostility factor items indicated that improvements in the Hostility, Poor Impulse Control, and Uncooperativeness items with aripiprazole 30 mg/day separated statistically from placebo; however, the individual contribution of each item to overall improvement was not evaluated. Statistically significant improvements in PANSS Hostility factor scores were seen from week 3 of treatment with aripiprazole 30 mg/day, an important finding given that rapid resolution of symptoms is critical in the acute treatment setting.

Aripiprazole has been shown to be effective in adolescents with schizophrenia, as measured by significantly greater improvement over placebo in PANSS Total scores as well as PANSS Positive and Negative subscale scores (Findling et al. 2008). It is possible that the pharmacodynamic actions of aripiprazole that generate changes in multiple domains may contribute to the improvement in hostility, and examination of such hypotheses should be the plan of future analyses. The results reported here extend these findings beyond the core positive and negative symptoms of schizophrenia and add to the growing evidence base supporting the use of aripiprazole as a treatment option in adolescents with schizophrenia. Furthermore, both aripiprazole treatment groups demonstrated significant improvement in Pediatric Quality of Life Enjoyment and Satisfaction Questionnaire (PQ-LES-Q) Overall scores and Children's Global Assessment Scale (CGAS) scores compared with placebo at end point; however, improvement in the PQ-LES-Q Total score did not significantly differ between the aripiprazole treatment groups and placebo at study end point (Findling et al. 2008). It is possible that improvement in the PANSS Hostility factor scores in the improvement observed in these functional domains.

Although the antihostility effects of aripiprazole and other atypical agents have been demonstrated previously in adults with schizophrenia (Czobor et al. 1995; Aleman and Kahn 2001; Citrome et al. 2001; Volavka et al. 2005; Janicak et al. 2009), this is the first analysis, to our knowledge, to investigate the antihostility effects of an atypical antipsychotic in adolescents. Given the relative lack of data from controlled clinical trials investigating the use of atypical antipsychotics in adolescents with schizophrenia, current recommendations for the use of atypical agents in adolescents are primarily based on the adult literature (AACAP 2000). This analysis is the first to suggest that aripiprazole has benefits for the treatment of hostility symptoms in adolescents with schizophrenia, one of the most worrying cluster of symptoms, and replicates findings seen in adults. Expansion of these findings to assess the efficacy of aripiprazole to treat hostility associated with other psychiatric diagnoses would be of clinical value.

The findings reported here should be considered in light of several limitations. First, while aripiprazole was associated with significantly greater improvements in Hostility factor scores than placebo, the clinical relevance of this level of symptom improvement is unknown. This was a post hoc analysis of primary efficacy end-point data, the post hoc nature of this analysis means that the results should be considered to be preliminary; prospectively designed studies are needed to confirm these findings. Second, this study was not designed to investigate the effects of aripiprazole on hostility, and investigators may have overlooked hostility symptoms. It should be noted that baseline Hostility scores were low and the subjects treated here do not represent a hostile and aggressive subject group. However, despite relatively low baseline Hostility factor scores, improvements in Hostility factor scores were evident during treatment and accounted for over 10% of the total improvement in PANSS Total symptom severity. In addition, these analyses did not control for secondary medication effects such as adverse events (e.g., sedation) or other clinical improvements (e.g., changes in positive symptoms) that may contribute to the outcomes. Finally, this was a fixed-dose study and subjects in the 10 mg/day dose group may have experienced greater improvements in PANSS Hostility factor items given the option to receive higher doses.

In conclusion, this post hoc analysis of data from adolescents with schizophrenia shows that aripiprazole can effectively alleviate symptoms of hostility and may be considered to be a useful therapeutic option in adolescents displaying these symptoms.

Footnotes

Disclosures

Adelaide S. Robb receives or has received research support from, acted as a consultant to, and/or served on a speaker's bureau for Abbott, Janssen, Forest, McNeil, Eli Lilly, Lundbeck, Pfizer, Otsuka, Bristol-Myers Squibb, GlaxoSmithKline, Supernus, Sepracor, Merck, the National Institute of Child Health and Human Development, and National Institute of Mental Health. William H. Carson, Margaretta Nyilas, Mirza Ali. Robert A. Forbes, and Taro Iwamoto are employees of Otsuka Pharmaceutical Development & Commercialization Inc. Sheila Assunção-Talbott is a former employee of Bristol-Myers Squibb. Richard Whitehead is a former employee of Otsuka America Pharmaceutical, Inc. Andrei Pikalov is an employee of Otsuka America Pharmaceutical, Inc.

Acknowledgments

Editorial support for the preparation of this manuscript was provided by Dr. Suzanne Patel of Ogilvy Healthworld Medical Education; Ogilvy's funding was provided by Otsuka America Pharmaceutical, Inc.

This study was supported by Otsuka Pharmaceutical Co., Ltd.

References

AACAP. AACAP official action. Summary of the practice parameters for the assessment and treatment of children and adolescents with schizophrenia. American Academy of Child and Adolescent Psychiatry. J Am Acad Child Adolesc Psychiatry, 39:1580–1582. 2000.

Aleman

, Kahn

. Effects of the atypical antipsychotic risperidone on hostility and aggression in schizophrenia: A meta-analysis of controlled trials. Eur Neuropsychopharmacol, 11:289–293. 2001.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 4 ^th. Washington (DC): American Psychiatric Association, 1994(DSM-IV).

Burris

, Molski

, Xu

, Ryan

, Tottori

, Kikuchi

, Yocca

, Molinoff

. Aripiprazole, a novel antipsychotic, is a high-affinity partial agonist at human dopamine D2 receptors. J Pharmacol Exp Ther, 302:381–389. 2002.

Citrome

, Volavka

, Czobor

, Sheitman

, Lindenmayer

, McEvoy

, Cooper

, Chakos

, Lieberman

. Effects of clozapine, olanzapine, risperidone, and haloperidol on hostility among patients with schizophrenia. Psychiatric Services, 52:1510–1514. 2001.

Connor

, Ozbayrak

, Kusiak

, Caponi

, Melloni

RH:

Jr . Combined pharmacotherapy in children and adolescents in a residential treatment center. J Am Acad Child Adolesc Psychiatry, 36:248–254. 1997.

Czobor

, Volavka

, Meibach

. Effect of risperidone on hostility in schizophrenia. J Clin Psychopharmacol, 15:243–249. 1995.

Findling

, Robb

, Nyilas

, Forbes

, Jin

, Ivanova

, Marcus

, McQuade

, Iwamoto

, Carson

. A multiple-center, randomized, double-blind, placebo-controlled study of oral aripiprazole for treatment of adolescents with schizophrenia. Am J Psychiatry, 165:1432–1441. 2008.

Frazier

, McClellan

, Findling

, Vitiello

, Anderson

, Zablotsky

, Williams

, McNamara

, Jackson

, Ritz

, Hlastala

, Pierson

, Varley

, Puglia

, Maloney

, Ambler

, Hunt-Harrison

, Hamer

, Noyes

, Lieberman

, Sikich

. Treatment of early-onset schizophrenia spectrum disorders (TEOSS): Demographic and clinical characteristics. J Am Acad Child Adolesc Psychiatry, 46:979–988. 2007.

10.

Hafner

, Maurer

, Loffler

, Riecher-Rossler

. The influence of age and sex on the onset and early course of schizophrenia. Br J Psychiatry, 162:80–86. 1993.

11.

Janicak

, Glick

, Marder

, Crandall

, McQuade

, Marcus

, Eudicone

, Assuncao-Talbott

. The acute efficacy of aripiprazole across the symptom spectrum of schizophrenia: A pooled post hoc analysis from 5 short-term studies. J Clin Psychiatry, 70:25–35. 2009.

12.

Kranzler

, Roofeh

, Gerbino-Rosen

, Dombrowski

, McMeniman

, DeThomas

, Frederickson

, Nusser

, Bienstock

, Fisch

, Kumra

. Clozapine: Its impact on aggressive behavior among children and adolescents with schizophrenia. J Am Acad Child Adolesc Psychiatry, 44:55–63. 2005.

13.

Lindenmayer

, Bernstein-Hyman

, Grochowski

. A new five factor model of schizophrenia. Psychiatr Q, 65:299–322. 1994.

14.

Lindenmayer

, Grochowski

, Hyman

. Five factor model of schizophrenia: Replication across samples. Schizophr Res, 14:229–234. 1995.

15.

Loranger

. Sex difference in age at onset of schizophrenia. Arch Gen Psychiatry, 41:157–161. 1984.

16.

Marder

, Davis

, Chouinard

. The effects of risperidone on the five dimensions of schizophrenia derived by factor analysis: Combined results of the North American trials. J Clin Psychiatry, 58:538–546. 1997.

17.

Shapiro

, Renock

, Arrington

, Chiodo

, Liu

, Sibley

, Roth

, Mailman

. Aripiprazole, a novel atypical antipsychotic drug with a unique and robust pharmacology. Neuropsychopharmacology, 28:1400–1411. 2003.

18.

Sikich

, Frazier

, McClellan

, Findling

, Vitiello

, Ritz

, Ambler

, Puglia

, Maloney

, Michael

, De Jong

, Slifka

, Noyes

, Hlastala

, Pierson

, McNamara

, Delporto-Bedoya

, Anderson

, Hamer

, Lieberman

. Double-blind comparison of first- and second-generation antipsychotics in early-onset schizophrenia and schizo-affective disorder: Findings from the treatment of early-onset schizophrenia spectrum disorders (TEOSS) study. Am J Psychiatry, 165:1420–1431. 2008.

19.

Steiner

, Saxena

, Chang

. Psychopharmacologic strategies for the treatment of aggression in juveniles. CNS Spectr, 8:298–308. 2003.

20.

Tadori

, Forbes

, McQuade

, Kikuchi

. Characterization of aripiprazole partial agonist activity at human dopamine D(3) receptors. Eur J Pharmacol, 597:27–33. 2008.

21.

Veroff

, Nyilas

, Bohn

, Kingsley

, Ramirez

. A positive international adolescent schizophrenia clinical trial: Ensuring inter-rater reliability. 48th Annual NCDEU Meeting, May 27–30 Phoenix: Arizona, 2008.

22.

Volavka

. Neurobiology of Violence, 2nd. Washington D, London: American Psychiatric Publishing, Inc., 2002.

23.

Volavka

, Czobor

, Citrome

, McQuade

, Carson

, Kostic

, Hardy

, Marcus

. Efficacy of aripiprazole against hostility in schizophrenia and schizoaffective disorder: Data from 5 double-blind studies. J Clin Psychiatry, 66:1362–1366. 2005.