Impulsive Aggression,Delay Discounting,and Adolescent Suicide Attempts: Effects of Current Psychotropic Medication Use and Family History of Suicidal Behavior

Abstract

Objective:

Impulsive-aggressive behaviors have been consistently implicated in the phenomenology, neurobiology, and familial aggregation of suicidal behavior. The purpose of this study was to extend previous work by examining laboratory behavioral measures of delayed reward impulsivity and impulsive aggression in adolescent suicide attempters and never-suicidal comparison subjects.

Methods:

Using the Point Subtraction Aggression Paradigm (PSAP) and the Delay Discounting Task (DDQ), the authors examined delay discounting and impulsive aggression in 40 adolescent suicide attempters, ages 13–18, and 40 never-suicidal, demographically matched psychiatric comparison subjects.

Results:

Overall, suicide attempters and comparison subjects performed similarly on the PSAP and DDQ. There was a significant group by current psychotropic medication use interaction (p=0.013) for mean aggressive responses on the PSAP. Group comparisons revealed that attempters emitted more aggressive responses per provocation than comparison subjects, only in those not on psychotropic medication (p=0.049), whereas for those currently treated with psychotropic medication, there were no group differences (p>0.05). This interaction effect was specific to current antidepressant use. Among all subjects, family history of suicidal behavior (suicide or suicide attempt) in first degree relatives was significantly correlated with both delay discounting (r=−0.22, p=0.049), and aggressive responding (r=0.27, p=0.015). Family history of suicidal behavior was associated with delay discounting, but not with aggressive responding on the PSAP, after controlling for relevant covariates.

Conclusions:

In this study, impulsive-aggressive responding was associated with suicide attempt only in those not being treated with antidepressants. Future work to replicate and extend these findings could have important therapeutic implications for the treatment of depressed suicide attempters, many of whom are affected by impulsive aggression.

Introduction

Adolescent suicide remains a serious public health concern. In 2010, suicide was the second leading cause of death in United States youth ages 12–17 years (Centers for Disease Control and Prevention 2013). A nonfatal suicide attempt is the strongest predictor of youth suicide (Marttunen et al. 1992; Shaffer et al. 1996; Brent et al. 1999). In 2011, 7.8% of high school students reported having attempted suicide in the past year, and 2.4% had made an attempt requiring medical treatment (Eaton et al. 2012). Numerous studies have identified risk factors for suicide attempts in adolescents (Pfeffer, et al. 1988; Beautrais et al. 1996; Gould et al. 1998; Beautrais et al. 1999; Borowsky et al. 2001; Fergusson et al. 2000), but our understanding of the etiology of adolescent suicidal behavior is still not well understood. Further research to better characterize adolescent suicide attempters is warranted to identify unique clinical, familial, and behavioral characteristics that may inform prevention and treatment strategies for youth at highest risk of suicide.

Impulsive aggression, or the tendency to react to frustration or provocation with hostility or aggression, is a behavioral trait that has been linked with suicide and attempted suicide in adolescents and young adults (Brent et al. 1993a; Stein et al. 1998; Dumais et al. 2005; McGirr et al. 2008). Neurobiological studies have shown a strong concordance between impulsive aggression and suicidal behavior, with both behaviors correlated with reduced serotonergic function (Mann 1998). Impulsive aggression also has been consistently implicated in the familial aggregation of suicidal behavior (Brent 2010), with family studies showing that higher levels of impulsive aggression in adolescents who attempted or completed suicide are associated with greater familial aggregation of suicidal behavior (Brent et al. 1996; Johnson et al. 1998). Moreover, higher levels of impulsive aggression in parents are associated with offspring suicidal behavior (Brent et al. 2002), and the familial aggregation of suicidal behavior appears to be mediated by the familial aggregation of impulsive aggression (Brent et al. 2003; McGirr et al. 2009).

Another commonly studied risk factor for suicide and suicidal behavior in young people is impulsivity (Beautrais et al. 1999; Kingsbury et al. 1999; Horesh 2001; Bridge et al. 2006). Impulsivity has been defined variously as a propensity to act on impulse rather than forethought, an indifference to future consequences, and an inability to inhibit behaviors that are inappropriate to the situation, which may or may not include aggressive behaviors (Eysenck and Eysenck 1980; Keilp et al. 2006; Reynolds et al. 2006). Findings relating impulsivity to suicidal behavior in young people have shown mixed evidence. Kingsbury and colleagues (1999), for example, reported a significant relationship between impulsivity and intentional overdose in adolescents, which persisted even after controlling for depression. However, in two other studies, the association between impulsivity and suicide attempts was attenuated after controlling for key covariates, including depression (Bridge et al. 2012) and hopelessness (Beautrais et al. 1999). In terms of behavioral measures of impulsivity, one study has reported increased impulsive responding in adolescent suicide attempters (Horesh 2001), whereas another study reported differences on a reward-directed measure but no differences on a measure of response inhibition (Dougherty et al. 2009).

Accumulating evidence from studies of adults suggests that delay discounting may also be a relevant vulnerability factor for suicidal behavior (Dombrovski et al. 2011; Liu et al. 2012). Delay discounting describes the extent to which an individual chooses a smaller immediate reward over a larger but delayed reward because the subjective value of the reward decreases as a function of time (Reynolds 2006). In this regard, delay discounting provides an index of impulsiveness in humans (Green and Myerson 2004). Dombrovski et al. (2011) reported, in a clinical sample with late-life depression, that high-lethality attempters were more willing to delay future monetary rewards (i.e., less impulsive), compared with low-lethality attempters and suicide ideators. Conversely, participants who made low-lethality attempts displayed an exaggerated preference for immediate rather than larger delayed rewards (i.e., were more impulsive). In an adult sample of substance users with and without suicide attempt history, Liu et al. (2012) found a significant interaction between history of attempted suicide and delay discounting based on reward size. Specifically, those without a history of suicide attempt discounted the value of delayed smaller rewards more than delayed larger rewards, whereas those with a history of suicide attempt discounted consistently across reward sizes, showing similar discounting rates for both small and large rewards.

To our knowledge, no study to date has compared adolescent suicide attempters and nonattempters using a laboratory behavioral measure of impulsive aggression. Laboratory tasks that directly assess impulsive-aggressive behaviors offer several potential advantages over self-report assessments. First, laboratory-based procedures measure more specific behavioral processes than self-report measures (Reynolds et al. 2008). Second, rater bias is eliminated when behaviors are recorded in laboratory tasks, whereas self-report measures require subjects to self-evaluate and disclose their own behaviors. Finally, laboratory approaches measure both trait- and state-dependent aspects of impulsive-aggression, making these measures potentially useful targets for testing the role of impulsive aggression in the prediction and mediation of treatment response of adolescent suicide attempters (Cherek et al. 1997; Reynolds et al. 2006).

With regard to delay discounting, there have been only two studies examining delayed reward impulsivity in adolescents with suicide attempt history (Dougherty et al. 2009; Mathias et al. 2011). Mathias et al. (2011), in an inpatient sample of adolescent girls, found that participants with multiple suicide attempts were distinguished from single attempt or nonattempter participants on self-ratings of depression and aggression, and that they performed more impulsively on a measure of delayed reward, as assessed by the Two Choice Impulsivity Paradigm (TCIP) (Dougherty, et al. 2005). Dougherty et al. (2009), in a sample of adolescents with nonsuicidal self-injury (NSSI) with or without attempted suicide, found that participants with NSSI and suicide attempt history displayed elevated impulsivity on the TCIP compared with NSSI-only participants. Together, these studies demonstrate that immediate gratification at the expense of longer-term gain may be more salient for girls with multiple attempts and adolescents with NSSI and suicide attempts. Nevertheless, youth who engage in NSSI and suicidal behavior may demonstrate a risk profile distinct from those who attempt suicide with no history of NSSI (Muehlenkamp and Gutierrez 2007).

The aim of the current study was to investigate laboratory behavioral measures of impulsive aggression and delay discounting in a previously described sample of adolescents with a history of attempted suicide (Bridge et al. 2012). Using a case–control design, Bridge et al (2012) found that adolescents with a history of suicide attempt performed significantly worse on the Iowa Gambling Task, a laboratory betting task that simulates real-life decision making (Bechara 2007), than never-suicidal comparison subjects. Findings of impaired decision making remained significant even after accounting for key clinical and behavioral risk factors. In the parent study, current psychotropic medication use was the only variable that remained a significant predictor of suicide attempt status after controlling for other factors that differed between the groups; the majority of psychotropic medication use among participants involved antidepressant treatment (Bridge et al. 2012). Although there is some evidence that serotonergic antidepressants may reduce impulsiveness, aggression, and other risk factors for suicidal behavior (Bond 2005), concerns about the use of antidepressants in children and adolescents were raised in 2004 when the United States Food and Drug Administration (FDA) reported an increased risk of suicidal thinking and behavior associated with their use (Hammad et al. 2006). However, the benefits of antidepressants may outweigh the risk of emergent suicidal thoughts and suicide attempts (Bridge et al. 2007).

In the current study, we hypothesized that suicide attempters would display more laboratory-induced impulsive aggression and discount more by delay in comparison with control subjects, controlling for the effects of current psychotropic medication use. Because study inclusion criteria allowed for a history of suicidal behavior in first degree relatives of both attempters and comparison subjects, we also explored the associations among impulsive aggression, delay discounting, and family history of suicidal behavior. Finally, we examined correlations between self-report measures of impulsivity/aggression and behavioral measures of delay discounting and impulsive aggression. These analyses address the basic question of whether self-report and behavioral measures assess the same processes. Typically, correlations between psychometric and behavioral measures of impulsiveness and impulsive aggression are low or nonexistent (Cherek et al. 1997; Horesh 2001; Reynolds et al. 2006). This lack of association is likely the result of differences in the breadth/specificity of behaviors assessed by self-report and behavioral methods, with performance-based measures modeling more specific behavioral processes.

Methods

Sample

This sample has been described in detail in a previous report (Bridge et al. 2012). Briefly, 40 youth, 13–18 years of age, who had attempted suicide, were compared with 40 youth, matched on age (±1 year), sex, and race, who had never engaged in suicidal behavior or had suicidal ideation. Suicide attempt was defined as self-injurious behavior with stated or inferred intent to die, within 1 year of the recruitment date. Youth with other types of self-injurious behavior without the intent to die were excluded. Comparison subjects were receiving treatment for psychiatric or behavioral health concerns at the same community behavioral health settings as attempters. Exclusion criteria for both groups included: Intelligence quotient (IQ)<70, being non-English-speaking, and having an out of home placement.

The study sample was primarily female (75%), white non-Hispanic (70%), and the mean (SD) age was 15.6 (1.3) at the index date (Bridge et al. 2012). Attempters and comparison subjects were comparable in terms of IQ; pubertal level; household income; rate of family history of suicidal behavior; and rates of anxiety, somatic, and behavioral disorders. Attempters had significantly higher rates of affective disorders and current psychotropic medication use, and had higher self-reported impulsivity and hostility than comparison subjects.

The study was approved by the Institutional Review Board of The Research Institute at Nationwide Children's Hospital. Informed consent and assent were obtained from all participants and their parents (if the participant was under age 18).

Assessments

Demographic information was elicited from all subjects and parents using the General Information Sheet (Brent et al. 1993a). IQ was assessed by the Kaufman Brief Intelligence Test, 2nd Edition (Kaufman and Kaufman 2004). Lifetime history of suicide attempts, number of suicide attempts, methods, medical lethality, and triggering events were assessed using the Columbia University Suicide History Form (Mann et al. 1992). Psychiatric problems in youth were established using the Diagnostic and Statistical Manual of Mental Disorders (DSM)-oriented scales of the Child Behavior Checklist (Achenbach and Rescorla 2001; Achenbach et al. 2003); the borderline cut-point (T score≥65) was used to indicate the presence of a DSM disorder (Achenbach and Rescorla 2001). Family history of suicidal behavior was assessed from a series of questions adapted from the Family History Screen (Weissman et al. 2000). Psychotropic medication history was assessed by the Services Assessment for Children and Adolescents (Stiffman et al. 2000). In this study, current psychotropic medication use refers to use of antidepressants, antipsychotics, antianxiety agents, mood stabilizers, or stimulants.

Self-reported aggression was rated using the Buss-Perry Aggression Questionnaire-Short Form (BPAQ-SF) (Diamond and Magaletta 2006). In this study, only the hostility subscale was used, because it was shown to distinguish attempters and comparison subjects in the primary study (Bridge et al. 2012). Self-reported impulsivity was assessed using the Barratt Impulsiveness Scale-Adolescent version (Fossati et al. 2002), a downward extension of the widely used adult Barratt Impulsivity Scale (Barratt 1965). In this study, only the total score was calculated, as it has been recommended as the most appropriate index of impulsivity for research with adolescents (Fossati et al. 2002).

Impulsive aggression was assessed for all participants using the Point Subtraction Aggression Paradigm (PSAP) (Cherek 1992; Cherek et al. 1997; Cherek and Lane 1999), a well-validated method of provoking aggression in a laboratory setting through a computer game. During the PSAP, participants have the option to press buttons on a response panel that will either accumulate points exchangeable for money (Button A; nonaggressive responding), subtract points from a fictitious person paired with the research participant (Button B; aggressive responding), or protect the participant from periodic point subtractions attributed to the fictitious subject (Button C; escape responding).

Prior to beginning the PSAP, participants were told that their response panel was linked to one of several other similar panels and they would be competing against other individuals located in other laboratories. Participants were informed that the main goal of the game was to earn as many points as possible by pressing Button A. Participants were instructed that pressing Button A 100 consecutive times would earn 10 cents, pressing Button B 10 times would subtract 10 cents from other participants but would not be added to the participant's own money, and pressing Button C 10 times would protect the participant's points for a period of time. Once Button A, B, or C was selected, then only that response option was available until the required ratio of 100 or 10 responses was finished, and then all three response options were available. Participants were provoked periodically during the session by having points taken away by the fictitious subject with whom they were paired. In fact, all point subtractions were presented randomly by the computer program. A running total was displayed at the top of the monitor, which allowed participants to know how much money was subtracted from their total; the running total was enlarged and flashed on the screen when money was being subtracted.

Each participant completed a single 25 minute PSAP session. The construct validity of a single session of the PSAP has been established (Golomb et al. 2007). The external validity of the PSAP for children and adolescents has been supported by prior studies of aggressive children with attention-deficit/hyperactivity disorder (ADHD) and in high school aged athletes participating in high physical contact versus those participating in low physical contact sports (Casat et al. 1995; Huang et al. 1999).

Delay discounting was assessed for all participants with a computerized, question-based delay discounting task (DDQ) (Richards et al. 1999). For this measure, participants were presented a choice of $10 available after a given delay (i.e., 1, 2, 30, 180, 365 days) and a smaller amount of money available immediately (e.g., “Would you rather have $10 in 2 days or $2 now?”). This task used an adjusting amount procedure that adjusts the immediate amount in increments of ±$0.50 to derive an indifference point between the delayed-standard and immediate-adjusting options for each of the five delays assessed. An indifference point reflected the smallest amount of money an individual chose to immediately receive instead of the delayed standard ($10) at the specific delay. The order of the questions was determined by the computer and presented at random. In the end, each participant answered 60 questions in total. Indifference points across the five delays were characterized with an area under the curve (AUC) method, with smaller area values indicating greater discounting and greater impulsivity (Myerson et al. 2001). This method avoids certain systematic errors in characterizing discounting data that occur when fitting these data to a hyperbolic decay function, another commonly reported method for analyzing discounting data (Mazur 1987). Participants were told that each choice they made was important, as at the end of the task one question would be chosen at random and they would get the dollar amount that they chose.

Statistical analysis

Aggressive responding per provocation on the PSAP and AUC data on the DDQ were inspected for normality using Shapiro–Wilk tests. Normality criteria for both measures were not satisfied. PSAP aggressive responding data were transformed using a square root function, and DDQ data were transformed using a log-10 function.

Paired-samples t tests and McNemar's χ² were used, as appropriate, to compare clinical characteristics, self-report measures of impulsivity and hostility, and behavioral measures of delay discounting and impulsive aggression between attempters and comparison subjects. Potential group differences in delay discounting and impulsive aggression as a function of current exposure to psychotropic medications were examined using separate between-subjects two way ANOVA. Specifically, a two way [(suicide attempters vs. comparison subjects)×(current psychotropic medication use vs. no psychotropic medication use)] ANOVA was run to determine the potential main and interactive effects of psychotropic drug exposure on impulsive aggression and delay discounting. Partial eta-squared (η_p ²) was provided as a measure of effect size. For η_p ², 0.01, 0.06, and 0.14 are considered small, medium, and large effect sizes (Cohen 1988).

Spearman's rank correlation coefficients were computed to assess associations between the measures of delay discounting and impulsive aggression, self-reported impulsivity and hostility, and variables that differed significantly between groups. Post-hoc analyses exploring the association among impulsive aggression, delay discounting, and family history of suicidal behavior were conducted in three phases. First, impulsive aggression and delay discounting were screened for their association with family history of suicidal behavior using Spearman's rank correlation. Next, participants with a family history of suicidal behavior were compared with those without on demographic and clinical characteristics by using t tests, χ², and Fisher's exact test, as appropriate. In the third phase, two way ANOVAs and analysis of covariance (ANCOVA) were used to examine possible the effect of group, and family history of suicidal behavior on impulsive aggression and delay discounting, covarying for potential confounds.

All statistical tests were two tailed, and p values <0.05 were considered statistically significant. Statistical analyses were conducted with SPSS, version 21.0 (IBM SPSS Statistics, Somers, NY) and STATA/IC, version 13.1 (StataCorp LP, College Station, TX).

Results

Group comparisons

Characteristics of attempters and comparison subjects are shown in Table 1. As reported previously (Bridge et al. 2012), attempters had higher rates of current psychotropic medication use and greater self-reported impulsivity and hostility than the comparison subjects. There were no significant differences between suicide attempters and comparison subjects in the mean square root transformed number of aggressive responses per provocation on the PSAP (7.17 [SD=3.92] vs. 6.80 [3.87]; t₃₉=0.40, p=0.69) or mean log-10 transformed total AUC on the DDQ (−0.57 [0.31] vs. −0.62 [0.32]; t₃₉=0.53, p=0.60).

Table 1.

Characteristics of Adolescent Suicide Attempters and Never-Suicidal Comparison Subjects

Characteristic	Suicide attempters (n=40)		Comparison subjects (n=40)		Analysis
	Mean	SD	Mean	SD	Statistic ^a	df	p Value
IQ	97.3	14.7	98.3	13.8	t=-0.30	39	0.77
BPAQ-SF hostility score	7.1	3.8	4.8	3.4	t=2.67	36	0.01
BIS-A total score	74.4	11.6	68.6	10.5	t=2.67	39	0.01
DDQ: Log-10 transformed AUC	−0.57	0.31	−0.62	0.37	t=0.53	39	0.60
PSAP: Square root transformed aggressive responding per provocation	7.17	3.92	6.80	3.87	t=0.40	39	0.69

	n	%	n	%
Race/ethnicity^b
White, non-Hispanic	28	70.0	28	70.0
Other race, ethnicity	12	30.0	12	30.0
CBCL DSM-oriented scales
Affective disorders	28	70.0	16	40.0	χ²=6.6	1	0.01
Current medication use
Any psychotropic medication use	30	75.0	13	32.5	χ²=13.8	1	<0.001
Family history of suicidal behavior,^c % yes	13	32.5	10	25.0	χ²=0.8	1	0.37

Paired t test or McNemar's χ².

Comparison subjects were matched to suicide attempters on race/ethnicity.

First-degree relatives.

IQ, intelligence quotient: BPAQ-SF, Buss–Perry Aggression Questionnaire—Short Form; BIS-A, Barratt Impulsivity Scale—Adolescent version; DDQ, Question-Based Delay Discounting Measure; AUC, area under the curve; PSAP, Point Subtraction Aggression Paradigm; CBCL, Child Behavior Checklist; DSM, Diagnostic and Statistical Manual of Mental Disorders.

Spearman rank correlations

The correlation between delay discounting and aggressive responding on the PSAP was not significant (p=0.28) (Table 2). Delay discounting correlated significantly with IQ (p=0.013). The correlation between delay discounting and IQ was significant in attempters (r=0.33, n=40, p=0.036) but not in comparison subjects (r=0.23, n=40, p=0.15). Aggressive responding on the PSAP correlated significantly with IQ (p<0.001) and self-reported hostility (p=0.022). The correlation between aggressive responding on the PSAP and IQ was significant in both attempters (r =−0.44 n=40, p=0.005) and comparison subjects (r =−0.37 n=40, p=0.019). The association between aggressive responding and hostility was significant in comparison subjects (r =−0.46 n=38, p=0.003) but not in attempters (r =−0.17 n=40, p=0.305).

Table 2.

Spearman Correlation Matrix

	1	2	3	4	5	6	7
1. DDQ: Log-10 transformed area under the curve	-
2. PSAP: Square root transformed aggressive responding per provocation	−0.12	-
3. Race/ethnicity	0.10	0.23^*	-
4. IQ	0.28^*	−0.39^**	−0.16	-
5. Current psychotropic medication use^a	−0.01	−0.02	−0.21	−.04	-
6. Family history of suicidal behavior^b	−0.22^*	.27^*	0.31^**	−0.01	0.04	-
7. Barratt Impulsivity Scale – Adolescent version total score	−0.09	−0.07	−0.10	−0.13	0.08	−0.13	-
8. Buss Perry Aggression Questionnaire – Short form hostility subscale	0.04	−0.26^*	0.09	0.15	0.11	−0.02	0.49^**

Includes antidepressants, antipsychotics, antianxiety agents, mood stabilizers, and stimulants.

First-degree relatives.

p<0.05; ^** p<0.01.

DDQ indicates Question-based Delay Discounting Measure; PSAP, Point Subtraction Aggression Paradigm.

Effect of psychotropic medication use on impulsive aggression and delay discounting

For the PSAP, the two way ANOVA revealed a significant group times psychotropic medication exposure interaction effect, F(1,76)=6.48, p=0.013, η_p ²=0.079, indicating that the groups differed significantly in aggressive responding per provocation by current exposure to psychotropic medication (Fig. 1). There was no significant main effect of group, F(1,76)=0.67, p=0.42, η_p ²=0.009, or psychotropic medication use, F(1,76)=0.30, p=0.59, η_p ²=0.004. Follow-up analyses showed that the only significant difference was between attempters and comparison subjects not currently using psychotropic medications, with attempters (n=10) emitting more aggressive responses per provocation than comparison subjects (n=27) (9.35 [5.36] vs. 6.19 [3.68], F(1,35)=4.18, p=0.049, η_p ²=0.107).

FIG. 1.

Current psychotropic medication use moderates the association between suicide attempt history and aggressive responding.

For the DDQ, the group times psychotropic medication use interaction term was not significant, F(1,76)=0.00, p=0.98, η_p ²=0.00, indicating that the groups did not differ in delay discounting by current exposure to psychotropic medications. There also was no significant main effect of group, F(1,76)=0.22, p=0.64, η_p ²=0.003, or psychotropic medication use, F(1,76)=0.00, p=0.95, η_p ²=0.00.

Sensitivity analyses

Additional analyses were performed to determine whether the group times psychotropic medication exposure interaction effect in aggressive responding on the PSAP was robust to the effects of IQ and self-reported hostility on the BPAQ-SF. In an ANCOVA model including IQ and hostility as covariates, the interaction between group and psychotropic medication use remained significant, F(1,72)=8.26, p=0.005, η_p ²=0.103. Only IQ was a significant covariate, explaining additional variance in aggressive responding to provocation, F(1,72)=16.19, p<0.001; η_p ²=0.184; lower IQ was associated with more impulsive aggression. This interaction effect appeared to be specific to current antidepressant use. An ANCOVA including IQ and hostility as covariates revealed a significant group times antidepressant medication use interaction effect, F(1,72)=4.70, p=0.034, η_p ²=0.061. In participants not currently using antidepressants, attempters (n=17) emitted more aggressive responses per provocation than comparison subjects, (n=32) (8.22 [4.53] vs. 6.54 [3.59], F(1,45)=3.01, p=0.089, η_p ²=0.063), although statistical significance was only at a trend level. Results for delay discounting were unchanged after accounting for the effects of IQ in an ANCOVA model.

Post-hoc analyses: Associations among family history of suicidal behavior, delay discounting, and impulsive aggression

Family history of suicidal behavior correlated significantly with both the PSAP (r=0.27, n=80, p=0.015) and DDQ (r =−0.22, n=80, p=0.049), indicating higher levels of impulsive aggression and greater discounting by delay in participants with a family history of suicidal behavior (Table 2). The correlation between aggressive responding on the PSAP and family history of suicidal behavior was significant at a trend level in attempters examined separately (r=0.28, n=40, p=0.075) but was not significant in comparison subjects (r=0.23, n=40, p=0.153). Conversely, the association between delay discounting and family history of suicidal behavior was a result of the significant negative correlation in comparison subjects (r=−0.31, n=40, p=0.050); delay discounting did not correlate significantly with family history of suicidal behavior in attempters (r=−0.10, n=40, p=0.523).

Characteristics of participants with and without a family history of suicidal behavior are shown in Table 3. Participants with a family history of suicidal behavior were more likely to be of minority race/ethnicity and to have higher rates of affective and anxiety disorders than participants with no family history of suicidal behavior.

Table 3.

Characteristics of Participants With and Without a Family History (FH) of Suicidal Behavior ^a

Characteristic	FH of suicidal behavior (n=23)		No FH of suicidal behavior (n=57)		Analysis
	Mean	SD	Mean	SD	Statistic	df	p Value
Age (years)	15.7	1.3	15.5	1.4	t=0.56	78	0.58
IQ	97.0	12.3	98.1	15.0	t=0.31	78	0.76

	n	%	n	%
Female	17	73.9	43	75.4	χ²=0.02	1	0.89
Race/ethnicity					χ²=7.6	1	0.006
White, non-Hispanic	11	47.8	45	78.9
Other race, ethnicity	12	52.2	12	21.1
CBCL DSM-IV-oriented scales
Affective disorders	17	73.9	27	47.4	χ²=4.7	1	0.03
Anxiety disorders	12	52.2	16	28.1	χ²=4.2	1	0.04
Somatic disorders	11	47.8	16	28.1	χ²=2.9	1	0.09
Any behavioral disorder	18	45.0	13	32.5	χ²=1.3	1	0.25
Euthymic participants^b	13	56.5	39	68.4	χ²=1.0	1	0.31
Current medication use
Any psychotropic medication use	13	56.5	30	52.6	χ²=0.1	1	0.75
Current smoker, % yes	4	17.4	22	38.6	χ²=3.6	1	0.07
Current alcohol use, % yes	6	26.1	17	29.8	χ²=0.1	1	0.74
Current substance use,^c % yes	6	26.1	15	26.3	χ²=0.0	1	0.98
Sexual abuse in the past year	2	8.7	4	7.0	FET		1.00

First-degree relatives.

Based on Beck Depression Inventory-FastScreen (Beck, Steer, Brown 2000) total score<4, indicating minimal depressive symptoms.

Includes marijuana, cocaine, amphetamine, LSD/mescaline, tranquilizers, heroin/opiates, PCP, or gases/fumes.

CBCL, Child Behavior Checklist; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, 4th ed.; FET, Fisher's exact test.

Additional analyses were conducted to quantify discounting by delay and impulsive aggression in attempters and comparison subjects as a function of family history of suicidal behavior. For the DDQ, a two way ANOVA revealed a significant effect of family history of suicidal behavior, F(1,76)=5.91, p=0.017, η_p ²=0.072, but no significant effect of group, F(1,76)=1.54, p=0.219, η_p ²=0.020, or the interaction between group and family history of suicidal behavior, F(1,76)=1.59, p=0.211, η_p ²=0.020 (Fig. 2). In an ANCOVA model including race/ethnicity, IQ, affective disorders, and anxiety disorders as covariates, the main effect of family history of suicidal behavior remained significant, F(1,72)=7.11, p=0.009; η_p ²=0.090.

FIG. 2.

Question-based delay discounting (DDQ) in adolescent suicide attempters and nonsuicidal comparison subjects stratified by family history (FH) of suicidal behavior in first-degree relatives.

For the PSAP, a two way ANOVA revealed a significant effect of family history of suicidal behavior, F(1,76)=4.29, p=0.042, η_p ²=0.053, but no significant effect of group, F(1,76)=0.31, p=0.578, η_p ²=0.004) or the group times family history of suicidal behavior interaction, F(1,76)=0.59, p=0.447, η_p ²=0.008. In an ANCOVA model including race/ethnicity, IQ, affective disorders, and anxiety disorders as covariates, the main effect of family history of suicidal behavior did not remain significant, F(1,72)=2.12, p=0.150; η_p ²=0.029).

Discussion

In the present study, adolescents with a history of suicide attempt and never-suicidal subjects were compared on laboratory behavioral measures of delay discounting and impulsive aggression. Contrary to expectations, we found no overall difference between attempters and comparison subjects in delay discounting or impulsive aggression. There was a significant interaction between suicide attempt status and current psychotropic medication use on the PSAP, a measure of impulsive aggression. The interaction was accounted for by attempters with no current use of psychotropic medications exhibiting more impulsive aggression than comparison subjects with no current psychotropic drug use, an effect not seen between attempters and comparison subjects who were currently using psychotropic medications. The interaction effect appeared to be specific to current antidepressant use. The reason why impulsive aggression was related to suicide attempts only in the absence of psychotropic drug use is unclear. Deficient serotonergic activity in the prefrontal cortex may be an important mechanism underlying the association between impulsive aggression and suicidal behavior (Mann 2003; Seo et al. 2008). Antidepressants, and specifically the selective serotonin reuptake inhibitors (SSRIs), may help some patients control the impulse to act aggressively (Bond 2005). Pharmacological research has shown that SSRIs can reduce impulsive aggression in both adults (Zanarini et al. 2004) and children and adolescents (Armenteros and Lewis 2002). In this study, the majority of youth on antidepressant medications were taking an SSRI, which supports the possibility of a clinical effect of SSRI treatment in reducing impulsive aggression. Given the exploratory nature of these findings, future research should clarify the moderating effect of antidepressant medication use and the underlying mechanisms involved in this association.

Contrary to expectation, the laboratory behavioral measure of delay discounting used in this study, the DDQ, also did not distinguish attempters from comparison subjects. This finding is at odds with those of prior studies that have found significant differences in delay discounting in subjects with and without history of suicide attempts (Dombrovski et al. 2011; Mathias et al. 2011; Liu et al. 2012). One plausible explanation that may explain the differences in findings relating delay discounting to suicidal behavior is that delay discounting may be important in only a subset of suicide attempters. All prior studies noted found that delay discounting distinguished specific subgroups of attempters (e.g., high-lethality vs. low-lethality attempters; multiple vs. single attempters) compared with control subjects. In our sample, most attempters (n=32, 80%) made nonviolent, low-lethality suicide attempts (Bridge et al. 2012), and delay discounting was unrelated to multiple attempt history (data available on request). Further studies including subjects with a history of suicidal ideation but no history of suicide attempt are needed to confirm or refute our findings.

The post-hoc analyses show a clear association between delay discounting and family history of suicidal behavior. Participants with a family history of suicidal behavior displayed more delayed reward impulsivity than participants with no family history of suicidal behavior. Findings of delay discounting remained significant even after accounting for IQ and differences in race/ethnicity, affective disorder, and anxiety disorder between those with and without a family history of suicidal behavior. One recent study of adolescent twins shows evidence of heritability of delay discounting (Anokhin et al. 2011). Taken together, these results suggest that delay discounting may be a promising endophenotype of suicidal behavior (Mann et al. 2009). Endophenotypes are defined as internal phenotypes (i.e., not obvious to the unaided eye) between gene and disease (Gottesman and Gould 2003). Future studies are warranted to establish whether delay discounting fulfills the criteria for an endophenotype.

One of the aims of this study was to examine the association between self-report and laboratory behavioral assessments of impulsivity and impulsive aggression. In general, correlation coefficients between self-report and laboratory behavioral assessments were not significant, with one notable exception. Those who rated themselves as more hostile on the BPAQ-SF displayed lower levels of impulsive aggression on the PSAP. This negative correlation runs counter to the notion in suicide research that hostility, a mood state (Buss and Durkee 1957), and impulsive aggression are analogous constructs (Keilp et al. 2006). Future research needs to better identify the distinct dimensions of aggressive behavior most closely associated with adolescent suicide risk. It will also be important to establish whether prediction models can show increased accuracy using a combination of laboratory behavioral measures and self-ratings, or whether one method more closely identifies suicide risk.

Limitations

This study has several potential limitations. First, all participants were seeking outpatient behavioral health treatment or emergency services at a single metropolitan children's hospital. Second, the sample of suicide attempters was drawn from a pool of patients who had agreed to be contacted about research opportunities, and only 60% of eligible attempters were studied. Third, the sample was mostly female and currently on psychoactive medications at the time of assessment. Together, these issues of sample selection and participation may affect the generalizability of our findings. Therefore, these findings must be confirmed in more representative samples of suicide attempters, preferably in youth not on psychoactive medications, to rule out potential drug effects. Fourth, assessment of youth psychiatric problems was obtained only by parent ratings of child behavior rather than by structured diagnostic interviews with both participants and parents. Fifth, we were unable to determine whether participants were on psychotropic medication at the time of the index attempt. This is an important shortcoming, because if participants were on a psychotropic medication, it suggests that the effect of the medication on impulsive aggression did not alter their risk. If on the other hand, they were prescribed the medication as a result of the attempt, then this is a cautionary note that if one does not control for medication use, the role of particular traits may be underestimated. Sixth, we did not include a measure of premeditated aggression, which may be associated with more carefully planned suicide attempts. Seventh, to reduce participant burden, the study used only a single session of the PSAP to assess impulsive aggression in participants. Although this approach has demonstrated acceptable construct validity (Golomb et al. 2007), it remains unclear whether assessment of impulsive aggression using the more common method of conducting multiple PSAP sessions until performance asymptotes (Cherek et al. 1997) would have yielded different results. Therefore, future studies would likely benefit from exploring the effect of single versus multiple administrations of the PSAP on adolescent suicidal behavior.

Clinical Significance

One clinically relevant finding was that group differences in impulsive aggression were influenced by current use of psychotropic medications, particularly antidepressants. These findings highlight the need for future research to explore the potential moderating role of antidepressant medication use, and to clarify mechanisms involved in the association between impulsive aggression and adolescent suicide attempts.

Conclusion

In summary, this study found qualified evidence that a laboratory model of impulsive aggression was associated with adolescent suicidal behavior and no evidence that adolescent suicide attempters discount more by delay than never-suicidal comparison subjects. Longitudinal studies with larger and more representative samples of youth suicide attempters will be needed to better understand the temporal association among specific subtypes of impulsivity and impulsive-aggressive behavior and suicide attempt risk in adolescents. To clarify the familial influence of suicidal behavior on child suicide attempt risk, future work should also seek to clarify the role of delay discounting as a potential endophenotype of adolescent suicidal behavior. The association between impulsive aggression and hostility also warrants further study.

Footnotes

Acknowledgements

The authors thank Dr. Scott Lane for sending the PSAP task and for providing technical assistance on its use. The authors also gratefully acknowledge the youth and parents who participated in this study and the staff of Community Behavioral Health Services at Nationwide Children's Hospital for their assistance with subject recruitment.

Disclosures

Dr. Brent has received royalties from Guilford Press and from eRT for a voice-activated version of the Columbia Suicide Severity Rating Scale, is an UpToDate Psychiatry Editor; and has received honoraria from presentations for continuing medical education events. All other authors report no biomedical financial interests or potential conflicts of interest.

References

Achenbach

, Dumenci

, Rescorla

: DSM-oriented and empirically based approaches to constructing scales from the same item pools. J Clin Child Adolesc Psychol, 32:328–340, 2003.

Achenbach

, Rescorla

: Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2001.

Anokhin

, Golosheykin

, Grant

, Heath

: Heritability of delay discounting in adolescence: A longitudinal twin study. Behav Genet, 41:175–183, 2011.

Armenteros

, Lewis

: Citalopram treatment for impulsive aggression in children and adolescents: An open pilot study. J Am Acad Child Adolesc Psychiatry, 41:522–529, 2002.

Barratt

: Factor analysis of some psychometric measures of impulsiveness and anxiety. Psychol Rep., 16:547–554, 1965.

Beautrais

, Joyce

, Mulder

: Personality traits and cognitive styles as risk factors for serious suicide attempts among young people. Suicide Life Threat Behav, 29:37–47, 1999.

Beautrais

, Joyce

, Mulder

: Risk factors for serious suicide attempts among youths aged 13 through 24 years. J Am Acad Child Adolesc Psychiatry, 35:1174–1182, 1996.

Bechara

: Iowa Gambling Task: Professional Manual. Lutz, FL: Psychological Assessment Resources, Inc.; 2007.

Beck

, Steer

, Brown

: Beck Depression Inventory-FastScreen for medical patients. Manual. San Antonio, TX: The Psychological Corporation, 2000.

10.

Bond

: Antidepressant treatments and human aggression. Eur J Pharmacol., 526:218–225, 2005.

11.

Borowsky

, Ireland

, Resnick

: Adolescent suicide attempts: Risks and protectors. Pediatrics, 107:485–493, 2001.

12.

Brent

: What family studies teach us about suicidal behavior: Implications for research, treatment, and prevention. Eur Psychiatry, 25:260–263, 2010.

13.

Brent

, Baugher

, Bridge

, Chen

, Chiappetta

: Age- and sex-related risk factors for adolescent suicide. J Am Acad Child Adolesc Psychiatry, 38:1497–1505, 1999.

14.

Brent

, Bridge

, Johnson

, Connolly

: Suicidal behavior runs in families. A controlled family study of adolescent suicide victims. Arch Gen Psychiatry, 53:1145–1152, 1996.

15.

Brent

, Johnson

, Bartle

, Bridge

, Rather

, Matta

, Connolly

, Constantine

: Personality disorder, tendency to impulsive violence, and suicidal behavior in adolescents. J Am Acad Child Adolesc Psychiatry, 32:69–75, 1993a.

16.

Brent

, Oquendo

, Birmaher

, Greenhill

, Kolko

, Stanley

, Zelazny

, Brodsky

, Bridge

, Ellis

, Salazar

and Mann

: Familial pathways to early-onset suicide attempt: Risk for suicidal behavior in offspring of mood-disordered suicide attempters. Arch Gen Psychiatry, 59:801–807, 2002.

17.

Brent

, Oquendo

, Birmaher

, Greenhill

, Kolko

, Stanley

, Zelazny

, Brodsky

, Firinciogullari

, Ellis

and Mann

. Peripubertal suicide attempts in offspring of suicide attempters with siblings concordant for suicidal behavior. Am J Psychiatry, 160:1486–1493, 2003.

18.

Brent

, Perper

, Moritz

, Allman

, Friend

, Roth

, Schweers

, Balach

, Baugher

: Psychiatric risk factors for adolescent suicide: A case–control study. J Am Acad Child Adolesc Psychiatry. 32:521–529, 1993b.

19.

Bridge

, Goldstein

, Brent

: Adolescent suicide and suicidal behavior. J Child Psychol Psychiatry, 47:372–394, 2006.

20.

Bridge

, Iyengar

, Salary

, Barbe

, Birmaher

, Pincus

, Ren

, Brent

: Clinical response and risk for reported suicidal ideation and suicide attempts in pediatric antidepressant treatment: A meta-analysis of randomized controlled trials. JAMA, 297:1683–1696, 2007.

21.

Bridge

, McBee–Strayer

, Cannon

, Sheftall

, Reynolds

, Campo

, Pajer

, Barbe

, Brent

: Impaired decision making in adolescent suicide attempters. J Am Acad Child Adolesc Psychiatry, 51:394–403, 2012.

22.

Buss

, Durkee

: An inventory for assessing different kinds of hostility. J Consult Psychol, 21:343–349, 1957.

23.

Casat

, Pearson

, Van Davelaar

, Cherek

: Methylphenidate effects on a laboratory aggression measure in children with ADHD. Psychopharmacol Bull, 31:353–356, 1995.

24.

Centers for Disease Control and Prevention: National Center for Injury Prevention and Control. Web-based Injury Statistics Query and Reporting System (WISQARS) Source of data from WISQARS is the National Vital Statistics System from the National Center for Health Statistics, 2013. Available at www.cdc.gov/ncipc/wisqars Accessed April 3, 2014 .

25.

Cherek

: A Manual for the Point Subtraction Aggression Paradigm. Houston: University of Texas; 1992.

26.

Cherek

, Lane

: Effects of d,l-fenfluramine on aggressive and impulsive responding in adult males with a history of conduct disorder. Psychopharmacology, 146:473–481, 1999.

27.

Cherek

, Moeller

, Schnapp

, Dougherty

: Studies of violent and non-violent male parolees: I. Laboratory and psychometric measurements of aggression. Biol Psychiatry, 41:514–522, 1997.

28.

Cohen

: Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.

29.

Diamond

, Magaletta

: The short-form Buss–Perry Aggression Questionnaire (BPAQ-SF): A validation study with federal offenders. Assessment, 13:227–240, 2006.

30.

Dombrovski

, Szanto

, Siegle

, Wallace

, Forman

, Sahakian

, Reynolds

3rd , Clark

: Lethal forethought: delayed reward discounting differentiates high- and low-lethality suicide attempts in old age. Biol Psychiatry, 70:138–144, 2011.

31.

Dougherty

, Mathias

, Marsh

, Jagar

: Laboratory behavioral measures of impulsivity. Behav Res Methods, 37:82–90, 2005.

32.

Dougherty

, Mathias

, Marsh–Richard

, Prevette

, Dawes

, Hatzis

, Palmes

, Nouvion

: Impulsivity and clinical symptoms among adolescents with non-suicidal self-injury with or without attempted suicide. Psychiatry Res, 169:22–27, 2009.

33.

Dumais

, Lesage

, Alda

, Rouleau

, Dumont

, Chawky

, Roy

, Mann

, Benkelfat

, Turecki

: Risk factors for suicide completion in major depression: A case–control study of impulsive and aggressive behaviors in men. Am J Psychiatry, 162:2116–2124, 2005.

34.

Eaton

, Kann

, Kinchen

, Shanklin

, Flint

, Hawkins

, Harris

, Lowry

, McManus

, Chyen

, Whittle

, Lim

, Wechsler

, Centers for Disease Control and Prevention. Youth risk behavior surveillance – United States, 2011. MMWR Surveill Summ, 61:1–162, 2012.

35.

Eysenck

SBG

, Eysenck

: Impulsiveness and venturesomeness in children. Pers Individ Dif, 1:73–78, 1980.

36.

Fergusson

, Woodward

, Horwood

: Risk factors and life processes associated with the onset of suicidal behaviour during adolescence and early adulthood. Psychol Med, 30:23–39, 2000.

37.

Fossati

, Barratt

, Acquarini

, Di Ceglie

: Psychometric properties of an adolescent version of the Barratt Impulsiveness Scale-11 (BIS-11-A) in a sample of Italian high school students. Percept Mot Skills, 95:621–635, 2002.

38.

Golomb

, Cortez–Perez

, Jaworski

, Mednick

, Dimsdale

: Point subtraction aggression paradigm: Validity of a brief schedule of use. Violence Vict, 22:95–103, 2007.

39.

Gottesman

, Gould

: The endophenotype concept in psychiatry: Etymology and strategic intentions. Am J Psychiatry, 160:636–645, 2003.

40.

Gould

, King

, Greenwald

, Fisher

, Schwab–Stone

, Kramer

, Flisher

, Goodman

, Canino

, Shaffer

: Psychopathology associated with suicidal ideation and attempts among children and adolescents. J Am Acad Child Adolesc Psychiatry, 37:915–923, 1998.

41.

Green

, Myerson

: A discounting framework for choice with delayed and probabilistic rewards. Psychol Bull, 130:769–792, 2004.

42.

Hammad

, Laughren

, Racoosin

: Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychiatry, 63:332–339, 2006.

43.

Horesh

: Self-report vs. computerized measures of impulsivity as a correlate of suicidal behavior. Crisis, 22:27–31, 2001.

44.

Huang

, Cherek

, Lane

: Laboratory measurement of aggression in high school age athletes: provocation in a nonsporting context. Psychol Rep., 85:1251–1262, 1999.

45.

Johnson

, Brent

, Bridge

, Connolly

: The familial aggregation of adolescent suicide attempts. Acta Psychiatr Scand, 97:18–24, 1998.

46.

Kaufman

, Kaufman

: Kaufman Brief Intelligence Test, 2nd ed. Minneapolis: NCS Pearson; 2004.

47.

Keilp

, Gorlyn

, Oquendo

, Brodsky

, Ellis

, Stanley

, Mann

: Aggressiveness, not impulsiveness or hostility, distinguishes suicide attempters with major depression. Psychol Med., 36:1779–1788, 2006.

48.

Kingsbury

, Hawton

, Steinhardt

, James

: Do adolescents who take overdoses have specific psychological characteristics? A comparative study with psychiatric and community controls. J Am Acad Child Adolesc Psychiatry, 38:1125–1131, 1999.

49.

Liu

, Vassileva

, Gonzalez

, Martin

: A comparison of delay discounting among substance users with and without suicide attempt history. Psychol Addict Behav, 26:980–985, 2012.

50.

Mann

: Neurobiology of suicidal behaviour. Nat Rev Neurosci, 4:819–828, 2003.

51.

Mann

: The neurobiology of suicide. Nat Med, 4:25–30, 1998.

52.

Mann

, Arango

, Avenevoli

, Brent

, Champagne

, Clayton

, Currier

, Dougherty

, Haghighi

, Hodge

, Kleinman

, Lehner

, McMahon

, Moscicki

, Oquendo

, Pandey

, Pearson

, Stanley

, Terwilliger

, Wenzel

. Candidate endophenotypes for genetic studies of suicidal behavior. Biol Psychiatry, 65:556–563, 2009.

53.

Mann

, McBride

, Brown

, Linnoila

, Leon

, DeMeo

, Mieczkowski

, Myers

, Stanley

: Relationship between central and peripheral serotonin indexes in depressed and suicidal psychiatric inpatients. Arch Gen Psychiatry, 49:442–446, 1992.

54.

Marttunen

, Aro

, Lonnqvist

: Adolescent suicide: Endpoint of long-term difficulties. J Am Acad Child Adolesc Psychiatry. 31:649–654, 1992.

55.

Mathias

, Dougherty

, James

, Richard

, Dawes

, Acheson

, Hill-Kapturczak

: Intolerance to delayed reward in girls with multiple suicide attempts. Suicide Life Threat Behav, 41:277–286, 2011.

56.

Mazur

: An adjusting procedure for studying delayed reinforcement. In: Quantitative Analysis of Behavior, Vol 5. The Effects of Delay and Intervening Events on Reinforcement Value, edited by Commons

M.L.

, Mazur

J.E.

, Nevin

J.A.

, Rachlin

Hillsdale, NJ: Erlbaum, 55–73, 1987.

57.

McGirr

, Renaud

, Bureau

, Seguin

, Lesage

, Turecki

: Impulsive-aggressive behaviours and completed suicide across the life cycle: A predisposition for younger age of suicide. Psychol Med, 38:407–417, 2008.

58.

McGirr

, Alda

, Seguin

, Cabot

, Lesage

, Turecki

: Familial aggregation of suicide explained by cluster B traits: A three-group family study of suicide controlling for major depressive disorder. Am J Psychiatry, 166:1124–1134, 2009.

59.

Muehlenkamp

, Gutierrez

: Risk for suicide attempts among adolescents who engage in non-suicidal self-injury. Arch Suicide Res, 11:69–82, 2007.

60.

Myerson

, Green

, Warusawitharana

: Area under the curve as a measure of discounting. J Exp Anal Behav, 76:235–243, 2001.

61.

Pfeffer

, Newcorn

, Kaplan

, Mizruchi

, Plutchik

: Suicidal behavior in adolescent psychiatric inpatients. J Am Acad Child Adolesc Psychiatry, 27:357–361, 1988.

62.

Reynolds

. A review of delay-discounting research with humans: relations to drug use and gambling. Behav Pharmacol, 17:651–667, 2006.

63.

Reynolds

, Ortengren

, Richards

, De Wit

: Dimensions of impulsive behavior: Personality and behavioral measures. Pers Individ Dif, 40:305–315, 2006.

64.

Reynolds

, Penfold

, Patak

: Dimensions of impulsive behavior in adolescents: Laboratory behavioral assessments. Exp Clin Psychopharmacol, 16:124–131, 2008.

65.

Richards

, Zhang

, Mitchell

, de Wit

: Delay or probability discounting in a model of impulsive behavior: Effect of alcohol. J Exp Anal Behav, 71:121–143, 1999.

66.

Seo

, Patrick

, Kennealy

: Role of serotonin and dopamine system interactions in the neurobiology of impulsive aggression and its comorbidity with other clinical disorders. Aggress Violent Behav, 13:383–395, 2008.

67.

Shaffer

, Gould

, Fisher

, Trautman

, Moreau

, Kleinman

, Flory

: Psychiatric diagnosis in child and adolescent suicide. Arch Gen Psychiatry, 53:339–348, 1996.

68.

Stein

, Apter

, Ratzoni

, Har-Even

, Avidan

: Association between multiple suicide attempts and negative affects in adolescents. J Am Acad Child Adolesc Psychiatry, 37:488–494, 1998.

69.

Stiffman

, Horwitz

, Hoagwood

, Compton

3rd , Cottler

, Bean

, Narrow

, Weisz

: The Service Assessment for Children and Adolescents (SACA): Adult and child reports. J Am Acad Child Adolesc Psychiatry, 39:1032–1039, 2000.

70.

Weissman

, Wickramaratne

, Adams

, Wolk

, Verdeli

, Olfson

: Brief screening for family psychiatric history: The family history screen. Arch Gen Psychiatry, 57:675–682, 2000.

71.

Zanarini

, Frankenburg

, Parachini

: A preliminary, randomized trial of fluoxetine, olanzapine, and the olanzapine-fluoxetine combination in women with borderline personality disorder. J Clin Psychiatry. 65:903–907, 2004.