Comorbid Internalizing and Disruptive Behavior Disorder in Adolescents

Abstract

This study examined differences between comorbid internalizing and disruptive behavior disorder (DBD), and those with either internalizing disorder or DBD. We focused on differences with regard to trauma exposure and offending characteristics in 8,431 juvenile justice youths. Self-reported, structured interview and official record data were used. Multinomial logistic regression analysis predicted disorder profile from traumatic exposure, suicide attempt, and offending characteristics, adjusting for background variables. Victimization by non-sexual violence was significantly higher in comorbid than in internalizing youth. Also, the number of DBDs, as well as rates of victimization via sexual and non-sexual assault, was significantly higher in the comorbid than in the DBD group. We conclude that a history of victimization, but not an early onset of criminal behavior, was associated with comorbid internalizing disorder and DBD. Findings emphasize the need for improving identification of this comorbid condition and referral for effective treatment.

Keywords

psychiatric disorder comorbidity traumatic exposure offending adolescents

Comorbidity, the co-occurrence of two or more different psychiatric disorders, has received increasing recent attention (Angold, Costello, & Erkanli, 1999; Cunningham & Ollendick, 2010; Faire & Ollendick, 2013; Wolff & Ollendick, 2006). Relatively strong associations have been noted between internalizing disorder and disruptive behavior disorder (DBD; Angold et al., 1999; Wolff & Ollendick, 2006). Reasons for this are not immediately obvious, given the seemingly divergent nature of these types of disorder: Youth with internalizing disorders, such as depression and anxiety, are considered to hold their emotions in, whereas those with DBDs, such as conduct disorder (CD; characterized by a repetitive and persistent pattern of antisocial behavior), and attention deficit hyperactivity disorder (ADHD; characterized by impulsive behavior, attention problems, restlessness), act their emotions out. Depressed adolescents in the general population are about 6 times more likely than those without depression to also meet criteria for either CD (odds ratio [OR] = 6.6) or ADHD (OR = 5.5), suggesting that this particular comorbid pattern may be more prevalent than could be accounted for by the occurrence of either DBD or internalizing disorder (Angold et al., 1999). Recognizing the co-occurrence of these concerns, the new Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association [APA], 2013) diagnosis of disruptive mood dysregulation disorder (DMDD) includes a range of disruptive behavior and internalizing disorders, such as chronic irritable or angry mood and anger outbursts.

Across a range of studies and samples, those with co-occurring internalizing disorder and DBD appear to suffer from long-term problems in functioning, more so than those with either internalizing or DBD alone. These long-term problems particularly concern criminal behavior. In a community study, children diagnosed with this disorder profile were more likely to be arrested during young adulthood than were non-disordered children (Copeland, Miller-Johnson, Keeler, Angold, & Costello, 2007). Among clinical outpatient children and adolescents, those with both CD and depression were more likely to commit crimes as adults, compared with those with depression but without CD (Harrington, Fudge, Rutter, Pickles, & Hill, 1991). Recently, we reported that co-occurring internalizing disorder and DBD, assessed at juvenile probation intake, increased the risk of later (young adult) reoffending (Hoeve, McReynolds, & Wasserman, 2013a). In adult offenders, among those meeting criteria for antisocial personality disorder (which requires a diagnosis of CD by adolescence), about one third also meet criteria for an anxiety disorder, with an age of onset of about 16 years (Hodgins, De Brito, Chhabra, & Coté, 2010).

It is unclear why children and adolescents with this comorbid condition are at higher risk of later offending than those with either internalizing disorder or DBD alone. Examining two possible pathways to later offending might help our understanding. One pathway leads from trauma exposure to offending behavior. The cycle of violence hypothesis posits that children exposed to violence might be at increased risk of violent behavior in adolescence and adulthood (e.g., Maxfield & Widom, 1996; Widom, 1989; Wilson, Stover, & Berkowitz, 2009). Traumatic exposure elevates risk of both internalizing disorder (e.g., Kerig, Ward, Vanderzee, & Moeddel, 2009) and DBD (Wilson et al., 2009). In one cross-sectional study of juvenile justice youth (Ford, Hartman, Hawke, & Chapman, 2008), various types of traumatic exposure (including physical abuse, domestic violence, and neglect) were associated with suicidal ideation and substance abuse. Chronic stress, in particular, has been found to impair brain development, perhaps increasing risk of mental health problems (Twardosz & Lutzker, 2010). Distress resulting from traumatic events may exhaust the youth mentally and physically, leading to emotional dysregulation, impulsivity, dysfunctional information processing, lower levels of empathy, and eventually to antisocial behavior (Ford, Chapman, Mack, & Pearson, 2006). These studies suggest that the consequences of exposure to traumatic events include a range of internalizing and externalizing mental health problems. However, the degree to which those consequences might include elevated rates of comorbid internalizing disorder and DBD has not been systematically examined.

The second relevant pathway leads from an early onset of antisocial behavior to persistent serious offending into adulthood (e.g., Farrington, 2006; Gendreau, Little, & Goggin, 1996; Moffitt, 1993; Patterson & Yoerger, 2002). Those who follow this life-course persistent antisocial pathway show an earlier onset of antisocial behavior in childhood, which continues over the life course. This pattern is associated with neurodevelopmental risk factors (Fairchild, Van Goozen, Calder, & Goodyer, 2013), poorer family functioning, and an increased rate of more serious and violent offenses (Hoeve et al., 2008; Moffitt, 2006). Developmentally, onset of disruptive behavior problems seems to precede that for internalizing problems (e.g., Loeber & Burke, 2011; Van Lier et al., 2012). Although research on psychopathology by age-of-onset subtype is scarce (see Vermeiren, 2003, for a review), rates of mental health disorder for those on the early onset pathway are higher than for those on the late onset pathway. Among incarcerated youth (Ruchkin, Koposov, Vermeiren, & Schwab-Stone, 2003), DBD, depression, and posttraumatic stress disorder (PTSD; characterized by recurring flashbacks, avoidance, or numbing of memories after a traumatic event) in particular were more prevalent in those with earlier onset of antisocial behavior. In contrast, in a high-risk community sample of youth (Aguilar, Sroufe, Egeland, & Carlson, 2000), those with a late, rather than early, onset of externalizing problems endorsed elevated levels of stress and internalizing problems. Thus, it is unknown whether those with comorbid internalizing disorder and DBD are more likely to start a course of antisocial behavior at an early age.

Several studies have focused on correlates of comorbid youth compared with those with either an internalizing disorder or DBD alone. Some of such studies report more impairment among comorbid youth (disorder history, functional impairment, physical health; Newman, Moffitt, Caspi, & Silva, 1998; poorer educational performance; Lewinsohn, Rohde, & Seeley, 1995; and lower social competence; Renouf, Kovacs, & Mukerji, 1997), whereas others do not (e.g., Ezpeleta, Domenech, & Angold, 2006; Steinhausen & Reitzle, 1996). Most prior studies have examined correlates of either internalizing or disruptive disorder alone, rather than comparing characteristics of comorbid individuals with individuals presenting with one or another component separately (for reviews, see Faire & Ollendick, 2013; Wolff & Ollendick, 2006). Furthermore, earlier research has most often focused on young children, although older children have greater risk of comorbidity (Wolff & Ollendick, 2006). Finally, studies of comorbidity have typically compared general comorbidity with single disorders. However, heterotypic comorbid conditions (i.e., co-occurrence of disorders in different diagnostic classes; Angold et al., 1999) are more difficult to explain than homotypic comorbidities (e.g., anxiety and depression), because methodological artifacts, such as criterion overlap, are less obvious sources for heterotypic comorbidity (e.g., Angold et al., 1999; Lilienfeld, 2003).

In the present study, we aim to increase clarification of the characteristics and etiology of comorbid internalizing disorder and DBD. Although various potential explanatory factors exist (Wolff & Ollendick, 2006), here we focus on factors that are most relevant in connection with offending behavior: traumatic exposure and age onset of offending. The present study examines (a) whether comorbid internalizing disorder and DBD are associated with higher levels of traumatic exposure and related clinical characteristics, such as PTSD, depression, and suicide attempts, and (b) whether comorbid internalizing disorder and DBD are associated with an earlier onset of antisocial behavior and related characteristics, such as age of first arrest, and higher rates of interpersonal offenses and number of DBDs. We examine whether youths with both internalizing disorder and DBD can be distinguished from those presenting with only one or the other component.

The prevalence of symptoms of both internalizing disorder and CD is elevated in juvenile justice youth (Teplin, Abram, McClelland, Dulcan, & Mericle, 2002) as is comorbidity in general (Wasserman, McReynolds, Schwalbe, Keating, & Jones, 2010), making this an efficient population in which to study these issues. In the present report, we examine the ways in which justice system youth with both internalizing disorder and DBD can be distinguished from those presenting with only one or the other component. On the basis of earlier studies, we expect that those with both internalizing disorder and DBD will report higher rates of traumatic exposure. Earlier studies have found that those with early and those with late onset of offending have higher rates of both internalizing and externalizing disorder. For this reason, we do not have a specific hypothesis with regard to the age of onset of justice-involved youth with internalizing disorder and DBD in comparison with counterparts with internalizing disorder or DBD alone.

Method

Subjects and Procedures

Juvenile justice agencies in 18 states (57 sites) participated in a collaboration with Columbia University’s Center for the Promotion of Mental Health in Juvenile Justice (CPMHJJ). Juvenile justice agencies used standardized data collection protocols and used universal or systematic random (by day of the week) sampling. Youths referred to juvenile justice agencies (N = 9,819) completed an audio computer-assisted diagnostic self-interview soon after intake into the site’s probation, detention, or secure care system. Sites de-identified information and provided assessment results along with demographic and offense characteristics to CPMHJJ. Procedures were approved by the Institutional Review Board of New York State Psychiatric Institute (NYSPI)/Columbia University.

Data were compiled into the National Archive of Mental Health in Juvenile Justice, resulting in a series of reports of the prevalence of psychiatric disorder across a range of juvenile justice settings (Wasserman et al., 2010). Data were collected on disorders that are most common among adolescents. First, data were collected on substance use disorder (SUD), including alcohol, marijuana, and other drug abuse and dependence. Second, data were collected on DBD, including CD, oppositional defiant disorder (ODD; characterized by irritable, angry, argumentative, and defiant behavior), and ADHD. Third, data were collected on anxiety disorder, including agoraphobia (anxiety in particular environments, such as crowds or wide open spaces), generalized anxiety (excessive, uncontrollable, and often irrational worry about everyday matters), panic disorder (recurring panic attacks and worry about having attacks), PTSD, and social phobia (strong fear in social situations). Finally, data were collected on affective disorder, including mania (elevated euphoric or irritable mood), hypomania (less severe than full mania), major depressive disorder (persistent low mood), and dysthymic disorder (less severe than major depressive disorder but symptoms are longer lasting). As reported earlier (Wasserman et al., 2010), about half the participants reported one or another disorder. About one third met criteria for more than one diagnosis. About 20% reported anxiety disorder, almost 10% reported affective disorder, more than a quarter reported DBD, and a third reported SUD.

Among those in the National Archive, full data were available for 8,431 youths on self-reported disorder, suicide attempt, and traumatic exposure, as well as on offense and demographic characteristics. The present report examines a subset of those participants, whose mental health characteristics reflected one of four profiles: those with no disorder, those with internalizing disorder (anxiety and affective disorder), those with DBD, and those with comorbid internalizing disorder and DBD (see Table 1 for sample characteristics). In this juvenile justice sample, 10.3% reported a comorbid internalizing disorder and DBD.

Table 1:

Sample Characteristics by Disorder Profile

	Total Sample (N = 6,691)		No Disorder (n = 3,228)		Internalizing (n = 766)		DBD (n = 1,940)		Comorbid (n = 757)
	n	%	n	%	n	%	n	%	n	%
Age (years; M, SD)^a	15.4	1.6	15.3	1.6	15.4	1.6	15.5	1.6	15.5	1.6
Male^a,b	5,149	77.0	2,539	78.7	507	66.2	1,588	81.9	515	68.0
Non-White^a,c	3,950	59.0	2,043	63.3	477	62.3	1,032	53.2	398	52.6
System entry^a,c	2,452	36.6	1,536	47.6	325	42.4	407	21.0	184	24.3
Interpersonal current offense^b	2,223	33.4	1,082	33.6	284	37.2	585	30.4	272	36.5
Repeat offender^a,c	4,858	73.7	2,136	66.7	539	71.1	1,575	83.1	608	82.2
Age at first offense (years; M, SD)^a	13.4	2.0	13.5	2.0	13.3	1.9	13.2	2.0	13.3	2.0
Any traumatic exposure^a,b,c	5,406	80.8	2,250	69.7	668	87.4	1,763	90.9	725	95.9
Exposure to forced sexual activity^a,b,c	784	11.7	207	6.4	171	22.4	190	9.8	216	28.6
Exposure to non-sexual violence^a,b,c	3,517	52.6	1,157	38.8	410	53.7	1,359	70.1	591	78.2
Other traumatic exposure^a,b,c	4,876	79.2	1,980	66.9	619	86.6	1,593	90.0	684	95.7
Past-month suicide attempt^a,b,c	191	2.9	21	0.7	39	5.1	42	2.2	89	11.9
Lifetime suicide attempt^a,b,c	1,062	15.9	242	7.5	181	23.7	333	17.0	306	40.4

Note. Figures are counts (n) and percentages (%). Characteristics regarding age denoted with “(years; M, SD)” are means (M), and standard deviations (SD) instead of n and %. DBD = disruptive behavior disorder.

Comorbid is significantly different from no disorder.

Comorbid is significantly different from DBD.

Comorbid is significantly different from internalizing.

Measures

Psychiatric Disorder and Suicide Attempt

Juvenile justice youth self-assessed mental health status on the Voice Diagnostic Interview Schedule for Children (V-DISC), which is based on the Diagnostic Interview Schedule for Children–IV (DISC-IV; Schaffer et al., 1996). The DISC-IV is highly structured and has been evaluated in clinical and community samples. DISC-IV test–retest reliability is as good as or better than previous versions (Schaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000). The V-DISC’s audio computer-assisted self-interview (ACASI) structure relies on a computer, with questions posed via headphones. The V-DISC has been widely used in juvenile justice settings (e.g., Hayes, McReynolds, & Wasserman, 2005; McReynolds, Wasserman, Fisher, & Lucas, 2007; Wasserman, Ko, & McReynolds, 2004; Wasserman, McReynolds, Ko, Katz, & Carpenter, 2005). No significant differences have been found in reliability of diagnoses between self-administered and interviewer-administered versions (Wasserman, McReynolds, Lucas, Fisher, & Santos, 2002). One-month reliability of most diagnoses (κ) ranged between 0.50 and 0.70 (Lucas, 2003).

The V-DISC measures 20 Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; APA, 1994) disorders in four clusters based on symptoms according to the DSM-IV: SUD, DBD, anxiety disorder, and affective disorder. We designated four groups. The first (Group 1) consisted of youth with no disorder. We considered two groups without heterotypic comorbidity: Group 2 included those with anxiety or affective disorder but without DBD (internalizing), those in Group 3 met criteria for DBD but not for internalizing disorder, and Group 4 included those with comorbid internalizing disorder and DBD. The present report then excluded youths meeting criteria solely for SUD (n = 1,740), although those with SUD appeared in all groups except for the no disorder group. The V-DISC also measures whether or not youths ever made a suicide attempt (lifetime suicide attempt) or tried to commit suicide in the past month (past-month suicide attempt).

Traumatic Exposure

The V-DISC queries about eight types of traumatic exposure. Consistent with others’ formulations (Breslau, Lucia, & Alvarado, 2006), youth who reported being attacked or beaten badly, experiencing forced sex (forced sexual activity), or being threatened by a weapon (non-sexual assault) were designated as exposed to traumatic events. Other traumatic exposure included being in a bad accident or natural disaster, seeing someone get badly hurt, or seeing a dead body. Any traumatic exposure included all items and a further query “thinking that you or others would be hurt badly or die,” that could not readily be assigned to the other categories (see Wasserman & McReynolds, 2011 for further details).

Demographic and Offense Characteristics

Background information, including age (in years), gender (males vs. females), and race (non-Whites vs. Whites), was recorded by local staff at baseline. Agencies also provided information regarding juvenile justice setting: system intake, such as court or probation settings, versus pre-trial detention and post-adjudicated secure care. Information on current offense, extracted from justice records, was also provided. For the current study, we used data on repeat offender status (repeat vs. first time offenders), age at first offense, and interpersonal (rape, assault, robbery, arson, homicide, and all weapons charges) versus non-interpersonal (property, substance) offense (details in Wasserman et al., 2010).

Data Analysis

First, bivariate associations were examined between disorder profile and each of the following: demographic background, offending, traumatic exposure, and suicide attempt. We used chi-squared tests and ANOVA (Table 1). Next, we analyzed associations between disorder profile, on one hand, and type and number of disorders, on the other hand, for descriptive purposes (Table 2). Using logistic regression, we examined potential differences among comorbid, internalizing, and DBD youth, adjusting for demographic variables.

Table 2:

Type and Number of Disorders by Disorder Profile

	Total Sample (N = 6,691)		Internalizing (n = 766)		DBD (n = 1,940)		Comorbid (n = 757)
	n	%	n	%	n	%	n	%
Number of internalizing disorders (M, SD)^a	2.1	1.4	1.9	1.2	—	—	2.4	1.4
Any anxiety disorder^a	1,214	18.2	643	84.1	—	—	571	75.6
Agoraphobia^a	569	8.5	331	43.3	—	—	238	31.5
Generalized Anxiety^a	214	3.2	77	10.1	—	—	137	18.1
Panic disorder^a	283	4.2	116	15.2	—	—	167	22.2
Posttraumatic stress disorder^a	304	4.5	133	17.4	—	—	171	22.6
Social phobia	402	6.0	199	26.0	—	—	203	26.9
Any affective disorder^a	686	10.3	254	33.5	—	—	432	57.7
Mania^a	97	1.5	20	2.6	—	—	77	10.2
Hypomania^a	102	1.5	37	4.9	—	—	65	8.7
Major depressive disorder^a	548	8.2	207	27.1	—	—	341	45.0
Dysthymic disorder	26	0.4	10	1.3	—	—	16	2.1
Number of DBDs (M, SD)^b	1.2	0.5	—	—	1.1	0.4	1.4	0.6
Any DBD	2,697	403	—	—	1,940	100	757	100
Conduct disorder^b	2,420	36.3	—	—	1,796	93.4	624	84.1
Oppositional defiant disorder^b	583	8.7	—	—	276	14.3	307	40.9
Attention deficit/ hyperactivity disorder^b	286	4.5	—	—	149	8.3	137	24.0
Any SUD^a	1,580	24.3	260	35.9	930	50.5	390	54.2
Alcohol abuse	340	5.2	72	9.9	193	10.4	75	10.4
Alcohol dependence^b	242	3.7	54	7.4	148	8.0	40	5.5
Marijuana abuse	350	5.4	86	11.8	184	9.9	80	11.1
Marijuana dependence^b	487	7.5	98	13.5	298	16.0	91	12.6
Other substance abuse^a	224	3.4	19	2.5	148	7.8	57	7.8
Other substance dependence^a,b	560	8.5	40	5.3	326	17.1	194	26.4

Note. Figures are counts (n) and percentages (%). Characteristics regarding the number of disorders denoted with “(M, SD)” are means (M), and standard deviations (SD) instead of n and %. DBD = disruptive behavior disorder; SUD = substance use disorder.

Comorbid is significantly different from internalizing.

Comorbid is significantly different from DBD. Analyses adjusted for gender, ethnicity, and juvenile justice setting.

Finally, we conducted a series of regression analyses to predict membership of the comorbid group from traumatic exposure and offense characteristics (Table 3). The multivariate models (logistic, non-linear mixed, and multinomial logistic) adjusted for gender, race, and justice setting. Variables with significant bivariate associations (p < .05, Table 1) were included in the multivariate models. To reduce redundancy, some measures were excluded (e.g., recent suicide attempt, because recent suicide attempts are a subset of lifetime attempts).

Table 3:

Odds Ratios (95% CIs) for Multivariate Group Comparisons Between Comorbid and Other Disorder Groups

	Comorbid (n = 613) vs. Internalizing (n = 659)		Comorbid (n = 613) vs. DBD (n = 1,575)
	OR	95% CI	OR	95% CI
Male	0.95	[0.72, 1.25]	0.78*	[0.60, 1.00]
Non-White	0.75*	[0.59, 0.95]	1.10	[0.90, 1.35]
System intake	0.57**	[0.42, 0.76]	1.57***	[1.20, 2.07]
Interpersonal offense	0.84	[0.66, 1.07]	1.22	[0.99, 1.50]
Repeat offender	1.03	[0.73, 1.45]	1.20	[0.88, 1.64]
Age at first offense	1.00	[0.93, 1.06]	1.02	[0.96, 1.07]
Any substance use disorder	1.85***	[1.45, 2.34]	1.08	[0.88, 1.32]
Exposure to forced sexual activity	0.98	[0.72, 1.33]	2.51***	[1.89, 3.32]
Exposure to non-sexual violence	2.49***	[1.87, 3.32]	1.33*	[1.02, 1.74]
Other traumatic exposure	1.39	[0.83, 2.32]	1.56	[0.97, 2.53]
Lifetime suicide attempt	1.76***	[1.35, 2.31]	2.67***	[2.12, 3.34]

Note. The reference category is the comorbid group. Because we were particularly interested in the odds of comorbid versus other groups instead of other groups versus comorbid, we converted the original ORs into 1/OR. OR = odds ratio; CI = confidence interval; DBD = disruptive behavior disorder.

p < .05. **p < .01. ***p < .001.

Because individual data were nested within facilities in the types of juvenile justice settings, we first examined how the clustering of youth within facilities affected results in a series of (non-hierarchical) logistic regression analyses, predicting membership in each of the defined disorder groups: comorbid disorder, internalizing disorder, and DBD. Next, we predicted disorder group membership in non-linear mixed models using SAS PROC NLMIXED, which accounts for the hierarchical structure of the data. Here, youth (the lowest level) were nested within facilities (the highest level).

Because findings from both non-linear mixed models and logistic regression models (available on request) were very similar, our final model was non-hierarchical (Table 3). These analyses used multinomial logistic regression (Hosmer & Lemeshow, 2000), which allows the dependent variable to have more than two categories, producing an OR for each dependent category compared with a set reference group. Here, internalizing disorder and DBD groups were each compared with the comorbid reference group. The advantage of using a single multinomial logistic regression model over multiple logistic regression models is that it corrects for family-wise error (i.e., the possibility of Type I errors, resulting from testing multiple hypotheses). Type and number of disorders (Table 2) were not considered in these models, because of the overlap between constructs, such as traumatic exposure and particular disorders (e.g., PTSD).

Results

Sample Characteristics

Our final sample consisted of 6,691 youths with no disorder, internalizing disorder, DBD, or both from 57 sites in 18 states. The average youth was about 15 years old at baseline; most were male and non-White (Black, Hispanic, or “Other”). About two thirds were adjudicated and remanded to correctional facilities or detained prior to adjudication (n = 4,239). About one third were from system intake settings (n = 2,452), such as probation or court intake. Sample characteristics are presented in Table 1.

Comorbid Versus Internalizing Youth

As Table 1 shows, compared with internalizing youth, comorbid youth were more likely to be White, detained or incarcerated (vs. at intake), and repeat offenders. Rates of traumatic exposure and suicide attempt history were substantially elevated in comorbid youth: for example, for any traumatic exposure, χ²(1) = 35.6, p < .001; for lifetime suicide attempts, χ²(1) = 49.1, p < .001.

Comorbid Versus DBD Youth

Compared with those with DBD, comorbid youth were more likely to be female and interpersonal offenders. Again, rates of traumatic exposure and suicide attempt history were substantially higher in comorbid than in DBD youth (e.g., any traumatic exposure), χ²(1) = 19.3, p < .001; for lifetime suicide attempts, χ²(1) = 162.9, p < .001.

Type and Number of Disorders

Overall, comorbid youth reported more disorders and higher rates of most types of disorders than youth in either the internalizing or DBD group (see Table 2). The comorbid group reported more internalizing disorders than the internalizing group and more DBD disorders than the DBD group.

Comorbid Versus Internalizing Youth

Comorbid youth reported significantly more types of internalizing disorders than did internalizing youth (OR = 1.3, p < .001) and were more than twice as likely to report one or another affective disorder as those in the internalizing group (OR = 2.6, p < .001). Net of gender, ethnicity, and juvenile justice setting, certain disorders were significantly more prevalent in comorbid youth than in the internalizing group: generalized anxiety, panic disorder, PTSD, mania, hypomania, major depressive disorder, and many types of SUDs. Comorbid youth were twice as likely to report one or another SUD (OR = 2.0, p < .001) and also showed a threefold and fivefold increase, respectively, in their rates of abuse (OR = 3.0, p < .001) and dependence (OR = 5.6, p < .001) for substances other than alcohol and marijuana.

Comorbid Versus DBD Youth

Comorbid youth reported significantly more types of DBD than did those in the DBD group (OR = 2.7, p < .001). There were varying patterns across disorder types: Compared with those in the DBD group, comorbid youth were less than half as likely to meet criteria for CD (OR = 0.4, p < .001), but were 3 to 4 times as likely to report either ODD (OR = 4.0, p < .001) or ADHD (OR = 3.5, p < .001). Patterns also varied for types of substance disorder: Although comorbid youth were more likely to report any SUD, (OR = 1.2, p < .001), they reported lower rates of alcohol (OR = 0.7, p < .05) and marijuana (OR = 0.7, p < .05) dependence, and higher rates of dependence on another substance (OR = 1.8, p < .05).

Predicting Disorder Group From Demographic, Offense, and Clinical Characteristics

Table 3 presents the multinomial logistic regression model, predicting disorder group from offense characteristics, traumatic exposure, and suicide attempt history, adjusting for demographics, with comorbid youth as the reference group. Overall, the comorbid group reported higher rates of traumatic exposure and suicide attempt than did those in either the internalizing or DBD group.

Comorbid Versus Internalizing Youth

Compared with the internalizing group, comorbid youth were less likely to be non-White (OR = 0.8, p < .05) and more likely to be detained or incarcerated (vs. system intake; OR = 0.6, p < .001). Adjusting for demographic and offense characteristics, comorbid youth were almost twice as likely to report one or another SUD (OR = 1.9, p < .001). They were more than twice as likely as internalizing youth to report either victimization by non-sexual violence (OR = 2.5, p < .001) or a lifetime suicide attempt (OR = 1.8, p < .001).

Comorbid Disorder Versus DBD Youth

Compared with the DBD group, those in the comorbid group were less likely to be male (OR = 0.8, p < .05) and more likely to be at system intake (OR = 1.6, p < .01). Adjusting for demographic and offense characteristics, comorbid youth were more likely to report victimization by either non-sexual (OR = 1.3, p < .05) or sexual (OR = 2.5, p < .01) violence as well as significantly higher rates of lifetime suicide attempts (OR = 2.7, p < .001).

Discussion

In the present study, our aim was to increase clarification of the characteristics and etiology of comorbid internalizing and DBD. Although various potential explanatory factors exist (Wolff & Ollendick, 2006), here we focused on those relevant to offending behavior: traumatic exposure and age onset of offending. We found that those meeting criteria for both types of disorder showed higher levels of traumatic exposure and were more likely to report prior suicide attempts than those with only one or the other component. However, comorbid youth did not differ in their age of onset of antisocial behavior from those with either internalizing or DBD alone. Furthermore, we found that rates of disorder were higher among the comorbid group: Comorbid youth reported higher rates of internalizing disorder, major depression, and SUD than did internalizing youth, and compared with those in the DBD group, persons in the comorbid group reported more DBDs, particularly ODD and ADHD.

Impairment and Trauma in Comorbid Youth

Comorbid youth not only met criteria for both DBD and internalizing disorder, but compared with youth with either type of psychiatric condition, they showed more extensive comorbidity within both internalizing and externalizing domains. First, comorbid youth were more likely to have an affective disorder than were internalizing youth and to have ODD and ADHD than were DBD youth. Second, comorbid youth were also more likely to have been exposed to sexual and non-sexual violence. Comorbid youth, then, appear to report higher rates of exposure to traumatic events, greater numbers of disorders, and more dependence on substances other than marijuana and alcohol than those with less complex pathology. Our finding is consistent with our earlier understanding of the role of this pattern of comorbidity as a marker for severity of clinical and functional impairment (Angold et al., 1999). Comorbid youth were significantly more likely to report victimization by violence than those with either internalizing disorder or DBD alone. This is consistent with the suggestion that traumatic exposure and other indicators of childhood adversity increase risk of both internalizing and DBD (Kerig & Becker, 2010; Kerig et al., 2009; Wilson et al., 2009). Distress resulting from traumatic events may have both mental and physical adverse consequences, leading to dysregulation of emotion, impulsiveness, dysfunctional information processing, lower levels of empathy, and antisocial behaviors (Ford et al., 2006).

The present findings are in disagreement with those who have argued that individuals with this comorbid pattern might be less impaired than those with DBD alone, because internalizing disorders could diminish antisocial behavior. For example, symptoms of anxiety and high baseline levels of arousal have been suggested as protective factors for antisocial behaviors (Faire & Ollendick, 2013). However, we found that comorbid youth were significantly more often referred to the justice system than DBD youth for violent interpersonal offenses, more serious indicators of antisocial behavior than are non-violent offenses (e.g., theft). In addition, the average number of DBDs was higher in the comorbid group than in the DBD alone group.

Comorbid Condition and Offending

Comorbid youth did not show an earlier onset of antisocial behavior or more serious antisocial history than did those with either internalizing or DBD alone. In addition, although they displayed higher rates of DBDs, they were less likely to meet criteria for CD than those with DBD alone. Our finding that comorbid youth were more likely to have ODD and ADHD than counterparts with DBD alone may suggest that their externalizing problems might be less severe. However, some earlier investigators reported that children and adolescents with both disruptive and internalizing problems were at increased risk of later offending in young adulthood (Copeland et al., 2007; Harrington et al., 1991; Hodgins et al., 2010; Hoeve et al., 2013a). Explanations for that increased risk remain unclear. One possibility might focus on a persistent antisocial pathway with an early onset; this pathway has been characterized by male gender, an early age of onset, and repeat and interpersonal offending (the strongest risk factors for later offending; Gendreau et al., 1996). Yet in the present study, youth with the comorbid pattern were more likely to be female than those with DBD, and offense history did not differentiate among groups when adjusting for rates of traumatic exposure or suicide attempt. Although this disorder profile may be a marker for severity of antisocial behavior (Angold et al., 1999), comorbid youth who engage in delinquent activities would be unlikely to be designated as life-course persistent offenders (Moffitt, 1993; Moffitt & Caspi, 2001; Moffitt, Caspi, Dickson, Silva, & Stanton, 1996), because in the present research, they did not necessarily begin offending at an early age, compared with non-comorbid youth.

As an alternative, traumatic exposure likely increases risk of both internalizing and DBDs. These comorbid conditions may interact to produce worsening course of behavior (Loeber & Keenan, 1994). An earlier study (Hoeve et al., 2013a) found that youth in the justice system with comorbid internalizing and DBD had a six-fold risk of young adult recidivism, whereas those with either disruptive behavior or internalizing disorder alone were not different from non-disordered youth, consistent with the proposition that these disorders may operate interactively.

Limitations and Future Research

Among the strengths of this study are the large sample size and the use of both self-report and official data to measure indicators of age of onset of antisocial behavior. Despite these strengths, a number of limitations should be noted. First, our analyses are based on DSM-IV diagnosis. DSM-5 (APA, 2013) has incorporated changes in the criteria for internalizing and DBDs. Revisions to the diagnostic nomenclature will likely affect prevalence rates of these disorders, but we estimate that the effects on correlates of disorder profiles are rather small. Second, although we considered a range of risk factors (consistent with the model of Wolff & Ollendick, 2006), we lacked information on potentially relevant indicators of childhood adversity and family characteristics. Future studies should examine childhood adversity indicators and systematically analyze shared and unique risk factors longitudinally. Third, our data were cross-sectional, and we had no information on the lifetime course of disorder patterns. Were mental health disorder to be measured longitudinally, developmental transactions and interactions between co-occurring disorders over time could be examined. Fourth, we relied on the DISC for measuring exposure to trauma. The DISC measures a limited number of traumatic events, and as a result, findings regarding the prevalence might be underestimations of actual rates of traumatic exposure in the investigated disorder groups. Fifth, this work is based on adolescent reports of disorder. Although including other informants might have provided higher rates of disorder, research comparing parent report with V-DISC diagnoses (Ko, Wasserman, McReynolds, & Katz, 2004) showed that fewer than one third of parents of justice system youths added substantial new information to youth report of mental health concerns.

We argue that further research on comorbid internalizing and DBD is warranted. DSM-5 now defines a new disorder, namely, DMDD (APA, 2013), characterized by temper outbursts, persistent irritability, and angry mood, as well as increased risk of suicidal and aggressive behavior. Individuals with DMDD are more likely to have a history of ODD, ADHD, or anxiety symptoms. Although our results are based on DSM-IV diagnosis, comorbid youth in our study had relatively high rates of affective disorders, ODD, and ADHD, and were at risk of suicide attempts, consistent with some characteristics of DMDD. However, those designated as comorbid for the present study were more likely to be female than those with no disorder or DBD alone, whereas DMDD is identified more often in males. However, the extent to which the comorbid group would meet criteria for DMDD is unknown. Given that those with DMDD are at risk of comorbid internalizing and DBD (Copeland, Angold, Costello, & Egger, 2013), we suggest that further research concentrates on characteristics of those with DMDD and their levels of offending. Also, in addition to environmental influences, such as childhood maltreatment, it would be interesting to examine which other risk factors (e.g., neurobiological influences) may increase the risk of a comorbid condition and offending. For example, studies on monoamine oxidase A (MAOA) genotype, maltreatment, and mental health problems have shown that genetic influences can enhance or decrease the negative impact of environmental stress (Kim-Cohen et al., 2006).

Implications for Policy and Practice

Clinical services should be keen to screen children and adolescents with comorbid internalizing and DBD adequately. First, those presenting with externalizing concerns should be screened for the presence of additional internalizing disorder. Symptoms of DBD in young persons have been found to interfere with the capacity of adults around them to notice commonly comorbid internalizing concerns. In a study of reasons for referral (Weiss, Jackson, & Susser, 1997), parents’ awareness of internalizing problems was lower when youth presented with externalizing difficulties than when youth presented without externalizing problems. Among youth in the juvenile justice system, those with one or another internalizing disorder are most likely to be unidentified and consequently, the least likely to access services (Hoeve, McReynolds, & Wasserman, 2013b; Wasserman et al., 2008). Externalizing problems may blind those who might recommend services such as family, friends, and school and juvenile justice authorities, to the more subtle internalizing problems, although their co-occurrence is associated with greater impairment and a higher risk of adult reoffending. Particularly, as comorbid youth were less likely to meet criteria of CD, and therefore their externalizing problems might seem to be less severe, a more extensive comorbidity might remain unidentified. Second, those with comorbid internalizing disorder and DBD should be screened for SUD. Those presenting with internalizing disorder are often not screened for SUD, and substance use treatment programs typically focus on substance use and externalizing problems, possibly because SUD is more strongly related to externalizing disorder (Chan, Dennis, & Funk, 2008). Therefore, those in the comorbid group comprise an at-risk group for whom assessment of SUD is needed.

When comorbid internalizing disorder and DBD have been identified, youth in juvenile justice settings should receive adequate treatment that targets both internalizing and externalizing problems. Interventions are very different for youth with different types of disorder profiles, and therefore, it is important that justice agencies effectively match youth to programs that can address their mental and behavioral health needs. For example, youth with CD and co-occurring internalizing disorder may need more intensive treatment programs than do those with other disorder profiles (Lambert, et al., 2001). Given that the comorbid group was more likely to have a history of victimization by interpersonal violence, those with this comorbid condition should receive trauma-focused treatment. Ford and Blaustein (2013) provide an overview of innovative trauma-focused interventions that have the potential to adjust the milieus in juvenile justice institutions and provide youth and the staff skills to cope with and resolve trauma-related symptoms and reactions.

Many youth commence juvenile justice system contact without having accessed mental health services in their communities (e.g., Novins, Duclos, Martin, Jewett, & Manson, 1999), so it is important to improve the identification of a range of mental health concerns via systematic screening. Prevention programs aimed at lawbreaking should focus not only on age of onset of offending but also on contributions of trauma exposure and comorbid internalizing conditions during childhood, perhaps via universal school-based screening (Essex et al., 2009) that captures both internalizing and externalizing concerns. Addressing trauma exposure and this comorbid pattern might prevent a pathway of persistent offending that continues into young adulthood.

Footnotes

This work was supported by the Carmel Hill Fund and by a Marie Curie grant of the European Commission awarded to Machteld Hoeve (FP7-PEOPLE-2010-IOF, project 274337).

Machteld Hoeve, PhD, is assistant professor of forensic child and youth care sciences at the Research Institute of Child Development and Education of the University of Amsterdam, the Netherlands. Her research interests involve family processes and psychopathology in relation to the development of criminal behavior over the life course.

Larkin S. McReynolds, PhD, is an assistant professor at Columbia University’s Departments of Epidemiology and Mental Health Services and Policy Research. As research director for the Center for the Promotion of Mental Health in Juvenile Justice, her work has focused on implementing and evaluating screening and assessment protocols across a range of juvenile justice settings.

Gail A. Wasserman, PhD, is a professor of clinical psychology in child psychiatry at Columbia University. She directs Columbia’s Center for the Promotion of Mental Health in Juvenile Justice (), a research and policy center offering guidance to juvenile justice agencies on protocols for efficient identification and service referral for those with mental health service needs. She has been conducting research on developmental psychopathology for 35 years. She has authored more than 100 articles for academic and practitioner audiences.

References

Aguilar

Sroufe

L. A.

Egeland

Carlson

(2000). Distinguishing the early-onset/persistent and adolescence-onset antisocial behavior types: From birth to 16 years. Development and Psychopathology, 12, 109-132.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Angold

Costello

E. J.

Erkanli

(1999). Comorbidity. Journal of Child Psychology and Psychiatry and Allied Disciplines, 40, 57-87.

Breslau

Lucia

V. C.

Alvarado

G. F.

(2006). Intelligence and other predisposing factors in exposure to trauma and posttraumatic stress disorder. Archives of General Psychiatry, 63, 1238-1245. doi:10.1001/archpsyc.55.7.626

Chan

Dennis

M. L.

Funk

R. R.

(2008). Prevalence and comorbidity of major internalizing and externalizing problems among adolescents and adults presenting to substance abuse treatment. Journal of Substance Abuse Treatment, 34, 14-24. doi:10.1016/j.jsat.2006.12.031

Copeland

W. E.

Angold

Costello

E. J.

Egger

(2013). Prevalence, comorbidity, and correlates of DSM-5 proposed disruptive mood dysregulation disorder. American Journal of Psychiatry, 170, 173-179.

Copeland

W. E.

Miller-Johnson

Keeler

Angold

Costello

E. J.

(2007). Childhood psychiatric disorders and young adult crime: A prospective, population-based study. American Journal of Psychiatry, 164, 1668-1675.

Cunningham

N. R.

Ollendick

T. H.

(2010). Comorbidity of anxiety and conduct problems in children: Implications for clinical research and practice. Clinical Child and Family Psychology Review, 13, 333-347. doi:10.1007/s10567-010-0077-9

10.

Essex

M. J.

Kraemer

H. C.

Slattery

M. J.

Burk

L. R.

Boyce

W. T.

Woodward

H. R.

Kupfer

D. J.

(2009). Screening for childhood mental health problems: Outcomes and early identification. Journal of Child Psychology and Psychiatry, 50, 562-570.

11.

Ezpeleta

Domenech

J. M.

Angold

(2006). A comparison of pure and comorbid CD/ODD and depression. Journal of Child Psychology and Psychiatry, 47, 704-712. doi:10.1111/j.1469-7610.2005.01558.x

12.

Fairchild

Van Goozen

S. H. M.

Calder

A. J.

Goodyer

I. M.

(2013). Research review: Evaluating and reformulating the developmental taxonomic theory of antisocial behavior. Journal of Child Psychology and Psychiatry, 54, 924-940. doi:10.1111/jcpp.12102

13.

Faire

M. G.

Ollendick

T. H.

(2013). Anxiety and oppositional defiant disorder: A transdiagnostic conceptualization. Clinical Psychology Review, 33, 229-240.

14.

Farrington

D. P.

(2006). Building developmental and life-course theories of offending. In Cullen

F. T.

Wright

J. P.

Blevins

K. R.

(Eds.), Taking stock: The status of criminological theory (pp. 335-364). New Brunswick, NJ: Transaction Publishers.

15.

Ford

J. D.

Blaustein

M. E.

(2013). Systemic self-regulation: A framework for trauma-informed services in residential juvenile justice programs. Journal of Family Violence, 28, 665-677. doi:10.1007/s10896-013-9538-5

16.

Ford

J. D.

Chapman

Mack

J. M.

Pearson

(2006). Pathways from traumatic child victimization to delinquency: Implications for juvenile and permanency court proceedings and decisions. Juvenile & Family Court Journal, 57, 13-26.

17.

Ford

J. D.

Hartman

Hawke

Chapman

J. F.

(2008). Traumatic victimization, posttraumatic stress disorder, suicidal ideation, and substance abuse risk among juvenile justice-involved youth. Journal of Child & Adolescent Trauma, 1, 75-92.

18.

Gendreau

Little

Goggin

(1996). A meta-analysis of the predictors of adult offender recidivism: What works! Criminology, 34, 575-607.

19.

Harrington

Fudge

Rutter

Pickles

Hill

(1991). Adult outcomes of childhood and adolescent depression: II. Links with antisocial disorders. Journal of the American Academy of Child & Adolescence Psychiatry, 30, 434-439.

20.

Hayes

M. A.

McReynolds

L. S.

Wasserman

G. A.

(2005). Paper and voice MAYSI-2: Format comparability and concordance with the voice DISC-IV. Assessment, 12, 395-403. doi:10.1177/1073191105280359

21.

Hodgins

De Brito

Chhabra

Coté

(2010). Anxiety disorders among offenders with antisocial personality disorders: A distinct subtype? Canadian Journal of Psychiatry, 55, 784-791.

22.

Hoeve

Blokland

Dubas

J. S.

Loeber

Gerris

J. R. M.

Van der Laan

P. H.

(2008). Trajectories of delinquency and parenting styles. Journal of Abnormal Child Psychology, 36, 223-235. doi:10.1007/s10802-007-9172-x

23.

Hoeve

McReynolds

L. S.

Wasserman

G. A.

(2013a). The influence of adolescent psychiatric disorder on young adult recidivism. Criminal Justice and Behavior, 40, 1462-1476. doi:10.1177/0093854813488106

24.

Hoeve

McReynolds

L. S.

Wasserman

G. A.

(2013b). Service referral for juvenile justice youths: Associations with psychiatric disorder and recidivism. Administration and Policy in Mental Health and Mental Health Services Research, 41, 379-389. doi:10.1007/s10488-013-0472-x

25.

Hosmer

Lemeshow

(2000). Applied logistic regression (2nd ed.). New York, NY: John Wiley.

26.

Kerig

P. K.

Becker

S. P.

(2010). From internalizing to externalizing: Theoretical models of the processes linking PTSD to juvenile delinquency. In Egan

S. J.

(Ed.), Post-traumatic stress disorder (PTSD): Causes, symptoms and treatment (pp. 33-78). Hauppauge, NY: Nova Science.

27.

Kerig

P. K.

Ward

R. M.

Vanderzee

K. L.

Moeddel

M. A.

(2009). Posttraumatic stress as a mediator of the relationship between trauma and mental health problems among juvenile delinquents. Journal of Youth Adolescence, 38, 1214-1225. doi:10.1007/s10964-008-9332-5

28.

Kim-Cohen

Caspi

Taylor

Williams

Newcombe

Craig

I. W.

Moffitt

T. E.

(2006). MAOA, maltreatment, and gene–environment interaction predicting children’s mental health: New evidence and a meta-analysis. Molecular Psychiatry, 11, 903-913. doi:10.1038/sj.mp.4001851

29.

S. J.

Wasserman

G. A.

McReynolds

L. S.

Katz

L. M.

(2004). Contribution of parent report to voice DISC-IV diagnosis among incarcerated youths. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 868-877.

30.

Lambert

Wahler

R. G.

Andrade

A. R.

Bickman

(2001). Looking for the disorder in conduct disorder. Journal of Abnormal Psychology, 110, 110-123. doi:10.1037/0021-843X.110.1.110

31.

Lewinsohn

P. M.

Rohde

Seeley

J. R.

(1995). Adolescent psychopathology: III. The clinical consequences of comorbidity. Journal of the American Academy of Child & Adolescent Psychiatry, 34, 510-519.

32.

Lilienfeld

S. O.

(2003). Comorbidity between and within childhood externalizing and internalizing disorders: Reflections and directions. Journal of Abnormal Child Psychology, 31, 285-291.

33.

Loeber

Burke

J. D.

(2011). Developmental pathways in juvenile externalizing and internalizing problems. Journal of Research on Adolescence, 21, 34-46. doi:10.1111/j.1532-7795.2010.00713.x

34.

Loeber

Keenan

(1994). Interaction between conduct disorder and its comorbid conditions: Effects of age and gender. Clinical Psychology Review, 14, 497-523.

35.

Lucas

C. P.

(2003). The use of structured diagnostic interviews in clinical child psychiatric practice. In First

M. B.

(Ed.), Review of psychiatry: Structured evaluation in clinical practice (Vol. 22, pp. 75-102). Washington, DC: American Psychiatric Press.

36.

Maxfield

M. G.

Widom

C. S.

(1996). The cycle of violence: Revisited 6 years later. Archives of Pediatrics and Adolescent Medicine, 150, 390-395.

37.

McReynolds

L. S.

Wasserman

G. A.

Fisher

P. W.

Lucas

C. P.

(2007). Diagnostic screening with incarcerated youths: Comparing the DPS and voice DISC. Criminal Justice and Behavior, 34, 830-845.

38.

Moffitt

T. E.

(1993). Adolescence-limited and life-course-persistent antisocial behavior: A developmental taxonomy. Psychological Review, 100, 674-701.

39.

Moffitt

T. E.

(2006). A review of research on the taxonomy of life-course persistent versus adolescence-limited antisocial behavior. In Cullen

F. T.

Wright

J. P.

Blevins

K. R.

(Eds.), Taking stock: The status of criminological theory (pp. 277-311). New Brunswick, NJ: Transaction Publishers.

40.

Moffitt

T. E.

Caspi

(2001). Childhood predictors differentiate life-course-persistent and adolescence-limited antisocial pathways among males and females. Development and Psychopathology, 13, 355-375.

41.

Moffitt

T. E.

Caspi

Dickson

Silva

Stanton

(1996). Childhood-onset versus adolescent-onset antisocial conduct problems in males: Natural history from ages 3 to 18 years. Development and Psychopathology, 8, 399-424.

42.

Newman

D. L.

Moffitt

T. E.

Caspi

Silva

P. A.

(1998). Comorbid mental disorders: Implications for treatment and sample selection. Journal of Abnormal Psychology, 107, 305-311.

43.

Novins

D. K.

Duclos

C. W.

Martin

Jewett

C. S.

Manson

S. M.

(1999). Utilization of alcohol, drug, and mental health treatment services among American Indian adolescent detainees. Journal of the American Academy of Child & Adolescent Psychiatry, 38, 1102-1108.

44.

Patterson

G. R.

Yoerger

(2002). A developmental model for early- and late-onset delinquency. In Reid

J. B.

Patterson

G. R.

Snyder

(Eds.), Antisocial behavior in children and adolescents: A developmental analysis and model for intervention (pp. 147-172). Washington, DC: American Psychological Association.

45.

Renouf

A. G.

Kovacs

Mukerji

(1997). Relationship of depressive, conduct, and comorbid disorders and social functioning in childhood. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 998-1004.

46.

Ruchkin

V. V.

Koposov

Vermeiren

R. R. J. M.

Schwab-Stone

M. E.

(2003). Psychopathology and age at onset of conduct problems in juvenile delinquents. Journal of Clinical Psychiatry, 64, 913-920.

47.

Schaffer

D. M.

Fisher

P. W.

Dulcan

M. K.

Davies

Piacentini

J. C.

Schwab-Stone

M. E.

(1996). The NIMH diagnostic interview schedule for children (DISC 2.3): Description, acceptability, prevalence, and performance in the MECA study. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 865-877.

48.

Schaffer

D. M.

Fisher

P. W.

Lucas

C. P.

Dulcan

M. K.

Schwab-Stone

M. E.

(2000). NIMH diagnostic interview schedule for children, version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of American Academy of Child & Adolescent Psychiatry, 39, 28-38.

49.

Steinhausen

H. C.

Reitzle

(1996). The validity of mixed disorders of conduct and emotions in children and adolescents: A research note. Journal of Child Psychology and Psychiatry, 37, 339-343.

50.

Teplin

L. A.

Abram

K. M.

McClelland

G. M.

Dulcan

M. K.

Mericle

A. A.

(2002). Psychiatric disorders in youth in juvenile detention. Archives of General Psychiatry, 59, 1133-1143. doi:10.1001/archpsyc.59.12.1133

51.

Twardosz

Lutzker

J. R.

(2010). Child maltreatment and the developing brain: A review of neuroscience perspectives. Aggression and Violent Behavior, 15, 59-68. doi:10.1016/j.avb.2009.08.003

52.

Van Lier

P. A. C.

Vitaro

Barker

E. D.

Brendgen

Tremblay

R. E.

Boivin

(2012). Peer victimization, poor academic achievement, and the link between childhood externalizing and internalizing problems. Child Development, 83, 1775-1788. doi:10.1111/j.1467-8624.2012.01802.x

53.

Vermeiren

R. R. J. M.

(2003). Psychopathology and delinquency in adolescents: A descriptive and developmental perspective. Clinical Psychology Review, 23, 277-318.

54.

Wasserman

G. A.

S. J.

McReynolds

L. S.

(2004, August). Assessing the mental health status of youth in juvenile justice settings (Juvenile Justice Bulletin, pp. 1-7). Retrieved from www.ojp.usdoj.gov/ojjdp

55.

Wasserman

G. A.

McReynolds

L. S.

(2011). Contributors to traumatic exposure and posttraumatic stress disorder in juvenile justice youths. Journal of Traumatic Stress, 24, 422-429.

56.

Wasserman

G. A.

McReynolds

L. S.

S. J.

Katz

L. M.

Carpenter

J. R.

(2005). Gender differences in psychiatric disorders at juvenile probation intake. American Journal of Public Health, 95, 131-137. doi:10.2105/AJPH.2003.024737

57.

Wasserman

G. A.

McReynolds

L. S.

Lucas

C. P.

Fisher

Santos

(2002). The voice DISC-IV with incarcerated male youths: Prevalence of disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 41, 314-321. doi:10.1097/00004583-200203000-00011

58.

Wasserman

G. A.

McReynolds

L. S.

Schwalbe

C. S.

Keating

J. M.

Jones

S. A.

(2010). Psychiatric disorder, comorbidity, and suicidal behavior in juvenile justice youth. Criminal Justice and Behavior, 37, 1361-1376. doi:10.1177/0093854810382751

59.

Wasserman

G. A.

McReynolds

L. S.

Whited

A. L.

Keating

J. M.

Musabegovic

Yanling

(2008). Juvenile probation officers’ mental health decision making. Administration and Policy in Mental Health and Mental Health Services Research, 35, 410-422.

60.

Weiss

Jackson

E. W.

Susser

(1997). Effect of co-occurrence on the referability of internalizing and externalizing problem behavior in adolescents. Journal of Clinical Child Psychology, 26, 198-204. doi:10.1207/s15374424jccp2602_8

61.

Widom

C. S.

(1989). The cycle of violence. Science, 244, 160-166.

62.

Wilson

H. W.

Stover

C. S.

Berkowitz

S. J.

(2009). The relationship between childhood violence exposure and juvenile antisocial behavior: A meta-analytic review. Journal of Child Psychology and Psychiatry, 50, 769-779. doi:10.1111/j.1469-7610.2008.01974.x

63.

Wolff

J. C.

Ollendick

T. H.

(2006). The comorbidity of conduct problems and depression in childhood and adolescence. Clinical Child and Family Psychology Review, 9, 201-220.