Deficient Emotional Self-Regulation in Preschoolers With ADHD: Identification,Comorbidity,and Interpersonal Functioning

Abstract

Objective: This study aims to develop an age-adjusted Child Behavior Checklist- (CBCL) and Teacher Report Form (TRF)-based method for the detection of deficient emotional self-regulation (DESR) in preschoolers with ADHD and to assess its incidence, comorbidities, and consequences on interpersonal functioning. Method: Eighty-six ADHD preschoolers and 104 controls were assessed using CBCL, TRF/1½ to 5, Psychiatric Interview With Preschool Age Psychiatric Assessment, Leiter-R, and ADHD rating scales. Results: Greatest sensitivity and specificity were obtained applying slightly lower threshold scores compared with school-age children (CBCL: Anxiety/Depression [A/D] ≥ 59, Attention Problems [AP] ≥ 60, Aggression Behaviors [AB] ≥ 58; TRF: A/D ≥ 59, AP ≥ 60, AB ≥ 60). DESR was detected in 33/86 (38.4%) and in 16/54 (29.6%) ADHD preschoolers versus 2/104 (1.9%) controls using CBCL and TRF, respectively. DESR is associated with significantly greater comorbidity and impairment in interpersonal functioning. Conclusion: Among ADHD preschoolers, DESR (a) requires lower CBCL and TRF threshold scores for detection, compared with school-age children, (b) displays similar incidence rates, and (c) is associated with enhanced psychiatric comorbidity and interpersonal difficulties.

Keywords

ADHD Child Behavior Checklist emotional self-regulation oppositional defiant disorder preschool children

Introduction

Deficient emotional self-regulation (DESR) is defined as an impairment in the ability to modulate intense emotions and their behavioral manifestations (Barkley, 1997; Barkley & Fischer, 2010). This overarching dysfunction encompasses at least four functionally distinct, yet strongly interrelated, processes: (a) deficits in self-regulating the physiological arousal caused by intense emotions, (b) difficulty in inhibiting behavior in response to either positive or negative emotions, (c) problems refocusing attention following intense emotions, and (d) disorganization of behavior in response to emotional activation (Spencer et al., 2011). Clinically, DESR includes low tolerance to frustration, impatience, irritability, dysphoria, and a general proneness to emotional reactions (Purper-Ouakil & Franc, 2011). DESR represents a trans-diagnostic dimension, present to a variable extent in the clinical presentation of several psychiatric entities. Among these, DESR appears especially linked to ADHD, as supported not only by clinical evidence but also by neuroimaging studies identifying impaired prefrontal cortical circuitry as the main neurological underpinning of deficits in emotional self-regulation and inhibition (Purper-Ouakil & Franc, 2011). A deeper understanding of this link between DESR and ADHD presently requires (a) the definition of appropriate evaluation tools and measurement methods for DESR, (b) its distinction from the emotional symptoms that characterize mood disorders, and (c) an appreciation of its influence on psychiatric disorders comorbid with ADHD, primarily oppositional defiant disorder (ODD).

In recent years, several authors have postulated that DESR can be detected through the Child Behavior Checklist (CBCL). In particular, a specific profile has been proposed consisting of scores >60 and <70 (i.e., intermediate between >1 and <2 SDs) on three CBCL scales reflecting DESR-related emotions and behaviors, namely, (a) Anxiety/Depression (A/D), (b) Aggression Behaviors (AB), and (c) Attention Problems (AP). Collectively, these three “A-A-A scales” should thus yield an aggregate threshold score >180 and <210 (Biederman et al., 2012; Spencer et al., 2011). By applying these CBCL criteria, 44% of 197 school-age ADHD children qualified for DESR, as compared with 2% of 224 age-matched controls (Spencer et al., 2011). This same CBCL profile predicted subsequent psychopathology and functional impairment in school-age children with ADHD at 4 years of follow-up, including higher rates of anxiety and disruptive behavior disorders, as well as significantly greater impairment in emotional and interpersonal functioning compared with ADHD children without DESR (Biederman et al., 2012; Spencer et al., 2011). Hence, the A-A-A dimensions of the CBCL could represent a potentially valid and reliable measure capable of identifying ADHD children with a psychopathological and functional profile consistent with the clinical definition of DESR.

ADHD is increasingly diagnosed at preschool age (Campbell, Pierce, March, Ewing, & Szumowski, 1994; Egger & Angold, 2006; Hardy et al., 2007; Healey, Miller, Castelli, Marks, & Halperin, 2008; Marakovitz & Campbell, 1998; Posner et al., 2007). In the United States, as many as 6% of preschoolers receive an ADHD diagnosis in community samples (Lavigne, Lebailly, Hopkins, Gouze, & Binns, 2009). A recent study on 2,475 preschoolers in Norway estimates the prevalence of ADHD at 1.9% (Wichstrøm et al., 2011). Even this lower rate poses ADHD as a significant public health challenge in the preschool population. In preschoolers, ADHD shares many features with the disorder diagnosed in school-age children and in adolescents (Sonuga-Barke, Stevenson, Thompson, & Viney, 1997). This similarity is not surprising, because factor analyses performed on symptoms from seven Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) disorders, including ADHD, showed that in preschoolers, psychopathology is largely differentiated according to DSM-IV syndromes strikingly similar to those found in older children and adolescents (Sterba, Egger, & Angold, 2007). In particular, preschool ADHD encompasses two subcomponents, namely, hyperactivity/impulsivity and inattention (Sterba et al., 2007). Most affected preschoolers display predominant hyperactivity/impulsivity or both subcomponents. ADHD symptoms in preschoolers are associated with developmental delay, pre-academic skill deficits, and problems with close interpersonal relationships (Hardy et al., 2007; Wilens et al., 2002). They show moderate stability over time, with rates and patterns of homotypic and heterotypic continuity similar to those observed in samples of children diagnosed at an older age (Bufferd, Dougherty, Carlson, Rose, & Klein, 2012; Lahey et al., 2004). Comorbidity is already common among 2- to 6-year-old children with ADHD, with 64% to 74% meeting diagnostic criteria for at least one additional psychiatric disorder, typically ODD, conduct disorder, generalized anxiety disorder, and/or mood disorders (Angold, Costello, & Erkanli, 1999; Posner et al., 2007; Wilens et al., 2002). Preschoolers with ADHD and/or ODD indeed fail to adequately control their anger, resulting in more frequent aggressive outbursts (Chang, Schwartz, Dodge, & McBride-Chang, 2003; Eisenberg et al., 2001). Interestingly, impairment in emotional, behavioral, and social domains may persist into school years and later in life, even when ADHD symptoms decrease (Lee, Lahey, Owens, & Hinshaw, 2008). The similarity of ADHD symptoms, functional impairment, and psychiatric comorbidity in preschool and school-age children suggests that age 3 to 5 years may well represent the most critical developmental time window for the early identification of both ADHD and DESR.

Despite the importance of DESR in preschoolers with ADHD, its reliable identification in this age group, as well as its relationship with psychiatric comorbidity and with long-term impairment, still remain two open questions. In particular, no “gold-standard” assessment method has been established to define the presence of DESR in ADHD preschoolers. In fact, the A-A-A CBCL profile described above has been applied starting from school age (i.e., >6 years old; Biederman et al., 2012; Spencer et al., 2011).

The present study aims at (a) optimizing the A-A-A CBCL profile for a DESR diagnosis before 6 years of age, by applying receiver operating characteristic (ROC) analyses to identify threshold scores endowed with greatest sensitivity/specificity in this age window and then (b) assessing the most frequent psychiatric comorbidities and consequences of DESR on interpersonal functioning in ADHD preschoolers, using the Psychiatric Interview With Preschool Age Psychiatric Assessment (PAPA). To maximize informativeness and strengthen reliability, both parent- and teacher-rated forms are used, because (a) the intensity of ADHD symptoms is context dependent and an ADHD diagnosis requires several distinct observers, (b) teachers tend to be more sensitive to disruptive behaviors, whereas parents are more sensitive to depression and anxiety (Abikoff, Courtney, Pelham, & Koplewicz, 1993), (c) parents and teachers tend not to regard DESR as an indicator of serious behavioral difficulties at this early age (Belden, Thomson, & Luby, 2008), and (d) different informant correspondence tends to be higher for externalizing than for internalizing problems, both in patient and in non-clinical samples (Achenbach, McConaughy, & Howell, 1987; Kolko & Kazdin, 1993).

Materials and Method

Participants

Our initial sample included 92 Caucasian Italian preschoolers, newly diagnosed with ADHD at several Child and Adolescent NeuroPsychiatry Outpatient Centers in Rome (Italy). Children with neurological sensory-motor deficits, cognitive impairment (IQ < 70), or autistic disorder were excluded. Six children reached scores above 70 in the three CBCL dimensions and an aggregate score above 210 and were thus excluded as in Spencer et al. (2011). Our age- and sex-matched control group consisted of 106 Caucasian Italian preschoolers, randomly recruited at a pediatric outpatient service. Two children did not complete the assessment and were thus excluded from the study. Hence, our sample ultimately consists of 86 preschoolers with a clinical diagnosis of ADHD (74 males, 12 females; M age = 54.8 ± 11.8 months) and 104 controls (80 males, 24 females; M age = 57.2 ± 7.8 months). Cases and controls do not significantly differ either in age (F = 2.649, df = 1,184, p = .105, ns) or in gender distribution (χ² = 3.053, p = .081, ns).

Clinical Assessment

Clinical observation and psychodiagnostic testing spanned three visits on separate days. During the first visit, the clinical and developmental history of the child was collected from the primary caregiver, while the child participated in an unstructured behavioral observation session. During the second visit, parents were administered an age-adapted semi-structured interview, the PAPA (Egger et al., 2006), while the child underwent cognitive assessment by the Leiter-R (Leiter, 1979). Only preschool children with ADHD underwent a third visit involving two ADHD rating scales (DuPaul, Power, Anastopoulos, & Reid, 1998) adapted for the Italian population, “Scala per l’individuazione di comportamenti di disattenzione e iperattività” (SDAI) and “Scala diretta ai genitori per l’individuazione di comportamenti di disattenzione e iperattività nel bambino” (SDAG), completed by teachers and parents, respectively (Marzocchi & Cornoldi, 2000). CBCL and TRF were filled out by parents and teachers, respectively (Achenbach & Rescorla, 2000; Frigerio, 2000). All the questionnaires used in the present study were collected as part of the routine clinical assessment of children diagnosed with ADHD. Preschool-age cases were recruited for the present study if they satisfied DSM-IV criteria for ADHD, as consistently documented by (a) direct observation by an expert clinician (M.G.M., B.M., E.V.), (b) patient history provided by parents, (c) PAPA scores, and (d) SDAI/SDAG rating scale scores. Verbal informed consent to participate in this study was thus provided by all parents for their children.

Psychometric Instruments

The CBCL 1½ to 5 and Teacher Report Form (TRF) 1½ to 5 were filled out by parents and teachers, respectively, for both ADHD cases and controls. CBCL and TRF for preschoolers are composed of 99 items inherent to seven dimensions for parents and six dimensions for teachers, designed to rate childhood emotional and behavioral problems. Items are scored by parents and teachers as either 0, 1, or 2 corresponding to definitely false, probably true, or definitely true, respectively (Achenbach & Rescorla, 2000). A standardized Italian version is available (Frigerio, 2000). The reliability and validity of the CBCL in predicting behavioral profiles (i.e., “externalizing” vs. “internalizing” patterns of scale elevations), which tend to remain stable throughout childhood, are well established (Mattison & Spitznagel, 1999).

The PAPA is a parent semi-structured interview to diagnose psychiatric disorders in preschool children aged 2 through 5.11 years and provides a comprehensive, age-appropriate assessment of all relevant Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; American Psychiatric Association, 2000) diagnoses (Egger et al., 2006). It uses an interviewer-based approach and includes a detailed glossary that provides guidelines for rating the symptom severity, frequency, and duration, as well as functional impairment rated as 0 = absent, 1 = infrequent, 2 = partial, 3 = severe. The PAPA has been demonstrated both reliable and valid, when diagnosing psychiatric disorders in preschool children (Egger et al., 2006).

Non-verbal cognitive abilities were assessed using the Leiter International Performance Scale–Revised (Leiter, 1979).

The SDAI and SDAG rating scales (Marzocchi & Cornoldi, 2000) were filled out by teachers and parents, respectively, for ADHD children only, to rate on a 4-point scale the frequency/intensity of each of the nine symptoms of inattention and of the nine symptoms of hyperactivity/impulsivity listed in DSM-IV. Each symptom can be scored as never or rarely, sometimes, often, or very often. The SDAI and SDAG scales display the following psychometric properties: test–retest reliability, r = .80, and interrater reliability, r = .80; construct validity of SDAI: r = .85 for hyperactivity and r = .50 for inattention; construct validity of SDAG: r = .47 (Donfrancesco et al., 2014; Marzocchi & Cornoldi, 2000).

CBCL and TRF Profile for DESR

The “A-A-A” CBCL and TRF profile is used to define the presence of DESR. It includes (a) intermediate elevations scores (>1 and <2 SDs) on the “A-A-A” scales, namely, A/D, AB, and AP, as well as (b) aggregate threshold score for the three A-A-A scales (>180 and <210; Biederman et al., 2012; Spencer et al., 2011). On the contrary, children were excluded if they displayed a CBCL or TRF profile consisting of very abnormal scores on the same A-A-A scales (i.e., >2 SDs in each scale and an aggregate score ≥210). This profile has been postulated to unveil the most profoundly dysregulated mood and behavior, associated with pediatric bipolar disorder (PBD) according to some (Biederman et al., 2009; Biederman et al., 1995), predictive of the most severe forms of the ADHD–ODD–conduct disorder spectrum according to others (Volk & Todd, 2007), or more broadly, linked with developmental trajectories leading to various forms of severe psychiatric impairment by other authors (Hudziak, Althoff, Derks, Faraone, & Boomsma, 2005; Meyer et al., 2009; for a systematic review, see Ratheesh, Berk, Davey, McGorry, & Cotton, 2015).

Statistical Procedure

The following statistical approach was applied:

Parameters were initially contrasted between the following three samples: (a) preschool ADHD children with a positive DESR profile at the CBCL applying school-age criteria (i.e., aggregate A-A-A score >180 and <210, and each A scale >60, as in Spencer et al., 2011, and Biederman et al., 2012), (b) preschool ADHD children not fulfilling school-age criteria for a DESR profile at the CBCL, and (c) control children without ADHD.

ROC analyses were performed contrasting preschool ADHD children with a positive DESR profile at the CBCL according to school-age criteria versus controls, to determine for each CBCL dimension (AB, A/D, AP) the threshold scores endowed with greatest sensitivity and specificity in preschoolers. In fact, ROC curves represent a diagram of true positive against false positive rates for all possible threshold scores of a diagnostic test or clinical scale. This statistical tool shows the trade-off between sensitivity and specificity, whereas the area under the curve is a measure of test accuracy.

The same ROC analyses were performed on TRF scores, available for 54 ADHD and 104 control children.

The correlation between parental and teacher scores for each dimension was calculated using Kendall’s tau; concordance rates for “presence” and “absence” of DESR were tested by kappa coefficient.

The entire sample was then reanalysed applying the threshold scores produced by ROC analyses to define the true incidence of DESR in ADHD preschoolers based on CBCL and TRF scores.

The presence of psychiatric comorbidity, the severity of DSM-IV symptoms, and the degree of functional impairment were then contrasted among (a) ADHD preschoolers with DESR, (b) ADHD preschoolers without DESR, and (c) controls, as previously done in school-age ADHD children (Biederman et al., 2012; Spencer et al., 2011). These analyses were repeated defining samples (a) and (b) above, initially, according to the previously published school-age threshold scores and, subsequently, according to the preschool-specific threshold scores provided by the initial ROC analyses (see below). Univariate parametric and non-parametric ANOVAs, or χ² tests, were used, depending on the outcome measure. Percentages and other statistics always refer to total sample sizes of n = 86 and 104 for ADHD cases and controls, respectively, with the only exception of TRF data (ADHD n = 54), due to teachers not returning the forms for 32 ADHD children. All analyses were performed using SPSS software 17.0.

Results

CBCL Parent Reports

Only 17/86 (19.8%) preschoolers with ADHD satisfied school-age CBCL criteria for the presence of DESR, namely, scores >60 in each of the three A-A-A dimensions with an aggregate score >180 and <210. As predictable, these 17 ADHD children showed significantly greater CBCL scores in all three A-A-A dimensions, when compared with the 104 controls (Table 1A). However, these incidence rates for DESR are largely lower than the 44% rates previously reported in school-age children (Biederman et al., 2012; Spencer et al., 2011). Most importantly, as many as 66 (76.7%) ADHD preschoolers showed an A-A-A aggregate score falling within the >180 and <210 interval, but not scores >60 in all three dimensions. ROC analyses were thus performed contrasting the 17 ADHD preschoolers with definite DESR versus the 104 controls, to verify whether different threshold scores could possibly provide a better fit for this age group. Greatest sensitivity and specificity were obtained with thresholds set at 59 for A/D, 58 for AB, and 60 for the AP dimension (Table 2A and Supplementary Figure 1, left panel). An aggregate CBCL score >178 was endowed with noticeable 100% sensitivity and 97.1% specificity. Applying these slightly lower thresholds (A/D ≥ 59, AP ≥ 60, AB ≥ 58), the presence of DESR was determined in 33/86 (38.4%) preschoolers with ADHD and in two (1.9%) controls (χ² = 41.62, df = 1, p < .0001). The severity of ADHD among preschoolers with DESR, defined according to these new thresholds, may be intermediate between preschoolers satisfying the more stringent school-age criteria and preschoolers without DESR (Supplementary Figure 2).

Table 1.

Mean (±SD) A-A-A Scores Obtained With (A) the CBCL and (B) the TRF by ADHD Children With A-A-A > 60, ADHD Children With A-A-A < 60, and CON.

A	Samples			K-W ANOVA	Pairwise comparisons
CBCL dimensions	ADHD/A-A-A > 60 (n = 17)	ADHD/A-A-A < 60 (n = 69)	CON (n = 104)	K-W ANOVA	ADHD/A-A-A > 60 versus CON	ADHD/A-A-A > 60 versus ADHD/A-A-A < 60	ADHD/A-A-A < 60 versus CON
Anxiety/Depression	65.94 ± 3.53	55.14 ± 5.09	53.15 ± 4.56	K-W = 48.174 (2df)	U = 70.000	U = 58.000	U = 2,579.000
				p = 3.46 × 10⁻¹¹	p = 5.77 × 10⁻¹⁰	p = 7.56 × 10⁻⁹	p = .001
Attention Problems	69.41 ± 5.54	67.46 ± 4.87	53.02 ± 4.75	K-W = 125.341 (2df)	U = 42.000	U = 508.000	U = 257.000
				p = 6.06 × 10⁻²⁸	p = 8.62 × 10⁻¹¹	p = .384	p = 1.26 × 10⁻²⁵
Aggression Behaviors	66.35 ± 4.96	62.00 ± 7.19	51.52 ± 3.30	K-W = 114.106 (2df)	U = 31.000	U = 317.500	U = 551.500
				p = 1.67 × 10⁻²⁵	p = 6.97 × 10⁻¹²	p = .003	p = 3.46 × 10⁻²²
B	Samples			K-W ANOVA	Pairwise comparisons
TRF dimensions	ADHD/A-A-A > 60 (n = 12)	ADHD/A-A-A < 60 (n = 42)	CON (N = 104)	K-W ANOVA	ADHD/A-A-A > 60 versus CON	ADHD/A-A-A > 60 versus ADHD/A-A-A < 60	ADHD/A-A-A < 60 versus CON
Anxiety/Depression	64.17 ± 3.64	55.78 ± 5.49	53.64 ± 4.88	K-W = 31.656 (2df)	U = 77.500	U = 41.500	U = 1,598.000
				p = 1.33 × 10⁻⁷	p = 4.02 × 10⁻⁷	p = 1.01 × 10⁻⁵	p = .010
Attention Problems	67.83 ± 4.64	66.62 ± 8.79	52.79 ± 4.51	K-W = 82.774 (2df)	U = 36.000	U = 208.000	U = 341.500
				p = 1.06 × 10⁻¹⁸	p = 3.18 × 10⁻⁸	p = .357	p = 4.86 × 10⁻¹⁶
Aggression Behaviors	66.33 ± 4.29	61.86 ± 7.75	52.09 ± 3.70	K-W = 82.610 (2df)	U = 16.500	U = 135.500	U = 437.500
				p = 1.152 × 10⁻¹⁸	p = 1.94 × 10⁻⁹	p = .0156	p = 1.91 × 10⁻¹⁵

Note. A-A-A = Anxiety/Depression, Aggression Behaviors, Attention Problems; CBCL = Child Behavior Checklist; TRF = Teacher Report Form; CON = controls; K-W = Kruskal–Wallis. Sample sizes, K-W non-parametric ANOVAs, and pairwise contrasts by Mann–Whitney U Tests are reported.

Table 2.

ROC Parameters Obtained in Preschoolers for the A-A-A Dimensions, by Contrasting: (Top) CBCL Scores in 17 ADHD Children With A-A-A > 60 (i.e., DESR Defined as in School-Age Children) Versus 104 Age-Matched Controls and (Bottom) TRF Scores in 12 ADHD Preschooler With A-A-A > 60 Versus 104 Age-Matched Controls.

	Best cutoff	Sensitivity	Specificity	AUC	95% CI	p value
CBCL dimensions
Anxiety/Depression	59	100.0	90.4	0.960	[0.929, 0.992]	1.27 × 10⁻⁹
Attention Problems	60	100.0	88.5	0.976	[0.953, 1.000]	3.38 × 10⁻¹⁰
Aggression Behaviors	58	100.0	93.3	0.982	[0.962, 1.000]	1.99 × 10-⁻¹⁰
TRF dimensions
Anxiety/Depression	59	100.0	89.4	0.943	[0.901, 0.984]	5.48 × 10⁻⁷
Attention Problems	60	100.0	92.3	0.976	[0.952, 1.002]	6.72 × 10⁻⁸
Aggression Behaviors	60	100.0	95.2	0.984	[0.965, 1.003]	4.36 × 10⁻⁸

Note. A-A-A = Anxiety/Depression, Aggression Behaviors, Attention Problems; ROC = receiver operating characteristic; CBCL = Child Behavior Checklist; DESR = deficient emotional self-regulation; TRF = Teacher Report Form; AUC = area under the curve; CI = confidence interval.

TRF Teacher Reports

Teacher reports were obtained for 54/86 ADHD children and for all 104 controls. Among the ADHD preschoolers, an aggregate A-A-A profile >180 and <210 with threshold scores >60 in all three dimensions was seen in 12 (21.8%), who again predictably displayed significant differences in all three dimensions when compared with 104 controls (Table 1B). ROC analyses contrasting these 12 ADHD preschoolers satisfying school-age criteria for the presence of DESR versus the 104 controls yielded greatest sensitivity and specificity for thresholds set at A/D ≥ 59, AP ≥ 60, AB ≥ 60 (Table 2B and Supplementary Figure 1, right panel), whereas an aggregate TRF score >180 was still endowed with best 100.0 sensitivity and 94.2 specificity. Applying these cutoffs, DESR was identified in 16/54 (29.6%) preschoolers with ADHD and in 2/104 (1.9%) controls (χ² = 27.63, df = 1, p < .0001).

Concordance Between CBCL and TRF

Although ADHD and DESR may be more prominent either in school or in the family environment, concordance between parent and teacher reports enhances confidence in the appropriateness of the diagnosis. Highly significant concordance was detected between CBCL and TRF for “presence/absence of DESR,” defined according to our novel threshold scores (κ coefficient = 0.716; p = 8.85 × 10⁻⁸). In general, A-A-A CBCL and TRF scores were quite consistent in ADHD with DESR, ADHD without DESR, and controls, when DESR was defined according to these age-adjusted threshold scores (Table 3). Collectively, the 86 preschoolers with ADHD displayed significantly correlated CBCL and TRF scores in all three dimensions (A/D: τ = 0.363, p = 4.43 × 10⁻⁴; AP: τ = 0.261, p = .012; AB: τ = 0.230, p = .018). Full concordance between CBCL and TRF for the presence of DESR was detected in 39/54 (72.2%) and in 41/54 (75.9%) ADHD children, applying threshold scores for school age and for preschoolers, respectively. Hence, using either CBCL or TRF, the threshold scores obtained by ROC analyses detect DESR with an incidence much closer to the rates previously reported for school-age children (Biederman et al., 2012; Spencer et al., 2011).

Table 3.

Mean (±SD) A-A-A Scores Obtained With (A) the CBCL and (B) the TRF by ADHD Children With DESR (DESR+), ADHD Children Without DESR (DESR−), and CON, When DESR Is Defined According to Preschool-Specific Threshold Scores Obtained by ROC Analyses.

A	Samples			K-W ANOVA	Pairwise comparisons
CBCL dimensions	ADHD/DESR+ (n = 33)	ADHD/DESR− (n = 53)	CON (n = 104)	K-W ANOVA	ADHD/DESR+ versus CON	ADHD/DESR+ versus ADHD/DESR−	ADHD/DESR− versus CON
Anxiety/Depression	63.06 ± 4.31	53.68 ± 4.74	53.15 ± 4.56	K-W = 63.021 (2df)	U = 244.000	U = 122.500	U = 2,405.000
				p = 2.07 × 10⁻¹⁴	p = 5.41 × 10⁻¹⁴	p = 1.66 × 10⁻¹¹	p = .181
Attention Problems	69.00 ± 4.63	67.13 ± 5.18	53.02 ± 4.75	K-W = 126.675 (2df)	U = 78.500	U = 727.000	U = 220.500
				p = 5.13 × 10⁻²⁸	p = 3.31 × 10⁻¹⁷	p = .180	p = 1.13 × 10⁻²¹
Aggression Behaviors	64.06 ± 4.96	62.11 ± 7.97	51.52 ± 3.30	K-W = 112.672 (2df)	U = 108.000	U = 671.500	U = 474.500
				p = 3.42 × 10⁻²⁵	p = 1.68 × 10⁻¹⁷	p = .071	p = 1.36 × 10⁻¹⁸
B	Samples			K-W ANOVA	Pairwise comparisons
TRF dimensions	ADHD/DESR+ (n = 16)	ADHD/DESR− (n = 38)	CON (n = 104)	K-W ANOVA	ADHD/DESR+ versus CON	ADHD/DESR+ versus ADHD/DESR−	ADHD/DESR− versus CON
Anxiety/Depression	62.88 ± 3.88	55.29 ± 5.66	53.64 ± 4.88	K-W = 33.766 (2df)	U = 131.500	U = 61.000	U = 1,538.000
				p = 4.65 × 10⁻⁸	p = 3.64 × 10⁻⁸	p = 6.61 × 10⁻⁶	p = .040
Attention Problems	69.12 ± 5.65	65.95 ± 8.63	52.79 ± 4.51	K-W = 83.626 (2df)	U = 34.000	U = 198.500	U = 337.000
				p = 6.93 × 10⁻¹⁹	p = 2.09 × 10⁻¹⁰	p = .068	p = 1.29 × 10⁻¹⁴
Aggression Behaviors	66.44 ± 3.81	61.34 ± 7.95	52.09 ± 3.69	K-W = 84.051 (2df)	U = 24.000	U = 134.500	U = 433.500
				p = 5.60 × 10⁻¹⁹	p = 1.02 × 10⁻¹¹	p = .002	p = 5.62 × 10⁻¹⁴

Note. A-A-A = Anxiety/Depression, Aggression Behaviors, Attention Problems; CBCL = Child Behavior Checklist; TRF = Teacher Report Form; DESR = deficient emotional self-regulation; CON = controls; ROC = receiver operating characteristic; K-W = Kruskal–Wallis. Sample sizes, K-W non-parametric ANOVAs, and pairwise contrasts by Mann–Whitney U Tests are reported.

Comorbidity and Level of Interpersonal Functioning

Prior to assessing the overall incidence of comorbidity and the level of interpersonal functioning in ADHD preschoolers with DESR, we sought an initial validation of the preschool-specific CBCL thresholds identified here by ROC analysis. To this aim, PAPA scores were contrasted between (a) 16 ADHD children satisfying preschool but not school-age thresholds, (b) 17 ADHD children satisfying also school-age thresholds for DESR, (c) 53 ADHD preschoolers without DESR, and (d) 104 controls. This procedure made use of CBCL scores only, because of the more limited statistical power of TRF analyses due to the smaller sample size. These contrasts provided the following results: (a) preschool children diagnosed with DESR according to preschool and school-age thresholds display equally enhanced rates of psychiatric comorbidity for ODD and for anxiety disorders (both p < .01 compared with ADHD without DESR; see Figure 1A), (b) impairment in social functioning is not significantly different between the two DESR groups, which both differ largely from controls and, at least in some items, from ADHD preschoolers without DESR (Figure 1B), (c) the severity of ADHD among preschoolers with DESR according to the new thresholds is close, though somewhat lower, compared with preschoolers satisfying the more stringent school-age criteria, while both appear more severely affected as compared with ADHD preschoolers without DESR (Supplementary Figure 2). Collectively, these results support the proposal that ADHD preschoolers who fulfill DESR criteria set forth by ROC analyses indeed represent “true positives.”

Figure 1.

DESR and psychopathology recorded using the PAPA interview: (A) incidence of comorbidity and (B) intensity of social problems.

Finally, adopting the proposed CBCL thresholds to identify the presence of DESR in preschool children with ADHD, the overall incidence of ODD and anxiety disorders was 75.8% (25/33) and 54.5% (18/33) in ADHD children with DESR, as compared with 37.7% (20/53) and 15.1% (8/53) in ADHD children without DESR (χ² = 11.79 and 15.01, respectively; both p < .001). ADHD children with DESR were significantly more impaired in social functioning compared with ADHD children without DESR (n = 33 and 53, respectively; F = 20.244, 1 df, p = 2.23 × 10⁻⁵). ADHD symptoms were more severe if comorbid with DESR, as compared with ADHD in the absence of DESR (n = 33 and 53, respectively; F = 5.588, 1 df, p < .05). Therefore, also among preschoolers, as in school-age children diagnosed with ADHD, the presence of DESR is associated with a more severe clinical presentation.

Discussion

To our knowledge, the present study, for the first time, addresses the early detection of DESR among preschoolers with ADHD and explores its influence on comorbidities, level of impairment in interpersonal functioning, and severity of ADHD symptoms. Our data indicate that DESR can be detected using the CBCL and TRF A-A-A dimensions, although differences between preschoolers and school-age ADHD children require small but clinically meaningful adjustments in threshold scores. A 3-point difference in CBCL threshold score may not appear sizable at first. Yet, by applying the slightly lower CBCL cutoff identified here by ROC analyses (A/D ≥ 59, AP ≥ 60, AB ≥ 58, with aggregate score >178), instead of the cutoff used in school-age children (>60 in each dimension and aggregate score >180), we define a consistent subgroup of ADHD preschoolers also affected with DESR, as supported by comorbidity rates, level of impairment in interpersonal functioning, and severity of ADHD symptoms (Figure 1A and 1B, Supplementary Figure 2). Furthermore, applying these age-specific CBCL thresholds, DESR prevalence rates move from 19.8% to 38.4%, much closer to those 44% previously described in school-age children with ADHD (Biederman et al., 2012; Spencer et al., 2011). It thus seems highly plausible that the preschoolers with ADHD satisfying the cutoff scores identified by ROC analyses in the present study do indeed have a DESR and would represent false negatives, if school-age thresholds were applied to this age group.

The need to apply lower threshold scores to detect clinically meaningful behavioral conditions among preschoolers, as compared with school-age children, is not at all unique to DESR in ADHD and may stem from at least two distinct factors. On one hand, the range of normal emotional regulation may be wider in early than in late childhood, whereas the boundaries between typical and atypical behavior may be more ill defined, especially for DESR. On the other hand, both parents and teachers have been shown to apply less stringent criteria in their evaluation of this age group, as compared with older children, in reference to depressed symptoms (Luby, Belden, Pautsch, Si, & Spitznagel, 2009) and disruptive behaviors (Studts & van Zyl, 2013). This tendency by parents to downplay atypical affective and externalizing reactions in preschoolers not only supports our general conclusions but also may seemingly explain why our ROC analyses interestingly identify the need to apply lower thresholds for the A/D and Aggressive Behaviors CBCL dimensions, but not for the AP dimension where the threshold set at >60 in school-age children retains its validity also among preschoolers (Table 2). These two factors, that is, broader normality range and more lenient criteria applied by parents and teachers, likely also explain the slightly lower, and not higher, incidence of DESR, we record in our preschoolers as compared with published rates in school-age children (Biederman et al., 2012; Spencer et al., 2011). Furthermore, teacher and parent scores do not necessarily overlap, as teachers tend to provide higher scores than parents. This is due to the well-known combination of two issues, namely, (a) a child’s behavior is setting specific and ADHD children show worse behavior when they are involved in group situations with no or minimal adult supervision, as compared with one-to-one interaction with a parent or caregiver, and (b) parents and teachers differ in sensitivity, with the former more keen on reporting internalizing behaviors, such as depression and anxiety, and the latter more sensitive to externalizing disruptive behaviors (Kolko & Kazdin, 1993). Despite these predictable differences, CBCL parent and TRF teacher scores are significantly correlated and in good agreement, both supporting the need to apply lower threshold scores to reliably identify DESR among preschoolers as compared with school-age children.

The presence of DESR among preschoolers with ADHD is associated here with considerable levels of behavioral comorbidity and social dysfunction, analogous to those previously recorded in school-age ADHD children with DESR (Biederman et al., 2012; Spencer et al., 2011). DESR is indeed a component of several developmental psychopathologies, with increasing evidence for prediction of ADHD, disruptive, and mood syndromes (Burke, Hipwell, & Loeber, 2010; Stringaris, Cohen, Pine, & Leibenluft, 2009; Stringaris & Goodman, 2009). ADHD preschoolers with DESR show, here, higher rates of psychiatric comorbidity, especially disruptive disorders and anxiety (Figure 1A). These results are in line with recent evidence that DESR in early childhood is a predictive factor for both preschool ADHD and for comorbid anxiety (Overgaard et al., 2014). Moreover, preschoolers with DESR show significantly higher impairment also in several CBCL and TRF dimensions other than those included in the A-A-A profile, especially withdrawal and emotional reactivity. This most likely reflects abnormal emotional reactivity, which has been identified in older ADHD children as one of the hallmarks of DESR (Purper-Ouakil & Franc, 2011). High emotional reactivity was found associated in ADHD with co-occurring aggressive symptoms (Graziano, McNamara, Geffken, & Reid, 2013), parasympathetic dysregulation in emotional arousal (Musser et al., 2011), and explosive reactions to unpleasant emotional stimuli (Pishyareh et al., 2012). Abnormal emotional reactivity also underlies the broad-based and significant impairment recorded in social function among our preschoolers with a CBCL profile positive for DESR (Figure 1B). In our study, preschoolers with ADHD and DESR appear significantly more impaired in their relationships with peers, parents, and teachers, albeit to a slightly different extent depending on whether they satisfy the more stringent school-age thresholds or the lower age-specific thresholds. The present data are consistent with the suggestion that DESR might account for a significant portion of social dysfunction in ADHD (Barkley & Fischer, 2010).

The interpretation of the CBCL-PBD profile, that is, CBCL and/or TRF aggregate scores ≥210, represents an additional challenge. In older children, this CBCL profile has been associated with juvenile bipolar disorder (Ayer et al., 2009; Biederman et al., 2009; Faraone, Althoff, Hudziak, Monuteaux, & Biederman, 2005; Mick, Biederman, Pandina, & Faraone, 2003; Papolos, Hennen, Cockerham, Thode, & Youngstrom, 2006). Some evidence suggests that early onset bipolar disorder may be identifiable also at preschool age (Danielyan, Pathak, Kowatch, Arszman, & Johns, 2007; Dilsaver & Akiskal, 2004; Ferreira Maia, Boarati, Kleinman, & Fu-I, 2007; Luby & Belden, 2006; Luby, Tandon, & Belden, 2009; Scheffer & Niskala Apps, 2004). However, studies largely differ in methodology, sample size, age range, and psychiatric assessment, raising major controversy in the field. Volk and Todd (2007) found the CBCL-PBD profile to be related not so much with bipolar disorder, but rather with ADHD, ODD, conduct disorder, and enhanced risk for suicidal behavior, highly compatible with excess impulsivity. Other studies have described the CBCL-PBD profile to predict broad psychiatric impairment rather than specific DSM diagnoses (Hudziak et al., 2005; Meyer et al., 2009; for a systematic review, see Ratheesh et al., 2015). In our sample, full concordance between CBCL and TRF scores was found for five out of six ADHD children displaying this extreme profile. Moreover, among another 10 ADHD children classified with this extreme profile by teachers, eight satisfied both preschool and school-age thresholds for DESR according to their parents. Although it is possible that these preschoolers suffer from very early onset bipolar disorder, conceivably, they could rather present with an extremely severe form of ADHD with DESR. Nonetheless, based on previous studies of school-age children with ADHD (Biederman et al., 2012; Spencer et al., 2011), this extreme profile has been maintained here as an exclusion criterion.

The present study displays some limitations, which must be duly acknowledged. First, reliably diagnosing ADHD in small children can be challenging. Here, we have adopted a stringent recruitment strategy, requiring consistent diagnostic positivity from multiple sources of information. Although minimizing the incidence of false positives, there is no guarantee this approach has been completely successful at achieving diagnostic validity in every case. Second, recruiting large samples of preschoolers with ADHD and devoid of other diagnoses, such as autism, is indeed cumbersome. Consequently, our sample size, though substantial within this age group, is not sufficient to randomly split cases into two separate subsamples (i.e., an experimental and a replica sample), allowing an internal replication of the proposed threshold scores. External replications in independent samples will thus be necessary to conclusively confirm the validity of the present results. Third, no longitudinal data allow at this time to draw firm conclusions on the developmental trajectories of these children. The prospective follow up of the present cohort will thus be crucial to ultimately validate the proposed thresholds. Fourth, TRF threshold scores should be viewed with caution, due to their smaller sample size, whereas CBCL threshold scores are endowed with greater reliability. In addition, statistical analyses on comorbidity and social functioning, here limited to CBCL scores, will have to involve TRF scores once a larger sample size becomes available. Finally, we have strictly applied the three-dimensional approach of the A-A-A profile previously proposed for school-age children (Biederman et al., 2012; Spencer et al., 2011). Other approaches using pre-selected CBCL items drawn from multiple additional dimensions have been shown to possess greater predictive power in reference to a bipolar disorder outcome compared with the CBCL-PBD profile (Papachristou et al., 2013). It will be interesting to assess prospectively whether, and to what extent, other CBCL and TRF dimensions, such as withdrawal and emotional reactivity, can contribute to enhance sensitivity and specificity in the detection of DESR among preschoolers, when applied in conjunction with the A-A-A dimensions.

Conclusion

This study provides threshold scores able to maximize the identification of DESR in preschoolers with ADHD using the A-A-A profile of the CBCL (A/D ≥ 59, AP ≥ 60, AB ≥ 58, aggregate score ≥ 178) and TRF (A/D ≥ 59, AP ≥ 60, AB ≥ 60, aggregate score ≥ 180). The psychopathological and functional profile of ADHD preschoolers with DESR, identified by applying this methodology, is highly consistent with the clinical concept of DESR, as previously developed in school-age children. Also in early infancy, DESR is significantly associated with greater psychiatric comorbidity (ODD and anxiety disorders) and social dysfunction in ADHD. We hypothesize that the presence of DESR among preschoolers may represent a continuum, ranging from mild forms, yet clinically relevant and possibly better detected by the threshold scores described in this study, to moderate forms able to satisfy CBCL criteria already set forth among school-age children, to severe forms reaching aggregate scores >210. It will be very interesting to longitudinally explore this hypothesis and the influence of these different CBCL-defined severity levels on long-term outcome and treatment response, as well as to better understand their clinical, developmental, and pathophysiological characteristics.

Footnotes

Acknowledgements

The authors gratefully acknowledge all the families who participated in this study and Martina Giallonardo for helping with the graphics.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material is available for this article online.

Author Biographies

Maria Grazia Melegari operates as a child and adolescent psychiatrist on psychiatric and neuro-developmental disorders of early childhood in an NHS Child and Adolescence NeuroPsychiatry Outpatient Center in Rome, Italy. In addition, she coordinates the local ADHD Reference Center where both clinical and research activity is carried out. She is coauthor in 35 national and international published studies.

Roberto Sacco works as a clinical psychologist, mainly on developmental neuropsychology and psychodiagnostics. His primary interests are clinical and neurodevelopmental neuroscience, neurocognitive rehabilitation, genetic epidemiology and biostatistics. He is co-author on 28 international publications listed in PubMed.

Barbara Manzi works as a child and adolescent psychiatrist in Child and Adolescence NeuroPsychiatry outpatient Center of ASL RM/A, in Rome (Italy). In the last 10 years she also performed research in autism spectrum disorder, partecipating as coauthor in 18 papers published in PubMed. She is presently interested in early diagnosis of psychiatric disorders.

Elena Vittori works as child and adolescent psychiatrist at CRC Balbuzie, an important rehabilitation center in Rome (Italy). She performs diagnostic evaluations and treatment interventions for children with neurodevelopmental disorders. Her primary interest is Attention Deficit and Hyperactivity Disorder.

Antonio M. Persico is full professor of Child & Adolescent NeuroPsychiatry at University of Messina, Italy and Scientific Director of the Mafalda Luce Center for Pervasive Developmental Disorders in Milan, Italy. He has a long-standing interest in the genetic and neurobiological underpinnings of neurodevelopmental disorders, including autism and ADHD, as well as in biomarkers for disease subtyping and early diagnosis. He is co-author in 111 papers listed in PubMed.

References

Abikoff

Courtney

Pelham

W. E.

Jr. Koplewicz

H. S.

(1993). Teachers’ ratings of disruptive behaviors: The influence of halo effects. Journal of Abnormal Child Psychology, 21, 519-533.

Achenbach

T. M.

McConaughy

S. H.

Howell

C. T.

(1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213-232.

Achenbach

T. M.

Rescorla

L. A.

(2000). Manual for the ASEBA preschool forms & profiles. Burlington: University of Vermont, Research Center for Children, Youth & Families.

Angold

Costello

E. J.

Erkanli

(1999). Comorbidity. Journal of Child Psychology and Psychiatry, 40, 57-87.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

Ayer

Althoff

Ivanova

Rettew

Waxler

Sulman

Hudziak

(2009). Child Behavior Checklist Juvenile Bipolar Disorder (CBCL-JBD) and CBCL Posttraumatic Stress Problems (CBCL-PTSP) scales are measures of a single dysregulatory syndrome. Journal of Child Psychology and Psychiatry, 50, 1291-1300.

Barkley

R. A.

(1997). Attention-deficit/hyperactivity disorder, self-regulation, and time: Toward a more comprehensive theory. Journal of Developmental & Behavioral Pediatrics, 18, 271-279.

Barkley

R. A.

Fischer

(2010). The unique contribution of emotional impulsiveness to impairment in major life activities in hyperactive children as adults. Journal of the American Academy of Child & Adolescent Psychiatry, 49, 503-513.

10.

Belden

A. C.

Thomson

N. R.

Luby

(2008). Temper tantrums in healthy versus depressed and disruptive preschoolers: Defining tantrum behaviors associated with clinical problems. Journal of Pediatrics, 152, 117-122.

11.

Biederman

Petty

C. R.

Monuteaux

M. C.

Evans

Parcell

Faraone

S. V.

Wozniak

(2009). The Child Behavior Checklist Paediatric Bipolar Disorder profile predicts a subsequent diagnosis of bipolar disorder and associated impairments in ADHD youth growing up: A longitudinal analysis. Journal of Clinical Psychiatry, 70, 732-740.

12.

Biederman

Spencer

T. J.

Petty

Hyder

L. L.

O’Connor

K. B.

Surman

C. B.

Faraone

S. V.

(2012). Longitudinal course of deficient emotional self-regulation CBCL profile in youth with ADHD: Prospective controlled study. Neuropsychiatric Disease and Treatment, 8, 267-276.

13.

Biederman

Wozniak

Kiely

Ablon

Faraone

Mick

. . . Kraus

(1995). CBCL clinical scales discriminate prepubertal children with structured interview-derived diagnosis of mania from those with ADHD. Journal of the American Academy of Child & Adolescent Psychiatry, 34, 464-471.

14.

Bufferd

S. J.

Dougherty

L. R.

Carlson

G. A.

Rose

Klein

D. N.

(2012). Psychiatric disorders in preschoolers: Continuity from ages 3 to 6. American Journal of Psychiatry, 169, 1157-1164.

15.

Burke

J. D.

Hipwell

A. E.

Loeber

(2010). Dimensions of oppositional defiant disorder as predictors of depression and conduct disorder in preadolescent girls. Journal of the American Academy of Child & Adolescent Psychiatry, 49, 484-492.

16.

Campbell

S. B.

Pierce

E. W.

March

C. L.

Ewing

L. J.

Szumowski

E. K.

(1994). Hard-to-manage preschool boys: Symptomatic behavior across contexts and time. Child Development, 65, 836-851.

17.

Chang

Schwartz

Dodge

K. A.

McBride-Chang

(2003). Harsh parenting in relation to child emotion regulation and aggression. Journal of Family Psychology, 17, 598-606.

18.

Danielyan

Pathak

Kowatch

R. A.

Arszman

S. P.

Johns

E. S.

(2007). Clinical characteristics of bipolar disorder in very young children. Journal of Affective Disorders, 97, 51-59.

19.

Dilsaver

S. C.

Akiskal

H. S.

(2004). Preschool-onset mania: Incidence, phenomenology and family history. Journal of Affective Disorders, 82, S35-S43.

20.

Donfrancesco

Marano

Calderoni

Mugnaini

Thomas

Di Trani

. . . Vitiello

(2014). Prevalence of severe ADHD: An epidemiological study in the Italian regions of Tuscany and Latium. Epidemiology and Psychiatric Sciences, 15, 1-9.

21.

DuPaul

G. J.

Power

T. J.

Anastopoulos

A. D.

Reid

(1998). ADHD Rating Scale-IV: Checklists, norms, and clinical interpretation. New York, NY: Guilford Press.

22.

Egger

H. L.

Angold

(2006). Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiology. Journal of Child Psychology and Psychiatry, 47, 313-337.

23.

Egger

H. L.

Erkanli

Keeler

Potts

Walter

B. K.

Angold

(2006). Test-retest reliability of the Preschool Age Psychiatric Assessment (PAPA). Journal of the American Academy of Child & Adolescent Psychiatry, 45, 538-549.

24.

Eisenberg

Cumberland

Spinrad

T. L.

Fabes

R. A.

Shepard

S. A.

Reiser

. . . Guthrie

I. K.

(2001). The relations of regulation and emotionality to children’s externalizing and internalizing problem behavior. Child Development, 72, 1112-1134.

25.

Faraone

S. V.

Althoff

R. R.

Hudziak

J. J.

Monuteaux

Biederman

(2005). The CBCL predicts DSM bipolar disorder in children: A receiver operating characteristic curve analysis. Bipolar Disorders, 7, 518-524.

26.

Ferreira Maia

A. P.

Boarati

M. A.

Kleinman

Fu-I

(2007). Preschool bipolar disorder: Brazilian children case reports. Journal of Affective Disorders, 104, 237-243.

27.

Frigerio

(2000). Questionario sul comportamento del bambino [Questionnaire on child behaviour]. Bosisio Parini, Italy: Istituto Scientifico “E. Medea”, Assoc. “La Sacra Famiglia.”

28.

Graziano

P. A.

McNamara

J. P.

Geffken

G. R.

Reid

A. M.

(2013). Differentiating co-occurring behavior problems in children with ADHD: Patterns of emotional reactivity and executive functioning. Journal of Attention Disorders, 17, 249-260.

29.

Hardy

K. K.

Kollins

S. H.

Murray

D. W.

Riddle

M. A.

Greenhill

Cunningham

. . . Chuang

S. Z.

(2007). Factor structure of parent- and teacher-rated attention-deficit/hyperactivity disorder symptoms in the Preschoolers With Attention-Deficit/Hyperactivity Disorder Treatment Study (PATS). Journal of Child and Adolescent Psychopharmacology, 17, 621-634.

30.

Healey

Miller

Castelli

Marks

Halperin

(2008). The impact of impairment criteria on rates of ADHD diagnoses in preschoolers. Journal of Abnormal Child Psychology, 36, 771-778.

31.

Hudziak

J. J.

Althoff

R. R.

Derks

E. M.

Faraone

S. V.

Boomsma

D. I.

(2005). Prevalence and genetic architecture of Child Behavior Checklist-juvenile bipolar disorder. Biological Psychiatry, 58, 562-568.

32.

Kolko

D. J.

Kazdin

A. E.

(1993). Emotional/behavioral problems in clinic and nonclinic children: Correspondence among child, parent, and teacher reports. Journal of Child Psychology and Psychiatry, 34, 991-1006.

33.

Lahey

B. B.

Pelham

W. E.

Loney

Kipp

Ehrhardt

Lee

S. S.

. . . Massetti

(2004). Three-year predictive validity of DSM-IV attention deficit hyperactivity disorder in children diagnosed at 4-6 years of age. American Journal of Psychiatry, 161, 2014-2020.

34.

Lavigne

J. V.

Lebailly

S. A.

Hopkins

Gouze

K. R.

Binns

H. J.

(2009). The prevalence of ADHD, ODD, depression, and anxiety in a community sample of 4-year-olds. Journal of Clinical Child and Adolescent Psychology, 38, 315-328.

35.

Lee

Lahey

Owens

Hinshaw

(2008). Few preschool boys and girls with ADHD are well-adjusted during adolescence. Journal of Abnormal Child Psychology, 36, 373-383.

36.

Leiter

R. G.

(1979). Instruction manual for the Leiter International Performance Scale. Wood Dale, IL: Stoelting.

37.

Luby

J. L.

Belden

(2006). Defining and validating bipolar disorder in the preschool period. Development and Psychopathology, 18, 971-988.

38.

Luby

J. L.

Belden

A. C.

Pautsch

Spitznagel

(2009). The clinical significance of preschool depression: Impairment in functioning and clinical markers of the disorder. Journal of Affective Disorders, 112, 111-119.

39.

Luby

J. L.

Tandon

Belden

(2009). Preschool bipolar disorder. Child & Adolescent Psychiatric Clinics of North America, 18, 391-403.

40.

Marakovitz

S. E.

Campbell

S. B.

(1998). Inattention, impulsivity, and hyperactivity from preschool to school age: Performance of hard-to-manage boys on laboratory measures. Journal of Child Psychology and Psychiatry, 39, 841-851.

41.

Marzocchi

G. M.

Cornoldi

(2000). Una scala di facile uso per la rilevazione dei comportamenti problematici dei bambini con Deficit di Attenzione e Iperattività [An easy-to-use scale to record problem behaviors in children with attention deficit and hyperactivity]. Psicologia Clinica dello Sviluppo, 4, 43-63.

42.

Mattison

R. E.

Spitznagel

E. L.

(1999). Long-term stability of Child Behavior Checklist profile types in a child psychiatric clinic population. Journal of the American Academy of Child & Adolescent Psychiatry, 38, 700-707.

43.

Meyer

S. E.

Carlson

G. A.

Youngstrom

Ronsaville

D. S.

Martinez

P. E.

Gold

P. W.

. . . Radke-Yarrow

(2009). Long-term outcomes of youth who manifested the CBCL-pediatric bipolar disorder phenotype during childhood and/or adolescence. Journal of Affective Disorders, 113, 227-235.

44.

Mick

Biederman

Pandina

Faraone

S. V.

(2003). A preliminary meta-analysis of the Child Behavior Checklist in pediatric bipolar disorder. Biological Psychiatry, 53, 1021-1027.

45.

Musser

E. D.

Backs

R. W.

Schmitt

C. F.

Ablow

J. C.

Measelle

J. R.

Nigg

J. T.

(2011). Emotion regulation via the autonomic nervous system in children with attention-deficit/hyperactivity disorder (ADHD). Journal of Abnormal Child Psychology, 39, 841-852.

46.

Overgaard

K. R.

Aase

Torgersen

Reichborn-Kjennerud

Oerbeck

Myhre

Zeiner

(2014). Continuity in features of anxiety and attention deficit/hyperactivity disorder in young preschool children. European Child & Adolescent Psychiatry, 23, 743-752.

47.

Papachristou

Ormel

Oldehinkel

A. J.

Kyriakopoulos

Reinares

Reichenberg

Frangou

(2013). Child Behavior Checklist-Mania Scale (CBCL-MS): Development and evaluation of a population-based screening scale for bipolar disorder. PLoS ONE, 8, e69459.

48.

Papolos

Hennen

Cockerham

M. S.

Thode

H. C.

Jr. Youngstrom

E. A.

(2006). The Child Bipolar Questionnaire: A dimensional approach to screening for pediatric bipolar disorder. Journal of Affective Disorders, 95, 149-158.

49.

Pishyareh

Tehrani-Doost

Mahmoodi-Gharaie

Khorrami

Joudi

Ahmadi

(2012). Attentional bias towards emotional scenes in boys with attention deficit hyperactivity disorder. Iranian Journal of Psychiatry, 7, 93-96.

50.

Posner

Melvin

G. A.

Murray

D. W.

Gugga

S. S.

Fisher

Skrobala

. . . Greenhill

L. L.

(2007). Clinical presentation of attention-deficit/hyperactivity disorder in preschool children: The Preschoolers With Attention-Deficit/Hyperactivity Treatment Study (PATS). Journal of Child and Adolescent Psychopharmacology, 17, 547-562.

51.

Purper-Ouakil

Franc

(2011). Emotional dysfunctions in attention deficit hyperactivity disorder. Archives de Pédiatrie, 18, 679-685.

52.

Ratheesh

Berk

Davey

C. G.

McGorry

P. D.

Cotton

S. M.

(2015). Instruments that prospectively predict bipolar disorder: A systematic review. Journal of Affective Disorders, 179, 65-73.

53.

Scheffer

R. E.

Niskala Apps

J. A.

(2004). The diagnosis of preschool bipolar disorder presenting with mania: Open pharmacological treatment. Journal of Affective Disorders, 82, S25-S34.

54.

Sonuga-Barke

Stevenson

Thompson

Viney

(1997). The structure of British pre-school and children’s behaviour problems. Psychological Medicine, 27, 909-918.

55.

Spencer

T. J.

Faraone

S. V.

Surman

C. B.

Petty

Clarke

Batchelder

. . . Biederman

(2011). Toward defining deficient emotional self-regulation in children with attention-deficit/hyperactivity disorder using the Child Behavior Checklist: A controlled study. Postgraduate Medicine, 123, 50-59.

56.

Sterba

Egger

H. L.

Angold

(2007). Diagnostic specificity and nonspecificity in the dimensions of preschool psychopathology. Journal of Child Psychology and Psychiatry, 48, 1005-1013.

57.

Stringaris

Cohen

Pine

Leibenluft

(2009). Adult outcomes of youth irritability: A 20-year prospective community-based study. American Journal of Psychiatry, 166, 1048-1054.

58.

Stringaris

Goodman

(2009). Longitudinal outcome of youth oppositionality: Irritable, headstrong, and hurtful behaviors have distinctive predictions. Journal of the American Academy of Child & Adolescent Psychiatry, 48, 404-412.

59.

Studts

C. R.

van Zyl

M. A.

(2013). Identification of developmentally appropriate screening items for disruptive behavior problems in preschoolers. Journal of Abnormal Child Psychology, 41, 851-863.

60.

Volk

H. E.

Todd

R. D.

(2007). Does the Child Behavior Checklist juvenile bipolar disorder phenotype identify bipolar disorder? Biological Psychiatry, 62, 115-120.

61.

Wichstrøm

Berg-Nielsen

T. S.

Angold

Egger

H. L.

Solheim

Sveen

T. H.

(2011). Prevalence of psychiatric disorders in preschoolers. Journal of Child Psychology and Psychiatry, 53, 695-705.

62.

Wilens

T. E.

Biederman

Brown

Tanguay

Monuteaux

M. C.

Blake

Spencer

T. J.

(2002). Psychiatric comorbidity and functioning in clinically referred preschool children and school-age youths with ADHD. Journal of the American Academy of Child & Adolescent Psychiatry, 41, 262-268.