Congruent Validity of the Strengths and Difficulties Questionnaire to Screen for Comorbidities in Children With ADHD

Abstract

Objective: This study aimed to determine whether the Strengths and Difficulties Questionnaire (SDQ) is an effective screening tool for identifying comorbid mental health difficulties in children with ADHD. Method: Parents of children with ADHD (5-13 years) completed the SDQ and the Anxiety Disorders Interview Schedule for DSM-IV–Parent Version (ADIS-C/P-IV). Data from both the measures were compared to determine congruent validity. Results: Analyses revealed that significant relationships exist between the SDQ total score and the total number of comorbidities on the ADIS-C/P-IV. The SDQ emotional problem and conduct problem scales were significantly related to internalizing and externalizing comorbidities on the ADIS-C/P-IV, respectively. Conclusion: While significant relationships were found between the SDQ and ADIS-C/P-IV across various domains, this relationship was stronger for externalizing comorbidities. Additional screening questions are required to effectively screen for less common internalizing comorbidities in children with ADHD, for example, major depression and panic disorder.

Keywords

ADHD comorbidity assessment children

ADHD is a common childhood disorder affecting approximately 5% of children worldwide (Polanczyk, Silva de Lima, Horta, Biederman, & Rohde, 2007). Comorbidities occur commonly in children with ADHD (Spencer, Biederman, & Wilens, 1999). A large population-based study found that approximately 67% of children with ADHD present with one or more other psychological or neuro-developmental comorbidities compared with 11% of children without ADHD (Larson, Russ, Kahn, & Halfon, 2011). Of children with ADHD, 33% had a single comorbidity, 16% had two, and 18% had three or more (Larson et al., 2011). This suggests that instances of children suffering from ADHD alone are the exception not the rule (Jensen et al., 2001).

Multiple studies have reported that internalizing and externalizing conditions occur in children with ADHD at a much higher rate than the general population. (Biederman et al., 1993; Biederman, Newcorn, & Sprich, 1991; Wilens, Biederman, & Spencer, 2002).

Prevalence of Internalizing and Externalizing Comorbidities in Children With ADHD

Internalizing comorbidities are conceptualized by the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) as including anxiety (e.g., Separation Anxiety Disorder [SAD] and Generalized Anxiety Disorder [GAD]) and mood disorders (e.g., Major Depressive Disorder [MDD] and dysthymia). Studies examining the prevalence of internalizing comorbidities in children with ADHD estimate that anxiety can occur in up to 20% to 25% of cases (Jarrett & Ollendick, 2008) and mood disorders occur in 15% to 20% (Goldman, Genel, Bezman, & Slanetz, 1998; Jensen, Martin, & Cantwell, 1997). Larson et al. (2011) found that 18% of children with ADHD had an anxiety disorder versus 2% of children without ADHD and 14% of children with ADHD had a depressive disorder versus 1% of children without ADHD.

Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD) are two DSM-IV disorders that can be conceptualized as externalizing comorbidities. The prevalence of ODD among children with ADHD ranges from 40% to 80% (Bartholomew & Owens, 2006; Goldman et al., 1998; Jensen et al., 1997). Larson et al. (2011) found the prevalence rates of CD to be present in 27% of children with ADHD compared with 2% of children without ADHD.

The Impact of Mental Health Comorbidities on Child Functioning

Mental health comorbidities greatly influence the presentation, diagnosis, and prognosis of children with ADHD as comorbidities increase the overall level of disease burden (Kunwar, Dewan, & Faraone, 2007; The MTA Cooperative Group, 1999). Comorbid disorders have an accumulative effect on dysfunction (Gillberg et al., 2004). This in turn, leads to poorer outcomes for children and families with ADHD and mental health comorbidities, relative to children with ADHD alone (Gillberg et al., 2004; Jensen et al., 2001; Jensen et al., 1997).

Numerous studies have examined the impact of internalizing or externalizing comorbidities on the functioning of children with ADHD. Comorbid anxiety, for example, has been associated with greater impairments in social functioning (e.g., poorer social skills, lower peer acceptance, greater peer neglect, and more solitary play; Ransone, 2009). Children with ADHD and comorbid anxiety have more severe anxiety-based symptoms and overall impairment than children with either anxiety or ADHD alone (Bowen, Chavira, Bailey, Stein, & Stein, 2008). In addition, depressive disorders in children with ADHD are often recurring and quite debilitating (Biederman, Mick, & Faraone, 1998). Furthermore, adolescents with ADHD and comorbid depressive disorders have an increased risk for suicidal behavior compared with those with only a depressive disorder (Biederman et al., 1991; Brent et al., 1988).

Various studies have found that children with ADHD and comorbid ODD or CD engage in more aggressive and delinquent behaviors. These children are also at risk of academic underachievement, substance abuse, lower self-esteem, and social maladjustment (Barkley, Fischer, Edelbrock, & Smallish, 1990; Bukstein, 2000; Faraone, Biederman, Keenan, & Tsuang, 1991; Kuhne, Schachar, & Tannock, 1997; Waschbusch et al., 2002). Similarly, Murphy, Barkley, & Bush (2002) noted that children with ADHD and comorbid ODD or CD are also more likely to be violent and antagonistic in their contact with others.

Assessment of Comorbidities in Children With ADHD

Given the high rates of comorbidity in children with ADHD, it is essential that these are effectively identified to direct appropriate treatments. There is some evidence that comorbidities may go undetected and therefore untreated in children with ADHD. A recent audit of 1,528 pediatric consultations for ADHD found that 8% were reported by pediatricians to have an anxiety disorder and 15%, to have ODD; these rates are much lower than have been identified when assessed via structured diagnostic interviews (Efron, Davies, & Sciberras, 2013). Screening for coexisting comorbidities in children with ADHD is necessary to ensure appropriate treatment plans are developed. This is especially important given that comorbidities have adverse impacts on the long-term outcomes for children with ADHD.

Currently, there are a range of clinical tools used in the assessment of ADHD and mental health comorbidities, including structured and semistructured interviews, behavior checklists, rating scales, and questionnaires (Biederman et al., 1991). Semistructured diagnostic interviews such as the Anxiety Disorders Interview Schedule (ADIS-C/P-IV) are often used for their well-established validity and alignment with DSM-IV categories. However, such measures can be very time-consuming to administer. Heisel, Duberstein, Lyness, and Feldman (2010) argued that for any psychological screening tests to be viable in healthcare settings, they must be valid, brief, and easy to administer and score. This serves multiple purposes including (a) identifying potential psychological conditions, (b) reducing the costs associated with lengthy assessments, and (c) saving clinicians’ time. Despite the diagnostic accuracy of more well-established and comprehensive diagnostic measures in primary care practice (Spitzer et al., 1994), they are often time-consuming; thus they do not enjoy widespread use among physicians (Carlat, 1998).

One efficient and commonly used option is to screen for comorbidities using a brief measure (e.g., the Strengths and Difficulties Questionnaire [SDQ]) and to then conduct a more detailed assessment if indicated by the screening measure. It is important to understand how the results from brief screening measures relate to findings from more detailed assessments of comorbidities, including structured diagnostic interviews, to ensure the validity of this clinical approach for children with ADHD.

Strengths and Difficulties Questionnaire

The SDQ is a brief, 25-item validated parent-rated screening questionnaire divided into five subscales of behavioral and emotional problems for children aged 4 to 16 (Goodman, 1997). The majority of research relating to the SDQ concerns itself primarily with the validity and reliability of the measure, thereby investigating internal factorial analysis or the instrument’s transferability to reproduce results in various national population groups. Using a large community sample of young Australian children, Hawes and Dadds (2004) found moderate to strong internal reliability for the SDQ subscales, utilizing the parent-rated form, and good correspondence with comorbid internalizing and externalizing disorders measured by diagnostic interview. For example, children scoring in the clinical range on the emotional problem subscale of the SDQ were 12 times more likely to meet DSM-IV criteria for an internalizing disorder. Similarly, children scoring in the clinical range on the conduct problem subscale were 31 times more likely to meet DSM-IV criteria for an externalizing behavior disorder. They also found strong parent-rated test-retest correlations ranging between r = .61-.77 after a 12-month period.

Previous studies have explored potential links between the SDQ and other established measures. Hawes and Dadds’s (2004) results indicated that the pattern of correlation between the SDQ subscales and the Diagnostic Interview Schedule for Children, Adolescents, and Parents (DISCAP) demonstrated sound external validity. In a British study comparing the parent-rated SDQ and the Child Behavior Checklist (CBCL), results indicated that there was a high correlation of the scores between the SDQ and the CBCL, with the SDQ at least as good at detecting internalizing and externalizing problems (Goodman & Scott, 1999). Klasen et al. (2000) conducted a similar study in Germany, producing findings that also indicated a high correlation between SDQ and CBCL scores as well as the Youth Self Report (YSR) with the ability to distinguish between clinical and community populations. The study used parent- and self-rated questionnaires, although the self-rated version was only available to their community sample. However, it is still unclear how well the SDQ screens for comorbid mental health difficulties in children with ADHD specifically.

Anxiety Disorders Interview Schedule

The ADIS-C/P-IV is a comprehensive, semistructured diagnostic interview that assesses for DSM-IV diagnoses in children, including internalizing and externalizing disorders (Lyneham & Rapee, 2005). It has been found to be a valid and reliable measure across several studies with good to excellent test–retest reliability (κ = 0.61-1) over an interval of 7 to 14 days (Silverman & Nelles, 1988; Silverman, Saavedra, & Pina, 2001).

Studies have also been conducted examining the ADIS-C/P-IV against the YSR and the CBCL. A study by Ferdinand (2008) examined the validity of the CBCL and YSR against DSM-IV diagnoses as determined by the ADIS-C/P-IV. The results indicated a moderate relationship existed between scores relating to anxieties and a strong relationship existed between scores relating to MDD and dysthymia for the YSR and CBCL.

The YSR and the CBCL are established, widely accepted screening tools, often used for identifying comorbidities. The relevance of considering these previous studies in reference to the current study lies in the fact that findings have indicated a correspondence to the SDQ and the ADIS-C/P-IV, albeit independently. These results bridge the proposed theoretical link between the SDQ and the ADIS-C/P-IV as no previous studies have directly examined the congruent validity of the SDQ and the ADIS-C/P-IV.

Aims and Hypotheses

Given the high rates of comorbidity in children with ADHD and the negative effects comorbidities have on the long-term outcomes for children with ADHD, it is imperative that comorbid conditions are readily identified and treated in clinical practice. The current study aims to examine how well the SDQ screens for internalizing and externalizing comorbidities in children with ADHD. This will be determined by examining the relationship between the SDQ and the ADIS-C/P-IV. The ADIS-C/P-IV was selected as a comparative measure because of its ability to determine DSM-IV comorbid diagnoses in children and its well-established validity and reliability. The SDQ has an administration time of approximately 5 minutes, making it a time efficient clinical screening tool.

The following hypotheses were postulated:

Hypothesis 1: Participants who have a higher SDQ total score will be more likely to exhibit a greater number of comorbidities, as identified by the ADIS-C/P-IV.

Hypothesis 2: Participants who are rated as having emotional problems, according to the SDQ, will be more likely to meet DSM-IV criteria for an internalizing comorbidity as identified by the ADIS-C/P-IV.

Hypothesis 3: Participants who are rated as having conduct problems, according to the SDQ, will be more likely to meet DSM-IV criteria for an externalizing comorbidity as identified by the ADIS-C/P-IV.

Method

Participants

The current study used a subset of data obtained through the “Sleeping Sound with ADHD” project (Sciberras et al., 2010). The study aimed to investigate the efficacy of a behavioral sleep intervention in treating sleep problems experienced by children with ADHD. Participants (N = 244) consisted of the families of children who had pediatrician-diagnosed ADHD (inclusive of inattentive, hyperactive-impulsive and combined subtypes) and parent-reported “moderate” or “severe” sleep problems (Sung, Hiscock, Sciberras, & Efron, 2008). The age of participants ranged from 5 to 13, with a mean age of 10.11 (SD = 1.97). Of the participants, 208 (85.2%) were male and 36 (14.8%) were female. The DSM-IV criteria for ADHD-Combined type were met by 174 (71.9%) participants, 62 (25.6%) met the DSM-IV criteria for ADHD-Inattentive type, and 6 (2.5%) met the DSM-IV criteria for ADHD-Hyperactive/Impulsive type. Most children (n = 211, 87.9%) were taking medication for the treatment of ADHD, including Ritalin/Ritalin LA 111 (45.5%), Concerta 83 (34%), Strattera 14 (5.7%), Catapres 15 (6.1%), and other medications 23 (9.4%).

Exclusion criteria for the Sleeping Sound with ADHD study included the following: (a) participants receiving additional professional help for their child’s sleep, apart from their pediatrician, such as a psychologist or a specialized sleep clinic; (b) children identified as having a serious medical condition or an intellectual disability (children with learning disabilities or mental health comorbidities were not excluded); and (c) children with suspected obstructive sleep apnea.

Measures

The ADHD Rating Scale IV (parent version; DuPaul, Power, Anastopoulos, & Reid, 1998) was used to confirm the child’s ADHD diagnosis according to DSM-IV criteria. This measure is an 18-item validated scale measuring the core symptoms of ADHD in children. The measure consists of two scales assessing inattention and hyperactivity/impulsiveness symptoms, with nine behaviors listed in each section. Parents rated each behavior, with its corresponding numerical value, as either never or rarely (0), sometimes (1), often (2), or very often (3). Positive ADHD diagnoses were determined by a frequency of six or more 2 to 3 value ratings in either domain. Furthermore, study-designed questions assessing the symptom duration, onset, and impairment were required to confirm the diagnosis of ADHD. This measure has good psychometric properties including adequate reliability and validity (Collett, Ohan, & Meyers, 2003).

The SDQ (Goodman, 1997) provides standard scores on five subscales: hyperactivity/inattention (α = .60), conduct problems (α = .72), emotional symptoms (α = .69), peer-relationship problems (α = .62), and pro-social behavior (α = .69); a total problems score (α = .74) is derived from the first four subscales. It has an average completion time of 5 minutes. The parent-reported SDQ versions for children aged 4 to 10 and 11 to 17 were used in this study. The participants rated each item on a 3-point scale including not true (0), somewhat true (1), and certainly true (2), with each answer receiving a 0 to 2 point value. Scores were classified as “normal” or “borderline/abnormal” for each participant. These parent-completed scores were normal (0-13) and borderline/abnormal (14-40) for total difficulties, normal (0-3) and borderline/abnormal (4-10) for emotional symptoms, and normal (0-2) and borderline/abnormal (3-10) for conduct problems. These calculations and categories were assigned using the SDQ scoring instrument (Goodman, 1997).

The ADIS-C/P-IV was used to assess internalizing/externalizing comorbidities and was administered over the telephone by a trained research assistant with the child’s parent, the average completion time being 45 minutes. The following comorbidities were assessed: SAD, GAD, social phobia, specific phobia, panic disorder (PD), obsessive compulsive disorder (OCD), posttraumatic stress disorder (PTSD), MDD, dysthymia, CD, and ODD. The measure presents each disorder in separate sections; these contain questions assessing whether the child meets diagnostic criteria for the disorder being assessed. Children can be classified as meeting criteria for a particular diagnostic category (yes/no) based on the responses parents gave to each question. To meet diagnostic criteria, children needed to meet the symptom threshold for diagnosis and parents needed to rate children as having an interference rating of 4 or more in daily functioning, as determined by the feelings thermometer, a 0- to 8-dimensional scale with a corresponding nominal rating, that is 0 (not at all), 2 (a little bit), 4 (some), 6 (a lot), and 8 (very, very much).

Lyneham and Rapee (2005) reported that the ADIS-C/P-IV is valid for telephone administration. They found a good to excellent range in the level of agreement between standard and telephone administration in determining the presence of individual anxiety (κ = .63-.86) and other disorders (κ = .79-.91). Following data collection, the interrater reliability of researchers who conducted the ADIS-C/P-IV was assessed to ensure consistency between interviewers and their coding of parents’ responses on the ADIS-C/P-IV. In the present study, interrater agreement was good for OCD (κ = .69; p < .001) and excellent for all other diagnoses examined (κ = .83-1.0; p < .001).

Procedure

Participants were recruited from 50 Victorian pediatricians whose patients met the study’s inclusion criteria outlined earlier. A study-designed letter was sent by the pediatrician to the child’s primary caregiver, asking them whether they wished to take part in the study. The pediatrician provided the research team with the telephone details of families who had not declined. The research team then telephoned families to see if they were willing to further assess eligibility and see whether they were interested in participating.

Eligible and interested families were then posted a participant information statement, a consent form, and the SDQ as part of a larger baseline survey. Upon receipt of parental consent and the baseline survey, parents were telephoned to complete the ADIS-C/P-IV. Ethics approval was obtained from The Royal Children’s Hospital (#30033) Human Research Ethics Committee.

Data Analysis

Statistical analyses were performed using the SPSS program version 20.0. Descriptive statistics were reported for each variable of interest. Frequencies and percentages were used for categorical variables, whereas means and standard deviations were used for continuous variables. The primary measure of the current study, the SDQ, was compared with the ADIS-C/P-IV to examine its congruent validity for determining internalizing, externalizing, and total number of comorbidities. An alpha level of α = .05 was used to determine statistical significance. Analyses tested the following postulated hypotheses:

First, linear regression was used to determine whether participants’ SDQ total score (continuous) predicted the total number of comorbidities on the ADIS-C/P-IV (continuous). Binomial logistic regression was then used to determine whether the presence of a clinically significant score (yes/no) on the SDQ emotional problem scale predicted the presence of one or more internalizing comorbidities (yes/no) on the ADIS-C/P-IV. Similarly, binomial logistic regression was used to determine whether the presence of a clinically significant score on the SDQ conduct problems scale (yes/no) predicts the presence of one or more externalizing comorbidities (yes/no) on the ADIS-C/P-IV.

The ability of the SDQ to differentiate between true/false positives and negatives was examined against corresponding ADIS-C/P-IV diagnoses. The sensitivity and specificity of the SDQ subscales were calculated, giving the proportion of participants within normal and abnormal ranges of the SDQ’s emotional and conduct subscales with a corresponding (present or absent) ADIS-C/P-IV internalizing/externalizing diagnosis. Positive and negative predictive values were then calculated from the data, providing the probabilities that the SDQ will accurately determine correct and incorrect DSM-IV comorbid diagnoses.

Receiver operating characteristic (ROC) analyses were also conducted to determine whether the optimal cutoff points for the SDQ emotional and conduct subscales against the ADIS-C/P-IV internalizing and externalizing comorbidities differed from Goodman’s predetermined cutoffs of normal (0-3) and borderline/abnormal (4-10) for emotional symptoms and normal (0-2) and borderline/abnormal (3-10) for conduct problems for the current population.

Results

The frequency and percentages of internalizing and externalizing comorbidities according to the ADIS-C/P-IV are provided in Table 1. Results indicated there were a much greater proportion of participants presenting with anxiety-based disorders compared with depressive disorders. The most prevalent comorbidities within the sample were social (53%) and specific phobia (52%).

Table 1.

Frequency and Percentages of Positive Diagnoses as Determined by the ADIS-C/P-IV (Total n = 226).

Comorbidity	n (%) Yes
Internalizing	186 (82.3)
Social phobia	119 (52.7)
Specific phobia	118 (52.2)
Separation anxiety disorder	96 (42.5)
Generalized anxiety disorder	84 (37.2)
Major depressive disorder	30 (13.4)
Dysthymia	27 (12.0)
Obsessive compulsive disorder	25 (11.1)
Post-traumatic stress disorder	15 (6.6)
Panic disorder	8 (3.5)
Externalizing	153 (68.3)
Oppositional defiant disorder	81 (36.2)
Conduct disorder	75 (33.5)

Note. ADIS-C/P-IV = Anxiety Disorders Interview Schedule for DSM-IV- Parent Version.

The SDQ total score (M = 22.13, SD = 5.63) significantly predicted the total number of comorbidities on the ADIS-C/P-IV (M = 3.02, SD = 1.96), β = .51, t(222) = 8.82, p = < .001. The SDQ total score also explained a significant proportion of variance in the total number of comorbidities on the ADIS-C/P-IV, R² = .26, F(1, 222) = 77.78, p = < .001.

Participants scoring in the clinical range on the SDQ emotional problems subscale were not more likely to meet the criteria for an internalizing comorbidity on the ADIS-C/P-IV, Nagelkerke’s R² = .02, β = .62, Wald χ² = 3.01, p = .083, OR 1.85 (95% CI = 0.92 to 3.72). The model accuracy of SDQ emotional problems against internalizing comorbid diagnoses represents a 30% improvement in correct classification over random allocation alone, as determined by Klecka’s τ = .3.

Post hoc analyses were conducted to examine whether scoring in the clinical range on the SDQ emotional problem scale predicted the presence of specific internalizing comorbidities on the ADIS-C/P-IV.

Participants scoring in the clinical range on the SDQ emotional problem subscale were more likely to meet the criteria for SAD: Nagelkerke’s R² = .09, β = −1.22, Wald χ² = 14.92, p = < .001, GAD: Nagelkerke’s R² = .21, β = −2.14, Wald χ² = 27.48, p = < .001, social phobia: Nagelkerke’s R² = .08, β = −1.11, Wald χ² = 14.08, p = < .001 and specific phobia: Nagelkerke’s R² = .03, β = −.66, Wald χ² = 5.26, p = .022 on the ADIS-C/P-IV. The findings were not significant for MDD, dysthymia, OCD, PTSD, and PD.

The model accuracy of SDQ emotional problems against SAD, GAD, and social and specific phobia diagnoses represents a respective 19%, 27%, 25%, and 16% improvement in correct classification over random allocation alone, as determined by the following Klecka’s τ values: .19, .27, .25, and .16. Table 2 lists odds ratios (OR) showing the effect of changes in group membership from normal to borderline/abnormal in SDQ emotional problems and the respective likelihood that a participant will also have a positive diagnosis on the ADIS-C/P-IV

Table 2.

Odds Ratio of SDQ Emotional Problems Against Significant Internalizing Comorbidities as Determined by the ADIS-C/P-IV.

Comorbidity^a	Odds ratio (95% CI)	p value
Generalized anxiety disorder	8.49 [3.82, 18.88]	< .001
Separation anxiety disorder	3.38 [1.82, 6.28]	< .001
Social phobia	3.02 [1.70, 5.39]	< .001
Specific phobia	1.93 [1.10, 3.38]	.02
Major depressive disorder	2.76 [1.01, 7.53]	.05
Post-traumatic stress disorder	3.44 [0.76, 15.65]	.11
Obsessive compulsive disorder	2.14 [0.77, 5.94]	.15
Dysthymia	1.50 [0.60, 3.71]	.39

Note. SDQ = Strengths and Difficulties Questionnaire; ADIS-C/P-IV = Anxiety Disorders Interview Schedule for DSM-IV–Parent Version.

Panic disorder has been excluded for this analysis because of insufficient available cases (n = 8).

Participants scoring in the clinical range on the SDQ conduct problem subscale were significantly more likely to have an externalizing comorbidity on the ADIS-C/P-IV, Nagelkerke’s R² = .32, β = −3.54, Wald χ² = 31.49, p = < .001, OR = 34.52 (95% CI = 10.02 to 118.94). The model accuracy of SDQ conduct problems against externalizing comorbid diagnoses represents a 60% improvement in correct classification over random allocation alone, as determined by Klecka’s τ = .6.

The ability of the SDQ emotional and conduct problems subscales to correctly identify true positives and negatives against internalizing and externalizing comorbidities on the ADIS-C/P-IV is shown in Table 3. Sensitivity and specificity results indicated that 69% of participants within the borderline/abnormal SDQ emotional banding will have a positive diagnosis of an internalizing comorbidity and 45% of those within the normal SDQ emotional banding will have a negative diagnosis of an internalizing comorbidity. In addition, 98% of participants within the borderline/abnormal SDQ conduct banding will have a positive diagnosis of an externalizing comorbidity and 41% of those within the normal SDQ conduct banding will have a negative diagnosis of an externalizing comorbidity.

Table 3.

Sensitivity, Specificity, Positive Predictive Values and Negative Predictive Values of SDQ Emotional and Conduct Problems Against Internalizing and Externalizing Comorbidities as Determined by the ADIS-C/P-IV.

SDQ Subscale	Sensitivity (95% CI)	Specificity (95% CI)	PPV	NPV
Emotional problems	0.69 [0.62, 0.76]	0.45 [0.31, 0.60]	0.85	0.24
Conduct problems	0.98 [0.94, 0.99]	0.41 [0.30, 0.52]	0.78	0.91

Note. SDQ = Strengths and Difficulties Questionnaire; ADIS-C/P-IV = Anxiety Disorders Interview Schedule for DSM-IV–Parent Version; PPV = positive predictive value; NPV = negative predictive value.

Positive predictive value (PPV) results indicated that 85% of participants with a true positive test result within the SDQ emotional borderline/abnormal banding are correctly identified as having an internalizing comorbidity and 78% of those with a true positive test result within the SDQ conduct borderline/abnormal banding are correctly identified as having an externalizing comorbidity. Negative predictive value (NPV) results indicated that 24% of participants with a true negative test result within the SDQ emotional normal banding are correctly identified as not having an internalizing comorbidity and 91% of those with a true negative test result within the SDQ conduct normal banding are correctly identified as having an externalizing comorbidity.

ROC analyses were conducted to determine the optimal cutoff points for the SDQ emotional (M = 4.79, SD = 2.46) and conduct (M = 5.01, SD = 2.39) subscale scores against the ADIS-C/P-IV internalizing and externalizing comorbidities, respectively, thereby establishing at what level participants of the current study were coded as either normal or borderline/abnormal in these specific domains. Two separate ROC curves for each subscale were plotted graphically; the following area under the curve values were obtained: SDQ emotional scale C = .65, SE = .043, 95% CI from .56 to .73 and SDQ conduct scale C = .87, SE = .024, 95% CI from .82 to .92.

The value for the area under the curve can be interpreted as follows: A randomly selected participant with a positive comorbid diagnosis has an SDQ subscale score higher than that for a randomly chosen participant with a negative comorbid diagnosis. This occurs 87% of the time for the SDQ conduct scale and 65% of the time for SDQ emotional scale. Thus, an ideal comparative measure will have an area under the curve value equal to 1; whereas a measure unable to distinguish between two conditions (i.e., positive or negative comorbid diagnosis) will have an area under the curve value equal to 0.5. Based on these values, the SDQ conduct subscale is a relatively good model while the SDQ emotional subscale is considerably weaker.

Coordinates of the curve results of the analyses found that the optimal cutoff scores are 0 to 5 for normal and 6 to 10 for borderline/abnormal on the SDQ emotional subscale and 0 to 4 for normal and 5 to 10 for borderline/abnormal on the SDQ conduct subscale. These cut points were determined by selecting the point within each subscale that maximized the sum of both sensitivity and specificity values. These differed from Goodman’s original cutoff scores reported earlier.

Discussion

This study found that there was a significant relationship between the SDQ total score and the total number of comorbidities on the ADIS-C/P-IV. There was also a significant relationship found between the SDQ conduct problem subscale and the externalizing comorbid diagnoses as determined by the ADIS-C/P-IV. However, there was no significant relationship found between the SDQ emotional problem subscale and the internalizing comorbid diagnoses as determined by the ADIS-C/P-IV.

Consistent with our expectation, this study found that there was a significant relationship between the SDQ total score and the number of positive comorbidities on the ADIS-C/P-IV. This result is consistent with previous findings reporting relationships between the SDQ and other diagnostic measures. Hawes and Dadds (2004) found a strong correlation between the SDQ total difficulty score and comorbidities identified via diagnostic interview. Goodman and Scott (1999) and Klasen et al. (2000) also found a strong correlation between the SDQ and CBCL total scores. In addition, Klasen et al. (2000) found a similar strength of correlation existing between the SDQ and YSR total scores. Ferdinand (2008) determined that higher scores on the CBCL and YSR can be used to effectively screen for DSM-IV disorders, thus, providing the link that a general significant relationship exists between the SDQ and the ADIS-C/P-IV.

Unexpectedly, this study did not find a significant relationship existing between SDQ emotional problems and a positive internalizing ADIS-C/P-IV comorbid diagnosis. These results are in contrast to the findings from Hawes and Dadds (2004), who found significant correlations between SDQ emotional problem scores and their equivalent domains as determined by diagnostic interview.

The findings from the current study are slightly in line with Goodman and Scott’s (1999) findings, as they found that a weaker strength relationship existed between the SDQ and the CBCL equivalent internalizing subscales than the externalizing subscales.

It is important to note that post hoc analyses revealed a significant relationship between SDQ emotional problems and specific internalizing comorbidities including SAD, GAD, and social and specific phobias. However, there was not a significant relationship between SDQ emotional problems and less common comorbidities observed in children including PD, OCD, PTSD, MDD, and dysthymia. This finding is in contrast to previous research that has found a significant relationship between the SDQ emotional subscale and mood disorders (Ferdinand, 2008). It should be noted that the CBCL and YSR have distinct affective and anxiety subscales operating under the umbrella of internalizing disorders. Whereas, a broader banding of internalizing disorders was applied to the SDQ emotional subscale in the current study, thereby encompassing depressive and anxiety disorders without distinction. This could account for the differing results found in previous studies. Overall, the findings from the current study suggest that the SDQ emotional subscale is better at identifying more commonly occurring anxiety disorders in children with ADHD (Cartwright-Hatton, McNicol, & Doubleday, 2006; Souza, Pinheiro, & Mattos, 2005) as opposed to less common anxiety disorders (e.g., OCD, PTSD) and mood disorders.

As hypothesized, this study found a significant relationship between SDQ conduct problem scale and a positive externalizing comorbid diagnosis on the ADIS-C/P-IV. These results are in line with Hawes and Dadds (2004) study who also found that higher SDQ conduct scores were associated with a greater probability of being assigned an externalizing DSM-IV diagnosis. Furthermore, it is in line, specifically, with Goodman and Scott (1999) and Klasen et al.’s (2000) findings, which indicated a strong correlation existing between the SDQ conduct subscale and the equivalent CBCL and YSR scales.

It should be noted that the aforementioned studies (Ferdinand, 2008; Goodman & Scott, 1999; Hawes & Dadds, 2004; Klasen et al., 2000) used non-ADHD samples for their research, which has formed the basis for the comparative analysis of the SDQ in an ADHD population in the current study.

With regard to sensitivity and specificity results of SDQ emotional and conduct subscales, sensitivity was stronger across both the subscales while specificity was quite poor. This suggests that, as a whole, the SDQ is better able to correctly identify true positives as opposed to true negatives in children with ADHD. That being said, sensitivity is more useful for a screening measure, as the possibility of missing potential comorbid diagnoses that exist is more important than accuracy of negative diagnoses.

Findings from the ROC analyses within the current study determined that Goodman’s SDQ emotional and conduct subscale cutoff scores (based on normative, non-ADHD data) did not provide optimal sensitivity and specificity in determining internalizing and externalizing comorbidities on the ADIS-C/P-IV for children with ADHD in our sample. If sensitivity and specificity are viewed as percentages, by using the ROC cutoff scores as opposed to Goodman’s, the overall clinical accuracy of the SDQ against the ADIS-C/P-IV is increased by 13% for emotional problem scale and 22% for conduct problem scale.

There were a number of limitations within the current study, which should be noted. As the ADIS-C/P-IV only provides positive or negative comorbid diagnoses, this may have led to overreliance on dichotomous variables, which use less powerful statistical techniques. Ideally, SDQ subscale scores would have been evaluated against continuous data of specific ADIS-C/P-IV subscales; however, this was not possible within the current study.

Furthermore, our sample consisted of children with ADHD and moderate/severe sleep problems. It is likely that our sample has more severe ADHD and comorbidities than the general population of children with ADHD. However, this should not have affected the relationships between the screening and diagnostic tools being examined. It is also important to note that sleep problems are actually quite common in children with ADHD, with up to 70% of parents of children with ADHD reporting that their child’s sleep is a problem (Sung et al., 2008). Finally, a better balance or greater number of participants within categories of positive and negative diagnoses of certain comorbidities may have yielded alternative results.

Implications

The current study investigated the congruent validity of the SDQ against the ADIS-C/P-IV and its ability to screen for comorbidities in children with ADHD. Clinically, the SDQ conduct problem scale appears to particularly capture children with externalizing comorbidities, with 98% of children with ADHD and an externalizing disorder being identified on this measure, indicating that the SDQ conduct problem subscale could be employed to reliably determine whether ODD or CD exists in the child. In contrast, the overreliance on the SDQ emotional problems scale may miss some children with ADHD who also present with an internalizing comorbidity. Therefore, it is essential that additional screening questions are used in the clinical context to assess for the internalizing disorders not as well captured by the SDQ emotional problem scales, namely, MDD, dysthymia, OCD, PTSD, and PD.

Conclusion

In conclusion, the current study found that there was a significant relationship between the SDQ and the ADIS-C/P-IV. This was particularly apparent in relation to identifying externalizing comorbidities in children with ADHD, thereby making the SDQ conduct problems subscale an ideal screening measure for ODD and CD. Furthermore, the SDQ was able to effectively screen for commonly occurring internalizing comorbidities, such as SAD, GAD, and social and specific phobias. However, its inability to detect less commonly occurring internalizing comorbidities, including PD, OCD, PTSD, MDD, and dysthymia, suggests that alternative means are needed to screen for these diagnostic categories.

Footnotes

Acknowledgements

We thank all families and pediatricians for taking part in the study. We would also like to thank the following project staff for their contribution in data collection for this study: Bronwyn Campbell, Bibi Gerner, Grace Gordon, Sonia Khano, Kate Lycett, Rebecca Mitchell, Rebecca Palmieri, and Jane Sheehan. We would also like to thank the investigators on the broader Sleeping Sound with ADHD study for their support: Harriet Hiscock, Daryl Efron, Frank Oberklaid, and Fiona Mensah.

Declaration of Conflicting Interests

The author(s) declared that they had no conflicts of interests with respect to their authorship or the publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study has been funded by a project grant from the Australian National Health and Medical Research Council (NHMRC; No. 607362). Dr Sciberras is funded by an NHMRC Early Career Fellowship in Population Health (No. 1037159). The Murdoch Childrens Research Institute is supported by the Victorian Government’s Operational Infrastructure Support Program.

Author Biographies

Joshua Bekker is a psychology honors graduate from Victoria University (VU).

Dorothy Bruck is a psychology academic at VU, a psychologist and research program leader within the VU Centre for Cultural Diversity and Wellbeing.

Emma Sciberras is a postdoctoral research fellow and clinical psychologist at the Murdoch Childrens Research Institute and The Royal Children’s Hospital.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Barkley

R. A.

Fischer

Edelbrock

C. S.

Smallish

(1990). The adolescent outcome of hyperactive children diagnosed by research criteria: I. An 8-year prospective follow-up study. Journal of the American Academy of Child & Adolescent Psychiatry, 29, 546-557. doi:10.1097/00004583-199007000-00007

Bartholomew

Owens

(2006). Sleep and ADHD: A review. Medicine and Health, Rhode Island, 89(3), 91-93.

Biederman

Faraone

S. V.

Spencer

Wilens

Norman

Lapey

K. A.

Doyle

(1993). Patterns of psychiatric comorbidity, cognition, and psychosocial functioning in adults with attention deficit hyperactivity disorder. The American Journal of Psychiatry, 150, 1792-1798.

Biederman

Mick

Faraone

S. V.

(1998). Depression in attention deficit hyperactivity disorder (ADHD) children: “True” depression or demoralization? Journal of Affective Disorders, 47(1-3), 113-122. doi:10.1016/S0165-0327(97)00127-4

Biederman

Newcorn

Sprich

(1991). Comorbidity of attention deficit hyperactivity disorder with conduct, depressive, anxiety, and other disorders. The American Journal of Psychiatry, 148, 564-577.

Bowen

Chavira

D. A.

Bailey

Stein

M. T.

Stein

M. B.

(2008). Nature of anxiety comorbid with attention deficit hyperactivity disorder in children from a pediatric primary care setting. Psychiatry Research, 157(1-3), 201-209.

Brent

D. A.

Perper

J. A.

Goldstein

C. E.

Kolko

D. J.

Allan

M. J.

Allman

C. J.

Zelenak

J. P.

(1988). Risk factors for adolescent suicide: A comparison of adolescent suicide victims with suicidal inpatients. Archives of General Psychiatry, 45, 581-588. doi:10.1001/archpsyc.1988.01800300079011

Bukstein

O. G.

(2000). Disruptive behavior disorders and substance use disorders in adolescents. Journal of Psychoactive Drugs, 32, 67-79. doi:10.1080/02791072.2000.10400213

10.

Carlat

D. J.

(1998). The psychiatric review of symptoms: A screening tool for family physicians. American Family Physician, 58, 1617-1624.

11.

Cartwright-Hatton

McNicol

Doubleday

(2006). Anxiety in a neglected population: Prevalence of anxiety disorders in pre-adolescent children. Clinical Psychology Review, 26, 817-833. doi:10.1016/j.cpr.2005.12.002

12.

Collett

B. R.

Ohan

J. L.

Meyers

K. M.

(2003). Ten-year review of rating scales. V: Scales assessing attention-deficit/hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 42, 1015-1037. doi:10.1097/01.CHI.0000070245.24125.B6

13.

DuPaul

G. J.

Power

T. J.

Anastopoulos

A. D.

Reid

(1998). ADHD rating scale IV: Checklists, norms, and clinical interpretation. New York, NY: Guilford.

14.

Efron

Davies

Sciberras

(2013). Current Australian paediatric practice in the assessment and treatment of Attention Deficit Hyperactivity Disorder: A survey and practice audit. Academic Pediatrics,13(4), 328-333. doi:10.1016/j.acap.2013.03.009.

15.

Faraone

S. V.

Biederman

Keenan

Tsuang

M. T.

(1991). Separation of DSM-III attention deficit disorder and conduct disorder: Evidence from a family-genetic study of American child psychiatric patients. Psychological Medicine, 21, 109-121. doi:10.1017/S0033291700014707

16.

Ferdinand

R. F.

(2008). Validity of the CBCL/YSR DSM-IV scales anxiety problems and affective problems. Journal of Anxiety Disorders, 22, 126-134. doi:10.1016/j.janxdis.2007.01.008

17.

Gillberg

I. C.

Rasmussen

Kadesjö

Söderström

Råstam

Niklasson

(2004). Co-existing disorders in ADHD—implications for diagnosis and intervention. European Child & Adolescent Psychiatry, 13, 80-92. doi:10.1007/s00787-004-1008-4

18.

Goldman

L. S.

Genel

Bezman

R. J.

Slanetz

P. J.

(1998). Diagnosis and treatment of attention-deficit/hyperactivity disorder in children and adolescents. Journal of the American Medical Association, 279, 1100-1107. doi:10.1001/jama.279.14.1100

19.

Goodman

(1997). The strengths and difficulties questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581-586. doi:10.1111/j.1469-7610.1997.tb01545.x

20.

Goodman

Scott

(1999). Comparing the strengths and difficulties questionnaire and the child behavior checklist: Is small beautiful? Journal of Abnormal Child Psychology, 27, 17-24. doi:1023/A:1022658222914

21.

Hawes

D. J.

Dadds

M. R.

(2004). Australian data and psychometric properties of the strengths and difficulties questionnaire. Australian and New Zealand Journal of Psychiatry, 38, 644-651. doi:10.1111/j.1440-1614.2004.01427.x

22.

Heisel

M. J.

Duberstein

P. R.

Lyness

J. M.

Feldman

M. D.

(2010). Screening for suicide ideation among older primary care patients. Australian & New Zealand Journal of Psychiatry, 23, 260-269. doi:10.3122/jabfm.2010.02.080163

23.

Jarrett

M. A.

Ollendick

T. H.

(2008). A conceptual review of the comorbidity of attention-deficit/hyperactivity disorder and anxiety: Implications for future research and practice. Clinical Psychology Review, 28, 1266-1280. doi:10.1016/j.cpr.2008.05.004

24.

Jensen

P. S.

Hinshaw

S. P.

Kraemer

H. C.

Lenora

Newcorn

J. H.

Abikoff

H. B.

Vitiello

(2001). ADHD comorbidity findings from the MTA study: Comparing comorbid subgroups. Journal of the American Academy of Child & Adolescent Psychiatry, 40, 147-158. doi:10.1097/00004583-200102000-00009

25.

Jensen

P. S.

Martin

Cantwell

D. P.

(1997). Comorbidity in ADHD: Implications for research, practice, and DSM-V. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 1065-1079. doi:10.1097/00004583-199708000-00014

26.

Klasen

Woerner

Wolke

Meyer

Overmeyer

Kaschnitz

Goodman

(2000). Comparing the German versions of the Strengths and Difficulties Questionnaire (SDQ-Deu) and the Child Behavior Checklist. European Child & Adolescent Psychiatry, 9, 271-276. doi:10.1007/s007870070030

27.

Kuhne

Schachar

Tannock

(1997). Impact of comorbid oppositional or conduct problems on attention-deficit hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 1715-1725. doi:10.1097/00004583-199712000-00020

28.

Kunwar

Dewan

Faraone

S. V

. (2007). Treating common psychiatric disorders associated with attention-deficit/hyperactivity disorder. Expert Opinion on Pharmacotherapy, 8, 555-562. doi:10.1517/14656566.8.5.555

29.

Larson

Russ

S. A.

Kahn

R. S.

Halfon

(2011). Pattern of comorbidity functioning and services uses for U.S. children with ADHD, 2007. Pediatrics, 127, 462-470. doi:10.1542/peds.2010-0165

30.

Lyneham

Rapee

R. M.

(2005). Agreement between telephone and in-person delivery of a structured interview for anxiety disorders in children. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 274-282. doi:10.1097/00004583-200503000-00012

31.

Murphy

K. R.

Barkley

R. A.

Bush

(2002). Young adults with attention deficit hyperactivity disorder: Subtype differences in comorbidity, educational, and clinical history. Journal of Nervous and Mental Disorders, 190, 147-157. doi:10.1097/00005053-200203000-00003

32.

Polanczyk

G. V.

Silva

Lima

Horta

B. L.

Biederman

Rohde

L. A.

(2007). The worldwide prevalence of ADHD: A systematic review and metaregression analysis. The American Journal of Psychiatry, 164, 942-948. doi:10.1176/appi.ajp.164.6.942

33.

Ransone

M. L.

(2009). Anxiety in children with attention-deficit/hyperactivity disorder: Influences on peer relationships and social skills (Distinguished majors thesis). Charlottesville: University of Virginia.

34.

Sciberras

Efron

Gerner

Davey

Mensah

Oberklaid

Hiscock

(2010). Study protocol: The sleeping sound with attention-deficit/hyperactivity disorder project. BMC Pediatrics, 10, 101. doi:10.1186/1471-2431-10-101

35.

Silverman

W. K.

Nelles

W. B.

(1988). The Anxiety Disorders Interview Schedule for children. Journal of the American Academy of Child & Adolescent Psychiatry, 27, 772-778.

36.

Silverman

W. K.

Saavedra

L. M.

Pina

A. A.

(2001). Test–retest reliability of anxiety symptoms and diagnoses with the Anxiety Disorders Interview Schedule for DSM-IV: Child and parent versions. Journal of the American Academy of Child & Adolescent Psychiatry, 40, 937-944. doi:10.1097/00004583-200108000-00016

37.

Souza

Pinheiro

M. A.

Mattos

(2005). Anxiety disorders in an attention-deficit/hyperactivity disorder clinical sample. Arquivos de Neuro-Psiquiatria, 63(2B), 407-409. doi:10.1590/S0004-282X2005000300008

38.

Spencer

Biederman

Wilens

(1999). Attention-deficit/hyperactivity disorder and comorbidity. Pediatric Clinics of North America, 46, 915-927. doi:10.1016/S0031-3955(05)70163-2

39.

Spitzer

R. L.

Williams

J. B.

Kroenke

Linzer

deGruy

F. V.

III Hahn

S. R.

. . . Johnson

J. G.

(1994). Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study. Journal of the American Medical Association, 272(22), 1749-1756. doi: 10.1001/jama.1994.03520220043029

40.

Sung

Hiscock

Sciberras

Efron

(2008). Sleep problems in children with attention-deficit/hyperactivity disorder: Prevalence and the effect on the child and family. Archives of Pediatrics and Adolescent Medicine, 162, 336-342. doi:10.1001/archpedi.162.4.336

41.

The MTA Cooperative Group. (1999). A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder: Multimodal treatment study of children with ADHD. Archives of General Psychiatry, 56, 1073-1086.

42.

Waschbusch

D. A.

Pelham

W. E. J.

Jennings

J. R.

Greiner

A. R.

Tarter

R. E.

Moss

H. B.

(2002). Reactive aggression in boys with disruptive behavior disorders: Behavior, physiology, and affect. Journal of Abnormal Child Psychology, 30, 641-656. doi:10.1023/A:1020867831811

43.

Wilens

T. E.

Biederman

Spencer

T. J.

(2002). Attention deficit/hyperactivity disorder across the lifespan. Annual Review of Medicine, 53, 113-131. doi:10.1146/annurev.med.53.082901.103945