Abstract
ADHD is the most common neurobehavioral disorder of childhood, affecting an estimated 5-7% of school-aged children (American Psychiatric Association [APA], 2000). In children, ADHD is associated with disruptive behaviors, academic underachievement, peer rejection, noncompliance with adults, and poor social skills (Barkley, 2006; DuPaul et al., 2004). Early detection, assessment, and treatment of ADHD can help to ameliorate the negative educational and psychosocial consequences for most children with ADHD, and may reduce the long-term impact of the disorder (Sonuga-Barke, Koerting, Smith, McCann, & Thompson, 2011). As such, it is important that the most effective treatment possible is implemented and the first step to effective treatment is an accurate diagnosis.
The American Academy of Child and Adolescent Psychiatry (AACAP) recommends that the evaluation of a child suspected to meet criteria for ADHD consist of (a) interviews with the parent and child, (b) acquiring information from the school, (c) obtaining information about overall functioning, (d) evaluation for comorbid disorders, and (e) collecting information pertaining to the child’s medical, social, and family histories (Pliszka, 2007). As part of the assessment, it is recommended that the clinician conduct an in-depth interview with the parent(s) and teacher to determine the presence, duration, frequency, severity, and the onset of ADHD symptoms (Pliszka, 2007). As specified by the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; APA, 2000) criteria, to warrant a diagnosis of ADHD the individual must present with six of nine symptoms from the inattention cluster and/or six of nine symptoms from the hyperactive/impulsive cluster (APA, 2000). The symptoms must have been present for at least 6 months, and are required to be disruptive and inappropriate relative to the individual’s development level (APA, 2000). Each of these symptoms must be present on more days than not, the onset of symptoms must occur during childhood (before age 7 years), and the symptoms must follow a chronic course (APA, 2000). Furthermore, the symptoms cannot occur exclusively during the course of a pervasive developmental disorder, schizophrenia, or other psychotic disorder, or be better accounted for by another mental health disorder (APA, 2000). Importantly, the DSM-IV-TR requires documentation of impairment in at least two settings (e.g., home and school).
While accurate diagnosis of ADHD may seem like a rather simple task given the observable nature of the symptoms, it can in fact be quite challenging. It is relatively easy to rule in ADHD symptoms but difficult to rule out other origins of these symptoms. As such, a diagnosis of ADHD is complicated by the fact that it is often difficult to make a differential diagnosis, since the symptoms characteristic of ADHD are also often characteristic of other disorders. For example, a child with a learning disability when confronted with difficult academic tasks may display inattention that can be mistaken for ADHD (Schachar, 2009). To complicate matters further, children with ADHD are at a significantly increased risk of developing a range of comorbid disorders (e.g., oppositional defiant disorder [ODD], mood and anxiety disorders, learning disabilities) compared with children without ADHD (Barkley, 2006; Yoshimasu et al., 2012). In fact, it is uncommon to see an individual diagnosed with ADHD who does not present with at least one other psychiatric disorder; as many as 87% of children with ADHD may have a second disorder and more than 67% may have two or more additional disorders (Kadesjo & Gillberg, 2001). For these reasons, it is important that the evaluation of ADHD include the assessment of disorders that may be differentially diagnosed as well as comorbid conditions, including physical disorders (e.g., sleep apnea), developmental disorders (e.g., learning and language disorders), and emotional or behavioral disorders (e.g., anxiety, mood disorders, disruptive behavior disorders; American Academy of Pediatrics [AAP], 2011). As such, clinicians should be prepared to deal with a broad range of psychiatric symptoms while assessing and treating individuals with ADHD (Pliszka, 2007).
Although there has been some consensus regarding what an ADHD assessment should consist of (i.e., information on development and general health; clinical interviews with parents and teachers, and possibly the child; and psychoeducational assessment), these recommendations are not always followed (AAP, 2000; Furman & Berman, 2004; Root & Resnick, 2003). Some clinicians and researchers rely heavily on rating scales for the diagnosis of ADHD (Robinson, 2005). In one study, 70% of physicians reported relying on ADHD-specific rating scales, and 60% reported using global rating scales to diagnose ADHD (Chan, Hopkins, & Perrin, 2002). In addition, Handler and DuPaul (2005) found that 85% of practicing psychologists did not regularly follow assessment procedures consistent with guidelines for best practice. While it has been advised that parents and teachers complete a standardized behavior rating scale, it is important to stress that these rating scales by themselves are not recommended as the sole diagnostic tool (AAP, 2000; Pliszka, 2007; Re & Cornoldi, 2009; Tripp, Schaughency, & Clarke, 2006). It is recommended that broad-band rating scales (i.e., questionnaires that tap into ADHD as well as symptoms of other psychiatric disorders), rather than narrow, ADHD- only, focused rating scales should be used as only one component of a multimethod assessment (Pliszka, 2007).
Using rating scales as a sole diagnostic tool is a cause for concern, as there is very little research conducted to establish the specificity and sensitivity of these ratings scales in comparison to diagnoses based on best-practices assessment for ADHD. The most commonly used questionnaires are the Conners’ Parent and Teacher Rating Scales (CPRS/CTRS; Conners, 1997). While the Conners’ Rating Scales are recommended for clinical use, the author of these scales cautions clinicians about the risk of false-positive and false-negative errors, which may ultimately lead to ADHD over or under diagnosis. Consistent with the AACAP practice parameters for the assessment and treatment of children and adolescents with ADHD, the author of the CPRS/CTRS encourages practitioners to use theses rating scales in conjunction with several other sources of information (Conners, 2001). An additional source of information, which is recommended for best-practice assessment, is parent and teacher diagnostic interviews. For example, the Parent Interview for Child Symptoms (PICS; Schachar, Ickowicz, & Sugarman, 2000) is a semistructured clinical interview for use with parents that is designed to diagnose ADHD. Similarly, the Teacher Telephone Interview (TTI; Tannock, Hum, Masellis, Humphries, & Schachar, 2002) is a semistructured clinical interview for use with teachers that is used to diagnose ADHD and is designed to be used in conjunction with the PICS. The PICS and TTI are based on DSM-IV (4th ed.; APA, 1994) criteria and have been validated for the use of assessing ADHD in children (Ickowicz et al., 2006; Tannock et al., 2002).
There have been a few published studies that have investigated the diagnostic utility of rating scales (e.g., DuPaul, Power, Anastopoulis, & Reid, 1998; Bussing, Schuhmann, Belin, Widawski, & Perwien, 1998; Luk & Leung, 1989; McCann, Scheele, Ward, & Roy-Byrne, 2000; Rucklidge & Tannock, 2002; Snyder, 2004; Vaughn, Riccio, Hynd, & Hall, 1997). There is considerable disagreement concerning the diagnostic accuracy of these rating scales (see Collett, Ohan, & Myers, 2003; Snyder, Drozd, & Xenakis, 2004). However, a review of the literature conducted by Snyder, Hall, Cornwell, and Quintana (2006) found that when taking into consideration the statistical methods and experimental designs, the expected range of accuracy for rating scales was 59% to 79% with a pooled mean of 69%. In another study, Quintana, Snyder, Purnell, Aponte, and Sita (2007) found that while rating scales were sensitive measures of ADHD, the rating scales produced false positives for a large percentage of participants without ADHD in a diverse clinical sample (i.e., a specificity of 22%). These results indicate that the diagnostic accuracy of rating scales is uncertain and further research with representative clinical samples is needed to determine their utility.
There have been very few published studies that have specifically examined the diagnostic utility of the CPRS/CTRS. One study evaluated the diagnostic accuracy of the CTRS DSM-IV subscales in comparison to the TTI (Charach, Chen, Hogg-Johnson, & Schachar, 2009). Participants in the study were children aged 6 to 12 years referred to a clinic for the assessment of attention, learning, and behavior problems. Participants completed the CTRS and the TTI. Charach, Chen, Hogg-Johnson, and Schachar (2009) found that high T-scores (60 and above) corresponded to high sensitivity. They also found that T-scores of 80 or higher offered high specificity. In contrast, their results indicated that high scores on the CTRS did not indicate the presence of impairing symptoms reaching diagnostic threshold as reported by the clinical teacher interview (Charach et al., 2009). Charach and colleagues concluded that the results from the CTRS must be interpreted with caution, as the CTRS is limited in its ability to predict whether children referred for an ADHD assessment reach DSM-IV criteria for ADHD at school.
While examining the convergence of the PICS with the CPRS-ADHD index, Ickowicz and colleagues (2006) found that two thirds of clinic cases were rated similarly on both measures. However, individuals identified as having ADHD by the PICS demonstrated more impairment in inhibitory control than individuals identified via the CPRS-ADHD index, suggesting that the PICS may be more capable than the CPRS at identifying children with ADHD who have impaired inhibitory control (Ickowicz et al., 2006).
Furthermore, McGonnell et al. reviewed files of 198 children that had been referred to and seen at our specialty ADHD Clinic. Although all of those 193 children met criteria for ADHD diagnosis based on questionnaires for ADHD, only 58% actually received an ADHD diagnosis after a comprehensive ADHD assessment that included parent and teacher diagnostic interviews, observations, psychoeducational assessments, and collection of historical information about the child’s development and school functioning (McGonnell et al., 2009).
The purpose of the current study was to investigate the validity of the CPRS/CTRS and semistructured interviews (PICS/TTI) in relation to best-practices clinical diagnosis of ADHD. Participants were clients seen at the Colchester East Hants ADHD Clinic (n = 279) who had all received comprehensive assessments of ADHD (based on AACAP guidelines), including diagnostic interviews with parents and teachers (PICS/TTI), along with ratings scales (CPRS/CTRS), psychoeducational assessments, classroom observations, and a collection of historical information. The information collected at this clinic allowed us to test the specificity and sensitivity of questionnaires (i.e., CPRS/CTRS) and clinical interviews (i.e., PICS/TTI) within this sample of children. The sample was diagnosed with a wide range of clinical diagnoses, rather than just ADHD, as has often been the case in much of the past research. It has been suggested that rating scales may be less able to distinguish between ADHD and individuals without ADHD in a clinic setting than in a community-based sample (AAP, 2000). We hypothesized that the rating scales and clinical interviews used for the assessment of ADHD would be highly sensitive (i.e., would correctly identify the majority of patients with ADHD), but that the clinical interviews would also demonstrate high specificity compared with the questionnaires.
Method
Participants
Data used in the current study were drawn from a research database comprising children assessed at the Colchester East Hants ADHD Clinic (referred to as the ADHD Clinic throughout this article). The ADHD Clinic is located at a hospital Colchester East Hants Health Centre in a mid-size town in Nova Scotia, Canada. The ADHD Clinic operates under a formal partnership between the Colchester East Hants Health Authority and the Chignecto-Central Regional School Board (More information about this clinic can be found in the article by McGonnell et al., 2009). To receive services from the ADHD Clinic, and thus participate in the current study, children were required to be between the ages of 6 and 12 at the time of the assessment, and attend school within the designated health district catchment area. The ADHD Clinic is restricted to children who have no previous diagnosis of ADHD, are psychotropic medication-naïve, and have not received a psychoeducational assessment within the past 2 years. The ADHD Clinic has been open since 2000; at the time of the study (July 2012) there were 279 ADHD Clinic clients in the database eligible to participate. Participants were restricted to clients who consented to have the clinical information collected during their visit to the ADHD Clinic used for research purposes. Five clients did not sign consent forms and thus their data were not included in the current research study.
Measures
PICS (Schachar et al., 2000)
The PICS is a semistructured diagnostic interview based on DSM-IV criteria for use with parents. The PICS was developed specifically to diagnose childhood disruptive behavior disorders (i.e., ADHD, ODD, and conduct disorder [CD]) and to screen for other emotional and psychiatric disorders (e.g., anxiety disorders, mood disorders; Schachar, Ickowicz, & Sugarman, 2005). The PICS contains three modules: (a) the General Information module, (b) the Disruptive Disorders module, and (c) the General Psychopathology module. The General Information module takes approximately 1 hour to administer, and asks information about family history and composition, medical history, developmental milestones, and school history. Demographic variables such as child age and sex were drawn from this module to describe the sample. The Disruptive Behaviors module taps into behaviors associated with the diagnosis of ADHD, ODD, and CD. This section elicits descriptions of behavior (over the past 6 months for the ADHD and ODD sections, and over the past 6-12 months for the CD section) in specific situations rather than simply listing symptoms. The General Psychopathology module asks about symptoms relating to a range of other mental health disorders, such as anxiety disorders and mood disorders.
During this interview, the clinician rates the presence and severity of symptoms based on parent descriptions of the child’s behavior in various situations. However, the context and the symptoms probed in each situation are standardized (Schachar et al., 2005). It is the job of the interviewer to encourage the informant to provide enough detailed information about their child’s behavior so that the interviewer is capable of determining whether the criteria for a symptom is reached (Schachar et al., 2005). The severity of symptoms is rated on a 4-point scale with the following anchors, 1 0 = absent, 1 = dubious or trivial abnormality, 2 = definite abnormality, 3 = marked abnormality, and 9 = not known or unable to rate. Administration usually takes from 1.5 to 2.5 hours. Ratings must be given “2” or greater for the individual symptom to be regarded as clinically significant.
The PICS has been shown to be a reliable and valid measure for a variety of diagnoses (Ickowicz et al., 2006). Interrater reliability of 48 randomly selected videotaped interviews of the PICS indicated good reliability for ADHD diagnosis and CD, and excellent reliability for ODD. In addition, intraclass correlation coefficients for symptom scores of all three disorders are excellent (Ickowicz et al., 2006).
TTI (Tannock et al., 2002)
The TTI is a semistructured diagnostic interview based on DSM-IV criteria, administered by the ADHD Clinic’s mental health psychologist with the child’s teacher over the phone. The structure and format of this interview are similar to those of the PICS, and it is designed to be used in conjunction with the PICS. The TTI was developed to systematically assess children’s disruptive behavior and impairment (ideally over a time frame of 6 months) in the school setting and as such, to aid in the diagnosis of ADHD, ODD, and CD (Tannock et al., 2002). The TTI also allows for the assessment of other major mental health disorders (e.g., anxiety, depression). The TTI consists of three components (a) the academic placement review, (b) the symptom review, and (c) the impact review (Tannock et al., 2002). The purpose of the academic placement review is to determine the degree of structure in the current classroom, decide whether other teachers need to be interviewed, and to learn which supports and interventions are already in place for the child. In the symptom review section, the teacher describes the student’s behavior in five different situations (i.e., school arrival, group instruction, individual seatwork, transitions, peer interactions) which allows for collection about the child’s ADHD symptoms as well as their academic functioning, classroom behavior, peer relations, and relationships with adults in the school. Based on teacher descriptions, the interviewer rates the severity and frequency of symptoms on a 4-point scale 1 (0 = no problem, 1 = mild, 2 = moderate, 3 = severe). Finally, the impact review is conducted by asking the teacher several questions relating to their main concerns regarding the child, the chronicity of the problem, the child’s medication status, and the extent to which these concerns affect the teacher and other students in the class (Tannock et al., 2002). Preliminary analyses have been conducted and have shown high interrater reliabilities (Hum, 2005).
CPRS/CTRS-Revised (Long Form; Conners, 1997)
The CPRS is a standardized behavior rating scale completed by parents of children and adolescents ages 3 to 17 to assess an array of problem behaviors. The CPRS consists of 80 items that comprise 14 scales designed to tap into externalizing (e.g., conduct problems, aggression) and internalizing (e.g., anxiety, depression) symptomatology. The CTRS is the teacher version and consists of 59 items that comprise 13 scales. Parents and teachers are asked to consider their child’s/student’s behavior over the past month when completing the Conners’ Rating Scales. The parent and teacher versions are designed to provide an assessment of ADHD; however, these rating scales also contain subscales for the assessment of conduct, family, emotional, anger control, and anxiety problems (Conners, 1997). The CPRS/CTRS are considered reliable and valid. When the Conners’ Rating Scales are compared with other measures of ADHD symptoms, concurrent validity ranges from .68 to .80 (Pelham, Fabiano, & Massetti, 2005). For the current study, we were particularly interested in two subscales of the CTRS/CPRS, the DSM-IV Inattentive Symptoms subscale (9 items) and the DSM-IV Hyperactive-Impulsive Symptoms subscale (9 items). High scores (T-scores of 65 or above) on these subscales indicate clinical significance. All T-scores were capped at 90 for the analyses.
Procedure
Ethical approval for the study was obtained from the Colchester East Hants Health Authority and the IWK Health Centre. The diagnostic procedures at the ADHD Clinic were provided by an interdisciplinary team consisting of clinical psychologists, school psychologists, and pediatricians. Prior to the clinic day, screening questionnaires (CPRS/CTRS) were completed by the parent(s) and the teacher. Parents also completed a demographic questionnaire. In addition, the ADHD Clinic psychologist conducted a diagnostic telephone interview with the child’s teacher (i.e., TTI), and the school psychologist completed an in-class observation of the child, as well as a review of the child’s school record. On the actual clinic day, parents completed a semistructured diagnostic interview (i.e., PICS) with the child psychologist and pediatrician. Moreover on the clinic day, the child received a standard psychoeducational assessment battery by the school psychologist, measuring the child’s intellectual abilities, academic achievement, and some processing skills such as visual-motor integration. After collection of all information, the ADHD Clinic team met to discuss possible diagnoses based on the results of the above measures. Once a diagnosis had been made, the ADHD Clinic team met with the parents/guardians to discuss the results of the assessment, provide the diagnosis/diagnoses (if any diagnoses where reached), and to recommend potential treatments and intervention strategies.
At the end of the assessment procedures, informed consent from parents/guardians was obtained to allow data collected during the assessment to be used for future research purposes. Only the data from those children whose parents/guardians provided consent were included in the research database. Once all of the data were entered into the database, data labeled as missing from the database was double checked against hard copies of data files to fill in missing values, and to check the accuracy. Spot-checking was also completed for 80 data files to ensure their accuracy. Once the data were checked, deidentified Excel data files were downloaded to a password-protected electronic file storage.
Data Analysis
All of the data used in this study were first organized in Microsoft Excel. The data were then uploaded into SPSS (version 17) for analysis. The sensitivity and specificity were computed for clinical interviews (i.e., PICS, TTI) and rating scales (i.e., CPRS/CTRS) in relation to the clinical diagnosis. The ADHD Clinic diagnosis is considered the “true” diagnosis as it is based on comprehensive assessment and diagnostic procedures that are consistent with best practice.
To compute sensitivity and specificity, scores for the PICS and the TTI were recoded as follows: 0 = did not reach criteria for ADHD (less than 4 on the inattentive and hyperactive-impulsive symptom clusters), 1 = reached criteria for ADHD (6 or more on one or both of the symptom clusters), and 2 = showed some evidence of ADHD (scored 4-5 on either of the symptom clusters but below 6 on both clusters). Using the ADHD Clinic criteria for diagnosing ADHD (i.e., child meets criteria on either the PICS or the TTI and has at least four symptoms on the other interview), we then determined who met criteria for ADHD based on the clinical interviews.
Similarly, the CPRS/CTRS were both recoded as follows: 0 = did not reach criteria for ADHD (T-score less than 65 on both symptom clusters), 1 = reached criteria for ADHD (T-score greater than or equal to 65 on at least one symptom cluster), 2 = showed some evidence of ADHD (scored 60-64 on either symptom cluster and below 65 on both symptom clusters). Using similar criteria to the PICS/TTI algorithm (meeting criteria in at least one setting and demonstrating some symptoms in the other setting), we were able to determine who met criteria for ADHD based on a combination of the data from the CPRS/CTRS. Children were required to have a T-score of 65 or higher on either symptom cluster of the CPRS or CTRS, and a T-score of at least 60 on the other questionnaire. For example, if a child had a T-score of 70 on the CPRS, he or she must also have a T-score of at least 60 on the CTRS to meet criteria for ADHD based on the questionnaires.
Two chi-square tests were completed: clinical interviews/clinic diagnosis and questionnaires/clinic diagnosis. Sensitivity (i.e., the measure’s accuracy in correctly identifying clients as having ADHD) was determined by calculating the number of true positives divided by the number of true positives plus the number of false negatives. Specificity (i.e., the measure’s accuracy in correctly identifying when a client does not have ADHD) was determined by calculating the number of true negatives divided by the number of true negatives plus the number of false positives.
Results
Sample Characteristics
Of the 289 clients assessed at the ADHD Clinic, 5 were excluded from analyses as they did not consent to allow their data to be used for research purposes, and 5 clients did not have TTI data, and thus were also excluded. Therefore, the final sample consisted of 279 children. The mean age of the children seen at the ADHD Clinic was 8.49 years (SD = 1.70), ranging from 5.95 to 12.67 years. The majority (i.e., 69.2%) of clients were male. Approximately 22% of parents indicated that they had completed university and 27.6% reported completing community or technical college. See Table 1 for more detailed demographic information.
Demographic Characteristics of Clinical Sample Referred for Attentional and Behavioral Difficulties.
Of the clients assessed at the ADHD Clinic, 52.3% (n = 146) were diagnosed with ADHD, which included the following subtypes: 66.4% (n = 97) were diagnosed with ADHD Combined-type, 26.0% (n = 38) received a diagnosis of ADHD Inattentive-type, 6.8% (n = 10) were diagnosed with ADHD Hyperactive-Impulsive type, and 0.7% (n = 1) were diagnosed with ADHD–not otherwise specified (ADHD-NOS). Of those diagnosed with ADHD, 28.8% (n = 42) were also diagnosed with a comorbid mental health condition and 47.9% (n = 70) were diagnosed with a learning disability, or were identified as being at significant risk for a learning disability. The remaining 47.7% (n = 133) of the entire sample did not receive a diagnosis of ADHD, but 66.2% (n = 88) of these individuals received another diagnosis (20% [n = 55] were diagnosed with another mental health disorder 2 and 18% [n = 49] were diagnosed with only a learning disability). The remaining 10.0% (n = 29) of clients assessed at the ADHD Clinic were not diagnosed with ADHD, a learning disability, or any other mental health disorder.
Sensitivity and Specificity
The values evaluating diagnostic accuracy of the questionnaires and diagnostic interviews are summarized in Table 2 and Table 3, respectively. The sensitivity of clinical interviews (PICS/TTI) in comparison to the clinical diagnosis was 91.8%, and the specificity was 70.7%. In contrast, while the sensitivity of questionnaires (i.e., CPRS/CTRS) with clinical diagnosis of ADHD was relatively high 83.5%, the specificity was much lower, at 35.7%.
Sensitivity and Specificity of Rating Scales (CPRS/CTRS) in a Diverse Clinical Sample, as Compared With a Best-Practice Clinic Diagnosis of ADHD.
Note. CPRS/CTRS = Conners’ Parent and Teacher Rating Scales.
Sensitivity and Specificity of Diagnostic Interviews Used to Diagnose ADHD (PICS/TTI) in a Diverse Clinical Sample, as Compared With a Best-Practice Clinic Diagnosis of ADHD.
Note. PICS = Parent Interview for Child Symptoms; TTI = Teacher Telephone Interview.
Discussion
Accurate ADHD diagnosis is the first step to appropriate treatment. Inaccurate diagnosis may result in either lack of treatment or inappropriate treatment, which could in turn result in worsening of symptoms and ultimately worsen the child’s prognosis. The main goal of this study was to determine whether the sensitivity and specificity of the CPRS/CTRS are adequate for diagnostic utility. More specifically, we wanted to compute the sensitivity and specificity of the CPRS/CTRS in comparison to a comprehensive clinical assessment of ADHD.
As expected, we found high values for the sensitivity and specificity of diagnostic parent and teacher clinical interviews in relation to the comprehensive diagnosis given at the ADHD Clinic (i.e., 91.8% and 71.8%, respectively). This was not surprising considering the diagnosis given by the ADHD Clinic depends highly on the parent and teacher clinical interviews. However, the ADHD Clinic team also takes into account various other sources of information including psychoeducational assessments, classroom observations, behavioral rating scales, medical information, and school records when determining diagnosis. The sensitivity and specificity values we calculated are consistent with this model, indicating that the PICS and TTI were capable of diagnosing most individuals correctly. The results indicate that the clinical interviews did not have perfect diagnostic accuracy, which would be expected in a multimethod assessment. The lack of perfect diagnostic accuracy means that at times the ADHD Clinic team found it necessary to overrule (i.e., rely more heavily on other aspects of the data such as the psychoeducational assessment results, mental health, and environmental considerations) the results of the diagnostic interviews. For example, the clinicians may have had good reason to believe that a child who presented with clinically significant levels of ADHD symptoms did not warrant a diagnosis of ADHD, as the symptoms were thought to be the result of another underlying cause (e.g., sleep disorder, chaotic family life, etc).
Similar to the clinical interviews, the CPRS/CTRS demonstrated high sensitivity in relation to the overall clinic diagnosis, yet specificity was found to be low (i.e., 83.5% and 35.7%, respectively). These findings support our hypotheses and demonstrate that while the CPRS/CTRS correctly identified the majority of patients with ADHD, the rating scales were unable to correctly identify when a patient did not have ADHD (i.e., there were a large number of false positives). As such the CPRS and CTRS are not appropriate tools to aid in a differential diagnosis.
The above findings are consistent with literature in the field and highlight the importance of refraining from relying on rating scales for the assessment and diagnosis of ADHD. Charach and colleagues (2009) calculated specificity and sensitivity values for a clinical sample for individuals whose teachers completed the CTRS and TTI. They found that T-scores of 80 on the CTRS or higher were needed before high specificity was demonstrated (Charach et al., 2009). Similar to the current study, Quintana et al. (2007) found that rating scales were sensitive measures of ADHD (81%), the rating scales produced false positives for a large percentage of participants without ADHD in a diverse clinical sample (i.e., specificity of 22%). Combining our results with those of previous studies highlights the weak specificity of the CPRS/CTRS and therefore brings into question the appropriateness of using these questionnaires for diagnostic purposes.
There are a number of strengths of the current study. Although there have been some similar studies, to date there is no known research investigating the diagnostic utility of the CPRS/CTRS in comparison to a clinical diagnosis based on best-practice assessment methods as outlined by AACAP. As a result, this study provides useful information and recommendations regarding the assessment and diagnosis of ADHD. An additional strength of this study is the large sample size (n = 279 participants), which allows us to be more confident in the generalizability and accuracy of our results. Finally, children were referred specifically to the ADHD Clinic for attentional and behavioral difficulties, and as such we were able to test the sensitivity and specificity of the CPRS/CTRS in a clinically diverse sample. In terms of limitations, children assessed at the ADHD Clinic were all between the ages of 6 and 12, which confines the generalizability to this age group. In addition, all of the participants in the study were required to live in a specific geographic location to qualify for services from the ADHD Clinic. As such, it is difficult to say whether the results of the current study would generalize to other geographic regions. Finally, since the current study administered the Conners’ Rating Scales to a clinical sample, the findings of this study may not generalize to administrations of the CPRS/CTRS in a community-based sample.
Clinical Implications
Our findings highlight the importance of using best-practice assessment procedures, including clinical interviews, to diagnose ADHD. The Conners’ Rating Scales are useful tools when used in conjunction with other diagnostic measures. However, the specificity of this rating scale is not sufficient to be used alone to diagnose ADHD. Given these results, it is alarming when we consider the large percentage of clinicians relying heavily on rating scales to diagnose ADHD (Chan et al., 2002). ADHD is a complex disorder, that can be a challenge to diagnose, and as such, accurate ADHD diagnosis requires more information than can be obtained from a simple rating scale. The results of this study indicate that while the Conners’ Ratings Scales are likely to correctly identify most clients with ADHD, it will also yield many false positives. This is problematic because diagnosis affects treatment. If the diagnosis given is not accurate, there is a good chance that the best treatment available will not be provided, and thus will likely be ineffective.
Conclusion
While the CPRS/CTRS are useful tools when used in combination with other diagnostic measures, the specificity of this rating scale is not sufficient to be used as the sole diagnostic instrument. If the CPRS/CTRS were used as the sole diagnostic measure, this would potentially result in incorrectly identifying those with other mental health disorders that often display symptoms of inattention and/or hyperactivity-impulsivity as having ADHD. This is problematic because diagnosis affects treatment decisions. Based on our results and AACAP guidelines for best practice, we recommend that symptom information be initially gained via broad-band rating scales and cases that are flagged by these scales be then further evaluated with a best-practices comprehensive assessment of ADHD.
Footnotes
Acknowledgements
The authors would like to extend their sincere gratitude to the IWK Health Centre and Dalhousie University for providing funding for the current research study, the children and families who participated, and the ADHD Clinic staff (Rachel Panton, Naomi Aquino, Marilyn MacPherson, and Tracey Williams, as well as all school psychologists and past mental health clinicians who have worked at the ADHD Clinic).
Authors’ Note
Ashton Parker is now at the Department of Education, Mount Saint Vincent University, Halifax, Nova Scotia, Canada, B3M 2J6.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for the research study was provided by the IWK Health Centre and Dalhousie University.