Abstract
Introduction
ADHD symptoms are commonly classified with scales such as the ADHD Rating Scale (ADHD-RS) (DuPaul et al. 1998) and the Conners-Wells' Adolescent Self-Report Scales (CASS) (Conners 1997), which describe core symptoms (both of the above scales) and associated features (CASS only) of the disorder. Both of these scales have been validated with adolescent populations; however, the ADHD-RS uses domain symptoms without adolescent-specific modifiers for content specificity. The Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) (American Psychiatric Association 1994) presents the symptoms of ADHD in language that is particular to children, but adolescents and adults often manifest symptoms differently than children. Adolescents and adults typically experience a relative increase in inattentive symptoms compared with hyperactive/impulsive symptoms, which is partially due to the higher cognitive demands of adolescence and adulthood and that overt hyperactive symptoms are more often felt internally rather than externally manifested (Millstein et al. 1997; Weiss and Jain 2000; Murphy and Adler 2004; Kessler et al. 2010; Goodman et al. 2011). Consequently, the clinician is left to interpret the presentation of the disorder for these older populations (Adler and Cohen 2004).
The Adult ADHD Self-Report Scale (ASRS) v1.1 Symptom Checklist is an 18-item scale developed by the workgroup on adult ADHD for the World Health Organization (WHO) as a means for providing a valid self-assessment of ADHD symptoms (Kessler et al. 2005). The symptom presentation of the ASRS v1.1 Symptom Checklist uses adult-specific language in a self-rated, frequency based format with symptoms rated on a 0–4 scale (0 “never,” 1 “rarely,” 2 “sometimes,” 3 “often,” and 4 “very often”; (Adler et al. 2010). The language of the items is described in Table 1. The ASRS v1.1 Symptom Checklist has shown high internal consistency and concurrent validity with the clinician-administered ADHD-RS in community- and clinic-based samples of adults with ADHD (Kessler et al. 2005; Adler et al. 2006) and as a measure of treatment response (Adler et al. 2009). The scale, which is copyrighted by the WHO, has been translated into 10 languages and is available in the public domain at
ADHD=attention-deficit/hyperactivity disorder; ASRS=Adult ADHD Self-Report Scale; ADHD-RS=ADHD Rating Scale.
As it could be advantageous for clinicians to have a valid, self-report scale that uses age-specific language and provides a context-basis to assess symptoms of adolescent ADHD, we sought to validate and evaluate the psychometric properties of the ASRS v1.1 Symptom Checklist in a sample of adolescents with ADHD by: (1) Evaluating the internal consistency of the ASRS v1.1 Symptom Checklist and (2) Examining the concurrent validity of the ASRS v1.1 Symptom Checklist by comparing it to the clinician-administered ADHD-RS.
Methods
Study participants
Adolescent outpatients between the ages of 13 and 17 years inclusive, who met Diagnostic and Statistical Manual of Mental Disorders, 4th edition Text Revision (DSM-IV-TR) (American Psychiatric Association 2000) criteria for a primary diagnosis of ADHD, were eligible to participate. The diagnosis of ADHD was based on findings from DSM-IV-TR-based clinician evaluation supplemented by the Kiddie Schedule for Affective Disorders & Schizophrenia (K-SADS) (Kaufman et al. 1997). Individuals who were unable to reliably report ADHD symptoms as determined by study investigators were excluded. To obtain the broadest applicability to these findings, we included subjects both on medication and off medication and did not control for medication selection or titration. Participants were recruited for this study from the following sources: (1) advertisements in local media; (2) professional referrals for evaluation of ADHD; (3) participants from previous ADHD studies who had consented to be contacted for future studies; or (4) adolescents receiving treatment for ADHD at the study site.
Rating scales
The ASRS v1.1 Symptom Checklist is described above.
The clinician-administered ADHD-RS was designed and validated to assess current ADHD symptomatology (DuPaul et al. 1998; Faries et al. 2001). The scale consists of 18 items that directly correspond to the 18 DSM-IV symptoms of ADHD and rates the frequency and severity of each symptom on a 4-point scale from 0 (none) to 3 (severe). Although originally validated as a teacher-administered scale, the clinician-administered ADHD-RS has been validated for use with adolescents (Faries et al. 2001) and used as the primary outcome measure in clinical trials of osmotic-release oral system (OROS) methylphenidate (Wilens et al. 2006a, 2006b), atomoxetine (Michelson et al. 2001; Spencer et al. 2002; Kelsey et al. 2004), and lisdexamfetamine dimesylate (Biederman et al. 2007; Findling et al. 2008, 2011) in children and adolescents with ADHD. Table 1 shows the symptom language from both scales. To minimize inter-rater variability, raters at all study sites had extensive experience in the assessment of ADHD and were required to complete a rater training program, which has been detailed elsewhere (Adler et al. 2005).
Study design
The study was conducted from April 2006 to April 2009 at four study sites: (1) the ADHD Program at the NYU School of Medicine, (2) the Child Mental Health Outpatient Clinic at Bellevue Hospital, (3) the ADHD Clinical Trials Program at the Mount Sinai School of Medicine, and (4) the Research Program in Pediatric Psychopharmacology at Massachusetts General Hospital (MGH). Institutional review board approval was obtained through the local board at each site. Study procedures were consistent with the Declaration of Helsinki and Good Clinical Practice according to the International Conference on Harmonization guidelines. All parents and adolescents provided written informed consent and assent, respectively, before completing any study-related procedures.
The study was completed in either one or two visits that occurred one to nine weeks apart. Prospective participants were screened for eligibility and enrolled into the study at Visit 1. Clinicians completed the ADHD-RS and participants were asked to complete the ASRS v1.1 Symptom Checklist based on the preceding month at Visit 1 and the interval of time between visits at Visit 2.
Data analysis
Internal consistency of the ASRS v1.1 Symptom Checklist was assessed by Cronbach's alpha (Cronbach's α) (Cronbach 1951). Concurrent validity was determined by calculating Pearson's correlation coefficients between the total mean item scores for the ASRS v1.1 Symptom Checklist and the ADHD-RS. Analyses of internal consistency and concurrent validity were also performed with the sample stratified by age (ages 13–15 and 16–17) and gender. The Kappa coefficient of agreement assessed item-by-item agreement between the ASRS v1.1 Symptom Checklist and the ADHD-RS. A one-way analysis of variance tested for site differences in age and socioeconomic status (SES) and a chi-square tested for site differences in gender. To allow for comparisons between the scales for Pearson's correlation coefficient and Kappa correlation computations, responses of “0” (none) and “1” (rarely) for the ASRS v1.1 Symptom Checklist were combined into a single category to match the “0–3” scoring of the ADHD-RS, since the ADHD-RS is rated 0–3 and the ASRS v1.1 Symptom Checklist is rated 0–4.
Results
A total of 88 participants were enrolled: 27 at NYU/Bellevue, 40 at Mt. Sinai, and 21 at MGH. Of the sample, 65.9% (n=58) of participants completed both visits. The study was designed prehoc so that at least 60% of participants would complete two visits to allow repeat examination of the scale. Analyses of internal consistency and concurrent validity were separately performed for all 88 participants at Visit 1 and all 58 participants at Visit 2. The item-by-item analysis was performed with data from Visits 1 and 2 combined.
The mean age of participants was 14.9±1.5 SD years; 76.1% (n=67) of the sample was male. About 71.6% (n=63) of the sample were younger adolescents and 28.4% (n=25) were older adolescents. About 64.8% (n=57) of the sample identified themselves as Caucasian, 14.8% (n=13) as African-American, 12.5% (n=11) as Hispanic, 3.4% (n=3) as Asian-American, 1.1% (n=1) as American Indian, and 3.4% (n=3) as other/mixed ethnicity. The mean SES, as determined by the Hollingshead Four Factor Index of Social Status scale (Hollingshead 1975), was 47.7±9.6 SD. About 73.9% (n=65) of the sample was currently receiving medication for ADHD, with 47.7% (n=42) taking psychostimulants, 9.1% (n=8) taking atomoxetine and 17.1% (n=15) taking a combination of the two medications. There were no significant differences across sites in age [F (2, 85)=0.30, p=0.74], gender [χ 2(2)=0.064, p=0.97], or SES [F (2, 70)=0.49, p=0.62].
Internal consistency
Internal consistency of ASRS v1.1 Symptom Checklist items was high, with Cronbach's α coefficients of 0.93 at Visit 1 and 0.94 at Visit 2. The Cronbach's α coefficients for the young adolescent cohort (ages 13–15) were 0.92 at Visit 1 and 0.94 at Visit 2. The old adolescent cohort (ages 16–17) had Cronbach's α coefficients of 0.94 for Visit 1 and 0.96 for Visit 2. The Cronbach's α coefficients for the boys were 0.93 for Visit 1 and 0.94 for Visit 2. The Cronbach's α coefficients for the girls were 0.92 for Visit 1 and 0.95 for Visit 2.
Concurrent validity
Pearson's correlation coefficients between the ASRS v1.1 Symptom Checklist and ADHD-RS were significant (p<0.0001) at Visit 1 and Visit 2 (Table 2). For the analyses stratified by age and gender, Pearson's correlation coefficients between the scales were significant (p<0.0001) at Visit 1 and Visit 2 for all subgroups (Table 2).
The item-by-item analysis between the scales found moderate agreement for individual items (% agreement: 35.2%–63.4%) and statistically significant kappa coefficients (p=0.071–0.339) for 17 of the 18 items (Table 3).
Analysis performed with data from Visits 1 and 2 combined.
Discussion
The ASRS v1.1 Symptom Checklist showed high internal consistency and strong evidence for concurrent validity with the clinician-administered ADHD-RS. However, the scales are also fairly distinct; Pearson correlation coefficients indicate that the shared variance is only 25%–30%. Since the ASRS v1.1 Symptom Checklist is a self-report scale that uses adult-specific, context-based language and the ADHD-RS is a clinician-rated instrument with brief descriptions of symptom domains, some of the disagreement is to be expected and is not necessarily related to properties of the scales. The internal consistency and concurrent validity analyses performed with the sample stratified by age and gender did not reveal an effect for either measure, as the scores were quite similar across groups. The absence of an age effect is of particular significance given the differential symptom presentations based on differing demands specific to each age group and that older adolescents are chronologically closer to young adults and actually considered part of the transitional adult cohort (The MTA Cooperative Group 1999). The discrepancy between the Pearson correlation coefficients between younger and older adolescents at Visit 1 (r=0.73 and 0.70, respectively) and Visit 2 (r=0.65 and 0.92, respectively) may be due, in part, to the ability of older adolescents to self-report better than younger adolescents (Wilens et al. 2006a). The older adolescent cohort may have been better able to self-observe at Visit 2 due to increased familiarity with the scales and their own symptoms, leading to better correlations with clinician ratings at Visit 2.
The moderate item-by-item agreement between the ASRS v1.1 Symptom Checklist and the ADHD-RS was not surprising as the ASRS v1.1 Symptom Checklist is scored based on symptom frequency, whereas the ADHD-RS is scored on both the frequency and severity of symptoms. The ability to assess multiple items when diagnosing ADHD is reinforced by the general, but not identical agreement of items. There was less agreement for Items 7, 9 and 11, which assess problems associated with executive functioning and suggests that adolescents may be less likely to accurately assess deficits in these areas. Since these symptoms have significant clinical impact and are therefore of primary concern, clinicians might consider gathering information from other sources such as parents or teachers. The lack of significance for Item 8 could be attributed to differences in content between the scales, as the ASRS v1.1 Symptom Checklist item assesses difficulty unwinding or relaxing during free time and the ADHD-RS assesses difficulty playing or engaging in leisure activities quietly which may have resulted in a high degree of variability. Consequently, there was either very high or very low agreement for item 8 leading to moderate agreement between the scales but a low kappa.
The ASRS v1.1 Symptom Checklist was not originally designed for diagnostic validity and our goal was to assess the internal consistency and concurrent validity of the ASRS among ADHD adolescents. Lacking a non-ADHD group, we were not able to assess the predictive validity of the ASRS for a diagnosis of ADHD.
The sample in this study was relatively small and the majority was receiving medication for ADHD at the time of their participation in the current study, which may limit generalizability of the results. Additionally, selection bias may have occurred by not controlling for medication selection and titration. It is possible that treatment reduced the variability of symptoms in our sample, which would be expected to reduce our measures of internal consistency and validity. Other limitations include the fact that the ADHD-RS is a clinician-administered scale whereas the ASRS v1.1 Symptom Checklist is a self-report scale and that the order of administration of the scales was not specified. Lastly, although the study population included a mix of patients recruited from advertisements, ongoing clinical trials and existing clinical populations, the sample was primarily a referred one which may raise concerns about the potential impact of referral bias.
Conclusions
Although the ASRS v1.1 Symptom Checklist was originally developed and validated for use in adult ADHD, this preliminary validation of the scale extends the utility of the scale beyond adults to adolescents. While we did not evaluate the validity of the ASRS v1.1 Symptom Checklist as a measure of treatment response, the test-retest reliability between Visits 1 and 2 was good and the scale is valid for repeat administration, suggesting that it may be a good measure of treatment response. Future directions of research will include further replication of the ASRS v1.1 Symptom Checklist with larger numbers of participants, contain samples of ADHD and non-ADHD adolescents, and explore the validity of the scale as a measure of treatment response, which will allow clinicians to use the scale during the planning and evaluation of treatment regimens for adolescents with ADHD.
Clinical Significance
Clinicians should find the age-specific, context-based language of the ASRS v1.1 Symptom Checklist useful for interpreting the presentation of ADHD in both younger and older adolescents. Other potential advantages of the scale include its ready availability, translation in multiple languages, and ability to offer longitudinal assessment of symptoms from adolescence into adulthood.
Footnotes
Disclosures
Dr. Lenard Adler has received grant/research support from the following sources: Bristol-Myers Squibb, Shire, Chelsea Therapeutics, Eli Lilly and Company, and the National Institute of Drug Abuse. Dr. Adler has served on the advisory board for AstraZeneca Pharmaceuticals, Eli Lilly and Company, Major League Baseball, Mindsite, Shire, and i3 Research. Dr. Adler has consulted with Abbott Laboratories, AstraZeneca Pharmaceuticals, Cortex Pharmaceuticals, Cephalon, Eli Lilly and Company, Epi-Q, i3 Research, INC Research, Major League Baseball, Merck and Company, Mindsite, Novartis Pharmaceuticals, Organon, Ortho McNeil/Jannsen/Johnson and Johnson, Otsuka Pharmaceuticals, Psychogenics, Sanofi-Aventis Pharmaceuticals, Shire, and United Biosource. Dr. Adler has received royalty payments (as inventor) from the NYU School of Medicine for license of adult ADHD scales and training materials since 2004.
David Shaw has no financial interests or potential conflicts of interest to declare.
Dr. Thomas Spencer has received research support from the following sources: Shire Laboratories, Inc., Cephalon, Eli Lilly and Company, Glaxo-Smith Kline, Janssen, McNeil Pharmaceutical, Novartis Pharmaceuticals, Pfizer, and NIMH. Dr. Thomas Spencer has been a speaker or on a speaker's bureau for the following pharmaceutical companies: Shire Laboratories, Inc., Eli Lilly and Company, Glaxo-Smith Kline, Janssen, McNeil Pharmaceutical, and Novartis Pharmaceuticals. Dr. Thomas Spencer has been an advisor or on an advisory board for the following pharmaceutical companies: Shire Laboratories, Inc., Cephalon, Eli Lilly and Company, Glaxo-Smith Kline, Janssen, McNeil Pharmaceutical, Novartis Pharmaceuticals, and Pfizer.
Dr. Jeffrey Newcorn has received research support from the following sources: Eli Lilly and Company, Ortho-McNeil-Janssen, and Shire Laboratories, Inc. Dr. Newcorn has consulted with Eli Lilly and Company, Ortho-McNeil-Janssen, and Shire Laboratories, Inc. Dr. Newcorn has served on the advisory board for Eli Lilly and Company, Ortho-McNeil-Janssen, Schering-Plough, and Shire Laboratories, Inc.
Dr. Paul Hammerness has received research funds or participated in CME activities/professional talks supported by the following pharmaceutical companies: Abbott, Eli Lilly and Company, Forest, McNeil, and Shire. Dr. Hammerness has also received research funds from Elminda Ltd and has participated in speaker training and served on the advisory board for Shire. Dr. Hammerness has participated, as an investigator, in research studies funded by the following pharmaceutical companies: Abbott, Bristol Myers Squibb, Cephalon, Eli Lilly, Glaxo-SmithKline, Johnson and Johnson, McNeil, Merck, New River, Organon, Pfizer, Shire, and Takeda. Dr. Hammerness has also received honoraria from Reed Medical Education (a logistics collaborator for the MGH Psychiatry Academy). Commercial entities supporting the MGH Psychiatry Academy are listed on the Academy's website,
Dr. David Sitt has no financial interests or potential conflicts of interest to declare.
Dr. Ana Christina Minerly has no financial interests or potential conflicts of interest to declare.
Jennifer Davidow has no financial interests or potential conflicts of interest to declare.
Dr. Stephen Faraone has consulted with, has served the advisory board, or has been a speaker for the following sources: Shire, McNeil, Janssen, Novartis, Pfizer, Ortho-McNeil, and Eli Lilly and Company. Dr. Faraone has received research support from Eli Lilly and Company, Shire, Pfizer, and the National Institutes of Health.
