Abstract
Introduction
Currently, it is presumed that ADHD affects approximately 4.4% of U.S. adults (Kessler et al., 2006). Those with adult ADHD have increased rates of psychiatric comorbidities, workplace issues, and accidents with injuries (Biederman, Faraone, et al., 2006; de Graaf et al., 2008; Kupper et al., 2012; London & Landes, 2016). The Adult ADHD Investigator Symptom Rating Scale (AISRS) is a clinician-administered semistructured interview methodology developed to evaluate treatment responses (Spencer et al., 2010). The measure contains prompts and stem questions designed to capture the Diagnostic and Statistical Manual of Mental Disorders (DSM) symptoms of the disorder ADHD as they present in adulthood. Based on the responses from the patient, the administering clinicians rate the symptom severity as 0 = none, 1 = mild, 2 = moderate, or 3 = severe. The scale has been used and validated many times in a variety of clinical drug trials (Adler & Gorny, 2015; Adler et al., 2006; Adler, Zimmerman, Starr, Silber, Palumbo, Orman & Spencer 2009; Arnold, Feifel, Earl, Yang, & Adler, 2014; Goodman et al., 2017; Spencer et al., 2010).
The Adult ADHD Self-Report Scale (ASRS) Symptom Checklist is a self-report that presents the 18 DSM ADHD symptoms in adult context and rated on a frequency basis. It was developed by the World Health Organization (WHO) work group on adult ADHD, and comprises an 18-item Symptom Checklist (corresponding to the 18 symptoms found in the DSM) and a six-item screener (items extracted from the symptom checklist) to help identify adults at risk of ADHD (Kessler et al., 2005; Ustun et al., 2017). The Screener and the Symptom Checklist use a 5-point Likert-type scale to rate ADHD symptoms (0 = never, 1 = rarely, 2 = sometimes, 3 = often, and 4 = very often). Depending on the question, “sometimes,” “often,” or “very often” suggests clinical impairment for that specific item. The ASRS v1.1 Symptom Checklist is designed to provide clinicians with an inventory of adult ADHD symptoms as the first part of a diagnostic evaluation or to be used in monitoring treatment response (Adler, Shaw, & Alperin, 2015; Adler et al., 2006).
The ASRS and AISRS have both been expanded to include additional symptoms intended to assess executive function deficits (EFDs; nine items) and emotional dyscontrol (EC; four items). EFDs are deficiencies of high order cognitive processes, such as self-control, self-regulation, and ability to prioritize and to plan multiple tasks. EC includes symptoms of mood lability, irritability, and emotional overreactivity. These additional items were developed by the WHO adult ADHD work group by examining other scales that measure ADHD non-DSM symptoms, including the Brown Attention Deficit Disorder Scale, the Conners’ Adult ADHD Rating Scale (CAARS)–Self-Report: Long Version (Brown, 1996; Conners, Erhardt, & Sparrow, 1998), and the Utah Criteria (Wender, 1998). As with the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association [APA], 2013) items, specific prompts have been written for the AISRS to help guide the rater to explore the full manifestations of symptoms in an adult with ADHD.
There is much disagreement in the literature if EFD/EC should be viewed as core symptoms of adult ADHD, associated features, or merely the result of co-traveling symptoms (Adler et al., 2017; Barkley & Murphy, 2010; Biederman et al., 2015; Biederman, Petty, et al., 2006; Brown, 2007; Corbisiero, Morstedt, Bitto, & Stieglitz, 2017; Faraone, 2000; Kessler et al., 2010). Nevertheless, as both EFD and EC are associated with adult ADHD, these symptoms are often measured in adult ADHD treatment studies. The effects of medication treatment in adults with ADHD in addressing symptoms of EFDs and EC have been examined using a variety of measures (Adler et al., 2013; Biederman et al., 2017; Brown et al., 2011; Marchant, Reimherr, Halls, Williams, & Strong, 2010; Reimherr et al., 2007; Rosler et al., 2010). We now present data as to the psychometrics of the expanded EFDs and EC portions of the AISRS and ASRS v1.1 Symptom Checklist as prior validations focused on the DSM portions of the scales (Adler et al., 2006; Spencer et al., 2010). We also present pilot normative data for both scales, which could be quite useful in classifying patient and participant scores when examining treatment response and in diagnosing adult ADHD.
Method
The present article presents secondary analysis of data from a larger study to update and validate the ASRS Screener for DSM-5 (Ustun et al., 2017).
Participants
Two samples were pooled together: (a) a referred sample of adults recruited as part of the New York University Adult ADHD Program and (b) patients screened for ADHD at a primary care physician (PCP) practice in the New York area.
Referred sample
The referred sample was composed of adults (aged 18-54, inclusive) recruited as part of the New York University Adult ADHD Program (Adler et al., 2017; Ustun et al., 2017). These participants were administered the Structured Clinical Interview for DSM-IV-TR Axis I Disorders (SCID), the ASRS Symptom Checklist–Expanded Version to measure self-reported ADHD symptoms, and the Adult ADHD Clinical Diagnostic Scale (ACDS) v1.2 to assess DSM-5 adult ADHD. A subset of participants (n = 85) were also administered the AISRS-expanded version. Participants self-reported medical history, psychiatric history, and demographics.
PCP practice sample
This sample was composed of adults (18-60 years, inclusive) who were screened negative ADHD at a PCP practice in the New York area affiliated with New York University Langone Medical Center (see Silverstein, Alperin, Faraone, Kessler, & Adler, 2017, for a full description of recruitment and assessment methods).
After the initial screening at the PCP waiting room, ADHD screen negative participants were contacted by trained research staff between 0 and 21 days after initial screening. These phone assessments included the Mini International Neuropsychiatric Interview 7.0 (MINI) to screen for DSM-5 psychiatric comorbidities, the ASRS Symptom Checklist–Expanded Version to measure self-reported ADHD symptoms, and the ACDS v1.2 to confirm lack of current DSM-5 adult ADHD diagnosis. Demographics, medical, and psychiatric histories were also collected.
All conduct of both studies was approved by the Institutional Review Board Associates (IRBA) of New York University School of Medicine.
Rating Scales
In this study, adult ADHD was evaluated with ACDS v1.2 (Adler & Cohen, 2004; Adler et al., 2015), a semistructured diagnostic interview that is widely used in adult ADHD studies (Fayyad et al., 2017; Houston et al., 2011; Spencer et al., 2001; Spencer et al., 1998; Spencer et al., 1995; Ustun et al., 2017). The ACDS v1.2 interview begins with a retrospective assessment of childhood symptoms of ADHD and then an expanded set of recent (past year, following DSM-5) symptoms, including all DSM-5 Criterion A1 and A2 symptoms.
The AISRS is described above. The adult portion of the ACDS v1.2 and the AISRS has the same prompts and structure, except that the AISRS asks for a retrospective reporting of the 2 weeks and the ACDS v1.2 past year.
The ACDS v1.2, AISRS, and ASRS all have expanded versions (with prompts for the ACDS v1.2 and AISRS) that consist of 13 non-DSM symptoms of EFDs and EC noted above. For both samples, raters for the clinical assessment were trained per standard rater training procedures prior to initiation of study (Adler et al., 2005).
Psychiatric comorbidities for the referred sample were evaluated using the SCID and with the MINI for the PCP sample (First, Spitzer, Gibbon, & Williams, 2002; Sheehan et al., 1998). The SCID is a widely used clinical interview used to measure psychiatric comorbidities and has been used for many adult ADHD studies (e.g., Adler et al., 2008; Kessler et al., 2004; Oddo, Knouse, Surman, & Safren, 2016). The MINI is a short-structured clinical interview used to screen for DSM-5 psychiatric comorbidities. It has been well validated and is widely used to evaluate for psychiatric comorbidities in research studies, including adult ADHD studies (Bradley et al., 2017; Spencer et al., 2006; Yoshimasu et al., 2016).
All of the evaluations for the referred sample were conducted in person; all the assessments for the PCP sample that are relevant for the present analysis PCP sample were conducted via telephone. Referred sample interviews were conducted by two clinical psychology trainees (a PhD candidate with an MA and an MA candidate with a BA), and the PCP sample interviews were administered by two research coordinators with experience in adult ADHD (one with a BA and one with a BS). The clinical interviewers for both samples were trained by one of the investigators (L.A.A.). Validity of the referred sample interviews was established by regular direct observation of interviews. To prevent rater drift, calibration meetings were held between the coordinators, and consistent ratings and validity of these interviews were established by meetings with one of the investigators (L.A.A.).
Data Analysis
All statistical analyses were conducted using SPSS (Version 22). For participants who only had an ACDS v1.2 and did not have an AISRS (n = 214), AISRS scores were extracted from the ACDS ratings. The AISRS and ACDS v1.2 have the same prompts and 4-point rating scale (“none/never,” “mild,” “moderate,” and “severe,” with the latter two being the threshold for clinical impairment for each item for both scales). Therefore, no transformations of the ACDS v1.2 ratings to extract AISRS scores for each participant were required. Past analysis on the present data set has demonstrated the statistical rationalization (Silverstein, Faraone, et al., 2017), and Cohen’s Kappa’s examination of item-by-item agreement demonstrates high agreement between AISRS and ACDS scores. All subsequent discussion of AISRS data will refer to the combination of the AISRS scores obtained from the actual scale and the extracted scores from the ACDS v1.2.
Internal consistencies for the ASRS and AISRS were calculated with Cronbach’s alpha. To further evaluate the psychometrics of the additional EFDs and EC items, item-total correlations were also calculated for the items of the ASRS and AISRS EFDs and EC subscales. Item-total correlations refer to the correlation between that item and the other items of the scale when that item is excluded.
To create normative data for this sample, histograms of ADHD controls for ASRS 31-item score (nine inattentive [IA], nine hyperactive–impulsive [H-I], four EC, and nine EFD items) and AISRS total DSM 18-item (nine IA and nine H-I) score were plotted. In addition, to create a normative data representative of the adult population with and without adult ADHD (~4.4%), six adult ADHD cases were randomly selected (three males and three females). Histograms of this pilot nationally representative sample of ASRS 31-item scores and AISRS 18-item scores were plotted.
All tests were two-tailed and used a significance level of p < .05, unless indicated.
Results
Demographic Characteristics of the Samples
Data from 297 participants were analyzed for this study. The mean age of the total sample was 32.0 ± SD 10.5 years, 44.5% (n = 133) male and 55.5% (n = 166) female; 57.2% (n = 171) of the sample met criteria for current adult ADHD, 11.7% met criteria for current Generalized Anxiety Disorder, and 35.8% met criteria for past Major Depressive Disorder (see Silverstein, Faraone, et al., 2017, for full presentation of demographics of the sample).
Internal Consistency
Cronbach’s alpha for the ASRS and AISRS DSM 18-item scales and individual subscales ranged from .84 to .96 and .85 to .97, respectively (Table 1). These remained high controlling for age and gender (Table 1). Item-total correlations for both scales’ EFD items ranged from .62 to .89 (Table 2), for EC from .63 to .72 (Table 2).
Internal Consistency of ASRS and AISRS Total 18-Item Scale and Subscale.
Note. ASRS = Adult ADHD Self-Report Scale; AISRS = Adult ADHD Investigator Symptom Rating Scale; DSM-5 = Diagnostic and Statistical Manual of Mental Disorders (5th ed.; American Psychiatric Association, 2013); IA = inattentive; H-I = hyperactive–impulsive; EFDs= executive function deficits; EC = emotional dyscontrol.
Item-Total Correlations for the EC and EFD items of the ASRS and AISRS.
Note. ASRS = Adult ADHD Self-Report Scale; AISRS = Adult ADHD Investigator Symptom Rating Scale. EC = emotional dyscontrol; EFDs= executive function deficits.
The ASRS and AISRS have the same symptom domains. The symptom domains, and not the actual questions or prompts, have been printed above.
Indicates EC item.
Indicates EFD item.
ASRS and AISRS Pilot Normative Data
ASRS and AISRS cutoff score
Examining the histograms of ASRS and AISRS scores for ADHD controls, 95% cutoff scores were 70 and 23, respectively (Figures 1 and 2). When randomly selecting six ADHD+ cases (three males and three females) to create a pilot sample of adult ADHD in the general population (~5% ADHD prevalence rate in our sample), ASRS 95% cutoff score was 82. Using this pilot sample method, AISRS cutoff score was between 25 and 27 (between 94.8 and 95.5 cumulative %), and therefore presumed to be 26 (Figures 3 and 4).
Frequency of ASRS 31-item total score for ADHD(–) controls.
Frequency of AISRS 18-item total score for ADHD(–) controls.
Frequency of ASRS 31-item total score for nationally representative sample.
Frequency of AISRS 18-item total score for nationally representative sample.
Adult ADHD controls with high AISRS scores
Seven adult ADHD controls, determined by the ACDS v1.2 to be ADHD negative, had AISRS scores ≥23 (range = 23-30). Of note, two patients (both of whom had AISRS score = 23) had insufficient levels of impairment due to their adult ADHD symptoms noted on their ACDS v1.2 evaluation, and therefore failed to meet criteria for adult ADHD. Six of the patients had psychiatric conditions (depression or Generalized Anxiety Disorder) that could explain their impairment, and four failed to meet childhood onset of significant symptoms criterion (therefore failing to meet DSM-5 Criterion A1).
Discussion and Conclusion
Our analyses demonstrate that the ASRS and AISRS 31-item expanded versions have high validity in their assessment of 18 core DSM-5 adult ADHD symptoms, EFD, and EC symptoms. In addition, the expanded versions have good psychometric properties in the assessment of EFDs and EC. Pilot data suggest that AISRS 18-item DSM IA and H-I scores ranging from 23 to 26 have a high likelihood of adult ADHD. The AISRS cutoff score finding is consistent with clinical trials generally requiring an AISRS 18-item total score of at least 24 to 26 (Adler et al., 2011; Biederman, Mick, Spencer, Surman, & Faraone, 2012; Goodman et al., 2017). Our pilot data also show that patients with an ASRS 31-item total score ranging from 70 to 82 have a high likelihood of adult ADHD.
The Cronbach’s alpha for EC items was lower than EFD items, which suggests that EC is a less homogeneous and, perhaps, less reliable construct than EFD. The lower alpha for EC items is consistent with previous findings of EC symptoms having less robust factor loadings, and that EC may be a somewhat less robust core feature of adult ADHD, as compared with EFDs (Adler et al., 2017).
It is interesting that the AISRS cutoff score for high likelihood of adult ADHD is 24 because using DSM-5 adult ADHD criteria it is possible for an adult with ADHD who only meets criteria for 5 IA or H-I symptoms of moderate severity to have a score of 10. It is possible that symptom loading with few moderate symptoms is not sufficient to create enough impairment (DSM-5 Criterion D) to make the diagnosis. This may be a result of 97% of the ADHD+ sample presenting with at least seven IA or H-I symptoms of at least moderate severity. Or, it is possible that most adults with ADHD do not have only five or six IA or H-I symptoms of moderate severity. Nevertheless, the present findings underscore the importance of considering general impairment due to ADHD (Criterion D) beyond merely looking at individual impairing symptoms when evaluating for adult ADHD.
Recently, there has been an increased demand and subsequent attempts in the literature to refine diagnostic evidence-based procedures for adult ADHD (Martel, Nigg, & Schimmack, 2017; Sibley, Mitchell, & Becker, 2016). The design of clear and simple methods for diagnosis (e.g., cutoff scores on a scale) is especially important considering that many providers, especially those in primary care, lack confidence in evaluation and diagnosis of the disorder (Adler, Shaw, Sitt, Maya, & Morrill, 2009; Faraone, Spencer, Montano, & Biederman, 2004; Goodman, Surman, Scherer, Salinas, & Brown, 2012). The ASRS cutoff score can be particularly useful for those less confident in their clinical evaluation or clinical ratings of their patient’s symptoms.
Nonetheless, there are numerous methodological concerns to consider when both establishing cutoffs and using self-report or clinician-rated scales for adult ADHD. It is extremely important to note that using cutoff scores does not mean that the patient meets DSM-5 criteria for adult ADHD, as neither the ASRS nor AISRS evaluate the DSM requirement impairment criteria. In addition, including only a referred sample (which generally serves as the source of many patients when evaluating scale psychometrics) as the sample set to establish the psychometric validity reduces the generalizability of findings as it may result in a biased sample. For example, referred samples are unlikely to include those who are less aware of their symptoms or less classic ADHD presentations that other providers do not identify (see Sibley et al., 2016). Many studies do not include randomly selected participants (and not just referred sample). The present analysis is less likely to be fraught with the issue of missing those who are unaware that they have adult ADHD as the analysis included a large randomly selected PCP sample of ADHD negatives recruited from a PCP practice.
An additional limit of selecting a cutoff score is that it does not consider the influence of overlapping symptoms. For example, there may be overlap between the symptoms of inattention and EFD as evaluated by the AISRS (e.g., conflation of DSM symptom of losing things and EFD symptoms of organization). Using a cutoff score may give more weight to these symptoms. This may be particularly concerning when evaluating atypical cases of adult ADHD. These concerns are why we have suggested elsewhere (e.g., Adler et al., 2008; Faraone, 2000; Kessler et al., 2005) that clinical evaluations of adult ADHD should consider multiple sources of information when establishing a diagnosis. We therefore believe that the present study evidences that cutoff scores can be an additional source of information to increase clinician confidence and diagnostic accuracy, but stress that cutoff scores should not be solely used as a single factor in making the diagnosis.
Limitations
Although the sample was robust, investigator symptom ratings were, as noted above, obtained from a combination of AISRS and ACDS v1.2 ratings, which was psychometrically valid. The noted ASRS 31-item expanded cutoff range predictive of adult ADHD of 70 to 82 was somewhat large; the noted variability in EC ratings may have contributed to this larger range. In addition, the clinical interviews did not include informant reports, which could have improved diagnostic accuracy (Adler et al., 2006; Martel et al., 2017; Molina & Sibley, 2014; Sibley et al., 2016; Zucker, Morris, Ingram, Morris, & Bakeman, 2002). Finally, the psychometrics of the expanded scales to evaluate EFD and EC symptoms did not include neuropsychological tests to validate findings. Nevertheless, data support higher yield of symptom recognition via EFD symptom scales in ADHD than neuropsychological test (Biederman et al., 2008).
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Samuel Alperin, Ms. Terry Leon, and Mr. Michael Silverstein have no conflicts to disclose. In the past 5 years, Dr. Faraone received income, potential income, travel expenses, continuing education support, and/or research support from Otsuka, Lundbeck, Rhodes, Arbor, KenPharm, Ironshore, Shire, Akili Interactive Labs, CogCubed, Alcobra, VAYA, Sunovion, Genomind, Neurolifesciences, Neurovance, Alcobra, Otsuka, and Pfizer. Dr. Joseph Biederman is currently receiving research support from the following sources: American Academy of Child and Adolescent Psychiatry (AACAP); The Department of Defense; Food & Drug Administration; Headspace; Lundbeck; Neurocentria, Inc.; National Institute on Drug Abuse (NIDA); PamLab; Pfizer; Shire Pharmaceuticals, Inc.; Sunovion; and National Institutes of Health (NIH). Dr. Biederman has a financial interest in Avekshan LLC, a company that develops treatments for ADHD. His interests were reviewed and are managed by Massachusetts General Hospital (MGH) and Partners HealthCare in accordance with their conflict of interest policies. Dr. Biederman’s program has received departmental royalties from a copyrighted rating scale used for ADHD diagnoses, paid by Ingenix, Prophase, Shire, Bracket Global, Sunovion, and Theravance; these royalties were paid to the Department of Psychiatry at MGH. In 2017, Dr. Biederman is a consultant for Aevi Genomics, Akili, Guidepoint, Ironshore, Medgenics, and Piper Jaffray. He is on the scientific advisory board for Alcobra and Shire. He received honoraria from the MGH Psychiatry Academy for tuition-funded Continuing medical education (CME) courses. Through MGH corporate licensing, he has a U.S. Patent (#14/027,676) for a nonstimulant treatment for ADHD and a patent pending (#61/233,686) on a method to prevent stimulant abuse. In 2016, Dr. Biederman received honoraria from the MGH Psychiatry Academy for tuition-funded CME courses, and from Alcobra and The American Professional Society of ADHD and Related Disorders (APSARD). He was on the scientific advisory board for Arbor Pharmaceuticals. He was a consultant for Akili and Medgenics. He received research support from Merck and Sertraline Pediatric Registry for the Evaluation of Safety (SPRITES). In 2015, Dr. Biederman received honoraria from the MGH Psychiatry Academy for tuition-funded CME courses and from Avekshan. He received research support from Ironshore; Magceutics, Inc.; and Vaya Pharma/Enzymotec. In 2014, Dr. Biederman received honoraria from the MGH Psychiatry Academy for tuition-funded CME courses. He received research support from AACAP, Alcobra, Forest Research Institute, and Shire Pharmaceuticals, Inc. In previous years, Dr. Biederman received research support, consultation fees, or speaker’s fees for/from the following additional sources: Abbott; Alza; APSARD; AstraZeneca; Boston University; Bristol Myers Squibb; Cambridge University Press; Celltech; Cephalon; The Children’s Hospital of Southwest Florida/Lee Memorial Health System; Cipher Pharmaceuticals, Inc.; Eli Lilly and Co.; Esai; ElMindA; Fundacion Areces (Spain); Forest; Fundación Dr. Manuel Camelo A.C.; Glaxo; Gliatech; Hastings Center; Janssen; Juste Pharmaceutical Spain; McNeil; Medice Pharmaceuticals (Germany); Merck; MGH Psychiatry Academy; MMC Pediatric; National Alliance for Research on Schizophrenia and Depression (NARSAD); NIDA; New River; National Institute of Child Health and Human Development (NICHD); National Institute of Mental Health (NIMH); Novartis; Noven; Neurosearch; Organon; Otsuka; Pfizer; Pharmacia; Phase V Communications; Physicians Academy; The Prechter Foundation; Quantia Communications; Reed Exhibitions; Shionogi Pharma, Inc.; Shire; the Spanish Child Psychiatry Association; The Stanley Foundation; UCB Pharma, Inc.; Veritas; and Wyeth. Dr. Thomas Spencer received research support or was a consultant from the following sources: Alcobra; Avekshan; Cephalon; Eli Lilly & Company; Enzymotec Ltd.; Heptares; Impax; Ironshore; Janssen; Lundbeck; McNeil Pharmaceutical; Novartis; Shire Laboratories, Inc.; Sunovion; VayaPharma; the Food and Drug Administration (FDA), and the Department of Defense. Consultant fees are paid to the MGH Clinical Trials Network and not directly to Dr. Spencer. Dr. Thomas Spencer has been on an advisory board for the following pharmaceutical companies: Alcobra. Dr. Spencer received research support from Royalties and Licensing fees on copyrighted ADHD scales through MGH Corporate Sponsored Research and Licensing. Through MGH corporate licensing, Dr. Spencer has a U.S. Patent (#14/027,676) for a nonstimulant treatment for ADHD and a patent pending (#61/233,686) for a method to prevent stimulant abuse. In the past 3 years, Dr. Lenard Adler has received income, potential income, travel expenses, support for serving on the advisory board on ADHD, consulting support, payment for development of educational presentations, grant support, and/or research support from Cortex/Respire Pharmaceuticals, Sunovion Pharmaceuticals, Enzymotec, Shire Pharmaceuticals, National Football League, Major League Baseball, Shire Pharmaceuticals, Alcobra Pharmaceuticals, Lundbeck, The State University of New York Upstate, APSARD, and Purdue. He also has received royalty payments (as inventor) from New York University for the license of adult ADHD scales and training materials since 2004.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for the current analyses from DataStat, Inc., Ann Arbor, MI and New York University School of Medicine Office of Industrial Liaison.
