Abstract
The evidence-based assessment of attention-deficit/hyperactivity disorder (ADHD) depends on adherence to Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5) diagnostic criteria and reliance on multi-method/multi-informant data. Although nearly all psychologists endorse these practices, college students with ADHD may lack documentation supporting their diagnoses. We reviewed the documentation submitted by 214 undergraduates diagnosed with ADHD and receiving academic accommodations for this condition. Their clinicians also completed a checklist that described their assessment procedures. Relatively few psychologists assessed all DSM-5 criteria, based on either the psychologist’s self-reported assessment procedures (23.4%), written documentation (14.0%), or multi-method/multi-informant data (10.3%) such as educational/medical records, results of rating scales, or interviews with other informants. Psychologists were least likely to assess students’ areas of impairment or to rule out alternative causes for students’ self-reported symptoms. This lack of adherence to DSM-5 criteria and overreliance on students’ self-reports can threaten the reliability of diagnosis and the appropriateness of medication and accommodations that follow.
Keywords
Attention-deficit/hyperactivity disorder (ADHD) is a serious, neurodevelopmental condition that affects approximately 5% of postsecondary students (Ramsay & Rostain, 2015). In primary and secondary school, students with ADHD are more likely to earn lower grades, fail a course, or be referred to special education or remedial tutoring than their classmates (Barkley, 2015; Frazier, Youngstrom, Glutting, & Watkins, 2007). In college, they report more problems with studying, test-taking, and time management than their peers (DuPaul, Pinho, Pollack, Gormley, & Laracy, 2017; Roberts, Milich, & Barkley, 2015). College students with ADHD tend to have lower grade point averages (GPAs) and are less likely to complete their degrees than either their typically developing classmates or students with many other psychiatric disorders (Gormley, DuPaul, Weyandt, & Anastopoulos, 2016; Weyandt et al., 2013). Perhaps unsurprisingly, college students with ADHD also endorse more problems with anxiety, depression, motivation, and academic self-efficacy than other students (Barkley, 2015; Ramsay & Rostain, 2015).
Medication is a primary, evidence-based treatment for adults with ADHD. Psychostimulants, in particular, are efficacious in reducing core symptoms of the disorder and improving academic functioning (Prince, Wilens, Spencer, & Biederman, 2015). However, these medications have high abuse potential and are candidates for misuse or diversion. Approximately 17% of college students without ADHD use psychostimulants for nonmedical purposes, usually to increase attention and to improve academic performance. Although medication is typically acquired from peers with valid prescriptions (Benson, Flory, Humphreys, & Lee, 2015), approximately 10% of college students seeking an ADHD diagnosis misreport symptoms to obtain medication (Musso & Gouvier, 2014).
Students with ADHD may also receive academic accommodations under the auspices of Section 504 of the Rehabilitation Act (1973) and the Americans With Disabilities Act Amendments Act (ADAAA; 2008). New regulations to Titles II and III of the ADAAA identify ADHD as a potentially disabling condition that can merit accommodations in college (U.S. Department of Justice, 2016). The goal of accommodations is to mitigate the effects of ADHD symptoms on major life activities. Accommodations allow students with disabilities to learn, and to demonstrate their learning, in a manner similar to their classmates without disabilities. The most common accommodations are extended time on exams and testing in a separate, distraction-reduced setting (Lovett & Lewandowski, 2015). Although both disability directors and students regard these accommodations as helpful, there is little empirical data showing that they are effective and specific for students with ADHD (Gordon, Lewandowski, & Lovett, 2015; Lewandowski, Lambert, Lovett, Panahon, & Sytsma, 2014). For example, students given additional time on exams tend to outperform their classmates who complete exams under standard conditions, regardless of their disability status. Furthermore, students without disabilities seem to benefit more from extra time than their classmates with ADHD and other disabling conditions (Gregg & Nelson, 2012; Lewandowski, Lovett, Parolin, Gordon, & Codding, 2007; Lewandowski, Lovett, & Rogers, 2008).
The accurate identification of ADHD in postsecondary students is important; classification errors can have significant, real-world consequences for students, classmates, and colleges (Lindstrom & Lindstrom, 2017). False-negative errors limit students’ access to efficacious medications and risk discrimination by denying students equal access to higher education. False-positive errors, however, may lead to the inappropriate provision of medication and accommodations to students without disabilities, trivialize the experiences of students with actual ADHD, diminish university resources, and erode academic standards by providing some students an unfair advantage over their classmates (Gordon et al., 2015; Lindstrom, Nelson, & Foels, 2015).
Evidence-Based ADHD Assessment
The Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association [APA], 2013) describes the essential features of ADHD. Several guidelines have been developed to establish best practices in the assessment of ADHD in children and adolescents (American Academy of Pediatrics, 2011; National Association of School Psychologists, 2012; Pliszka & AACAP Work Group, 2007), and college students in particular (Gordon et al., 2015; National Resource Center on ADHD, 2015; Ramsay & Rostain, 2015). These guidelines emphasize adherence to DSM-5 criteria and reliance on multiple methods of information gathered from multiple informants. Although a diagnostic interview with the student remains the cornerstone of evidence-based assessment, five other methods are widely accepted to supplement students’ self-reported data.
First, individuals with ADHD must experience significant symptoms (Criterion A; APA, 2013). Most professional guidelines recommend the use of norm-referenced rating scales to determine the presence of significant symptoms that are inconsistent with students’ developmental level (Adler, Shaw, & Alperin, 2015). Elevated scores on these rating scales, usually exceeding the 93rd percentile for age and gender, can establish developmentally atypical symptom presentation in adults (Roberts et al., 2015).
Second, DSM-5 recommends that clinicians rely on ancillary information, rather than students’ self-reports, to establish symptom onset prior to age 12 (Criterion B; APA, 2013). Previous research has shown that young adults’ recollections of childhood ADHD symptoms are prone to error (Miller, Newcorn, & Halperin, 2010). For example, Mannuzza, Klein, Klein, Bessler, and Shrout (2002) showed the inaccuracy of young adults’ recall of childhood ADHD symptoms. Only 78% of adults with well-documented histories of ADHD recalled significant symptoms in childhood, whereas 11% of adults without histories of ADHD reported significant childhood symptoms. More recently, Sibley and colleagues (Sibley, Pelham, Molina, Gnagy, Waschbusch, et al., 2012; Sibley, Pelham, Molina, Gnagy, Waxmonsky, et al., 2012) found modest correlations between young adults’ recollection of their ADHD symptoms in childhood and their actual childhood symptoms as reported by others. Adults with well-documented histories of ADHD tended to underreport childhood symptoms whereas adults without histories of ADHD tended to overreport childhood symptoms (K. R. Murphy, Gordon, & Barkley, 2002; P. Murphy & Schachar, 2000). Consequently, clinicians should corroborate adults’ recollections of childhood symptoms with other sources of data such as school records showing a history of academic or behavioral problems, medical records showing a history of pharmacotherapy, or an interview with parents about the students’ developmental history (Gibbins & Weiss, 2007; K. R. Murphy & Gordon, 2006; Ramsay, 2015).
Third, DSM-5 directs clinicians to gather data from multiple informants to establish the presence of symptoms across settings (Criterion C; APA, 2013). For example, clinicians may interview or administer rating scales to teachers, roommates, or employers who are familiar with students’ behavior in academic, social, and occupational settings, respectively.
Fourth, DSM-5 requires evidence that students’ ADHD symptoms limit their everyday functioning (Criterion D; APA, 2013). Although symptom severity and degree of impairment may seem synonymous, they are distinct constructs that are only moderately correlated (Lovett, Gordon, & Lewandowski, 2016). Studies investigating the association between symptom severity and impairment in adults with ADHD have yielded median bivariate correlations ranging from .25 for individual items to .70 for composite measures. Overall, symptom severity explains less than one half of the variance of impairment in adults with this condition (Lewandowski, Lovett, & Gordon, 2016). Disability counselors also recognize discrepancies between symptom severity and impairment in college students with ADHD (D’Alessio & Banerjee, 2016; Ofiesh, Moniz, & Bisagno, 2015). Some students experience significant symptoms of inattention and hyperactivity-impulsivity, but use compensatory strategies to function effectively at school. Other students with ADHD experience limitations in certain classes, but not others, depending on the demands these classes place on attention, concentration, and inhibition. Still other students fall short of the number of symptoms required for an ADHD diagnosis, but experience substantial limitations in their academic achievement or attainment, nonetheless. Whereas many students with ADHD symptoms struggle to complete coursework, manage a family, or maintain employment, others show fewer problems with school, relationships, or work (Gathje, Lewandowski, & Gordon, 2008). The assessment of impairment, independent of symptom count or severity, is therefore essential to the DSM-5 conceptualization of ADHD.
Finally, DSM-5 requires clinicians to rule out alternative causes for students’ ADHD symptoms (APA, 2013). Alternative causes include other psychiatric disorders, physical illnesses, and the effects of medications and other substances (Post & Kurlansik, 2013). Similarly, students with inadequate academic skills or low motivation to pursue postsecondary education may experience problems with attention, concentration, and academic functioning that resemble ADHD (Diller, 2010). Finally, some students may intentionally feign symptoms to obtain medication or accommodations (Cook et al., 2018). These alternative explanations must be ruled out prior to assigning a diagnosis (Criterion E).
Clinician’s Adherence to Best Practices
Studies investigating the degree to which clinicians adhere to DSM-5 criteria and multi-method/multi-informant assessment have yielded mixed results. Survey studies suggest that most clinicians follow DSM standards. In an early study, Handler and DuPaul (2005) surveyed 230 psychologists about their typical diagnostic methods when assessing ADHD in children. Approximately 93% of psychologists reported using Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; APA, 1994) criteria frequently or very often. Furthermore, most reported interviewing parents (93%) and children (92%), administering norm-referenced rating scales (91%), gathering information from multiple informants (82%), and obtaining ancillary information about children’s academic and behavioral functioning (95%).
More recently, Ogg and colleagues (2013) surveyed 217 school psychologists regarding their typical ADHD assessment practices for preschool and school-age children. On average, respondents reported that they “strictly” adhered to DSM-IV diagnostic criteria and relied on multi-method/multi-informant assessment approximately 80% of the time. Furthermore, school psychologists rated the practice of administering ADHD rating scales, gathering information about impairment at home and school, and ruling out alternative explanations for children’s symptoms prior to assigning a diagnosis as “important” (rating four out of five) when assessing children suspected of the disorder.
In contrast to the results of these surveys, studies examining clinicians’ assessment reports suggest a lack of adherence to DSM criteria and reliance on multi-method/multi-informant data. Notably, these studies involved postsecondary students rather than children.
In the first study, researchers examined the reports submitted by 50 medical students seeking accommodations on a licensing exam (Joy, Julius, Akter, & Baron, 2010). Only 14% of reports provided sufficient data meeting DSM-IV criteria for ADHD. Although most (58%) reports described areas of impairment, relatively few provided evidence of significant symptoms (33%), established childhood onset (28%), or documented symptoms across settings (28%). Few clinicians (16%) indicated that they ruled out alternative causes, despite the fact that most students experienced comorbid learning problems (50%), anxiety (47%), or depression (10%).
In the second study, researchers reviewed 100 psychological reports submitted by college students seeking academic accommodations (Nelson, Whipple, Lindstrom, & Foels, 2014). In contrast to the previous study, the researchers only included students with an ADHD diagnosis and excluded students with comorbid learning disabilities. Only 1% of reports provided sufficient evidence meeting all DSM-5 criteria. Although 84% of psychologists documented significant ADHD symptoms, only 59% described current impairment and 55% established childhood onset. Few psychologists documented symptoms in two or more settings (28%) or indicated that they ruled out alternative causes (14%). Although nearly all clinicians (98%) interviewed students, only 69% provided evidence that they gathered information from other informants, 55% provided the results of rating scales, and 32% reviewed students’ academic/medical records.
The Current Study
Previous research, therefore, provides conflicting data regarding the degree to which clinicians adhere to DSM criteria and adopt best practices when assessing ADHD. On the surface, there seems to be a discrepancy between clinicians’ self-reports and their actual assessment practices. However, this discrepancy could be due to a deficiency in documentation rather than to a lack of adherence to diagnostic criteria (Nelson et al., 2014). It is possible that psychologists’ self-reports are accurate; clinicians may assess all diagnostic criteria but simply fail to document their methods and findings in their reports. If so, psychologists need to do a better job “showing their work” when assessing postsecondary students. However, it is also possible that some psychologists might neglect to assess all diagnostic criteria or assign an ADHD label in the absence of supporting evidence. Such practices could reduce the reliability and validity of ADHD diagnoses, contradict principles of evidence-based assessment, and provide students with unwarranted access to medication and accommodations (Gordon et al., 2015).
The purpose of our study was to replicate and extend previous research examining clinicians’ adherence to diagnostic standards. We examined the psychological reports submitted by undergraduates to the disability offices of two private colleges. In contrast to previous research, all students had been assigned the primary diagnosis of ADHD in their reports and all were already receiving academic accommodations.
We also reviewed other documentation submitted by psychologists with their reports. Supporting documentation included students’ educational, medical, and previous psychological records. Such supporting evidence, although not part of the report itself, can be helpful to establish childhood onset, previous or current impairment, or a history of accommodations.
Finally, each clinician completed a checklist that described his or her assessment practices. Even if the clinician failed to document a specific method, informant, or criterion in the report, he or she could indicate its presence on the checklist.
We hoped that our study would allow us to resolve the apparent discrepancy between clinicians’ self-reported ADHD assessment practices and data in their professional reports. Whereas a lack of documentation of clinicians’ assessment practices might suggest the need for greater care in report writing, a lack of adherence to diagnostic standards would indicate more serious deviation from evidence-based practice.
Method
Participants
Participants (N = 214) were full-time undergraduates, aged 17 to 22 years, attending one of two private colleges. Approximately 55.6% were men. Ethnicities included White (88.3%), African American (4.7%), Latino (3.3%), and Other (3.7%). Students’ most recent diagnoses were ADHD, inattentive type/presentation (43.0%), ADHD, combined type/presentation (27.1%), ADHD, not otherwise specified (17.7%), “attention deficit disorder” (8.4%), and ADHD, Hyperactive-Impulsive Type/Presentation (1.4%). Approximately 19% of students were first diagnosed with ADHD in preschool or primary school, 26% were first diagnosed in secondary school, and 55% had not been diagnosed with ADHD until after beginning college. Approximately 88% of students had been prescribed medication for ADHD and 80% were currently taking medication. Secondary diagnoses included learning disability/disorder (31.8%), anxiety disorder (23.4%), depressive disorder (11.7%), and communication disorder (4.7%).
Reports were completed by psychologists (78%) or school psychologists (22%) not affiliated with the colleges. Approximately 58.9% of clinicians were women. Degrees included PhD (63.6%), PsyD (15.4%), EdS (14.5%), MA/MS/MEd (5.1%), and EdD (1.4%). Clinicians worked in private practice, clinics, hospitals, or universities (79%) or public secondary schools (21%). Years of professional experience included ≤5 years (11.7%), 6 to 10 years (49.5%), 11 to 15 years (30.4%), and ≥16 years (8.4%). A different clinician evaluated each student. If the same clinician evaluated more than one student, only the student evaluated most recently was included in the study.
Reports were submitted to the colleges during the previous four academic years and written between the years 2011 and 2017. Most reports (86%) were written within the previous 3 years with the remainder written within the previous 4 (5%) or 5 (9%) years, as required by the colleges. Most reports were written after publication of DSM-5 and included a DSM-5 diagnosis (67.3%).
Preliminary contingency analyses indicated that the school psychologists in our study (74.5%) were more likely than the other psychologists (54.5%) to be women, χ2(1) = 6.05, p = .014, Φ = .17. School psychologists (91.5%) were also more likely than other psychologists (1.2%) to be working in a public school setting, χ2(1) = 180.06, p < .001, Φ = .91. In contrast, psychologists (100%) were more likely than school psychologists (10.6%) to hold a doctoral degree, χ2(1) = 185.68, p < .001, Φ = .93. The reports of psychologists (91.0%) tended to be more recent (i.e., written in the previous 3 years) than the reports of school psychologists (66.0%), χ2(1) = 18.60, p < .001, Φ = .29. Consequently, psychologists (71.9%) were more likely than school psychologists (51.1%) to assign a DSM-5 (rather than DSM-IV) diagnosis, χ2(1) = 7.20, p = .007, Φ = .18.
Procedure
All students had participated in a psychological evaluation, had been assigned the primary diagnosis of ADHD, and had submitted this documentation to the disability office of their college. Students were also receiving academic accommodations for limitations associated with their ADHD diagnosis.
Disability specialists provided redacted reports and supporting documents for each student. Supporting documents included educational records (e.g., 504 Plan, Individualized Education Program [IEP], Summary of Performance [SOP]), medical records (e.g., history of pharmacotherapy), documentation of the provision of previous accommodations (e.g., letters from teachers/testing agencies), and the results of previous evaluations.
Disability specialists also required each clinician to complete a checklist that described his or her assessment practices. The checklist consisted of the five DSM-IV/5 criteria for ADHD and various assessment procedures: interview with student, interview with another informant, ADHD rating scale completed by student, ADHD rating scale completed by another informant, impairment rating scale completed by student, impairment rating scale completed by another informant, psychoeducational testing, neuropsychological testing, review of educational records, review of medical records, and results of a symptom validity test (SVT). The clinician could use the checklist to indicate that a diagnostic criterion was met using a specific method, even if it was not included in his or her report. The checklist also solicited demographic information about the clinician including name, degree, years of practice, location, DSM-IV/5 diagnoses, and date of diagnoses.
Two research assistants independently reviewed each student’s documentation for data supporting DSM-5 criteria. Each criterion could be satisfied based on three levels of evidence: (a) clinician self-report, based on the assessment checklist; (b) documentation review, based on any written information including students’ self-reports during clinical interview; or (c) multi-method/multi-informant data, based on the results of rating scales, information provided by other informants, or educational/medical records. Operational definitions for each criterion are provided below.
ASignificant symptoms: DSM-5 requires adults to show significant symptoms that are inconsistent with developmental level. In our study, this criterion could be met if (a) the clinician reported assessing significant symptoms, even if such information was not included in the report; (b) the documentation indicated that the student endorsed significant symptoms; or (c) any documentation indicated that the student endorsed a significant number or severity of symptoms on an ADHD rating scale. 1
Childhood onset: DSM-5 requires the presence of several symptoms prior to age 12. However, several experts have argued for a more lenient cutoff when diagnosing ADHD in adults, given the difficulty of obtaining historical records or information from caregivers (Roberts et al., 2015). Consequently, in our study, this criterion could be met if (a) the clinician reported assessing onset prior to college, even if such information was not included in the report; (b) the documentation indicated onset prior to college, even if based exclusively on student self-report; or (c) onset prior to college was established by educational/medical records or an interview/rating scale completed by a caregiver about the student’s functioning in childhood or adolescence.
Symptoms across settings: DSM-5 requires symptoms in multiple settings. In our study, this criterion could be satisfied if there was evidence that the clinician assessed symptoms in multiple settings based on (a) the clinician’s self-report; (b) any information presented in the documentation, including the student’s self-report; or (c) multi-method/multi-informant data such as an interview with another informant (e.g., employer, teacher) about the student’s current functioning or results of an ADHD rating scale completed by another informant. 1
Impairment: DSM-5 requires evidence that the individual’s ADHD symptoms limit academic, occupational, or social functioning. In our study, we accepted (a) the clinician’s self-report that he or she assessed impairment, separate from symptom number or severity; (b) any documentation that described current limitations in functioning (e.g., problems completing coursework, managing finances, or being on-time for work), even if based exclusively on the student’s self-report; or (c) multi-method/multi-informant data indicating current impairment or need for accommodations, a history of impairment or accommodations, or results of an impairment rating scale.
Alternative Causes Ruled Out: DSM-5 requires diagnosticians to rule out alternative explanations for individuals’ symptoms. In our study, we considered this criterion met if (a) the clinician reported that he or she ruled out alternative causes or (b) there was any rule-out statement in the documentation. We also looked for evidence that the clinician administered an SVT to rule out low-effort test-taking or malingering, although it was not required to meet this criterion.
We determined interrater reliability for students’ documentation by calculating the percentage agreement among reviewers. Agreement was highest for evidence of significant symptoms in the documentation (.95) and lowest for evidence of impairment in the documentation (.89). Discrepancies were resolved by review and discussion.
Results
Figure 1 shows the percentage of students meeting each DSM-5 criterion based on (a) clinicians’ self-reported assessment practices; (b) written documentation, including information based exclusively on students’ self-reports; and (c) multi-method/multi-informant data (e.g., interviews with other informants, records, rating scales).
Percentage of students meeting DSM-5 criteria for ADHD based on three types of evidence: (a) clinician self-report, (b) written documentation including information provided by the student during diagnostic interview, and (c) multi-method/multi-informant data such as educational/medical records, interviews with other informants, or rating scales.
The strongest support for students’ diagnoses came from clinicians’ self-reported assessment practices. All clinicians reported that they reviewed DSM-5 symptoms and determined that students experienced a significant number to merit the ADHD diagnosis. Most psychologists also reported that they established childhood onset (56.5%) and the presence of symptoms in multiple settings (71.0%). Fewer psychologists reported that they assessed impairment (48.6%) or ruled out alternative causes (40.2%). Only 23.4% of psychologists reported that they assessed all DSM-5 criteria for ADHD.
Psychologists’ reports and supporting documentation provided less support for students’ diagnoses. Only 14.0% of reports and supporting documentation included evidence meeting all DSM-5 criteria. Although nearly all (96.3%) documentation indicated that students had significant ADHD symptoms, only about one half (55.6%) indicated that students experienced symptoms in multiple settings. Most documentation did not provide evidence of childhood onset (45.2%) or impairment (34.1%), even if based exclusively on students’ self-reports. Relatively few documents (29.0%) ruled out alternative causes.
Multi-method/multi-informant data supporting students’ ADHD diagnoses were also sparse. Only 10.3% of psychologists provided data generated by multiple methods or from multiple informants meeting all DSM-5 criteria. Relatively few clinicians described the results of self-report rating scales (37.9%; Criterion A) or consulted another informant to establish symptom presentation across settings (39.7%; Criterion C). Less than one third of psychologists indicated that they established childhood symptom onset based on educational/medical records or information provided by a caregiver (29.0%; Criterion B). Only 21.5% of psychologists provided evidence of impairment based on current or previous accommodations, reports from other informants, or results of an impairment rating scale.
We conducted a series of contingency analyses to identify relationships between clinicians’ characteristics and their likelihood of adherence to DSM-5 diagnostic criteria. Results showed that school psychologists (36.2%) were more likely than other psychologists (19.8%) to report assessing all criteria, χ2(1) = 5.52, p = .019, Φ = .17. Follow-up analyses showed that school psychologists (74.5%) were more likely than other psychologists (41.3%) to report gathering data regarding students’ impairment, χ2(1) = 16.14, p < .001, Φ = .27, and school psychologists (61.7%) were more likely than other psychologists (34.1%) to report ruling out alternative causes, χ2(1) = 11.60 p = .001, Φ = .23.
Contingency analyses also showed that school psychologists (21.3%) were more likely than other psychologists (7.2%) to assess all criteria based on multi-method/multi-informant data, χ2(1) = 7.89, p = .005, Φ = .19. Follow-up analyses showed that school psychologists (44.7%) were more likely than other psychologists (17.4%) to gather objective evidence of impairment, χ2(1) = 15.28, p < .001, Φ = .26, and school psychologists (45.3%) were more likely than other psychologists (21.1%) to specifically mention ruling out alternative causes in their reports, χ2(1) = 9.26, p = .002, Φ = .21.
Additional contingency analyses showed no significant associations between (a) clinicians’ degrees (i.e., doctoral, nondoctoral), (b) clinicians’ years of experience, (c) age of reports, or (d) diagnostic system (DSM-IV vs. DSM-5) and their likelihood of assessing all DSM-5 criteria based on clinician self-report, student self-report, or multi-method/multi-informant data, respectively.
Discussion
Our findings showed deficiencies in the written documentation supporting students’ ADHD diagnoses. Only 10% of students had sufficient multi-method, multi-informant data meeting all DSM-5 criteria. Furthermore, only 14% of students had documentation meeting all diagnostic criteria when we expanded our analyses to include data based exclusively on students’ self-reports. Our findings are consistent with the results of previous studies that also showed a low percentage of college students with documentation supporting their ADHD diagnoses (Joy et al., 2010; Nelson et al., 2014). Altogether, these results yield converging data that clinicians should provide more thorough evidence supporting their diagnostic decisions and recommendations for treatment.
More importantly, our results also indicated deficiencies in clinicians’ methods of assessment. Although all students were diagnosed with ADHD and receiving accommodations for this condition, only 23% of clinicians reported assessing all DSM-5 criteria. This finding suggests more serious deviation from best practices in evidence-based assessment. To the extent that psychological assessment is a scientific endeavor, deviation from DSM-5 criteria threatens the reliability of diagnosis and the validity of clinical decisions that follow (Jensen-Doss & Hawley, 2010).
Impairment and Alternative Causes
Regardless of the level of evidence we used to evaluate adherence, clinicians were least likely to assess impairment or to rule out alternative causes for students’ symptoms. Specifically, only 49% of clinicians reported assessing impairment, 34% described areas of impairment in their reports, and 22% provided multi-method/multi-informant data indicating impairment, such as school records showing low grades, evidence of a history of accommodations, or results of an impairment rating scale. Similarly, only 40% of clinicians reported ruling out alternative causes for student’s symptoms, and 29% included a rule-out statement in their reports.
The assessment of impairment, independent of symptom presentation, is critical to both the psychiatric definition of ADHD and the legal definition of a disability (Lovett et al., 2016). DSM-5 requires individuals with ADHD to show “clear evidence that symptoms interfere with, or reduce the quality of, social, academic, or occupational function” (APA, 2013, p. 60). Its authors give examples of impairment in adults including poor occupational performance or difficulty sustaining employment, reduced school performance or academic attainment, and conflicts in relationships. Similarly, the ADAAA considers ADHD as a disability if it “substantially limits the ability of the individual to perform major life activities as compared to most people in the general population” (U.S. Department of Justice, 2016, p. 3). Indeed, college students are entitled to accommodations under the auspices of the ADAAA based on their degree of impairment, not on their diagnostic classification, per se (Lovett, 2014).
Evidence of impairment is essential to decision-making regarding treatment and mitigation (Weis, Dean, & Osborne, 2016). Interventions for ADHD, either behavioral or pharmacological, focus chiefly on improving students’ functioning at school, work, or in social settings. Information about how students’ symptoms limit their functioning in these situations is critical to planning treatment. Similarly, accommodations are designed to mitigate the effects of ADHD on students’ academic functioning. Information about the nature and severity of these limitations is necessary to tailor accommodations to meet students’ needs. Physicians, behavior therapists, and disability specialists need a thorough description of students’ current functioning and degrees of impairment to effectively plan, implement, and monitor these interventions.
Clinicians also must rule out alternative causes for students’ ADHD symptoms and academic problems. Symptoms of ADHD are shared with other psychiatric disorders and physical illnesses, especially anxiety disorders, learning disabilities, and substance use problems—all of which are prevalent among college students (Barkley, 2015; Post & Kurlansik, 2013). Similarly, students with low self-efficacy regarding their academic skills or low motivation to engage in college-level work might experience academic problems that mimic those reported by students with ADHD (Lewandowski, Gathje, Lovett, & Gordon, 2013). Typically developing students can also misinterpret academic stress, anxiety, and mood problems as symptoms of ADHD (Alexander & Harrison, 2013). Although relatively few students in our study had extensive histories of ADHD, most were experiencing other psychiatric conditions. Few clinicians examined whether these other conditions might better explain students’ self-reported ADHD symptoms.
Failure to rule out alternative explanations could contribute to ineffective or inappropriate interventions. For example, students whose problems with attention and concentration are better explained by anxiety or mood disorders, rather than ADHD, would likely benefit from pharmacological and/or psychosocial therapies designed for these conditions. Students whose academic struggles are attributable to substance use problems, sleep disturbance, or learning disabilities would better be served by evidence-based interventions targeting these specific problems. Students with low motivation or problems adjusting to the demands of college should be offered academic or psychosocial counseling, rather than psychostimulants and extra time on exams (Diller, 2010).
Clinician Characteristics and Adherence
Contingency analyses revealed several significant associations between clinicians’ characteristics and their likelihood of adhering to DSM-5 criteria. Specifically, school psychologists, who worked almost exclusively in public school settings, were more likely to assess all diagnostic criteria than other psychologists, who overwhelmingly worked in private practices, clinics, hospitals, and universities. Furthermore, school psychologists were more likely than other psychologists to assess all criteria based on either their self-reported assessment practices or the multi-method/multi-informant documentation they submitted to the college. Follow-up analyses showed that school psychologists were more likely than other psychologists to assess two criteria in particular: impairment (Criterion D) and alternative causes (Criterion E).
At least three factors might explain why school psychologists showed better adherence to DSM-5 criteria and multi-method/multi-informant assessment than the other psychologists in our study: (a) training, (b) setting, and (c) motivation. First, there might be something about the training that school psychologists receive that makes them more likely to adhere to best practices. For example, training in school-based consultation and liaison, functional assessment, and psychoeducational testing might better prepare school psychologists to gather information from multiple informants, document functional impairment, and rule out alternative causes for academic problems, such as learning disabilities (Fagan & Wise, 2007; Merrell & Ervin, 2011).
Second, psychologists working in schools might have greater access to information about students’ functioning in academic settings than psychologists employed in clinics and hospitals. For example, it is likely that school psychologists can more easily interview parents and teachers, review school records, or observe students in the classroom than clinicians who work in the community.
Third, some clinicians might be motivated to assign an ADHD diagnosis in the absence of sufficient evidence (Gordon, Lewandowski, Murphy, & Dempsey, 2002; Harrison, Lovett, & Gordon, 2013). Although students might lack evidence of impairment, or experience other factors that might better account for their problems (e.g., low motivation, poor sleep hygiene, substance use problems), clinicians might assign a diagnostic label out of a genuine desire to help students succeed. Clinicians in private practice might experience particular pressure to diagnose, given the large fee paid by students (or their families) for disability evaluations. It may be difficult for psychologists in private practice to deny students the diagnostic label they seek, given the time and cost associated with testing in private clinical settings.
Access to Medication and Accommodations
Although not assessed directly in our study, our findings have implications for the welfare of postsecondary students and institutions more generally. Perhaps most salient is the possibility that the lack of adherence to diagnostic criteria might lead to the inappropriate provision of psychostimulants by physicians and contribute to problems with misuse or diversion. Meta-analysis indicates that 17% of postsecondary students misuse psychostimulants. Although most college students report using these medications as a “study aid,” misuse is associated with decreased academic performance. Students who misuse psychostimulants tend to acquire these medications from peers with valid prescriptions. Clinicians’ failure to adhere to DSM-5 diagnostic criteria, and judiciously prescribe medication only to students with genuine impairment, has been identified as a primary contributor to misuse on college campuses (Benson et al., 2015).
A related concern is that some students may misreport ADHD symptoms to gain access to medication (Cook et al., 2018; Harrison & Edwards, 2010). The base rate for malingering among college students seeking medication for ADHD is approximately 10% (Pella, Hill, Shelton, Elliott, & Gouvier, 2012). Simulation studies indicate that students can feign symptoms with little coaching; neither clinicians nor norm-referenced rating scales are effective in differentiating simulators from students honestly reporting symptoms (Musso & Gouvier, 2014). In contrast, many students with genuine ADHD fail to report significant symptoms because of unrealistically positive self-appraisals (Prevatt et al., 2012). To increase diagnostic accuracy, many professionals recommend administering SVTs when assessing postsecondary students for disabilities (Harrison, Green, & Flaro, 2012). Although SVTs are effective in identifying malingering, only 3% of clinicians in our study administered one.
Students misdiagnosed with ADHD might also be provided inappropriate or ineffective accommodations. Accommodation decision-making is important because it can affect the academic performance of students and the integrity of postsecondary education (Lovett & Lewandowski, 2015). The most common accommodation, additional time on exams, tends to improve test performance in all students, regardless of their disability status (Gordon et al., 2015; Lewandowski et al., 2008; Lovett, 2010; Miller, Lewandowski, & Antshel, 2015). The indiscriminate provision of additional time can give students an unfair advantage over their classmates who do not receive this accommodation. For example, Lewandowski, Cohen, and Lovett (2013) found that college students without disabilities benefited more than their peers with disabilities from additional time on exams. Furthermore, students classified with disabilities who were given 100% additional time on an exam outperformed their classmates who took the exam under standard conditions. Rather than “level of playing field” for students with disabilities, additional time can threaten the validity of test scores. It is important, therefore, that this accommodation be restricted to students with actual disabilities, and limited in duration, to avoid giving certain students an unfair advantage over their classmates.
Other accommodations may have negligible or deleterious effects on performance (Gordon et al., 2015). For example, only one published study has examined the effects of testing in a separate room on performance. Contrary to expectations, students earned slightly higher scores in a group, rather than private, setting (Lewandowski, Wood, & Lambert, 2015). Providing students’ access to a calculator during math exams or a word processor on essay tests can actually increase anxiety and decrease performance for students with disabilities (Lovett & Lewandowski, 2015; Lovett, Lewandowski, Berger, & Gathje, 2010). Clearly, more research is needed to make sure that accommodations effectively mitigate students’ disabilities and avoid inadvertently harming their performance (Gregg & Nelson, 2012).
Implications and Limitations
Why did so few students in our study have evidence supporting their diagnoses? One possibility is that clinicians face practical obstacles to assessing ADHD in adults. When surveyed about their typical assessment practices with school-age children, psychologists reported evaluating all criteria and relying heavily on multi-method/multi-informant data (Handler & DuPaul, 2005; Ogg et al., 2013). However, when asked to report their actual assessment practices with a specific college student, most psychologists in our study fell short of these standards. It is likely that psychologists have greater difficulty assessing certain criteria in adults. Many college students may be unable to provide childhood educational/medical records or unwilling to allow clinicians to interview parents and teachers regarding their developmental histories or current academic functioning. Clinicians may be unable to rule out alternative causes for college students’ self-reported academic problems because of time and cost constraints. Whereas children might be assessed for learning disabilities or communication disorders by psychologists at their schools, college students must often bear the burden of these time- and cost-intensive evaluations themselves.
A second possibility is that some clinicians may lack knowledge of DSM-5 criteria and the ADAAA. For example, Harrison and colleagues (2013) surveyed clinicians who conducted adult disability evaluations. Almost 14% incorrectly reported that significant elevations on a self-report ADHD rating scale were sufficient for the ADHD diagnosis; 36% reported that a student could be diagnosed with ADHD without experiencing symptom onset in childhood; and 70% believed that impairment could be demonstrated by showing deficits compared with other high-functioning college students, rather than to individuals in the general population. Fortunately, several publications are available to help clinicians update their knowledge of adult ADHD assessment (Gordon et al., 2015; Roberts et al., 2015) and accommodation decision-making (see Lovett et al., 2016; Lovett & Lewandowski, 2015).
A third possibility is that professionals see their role chiefly as advocates for their clients rather than primarily as psychological scientists (Nelson, Lovett, & Lindstrom, 2015). In the same survey, 45% of clinicians reported that the purpose of evaluating students is to help them secure the accommodations they want, and 14% reported that it is appropriate to diagnose a student with ADHD, even if the student does not meet criteria, to help him or her acquire accommodations (Harrison et al., 2013). Clinicians, like all humans, are susceptible to confirmation biases and their desire to help others (Youngstrom, Choukas-Bradley, Calhoun, & Jensen-Doss, 2015). Although well intentioned, the unwarranted provision of accommodations runs contrary to evidence-based practice and is unfair to students who may lack the academic savviness and financial resources necessary to pursue ADHD testing and to gain access to medication and accommodations not afforded their peers (Pellegrino, Sermons, & Shaver, 2011).
Results of our study point to at least four, practical ways clinicians can better adhere to DSM-5 criteria and increase the reliability and validity of their diagnoses. First, and most importantly, clinicians can renew their commitment to comprehensive assessment when evaluating ADHD in adults. At the very least, comprehensive assessment involves adhering to DSM-5 criteria, gathering data from multiple informants, and relying on historical documents and contemporary records to establish symptom presentation and impairment (Wei & Suhr, 2015). Second, clinicians should assess impairment independent of symptom number or severity. Fortunately, several brief, norm-referenced impairment rating scales for adults are available (see Lewandowski et al., 2016). Many assess functioning across multiple domains (e.g., school, social, work) and can be completed by other informants (e.g., clinicians, employers), thereby permitting a time- and cost-effective means of multi-method, multi-informant assessment of impairment. Third, clinicians can avoid confirmatory biases in their diagnostic decision-making by exploring alternative causes for students’ problems. At the very least, clinicians can screen for other problems by supplementing ADHD rating scales with an omnibus measure of behavioral and social-emotional functioning. For example, the Behavior Assessment System for Children–Third Edition (BASC-3; Reynolds & Kamphaus, 2015) is a group of rating scales that assess a wide range of externalizing problems, internalizing problems, academic difficulties, and adaptive skills. Scales can be administered to parents, teachers, or students themselves. Furthermore, norms allow clinicians to quickly identify potential problems compared with other young adults (through age 21 for parent/teacher reports) or college students (through age 25 for self-reports). Finally, we recommend that clinicians routinely administer an SVT when conducting adult disability evaluations. Administering an SVT is especially important when clinicians rely primarily on students’ self-report data (Suhr, Cook, & Morgan, 2017). Results can be used to confirm clinicians’ ADHD diagnosis or identify other potential causes, such as symptom exaggeration, malingering, or low-effort test-taking (Bush et al., 2005; Heilbronner et al., 2009). Several SVTs are especially useful in ADHD evaluations for college students (see Jasinski et al., 2011).
One threat to our study’s internal validity lies in the manner with which we operationalized the evidence used to support students’ diagnoses. In all cases, we accepted a wide range of data, giving clinicians the benefit-of-the-doubt regarding their diagnostic decision-making. For example, we accepted clinicians’ self-reported assessment procedures, in addition to information in their reports, as evidence that students met each criterion. We also accepted supporting documentation, such as educational and medical records submitted with the report, as evidence. Nevertheless, most students lacked sufficient support for their diagnoses.
The main threat to our study’s external validity is the representativeness of our sample. Although large, it reflects students receiving accommodations for ADHD at only two private colleges. Students with well-documented childhood histories of ADHD tend to enroll in 2-year public colleges rather than 4-year private colleges or universities (Newman et al., 2011). Unlike most of the students in our study, students with well-documented histories of ADHD are typically first diagnosed in childhood, experience academic difficulties in primary and secondary school, and continue to experience deficits in attention, concentration, and executive functioning that limit their functioning as adults (Newman et al., 2012; Weyandt et al., 2013). It is possible that community college students with ADHD might have more extensive histories of impairment and clearer evidence that they meet DSM-5 criteria. Indeed, community college students diagnosed with learning disabilities are more likely to have histories of academic problems and current academic impairments than students diagnosed with learning disabilities at 4-year private colleges (Weis, Speridakos, & Ludwig, 2014). It is possible that the students in our study reflect a subgroup of postsecondary students who first seek the ADHD label in adulthood as an explanation for problems meeting the demands of a rigorous, postsecondary education (Suhr & Wei, 2013, 2017). Future research should include students attending other postsecondary institutions to determine the generalizability of our findings.
Another threat to external validity is the limited demographic information about the clinicians in our study. Because our study was archival in nature, we were unable to report clinicians’ age and ethnicity. The absence of these demographic data might limit the generalizability of our results.
Despite these limitations, our study provides modest support for students’ ADHD diagnoses. Although all students in our study had been assigned the ADHD label and were receiving accommodations for limitations associated with that condition, very few had a history of symptoms prior to college, evidence of impairment that would merit mitigation, and alternative causes for their academic problems evaluated and ruled out. The lack of support for students’ ADHD diagnoses is not merely a deficit in documentation; it appears to reflect a lack of adherence to best practices in evidence-based assessment. However well intentioned, the unsupported or indiscriminate assignment of the ADHD label, and provision of medication and accommodations that follow, can negatively affect students, their classmates, and postsecondary institutions more generally. Psychologists and disability specialists alike must renew their commitment to evidence-based practice to make sure that services are directed to students most in need.
Footnotes
Authors’ Note
Christina H. Till is now at Duquesne University. Celeste P. Erickson is now at Tufts University.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the Laurie Bukovac Hodgson and David Hodgson Endowed Fund for Disability Research.