Abstract
ADHD is a neurodevelopmental chronic mental health disorder associated with substantial physical, behavioral, cognitive, and social impairments throughout the life span (American Psychiatric Association [APA], 2000; Barkley, 2006; Barkley, Fischer, Smallish, & Fletcher, 2002; Faraone & Antshel, 2008). About 4.4% of the adult U.S. population, or about 9 million adults, are estimated to have ADHD (Faraone & Biederman, 2005; Kessler et al., 2006). The primary screening tool used by clinicians to make decisions regarding ADHD diagnosis in adults is self-report (Wender, Wolf, & Wasserstein, 2001), which is a method believed to underestimate the incidence of the disorder (Murphy & Schachar, 2000; Zucker, Morris, Ingram, Morris, & Bakeman, 2002).
The extent to which ADHD is under- or overdiagnosed in adolescents has long been investigated (Desgranges, Desgranges, & Karsky, 1995; Kessler et al., 2006), with most research focused on the ability of therapists to properly identify sex differences in the expression of ADHD (Gaub & Carlson, 1997; Gershon, 2002), to properly adhere to diagnostic criteria in the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; APA, 1994; Bruchmüller, Margraf, & Schneider, 2012), and to be aware of the heritability of ADHD (Able, Johnston, Adler, & Swindle, 2007; Waite & Ramsay, 2010). However, little effort has been focused on the extent to which patients’ beliefs about their diagnosis prior to a diagnostic assessment influence diagnosis.
Recent evidence suggests that self-reports, which are commonly used as screening tools to help clinicians make decisions regarding ADHD diagnosis with adolescents and adults, are less diagnostically sensitive than informant reports in assessing ADHD symptoms (Sibley et al., 2012; Sibley et al., in press). The evidence suggests that young adults with ADHD tend to underreport symptoms and young adults without ADHD tend to overreport symptoms using self-report assessments (Barkley et al., 2002; Sibley et al., 2010). In fact, Sibley et al. (in press) suggest that the use of self-report measures in assessing ADHD is so inconsistent that it adds little benefit compared with parent reports alone. Therefore, it would be beneficial to identify potential factors that contribute to inconsistencies in self-reports to alleviate the problems associated with using these measures to reliably screen for ADHD.
One possible factor contributing to inconsistent patient self-reports is patient beliefs about diagnosis. Concerns regarding the influence of self-beliefs on the expression of ADHD symptoms have been raised (Bell, Long, Garvan, & Bussing, 2011). Studies show that negative stigmas held by teachers and parents can negatively influence the behavior and self-efficacy of children (Eisenberg & Schneider, 2007; Madon, Guyll, Spoth, & Willard, 2004). For children with ADHD, teachers and parents can often hold more negative perceptions of the abilities of these children compared with those without ADHD, even when no child differences exist (Eisenburg & Schneider, 2007). Given that adults can hold negative stigmas that change their behavior toward children known to be diagnosed with ADHD, it is certainly possible that young adults may likewise be influenced by their perceptions or beliefs of their own diagnosis. If true, then these beliefs could contribute to the purported inconsistent self-reports of young adults using these self-report assessments (Barkley et al., 2002; Sibley et al., 2010).
In this experiment, we tested for the first time the hypothesis that self-beliefs prior to ADHD assessment can influence self-report ratings. In this experiment, we used the World Health Organization’s (WHO) Adult ADHD Self-Report Scale (ASRS) screener (Adler et al., 2006; Kessler et al., 2005; Kessler et al., 2007), which is commonly used in clinical practice.
Method
Participants
A sample of undergraduate college students enrolled in introductory level psychology courses were recruited using classroom visits. A final sample was selected who had no prior knowledge of, diagnosis, treatment, or other medical or clinical history of a mental or behavioral disorder, as determined in a preliminary medical and demographic survey. Participants also completed the WHO ASRS screener (Kessler et al., 2005), which was used to select participants and establish baseline scores on this screener. Of the 74 participants who completed this screener, 54 participants scored below clinical significance on the ASRS screener (see “ASRS Measure” section) and were therefore selected to participate in this experiment. For the 54 participants, key additional characteristics were as follows: 24 women, 30 men; 63% White, 37% Black; age: M = 20.4, SD = 1.2 years. The university’s Institutional Review Board approved the procedures for this study.
ASRS measure
The ASRS is a valid and reliable screener that is commonly used in clinical practice to screen for ADHD in adults (Adler et al., 2006; Kessler et al., 2007). The ASRS screener is used to determine the likelihood of diagnosis for ADHD with 18 items on the following 5-point scale: 0 (never), 1 (rarely), 2 (sometimes), 3 (often), and 4 (very often). The screener can be used to determine clinical significance of diagnosis by scoring six items found to be the most predictive of symptoms consistent with clinical diagnosis of adult ADHD (Adler et al., 2006; Kessler et al., 2007). Clinical significance is identified when a participant scores in the clinical range (i.e., has symptoms highly consistent with adult ADHD) on at least four of these six items. To also identify the frequency of symptoms, scores were summed for all items and for each domain: inattentive domain (sum questions 1-4, 7-8) and hyperactive/impulsive domain (sum questions 5-6, 12-18). Each method of scoring the ASRS was applied here.
Procedures
Participants completed the ASRS in a pretest and again 1 week later in a posttest. The ASRS pretest was taken at the time of participant selection (see “Participants” section). The 54 participants who scored below clinical significance on the ASRS constituted our sample. These participants were matched based on their ASRS pretest scores and randomly assigned to one of three feedback manipulation groups, such that the same proportion of men and women from each ethnic category were assigned to each group, and ASRS pretest scores were identical between groups.
One week following the initial ASRS pretest, participants were brought back for a follow-up session. All participants who completed the pretest returned for the 1-week follow-up session. In that session, participants received the ASRS with the order of items reversed to minimize testing effects. In this session, all groups were told that they were taking a follow-up test that was different from their pretest and that the results of their pretest were indicated on the back of their follow-up test, if those scores were available. Participants were then randomly assigned to one of three feedback manipulation groups. Group Negative had “Negative” written on their follow-up test to indicate that they did not have symptoms consistent with ADHD diagnosis. Group Positive was deceived and had “Positive” written on their follow-up test to indicate that they did have symptoms consistent with ADHD diagnosis, which was not true. Group No Feedback received no feedback about their pretest scores. The feedback was written down instead of spoken to standardize the delivery of feedback between groups and reduce the likelihood of demand characteristics.
After completing the ASRS posttest, all participants confirmed in a follow-up survey that they felt the feedback given was honest. All participants were then debriefed, thanked for their time, and dismissed. Differences between pretest and posttest scores on the ASRS were compared.
Data Analyses
A Kruskal–Wallis H Test was used to compare the number of participants in each group who scored in the clinical significance range on the ASRS. Using this test, we tested the null hypothesis that the distribution of those scoring in the clinical significance range was the same in each group. Because the ASRS was taken at two times, test–retest reliability was computed using a binomial test. Like the pretest, no participants should score at or above clinical significance on the follow-up test. However, for the binomial test, we tested the null hypothesis that the proportion of participants scoring at or above clinical significance in each group is .05, to allow for the standard .05 error rate. All tests were conducted at a .05 level of significance.
An ANOVA with groups, gender, and race as factors was also computed for three dependent measures: difference scores (pretest–posttest) in the attentive domain, in the hyperactive/impulsive domain, and totals for the sum of all 18 items. If significant, 95% confidence intervals (CIs) were drawn around the difference scores for each group. A significant difference was identified when the difference scores did not envelop zero.
Results
The distribution of those scoring in the clinical significance range (i.e., those having symptoms highly consistent with adult ADHD) was not equal between groups, χ2(2) = 11.81, p = .003. Group Positive showed a significantly higher proportion of participants scoring in the clinical significance range on the posttest (8/18; 44%) compared with other groups. A binomial test showed that this result was significantly disproportionate, p < .001, and therefore would fail to meet criteria needed to establish test–retest reliability. By comparison, Group Negative and Group No Feedback had only one participant in each group score in the clinical significance range on the posttest (1/18; 6%). A binomial test showed that this result matches expected proportions, p = .60, and therefore does meet criteria needed to establish test–retest reliability.
An ANOVA showed that changes from pre- to posttest were significant for total scores on the ASRS, F(2, 53) = 3.85, p = .03 (R2 = .13), and for scores in the inattentive domain, F(2, 53) = 6.00, p = .005 (R2 = .19), but no significance was evident for scores in the hyperactive/impulsive domain, p = .36. For scores on the ASRS, 95% CIs were drawn. As shown in Figure 1, total scores significantly increased from pre- to posttest only for Group Positive, 95% CI = [0.19, 9.03]. For Group Positive, scores significantly increased in the inattentive domain, 95% CI = [1.89, 6.56], but not in the hyperactive/impulsive domain. All other CIs enveloped zero and therefore no further significant differences were evident. No effects of gender or race were significant (p > .35 for all tests).
The 95% confidence intervals by group for differences in pretest to posttest on all 18 items of the ASRS (total scores), on the inattentive domain items (sum questions 1-4, 7-8), and on the hyperactive/impulsive domain items (sum questions 5-6, 12-18)
Discussion
This experiment was conducted to test the hypothesis that feedback about an ADHD diagnosis influences how a nonclinical sample scores on an ASRS screener. In this experiment, all participants scored below clinical significance on the ASRS in a pretest and had no history of ADHD or any other mental or behavioral disorder. When participants received no feedback or were informed that they did not show symptoms consistent with ADHD diagnosis, ASRS scores were consistent in a posttest and the screener met criteria needed to establish test–retest reliability. However, when participants were deceived and informed that they did have symptoms consistent with ADHD diagnosis, ASRS scores were remarkably unreliable, with 8 of 18 participants scoring in the clinically significant range of the ASRS in a posttest, despite scoring below clinical significance in a pretest 1 week earlier.
The results presented here illuminate that positive feedback about having a disorder can increase inconsistencies in ASRS scores. In this study, we manipulated participant self-beliefs by giving positive feedback, negative feedback, or no feedback prior to the ASRS retest. The manipulation check showed that all participants found the feedback to be honest. Therefore, we surmise that participants receiving the positive feedback changed their self-belief of their own likelihood of diagnosis, which led to the greater inconsistency in ASRS ratings from Time 1 to Time 2. These findings suggest that one possible reason that adults without ADHD can overreport symptoms is that they have self-beliefs regarding their own likelihood of disorder. Because patients are often told of their possibility of having ADHD or are inherently convinced they have the disorder prior to assessment (Guyll, Madon, Prieto, & Scherr, 2010; Weisler & Goodman, 2008), one possible implication is that positive feedback given prior to assessment may lead many patients to overreport their symptoms (Barkley et al., 2002; Sibley et al., 2010), thereby leading to the possibility of overdiagnosing the disorder in these cases.
Interestingly, participants in our study overreported symptoms in the inattentive domain in the posttest, but not in the hyperactive/impulsive domain. These findings are consistent with studies showing that the inattentive domain is prominent in adults and that hyperactive/impulsive symptoms can decline significantly with age, even if these symptoms were expressed in childhood (Biederman, Mick, & Faraone, 2000; Millstein, Wilens, Biederman, & Spencer, 1997). Inattentive symptoms are also more strongly related to academic performance than hyperactive/impulsive symptoms (Gordon et al., 2006), which may explain why college students in our study specially overreported symptoms in this domain.
The extent to which these findings can be applied to patients who express ADHD or can be controlled in a clinical setting cannot be determined here. However, the results do show that self-beliefs regarding the likelihood of having ADHD can lead young adults without ADHD to overreport their symptoms, potentially leading to overdiagnosis in these cases. These findings provide one likely explanation for why self-report measures to assess adult ADHD can be diagnostically insensitive (Sibley et al., 2012; Sibley et al., in press) inasmuch as patient beliefs prior to diagnostic assessment influence ASRS self-report ratings and therefore decrease the reliability of those ratings.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research and/or publication of this article.