Development and Evaluation of the ADHD Cognitions Scale for Adults

Abstract

Objective: The clinical literature on ADHD in adults suggests that “overly positive” or optimistic cognitions may contribute to impairment and failure to use self-regulation skills in this population, yet the research literature on this topic is limited. We developed the ADHD Cognitions Scale (ACS), a brief self-report measure of ADHD-related thoughts, and evaluated its psychometric properties. Method: We collected self-report measures, inculding the ACS, from two large community samples (Ns = 262, 304). Results: The measure demonstrated a one-factor solution that replicated in the second sample. Evidence of good internal consistency and also convergent and divergent validity was obtained for both samples. Scores on the ACS correlated with functional impairment, time management problems, and avoidant coping strategies. Conclusion: With additional study, the ACS may be useful to identify and track maladaptive ADHD-related cognitions during cognitive-behavioral treatment, and to further study the role of these thoughts in ADHD-related impairment.

Keywords

adult ADHD rating scale automatic thoughts cognitive-behavioral therapy psychometrics

ADHD in adults is characterized by developmentally inappropriate and impairing symptoms of inattention and/or hyperactivity-impulsivity. In about two thirds of cases, the disorder and its related impairments persist from childhood into adulthood, leading to new impairments in adult domains of functioning (Barkley, Murphy, & Fischer, 2008). Cognitive-behavioral therapy (CBT) is an efficacious treatment for adults with the disorder that teaches strategies to reduce functional impairment associated with symptoms (Safren et al., 2010; Solanto et al., 2010). In addition to behavioral self-management strategies, many of these approaches involve management of depressive and anxious cognitions that may affect clients’ functioning (Ramsay & Rostain, 2008; Safren, Perlman, Sprich, & Otto, 2005; Solanto, 2011). As is the case with CBT in general, the main focus has thus far been on the effects of overly negative maladaptive thoughts associated with internalizing disorders.

Targeting such patterns of negative maladaptive thoughts is important as there is strong evidence that ADHD in adults is associated with increased risk of depression and anxiety disorders and, more specifically, that ADHD in adults is also associated with patterns of negative and distorted thinking associated with these disorders (Abramovitch & Schweiger, 2009; Knouse, Zvorsky, & Safren, 2013; Mitchell, Benson, Knouse, Kimbrel, & Anastopoulos, 2013; Strohmeier, Rosenfield, DiTomasso, & Ramsay, 2016). However, there is growing recognition that maladaptive patterns of thinking are not restricted to these “negative” thinking styles in adult ADHD (Knouse & Mitchell, 2015). On the contrary, recurrent problematic thoughts in adults with ADHD may appear quite optimistic on the surface—for example, “I’m the kind of person who does my best work at the last minute.” Such cognitions are topographically different from typical depressive and anxious cognitions targeted in most manualized CBT approaches for adults diagnosed with ADHD but are thought to have a similar functional outcome: failure to engage in self-management strategies. This issue has been repeatedly described in the clinical literature on adult ADHD (Ramsay & Rostain, 2008; Sprich, Knouse, Cooper-Vince, Burbridge, & Safren, 2010; Zylowska, 2012), and the issue of bias in the cognitions of some children with ADHD—the positive illusory bias—has also been studied extensively (see Owens, Goldfine, Evangelista, Hoza, & Kaiser, 2007 for a review). Evidence for these biases and their effects in adults with the disorder is compelling but more limited (Golden, Owens, Evangelista, & Micheli, 2006; Knouse, Bagwell, Barkley, & Murphy, 2005; Lui, Johnston, Lee, & Lee-Flynn, 2013; Prevatt et al., 2012).

In seeking to further investigate maladaptive thoughts in adult ADHD, at least two distinct approaches are possible. As in the research on the positive illusory bias, researchers could focus on the accuracy of participants’ domain-specific self-assessments relative to some standard (e.g., Knouse et al., 2005; Lui et al., 2013), and then examine whether individuals with miscalibrated opinions are more likely to experience negative outcomes. An alternative approach and one that is more consistent with the way that maladaptive cognitions are approached in CBT is to measure the frequency or severity with which overly positive thoughts occur in the daily lives of people with ADHD and to evaluate whether these thoughts are associated with negative outcomes. This approach has been crucial to understanding the role of negative automatic thoughts in internalizing disorders (R. Beck & Perkins, 2001), and self-report measures such as the Automatic Thoughts Questionnaire (ATQ; Hollon & Kendall, 1980) have been important tools in these investigations. A similar instrument for studying maladaptive cognitions associated with ADHD and its impairments would aid further investigation in this area, and the lack thereof is currently a barrier for researchers. Furthermore, alteration of problematic ADHD-related cognitions may be a mechanism of change in CBT for ADHD (Ramsay, 2010), and the ability to measure the frequency of these thoughts as treatment progresses could benefit both clinical research and clinical practice.

Study Objectives

The objective of the present study was to develop and evaluate a brief measure of ADHD-related automatic thoughts, which we refer to as the ADHD Cognitions Scale (ACS). The ACS was originally developed by the study authors as a clinical measure used with adult clients who were being assessed and treated for ADHD in an ADHD specialty clinic. The earliest version of the measure contained 33 items that were generated by the study authors in a rational-intuitive manner. Over time, the number of items that we administered to clients was reduced from 33 items to 20 items based on item performance in the clinic (e.g., associations with ADHD symptoms; item-total correlations; Anastopoulos et al., 2012). The objective of the present research was to further refine the ACS by developing a brief, reliable, valid, and unidimensional version that could be used by clinicians and researchers alike in a wide range of settings. Consistent with the cognitive-behavioral model of ADHD, we expected that ACS scores would be positively associated with ADHD symptoms and functional impairment. Furthermore, consistent with the view that ADHD-related cognitions may function to avoid negative affect (Knouse & Mitchell, 2015), we predicted that scores on the ACS would be positively associated with disengagement coping strategies (cognitive and behavioral avoidance, self-distraction, denial) and negatively associated with engagement coping (active coping, positive reframing, acceptance, planning).

Method

Participants

Study 1

Participants were 262 adults completing online self-report measures recruited through Amazon’s Mechanical Turk (MTurk; see Procedure). The mean age of the sample was 35.22 years (SD = 11.56) and ranged from 18 to 74 years with a median of 32. See Table 1 for additional sample demographics and descriptive statistics.

Table 1.

Sample Demographics.

	Sample 1	Sample 2
N	262	304
Age (SD)	35.22 (11.56)	33.99 (9.80)
Gender (%)
Men	110 (42)	146 (48)
Women	156 (58)	158 (52)
Race (%)
White/Caucasian	217 (83)	258 (84.9)
Black/African American	19 (7.3)	23 (7.6)
Asian/Pac. Isl./Asian American	22 (8.4)	24 (7.9)
Native American	8 (3.1)	7 (2.3)
Other	4 (1.5)	4 (1.3)
Ethnicity (%)
Hispanic	12 (4.6)	27 (8.9)
Non-Hispanic	250 (95.4)	277 (91.1)
Highest level of education (%)
No high school diploma	1 (0.4)	2 (0.7)
High school	38 (14.5)	46 (15.1)
Some college	65 (24.8)	71 (23.4)
Associate’s or technical degree	30 (11.4)	36 (11.8)
Bachelor’s degree	102 (38.9)	118 (38.8)
Master’s degree	22 (8.4)	26 (8.6)
Doctorate/medical/law degree	3 (1.1)	5 (1.6)
Employment status (%)
Full-time	133 (50.8)	202 (66.4)
Part-time	50 (19.1)	53 (17.4)
Full-time student	23 (8.8)	10 (3.3)
Part-time student	10 (3.8)	6 (2.0)
Stay-at-home parent or homemaker	27 (10.3)	26 (8.6)
Unemployed	35 (13.4)	19 (6.3)
Receiving disability benefits	7 (2.7)	4 (1.3)
Receiving welfare benefits	2 (0.8)	1 (0.3)
Retired	8 (3.1)	4 (1.3)
Marital status (%)
Single	103 (39.3)	138 (45.4)
Living with partner	44 (16.8)	47 (15.5)
Married	91 (34.7)	99 (32.6)
Divorced	22 (8.4)	17 (5.6)
Widowed	2 (0.8)	3 (1.0)
With children (%)	109 (41.6)	112 (36.8)

Note. Participants were able to check more than one race category and employment descriptor.

With respect to ADHD status, 7.3% of the sample reported receiving an ADHD diagnosis at some point in their lives and 5.7% endorsed a current diagnosis. Among those ever diagnosed with ADHD, 95% had taken medications for ADHD at some point in their lives and 61% were currently taking medications for the disorder. Sixteen percent of the sample reported current nonmedication treatment for ADHD. Using responses on the Barkley Adult ADHD Rating Scale (BAARS) and applying analogue Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; APA, 2013) criteria of five or more symptoms on either list and related impairment, 13% of the sample met these criteria as follows: 5.7% of the sample met criteria for the Predominantly Inattentive presentation, 3.4% for the Predominantly Hyperactive-Impulsive presentation, and 3.5% for the Combined presentation. With the addition of the requirement that participants endorse of “most of the problems” starting before age 12, the rate fell to 4.2% (i.e., 1.1% Inattentive, 2.3% Hyperactive-Impulsive, and 0.8% Combined). In this sample, participants reporting that they were ever diagnosed with ADHD were more likely to be male (p = .05), whereas participants currently meeting proxy DSM-5 criteria without age of onset were more likely to be female (p = .05). A greater proportion of adults meeting proxy DSM-5 criteria that included the age of onset were Hispanic/Latino (p < .001). No other differences in gender, education level, race, or ethnicity were detected for ADHD groups.

Study 2

Participants were 304 adults completing online self-report measures recruited through MTurk using similar procedures to Study 1. The mean age of the sample was 33.99 years (SD = 9.80) and ranged from 19 to 69 years with a median of 32. See Table 1 for additional sample demographics and descriptive statistics.

With respect to ADHD status, 7.6% of the sample reported ever receiving an ADHD diagnosis and 4.9% endorsed a current diagnosis. Among those ever diagnosed with ADHD, 78% had taken medications for ADHD at some point in their lives and 44% were currently taking medications for the disorder. Seventeen percent of those ever diagnosed reported currently receiving nonmedication treatment for the disorder. Using responses on the BAARS and applying analogue DSM-5 criteria of five or more symptoms on either list and related impairment, 10.2% of the sample met these criteria as follows: 4.9% of the sample met criteria for the Predominantly Inattentive presentation, 1.6% for the Predominantly Hyperactive-Impulsive presentation, and 3.6% for the Combined presentation. When endorsement of “most of the problems” starting before age 12 was added to these criteria, the rate fell by 4.6% (i.e., 2.0% Inattentive, 0.7% Hyperactive-Impulsive, and 2.0% Combined). In this sample, participants reporting that they had ever been diagnosed with ADHD were more likely to be female (p = .08). No other differences in gender, education level, race, or ethnicity were detected for ADHD groups.

Measures

Study 1

ACS

The ACS was modeled on the ATQ (Hollon & Kendall, 1980), a measure of negative automatic thoughts that were associated with depression. As noted above, ACS items were originally generated on a rational-intuitive basis by the study authors through their clinical work with patients with ADHD. Examples of items are “I’ll just do this one thing first” and “I do better waiting until the last minute.” Participants are instructed to rate how often each of these automatic thoughts occurs to them using the following Likert-type scale: 1 = not at all; 2 = sometimes; 3 = often; 4 = very often; 5 = all the time. The psychometric properties of the ACS are detailed below, and the final version of the scale can be found in the appendix.

BAARS-IV

The BAARS-IV (Barkley, 2011a) is a self-report scale in which participants report on the frequency with which they have experienced ADHD symptoms over the past 6 months on a 4-point Likert-type scale (never or rarely, sometimes, often, very often). The scale includes the 18 DSM-IV ADHD items. The scale also asks participants to report on the age of onset of endorsed symptoms as well as domains in which the symptoms impaired their functioning. Evidence of reliability and validity comes from a normative sample of 1,249 adults (test–retest reliability = .75 at 2-3 weeks; internal consistency = .91). Internal consistency for the 18 DSM-IV ADHD items in this sample was excellent (α = .92).

Barkley Functional Impairment Scale (BFIS)

The BFIS (Barkley, 2011c) is a 15-item measure that asks participants to rate how impaired they are (i.e., how much difficulty they have in functioning) in a variety of domains of life, including home, work, romantic relationships, driving, and parenting. Participants rate their impairment on a scale from 0 (not at all) to 9 (severe). Evidence of reliability and validity comes from the same normative sample as the BAARS-IV above, and includes good internal consistency (α = .969) and test–retest reliability at 2 to 3 weeks (.72) for the total impairment score. In Study 1, we observed a higher-than-expected number of missing values on this measure, which we believe was due to its formatting in the online survey. Specifically, participants responded by clicking and moving a slider to indicate their level of impairment. If participants wished to indicate no impairment (score of 0), they had to click (but not move) the slider, as 0 was its default position. Thus, some participants who intended to indicate no impairment may have had missing data. In addition, we did not include a “does not apply” option, and so participants may have left items blank for this reason. Despite these missing values, the internal consistency for the scale was strong (α = .93), and use of mean scores instead of totals attenuated the impact of missing data on this measure. Note that formatting issues were corrected in Study 2.

ATQ

The 30-item ATQ (Hollon & Kendall, 1980) was designed to measure the occurrence of automatic negative thoughts associated with depression. Respondents rate how frequently a thought occurred over the last week on a scale ranging from 1 (not at all) to 5 (all the time). The measure has good reliability, and distinguishes between depressed and nondepressed groups (Hollon & Kendall, 1980). Internal consistency in Study 1 was excellent (α = .99).

Center for Epidemiologic Studies–Depression (CES-D) Scale

The CES-D is a self-report scale that measures the frequency of depressive symptoms (Radloff, 1977). Participants rate the frequency of each item over the past week on a 4-point scale ranging from “Rarely or none of the time (Less than 1 day)” to “Most or all of the time (5-7 days).” Internal consistency of the CES-D in Study 1 was excellent (α = .94).

GAD-7

The GAD-7 (Spitzer, Kroenke, Williams, & Löwe, 2006) is a short scale designed to screen for and measure symptoms of generalized anxiety over the past 2 weeks. Participants respond on a 4-point scale from 0 (not at all) to 3 (nearly every day). The GAD-7 correlates with functional impairment and measure anxiety that is related to but distinct from depression (Spitzer et al., 2006). Internal consistency of this measure in Study 1 was excellent (α = .93).

Barkley Deficits in Executive Functioning Scale–Short Form (BDEFS-S)

The 20-item BDEFS (Barkley, 2011b) measures problems with executive functioning in daily life. Participants rate the frequency with which they have experienced each problem over the past 6 months on a 4-point scale from 1 (“Never or Rarely”) to 4 (“Very Often”). The BDEFS has five subscales measuring deficits in self-management to time, self-organization/problem solving (information-processing difficulties/cognitive inflexibility), self-restraint, self-motivation, and self-regulation of emotions. The BDEFS was normed on a large nationally representative sample of adults (n = 1,240), and demonstrated strong reliability and validity. Internal consistency of the BDEFS in Study 1 was excellent (α = .94).

Cognitive-Behavioral Avoidance Scale (CBAS)

On the CBAS (Ottenbreit & Dobson, 2004), participants rate 31 statements that describe how they might deal with situations and problems in their lives on a scale from 1 (not at all) to 5 (extremely true for me). The CBAS has subscales for both cognitive and behavioral avoidance. CBAS scores showed good psychometric properties, including high internal consistency and test–retest reliability (Ottenbreit & Dobson, 2004), and total CBAS scores showed excellent internal consistency in Study 1 (α = .97).

Study 2

Measures in Study 2 included the following measures described above for Study 1: ACS, BAARS-IV (Barkley, 2011a; Study 2, α = .94); BFIS (Barkley, 2011c; α = .94); ATQ (Hollon & Kendall, 1980; α = .99); CES-D (Radloff, 1977; α = .95); GAD-7 (Spitzer et al., 2006; α = .93); BDEFS-S (Barkley, 2011b; α = .95); and the CBAS (Ottenbreit & Dobson, 2004; α = .97).

In addition, the Study 2 self-report battery included items from the Weiss Functional Impairment Rating Scale (WFIRS; Weiss, 2000). We included the WFIRS as an additional, more fine-grained index of functional impairment, including items from the following subscales: Family (α = .88), Work (α = .88), School (α = .91), Life Skills (α = .90), Self-Concept (α = .96), and Social (α = .89). We omitted items from the Risk subscale due to concerns about asking participants to report on potentially illegal activities. For Study 2, we also included the brief form of the COPE Inventory (Carver, 1997), which provides an efficient way to measure a broad array of coping strategies. Each subscale of the brief COPE contains two items, and internal consistencies ranged from poor to excellent as follows: Self-Distraction (α = .46), Active Coping (α = .80), Denial (α = .75), Substance Use (α = .97), Emotional Support (α = .91), Instrumental Support (α = .93), Behavioral Disengagement (α = .73), Venting (α = .64), Positive Reframing (α = .82), Planning (α = .84), Humor (α = .91), Acceptance (α = .53), Religion (α = .94), Self-Blame (α = .86).

Procedure

Study 1

Data were collected using Amazon’s Mechanical Turk between May 30 and June 2, 2014. Mechanical Turk (MTurk; http://www.mturk.com) is an online crowdsourcing platform where requesters recruit workers to complete tasks and pay those workers for successful task completion (see Mason & Suri, 2012 for an introduction). MTurk samples are often more diverse than typical Internet and college student samples, and data reliability appears to be generally comparable with traditional methods (Buhrmester, Kwang, & Gosling, 2011). Relative to the U.S. population, MTurk samples, however, tend to be younger, better educated, and less racially and ethnically diverse, although they also tend to contain more participants who are unemployed or underemployed (Chandler, & Shapiro, 2016). Mechanical Turk has been used to recruit clinical samples (Shapiro, Chandler, & Mueller, 2013) and to investigate ADHD in adults (Wymbs & Dawson, 2015). Specifically, Wymbs and Dawson (2015) found that the proportion of adults who self-reported an ADHD diagnosis in an MTurk sample was similar to other non-Internet community samples of adults.

We used selection parameters for our MTurk sample and a validity scale to increase the likelihood of obtaining reliabile and high-quality data in the current study. The study was only available to MTurk workers with U.S. residency and who had at least 95% of their prior work on MTurk accepted by the requester. These selection parameters were intended to increase the likelihood that participants would complete the survey in a valid way. To further enhance data quality, we included a 13-item Infrequency scale that was placed randomly within the survey battery. This scale consisted of the 13 infrequency items developed by Chapman and Chapman (1983) randomly intermixed with another measure that is not reported in this study. Consistent with the authors’ recommendations, participants who endorsed more than two Infrequency scale items were dropped from further study (n = 43; 14% of the original sample). Thus, our analyzed sample of 262 includes only those participants who passed the validity screen.

The survey battery was administered using Qualtrics online survey software (www.qualtrics.com). Participants read a consent form, and clicked to indicate their consent and to attest that they were 18 years or older. They then completed a demographic questionnaire, the ACS, questions about ADHD diagnosis and treatment history, the BAARS, and the BFIS. Next, participants completed the other measures in a randomized order. Participants were paid US$2.00 through the MTurk platform.

Study 2

Data for Study 2 were collected using Amazon’s Mechanical Turk between January 12 and 14, 2016. We employed procedures nearly identical to Study 1, including selection parameters of U.S. residency and at least 95% of prior work on MTurk accepted by the requester. We again included the 13-item Infrequency scale, and participants who endorsed more than two items were dropped from further study (n = 49; 14% of the original sample). Study 2 participation was unavailable to Study 1 participants to prevent duplicate survey takers in the samples. Participants were paid US$2.00.

Results

Factor Analysis

Sample 1

An exploratory principal axis factor (PAF) analysis was conducted with Sample 1 to examine the initial factor structure of the full 20-item version of the ACS. As expected, examination of the scree plot revealed that a one-factor solution was most appropriate. Accordingly, the PAF was rerun, and a single factor was extracted. A total of nine items with factor loadings of 0.50 or higher were retained from the PAF for further analysis.

Exploratory/confirmatory factor analysis (E/CFA) was used to further evaluate the factor structure of the ACS, as use of E/CFA as an intermediate step between PAF and CFA frequently identifies additional sources of model misspecification that are not able to be identified with PAF alone (Brown, 2006; Jöreskog, 1969; Kimbrel et al., 2011). Root mean square error of approximation (RMSEA), comparative fit index (CFI), Tucker–Lewis index (TLI), and standardized root mean square residual (SRMR) were used to assess model fit. RMSEA values near or below 0.06, CFI and TLI values near or above 0.95, and SRMR values near or below 0.10 (Hu & Bentler, 1999; Kline, 2011) are generally considered to be indicative of close fit. A one-factor model that included the nine items with the highest factor loadings from the PAF was evaluated first. Examination of the fit indices from the E/CFA revealed that the proposed one-factor solution fits the data poorly, χ²(27) = 182.072, p < .001, RMSEA = 0.15, CFI = 0.823, TLI = 0.764, SRMR = 0.068. Examination of modification indices indicated that nearly all of the model misfit was due to covariance between the error terms of two of the items and the error terms of the remaining items. Accordingly, to further increase scale brevity, the two problematic items were removed from the scale, and the one-factor E/CFA was repeated on the remaining seven items. The resulting one-factor solution fits the data well, falling within the range of values generally accepted to indicate close model fit, χ²(14) = 27.410, p = .017, RMSEA = 0.060, CFI = 0.975, TLI = 0.963, SRMR = 0.032. Factor loadings were also high, ranging from 0.51 to 0.80, with an average factor loading of 0.63 (Table 2).

Table 2.

Factor Loadings for the ACS Items in Samples 1 and 2.

Items	Sample 1	Sample 2
I’ll just do this one thing first.	0.58	0.62
I do better waiting until the last minute.	0.57	0.51
I can’t stop right now.	0.56	0.48
Though this usually “sucks me in,” I’ll just do it for a minute.	0.78	0.71
I have plenty of time—I’ll just do one more thing before I go.	0.64	0.71
Being impulsive is a big part of who I am.	0.51	0.62
I know I’m supposed to be doing something else but I want to do this now.	0.80	0.78
M	0.63	0.63

Note. ACS = ADHD Cognitions Scale.

Sample 2

Maximum-likelihood-based CFA was used to further evaluate the factor structure of the seven-item version of the ACS in Sample 2. Examination of the fit indices revealed that a one-factor solution continued to provide good fit to the data for the seven-item version of the ACS in Sample 2, χ²(21) = 637.771, p < .001, RMSEA = 0.065, CFI = 0.970, TLI = 0.956, SRMR = 0.033. Factor loadings also continued to be high in Sample 2 (Table 2), ranging from 0.48 to 0.78, with an average factor loading of 0.63. Notably, the item “I know I’m supposed to be doing something else, but I want to do this now” had the highest factor loading across both samples. Given that the seven-item version of the ACS was found to demonstrate good factor structure across samples, all remaining analyses were limited to this version of the measure, which is provided in the appendix.

Reliability

The seven-item version of the ACS demonstrated good internal consistency (α = .82) and good split-half reliability (Spearman–Brown Coefficient for Unequal Length = .84) in Sample 1. Internal consistency (α = .82) and split-half reliability (Spearman–Brown Coefficient for Unequal Length = .84) were identical for the ACS in Sample 2, suggesting that internal consistency reliability for the ACS was stable across samples.

Validity

Sample 1

To examine validity in terms of the ACS’s relationship to ADHD, we used both a categorical and a dimensional approach. We examined scores on the measure among participants with and without ADHD based on several different methods of operationalizing ADHD (Table 3). Participants who self-reported ever having been diagnosed with ADHD or having a current diagnosis did not score significantly higher on the ACS than other participants, although the means were in the hypothesized direction, albeit with a small effect size. However, when participants were placed in diagnosis proxy groups based upon their BAARS scores (see Participants section), participants who met analogue DSM-5 criteria both with and without the age of onset criterion applied showed higher mean ACS scores than participants who did not with a very large effect size (Table 3). When combining proxy diagnosis methods to create a group who reported being diagnosed with ADHD and meeting DSM-5 criteria on the BAARS without age of onset (n = 7), this group also had a significantly higher mean ACS score with a large effect size. When examining associations with ADHD symptoms dimensionally, as shown in Table 4, ACS scores showed significant and moderate correlations with the Total and Symptom subscales of the BAARS.

Table 3.

Mean ACS Scores by ADHD Status—Study 1.

Basis of ADHD grouping	ADHD			Comparison			P	d
Basis of ADHD grouping	n	M	SD	n	M	SD	P	d
Self-report: Ever diagnosed	19	2.46	0.85	243	2.24	0.69	.187	0.28
Self-report: Currently diagnosed	15	2.45	0.77	247	2.24	0.70	.272	0.29
BAARS DSM-5 analogue, no age of onset	34	3.11	0.65	228	2.12	0.62	<.001	1.56
BAARS DSM-5 analogue with age of onset	11	2.94	0.60	251	2.22	0.69	.001	1.14
Ever diagnosed + BAARS DSM-5, no age of onset	7	3.00	0.86	255	2.23	0.69	.004	0.99

Note: ACS = ADHD Cognitions Scale; BAARS = Barkley Adult ADHD Rating Scale; DSM = Diagnostic and Statistical Manual of Mental Disorders.

Table 4.

Correlations Between ACS and Constructs Related to Convergent Validity, Divergent Validity, and Functional Impairment.

Convergent validity	Study 1		Study 2
Convergent validity	r	p	r	p
ADHD symptoms (BAARS Total)	.61	<.001	.56	<.001
Inattentive symptoms (BAARS INATT)	.60	<.001	.55	<.001
Hyperactive-impulsive symptoms (BAARS HY-IM)	.50	<.001	.49	<.001
Self-management to time (BDEFS)	.64	<.001	.60	<.001
Self-organization/problem solving (BDEFS)	.36	<.001	.36	<.001
Impulsivity (BDEFS)	.53	<.001	.49	<.001
Self-motivation (BDEFS)	.48	<.001	.44	<.001
Emotion regulation (BDEFS)	.44	<.001	.37	<.001
Cognitive-behavioral avoidance (CBAS)	.49	<.001	.37	<.001
Cognitive avoidance (CBAS—Cognitive)	.51	<.001	.40	<.001
Behavioral avoidance (CBAS—Behavioral)	.42	<.001	.31	<.001
Self-distraction (COPE)	—	—	.19	.001
Active coping (COPE)	—	—	−.25	<.001
Denial (COPE)	—	—	.31	<.001
Substance use (COPE)	—	—	.32	<.001
Use of emotional support (COPE)	—	—	.04	.473
Use of instrumental support (COPE)	—	—	.05	.393
Behavioral disengagement (COPE)	—	—	.30	<.001
Venting (COPE)	—	—	.14	.018
Positive reframing (COPE)	—	—	−.11	.047
Planning (COPE)	—	—	−.21	<.001
Humor (COPE)	—	—	.13	.021
Acceptance (COPE)	—	—	−.13	.023
Religion (COPE)	—	—	−.07	.216
Self-blame (COPE)	—	—	.35	<.001
Divergent validity	r	p	r	p
Negative automatic thoughts (ATQ)	.40	<.001	.39	<.001
Depressive symptoms (CES-D)	.41	<.001	.38	<.001
Generalized anxiety symptoms (GAD-7)	.45	<.001	.37	<.001
Correlation with functional impairment	r	p	r	p
Overall functional impairment (BFIS)	.48	<.001	.40	<.001
Family impairment (WFIRS Family)	—	—	.32	<.001
Work impairment (WFIRS Work)	—	—	.37	<.001
School impairment (WFIRS School)	—	—	.44	<.001
Life skills impairment (WFIRS Life Skills)	—	—	.43	<.001
Self-concept impairment (WFIRS Self-Concept)	—	—	.31	<.001
Social impairment (WFIRS Social)	——	—	.27	<.001

Note. BAARS = Barkley Adult ADHD Rating Scale; BDEFS = Barkley Deficits in Executive Functioning Scale; CBAS = Cognitive-Behavioral Avoidance Scale; COPE = Brief COPE; ATQ = Automatic Thoughts Questionnaire; CES-D = Center for Epidemiologic Studies–Depression scale; BFIS = Barkley Functional Impairment Scale; WFIRS = Weiss Functional Impairment Rating Scale.

In Sample 1 (Table 4), the ACS also correlated significantly with self-reported executive functioning problems in daily life—particularly with problems related to Self-Management to Time (r = .64). ACS also correlated moderately with cognitive and behavioral avoidance. Finally, ACS scores correlated to a moderate degree with overall functional impairment as measured by the BFIS.

In terms of divergent validity, we examined correlations of ACS with related but distinct constructs and found significant but not overly inflated correlations with negative automatic thoughts, depressive symptoms, and anxiety symptoms (Table 4), suggesting that the construct measured by ACS is distinct from the other constructs.

Sample 2

In this sample, participants who self-reported ever receiving an ADHD diagnosis or currently being diagnosed with ADHD scored significantly higher on the ACS than other participants (Table 5). Compared with Sample 1, the between-groups effect size in Sample 2 was larger. As in Sample 1, when participants were placed in diagnosis proxy groups based upon their BAARS scores, participants who met analogue DSM-5 criteria both with and without the age of onset criterion applied showed higher mean ACS scores than participants who did not with large to very large effect sizes (Table 5). When combining proxy diagnosis methods to create a group who reported being diagnosed with ADHD and meeting DSM-5 criteria on the BAARS without the age of onset (n = 9), this group also had a significantly higher mean ACS score with a large effect size. Also as in Sample 2, ACS scores showed significant and moderate correlations with the Total and Symptom subscales of the BAARS, and the ACS continued to correlate significantly with self-reported executive functioning problems in daily life. In this sample, correlations with cognitive and behavioral avoidance were weaker but still significant.

Table 5.

Mean ACS Scores by ADHD Status—Study 2.

Basis of ADHD grouping	ADHD			Comparison			p	d
Basis of ADHD grouping	n	M	SD	n	M	SD	p	d
Self-report: Ever diagnosed	23	2.61	0.94	281	2.23	0.68	.012	0.46
Self-report: Currently diagnosed	15	2.68	0.92	289	2.23	0.69	.018	0.55
BAARS DSM-5 analogue: No age of onset	31	3.08	0.72	273	2.16	0.64	<.001	1.35
BAARS DSM-5 analogue with age of onset	14	2.92	0.91	290	2.22	0.68	<.001	0.87
Ever diagnosed + BAARS DSM-5, no age of onset	9	3.12	1.08	295	2.23	0.68	<.001	0.99

Note. ACS = ADHD Cognitions Scale; BAARS = Barkley Adult ADHD Rating Scale; DSM = Diagnostic and Statistical Manual of Mental Disorders.

In Sample 2, ACS scores correlated with overall functional impairment as measured by the BFIS and with specific dimensions of impairment as measured by the subscales of the WFIRS. Correlations were strongest with school and life skills impairment, and weaker with social impairment. As in Sample 1, significant but not excessive correlations with negative automatic thoughts, depressive symptoms, and anxiety symptoms provided evidence of divergent validity. As predicted, ACS scores correlated positively with coping strategies characterized by disengagement, including cognitive (r = .40) and behavioral avoidance (r = .31) and behavioral disengagement (r = .30), denial (r = .31), and self-distraction (r = .19). As predicted, ACS scores also correlated negatively with engagement-oriented coping strategies, including active coping (r = −.25) and planning (r = −.21), while correlations were weaker but still negative with positive reframing (r = −.11) and acceptance (r = −.13). It should be noted that correlations for self-distraction and acceptance may be attenuated due to poor scale internal consistency, as disattenuated correlations were .34 and .22, respectively. Unexpectedly, some of the strongest scores between ACS and coping were for substance use (r = .32) and self-blame (r = .35).

Discussion

The present research evaluated a brief measure of ADHD-related cognitions. This measure, known as the ACS, demonstrated good reliability, factor structure, and validity across two large community samples. As expected, scores on the ACS correlated with ADHD symptoms. ACS scores were significantly higher in adults with DSM-5 proxy diagnoses based upon self-report and, in one sample, were higher for people ever diagnosed and currently diagnosed with ADHD. Higher ACS scores were also associated with greater global and domain-specific functional impairment. In both samples, ACS scores correlated with executive functioning problems in daily life—in particular with managing behavior with respect to time. Indeed, on their face, many of the ACS items relate to difficulties disengaging from preferred tasks and shifting to higher priority or more urgent but less preferred tasks. Relatedly, exploratory analyses of the relationship between the ACS and coping strategies suggest that the thoughts tapped by the measure may be associated with avoidant or disengagement coping as opposed to active strategies.

In considering the possible function of cognitions tapped by the ACS, there is considerable similarity of ACS items to permission-giving thoughts as described in the literature on cognitive therapy for substance abuse (A. T. Beck, Wright, Newman, & Liese, 2001) and trichotillomania (Rehm, Nedeljkovic, Thomas, & Moulding, 2015). In their model of cognitions in trichotillomania, Rehm and colleagues (2015) describe a process by which a “permission giving narrative” precedes hair pulling and justifies the behavior to the client (e.g., “I can just pull one hair and then stop”), whereupon the act of pulling itself serves an emotion regulation function.

Similarly, Knouse and Mitchell (2015) have proposed that “overly positive” thoughts associated with ADHD often constitute avoidance behavior—that is, the act of thinking these thoughts may be reinforced by escape from negative affect that is triggered when clients are reminded of the need to engage in nonpreferred tasks or similar anxiety-provoking situations. For example, a thought, such as “I’ll just do this one thing first,” gives the client permission to avoid a task that may begin to evoke negative affect, and the act of thinking the permission-giving thought is subsequently reinforced by the temporary removal of negative affect. One implication of this view is that the types of thoughts measured by the ACS are by no means specific to people with ADHD, although they may represent an important link in the behavioral chain connecting ADHD symptoms to impairment for this population. Another implication of this view is that topographically “positive” and “negative” cognitions may often have the same avoidance function, and so they may be positively correlated, as was observed in the current study (r = .39 with ATQ in the Study 1 sample, .40 in the Study 2 sample). Future studies will need to more rigorously evaluate these possibilities, as suggested below.

Our findings regarding the ACS should be considered in light of the limitations of the data presented here. First, we have presented results from two community samples of adults and, although the samples contained some adults who met our research criteria for ADHD, participants with the disorder were not specifically recruited for this study. As a result, our sample size of adults with ADHD (previous diagnosis or diagnosis proxy based on self-report) is relatively small. Furthermore, information on ADHD diagnoses, symptoms, impairment, and onset was all based upon self-report. The psychometrics of the ACS and evidence of clinical utility and sensitivity of the measure to treatment-related change will need to be evaluated in larger clinical samples and treatment studies that use multiple methods of assessment. Second, we did not use a prospective design, and thus we cannot provide evidence of the stability of ACS scores (test–retest reliability) nor of the predictive validity of scores across time. Finally, our results cannot shed light on the degree to which the cognitions tapped by the ACS are actually associated with avoidance behavior in daily life or the degree to which people’s perceptions of the frequency of these thoughts are accurate. Importantly, this is a general limitation of using any self-report rating scale to assess cognitive and behavioral processes that may be fleeting and dependent upon the respondent’s perception and recollection. In future studies, this limitation might be addressed by using experience sampling methodology to evaluate the frequency and consequences of thoughts tapped by the ACS in daily life.

In addition to studies further evaluating the validity of the ACS, the measure may also potentiate additional research into the role of cognitions in the manifestation of ADHD symptoms in daily life and the difficulties adults with the disorder experience in implementing the self-regulation skills they “know” but may not consistently “show.” For example, it would be interesting to investigate whether frequency of thoughts as measured by the ACS is related to the accuracy of more global self-ratings, as evaluated in studies on Positive Bias. These measures may tap two aspects of the same “overly optimistic” cognitive biases or may represent two distinct cognitive phenomena associated with ADHD. The ACS may also be useful to clinicians conducting CBT with adults with the disorder. For example, the measure could be used to identify problematic thoughts associated with failure to use self-regulation skills, and awareness of these thoughts could then serve as a cue for use of specific skills (see Knouse & Mitchell, 2015 for a detailed description). Completing the measure and discussing it may also help clients identify their own idiosyncratic “permission-giving thoughts” that tend to precede problematic avoidance.

In sum, we have presented data from two large community samples that lend preliminary support for the reliability and validity of a short self-report measure of ADHD-related cognitions. Scores on the measure are associated with self-reported ADHD symptoms and related impairment, as well as time management problems and the tendency to engage in problematic avoidance and disengagement coping. We hope that the measure will be the subject of future study in well-defined ADHD samples, and that it may prove useful in studying the contribution of maladaptive thoughts to ADHD-related impairment.

Footnotes

Appendix

Authors’ Note

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Veteran Affairs (VA), the National Institute on Drug Abuse, or the U.S. government.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Dr. Knouse was supported by a Faculty Summer Research Fellowship at the University of Richmond during the completion of this work. Dr. Mitchell was supported by a grant from the National Institute on Drug Abuse (NIDA; K23 DA032577) and has received royalties from New Harbinger Press. Dr. Kimbrel was supported by a Career Development Award (IK2 CX000525) from the Clinical Science Research and Development Service of the Department of Veterans Affairs (VA) Office of Research and Development (ORD), the VA Mid-Atlantic Mental Illness Research, Education, and Clinical Center, and the Research & Development and Mental Health Services of the Durham VA Medical Center. Dr. Anastopoulos is currently supported by an R01 grant award (MH094435-05) from the National Institute of Mental Health and by a Goal 3 Efficacy grant award (R305A150207) from the Institute of Education Sciences, U.S. Department of Education.

Author Biographies

Laura E. Knouse, PhD, is an associate professor of Psychology at the University of Richmond. Her research interests include cognitive-behaviral therapy (CBT) for adult ADHD and self-regulation strategies for academic succeess in college students.

John T. Mitchell, PhD, is an assistant professor of Psychiatry and Behavioral Sciences at Duke University School of Medicine and the Duke ADHD Program. His research interests include ADHD in adulthood, ADHD and substance use, treatment development, and mobile health.

Nathan A. Kimbrel, PhD, received his doctorate in clinical psychology from the University of North Carolina at Greensboro. He is the Assistant Director of the Genetics Research Laboratory and Co-Assistant Director of the Clinical Core of the Department of Veterans Affairs’ Mid-Atlantic Mental Illness Education, Research, & Clinical Center (MIRECC), located in Durham, NC. He is also an assistant professor in the Department of Psychiatry and Behavioral Sciences at Duke University. He primarily conducts research on suicidal and nonsuicidal self-injury, PTSD, and depression. He also has a long-standing interest in improving understanding of the association between each of these conditions and ADHD.

Arthur D. Anastopoulos, PhD, received his doctorate in clinical psychology from Purdue University and currently holds the position of Professor of Human Development and Family Studies at the University of North Carolina at Greensboro, where he also directs a specialty ADHD Clinic. Throughout his career Dr. Anastopoulos has maintained clinical and research interest in the assessment and treatment of ADHD across the life span, with a current focus on emerging adults with ADHD attending college.

References

Abramovitch

Schweiger

(2009). Unwanted intrusive and worrisome thoughts in adults with attention deficit\hyperactivity disorder. Psychiatry Research, 168, 230-233. doi:10.1016/j.psychres.2008.06.004

Anastopoulos

A. D.

Benson

J. W.

Mitchell

J. T.

Knouse

L. E.

Kimbrel

N. A.

Bray

A. C.

(2012, August). Development of a measure for assessing AD/HD-related automatic thoughts. Poster session presented at the annual meeting of the American Psychological Association (APA), Orlando, FL.

Barkley

R. A.

(2011a). Barkley adult ADHD rating scale-IV. New York, NY: Guilford Press.

Barkley

R. A.

(2011b). Barkley deficits in executive functioning scale. New York, NY: Guilford Press.

Barkley

R. A.

(2011c). Barkley functional impairment scale. New York, NY: Guilford Press.

Barkley

R. A.

Murphy

K. R.

Fischer

(2008). ADHD in adults: What the science says. New York, NY: Guilford Press.

Beck

A. T.

Wright

F. D.

Newman

C. F.

Liese

B. S.

(2001). Cognitive therapy of substance abuse. New York, NY: Guilford Press.

Beck

Perkins

T. S.

(2001). Cognitive content-specificity for anxiety and depression: A meta-analysis. Cognitive Therapy and Research, 25, 651-663. doi:10.1023/a:1012911104891

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.

10.

Buhrmester

Kwang

Gosling

S. D.

(2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3-5. doi:10.1177/1745691610393980

11.

Carver

C. S.

(1997). You want to measure coping but your protocol’s too long: Consider the brief cope. International Journal of Behavioral Medicine, 4, 92-100. doi:10.1207/s15327558ijbm0401_6

12.

Chandler

Shapiro

(2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12, 53-81. doi:10.1146/annurev-clinpsy-021815-093623

13.

Chapman

L. J.

Chapman

J. P.

(1983). Infrequency scale for personality measures (Unpublished scale). University of North Carolina at Greensboro.

14.

Golden

C. M.

Owens

J. S.

Evangelista

Micheli

(2006, November). Self-perceptions in adults with ADHD symptomatology. Poster presented at the Association of Behavioral and Cognitive Therapies conference, Chicago, IL.

15.

Hollon

S. D.

Kendall

P. C.

(1980). Cognitive self-statements in depression: Development of an automatic thoughts questionnaire. Cognitive Therapy and Research, 4, 383-395.

16.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55. doi:10.1080/10705519909540118

17.

Jöreskog

K.G.

(1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 36, 183-202. doi:10.1007/BF02289343

18.

Kimbrel

N. A.

Steffen

L. E.

Meyer

E. C.

Kruse

M. I.

Knight

Zimering

R. T.

Gulliver

S. B.

(2011). A revised measure of occupational stress for fire fighters: Psychometric properties and relationship to PTSD, depression, and substance abuse. Psychological Services, 8, 294-306.

19.

Kline

R. B.

(2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.

20.

Knouse

L. E.

Bagwell

C. L.

Barkley

R. A.

Murphy

K. R.

(2005). Accuracy of self-evaluation in adults with ADHD: Evidence from a driving study. Journal of Attention Disorders, 8, 221-234. doi:10.1177/1087054705280159

21.

Knouse

L. E.

Mitchell

J. T.

(2015). Incautiously optimistic: Positively-valenced cognitive avoidance in adult ADHD. Cognitive and Behavioral Practice, 22, 192-202. doi:10.1016/j.cbpra.2014.06.003

22.

Knouse

L. E.

Zvorsky

Safren

S. A.

(2013). Depression in adults with Attention-Deficit/Hyperactivity Disorder (ADHD): The mediating role of cognitive-behavioral factors. Cognitive Therapy and Research, 37, 1220-1232. doi:10.1007/s10608-013-9569-5

23.

Lui

J. H. L.

Johnston

Lee

C. M.

Lee-Flynn

S. C.

(2013). Parental ADHD symptoms and self-reports of positive parenting. Journal of Consulting and Clinical Psychology, 81, 988-998. doi:10.1037/a0033490

24.

Mason

Suri

(2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1-23. doi:10.3758/s13428-011-0124-6

25.

Mitchell

J. T.

Benson

J. W.

Knouse

L. E.

Kimbrel

N. A.

Anastopoulos

A. D.

(2013). Are negative automatic thoughts associated with ADHD in adulthood? Cognitive Therapy and Research, 37, 851-859. doi:10.1007/s10608-013-9525-4

26.

Ottenbreit

N. D.

Dobson

K. S.

(2004). Avoidance and depression: The construction of the Cognitive-Behavioral Avoidance Scale. Behaviour Research and Therapy, 42, 293-313. doi:10.1016/S0005-7967(03)00140-2

27.

Owens

J. S.

Goldfine

M. E.

Evangelista

Hoza

Kaiser

N. M.

(2007). A critical review of self-perceptions and the positive illusory bias in children with ADHD. Clinical Child and Family Psychology Review, 10, 335-351. doi:10.1007/s10567-007-0027-3

28.

Prevatt

Proctor

Best

Baker

Van Walker

Taylor

N. W.

(2012). The positive illusory bias: Does it explain self-evaluations in college students with ADHD? Journal of Attention Disorders, 16, 235-243. doi:10.1177/1087054710392538

29.

Radloff

L. S.

(1977). The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385-401. doi:10.1177/014662167700100306

30.

Ramsay

J. R.

(2010). CBT for adult ADHD: Adaptations and hypothesized mechanisms of change. Journal of Cognitive Psychotherapy, 24, 37-45. doi:10.1891/0889-8391.24.1.37

31.

Ramsay

J. R.

Rostain

(2008). Cognitive-behavioral therapy for adult ADHD: An integrative psychosocial and medical approach. New York, NY: Routledge.

32.

Rehm

I. C.

Nedeljkovic

Thomas

Moulding

(2015). The role of cognitions and beliefs in trichotillomania: A qualitative study using interpretative phenomenological analysis. Behaviour Change, 32, 209-230. doi:10.1017/bec.2015.11

33.

Safren

S. A.

Perlman

C. A.

Sprich

Otto

M. W.

(2005). Mastering your adult ADHD: A cognitive-behavioral treatment program, therapist guide. New York, NY: Oxford University Press.

34.

Safren

S. A.

Sprich

Mimiaga

M. J.

Surman

Knouse

L. E.

Groves

Otto

M. W.

(2010). Cognitive behavioral therapy vs. relaxation with educational support for medication-treated adults with ADHD and persistent symptoms. Journal of the American Medical Association, 304, 857-880. doi:10.1001/jama.2010.1192

35.

Shapiro

D. N.

Chandler

Mueller

P. A.

(2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1, 213-220. doi:10.1177/2167702612469015

36.

Solanto

M. V.

(2011). Cognitive-behavioral therapy for adult ADHD: Targeting executive dysfunction. New York, NY: Guilford Press.

37.

Solanto

M. V.

Marks

D. J.

Wasserstein

Mitchell

Abikoff

Alvir

Kofman

M. D.

(2010). Efficacy of metacognitive therapy for adult ADHD. American Journal of Psychiatry, 167, 958-968. doi:10.1176/appi.ajp.2009.09081123

38.

Spitzer

R. L.

Kroenke

Williams

J. W.

Löwe

(2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166, 1092-1097. doi:10.1001/archinte.166.10.1092

39.

Sprich

Knouse

L. E.

Cooper-Vince

Burbridge

Safren

S. A.

(2010). Description and demonstration of CBT for ADHD in adults. Cognitive and Behavioral Practice, 17, 9-15. doi:10.1016/j.cbpra.2009.09.002

40.

Strohmeier

C. W.

Rosenfield

DiTomasso

R. A.

Ramsay

J. R.

(2016). Assessment of the relationship between self-reported cognitive distortions and adult ADHD, anxiety, depression, and hopelessness. Psychiatry Research, 238, 153-158. doi:10.1016/j.psychres.2016.02.034

41.

Weiss

M. D.

(2000). Weiss Functional Impairment Rating Scale. Retrieved from http://naceonline.com/AdultADHDtoolkit/assessmenttools/wfirs.pdf

42.

Wymbs

B. T.

Dawson

A. E.

(2015). Screening Amazon’s Mechanical Turk for adults with ADHD. Journal of Attention Disorders. doi:10.1177/1087054715597471

43.

Zylowska

(2012). The mindfulness prescription for adult ADHD: An 8-step program for strengthening attention, managing emotions, and achieving your goals. Boston, MA: Shambhala.