Neuropsychological Functioning in Adults With ADHD and Adults With Other Psychiatric Disorders

Abstract

Objective: The aim was to investigate how well neuropsychological measures can discriminate between adults with ADHD and those with other psychiatric disorders. Method: Adults with ADHD and a clinical control group (n = 110) were included. Neuropsychological functioning was investigated using measures of inhibition, working memory, set shifting, planning, fluency, reaction-time variability, and delay aversion. Results: Adults with ADHD performed more poorly compared with clinical controls with regard to all constructs. The effects of verbal memory, inhibition, set shifting, fluency, and delay aversion remained significant when controlling for IQ. However, when controlling for basic cognitive functions, only the effects of inhibition, fluency, and delay aversion were significant. Sensitivity ranged between 64% and 75%, and specificity between 66% and 81%. Conclusion: Neuropsychological tests have a relatively poor ability to discriminate between adults with ADHD and clinical controls, but they may be used to identify individuals at particularly high risk for poor daily functioning.

Keywords

ADHD executive functions delay aversion specificity sensitivity

ADHD (American Psychiatric Association [APA], 1994) is a psychiatric disorder characterized by inattention, hyperactivity, and impulsivity. It is believed to be a heterogeneous disorder that is related to multiple neuropsychological deficits, including executive function (EF) deficits, delay aversion, and high reaction time (RT) variability (e.g., Castellanos, Sonuga-Barke, Milham, & Tannock, 2006; Nigg, Willcutt, Doyle, & Sonuga-Barke, 2005). However, the theoretical propositions within the ADHD research area are primarily based on studies of children. During recent years, there has been an increasing number of studies of ADHD in adulthood, but comparisons have most often been made with a normal control group. Thus, we still know very little about which neuropsychological deficits are truly specific to ADHD. The overall aim of the present study was therefore to compare adults diagnosed with ADHD with adults with other psychiatric disorders. In line with the view that ADHD is a heterogeneous disorder, we included a broader range of neuropsychological functions compared with most previous studies. In addition, we addressed important limitations of previous studies by including control tasks measuring basic cognitive functions (e.g., speed) and by complementing our group differences with measures of sensitivity and specificity.

Neuropsychological Functioning in ADHD

Previous studies of children with ADHD have found that they perform more poorly compared with controls on various EF tests (e.g., Barkley, 2006 for a review). There are far fewer studies on adults with ADHD, but recent research suggests that they also perform significantly worse than controls on EF tests measuring, for example, inhibition, working memory, set shifting, and planning (e.g., Alderson, Kasper, Hudec, & Patros, 2013; Boonstra, Kooij, Oosterlaan, Sergeant, & Buitelaar, 2010; Halleland, Haavik, & Lundervold, 2012; Rohlf et al., 2012; Woods, Lovejoy, & Ball, 2002). With regard to other neuropsychological deficits associated with ADHD, the “dual-pathway model” (Sonuga-Barke, 2002) suggests that, in addition to deficient EF, ADHD is also characterized by delay aversion (i.e., a tendency to choose a small immediate reward rather than a greater reward presented later). Previous studies of children have found mixed results, with some studies demonstrating significant differences between children with ADHD and controls (e.g., Dalen, Sonuga-Barke, Hall, & Remington, 2004; Solanto et al., 2001; Sonuga-Barke, Dalen, & Remington, 2003) and some failing to do so (e.g., Karalunas & Huang-Pollock, 2011; Sjöwall, Roth, Lindqvist, & Thorell, 2013; Solanto et al., 2007). To our knowledge, very few previous studies have examined delay aversion in adults with ADHD. However, Marx and colleagues (2010) found that adults with ADHD were more delay averse compared with controls on a computerized task, and Clare, Helps, and Sonuga-Barke (2010) found that self-ratings of delay aversion and delay discounting were significantly related to ratings of ADHD symptoms in normally developing adults.

Another factor of importance for ADHD is RT variability. Sergeant (2005) has defined RT variability as a measure of the energy necessary to meet task demands. High RT variability has been consistently associated with ADHD among children (e.g., Castellanos et al., 2005 for a review), but there are far fewer studies on adults. However, existing studies do indicate that high RT variability is a prominent feature of ADHD also in adulthood (see Klein, Wendling, Huettner, Ruder, & Peper, 2006, for a review). One study even reported that measures of RT variability showed the largest group differences when comparing adults with ADHD with normal controls and when comparing them with adults with anxiety disorders (Epstein, Johnson, Varia, & Conners, 2001).

The Issue of Specificity

One important limitation of previous research is that most studies have only investigated group differences. As emphasized by, for example, Doyle and colleagues, group differences alone are insufficient indices of the discriminant ability of neuropsychological measures (Doyle, Biederman, Seidman, Weber, & Faraone, 2000). As a complement to group differences, discriminant ability is preferably examined using measures of sensitivity and specificity. Studies of children that have conducted such analyses have generally found that neuropsychological tasks are better at excluding normal children from the ADHD category than at confirming ADHD in children diagnosed with the disorder (e.g., Barkley & Grodzinsky, 1994; Doyle et al., 2000). Thus, the specificity (i.e., the probability of a normal test score given that a person does not have the diagnosis) has been relatively high in these studies, whereas the sensitivity (i.e., the probability of an abnormal test score given that the person has the diagnosis) has been low. Similar conclusions have been drawn based on the few studies examining this issue in samples of ADHD adults (e.g., Lovejoy et al., 1999).

If a neuropsychological test is regarded as having good discriminatory ability for ADHD, it should be able to discriminate not only between individuals with ADHD and normally developing controls but also between in ADHD and other psychiatric disorders. Compared with normal controls, EF deficits have been reported in patients with depression (e.g., Godard, Grondin, Baruch, & Lafleur, 2011; Gohier et al., 2009; Hammar et al., 2011; Rose & Ebmeier, 2005), bipolar disorder (e.g., Godard et al., 2011; Robinson et al., 2006), general anxiety disorder (Gualtieri & Dexter, 2008), and obsessive compulsive disorder (Bannon, Gonsalvez, Croft, & Boyce, 2006). Of greater importance are the few studies that have reported measures of specificity and sensitivity when making a direct comparison between adults with ADHD and those with other psychiatric disorders. One previous study included tasks measuring attention, executive functions, psychomotor speed, and arithmetic skills, and the results showed high sensitivity (.93) and specificity (.90) when comparing adults with ADHD with normal controls (Walker, Shores, Trollor, Lee, & Sachdev, 2000). However, especially the sensitivity (.63), but also the specificity (.80), was lower when trying to distinguish between adults with ADHD and those with depression or anxiety disorders. Similar findings were presented by Taylor and Miller (1997). In addition, Katz, Wood, Goldstein, Auchenbach, and Geckle (1998) found a low overall classification rate when trying to discriminate between adults with ADHD and adults with depression, although in their study, it was the specificity that was particularly low (.40). Low overall classification rates have also been found in studies comparing adults with ADHD and psychiatric controls using questionnaire data measuring functions such as attention and memory (Solanto, Etefia, & Marks, 2004; Voorhees, Hardy, & Kollins, 2011). Finally, there is one study (Kovner et al., 1998) that stands out from the rest by demonstrating a very high overall classification rate (> 90%), but these findings must be interpreted with care due to the very small sample size used.

One serious limitation of the studies mentioned above is that they have seldom included a large range of neuropsychological functions, which may explain the low specificity and/or sensitivity. There is clearly a need to examine whether measures based on current models of heterogeneity used in the ADHD research (e.g., Castellanos et al., 2006; Nigg et al., 2005) can better discriminate between adults with ADHD and those with other psychiatric disorders. As described above, such an approach should include measures of executive functions (i.e., inhibition, working memory, set shifting, and planning), delay aversion, and RT variability. In addition, previous studies are limited in that they have not controlled for basic cognitive processes such as speed, perception, and memory. However, it has been argued that performance on EF tasks is dependent on these basic processes, and it is therefore necessary to use adequate control variables to conclude that EF deficits are of central importance to ADHD (Boonstra et al., 2010). To our knowledge, only one study of adult ADHD has used performance on non-EF tasks as control variables, and this study concluded that adult ADHD is primarily related to deficits in inhibition and set shifting (Boonstra et al., 2010). In addition, it can be noted that some studies of adults have found group differences in RT variability but not in mean RT (Klein et al., 2006), suggesting that increased RT variability is primarily related to ADHD and not secondary to overall slower processing.

Aim of the Present Study

The results presented above suggest that neuropsychological deficits are linked not only to ADHD but also to other psychiatric problems such as depression, anxiety, and bipolar disorder. However, few studies have made a direct comparison between adults with ADHD and those with other psychiatric problems, and, thus, we do not know to what extent neuropsychological deficits are specifically linked to ADHD. As emphasized by, for example, Tamm and colleagues (2012), this would be of value in terms of specificity and for providing in-depth information on the phenomena under investigation. The overall aim of the present study was therefore to investigate how well measures of neuropsychological functioning can discriminate between adults with ADHD and those with other psychiatric disorders. In contrast to previous studies, we (a) included a broad range of neuropsychological functions to account for the fact that ADHD has been described as a heterogeneous disorder, (b) included measures of sensitivity and specificity, and (c) controlled for basic cognitive processes and IQ. In line with Boonstra et al. (2010), we hypothesized that adult ADHD would primarily be related to deficits in inhibition and set shifting after controlling for basic cognitive functions and IQ.

Method

Participants

The present study included 110 participants: 57 participants (24 men/33 women) diagnosed with ADHD and 53 (16 men/37 women) in a clinical control group. The age of the participants ranged between 18 and 44 years, with a mean age of 26 years in both groups (M = 26.8, SD = 5.9 in the ADHD group, M = 25.5, SD = 5 in the clinical control group, t = 1.184, ns). Participants in both groups were recruited from three outpatient psychiatric clinics. They underwent a neuropsychiatric assessment conducted by a licensed psychologist. The assessment included a clinical judgment using the second version of the Diagnostic Interview for ADHD in Adults (DIVA; Kooij, 2013). This semistructured interview consists of two parts: one for assessing the presence of all 18 Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; APA, 1994) criteria in childhood (primary school, age 6-12) and the present time; the other for assessing impairment in five areas of functioning (i.e., education, work, family, social/relationships, and self-confidence) in childhood and at the present time. In addition, current levels of ADHD symptoms were assessed using self-report on the Adult ADHD Self-Report Scale (ASRS-v1.1; Kessler et al., 2005). The psychologist also interviewed a close relative of the participant, in most cases the mother, to obtain a detailed anamnesis. All participants in the ADHD group met the full diagnostic criteria according to the DSM-IV (APA, 1994). Finally, all participants underwent testing of global intellectual ability using the fourth edition of Wechsler Adult Intelligence Scale (WAIS-IV; Wechsler, 2008). Exclusion criteria were an IQ score of <.80 on WAIS-IV and the presence of substance-related disorders. In addition to a primary ADHD diagnosis, the participants in the ADHD group also met the DSM-IV criteria for the following comorbid diagnoses: mood disorders including “major depression” (15.8%), bipolar disorder (5.3%), unspecified anxiety disorder (UNS) (5.3%), panic disorder (3.5%), obsessive compulsive disorder (1.7%), social phobia (1.7%), and personality disorders (5.3%). Five of the participants had more than one comorbid diagnosis. The diagnoses in the clinical control group were the following: mood disorders including “major depression” (43.4%), bipolar disorder (11.3%), anxiety disorder UNS (15.1%), social phobia (9.4%), panic disorder (1.8%), obsessive compulsive disorder (5.7%), general anxiety disorder (5.7%), posttraumatic stress disorder (5.7%), eating disorders (1.8%), and personality disorders (11.3%). Fifteen participants had more than one diagnosis.

Procedures and Measures

Participants were recruited from the psychiatric clinics’ waiting rooms, and individuals interested in participating were thereafter contacted by phone or letter. At the first visit, they were given more detail and were asked to sign a written consent form. Participants visited the clinic on two occasions to perform the neuropsychological testing (see detailed descriptions below), and they were also asked to complete two questionnaires. As compensation for participating, the individuals in the ADHD group received two movie tickets (value approx. 20 Euros) and those in the control group received 50 Euros. The local ethics committee approved the study.

Neuropsychological assessment of executive functions

Most of the neuropsychological tests used in the present study were selected from either Delis Kaplan Executive System (D-KEFS; Delis, Kaplan, & Kramer, 2001) or WAIS-IV (Wechsler, 2008). In addition, a few computerized EF tests used in previous studies on ADHD were used. Below follows a detailed description of all included measures.

Verbal working memory was measured by two subtests from WAIS-IV: Letter-Number Sequencing and Digit Span. In Letter-Number Sequencing, participants have to repeat a series of randomly mixed letters and numbers, starting with the numbers in ascending order, followed by the letters in alphabetical order. In Digit Span Backward, participants have to repeat the series in a backward order, and in Digit Span Sequencing, the numbers are randomly presented and must be repeated in the correct number order. Digit Span Forward was not included as this test primarily measures short-term memory.

Spatial working memory was measured using the Find-the-Phone Task (Delosis, London). This task is similar in design to the spatial working memory task included in the Cambridge Neuropsychological Test Automated Battery (CANTAB; Owens, Downes, Sahakian, Polkey, & Robbins, 1990). In the version used in the present study (Sjöwall et al., 2013), a number of telephones are shown on the computer screen. Participants are instructed to find the telephone that is ringing by clicking on the phones using the computer mouse. If they find the correct telephone, the signal stops and a new telephone starts ringing until all telephones on the screen have rung once. Participants are told that each phone will only ring once and that the goal of the task is to find all the ringing phones without selecting the same phone twice. The adult version used in this study included six sessions: two with six telephones, two with eight telephones, and two with ten telephones. The number of incorrect answers was used as a measure of spatial working memory.

Inhibition was measured using the Color Word subtest from D-KEFS and a Navon-like task. In the Color Word Test, only the third trial (i.e., interference trial) was used. In this trial, participants are presented rows of words printed in dissonant colors and are instructed to inhibit reading the words and, instead, name the dissonant colors in which the words are printed. The number of seconds needed to complete the trial was used as a measure of inhibition. The Navon paradigm has been used previously (e.g., Miyake, Friedman, Emerson, Witzki, & Howerter, 2000). In the present version (Delosis, London; Sjöwall et al., 2013), a circle consisting of small squares, or the opposite, a square consisting of small circles, is displayed on the computer screen. In one session, the participants are asked to respond to the local stimuli (i.e., the small squares making up the circle), and in the other session, they are asked to respond to the global stimuli (i.e., the circle made up by the squares). In total, 20 objects (10 squares and 10 circles) were shown. The score used was mean reaction time.

Set shifting was measured using the shifting trials from the Color Word Task and the Verbal Fluency Task from D-KEFS, and a third trial of the Navon task. During the shifting condition of the Color Word Task, the participants are asked to switch back and forth between naming the dissonant ink colors and reading the words. Completion time was used as a measure of set shifting. In the shifting condition of the Verbal Fluency Task, participants are instructed to alternate between saying words from two different semantic categories as quickly as possible for 60 s. Number of correct shifts was used as a second measure of set shifting. Finally, set shifting was measured using the Navon task (see description above under the heading “inhibition”). A third trial was performed in which participants had to shift between responding to the local or the global stimuli. Mean reaction time for the third trial was used as a measure of shifting.

Verbal fluency was measured using the Fluency Task from D-KEFS. During 60 s/trial, the participants are requested to say as many words as possible that begin with a specified letter (F, A, or S) or a designated semantic category (animals’ or boys’ names). The mean standard score on the two conditions (i.e., letter fluency and category fluency) was used as a measure of verbal fluency.

Planning was measured by the Sorting test and the Tower test from D-KEFS. In Sorting test (i.e., free sorting), the participants are instructed to sort cards into two groups according to as many different categorization rules as possible and to describe the concepts or the rule of categorization. In Condition 2 (i.e., sort recognition), the examiner sorts the cards into two groups, and the participant has to identify the correct categorization rule. The mean number of correct sorts was used as a measure of planning. For the Tower Test, the participants are instructed to build towers with disks (varying in size) in the fewest number of moves possible using prespecified rules. The Total Achievement Score, which is the mean of three measures (i.e., number of moves to completion, the item-completion time, and correct number of towers), was used as a measure of planning.

Delay aversion was measured using the “Quick Delay Questionnaire” (QDQ) developed by Clare et al. (2010). The QDQ is a 10-item self-rating instrument for adults that measures delay aversion and discounting. Ratings are made on a scale from 1 (do not agree at all) to 5 (agree fully), and high values indicate high levels of delay-related behaviors.

Reaction time variability was measured by the standard deviation of the participants’ reaction time for correct responses on the two non-set-shifting trials of the Navon-like task (see task description under the heading “inhibition” above).

Control variables

To control for more basic cognitive functioning such as speed, verbal abilities, and memory, the following three subtests from the WAIS-IV (Wechsler, 2008) were used: Block Design, Vocabulary, and Digit Span Forward. In addition, measures of response speed (mean RTs) were collected from D-KEFS (i.e., Color-Word and Fluency Subtests) and the Navon Task. Finally, and in line with the D-KEFS manual (Delis et al., 2001), two of the measures of inhibition and the category fluency measure (see description above) were used as control variables when studying the effects of set shifting. Table 1 describes in more detail which control variable is used for each task.

Table 1.

Overview of all Neuropsychological Tests Included in the Study and Their Respective Control Tests.

Neuropsychological domain	Neuropsychological test	Control domain	Control test
Verbal working memory	WAIS-IV, Letter-number sequencing	Verbal short-term memory	WAIS-IV, Digit Span—Forward
	WAIS-IV, Digit Span—Backward	Verbal short-term memory	WAIS-IV, Digit Span—Forward
	WAIS-IV, Digit Span—Sequencing	Verbal short-term memory	WAIS-IV, Digit Span—Forward
Spatial working memory	Find the phone task, errors	Verbal short-term memory	WAIS-IV, Digit Span—Forward
Inhibition	D-KEFS C-W test, Inhibition trial	Response speed	D-KEFS C-W test, Color naming
Inhibition	Navon task, N-ST	Response speed	D-KEFS, Trail making—Motor speed
Set shifting	Mean RT, Navon task, ST	Inhibition	Navon task, mean RT, N-ST
	D-KEFS, C-W test, switching trial	Inhibition	D-KEFS C-W test, Inhibition trial
	D-KEFS—Category switching	Fluency	D-KEFS category fluency
Fluency	D-KEFS, Letter fluency	Vocabulary	WAIS-IV, Vocabulary
Fluency	D-KEFS, Category fluency	Vocabulary	WAIS-IV, Vocabulary
Planning	D-KEFS, Tower test	Visual-constructive abilities	WAIS-IV, Block design
Reaction time variability	Navon task, Standard deviation in reaction time, N-ST	Response speed	Navon task, mean RT, N-ST
Delay aversion	Quick delay questionnaire	Inhibition	D-KEFS C-W test, Inhibition trial

Note. WAIS-IV = Wechsler Adult Intelligence Scale–4th edition; D-KEFS = Delis Kaplan Executive Function System; C-W Test = Color Word Test; NST = nonswitch trials; ST = Switch trial.

Finally, intelligence was estimated using the General Ability Index (GAI) from the WAIS-IV (Wechsler, 2008). GAI is composed of the following subtests: Similarities, Vocabulary, Information, Matrix Reasoning, Block Design, and Visual Puzzles. Previous studies of the WAIS-III have found a very high correlation (r = .96-.97) between GAI and Full-Scale IQ in clinical samples (Iverson, Lange, Viljoen, & Brink, 2006; Tulsky, Saklofske, Wilkins, & Weiss, 2001), indicating that GAI is a good measure of general mental ability (g).

Statistical Analysis

Means and standard deviations were computed for all measures, and group differences were calculated using two-tailed independent t-tests. Effect sizes were calculated using partial eta squared (η²). We considered results with small effect sizes (below .06) to be negligible. We further considered effect sizes between .06 and .14 as medium and those larger than .14 as large. Analysis of covariance (ANCOVA) was then performed comparing the groups on EF measures while controlling first for sex only, second for sex and IQ, and third for sex and control tests. Next, a Binary logistic regression analysis was performed, including the variables for which a significant group difference had been found. This analysis allowed us to study independent effects of the different neuropsychological variables, as well as to investigate how well different models could discriminate between the groups in terms of sensitivity and specificity.

Results

Group Differences

The results of the ANCOVAs are presented in Table 2. For working memory, the results showed that the adults with ADHD performed more poorly compared with the clinical controls with regard to two of the tasks: one verbal (i.e., the Letter-Number Sequencing Task) and one spatial (i.e., Find-the-Phone Task). However, no significant group differences were found for the two Digit Span subtests (i.e., Backwards or Sequencing). The size of the effects was small for all working memory tasks, except for the Letter-Number Sequencing Task, which showed a medium-sized effect. When controlling for short-term memory, the results showed that the group difference found for the Letter-Number Sequencing Task was no longer significant. The effect for the spatial working memory task just missed significance (p = .052) when controlling for short-term memory and was nonsignificant when controlling for IQ.

Table 2.

Means and Standard deviations for all Major Variables Included in the Study and Results of ANCOVAs.

	ADHD	Control group	ANCOVA sex	ANCOVA sex + IQ	ANCOVA sex + Control tests
	M (SD)	M (SD)	F	F	F
Verbal working memory
Letter-number sequencing (standard score)	8.5 (1.9)	9.8 (2.8)	6.18 (0.06)*	6.92 (0.07)*	1.05 (0.01)
Digit Span Backward (standard score)	9.2 (2.6)	9.7 (2.9)	0.71 (0.01)	0.56 (0.01)	0.37 (0.01)
Digit Span Sequencing (standard score)	7.7 (2.3)	8.4 (2.2)	2.43 (0.02)	1.75 (0.02)	0.24 (0.01)
Spatial working memory
Find the phone task (errors)	29.3 (20.1)	21.2 (18.2)	4.56 (0.04)*	3.85 (0.04)^† a	1.99 (0.02)
Inhibition
Color-word, inhibition trial (errors)	7.1 (3.8)	9.6 (3.3)	11.87 (0.10)***	12.25 (0.11)*	9.22 (0.08)*
Navon, inhibition trials (reaction times)	809 (321)	718 (223)	2.96 (0.03)^a	1.93 (0.02)	.81 (0.01)
Set shifting
Color word, shifting trial (standard score)	7.1 (3.5)	9.5 (2.9)	13.78 (0.12)***	12.65 (0.11)*	3.27 (0.03)^†
Category fluency, shifting (standard score)	10.4 (3.2)	12.1 (3.5)	3.96 (0.04)*^b	4.09 (0.04)*^b	1.65 (0.02)
Navon task, shifting trial (reaction times)	1,347 (479)	1,151 (358)	6.54 (0.06)*	5.49 (0.05)*	0.39 (0.01)
Fluency
Letter fluency (standard score)	10.79 (3.9)	12.4 (3.7)	4.10 (0.04)*^b	4.32 (0.04)*^b	5.76 (0.06)*^b
Category fluency (standard score)	11.2 (3.9)	12.6 (4.3)	2.20 (0.02)	2.02 (0.02)	3.18 (0.03)^†
Planning
Tower test (standard score)	10.2 (2.6)	11.2 (2.6)	3.99 (0.04)*	3.70 (0.03)^†	3.69 (0.04)^†
Reaction time variability
Navon task (SD in reaction time)	1,848 (574)	1,630 (547)	4.15 (0.04)*	2.79 (0.03)^†	0.53 (0.01)
Delay aversion
Delay aversion questionnaire	3.3 (0.66)	2.8 (0.70)	10.8 (0.10)***	9.99 (0.10)*	7.06 (0.071)**

Significant when excluding patients on psychopharmacological medication.

Nonsignificant when excluding patients on psychopharmacological medication.

†

p < .10. *p < .05. **p < .01. ***p < .001.

For inhibition, the results showed a significant group difference for the Color Word task with a medium effect size but no significant difference for the Navon task, and this effect size was small. When controlling for either IQ or response speed, the results remained the same.

For set shifting, the results showed that the adults with ADHD performed more poorly than the clinical controls on all three tasks (i.e., Color Word, Category Switching, and the Navon Task). The size of these effects was medium, and all group differences remained significant when controlling for IQ. However, when controlling for inhibition, the effects were no longer significant, except for a tendency toward a significant effect for the Color Word Task (p = .073).

For fluency, the results showed that the ADHD group performed more poorly than the controls on letter fluency but not category fluency. All effect sizes were in the small range, but the significant group effect for letter fluency remained significant when controlling for either IQ or vocabulary. The results of the second task (i.e., Category Fluency) was marginally significant (p = .078) when controlling for vocabulary.

For planning, the ADHD group was shown to perform more poorly than the controls, with an effect size in the small range. When controlling for IQ or visual constructive abilities, there was a tendency toward a significant group difference (both ps < .06).

For RT variability, the ADHD group showed significantly higher variability compared with the controls, with a small effect size. When controlling for IQ, the significant group difference became only marginally significant (p = .098), and group differences were nonsignificant when controlling for response speed.

For delay aversion, which was measured using self-ratings, the adults with ADHD reported a higher degree of delay aversion than the clinical controls did. The effect size for this comparison was medium, and the effect remained significant when controlling for either IQ or inhibition.

Logistic Regression Analyses

To determine how well the neuropsychological variables could classify the participants into the correct group, a set of binary logistic regression analyses was performed (see Table 3). First, we tested a full model with all the 10 variables for which significant group differences had been found in the ANCOVAs (Model 1). This model was shown to be statistically significant, χ² = 28, 37, p < .001, indicating that the predictors as a set reliably distinguished between adults with ADHD and clinical controls. The measures included in Model 1 correctly classified 66% of the participants, with a sensitivity of 64 and a specificity of 67. However, only the effect of delay aversion significantly predicted group membership, with a marginally significant effect for Letter Number Sequencing (see Table 4). A model with only these two variables was also significant, χ² = 22,15, p < .001. This model (Model 2) classified 71% of the participants correctly. Compared with Model 1, the sensitivity of Model 2 was higher (75), whereas the specificity was lower (66). Finally, we examined a third model (Model 3), which was the same as Model 1, although we excluded delay aversion, as the importance of this variable may be inflated given that this function was measured using self-ratings rather than a laboratory task. When excluding delay aversion, none of the variables contributed significantly to discriminating between the two groups. Model 3 was significant, χ² = 17.99, p < .05, and classified 64.2% of the participants correctly. Compared with Model 1, the sensitivity was lower (54) and the specificity higher (73).

Table 3.

Sensitivity and Specificity for Model 1-3.

Description	Sensitivity	Specificity	Overall classification
Model 1
All 10 variables for which significant group differences were found in the ANCOVAs	64.3	67.4	65.9
Model 2
Only the two variables that had significant or marginally significant effects in Model 1	75.0	65.9	70.5
Model 3
Same as Model 1 described above, except that delay aversion was excluded	66.7	81.4	75.7

Table 4.

Results of the Logistic Regression Analysis, Including the 10 Variables for Which Significant Group Differences Had Been Found.

Variables	Estimate	Wald χ²
Verbal working memory
Letter-number sequencing (standard score)	.17	3.14^†
Spatial working memory
Find the phone task (errors)	.01	0.09
Inhibition
Color word, inhibition trial (errors)	.10	0.03
Set shifting
Color word, shifting trial (standard score)	.11	1.26
Category fluency, shifting (standard score)	.09	0.75
Navon task, shifting trials (reaction times)	.00	1.12
Fluency
Letter fluency (standard score)	.09	0.29
Planning
Tower test (standard score)	.11	0.68
Reaction time variability
Navon task (SD in reaction time)	.00	1.35
Delay aversion
Delay aversion questionnaire	.43	6.53*

†

p < .10. *p < .05.

Effects of Stimulant Medication

Previous studies have shown mixed results regarding the effects of psychostimulant medication on cognitive functioning in adults (e.g., Aron, Dowson, Sahakian, & Robbins, 2003; Boonstra, Kooij, Oosterlaan, & Sergeant, 2005; Tucha et al., 2011). However, as stimulants could possibly have affected our results, we performed some complimentary analyses where we excluded the participants in the ADHD group who were taking psychostimulant medication (n = 18). If stimulant medication improves cognitive functioning, group differences between the ADHD group and the clinical control group would be expected to increase when excluding the individuals on medication. However, the results showed just the opposite, as the effects mostly changed from significant to nonsignificant (see footnote in Table 2). It is worth noting that all effects that changed had small effect sizes when all participants were included, which indicates that the changes in the results are probably due to insufficient power in these complimentary analyses to detect group differences of small sizes. Only two effects (i.e., inhibitory errors on the Navon task and the effect of spatial working memory when controlling for IQ) changed from nonsignificant to significant when excluding the individuals on medication. With regard to the logistic regression analyses, the results showed that when only including medication-naïve participants, there was an increase in specificity (81%-88%), whereas the sensitivity remained the same or decreased (48%-67%). In total, there was only a small increase in the overall classification rate for Model 1 (76%) and Model 2 (75%) but not for Model 3 (72%) when excluding participants taking psychostimulant medication.

Discussion

The overall aim of the present study was to investigate to what extent measures of neuropsychological functioning can discriminate between adults with ADHD and those with other psychiatric disorders. Without controlling for IQ or control tests, we found significant group differences with regard to the following functions: verbal and spatial working memory, inhibition, set shifting, fluency, planning, RT variability, and delay aversion. The effect sizes were small (below .06) to medium (between .06 and .14). After controlling for IQ, the significant effects remained for inhibition, set shifting, fluency, and delay aversion. When controlling for basic cognitive functions, significant group differences were only found for inhibition, fluency, and delay aversion.

Critical Issues in Research on ADHD

The present study addressed a number of important limitations of previous research. First, we compared individuals with ADHD with a psychiatric control group to determine which neuropsychological deficits are specific to the disorder. Second, we studied not only simple group differences but also measures of sensitivity and specificity to examine the discriminatory ability of neuropsychological measures in adult ADHD. In line with the view that ADHD is a heterogeneous disorder (Castellanos et al., 2006; Nigg et al., 2005), we also included a broader range of neuropsychological functions compared with most previous studies. A final critical issue that was addressed concerned the fact that we controlled for IQ and basic cognitive functions.

Neuropsychological Functions in Adults With ADHD

We found significant group differences within all the neuropsychological domains included in the study (i.e., inhibition, working memory, set shifting, fluency, planning, and delay aversion). However, results were not totally consistent, as we included several measures for most functions, and for verbal working memory, inhibition, and fluency, significant group differences were not found for all the included measures. In addition, it should be noted that most group differences were in the small range, except for one of the measures for each verbal working memory, inhibition, fluency, and delay aversion, for which medium effect sizes were found.

Our finding that adult ADHD is associated with a range of different neuropsychological deficits is in line with previous studies comparing adults with ADHD and normal controls (e.g., Alderson et al., 2013; Boonstra et al., 2010; Halleland et al., 2012; Rohlf et al., 2012; Woods et al., 2002), and extends these findings by showing that adults with ADHD perform more poorly also compared with a psychiatric control group. However, in contrast to previous studies, we only found a significant group difference for the Letter-Number Sequencing Task and not the two subtests from the Digit Span Task (e.g., Kovner et al., 1998; Rohlf et al., 2012; Walker et al., 2000). It should be noted that, for example, Crowe (2000) has argued that the Letter-Number Sequencing Task assesses not only verbal working memory but also visuospatial functions. Thus, it may be that adults with ADHD are primarily deficient with regard to spatial working memory. This interpretation is supported by the present study, which showed a significant group difference for spatial working memory, and by studies of ADHD in children, which have demonstrated larger effect sizes for spatial compared with verbal working memory (Martinussen, Hayden, Hogg-Johnson, & Tannock, 2005). However, a recent meta-analytic review of adults (Alderson et al., 2013) showed similar effect sizes for the verbal and visuospatial domain and instead suggested that verbal tasks that place greater demands on the central executive (i.e., tasks requiring storage and manipulation) yield larger effect sizes, at least when comparing adults with ADHD with normal controls. As it could be argued that the Letter-Number Sequencing task places higher demands on the central executive than the Digit Span Task does, this may be an alternative explanation for our failure to find significant group differences for the two Digit Span subtests.

Controlling for IQ and Basic Cognitive Functions

The issue of whether it is advisable to control for intelligence has been debated. It has been argued that controlling for IQ may eliminate some of the differences between ADHD and controls that are a result of the variable of interest: ADHD (Nigg, 2001). However, a stronger case for the importance of EF deficits in ADHD could certainly be made if the deficits remain when controlling for IQ. Consequently, we feel that researchers might be best off reporting their data with and without controlling for IQ, letting the reader make his or her own interpretation of the results.

In the present study, the effects of inhibition, verbal working memory, fluency, set shifting, and delay aversion remained significant when controlling for IQ. These are exactly the same functions that remained significant (i.e., p < .05) when controlling for IQ in the study by Boonstra and colleagues (2010), except that they did not include delay aversion and made a somewhat different interpretation of the results, as they used a more conservative alpha level. Interestingly, the results are similar even though Boonstra and colleagues used a normal control group.

When controlling for basic cognitive functions (i.e., speed, verbal abilities, and memory), only the significant effects of inhibition, fluency, and delay aversion remained significant. Almost no previous studies have controlled for basic cognitive functions, and it should be emphasized that this is a very strict control. However, conducting this control enabled us to examine whether adult ADHD is specifically linked to executive deficits or to cognitive deficits more in general. The fact that the effect of inhibition remained significant when controlling for basic cognitive processes is in line with the study by Boonstra and colleagues (2010) and Barkley’s (1997) hybrid model of ADHD. In Barkley’s model, deficient inhibition is seen as most central to ADHD, and this deficit leads to secondary impairments in other executive functions. This could be interpreted to mean that inhibition should survive control for IQ and basic cognitive functions, although it shares substantial variance with other executive functions. The proposed overlap between inhibition and other executive functions may explain why the effect of set shifting became nonsignificant when controlling for inhibition as well as our finding that inhibition did not contribute independently in the logistic regression analysis.

With regard to the overlap between different neuropsychological functions, it is also interesting to note that the significant group difference for delay aversion remained significant when controlling for inhibition. This finding is in line with the dual-pathway model, suggesting two separate pathways to ADHD: one motivational pathway, which is characterized by delay aversion, and one executive pathway, which is characterized by poor inhibitory control (Sonuga-Barke, 2002). In studies of children, support for this theory has been mixed, with larger effect sizes generally being found for preschool children than for school-aged children (Karalunas & Huang-Pollock, 2011). However, Marx and colleagues (2010) found the largest effect sizes for adults when comparing them with adolescents (age 13-18 years) and school-aged children (8-12 years). In summary, there is some support for a deficit in delay aversion among adults with ADHD, although more studies of this age group are clearly needed before any conclusions can be drawn.

The Issue of Specificity

As a complement to group differences, we examined discriminant ability using measures of sensitivity and specificity. As mentioned in the introduction, studies of children have generally found that neuropsychological tasks are better at excluding normal children from the ADHD category than at confirming ADHD in children diagnosed with the disorder (e.g., Barkley & Grodzinsky, 1994; Doyle et al., 2000). Stated in another way, the sensitivity of neuropsychological tests is low, whereas the specificity is high, or at least higher. Some studies of adults with ADHD have drawn the same conclusion (Lovejoy et al., 1999). However, practically all studies of adults with ADHD have used normal controls as the comparison.

With a sensitivity ranging between 64% and 75% and a specificity ranging between 66% and 81% when including the whole sample, the present study found rates similar to that found by Walker et al. (2000). However, compared with Katz and colleagues (1998), the sensitivity was lower, whereas the specificity was higher, and the sensitivity and specificity were considerably lower compared with the results found by Kovner and colleagues (1998). Except for the study by Kovner and colleagues, which used a very small sample size, the results from the present study and other previous studies demonstrate that neuropsychological tests are generally not very good at discriminating between adults with ADHD and adults with other psychiatric disorders. It is important to emphasize that these results were found even though many of the tests demonstrated significant group differences. In addition, the present study was not able to increase the overall classification rate by including a much larger range of neuropsychological functions compared with most previous studies. In contrast to previous studies, a self-rating measure of delay aversion was, for example, included. However, even though the largest group differences were found for this measure, the specificity increased from 67% to 81% when excluding delay aversion. Thus, demonstrating significant group differences and medium effect sizes is clearly not the same as demonstrating good ability to discriminate between groups.

Limitations and Future Directions

There are a few limitations worth mentioning. First, participants who were on medication with central stimulants were included in the study together with medication-naïve participants. Some previous studies have found that medication improves cognitive performance in adults with ADHD (e.g., Aron et al., 2003; Boonstra et al., 2005). However, we conducted some complimentary analyses that excluded participants on medication, and group differences were generally not found to be larger. In addition, the overall classification rate was relatively similar in these complimentary analyses. Thus, including participants on medication did not appear to affect the results substantially. A second limitation was that not all the included functions were studied using multiple measures. It would have been valuable to include more measures of inhibition and RT variability, and it would also have been valuable to include a laboratory test of delay aversion. However, in line with recent theories emphasizing the neuropsychological heterogeneity within ADHD (e.g., Castellanos et al., 2006; Nigg et al., 2005), we decided to prioritize studying a broad range of functions over studying fewer functions in more detail. Therefore, the present study is the first investigation of adult ADHD to include executive functions, RT variability, and delay aversion in the same study.

With regard to future research, there is clearly a need for more studies comparing adults with ADHD and those with other psychiatric disorders. Preferably, these studies should include measures of sensitivity and specificity. Another important issue for future research is the role of emotion regulation in adult ADHD, as recent studies of children with ADHD have emphasized the need to view emotion regulation deficits as a central part of the disorder (Martel, 2009; Sjöwall et al., 2013).

In conclusion, the present results suggest that adults with ADHD differ significantly from those with other psychiatric disorders on a range of different neuropsychological functions. However, effect sizes were relatively small and only the effects of inhibition, fluency, and delay aversion remained significant when controlling for IQ or basic cognitive functions. In addition, the results demonstrated that, despite significant group differences, the ability of these tests to discriminate between groups was relatively poor, with about 25% to 30% of participants being misclassified. These results could be interpreted to mean that neuropsychological tests cannot be used to distinguish between different psychiatric groups. Instead, these measures might be of greater value for identifying strengths and difficulties for individuals with psychiatric disorders, thereby determining to what extent they are at greater risk for developing functional impairments in daily life. An important avenue for future research will therefore be to establish to what extent different neuropsychological deficits can explain the link between adult ADHD and functional impairments such as poor academic achievement, problems with social relations, unemployment, criminality, and substance use.

Footnotes

Acknowledgements

The authors wish to thank Tobias Johansson for valuable help with data collection.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Thorell has served as a consultant for Shire and is on the advisory board of PRIMA. Ms. Holst has no potential conflict of interest to report.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the Swedish Research Council to the second author.

Author Biographies

Ylva Holst is a licensed psychologist and a specialist in clinical neuropsychology, working at Psychiatry South, Stockholm County Council, Sweden. She has large clinical experience working with adults with ADHD.

Lisa B. Thorell is an associate professor at the Department of Clinical Neuroscience at the Karolinska Institutet, Stockholm, Sweden. She has published a large number of research papers related to many different aspects of ADHD (e.g., development, neuropsychological deficits, gender differences, psychosocial treatment).

References

Alderson

R. M.

Kasper

L. J.

Hudec

K. L.

Patros

C. H.

(2013). Attention-deficit/hyperactivity disorder (ADHD) and working memory in adults: A meta-analytic review. Neuropsychology, 27, 287-302.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Aron

A. R.

Dowson

J. H.

Sahakian

B. J.

Robbins

T. W.

(2003). Methylphenidate improves response inhibition in adults with attention-deficit/hyperactivity disorder. Society of Biological Psychiatry, 54, 1465-1468.

Bannon

Gonsalvez

C. J.

Croft

R. J.

Boyce

P. M.

(2006). Executive functions in obsessive-compulsive disorder: State or trait deficits? Australian and New Zealand Journal of Psychiatry, 40, 1031-1038.

Barkley

R. A.

(1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121, 65-94.

Barkley

R. A.

(2006). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment. New York, NY: Guilford Press.

Barkley

R. A.

Grodzinsky

G. M.

(1994). Are tests of frontal lobe functions useful in the diagnosis of attention disorders? Clinical Neuropsychologist, 8, 121-139.

Boonstra

A. M.

Kooij

J. J. S.

Oosterlaan

Sergeant

J. A.

(2005). Does methylphenidate improve inhibition and other cognitive abilities in adults with ADHD? Journal of Clinical and Experimental psychology, 27, 278-298.

Boonstra

A. M.

Kooij

J. J. S.

Oosterlaan

Sergeant

J. A.

Buitelaar

J. K.

(2010). To act or not to act, that’s the problem: Primarily inhibition difficulties in adult ADHD. Neuropsychology, 24, 209-221.

10.

Castellanos

F. X.

Sonuga-Barke

E. J. S.

Milham

M. P.

Tannock

(2006). Characterizing cognition in ADHD: Beyond executive dysfunction. TRENDS in Cognitive Sciences, 10, 117-123.

11.

Castellanos

F. X.

Sonuga-Barke

E. J. S.

Scheres

Di Martino

Hyde

Walters

J. R.

(2005). Varieties of attention/hyperactivity intra-individual variability. Biological Psychiatry, 57, 1416-1423.

12.

Clare

Helps

Sonuga-Barke

E. J. S.

(2010). The Quick Delay Questionnaire: A measure of delay aversion and discounting in adults. Attention Deficit Hyperactivity Disorder, 2, 43-48.

13.

Crowe

S. F.

(2000). Does the letter-number-sequencing task measure anything more than digit span? Assessment, 7, 113-117.

14.

Dalen

Sonuga-Barke

E. J. S.

Hall

Remington

(2004). Inhibitory deficits, delay aversion, and preschool AD/HD: Implications for the dual-pathway model. Neural Plasticity, 11, 1-11.

15.

Delis

Kaplan

Kramer

(2001). Delis-Kaplan executive function system. San Antonio, TX: The Psychological Corporation.

16.

Doyle

A. E.

Biederman

Seidman

L. J.

Weber

Faraone

S. V.

(2000). Diagnostic efficiency of neuropsychological test scores for discriminating boys with and without attention deficit-hyperactivity disorder. Journal of Consulting and Clinical Psychology, 68, 477-488.

17.

Epstein

Johnson

D. E.

Varia

I. M.

Conners

(2001). Neuropsychological assessment of response inhibition in adults with ADHD. Journal of Clinical and Experimental Neuropsychology, 23, 362-371.

18.

Godard

Grondin

Baruch

Lafleur

(2011). Psychosocial and neurocognitive profiles in depressed patients with major depressive disorder and bipolar disorder. Psychiatry Research, 190, 244-252.

19.

Gohier

Ferracci

Surguladze

S. A.

Lawrence

El Hage

Zied Kefi

. . . Le Gall

(2009). Cognitive inhibition and working memory in unipolar depression. Journal of Affective Disorders, 116, 100-105.

20.

Gualtieri

C. T.

Dexter

(2008). The frequency of cognitive impairment in patients with anxiety, depression, and bipolar disorder: An unaccounted source of variance in clinical trials. Journal of Clinical Psychiatry, 69, 1122-1130.

21.

Halleland

H. B.

Haavik

Lundervold

A. J.

(2012). Set shifting in adults with ADHD. Journal of the International Neuropsychological Society, 18, 728-737.

22.

Hammar

Å.

Strand

Årdal

Schmid

Lund

Elliott

(2011). Testing the cognitive effort hypothesis of cognitive impairment in major depression. Nordic journal of Psychiatry, 65, 74-80.

23.

Iverson

G. L.

Lange

R. T.

Viljoen

Brink

(2006). WAIS-III General Ability Index in neuropsychiatry and forensic psychiatry inpatient samples. Archives of Clinical Neuropsychology, 21, 77-82.

24.

Karalunas

S. L.

Huang-Pollock

C. L.

(2011). Examining relationships between executive functioning and delay aversion in attention deficit hyperactivity disorder. Journal of Clinical Child & Adolescent Psychology, 40, 837-847.

25.

Katz

L. J.

Wood

D. S.

Goldstein

Auchenbach

R. C.

Geckle

(1998). The utility of neuropsychological tests in evaluation of attention-deficit/hyperactivity disorder (ADHD) versus depression in adults. Assessment, 5, 45-51.

26.

Kessler

R. C.

Adler

Ames

Demler

Faraone

Hiripi

. . . Walters

E. E.

(2005). The World Health Organization Adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychological Medicine, 35, 245-256.

27.

Klein

Wendling

Huettner

Ruder

Peper

(2006). Intra-subject variability in attention-deficit hyperactivity disorder. Biological Psychiatry, 60, 1088-1097.

28.

Kooij

J. J. S.

(2013). Adult ADHD: Diagnostic assessment and treatment (3rd Ed). New York, NY: Springer-Verlag Publishing

29.

Kovner

Budman

Frank

Sison

Lesser

Halperin

(1998). Neuropsychological testing in adult attention deficit hyperactivity disorder: A pilot study. International Journal of Neuroscience, 96, 225-235.

30.

Lovejoy

D. W.

Ball

J. D.

Keats

Stutts

M. L.

Spain

E. H.

Janda

Janusz

(1999). Neuropsychological performance of adults with attention deficit hyperactivity disorder (ADHD): Diagnostic classification estimates for measures of frontal lobe/executive functioning. Journal of the International Neuropsychological Society, 5, 222-233.

31.

Martel

M. M.

(2009). Research Review: A new perspective on attention-deficit/hyperactivity disorder: Emotion dysregulation and trait models. Journal of Child Psychology and Psychiatry, 50, 1042-1051.

32.

Martinussen

Hayden

Hogg-Johnson

Tannock

(2005). A meta-analysis of working memory impairments in children with attention-deficit hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 377-384.

33.

Marx

Hübner

Herpertz

S. C.

Berger

Reuter

Kircher

Herpertz-Dahlmann

(2010). Cross-sectional evaluation of cognitive functioning in children, adolescents, and young adults with ADHD. Journal of Neural Transmission, 117, 403-419.

34.

Miyake

Friedman

N. P.

Emerson

M. J.

Witzki

A. H.

Howerter

(2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49-100.

35.

Nigg

J. T.

(2001). Is ADHD a disinhibitory disorder? Psychological Bulletin, 127, 571-598.

36.

Nigg

J. T.

Willcutt

E. G.

Doyle

A. E.

Sonuga-Barke

E. J. S.

(2005). Causal heterogeneity in attention-deficit/hyperactivity disorder: Do we need neuropsychologically impaired subtypes? Biological Psychiatry, 57, 1224-1230.

37.

Owens

A. M.

Downes

J. J.

Sahakian

B. J.

Polkey

C. E.

Robbins

T. W.

(1990). Planning and spatial working memory following frontal lobe lesions in man. Neuropsychologia, 28, 1021-1034.

38.

Robinson

L. J.

Thompson

J. M.

Gallagher

Goswami

Young

A. H.

Ferrier

I. N.

Moore

P. B.

(2006). A meta-analysis of cognitive deficits in euthymic patients with bipolar disorder. Journal of Affective Disorders, 93, 105-115.

39.

Rohlf

Jucksch

Gawrilow

Huss

Hein

Lehmkuhl

Salbach-Andrae

(2012). Set shifting and working memory in adults with attention-deficit/hyperactivity disorder. Journal of Neural Transmission, 119, 95-106.

40.

Rose

E. J.

Ebmeier

K. P.

(2006). Pattern of impaired working memory during major depression. Journal of Affective Disorders, 90, 149-161.

41.

Sergeant

J. A.

(2005). Modeling attention-deficit/hyperactivity disorder: A critical appraisal of the cognitive-energetic model. Biological Psychiatry, 57, 1248-1255.

42.

Sjöwall

Roth

Lindqvist

Thorell

L. B.

(2013). Multiple deficits in ADHD: Executive dysfunction, delay aversion, reaction time variability and emotional deficits. Journal of Child Psychology and Psychiatry, 54, 619-627.

43.

Solanto

M. V.

Abikoff

Sonuga-Barke

E. J. S.

Schachar

Logan

G. D.

Wigal

Turkel

(2001). The ecological validity of delay aversion and response inhibition as measures of impulsivity in AD/HD: A supplement to the NIMH multimodal treatment study of AD/HD. Journal of Abnormal Child Psychology, 29, 215-228.

44.

Solanto

M. V.

Etefia

Marks

D. J.

(2004). The utility of self-report measures and the continuous performance test in the diagnosis of ADHD in adults. CNS Spectrum, 9, 649-659.

45.

Solanto

M. V.

Gilbert

S. N.

Raj

Zhu

Pope-Boyd

Stepak

Newcorn

J. H.

(2007). Neurocognitive functioning in AD/HD, predominantly inattentive and combined subtypes. Journal of Abnormal Child Psychology, 35, 729-744.

46.

Sonuga-Barke

E. J. S.

(2002). Psychological heterogeneity in AD/HD—A dual pathway model of behavior and cognition. Behavioural Brain Research, 130, 29-36.

47.

Sonuga-Barke

E. J. S.

Dalen

Remington

(2003). Do executive deficits and delay aversion make independent contributions to preschool attention-deficit/hyperactivity disorder symptoms? Journal of the American Academy of Child & Adolescent Psychiatry, 42, 1335-1342.

48.

Tamm

Narad

M. E.

Antonini

T. N.

O’Brien

K. M.

Hawk

L.W.

Jr. Epstein

J. N.

(2012). Reaction time variability in ADHD: A review. Neurotherapeutics, 9, 500-508.

49.

Taylor

C. J.

Miller

D. C.

(1997). Neuropsychological assessment of attention in ADHD adults. Journal of Attention Disorders, 2, 77-88.

50.

Tucha

Sontag

T. A.

Stasik

Laufkötter

Lange

K. W.

(2011). Differential effects of methylphenidate on problem solving in adults with ADHD. Journal of Attention Disorders, 15, 161-173.

51.

Tulsky

D. S.

Saklofske

D. H.

Wilkins

Weiss

L. G.

(2001). Development of a general ability index for the Wechsler Adult Intelligence Scale–Third edition. Psychological Assessment, 13, 566-571.

52.

Voorhees

E. E.

Hardy

K. K.

Kollins

C. H.

(2011). Reliability and validity of self-and other-ratings of symptoms of ADHD in adults. Journal of Attention Disorders, 15, 224-234.

53.

Walker

A. J.

Shores

E. A.

Trollor

J. N.

Lee

Sachdev

P. S.

(2000). Neuropsychological functioning of adults with attention deficit hyperactivity disorder. Journal of Clinical and Experimental Neuropsychology, 22, 115-124.

54.

Wechsler

(2008). Wechsler Adult Intelligence Scale–Fourth edition. San Antonio, TX: The Psychological Corporation.

55.

Woods

S. P.

Lovejoy

D. W.

Ball

J. D.

(2002). Neuropsychological characteristics of adults with ADHD: A comprehensive review of initial studies. The Clinical Neuropsychologist, 16, 12-34.