On the relationship between autistic traits and executive functioning in a non-clinical Dutch student population

Abstract

We examined the association between autistic traits and different aspects of executive functioning (EF), using non-clinical Social Science and Science students as participants. Autistic traits, and associated personality traits, were measured using the Autism Quotient (AQ) and the Temperament and Character Inventory (TCI), respectively. EF was examined by means of a random number generation test and a phonemic fluency test. Using appropriate dependent measures, the following EF components were examined: 1) inhibition of prepotent responding, 2) simple output inhibition, 3) working memory monitoring and updating, and 4) switching. No significant relationship was found between the AQ and each of the four components of EF. However, two TCI subscales were reliably correlated with either the working memory or the shifting component. These results were discussed in view of the concept of an autism spectrum with respect to executive abilities.

Keywords

Autism-Spectrum Quotient executive functioning non-clinical student population phonemic fluency random number generation Temperament and Character Inventory

Autism spectrum disorders (ASD) have been linked to deficits in executive functioning (EF), although recent research questions the existence of a simple relationship (e.g. Geurts et al., 2009; Hill, 2004; Hill and Bird, 2006). Executive functions refer to a set of higher cognitive processes that allow an individual to exert control over lower cognitive processes. A large number of different processes have been suggested to be part of this set, such as mental flexibility, inhibition, problem solving, working memory (WM) and action initiation and monitoring. These higher cognitive processes enable behavioural adaptation to changing environmental demands and the display of goal-directed behaviour. Depending on the variables such as the specific type of EF process or component that is assessed, the task used to tap this process or component and when in the developmental trajectory the task is performed, evidence for executive dysfunction may, or may not, be found. For example, with respect to the type of task, recent evidence suggests that especially the ill-defined tasks that resemble daily life challenges (e.g. so-called ecologically valid tasks, see Burgess et al., 2006; Kenworthy et al., 2008) and self-reports on EF may be suitable. The latter may reveal EF impairments even in non-clinical autistic populations (e.g. Christ et al., 2010).

The autism spectrum hypothesis (Baron-Cohen, 1995) proposes a continuum or dimension of ‘autistic traits’. Accordingly, using an appropriate questionnaire, each individual can be placed somewhere on this continuum. Baron-Cohen et al. (2001) developed a self-administered scale for this purpose, the Autism-Spectrum Quotient (AQ). Based on the autism spectrum notion, Kunihira et al. (2006) used the AQ to examine whether variations in autistic traits that occur in non-clinical Japanese student populations (humanities and social science vs. science students) are significantly associated with specific personality traits and/or cognitive abilities. This study revealed reliable correlations between the AQ score and personality traits that are characteristic of an obsessional personality. However, no significant correlations were found between the AQ score, on the one hand, and measures of cognitive abilities related to Theory of Mind, central coherence and EF, on the other hand.

A limitation of the study by Kunihira et al. (2006) is that their examination of EF consisted of only a single test, the Wisconsin Card Sorting Test (WCST; Heaton et al., 1993). This test is one of the most commonly used tests to assess EF, but it in fact captures only one of the three basic cognitive component processes that have been proposed to underlie EF. Specifically, based on a latent variable analysis, Miyake et al. (2000) proposed that most traditional tests of EF call upon (a) inhibition of prepotent responses, (b) information monitoring and updating and/or (c) mental set shifting. With respect to the WCST, structural equation modelling suggested that this test is related most strongly to the shifting component (Miyake et al., 2000).

In the present study, we aimed at further examining the relationship between autistic traits in a non-clinical student population (social science and science students), as measured by the AQ, and EF, as measured by objective tests. Specifically, we focused on examining the relationship between the AQ and each of the basic components of EF (and not only the shifting component as in Kunihira et al., 2006). One task that we used for this purpose was the random number generation (RNG) task (see Baddeley, 1966, for an early report). In a typical RNG task, participants are asked to produce a random sequence of digits at a paced rate. This task places a heavy demand on EF because the participant must integrate and keep information from long-term memory active in WM (e.g. task instructions, set of possible response options), monitor previous output and change production strategies, for example, by suppressing prepotent responses. Moreover, imaging studies have shown the involvement of a neural circuitry that incorporates the frontal lobes (e.g. Jahanshahi et al., 1998, 2000; Knoch et al., 2005). Frontal lobes are traditionally linked to executive functions, although here too, there is no simple one-to-one relationship: frontal lobe damage may neither be necessary nor sufficient for impaired performance on tests of EF (e.g. Alvarez and Emory, 2006).

In terms of the three basic components of EF, the RNG has been shown to tap inhibition and updating but not set shifting (Maes et al., 2011a; Miyake et al., 2000; Poljac et al., 2010; Towse and Neil, 1998). The updating aspect is reflected in a measure of randomness that is sensitive to the equality of use of the different response options (digits). The participant can only avoid systematically omitting particular digits, or choosing particular digits too frequently, if he is able to keep track of previously chosen digits and to temporarily discard choosing digits that have just been chosen. The inhibition component is reflected in (a) a measure that is sensitive to the extent to which the subject simply repeats digits and (b) a measure that is sensitive to the extent to which the subject produces ‘prepotent’ sequences, such as those implied in counting forward or backwards. The first and second measure are linked to the concepts of, respectively, simple output inhibition and the inhibition of prepotent responding. More generally, a distinction between these two types of inhibition is substantiated by neuroanatomical considerations (e.g. Alexander et al., 1986).

As also indicated above, one recent suggestion put forward is that cognitive deficits in ASD are most evident in tasks that are relatively ill defined (White et al., 2009). These are tasks that are often also assumed to have high ecological validity (e.g. Geurts et al., 2009; Hill and Bird, 2006). The RNG may be considered to be an ill-defined or ‘open-ended’ EF task (but perhaps not an ecologically valid one) because it requires spontaneous, as opposed to reactive, flexibility (see Eslinger and Grattan, 1993, for a more detailed discussion of this distinction). In reactive flexibility tasks, like the WCST, participants are required to switch attention and respond to environmental cues, whereby this requirement is explicitly cued by an environmental stimulus that consists of negative feedback. Instead, in spontaneous flexibility tasks, the necessity to show cognitive flexibility is much more implicit. It must be initiated by the participant in the absence of any further environmental cuing over and above the instruction given at the outset of the task. Therefore, this task may be especially sensitive to detect associations between autistic traits and EF, in both clinical and non-clinical populations.

To the authors’ knowledge, there are only two previous studies that explicitly assessed the relationship between (clinical) ASD and RNG (Rinehart et al., 2006; Williams et al., 2002). The results of these two studies suggest that, in RNG, both low-functioning adults with autism (LFA) and high-functioning children with autism (HFA) display impaired output inhibition but no deficits in inhibition of prepotent responses. Instead, Asperger’s syndrome (AS) in children seems to be associated with impaired inhibition of prepotent responses but not with impaired output inhibition. LFA in adults does not seem to be associated with impaired WM monitoring and updating; no other information is available concerning the relationship between other ASD populations and WM monitoring and updating, as assessed with appropriate RNG measures. There is some evidence for impaired WM capacity in ASD, but this evidence is based on other tests, for example, in studies examining spatial WM involving a relatively high memory load (e.g. Steele et al., 2007) or in those examining verbal memory (e.g. Alloway et al., 2009) . It remains to be seen whether autistic traits in clinical and non-clinical populations are associated with compromised WM capacity, as assessed with the RNG.

One component of EF that is not captured by RNG tasks is shifting (e.g. Miyake et al., 2000). ‘Open-ended’ tasks that are suitable to assess this aspect of EF are verbal fluency tests, such as the category fluency test (CFT) and, especially, the letter fluency test (LFT; see Lezak et al., 2004). Verbal fluency tests are thought to invoke at least two cognitive components (Troyer et al., 1997). The first is clustering, reflecting the ability to generate words in certain semantic or phonemic subcategories. The second is switching, which implicates the ability to switch to a new cluster once another cluster has been ‘exhausted’. With respect to the relationship between ASD and verbal fluency performance, the results are partly mixed. In general, performance on the CFT, which for a substantial part relies on overlearned semantic knowledge, is not impaired in ASD. However, LFT performance, which demands the use of efficient lexical retrieval strategies, has been found to be both impaired (e.g. Rumsey and Hamburger, 1988; Turner, 1999) and not impaired (e.g. Boucher et al., 2005) in ASD individuals. It remains to be seen whether variations in autistic traits in non-clinical populations are reliably correlated with verbal fluency performance, especially with letter fluency performance.

Autistic traits in a normal Japanese population, as measured by the AQ, have been shown to be strongly correlated with specific personality traits (e.g. Kunihira et al., 2006), as measured by the Temperament and Character Inventory (TCI; Cloninger, 1994), such as harm avoidance (HA) and novelty seeking (NS). The participants in our study also completed the TCI, next to completing the AQ, a RNG task and a LFT. The question of interest was whether or not autistic traits in a normal Dutch population, as assessed directly by the AQ and somewhat more indirectly by TCI personality traits, are associated with the basic inhibition, updating and switching components of EF, as measured by two ‘open-ended’ tasks presumed to be especially sensitive to the detection of EF deficits in autism.

Methods

Participants

The participants were 19 (9 females and 10 males) social science students and 19 (7 females and 12 males) science students. After briefly being informed about the tests to be performed in the study (see below), all students agreed to participate without receiving any financial or study compensation. All participants completed the following questionnaires and cognitive tests in a fixed order.

Questionnaires

The AQ (Baron-Cohen et al., 2001) consists of 50 forced-choice statements with a 4-point Likert response scale. The statements, which were translated in Dutch (Hoekstra et al., 2008), cover different domains associated with ASD, such as social and communication skills, imagination, attention to detail and tolerance of change. Scores range from 0 to 50 (the higher the score, the stronger the autistic traits), and a frequently used cut-off score for AS or HFA diagnosis is 32.

All participants completed a Dutch translation of the TCI. Basic data on psychometric properties and validity in the Dutch language area are provided by, for example, Duijsens et al. (2000) and Egger et al. (2007). The TCI consists of 240 statements, with a binary forced-choice statement (correct/incorrect). The questionnaire consists of four main temperament scales and three main character scales. In the present study, we focused on the temperament scales, NS, HA and reward dependence (RD). These scales have previously been shown to be significantly correlated with the AQ in non-clinical Japanese student populations (Kunihira et al., 2006).

Cognitive tests

Two tests were employed to assess the participants’ IQ. The Raven Coloured Progressive Matrices (RCPM; Raven et al., 1990) consists of 3 sets of 12 items each and were used to assess non-verbal reasoning ability. The test provides a raw score, ranging from 0 to 36, which was transformed into IQ scores based on norm tables. The Dutch Reading Test for Adults (Nederlandse Leestest Voor Volwassenen (NLV); Bouma et al., 1995) is a short test commonly used in the Netherlands to estimate verbal intelligence in neuropsychological and psychiatric patients. The participant is required to read out 50 words aloud. The raw score, the total number of correctly pronounced words, was transformed into an IQ score on the basis of norm tables.

In the ‘click’ RNG (Maes et al., 2011a), the digits 1–9 are shown on a computer screen to the participant. The digits are laid out orderly in a 3 × 3 grid. The participant is required to click on one digit every second, using a standard computer mouse. The generation of responses is paced by a ‘click’ sound that is presented at a rate of 1/s. The total number of to-be chosen digits is 81. The written instruction informs the participant to choose the digits as randomly as possible. The concept of randomness is explained using the ‘hat analogy’ (see Maes et al., 2011a, for more details). Three dependent measures were examined (see Towse and Neil, 1998, for more details). The first is the Phi-2-gram index (Phi2). The Phi2 is a measure of the tendency to emit response repetitions over different lengths (‘d-gram’ lengths). Therefore, the Phi2 reflects the tendency to repeat values over a distance of two responses, which equals simple response repetition. The index has a negative value if more d-grams are different (less repetition) than are statistically predicted and has a positive value in case of more than predicted repetitions. Therefore, a high Phi2 value reflects impaired output inhibition. The Adjacency measure represents the number of ‘neighbouring’ ascending (e.g. 1 followed by 2) or descending (e.g. 4 followed by 3) pairs. This measure can range between 0% (no adjacent pairs) and 100% (solely neighbouring pairs). A high percentage reflects an impaired ability to inhibit prepotent responses. Finally, the Coupon measure reflects the mean number of responses across the complete response set (in this case 81 responses) that are emitted before all the response options have been used. A relatively low Coupon measure value implies that the participant has systematically worked through the set of responses (cycling). This in turn implies a high equality of use of the different response options, which might indicate a good WM monitoring and updating ability.

The LFT was used to assess the shifting aspect of EF. The LFT requires the generation of words, cued with a specific letter. In the present study, the letters D, A and T were used. The total number of correct, non-repeated words was used as a dependent measure for this test.¹

Data analysis

The association among the various measures was assessed using Pearson correlation coefficients.

Results

Table 1 summarizes the details of the two groups included in the study. Analyses of variance revealed that the science students had higher AQ and non-verbal intelligence (RCPM) scores than the social science students (F(1, 36) = 10.33, p < 0.01 and F(1, 35) = 11.84, p < 0.01). There was no significant difference between the two groups for any of the other measures, although the difference for the RNG-Coupon measure approached significance (F(1, 36) = 3.89, p = 0.056; other F < 1.75).

Table 1.

Values on the different measures for the two student groups.

Measure	Social science students	Science students
Age	20.74 (3.03)	20.53 (2.48)
AQ***	13.53 (3.53)	18.63 (5.96)
TCI-NS	17.58 (5.64)	18.26 (5.33)
TCI-HA	15.95 (6.52)	15.32 (5.73)
TCI-RD	16.26 (1.37)	15.11 (3.56)
RCPM***	112.44 (9.90)	123.00 (8.75)
NLV	106.0 (8.70)	107.32 (8.40)
RNG-Phi2	−2.94 (1.17)	−3.14 (1.57)
RNG-Adjacency	24.56 (6.17)	24.02 (6.24)
RNG-Coupon	14.04 (2.21)	16.19 (4.22)
LFT	40.61 (9.06)	40.42 (8.79)

AQ: Autism-Spectrum Quotient; TCI: Temperament and Character Inventory; NS: novelty seeking; HA: harm avoidance; RD: reward dependence; RCPr: Raven Coloured Progressive Matrices; NLV: Nederlandse Leestest Voor Volwassenen; RNG: random number generation; Phi2: Phi-2-gram index; LFT: letter fluency test; SD: standard deviation; ANOVA: analysis of variance.

Scores represent mean (+ SD); means are based on n = 19, except for the RCPM and the LFT in the social science group (n = 18 in each of these cases, due to missing values).

***

Difference between groups is significant at a p level < 0.01 (ANOVA).

Table 2 displays the Pearson correlation coefficients between the AQ, TCI and each of the RCPM, NLV and EF test measures. The correlations between the AQ and the TCI-HA (r = 0.43) and TCI-RD (r = −0.39) scores were significant (p < 0.05), whereas the correlation between the AQ and the TCI-NS (r = −0.31) just failed to reach statistical significance (p = 0.055). The RNG-Coupon measure was significantly negatively correlated with the TCI-HA score (p = −0.32); the LFT score was near significantly positively associated with the TCI-RD score (p = 0.31). None of the other correlations were significant (r ≤ 0.25, p > 0.10). This pattern of (near) significant and insignificant correlations was maintained when including the RCPM score as a controlled variable in partial correlations.

Table 2.

Pearson correlation coefficient between the AQ, TCI and test scores.

Measure	AQ	TCI-NS	TCI-HA	TCI-RD
TCI-NS	−0.31*
TCI-HA	0.43***	−0.43***
TCI-RD	−0.39**	0.10	−0.06
RCPM	0.25	−0.07	0.03	−0.03
NLV	0.14	0.04	−0.02	−0.25
RNG-Phi2	−0.04	0.15	0.00	0.10
RNG-Adjacency	0.10	−0.25	0.01	−0.04
RNG-Coupon	−0.02	0.17	−0.32**	−0.12
LFT	−0.18	0.05	0.16	0.31*

AQ: Autism-Spectrum Quotient; TCI: Temperament and Character Inventory; NS: novelty seeking; HA: harm avoidance; RD: reward dependence; RCPM: Raven-Coloured Progressive Matrices; NLV: Nederlandse Leestest Voor Volwassenen; RNG: random number generation; LFT: letter fluency test.

The correlation coefficients are based on n = 38 except for all correlations involving the RCPM and the LFT (n = 37 for each; missing values).

0.05 < p < 0.10, **p < 0.05, ***p < 0.01.

Discussion

The science students had higher AQ scores than the social science students, a result that confirms previous findings in Dutch and other populations (Austin, 2005; Baron-Cohen et al., 2001; Hoekstra et al., 2008; Wakabayashi, 2003). In the present study, the science students also had a higher RCPM score compared to the social science students, suggesting that the former students had a greater non-verbal reasoning ability. The correlation analysis revealed a (near) significant association between the AQ score and each of the TCI personality subscales. Specifically, the correlation with the HA subscale was the strongest, followed by the correlations with the RD and NS subscales. This pattern of results is identical to that found in the Kunihira et al. (2006) study, examining a much larger sample of non-autistic Japanese students. This correspondence confirms the capability of the AQ to capture, in general populations, personality traits that traditionally are associated with ASD.

However, importantly, correlation analyses clearly failed to reveal a significant association between the AQ score and each of the different component EF measures. The absence of an association between the AQ and shifting capacity in our study, as measured by the LFT, is in accordance with the null results reported for a non-autistic adult Japanese student population (Kunihira et al., 2006), in which the WCST was used to measure shifting capacity. This null result can be contrasted with some previous reports that did find clinical ASD to be reliably associated with impaired LFT performance, although, as outlined in the introduction, there are also studies reporting no impairments. Importantly, even in studies finding a reduced production of words in LFT in ASD individuals relative to controls, the impaired performance may be explained by referring to processes other than ‘switching/shifting’, such as a low processing speed (e.g. Spek et al., 2009). More generally, there clearly are other reports of compromised performance of diverse autistic populations on other tasks that are intended to capture the shifting aspect of EF, such as the WCST. However, here too, there are reasons to doubt whether these deficits are specifically due to the shifting component involved in this task (see Geurts et al., 2009, for a review; see also Maes et al., 2011b). Next to this confirmation of a lack of correlation with shifting found in the Japanese study, one major additional implication of the present results is that, in non-clinical populations, variations in the AQ score are also not reliably associated with the inhibition and updating EF components, as assessed with the RNG task.

One could argue that the present null results with respect to the AQ were due to the present EF tests being insufficiently sensitive, for example, because they did not reveal large inter-individual differences. In addition, the inter-individual differences in AQ score too might have been insufficiently large. In this respect, it is worthy of note that the individuals in the present sample were at the lower end of the AQ scale and had a lower overall mean AQ score (16) than was the case for, for example, the population of students examined in the Kunihira et al.’s (2006) study (mean = approximately 22). This presumed lack of variances could then have been responsible for the lack of significant correlations with the AQ measure. However, it is important to note that the ranges and variances for the different scores were, in fact, quite considerable. For example, the range (and variance) of the AQ score, the RNG-Adjacency measure and the LFT were, respectively, 6 -30 (variance = 30.02; cf. also the variance in Kinihira et al.’s study: approximately 39), 11.11 -40.74 (variance = 37.53) and 21 -57 (variance = 77.42). These ranges and variances suggest a sufficient sensitivity for detecting significant correlations with the AQ measure.

Another argument against a presumed insensitivity of the present EF tests is provided by the fact that at least the RNG-Coupon and LFT scores were each reliably associated with a personality trait that in itself was shown to be linked to autistic traits as measured by the AQ (see below for further discussion of these correlations).

A final argument against the possibility of an insufficient test sensitivity of the AQ and EF measures is provided by additional, preliminary, data that we gathered from a small sample of adults with ASD (four diagnosed with AS and four with pervasive developmental disorder–not otherwise specified (PDD-NOS)). These individuals, who performed the same tests as included in the present study, had a mean AQ score of 32.62. Their mean scores on the RNG-Phi2, RNG-Adjacency, RNG-Coupon and LFT measures were 4.21, 39.97, 16.28 and 34.25, respectively. Combining the data from these individuals with those from an IQ-matched sample from each of the two student populations included in the present study did reveal significant (non-parametric) correlations between the AQ and the RNG-Phi2 (r = −0.42), RNG-Adjacency (r = 0.57) and LFT (r = −0.46) tests. In other words, adding a clinical sample of individuals to the present non-clinical participants did reveal significant results using the present RNG and fluency tests, and associated dependent variables, as measures of EF. In combination with the large inter-subject variances, this suggests that the present RNG and LFT are, in principle, suitable for detecting ASD-related (and AQ-related) EF deficits.

Although for the different EF measures no significant correlations were found with the AQ score, the RNG-Coupon score was significantly negatively correlated with the HA trait of the TCI, and the LFT score was positively correlated with the TCI-RD trait. These correlations must be treated with some caution because both correlations would be insignificant had we corrected for the multiple correlations that we performed (to reduce Type 1 error). However, at least compared to the remaining correlations, these correlations are relatively high. At present, we can only speculate about the process(es) underlying these correlations. For example, why is a low Coupon score associated with strong HA? A low Coupon score could reflect better WM monitoring and updating than a large score. However, we cannot think of any plausible reason why a good WM should favour a behavioural trait that is characterized by HA. However, a low RNG-Coupon score might also reflect strong ‘systematic’ responding, such as more-or-less systematic ‘cycling’ through all response options. Notably, the use of such a strategy does not necessarily imply a memory load. To examine this possibility further, we performed an additional analysis, using a statistic that is sensitive to the degree of response ‘systematicity’. Specifically, we used a measure based on the correlation between an individual’s separate responses at different response lags (autocorrelations). For each individual, we determined the maximum autocorrelation from lags 1–9 (see Maes, 2003, for further details: the MaxAutstatistic (see Maes, 2003, for further details: the MaxAut statistic, i.e. the maximum of all computed autocorrelations)). The mean MaxAut correlation was 0.26. Based on computer-generated random sequences (19 sets of random sequences of 81 digits each, using the digits 1–9), the mean MaxAut correlation was 0.19. These results suggest that the participants of both groups clearly (and statistically significantly) responded more ‘systematically’ than expected for truly, computer-generated, random digit sequences. We might suggest that individual differences in this ‘cycling’ tendency might reflect differences in behavioural rigidity, which in turn is conducive to a focus on (the avoidance of) potentially harmful situations (high TCI-HA score).

Whatever the merit of such speculations, the significant correlations with the TCI scales do have one important implication. They indirectly provide evidence for the claim that the AQ might not be ideally suited for the measurement of non-clinical autistic traits (e.g. also see Ingersoll et al., 2011) and specifically of those traits that might cause EF impairments. The present positive results with respect to the TCI suggest that there might be other autistic traits (or personality traits that are in turn associated with these traits), and/or measures of such traits, that are more directly relevant for EF, especially when they are measured in non-clinical samples. Therefore, the present null results with respect to the AQ found in this, and in the Kunihira et al.’s study, do not necessarily invalidate the notion of an autism spectrum; the AQ might merely insufficiently tap variations in those traits that cause EF problems in clinical populations. According to this line of reasoning, only in the case of the inclusion of more ‘extreme’ (i.e. clinical) cases is the variation in those EF-relevant autistic traits, as measured by the AQ, sufficiently large to be able to detect significant correlations with measures of EF. However, clearly more research is needed to establish which specific ‘autistic’ traits are most directly linked to which aspects of EF in clinical and non-clinical populations and how these traits and EF aspects can be objectively assessed best.

Footnotes

Acknowledgements

We would like to thank all participants for their cooperation, and Sanne de Ronde and Maud Bijaard for their assistance in collecting the data. We also want to thank two anonymous reviewers for their helpful comments.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Notes

References

Alexander

DeLong

Strick

(1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neurosciences 9: 357–381.

Alloway

Rajendran

Archibald

LMD

(2009) Working memory in children with developmental disorders. Journal of Learning Disabilities 42: 372–382.

Alvarez

Emory

(2006) Executive function and the frontal lobes: a meta-analytic review. Neuropsychology Review 16: 17–42.

Austin

(2005) Personality correlates of the broader autism phenotype as assessed by the Autism Spectrum Quotient (AQ). Personality and Individual Differences 38: 451–460.

Baddeley

(1966) The capacity for generating information randomization. Quarterly Journal of Experimental Psychology 18: 119–129.

Baron-Cohen

(1995) Mindblindness: An Essay on Autism and Theory of Mind. Cambridge, MA: MIT Press/Bradford Books.

Baron-Cohen

Wheelwright

Skinner

. (2001) The Autism-Spectrum Quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders 31: 5–17.

Boucher

Cowell

Howard

. (2005) A combined clinical, neuropsychological and neuroanatomical study of adults with high functioning autism. Cognitive Neuropsychiatry 10: 165–213.

Bouma

Mulder

Lindeboom

(1995) Nederlandse Leestest Voor Volwassenen (NLV). Lisse: Swets & Zeitlinger.

10.

Burgess

Alderman

Forbes

. (2006) The case for the development and use of ‘ecologically valid’ measures of executive function in experimental and clinical neuropsychology. Journal of the International Neuropsychological Society 12: 194–209.

11.

Christ

Kanne

Reiersen

(2010) Executive function in individuals with subthreshold autism traits. Neuropsychology 24: 590–598.

12.

Cloninger

(1994) The Temperament and Character Inventory (TCI): A guide to Its Development and Use. St Louis, MO: Center for Psychobiology of Personality, Washington University.

13.

Duijsens

Spinhoven

Goekoop

. (2000) The Dutch temperament and character inventory (TCI): dimensional structure, reliability and validity in a normal and psychiatric outpatient sample. Personality and Individual Differences 28: 487–499.

14.

Egger

JIM

Gringhuis

Breteler

. (2007) MMPI-2 clusters of alcohol dependent patients and the relation with Cloninger’s temperament-character inventory. Acta Neuropsychiatrica 19: 238–243.

15.

Eslinger

Grattan

(1993) Frontal lobe and frontal-striatal substrates for different forms of human cognitive flexibility. Neuropsychologia 31: 17–28.

16.

Geurts

Corbett

Solomon

(2009) The paradox of cognitive flexibility in autism. Trends in Cognitive Sciences 13: 74–82.

17.

Heaton

Chelune

Talley

. (1993) Wisconsin Card Sorting Test Manual – Revised and Expanded. Odessa, FL: Psychological Assessment Resources, Inc.

18.

Hill

(2004) Evaluating the theory of executive dysfunction in autism. Developmental Review 24: 189–233.

19.

Hill

Bird

(2006) Executive processes in Asperger syndrome: patterns of performance in a multiple case series. Neuropsychologia 44: 2822–2835.

20.

Hoekstra

Bartels

Cath

. (2008) Factor structure, reliability and criterion validity of the Autism-Spectrum Quotient (AQ): a study in Dutch population and patient groups. Journal of Autism and Developmental Disorders 38: 1555–1566.

21.

Ingersoll

Hopwood

Wainer

. (2011) A comparison of three self-report measures of the broader autism phenotype in a non-clinical sample. Journal of Autism and Developmental Disorders 41(12): 1646–1657.

22.

Jahanshahi

Dirnberger

Fuller

. (2000) The role of the dorsolateral prefrontal cortex in random number generation: a study with positron emission tomography. NeuroImage 12: 713–725.

23.

Jahanshahi

Profice

Brown

. (1998) The effects of transcranial magnetic stimulation over the dorsolateral prefrontal cortex on suppression of habitual counting during random number generation. Brain 121: 1533–1544.

24.

Kenworthy

Yerys

Anthony

. (2008) Understanding executive control in autism spectrum disorders in the lab and in the real world. Neuropsychology Review 18: 320–338.

25.

Knoch

Brugger

Regard

(2005) Suppressing versus releasing a habit: frequency-dependent effects of prefrontal transcranial magnetic stimulation. Cerebral Cortex 15: 885–887.

26.

Kunihira

Senju

Dairoku

. (2006) ‘Autistic’ traits in non-autistic Japanese populations: relationship with personality traits and cognitive ability. Journal of Autism and Developmental Disorders 36: 553–566.

27.

Lezak

Howieson

Loring

(2004) Neuropsychological Assessment. 4th ed. New York: Oxford University Press.

28.

Maes

JHR

(2003) Response stability and variability induced in humans by different feedback contingencies. Learning and Behavior 31: 332–348.

29.

Maes

JHR

Eling

PATM

Reelick

. (2011a) Assessing executive functioning: on the validity, reliability, and sensitivity of a click/point random number generation task in healthy adults and patients with cognitive decline. Journal of Clinical and Experimental Neuropsychology 33: 366–378.

30.

Maes

JHR

Eling

PATM

Wezenberg

. (2011b) Attentional set shifting in autism spectrum disorder: differentiating between the role of perseveration, learned irrelevance, and novelty processing. Journal of Clinical and Experimental Neuropsychology 33: 210–217.

31.

Miyake

Friedman

Emerson

. (2000) The unity and diversity of executive functions and their contribution to complex ‘frontal lobe’ tasks: a latent variable analysis. Cognitive Psychology 41: 49–100.

32.

Poljac

Simon

Ringlever

. (2010) Impaired task switching performance in children with dyslexia but not in children with autism. The Quarterly Journal of Experimental Psychology 63: 401–416.

33.

Raven

Court

Raven

(1990) Manual for Raven’s Progressive Matrices and Vocabulary Scales – Section 2: Coloured Progressive Matrices. Oxford: Oxford Psychologists Press.

34.

Rinehart

Bradshaw

Moss

. (2006) Pseudo-random number generation in children with high-functioning autism and Asperger’s disorder: further evidence for a dissociation in executive functioning? Autism 10: 70–85.

35.

Rumsey

Hamburger

(1988) Neuropsychological findings in high-functioning men with infantile autism, residual state. Journal of Clinical and Experimental Neuropsychology 10: 201–221.

36.

Spek

Schatorjé

Scholte

. (2009) Verbal fluency in adults with high-functioning autism or Asperger syndrome. Neuropsychologia 47: 652–656.

37.

Steele

Minshew

Luna

. (2007) Spatial working memory deficits in autism. Journal of Autism and Developmental Disorders 37: 605–612.

38.

Towse

Neil

(1998) Analyzing human random generation behavior: a review of methods used and a computer program for describing performance. Behavior Research Methods, Instruments, & Computers 30: 583–591.

39.

Troyer

Moscovitch

Winocur

(1997) Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. Neuropsychology 11: 138–146.

40.

Turner

(1999) Generating novel ideas: fluency performance in high-functioning and learning disabled individuals with autism. Journal of Child Psychology and Psychiatry and Allied Disciplines 40: 189–201.

41.

Wakabayashi

(2003) Validity of Autism-Spectrum Hypothesis in normal adults: evidence from the relation between types of Sciences and AQ scores in university students. The Japanese Journal of Autistic Spectrum 2: 11–20.

42.

White

Burgess

Hill

(2009) Impairments on ‘open-ended’ executive function tests in autism. Autism Research 2: 138–147.

43.

Williams

Moss

Bradshaw

. (2002) Random number generation in autism. Journal of Autism and Developmental Disorders 32: 43–47.