Deficits in metacognitive monitoring in mathematics assessments in learners with autism spectrum disorder

Abstract

Children and adults with autism spectrum disorder have been found to have deficits in metacognition that could impact upon their learning. This study explored metacognitive monitoring in 28 (23 males and 5 females) participants with autism spectrum disorder and 56 (16 males and 40 females) typically developing controls who were being educated at the same level. Participants were asked a series of mathematics questions. Based upon previous research, after each question they were asked two metacognitive questions: (1) whether they thought they had got the answer correct or not (or ‘don’t know’) and (2) whether they meant to get the answer correct or not (or ‘don’t know’). Participants with autism spectrum disorder were significantly more likely than the typically developing group to erroneously think that they had got an incorrect answer correct. Having made an error, those with autism spectrum disorder were also significantly more likely to report that they had meant to make the error. Different patterns in the types of errors made were also identified between the two groups. Deficits in metacognition were identified for the autism spectrum disorder group in the learning of mathematics. This is consistent with metacognitive research from different contexts and the implications for supporting learning in autism spectrum disorder are discussed.

Keywords

autism spectrum disorder mathematics learning metacognition

Autism spectrum disorder (ASD) is estimated to affect around 1% of the population (Baird et al., 2006), and is characterised by impairments in social communication and interaction as well as behavioural and cognitive inflexibility (American Psychiatric Association (APA), 2013). A deficit in ‘theory of mind’ has been proposed to account for the social deficits in ASD, which was originally defined as difficulty in attributing mental states to oneself and others (Premack and Woodruff, 1978). The distinction between self-referential metacognition (termed ‘metacognition’) and other-referential metacognition (termed ‘mindreading’) is pertinent as a wealth of research has explored deficits in people with ASD attributing mental states to others that are different to their own (see Baron-Cohen, 1997; Baron-Cohen et al., 1985; Castelli et al., 2002; Frith, 2001); however, there is a relative paucity of research exploring an awareness of one’s own metacognitive states in ASD (see, for review, Carruthers, 2009; Williams, 2010).

Those with ASD have impairments in both self- and other-referential cognition (Lombardo and Baron-Cohen, 2011; Lombardo et al., 2007; see also Pfeifer et al., 2013) which may indicate the possibility that the same cognitive and neural functions underpin both self- and other-referential metacognition (see Frith and Happé, 1999; Happé, 2003; Hobson, 1990; Williams, 2010). Williams et al. (2009) and Williams and Happé (2009) argue that self-referential metacognition may be more impaired than other-referential metacognition in ASD (see also Lombardo et al., 2010). The available research suggests that individuals with ASD do have a deficit in self-referential metacognition (hereafter ‘metacognition’). Metacognition encompasses two components: first, metacognitive knowledge of cognition (‘declarative’), which is stored acquired knowledge of cognition, such as ‘I am better at arithmetic than I am at spelling’. Second, metacognitive experience of cognition (‘procedural’), which encompasses the awareness of one’s own cognition (metacognitive monitoring) and the capacity to control appropriate cognitive responses to the monitoring (metacognitive regulation: Flavell, 1979; Lockl and Schneider, 2002; Nelson and Narens, 1990; Schraw and Moshman, 1995). Metacognitive monitoring, in particular, is considered essential for a sense of ‘self-concept’ (Roebers et al., 2012) and for day-to-day behavioural functioning, because accurate monitoring of one’s internal states facilitates the regulation of and control over those states and over learning and behaviour (Nelson and Leonesio, 1988). As an example, an accurate ‘feeling of knowing’ for what one knows (and does not know) would be an indicator of metacognitive monitoring as would a reliable ‘judgement of confidence’ in what one knows. Wojcik et al. (2013) explored a ‘feeling of knowing’ for both episodic and semantic memory and report metacognitive impairments for episodic memory only. Grainger et al. (2014) also report diminished feeling of knowing for accurate memory recall in those with ASD (see also Farrant et al., 1999a, 1999b). Sawyer et al. (2014) argued that social difficulties in ASD are not due to the ability to accurately make emotion recognition decisions but instead are due to those with ASD being less able to assess whether they have correctly recognised the emotion or not (see also Wilkinson et al., 2010).

One area in which metacognition is argued to be a powerful predictor of performance is school-based learning. For example, research has highlighted that metacognition predicts mathematical performance more powerfully than intellectual abilities and there is extensive evidence that developing metacognition is an effective intervention in school children within and below the ‘normal’ range of mathematical ability (Higgins et al., 2013; Iuculano et al., 2014; Maxwell and Grenier, 2014; Schneider and Artelt, 2010; Van der Stel et al., 2010; see also Roebers et al., 2012, 2014). On average, mathematics ability is substantially lower among people with ASD than would be expected on the basis of intelligence quotient (IQ) (Chiang and Liu, 2007; Mayes and Calhoun, 2003, 2006; see also Jones et al., 2009), with educational underachievement in mathematics contributing directly to less-than-optimal economic outcomes (e.g. low levels of employment) and life chances among people with ASD (Estes et al., 2011). A better understanding of metacognition in ASD with reference to the learning of mathematics may therefore have potentially far-reaching benefits.

One methodology for assessing metacognition, specifically metacognitive monitoring, that is applicable to mathematics learning was proposed by Russell and Hill (2001). After a child’s drawing was manipulated by an experimenter so that it differed from what the child had actually had drawn (without the knowledge of the child), children were asked ‘what did you mean to draw?’ and ‘what did you think you were drawing?’ (after Russell and Hill, 2001). Russell and Hill found that when the ‘Mean’ question was asked before the ‘Think’ question, only around half of the children with ASD correctly recognised that they had not intended the actual outcome, although other experiments did not reveal a consistent pattern of metacognitive monitoring deficits. This methodology was developed by Williams and Happé (2010) who found around one-third of participants with ASD (mistakenly) reported that they had meant to draw the experimenter-manipulated picture and that they thought they were drawing the experimenter-manipulated picture (erroneously as this was not possible: A second study found a similar proportion of children with ASD erroneously reported that they had intended a knee reflex). The authors provide evidence that the ‘Mean’ and ‘Think’ questions are comparably difficult for those with ASD. Overall, the authors suggest that those with ASD have limited awareness of their own metacognition, and this deficit was independent of both age and verbal ability (see also Phillips et al., 1998; Russell and Jarrold, 1998).

Thinking that you meant to make an error which relates directly to a failure of metacognitive monitoring. An internally generated monitoring system is argued to compare a representation of the correct or intended response with a representation of the actual response – with discrepancies leading to remedial actions (metacognitive regulation: see Henderson et al., 2006). A deficit in online assessments of ‘have I got this correct?’ and ‘did I mean to get this correct?’ could have a significant impact upon the capacity to learn from errors. An event-related potential (ERP) study of decision-making errors found that there were no differences between those with and without ASD when the correct decision was made. However, there was reduced activity in those with ASD when an error was made, suggesting a specific insensitivity to error awareness (Vlamings et al., 2008; see also Henderson et al., 2006). The authors also noted a lack of post-error slowing in ASD (compared to controls) and suggest this impairment in error awareness (metacognitive monitoring) may lead to a failure to reflect and evaluate the error and change strategy to deal with the situation (metacognitive regulation). Error-correction in learning in ASD has received little scrutiny, although it has been suggested that people with ASD ‘learn how to learn’ differently (Powell and Jordan, 1993). The types of errors made may therefore differ between those with and without ASD.

This study asked, if those with ASD ‘learn how to learn’ differently, do these metacognitive differences impact upon learning, and how does this affect the learning of mathematics in particular? This study examined the role of metacognitive monitoring in learning in ASD – specifically in mathematics. In addition to mathematics potentially being particularly problematic for those with ASD, the advantage of researching mathematics learning is that multiple questions can be asked, rather than a single instance. Based upon previous research (Williams and Happé, 2010), we predicted that learners with ASD would be more likely to report that an erroneous answer was correct and more likely to report that they meant to make an error in their mathematics learning (after an error had been made). We also explored the errors made in mathematics learning to identify any differences between learners with and without ASD.

Methods

Participants

The project was approved by the Departmental Research Ethics Committee which ensures adherence to the ethical principles of the British Psychological Society. Participants comprised 23 males and 5 females with a diagnosis of ASD attending an educational unit specifically for young people with ASD, attached to a mainstream school. Participants had been diagnosed by international standards (Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV) by clinicians as an entry requirement for the unit. An index of IQ (Wechsler Abbreviated Scale of Intelligence (WASI)-UK, Wechsler, 1999) was also taken. Verbal IQ ranged from 63 to 108 (mean = 89.69, standard deviation (SD) = 10.59) and performance IQ ranged from 60 to 126 (mean = 96.42, SD = 16.46). Two participants were excluded on the basis of IQ being less than 70, resulting in the data from 21 males and 5 females being analysed. Of the retained sample, the verbal IQ ranged from 71 to 108 (mean = 91.08, SD = 9.31) and performance IQ ranged from 80 to 126 (mean = 99.25, SD = 13.62). The mean age was 13.7 years (SD = 1.3 years).

The United Kingdom has a National Curriculum for mathematics that prescribes what should be taught at what age. Students at this age would typically be expected to be at ‘Key Stage 3’. Despite participants with ASD having a mean IQ within the normal range, teaching staff reported that these participants were working at a Key Stage 2 level for mathematics. Key Stage 2 typically relates to 7–11 years of age. As a consequence, we employed a typically developing (TD) control group who were also working at Key Stage 2. The metacognitive monitoring processes of ‘Thinking’ and ‘Meaning’ to get an answer correct described above would typically expect to be developed by the age of 5½ years (see Wilkinson et al., 2010) and predict ability around 11 years of age (Roebers et al., 2014). We therefore recruited 16 males and 40 females (the number in the school year) from a mainstream school in a similar location to the ASD unit. The TD participants were in the final year of Key Stage 2 (‘Year 6’), aged 10.5 years (SD = 0.5). The TD group were therefore younger than the ASD group but being taught the same level of National Curriculum mathematics material. There were no reported disorders in the TD group and no teacher reports of abnormal intellect (such as special educational needs, which was reflected in the end of year national assessments). Due to time constraints, an independent assessment of IQ could not be undertaken. Independent assessments of metacognition and mathematical ability were not undertaken as it has been found that the former is highly predictive of the latter and matching on this basis may obscure group differences (Higgins et al., 2013; Iuculano et al., 2014; Van der Stel et al., 2010). We anticipated specific deficits in the ASD sample that could not be matched to the cognitive profile of another sample (e.g. Chiang and Liu, 2006; Mayes and Calhoun, 2003, 2006). An ASD and TD group was therefore matched on being exposed to the same level of mathematics curriculum material, although this does not necessitate both groups would be of the same mathematical ability. The stepped procedure described below provided an index of level of mathematical ability.

Procedure

Each participant was seated with the experimenter in a one-on-one situation in a quiet room adjacent to the classroom. Participants had paper and a pencil and could write down workings-out, but could not use a calculator. Participants were each asked three sets of five mathematical questions, totalling 15 questions. The questions were typed on a sheet of paper and read to participants. The first set of five questions were always level 1 questions (based on the UK National Curriculum) – multiplication of numbers both of which were between 1 and 5 (e.g. 3 × 4 = ?; ? × 5 = 10). If errors were made, the second set of questions were level 1 questions (and again if errors were made, the third set of questions were at level 1). If no errors were made in the first set of questions, the next set of questions were level 2 – multiplication of numbers both of which were greater than 5 (e.g. 8 × ? = 48; 7 × 9 = ?; again from the National Curriculum). If errors were made, the third set of questions were again at level 2. If no errors were made with level 2 questions, the third set comprised five difficult questions, again drawn from examples from the National Curriculum, specifically (1) 23.1 × 8 = ? (2) 364 ÷ 7 = ? (3) 45.3 × 6 = ? 48 ÷ 3 = ? (4) 602 × 57 = ? This stepped approach was to present students with stimuli in which they were able to achieve correct answers and making an error was a realistic possibility. Obtaining no correct answers would have been an exclusion criteria but this did not occur. Note that the stepped methodology does mean that students were exposed to different stimuli.

To specifically address the role of metacognitive monitoring, participants were asked the following question after they had given their answer but before they had received the feedback as to whether their answer was correct or not:

Do you think you got the answer to that question right or wrong? (The order of the words ‘right’ and ‘wrong’ was varied between participants).

Possible responses were right, wrong or don’t know.

The participants were then told if they had got the answer right or wrong. They were then asked:

Did you mean to get that question right/wrong? (depending upon if they had been correct).

Possible responses were: yes, no or don’t know.

The wording was taken from Williams and Happé (2010). The questions were designed to ascertain first whether participants were aware they had made errors (question 1 – ‘Think’) and second whether they then believed they had intended to get the answer correct or make an error (question 2 – ‘Mean’). Importantly, the two responses require different words to be produced. In addition, the errors made were analysed using a framework of errors (Ryan and Williams, 2007; see McCloskey et al., 1985). Two judges rated the errors using the Ryan and Williams (2007) framework. Inter-rater reliability was high (>80%) and disagreements over how to categorise an error were resolved through discussion.

Statistical analysis and exclusion criteria

As noted above, two participants from the ASD group were excluded as their IQ was below 70. Two ASD participants were unavailable for IQ testing. These two participants with ASD and undetermined IQ were retained as IQ was also undetermined in the TD group. There was no anecdotal evidence from teachers that any of those without IQ data would be considered as having learning difficulties. As there was most likely to be a difference between those with and without ASD in the number or errors made, it was appropriate to report the proportion of metacognitive errors. The prediction of more metacognitive errors allowed for one-tailed t-tests to be conducted for the error analysis. Where equalities of variance cannot be assumed, there is a decrease in the degrees of freedom.

Results

Overall, the ASD group and the TD group did not significantly differ on the number of correct responses made for the mathematics questions (11.35 and 12.70, respectively; t(31.89) = 1.7, ns). In all, 126 errors were made by the ASD group and 115 by the TD group. As each participant was asked 15 questions, this equates to around 20% of responses. The proportion of ‘think’ and ‘mean’ metacognitive responses after making the errors is reported in Table 1.

Table 1.

Proportion of metacognitive errors made by ASD and typically developing (TD) learners.

Mean proportion incorrect and …	ASD	TD	t-test
thought correct	0.35(0.40)	0.19(0.30)	t(60) = 1.81, p < 0.05
thought incorrect	0.31(0.41)	0.35(0.37)	t(60) = 0.36, ns
don’t know if correct	0.34(0.38)	0.47(0.42)	t(60) = 1.13, ns
meant to get correct	0.62(0.45)	0.86(0.30)	t(28.93) = 2.23, p < 0.05
meant to get incorrect	0.27(0.39)	0.03(0.14)	t(22.57) = 2.75, p < 0.05
don’t know if meant to	0.11(0.24)	0.11(0.27)	t(62) = 0.01, ns

ASD: autism spectrum disorder; ns: non-significant.

Table 1 highlights that when an answer is incorrect, those with ASD were (erroneously) more likely to think they had provided the correct answer. There were no significant group differences in the proportions who thought they had provided an incorrect response or did not know if they had provided an incorrect response. In addition, when an answer was revealed to be incorrect, the ASD group were significantly more likely to report that they meant to get the answer wrong. The TD group were more likely to report meaning to get the answer right, and very few participants reported not knowing whether they meant to get the answer right or wrong.

To explore whether age was an overall factor that impacted upon the results, partial correlations were conducted controlling for group (ASD and TD). In addition, Pearson correlations were conducted for the ASD group between verbal and performance IQ and the variables in Table 1. Age, verbal and performance IQ did not significantly relate to whether participants thought an incorrect response was right, wrong or did not know; whether participants meant to get an answer right, wrong or did not know; or the total number of correct answers (all p > 0.05) – although there was one exception: A higher performance IQ significantly related to less ‘don’t know if thought correct’ responses after an error had been made (r(22) = −0.61, p < 0.01). There were few females with ASD. As a consequence, potential sex differences were investigated by conducting t-tests between males and females from the TD group. No significant sex differences were identified in whether participants thought an incorrect response was right, wrong or did not know, nor whether participants meant to get an answer right, wrong or did not know, nor the number of correct answers (all p > 0.05).

A second analysis explored whether the metacognitive variables related to the number of errors made in the mathematics task. As there were group differences, Pearson correlations were run separately for the TD and ASD groups. For the TD group, the number of incorrect answers correlated positively with thinking a wrong answer was wrong (r = 0.44, p < 0.01) and negatively with not knowing whether a wrong answer was correct (r = −0.35, p < 0.05), but not thinking a wrong answer was correct (p > 0.05). Meaning to get an incorrect answer positively correlated with the number of incorrect answers (r = 0.52, p < 0.001 - note this occurred very rarely in the TD group), but not meaning to get a wrong answer correct or not knowing if they meant to get the answer correct (both p > 0.05). There were no significant correlates with the number of incorrect, correct or don’t know responses for the ASD group (all p > 0.2).

A third analysis explored the types of errors that had been made. The stepped methodology allowed both groups to get a comparable number of questions correct and make a comparable number of errors and both groups answered questions from levels 1, 2 and 3 (see ‘Method’ section). However, substantially more of the TD group (78%) reached level 3 than the ASD group (41%), indicating a higher level of ability for the TD group. As the same mathematical processes were embedded within each level (e.g. multiplication), Table 2 highlights the proportions of each error made for TD and ASD groups, remembering that the TD group attempted more of the harder versions than the ASD group.

Table 2.

Types of errors made by typically developing (TD) learners and those with ASD.

Error type	ASD	TD
Erroneous multiplication facts (e.g. 8 × 8 = 62)	0.65	0.46
Choose inappropriate operation to solve problems (e.g. 2 × 3 = 5)	0.15	0.00
Multiplications of tens are left out or added (e.g. 6 × 5 = 3 or 300)	0.02	0.15
Decimal place value misconception (e.g. 45.3 × 6 = 2718)	0.00	0.06
Ignores one (or more) parts (e.g. 3 × 4 = 3)	0.00	0.04
Neglects operators to conjoin numbers (e.g. 3 × 3 = 33)	0.03	0.00
In working out, error in addition of multiples (e.g. 602 × 57 = (600 × 57 = 34,200) + (2 × 57 = 114) total = 34,200 + 114 = 34,114)	0.01	0.05
Makes a carry error in a vertical layout	0.00	0.02
Did not attempt question	0.04	0.05
Unclassified error	0.10	0.15

ASD: autism spectrum disorder.

Table 2 contains an error analysis of 126 errors made by the TD group and 115 errors made by the ASD group. By far the most common error was consistent across both groups and was an error recounting multiplication facts. It is important to bear in mind the error differences relate to less than one-third of the errors. It is interesting that the second and third most common errors made by the ASD group were not made at all by the TD group. Both of these errors related to the processing of the operators (e.g. the multiplication sign in 3 × 3 = ?). Similarly, the second and third most common errors made by the TD group were hardly made at all by the ASD group. Both of these errors relate to the scale of the answer (e.g. units, tens, hundreds).

Finally, it has been found that metacognitive differences emerge in ASD when errors are made but not when correct responses are made. A final analysis explored metacognitive ‘think’ and ‘mean’ responses to correct answers (reflecting Table 1 above, but when the answers were correct). There was one significant difference: The ASD group were significantly more likely to think that they had got a correct answer right than the TD group (t(75) = 2.45, p < 0.05; all other comparisons, p > 0.05).

Discussion

Deficits in metacognition have been reported in ASD. Children with ASD have reported that they meant to make errors, that is, the observed erroneous behaviour was intended. Such a deficit in metacognitive monitoring limits the likelihood of remedial compensatory strategies being employed (metacognitive regulation). This study explored whether learners with ASD thought they had made an error and meant to make an error in a series of mathematical questions. This study found that the ASD group were both more likely to think an erroneous answer was correct, and when told they had made an error were significantly more likely to report that they meant to make that error. Whereas these indices of metacognitive monitoring significantly correlated with actual performance in the TD group, this was not the case for the ASD group. These findings are consistent with a metacognitive monitoring deficit in ASD (rather than genuinely intending to make errors at the time of doing the task).

This is the first study to report this deficit in the context of learning. The findings are consistent with previous research from other contexts, such as face recognition, memory and intention with respect to drawing (Sawyer et al., 2014; Wilkinson et al., 2010; Williams and Happé, 2010) and electroencephalogram (EEG) evidence of reduced error awareness in ASD (Henderson et al., 2006; Vlamings et al., 2008). The deficits in metacognitive monitoring in this study were only in evidence when an error was made. Indeed when no errors were made, those with ASD correctly identified they had achieved a correct response to a greater degree than the TD group. Thus, the findings of this study are consistent with the EEG research that the metacognitive monitoring deficits in ASD are most pronounced when errors are made. It is interesting to note one-third of ASD responses indicated that an incorrect response was correct, which is the same proportion reported by Williams and Happé (2010) within a non-learning context, which may be consistent with metacognitive monitoring deficits being domain-general and applying to all tasks (Gourgey, 1998; Schraw, 1998), although this needs to be empirically evaluated. The context of learning is particularly pertinent; however, as deficits in error awareness are argued to have downstream effects on responding appropriately to errors to result in learning (Henderson et al., 2006; Vlamings et al., 2008). This study suggests that highlighting when an error has occurred is the first stage in addressing metacognitive monitoring deficits in ASD.

Previous research has indexed metacognitive monitoring through a ‘feeling of knowing’ (Grainger et al., 2014; Wojcik et al., 2013, 2014) and presumably, therefore, a sense of not knowing (see also Sawyer et al., 2014). The findings of this study are consistent with this and suggest a metacognitive monitoring deficit in ASD that extends at least from meta-memory to meta-learning. Metacognitive monitoring has been associated with a sense of self-concept (Roebers et al., 2012) and is argued to inform metacognitive regulation (Nelson and Leonesio, 1988), that is, the capacity to control appropriate cognitive responses to the monitoring. Metacognitive regulation relates to executive functioning (Roebers et al., 2012) which may be consistent with executive dysfunction accounts of ASD, such as perseverating with erroneous responses (see Wilkinson et al., 2010). Whether metacognitive monitoring and metacognitive regulation provide a framework within which both meta-representation of mental states and executive dysfunction accounts of ASD overlap is an open question. Hill and Russell (2002) draw a distinction between an executive function self-monitoring as a ‘subpersonal’ process (knowing oneself to be the source of self-determined change) with a ‘personal-level’ self-referential perspective-taking process (akin to a ‘personal’ metacognitive monitoring).

It has been argued that this self-referential metacognition may relate to other-referential metacognition, both of which may share common cognitive and neural mechanisms (see Frith and Happé, 1999; Happé, 2003; see Williams, 2010). The results are consistent with a decrease in the ASD group to attribute false belief to themselves. Thus, ASD may be characterised as a deficit in self- and other- referential cognition (see Williams and Happé, 2010), or metacognition and mindreading (respectively, the latter not assessed in this study). It is also possible, however, that with a mean age of 13.7 years, such a common cognitive deficit is no longer present, but that the development of the metacognitive skills necessary in an academic situation are delayed for people with ASD. As mindreading was not assessed in this study, this possibility cannot be ruled out. One avenue to explore this hypothesis could be to explore if metacognitive monitoring during mathematics tasks relates to mindreading tasks (see Carruthers, 2009). In addition to this potential theoretical implication, Thompson et al. (2011) (see also Alter et al., 2007; Simmons and Nelson, 2006) have suggested that a lack of a metacognitive feeling of knowing serves to override intuitive processing with deliberative processing within Dual Process Theory (see Evans and Stanovich, 2013 for review). Brosnan et al. (2014), for example, report that those with ASD request significantly more information than controls to reach the same level of confidence in decision-making. This would be consistent with an ASD profile engaging in greater deliberation in response to a diminished metacognitive monitoring sense of a feeling of knowing.

Taken together, it seems that there is a deficit in error awareness in ASD, indexed by the ‘think’ question in this study. This is consistent with Russell and Jarrold (1998) who found that children with ASD were less likely to correct errors they had made, whether there was external feedback that the error had occurred or whether internal monitoring was required to be aware of the error. The ‘mean’ question is also of interest as it suggests that once an error has been made, there is a tendency in people with ASD to report that the error was intended. One possibility was that those with ASD genuinely intended to make errors, but this seems unlikely as an apparent intention to make errors did not relate to actual errors. Other possibilities include a deficit in (1) monitoring the outcome of intention, (2) discriminating between intended and unintended outcomes and (3) reporting on the nature of an intended outcome when it was unexpected (see Russell and Hill, 2001). Russell and Hill did not find evidence for these possibilities in ASD, although Phillips et al. (1998) report that children with ASD tend to report that an unintended outcome had in fact been intended, which is consistent with this study and Williams and Happé (2010). Phillips et al. and Williams and Happé, however, had a very different (shooting and drawing, respectively) tasks. One possibility is that there is a domain-general tendency to infer one’s own intention from one’s own behaviour in ASD, that is, inferring intention from knowing oneself to be the source of self-determined change (see Hill and Russell, 2002). Russell and Jarrold (1998) speculate it is this (subpersonal) executive difficulty in monitoring one’s own actions that can lead to a failure in developing a sense of being responsible for the outcomes of one’s own actions and ultimately an impoverished self-concept (see also Roebers et al., 2012). Importantly, however, this study suggests that such deficits only occur when errors are made, the ASD group did not make more errors when the ‘think’ and ‘mean’ questions were asked about their correct responses (indeed they were more accurate). Thus, typically people seem to know if they are right or have made an error and adjust their behaviour accordingly to conform to an intended outcome. Those with ASD can be characterised as knowing when they are right, but not knowing when they have made an error (so no conflict with intention) and have an associated diminished capacity to adjust behaviour accordingly.

A provisional error analysis also highlighted potential similarities and differences in the types of errors made by those with and without a diagnosis of ASD. Both groups tended to recount erroneous multiplication facts as their most common error. It is potentially interesting for future research that those with ASD also had a unique error profile that included problems processing the operator – errors that were not made by the TD group. Difficulties in mathematics specifically in people with ASD who have a normal to high IQ have been previously identified (Mayes and Calhoun, 2003; see Chiang and Lin, 2007).

Jones et al. (2009) report that 6% of adolescents with ASD demonstrated an ‘Arithmetic Dip’, which was determined by a specific underperformance upon numerical operations (16% showed the opposite pattern of an ‘Arithmetic Peak’ – both of which were largely independent of ‘Reading Dips and Peaks’). When using words rather than operators (e.g. ‘If you have two red balloons and three blue balloons, how many balloons do you have altogether?’), Titeca et al. (2014) found no deficits in children with ASD. Speculatively, therefore, there may be a small subgroup within the ASD population who have difficulties with numeric operations due to processing numeric operators. Error analysis may facilitate a better understanding of this cognitive profile, which is the cognitive profile evident in most people with ASD (with IQ > 70). Language impairment has also been associated with difficulties with mathematics. Phonological processing skills specifically impact upon numeric operations (i.e. arithmetic computation skills, see Hecht et al., 2001), although whether this relates to the processing of the numeric operator specifically remains to be determined. Pimperton and Nation (2010) found a group of poor comprehenders who also scored significantly lower on the vocabulary subscale of the WASI compared to a matched control group, did not show a deficit in processing numerical operations (though there was a specific deficit in mathematical reasoning). Thus verbal IQ (as assessed by the WASI) or reading comprehension would not be expected to account for the differences in errors between the groups. Within this study, verbal and performance IQ did not significantly relate to the ‘think’ or ‘mean’ question for the ASD group (except a higher performance IQ related to fewer ‘don’t know responses). IQ data were not available for the TD group. Verbal comprehension deficits, however, may interfere with the ability to understand fully what the metacognitive questions entail.

McCloskey et al. (1985) propose a model of mathematical processing based upon two elements, namely number processing (comprehension and production) and calculation. The calculation system includes a discrete module for processing operational symbols specifically. The model is based upon neuropsychological evidence of patients who have intact number processing but impaired processing of visually presented operators (e.g. 3 × 5 = 8). Within McCloskey et al.’s model, the present data would suggest that it is the calculation rather than the number processing system that may reflect differences between the ASD and control groups. Future research can explore the extent to which this effect may relate to the inaccurate perception or processing of operators or a more literal conception of combining numbers (e.g. 3 + 3 = 33). Directly exploring links with indices of ability beyond verbal and performance IQ as well as attention also represent avenues for future research as attentional problems have been reported in many of those with a diagnosis of ASD (for example, see Ames and White, 2011).

There are a range of limitations to this study. First, the age varied between the two groups. The metacognitive processes under investigation are typically established by age of 5½ years, so we can be fairly confident they would be established in our TD group, despite their younger age. The deficits were identified in the older group and there were no correlates with age (when controlling for diagnostic group). The younger group were selected as they were receiving the same level of mathematics tuition as those with ASD (Key Stage 2). Despite this, there was evidence that the TD group had a higher level of mathematical ability than those with ASD. The ASD group had an IQ within the normal range (performance IQ had a mean of 99.25) but impaired mathematical ability, which is the typical ASD profile (Mayes and Calhoun, 2003; 2006; see Chiang and Lin, 2007). Chiang and Lin also report there are a small group of mathematically gifted people with ASD (see also Iuculano et al., 2014; Jones et al., 2009), and the present finding may not extend to such a group, or those with an IQ outside of the normal range. It would be interesting to identify whether metacognition differentiated this mathematically gifted subgroup from the typical ASD profile. This could be assessed by presenting more complex material upon which mathematically gifted learners with and without ASD could be compared for their responses to errors. The stepped approach to increasing the difficulty of mathematics questions in this study also meant that different participants answered different questions. This needs to be borne in mind, especially with regard to the analysis of errors, although both groups were exposed to the full range of potential questions. This was structured so that participants were presented with questions that would elicit errors, in the context of also correctly answering questions. This was largely successful as all the participants answered some questions correctly and most made errors.

However, a major limitation is the matching of the samples. There was not an independent assessment of mathematical ability prior to the study undertaken by all participants. The unique cognitive profile of ASD makes matching to other clinical groups problematic (Mayes and Calhoun, 2003, 2006). Matching based upon mathematics ability may incorporate a usual low ability ASD profile with a gifted subgroup. An ideal control group would be those with a normal IQ and a specific deficit in mathematics ability; however, there are numerous methods for assessing mathematical ability and diagnostic criteria for a ‘mathematical learning disability’ remain unresolved (see Geary et al., 2007). Given the pattern of errors made, another comparison group of interest would be participants with specific language impairment and/or poor text comprehension skills. An additional option for future research could be to use even younger controls. Key Stage 2 tuition starts at the age of 7 years, although the metacognitive processes under investigation would only have been established relatively recently for this group and may not consistently predict ability until around 11 years of age (see Roebers et al., 2014; Wilkinson et al., 2010). It is of potential interest that the ASD group differed from a TD group, as the significant differences identified by Williams and Happé (2010) were largely between their ASD and ‘developmentally disabled’ groups, not between their ASD and TD groups. However, this major matching limitation needs to be borne in mind when interpreting the data, although the findings of metacognitive deficits are consistent with the literature from other contexts (Grainger et al., 2014; Sawyer et al., 2014; Williams and Happé, 2010) which may suggest a generalised weakness in metacognitive monitoring (although see Wojcik et al., 2013, 2014).

The groups were not balanced for gender. This is typical for ASD populations, but the male–female bias was unexpected in the control sample. No gender differences were identified in this study, but the imbalance of numbers means this should be interpreted with caution. There were no reported diagnoses or extreme abilities reported for the TD group and a normal IQ was assumed as it was not possible to formally assess within the classroom setting, which again is a limitation of the study. Finally, while we were able to view the ASD students’ diagnostic records to confirm their diagnosis, we were not able to administer any formal assessments of ASD. A formal clinical diagnosis was required for access to the ASD unit, but it would be interesting to explore whether metacognitive differences related to variations in severity.

Footnotes

Declaration of conflicting interests

The authors declare that they have no conflict of interest.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

References

Alter

Oppenheimer

Epley

(2007) Overcoming intuition: metacognitive difficulty activates analytic reasoning. Journal of Experimental Psychology: General 136: 569–576.

American Psychiatric Association (APA) (2013) Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Publishing.

Ames

White

(2011) Brief report: are ADHD traits dissociable from the autistic profile? Links between cognition and behaviour. Journal of Autism and Developmental Disorders 41: 357–363.

Baird

Simonoff

Pickles

(2006) Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: the Special Needs and Autism Project (SNAP). The Lancet 368: 210–215.

Baron-Cohen

(1997) Mindblindness: Essay on Autism and the Theory of Mind. London: Bradford Books.

Baron-Cohen

Leslie

Frith

(1985) Does the autistic child have a ‘theory of mind’? Cognition 21: 37–46.

Brosnan

Chapman

Ashwin

(2014) Adolescents with autism spectrum disorder show a circumspect reasoning bias rather than ‘Jumping-to-Conclusions’. Journal of Autism and Developmental Disorders 44: 513–520.

Carruthers

(2009) How we know our own minds: the relationship between mindreading and metacognition. Behavioral and Brain Sciences 32: 138–182.

Castelli

Frith

Happé

(2002) Autism, Asperger syndrome and brain mechanisms for the attribution of mental states to animated shapes. Brain 125: 1839–1849.

10.

Chiang

Lin

(2007) Mathematical ability of students with Asperger syndrome and high-functioning autism: a review of literature. Autism 11: 547–556.

11.

Estes

Rivera

Bryan

(2011) Discrepancies between academic achievement and intellectual ability in higher-functioning school-aged children with autism spectrum disorder. Journal of Autism and Developmental Disorders 41: 1044–1052.

12.

Evans

JSBT

Stanovich

(2013) Dual-process theories of higher cognition: advancing the debate. Perspectives on Psychological Science 8: 223–241.

13.

Farrant

Blades

Boucher

(1999a) Recall readiness in children with autism. Journal of Autism and Developmental Disorders 29: 359–366.

14.

Farrant

Boucher

Blades

(1999b) Metamemory in children with autism. Child Development 70: 107–131.

15.

Flavell

(1979) Metacognition and cognitive monitoring: a new area of cognitive–developmental inquiry. American Psychologist 34: 906–911.

16.

Frith

(2001) Mind blindness and the brain in autism. Neuron 32: 969–979.

17.

Frith

Happé

(1999) Theory of mind and self-consciousness: what it is like to be autistic. Mind & Language 14: 1–22.

18.

Geary

Hoard

Byrd-Craven

(2007) Cognitive mechanisms underlying achievement deficits in children with mathematical learning disability. Child Development 78: 1343–1359.

19.

Gourgey

(1998) Metacognition in basic skills instruction. Instructional Science 26: 81–96.

20.

Grainger

Williams

Lind

(2014) Metacognition in autism: diminished feelings of knowing and mindreading. Journal of Abnormal Psychology 123: 650–659.

21.

Happé

(2003) Theory of mind and the self. Annual New York Academy of Science 1001: 134–144.

22.

Hecht

Torgesen

Wagner

(2001) The relations between phonological processing abilities and emerging individual differences in mathematical computation skills: a longitudinal study from second to fifth grades. Journal of Experimental Child Psychology 79: 192–227.

23.

Henderson

Schwartz

Mundy

(2006) Response monitoring, the error-related negativity, and differences in social behavior in autism. Brain and Cognition 61: 96–109.

24.

Higgins

Katsipataki

Kokotsaki

(2013) The Sutton Trust – Education Endowment Foundation Teaching and Learning Toolkit: Metacognition and Self-Regulation. London: Education Endowment Foundation.

25.

Hill

Russell

(2002) Action memory and self-monitoring in children with autism: self versus other. Infant and Child Development 11: 159–170.

26.

Hobson

(1990) On the origins of self and the case of autism. Development and Psychopathology 2: 163–181.

27.

Iuculano

Rosenberg-Lee

Supekar

(2014) Brain organization underlying superior mathematical abilities in children with autism. Biological Psychiatry 75: 223–230.

28.

Jones

CRG

Happé

Golden

(2009) Reading and arithmetic in adolescents with autism spectrum disorders: peaks and dips in attainment. Neuropsychology 23: 718–728.

29.

Lockl

Schneider

(2002) Developmental trends in children’s feeling-of-knowing judgements. International Journal of Behavioral Development 26: 327–333.

30.

Lombardo

Baron-Cohen

(2011) The role of the self in mindblindness in autism. Consciousness and Cognition 20: 130–140.

31.

Lombardo

Barnes

Wheelwright (2007) Self-referential cognition and empathy in autism. PLoS ONE. Available at: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0000883

32.

Lombardo

Chakrabarti

Bullmore

(2010) Atypical neural self-representation in autism. Brain 133: 611–624.

33.

McCloskey

Caramazza

Basili

(1985) Cognitive mechanisms in number processing and calculation: evidence from dyscalculia. Brain and Cognition 4: 171–196.

34.

Maxwell

Grenier

(2014) The effects of metacognitive treatments on the academic performance of students with learning disabilities: a meta-analysis. Learning and Individual Differences 29: 141–149.

35.

Mayes

Calhoun

(2003) Analysis of WISC-III, Stanford-Binet: IV, and academic achievement test scores in children with autism. Journal of Autism and Developmental Disorders 33: 329–341.

36.

Mayes

Calhoun

(2006) Frequency of reading, math, and writing disabilities in children with clinical disorders. Learning and Individual Differences 16: 145–157.

37.

Nelson

Leonesio

(1988) Allocation of self-paced study time and the ‘labor-in-vain’ effect. Journal of Experimental Psychology: Learning, Memory, and Cognition 14: 476–486.

38.

Nelson

Narens

(1990) A theoretical framework and new findings. In: Bower

(ed.) The Psychology of Learning and Motivation: Advances in Research and Theory, vol. 26. New York: Academic Press, pp. 125–141.

39.

Pfeifer

Merchant

Colich

(2013) Neural and behavioral responses during self-evaluative processes differ in youth with and without autism. Journal of Autism and Developmental Disorders 43: 272–285.

40.

Phillips

Baron-Cohen

Rutter

(1998) Understanding intention in normal development and autism. British Journal of Developmental Psychology 16: 337–348.

41.

Pimperton

Nation

(2010) Understanding words, understanding numbers: an exploration of the mathematical profiles of poor comprehenders. British Journal of Educational Psychology 80: 255–268.

42.

Powell

Jordan

(1993) Being subjective about autistic thinking and learning to learn. Educational Psychology 13: 359–370.

43.

Premack

Woodruff

(1978) Does the Chimpanzee have a theory of mind? Behavioral and Brain Sciences 4: 515–526.

44.

Roebers

Cimeli

Röthlisberger (2012) Executive functioning, metacognition, and self-perceived competence in elementary school children: an explorative study on their interrelations and their role for school achievement. Metacognition and Learning 7: 151–173.

45.

Roebers

Krebs

Roderer

(2014) Metacognitive monitoring and control in elementary school children: their interrelations and their role for test performance. Learning and Individual Differences 29: 141–149.

46.

Russell

Hill

(2001) Action monitoring and intention reporting in children with autism. Journal of Child Psychology and Psychiatry 42: 317–328.

47.

Russell

Jarrold

(1998) Error-correction problems in autism: evidence for a monitoring impairment? Journal of Autism and Developmental Disorders 28: 177–188.

48.

Ryan

Williams

(2007) Children’s Mathematics 5–15: Learning from Errors and Misconceptions. Maidenhead: Open University Press.

49.

Sawyer

ACP

Williamson

Young

(2014) Metacognitive processes in emotion recognition: are they different in adults with Asperger’s disorder? Journal of Autism and Developmental Disorders 44: 1373–1382.

50.

Schneider

Artelt

(2010) Metacognition and mathematics education. ZDM: The International Journal of Mathematics Education 42: 149–161.

51.

Schraw

(1998) Promoting general metacognitive awareness. Instructional Science 26: 113–125.

52.

Schraw

Moshman

(1995) Metacognitive theories. Educational Psychology Review 7: 351–371.

53.

Simmons

Nelson

(2006) Intuitive confidence: choosing between intuitive and nonintuitive alternatives. Journal of Experimental Psychology: General 135: 409–428.

54.

Thompson

Prowse Turner

Pennycook

(2011) Intuition, reason, and metacognition. Cognitive Psychology 63: 107–140.

55.

Titeca

Roeyers

Josephy

(2014) Preschool predictors of mathematics in first grade children with autism spectrum disorder. Research in Developmental Disabilities 35: 2714–2727.

56.

Van der Stel

Veenman

MVJ

Deelen

(2010) The increasing role of metacognitive skills in math: a cross-sectional study from a developmental perspective. ZDM: The International Journal of Mathematics Education 42: 219–229.

57.

Vlamings

PHJM

Jonkman

Hoeksma

(2008) Reduced error monitoring in children with autism spectrum disorder: an ERP study. European Journal of Neuroscience 28: 399–406.

58.

Wechsler

(1999) Wechsler Abbreviated Scale of Intelligence (WASI). San Antonio, TX: Harcourt Assessment.

59.

Wilkinson

Best

Minshew

(2010) Memory awareness in individuals with autism. Journal of Autism and Developmental Disorders 40: 1371–1377.

60.

Williams

(2010) Theory of own mind in autism: evidence of a specific deficit in self-awareness? Autism 14: 474–494.

61.

Williams

Happé

(2009) What did I say? Versus what did I think? Attributing false beliefs to self amongst children with and without autism. Journal of Autism and Developmental Disorders 39: 865–873.

62.

Williams

Happé

(2010) Representing intentions in self and other: studies of autism and typical development. Developmental Science 13: 307–319.

63.

Williams

Lind

Happé

(2009) Metacognition may be more impaired than mindreading in autism. Behavioral and Brain Sciences 32: 162–163.

64.

Wojcik

Moulin

CJA

Souchay

(2013) Metamemory in children with autism: exploring ‘feeling-of-knowing’ in episodic and semantic memory. Neuropsychology 27: 19–27.

65.

Wojcik

Waterman

Lestié

(2014) Metacognitive judgments-of-learning in adolescents with autism spectrum disorder. Autism 18: 393–408.