Abstract
For autistic students receiving special education services, little is known about their relative strengths, weaknesses, and enjoyment across different math content areas; their overall math interest and persistence are also not well-studied. Using the 2017 eighth-grade National Assessment of Education Progress data, this study finds, relative to general education peers with the same math proficiency level, autistic students scored higher and exhibited faster speed in solving visuospatial problems (e.g. identifying figures), but scored lower on math word problems with complex language or social context. Autistic students reported a higher level of enjoyment in solving math problems related to finding areas of shapes or figures but a lower level of persistence than their non-autistic, general education peers. Our work points out the need to help autistic students overcome their weaknesses in word problems and develop their mathematical persistence.
Although popular media portrayals often stereotype autistic individuals as more likely than the general population to be good at math (Moore, 2006; Morton, 2001; Ross, 2006; Safer, 2012), research evidence is mixed on the general math abilities of autistic individuals. A study summarizing the literature shows they may have lower math ability than the general population (Chiang & Lin, 2007). In contrast, some studies find no difference in math abilities between autistic individuals and their typically developing peers across a wide age range, from preschools (Titeca et al., 2015; Titeca et al., 2014) and school-age children (May et al., 2015; Mayes & Calhoun, 2007) to adults (Goldstein et al., 2001; Tops et al., 2017). A study finds that autistic students excel in math, with scores of 1.3 standard deviations (SDs) above the normed sample on overall math proficiency (Assouline et al., 2012).
In addition to general math performance, studies that involve subgroups of autistic students reveal strengths and weaknesses in specific areas of math skills. Some autistic individuals show strong math ability in doing rapid numerical calculations (Baron-Cohen et al., 2007; Chia, 2012; Iuculano et al., 2014; Wei, Christiano, Yu, Wagner, & Spiker, 2014) but others show either equivalent performance (Gagnon et al., 2004) or worse performance (Aagten-Murphy et al., 2015; Bullen et al., 2020) than the non-autistic group. Some studies found autistic individuals good at visuospatial abilities (Caron et al., 2004; Mitchell & Ropar, 2004; Shah & Frith, 1983, 1993; Soulières et al., 2011; Stevenson & Gernsbacher, 2013). For example, Shah and Frith (1983, 1993) report better performance on embedded figure tests among 20 autistic adolescents and block design tests among 20 autistic adults ages 15–25. Mitchell and Ropar (2004) found that autistic students appear faster at searching for feature and conjunctive targets in a visual array when compared with their non-autistic peers. Despite these strengths in numerical calculation and visuospatial awareness, studies reveal that some autistic students appear to struggle with other math skills. Some struggle with word problems and everyday math knowledge (Bae et al., 2015; Wei, Christiano, Yu, Wagner, & Spiker, 2014). Some of them also had lower processing speeds (Mayes & Calhoun, 2007). For instance, Hedvall et al. (2013) report that preschool autistic children have slower processing speed when compared to their verbal skills. Similarly, Oliveras-Rentas et al. (2012) found that autistic children with an IQ above 70 appeared to have a lower processing speed score than their general IQ score. While these studies have looked at processing speed of general math problem-solving, no studies have examined processing speed pertaining to different math content areas.
Math interest is correlated with science, technology, engineering, and mathematics (STEM) participation and career choice (Wei et al., 2015). Studies involving a nationally representative sample of autistic college students report that autistic students were more likely to pursue STEM majors, including math, when compared to other disability groups and the general population (Wei, Christiano, Yu, Blackorby, et al., 2014; Wei et al., 2013). However, there is a dearth of evidence on the characteristics that lead to STEM career choices among autistic students—namely, their enjoyment, interests, and persistence in STEM. In one study, though, Martin et al. (2020) compared self-reported technology and engineering interest between 16 middle school autistic students and 30 peers without disabilities and found no differences between the two groups. Cooper and Farkas (2022) found that the extent to which autistic students see themselves as a “math person” is the strongest predictor of high school math grade point average (GPA) among those with above-average math scores. Georgiou et al. (2018) documented that autistic students report greater fear of failure than non-autistic peers, and fear of failure is negatively associated with their math interest. All previous research focuses on overall math interest. So far, we have found no studies comparing enjoyment in the different content areas of math between autistic students and their non-autistic peers.
To fill the gap in the literature, this study is the first to evaluate how autistic students compare to their non-autistic, general education peers with respect to how well they perform on math problems in different content areas, how quickly they answer such questions, and how much they enjoy performing math tasks in different content areas. Our research questions are the following:
RQ1. What types of problems do eighth-grade autistic students demonstrate better performance and more rapid response times than their general education peers?
RQ2. What types of problems do eighth-grade autistic students demonstrate lower performance and longer response times than their general education peers?
RQ3. What types of problems do eighth-grade autistic students report higher levels of enjoyment than their general education peers?
RQ4. Do eighth-grade autistic students report a higher overall math interest and persistence than their general education peers?
Methods
Data sources
Our study used restricted data from the National Assessment of Educational Progress (NAEP). This nationally administered low-stakes test represents student achievement across the United States, individual states, and some large urban districts. NAEP has long been considered the “gold” standard of assessment, and the federal government has used it as its official measure of how students perform academically in the United States (National Center for Education Statistics, 2009). In each participating state, NAEP assesses a representative sample of schools’ and students’ performance in reading and mathematics. The 2017 NAEP math assessment sampled 144,900 eighth-grade students from 6500 schools. Participating students took two blocks of the NAEP math assessment on tablets.
Study sample
The NAEP program released 2 out of 10 blocks of math items used in the 2017 digital assessment and associated restricted-use student data. One released math block includes 28,200 students, and we analyzed this dataset. The other released math block includes only 2800 students. We did not use this dataset for analysis because the sample size for autistic students is too small for any meaningful analysis.
NAEP requires students with disabilities to be fully included in the assessment. About 90% of students with disabilities in eighth grade were assessed in the NAEP math assessment (U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress, 2019). The rest of the students with disabilities could not participate even with accommodations. Autistic students in this study were identified by their schools as receiving special education services under the autism disability category. General education peers were students who were from the same schools as autistic students and who did not have a disability.
For each student with disabilities who was sampled to participate in the NAEP test, a principal/assistant principal, special education teacher, bilingual education/ESL teacher, or classroom teacher filled out a disability questionnaire. This questionnaire collected information about the student’s disability category and whether the student participated in the state assessment with accommodations (and what type of accommodations should be provided), without accommodations, or met the participation criteria for the state alternate assessment. Most students with disabilities participated in the NAEP test except those who met the participation criteria for the alternate assessment. The individualized education program teams at the students’ schools determined what testing accommodations the students should receive, including granting extended-time accommodations. Students with moderate disabilities were more likely to receive extended-time accommodations than those with mild or severe disabilities (Wei & Zhang, 2023). The extended time allows for accurate measurement of the student’s abilities instead of their weakness in reading or math processing speed due to their disabilities (Duncan & Purcell, 2020). NAEP assessment followed the test accommodation decision based on the individualized education program teams’ decisions.
Our analysis sample included 70 unaccommodated autistic eighth graders who completed the 15-item test within 30 min, 80 autistic eighth graders who received extended-time accommodations to finish the test within 90 min, and 24,870 general education peers who completed the test within 30 min.
Measures
The 2017 eighth-grade NAEP digital math assessment process data provide a unique opportunity to answer our research questions. NAEP collects not only students’ performance data but also their response time on each item. In addition, a post-assessment survey collects students’ self-report of enjoyment by content areas, overall math interest, and persistence. We take advantage of the available data on item performance, item response time, and survey responses to understand the strengths, weaknesses, and enjoyment in solving problems in different math areas among eighth-grade autistic students.
Item-level performance and overall proficiency
We analyzed all 15 items on this math assessment. NAEP math items were developed by math experts based on the NAEP assessment framework that describes math content that the US students in that grade level should master. NAEP assessment has achieved the highest standard for reliability and validity. The Cronbach’s alpha internal consistency reliability of the total score on this release math block is 0.80 (Witmer et al., 2022).
Each item falls into one of the five content areas described in Table 1. These items include fractions, lines, rotating shapes, products of two 2-digit decimals, x/y intercept, graphs and plots, measurement, diameter and circumference of a circle, and area of a shape. Six binary items have a maximum score of 1: A student scored 1 if they answered an item correctly or 0 if they answered it incorrectly. Eight items have a maximum score of 2: A student scored 2 if they answered an item correctly, 1 if they answered partially correctly, or 0 if they answered incorrectly. One item has a maximum score of 4, where 0 is incorrect, 1 is minimal, 2 is partial, 3 is satisfactory, and 4 is extended. NAEP reports a student’s math proficiency level based on their overall performance on two blocks of the math items, adjusting for the difficulty of the items. The levels range from 1 to 4, where 1 is below basic, 2 is basic, 3 is proficient, and 4 is advanced.
Item content, score range, readability, and social context complexity.
Source: All 15 math test items can be found at https://www.nationsreportcard.gov/nqt/searchquestions
Item 3 does not include any text that can be used to calculate the Flesch Read Ease score.
Total response time and item response time
Item response time and total response time on the assessment were recorded in the NAEP process data. Because the NAEP math assessment allows students to jump between items, a student’s time on task for an item is often not a single data point but rather the cumulative time spent during each visit to the item across multiple visits. Item response time is defined as the total time in seconds that a student worked on an item, including initial visits and all revisits of the same item. Total response time is the total time in seconds a student worked on all 15 items in this math assessment. The item response time on an unvisited question is 0 s. The total response time on the math block did not include the unvisited questions.
Enjoyment by math content area and overall interest and persistence
On the post-assessment survey, NAEP asks each student to rate on a 5-point Likert-type scale (1 = not at all; 2 = a little bit; 3 = somewhat; 4 = quite a bit; 5 = enjoy a lot) whether the student enjoyed solving math problems in six math content areas (arithmetic, finding areas of shapes/figures, solving probabilities and events, solving equations, constructing and building graphs, or working with geometric figures). Math interest index score is based on six survey items: (1) I enjoy doing math; (2) I look forward to my math class; (3) I am interested in the things I learned in math; (4) I think making an effort in math is worthwhile; (5) I think math will help me even when I am not in school; (6) I think it is important to do well in math.
The persistence index score is based on four survey items: (1) I finish whatever I begin; (2) I try very hard even after making mistakes; (3) I keep working hard even when I feel like quitting; (4) I keep trying to improve self, even if it takes a long time to get there.
Students rate each item under the math interest index and persistence index on a 5-point Likert-type scale (1 = not at all like me; 2 = a little bit like me; 3 = somewhat like me; 4 = quite a bit like me; 5 = exactly like me). NAEP uses an item response theory partial-credit scaling model to estimate an index score and place that score on a scale ranging from 0 to 20.
Student demographic characteristics
Demographic variables included student gender; age in years at the time of testing; and race/ethnicity, coded as four dichotomous variables for African American, Hispanic, White, or Other (Asian, American Indian, Pacific Islander, or multiple races).
Linguistic and social content analysis of math items
The text of math items was typed into Microsoft Word, and Flesch Reading Ease scores were calculated for each math item using Microsoft Word readability statistics. Flesch Reading Ease score has been used to measure the difficulty level of a passage for more than 70 years (Farr et al., 1951). It has been validated and improved over the years (Eleyan et al., 2020). According to Stockmeyer (2009), the Microsoft Word readability program is an easy and accurate way to obtain Flesch Reading Ease scores. It gives a text a score between 1 and 100, with 90–100 being very easy to read (fifth-grade level) and 0–30 being very difficult to read (college level). The higher the score, the easier to read the text.
Three subject matter experts with PhD degrees in special education, educational psychology, and psychometrics independently rated the complexity of the social context of each item on three levels. In the first level,
Statistical analysis
Descriptive analysis
All analyses were performed using R version 4.1.0 (R Core Team, 2021). Descriptive analyses of demographic characteristics, math total scores, item scores, math proficiency level, total response time, and item response time were performed for unaccommodated autistic students, autistic students who received extended-time accommodations, and their unaccommodated general education peers.
Welch’s t-test and the Z-test for two proportions
For binary variables, the Z-test for two proportions was used to test whether the autistic group was significantly different from the general education group in demographic composition or the proportion of students who answered an item correctly. Welch’s t-test was used for continuous variables to test whether either autistic group significantly differed from the general education group in the mean value. Welch’s t-test, also known as the unequal variances t-test, is preferred over the Student’s t-test when the two groups’ sample sizes and variance are unequal (Derrick et al., 2016; Ruxton, 2006). Cohen’s d effect size is reported.
Standardized mean difference test
Differential item functioning (DIF) is a well-known method to compare the item-level test performance of two groups after the two groups are matched on the measured trait (Camilli & Shepard, 1994; Holland & Wainer, 2012). If there is no DIF, members from each group at each ability level should have the same probability of answering the item correctly. If a test item is identified as showing DIF, it implies that the two groups respond to the item differently, and the psychometric properties vary by group characteristics (Ercikan, 2002). Significant DIF items that favor autistic students indicate that autistic students had relative strengths in the areas these items are measuring. Significant DIF items that favor general education peers indicate that autistic students had relative weakness in these math areas.
With the availability of item response time data, DIF analyses have been expanded to study differential response time (DRT; Ercikan et al., 2020), comparing differences in item response time conditioning on math proficiency. This study used standardized mean difference (SMD) statistics (Ercikan et al., 2020) to calculate DIF and DRT. Specifically, we denote general education peers as the reference group
where
Community of involvement
There was no specific community input from autistic individuals or family members on the analysis presented in this study.
Results
Descriptive analysis
Table 2 provides descriptive statistics for two groups of autistic students and their general education peers. One notable demographic difference between autistic students and their general education peers is gender. Autistic students had a significantly higher proportion of male students in both the unaccommodated autistic group (83%) and the group with extended-time accommodations (81%), compared to their general education peers (49%) in both cases (p < 0.001). Another notable demographic difference is race. In the unaccommodated autistic group, 64% of students were White, which was significantly higher than the proportion of White students (48%) in the general education peers group (p < 0.01). These demographic findings are consistent with national autism prevalence statistics (Centers for Disease Control and Prevention, 2020), which also present a gender and racial disparity that is biased toward male and White autistic individuals.
Sample characteristics for autistic students and their general education peers.
Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (2019), Response Process Data From the 2017 NAEP Grade 8 Mathematics Assessment.
ETA: extended-time accommodation.
Samples sizes were rounded to the nearest 10 following the NAEP restricted data use agreement. Significance notations are between each autistic group and the general education group.
p < 0.05, **p < 0.01, ***p < 0.001.
In addition, the unaccommodated autistic group had a significantly higher proportion of students who scored at the highest proficiency level (advanced) compared to their general education peers (17% vs 10%, p < 0.05). However, the autistic group with extended-time accommodations had a significantly higher proportion of students who scored at the lowest proficiency levels relative to their general education peers (52% vs 25%, p < 0.001). Although the unaccommodated autistic group had similar total math scores as their general education peers, the autistic group with extended-time accommodations had significantly lower total math scores than their general education peers (7.51 vs 9.16, p < 0.05).
In terms of the total time spent on the 15-item math test, the unaccommodated autistic group spent less time than their general education peers (1320.35 vs 1403.87 s, p < 0.05), but the autistic group with extended-time accommodations spent more time than their general education peers (1580.97 vs 1403.87 s, p < 0.05).
Linguistic and social complexity analysis of NAEP math items
In addition to classifying the math content area measured by each item (column “Content” in Table 1), we also analyzed the linguistic (column “Flesch Reading Ease Score”) and social complexity (column “Social Context Complexity Score”) of each item. Eleven of the 15 math items had a readability score higher than 70—below the eighth-grade level. Three items had a readability score less than 70—at or above eighth-grade level.
Results of our item social content analysis showed that 10 of the 15 assessment items had no social context. For instance, Item 3 had no social context and simply asked the student to multiply two numbers together. Meanwhile, of the five items with social context, one item had a low level of social context complexity while four items had a high level of social context complexity: Item 1 had low social context and presented students with a setting in which a grocery store has milk on sale at 25% off; students were asked to find the correct fraction that equals to 25%. By contrast, four items had high social complexity:
Item 2, which asked students to complete a circle chart to present survey data collected by a teacher from their students;
Item 9, which asked students to figure out the distance between town A and town B on a map;
Item 11, which presented a linear equation problem in the social context of comparing prices of two gym membership plans; and
Item 13, which asked students to evaluate a circle graph and a bar graph based on data collected about pets at home.
These four items were presented in everyday life experiences relating to attending schools, driving, going to a gym, and shopping for groceries. To succeed on these types of questions, students needed to understand the social experience described and then be able to apply their math skills to solve real-world problems.
DIF
Table 3 shows the mean and SDs of item-level scores for four groups (unaccommodated autistic group, autistic group with extended-time accommodations, students with disabilities with extended-time accommodations, 1 and general education peers), along with the SMD comparing the unaccommodated autistic group to their general education peers, and the SMD comparing the autistic group who received extended-time accommodations to their general education peers. Our SMD procedure to identify DIF items indicates that the unaccommodated autistic group on average scored lower on Items 2, 9, and 10 than the general education peers given the same level of math proficiency. The subject matter experts rated two of these three items as having high social content complexity. The DIF analysis comparing the autistic group who received extended-time accommodations and the general education peers suggests that the autistic group on average scored higher on Items 7 (calculate the diameter of a circle) and 12 (identify figures) than the general education peers given the same level of math proficiency.
Item scores and DIF analysis for autistic students and their general education peers.
Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (2019), Response Process Data From the 2017 NAEP Grade 8 Mathematics Assessment.
DIF: differential item functioning; ETA: extended-time accommodation; SWD: students with disabilities; SMD: standardized mean difference.
Significance notations in columns “Autistic Unaccommodated (n = 70)” and “Autistic ETA (n = 80)” are between each autistic group and the general education group. Significance notations in columns “SWD ETA (n = 2760)” are between the autistic ETA group and the SWD ETA group. Samples sizes were rounded to the nearest 10 following the National Assessment of Educational Progress restricted data use agreement.
p < 0.05, ** p < 0.01, ***p < 0.001.
DRT
The SMD results in Table 4 show that the unaccommodated autistic group spent less time on Items 5, 8, 11, 12, and 14 than their general education peers with the same proficiency level. These five items include tasks to compare measurements, mentally rotate a triangle, interpret linear equations, identify figures, and construct data analysis plots. An alternative hypothesis for why autistic students in the standard condition are faster is that they are more likely to input a guess quickly. To alleviate this concern, we checked that the proportion of students in each group who answered each item correctly was similar on these five items (Table 4). There were no statistically significant differences in mean score between the two groups except that the unaccommodated autistic group had a significantly higher percentage of students who answered Item 12 (identify figures) correctly than the general education peers. This finding suggests that it is unlikely that the unaccommodated autistic group’s shorter response time was due to random guessing on these items.
Item response time and DRT analysis for autistic students and their general education peers.
Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (2019), Response Process Data From the 2017 NAEP Grade 8 Mathematics Assessment.
DRT: differential response time; ETA: extended-time accommodation; SMD: standardized mean difference.
Samples sizes were rounded to the nearest 10 following the National Assessment of Educational Progress restricted data use agreement.
p < 0.05, **p < 0.01, ***p < 0.001.
The autistic group with extended-time accommodations spent more time on Items 1, 13, 14, and 15 than the general education group with the same proficiency level. Although the autistic group with extended-time accommodation performed worse on these four items than the general education group, only Item 1 reached a statistically significant difference at alpha = 0.05.
Enjoyment by content areas, math interest, and persistence
Table 5 shows that, as compared with general education peers, both unaccommodated and accommodated autistic students reported a higher level of math enjoyment in solving math tasks relating to finding areas of shapes or figures (with a Cohen’s d effect size of 0.25 and 0.15, respectively), but a lower level of persistence (with a Cohen’s d effect size of −0.45 and −0.23, respectively). Furthermore, autistic group with extended-time accommodations reported less enjoyment in solving math equations than the general education group (Cohen’s d = –0.28). Please note that only 3 out of 12 comparisons on enjoyment outcomes were statistically significant at the 0.05 level possibly due to small sample sizes.
Survey responses for autistic students and their general education peers.
Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress 2019, Response Process Data From the 2017 NAEP Grade 8 Mathematics Assessment.
ETA: extended-time accommodation.
Two independent-sample t-tests were used to test the difference between the two groups. Samples sizes were rounded to the nearest 10 following the National Assessment of Educational Progress restricted data use agreement.
p < 0.05, **p < 0.01, ***p < 0.001.
Discussion
This study is the first to delve deeper to identify the relative strengths, weaknesses, and enjoyment across different math content areas among autistic students, as well as their overall math interest and level of persistence, using national NAEP data. Previous research has shown that some autistic individuals appear to have strengths in the calculation ( Baron-Cohen et al., 2007; Chia, 2012; Wei, Christiano, Yu, Wagner, & Spiker, 2014) and abstract spatial reasoning (Caron et al., 2004; Mitchell & Ropar, 2004; Shah & Frith, 1983, 1993; Soulières et al., 2011; Stevenson & Gernsbacher, 2013), and may rely more on visual-mental representations instead of verbal ones in understanding abstract concepts (Grandin, 2006; Kim & Cameron, 2016). This study provides additional evidence of the strength of autistic students in visuospatial cognitive skills. As compared with their general education peers, unaccommodated autistic students not only scored higher but also more quickly solved a math item that asked them to identify figures. Unaccommodated autistic students also exhibited shorter response times on math tasks comparing measures using unit conversions, mentally rotating a triangle, interpreting linear equations, and constructing data analysis plots. Although the autistic group with extended-time accommodations on average had a lower total math score and a much higher percentage in the lowest proficiency category than their general education peers, they had a higher accuracy rate on items involving calculating diameters of a circle and identifying figures.
Corresponding to the more challenging math skills, previous studies have shown that autistic students struggle with math word problems and everyday math knowledge (Bae et al., 2015; Miller et al., 2017). A study attributes this struggle to an underlying impairment in processing and comprehending linguistic information (Miller et al., 2017). Other studies suggest that their challenge with working memory and processing speed—not their reading comprehension skills—is responsible for their relatively lower performance on math word problems (Assouline et al., 2012; Oswald et al., 2016). Other studies believe that a lack of inhibitory control restricts their ability to suppress irrelevant literal and numerical information of math word problems from their working memory (Sabagh-Sabbagh & Pineda, 2010; Schmitt et al., 2018). Still, other studies suggest that the limited community experiences (Nahmias et al., 2014) and challenges with communication and social interaction (American Psychiatric Association, 2013) make everyday math knowledge particularly difficult for autistic students (Bae et al., 2015).
Our findings suggest that linguistic complexity could be one of the reasons that autistic students struggle with math word problems. For example, although the Flesch Reading Ease score (70.6) for Item 10 indicates it is appropriate for a seventh-grade level, this item includes the word “transversal,” which is crucial to answering the item correctly but may be unfamiliar to many students. However, language alone is unlikely to fully explain why autistic students struggle with certain word problems, as Items 2 and 9—difficult for unaccommodated autistic students—were not particularly difficult when measured by the Flesch Reading Ease feature. An alternative hypothesis is that autistic students struggled with these items because of the items’ substantial social context complexity as rated by our subject matter experts. Social context complexity could be a construct-irrelevant barrier that prevents autistic students from performing as well as their general education peers, even if they have the same level of math proficiency. These items required students to have adequate knowledge about the social situation and to understand the social interactions described in the word problem, formulate the word problem’s social context into mathematical language, and then use their math skills to solve the problem.
Math instruction will better support learning for autistic students by including approaches that help them comprehend real-life math problems. Educators can borrow meta-cognitive and explicit schema instruction that is effective for students with learning disabilities (Powell et al., 2020), focusing on teaching autistic students to connect math concepts and real-life math problems.
Consistent with the performance strength in visuospatial skills, our survey results show that autistic students reported a significantly higher level of enjoyment working with shapes and figures, which is related to their strength in identifying figures for both autistic groups. By contrast, math tasks related to solving equations were not particularly appealing to autistic students in the extended-time accommodation condition, who reported a lower level of enjoyment on these math tasks than their general education peers. In addition, the lower level of persistence reported by both groups of autistic students as compared to general education peers suggests that educators, K–12 schools, and postsecondary institutions need to provide additional instructional support and services for autistic students to inspire and protect their innate interest in visuospatial reasoning. One way to provide such support may be to have future studies incorporate persistence and resilience training into the social skills training for autistic students.
The findings of this study also have implications for math test developers. Although the performance and response time differences found between autistic students and general education peers may be due to true math ability differences, the content and design of certain items (e.g. items with complex language or social contexts) may make them particularly difficult for autistic students. Math test developers could simplify the language and social context of math word problems to make the math assessment more equitable, fair, and accessible for autistic students.
There are several limitations to consider when interpreting these findings: First, a major limitation of this study is that the autistic students in the NAEP dataset are not representative of the national population of autistic students. The estimated prevalence of autism among children of age 8 in the United States was 2.3% in 2018 (Centers for Disease Control and Prevention, 2020). However, the proportion of autistic students in our analytic sample was 0.6%, which is lower than the national average. Our sample includes autistic students and their general education peers who took 1 of 10 blocks of the grade 8 math assessment. The smaller-than-expected percentage of the autistic students in our sample could be due to random variation. Second, the study had only a small sample of autistic students (70 in the unaccommodated group and 80 in the extended-time accommodation group) for the SMD analysis. The recommended sample size for DIF analysis is for the smaller group to comprise at least 100 examinees, with the total number of examinees equal to 500 (Buzick & Stone, 2011). Although our total sample size is larger than 500, our autistic student sample size is smaller than the 100 recommended sample size. Third, the autism diagnosis was based on district reports of students receiving special education services under the autism category. The general education peers in this study were eighth-grade students who did not have a disability. A lack of screening procedures limits this study to determining learning difficulties or neurodevelopmental or psychiatric disorders among the comparison group. Fourth, the NAEP study did not assess students’ language skills, working memory, or executive function. Future studies must investigate how these variables are associated with math performance across different content areas for autistic students. Finally, to take full advantage of the performance and time usage data on all 15 items, our research required testing multiple hypotheses simultaneously, which could potentially inflate the rate of false discoveries. Although we set the significance level to 0.05 originally, a more conservative p value of 0.01 should be used to reduce the risk of committing the type I error. When a more stringent p-value cutoff of 0.01 is used, the SMD statistics to test the DIF between unaccommodated autistic students and general education peers still achieve statistical significance, suggesting that autistic students showed weakness in math word problems (Items 2, 9, and 10). The SMD statistics to test the DRT between unaccommodated autistic students and general education peers are also still significant for Items 5 and 14, confirming their strength in speed on solving measurement problems and constructing data analysis plots. Moreover, unaccommodated autistic students have lower persistence levels than their general education peers, which reach statistical significance at 0.01. However, our other results do not achieve the higher significance level.
Despite these limitations, our findings represent the first study to identify the relative strengths and weaknesses and enjoyment by specific math content areas—as well as overall math interest and persistence—among autistic students using NAEP data. Visuospatial ability is a critical component of human intelligence (Park et al., 2010). It is associated with success in many high-paying occupations, including architecture, surgery, engineering, and science (Park et al., 2010). This study suggests that the enjoyment and strengths in visuospatial reasoning of autistic individuals may indeed provide them with the potential to become sources of STEM talent. However, their weakness in word problems and low levels of persistence suggests that further research is needed to understand why autistic students struggle with word problems, how these weaknesses in math word problems skills and persistence are associated with future persistence and academic success, and what educators can do to better support autistic students’ math learning given what we know.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported here was supported by the Institute of Education Sciences, US Department of Education, through Grant R324P210005 to SRI International and University of Illinois, Urbana–Champaign. The opinions expressed are those of the authors and do not represent views of the Institute or the US Department of Education.
