Development of Mathematical Practices Through Word Problem–Solving Instruction for Students With Autism Spectrum Disorder

Abstract

The Common Core State Standards for Mathematics highlight the importance of not only content standards for mathematics but also mathematical practices such as communication, representation, and reasoning, skills that are often difficult for students with autism spectrum disorder (ASD). Through a single-case multiple-probe-across-participants design, this study found modified schema-based instruction (MSBI) to be an effective strategy to increase the use of mathematical practices for middle school students with ASD when solving multiplicative word problems. Four students eligible for special education services under the area of autism enrolled in sixth-grade general education mathematics classes increased their use of mathematical practices for two problem types (multiplicative comparison and proportion) and maintained the use of mathematical practices 4 to 8 weeks after intervention. Additionally, all participants generalized their use of mathematical practices to novel multiplicative comparison problems containing extraneous information, and three of the participants generalized mathematical practice skills to proportion problems containing extraneous information. Implications for practice are discussed.

The National Council on Disability (2017) suggests the preparation individuals with disabilities receive throughout K–12 education does not provide them with the skills necessary to obtain gainful employment. This is particularly alarming considering mathematical knowledge is said to be “a powerful vehicle for social access and social mobility. . . . Lack of access to mathematics is a barrier . . . that leaves people socially and economically disenfranchised” (Schoenfeld, 2004, p. 255). Engaging students with disabilities in mathematical content and practice standards is an important component to helping them develop skills for future participation in higher education and work environments. The ability to think mathematically facilitates student engagement in our technologically advancing society and prepares them to participate in occupations requiring critical analysis. Computers and calculators are replacing the need for individuals to quickly calculate advanced mathematical algorithms. Instead, we need people to think and communicate about mathematical ideas and apply them to new and unique situations.

Instead, we need people to think and communicate about mathematical ideas and apply them to new and unique situations

Mathematics for Students With Autism Spectrum Disorder (ASD)

There has been some debate about the mathematical profile of individuals with ASD (Oswald et al., 2016). Individuals with ASD share similar characteristics (social communication difficulties and repetitive or restrictive behaviors and interests), but the severity of their symptoms and cognitive capabilities are highly variable. Although it has been previously assumed most individuals with ASD are strong mathematical thinkers, research suggests the mathematical profile of students with ASD is uneven (Jones et al., 2009), and compared with the general population, individuals with ASD are up to 5 times more likely to exhibit a mathematics disability than typically developing peers (Oswald et al., 2016).

Students with ASD may experience barriers to developing mathematical proficiency as a result of their unique cognitive profile. Compared with typically developing peers, students with ASD often have deficits in (a) language, (b) mathematical knowledge, (c) executive functioning, and (d) central coherence. Language deficits may include sentence-level comprehension difficulties or vocabulary deficits. The application of mathematical knowledge in daily life experiences is known as everyday mathematical knowledge (Bae et al., 2015). Mathematical word problems require students to use everyday mathematical knowledge to both conceptually understand the context of the word problem and use processes to solve it. Students with ASD often experience difficulty with executive functioning in areas related to planning, working memory, and monitoring, which are all necessary to engage in processes used to solve mathematical word problems. Finally, individuals with ASD inherently focus on specific details rather than the “big picture,” which can impede their success when a holistic understanding of the problem is required.

Considering the variability of mathematical performance among students with ASD, researchers have investigated the effectiveness of several interventions to improve mathematical outcomes. In a review of evidence-based interventions designed to teach mathematics to students with ASD, Hart Barnett and Cleary (2015) found interventions utilizing cognitive strategies as well as visual representations were successful in improving mathematical outcomes for students with ASD. The majority of the articles reviewed focused on procedural skills (e.g., computation problems, count-on strategy) and additive relationships (addition and subtraction). In a 2016 review, King et al. likewise found interventions focused on teaching computational or functional mathematical skills. To prepare students with ASD to participate in general education curriculum standards, authors of these reviews called for an intentional focus on higher-level mathematics.

Mathematical Problem Solving

The body of research informing mathematics instruction for students with ASD is rapidly expanding but is still limited in comparison to what we know about effective instruction for students with or at risk for mathematics learning disabilities. Gevarter et al. (2016) suggest students with ASD who do not have a comorbid intellectual disability are likely to benefit from strategies that have been successful for teaching students with learning disabilities, specifically, visual representations, think-alouds, and heuristics. Schema-based instruction (SBI) is one evidence-based practice that incorporates all three of these strategies (e.g., Jitendra et al., 2002, 2017; Peltier & Vannest, 2017.)

In SBI, students learn to use a heuristic to recognize and model the problem structure via a visual representation (schema) that depicts the mathematical relationship in the problem. SBI is an evidence-based practice to teach mathematical word problem solving for students with mathematics disabilities (Peltier & Vannest, 2017), with the research base including both additive and multiplicative relationships for students in elementary and middle grades. Jitendra et al. (2002) found providing strategy instruction relevant to specific types of mathematical word problems helped students with learning disabilities develop a conceptual understanding of the mathematical relationships. In this study, Jitendra et al. (2002) found evidence that some students were able to generalize problem-solving strategies to untaught problem types and multistep problems, supporting the likelihood SBI helped participants formulate conceptual understanding of the multiplicative relationships. In a 6-week SBI unit to teach proportional problem solving to students with mathematics difficulties, Jitendra et al. (2017) used a retrospective randomized cluster design to investigate the effectiveness of SBI on mathematical problem solving. The SBI treatment was effective in improving the immediate and long-term proportional problem solving for students with identified mathematical problem-solving difficulties.

Traditional SBI can support students who have difficulties with executive functioning (through the heuristic) and mathematical knowledge (through teacher think-alouds). Given the additional difficulties students with ASD have with language and central coherence, modified SBI (MSBI) incorporates the core components of SBI (problem-type instruction, visual representations, and teacher modeling with guided practice; Jitendra et al., 2002) while incorporating additional evidence-based practices aimed to support executive functioning and language comprehension (Spooner et al., 2017). Traditional SBI asks students to draw their own diagrams, whereas MSBI provides the option of using premade diagrams to reduce fine-motor requirements. Whereas SBI utilizes mnemonic strategies, such as RUNS (Read the problem, Use a diagram, Number sentence, State the answer), to help students during the problem-solving process, MSBI removes the potential language barrier by presenting steps of a heuristic as a visual. Additionally, MSBI provides immediate corrective feedback at the step level to prevent students from practicing incorrect strategies.

Previous research has consistently demonstrated the effectiveness of MSBI to improve mathematical problem-solving skills for students with ASD and comorbid intellectual disability (ID; e.g., Root et al., 2017, 2018; Root & Browder, 2019). Yet researchers have called for an increased focus on mathematics interventions for students with less extensive support needs (Gevarter et al., 2016; King et al., 2016). Few studies to date have investigated the effects of MSBI for students with ASD who have an average to above-average IQ. Preliminary research with this population has focused on examining the effects of MSBI on mathematical practices (e.g., flexibility and communication) of two students with ASD without ID. Cox and Root (2020) taught two middle school students to show their work, solve proportional problems multiple ways, and explain their mathematical reasoning when solving proportion problems containing extraneous information. The ABAB design found performance increased when participants had access to the visual heuristic outlining the problem-solving process, but mathematical communication and flexibility decreased when visual supports were removed. Although these findings suggest MSBI with visual supports may be effective for improving mathematical communication and flexibility in proportional problems for some students with ASD, design limitations did not allow for interpretation of which components of MSBI (e.g., visuals vs. explicit instruction) were more effective.

To assess the role of the visual supports, Cox and Root (in press) investigated the effect of MSBI on the mathematical practices of one 13-year-old student with ASD without ID across three multiplicative problem types (i.e., equal group, proportion, and percentage of change). Visual supports were provided in both baseline and intervention. Results of the multiple-probe-across-behaviors (problem types) design found a functional relation between MSBI and mathematical practices and confirmed visuals alone were insufficient. However, the participant demonstrated overgeneralization in that he solved each word problem using the procedures of the most recently taught problem type. After discrimination training, the participant reached ceiling performance for all three problem types. Due to the carryover effects of the MSBI training across behaviors and evaluation for just one individual, additional research is warranted.

Purpose

The Common Core State Standards for Mathematics (CCSSM; National Governors Association for Best Practices & Council of Chief State School Officers, 2010) set forth a rigorous outline of both what students should know (i.e., content) and how students should participate in mathematical content (i.e., math practices). Students with ASD are likely to experience difficulties engaging in mathematical practices as a result of their deficits in executive functioning and social communication. Researchers have identified instructional strategies to improve mathematical problem solving for students with ASD with comorbid intellectual disability (e.g., Root et al., 2018; Root & Browder, 2019), but further research is needed to investigate the effects of instructional strategies on engagement in mathematical practices for students with ASD, as those practices have been identified by the CCSSM as important for the development of mathematical proficiency.

Therefore, the purpose of this study was to investigate the effects of MSBI on the use of mathematical practices for four students with ASD, extending previous research by providing evidence of generalizability of prior findings across different participants, settings, and problem types. Additionally, the measure of problem-solving ability was explicitly aligned to the mathematical practices outlined in the CCSSM (National Governors Association for Best Practices & Council of Chief State School Officers, 2010), and generalization to untaught problem types and generalization to similar problems containing extraneous information were evaluated. Finally, measures of social validity provide information about the feasibility, perceived usefulness, and maintenance of learned behaviors. The following research questions were addressed: (a) Does MSBI increase the use of mathematical practices (measured by a researcher-created rubric) demonstrated by four middle school students with ASD without ID? (b) Do students with ASD without ID generalize their use of mathematical practices when solving similar word problems containing extraneous information? (c) Do students with ASD without ID discriminate between problem types following MSBI?

Method

Participants

Participants were recruited following procedures approved by the university institutional review board. Inclusion criteria were as follows: (a) enrolled in fifth-through-eighth-grade general education mathematics class, (b) eligible for special education under the Individuals With Disabilities Education Improvement Act category of autism, (c) did not meet eligibility criteria for or have a diagnosis of ID, (d) able to communicate in multiword sentences in written or verbal form, (e) able to write numbers in a box 2 cm by 2 cm, and (f) able to accurately multiply two-digit numbers with or without a calculator. Criteria (d), (e), and (f) were assessed using a researcher-created screening activity to measure verbal and written communication, fine-motor, and multiplication skills.

Researchers recruited students by sending a flyer to all middle-grade principals across the district (approximately 15), who shared with teachers. Teachers were asked to recommended students for this study based on difficulties with mathematical practices (e.g., explaining mathematical reasoning, selecting appropriate strategies to solve problems, and making sense of word problems). A total of seven teachers nominated 12 students based on their inadequate performance during classroom instruction. Researchers screened all students (four) who returned parent consent and completed student assessment. Four male students who were not showing their work or communicating their mathematical reasoning participated in the study. Table 1 provides demographic information. Each had a medical or educational diagnosis of ASD and was enrolled in sixth-grade general education mathematics courses. Scores from state standardized achievement tests were not available. Therefore, the researcher administered the full Test of Mathematical Achievement—Third Edition (TOMA-3; Brown et al., 2012) prior to intervention to provide an overall assessment of the participants’ mathematical achievement. Teachers reported all participants had a strength in following step-by-step procedures and learning routines. MSBI leverages this by providing students with a routine for solving mathematical word problems, supporting language, communication, and flexibility.

Table 1.

Participant Characteristics.

Characteristic	Bob	Al	Kahoot	Jacob
Age	10	11	12	11
Grade	5th	6th	6th	6th
Diagnosis	ASD	ASD	ASD	ASD
TOMA Mathematical Symbols and Concepts	63% (average)	50% (average)	50% (average)	2% (poor)
TOMA Computation	91% (above average)	63% (average)	16% (below average)	25% (average)
TOMA Mathematics in Everyday Life	91% (above average)	50% (average)	75% (average)	<1% (very poor)
TOMA Word Problems	98% (superior)	75% (average)	84% (above average)	9% (below average)
TOMA mathematical ability index	94% (superior)	55% (average)	55% (average)	3% (poor)
Mathematics instructional setting	6th-grade general education	General education	General education	General education
Ethnicity	Asian and White	White	White	White

Note. ASD = autism spectrum disorder; TOMA = Test of Mathematical Achievement.

Setting

Instruction was delivered in a one-on-one format by the first author. The interventionist was a certified former middle school mathematics teacher and a doctoral candidate in special education at the time of intervention. The study took place in public schools in a university town in the southeastern United States. All sessions were conducted in a quiet space located in the participants’ schools allocated by school personnel. Jacob, Al, and Kahoot met with the interventionist in the teacher’s lounge or the kitchen area of their school. Bob’s sessions were held at a private table in the common area of the media center at his school.

Materials

In baseline and intervention sessions, participants were provided with a laminated heuristic, one word problem at a time (attached to heuristic with sticky tack), and three schemas (see Figure 1), all attached to a legal-size clipboard. The heuristic provided a systematic five-step plan for students to follow based on Jitendra et al.’s (2002) recommendations, which include (a) underline the components of the problem type, (b) identify the type of problem, (c) diagram the mathematical relationship, (d) write an equation, (e) solve and answer the question, and (f) explain mathematical reasoning. Two types of multiplicative problems were targeted: multiplicative comparison (MC) and proportional (PROP), as shown in Table 2. All word problems contained whole numbers and were evaluated by an expert in mathematics education for content validity. In each baseline and intervention session, participants had access to three schemas (i.e., MC, PROP, and distractor). Participants used dry-erase markers to write on the laminated sheets. To reduce cognitive load and procedural computation errors, students were provided a basic TI-10 calculator in all sessions across all phases. During screening sessions, all students demonstrated an ability to use the calculator to multiply and divide three-digit numbers.

Figure 1.

Heuristic and schemas.

Table 2.

Example Word Problems.

Type	Multiplicative comparison	Proportion
Word problems	Kaylee has 17 dolls. Vibha has 5 times as many dolls as Kaylee. How many dolls does Vibha have?	Daisy was making a puzzle with her father. If the duo adds 30 pieces every 2 nights, how many nights will it take to build the 150 piece puzzle?
	Logan has 100 trains. Hunter has 2 times as many trains as Logan. How many trains does Hunter have?	Sallie spends 12 hours in 1 week practicing her clarinet. If she practices her clarinet at the same rate every week, how many hours would she practice in 5 weeks?
	Brad slept 5 hours last night. Sarah slept 2 times as long as Brad. How many hours did Sarah sleep last night?	It takes Lucy 25 minutes to knit 5 rows of her blanket. At that rate, how many minutes will it take Lucy to knit 45 rows?
Generalization word problems	Ted and Joseph are 13 years old. Joseph has 15 DVDs. Ted has 4 times more DVDs than Joseph. How many DVDs does Ted own?	Thor leaves through a portal at 10:30 a.m. and travels 300 miles in 10 seconds. If it travels at the same speed, how many miles will it travel in 30 seconds?
	There are 30 children in Ms. Robinson’s class and 32 children in Mrs. Koerner’s class. If there are 11 times more children in the school than in Ms. Robinson’s class, how many children are there in the school?	Carlos is driving to visit some relatives who live 320 miles away. He drives exactly 40 miles per hour the whole way. If Carlos leaves at 9:00 a.m., how many hours will it take Carlos to reach his relatives?
	Will owns 10 Nike shirts and 20 Under Amour shirts. If Will owns 3 times more Nike shorts than Nike shirts, how many Nike shorts does Will own?	Over 3 days, Wonder Woman flew 99 kilometers in her invisible jet. During her trip, she used 9 liters of fuel. How many kilometers can Wonder Woman fly with 12 liters of fuel?

Dependent Variables

The primary dependent variable was mathematical practices, as measured by a researcher-created rubric that aligned with the heuristic presented to the participants (see Table 3). The steps were aligned to mathematical practices identified in the CCSSM (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010). Participants earned 1 point for each independent correct response for an opportunity to earn up to 7 points for each problem (see Table 3). Table 2 provides examples of word problems used in baseline, intervention, and generalization probes for both multiplicative problem types (MC and PROP). Both problem types were measured using the same criteria.

Table 3.

Math Practices Aligned With Researcher-Created Rubric.

Math practice	Step of the heuristic	Measured behavior
Make sense of problems	1. Identify the question	1. Indicates what the question is (can be verbal, written, or underlined)
Look for and express regularity in repeated reasoning	2. Identify the problem type	2. Writes or states the type of problem (MC or PROP)
Use appropriate tools strategically	3. Represent the mathematical relationship	3a. Diagrams the mathematical relationship with correct schema and values
Model with mathematics	3. Represent the mathematical relationship	3b. Writes the equation or number sentence accurately
Attend to precision	4. Calculate	4a. Correctly calculates for given word problem
Make sense of problems and preserver in solving them	4. Calculate	4b. Correctly answers the question (including label)
Reason abstractly and quantitatively	5. Explain mathematical reasoning	5. Explains reasoning (verbal explanation, written explanation, or number sentence)

Note. MC = multiplicative comparison; PROP = proportional.

During baseline and probe sessions, participants had the opportunity to solve four problems (two of each problem type), resulting in an opportunity to earn 14 points for each problem type in each session. During intervention sessions, participants were given only questions for the targeted problem type, with an opportunity to earn 14 total points in each session. Participants entered a third intervention phase (discrimination of problem types [DISC]) if their data demonstrated overgeneralization. The second and third dependent variables, generalization of mathematical practices and discrimination, were measured using the same 7-point rubric.

Procedures

In each session, participants were reminded the importance of showing their work and explaining their reasoning so their teachers knew what they were thinking. They were then presented with a clipboard containing the heuristic, a word problem, and three schemas (see Materials). Participants were given one problem at a time.

Baseline and probe sessions

Participants solved four problems (two of each type) during each baseline and probe session. On-task behaviors were reinforced using verbal praise, but no feedback regarding correct and incorrect responses was provided.

Intervention

The interventionist used scripted lessons to deliver instruction that included explicit modeling, guided practice, and corrective feedback during all intervention lessons, which lasted between 14 and 30 min. Following suggestions from prior MSBI research (e.g., Browder et al., 2018; Root et al., 2018), each problem type was taught individually to mastery, with a discrimination phase provided for any participant whose data showed overgeneralization (i.e., using the most recently taught procedure for all problems).

Intervention for each problem type began with a 20-min modeled lesson to help students identify essential components of the problem and map them onto the appropriate schema (e.g., Jitendra et al., 2002). No data were collected during the model lesson, as the participants did not have the opportunity to independently respond. After the problem identification model lesson, intervention data collection began. Participants were given the opportunity to solve the targeted problem type independently before the interventionist used a system of least prompts. The hierarchy included gestural prompt (i.e., pointing to the step the participant should complete), specific verbal prompt (i.e., verbally stating what to do for that step), and a model-retest (i.e., modeling how to complete step, followed by student repeating the behavior). If the participant answered incorrectly at any time, the interventionist immediately provided error correction with a model-retest. Only independent correct responses were recorded and graphed.

Discrimination

Discrimination training sessions were held for participants if data in the second probe condition indicated they were not distinguishing between problem types. The purpose of this training was to directly teach the participant to identify the problem type based on the structure of the word problem. The interventionist used a T-Chart with examples of the two problem types and used think-alouds to model how to determine problem type. The participant then practiced categorizing previously solved problems into the two problem-type columns.

Generalization and maintenance

Generalization was measured once during each probe phase. During these sessions, participants solved four word problems (two of each type) containing extraneous information (i.e., irrelevant quantitative information). See Table 2 for examples. Maintenance probes followed baseline procedures and were administered 3 to 8 weeks following the final probe phase for each participant. A minimum of two maintenance probes were conducted within 1 week of each other for each participant.

Research Design

A multiple-probe-across-participants design (Ledford & Gast, 2018) was used to measure the effects of MSBI on the mathematical practices of four students with ASD. Data were collected across four experimental conditions: (a) baseline, (b) intervention, (c) probe, and (d) maintenance. Intervention consisted of a minimum of two phases, MC and PROP, with an optional third phase of DISC that participants entered if primary dependent variable (mathematical practices) indicated it was needed. A three-session probe was conducted between each intervention phase. All participants began baseline at the same time, and data were collected concurrently for a minimum of five sessions and until the first participant demonstrated a stable trend. Once the first participant met mastery criteria in the first problem type (MC), all participants were probed for a minimum of one session, with the participant entering intervention completing three probes. Participants continued in intervention phases until they met mastery criteria (i.e., 12 out of 14 points for two sessions). Three probes were administered between each intervention phase to measure participants’ mathematical practices without feedback. Participants continued to enter intervention in a staggered pattern after the previous participant demonstrated an increase in mathematical practices when solving MC problems.

Data analysis

Effectiveness of the intervention was evaluated using both visual and statistical analysis (i.e., between-case standardized mean difference [BC SMD]). Within- and between-condition visual analysis was conducted to determine the presence or absence of a functional relation by following the four steps outlined by What Works Clearinghouse (Kratochwill et al., 2010): (1) document a predicable baseline pattern of data, (2) examine data within each phase to assess within-phase patterns (i.e., level, trend, variability), (3) compare data from each phase with data from adjacent phase to assess for presence of an effect (i.e., overlap, immediacy of effect, and consistency of patterns in similar phases), and (4) integrate all information to determine if there are at least three demonstrations of an effect at three different points in time.

Although visual analysis served as the primary method for analyzing data, the online BC SMD application (Gierut et al., 2015) was used to calculate an estimated effect size. BC SMD is a statistical analysis that compares both between- and within-group means to calculate an effect size measure that is comparable to a group-design Cohen’s d. BC SMD was calculated by comparing participant performance in adjacent comparable conditions. Baseline scores were compared with all data collected after implementation of the intervention. According to Gierut et al. (2015), BC SMD effects can be interpreted as small (1.4), medium (3.6), or large (10.1).

Reliability and Fidelity

The first author served as the primary interventionist. Two undergraduate research assistants were trained to code for reliability and fidelity via information sessions, examples and nonexamples, videos, and practice sessions. A minimum of 30% of all sessions across all participants and conditions were assessed for both interrater reliability (IRR) and procedural fidelity (PF).

Point-by-point agreement (Ledford & Gast, 2018) was used to calculate IRR. IRR was monitored by an independent observer on a weekly basis, and disagreement discussions were held anytime IRR was under 80%. Agreement was calculated for both problem types for all participants. PF in baseline and probes was measured with a checklist. PF for intervention sessions measured appropriate implementation of the system of least prompts for each step. PF was calculated by adding the number of steps completed correctly and dividing by the total number of steps.

Social Validity

Following suggestions from Ledford and Gast (2018), social validity data were collected from a variety of sources. Participants completed the Attitude Toward Math subsection of the TOMA-3 pre- and postintervention and answered open-ended questions about their perception of the intervention after the final probe. Participants’ parents answered questions about the importance of mathematical practices and their observations of their child’s engagement with mathematics after the intervention. Their general education mathematics teachers answered questions about the importance of mathematical practices.

Results

Mathematical Practices

Visual analysis of data displayed in Figure 2 supports a functional relation between MSBI and the number of mathematical practices used during both MC and PROP problems, as there were four demonstrations of effect at four different points in time. During baseline probes, three of the four participants (Bob, Al, and Jacob) displayed a variable use of mathematical practices for both problem types (MC and PROP; see Figure 2), and Kahoot’s data were floor level (0) during baseline (see Table 4 for step-by-step averages). After implementation of the intervention, an immediate effect was observed in a clear level change for all four participants. During Probe 1, all four participants maintained their use of mathematical practices during taught problem types (MC) and increased their use of mathematical practices for untaught problem types (PROP) to varying degrees. Additionally, all four participants successfully used mathematical practices when solving generalization MC problems containing extraneous information. After PROP intervention, all four participants increased their use of mathematical practices from both baseline performance as well as Probe 1. In Probe 2, three of the four participants (Bob, Jacob, and Kahoot) were able to discriminate between the two problem types and used mathematical practices successfully for both problem types, whereas Al’s variable data trend suggested less stable use of mathematical practices for both problem types. Discrimination training was administered with Al for two sessions, after which time he met mastery criteria in Probe 3. Observed variance within and between participants in the maintenance phase suggests inconstant use of mathematical practices 4 to 8 weeks after intervention. Kahoot was the only participant to regularly use the calculator. Jacob used the calculator for proportion problems to check his work periodically. We will discuss specific notable problem-solving behavior patterns for participants next.

Figure 2.

Graph of mathematical practices.

Table 4.

Average Correct Responses by Step (in percentages).

	Step	Behavior	Baseline		IV MC	Probe 1		IV PROP	Probe 2		IV DISC		Probe 3		Maintenance
	Step	Behavior	MC	PROP	MC	MC	PROP	PROP	MC	PROP	MC	PROP	MC	PROP	MC	PROP
Al	5	Reasoning	0	0	100	67	67	100	50	50	100	100	83	83	100	100
	4b	Answer	0	0	80	17	17	50	33	17	75	50	50	67	67	33
	4a	Calculation	69	50	80	83	100	83	83	67	100	75	83	83	100	100
	3b	Equation	0	0	100	67	0	100	50	67	100	100	83	100	100	100
	3a	Diagram	0	0	80	83	0	100	50	50	75	100	100	100	100	100
	2	Type	0	0	80	100	0	100	50	67	75	100	100	100	100	83
	1	Components	0	0	80	100	100	83	100	100	100	100	100	100	100	100
Bob	5	Reasoning	0	10	88	67	67	83	83	100	—	—	—	—	100	100
	4b	Answer	20	50	50	33	83	67	100	100	—	—	—	—	67	67
	4a	Calculation	60	90	100	83	83	100	100	100	—	—	—	—	100	100
	3b	Equation	30	10	100	100	100	100	100	100	—	—	—	—	100	83
	3a	Diagram	0	0	75	100	100	100	100	100	—	—	—	—	100	100
	2	Type	0	0	100	83	0	100	100	83	—	—	—	—	100	100
	1	Components	0	0	100	100	100	100	100	100	—	—	—	—	100	100
Jacob	5	Reasoning	0	0	100	83	33	75	100	100	—	—	—	—	83	83
	4b	Answer	36	9	100	50	50	100	100	100	—	—	—	—	33	67
	4a	Calculation	82	50	100	83	50	100	100	100	—	—	—	—	83	83
	3b	Equation	27	5	100	83	17	100	100	100	—	—	—	—	67	100
	3a	Diagram	0	18	100	100	33	100	100	100	—	—	—	—	33	83
	2	Type	0	0	75	100	0	75	100	100	—	—	—	—	100	100
	1	Components	0	0	75	100	100	100	100	100	—	—	—	—	67	0
Kahoot	5	Reasoning	0	0	83	33	0	25	100	83	—	—	—	—	83	17
	4b	Answer	0	0	50	17	0	83	0	83	—	—	—	—	33	33
	4a	Calculation	0	0	100	83	0	25	100	83	—	—	—	—	100	67
	3b	Equation	0	0	100	83	0	92	100	83	—	—	—	—	83	67
	3a	Diagram	0	0	83	83	0	75	100	83	—	—	—	—	83	100
	2	Type	0	0	67	100	0	100	100	100	—	—	—	—	100	67
	1	Components	0	0	75	100	100	100	83	100	—	—	—	—	100	100

Note. IV = intervention phase; MC = multiplicative comparison; PROP = proportional; DISC = discrimination.

Bob

The mathematical practice point Bob most frequently missed during MC intervention and Probe 1 phase was labeling his numerical answer (see Table 4). When prompted to label, Bob would use an incorrect label at first, needing error correction (model-retest.) Without the explicit proportion instruction, Bob was unable to identify the problem type (he named them multistep problem) and was variable in his application of the other mathematical practices when solving PROP problems. Bob’s use of mathematical practices for PROP problems increased and stabilized at 12 during the PROP intervention phase.

Al

When the system of least prompts was withdrawn after the MC intervention phase, and both problem types were presented in Probe 1, Al tried to solve all four problems as though they were MC word problems. Sessions more than doubled in length, from an average of 8 min to an average of 20 min. When systematic prompting was withdrawn again during Probe 2, Al initially solved all four problems as though they were all PROP (the most recently taught type). Al inconsistently identified the correct problem type and therefore received discrimination training. Notably, his reasoning (Step 5) increased from 50% for both problem types in Probe 2 to 100% for both problem types in the discrimination phase. As shown in Table 4, the mathematical practice point Al most frequently missed was labeling his answer.

Jacob

During baseline, Jacob was able to generate diagrams, write number sentences, and correctly calculate for some of the MC and PROP word problems (see Figure 2 and Table 4). After the MSBI for MC word problems, Jacob’s use of mathematical practices immediately jumped to a score of 13 for two consecutive sessions. During Probe 1, Jacob’s use of mathematical practices for PROP problems increased slightly, demonstrating generalization as he had received instruction only in MC problems. Jacob was able to draw his own diagram and write appropriate number sentences for some of the PROP problems after the MC intervention session. Jacob successfully discriminated between MC and PROP problems during the Probe 2, reaching ceiling performance for all three sessions.

Kahoot

During baseline probes, Kahoot consistently solved all word problems by multiplying the first two quantities in the word problem, never earning mathematical practice points. After MSBI for MC word problems, Kahoot’s use of mathematical practices immediately jumped from zero to eight, and he reached mastery criteria after three intervention sessions. Kahoot required the greatest number of sessions to reach mastery criteria for any intervention phase, with six sessions needed for the PROP intervention. In Probe 2 sessions, Kahoot maintained or surpassed his use of mathematical practices from previous phases. Without prompting or feedback, Kahoot was able to independently use an average of 11.67 (range = 11–12) mathematical practice steps when solving MC problems and an average of 13 (range = 11–14) mathematical practice steps when solving PROP problems. The mathematical practice Kahoot most frequently skipped during Probe 2 was labeling his answer. In the final maintenance session, Kahoot expressed frustration that did not learn to use the distractor diagram and that he was going to use it (even though he knew it was incorrect) to solve the PROP problems.

Statistical Analysis of Change

The functional relation established through visual analysis is supported by medium effect sizes (Gierut et al., 2015) for MC (BC SMD = 4.78, confidence interval [CI] = [3.10, 6.46]) and PROP (BC SMD = 5.28, CI = [2.11, 8.44]). These estimated effect sizes suggest the intervention produced a meaningful change when comparing the mean difference of performance across conditions.

Fidelity and Reliability

PF was calculated as 99.25% for Bob’s baseline (4/11 sessions), 97% for intervention (3/7), 100% for generalization (3/4), and 100% for maintenance (1/3). PF was calculated as 100% for Al’s baseline (8/17 sessions), 98.25% for intervention (3/10), 100% for generalization (2/4), and 100% for maintenance (1/3). PF was calculated as 100% for Jacob’s baseline (5/16 = 31%), intervention (2/4 = 50%), generalization (2/3 = 67%), and maintenance (1/3 = 33%) sessions. PF was calculated as 99.5% for Kahoot’s baseline (6/17 sessions), 94.75% for intervention (4/10), and 100% for generalization (2/4) and maintenance (1/3) sessions.

Agreement was slightly higher during baseline conditions for most participants (Bob, µ = 97; Al, µ = 99%; Jacob, µ = 90%; Kahoot, µ = 100) than for intervention conditions (Bob, µ = 91%; Al, µ = 93%; Jacob, µ = 97%; Kahoot, µ = 93). Agreement was also generally higher for MC problems (µ = 98%; range = 86%–100%) than proportion problems (µ = 95%; range = 79%–100%). Agreement for only one proportion session fell below 80%, which occurred when the intervention agent miscalculated the solution and did not appropriately prompt the student to fix his work. Subsequently her scores did not reflect his error.

Social Validity

All four participants indicated they enjoyed participating in the study, would participate in another similar study in the future, and would recommend the intervention to peers. During the interview, only three of the four participants (Bob, Al, and Kahoot) felt their ability to solve mathematical problems improved after the intervention. Additionally, Bob, Al, and Kahoot indicated they would use the problem-solving strategy in the future. When asked what components of the intervention were most helpful, Bob felt the step-by-step process helped him to “slow down and think through my answers,” and Al stated that the diagram helped him the most. Both Jacob and Al indicated that explaining their reasoning was the most difficult part of solving the word problems, and they felt that it was useful only if the problem was difficult.

On a written survey that was sent home at the conclusion of the study, parents of all four participants indicated that they would agree to have their child participate in a similar study in the future. When asked about their child’s strengths in mathematics, the parents shared skills such as finding patterns (Kahoot), an understanding of numbers (Jacob), memorization (Al), and creativity (Bob) as current strengths. When asked about observed weaknesses, parents mentioned the ability to show work and think abstractly several times. Three of the four parents indicated that they saw some improvement in their child when working on word problems (i.e., improved grades, showing work, slowing down when solving problems) after the intervention.

All four of the general education teachers were familiar with the term “mathematical practices” from the CCSSM (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010). Three of the four rated the ability to explain reasoning when solving mathematical word problems as “very important,” and the fourth teacher rated it as “important.” All four of the teachers identified the following skills as “very important” when solving word problems: (a) identifying the type of mathematical relationship depicted, (b) selecting an appropriate problem-solving strategy, (c) correctly calculating a specific answer, and (d) labeling the answer with the appropriate noun.

Discussion

This study sought to extend previous research on MSBI for students with ASD. Extant literature supports MSBI as an effective strategy to increase problem-solving skills of students with ASD and ID (e.g., Root et al., 2018; Root & Browder, 2019) and to improve flexibility and communication for students with ASD (Cox & Root, 2020) but has not specifically measured student engagement in mathematical practices as outlined by the CCSSM. In this study, all four participants increased their use of mathematical practices after MSBI for both problem types. Participants also maintained use of mathematical practices 3 to 8 weeks after intervention. Four demonstrations of an effect support a functional relation between MSBI and the use of mathematical practices, providing evidence MSBI is an effective strategy to increase the use of mathematical practices for some students with ASD.

Acquisition of Mathematical Practices

Although mathematical proficiency for all students is a primary goal of the new mathematical standards, it is unclear what specific role mathematical practices play for students with communication difficulties, such as students with ASD. Mathematics education experts (e.g., National Council of Teachers of Mathematics [NCTM], 2000) suggest engagement in mathematical practices helps students form connections between procedural skills and a conceptual understanding (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010). Empirical research has confirmed a correlation between procedural skills and conceptual understanding in typically developing students (Rittle-Johnson & Alibali, 1999). Such a correlation has not been explored for students with ASD. Reviews of the literature suggest the majority of intervention research for students with ASD has focused on teaching computational skills (Gevarter et al., 2016; King et al., 2016) without any mention of mathematical practices.

The results of this study suggest MSBI can help some middle school students with ASD increase their use of mathematical practices (i.e., procedures and process; National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010) when engaged in word problem solving. Table 4 demonstrates all four participants increased their average scores for each step when comparing their scores in baseline with their final probe. Although access to the heuristic alone was not sufficient for students to meaningfully engage in practices meaningfully, instruction on how to use the heuristic along with explicit instruction in underlying problem structures led to student engagement in reasoning and self-monitoring. The interventionist observed several participants reconsider the reasonableness of their answers and recalculate their answers after explaining their mathematical reasoning. During interviews, three of the students acknowledged using the heuristic (Jacob) or explaining their reasoning (Al and Bob) helped them to recognize their own errors. Problem-solving instruction for students with ASD should encourage participation in math practices to promote reasoning that leads to generalization and maintenance.

Problem solving instruction for students with ASD should encourage participation in math practices to promote reasoning that leads to generalization and maintenance

Generalization of Mathematical Practices

Students in middle-grade mathematics should be given opportunities to generalize their understanding to new situations (NCTM, 2000). The ability to generalize a learned behavior to a new context is particularly challenging for students with ASD (Sperry et al., 2010) but often neglected when assessing the effects of an intervention for students with ASD (Machalicek et al., 2008). Measuring generalization of an experimental effect beyond the specific instructional setting increases confidence that these effects would occur in more naturalized settings (Ledford & Gast, 2018). This study specifically measured students’ ability to generalize to problems containing extraneous information.

The ability to distinguish relevant versus irrelevant information is another crucial skill to promote generalization of problem-solving skills to real-life application of mathematical reasoning. Cox and Root (2020) found MSBI with visual supports was effective for teaching two students with ASD without ID to solve proportional problems containing extraneous information, but the students in that study were explicitly taught to identify relevant information from the problem. The current study evaluated students’ ability to solve multiplicative comparison and proportional word problems containing extraneous information without explicit instruction to categorize relevant versus irrelevant information. Results suggest MSBI may help students with ASD without ID map relevant information into the appropriate schema, aiding in the students’ ability to utilize key information while ignoring or excluding irrelevant information.

Maintenance of Mathematical Practices

Previous research investigating the effects of MSBI on problem-solving behaviors of students with ASD without ID did not collect maintenance data (Cox & Root, 2020, in press). Measuring student performance weeks after the final intervention provides information on the likelihood that the student will continue to use the measured behavior without additional instruction. This information is pertinent for teachers as they make instructional decisions of when to move to a new concept or which concepts are most likely to need refresher instruction. This study measured student use of mathematical practices 3 to 8 weeks after the final intervention session. Participants’ use of mathematical practices during maintenance sessions varied and suggest that some students with ASD without ID may benefit from booster sessions or prompts to maintain use of mathematical practices after the instructional period. Three of the four participants exhibited a decreasing trend during maintenance for PROP problems, suggesting proportional reasoning may be a particularly challenging content that is likely to need additional instruction, a notion consistent with extant literature (e.g., Vergnaud, 1994). This is particularly important because although proportional reasoning is among the more challenging concepts taught during middle-grades mathematics (Vergnaud, 1994), it is also a key component of middle-grade mathematics content (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010; NCTM, 2000). Therefore, it is important for teachers to continue to monitor student use of proportional reasoning.

Limitations and Future Research

Although this study provides evidence of the effectiveness of MSBI to increase the use of mathematical practices for some students with ASD, several limitations need to be considered when interpreting the results. First, there was variability in generalization, maintenance, and discrimination performance across students. The design of this study does not allow for interpretation of why particular students were able to generalize, discriminate, or maintain performance whereas others were not. Replications are needed to examine the level of supports students need and what factors may contribute to variability. Second, the intervention took place in a one-on-one setting with an intervention agent who was a trained doctoral student in special education. These conditions are not consistent with the instructional delivery common in traditional general education environments, and therefore it is unclear if the effects of MSBI would extend into a group environment or with more natural intervention agents (i.e., teachers).

Prior MSBI research found student performance was less stable when the word problems did not follow a consistent format (e.g., Cox & Root, 2020, in press). We chose to use a consistent format for all word problems using the same missing variable to make comparison between probes most similar. Although this decision increased confidence that variability in the data was not related to difficulty of the problem, it also decreased confidence that similar results would be found in a wider range of multiplicative word problems. For example, all MC word problems provided the multiplier and the product, and the original amount was the missing variable. It is unclear if the participants would have shown the same amount of growth if the word problems had provided different missing variables each time. The word problems also represented only two types of word problems, MC and PROP, which are only a portion of the types of word problems students will see in middle-grades mathematics courses. These findings extend previous research that found MSBI effective to teach additive (e.g., group, whole, change, compare) and multiplicative (proportion, percentage of change, equal group) to students with ASD. Although SBI has shown to be effective to improve both proximal and distal measures of math performance for students with high-incidence disabilities (Gevarter et al., 2016), future research is needed to evaluate the distal effects of MSBI on the overall mathematical performance of students with ASD.

Practical Implications

The body of research to support the effectiveness of MSBI to improve mathematical problem-solving skills, including mathematical practices, for students with ASD continues to grow. Teachers should consider using MSBI to help students with ASD solve mathematical word problems and explain their mathematical reasoning. Supports from MSBI, such as the heuristic or a more specific task analysis, should be selected based on specific student characteristics, and consideration of how and when to fade these supports will be necessary. A heuristic or task analysis can support the student in monitoring their own progress in that problem-solving process. A general heuristic can emphasize problem-solving steps, such as identifying the problem type, representing the relationship in a diagram or number sentence, and checking that the answer is reasonable. General heuristics are effective for students who thrive on structure or routine and are easier to fade than more specific task analyses. Using a visual support, such as a heuristic, to guide the student through the problem-solving process can support students with ASD who experience difficulties with executive functioning required during problem solving. The students in this study all eagerly checked off steps as they completed them in the heuristic. Teachers could create heuristics to emphasize the mathematical practices each student needs the most help with. Teachers should consider individual students’ needs to determine what level of supports are needed and when and how to fade those supports while maintaining student engagement in mathematical practices.

A heuristic or task analysis can support the student in monitoring their own progress in the problem solving process

Students with ASD can engage in mathematical practices with appropriate supports. Explicit instruction in the problem-solving process can help students with ASD learn mathematical problem solving is an iterative process that includes understanding the problem, devising a plan to solve the problem, using tools to solve the problem, and checking their work. Teachers can use a model-lead-test format to provide scaffolded supports with immediate feedback. Exposing students to multiple ways of representing the mathematical relationship can help students deepen their understanding of the concept. In this strategy, students listen to the problem-solving process from the perspective of the teacher, practice the problem-solving process with immediate feedback, and then independently complete the process to demonstrate mastery. The instructor can use information from the final phase to make instructional decisions. Communication deficits are a trademark for all individuals with an ASD diagnosis; therefore, instruction should focus on ways to explicitly teach communicative behaviors (e.g., self-talk, reasoning) that promote a deep understanding of academic concepts.

Exposing students to multiple ways of representing the mathematical relationships can help students deepen their understanding of the concept

Conclusion

The CCSSM (National Governors Association for Best Practices & Council of Chief State School Officers, 2010) set forth a rigorous outline of both what students should know (i.e., content) and how students should participate in mathematical content (i.e., math practices). Students with ASD are likely to have a difficult time engaging in mathematical practices as a result of their deficits in executive functioning and social communication. Researchers have identified instructional strategies to improve mathematical problem solving for students with ASD and ID (e.g., Root et al., 2018; Root & Browder, 2019), but further research is needed to investigate the effects of instructional strategies on how students with ASD without ID are engaging in mathematical practices identified in the CCSSM as important for the development of mathematical proficiency.

Footnotes

Correction (June 2025):

Article updated online; for further details please see correction notice

ORCID iDs

Sarah K. Cox

Jenny R. Root

References

Bae

Y. S.

Chiang

H. M.

Hickson

(2015). Mathematical word problem solving ability of children with autism spectrum disorder and their typically developing peers. Journal of Autism and Developmental Disorders, 45(7), 2200–2208. https://doi.org/10.1007/s10803-015-2387-8

Browder

D. M.

Spooner

Y-y

Saunders

A. F.

Root

Ley Davis

Brosh

(2018). Teaching students with moderate intellectual disability to solve word problems. The Journal of Special Education. Advance online publication. https://doi.org/10.1177/0022466917721236

Brown

V. L.

Cronin

M. E.

Bryant

D. P.

(2012). Test of mathematical abilities (3rd ed.). PRO-ED.

Cox

S. K.

Root

J. R.

(2020). Modified schema-based instruction to develop flexible mathematics problem solving strategies for students with autism spectrum disorder. Remedial and Special Education, 41(3). https://doi.org/10.1177/0741932518792660

Cox

S. K.

Root

J. R.

Goetz

Taylor

(2021). Modified schema-based instruction to encourage mathematical practice use for a student with autism spectrum disorder. Education and Training in Autism and Developmental Disabilities, 56(2), 190–204. https://www-proquest-com-s.web.bisu.edu.cn/docview/2599642704

Gevarter

Bryant

D. P.

Bryant

Watkins

Zamora

Sammarco

(2016). Mathematics interventions for individuals with autism spectrum disorder: A systematic review. Review Journal of Autism and Developmental Disorders, 3(2), 224–238. https://doi.org/10/1007/s40489-016-0078-9

Gierut

J. A.

Morrisette

M. L.

Dickinson

S. L.

(2015). Effect size for single-subject design in phonological treatment. Journal of Speech Language and Hearing Research, 58(5), 1464–1481. https://doi.org/10.1044/2015_JSLHR-S-14-0299

Hart Barnett

J. E.

Cleary

. (2015). Review of evidence-based mathematics interventions for students with autism spectrum disorders. Education and Training in Autism and Developmental Disabilities, 50(2), 172–185. https://www-jstor-org.web.bisu.edu.cn/stable/24827533

Jitendra

DiPipi

C. M.

Perron-Jones

(2002). An exploratory study of schema-based word-problem solving instruction for middle school students with learning disabilities: An emphasis on conceptual and procedural understanding. The Journal of Special Education, 36(1), 23–38. https://doi.org/10.1177/00224669020360010301

10.

Jitendra

A. K.

Harwell

M. R.

Dupuis

D. N.

Karl

S. R.

(2017). A randomized trial of the effects of schema-based instruction on proportional problem-solving for students with mathematics problem-solving difficulties. Journal of Learning Disabilities, 50(3), 322–336. https://doi.org/10.1177/0022219416629646

11.

Jones

C. R. G.

Happe

Golden

Marsden

A. J. S.

Tregray

Simonoff

Pickles

Baird

Charman

(2009). Reading and arithmetic in adolescents with autism spectrum disorders: Peaks and dips in attainment. Neuropsychology, 23, 718–28. https://doi.org/10.1037/a0016360

12.

King

S. A.

Lemons

C. J.

Davidson

K. A.

(2016). Math interventions for students with autism spectrum disorder: A best-evidence synthesis. Exceptional Children, 82(4), 443–462. https://doi.org/10.1177/0014402915625066

13.

Kratochwill

T. R.

Hitchock

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2010). Single-case designs technical documentation. http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.

14.

Ledford

J. R.

Gast

D. L.

(Eds.). (2018). Single case research methodology: Applications in special education and behavioral sciences. Routledge.

15.

Machalicek

O’Reilly

M. F.

Beretvas

Sigafoos

Lancioni

Sorrells

Lang

Rispoli

(2008). A review of school-based instructional interventions for students with autism spectrum disorders. Research in Autism Spectrum Disorders, 2(3), 395–416. https://doi.org/10.1016/j.rasd.2007.07.001

16.

National Council on Disability. (2017). National disability policy: a progress report - October 2017. https://ncd.gov/progressreport/2017/national-disability-policy-progress-report-october-2017

17.

National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics.

18.

National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for Mathematics: Standards for mathematical practice. http://www.corestandards.org/Math/Practice/

19.

Oswald

T. M.

Beck

J.S.

Iosif

A. M.

McCauley

J. B.

Gilhooly

L. J.

Matter

J. C.

Solomon

(2016). Clinical and cognitive characteristics associated with mathematics problem solving in adolescents with autism spectrum disorder. Autism Research, 9(4), 480–490. https://doi.org/10.1002/aur.1524

20.

Peltier

Vannest

K. J.

(2017) A meta-analysis of schema instruction on the problem-solving performance of elementary school students. Review of Educational Research, 87(5). https://doi.org/10.3102/0034654317720163

21.

Rittle-Johnson

Alibali

M. W.

(1999). Conceptual and procedural knowledge of mathematics: Does one lead to the other? Journal of Educational Psychology, 91(1), 175–189. https://doi.org/10.1037/0022-0663.91.1.175

22.

Root

J. R.

Browder

D. M.

(2019). Algebraic problem solving for middle school students with autism and intellectual disability. Exceptionality, 27(2), 118–132. https://doi.org/10.1080/09362835.2017.1394304

23.

Root

J. R.

Cox

S. K.

Hammons

Saunders

A. F.

Gilley

(2018). Contextualizing mathematics: Teaching problem solving to secondary students with intellectual and developmental disability. Intellectual and Developmental Disabilities, 56(6), 442–457. https://doi.org/10.1352/1934-9556-56.6.442

24.

Root

J. R.

Saunders

Spooner

Brosh

(2017). Teaching personal finance mathematical problem solving to individuals with moderate intellectual disability. Career Development and Transition for Exceptional Individuals, 40(1), 5–14. https://doi.org/10.1177/2165143416681288

25.

Schoenfeld

A. H.

(2004). The math wars. Educational Policy, 18(1), 253–286. https://doi.org/10.1177/0895904803260042

26.

Sperry

Neitzel

Engelhardt-Wells

(2010). Peer-mediated instruction and intervention strategies for students with autism spectrum disorders. Preventing School Failure: Alternative Education for Children and Youth, 54(4), 256–264. https://doi.org/10.1080/10459881003800529

27.

Spooner

Saunders

Root

Brosh

(2017). Promoting access to common core mathematics for students with severe disabilities through mathematical problem solving. Research & Practice for Persons with Severe Disability, 42(3), 1–16. https://doi.org/10.1177/1540796917697119

28.

Vergnaud

(1994). Multiplicative conceptual field: What and why. In Harel

Confey

(Eds.), The development of multiplicative reasoning (pp. 41–60). State University of New York Press.

29.

Xin

Y. P.

(2008). The effect of schema-based instruction in solving word problems: An emphasis on pre-algebraic conceptualization of multiplicative relations. Journal for Research in Mathematics Education, 39(5), 526–551. https://www-jstor-org.web.bisu.edu.cn/stable/40539313