The Effect of Model-Based Problem Solving on the Performance of Students Who are Struggling in Mathematics

Abstract

The U.S. Nation’s Report Card reveals that lower performing students exhibited greater achievement decline than their average/high performing peers based on 2022 long-term trend mathematics assessments for age 9 students. Technology, including computer-assisted instruction, plays an important role in today’s dynamic learning environments. Currently, there is a lack of computer-assisted intervention programs that systematically teach generalized word problem-solving skills that are driven by mathematical models. Model-based problem solving (MBPS) is one of the essential emphases in the Common Core mathematical practice standards. This study investigated the effects of a web-based computer tutor, MBPS, on enhancing word problem-solving performance of elementary students who are struggling in mathematics. The MBPS tutor incorporates best practices that are identified by the Institute of Educational Sciences’ (IES) latest practice guide, including providing systematic instruction, visual and verbal supports, and teaching of precise mathematical language. Findings indicate that the MBPS tutor boosted participants’ performance above and beyond the business-as-usual comparison group.

Keywords

algebra readiness conceptual understanding mathematics word problem solving model-based problem solving learning disabilities at-risk students elementary mathematics intervention programs computer-assisted instruction web-based programs

While mathematics skills are crucial for functioning in the 21st century, students with learning disabilities (LD) or at risk for LD in mathematics (LDM) lag well behind their peers from very early on in their educational trajectory (U.S. Department of Education, 2022). They often continue to fall further behind as they transition from elementary to secondary schools (Carcoba Falomir, 2019). Over the past decade, the mathematical performance of students with LDM has not improved and in fact declined since 2020 (U.S. Department of Education, 2022), despite laws that demand that all students be provided with equal opportunities to access learning resources and meet the same high standards (e.g., Common Core State Standards Initiative [CCSSI], 2012; Every Student Succeeds Acts, 2015; National Council of Teacher of Mathematics [NCTM], 2000).

Today’s curriculum standards emphasize conceptual understanding in problem solving, mathematical modeling, algebra readiness, and higher order thinking and reasoning (CCSSI, 2012). This poses significant challenges for students with LDM. To guide instructional practice for students with LDM, Institute of Educational Sciences’ (IES) latest Practice Guide suggests the use of systematic instruction through sequencing, using worked-out examples, providing visual and verbal supports, and teaching of precise mathematical language in elementary mathematics to students with LDM (Fuchs et al., 2021).

Operational Versus Relational Paradigm in Word Problem Solving

Students with LDM experience significant difficulties in solving word problems (Geary, 1994) as it involves various skills such as comprehend the problem, generate the model/equation for solution, and carry out the calculation to solve the problem. According to existing literature, there are two paradigms in word problem-solving instruction. The operational paradigm focuses on semantic analysis of the word problem involving transforming the words of the problem into an arithmetical operation (Polotskaia & Savard, 2018). Historically, it has been believed that selecting and applying an appropriate arithmetic operation is crucial to success in problem solving. For instance, the “keyword” strategy specifically focused on translating “cue” words into operation signs (Xin, 2016). The keyword strategy directs students’ attention toward isolated “key words” (e.g., “Together”) without first comprehending the entire word problem. The keyword strategy might be a “quick and dirty” way to “fix” word problem solving; however, it does not orient students’ attention to a problem’s underlying mathematical relations that is critical to solution planning and accurate problem solving (CCSSI, 2012). In contrast, applying the keyword strategy might contribute to students being prone to “reversal operation” errors when encountering “inconsistent language” problems (e.g., “Tara solved 21 problems. She solved 3 times as many problems as Pat. How many problems did Pat solve?”), where students might mistakenly multiply, per the key word “times,” when they need to divide for solution (Xin, 2007).

Most intervention strategies developed for helping students with LDM centered on translating a word problem “storyline” into numbers connected with an operation sign. For instance, stories such as “making purchases” or “spending money” would be associated with or translated into a “minus” sign in a mathematics sentence or equation. In short, the operational paradigm relies on learners’ development of understanding that is driven by arithmetical operations rather than mathematical relationships (Polotskaia & Savard, 2018). According to scholars in mathematics education (Davydov, 1982; Thompson, 1993), focusing on the operation (e.g., whether to “add” or “subtract”) relevant to calculation is likely to distract learners from mathematical relations depicted in the word problem. In fact, it is the mathematical relation, on which the mathematical model is conceptualized, that contributes to generalized problem solving (Jonassen, 2003).

In the relational paradigm, the idea of mathematical relationship constitutes the conceptual basis for problem solving. Learners need to read the entire story to understand the mathematical relations in the word problem. In contrast to the operation paradigm, the relational paradigm focuses on the learner representing and describing decontextualized mathematical relations before applying an operation to solve the problem. The relational paradigm is in line with current curriculum standards for mathematical practice. For instance, NCTM calls for developing essential understanding of addition and subtraction and emphasizes teaching mathematical relations through making connections between different problem variations or variants of number sentences (e.g., a + b = c; c– b = a or a = c – b; c – a = b or b = c – a), however, with the same mathematical relations (Caldwell et al., 2011).

Intervention Research in Mathematics Word Problem Solving for Struggling Students

Existing meta-analyses on word problem solving for students with learning difficulties (Myers et al., 2022) have shown that heuristic strategy instruction (e.g., Woodward et al., 2001), schema-based instruction (Xin et al., 2005), and model-based problem solving (MBPS, Witzel et al., 2022; Xin, 2012) were effective for enhancing word problem-solving performance of students with LDM.

Literature syntheses and meta-analyses have analyzed mathematics interventions involving students with LDM. For instance, Zhang and Xin (2012) conducted a meta-analysis of 39 studies that investigated the effect of word problem solving interventions for students with LDM. Findings indicated that problem structure representation resulted in the highest effect size (ES = 2.64) compared with cognitive strategy training (ES = 1.86). Hughes et al. (2014) conducted a meta-analysis of 12 mathematics intervention studies (1983–2013) involving algebra. The findings from this study indicate that model-based interventions had the strongest effectiveness (ES = 0.68) when compared with other strategies.

Most recently, Kim and Xin (2022) conducted a review of intervention studies (1981–2019) that applied computer-assisted instruction (CAI) to facilitate the learning of mathematical word problem solving of students with LDM. Findings from 13 studies indicate that CAI programs developed based on MBPS yielded the largest effect size (ES = 1.42) when compared with other strategies such as cognitive/metacognitive strategy (ES = .99) or direct instruction /guided practice (ES = 0.17). In summary, findings from this literature review support the use of MBPS, which emphasizes expressions of mathematical relations in algebraic equations.

Existing MBPS CAI has been focusing on multiplicative reasoning and problem solving of students with LDM. Field studies (e.g., Xin et al., 2017, 2020) have shown that students who learned from the MBPS CAI improved their performance not only on researcher-developed criterion tests but also on commercial published standardized tests (Xin et al., 2017). Given the effectiveness of MBPS in enhancing the performance of elementary students with LDM in multiplicative word problem solving, the purpose of this study was to examine a web-based MBPS computer tutor, developed through support by the National Science Foundation, that focuses on nurturing students’ additive reasoning and problem solving. Specific research questions were as follows: (RQ1) Did participants who received the MBPS intervention outperform the business-as-usual (BAU) group? Did the MBPS group maintain their performance after the termination of the intervention? (RQ2) Did participants in the MBPS group improve their performance on a transfer measure that was designed to assess students’ algebraic knowledge and skills? Did participants in the MBPS group improve their performance on solving problems taken from commercially published math textbooks? And (RQ3) Did participants in the MBPS group improve their performance on a distal measure, a standardized test?

Method

Participants and Setting

This study was conducted within the larger context of the National Science Foundation (NSF)-funded project¹ (Xin, 2015). Participants were 17 third graders with LDM from one elementary school in the mid-western United States. To determine sample size, a power analysis using a Cronbach’s alpha level of .05 and an effect size of 1.25 based on existing related research (Xin et al., 2011) was conducted, which indicated that a minimum of 8 or 9 participants in each group was sufficient to obtain a power of .87 or .91 for a 2 × 2 repeated measures analysis of variance (Friendly, 2000).

The participating school recommended nine third graders with LDM who were at the most risk for failure in mathematics, including three students with school-identified LD, for the MBPS CAI condition. The remaining eight third graders with LDM were placed in the business as usual (BAU) condition. The decision on assigning students into respective conditions was made by the schools in accordance with the needs of each participant for intensified support given parent consent (along with student assent) for participation in one of the intervention conditions. The participant sample in each condition was representative of the student population for differentiated instruction (What Works Clearinghouse, 2020). Tables 1 and 2 present demographic information of the two groups of students. All the instruction and testing were conducted in the school’s library during the afterschool program Monday through Thursday.

Table 1.

MBPS Group Student Demographics.

Name/variable	S1	S2	S3	S4	S5	S6	S7	S8	S9
Gender	Female	Female	Male	Male	Male	Female	Male	Male	Female
Ethnicity	Hispanic	Black	Hispanic	Hispanic	White	White	White	Black	Black
Age (year-month)	8-11	9-5	8-8	8-8	8-6	8-3	8-9	9-1	8-1
Socioeconomic status	Low	Low	Low	Low	Low	Low	Low	low	Low
Years in special ed.	0	0	1 (LD)	0	3(LD)	0	0	3(LD)	0
RtI support	Tier 2	Tier 2	Tier 3	Tier 2	Tier 3	Tier 2	Tier 2	Tier 3	Tier 2
% in general class	100%	100%	>80	100%	>80	100%	100%	>80	100%
OtisLennon/Full-scale	No test	76	77	61	90	79	72	82	No test
Verbal		74	79	73	92	81	73	77
Performance		79	77	50	89	79	75	87

Note. MBPS = model-based problem solving; LD = learning disabilities; RtI = response to intervention; OtisLennon = Otis-Lennon School Ability Test (Otis & Lennon, 2003).

Table 2

BAU Group Student Demographics.

Name/variable	S1	S2	S3	S4	S5	S6	S7	S8
Gender	Female	Male	Female	Female	Male	Female	Female	Male
Ethnicity	Black	Hispanic	Hispanic	White	Hispanic	Bi-racial	Hispanic	Black
Age(year-month)	8-5	8-5	8-11	8-10	9-4	8-7	8-7	8-9
Socioeconomic status	Low	Low	Low	Low	Low	Medium	Low	Low
RtI support	Tier2	Tier2	Tier2	Tier2	Tier2	Tier2	Tier2	Tier2
% in general class	100	100	100	100	100	100	100	100
Otis-Lennon/Full scale	86	76	92	96	85	99	No test	85
Verbal	89	87	87	92	90	95		82
Performance	84	67	98	99	82	103		90

Note. BAU = business-as-usual; RtI = response to intervention; OtisLennon = Otis-Lennon School Ability Test (Otis & Lennon, 2003).

Dependent Measures

Word Problem Solving Criterion Test (WPS)

The primary dependent measure was a researcher-developed 14-item WPS test; it involves eight part-part-whole problems (including combine, join-in, and take-away story situations) with either the part or the whole as the unknown, and six additive compare problems (including “more than . . .” or “less than . . .” word problem situations) with either the smaller, bigger, or the difference quantity as the unknown. The WPS test was designed in alignment with the NCTM (2000) and CCSSI (2012), which emphasize varying construction of word problems for assessing conceptual understanding and modeling with different representations in equations involving an unknown quantity in different positions. Cronbach’s alpha of the WPS test was .86 and the test–retest reliability was .93. Alternate forms were created for the use during pre- and postintervention assessment. Parallel form reliability between two randomly selected forms was .84.

Algebraic Model Expression Test (AME)

The AME test was used to assess potential improvement of students’ algebraic concepts, specifically, the algebraic expression of mathematical relations or ideas. It includes 10 items taken directly from commercially published mathematics textbooks and is in line with Common Core State Standards for Mathematics (CCSSI, 2012) that require model expression rather than solving problems arithmetically (e.g., Write an equation. Choose a variable for the unknown: Javier has 420 points. He has 30 fewer points than Maria). Test–retest reliability of the AME test was .90.

Curriculum-based Test (CBT)

To assess the students’ ability to generalize the skills they learned during the intervention to solve similar problems (structurally similar word problems but may include irrelevant information or numbers) appearing in school adopted commercial published math textbooks, the CBT test includes 14 problems that have similar mathematical problem structures as the WPS test. Test–retest reliability of CBT was .91.

Standardized Measurement

In addition to the WPS test and the CBT proximal measure, we used the Mathematics Problem Solving subtest of the Stanford Achievement Test (SAT-10, Pearson, 2004) as a distal measure for evaluating the impact of the interventions. The SAT-10 is a norm-referenced, criterion-referenced, and standardized achievement test with established reliability and validity. All items are presented in a multiple-choice format. Per the technical manual of the test, its internal consistency reliability for Primary 3 (Grade 3, Fall) for the Mathematics Problem Solving subtest was 0.88 to 0.89.

Scoring

The percentage of problems solved correctly was used as the dependent measure and calculated as the total points earned divided by the total possible points. Specifically, if the correct answer was given to a problem, one point was given. In the case that the answer to a problem-solving item (not multiple-choice items as in SAT) was incorrect, but the algorithm or model equation was correctly set up, half a point was given. The second and third authors scored all the tests independently. Interrater reliability was computed by dividing the number of agreements by the number of agreements and disagreements and multiplying by 100. Interrater reliability for scoring was 89% across all the tests. The disagreement between the two independent raters was resolved by the first three authors meeting to reexamine the test sheet and reaching a consensus.

Procedure

Both the MBPS and the BAU groups received the regularly scheduled 60-min mathematics class plus 30-min response-to-intervention (RtI) remediation during each school day. In addition, each student in the MBPS condition worked with the MBPS computer tutor, a web-based interactive program, during the afterschool, Monday through Thursday, for a total of 18 sessions (ranged from 15 to 23 across different individuals), with each session lasting for about 15 to 25 min throughout the fall semester. In contrast, the BAU group worked on solving similar math problems during the same afterschool programs monitored by participating schoolteachers for a total of 18 sessions. Both groups of students took the WPS test before and after the intervention. The MBPS group took two more follow-up tests, at 1 and 2 weeks after the termination of the intervention for skill maintenance check. The schoolteachers who led the BAU group did not administer the follow-up tests to the BAU group as they were busy wrapping up the semester. Both groups of students were allowed to use calculator to solve the problems.

MBPS Intervention

Students in the MBPS condition worked with the computer tutor one-on-one. Sessions were monitored by supervisors (including trained graduate students majoring in elementary education, special education, or computer science), with a supervisor–student ratio of about one (supervisor) to four (students). The session supervisor helped each of the participants log onto the MBPS computer tutor program in the beginning of each of the sessions. The student followed the direction of the computer tutor and engaged in the activities in Modules A through C. The learning and instruction were conducted through visual (screen display including animations) along with audio output as well as interactions between the learner (mouse click and input) and the computer tutor (audio and visual feedback). Each word problem was read to the learner to accommodate potential reading difficulties of the learner. During the session, the primary role of the supervisor was to make sure students followed the direction of the MBPS tutor and completed all modules in sequences. Supervisors also took responsibilities for noting any technical-related issues such as screen “freeze” and redirecting students to appropriate parts of the program after any unexpected interruptions.

Module A

Before solving real-world problems, Module A engaged students in a series of activities involving the use of virtual manipulatives such as unifix cubes to nurture fundamental mathematical ideas that are crucial for the development of additive reasoning and problem solving. Virtual manipulations have been supported by research in enhancing mathematics problem solving for students with special needs (Bouck et al., 2014). Specifically, Module A focuses on students’ conception of “number as the composite unit” (e.g., any number that is larger than 1 can be decomposed into a combination of two numbers, for instance, 4 is made of 3 and 1, or 2 and 2, or 0 and 4) and the development of multi-digit numbers as quantities of tens and ones. The aim of Module A is to challenge children’s counting acts to provoke changes in their mental operations, which will bring about the development of the composite unit (Kim et al., 2022). Module A consists of seven lessons Lesson 1: assessment of composite unit development; Lesson 2: counting using 5 and 10; Lesson 3: composing/decomposing numbers from 1 to 10; Lesson 4: building counting by ones strategies; Lesson 5: building tens and ones structures for numbers from 1 to 20; Lesson 6: using tens and ones structures to add and subtract with multiples of 10; and Lesson 7: using tens and ones structures to add and subtract within 100. Existing literature suggests that counting schemes are critical for the development of fundamental mathematics concepts as well as advanced mathematics reasoning (Steffe et al., 1983). Figure 1A presents sample screenshots in Module A of the MBPS tutor.

Figure 1.

Sample Screenshots of Modules A, B and C in the MBPS Computer Tutor.

Module B and Module C

Building on the fundamental mathematical ideas addressed in Module A, Module B engaged students in representing and solving various combine and change problem types using one cohesive mathematical model equation (part and part makes up the whole, or p + P = W). Module C engaged students in representing and solving a range of additive compare problems with the same mathematical model equation, although the denotation of each element in the PPW diagram equation changed accordingly (e.g., P_{smaller quantity} + P_{difference quantity} = W_{bigger quantity}). After solving the comparison problems, students were given opportunities to represent and solve mixed additive word problems to facilitate their construction of the mathematical model, p + P = W, for generalized problem solving. Figure 1B presents sample screenshots from Modules B and C of the MBPS tutor.

As shown in Figure 1B (left panel), in the beginning stage of Module B and Module C, before students learned how to solve real problems, the computer tutor provided students with word problem stories (e.g., “Emily had 8 marbles. She gave away 3 marbles. Now she has 5 marbles left.”) in which all three quantities in a problem were given; students did not need to solve for an unknown quantity. The purpose of presenting word problem stories in the beginning stage of the learning was for students to see the mathematical relations between the three quantities that make up the model equation (i.e., p + P = W). As all three quantities were given, students were able to see a balanced equation only if the information from the problem was mapped correctly onto the diagram equation. Through this word problem story representation stage, students were expected to make the connection between the concrete/semi-concrete bar model and the abstract part-part-whole model equation and, therefore, construct the concept of “part and part makes up the whole” or “p + P = W” model equation.

Following presentation of word problem stories with all known quantities, the computer tutor presented the students with real-world problems with an unknown quantity. During the problem-solving stage, students were guided to use letter “a” to represent the unknown quantity in the diagram equation. Specifically, in Module B, students were learning to solve problems such as “join-in” (e.g., Sam had 8 candy bars. Then Lucas gave him some more candy bars. Now he has 15 candy bars. How many candy bars did Lucas give Sam?); “take away” (e.g., Alex had many dolls. Then she gave away 12 of her dolls to her sister. Now Alex has 26 dolls. How many dolls did Alex have in the beginning?); or “combine” (e.g., Mr. Samir had 61 flashcards for his students. Mrs. Jones had 27 flashcards. How many flashcards do they have altogether?).

In Module C, students were introduced to additive compare problems. As shown in Figure 1C (right panel), the representation process was anchored with students fixing the “name tags” first before mapping quantities onto the diagram equation. Using the problem in Figure 1C (right panel) as an example, from the relational statement in this problem (i.e., He [Bobby] has 13 more basketball cards than Jeff), students would know Bobby had more basketball cards than Jeff and therefore were expected to place the name tag “Bobby” on the box for “Bigger” on one side of equation by itself, and the name tag “Jeff” on the box for “Smaller” (as one of the two parts) on the other side of the equation, together with the quantity for “Difference” (the other “part”). After students fixed the name tags they were asked to read the problem again and find corresponding numbers that were associated with Bobby, Jeff, or the difference quantity and map them onto the diagram equation. Once the representation of information in the diagram was complete, students were encouraged to use multiple strategies to solve for the unknown quantity in the diagram equation. For instance, students could solve for the unknown part using “mental map” or the math equation box to rewrite the equation and solve for the unknown part by subtracting the given part from the total. For those students who were unable to carry out the calculation, they were allowed to use calculator, as an accommodation, to solve for the unknown quantity after setting up the equation in the Math Equation box.

After the instruction in solving problems combine, join in, and take away in Module B, as well as additive compare problems in Module C, students were given opportunities to solve a mix of additive word problem variations. The purpose of engaging students in representing and solving variously constructed word problems was to reinforce the idea that although the cover stories of word problems (e.g., combine, join in, take away, compare problems involving “more than” or “less than”) might differ one from another, the underlying mathematical relation is the same: “part and part makes up the whole” or “p + P = W.”

Treatment Fidelity

For the MBPS group, the intervention for each of the students was prescribed by the computer tutor and the session supervisors ensured that the students followed the sequence of the three modules as described for the MBPS condition. The session supervisors were asked to take notes about computer “glitches” encountered during the session and report them to the Tech support member who would fix the “glitches.” The supervisors were required to report if any human-delivered instruction was provided during the session. In addition, all the sessions were recorded through a free and open-source software, OBS (Open Broadcaster Software) Studio, installed on each laptop to monitor students’ engagement as well as any human interaction with the learner during the session. Triangulated data showed no human-delivered problem-solving instruction during the sessions.

In contrast, the math instructional/learning activities carried out in the afterschool program defined the BAU condition. The researchers asked the schoolteachers who led the math afterschool program about their teaching strategies. These teachers reported that they often asked students to draw pictures to help understand the problem; they also taught students to use “cue” words to help with the operation sign in the math sentence. For instance, when seeing words such as “give away” in the word problem, students were taught to translate that into a minus sign in the math sentence.

Results

Equality Between the Groups During Pretest

To test the group equivalence before the intervention, we conducted the independent Samples t test on the two groups (i.e., MBPS and BAU) of students’ performance on the WPS criterion test. Results indicate that there were no significant differences between the two groups (p = .149) before the intervention. We also conducted homogeneity tests on gender and ethnicity between the two groups, Chi-square analyses showed no significant differences between the two groups (p = .80 for gender and p = .44 for ethnicity).

Comparing the Intervention Effect between the Two Groups

Table 3 presents the mean values and standard deviations on the WPS test by conditions. To assess the impact of the MBPS program, in reference to the BAU condition, we conducted an analysis of variance (ANOVA, 2 groups × 2 times of testing) with repeated measures on time (pretest and posttest) on the WPS test. The results show a statistically significant effect of time (p < .001), which indicates that both groups of students improved their performance on these measures after the intervention or remediation. More importantly, there is a statistically significant group–time interaction effect (p =.027). That is, although both groups improved their performance from pre- to posttest, the improvement rate of the students in the MBPS group (effect size [ES]_MBPS = 1.88) is much larger than that of the students in the BAU group (ES_BAU = 0.98). Figure 2 presents the two groups’ performance on the WPS test before and after the intervention or remediation. Using gain scores as the measure, the effect size between the two groups was 1.08 favoring the MBPS group.

Table 3

Mean Values and Standard Deviations by Treatment Condition for the Word Problem Solving Criterion Test.

Phase	M		SD		n
Phase	MBPS	BAU	MBPS	BAU	MBPS	BAU
Pretest	5.00	5.75	.707	.886	9	8
Posttest	9.61	7.69	2.51	1.58	9	8
Follow-up 1	9.33		2.45		9
Follow-up 2	8.17		2.29		9

Note. Scores are raw scores. MBPS = model-based problem solving; BAU = business-as-usual.

Figure 2.

Two Groups’ Performance on WPS Test Before (Time = 1) and After (Time = 2) the Intervention.

Maintenance Effect of the MBPS Group

We also conducted post hoc analyses of repeated measure ANOVA on the MBPS group’s posttest and follow-up test scores to evaluate potential maintenance effect. Results demonstrate that main effect on time shows a nonsignificant change (p = .150) in performance from posttest to follow-up Tests 1 and 2, indicating that students in the MBPS group maintained their posttest performance 1 and 2 weeks after the termination of the intervention.

Generalization Effect of the MBPS Group

To measure the potential generalization effect of the MBPS intervention, we conducted paired samples t tests to evaluate the MBPS group’s performance change on the AME test before and after the intervention. Results indicate that the students in the MBPS group significantly improved their performance on the algebraic model expression tasks after the MBPS intervention (M_pre = 2.56, M_post = 4.56, t = 3.000, p = .009).

To evaluate the impact of the MBPS intervention on the MBPS group’s performance on the CBT, we also used paired-samples t tests. The results indicated that the MBPS group students significantly improved their performance from pretest to posttest (M_pre = 2.56, M_post = 4.56, t = 8.043, p < .001).

To evaluate potential impact of MBPS intervention on performance on a distal measure, the MBPS group also took the Mathematics Problem Solving subset of the SAT-10 (Pearson, 2004) before and after the MBPS intervention. Results on SAT scores indicate that five out of nine (56%) participants in the MBPS group improved their SAT percentile rank by 3, 9, 9, 12, and 30 respectively.

Discussion

The purpose of this study was to evaluate the effect of a computer tutor that emphasizes MBPS on the performance of students with LDM. The MBPS tutor included two major components: Module A focuses on nurturing fundamental mathematical ideas pertaining to composing and decomposing a number and the concept of composite unit, which leads naturally to the concept of “part and part makes up the whole.” Modules B and C engaged students in representing and solving a range of additive word problems using the MBPS strategy. Results indicate that students in the MBPS group outperformed the BAU group who received “businesses as usual” intervention from the teachers.

The Effect of MBPS

After carefully examining students’ work before and after the intervention, it seems that during the pretest, most of the participants either relied on a keyword strategy to solve the problem or mapping the numbers from the word problem into the PPW diagram equation based on the sequence of each of the numbers appearing in the problem. Figure 3 presents students’ sample work before and after the MBPS intervention. As shown in the Figure 3a and b, during the pretest, the students mapped numbers into the PPW diagram equation based on the sequence of the numbers appearing in the word problem, without understanding the mathematical relations between the quantities. The decision to use “add” as shown in the vertical forms perhaps reflected the use of the keyword strategy (the word “Together” in the word problem as in Figure 3a or the word “more” as in the problem in Figure 3b). However, when the sequence of the numbers appearing in the problem is “consistent” with the layout of the PPW diagram equation, or the keyword (“more”) in the problem is consistent with the operation (+) needed for solution, then the sequential mapping or the keyword strategy would work coincidently, as shown in Figure 3c. Many students applied addition to solve all problems regardless of the problem situations.

Figure 3

Participating Students Sample Work Before and After MBPS Intervention.

Following the MBPS, as shown in Figure 3d–f, the students seem to represent the numbers in the PPW diagram equation based on their understanding of the “part” or “whole” in the story. For instance, in Figure 3d, the student’s mapping of the numbers into the diagram equation was not based on the sequence of the numbers appearing in the problem (i.e., 62, 29, ?).

It should be noted that the students used the “labeling” and/or “name tag” strategy they learned from the MBPS tutor to help anchor the representation of information from the word problem to the PPW diagram equation (as shown in Figure 3e for instance, “S” [small quantity], “d” [difference quantity], “B” [bigger quantity], as well as “E” for Elise, “T” for Tiffany). After correctly mapping the numbers in the PPW diagram equation, the model equation drives the solution plan as shown in Figure 3e (i.e., a + 20 = 42, a = 42 – 20 = 22).

In contrast, similar to the MBPS group during the pretest, the BAU group students often sequentially mapped numbers to the PPW diagram equation, relied on keyword strategies, or applied addition to solve all the problems. After the BAU instruction, some students used drawing (tick marks, mixed with pictures of the characters) to represent the problem situation. With teachers’ correction and coaching, some students were able to use a vertical algorithm to correctly solve some problems, especially those “consistent language problems” (e.g., Davis had 62 toy army men, then, one day he lost 29 of them. How many toy army men does Davis have now?) where the operation sign (e.g., “minus”) was consistent with the keyword (“lost”) and sequential translation of the word problem to a math sentence worked well (i.e., 62 – 29 = ?). However, students in the BAU group continued to struggle on problems such as inconsistent language problems (see Figure 3f for an example) where the referent unit, the quantity that the student would start with (from which 29 would be taken away), was the unknown quantity and the keyword strategy (“fewer” signifies “minus”) would not work.

Maintenance and Generalization Effect

The findings from this study showed that students in the MBPS group were able to maintain their improved performance on the WPS test one to two weeks after the termination of the intervention. As for generalization effect, after the MBPS intervention, participants improved their performance, to certain degrees, on the AME test as well as a proximal measure CBT. As the nature of the MBPS intervention was to facilitate algebraic reasoning and model representation, it was expected that students in the MBPS group would show some improvement in representing word problem situations in mathematical model equations without solving the problems. Nevertheless, as the problem presentation format of the test items in the AME (e.g., “Write an equation. Choose a variable for the unknown: Javier has 420 points. He has 30 fewer points than Maria.”) was different from the word problems presented in MBPS tutor (e.g., Ellen ran 62 miles in one month. Ellen ran 29 fewer miles than her friend Cooper. How many miles did Cooper run?), although the number of problems solved correctly doubled following the MBPS intervention, students’ posttest performance did not surpass 60% correct on the AME generalization test. Providing opportunities for students to solve similar problems as presented in AME test through worked-out examples (Fuchs et al., 2021) would perhaps help students become familiar with the presentation format of model expression problems and further enhance their performance on the AME test.

Similarly, although students significantly improved their performance on the CBT measure by doubling the number of problems they solved correctly compared with the pretest, their posttest scores did not reach 60% correct. Existing research shows that students with LDM experience difficulties in identifying relevant versus irrelevant information in word problems (Fuchs et al., 2008). Explicit instruction in how to identify relevant and irrelevant information is necessary to further enhance students’ performance on solving word problems involving irrelevant information.

Regarding far transfer effect on distal measure SAT, it should be noted that SAT covers a wide range of problem-solving skills, not just addition and subtraction word problem solving. Although only 56% of the students improved their performance on this commercially published standardized test, it is still encouraging given the fact that the majority of the students showed improvement to different degrees on this far transfer measurement. In addition to the additive reasoning and problem-solving students constructed through Modules B and C, the fundamental mathematical ideas nurtured in Module A might also have contributed to students’ general knowledge in basic number concept or number sense, which might have contributed to the partial generalization effect. The results from this study are consistent with the findings from previous research (e.g., Xin, 2019) as well as studies that examined the effect of another MBPS intelligent tutor designed to nurture multiplicative reasoning and problem solving of students with LDM (e.g., Xin et al., 2017). Model-based problem-solving underscores connections between mathematical ideas, which might promote students generalized problem-solving skills.

Limitations of the Study and Future Research

One limitation of the current study was that we were unable to randomly assign the students to two different conditions—it was largely determined by the schools in terms of which students with LDM needed and signed off for the specific intervention. To ensure the equality of the two comparison conditions, we conducted homogeneity tests on gender, ethnicity, and pretest performance. Future randomized control trial studies are needed to enhance the rigor of the experimental design. Another limitation of this study was the relatively small sample size. In fact, we exhausted the potential participants from one participating school and therefore we purposefully used a repeated measures design to boost the statistical power of the study. Future research is needed to expand the intervention to various school settings with larger sample sizes to validate the efficacy as well as the feasibility of the MBPS program across different learning environments. Lastly, it should be noted that participants in both groups also received Tier I and Tier II instruction (the participating school combined 60-min Tier I instruction and 30-min Tier II hours into one block of 90-min math class hours during the school day) and the three school-identified students with LDs received Tier III intervention. As such, the effect observed in this study pertaining to each of the comparison conditions might be a cumulative effect of the tiers of instruction each participant received during the school day plus the respective intervention during the afterschool program. Future research that includes more homogeneous groups of participants would help tease out the intervention effect.

Implications for Practice

The challenge of meeting the expectations of today’s high curriculum standards is compounded by a serious shortage of effective curricular materials and tools to empower the teachers who work with students with LDM. Our interviews with elementary mathematics teachers who worked with students with diverse needs suggest that these teachers face significant challenges in their work with children with LDM (Xin et al., 2018). Following are two quotes representing the challenges that many teachers are facing:

The biggest challenge schoolteachers encountered when working with students with learning difficulties is concept building” (RH, fifth-grade teacher)

Using regular math programs, teachers do not know how to adapt them to meet the needs of students with special needs. (JB, fourth-grade teacher/worked with students with LDM)

Currently, there is a lack of CAI intervention programs that focus on fundamental mathematical ideas to help students make sense of mathematics and enable generalized word problem-solving skills. The MBPS web-based program will provide teachers with tools guided by evidence-based instructional practice to facilitate all students’ meeting the curriculum standards. MBPS is one of the essential emphases in the Common Core mathematical practice standards (CCSSI, 2012).

Existing literature shows that students had the most difficulty in making the mental leap from the real situational model to mathematical model representation (Blum & Leiss, 2005). The preliminary findings of this study are encouraging. The web-based MBPS program, which emphasizes mathematical MBPS, seems to boost participants’ performance above and beyond the BAU instruction. The MBPS tutor incorporates best practices that are identified by the IES’s latest practice guide (Fuchs et al., 2021). First, MBPS provides systematic instruction through sequencing. For instance, the program nurtures students’ fundamental mathematical ideas that are critical to additive reasoning before introducing abstract mathematical models to students for word problem solving so that the students will be able to make sense of MBPS. Second, the MBPS tutor teaches precise mathematical languages and provides verbal support, including teaching word-problem story grammar (Xin et al., 2008) that help students understand mathematical relations. Third, the MBPS provides visuals such as the PPW diagram equation and name tags to anchor the problem representation. Through visual representation of the mathematical model equation (e.g., Part + Part = Whole) supported by scaffolds such as name tags or “labels,” students’ attention was directed to the understanding of mathematical relations depicted in the word problem, rather than jumping to searching for the keyword for operation or grabbing numbers for calculation.

Finally, the web-based MBPS tutor can facilitate the accessibility and usability of the program whether in inclusive classrooms or learning pods during school, in after-school programs, or in students’ homes. Since the pandemic, homeschooling/learning pod including hybrid homeschooling has grown noticeably (National School Choice Week Team, 2022). Because of their small size and personalized environment, the web-based MBPS tutor might also benefit these “micro-schools” that embrace students with learning differences.

Footnotes

Acknowledgements

The authors thank the administrators (in particular, Dr. John Layton, associate superintendent; Mrs. Megan Hatke, the principal), teachers, and students at Lafayette School Corporation who facilitated this study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This research was funded by the National Science Foundation, under grant #1503451. The opinions expressed do not necessarily reflect the views of the Foundation.

ORCID iDs

Yan Ping Xin

Qingli Lei

References

Blum

Leiss

(2005). “Filling up” – The problem of independence-preserving teacher interventions in lesson with demanding modeling tasks. Proceedings of CERME 4. http://www.erme.tu-dortmund.de/~erme/CERME4/CERME4_WG13.pdf#page=13

Bouck

E. C.

Satsangi

Doughty

T. T.

Courtney

W. T.

(2014). Virtual and concrete manipulatives: A comparison of approaches for solving mathematics problems for students with autism spectrum disorder. Journal of Autism and Developmental Disabilities, 44, 180–193. https://doi.org/10.1007/s10803-013-1863-2

Caldwell

J. H.

Karp

Bay-Williams

J. M.

(2011). Essential understanding series: Developing essential understanding of addition and subtraction for teaching mathematics in prekindergarten–grade 1. National Council of Teachers of Mathematics.

Carcoba Falomir

G. A.

(2019). Diagramming and algebraic word problem solving for secondary students with learning disabilities. Intervention in School and Clinic, 54(4), 212–218. https://doi.org/10.1177/1053451218782422

Common Core State Standards Initiative. (2012). Introduction: Standards for mathematical practice. https://learning.ccsso.org/wp-content/uploads/2022/11/ADA-Compliant-Math-Standards.pdf

Davydov

V. V.

(1982). Psychological characteristics of the formation of mathematical operations in children. In Carpenter

T. P.

Moser

J. M.

Romberg

T. A.

(Eds.), Addition and subtraction: Cognitive perspective (pp. 225–238). Lawrence Erlbaum.

Every Student Succeeds Act. (2015) Pub. Law 114-95, 129 Stat 1801.

Friendly

(2000). Power analysis for ANOVA designs. http://davidmlane.com/hyperstat/power.html

Fuchs

L. S.

Newman-Gonchar

Schumacher

Dougherty

Bucka

Karp

K. S.

Morgan

(2021). Assisting students struggling with math: Intervention in the elementary grades (WWC 2021006). National Center for Education Evaluation and Regional Assistance (NCEE), Institute of Education Sciences, U.S. Department of Education. http://whatworks.ed.gov/

10.

Fuchs

L. S.

Seethaler

P. M.

Powell

S. R.

Fuchs

Hamlett

C. L.

Fletcher

J. M.

(2008). Effects of preventative tutoring on the mathematical problem solving of third-grade students with math and reading difficulties. Exceptional Children, 74(2), 155–173. https://doi.org/10.1177/001440290807400202

11.

Geary

D. C.

(1994). Children’s mathematical development: Research and practical applications. American Psychological Association. https://doi.org/10.1037/10163-000

12.

Hughes

E. M.

Witzel

B. S.

Riccomini

P. J.

Fries

K. M.

Kanyongo

G. Y.

(2014). A meta-analysis of algebra interventions for learners with disabilities and struggling learners. The Journal of the International Association of Special Education, 15(1), 36–47. Retrieved February 28, 2023 from https://www.learntechlib.org/p/159660/

13.

Jonassen

D. H.

(2003). Designing research-based instruction for story problems. Educational Psychology Review, 15, 267–296.

14.

Kim

S. J.

Kastberg

S. E.

Xin

Y. P.

Lei

Liu

Wei

Chen

(2022). Counting strategies of students struggling in mathematics in a computer-based learning environment. The Journal of Mathematical Behavior, 68, Article 101007.

15.

Kim

S. J.

Xin

Y. P.

(2022). A synthesis of computer-assisted mathematical word problem-solving instruction for students with learning disabilities or difficulties. Learning Disabilities: A Contemporary Journal, 20(1), 27–45. https://www.ldw-ldcj.org/images/Kim__Xin_2022.pdf

16.

Myers

J. A.

Witzel

B. S.

Powell

S. R.

Pigott

Xin

Y. P.

Hughes

E. M.

(2022). A meta-analysis of mathematics word-problem solving interventions for elementary students who evidence mathematics difficulties. Review of Educational Research, 92(5), 695–742. https://doi.org/10.3102/00346543211070049

17.

National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics.

18.

National School Choice Week Team. (2022). Pandemic pods are here, are you in? https://schoolchoiceweek.com/learning-pods/

19.

Pearson. (2004). Stanford achievement test (10th ed., SAT-10).

20.

Polotskaia

Savard

(2018). Using the relational paradigm: Effects on pupils’ reasoning in solving additive word problems. Research in Mathematics Education, 20(1), 70–90. https://doi.org/10.1080/14794802.2018.1442740

21.

Steffe

L. P.

von Glasersfeld

Richards

Cobb

(1983). Children’s counting types: Philosophy, theory, and application. Praeger.

22.

Thompson

P. W.

(1993). Quantitative reasoning, complexity, and additive structures. Educational Studies in Mathematics, 25, 165–208. https://doi.org/10.1007/BF01273861

23.

U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress. (2022). Mathematica assessment.

24.

What Works Clearinghouse. (2020). What works clearinghouse standards handbook (Version 4.1). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/handbooks

25.

Witzel

B. S.

Myers

J. A.

Xin

Y. P.

(2022). Conceptually intensifying word problem solving for students with math difficulties. Intervention in School and Clinic, 58(1), 9–14. https://doi.org/10.1177/10534512211047580

26.

Woodward

Monroe

Baxter

(2001). Enhancing student achievement on performance assessment in mathematics. Learning Disability Quarterly, 24(1), 33–46. https://doi.org/10.2307/1511294

27.

Xin

Y. P.

(2007). Word-problem-solving tasks presented in textbooks and their relation to student performance: A cross-curriculum comparison case study. The Journal of Educational Research, 100, 347–359. https://doi.org/10.3200/JOER.100.6.347-360

28.

Xin

Y. P.

(2012). Conceptual model-based problem solving: Teach students with learning difficulties to solve math problems. Sense. https://doi.org/10.1007/978-94-6209-104-7_1

29.

Xin

Y. P.

(2015). Research related to modeling and problem solving: Conceptual model-based problem solving: Emphasizing pre algebraic conceptualization of mathematical relations. In Silver

E. A.

Kenney

P. A.

(Eds.), More lessons learned from research: Useful and useable research related to core mathematical practices (pp. 235–246). National Council of Teachers of Mathematics (NCTM).

30.

Xin

Y. P.

(2016). Conceptual model-based problem solving. In Fermer

Kilpatrick

Pehkonen

(Eds.), Posing and solving mathematical problems: Advances and new perspectives (pp. 231–254). Springer.

31.

Xin

Y. P.

(2019). The effect of a conceptual model-based approach on “additive” word problem solving of elementary students who are struggling in mathematics. ZDM: Mathematics Education, 51(1), 139–150. https://doi.org/10.1007/s11858-018-1002-9

32.

Xin

Y. P.

Jitendra

Deatline-Buchman

(2005). Effects of mathematical word problem-solving instruction on middle school students with learning problems. The Journal of Special Education, 39, 181–192. https://doi.org/10.1177/00224669050390030501

33.

Xin

Y. P.

Kastberg

Chen

(2015). Conceptual model-based problem solving: A response to intervention program for students with learning difficulties in mathematics (COMPS-RtI). National Science Foundation (NSF) funded project.

34.

Xin

Y. P.

Park

Tzur

(2020). The impact of a conceptual model-based mathematics computer tutor on multiplicative reasoning and problem-solving of students with learning disabilities. The Journal of Mathematical Behavior, 58, Article 100762. https://doi.org/10.1016/j.jmathb.2020.100762

35.

Xin

Y. P.

Tzur Si

Hord

Liu

Park

J. Y.

(2017). An intelligent tutor-assisted math problem-solving intervention program for students with learning difficulties. Learning Disability Quarterly, 40(1), 16. https://doi.org/10.1177/07319487166487400

36.

Xin

Y. P.

Wang

Sanders

(2018). Conceptual Model-based Mathematics Intelligent Tutors (COMMIT). NSF I-Corps Funded Project.

37.

Xin

Y. P.

Wiles

Lin

(2008). Teaching conceptual model-based word-problem story grammar to enhance mathematics problem solving. The Journal of Special Education, 42, 163–178. https://doi.org/10.1177/0022466907312895

38.

Xin

Y. P.

Zhang

Park

J. Y.

Tom

Whipple

(2011). A comparison of two mathematics problem-solving strategies: Facilitate algebra-readiness. The Journal of Educational Research, 104, 381–395. https://doi.org/10.1080/00220671.2010.487080

39.

Zhang

Xin

Y. P.

(2012). A follow-up meta-analysis of word problem solving interventions for students with learning problems. The Journal of Educational Research, 105(5), 303–318. https://doi.org/10.1080/00220671.2011.627397