A Randomized Trial of the Effects of Schema-Based Instruction on Proportional Problem-Solving for Students With Mathematics Problem-Solving Difficulties

Abstract

This article reports results from a study investigating the efficacy of a proportional problem-solving intervention, schema-based instruction (SBI), in seventh grade. Participants included 806 students with mathematical difficulties in problem solving (MD-PS) from an initial pool of 1,999 seventh grade students in a larger study. Teachers and their students in the larger study were randomly assigned to an SBI or control condition and teachers in both conditions then provided instruction on the topics of ratio, proportion, and percent. We found that students with MD-PS in SBI classrooms scored on average higher than their counterparts in control classrooms on a posttest and delayed posttest administered 9 weeks later. Given students’ difficulties with proportional problem-solving and the consequences of these difficulties, an important contribution of this research is the finding that when provided with appropriate instruction, students with MD-PS are capable of enhanced proportional problem-solving performance.

Keywords

schema-based instruction proportional reasoning problem solving mathematics difficulties

Proportional reasoning, which requires understanding the multiplicative relations between quantities (ratios) as well as the “covariance of quantities and invariance of ratios” (Lamon, 2007, p. 638), is important in understanding advanced mathematics topics and problems in science and technology (Beckmann & Izak, 2015; Fujmura, 2001; Lobato, Ellis, Charles, & Zbiek, 2010; National Mathematics Advisory Panel [NMAP], 2008). Despite its importance, learning and understanding ratios and proportional relationships has proven to be challenging for most learners (Adjiage & Pluvinage, 2007; Fujmura, 2001; Jitendra, Woodward, & Star, 2011; Lamon, 2007; Lobato et al., 2010; NMAP, 2008) and “transcends topical barriers in adult life” (Ahl, Moore, & Dixon, 1992, p. 81). The consequences of these difficulties are pronounced for students in an increasingly competitive job market, where the demand for mathematics-intensive science and engineering jobs are outpacing overall job growth three-to-one (NMAP, 2008).

Instructional time focused on ratios and proportional relationships in the Common Core State Standards (CCSS; National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010) occurs in middle school when students learn “to solve single and multi-step problems … solve a wide variety of percent problems, including those involving discounts, interest, taxes, tips, and percent increase or decrease” (p. 46). Solving even simple proportion problems is challenging for many children and adolescents, especially students with mathematics difficulties (MD). These students’ difficulty may be related to not only understanding the problem situation but also whether a particular solution strategy is appropriate. For example, a problem might involve two situations that appear to have different quantities (e.g., $3 for 6 apples, $6 for 12 apples), but the relations between the quantities remain constant (i.e., each apple costs $0.50). To solve this problem, students would need to recognize how relationships between quantities covary and identify relevant information to create an adequate representation of the problem (Lamon, 1999; Mayer, 1999) as well as demonstrate procedural flexibility to “choose strategically from among multiple solution methods based on their relative efficiency with respect to a given problem” (Berk, Taber, Gorowara, & Poetzl, 2009, p. 114).

Although several problem-solving interventions have been developed to help students with MD be more effective problem solvers (e.g., Xin & Jitendra, 1999; Zhang & Xin, 2012), the majority of research has focused on arithmetic and arithmetic story problems. As such, there is a general need for interventions focusing on complex problems (e.g., ratios and proportions, percent). Instructional interventions intended to promote understanding of ratios and proportional relationships are most useful when they provide opportunities for recognizing the underlying problem structure, using representations (e.g., diagrams) that illustrate the mathematical relations among key elements in the problem, facilitating problem solving and metacognitive strategy skills, and developing procedural flexibility (see Woodward et al., 2012).

An example of an instructional approach possessing these characteristics that was developed with the aim of improving student learning of ratios and proportional relationships is schema-based instruction (SBI). SBI is an instructional approach that has its roots in schema theory (Briars & Larkin, 1984; Carpenter, Hiebert, & Moser, 1981; Fennema, Carpenter, & Peterson, 1989; Riley, Greeno, & Heller, 1983) and incorporates cognitive models of mathematical problem solving (see Marshall, 1995; Mayer, 1999). In addition, SBI integrates several instructional features (e.g., explicit and systematic instruction, scaffolding instruction with guided questions to help clarify and refine student thinking, corrective feedback, frequent and cumulative review of key concepts) that are considered conducive to promoting problem solving for students with MD (Gersten, Beckmann et al., 2009; Gersten, Chard et al., 2009).

Since the initial studies of SBI that focused on teaching students to solve arithmetic problems (e.g., Jitendra et al., 1998; Jitendra, Dupuis et al., 2013), SBI has been developed to embed its instructional components (e.g., priming the underlying problem structure) into the content instruction for ratio, proportion, and percent that is aligned with state standards. The findings of several randomized controlled studies provide evidence of the effectiveness of SBI for typically achieving middle school students (Jitendra et al., 2009; Jitendra, Woodward et al., 2011; Jitendra, Star, Dupuis, & Rodriguez, 2013; Jitendra et al., 2015). However, only one randomized controlled study assessed the efficacy of SBI for students with MD on word problem solving involving multiplicative compare and equal groups problems (Xin, Jitendra, & Deatline-Buchman, 2005). Results indicated that students in the SBI group made significant gains at immediate posttest (d = 1.69) and on retention tests (d = 2.53) compared to students in the comparison condition when provided with about 720 min (12 sessions of 60 min) of remedial intervention.

In a limited number of SBI studies conducted in middle schools, students met the criterion for mathematical difficulties (i.e., low average performance on a standardized math achievement test) and were categorized as having MD (Jitendra et al., 2009; Jitendra & Star, 2012; Jitendra, Dupuis, Star, & Rodriguez, in press). These studies have yielded mixed results in that two studies suggested that the SBI intervention is not always promising for students with MD (Jitendra et al., 2009; Jitendra & Star, 2012); specifically, differences between SBI and business-as-usual control conditions on a ratio and proportion or a percent problem-solving test were not significant. One explanation for the nonsignificant results may be due to the relatively short duration of the intervention, which was about 450 to 500 min (9–10 sessions of 50 min). The authors argued that for these students to show gains in solving a wide range of problems, they may have needed more time and support to recognize the underlying problem structure, select and use the appropriate diagram to represent the problem, choose from among several methods, as well as monitor and reflect on the problem-solving processes (essential features of SBI).

The Jitendra et al. (in press) study, with a longer time frame, provided evidence that it is possible to have positive effects on the proportional problem-solving (PPS) performance of students with MD only (scored 25th percentile in mathematics and scored >25th percentile in the reading subtest on end-of-the-year state mathematics and reading achievement tests) and students with mathematics and reading difficulties (MDRD, scored 25th percentile in both mathematics and reading) when SBI instruction occurred over the course of 6 weeks of daily 45 min sessions (about 1,350 min in total). Due to the intensity of the support, students with MD and MDRD in the SBI condition not only outperformed their counterparts in the control condition on a PPS immediate posttest (g = 0.40) but also sustained the effects 6 weeks after the intervention (g = 0.42). However, SBI students’ responsiveness to treatment was differentiated by their performance on items related to only ratio and proportion versus percent. For students with MDRD, positive immediate and sustained effects were found on both ratio and proportion and percent problems. Similar positive immediate and sustained effects were found for students with MD on percent problem solving only. In sum, the evidence from the few investigations underscores the need for interventions that support students with MD become proficient in PPS.

The Present Study

Building on prior studies, we evaluated the effectiveness of SBI on proportional reasoning skills for seventh-grade students with mathematical difficulties in problem solving (MD-PS). Specifically, we examined these students’ responsiveness to a 6-week treatment (SBI vs. control group) on the immediate and retention tests of PPS. The research questions this study addresses are whether SBI is effective in enhancing the PPS skills of seventh graders’ with MD-PS (below the 35th percentile) immediately after the treatment is completed and whether the effects of SBI are maintained 9 weeks after the treatment is completed. In addition, the study examined the effect of SBI on a student’s growth over time. Based on previous research described above, we examined the following hypotheses:

SBI involves effective instructional practices (e.g., activating the mathematical structure of problems, representing the problem situation using an appropriate diagram, strategically selecting from among a variety of methods for a given problem that is efficient, monitoring and reflecting on the problem-solving processes) that will have positive immediate and sustained effects on students’ PPS performance. More specifically, students with MD-PS receiving SBI will outperform students with MD-PS in a control condition on both immediate and retention tests of PPS. Furthermore, we expected that SBI would yield similar positive effects on both ratio and proportion, and percent problems.

As SBI supports student learning of ratios and proportional relationships, it should have a pronounced positive effect on a student’s growth over time.

We also assessed whether students and their teachers who participated in the SBI intervention reported benefits of the SBI intervention (e.g., use of schematic diagrams, problem solving procedure) in solving proportion problems.

Method

Research Context and Design

The sample in the current study was taken from a larger study (Jitendra et al., 2015) that included a heterogeneous pool of seventh-grade students. The target population was middle school students and teachers in the upper Midwest of the United States. Eighty-two teachers from 58 middle schools in 50 public school districts in Minnesota participated in the Jitendra, Harwell et al. study (for information about demographics of the schools/school districts, see Jitendra et al., 2015). Once a teacher had been selected, one of their seventh-grade mathematics classes was selected at random to participate. Jitendra et al. (2015) used a randomized cluster design with longitudinal data (pretest, posttest, delayed posttest) in which 82 classrooms were initially assigned at random to SBI or control (“business as usual”) conditions. Teachers/classrooms served as clusters.

The current study used a subset of data from the Jitendra et al. (2015) study that consisted of students at risk for MD-PS (see Participants section). The classrooms of these students were assigned at random to SBI or control conditions, and students had already completed their participation in the Jitendra, Harwell et al. study. Thus, the research design for the current study was a retrospective randomized cluster design with longitudinal data. We argue that the strengths of randomly assigning the original 82 classrooms to SBI or control conditions are present in the current study. Specifically, we assume that whether a classroom was randomly assigned to the SBI or control condition in Jitendra, Harwell et al. was unrelated to whether students within that classroom were subsequently categorized as students with MD-PS. Thus, classrooms in the SBI or control condition should be approximately equal on background variables at pretest.

Participants

In the 82 classrooms, a total of 1,999 students participated in Jitendra et al. (2015). As described above, the current study focuses on a subgroup of those students with MD-PS. There is a lack of consensus on the definition of MD, “especially as it applies to identifying risk factors related to problem solving” (Swanson, Moran, Lussier, & Fung, 2014, p. 113). Because the focus of this study was on problem-solving deficit, we operationalized MD-PS as scores below the 35th percentile on a general measure of mathematical problem solving (Process and Applications subtest of the Group Mathematics Assessment and Diagnostic Evaluation [GMADE]; Pearson Education, 2004). This criterion was selected to ensure broad representation of students who may be at low to high risk of developing MD (see Bottge et al., 2015; Fuchs et al., 2014; Mazzocco, 2007). Based on scores on the GMADE measure administered at pretreatment, 806 students from among the 1,999 seventh graders who participated in the larger study were identified as having MD-PS. Of the 806 students with MD-PS, 399 were in SBI classrooms and 407 were in business as usual control classrooms. On the screening measure, the performance of SBI students was comparable to that of the control students. The mean GMADE score was 9.73 (SD = 2.02) for SBI and 9.80 (SD = 2.07) for the control group.

Table 1 presents student and teacher demographic data. In general, students were largely White with more than one half male and demographic patterns that were similar across the treatment and control groups. For example, the percentages of White, Black, Hispanic, and Asian students in the treatment group were 77.6, 8.9, 7.0, and 6.4, respectively; for the control group, these percentages were 77.1, 8.4, 7.9, and 6.6, respectively. Similar results emerged for teacher characteristics in the treatment and control groups. These findings support our assumption that students with MD-PS were equally distributed across the treatment and control classrooms.

Table 1.

Participant Demographic Information by Treatment.

		Treatment
		SBI				Control				Total
Variable		n	%	M	SD	n	%	M	SD	n	%	M	SD
Student information
Age				12.63	0.40			12.56	0.35			12.60	0.38
Sex	Female	179	44.9			198	48.6			377	46.8
	Male	205	51.4			186	45.7			391	48.5
Missing (age and sex)		15	3.8			23	5.7			38	4.7
Race	Asian	22	5.5			30	7.4			52	6.5
	Black	49	12.3			51	12.5			100	12.4
	Hispanic	31	7.8			37	9.1			68	8.4
	White	280	70.2			266	65.4			546	67.7
FRL	Yes	181	45.4			190	46.7			371	46.0
	No	201	50.4			194	47.7			395	49.0
LEP	Yes	30	7.5			37	9.1			67	8.3
	No	352	88.2			347	85.3			699	86.7
SpEd	Yes	58	14.5			48	11.8			106	13.2
	No	324	81.2			336	82.6			660	81.9
Missing (race, FRL, LEP, SpEd)		17	4.3			23	5.7			40	5.0
Teacher information
Sex	Female	26	65.0			29	69.0			55	67.1
	Male	14	35.0			13	31.0			27	32.9
Math courses taken				8.60	3.77			8.70	4.20			8.65	3.97
Education courses taken				4.24	4.81			2.87	2.65			3.54	3.89
Years experience in math				11.95	6.38			12.42	7.02			11.93	6.35
PD hours in math				24.88	30.94			23.65	17.61			24.25	24.86

Note. FRL = eligible for free or reduced price lunch; LEP = limited English proficiency; PD = professional development; SBI = schema-based instruction; SpEd = students qualified for special education services.

Screening Measure

The Process and Applications subtest of the GMADE (Pearson Education, 2004), a standardized broad measure of problem solving, evaluates students’ ability to comprehend mathematical language and concepts and apply relevant operations to solve word problems across multiple content areas (e.g., algebra, geometry, number, and operations). Reliability for this measure was 0.68 for fall and 0.77 for spring standardization samples. Several studies support the criterion-related validity of the GMADE. For example, correlations between the GMADE and KeyMath Revised–Normative Update (Connolly, 2007) were above .80.

Outcome Measure

The PPS test (Jitendra et al., 2015), a researcher-developed measure is comprised of released items from the National Assessment of Educational Progress (NAEP), Trends in International Mathematics and Science Study (TIMSS), and state mathematics assessments related to the topics of ratio/rate, proportion, and percent. The assessment consists of 23 multiple-choice (13 ratio and proportion, 10 percent) and four short-response items. The multiple-choice items were machine-scored dichotomously as correct/incorrect. The research team, blind to study conditions, scored the four short-response items on a 0 to 2 point scale using a rubric (e.g., using sample student responses taken from a previous study), which emphasized correct reasoning. Interrater reliability was estimated using an intraclass correlation and averaged 0.85, 0.91, and 0.89 at pretest, posttest, and delayed posttest, respectively.

To assess the reliability of the PPS items, we used the jMetrik software (Version 2.1.0; Meyer, 2011) to fit the congeneric model assuming a single continuous latent factor underlies the dichotomously and trichotomously scored PPS items (McDonald, 1999). The coefficient omega (Dunn, Baguley, & Brunsden, 2014) values for the PPS pretest, posttest, and delayed posttest of 0.54, 0.69, and 0.68, respectively, represent reliabilities estimated as the ratio of true score variance to observed score variance (Dunn et al., 2014; Revelle & Zinbarg, 2009).

Treatment Acceptability Rating Scale-Revised (TARF-R)

Students in the treatment condition completed a modified version of the TARF-R (Reimers & Wacker, 1988) at the end of the study. Students were asked to rate four items each related to diagrams (e.g., helped to organize information and understand how to solve problems) and problem-solving procedures (e.g., helpful in checking understanding of how to solve word problems) as well as one item on multiple solution strategies (e.g., enjoyed solving word problems using the different strategies—cross multiplication, unit rate, equivalent fractions) on a 1 to 4 scale (4 = strongly agree and 1 = strongly disagree). Coefficient alpha was 0.83.

Teacher Satisfaction Survey

Treatment teachers completed a survey of their perceptions of the SBI intervention at the end of the study. This instrument included 22 items that focused on the (a) overall benefits of the SBI intervention (e.g., the SBI curriculum will produce a lasting improvement in most students’ problem-solving skills); (b) ratio, proportion, and percent diagrams (e.g., diagrams are effective in highlighting the underlying mathematical structure of problems and mediating problem solution); and (c) problem-solving procedures, including focus on multiple solution strategies. The items were evaluated on the same 1 to 4 scale as the TARF-R. For this sample, coefficient alpha was 0.92.

Procedure

In mid-December, treatment teachers received professional development (see Professional Development section), and all teachers administered the GMADE and the PPS pretest following a standardized administration protocol. Teachers then provided instruction from early January to mid-February on the topics of ratio, proportion, and percent for 6 weeks, five times per week, using either SBI or their district-adopted curriculum. In mid-February to early March (within 3 weeks of the end of intervention), teachers readministered the PPS (posttest) and again approximately 9 weeks later (delayed posttest) to assess intervention effects. Student demographic information (e.g., sex, age, race, free or reduced lunch status) was also collected from the Minnesota Department of Education.

Professional Development

Treatment teachers in Jitendra et al. (2015) participated in an intensive 2-day (16 hr) professional development training covering implementation of the SBI intervention. The following were covered during professional development: (a) a detailed description of key features of the SBI intervention to support student learning of ratio, proportion, and percent; (b) implementation of SBI intervention components, including sorting problems by problem types, using schematic diagrams to represent information in the problem, generating “ballpark” estimates (quick and easy based on benchmark numbers and fractions), and selecting an appropriate solution method from among several strategies to solve problems; (c) procedures to facilitate student discussion, with an emphasis on developing their proportional reasoning skills using video segments of SBI teachers from previous studies; and (d) a discussion of implementing SBI and assessments faithfully as well as not sharing project materials and/or strategies with any control group teachers in their building.

Intervention Procedures

Teachers in the Jitendra et al. (2015) study provided all students in the treatment and “business as usual” control classrooms instruction during their regularly scheduled seventh-grade mathematics classes. In both conditions, instruction included two instructional units on ratio/proportion and percent, which was delivered daily over the course of 6 weeks. Students in control classes received business-as-usual classroom instruction that addressed the same topics as in the units taught in the treatment classes. Thus, all students were provided the opportunity to learn the same content over the same period of time regardless of random assignment to treatment or control classes. However, the delivery of the content differed in that treatment classes used the SBI components described below and the control classes received instruction that would typically occur in a seventh-grade mathematics class.

Description and Implementation of SBI

The instructional components of the SBI treatment are described in the original study (Jitendra et al., 2015) and include (a) activation of the mathematical structure of problems, (b) representation of information in the problem using diagrams, (c) development of procedural flexibility through selection and use of appropriate solution strategy based on the numbers in the problem, and (d) problem-solving and metacognitive (e.g., monitoring) strategy instruction. The problem-solving and metacognitive strategy instruction component allows students to engage in applying learned content (e.g., ratios/rates, percent) in problem-solving activities (e.g., recognizing the problem type, identifying and representing critical information in the problem using an appropriate diagram, connecting the problem to what is already known, estimating the answer, selecting a strategy to solve the problem, checking the reasonableness of the solution) and metacognitive activities (i.e., monitoring and reflecting on the problem-solving process).

Activating the mathematical problem structure

Teachers engaged students through deep-level questions to identify the type of problem by reading, retelling, and examining information in the problem as well as thinking about how problems within and across types are similar or different. Students critically evaluated whether a problem belongs to a particular category of problems (i.e., ratio, proportion, or percent) based on the problem features (e.g., proportion problems describe a statement of equality between two ratios/rates that allows one to think about the ways that the two situations are the same).

Representing the problem using diagrams

Teachers provided instruction on representing information in the problem using schematic representations that effectively linked the relationships between the relevant quantities in the problem. During problem representation, students were provided opportunities to critically reason why the same ratio schematic diagram can be used to represent information in both ratio and percent problems (a percent is a special type of ratio). Furthermore, instruction emphasized that even though ratio diagrams work well for some percent of change problems in representing the relation between the change amount and original amount, more complex percent of change problems (including simple interest) elicit the need for representations that depict both multiplicative and additive relationships (see Ratio and Percent diagrams in Figure 1).

Figure 1.

Ratio and Percent Diagrams

Using multiple solution strategies

Teachers encouraged students to deeply process information about when, how, and why to use a broad range of methods (e.g., equivalent fractions, unit rate, cross multiplication) for a given class of problems, thereby improving procedural knowledge. Instruction supported using and contrasting multiple solution methods so that students become cognizant of specific methods that are more efficient than others and select the strategy that is most efficient based on the numbers in the problem.

Teachers implemented the two replacement units (Ratio/Proportion and Percent) that consisted of the SBI instructional components embedded within their instruction. They were provided with a detailed teacher guide as a resource, along with teaching materials (e.g., visual diagrams and problem-solving checklists) to support implementation of activities to develop critical concepts and skills. Students received a set of materials (i.e., workbook and homework book) that included problems and application activities.

Instruction in Control (“Business-as-Usual”) Classes

Information on textbooks used in the control classrooms was also gathered in Jitendra et al. (2015) from (a) a written teacher questionnaire in which teachers listed the mathematics textbooks they used, (b) a review of ratio/proportion and percent lessons in the textbooks, and (c) an observation of one videotaped lesson of each teacher’s activity related to the target topics. Overall, the control classrooms used 10 different textbooks published from 2001 to 2012 by one of three publishers: Houghton, Mifflin, Harcourt; Glencoe/McGraw Hill; and Pearson Education. We examined the textbooks for the presence of core SBI instructional components (i.e., identifying the problem type, visual representations, modeling of problem-solving and metacognitive strategies, multiple solution strategies) and found that the instructional components covered did not overlap with those in treatment classrooms in ways that would distort estimates of the effects of SBI.

Fidelity of Treatment

Jitendra et al. (2015) developed a measure based on guidelines proposed by O’Donnell (2008) to measure fidelity of implementation (Dane & Schneider, 1998) and overall quality of instruction in treatment and control classrooms based on attributes of effective teaching. Fidelity information was generated by videotaping an entire lesson on proportion problem-solving for each teacher in the treatment and control conditions during the 6 weeks of the study. A seven-item measure was developed to determine the extent to which SBI teachers implemented the key components of the treatment: (a) identified the problem type by focusing on the key problem features, (b) connected the new problem to previously solved problems, (c) represented critical information in the problem text using an appropriate diagram, (d) generated an estimate prior to solving the problem, (e) discussed multiple solution strategies, (f) solved the problem and presents the solution within the context of the problem, and (g) evaluated the solution. The same measure was used in the control condition to evaluate program differentiation (Dane & Schneider, 1998).

Coders reviewed the videotapes and assigned a rating for each component. In addition, four global quality ratings were assigned based on the entire observation with regard to the teacher’s ability to (a) clarify the lesson purpose, (b) provide lesson closure, (c) manage instructional time (i.e., how well the teacher managed student behavior), and (d) minimize mathematical errors. Procedural fidelity and overall quality of instruction items were coded on a 0 to 3 scale (3 = high level of implementation and 0 = did not implement).

Coder training and reliability

One of the authors, in consultation with the developer of the SBI program, developed the coding scheme for the fidelity measure. The measure was finalized after multiple rounds of independent video coding by six coders, discussion, and reoperationalization of the codes. During coder training, videotapes were coded until agreement of codes was 90% or higher. Two coders independently assessed fidelity for each classroom video. Disagreements in coding were resolved through discussion and review of the videotapes. We estimated interrater reliability by computing intraclass correlations for the coder ratings, which averaged 0.98 across the seven procedural items (range 0.97 to 0.99) and 0.99 across the four quality instruction items (range 0.96 to 1.00).

Analysis of fidelity

We conducted t tests to test group differences on both the fidelity and quality of instruction data and used the Dunn-Bonferroni correction to control for compounding of Type 1 error. Results indicated statistically significant and substantial differences between the treatment and control groups on the total fidelity of implementation score, t(78) = 8.82, p <.001, and all individual items (p < .005) except for Item 6 (i.e., solves the problem and presents the solution within the context of the problem), with treatment teachers (M = 14.33, SD = 3.86) implementing SBI elements more than control teachers (M = 7.43, SD = 3.00). For the quality of instruction, there were no statistically significant differences between the groups (SBI: M = 9.50, SD = 1.28; Control: M = 9.28, SD = 1.71). These data provide evidence of program differentiation (Dane & Schneider, 1998) in that there were clear differences in SBI instructional elements across the two groups, whereas the general quality of instruction was similar in both conditions.

Data Analyses

We used two-level (students within classrooms) multilevel models to capture the hierarchical nature of the cross-sectional data (see Raudenbush & Bryk, 2002). The outcome variables included in the analyses were the PPS posttest (total score, ratio and proportion subscore, percent subscore) and PPS delayed posttest (total score, ratio and proportion subscore, percent subscore), which were analyzed separately. We examined students’ performance on the Ratio and Proportion and the Percent subscales separately on our PPS measure to address the two subdomains addressed by the intervention. A two-level model was fitted for each outcome variable using the HLM 6 software (Raudenbush, Bryk, & Congdon, 2004). In addition, we performed an ancillary analysis of the PPS longitudinal data to explore student change over time and whether change was related to the treatment.

We controlled for compounding Type I error rates using the Dunn-Bonferroni correction (Miller, 1966) procedure in which an overall Type I error rate (α = .20) was divided among all statistical tests for an outcome with no requirement that the error rate be divided equally. We assigned .05 to the test of the treatment because this was the most important effect in the model and divided the remaining .15 equally among tests of the remaining fixed effects, producing $α = \frac{. 15}{13} = . 0115$ . Because of the importance of identifying variation to be modeled, tests of variance component used $α = . 05$ .

There was a modest amount of missing data, generally ranging from 4% to 12% across variables, with approximately equal percentages of missing data for the treatment and control groups. Analyses of cases with complete data produced the same pattern of findings as analyses based on available data, and the latter are reported.

Power Analysis

An a priori power/sample size analysis using the Optimal Design software (Spybrook et al., 2011) that focused on testing the SBI versus control effect for cross-sectional data indicated that 82 clusters and 800 students would allow us to detect a standardized effect of .40 (a small-medium effect following Cohen, 1988) with a power exceeding .90 for an intraclass correlation of .19 (taken from Hedges & Hedberg, 2007). The effect size formula used in Optimal Design is $δ = \frac{γ_{01}}{{[τ_{00} + σ^{2}]}^{1 / 2}}$ , where $γ_{01}$ is the slope capturing the impact of the treatment, $τ_{00}$ is the intercept variance, $σ^{2}$ is the within-cluster (classroom) error variance, and $δ$ is the standardized effect size (i.e., .40) (Spybrook et al., 2011). These calculations were assumed to apply to each of the outcome variables (PPS, Ratio/Proportion subscale, Percent subscale).

Results

Descriptive Results

Descriptive results for the PPS variables by treatment are reported in Table 2. We conducted descriptive analyses that included exploring preexisting differences between the SBI and control group students and examining the correlations between measures. As expected, the SBI and control groups did not differ significantly at pretest (t = 0.15, p > .05) on the total PPS scores, the Ratio/Proportion (t = 0.76, p > .05), or the Percent subscale scores (t = 1.71, p > .05).

Table 2.

Descriptive Statistics for PPS Measure by Treatment.

	Treatment
	SBI			Control			Total
Variable	M	SD	n	M	SD	N	M	SD	N
Total PPS pretest	11.47	3.70	398	11.43	3.59	404	11.45	3.64	802
Missing			1			3			4
Total PPS posttest	13.87	4.73	381	12.69	4.11	374	13.29	4.47	755
Missing			18			33			51
Total PPS delayed	13.55	4.46	372	12.76	4.37	357	13.16	4.43	729
Missing			27			50			77
Ratio/Proportion pretest	7.92	2.93	398	8.07	2.85	404	8.00	2.89	802
Ratio/Proportion posttest	9.46	3.26	381	8.92	3.21	374	9.19	3.25	755
Ratio/Proportion delayed	9.41	3.21	372	8.89	3.29	357	9.16	3.26	729
Percent pretest	3.55	1.59	398	3.36	1.63	404	3.46	1.61	802
Percent posttest	4.41	2.25	381	3.77	1.83	374	4.09	2.08	755
Percent delayed	4.14	2.02	372	3.87	1.85	357	4.01	1.95	729

Note. PPS = proportional problem-solving; SBI = schema-based instruction. All test statistics are based on the total number of items correct; the maximum possible points for the total score, ratio and proportion subscore, and percent subscore are 31, 17, and 14, respectively.

Results of the bivariate correlation analyses showed that the correlations between the total PPS pretest–posttest, pretest–delayed posttest, and posttest–delayed posttest were 0.52, 0.49, and 0.65, respectively. For the PPS subscales, the correlations between the PPS Ratio/Proportion pretest–posttest, pretest–delayed posttest, and posttest–delayed posttest were 0.51, 0.49, and 0.60, respectively; the correlations between the PPS Percent pretest–posttest, pretest–delayed posttest, and posttest–delayed posttest were 0.24, 0.22, and 0.45, respectively.

Treatment Effects on PPS

Tables 3 and 4 show the HLM results for the PPS total score posttest and delayed posttest. Figure 2 shows percentage correct improvement from pre- to posttreatment on the MPS posttest and delayed posttest.

Table 3.

HLM Analyses of PPS Posttest.

	Total			Ratio/proportion			Percent
Fixed effect	B	SE	p	B	SE	p	B	SE	p
Between-student model (n = 725)
Sex	0.25	0.271	.360	−0.03	0.198	.881	0.30	0.147	.041
Asian	0.28	0.565	.619	0.11	0.412	.787	0.15	0.306	.632
Black	−0.55	0.447	.218	−0.43	0.325	.186	−0.12	0.242	.607
Hispanic	0.14	0.506	.777	−0.15	0.368	.676	0.17	0.273	.526
Pretest	0.53	0.040	<.001	0.49	0.036	<.001	0.24	0.047	<.001
Between-classroom model (n = 82)
Intercept	14.16	0.903	<.001	9.50	0.634	<.001	4.60	0.445	<.001
Treatment	1.17	0.391	.004	0.62	0.274	.026	0.63	0.193	.002
Sex	−0.11	0.422	.800	0.28	0.296	.352	−0.31	0.208	.140
Math courses	0.03	0.050	.590	0.02	0.035	.486	0.02	0.025	.545
Education courses	−0.05	0.052	.360	−0.02	0.036	.557	−0.02	0.026	.476
Years experience	−0.01	0.033	.663	−0.00	0.023	.975	−0.02	0.016	.360
PD hours	−0.00	0.007	.636	−0.00	0.005	.618	−0.00	0.004	.675
FRL	−0.17	0.176	.339	−0.05	0.124	.667	−0.11	0.087	.196
LEP	−0.48	0.161	.004	−0.39	0.113	.001	−0.13	0.079	.096
SpEd	−0.01	0.147	.957	−0.06	0.103	.565	0.00	0.073	.951
Random effect		VC	SD	 ²		df		p
Total	Classroom	1.20	1.09	151.29		72		<.001
	Student	12.36	3.52
Ratio/Proportion	Classroom	0.53	0.73	139.00		72		<.001
	Student	6.59	2.57
Percent	Classroom	0.21	0.46	128.90		72		<.001
	Student	3.70	1.92

Note. VC = variance component. Sex (1 = female, 0 = male), Asian (1 = yes, 0 = no), Black (1 = yes, 0 = no), Hispanic (1 = yes, 0 = no), PPS pretest, and PPS posttest are student-level variables; treatment (1 = SBI, 0 = Control), sex (1 = female, 0 = male), math courses (number of undergraduate/graduate mathematics courses), education courses (number of undergraduate/graduate pedagogy courses), years experience (years teaching experience in mathematics), PD hours (number of professional development hours in mathematics in the last year), FRL (% free/reduced price lunch in quintiles), LEP (% LEP in quintiles), and SPED (% special education in quintiles) are teacher-level variables.

Table 4.

HLM Analyses of PPS Delayed Posttest.

	Total				Ratio/proportion				Percent
Fixed effect	B	SE	p	B	SE	p	B	SE	p
Between-student model (n = 702)
Sex	−0.13	0.282	.638	−0.28	0.205	.173	0.18	0.142	.195
Asian	0.81	0.599	.179	0.66	0.436	.132	0.11	0.300	.702
Black	−0.05	0.477	.913	0.11	0.347	.752	−0.17	0.239	.481
Hispanic	0.10	0.529	.854	−0.02	0.385	.966	−0.01	0.264	.974
Pretest	0.51	0.041	<.001	0.48	0.037	<.001	0.21	0.046	<.001
Between-classroom model (n = 80)
Intercept	14.00	0.962	<.001	9.97	0.709	<.001	4.04	0.432	<.001
Treatment	0.93	0.404	.025	0.81	0.298	.008	0.20	0.182	.286
Sex	0.24	0.451	.599	0.07	0.333	.825	0.19	0.202	.360
Math courses	0.02	0.053	.751	−0.00	0.039	.988	0.02	0.024	.300
Education courses	−0.00	0.053	.993	−0.02	0.039	.620	0.03	0.024	.273
Years experience	−0.01	0.034	.681	0.00	0.025	.893	−0.02	0.015	.252
PD hours	−0.01	0.008	.390	−0.01	0.006	.137	0.00	0.003	.726
FRL	−0.21	0.182	.261	−0.07	0.134	.617	−0.14	0.082	.093
LEP	−0.39	0.166	.023	−0.35	0.122	.005	−0.06	0.074	.443
SPED	−0.08	0.153	.589	−0.15	0.113	.198	0.01	0.069	.866
Random effect		VC	SD	 ²		df		p
Total	Classroom	1.26	1.12	145.93		70		<.001
	Student	12.91	3.59
Ratio/Proportion	Classroom	0.71	0.84	153.21		70		<.001
	Student	6.81	2.61
Percent	Classroom	0.18	0.42	117.71		70		<.001
	Student	3.31	1.82

Note. Sex (1 = female, 0 = male), Asian (1 = yes, 0 = no), Black (1 = yes, 0 = no), Hispanic (1 = yes, 0 = no), PPS pretest, and PPS posttest are student-level variables; treatment (1 = SBI, 0 = Control), sex (1 = female, 0 = male), math courses (number of undergraduate/graduate mathematics courses), education courses (number of undergraduate/graduate pedagogy courses), years experience (years teaching experience in mathematics), PD hours (number of professional development hours in mathematics in the last year), FRL (% free/reduced price lunch in quintiles), LEP (% LEP in quintiles), and SPED (% special education in quintiles) are teacher-level variables.

Figure 2.

Proportional Problem-Solving Posttest and Delayed Posttest Scores by Condition

PPS total score posttest

The ICC for the unconditional model was .21 and was statistically significant (p < .001). Thus 21% of the variation in the PPS total score posttest was between classrooms, a value consistent with results for mathematics scores reported in Hedges and Hedberg (2007). The two-level model that was fitted had five student predictors (gender, Black, Asian, Hispanic, PPS pretest), which were grand-mean-centered, and seven teacher predictors (treatment, number of undergraduate/graduate mathematics courses, number of undergraduate/graduate pedagogy courses, % eligible for free/reduced price lunch in quintiles, % limited English proficient [LEP] in quintiles, % special education in quintiles), which were not centered.

The percentage of free/reduced price lunch, LEP, and students qualified for special education services were aggregated to the classroom level (e.g., % LEP students in a classroom) because for many classrooms there was little or no variation on these characteristics (e.g., one half of the classrooms had no LEP students), compromising estimation of model parameters within each classroom. Moreover, the distributions of these percentages were ragged and discontinuous and were rescaled to quintiles and the rescaled versions used as classroom-level predictors. Only intercepts were modeled at Level 2 because slope variances were statistically equal to zero. The HLM results for the PPS posttest are reported in Table 3.

Table 3 shows that with other predictors held constant, treatment was a significant predictor of PPS total posttest scores ( ${\hat{γ}}_{01}$ = 1.17), which translates to SBI students scoring on average 0.32 SDs higher than those in the control group, and a 22% reduction in the conditional intercept variance attributable to treatment. That is, treatment accounted for 22% of intercept variance beyond that attributable to other predictors in the model (Raudenbush & Bryk, 2002). Another significant result in Table 3 was for the classroom LEP variable (expressed in quintiles; ${\hat{γ}}_{08}$ = −0.48, −0.11 SD), which shows that moving from one LEP quintile to the next was associated with an expected decline of 0.48 points in classroom PPS total score posttest intercepts (i.e., classrooms with higher concentrations of LEP students were associated with lower PPS total posttest scores). PPS pretest was also a significant predictor ( ${\hat{γ}}_{50}$ = 0.53, 0.12 SD), meaning that increases in pretest scores were associated with higher posttest scores. The remaining effects in Table 3 were not significant.

Analyses of the PPS Ratio/Proportion and Percent subscales for the fully unconditional model produced similar variation between classrooms (19%, 13%), similar treatment effects (0.62, 0.23 SD; 0.63, 0.32 SD), and similar variance attributable to treatment (16%, 29%). LEP was a significant predictor of the ratio/proportion outcome (-0.39, -0.12) but not the percent outcome.

PPS posttest–delayed total score

The ICC for the unconditional model was .19 and was statistically significant (p < .001). Thus 19% of the variation in PPS posttest–delayed total score was between classrooms. The same predictors fitted to the PPS total posttest score data were used and the results are reported in Table 4. Once again, none of the student model predictors had slopes that varied significantly across classrooms and only the classroom intercept model had predictors.

The treatment effect was significant ( ${\hat{γ}}_{01}$ = 0.93) and translates to SBI students scoring on average 0.25 SDs higher than those in the control group or treatment accounted for 16% of intercept variance beyond that attributable to other predictors in the model. The PPS pretest was again a significant predictor ( ${\hat{γ}}_{50}$ = 0.51).

Treatment was also a significant predictor of the Ratio/Proportion subscale on the PPS delayed posttest $({\hat{γ}}_{01} = 0.81, 0.30 SD)$ , with 14% of the variance due to treatment (ICC = 18% for fully unconditional model). LEP was again a significant predictor (−0.35, −0.11 SD), with increases in the concentration of LEP students in a classroom associated with a decline in average performance on the delayed posttest for the Ratio/Proportion subtest. However, treatment was not a significant predictor of percent scores on the PPS delayed posttest.

Longitudinal analyses of PPS

We explored the ability of treatment to moderate student growth over time by fitting a three-level model in which Level 1 (repeated measures) was within-students and consisted of the PPS pretest, posttest, and delayed–posttest total scores; Level 2 was between students; and Level 3 was between classrooms. The average growth rate (linear slope) was 0.77 (p < .001), indicating that on average students’ PPS scores increased over time. Growth rates did not vary significantly between students, and thus, treatment did not moderate student growth. The only significant effect in this analysis indicated that Hispanic students on average scored about 1.25 points lower on the PPS pretest than White students.

TARF-R

The mean score for SBI students with MD-PS was 3.16 (SD = 0.66) on diagrams and 2.59 (SD = 0.58) on the problem-solving procedure. On the multiple solution strategies item, the mean score was 2.95 (SD = 0.98)

Teacher satisfaction survey

The mean scores were 3.34 (SD = 0.35) for the total score, 3.41 (SD = 0.42) for the overall benefits of the SBI intervention, 3.54 (SD = 0.42) for diagrams, and 3.12 (SD = 0.47) for problem-solving procedures.

Discussion

The goal of this retrospective randomized controlled design study was to determine whether the positive effects of the SBI intervention in the previous efficacy study (Jitendra et al., 2015) would uphold for students with MD-PS. Overall, our SBI intervention, which targeted understanding of ratios and proportional relationships, significantly improved PPS outcomes in students at risk for MD-PS at both immediate and delayed posttests. Students in treatment classes performed approximately one third of a standard deviation higher than students receiving typical mathematics instruction in the same content on the PPS total score at immediate posttest. This translates to approximately 63% of treatment classrooms scoring above the mean of control classrooms (Lipsey et al., 2012). It was also encouraging that treatment students maintained these gains 9 weeks following the intervention. Students in treatment classes performed one quarter of a standard deviation higher than students receiving typical mathematics instruction on the PPS total score at delayed posttest, meaning that approximately 60% of treatment classrooms scored above the mean of control classrooms (Lipsey et al., 2012). As predicted, our findings revealed that the SBI intervention’s focus on promoting deep conceptual and procedural knowledge was effective in enhancing the PPS performance of students at risk for MD-PS. These findings expand previous results showing that proportional reasoning can be supported effectively by the SBI intervention. Although the effects in this study are smaller than those in Jitendra et al. (2015; g = 0.46 and 0.32 for posttest and delayed posttest) and Jitendra et al. (in press; g = 0.40 and 0.42 for posttest and delayed posttest), the effect sizes of .25 and above are considered substantively important (see What Works Clearinghouse, 2014).

We were also interested in knowing whether the findings would provide support for the SBI treatment effects in improving students’ performance on both Ratio and Proportion and Percent PPS subscales. Although the SBI group outperformed the control group on the Ratio and Proportion subscale at both posttest and delayed posttest (.23 SDs and .30 SDs higher than the control group), we found treatment effects only at immediate posttest (.32 SDs higher than the control group) for the Percent subscale. These findings partially support our hypothesis that the SBI intervention’s focus on effective instructional practices (e.g., activating the mathematical structure of problems, representing the problem situation using an appropriate diagram, monitoring and reflecting on the problem-solving processes) would have similar positive effects on improving students’ scores on the PPS subscales. Our results build on and contrast those obtained by Jitendra et al. (in press), who found positive treatment effects for students at risk for both MD only and MDRD on the Ratio and Proportion subscale (g = .29) at delayed posttest only and for the Percent subscale at both posttest (g = 0.42) and delayed posttest (g = 0.39). It should be noted, however, that the current study is not a direct comparison to Jitendra et al. (in press), which used scores on end-of-the-year state mathematics and reading achievement to identify students with MD only and MDRD. In contrast, we used cutoff scores on a diagnostic measure of mathematical problem solving to specifically identify risk factors related to problem solving. Students’ difficulties with percent word problems in our study may be due to several reasons. In particular, it is well known that within the larger category of proportion reasoning problems, one particularly troublesome topic for students is percent (Lembke & Reys, 1994; Parker & Leinhardt, 1995). The many different ways that mathematically similar percent problems can be expressed highlights the importance of looking beyond surface features of word problems to identify and analyze underlying mathematical relations (Marshall, 1995). As such, additional focus and careful, targeted instruction of percent problems may be needed to demonstrate stability of acquisition of percent problems for students at risk for MD-PS receiving the treatment.

We also examined whether treatment moderated student growth over time. Although average student’s PPS scores increased over time, growth rates did not vary significantly between students and thus were unrelated to being in the treatment or control condition. The findings did not support our hypothesis that SBI would have a pronounced positive effect on a student’s growth over time. Given that students in the treatment condition did not significantly outperform students in the control condition on percent problem solving at delayed posttest, it could possibly explain the lack of difference in growth rates over time. More intensive attention to percent problem solving and the supports students require to sustain the gains over time may be needed.

We also documented fidelity of implementation in both treatment and control classes. The data suggest that the SBI treatment was implemented within a middle range, and the evidence for the presence of the SBI elements in the comparison classes was in the low range, confirming that the SBI treatment caused improvement in students’ PPS performance.

Limitations and Implications for Future Research and Practice

The findings of this study suggest the efficacy of the SBI intervention to improve proportional reasoning skills for middle school students. The consistent findings for improved proportional reasoning skills from this study and two other studies examining SBI implementation in general education and special education middle school classrooms for students at risk for MD (Jitendra et al., in press; Xin et al., 2005) provide an evidence base for SBI as a useful practice.

However, several limitations of the study require caution in interpreting the findings. One limitation that should be considered is the issue of low reliability estimates for the outcome measure, especially the pretest. Although the reliability estimates for the larger sample in Jitendra et al. (2015) were adequate, the low reliability for the current study sample may be due to the homogeneity of the sample of students at risk for MD. The reliability coefficients for our sample, although sufficient for making group comparisons, may underestimate the treatment effects (see Thorndike, 1977). That is, low reliability may not be a concern especially in light of the significant treatment effects we found in this study.

Another potential limitation is that only one lesson per teacher was videotaped to assess fidelity. However, it is worth noting that given the relatively brief period of the intervention (6 weeks), one video-recorded observation may be sufficient to provide a representative sample of participant functioning (Breitenstein et al., 2010). Unlike direct observation or audiotaping of lessons, video-recorded data provided several important advantages (e.g., ability to capture complex interactions, allow multiple viewings) and helped us maintain the quality of the coding that was done.

Our study did not address exactly what features of the SBI intervention were most effective in helping students develop proportional reasoning skills. Although effective instructional components and appropriate scaffolds (e.g., schematic diagrams, problem-solving checklists) might account for SBI’s positive effect for students with MD-PS in this study, it is not known whether certain components are more relevant for some students than others. Effectively scaffolding the representation and solution processes by providing diagrams that are appropriate for the task and teaching students to represent the problem situation as well as using problem-solving checklists to support them as they solved problems may have reduced the cognitive load placed on students with MD-PS (Berends & van Lieshout, 2009; Lee, Ng, & Ng, 2009). Evidence of the benefits of diagrams and checklists was demonstrated through SBI students’ ratings. Although students liked using a variety of strategies to solve the problems, it is not known whether teaching multiple solution strategies placed additional demands on students with MD-PS. Future research could test individual components for effectiveness so that interventions would include only the most effective components.

This study occurred over a relatively brief period (6 weeks of daily instruction, 5 days a week) and did not meet the recommendation in the best evidence syntheses of a 12-week criterion to ensure external validity (Slavin, 2008). Even though the duration of our study is consistent with the time commonly allocated to the topics of ratio, proportion, and percent in middle schools, the differential results for ratio and proportion and percent problem solving on the delayed posttest indicates a need for longer duration of the intervention in sustaining the SBI effects particularly for percent problem solving. As such, future research examining impacts from longer implementations for students with MD-PS is warranted.

The findings of the study have potentially important implications for practice. First, this study provides evidence of the promise of the SBI practices, including recognizing the problem structure, using schematic diagrams to illustrate the mathematical relations among key elements in the problem, monitoring and reflecting on the problem-solving process, and using a variety of strategies, for improving student learning. The findings suggest that when teachers incorporate these practices in their instruction, it can result in important and sustaining benefits for their students, including students with MD-PS. Second, it is important to try to understand why SBI did not moderate student growth over time. Results suggest that the percent content was complex and may have implications for the design of interventions to meet the needs of students with MD-PS. This could include intensive instruction (i.e., small group instruction, sustained and elaborated instruction) for students with MD-PS to master this content.

In conclusion, our SBI intervention approach is effective for enhancing the ability of seventh-grade students at risk for MD-PS to master PPS. Given the context of the “Common Core” framework of high standards, this study provides evidence that careful, targeted instruction of complex content and ambitious mathematics practices may need to occur over a sustained period of time for students with MD-PS to benefit from the treatment.

Footnotes

Authors’ Notes

We wish to acknowledge Susan Slater, the project coordinator, and research assistants Amy Lein and Gregory Simonson for their contributions to data collection as well as thank the seventh-grade math teachers and students throughout Minnesota who participated in the study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported here was supported by Grant R305A110358 from the Institute of Education Sciences, U.S. Department of Education. The ideas, findings, and conclusions or recommendations in this paper are those of the authors and do not represent the official views of the Institute of Education Sciences or the U.S. Department of Education.

References

Adjiage

Pluvinage

(2007). An experiment in teaching ratio and proportion. Educational Studies in Mathematics, 65, 149–175. doi: 10.1007/s10649-006-9049-x

Ahl

V. A.

Moore

C. F.

Dixon

J. A.

(1992). Development of intuitive and numerical proportional reasoning. Cognitive Development, 7, 81–108.

Beckmann

Izak

(2015). Two perspectives on proportional relationships: Extending complementary origins of multiplication in terms of quantities. Journal for Research in Mathematics Education, 46, 17–38.

Berends

I. E.

van Lieshout

E. M.

(2009). The effect of illustrations in arithmetic problem-solving: Effects of increased cognitive load. Learning and Instruction, 19, 345–353.

Berk

Taber

S. B.

Gorowara

C. C.

Poetzl

(2009). Developing prospective elementary teachers’ flexibility in the domain of proportional reasoning. Mathematical Thinking and Learning: An International Journal, 11, 113–135.

Bottge

B. A.

Toland

M. D.

Gassaway

Butler

Choo

Griffen

A. K.

(2015). Impact of enhanced anchored instruction in inclusive math classrooms. Exceptional Children, 81, 158–175. doi:10.1177/0014402914551742

Breitenstein

S. M.

Gross

Garvey

C. A.

Hill

Fogg

Resnick

(2010). Implementation fidelity in community-based interventions. Research in Nursing & Health, 33, 164–173. doi: 10.1002/nur.20373

Briars

D. J.

Larkin

J. H.

(1984). An integrated model of skill in solving elementary word problems. Cognition and Instruction, 1, 245–296.

Carpenter

T. P.

Hiebert

Moser

J. M.

(1981). Problem structure and first-grade children’s initial solution processes for simple addition and subtraction problems. Journal for Research in Mathematics Education, 12, 27–39. Retrieved from http://www.jstor.org/stable/748656

10.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Lawrence Erlbaum.

11.

Connolly

A. J.

(2007). KeyMath-3 diagnostic assessment: Manual forms A and B. Minneapolis, MN: Pearson.

12.

Dane

A. V.

Schneider

B. H.

(1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45. doi: 10.1016/S0272-7358(97)00043-3

13.

Dunn

T. J.

Baguley

Brunsden

(2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology (Early View version). doi: 10.1111/bjop.12046

14.

Fennema

Carpenter

T. P.

Peterson

P. L.

(1989). Learning mathematics with understanding: Cognitively guided instruction. In Brophy

J. E.

(Ed.), Advances in research on teaching (pp. 195–221). Greenwich, CT: JAI press.

15.

Fuchs

L. S.

Schumacher

R. F.

Sterba

S. K.

Long

Namkung

Malone

… Changas

(2014). Does working memory moderate the effects of fraction intervention? An aptitude–treatment interaction. Journal of Educational Psychology, 106, 499–514. doi: 10.1037/a0034341

16.

Fujmura

(2001). Facilitating children’s proportional reasoning: A model of reasoning processes and effects of intervention on strategy change. Journal of Educational Psychology, 93, 589–603. doi: 10.1037/0022-0663.93

17.

Gersten

Beckmann

Clarke

Foegen

Marsh

Star

J. R.

… Witzel

(2009). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools (NCEE 2009–4060). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/publications/practiceguides/

18.

Gersten

Chard

D. J.

Jayanthi

Baker

S. K.

Morphy

Flojo

(2009). Mathematics instruction for students with learning disabilities: A meta-analysis of instructional components. Review of Educational Research, 79, 1202–1242.

19.

Hedges

L.V.

Hedberg

E. C.

(2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29, 60–87. doi: 10.3102/0162373707299706

20.

Jitendra

A. K.

Dupuis

D. N.

Rodriguez

M. C.

Zaslofsky

A. F.

Slater

Cozine-Corroy

Church

(2013). A randomized controlled trial of the impact of schema-based instruction on mathematical outcomes for third grade students with mathematics difficulties. The Elementary School Journal, 114, 252–276.

21.

Jitendra

A. K.

Dupuis

D. N.

Star

J. R.

Rodriguez

M. C.

(in press). The effects of schema-based instruction on the proportional thinking of students with mathematics difficulties with and without reading difficulties. Journal of Learning Disabilities. Advance online publication. doi: 10.1177/0022219414554228

22.

Jitendra

A. K.

Griffin

McGoey

Gardill

Bhat

Riley

(1998). Effects of mathematical word problem solving by students at risk or with mild disabilities. Journal of Educational Research, 91, 345–356.

23.

Jitendra

A. K.

Harwell

M. R.

Dupuis

D. N.

Karl

S. R.

Lein

A. E.

Simonson

Slater

S. C.

(2015). Effects of a research-based mathematics intervention to improve seventh-grade students’ proportional problem solving: A cluster randomized trial. Journal of Educational Psychology, 107, 1019–1034. doi: 10.1037/edu0000039

24.

Jitendra

A. K.

Star

J. R.

(2012). An exploratory study contrasting high- and low-achieving students’ percent word problem solving. Learning and Individual Differences, 22, 151–158.

25.

Jitendra

A. K.

Star

J. R.

Dupuis

D. N.

Rodriguez

M. C.

(2013). Effectiveness of schema-based instruction for improving seventh-grade students’ proportional reasoning: A randomized experiment. Journal of Research on Educational Effectiveness, 6, 114–136. doi: 10.1080/19345747.2012.725804

26.

Jitendra

A. K.

Star

J. R.

Rodriguez

Lindell

Someki

(2011). Improving students’ proportional thinking using schema-based instruction. Learning and Instruction, 21, 731–745. doi: 10.1016/j.learninstruc.2011.04.002

27.

Jitendra

A. K.

Star

Starosta

Leh

Sood

Caskie

Mack

T. R.

(2009). Improving students’ learning of ratio and proportion problem solving: The role of schema-based instruction. Contemporary Educational Psychology, 34, 250–264. doi: 10.1016/j.cedpsych.2009.06.001

28.

Jitendra

A. K.

Woodward

Star

J. R.

(2011). Middle school students’ thinking about ratios and proportions. In Gersten

Newman-Gonchar

(Eds.), Understanding RTI in mathematics: Proven methods and applications (pp. 127–150). Baltimore: Paul H. Brookes.

29.

Lamon

S. J.

(1999). Teaching fractions and ratios for understanding: Essential content knowledge and instructional strategies for teachers. Mahwah, NJ: Erlbaum.

30.

Lamon

S. J.

(2007). Rational numbers and proportional reasoning: Toward a theoretical framework for research. In Lester

F. K.

(Ed.), Second handbook of research on mathematics teaching and learning (pp. 629–667). Charlotte, NC: Information Age Publishing.

31.

Lee

S. F.

(2009). The contributions of working memory and executive functioning to problem representation and solution generation in algebraic word problems. Journal of Educational Psychology, 101, 373–387. doi: 10.1037/a0013843

32.

Lembke

L. O.

Reys

B. J.

(1994). The development of, and interaction between, intuitive, and school-taught ideas about percent. Journal for Research in Mathematics Education, 25, 237–259.

33.

Lipsey

M. W.

Puzio

Yun

Hebert

M. A.

Steinka-Fry

Cole

M. W.

Roberts

Anthony

K. S.

Busick

M. D.

(2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms (NCSER 2013-3000). Washington, DC: National Center for Special Education Research, Institute of Education Sciences, U.S. Department of Education.

34.

Lobato

Ellis

A. B.

Charles

R. I.

Zbiek

R. M.

(2010). Developing essential understanding of ratios, proportions & proportional reasoning. Reston, VA: National Council of Teachers of Mathematics.

35.

Marshall

S. P.

(1995). Schemas in problem solving. New York: Cambridge University Press.

36.

Mayer

R. E.

(1999). The promise of educational psychology Vol. I: Learning in the content areas. Upper Saddle River, NJ: Merrill Prentice Hall.

37.

Mazzocco

M. M. M.

(2007). Defining and differentiating mathematical learning disabilities and difficulties. In Berch

D. B.

Mazzocco

M. M.M.

(Eds.), Why is math so hard for some children? The nature and origins of mathematical learning difficulties and disabilities (pp. 29–47). Baltimore: Paul Brookes.

38.

McDonald

R. P.

(1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates.

39.

Meyer

J. P.

(2011). jMetrik: A free and open source software program for comprehensive psychometric analysis [Version 2.1.0] [computer software]. Charlottesville: University of Virginia.

40.

Miller

M. C.

(1966). Simultaneous statistical inference. New York: McGraw-Hill.

41.

National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for Mathematics. Washington, DC: Authors.

42.

National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the National Mathematics Advisory Panel. Washington, DC: U.S. Department of Education.

43.

O’Donnell

(2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of Educational Research, 78, 33–84. doi:10.3102/0034654307313793

44.

Parker

Leinhardt

(1995). Percent: A privileged proportion. Review of Educational Research, 65, 421–481.

45.

Pearson Education. (2004). Group mathematics assessment and diagnostic evaluation (GMADE). Boston: Author.

46.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.

47.

Raudenbush

S. W.

Bryk

A. S.

Congdon

(2004). HLM 6.02: Hierarchical linear and nonlinear modeling [computer software]. Lincolnwood, IL: Scientific Software International.

48.

Reimers

T. M.

Wacker

D. P.

(1988). Parents’ ratings of behavioral treatment recommendations made in an outpatient clinic: A preliminary analysis of the influence of treatment effectiveness. Behavior Disorders, 14, 7–15.

49.

Revelle

Zinbarg

R. E.

(2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74, 145–154. doi:10.1007/s11336-008-9102-z

50.

Riley

M. S.

Greeno

J. G.

Heller

J. I.

(1983). Development of children’s problem-solving ability in arithmetic. In Ginsburg

H. P.

(Ed.), The development of mathematical thinking (pp. 153–196). New York: Academic Press.

51.

Slavin

R. E.

(2008). Perspectives on evidence-based research in education—What works? Issues in synthesizing educational program evaluations. Educational Researcher, 37, 5–14.

52.

Spybrook

Bloom

Congdon

Hill

Martinez

Raudenbush

(2011). Optimal design for longitudinal and multilevel research: Documentation for the optimal design software version 3.0. Retrieved from www.wtgrantfoundation.org.

53.

Swanson

H. L.

Moran

Lussier

Fung

(2014). The effect of explicit and direct generative strategy training and working memory on word problem-solving accuracy in children at risk for math difficulties. Learning Disability Quarterly, 37, 111–123. doi: 10.1177/0731948713507264

54.

Thorndike

R. M.

(1977). Measurement and evaluation in psychology and education (6th ed.). Columbus, OH: Merrill Prentice-Hall.

55.

What Works Clearinghouse. (2014). WWC procedures and standards handbook (version 3.0). Retrieved May 27, 2014 from http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

56.

Woodward

Beckmann

Driscoll

Franke

Herzig

Jitendra

Koedinger

K. R.

Ogbuehi

(2012). Improving mathematical problem solving in grades 4 through 8: A practice guide (NCEE 2012-4055). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/PracticeGuide.aspx?sid=16

57.

Xin

Y. P.

Jitendra

A. K.

(1999). The effects of instruction in solving mathematical word problems for students with learning problems: A meta-analysis. The Journal of Special Education, 32, 207–225.

58.

Xin

Y. P.

Jitendra

A. K.

Deatline-Buchman

(2005). Effects of mathematical word problem solving instruction on students with learning problems. The Journal of Special Education, 39, 181–192.

59.

Zhang

Xin

Y. P.

(2012). A follow-up meta-analysis for word-problem-solving interventions for students with mathematics difficulties. Journal of Educational Research, 105(5), 303–318.