Abstract
Educator preparation in comprehension strategy instruction (CSI) is essential to meeting the needs of English learners (ELs) in contemporary schools. This article provides teacher educators and professional developers with concrete examples of practices to prepare classroom teachers in comprehensive strategy instruction, based on an experimental study that examined the effect (.37) of CSI on third-grade students’ (n = 78) word-problem-solving (WPS) performance. CSI included problem-solving strategies that (a) helped students to understand the relevant and irrelevant information in a word problem’s question; (b) helped students to restate the problem orally, which allowed them to understand, to construct meaning, and to clarify any misunderstanding about the question; and (c) provided instructional feedback with student collaboration for problem solving and solution. Findings indicate that a focus on CSI may help facilitate WPS skills developing for ELs at risk of mathematics-learning difficulties (MLD). Educator preparation recommendations are embedded throughout with implications for future research and practice also discussed.
Keywords
A large and increasing proportion of students in U.S. schools come from a home in which a language other than English is spoken. About 4.85 million students enrolled in public schools were not yet fully proficient in English in the 2012-2013 school year, representing nearly 10% of the total public-school student enrollment (National Center for Education Statistics [NCES], 2015). According to The Nation’s Report Card (NCES, 2015), the average mathematics score for an English learner (EL) fourth grader was 218 (with 43% below basic). The challenge for many ELs is not only overcoming a language barrier, but also achieving academically. This poor mathematics achievement places many ELs at risk of mathematics-learning difficulties (MLD) and may put them at risk of overidentification for special education (a topic that is beyond the focus of this study). ELs encounter unique mathematics-learning challenges because of the range of instructional needs they require for whom mathematics content in a second language is most challenging due to (a) they are increasingly included in general education classrooms where the demands to problem solve and learn from mathematics text are substantial and (b) they are unlikely to receive inadequate mathematics instruction such as lack of exposure to comprehension strategies to improve problem-solving efficiency (e.g., Martiniello, 2008, 2009; National Mathematics Advisory Panel, 2008). ELs benefit from teachers who are prepared in comprehension strategy instruction (CSI) in problem solving so that they can access the general education mathematics curriculum.
Although there is a great deal of professional development (PD) research that supports the value of CSI in reading (e.g., Blachowicz & Ogle, 2017), much less is understood about how to prepare school professionals to use CSI with ELs in mathematics (e.g., word problems). As an example, although many school personnel (e.g., teachers, paraprofessionals, and tutors) receive PD with CSI in reading and thus building confidence in this area, very few receive preparation with CSI in problem solving and ELs (Orosco & Abdulrahim, 2017; Kong & Orosco, 2016; Orosco, 2014a, 2014b; Orosco, Swanson, O’Connor, & Lussier, 2013; Orosco, 2013). We do not yet know the PD model best suited in preparing teachers to become more responsive to their ELs’ problem-solving needs in high poverty schools. The literature reflects a variety of pedagogical perspectives and approaches, often grounded in varying philosophical and theoretical perspectives and resulting in different instructional and educational practices (e.g., Baker, Gersten, & Lee, 2002; Gersten et al., 2009; Swanson & Jerman, 2006). We still need to learn which problem-solving components are most effective and how best to prepare school professionals so that they can provide optimal instructional scaffolding to ELs. This is certainly true with respect to implementing complex comprehension strategies with word problems that can be challenging for some teachers to learn.
ELs’ Word-Problem Development and Educator Preparation
ELs are often the poorest in terms of developing word-problem-solving (WPS) skills in English (Orosco & Abdulrahim, 2017; Kong & Orosco, 2016; Orosco, 2014a, 2014b; Orosco et al., 2013; Orosco, 2013). These students also demonstrate characteristics of inactive learners who do not monitor their learning or use comprehension strategies effectively. They have not developed the cognitive awareness necessary to assess their understanding as they read word problems and to recognize when their comprehension has broken down in formulating a solution (Bresser, Melanese, & Sphar, 2009). Preparing school personnel to teach ELs how to use comprehension strategies improves their comprehension. Many of the comprehension strategies associated with the highest effect sizes for students with MLD teach students strategies that prompt them to reflect before, during, and after reading a word problem (e.g., Gersten et al., 2009). These strategies ask students to (a) think about the question that the word problem is asking, (b) find and understand the relevant/irrelevant information in the word problem, and (c) collaborate with other students for accuracy and practicing applying the abovementioned problem-solving skills for solution. It is critical for teachers of ELs with problem-solving challenges to acquire strategies to help them understand word problems because (a) they are increasingly included in general and special education elementary classrooms where the demands to read and learn from mathematics text are substantial and (b) they are unlikely to receive intervention support in mathematics content.
Also, emerging mathematics research with teachers of ELs indicates that instruction in oral language development (e.g., vocabulary skills and math content knowledge) can affect WPS comprehension levels equivalent to native English speakers (Orosco & Abdulrahim, 2017; Kong & Orosco, 2016; Orosco, 2014a, 2014b; Orosco et al., 2013; Orosco, 2013). Well-developed English language proficiency may be a critical first step to improving WPS skills in ELs. Specifically, providing direct and explicit instruction that (a) provides modeling and oral language development with evidence-based math components, (b) provides questioning support that assists students in understanding and answering word-problem questions, and (c) providing engaging and motivating collaborative practices, which not only get students involved in all mathematics activities but also allow them to practice skills-based instruction by teaching these to one another orally.
In summary, ELs can benefit from teachers prepared with CSI that help them with their problem-solving skills. Thus, the purpose of this study was to solve a critical teacher preparation problem in mathematics. Building on what is already known from related bodies of mathematics research and following state-of-the-art research procedures, this study developed an educator preparation model applying CSI by tutors for ELs with MLD.
The Challenge
In spite of the growing problem-solving research on educating students who struggle with word problems, there is surprisingly little research on how teachers provide CSI with word problems in ELs with MLD (Orosco & Abdulrahim, 2017). One reason may be that we still need to learn for ELs, and in general, what instructional scaffolding models work best for delivering mathematics strategies. Much of the research implemented in this area has focused on English-dominant speaking students (Goldenberg, 2013). Few studies have examined the relation of teaching mathematics strategies to see if they are effective with ELs (Authors, 2016). Research, and associated educator preparation, is needed that identifies and explicates instructional scaffolding practices with mathematics strategies that provide sufficient opportunity for repeated practice with specific feedback and support upon which to improve student WPS achievement (National Mathematics Advisory Panel, 2008). Also, it is important to note that the review of the mathematics strategy literature examined for cultural relevancy found little application of this construct (partly due to the difficulty of interpreting and applying this construct in experimental design) in the problem-solving literature. The authors of this study were aware of this, and because of this, this study’s intervention lacked cultural relevancy yet it was important to conduct an experimental study applying CSI to begin to establish a mathematics research foundation on ELs in the United States as it is often recommended for all students.
Although there is a dearth of WPS experiment research on ELs with MLD (e.g., Orosco et al., 2013; Orosco, 2014a, 2014b; Kong & Orosco, 2016), there is a vast literature on effective evidence-based teaching practices with English-speaking students likely to result in improving mathematics achievement, and WPS development in ELs via teacher preparation. Several recent studies have found support for teaching strategies to improve mathematics achievement, and these instructional scaffolding behaviors can be applied in promoting WPS skills in students with mathematics challenges and learning difficulties. Intervention research has identified core instructional methods effective for students with MLD (Gersten et al., 2009): (a) strategy training with problem representation, (b) modeling with irrelevant and relevant mathematics information, and (c) providing corrective feedback and student verbalization with small group collaboration for problem-solving solution. That is, these intervention studies support explicit instruction that provide (a) instructional strategies designed to create a connection between verbal input and conceptual understanding by creating a mental model (e.g., Kolloffel, Eysink, de Jong, & Wilhelm, 2009; Xin, Jitendra, & Deatline-Buchman, 2005), (b) verbal strategies that teach conceptual understanding of irrelevant and relevant concepts and principles of word problems (e.g., Griffin & Jitendra, 2009; Krawec, Huang, Montague, Kressler, & de Alba, 2013), and (c) instructional feedback with student collaboration and verbalization with the problem-solving process (e.g., Swanson, Moran, Bocian, Lussier, & Zheng, 2013; Swanson, Moran, Lussier, & Fung, 2014).
To examine this topic, a study was completed to investigate the effects of CSI on mathematical WPS achievement via an instructional scaffolding model in third-grade ELs at risk of MLD.
Conceptual Framework
In considering the importance of the use of the research-based practice provided by direct and explicit instructional scaffolding approaches (e.g., Gersten et al., 2009; Kolloffel et al., 2009; Swanson et al., 2014), EL mathematics research indicates that teachers must be prepared to help ELs take this information and create a problem-solving process as illustrated in Figure 1 (Orosco et al., 2013; Orosco, 2014a, 2014b; Orosco & Abdulrahim, 2017). As shown, teachers must provide the instructional scaffolding that (a) teaches the language of mathematics (e.g., vocabulary) so that students can contextualize and bring meaning to this language, (b) teaches students to monitor comprehension when WPS challenges arise, and (c) provides collaborative learning practices so that ELs practice their language and problem-solving skills (Kong & Orosco, 2016; Orosco & Abdulrahim, 2017). The teaching of WPS in a second language is a complex task that requires integrating WPS theory and instructional pedagogy. Therefore, conceptually, WPS for ELs is represented as a two-part comprehension model (e.g., Kintsch & Greeno, 1985). Phase one of WPS comprehension involves the translation of printed words into verbal input (e.g., linguistic processing and reading) involving both accuracy and efficiency. Problem translation is the ability to understand the relevant and irrelevant information in a problem, which can also be seen as text comprehension. As an example, as ELs are learning English as a second language, they may need to be taught to restate the problem orally, which helps them understand, construct meaning, and clarify any misunderstanding about the word problem through instructional scaffolding dialogue. The second conceptual phase (i.e., abstract-problem representation) reflects the cognitive processes, mathematics skills, and knowledge involved in understanding the word problem. In this vein, teachers must provide the instructional scaffolding that allows the student to comprehend the word-problem information and translating this into a written statement for a solution. Finally, WPS via instructional scaffolding can be improved through collaboration with teacher support or collaboration with more capable peers for accuracy and practice applying word-problem strategies for solution. During this time, teachers scaffold ELs how to provide feedback to each other and check for understanding by generating and answering questions about each other’s problem-solving process to assigned word problems (Kong & Orosco, 2016; Orosco & Abdulrahim, 2017). The conceptual model serves as (a) a foundation for examining ELs’ mathematics comprehension skills and (b) core content for educator preparation.
WPS conceptual framework.
Method
Seventy-eight third-grade Hispanic EL children (42 boys, 36 girls) at risk of MLD from a west coast state public urban school district participated in this study and were assigned to treatment (n = 48) or control (n = 30) condition within each classroom. The school district’s population consisted of 55% Hispanic (all Latino ELs), 22% African American, 14% White, 5% Asian, and 4% Other. The school district was considered a high-poverty school, as it had approximately 75% of its population in the free or reduced-price lunch program. The term Latino EL is used because the student population had been identified as coming from Latin American descendants (e.g., Mexican, Mexican American), and they were acquiring English as a second language. The school district’s instructional curriculum functioned as English as a second language program. Given that English as a second language instruction was used with the population, this project examined the use of CSI in English. However, though beyond the scope of this project, additional research in bilingual settings would inform effects of CSI use in Spanish. Also, because the school functioned as an English as a second language program, the participant’s school district did not disclose native language proficiency scores. The English level of the participants was categorized by their state’s english language development test (ELDT) as EL intermediate. The ELDT is a standardized measure of English listening, speaking, reading, and writing with reliability scores between 0.73 and 0.94 across elementary grade levels. According to the ELDT, students at the EL intermediate level begin to tailor their English-language skills to meet communication and learning demands with increasing accuracy. In addition, ELs at this stage of English development are able to identify and understand more concrete details, some major abstract concepts and some details of extended discourse in English. They are able to respond with increasing ease to more varied communication and learning demands with a reduced number of errors. Oral and written production has usually expanded to sentences, paragraphs, and original statements and questions in the English.
Identification of EL at Risk of MLD
This study followed the general agreement by researchers that it is best to use an absolute definition of MLD (cutoff score on achievement) rather than discrepancy between achievement and IQ (Fletcher et al., 1989; Siegel & Ryan, 1989). In determining the at-risk criteria for MLD for the chosen intervention sample, this study considered students who (a) had continued to experience WPS challenges and who had not responded well to general mathematics instruction, district, and state mandated tests over a 3-year period; and (b) who had performed below the 25th percentile on a standardized WPS achievement measure (e.g., see Table 1 for a description). In addition, students were considered for the sample if Spanish was spoken as their native language, as determined by the school’s home language survey and parent consent.
Pretest Information for Children by Treatment Conditions.
Note. EL = English learner; WIAT = Wechsler Individual Achievement Test; WRAT = Wide Range Achievement Test; KM = KeyMath Revised Diagnostic Assessment; TOMA = Test of Mathematical Abilities.
Experimental Design
General education instruction
Classroom observations indicated that general education instruction consisted of homeroom teachers using a direct instructional approach with district standards-based mathematics curriculum. Teachers followed a hierarchical instructional sequence, in which the majority of the instructional time focused on teaching basic skills (e.g., procedural skills) and less time on more complex skills such as explicitly teaching problem-solving skills (e.g., mathematics strategies). General problem-solving steps included the following: (a) understand, (b) plan, (c) carry out the plan, and (d) look back. A number of the components within the general education curriculum were also utilized in this study (e.g., checking each step of the plan, etc.). However, in contrast to general education instruction, this study’s intervention directly focused on specific components of CSI over consecutive sessions presented in a predetermined order, which is necessary to support ELs who are struggling with mathematics.
Intervention
This study’s treatment model supplemented general education mathematics teaching by providing small-group pullout instruction (three to five students per group) to the intervention group during the school day. Participants were randomly selected and categorized based on teacher recommendation (all students had low mathematics and research scores and a need for intervention), and a list was randomly generated (based on a standardized WPS scores) to assign students to a small group. Word-problem types (e.g., joining problems, separating problems, part-part-whole problems, and comparing problems) followed those used in the general-education curriculum. In this study, each participant received a booklet of 20 lessons consisting of five word problems per lesson from the general education curriculum. These word problems were verbally modified based on a scaffolding ladder with word problems that increasingly became more challenging over the course of the study. That is, in this approach the students were first taught strategies with basic word problems. As students became proficient in general problem-solving strategies and solved basic word problems correctly (e.g., 100% WPS accuracy), the intervention introduced more challenging word problems, which required further CSI. If not, the students were given more practice at the current level until 100% mastery was achieved. Because the intervention was administered in a small group, students were not moved up to the next level immediately to allow for student pairing. Fidelity checks (20% of lessons implemented) were conducted by trained graduate and undergraduate students to ensure that the students were using comprehension strategies proficiently in their notebooks. The observer coded for fidelity and scored observed or did not observe for each strategy observed in student’s notebooks. A total agreement calculation method indicated the stable presence of strategy behaviors for each lesson (97%). If fidelity of implementation fell below 80%, additional instructional scaffolding was provided to the students. Students in the treatment condition received intervention by trained tutors for 30 minutes twice a week for a period of 10 weeks, for a total of 20 lessons. Students in the control condition received 50 minutes of daily general-education mathematics direct instruction only and were not responding well to this instruction. CSI was selected as the intervention because it has been built on a foundation of direct explicit instructional scaffolding and associated with effective teaching such as collaborative group work, interactive dialogue, and procedural strategies.
Intervention procedure
In this study, trained undergraduate and graduate tutors with a background in education (e.g., elementary and special education majors) followed scripted comprehension strategies (see instructional scaffolding that follows below) with WPS practice that lasted for 30 minutes per session. Tutors participated in a 4-hour training session to become familiar with intervention materials and instructional techniques. The participants needed to see concrete examples of how mathematics strategies were related to ELs. The tutors were taught not only how to implement mathematics strategies, but also why, so that they could develop an understanding of the underlying conceptual rationale for each of the strategies and collaborative-based teaching components that made up this PD. Prior to implementation of interventions with students, tutors practiced delivering small group lessons to other tutors receiving feedback. Tutors were provided with scripted lessons (see the appendix for an example of lesson dialogue) and student booklets for implementation of lessons. In addition, tutors met with researchers to receive feedback on lesson implementation, review future lessons, and address any student-learning issues. The students discussed each strategy and if they demonstrated 100% mastery (via verbal discussion), the tutor proceeded to the next treatment. Students were provided multiple opportunities to practice the strategy to demonstrate mastery. If students did not comprehend a strategy, the tutor provided explicit instruction until students understood before providing the next strategy. Instructional scaffolding was as follows.
Pre- and Posttest Measures
Calculation accuracy
Calculation accuracy (i.e., basic mathematics skills) was assessed at both pretest and posttest. Calculation skills have been found to have a direct effect on WPS (Swanson, Jerman, & Zheng, 2008). The arithmetic computation subtest for the Wide Range Achievement Test-Third Edition (WRAT; Wilkinson, 1993) and the numerical operations subtest for the Wechsler Individual Achievement Test (WIAT; Psychological Corporation, 1992b) were administered to measure calculation ability. Two forms of the tests were counterbalanced across participants at pretest and posttest. The subtests required written computation to problems that increased difficulty. The dependent measure was the number of problems correct, which yielded a standard score (M = 100, SD = 15).
Word-problem accuracy
Two measures were individually administered to assess pre- and posttest problem solving ability: (a) the problem-solving subtests from the Test of Mathematical Abilities (TOMA; Brown, Cronin, & McIntire, 1994) and the KeyMath Revised Diagnostic Assessment (KM; Connolly, 1998). A composite score based on the summation of the two subtests’ means was created as the outcome measure of problem-solving ability.
Reading comprehension
Reading comprehension (RC) was assessed by the Passage Comprehension subtest from the Test of Reading Comprehension–Third Edition (TORC; Brown, Hammill, & Weiderholt, 1995).
Statistical analysis
Multilevel regression analysis (with pretest and RC as covariates) was used in this study to analyze treatment effects to help equate intact groups and is useful in reducing systematic bias and error variance (Bryk & Raudenbush, 2002). The fixed- and random-effect parameter estimates were obtained using PROC MIXED in SAS 9.3 (SAS Institute, 2010). A full maximum likelihood (ML) estimation was used to compute the parameters at posttest because of some attrition in sample size. Despite unequal sample sizes, the assumption of equal covariance matrices was met in all analyses. The homogeneity of regression slopes assumption was examined by testing the covariate by intervention interaction effect, and it was met in all analyses. The convergence criterion was found to be satisfied. The criterion variables in the regression analysis were grand mean centered (Singer & Willett, 2003) posttest scores for problem-solving accuracy.
Results
Table 1 presents the descriptive data of students at pretest by treatment groups, and Table 2 presents the correlation data for intervention group. No significant difference was found in terms of gender status between the treatment and control condition, χ2 (1, n = 77) = 0.11, p > .05. The composite scores for WPS accuracy were based on the mean z score computed for all three assessments (KM and TOMA).
Hierarchical Linear Models Predicting Posttest PSA.
Note. PSA = problem-solving accuracy (composite score of word-problem solving, KeyMath and TOMA); CSI = comprehension strategy instruction composite score; RC = reading comprehension; Pretest = composite score of word-problem solving (KeyMath and TOMA); AIC = Akaike’s information criterion; BIC = Bayesian information criterion; KM = KeyMath Revised Diagnostic Assessment; TOMA = Test of Mathematical Abilities.
p < .05. **p < .01. ***p < .001.
Pretest Comparisons
A MANOVA was computed among the control and treatment groups on composite pretest mean z scores for WPS accuracy assessments (KM and TOMA). The MANOVA was significant showing a WPS advantage for students in the control condition relative to student in treatment condition, Wilks’s Lambda = .91, F(1, 77) = 5.29, p < .01. A univariate test was computed on the composite pretest mean z scores for WPS measures. The univariate test also favored students in the control condition when compared with treatment conditions on the measures of problem-solving accuracy, (1, 77) = 2.81, p < .01, η2 = .09, p < .01. The post hoc comparison (Tukey test) indicated that pretest differences among the control and treatment conditions were significant (p < .05) on problem-solving components.
Intervention Outcomes on Standardized Measures
The design of this study allowed for the determination whether the level of posttest problem-solving performance achieved by participants in the intervention was statistically comparable with posttest performance in students in the control condition (Table 2).
Unconditional model
As shown on the left side of Table 2 for the unconditional model the random effect for the intercept between classrooms was significant, yielding an intraclass correlation of .15 (.10 / .68). The fixed effects provided an estimate of the average intercept for the total sample. The average posttest problem-solving accuracy z score for Grade 3 children without CSI treatment was 0.08.
Conditional Model 1
Table 2 shows the first conditional model (Conditional Model 1) entered with treatment condition CSI without controlling for pretest or RC. Overall, the model indicated that the mean wave problem-solving z score for the participants was 0.07. Posttest problem-solving accuracy score was 0.46 for CSI condition. This result indicated a significant posttest score when students received CSI. The model also reduced the significant random effects related to intercepts between classrooms when compared with the unconditional means model and accounted for 20% of the explainable variance (0.10 – .08 / 0.10). No significant advantage emerged at posttest for students receiving general mathematics instruction. The deviance score was lower than the unconditional model, as was Akaike’s information criterion (AIC) is conservative for the number of parameters estimated), suggesting that Conditional Model 1 was a good fit to the data.
Conditional model 2
The final conditional model sought to determine whether entering covariates (e.g., RC and pretest scores) improved the model fit. The results indicated a significant posttest score advantage for students receiving CSI treatment. Posttest problem-solving accuracy score was 0.37 for the CSI condition when controlling for student RC and pretest mathematics scores. The model also reduced the random effects related to intercepts between classrooms when compared with unconditional means model. This model yielded the lowest deviance (98.6) and AIC (110.6) values when compared with other models, indicating a better fit. No significant advantage emerged at posttest for students in control condition.
Effect size
Finally, because this was a small-scale study, effect size was computed for treatment condition applying Hedges g (Hedges, 1981) and indicated a significant treatment effect for the CSI (.37) condition.
Discussion-Implications for Educator Preparation
This study investigated whether CSI within word problems facilitated posttest problem-solving accuracy. Overall, the findings indicated that CSI that focused on directly and explicitly: (a) modeling reading and having students restate the word problem, (b) identifying irrelevant information, (c) identifying relevant information (e.g., question, vocabulary, and numbers), and (d) collaborating for problem solution facilitated posttest solution accuracy performance when compared with the control condition that included ELs with similar qualities. The results will now be discussed in terms of the research question that directed the study.
The results indicated a small but significant effect at posttest for CSI (.37) when compared with control condition. For the treatment condition, one of the reasons for this significance maybe that CSI helps student’s link relevant mathematics information more efficiently to prior knowledge by providing a model in which problems solving serves primarily to interpret the word problem. That is, CSI helps students make connections, ask questions, visualize, infer and predict, synthesize, and monitor their thinking about word problems. Finally, applying CSI allows for the integration of reading, language, and mathematics cognition that can help students to think effectively and understand ideas through reading and language and apply these skills toward solving word problems. Overall, the findings indicated that CSI may be an effective way to improving WPS performance among students at risk of MLD.
Implications
The implications from this study are clear that institutes of higher education (IHE) faculty need to provide future and current teachers with explicit research-based practices that have been validated with ELs. This would give preservice teachers a good start to feeling comfortable with providing this type of instruction to ELs. Educator preparation recommendations are as follows.
Limitations
Although the findings of this study demonstrated a small but significant effect of CSI (.37) on ELs’ WPS accuracy, there were limitations to this study. First this was a small-scale experiment, and data were collected on a small sample (n = 78) across a total of 20 lessons (10 weeks), and because of this, generalizing intervention effectiveness to other EL populations is limited at this time. Next, ELs were provided small group instruction by highly skilled and well-trained tutors; however, caution must be noted to the extent that strategies to improve students’ problem-solving accuracy with teachers who have less training and provide instruction in general education classrooms with a larger number of students have not been tested. This limitation highlights need for effective educator preparation in this content area as summarized above.
Summary
The findings from this study support the use of comprehension strategies for improving WPS performance in ELs at risk of MLD and indicate needed teacher education training. In particular, educator preparation should focus on relevant language and strategies to provide collaborative support to significantly improve WPS solution.
Footnotes
Appendix
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.