Abstract
Word problems are prevalent on high-stakes assessments, and success on word problems has implications for grade promotion and graduation. Unfortunately, English Language Learners (ELLs) continue to perform significantly below their native English-speaking peers on mathematics assessments featuring word problems. Little is known about the instructional needs and performance of ELLs at risk of mathematics difficulty (MD). In the present study, an exploratory quasi-experimental design was used to investigate word-problem instruction for ELLs in a culturally and linguistically diverse public elementary school. Specifically, we studied the efficacy of a word-problem intervention for ELLs with MD (N = 9) that combined culturally and linguistically responsive practices with schema instruction (CLR-SI). The study is unique in that it combines research on effective instruction for ELLs and students with MD; CLR-SI has not been investigated for either ELLs or students with MD. Results have implications for teachers, administrators, and researchers of ELLs with MD.
Keywords
Early success in mathematics is important because it is a strong predictor of later achievement in school (Morgan, Farkas, & Wu, 2011), and mathematics success leads to increased college and career opportunities (Murnane, Willett, Braatz, & Duhaldeborde, 2001). Achievement in mathematics is often measured by high-stakes standardized tests such as the National Assessment of Educational Progress (National Center for Education Statistics, 2013) that rely heavily on word problems. A word problem is a mathematics calculation embedded within sentences (Powell, 2011; Riley & Greeno, 1988). To solve word problems, students use text, typically presented in English, to identify missing information, make a plan to solve the problem, and perform one or more calculations to get the solution (Powell, 2011). The language and multi-step processes inherent in word problems can pose particular difficulties for English Language Learners (ELLs; Martiniello, 2008). The purpose of this study was to investigate the efficacy of a word-problem intervention for ELLs that incorporated culturally and linguistically responsive elements, which may be an important component of word-problem interventions for ELLs.
Mathematics Achievement and ELLs
The term ELL refers to a wide range of students with varying linguistic, cultural, and educational backgrounds. By federal definition, ELLs’ native language is a language other than English, and their level of English proficiency may impede academic achievement in classrooms where the language of instruction is English (Linquanti & Cook, 2013). English language is embedded within mathematics instruction, and ELLs often lag behind native English speakers in performance on standardized mathematics measures involving word problems at the elementary and secondary levels (Abedi & Lord, 2001; National Center for Education Statistics, 2013). The total number of words and academic vocabulary on standardized mathematics achievement measures can contribute to lower performance for ELLs compared with non-ELLs of similar mathematics ability, particularly for ELLs with lower English proficiency on test items with higher linguistic demands (Wolf & Leon, 2009).
Elementary mathematics can pose challenges for ELLs due to the linguistic complexity of instruction and assessment. In mathematics classrooms, students are expected to not only understand and solve problems but also explain their problem-solving process in written and verbal forms (Moschkovich, 1999; Powell & Hebert, in press). Many mathematical terms are new to learners (e.g., rhombus, subtract), and others may be familiar sounding (e.g., sum, value, product) but have specific and complex mathematical definitions (Freeman & Crawford, 2008). The latter can be just as unfamiliar for ELLs who may have limited knowledge of terms considered familiar to native English speakers. Syntactic and semantic features of mathematical discourse, such as the same as, take away, and how many go into, can also be confusing for students (McLeman, 2012). In addition, the symbols associated with mathematics (e.g., +, −, ×, ÷, =) and the interpretation of symbols can be challenging for students to understand (Freeman & Crawford, 2008; Gilmore, McCarthy, & Spelke, 2007; Powell & Fuchs, 2010).
Mathematics Difficulty
A mathematics disability is a type of specific learning disability (SLD) and is often referred to as dyscalculia in scientific literature (e.g., Butterworth, 2010; Mussolin, Mejias, & Noël, 2010). Approximately 3% to 6.5% of school-aged students struggle with a diagnosed mathematics disability (Shalev, 2004), but an even greater number of students demonstrate low mathematics performance without an official disability diagnosis. Of those students identified with a mathematics disability before fifth grade, 95% continue to struggle with mathematics at the high school level (Shalev, Manor, & Gross-Tsur, 2005). Both students with a diagnosed mathematics disability and students who display low mathematics performance are referred to in the literature as students with mathematics difficulty (MD; Vukovic, 2012). In MD research, participants may include students scoring in the bottom 10th, 25th, 31st, 35th, or 40th percentile (Mazzocco, 2005). The 25th percentile is a common cutoff point for defining MD (e.g., Bryant et al., 2008; Fuchs et al., 2009; Geary, Hoard, Nugent, & Byrd-Craven, 2008; Powell & Fuchs, 2010).
Students with persistent MD (i.e., continued difficulty across grade levels) are more likely than students without MD to have deficits in several mathematical areas (Vukovic & Siegel, 2010). Specifically, elementary students with MD may have difficulty counting (Geary, Hamson, & Hoard, 2000), understanding and comparing numbers (De Smedt & Gilmore, 2011), performing basic arithmetic facts (Jordan & Montani, 1997), and solving computation problems (Chong & Siegel, 2008). Specific to word problems, the language involved in word problems can pose challenges to students with MD. Bryant, Bryant, and Hammill’s (2000) study on the characteristics of students with MD found educators frequently rated students with MD as having difficulty with solving word problems. Fuchs et al. (2006) determined that language ability influenced mathematics performance on word problem solving. Overall, many researchers have demonstrated that word problems are problematic for students with MD (e.g., Fuchs et al., 2009; Jitendra et al., 2013; Powell et al., 2015; Swanson, Orosco, & Lussier, 2014).
Little is known about specific characteristics of ELLs with MD. Traditional classifications of MD focus on mathematics performance, which may or may not be appropriate for ELLs. Understanding the degree to which linguistic challenges influence mathematics performance can inform decisions regarding MD identification. There is a need for research on instructional techniques to promote mathematical problem solving for ELLs, particularly for ELLs with MD. This work is important because little is known about effective strategies to improve word problem solving for ELLs, despite the prevalence of word problems on high-stakes tests of achievement and the emphasis on problem solving in the national Common Core State Standards (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010) and from the National Council of Teachers of Mathematics (2014).
Culturally and Linguistically Responsive Mathematics Instruction
In the present study, we take a critical stance toward improving outcomes for ELLs who also identify with historically underserved racial and ethnic minorities. Specifically, we seek out an instructional approach that promotes academic achievement and affirms the identities of students whose educational opportunities and access may be influenced by a myriad of cultural, linguistic, and sociopolitical factors. Culturally responsive pedagogy is an approach to promote academic achievement for culturally and linguistically diverse (CLD) students and represents a variety of age, gender, geographic, class, and privilege in classroom instruction (Gay, 2002; Ladson-Billings, 1995). To increase the achievement of CLD students (i.e., ELLs), researchers support the use of culturally responsive instruction or culturally responsive teaching (CRT). A primary purpose of CRT is to achieve equitable educational outcomes for student populations who have been historically marginalized, thereby working toward larger goals of social justice (Hernandez, Morales, & Shroyer, 2013). A critical aspect of CRT includes knowing and incorporating student identities; therefore, this instructional approach lends itself for teachers working with a range of learner characteristics in their classrooms.
Pursuing instructional approaches that promote equitable outcomes is especially critical in mathematics, where, as stated previously, ELLs perform consistently lower on standardized achievement measures than their Caucasian, native English-speaking peers (National Center for Education Statistics, 2013). Culturally responsive mathematics instruction is defined in the literature as pedagogical knowledge, teacher beliefs, and instructional practices that promote mathematical thinking, value student funds of knowledge, and incorporate issues of power and social justice in mathematics education (Aguirre & del Rosario Zavala, 2013). Funds of knowledge, or identity, refer to experiences and understandings students bring into the classroom from their home community (e.g., measuring ingredients while cooking, economic practices of a local business), which can be leveraged in instruction (Esteban-Guitart & Moll, 2014; Moll, Amanti, Neff, & Gonzalez, 1992). To address complex issues of justice and equality in instruction, Turner and Strawhun (2007) state that the problem-solving contexts teachers present must be authentic to students’ lived experiences, and that students should find the problem worth solving (e.g., using a mathematics measurement unit to analyze issues of school overcrowding). On a large scale, mathematics education can be used as a tool to analyze relationships of power and privilege through social and economic structures.
Specifically for ELLs, research has focused on linguistically responsive teaching (LRT). Teachers implementing LRT incorporate linguistic supports (e.g., native language, grammatical supports, vocabulary development; Echevarria, Short, & Powers, 2006; Goldenberg, 2013) for their ELL students because of the additional challenges of learning academic content in a second. Cultural and linguistic diversity is seen as an asset to classroom learning experiences (Lucas, de Oliveira, & Villegas, 2014). Linguistically responsive mathematics instruction should be informed by ELLs’ prior experiences with mathematics content, language experiences and proficiencies, and educational histories (Moschkovich, 2013). Moschkovich’s (2013) recommendations for equitable mathematics instruction for ELLs include a focus on conceptual understanding and reasoning, strategic support for ELLs’ participation in mathematical discussions as they learn English by drawing on available resources (i.e., objects, drawings, graphs, and gestures), and the value of native language and home experiences in instruction. Teachers can make mathematics instruction comprehendible for ELLs by using familiar content and contexts, developing English vocabulary, using native language to support content understanding, and promoting collaborative discourse (Taube & Jasper, 2009).
In the present study, we view culturally and linguistically responsive instruction as complementary approaches that are interrelated. Within CRT and LRT, teachers should consider the unique learning characteristics of their students including native language, English language proficiency, race and ethnicity, home and community culture, and past educational experiences. Culturally and linguistically responsive mathematics instruction, therefore, should incorporate linguistic supports and effective strategies for the ELLs to make content accessible and promote academic achievement, while also incorporating aspects of students’ culture and experiences into mathematics content. In addition to facilitating mathematics achievement, culturally and linguistically responsive mathematics instruction may also help develop ELLs’ English language proficiency. Similarly, teachers should view diverse student experiences, perspectives, and languages as resources in their classroom. Word-problem instruction presents a unique opportunity to study culturally and linguistically responsive mathematics instruction, because of the role of context as well as linguistic complexities inherent in problems.
Word Problem Solving and ELLs
Of the existing studies on culturally and linguistically responsive mathematics instruction for ELLs, only a few focus on word-problem-solving instruction. The research focuses primarily on performance outcomes and associated problem-solving processes that ELLs use to solve word problems (e.g., Ambrose & Molina, 2010; Barwell, 2003, 2005; Bautista, Mulligan, & Mitchelmore, 2009; Cuellar, De La Colina, & Cmajdalka, 2005; Turner & Celedón-Pattichis, 2011). Less is known about instructional approaches to improve word problem solving for ELLs, including ELLs with MD. There is an alarming absence of empirical evidence to guide teachers’ word-problem instruction for ELLs, particularly for students who need additional support in mathematics.
Orosco (2014) and Orosco, Swanson, O’Connor, and Lussier (2013) tested the effectiveness of an instructional strategy to improve word problem solving for ELLs with MD. This strategy, Dynamic Strategic Math (DSM), incorporated aspects of LRT by pre-teaching vocabulary, providing instruction in small groups, and using a dynamic assessment approach to scaffold support according to individual student’s language proficiency and content understanding. Compared with the baseline phase, introduction of the DSM intervention increased the level of word problem solving for all participants in both studies. Orosco and colleagues’ work is promising; however, more research is needed to build an evidence base for ELLs with MD. Much of the existing ELL word-problem research assumes a deficit model that lumps all ELLs into one group, regardless of language and mathematics proficiency (Gutierrez & Orellana, 2006). Student populations should be well defined and distinguish language status from academic difficulties, as much as possible.
Schema-Based Word-Problem Instruction
Multiple studies have demonstrated the efficacy of schema instruction (SI) for students with MD (Fuchs et al., 2008; Fuchs et al., 2009; Jitendra et al., 1998; Powell & Fuchs, 2010). In SI, a schema is used as a framework in which students are taught to identify additive problem types (e.g., group, change, compare; Griffin & Jitendra, 2009). In SI, understanding the structure of a word problem is seen as critical to successful problem solving (Kalyuga, 2007). SI is a particularly promising approach for ELLs with MD because this evidence-based instruction includes both explicit strategy principles of MD research and grammatical structures to help students identify and solve word problems. SI can also be referred to in the literature as cognitively guided instruction (CGI; Ambrose & Molina, 2010), schema-based instruction (SBI; Jitendra et al., 2013), and schema-broadening instruction (Fuchs et al., 2008). Powell’s (2011) review on SI indicated effect sizes (ESs) favoring experimental conditions ranging from 0.28 to 6.84. Although ELLs have been included in past SI studies, there has not been a study to date investigating the effectiveness of this approach for this specific student population.
SI methods incorporate explicit strategy instruction and metacognitive approaches, which can help students navigate the linguistic complexity of word problems. Although ELLs have been included in student samples in previous SI research, effects of this approach for ELLs of varying ethnicity and language proficiency are unknown. Likewise, CRT or LRT have not been documented in existing SI research. Thus, the evidence base for SI warrants further investigation to determine whether this approach is beneficial for ELLs with MD.
Purpose of the Present Study
In the present study, we sought to explore the efficacy of a word-problem intervention for ELLs with MD. Because of the exploratory nature, we used a pretest and posttest design to investigate the influence of a culturally and linguistically responsive schema intervention (CLR-SI) for elementary ELLs with MD. The CLR-SI intervention was designed using CRT, LRT, and schema-based word-problem instruction for students with MD. Culturally and linguistically responsive approaches were integrated into an evidence-based intervention that used schemas (i.e., problem types) to teach students how to solve word problems. We sought to answer the following research questions (RQs):
Method
We conducted a quasi-experiment where we provided CLR-SI word-problem intervention to third-grade ELL students with MD. To answer RQs 1 and 2, we administered a pretest and posttest battery to each participant. To investigate RQ3, all participants responded to a social validity questionnaire at the conclusion of the project.
Setting
The study occurred at a culturally and linguistically diverse public elementary school (e.g., pre-kindergarten through fifth grade) in the mid-Atlantic region of the United States. The elementary school serves a high percentage of diverse learners, including refugee and ELL populations. School demographics were obtained from the district and are reported as such. At the time of the study, there were 606 students, and the student population was 53% male and 47% female. Racial composition included 33.1% Black, 22.2% Hispanic, 24.5% Caucasian, and 20.2% other. Student demographics also included 9.9% students with disabilities, 3.7% students identified as gifted or talented, and 76.6% receiving free or reduced lunch. In addition to the 9.9% students identified with disabilities, 28.2% of the student population was in Tier 2 or Tier 3 of the school’s Response to Intervention (RTI) framework. RTI was the process used to provide intensive intervention for students who demonstrated persistent academic difficulty. Students whom the school considered “at risk” for having a disability first engaged in Tier 2 and/or Tier 3 intervention before the identification process began.
Participants
For the present study, we focus on the mathematics achievement of ELL students who also identify with historically underserved racial and ethnic groups. All third-grade ELLs across three participating teacher classrooms were screened for MD to determine eligibility for word-problem tutoring. ELLs were administered a brief word-problem assessment (e.g., Pennies Test; Jordan & Hanich, 2000), and students who performed below the 25th percentile were considered to be at risk of MD. Scoring at or below the 25th percentile is a common identification practice in word-problem research for students with MD (Fuchs et al., 2009; Powell & Fuchs, 2010). For the present study, the 25th percentile was determined using a normed population. Nine ELLs across three classrooms were identified as at risk of MD and therefore eligible for the tutoring component of the study. Each of the third-grade teachers was provided a list of eligible students to confirm whether they felt (a) the proposed students would benefit from a word-problem intervention and (b) they were comfortable with each student being pulled out of the classroom during their scheduled morning meeting time. All teachers supported the inclusion of their students.
The elementary school used the World-Class Instructional Design and Assessment (WIDA) to measure English proficiency for ELLs. The WIDA is an English language assessment that determines student proficiency in reading, writing, listening, and speaking, and is used across a number of states in kindergarten through 12th grade (WIDA, 2014). Students are scored on their linguistic complexity, language forms and conventions, and vocabulary use. Although helpful for providing a picture of second-language acquisition, the WIDA only provides proficiency indicators for English and non-native language. The WIDA designates five levels of English language proficiency, with five being the highest level and one being the lowest level of English proficiency. The average WIDA level for participants was 3.1; the highest participant level was four, and the lowest level was one. The participant with a Level 1 proficiency was included in the study at the classroom teacher’s request.
Available demographic information is provided for all students (N = 9) who participated in the word-problem tutoring in Table 1. Participants’ classroom teachers provided all available demographic information. Surprisingly, reading level was unavailable for the majority of students. All students are assessed on district-mandated reading benchmarks multiple times a year; however, assessment data were held by the interventionist team. Teachers could only provide basic indicators for reading (e.g., Guided Reading Level F) on their demographic questionnaire. No participant had a school-diagnosed disability, although several were receiving early intervention through the school’s RTI framework and were considered at risk of SLD.
Participant Demographics.
Note. Participant I’s teacher expressed some confusion over which language, including English, was the student’s primary language. The school classified Participant I as an ELL, and he was identified as eligible for tutoring based on Pennies Test score. WIDA = World-Class Instructional Design and Assessment; ELL = English Language Learner.
Research Design
A quasi-experiment was used to determine the efficacy of the proposed intervention: CLR-SI. We used a quasi-experiment because all nine students participated in the intervention without a business-as-usual comparison. To understand whether intervention was efficacious, we compared the performance of the nine participants with a representative sample of third-grade students. The representative sample (N = 605) included data collected over multiple years at a site similar to the school in this study (Fuchs et al., 2009). ELLs comprised 16% of the representative sample.
There were two phases for each student: Before the intervention is introduced (e.g., Basic Strategy condition) and after it is introduced (e.g., CLR-SI condition). The nine eligible students were assigned to one of four tutoring groups to assess treatment procedures. The outcome behavior in this study was student word-problem performance. Five tutors (e.g., four Research Assistants [RAs] and the principal investigator) provided tutoring to four groups of ELLs with MD.
Four of the tutors were undergraduate students, and one tutor (i.e., principal investigator) was a doctoral student at a mid-Atlantic university. Four of the tutors were female, and one was male. Four of the tutors were Caucasian, and one was African American.
With the exception of the principal investigator, none of the tutors had prior teaching experience including instruction for ELLs or students with MD. Two of the four undergraduate tutors hoped to obtain teaching positions on graduation, and the other two were in other service-related fields (e.g., speech language pathology and audiology). One of the tutors was a Spanish major and could speak the language fluently. After the CLR-SI phase began, this tutor occasionally spoke in Spanish conversationally with students and provided academic praise statements in Spanish. She did not provide mathematics instruction in Spanish. It is unclear whether this tutor’s familiarity with students’ native language had a differential effect on their performance.
Measures
With the Pennies Test (Jordan & Hanich, 2000), students solved 14 word problems. All problems were read aloud by the examiner so that performance was not confounded with reading ability. Students were given a written copy of the test to solve problems. Word problems ranged from simple to complex and cover the following problem types: total/combine, change, difference/compare, and equalize (e.g., “Alex has 8 pennies. Kris has 6 pennies. What could Alex do to have as many pennies as Kris?” Jordan & Hanich, 2000, p. 571). Students responded to all of the abovementioned problem types. The Pennies Test was administered to all third-grade ELLs in participating teachers’ classroom to screen for students eligible for tutoring (i.e., students scoring at the 25th percentile or less). The Pennies Test was again administered as a posttest measure for students who participated in the intervention to determine whether word-problem performance changed after engaging in the tutoring project. Maximum score was 14, and the Cronbach’s α for this sample was .85.
The Pennies Test was administered in English. Prior to administration, the principal investigator met with participating teachers who confirmed students encountered similar word problems in class. Teachers were also given a list of their participating students and confirmed that English proficiency would not prohibit each student from comprehension of the word problems. In addition, there was not a word-problem measure in students’ native language available for use. We selected the Pennies Test for use because it contained representative problems for each additive schema type, had been previously used in schema instruction research, and had data from a normative sample available to compare student performance. We recognize that the language of the assessment may have posed additional difficulty; however, all students in the study answered school mathematics assessments in English on a regular basis, so the assessment for this study was not an unusual experience for the students.
We also included measures of computational fluency in the pretest and posttest to determine whether there was a relationship between word-problem, addition, and subtraction skills. With Addition Fluency (Fuchs, Hamlett, & Powell, 2003), students had 1 min to answer 25 addition facts with sums to 12. All problems were presented vertically, and students answered as many facts as possible during the 1-min time period. Maximum score was 25, and the Cronbach’s α for this sample was .92. With Subtraction Fluency (Fuchs et al., 2003), students had 1 min to answer 25 vertically presented subtraction facts with minuends to 12. Maximum score was 25, and the Cronbach’s α for this sample was .64. For both Addition Fluency and Subtraction Fluency, the principal investigator or an examiner read directions aloud prior to starting the timer, then allowed students to work independently until the end of 1 min.
Data collection
The Pennies Test, Addition Fluency, and Subtraction Fluency were administered twice, as an initial screening measure to determine eligibility for tutoring and after the final tutoring session (i.e., posttest). Each tutoring session concluded with students answering a word problem independently. Tutors scored and recorded student progress on this daily assessment after each tutoring session. The principal investigator administered the initial screening measures. Tutors and the principal investigator provided tutoring, administered small-group and individual measures, and scored word problems for analysis throughout both tutoring phases. For the present study, we focused our analysis on student responses on the Pennies Test.
Intervention
Tutoring lasted 10 weeks over the course of a semester, beginning in late September and concluding in mid-December. Tutoring sessions were conducted 3 times per week for 20 to 25 min a session. Tutors and the principal investigator provided tutoring in four small groups of two to three students.
Only basic addition and subtraction facts (with addends 0–9 and sums to 18) were used in each lesson embedded within the word problems. There were two phases of word-problem instruction. Tutors provided explicit and scaffolded instruction during every lesson in both conditions. The first phase was Basic Strategy Instruction, where students practiced word problems through strategy instruction and techniques they knew from their classroom. The second phase consisted of the word-problem intervention, CLR-SI. All tutors began by providing Basic Strategy Instruction, and changed to the CLR-SI intervention when instructed by the principal investigator. Tutoring groups began CLR-SI on a staggered schedule, to allow for preliminary analysis on the effectiveness of each phase. Basic Strategy sessions ranged from three (Students H and I) to 17 (Students C and D). CLR-SI sessions ranged from nine (Students C and D) to 23 (Students H and I). Throughout each tutoring session, students had the opportunity to earn puzzle pieces for following directions, working hard, and completing each of the activities during sessions to promote positive behavior.
Basic strategy instruction
During each tutoring session in the first phase, three activities occurred. The first activity was a flash card warm-up. Each flash card displayed either two numbers or two sets of pictures. The students took turns saying the total amount of numbers or pictures on the card. The numbers on a number flash card correspond to the pictures on a picture flash card. The tutor showed cards one at a time for 1 min. If the student answered correctly, the tutor placed the card in a correct pile. If the student answered incorrectly, the tutor asked the student to count to find the correct answer. After the student remediated an incorrect answer, the tutor placed the card in a correct pile. At the end of 1 min, the tutor and student counted the number of flash cards in the correct pile. Students graphed their individual or collective group flash card score on a graph daily.
After the flash card warm-up, the students demonstrated what they have retained from prior lessons on a word-problem review. Tutors gave students 3 min to answer as many problems as possible. Students completed the word-problem review every session, including the first day of tutoring, to establish baseline and performance growth across the two tutoring phases. The next part of each tutoring lesson was the actual word-problem instruction. All word problems were presented in English, which is consistent with their typical classroom instruction, and read aloud by the tutors. In the Basic Strategy Instruction phase, this consisted of general strategy instruction where students were taught to understand the problem, devise a plan, carry out the plan, look back, and check (Pólya, 1945). Specifically, tutors prompted students to use the RISE strategy where they (a) Read the problem, (b) Illustrated the problem by drawing a visual representation or writing the equation, (c) Solved for the unknown amount, and (d) Explained what they were solving for (e.g., four apples).
CLR-SI
Similar to the Basic Strategy phase, each CLR-SI lesson consisted of three activities: flash card warm-up, review, and instruction. The key difference in this condition was the word-problem instruction students received. Once tutors began the intervention phase, they started implementing CLR-SI. The CLR-SI condition included strategic support for ELLs’ participation in mathematics instruction as they learn English by drawing on available resources (i.e., objects, drawings, graphs, and gestures), allowing the use of native language and incorporating student experiences from outside of school.
Although SI covers a variety of problem types, the present study focused on change problem types. Ambrose and Molina’s (2010) comparison of Latino first graders’ performance on word problems revealed students had the most difficulty with change and difference/compare problems. Because of time constraints, tutors only provided CLR-SI instruction on change word-problem types.
CLR-SI follows traditional SI methods (e.g., Fuchs et al., 2008; Jitendra et al., 2013), where tutors instruct students to identify the problem type, identify the missing information, identify the known information, and set up the appropriate equation. Key differences between CLR-SI and the Basic Strategy Instruction phase include the incorporation of culturally and linguistically responsive elements and schema instruction using change–increase and change–decrease problem types. Students used an expanded version of the RISE strategy, which incorporated the use of manipulatives to illustrate and the use of schemas to solve word problems. Tutors followed Archer and Hughes (2011) explicit instruction model that has demonstrated benefit to students with MD. Tutors previewed the strategy and available resources, modeled the strategy, engaged students through questioning and opportunities to respond, facilitated student practice with frequent feedback, and allowed for independent practice within each tutoring session.
Culturally and linguistically responsive pedagogy was incorporated into traditional SI in several ways. Specifically, CLR-SI included the following elements of culturally and linguistically responsive mathematics instruction: (a) explicitly stating measurable lesson objectives, (b) facilitating oral discussions with students (i.e., encouraging peer discourse), (c) allowing use of native language, (d) using graphic organizers and manipulatives (i.e., colored motors) to help illustrate and compute each problem, (e) explicitly using students’ own ideas and experiences, and (f) providing relevant instructional examples to participants’ daily lives, pop culture, and cultural heritage.
Each CLR-SI lesson concluded with a word problem that solicited information from students’ personal lives. Students were asked to provide information relating to their experiences, interests, and pop culture (e.g., “Do you have any brothers or sisters?” “What is your favorite TV show?” “What did you do last weekend?”). This information was used to create word problems with student input, for the student to then solve. Students were allowed and encouraged to use native language if they desired. Tutors affirmed and encouraged native language as a mathematical resource. Only one of the tutors was fluent in Spanish; however, eight of the nine student participants spoke Spanish fluently. In the CLR-SI phase, students were encouraged to use either English or their native language while setting up the word problem to solve and calculating the unknown. As tutoring progressed, students became more comfortable speaking to each other in their native language while discussing word problems. Students used native language more often in groups where all students spoke Spanish.
Procedural fidelity
Fidelity of implementation was assessed across each phase of the intervention. Tutors participated in a 2-hr training to become familiar with and practice instruction in the two tutoring phases. Two members of the research team independently scored 100% of student responses. The discrepancies between the two databases were compared and rectified by the principal investigator to reflect the student’s original response.
All lessons were scripted to ensure tutors covered material in a similar manner. Tutors were not required to read scripts verbatim but were required to use scripts designated for each of the two phases. Scripts contained various prompts for tutors to facilitate and encourage peer discussion about word problem solving. Tutors became familiar with each lesson and delivered the lesson by following the framework, concepts, and vocabulary of the script. Tutors delivered the lesson using only the word problems, materials, and examples provided in the script. All tutors followed the same sequence of lessons within each condition.
To evaluate fidelity of implementation of the two tutoring phases (i.e., baseline and intervention), tutors digitally audio recorded all sessions. Of the recorded sessions, 20% were randomly sampled from each phase to ensure comparable representation of tutoring phase, tutors, and sessions. Two tutors listened to 20% of the tutoring sessions in each phase, for each group. Tutors did not assess fidelity of their own tutoring sessions. Tutors used a checklist to look for the presence or absence of different lesson components (i.e., 15 components in Basic Strategy Instruction phase; 16 to 19 components in CLR-SI phase) to determine whether instruction was implemented as intended. Checklist components included introducing the warm-up activities, lesson-specific word problems, and appropriate instruction provided for each phase of the tutoring project (e.g., use of manipulatives in CLR-SI phase) reinforcing key concepts through feedback. This was to ensure each lesson addressed all of the Basic Strategy and CLR-SI instructional components and lasted within the designated time limit (e.g., 20 to 25 min). After listening to 20% of the intervention sessions from each group, procedural fidelity was found to be 99% in the Basic Strategy Instruction phase and 94% in the CLR-SI phase.
Results
To address our first research question, we conducted a repeated-measures analysis of covariance (ANCOVA) to determine differences between students at the start and completion of the tutoring project on the Pennies Test. To control for English language proficiency, students’ WIDA scores were used as a covariate. To control for computational fluency, student pretest scores on Addition Fluency and Subtraction Fluency were also used as covariates in the full model. There was a significant effect of Pennies posttest performance, F(1, 5) = 19.069, p < .01. The ES, calculated using partial eta-squared, was 0.79, indicating a large effect in favor of the tutoring program (Cohen, 1992). English language proficiency (e.g., WIDA level) was not significantly related to Pennies performance, F(1, 5) = 2.173, p = .20. There was a significant interaction with students’ pretest score on Addition Fluency with Pennies performance, F(1, 5) = 10.383, p = .02. The ES was 0.68. Student pretest Subtraction Fluency scores were not significantly related to Pennies performance, F(1, 5) = 2.794, p = .16. The results indicate that students who participate in the Basic Strategy Instruction and CLR-SI improve on word-problem performance (see Table 2).
Pretest and Posttest Performance.
Due to the relatively small sample size, we were also interested in comparing our participants’ percentile growth on the Pennies Test with our second research question. We compared our participants’ Pennies Test results with a database of third-grade students (N = 605). The comparative sample represented ELL and non-ELL students, as well as students with and without MD (Fuchs et al., 2009). Using these normative percentiles, our mean participant score changed from approximately the 10th percentile (M = 4.56) at pretest to above the 40th percentile (M = 10.00) at posttest. At pretest, participant scores ranged from the third to the 18th percentiles. At posttest, participant percentiles ranged from the 33rd to 59th percentiles. Importantly, at posttest, no participant would have been found eligible for tutoring based on their posttest scores (i.e., scores at or below the 25th percentile).
Our third RQ dealt with the social validity of the tutoring project. Students responded to a five-question social validity questionnaire at the completion of posttest. This questionnaire was on the last page of the posttest battery administered by the tutor during the final session. Five statements were read aloud, and students were prompted to circle a smiley face if they agreed with the statement, a straight face if they were neutral or were not sure how they felt, or a sad face if they did not agree with the statement. Consistent with all other tutoring materials and measures, this questionnaire was administered in English. The questionnaire was shown to participating classroom teachers before administration to confirm the language and presentation would be understandable for all students.
For the first statement, “This tutoring was helpful,” nine students (100%) agreed with this statement. For the second statement, “I liked coming to tutoring,” seven students (78%) agreed with this statement. Two students (22%) felt neutral about this statement. The next statement was, “I learned how to solve story problems in tutoring,” and nine students (100%) agreed with this statement. The following statement was, “I can solve story problems correctly in class,” and six students (67%) agreed with this statement. Three students (33%) felt neutral about this statement. For the final statement, “I would like to continue learning how to solve different types of story problems,” nine students (100%) agreed. Students were able to write in additional comments at the bottom of their questionnaire. Overall, student comments were positive and expressed satisfaction with the tutoring. Examples of student comments were as follows: “It was really fun” and “I love to do tutoring.”
Discussion
In this exploratory study, we investigated the influence of CLR-SI on the word-problem performance of third-grade ELLs with MD. Our first RQ examined whether ELLs with MD demonstrated performance on word problems after receiving CLR-SI. From pretest to posttest, the change of word-problem performance was significant, with an ES of 0.79. Students who participated in the CLR-SI intervention demonstrated improved skill with solving word problems. Whether the improvement was due to the CLR component of the schema instruction or repeated practice and exposure to word problems needs to be investigated in future work. Likewise, further work is needed to determine the differential effects each phase of instruction (e.g., Basic Strategy and CLR-SI) had on student performance.
English language proficiency was not significantly related to participants’ word-problem performance. We did not expect a lot a variance, because of the homogeneous nature of participants’ WIDA scores. Participants’ subtraction fluency was also not significantly related to word-problem performance. At both pretest and posttest, participants attempted fewer problems than on Addition Fluency. The limited responses we obtained may have influenced both the low coefficient alpha for Subtraction Fluency and the lack of significance as a covariate for word-problem performance. The significant interaction of participants’ Addition Fluency scores indicates that students’ addition fluency influences their word problem. These results are consistent with literature indicating that students often demonstrate stronger performance on addition problems (De Corte & Verschaffel, 1981) and that an understanding of addition typically precedes understanding of subtraction (Canobi, 2004).
Our second research question concerned participants’ percentile growth compared with a normative sample of third-grade students. All participants increased their performance percentile, with the mean score changing from approximately the 10th percentile at pretest to above the 40th percentile at posttest. Importantly, when compared with data from a normative sample, students in the present study moved from being categorized as “at-risk” to performing similar to students without MD. Our third RQ dealt with participants’ perceptions of the intervention. All nine students found the tutoring helpful and felt that they had learned to solve word problems. The majority of students enjoyed the tutoring. Our results indicate that the combination of CLR and SI may be a viable approach for word-problem intervention for ELLs with MD and that students benefited from participating in the intervention.
Limitations
There are several limitations that should be noted in this study. First, inferences are made with a relatively small sample size. Future studies should study culturally and linguistically responsive word-problem instruction in settings with larger populations of ELLs with MD. Time, in terms of tutoring instruction and measuring student understanding, was another limitation of the study. Tutoring sessions were limited to 20 to 25 min, and all tutors expressed feeling rushed to cover all the necessary content in the tutoring scripts and allow students adequate time to practice. Students began the CLR-SI at different points in time, so some students had less exposure to this phase of the intervention. Further field testing and standardization of word-problem-solving measures are critical to improving inferences researchers are able to make regarding student performance.
The intervention also has several inherent limitations. Whereas culturally responsive elements are incorporated into the intervention, others were purposefully left out (i.e., home–school relationship). Additional elements of CRT could be infused into a word-problem intervention if delivered by students’ teachers, because a more permanent relationship would be established. Finding tutors who are also fluent in participants’ language would allow for explanation of content and facilitation of discussion in native language. There was also a degree of overlap between the Basic Strategy and CLR-SI. Students were similarly provided instructional scaffolds (i.e., flash card warm-up to improve computational fluency, RISE strategy, etc.) in both phases. Students also solved word problems in all variations of change problem types (i.e., addition problems with the starting amount unknown, subtraction problems with the change amount unknown, etc.). This variation was not consistent with classroom observations of mathematics instruction. Increased exposure and practice on word problems in both phases may have influenced student performance.
Due to the limited time available for tutoring over the course of the semester, only one problem type was explicitly taught in CLR-SI. Expanding the intervention to include multiple problem types (e.g., total, difference) would improve the inferences we were able to make regarding the efficacy of CLR-SI for ELLs with or at risk of MD. Future researchers should investigate the efficacy of including all problem types in the intervention. Despite the abovementioned limitations, the proposed study provides insight to the challenges and opportunities found in word-problem instruction for ELLs with or at risk of MD.
Implications for Practice
There are several implications of the present study for word-problem instruction, culturally and linguistically responsive mathematics instruction, teacher preparation, and the role of school context. It is relatively unclear whether CLR-SI is an effective approach for word-problem instruction for ELLs. The present study highlighted issues with measurement and methodological challenges associated with time. Word problem solving is a complex process; therefore, students should be given ample time to engage in instruction. The majority of word-problem intervention studies for students with MD measure change in performance over the course of weeks (i.e., Fuchs et al., 2008; Fuchs et al., 2009; Jitendra et al., 2013; Powell & Fuchs, 2010). Future intervention studies should consider the extensive time needed to assess changes in word-problem-solving performance and select research designs that will allow for maximum instruction with limited assessment (i.e., randomized control trials). Problem type and structural representation were very limited in the classroom observations and problems we observed students working on. To date, word-problem representation has not been systematically investigated in elementary curricula and assessment. Increasing the variation may help students’ word-problem-solving abilities and should be explored in future studies.
The results from the word-problem intervention confirm the need to better understand the variability of student characteristics within ELL populations. For example, the participant with the lowest English language proficiency (e.g., Student G) demonstrated consistently higher performance on her daily word-problem probe. According to her teacher’s reflections, this participant had received multiple years of schooling in her native country. Her teacher was confident that as her English proficiency increased, she would have minimal academic difficulties. Another participant, Student B, attended school in the United States longer and has a higher English proficiency than Student G; however, her teacher expressed concern over the limited information provided regarding her schooling in her native country. The teacher’s concern for Student B’s word problem solving had less to do with ability and more to do with the fact that she was still catching up on the school experience. Student A, in contrast, has attended the elementary school since pre-kindergarten. His teachers were concerned with his limited academic achievement. Student A’s English proficiency indicator was comparable with many of the other participants; however, he is classified at the most intensive tier of the RTI framework and would likely be identified with a learning disability before the end of third grade.
Academic performance, native and secondary language proficiency, the number of years ELLs have lived in the United States, and their academic experiences prior to arriving are a sample of the factors that should be considered in student mathematics achievement, including word problem solving. Teachers should recognize the complexity and influence of ELLs’ educational histories in mathematics instruction and assume an asset-based approach by leveraging student funds of identity in mathematics instruction (Esteban-Guitart & Moll, 2014; Moschkovich, 2013). ELLs with different educational strengths, needs, and experiences may respond differentially to tailored instruction. For example, Student G may have responded better to an intervention that included more linguistic supports and less mathematical concept supports. In contrast, Student A may have responded better to an intervention focused on computational fluency and basic mathematical concepts before moving on to more complex tasks such as word problem solving. Further qualitative research in this area may help develop future inventories to guide teachers’ instructional decision making for ELLs.
In summary, standardized mathematics items rely heavily on word problems to assess student knowledge and skill. Although there is evidence of discrepancies in mathematics performance between ELLs and their native English-speaking peers, there is limited research on effective culturally and linguistically responsive instruction to improve word problem solving for ELLs. The present study contributed to the literature through multiple-method study on mathematics instruction for ELLs with MD, with a focus on word problem solving. Further research at the student, teacher, and school levels is needed to better understand culturally and linguistically responsive word-problem instruction for ELLs with MD.
Footnotes
Acknowledgements
We extend thanks to the teachers, students, and research assistants who participated in this project. Statements do not reflect the position or policy of the university, schools, or persons, and no official endorsement should be inferred.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The University of Virginia supported this research.
