Abstract
The purpose of this study was to determine whether relationships existed between teachers’ implementation of two specific discourse-related instructional practices and students’ mathematics achievement in geometry and measurement as part of a research study on the effectiveness of an advanced mathematics curriculum for kindergarten and Grades 1 and 2. The mathematics units incorporated the following instructional practices: engaging students in verbal communication in mathematics and encouraging the use of appropriate mathematical vocabulary. Hierarchical linear modeling was used to determine the relationships between teachers’ use of the instructional practices and the students’ mathematics achievement. Results indicated that significant, positive relationships existed; the teachers’ implementation scores for the verbal communication and encouraging mathematical language instructional practices were predictors of student mathematics achievement as measured by students’ percentage gain scores on the Open-Response Assessments. Implications of these findings for mathematics instruction are discussed.
Keywords
Reform is a word commonly used to describe recommended mathematical practices in today’s educational environment (National Council of Teachers of Mathematics [NCTM], 1991, 2000; National Governors Association Center for Best Practices [NGA] & Council of Chief State School Officers [CCSSO], 2010). Although many definitions are currently used, generally speaking, reform in mathematics pedagogy involves a shift of instructional focus toward engaging students in mathematical reasoning and problem solving, encouraging students’ conceptual understanding, and developing classrooms as mathematical communities (NCTM, 1991; NGA & CCSSO, 2010). To facilitate and determine the effectiveness of these educational reforms in mathematics pedagogy, research has been conducted examining student achievement as related to instructional practices and curriculum. However, studies of the practices aligned with the reform recommendations tend to investigate instructional practices as a group as opposed to seeking to identify the effect a specific strategy may have on student achievement (Gimbert, Bol, & Wallace, 2007; Huffman, Thomas, & Lawrenz, 2003; Spillane & Zeuli, 1999). In addition, over 40 states have adopted the Common Core State Standards for Mathematics (CCSS-M; NGA & CCSSO, 2010) that includes Standards for Mathematical Practice. To date, research is needed to investigate the effectiveness of engaging students in the Mathematical Practices as specified by the CCSS-M. This study addresses two teacher instructional practices that are related to the CCSS Mathematical Practices of having students (a) construct viable arguments and critique the reasoning of others and (b) attend to precision, which includes use of appropriate mathematical vocabulary. The purpose of this study was to examine the relationship between teachers’ implementation of two instructional practices for mathematics (engaging students in verbal communication [discourse] and encouraging the use of appropriate mathematical vocabulary) and students’ mathematics achievement within the context of a larger study, Project M2. To provide a context for the current study, a brief summary of the Project M2 curriculum implementation research study as well as a review of the literature as related to the two instructional practices for mathematics are provided.
Project M2 was a quasi-experimental curriculum implementation research study funded by the National Science Foundation to develop and examine the effectiveness of six mathematics curriculum units, one in geometry and one in measurement with all students in the regular classroom each in three grade levels: kindergarten and Grades 1 and 2. Each of the units includes advanced content for the grade level as well as specific instructional practices. The mathematics achievement of the kindergarten and Grades 1 and 2 students in geometry and measurement was measured using the Iowa Tests of Basic Skills (ITBS) Mathematics subtest and a grade-level-specific Open-Response Assessment. Statistically significant differences between students in the intervention and comparison groups were found on the Open-Response Assessment but not on the ITBS for Grades 1 and 2 (Gavin, Casa, Adelson, & Firmender, 2013; Gavin, Casa, Firmender, & Carroll, 2013). However, the kindergarten intervention group scored significantly higher than the comparison group on both the Open-Response Assessment and the ITBS (Casa, Firmender, Gavin, & Carroll, 2013).
What the Research Says: Instructional Practices in Mathematics Education
The methods and instructional practices teachers use during instruction have the potential for influencing student achievement. In 1991, NCTM published the Professional Standards for Teaching Mathematics, which outlined a model of mathematics instruction that encouraged teachers to use instructional practices to promote the development of conceptual mathematical knowledge and skills. These practices involve the engagement of students in appropriate tasks to develop mathematical concepts through the use of classroom discourse, technology, and connections to previous knowledge (NCTM, 1991). More recently, the adoption of the CCSS (NGA & CCSSO, 2010) by most states has charged teachers with engaging students in the Standards for Mathematical Practice. These eight Standards for Mathematical Practices describe how students should interact with and engage in learning mathematical content (NGA & CCSSO, 2010). In addition, a position statement by the National Association for the Education of Young Children and National Association of Early Childhood Specialists in State Departments of Education (2003) stated that young students should be actively engaged in a curriculum that is challenging and in-depth and allows them to investigate mathematical content.
Many of these instructional practices are grounded in the Vygotskian tradition and socio-cultural theory that views learning as taking place through social interactions. Specifically related to mathematics education, the socio-cultural theory applies to instruction in which teachers engage students in tasks that require interactions between students and teachers and/or between students (van Oers, 1996). Students are also responsible for taking part in discussions related to the mathematics they are exploring. In this way, “meaningful learning is made dependent on the pupils’ opportunity to evaluate their own insights and ideas in critical comparison with culturally available concepts, norms, and methods” (van Oers, 1996, p. 92). Two questions that arise in response to this idea of engaging students in learning opportunities that promote social interaction are “[f]irst, in what kinds of practices do we wish students to participate; and second, what specific actions should a teacher take to improve students’ participation?” (Goos, 2004, p. 281).
A vast range of these mathematical instructional practices is available for teachers to include in their mathematics instruction, and as such is beyond the scope of this study. Instead, the review of mathematical instructional practices herein focuses on the instructional practices of engaging students in verbal communication and encouraging appropriate mathematical language as implemented during the Project M2 study (Casa et al., 2013; Gavin, Casa, Adelson, & Firmender, 2013; Gavin, Casa, Firmender, & Carroll, 2013).
Engaging Students in Verbal Communication
An important part of learning to be mathematical, whether in the primary school or in the university, is learning to take part in the discourses of mathematics, becoming both a consumer and a producer of texts that are recognized as legitimately mathematical within one’s community. (Burton & Morgan, 2000, p. 450)
For students, this mathematical community is the classroom where they can engage in both verbal and written forms of mathematical communication. In fact, NCTM (2000) identifies mathematical communication as one of the five mathematical processes that teachers should develop in students, and the Common Core Standards for Mathematical Practice (NGA & CCSSO, 2010) state that students should be provided opportunities to “construct viable arguments and critique the reasoning of others” (p. 6), as well as, “communicate precisely to others” (p. 7). Furthermore, the National Council of Teachers of English (NCTE; Whitin & Whitin, 2000) view verbal and written communication in mathematics as “tools for collaboration, discovery, and reflection” (p. 2). When students engage in mathematical communication, they can share their own ideas and analyze others’ ideas to further their understanding of mathematical concepts (NCTM, 2000).
Engaging students in mathematical communication requires the teacher to take on a specific role to facilitate discussions and foster the development of students’ communication skills. A teacher who implements such instructional practices listens to students’ ideas and challenges students’ thinking by asking them to justify their ideas. In addition, the teacher pursues certain ideas in more depth, provides additional information as necessary, and monitors students’ participation in discussions (NCTM, 1991; Walshaw & Anthony, 2008). For this to occur, students must view the classroom as a mathematical learning community or a community of practice in which they are “expected to propose and defend mathematical ideas and conjectures and to respond thoughtfully to the mathematical arguments of their peers” (Goos, 2004, p. 259). Furthermore, the student’s role in this mathematical learning community is to ask questions, pose problems, present and use varied strategies to justify solutions, consider examples and counterexamples, and examine mathematical evidence through listening and responding to others (NCTM, 1991). Engaging students in verbal communication or discourse about mathematics and encouraging students’ use of appropriate mathematical language are therefore two specific instructional practices related to the engagement of students as mathematicians during instruction and development of a community of learners.
In a classroom established as a community of practice, teachers engage students in discussions of mathematical ideas (Goos, 2004) or verbal communication. Teachers must teach students how to engage in these types of discussions, however (NCTM, 2000); and teachers’ use of this practice develops over time (Hufferd-Ackles, Fuson, & Sherin, 2004; Walshaw & Anthony, 2008). One way to facilitate students’ verbal communication in the mathematics classroom is the teacher’s use of talk moves (Chapin, O’Connor, & Anderson, 2009). These talk moves, developed by Chapin and colleagues (2009) and integrated previously in a mathematics curriculum for mathematically talented students (Gavin, Casa, Adelson, Carroll, & Sheffield, 2009; Gavin et al., 2007), have an identified purpose and help teachers facilitate students’ participation in mathematical discussions (see Table 1).
Talk Moves to Support Student Verbal Communication.
Note. Talk moves as implemented in Project M2 (Gavin, Casa, Chapin, & Sheffield, 2010, 2011a, 2011b, 2012a, 2012b, 2013) and adapted from Chapin, O’Connor, and Anderson (2009).
Although engaging students in verbal communication is a recommended instructional practice in mathematics education (NCTM, 1991; NGA & CCSSO, 2010), its use is varied. For example, approximately 36% of teachers (n = 99) indicated that they used critical discourse in their instruction very frequently, 24% frequently, 27% sometimes, and 13% seldom (McKinney, Chappell, Berry, & Hickman, 2009). Conversely, only 19% of teachers reported using “social interactions” in mathematics very frequently compared with 53% of teachers reporting that they seldom engage students in social interactions in mathematics (McKinney et al., 2009).
The body of research related to mathematics discourse is broad as the systematic reviews on the topic by Ryve (2011) and Walshaw and Anthony (2008) indicate. These reviews on mathematics discourse research focused on the conceptualization of discourse (Ryve, 2011) and how teachers promote discourse in the classroom (Walshaw & Anthony, 2008). Additional research, focused on how engaging students in verbal communication influences their mathematics achievement and concept development, has demonstrated that this instructional practice can be beneficial (Cross, 2009; Dixon, Egendoerfer, & Clements, 2009; Inagaki, Hatano, & Morita, 1998; Kazemi & Stipek, 2001). However, this research is often conducted with students beyond the primary grade level; only one of these studies (Dixon et al., 2009) examined its use with students in one of the primary grades (Grade 2).
Encouraging the Appropriate Use of Mathematical Language
A language specific to each academic discipline exists, and it is through this content-specific language that ideas are exchanged within each field. Wakefield (2000) proposed the idea that mathematics actually possesses attributes of a language such as “symbols and rules are uniform and consistent” (p. 272) and “communication requires encoding and decoding” (p. 273). This language of mathematics encompasses more than just numbers and symbols; it includes specific vocabulary that should be developed through instruction and experience.
Instructional practices for developing and encouraging the use of appropriate mathematical vocabulary include both direct and indirect methods (Marzano, 2004). Teachers can influence students’ mathematics achievement when they provide repeated exposure to vocabulary through modeling of the appropriate usage of mathematical language. Incidents of “teacher use” of words relating to number during instruction were significantly related to preschool-age students’ mathematics achievement (Ehrlich, 2007) and growth in mathematics knowledge (Klibanoff, Levine, Huttenlocher, Vasilyeva, & Hedges, 2006). By expecting students to engage in and correctly use mathematical vocabulary in both verbal and written communication, teachers can foster students’ development and appropriate use of mathematical vocabulary (Casa et al., 2013; Gavin et al., 2009; Gavin, Casa, Adelson, & Firmender, 2013; Gavin, Casa, Firmender, & Carroll, 2013; Thompson & Rubenstein, 2000). Another strategy for focusing on mathematical vocabulary development is the use of a “word wall.” A word wall may be used to display mathematical vocabulary related to a current unit of study (Rubenstein & Thompson, 2002), or can include both mathematical vocabulary words and corresponding picture cards that represent the words’ meanings (Casa et al., 2013; Gavin, Casa, Adelson, & Firmender, 2013; Gavin, Casa, Firmender, & Carroll, 2013).
Students may struggle with developing and using the language of mathematics for multiple reasons. For instance, some vocabulary words have different meanings in mathematical and non-mathematical contexts (e.g., volume as a measure of how much space a three-dimensional shape takes up, a measure of loudness, or an identification number for a periodical), vocabulary words that are homonyms in mathematical and non-mathematical contexts (e.g., sum and some), and vocabulary words that are used in more than one mathematical context (e.g., second to indicate a measure of time or an ordinal number; Rubenstein & Thompson, 2002; Schleppegrell, 2007).
The instructional practices of engaging students in verbal communication and encouraging students’ use of appropriate mathematical vocabulary share the characteristic of being related to language. However, each of these instructional practices also has distinguishing characteristics. When engaging students in verbal communication or discourse, the teacher must present students with a task worthy of discussion (Dixon et al., 2009; NCTM, 2000) and facilitate student discussion around making sense of the task, strategies for solving or working on the task, providing justification for strategies and solutions, and evaluating other students’ reasoning (NCTM, 2000). Although encouraging students’ use of appropriate mathematical vocabulary may be done during discussions, a distinction arises when teachers allow students’ use of informal language when discussing mathematical ideas instead of emphasizing mathematically correct vocabulary. In addition, the use of appropriate mathematical vocabulary is not limited to discussions. For example, teachers may use a word wall to display vocabulary words and corresponding iconic representations, students may represent mathematical vocabulary using pictures or symbols, and students may use appropriate vocabulary in their written mathematical communication.
Given the potential influence of these two instructional practices on students’ mathematical achievement and the distinctions between the two practices, the current study investigated the relationship between teachers’ implementation of them and student mathematics achievement as part of the Project M2 curriculum implementation study in separate analyses. The research questions that guided the study are as follows:
Is there a relationship between the level of teacher implementation of the verbal communication instructional practice during the Project M2 study and the change in students’ mathematics achievement as measured by the Open-Response Assessment?
Is there a relationship between the level of teacher implementation of the instructional practice to encourage mathematical language during the Project M2 study and the change in students’ mathematics achievement as measured by the Open-Response Assessment?
Research Design and Methods
The focus of the current study is the teachers’ implementation of the instructional practices embedded within the Project M2 mathematics units and how this may be related to the students’ mathematics achievement. Therefore, a summary of the Project M2 curriculum intervention and implementation is provided here to help situate the current study.
The Project M2 Curriculum Intervention and Implementation
As part of the quasi-experimental research design for the Project M2 study, teachers were randomly assigned to the intervention and control groups. Prior to the implementation of the curriculum, teachers in the intervention group attended a 4-day summer institute and participated in one additional day of professional development before the start of each unit. These professional development experiences focused on several aspects of the curriculum implementation, such as the key mathematical concepts in geometry and measurement addressed in the curriculum and the instructional practices embedded in the curriculum. Throughout the study, a member of the Project M2 professional development team made weekly visits to the classroom during instruction to monitor fidelity of implementation, which was documented. For each grade-level field test, the timeline for the implementation of the units was the same across classrooms for all grade levels during the specific Project M2 field test year.
The Project M2 units were developed to engage all students in the regular classroom in the investigation of advanced mathematics content. The project defines advanced-level curriculum as containing mathematics content that typically appears at higher grade levels and/or is studied in-depth with challenging tasks and problems. For example, the kindergarten measurement unit provides opportunities for students to explore the concept of the inverse relationship between the size of a unit used and the number of the unit needed to measure (i.e., the number of inches needed to measure a linear distance would be more than the number of feet to measure the same distance). This is a concept that would be considered advanced for kindergarten, as it does not appear in the CCSS (NGA & CCSSO, 2010) until Grade 2. In addition, only 36% of U.S. seventh graders and 48% of U.S. eighth graders were able to answer questions on this concept correctly during the 1995 Third International Mathematics and Science Study (TIMSS; Beaton et al., 1996).
The instructional practices embedded within the Project M2 units included engaging students in thinking and acting like mathematicians, creating a community of learners through mathematical communication, fostering verbal communication and written communication, using a talk frame graphic organizer to connect verbal and written communication, encouraging the use of appropriate mathematical vocabulary, and differentiating instruction (Casa et al., 2013; Gavin, Casa, Adelson, & Firmender, 2013; Gavin, Casa, Firmender, & Carroll, 2013). Although the Project M2 units encompassed all of the above instructional practices, the current study focuses on the teachers’ implementation of two of these: verbal mathematical communication and encouraging the use of appropriate mathematical vocabulary. As recommended by Ryve (2011), descriptions of these two instructional practices are provided.
The verbal communication (discourse) instructional practice is based on the communication process standard (NCTM, 2000), and the characteristics of this practice as it relates to Project M2 and the current study are as follows:
The communication is a community activity, so students discuss their ideas with both the teacher and their peers; students communicate to make sense of the problems and possible solutions; both the reasoning process (ideas on how to solve the problem) and the product (the answer) are valued, with the reasoning process overall being more highly regarded; students regularly justify their ideas; misunderstanding and misconceptions—both offered by students and introduced by teachers—are used as opportunities to guide the discussion; and several ideas are considered and revised as students get closer to accepted mathematical truths. (Gavin, Casa, Chapin, & Sheffield, 2011a, p. 13)
Through the curriculum, teachers were encouraged to engage students in verbal discussions of the mathematical concepts using five talk moves (see Table 1), as appropriate, to facilitate discussions about students’ mathematical ideas.
Encouraging the development and use of appropriate mathematical vocabulary is an instructional practice embedded in the Project M2 units that is implemented in several ways. For example, teachers are presented with the mathematical vocabulary that should be highlighted during each lesson, teachers model the use of correct mathematical vocabulary, and teachers encourage students to use the mathematical vocabulary words during discussions and in mathematical writing and representations. In addition, teachers display a mathematical word wall that includes two types of cards, mathematical vocabulary terms and pictorial representations of each term.
Sample
The sample for the current study included the 36 teachers and 601 students who previously participated in the Project M2 curriculum implementation research study as part of the field test intervention groups. Twelve teachers and 193 students from Grade 2, 12 teachers and 191 students from Grade 1, and 12 teachers and 217 students from kindergarten participated in the Project M2 field test intervention during the 2008-2009, 2009-2010, and 2010-2011 academic years, respectively. Missing data, however, is an issue with all educational research due to student mobility. In the cases where pre- and postassessment data were not available for a student, the data were eliminated listwise. As student mobility/absence was the reason for the missing data, it was assumed that missing data were random. In addition, the level of implementation for the instructional practices for these two Grade 2 classes had been documented using a previous version of the Project M2 Teacher Observation Scale that did not align exactly with the final version and which necessitated exclusion from the current study. Given that this was a paperwork issue, these data were assumed to be missing at random and eliminated listwise. Therefore, the final sample for the current study included 12 teachers and 210 students from kindergarten, 12 teachers and 186 students from Grade 1, and 10 of the teachers and 164 students from Grade 2 who participated in the Project M2 field test intervention during the 2008-2009, 2009-2010, and 2010-2011 academic years, respectively. This represents all of the kindergarten and Grade 1 teachers and students, but only 10 of the 12 Grade 2 teachers’ classes (Grade 2) that participated in the intervention of Project M2. Demographic details for the students and teachers in the original sample and the sample after missing data were eliminated listwise are presented in Tables 2 and 3.
Student Demographics for Project M2 Intervention Groups—Kindergarten and Grades 1 and 2.
Note. ESL = English as a second language.
Unavailable for 16 students.
Unavailable for 17 students.
Teacher Demographics for Project M2 Intervention Groups—Kindergarten and Grades 1 and 2.
Data Collection
The quantitative data analyzed in this study were collected as part of the Project M2 research study during the field test of the curriculum units. A description of the development and use of the instruments to collect these data are as follows.
Project M2 Open-Response Assessments
Due to the lack of national, standards-based, geometry and measurement assessments for kindergarten and Grades 1 and 2 that include open-ended questions, the Open-Response Assessment for each grade level was constructed by the researchers (Casa, Copley, & Gavin, 2010; Osiecki, Casa, & Gavin, 2009; Spinelli, 2008). These assessments were used as part of the Project M2 research study to assess the mathematics achievement in the areas of geometry and measurement for kindergarten, Grade 1, and Grade 2 students.
The development of the Open-Response Assessments for each grade level began with an extensive analysis of the geometry and measurement content for the appropriate grade level. Open-response style items for each of the assessments were constructed for a pilot test. The pilot versions of the Open-Response Assessments were sent to reviewers in the fields of mathematics, mathematics education, and early childhood education who rated each item on three characteristics: (a) identification of the appropriate mathematical content area, geometry, or measurement; (b) relevance to the mathematical content area; and (c) difficulty of the item for the specified grade level. The scores for each item were analyzed to determine which items would be retained and/or revised for the final versions of the Open-Response Assessments. The reviewers were also asked to comment on the wording and appropriateness of the questions, developmentally, for the specified grade level. Items were revised based on this review and content analysis and a pilot test was developed. The pilot tests of the Open-Response Assessments were conducted with two groups of students, a group who had experienced the pilot version of the Project M2 kindergarten and Grades 1 and 2 geometry and measurement units and a group who had experienced the regular mathematics curriculum. The reliability coefficients for each grade level’s Open-Response Assessment are α = .81 for kindergarten, α = .79 for Grade 1, and α = .82 for Grade 2.
The kindergarten Open-Response Assessment contained items that were administered in small group and individual formats; Grades 1 and 2 Open-Response Assessments were administered in a whole-class setting. In all cases, the classroom teacher was present during the administration of the assessment, but did not have access prior to or after the assessment. Two trained members of the Project M2 research team scored all students’ Open-Response Assessments using the previously established rubrics. Any discrepancy in item scores resulted in the item being scored by a third Project M2 research team member.
ITBS mathematics
The ITBS Mathematics assessment is a norm-referenced standardized assessment of student achievement in mathematics. The mathematics achievement of the Grade 2 students was assessed using the Mathematics Concepts subtest, Level 8, Form A. The reported reliability of this subtest is .81 (ITBS, 2003). The mathematics achievement of the Grade 1 students was assessed using the Mathematics test, Level 6, Form A. The reported reliability of this test is .79 (ITBS, 2003). The mathematics achievement of kindergarten students was assessed using the ITBS Mathematics test, Level 5R, Form A. The reported reliability of this test is .80 (ITBS, 2003). The assessment was done in a whole-class setting during which the classroom teacher was present.
Both the Open-Response and ITBS assessments for each individual grade level were administered across classrooms within the same 3-week time frame during the Project M2 field test year. For the Kindergarten Field Test, the assessments were administered in October for pretest and in May for posttest. The implementation of the kindergarten units took place in two 6-week time periods between January and April. For the Grades 1 and 2 Field Tests, the assessments were administered in September for pretest and in May for posttest in the respective field test years. The implementation of the Grade 1 and 2 units took place in two 6-week time periods between October and April.
Project M2 Teacher Observation Scale
This scale was developed as one of several measures used to monitor the fidelity of implementation of the Project M2 curriculum and embedded instructional practices and to assist with professional development during the field test. These items were teacher behaviors that would be evident if the teacher was implementing the specific instructional practices (Gavin & Casa, 2008). A trained Project M2 professional development staff member completed this treatment fidelity scale after each weekly classroom observation. The extent of the teachers’ implementation of the verbal communication instructional practice, including the talk moves, was recorded for the nine items (see Appendix A). For example, an item to measure the implementation of verbal communication was “[The talk move,] agree/disagree and why was used to have students apply their understanding to someone else’s thoughts and defend their position” (Gavin & Casa, 2008, p. 2). The verbal communication item scores were coded as 1 for “yes,” 0.5 for “somewhat,” and 0 for “no.” The mathematical language instructional practice was measured with three items (see Appendix B), an example of which is “The teacher or students referred to the word wall” (Gavin & Casa, 2008, p. 3). The use of the teachers’ implementation of this instructional practice was observed, and the items were coded as 1 for “yes” and 0 for “no.” See Appendices A and B for the complete set of observation items related to these two instructional practices.
Data Analysis
The students in the intervention group were nested within classrooms and experienced the Project M2 mathematics content and instructional practices as implemented by their teacher. It is therefore likely that students within classrooms experienced some level of statistical dependence. For this reason, the intra-class correlation (ICC) or the proportion of variance that is between classes on the dependent variable (Raudenbush & Bryk, 2002) was examined, and hierarchical linear modeling (HLM) was used in the data analysis to account for the non-independence (McCoach & Adelson, 2010; Raudenbush & Bryk, 2002) of the students on the dependent variable. ICCs of between .10 and .20 are common in educational research (McCoach, 2010). In the HLM analyses, the students from all grade levels (n = 560) were included at Level 1 and the classes/teachers represented the clusters (n = 34), or Level 2 units. With the sample including over 30 Level 2 clusters, the estimates of the parameter coefficients and variance components should not be biased (Maas & Hox, 2005).
The goal of the HLM analyses was to determine the relationship between the teachers’ implementation of the specific Project M2 instructional strategies and students’ mathematics achievement. To do this, we ran a series of multilevel models with HLM 7 (Raudenbush, Bryk, & Congdon, 2010) using restricted maximum likelihood (REML) estimation due to the small sample size (Raudenbush & Bryk, 2002). In addition, three variables were calculated, the students’ percentage gain scores pre- to posttest on the Open-Response Assessment and the teachers’ levels of implementation scores for each of the instructional practices being investigated, verbal communication and mathematical language. The students’ percentage gain scores pre- to posttest on the Open-Response Assessment were calculated using the kindergarten and Grades 1 and 2 student pre- and posttest scores on the Open-Response Assessment. Due to the possible unreliability of gain scores, the variances in the students’ scores on the Open-Response Assessment at pre- and posttest were examined and determined to be unequal. This means that the variance at posttest (0.037) was larger than variance at pretest (0.011; Fulcher & Willse, 2007), which would be expected of reliable gain scores. Descriptive statistics for variables entered in the HLM are provided in Table 4.
Descriptive Statistics for Variables in the Contextual Models.
Note. ITBS = Iowa Tests of Basic Skills.
The mean for the student percentage gain score on the Open-Response Assessment indicates that the average change in the students’ scores on the Open-Response Assessments was 0.43 or 43%.
The verbal communication and mathematical language instructional practice scores for each teacher were based on the number of observations completed. The average number of observations across grade levels and teachers was 10. The verbal communication instructional practice score for each teacher was calculated by using the mean of the verbal communication items for each teacher’s observation that was completed for at least seven of the nine items in the verbal communication section of the scale. The mathematical language instructional practice score was calculated for each teacher first by calculating the mean of the mathematical language section for each teacher’s observation that was completed for at least two of the three items in the mathematical language section of the scale. The descriptive statistics for these variables are provided in Table 4. In addition, the correlation between the teacher implementation scores for verbal communication and mathematical language was calculated (r = .500, p = .003), indicating a moderate correlation between the two variables. The verbal communication and mathematical language instructional practice scores for the teachers were entered into two separate HLM analyses as a Level 2 predictor of the students’ percentage gain score on the Open-Response Assessment.
After calculating the variables, we ran a completely unconditional model that contained only the dependent variable, student percentage gain score on the Open-Response Assessment at Level 1. Results from the unconditional model were used to determine the ICC, which was .30. This indicates that 30% of the variance in the percentage gain scores on the Open-Response Assessment is between classes. To enhance the precision with which estimates are made in the model (Raudenbush, 1997), the ITBS standard score at pretest was entered as a grand mean centered covariate into the random coefficients model at Level 1. This covariate explained an additional 10% of the between-class variance. We then estimated a full two-level model that included two dummy variables to represent the three grade levels of the teachers. The coding was such that Kindergarten was the referent grade level. Also included at Level 2 in the full, contextual model was the focal variable, teachers’ level of implementation score for verbal communication or encouraging mathematical language for the instructional practice, as a grand mean centered, Level 2 predictor.
Results
The separate models that were constructed to determine if a relationship exists between the students’ percentage gain score on the Open-Response Assessment and the teachers’ implementation scores for the instructional practices of verbal communication and mathematical language were based on the same unconditional and random coefficients models because the verbal communication and mathematical language implementation scores were entered into the model as Level 2 predictors.
The unconditional model with the students’ (K-Grade 2) percentage gain score on the Open-Response Assessments as the dependent variable and no predictors at either level in the model was analyzed first. The student’s percentage gain score on the Open-Response Assessment was a function of the intercept (γ00), the student-level residual (r), and the teacher-level variance (τ00). In the unconditional model with no other predictors, a student’s predicted percentage gain from pre- to posttest on the Open-Response Assessment would be 43%; this is statistically significantly different from zero (p < .001).
The students’ standard scores on the Kindergarten ITBS Mathematics, Grade 1 ITBS Mathematics, and Grade 2 ITBS Math Concepts subtests at pretest were entered at Level 1 in the random coefficients model as a predictor of the percentage gain score on the Open-Response Assessment and to account for differences between students on initial mathematics achievement. The ITBS Mathematics standard score at pretest was entered as a grand mean centered, Level 1 covariate. Analysis of the results of the random coefficients model indicated that the ITBS standard score at pretest was a statistically significant predictor of the percentage gain score on the Open-Response Assessment (p < .001); however, the slope for the ITBS standard score at pretest (τ11) was not statistically significant (p = 0.083). It was therefore determined that the ITBS pretest slope would not be allowed to randomly vary and the random coefficients model was run again. The results of the random coefficients model are displayed in Table 5. The addition of the ITBS standard score at the time of pretest as a covariate in the random coefficients model explains an additional 10% of the between-class variance over the unconditional model.
Summary of REML Parameter Estimates for the HLM Models for Percentage Gain Scores on the Open-Response Assessment.
Note. The chi-square test for homogeneity of variances indicated that the variances for the classes are homogeneous for both the verbal communication model, χ(.05, 33) = 7.00, p > .500, and the mathematical language model, χ(.05, 33) = 7.00, p > .500. REML = restricted maximum likelihood; HLM = hierarchical linear modeling; ITBS = Iowa Tests of Basic Skills.
p < .001. ***p < .001.
Verbal Communication
To determine if there is a relationship between the teachers’ implementation of the verbal communication instructional practice and the percentage gain score on the Open-Response Assessment, the teachers’ verbal communication implementation score variable (M = 0.71, SD = 0.16) from the Project M2 Observation Scale was entered as a grand mean centered, Level 2 predictor in a contextual model based on the final random coefficients model. The two dummy variables representing the grade level of the teachers were also entered into the model as a Level 2 predictor of the percentage gain score on the Open-Response Assessment.
Analysis of the results of the contextual model indicated that the teachers’ verbal communication implementation score and the two grade-level variables were not significant predictors of the slope for the ITBS standard score at pretest covariate. The fixed effects of verbal communication implementation score, and the two grade-level variables for the effect of the slope of the ITBS pretest were therefore eliminated from the model based on these results. This revised contextual model was then run, and the parameters from the final contextual model are shown in Table 5. The final contextual model including the verbal communication implementation score as a Level 2 predictor is as follows:
The final contextual model that includes the teachers’ verbal communication implementation score and the teachers’ grade level as predictors of the students’ percentage gain score on the Open-Response Assessment accounts for an additional 0.3% of the between-class variance over the random coefficients model, in the students’ percentage gain score on the Open-Response Assessment. Given that γ00 = 0.53, a kindergarten student with a mean ITBS standard score at pretest and whose teacher at the mean on the verbal communication scale would be predicted to gain 53% pre- to posttest on the Open-Response Assessment. γ02 is the parameter of interest in this analysis and represents the predicted change in the percentage gain score on the Open-Response Assessment as a teacher’s verbal communication implementation score increases, after controlling for grade level and the ITBS standard score at pretest. The verbal communication implementation score is a significant predictor of the student percentage gain score on the Open-Response Assessment; a kindergarten student at the average of the ITBS standard score at pretest and whose teacher was rated to have always implemented the verbal communication practice (scored a 1 on each item) on the Project M2 Observation Scale would be predicted to gain 80% pre- to posttest on the Open-Response Assessment.
Mathematical Language
To determine if there is a relationship between the teachers’ implementation of the mathematical language instructional practice and the percentage gain score on the Open-Response Assessment, the teachers’ mathematical language implementation score (M = 0.60, SD = 0.20) from the Project M2 Observation Scale was entered as a grand mean centered, Level 2 predictor in a contextual model based on the appropriate random coefficients model. The grade-level variables were also entered into the model as Level 2 predictors of the percentage gain score on the Open-Response Assessment.
Analysis of the results of the contextual model indicated that the teachers’ mathematical language implementation score and the grade-level variables were not significant predictors of the slope of the ITBS standard scores at pretest. Therefore, fixed effects of the grade-level variables and the teachers’ mathematical language implementation score on the effect of the slope of the ITBS standard scores at pretest were eliminated from the model. The revised contextual model was then run and the parameters from the final contextual model are shown in Table 5. The final contextual model that included the teachers’ implementation score for the mathematical language instructional practice is as follows:
The final contextual model that includes the teachers’ mathematical language implementation score and the teachers’ grade level as predictors of the students’ percentage gain score on the Open-Response Assessment accounts for an additional 0.3% of the between-class variance, over the random coefficients model, in the students’ percentage gain score on the Open-Response Assessment. The γ00 parameter is interpreted as the predicted percentage gain score on the Open-Response Assessment for a kindergarten student with a mean ITBS standard score at pretest and whose teacher is at the mean on the mathematical language scale. The predicted percentage gain from pre- to posttest on the Open-Response Assessment would be 54%. The parameter of interest in this analysis is γ03 and represents the predicted change in the percentage gain score on the Open-Response Assessment as a teacher’s mathematical language implementation score increases, after controlling for grade level and the ITBS standard score at pretest. The mathematical language implementation score is a significant predictor of the student percentage gain score on the Open-Response Assessment; a kindergarten student at the average of the ITBS standard score at pretest and whose teacher was rated to have always implemented the mathematical language practice (scored a 1 on each item) on the Project M2 Observation Scale would be predicted to gain an additional 72% pre- to posttest on the Open-Response Assessment.
Summary of the HLM Results
To determine if a relationship existed between the implementation of the verbal communication and mathematical language instructional practices and student achievement, as measured by the percentage gain score on the kindergarten and Grades 1 and 2 Open-Response Assessments, a series of multilevel models were constructed, and the data were analyzed. The results of the analyses indicated that both the scores for the implementation of the instructional practices, verbal communication and mathematical language, were statistically significant predictors of the student percentage gain scores on the Open-Response Assessments after accounting for the grade level of the teacher and the students’ ITBS standard scores at pretest. Therefore, the higher the teacher’s mean implementation score for the instructional practices of verbal communication and mathematical language, the higher the student percentage gain scores on the Open-Response Assessment would be for that teacher’s students.
Discussion
At a time when this country’s educational climate is focused on accountability and reform (NCTM, 2000; NGA & CCSSO, 2010), investigations into what may influence student performance and achievement are important. Investigating specific reform practices separately contrasts with other research on instructional practices of the reform movement in mathematics, which often has been conducted using a framework that examines the mathematical reform practices as a whole (Gimbert et al., 2007; Huffman et al., 2003; Spillane & Zeuli, 1999). The current study was conducted in an attempt to determine the relationship between specific instructional practices that teachers implemented as part of a curriculum research study and students’ achievement in geometry and measurement as measured by the kindergarten and Grades 1 and 2 Open-Response Assessments.
The body of research on verbal communication or discourse in mathematics classrooms attempts to address a number of issues related to this instructional practice. Some of these issues include how it is conceptualized (Ryve, 2011), how teachers develop the use of the practice (Hufferd-Ackles et al., 2004; Walshaw & Anthony, 2008), how teachers can engage students in verbal communication (Chapin et al., 2009; Walshaw & Anthony, 2008), how often teachers use the instructional practice (McKinney et al., 2009), and does the teachers’ use of the instructional practice influence student achievement or understanding (Cross, 2009; Dixon et al., 2009; Inagaki et al., 1998; Kazemi & Stipek, 2001). Engaging students in verbal communication is an instructional practice that varies in how often teachers report using it (McKinney et al., 2009). In the current study, the mean observed teacher implementation score for this instructional practice was 0.71 on a 0 to 1 scale, meaning that on average the teachers engaged students in verbal communication during 71% of the observations. However, the range of the scores was 35% to 96%, indicating a wide variety in the implementation of this instructional practice across teachers. While McKinney and colleagues (2009) noted a wide range based on teacher reported data, the current study also documented that teachers varied in their level of implementation of the practice even when it was expected.
It has been reported that engaging students in verbal mathematical communication may be beneficial for students’ mathematics achievement and understanding (Cross, 2009; Dixon et al., 2009; Inagaki et al., 1998; Kazemi & Stipek, 2001). These results represent a range of grade levels: Grade 2 (Dixon et al., 2009), Grades 4 and 5 (Inagaki et al., 1998; Kazemi & Stipek, 2001), and Grade 9 Algebra I (Cross, 2009). The results of the current study add to the evidence presented in these previous studies in support of engaging students in verbal communication as indicated by the positive relationship found between students’ mathematics achievement and the teachers’ use of the verbal communication instructional practice. However, the current study addresses the use of verbal communication with students in lower grade levels, specifically kindergarten and Grade 1. As a whole, previous and current results suggest that engaging students in verbal communication may be positively related to students’ mathematics achievement.
Previous research regarding the use of mathematical language has focused on teachers’ use of math talk during class discussions (Ehrlich, 2007; Klibanoff et al., 2006). The use of the mathematical language instructional practice in the context of the Project M2 curriculum field test differs from the previous research in that the focus is not solely on the teachers’ use of mathematical vocabulary. Instead, the mathematical language instructional practice is implemented when the teacher encourages students to use appropriate mathematical vocabulary in their verbal and written communication, exposes students to mathematical vocabulary through the use of a mathematical word wall, and models the use of vocabulary. Together, the current findings and previous research suggest that in addition to teachers’ use of mathematical vocabulary in discussions (Ehrlich, 2007; Klibanoff et al., 2006), encouraging students to use appropriate mathematical vocabulary in class discussions and in their writing may also be beneficial to growth in mathematics achievement.
Limitations to these results include that the intent of the Project M2 Observation Scale, as a part of the Project M2 field tests, was to serve as one of several measures to document the fidelity of implementation of the curriculum and instructional practices during the study and the level of professional development provided throughout the Project M2 study. Ideally, the instructional practices on the observation scale would be documented using a scale with the same number of categories. Also, although the members of the professional development team were trained to use the Project M2 Observation Scale, the establishment of interrater reliability would be helpful in the justification of results based on the observation scale.
The level of professional development during the Project M2 study and the presence of a Project M2 professional development staff member during implementation may be considered a limitation as it could have had an effect on the teachers’ use of the instructional practices during the lessons that were observed, recorded with the Project M2 Observation scale, and used to determine the level of implementation score for each of the instructional practices. It is possible that teachers were more likely to use the instructional practices during those lessons, depending on their perceptions of what was expected of them. However, the frequency of the visits (each professional development staff member made weekly visits to the classroom during the study) may have lessened this effect. In addition, the instructional practices were embedded in a curriculum that all of the participating teachers were using. It may be the case that the level of support during the field test and in the curricular materials had an influence on the teachers’ implementation of the instructional practices or that there are other aspects of the curriculum that influence the students’ mathematics achievement. Further investigations of the instructional practice across multiple curricula are warranted.
Despite these limitations, the current study provides initial evidence that engaging students in the practices of verbal communication and using appropriate mathematical vocabulary may be beneficial to students’ mathematics achievement. These results lend support to the integration of the CCSS for Mathematical Practice (NGA & CCSSO, 2010), which have been adopted by over 40 states across the country. To date, research on the effectiveness of the implementation of these Standards for Mathematical Practice still needs to be done. More specifically, the instructional practices investigated in this study, engaging students in verbal communication and encouraging use of appropriate mathematical vocabulary, relate to two of the Standards for Mathematical Practice: (a) construct viable arguments and critique the reasoning of others and (b) attend to precision, respectively (NGA & CCSSO, 2010). However, as the CCSS for mathematics are being implemented in classrooms across the country, further investigation into the effectiveness of these Standards for Mathematical Practice in relation to students’ mathematics achievement is necessary.
Footnotes
Appendix A
Appendix B
Authors’ Note
The opinions, conclusions, and recommendations expressed in this article are those of the authors and do not necessarily reflect the position or policies of the National Science Foundation.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Part of the work reported herein used archived data collected during a project funded by the National Science Foundation (Grant DRL-0733189).
