A Guide for Evaluating the Mathematics Programs Used by Special Education Teachers

Select a “Big Idea” of Mathematics

The first step is to identify a critical mathematical topic or big idea recognized in state (e.g., Texas Essential Knowledge and Skills; Texas Education Agency, 2012) or national (Common Core State Standards Initiative, 2010) mathematical content standards. Big ideas represent the critical concepts and skills of mathematics that are considered essential for students’ mathematics proficiency (Carnine, 1997; Coyne et al., 2011). An example of a big idea of early mathematics is place value. Knowledge of place value is critical to learning the base-10 system, composing and decomposing quantities, and using the arithmetic operations of addition, subtraction, multiplication, and division (Caldwell, Karp, Bay-Williams, & Zbiek, 2011).

Locate All Pertinent Lessons

The second step is to locate all lessons that address the targeted big idea, including the first lesson that introduces the concept or skill. The quality of introductory lessons is critical for ensuring high rates of student success. For example, if a program lacks sufficient instructional examples to introduce place value, the probability is high that students with or at risk for LD in mathematics will have difficulty gaining initial understanding of this big idea. In addition to the introductory lesson, it is also important to identify all subsequent lessons that address the targeted concept or skill.

Evaluate the Instructional Components

The third step is to evaluate the identified lessons to determine the extent to which they address three validated principles of explicit mathematics instruction. These principles are drawn from prior reviews of mathematics programs (Bryant et al., 2008; Carnine et al., 1997; Doabler et al., 2012; Sood & Jitendra, 2007) and the growing literature base on explicit mathematics instruction (Archer & Hughes, 2009; Becker, Engelmann, Carnine, & Rhine, 1981; Carnine, 1997; Chard & Jungjohann, 2006; Clarke et al., 2015; Clements, Agodini, & Harris, 2013; Coyne et al., 2011; Dennis et al., 2016; Gersten, Beckmann, et al., 2009; Gersten, Chard, et al., 2009). They include (a) instructional scaffolding, (b) student practice opportunities, and (c) judicious review.

To assess and evaluate the presence of the principles in the identified lessons, teachers should utilize the evaluation guide shown in Figure 1. Broadly, the guide addresses the three principles of explicit mathematics instruction. Across the principles are 13 evaluation questions that address key features of explicit mathematics instruction. Teachers should consider each question to obtain a comprehensive snapshot of their mathematics program.

OPEN IN VIEWER

Figure 1.

Mathematics Program Evaluation.

Each evaluation question is scored on a specific 3-point rating scale (see Figure 1). For example, a second grade mathematics program will receive a rating of 1, the lowest rating, if lessons pertaining to multidigit addition with regrouping offer a restricted range of instructional examples (i.e., one problem type). A rating of 2 suggests that an instructional principle is inconsistently present in the targeted lessons. For example, a program will achieve a score of 2 if it offers a limited number of instructional examples (i.e., 1 or 2) in the identified lessons. A rating of 3, the highest score, indicates that the principles are consistently applied across the identified lessons. For example, a program will achieve a score of 3 if frequent opportunities for student mathematics verbalizations (i.e., >10 per lesson) are included in the lessons targeting place value concepts.

After completing the evaluation, teachers should consider the rated score for each evaluation question. Ratings that fall at or below 2 may indicate a need to enhance the identified lessons. In the next section, we provide an example for how teachers can apply the evaluation guide to evaluate and assess a mathematics program used to teach students with or at risk for LD in mathematics.

Strengths of the Lesson

As depicted in Figure 2, there are several notable strengths of the lesson. First, the lesson offers a brief warm-up activity (3–5 minutes) to help students connect previously learned material (i.e., subtraction with no regrouping) with the concept to be introduced (i.e., subtraction with regrouping). A second strength of the lesson is that it provides an opportunity to directly teach key mathematics vocabulary terms (Powell & Driver, 2015). For example, to gain conceptual understanding of the targeted content, students need to understand how to apply key vocabulary such as tens, hundreds, and regrouping.

In addition, the lesson offers specific guidelines for teachers to overtly demonstrate for students how to solve multidigit subtraction with regrouping problems. Converging evidence suggests that students with disabilities in mathematics are more successful in developing mathematics proficiency when they are provided unambiguous demonstrations and explanations of new mathematical concepts and skills (Clarke et al., 2015; Gersten, Chard, et al., 2009). Teachers can utilize the lesson’s guidelines to make visible what they expect students to learn. One way teachers can do this is through “think-alouds.” Think-alouds make known a teacher’s thought processes for solving complex mathematics problems, such as multidigit subtraction problems with regrouping. When done well, think-alouds can serve as a valuable instructional tool for teachers to make their reasoning with mathematical concepts, procedures, and vocabulary both public and accessible (Archer & Hughes, 2011; Gersten, Beckmann, et al., 2009). Figure 3 shows an example of a teacher using a think-aloud to overtly demonstrate how to solve a multidigit subtraction problem with a zero in the tens place.

OPEN IN VIEWER

Figure 3.

Example of a Teacher Think-Aloud.

Shortfall 1: Instructional Examples

One critical shortfall is that the instructional examples offered in the lesson are limited in number (i.e., only two examples) and restricted to only one problem type (i.e., regrouping from the tens place). Consequently, it is rated a 1 for Questions 4 and 5 on the evaluation guide. When mathematics programs offer little to no instructional scaffolding, it often forces students with or at risk for LD in mathematics to solve complex mathematical problems on their own without sufficient support (Rosenshine, 2012). This can be problematic, particularly for students who have already faced a long line of failure and frustration with mathematics. Therefore, in the event of this shortfall it is critical that teachers include more guided teaching examples that initially offer a high level of instructional scaffolding with a gradual withdraw of that support as students became more independent in their learning. While the number of additional instructional examples to provide will vary based the complexity of the targeted mathematics content, the amount should be sufficient to promote students’ conceptual understanding of targeted mathematics content. Moreover, the examples should be purposefully sequenced across a series of lessons by introducing easier problem types before more complex ones. This way, instruction does not overwhelm students by introducing complex problem types during the initial lesson or activity.

For example, multidigit subtraction problems that contain zeroes in the tens place often pose difficulties for students because these problem types require regrouping across the zero (Fuson & Beckmann, 2012). Therefore, to better ensure that students obtain a high success rate with these particular subtraction problems, teachers should introduce them after students have mastered multidigit subtraction problems that contain numerals other than zeroes in the tens place. Figure 4 shows an example of how teachers can systematically sequence the introduction of different problem types across multiple days of instruction.

OPEN IN VIEWER

Figure 4.

Sequence of Subtraction with Regrouping Problems.

Shortfall 2: Use of Concrete Manipulatives

A second shortfall in the lesson is a lack of structured opportunities for students to receive hands-on experiences with visual representations of mathematical ideas or conceptual tools. The practice opportunities offered in the lesson only permitted students to solve subtraction problems on their worksheets; and it is rated a 1 for Questions 8 and 9 on the evaluation guide. Because such experiences are considered critical for developing conceptual understanding and building mathematical proficiency (NRC, 2001; Wu, 1999), it is important that teachers strategically integrate conceptual tools into their instruction when such opportunities are missing from mathematics programs. Teachers can do this by integrating the selected tools in the first lesson that introduces the targeted concept or skill. Teachers should explicitly demonstrate for students the purpose of the conceptual tools, such as using place value blocks to solve multidigit subtraction problems. Then, teachers should gradually withdraw such tools from instruction after students have begun to grasp a general understanding of the targeted concept or skill. Eventually, this systematic withdraw will help students with or at risk for LD in mathematics to build an important connection between the conceptual and abstract strands of mathematics (NRC, 2001).

Shortfall 3: Student Mathematics Verbalizations

A review also identified that the lesson’s independent practice and review activity was missing frequent opportunities for students to verbalize their mathematical thinking of targeted mathematics content. As such, it is rated a 1 for Question 10 on the evaluation guide. A growing body of research suggests that mathematics verbalizations are associated with increased student mathematics achievement (Clements et al., 2013; Gersten, Beckmann, et al., 2009; Gersten, Chard, et al., 2009). Therefore, when mathematics programs lack verbalization opportunities, it is critical for teachers to frequently engage students with or at risk for LD in mathematics in structured math talk. At the very least, teachers should require students to answer basic factual and procedural questions, such as “Can you tell me the next step in solving the problem?”

It is also important to engage struggling learners in more productive mathematical discourse or math talk. However, mathematics programs often fail to include more cognitively demanding verbalization opportunities (Bryant et al., 2008; Sood & Jitendra, 2007). In light of this, verbalization opportunities that help students access the conceptual level of mathematics may have to be incorporated into mathematics programs. For example, a teacher might have her students verbally justify why mathematical statements, such as “300 minus 246 equals 54,” are true. To stimulate deeper mathematics discussions, teachers might also provide follow-up generalization questions such as, “If 300 minus 246 equals 54 then what does 200 minus 146 equal?” Because this type of math talk can be difficult for students with or at risk for LD in mathematics, teachers should be prepared to guide mathematics verbalization opportunities by simultaneously stating the responses with their students (e.g., “Say it with me. We solved this problem by . . .”).