Abstract
The current article focuses on efforts to understand how a basic learning process—comparison—can be harnessed to improve learning, especially mathematics learning in schools. To harness the power of comparison in instruction, we must investigate three core decisions: what, when, and how to compare. Comparing different strategies for solving the same problem or easily confusable problem types is particularly effective for supporting mathematics learning. Comparing examples early in the learning process can be challenging, but delaying comparison can reduce procedural flexibility. Indeed, comparison is resource demanding, so it is more impactful when carefully supported (e.g., side-by-side visual presentation, explanation prompts). To bridge from research to practice, we communicated research findings to teachers and policymakers and developed curricular materials, instructional routines, and professional-development materials to help math teachers leverage these learning processes. We conclude this review with key open questions.
Students too often memorize ideas without understanding the ideas or being able to flexibly apply them to new contexts. For example, only 11% of 15-year-olds from around the world could work strategically using well-developed thinking and reasoning skills to solve math problems; in the United States, only 5% could (OECD, 2016). Cognitive-science research provides many insights into potential ways to improve teaching and learning in schools, but those insights infrequently make their way into classrooms (The National Academies of Sciences, Engineering, and Medicine, 2018). In the current article, we focus on our efforts to understand how one basic learning process—comparison—can be harnessed to improve learning, especially mathematics learning in schools.
We often learn through comparison. For example, we compare different brands of products, we compare one treatment option to another, and we compare new words, objects, and ideas with ones we already know. A study aggregating the results of previous studies found that comparison improved learning across a range of topics, including math, science, and language (Alfieri, Nokes-Malach, & Schunn, 2013). Further, in mathematics education, comparison is considered an important and effective instructional approach, on the basis of observations of expert teachers (National Council of Teachers of Mathematics [NCTM], 2014).
Theoretically, comparison promotes analogical reasoning. When studying individual examples, people often focus on unimportant, surface features that are not relevant to the target concepts and procedures (Gick & Holyoak, 1983). Comparing two examples leads people to create an analogy between them, helping them notice important, deep structural aspects of the examples; identify meaningful similarities and differences; and highlight their shared relational structure (Gentner, 1983; Gentner, Loewenstein, & Thompson, 2003; Schwartz & Bransford, 1998). In turn, this facilitates people’s transfer of the knowledge to new situations and problems (Gick & Holyoak, 1983). However, comparison often requires substantial mental effort (Richland, Begolli, Simms, Frausel, & Lyons, 2016). Thus, evidence-based guidelines are needed for using comparison effectively in instruction.
Research on engaging in comparison to promote mathematics learning in the past 15 years has revealed new insights about what, when, and how to compare. In this research, target learning outcomes were procedural knowledge (i.e., knowledge of what actions to take to solve problems, such as equation-solving procedures), procedural flexibility (i.e., knowledge of multiple procedures and when to use each), and conceptual knowledge (i.e., knowledge of abstract and general principles, such as equivalency; Star, Rittle-Johnson, & Durkin, 2016). Examples of items we have used to assess each knowledge type are shown in Table 1. Conceptual and procedural knowledge are typically the focus of mathematics instruction and assessment, whereas procedural flexibility is not, despite evidence that procedural flexibility is an important component of mathematics expertise (Star, 2005).
Sample Items for Assessing Procedural Knowledge, Procedural Flexibility, and Conceptual Knowledge
Note: Table adapted from Rittle-Johnson, Star, and Durkin (2009).
What to Compare?
Integrating the cognitive-science and mathematics-education literatures highlighted the importance of considering what is being compared. This is a fundamental aspect of comparison yet one that historically had not received focused attention in cognitive-science research. Comparing multiple strategies—that can both be correct or can vary in correctness—or comparing easily confusing problem types are the most promising avenues for promoting mathematics learning identified to date (see Rittle-Johnson & Star, 2011).
Comparing multiple correct strategies
Comparing multiple strategies for solving the same problem has rarely been studied in cognitive-science research, but it is the type of comparison most often used by expert mathematics teachers and recommended in mathematics-education standards (NCTM, 2014). Often, two correct strategies for solving the same problem are compared, such as two strategies for solving an equation.
To experimentally evaluate the impact of comparing multiple correct strategies on student mathematics learning, in a series of five studies, we redesigned two or three math lessons on a topic and implemented these lessons during mathematics classes. Students who compared multiple strategies saw the same problem solved two different, correct ways on each page of a workbook with questions asking them to compare the two (see Fig. 1). Students in the sequential condition studied one example per page with questions asking them to explain that individual strategy. Across these studies, with hundreds of students, students who compared multiple correct strategies gained greater procedural flexibility, often gained greater procedural knowledge, and sometimes gained greater conceptual knowledge than students who studied the same examples sequentially (Rittle-Johnson & Star, 2011; Star et al., 2016). Students’ explanations during the intervention confirmed that those who compared strategies often compared the similarities and differences in solution steps across examples and evaluated their efficiency and accuracy; in turn, frequency of making explicit comparisons during the intervention was predictive of learning outcomes. Overall, comparing correct strategies helped students differentiate important characteristics of strategies and when and why one strategy was better for solving a particular problem.
Sample pages from intervention packet for (a) compare and (b) sequential conditions. Figure adapted from Rittle-Johnson and Star (2007).
We evaluated the effectiveness of this type of comparison relative to the most commonly studied form of comparison in the cognitive-science literature—comparison of problems with different surface features, such as story context, but the same underlying solution strategy (i.e., isomorphic problems; Gick & Holyoak, 1983). We created two versions of isomorphic problems solved with the same strategy, given mixed evidence on how similar the problems should be (see Rittle-Johnson & Star, 2009). In one condition, the isomorphic problems were the same problem type and thus had very similar surface features and a very similar solution strategy, for example,
Comparing confusable problem types
Rather than comparing isomorphic problems, comparing easily confusing problem types (which are not isomorphic) has promise for promoting math learning. In laboratory research, college students who compared examples of algebra word problems from different categories (e.g., dilution vs. catch-up) were better able to sort new examples by problem category, rather than surface features, and to describe their structural features than students who studied examples one at a time (Cummins, 1992). Similarly, across two studies conducted in small groups, middle school students with little prior knowledge compared examples of addition with examples of multiplication of algebraic expressions (e.g.,
Comparing correct and incorrect strategies
Other cognitive-science research has highlighted the value of studying incorrect examples in combination with correct examples to increase depth of thinking about correct ideas and reduce use of incorrect ideas in the future (e.g., Booth, Lange, Koedinger, & Newton, 2013). We evaluated the impact of prompting students to compare correct and incorrect examples instead of comparing only correct examples. In a study of fourth- and fifth-grade students learning about decimal magnitude, students who compared incorrect and correct strategies gained greater conceptual and procedural knowledge than students who compared only correct strategies (Durkin & Rittle-Johnson, 2012). Comparing correct and incorrect examples helped students notice more conflicting ideas and focus more attention on the distinguishing features of the correct strategies, including the relevant concepts.
Overall, the cognitive-science literature on analogical reasoning provided theoretical and empirical guidance on comparison and how it aids learning, but applying these ideas to learning mathematics highlighted new types of comparison and helped specify the impact of different types of comparison on learning.
When to Compare?
Using comparison as an instructional method also highlighted the need to decide when in the learning process to use comparison. According to theories of analogical reasoning, prior knowledge of one of the to-be-compared examples may be important because analogical reasoning is particularly effective when learners can make inferences about a new idea by identifying its similarities and differences with a known idea and making predictions about the new idea on the basis of its alignment with the known idea (Gentner, 1983). Thus, should comparison be delayed until learners know one of the to-be-compared strategies? Indeed, instructional supports that help learners with prior knowledge are sometimes not effective with learners with little prior knowledge (Kalyuga, Ayers, Chandler, & Sweller, 2003).
Our initial evidence suggested that comparing strategies should be delayed until learners are familiar with one of the strategies. In our initial study with students with variable prior knowledge, comparing correct strategies was less effective than sequential study of the strategies for students who were not previously familiar with one of the strategies, although it was more effective for students with prior knowledge of one of the strategies (Rittle-Johnson, Star, & Durkin, 2009). This Condition × Prior Knowledge interaction was specific to prior knowledge of a strategy; general prior math knowledge did not interact with condition.
However, according to theories of analogical learning, people can learn from comparing two unfamiliar examples because it can help them notice potentially relevant similarities and differences (Gentner et al., 2003). In a follow-up study, we gave students with variable prior knowledge more time to learn a smaller amount of material. With this added support, students who compared correct strategies immediately gained greater procedural flexibility and similar conceptual and procedural knowledge relative to students who studied one strategy before exposure to additional strategies, regardless of prior knowledge (Rittle-Johnson, Star, & Durkin, 2012). Comparing two unfamiliar examples can aid learning, but this process requires sufficient support to avoid overwhelming learners.
How to Support Comparison?
Comparison often requires substantial mental effort. The cognitive-science literature has revealed key ways to support comparison and increase the probability that comparison will support learning (see also Richland et al., 2016). First, make the examples clear and visible and present both examples simultaneously, not one at a time. In math and some science topics, worked examples (a problem and step-by-step strategy for solving it) are very effective visual aids to help novices learn new procedures and related concepts (Atkinson, Derry, Renkl, & Wortham, 2000). Students make better comparisons and learn more when they do not have to rely on their memory of one example while comparing it with another example (Begolli & Richland, 2016). Second, present examples side by side and use common terminology, gestures, and other cues (e.g., highlight key parts in the same color) to guide attention to important similarities and differences. Visual cues, such as gesturing back and forth between similar aspects of two side-by-side examples, improve appropriate transfer of the demonstrated procedure to new contexts (Richland & McDonough, 2010). Labeling two examples with the same term makes it much more likely that learners will compare the examples and notice their underlying similarities (Namy & Gentner, 2002). Third, prompt for student explanation of key points about the comparison. Prompts to compare and explain specific aspects of two examples guide attention and improve learning from comparison more than generic prompts to compare (Gentner et al., 2003). It is also important to ask and allow students to answer higher level open-ended questions about the comparison (Star, Newton, et al., 2015). Finally, summarize the main points of the comparison after students reflect. Direct instruction on the key points after students compare supplements learners’ comparisons and improves learning from comparison (Gick & Holyoak, 1983; Schwartz & Bransford, 1998). Providing direct instruction after students compare examples can be more effective than providing it beforehand (Alfieri et al., 2013); comparison prepares students to learn more from direct instruction (Schwartz & Bransford, 1998).
Bridging to Practice
Cognitive-science-inspired research on mathematics learning in classroom contexts provided evidence-based recommendations for improving mathematics instruction. How do we bridge from research to practice to impact classroom instruction provided by teachers? One key is to have an interdisciplinary research team of psychology and education researchers who work directly with practitioners. Another key is to disseminate findings to broad audiences, including teachers and decision-makers, via practitioner-focused journals (e.g., Star, Kenyon, Joiner, & Rittle-Johnson, 2010), conferences (e.g., NCTM regional conferences), and webinars. Another is to influence consensus documents for educators on evidence-based instructional practices (Woodward et al., 2012), including investing the effort in chairing creation of these documents (Star, Caronongan, et al., 2015).
To promote high-quality adoption in classrooms, researchers will often need to develop curriculum and professional-development materials for teachers. We have developed a supplemental curriculum and designed teacher professional-development sessions titled Compare and Discuss Multiple Strategies in Algebra I (Star et al., 2019). This required us to substantially expand the number of, types of, and curricular coverage of our materials; develop and iteratively improve routines and teacher professional-development structures and materials; and expand our assessments. Teachers who adopted our curriculum and participated in our professional-development sessions improved their use of comparison during the summer professional-development sessions and the school year (Newton & Star, 2013). However, in our first yearlong study with Algebra I teachers, implementation of our materials was very infrequent, and students in the treatment condition did not learn more than students in the control condition (Star, Pollack, et al., 2015). In our second attempt, we revised the curriculum materials (for samples, see Fig. 2) and helped teachers align them with their curriculum. We refined and specified an instructional routine, as shown in Figure 3, including a think-pair-share routine for reflecting on the comparison. First, students think on their own for a minute about the discussion prompt. Next, each student pairs with another student to discuss the prompt, summarizing their ideas in writing. Then, students share their ideas in a whole-class discussion. We provided a graphic organizer for students to record their thinking in each phase. We also added professional-development sessions during the school year, during which we helped teachers plan and provided individualized feedback on their implementation. Overall, results suggest that these efforts greatly increased the quantity and quality of implementation and that teachers were generally positive about the approach (Durkin, Rittle-Johnson, Star, & Loehr, 2020). Students in the treatment condition learned more than students in the control condition for the unit on linear equation solving (Durkin et al., 2020). Thus, we have made substantial progress in promoting comparison of multiple strategies in mathematics classrooms, but much more work remains.
Sample Compare and Discuss Multiple Strategies in Algebra I curriculum materials for (a) comparing two correct strategies (“Which is better?”) and (b) comparing a correct and common incorrect strategy (“Which is correct?”). Figure taken from Star et al. (2019).
Instructional routine for promoting comparison and discussion of multiple strategies in the classroom. We recommend spending about 8 min in the comparison phase and 12 min in the discussion phase. Figure taken from Star et al. (2019).
Discussion
Comparison is a powerful learning process. In problem-solving domains such as mathematics, comparing multiple strategies and comparing confusable problem types promote conceptual knowledge, procedural knowledge, and procedural flexibility. However, comparison requires substantial mental effort by learners, and learners can become overwhelmed by it without adequate support, especially if all of the material is unfamiliar. Supports, such as presenting examples side by side and using cues to guide attention to important similarities and differences in the examples, facilitate learning from comparison. To help teachers use comparison more frequently and effectively in their classrooms, curricular materials, well-specified instructional routines, and sustained professional development are likely needed.
Despite the progress that has been made, many open questions remain. Theoretically, theories and formal models of analogical learning have not systematically considered or modeled comparison of two different strategies for solving the same problem. For instance, what impact do alignable differences (differences related to commonalities) compared with nonalignable differences (features in one strategy that have no corresponding feature in the other strategy) have on what people learn from the comparisons? How can this impact be modeled in alignment-based models of similarity? Additional open questions remain. For example, what are other effective ways to encourage and support comparison of multiple strategies (e.g., prompting students to generate a second way to solve a problem or to compare their own incorrect strategy with a correct strategy)? Also, does comparing multiple strategies impact students’ attitudes, such as their productive disposition toward mathematics (i.e., do they see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one’s own efficacy)? How can we support math teachers in learning to appropriately use and support comparison? Can they detect when their students need more support and adjust accordingly? Does developing teachers’ understanding of how and why comparison promotes learning improve their implementation? Further, how do teachers’ beliefs, such as their self-efficacy for helping all students learn challenging mathematics, impact how they use comparison, and do curriculum and professional-development materials such as ours change teachers’ beliefs? How can we integrate insights from using comparison to promote science learning (e.g., comparing observable and modeled events; see Jee & Anggoro, 2019) with insights from mathematics learning?
More broadly, for cognitive-science research to impact education, researchers must first make a clear connection to important educational outcomes that are valued by practitioners and clearly define and measure those outcomes. Then they must conduct research that provides evidence for the what, when, and how of the instructional intervention. Conducting such research informs theory as well as practice. For example, when choosing what to compare, our decision to focus on comparing multiple strategies was driven by best practices of teachers, and this research drove the need to extend theories of analogical reasoning to a type of comparison not previously considered. Bridging between theory and practice is not a one-way street from theory to practice; rather, it is bidirectional.
Recommended Reading
Alfieri, L., Nokes-Malach, T. J., & Schunn, C. D. (2013). (See References). Aggregates the results of previous studies on comparing examples and identifies factors that may improve learning from comparison.
Durkin, K., Star, J. R., & Rittle-Johnson, B. (2017). Using comparison of multiple strategies in the mathematics classroom: Lessons learned and next steps. ZDM, 49, 585—597. doi:10.1007/s11858-017-0853-9. Reviews empirical research on using comparison of multiple strategies to promote mathematics learning and provides instructional recommendations in more depth than this article.
Gentner, D., Loewenstein, J., & Thompson, L. (2003). (See References). Expands a theoretical acccount for analogical reasoning, based on structure-mapping theory, to learning from comparing two unfamiliar examples and provides evidence in support of the theory.
Richland, L. E., Begolli, K. N., Simms, N., Frausel, R. R., & Lyons, E. A. (2016). (See References). Reviews research on the cognitive challenges for students when comparing solution strategies and on teaching practices for helping students engage in comparison successfully.
Rittle-Johnson, B., & Star, J. R. (2007). (See References). The authors’ first study on using comparison to support mathematics learning, which provides details on the method and results.
Footnotes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
