Abstract
This article and my career as an educational researcher are grounded in two fundamental assumptions: (a) that research and practice can and should live in productive synergy, with each enhancing the other, and (b) that research focused on teaching and learning in a particular discipline can, if carefully framed, yield insights that have implications across a broad spectrum of disciplines. This article begins by describing in brief two bodies of work that exemplify these two fundamental assumptions. I then elaborate on a third example, the development of a new set of tools for understanding and supporting powerful mathematics classroom instruction—and by extension, powerful instruction across a wide range of disciplines.
I am honored and grateful to have been awarded the 2013 American Educational Research Association Distinguished Researcher award. I am gratified as well that the timing of the award allows me to unveil a new set of tools for understanding and supporting powerful classroom instruction. This article also allows me to reflect on, and exemplify, two themes that have been central to my work from my beginnings in educational research almost 40 years ago:
Research and practice can and should live in productive synergy, with each enhancing the other.
Research focused on teaching and learning in one particular discipline can, if carefully framed, yield insights that have implications across a broad spectrum of disciplines.
In what follows, I shall briefly review two main lines of my work over the years: a decade of work on problem solving and two decades of work on modeling the teaching process. My focus in those reviews will be on elaborating the research–practice dialectic and showing how results first derived in mathematics apply more generally. Having done so, I will turn my attention to work I have been involved with over the past half dozen years, which has been focused on understanding the attributes of powerful mathematics classrooms. Here, too, I will show how an expanding research-and-practice agenda has enriched both the research and the practices with which it has lived in happy synergy. Then, having elaborated on the nature of the work and framed it within mathematics, I will indicate how it yields a set of hypotheses for understanding and enhancing teaching in all disciplines.
Case 1: Problem Solving in Mathematics and Beyond
Perhaps the best way to summarize my problem-solving work (see, e.g., Schoenfeld, 1985, 1992) is that it consisted of a decade-long series of design experiments aimed at understanding and enhancing students’ mathematical problem solving.
Here a brief theoretical detour on the nature of design experiments is in order—my impression is that a significant number of researchers take design experiment to mean something like “cycles of design and implementation that result in an improved educational intervention.” Although design experiments do include such cycles, there is more to them than that. They are about improving both theory and practice.
So, just what is a design experiment? As I understand it (see, e.g., Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Schoenfeld, 2006),
One has a “local theory” about learning, which suggests some aspects of design.
One crafts a theory-based intervention (“the local theory says that this intervention ought to work in the following ways, to enhance understanding in these ways.”)
On the basis of implementing the intervention and carefully observing its impact, one (i) refines the local theory and (ii) refines the intervention.
That is, design experiments are as much about theory as they are about design. That said, let me briefly recap the evolution of my problem-solving work. Prior to building my first problem-solving course, I had conducted some experimentation, and done some theorizing, about what it would take to make effective use of particular problem-solving strategies. I took these ideas into the first version of the course, where I found that some things seemed to work, but some didn’t. Close observations of my students’ struggles provided a refined understanding of aspects of metacognition, which I could then better theorize. The same was the case for laboratory studies and ongoing instruction, aimed at understanding and then having an impact on belief systems. Over the course of a decade, my evolving understanding resulted in the following theoretical perspective:
If one seeks the reason(s) for someone’s success or failure in a mathematical problem-solving attempt, the cause of that success or failure will be located in one or more of that person’s
mathematical knowledge and resources,
access to productive “heuristic” strategies for making progress on challenging problems,
monitoring and self-regulation (aspects of metacognition), and
belief systems regarding mathematics and one’s sense of self as a thinker in general and a doer of mathematics in particular (in more current language, one’s mathematical identity). (See, e.g., Schoenfeld, 1985.)
At the same time, the problem-solving course improved, both profiting from the research and contributing to it. In that way, then, research and practice lived in productive synergy. (This is Fundamental Assumption 1.)
It goes without saying that the problem-solving work was not conducted in a theoretical vacuum. There was, of course, a large body of research on domain-specific knowledge in a wide variety of content areas. As I was elaborating on productive strategies (heuristics) for engaging in mathematics, others were elaborating on productive strategies in other domains, for example, in reading (Palincsar & Brown, 1984), writing (Scardamalia & Bereiter, 1983), science (diSessa, 1983), and artificial intelligence (Newell, 1983). Clearly, metacognition has a domain-specific component (that is, the more knowledge one has, the more effective one can be in monitoring and self-regulation), but aspects of it are domain general (e.g., Brown, 1987). Similarly, some beliefs about self may be general, some mathematics specific; but there are surely analogs in other domains. Thus, when the mathematical problem-solving work was well established, I could claim with confidence that (a) through (d) above were essential components of mathematical problem solving. But, given parallel work in other domains, I could already suggest the following with some confidence:
If one seeks the reason(s) for someone’s success or failure in a problem-solving attempt in any knowledge-rich domain, the cause of that success or failure will be located in one or more of that person’s
domain-specific knowledge and resources,
access to productive “heuristic” strategies for making progress on challenging problems in that domain,
monitoring and self-regulation (aspects of metacognition), and
belief systems regarding that domain and one’s sense of self as a thinker in general and a doer of that domain in particular (in more current language, one’s domain-specific identity).
That conjecture has held, in that resources, strategies, metacognition, and beliefs are generally acknowledged to be essential parts of sense making in every field, and no additional arenas have been declared as essential (see, e.g., Collins, Brown, & Newman, 1989). 1 I take this to be a case in point of Fundamental Assumption 2, that the research originally done in mathematics had direct analogues in other content domains.
Case 2: Decision Making in Mathematics Teaching and Beyond
My second example concerns the theory of decision making put forth in my book How We Think (Schoenfeld, 2010). My research group, known as the Functions Group, had been conducting a series of studies of mathematics tutoring (see, e.g., Schoenfeld et al., 1992) in order to develop the tools for studying teaching. My first case study of teaching occurred when a student in our teacher preparation program was dissatisfied with a lesson he had taught but couldn’t figure out why things had gone wrong. The coordinator of the program, who was also a member of Functions Group, suggested that he bring a video of his lesson to us for analysis.
My goal was to do more than help the student, although that was important. Ultimately, my intention was to be able to model this mathematics teacher’s classroom decision making on a moment-by-moment basis—and then others’.
Let me unpack the central ideas in the previous sentence. First, my intention was to build an analytical model of the student’s teaching. The reason for modeling is that modeling is a rigorous way to test theoretical ideas. It’s one thing to say, “I think this is why the teacher did what he did”; it’s quite something else to say, “Here is a general model of the decision-making process during teaching, into which I insert a characterization of the key aspects of this teacher’s teaching. The model makes the same decisions that the individual did.” Models are falsifiable, which means that one’s explanations are put to the test. 2 Second, the state of the art had advanced to the point where we might have a chance of understanding and modeling teaching as it happens rather than studying behavior in the laboratory. When the state of the art allows for it, the idea is to push the theory–practice dialectic. Third, yes, I began by modeling mathematics teaching—because my grounding is in mathematics and it is what I know best—but it was reasonable to expect that the essentials of teachers’ decision making (if framed the right way) would be akin to teachers’ decision making in other fields. 3
To summarize a book in a paragraph, the key idea in How We Think (Schoenfeld, 2010) is that people’s moment-by-moment decision making in teaching, in medicine, in fact, in all knowledge-rich domains can be modeled as a function of their
resources (especially their knowledge but also the tools at their disposal),
orientations (a generalization of beliefs, including values and preferences), and
goals (which are often chosen on the basis of orientations and available resources).
I will note, as before, that the theoretical ideas summarized above evolved in productive dialectic with practice. The first attempt at modeling was motivated by a problem from practice, and potential limits to the scope of our models were posed by other cases of “real-world” teaching. Moreover, as our understandings evolved, we embarked on experimental work related to professional development (see, e.g., Arcavi & Schoenfeld, 2008). It is one thing to say, theoretically, that orientations are central and belief change is slow; it is quite something else to try to achieve belief change. In attempting to do so and studying what happens, one develops a more nuanced understanding of the growth and change of belief systems. Thus, the work modeling decision making during mathematics teaching was, again, a case example of the productive research-and-practice synergy (that is, Fundamental Assumption 1).
Regarding Fundamental Assumption 2, I will note again that although I was focused on mathematics teaching in the early phases of modeling, the architecture of the decision making process was constructed so as to be general. As in the case of problem solving, what was known about decision making in a wide range of knowledge-intensive fields (e.g., teaching in other domains but also medical decision making; see, e.g., Groopman, 2007; Szolovits, Patil, & Schwartz, 1988) was entirely consistent with what was known in mathematics; it stood to reason that with the right framing, a theory of decision making during mathematics teaching could be seen as an instance of something more general (and the applications to other fields would be straightforward). How We Think makes a plausibility case, with ties to the broader literature and some fully worked out examples. Time will tell how robust the claims will be.
Case 3: Documenting and Supporting Productive Teaching in Mathematics
What Counts in Mathematics Teaching?
Here we come to the core example of this paper. The question here is, can one identify the key aspects of powerful mathematics classrooms—classrooms that produce students who do well on tests of mathematical content and problem solving?
The motivation for this work was the observation (a half dozen years ago, before the Measures of Effective Teaching study was conducted; Bill and Melinda Gates Foundation, 2012) that, although many of us have strong opinions about what makes for “good teaching” in mathematics and other classrooms, there was precious little evidence to support those beliefs. Ideally, one would like to have tools to measure classroom practices, and tools to measure student performance, to explore the relationship between the two: Do classrooms that score high on the dimensions of purported importance produce students who score high on tests of mathematical thinking?
To explore this issue, we needed a classroom measure that (a) was comprehensive, (b) focused on key aspects of mathematical sense making, (c) contained a relatively small number of important dimensions (so that, among other things, the implications for professional development would be clear), and (d) could be used in perhaps twice real time to code classroom data (that is, an hour of observations and note taking would require an additional hour to code the score), so that large-scale data analysis would be feasible. As explained in Schoenfeld (2013), although there were many schemes for classroom analysis (e.g., Beeby, Burkhardt, & Caddy, 1980; Danielson, 2011; Institute for Research on Policy Education and Practice, 2011; Junker et al., 2004; Marder & Walkington, 2012; PACT Consortium, 2012; Pianta, La Paro, & Hamre, 2008; University of Michigan, 2006), no schemes with attributes (a) through (d) above were available. Thus we set about building a theoretical framework and a classroom analysis rubric to match. The details of how we arrived at the theoretical frame are given in Schoenfeld (2013). In Table 1, I set forth our hypotheses regarding the key dimensions of powerful mathematics classrooms. 4 Again, this distillation is grounded in the literature (see Schoenfeld, 2013; Schoenfeld, Floden, & the Algebra Teaching Study and Mathematics Assessment Project, 2014). The key feature here is the distillation into a small number of dimensions.
The Five Dimensions of Mathematically Powerful Classrooms
To explore the hypothesis that these are indeed (one productive organization of) the dimensions of powerful classrooms, one needs a rubric for classroom observations. The team has produced such a rubric, called the Teaching for Robust Understanding of Mathematics (or TRU Math) rubric. 5 Before summarizing its essence, I must emphasize that this rubric was developed as a research tool and that it is not intended for administrative use in evaluating teachers. This is for at least three reasons: (a) Validation of the rubric through research is in its very early stages; (b) although the summary rubric given below seems straightforward, the actual use of the rubric requires training; and (c) we would much rather focus on working productively with teachers, as opposed to rating them. As seen later in this paper, we have an approach to professional development that focuses on engaging teachers in productive activities and conversations concerning the five dimensions of TRU Math.
The full TRU Math rubric contains subrubrics for characterizing episodes of whole-class instruction, small-group work, student presentations, and individual student work. Using the rubric involves parsing classroom activities into a sequence of “episodes” of no more than 5 minutes each in duration, assigning scores to each episode using the relevant subrubric, and then computing a weighted average of scores. The summary rubric, which is not used for scoring, is given in Figure 1. The summary rubric does provide a clear sense of the kinds of classroom activities that will score high or low along each of the dimensions.
The Teaching for Robust Understanding of Mathematics summary rubric
As noted above, to explore the relationship between classroom practices and student performance, one needs robust measures of student performance—in particular, measures of content, concepts, reasoning and problem solving. This, fortunately, has been a focus of our work since the early 1990s. The Balanced Assessment Project (Hugh Burkhardt, Alan Schoenfeld, Judah Schwartz, and Sandra Wilcox, principal investigators) was first funded in 1992 to construct assessments in line with the 1989 National Council of Teachers of Mathematics standards, and versions of the project (the Mathematics Assessment Resource Service and the Toolkit for Change, with Burkhardt, Schoenfeld, and Wilcox as PIs; and the Mathematics Assessment Project) have continued to the present day. 6 In sum, we now have a set of tools—both independent and dependent measures—for the empirical exploration of research-based hypotheses regarding powerful instruction and its impact. Very preliminary data analyses suggest that the relationship exists, but large-N studies are really what is called for.
Supporting Effective Mathematics Teaching
The next logical question is how to enhance teachers’ proficiency along those five dimensions (and study the impact of that work on teachers and students). Here the Mathematics Assessment Project and the Algebra Teaching Study have produced two sets of tools. The first is a set of “classroom challenges” or “formative assessment lessons” (FALs) designed to support teachers’ engagement in formative assessment focused on core content. To put things simply, doing formative assessment is hard: It calls for a set of pedagogical habits of mind and pedagogical content knowledge that most teachers do not have and that are not simple to acquire. For that reason, the Mathematics Assessment Project developed a series of 100 FALs whose purpose it is to scaffold teachers in teaching formatively. The content-oriented FALs are grounded in research on what students find difficult. They begin with tasks designed to elicit student thinking—to reveal bases of understanding that can be built upon and to reveal misunderstandings that need to be addressed. The lessons indicate common patterns of student responses to the tasks and ways to deal with them; they also contain activities that support the teacher in further assessing student understanding and building on it.
It is worth discussing the project’s design methodology here, for it, too, reflects a form of research-and-practice dialectic. Project designers based at the University of Nottingham craft a draft version of a lesson, which is then piloted in local schools. Team members observe the lessons (using a feedback rubric) and make suggestions for refinement to the design team, which then modifies the draft. When the draft is deemed solid, it is sent to three observation centers in the United States (in California, Michigan, and Rhode Island). The lessons are taught in a range of classrooms, and the observers document how they are working (or not) using a standard protocol. Observation forms are returned to Nottingham, where they are compiled by and discussed by the team. Revisions are then made by a team member other than the person who designed the lesson. This results in an “alpha” version. The alpha version is then distributed to the observation centers in the United States, where the observation process is repeated and the compiled feedback is used in the creation of a beta version. See Mathematics Assessment Project (2014) for detail.
To date, there have been more than 2 million FAL downloads from the project website, http://map.mathshell.org/.
The second set of materials we offer by way of support for teachers is the TRU Math Conversation Guide (Baldinger & Louie, 2014). This tool, intended for teacher-coach conversations (or, better, professional learning communities), addresses each of the five dimensions of TRU Math by raising a series of questions that teachers might consider with regard to lesson planning, debriefing, and thoughts about where to go next. In the conversation guide, the approach to the five dimensions is reframed as in Table 2.
Framing Questions in the Teaching for Robust Understanding of Mathematics Conversation Guide
These framing questions are mere overtures to conversations; the questions are elaborated in the conversation guide. Figure 2 shows the conversational elaboration of the third dimension, access to mathematical content.
Access to Mathematical Content, in the Teaching for Robust Understanding of Mathematics Conversation Guide
The FALs and the TRU Math Conversation Guide constitute the professional development part of the TRU Math suite.
Discussion: The Current State and Possibly Productive Next Steps
Figure 3 offers the “big-picture” view of the enterprise as I have described it thus far. The left-hand side of Figure 3 shows our research tools for the empirical exploration of research-based hypotheses regarding powerful instruction and its impact. The right-hand side of Figure 3 shows our tools for professional development, via instructional materials and support structures for systematic reflection on practice.
The research, practice, and development dialectic in mathematics
In concluding, I would like to review the enterprise described in Figure 3 with regard to the fundamental assumptions outlined at the beginning of this article.
Fundamental Assumption 1: Research and practice can and should live in productive synergy, with each enhancing the other.
The examples discussed in this article provide substantial backing for this assumption. In the problem-solving work, the course and the theory I was developing were mutually enriching, with the theory suggesting ideas for implementation and, at times, my intuitions as a teacher suggesting things for me to explore theoretically. The result was a “virtuous cycle” of discovery, including the refinement of ideas about strategy implementation, metacognition, and belief systems. The work on teacher modeling was inspired by problems of practice and refined by it; in turn, the theoretical work suggested avenues for the improvement of practice. In the current body of work, it is worth noting that every object and every arrow in Figure 3 represents or embodies a productive dialectic between theory and practice.
Fundamental Assumption 2: Research focused on teaching and learning in one particular discipline can, if carefully framed, yield insights that have implications across a broad spectrum of disciplines.
As indicated above, the problem-solving work was done in mathematics but had obvious analogues in other disciplines. The theoretical claims with regard to generality have stood the test of time. Similarly, the decision-making work was done in the context of a vast literature on decision making; mathematics teaching was the focus of my core examples, but the modeling was framed in ways that could be abstracted. I believe the same is the case with regard to the issues encapsulated in Figure 3. The TRU Math scheme is, of necessity, grounded in the specifics of mathematics teaching and learning. Thus, Dimension 1 (“the mathematics”) is fundamentally mathematical, just as the “resources” in the original problem-solving work and teacher modeling were fundamentally mathematical. But the other dimensions of TRU Math—cognitive demand; access to meaningful engagement with the content; agency, authority, and identity; and the uses of assessment—although “tinged” with mathematics when one looks at mathematics instruction, are general. That is, in a writing (or literature or physics) class, they would be “tinged” with writing (or literature or physics) in the same ways.
Hence, to use mathematical language, one can think of “the mathematics” in the TRU Math work as a variable—call it X, where X = “any particular discipline.” In the new scheme, called the “TRU X framework,” the first dimension, “the X,” would be “the extent to which this discipline (X) comes alive in the classroom as described in content standards or other documents, with students having the opportunity to develop productive disciplinary habits of mind.” The other three tools (represented by the dark boxes) in Figure 3 would be fleshed out analogously, resulting in Figure 4.
The research, practice, and development dialectic in general
This, of course, is just a conjecture at this point. I do think, however, that working on it will be a productive enterprise. I look forward to doing so, with colleagues from across the educational spectrum, in the years to come.