Educative Curriculum Materials: Uptake,Impact,and Implications for Research and Design

Abstract

The authors synthesize the findings of a research project to extend what is known about educative curriculum materials, or curriculum materials designed with the intent of supporting teacher learning as well as student learning. Drawing on a three-year program of research, including several close observational case studies and a large-scale quasi-experiment, the authors demonstrate how teachers use curriculum materials, what evidence there is of teachers’ uptake of ideas in educative curriculum materials, and what evidence there is of impact on teacher and/or student knowledge. These findings are situated in the literature, and the authors discuss how, taken together, the findings suggest design principles for educative curriculum materials. The authors close with implications for research.

Keywords

case studies curriculum elementary schools quasi-experimental analysis teacher education/development

Two decades ago, Ball and Cohen (1996) argued for curriculum materials as a lever for effecting change in classrooms, describing how these everyday resources could support teacher learning and teacher change. Remillard (2000) refers to such texts as materials that “speak to teachers, not merely through them” (p. 347). Building on the work of Ball and Cohen, Remillard, and others, Davis and Krajcik (2005) advanced design heuristics (or design guidelines) for educative curriculum materials, or curriculum materials designed explicitly to support teacher learning as well as student learning. Davis and Krajcik identified a gap in the literature when explaining their terminology:

Note that we use the term “design heuristics” rather than the more common “design principles.” Our recommendations are intended to be useful rules of thumb and not principles, which would imply a level of empirical testing that researchers have not yet undertaken. . . . Nonetheless, these design heuristics take us one step closer to the principled design of educative curriculum materials, necessary for this early stage of the research. (p. 4)

Here, we synthesize findings from our program of empirical research conceived to address this gap by integrating the results of a three-year research project in which we designed, tested, refined, and conducted a quasi-experiment with a set of educative curriculum materials. The work we describe spans design and development studies (including in-depth case studies) and efficacy¹ studies (a large quasi-experimental study) (Institute of Education Sciences & National Science Foundation, 2013)—an unusual scope for a relatively short project. While some findings have been presented in other research papers, they have not yet been synthesized to demonstrate the broad significance of the work.

Such syntheses across a program of research can provide useful guidance for the field. For example, Romance and Vitale (2001) reported on a multiyear study of science instruction; the authors used their project’s findings to discuss implications for policy and research. Hill and Charalambous (2012a) synthesized a set of related case studies, presented together in a special issue, to develop lessons learned about the relationships among teacher knowledge, mathematics instruction, and curriculum materials. Such research syntheses serve a different role than reviews of the literature or empirical reports on a single study and have the potential to provide important design, policy, and research guidance for the field.

The themes across our studies—situated within the literature—allow us to present a set of design principles for educative curriculum materials. To date, most design work specific to educative curriculum materials has been grounded in “best bets.” Curriculum developers rarely describe their full development process but are primarily guided by intuition, theory, and/or Davis and Krajcik’s (2005) design heuristics (see e.g., Beyer & Davis, 2009; Schneider & Krajcik, 2002). The empirically based and literature-grounded design principles presented here should interest curriculum developers seeking to make their materials supportive of teacher learning.

We organize our findings and design principles around three questions—questions that undergirded the entire program of research:

Research Question 1: How do teachers use and adapt curriculum materials in their enactment? What are some of the key influences on their decision making while using curriculum materials, including educative curriculum materials?

Research Question 2: What evidence do we have of teachers’ uptake of ideas from educative features in the educative curriculum materials?

Research Question 3: What evidence do we have of an impact of educative curriculum materials on teacher and student outcomes?

By curriculum materials, we mean resources designed to be used by teachers in classrooms to guide their instruction (Stein, Remillard, & Smith, 2007). By educative curriculum materials, we mean curriculum materials designed to support teacher learning as well as student learning (Davis & Krajcik, 2005). We use educative features to refer to the elements in curriculum materials specifically intended to provide support for teacher learning. These might take the form of “callout” boxes with teacher tips, graphics illustrating conceptual relationships among the ideas in a unit, guides to the use of readings, or suggestions for providing students feedback on their writing (e.g., Arias, Bismack, Davis, & Palincsar, 2016; Arias, Palincsar, & Davis, 2015; Cervetti, Kulikowich, & Bravo, 2015; Drake, Land, & Tyminski, 2014; Hill & Charalambous, 2012b). By uptake, we mean the ways teachers adopt, in language and/or action, ideas or practices recommended in the educative features.

We make two key assumptions in presenting our design principles. First, building on Ball and Cohen (1996), Remillard (2005), Davis and Krajcik (2005), Hill and Charalambous (2012a), and others, curriculum designers should support multiple domains of teacher knowledge and practice, including subject matter knowledge and pedagogical content knowledge as well as associated teaching practices for supporting the content and disciplinary practices students are learning. Second, building on Davis and Krajcik’s design heuristics, educative curriculum materials should reveal the rationales underlying recommendations. Davis and Krajcik argued that when teachers understand the rationale behind a recommendation, they are better able to apply the ideas across lessons, units, and contexts. These assumptions guide our design of educative features.

Our project focused on supporting elementary teachers of science. However, most of our implications should apply across content areas and grade bands given that educative curriculum materials are used across subject areas and grades (e.g., Collopy, 2003; Grossman & Thompson, 2008; McNeill, 2009). Four of the six design principles we propose are domain general. Furthermore, we conclude the paper with implications for the design and conduct of research regarding educative curricula; while grounded in our studies of science curriculum materials, these should apply to other areas as well.

Teachers, Curriculum Materials, and Elementary Science Education

Curriculum use entails transforming the written curriculum to the intended curriculum to the enacted curriculum (Stein et al., 2007). Teacher characteristics (e.g., knowledge and beliefs about teaching, learning, and learners) and characteristics of the curriculum materials (e.g., how the materials represent content) shape how teachers use curriculum materials. In our designs, we aim to support teachers’ engagement with curriculum materials by helping teachers recognize the rationales for recommendations and how they can productively adapt the recommendations in their classrooms. Indeed, any engagement with curriculum materials will involve adaptation to support teachers’ needs and contexts (e.g., Remillard, 2000). Curriculum materials are well suited to support teachers because they are situated in teachers’ daily work (Ball & Cohen, 1996; Putnam & Borko, 2000).

Educative curriculum materials can support positive outcomes across subject areas, grade levels, and contexts. For example, in secondary English language arts (Grossman & Thompson, 2008), elementary mathematics (Collopy, 2003; Hill & Charalambous, 2012a, 2012b), middle school science (Schneider & Krajcik, 2002; McNeill, 2009), and elementary science (Cervetti et al., 2015), teachers using educative curriculum materials changed their ideas about teaching and/or their practice, and/or their students experienced positive learning outcomes. Hill and Charalambous (2012b) found that educative features in curriculum materials seemed to support reform-oriented instruction. Grossman and Thompson (2008) found that curriculum materials provided opportunities for teacher learning, and for some teachers, this supported development of subject matter knowledge and pedagogical content knowledge. McNeill (2009) found that when teachers’ enactments departed from recommendations in educative features, their students’ learning suffered. Complementing these studies of impact on enactment is a study of use of specific educative features. Research with preservice teachers revealed differential use of educative features (Beyer & Davis, 2009). For example, they were more influenced by narratives (i.e., brief vignettes describing a teacher’s enactment of a lesson) than expository texts when identifying specific adaptations to teaching plans.

Elementary science teaching, with its associated challenges, provides an especially fruitful arena for studying the effects of educative curriculum materials. First, most elementary teachers teach all academic subject areas, including all science disciplines, even though they may not have robust science knowledge (Abell, 2007). Second, elementary science is often last in an ever-growing list of demands on teachers. It is both infrequently taught and inadequately supported through professional learning opportunities (Banilower et al., 2013; National Academies of Sciences Engineering and Medicine [NASEM], 2015). Supporting elementary science teachers is particularly critical now (NASEM, 2015) given current reforms in the United States that call for integrating disciplinary core ideas (e.g., heredity), crosscutting concepts (e.g. patterns), and scientific practices (e.g., arguing from evidence; National Research Council [NRC], 2012). Educative curriculum materials have potential to provide sustained, scalable, job-embedded, discipline-specific, professional learning opportunities that teachers need.

However, little is known about how specific features of educative curriculum materials support specific changes in practice among teachers. That is, how are educative features with particular characteristics (e.g., narratives, graphics, definitions) used, interpreted, and incorporated into instruction by teachers? Our first research question lays a foundation, and our second research question directly addresses this gap in the literature. Furthermore, while a handful of studies have attempted to show large-scale effects of educative curriculum materials (e.g., Cervetti et al., 2015; McNeill, Pimentel, & Strauss, 2013), continued work is needed—hence our third research question regarding impacts. Studying the effects of any instructional intervention, including educative curriculum materials, is challenging and involves tradeoffs in research design because so many factors can shape student outcomes (see e.g., Cheung, Slavin, Kim, & Lake, 2017). Thus, we also address the methodological lessons we have learned to provide guidance for future research.

Overview of Methods Used in Program of Research

Here we overview the methods we used throughout our program of research.

Overview of Context

We developed and tested the efficacy of educative features in the context of two kit-based, commercially available curricular units designed with National Science Foundation funding in the United States. These upper elementary units—Science and Technology Concepts’ (STC) Ecosystems and Electric Circuits units (National Science Resources Center [NSRC], 2004a, 2004b)—are intended to promote conceptual understanding through scientific investigation.

We worked with “typical” elementary teachers, not science specialists, because of our interest in educative curriculum materials as a way of effecting large-scale change. All teachers were certified in elementary education. None had much, if any, experience with STC curriculum materials, and they varied in their confidence in and experience with teaching science. We worked with ethnically diverse schools and districts that were mainly in underserved communities, especially in our initial work. We aspired to support those schools, and we anticipated that educative curriculum materials might be well suited to effect change in these contexts.

Overview of Three Phases of Work

Our work proceeded through three phases. Tables 1 and 2 summarize the research foci and data.

Table 1

Timeline and Research Summary

Phase	Focus	Resulting Studies
Year 1	Pilot study with original curriculum materials to inform the design of educative features; initial design of assessments	Bismack, Arias, Davis, and Palincsar (2014) Davis et al. (2014) Trygstad, Smith, Davis, and Palincsar (2012)
Year 2	Pilot study with educative curriculum materials to refine the educative features	Arias, Bismack, Davis, and Palincsar (2016) Bismack, Arias, Davis, and Palincsar (2015)
Year 3	Quasi-experimental efficacy study	Arias, Davis, and Palincsar (2014) Arias, Palincsar, and Davis (2015) Arias, Davis, Marino, Kademian, and Palincsar (2016) Arias, Smith, Davis, Marino, and Palincsar (2017) Kademian, Arias, Davis, and Palincsar (2015) Kademian, Arias, Davis, and Palincsar (2017) Smith and Smith (2014)

Table 2

Teacher and School Characteristics and Data Available in Years 1, 2, and 3

Year	Teacher	Years Teaching	Grade Level	School Characteristics (FRL%)	Hours of Video	Class Periods	Fieldnotes	Semi-Structured Interviews	N Student Notebooks
1	Watt	10	4	Urban fringe (86)	21.5	21	21	8	21
	Campbell	8	4	Urban fringe (86)	17	19	19	11	28
	Tenney	13	4	Urban fringe (66)	18	20	20	9	72
2	Tenney	14	4	Urban fringe (66)	18.5	24	24	4	26
	Levine	25	5	Urban fringe (66)	14	13	14	2	89
	Campbell	9	4	Urban fringe (86)	18	18	18	2	15
	Chagall	18	4	Urban fringe (86)	21	18	20	3	23
3	Decker	18	4	Urban fringe (83)	18	18	18	4	25
	Rosser	13	4	Urban fringe (54)	17	17	17	5	27
	Jay	18	4	Urban fringe (60)	16	16	16	4	29
	Arnold	15	4	Rural fringe (22)	10	10	10	5	30
	Beal	15	4–5	Suburban (43)	14	14	14	5	23
	Kilpatrick	4	4	Suburban (34)	3	3	3	1	23

Note. In Years 1 and 2, we video-recorded and took field notes for every lesson and collected as many student notebooks as were available. Year 3 refers to data for the case study teachers only. FRL = free and reduced price lunch.

In Year 1, we observed three teachers using the original STC curriculum materials to inform the design of the educative features we would overlay onto the materials. As a result of this pilot work, complemented by insights from the literature and support from content-area experts, we designed educative features to enhance the existing STC curriculum materials. (This theoretically and empirically informed design process is detailed elsewhere; Davis et al., 2014.)

In Year 2, four teachers used the enhanced curriculum materials, including the educative features we designed, and we collected a similar corpus of data as in Year 1. Based on Year 2 outcomes, we refined the educative features.

In Year 3, the design and development work culminated in a quasi-experimental efficacy study (Smith & Smith, 2014). We recruited 20 elementary schools (ranging from rural to urban fringe and from well-resourced to underresourced), randomly assigning them to the treatment condition, which received the enhanced curriculum materials, or the comparison condition, which received the original materials.² (See Figure 1 for a sampling of educative features in the enhanced materials.) The treatment group included 28 teachers and the comparison group 22 teachers. All but 1 teacher taught both units. We conducted one-day professional development workshops before each unit. We gathered common data for all 50 teachers (i.e., assessments of teacher content knowledge at multiple timepoints, lesson logs describing each enactment, pre- and postassessments of student content knowledge, and samples of student notebooks), and we supplemented those data with more extensive data (see Table 2) from 6 case study teachers: 4 from the treatment condition and 2 from the comparison condition. We purposefully selected these case study teachers to reflect the classrooms and teachers in the larger sample. In addition to school-level demographic characteristics, we considered each teacher’s content knowledge and beliefs, their willingness to have their instruction video-recorded, and logistics (e.g., travel time to the school). We oversampled from our treatment teachers to gain insight into the variability of the teachers’ enactment and because we already had data, from our pilot work, on teachers’ use of the original curriculum materials (as would be provided in the comparison condition).

Figure 1.

Sampling of the educative features developed: reading guide, content storyline, narrative, rubric

Overview of Data Analysis

In several of the smaller scale studies, we coded 2-minute segments of video-records; this convention enables analysis of teaching practices (Borko, Jacobs, Eiteljorg, & Pittman, 2008). We used partial-interval time sampling to mark each segment for the relevant code(s). We employed “tracers” (Duncan & Frymier, 1967), which allowed us detect the influence of educative features on teachers’ enactments. We used tracers as both a design element (seeding keywords or phrases in the design of the educative features) and an analytic tool (using those keywords or phrases in our coding to characterize uptake from the educative features). For example, we designed educative features that suggested teaching moves to discuss how and why to make justified predictions and included language such as justification and claim; these teaching moves and language were not in the original curriculum materials and thus served as tracers. To use these tracers as an analytic tool, we designed coding schemes that included the teaching moves and language tracers and analyzed the teachers’ enactments of lessons, teacher-created materials, and student work for evidence of these tracers. Our use of tracers is described in depth elsewhere (e.g., Arias, Bismack, et al., 2016; Bismack, Arias, Davis, & Palincsar, 2015), as are our analytic approaches in the small-scale studies more generally. The analyses for the quasi-experiment, which employed multilevel modeling, are described in Smith and Smith (2014).

Identifying Themes Across Studies

We took several steps to identify patterns across the studies within this larger program of research. We summarized the studies to identify main findings. We developed matrices that characterized case study teachers across studies (e.g., how a teacher used content, practice, or literacy-focused features). We developed initial themes, then reread the body of work to see if there was additional support or counterevidence for a theme. Throughout this process, we had ongoing conversations as a research group about the potential themes to sharpen our insights and identify counterevidence. We also used the themes to point toward potential design principles, which we tested in much the same manner.

Findings, Principles, and Connections to the Literature Base

Overall, we found that, as expected, teachers adapted the curriculum materials as they enacted them. Their decision making was informed by competing goals, which sometimes promoted positive changes and sometimes limited students’ opportunities to learn. We found extensive evidence of uptake of the ideas and recommendations in the educative features. Yet our evidence of impact on outcomes was more limited. In the following sections, we provide background literature related to each research question and then explicate the results and our design principles.

Use of Curriculum Materials and Key Influences on Decision Making

The literature documents how teachers adapt curriculum materials as they enact them (e.g., Collopy, 2003; McNeill, 2009; Remillard, 2000). Not all adaptations maintain students’ opportunities to learn (Hill & Charalambous, 2012a; Stein et al., 2007). Teachers’ decision making is shaped by factors related to the teachers themselves (e.g., knowledge, beliefs, goals, and identities), their curriculum materials (e.g., how inquiry-oriented they are; their look, structure, or voice), and the context (e.g., how supportive they are of innovative teaching) (for reviews of this literature, see Davis, Janssen, & Van Driel, 2016; Remillard, 2005). Here, we show how patterns in our findings complement the existing literature and inform our design principles.

Teachers adapted both the original and the enhanced curriculum materials as they enacted them. Table 3 summarizes the findings associated with Research Question 1. The changes teachers made sometimes maintained or augmented students’ opportunities to learn and sometimes limited those opportunities. Factors that shaped teachers’ decision making included time pressure, the strengths and limitations of the curriculum materials, and teachers’ understanding of the scientific practices and their students. Because most research related to this question stems from mathematics education, our findings extend the literature while being consistent with what is already understood.

Table 3

Findings Regarding Teachers’ Use of Curriculum Materials

Theme	Findings
Teachers’ adaptations: Examples of augmenting students’ opportunities to learn	• Teachers using the original curriculum materials in Year 1 enacted almost every type of scientific practice in the curriculum materials, emphasizing engaging students with scientific phenomena and making scientific observations (Bismack, Arias, Davis, & Palincsar, 2014). • Teachers in Year 3 (across both conditions) incorporated additional opportunities for students to work on the scientific practices (Kademian, Arias, Davis, & Palincsar, 2015), and some made the rationale for these practices explicit to their students (Arias, Davis, Marino, Kademian, & Palincsar, 2016). • The Year 3 teachers attended to their students and wanted to adjust their teaching to best support students’ learning (Kademian et al., 2015).
Teachers’ adaptations: Examples of limiting students’ opportunities to learn	• Teachers in Year 1 modified the sheets for recording ecosystems observations to include several observations per page. While this had the advantage of allowing comparison across time, it made the space for recording too small for students to make accurate and detailed observations (Bismack et al., 2014). • When students were supposed to design an investigation, Year 3 case study teachers (across both conditions) made many decisions for the students, giving them limited input into the investigation design (Kademian et al., 2015; see Bismack et al., 2014, for similar findings from Year 1).
Examples of factors shaping teachers’ adaptations	• Time: Teachers in Year 1 regularly noted concerns about how much time engaging with scientific phenomena and making and recording observations took; these teachers sometimes eliminated students’ opportunities to make predictions due to time constraints (Bismack et al., 2014; see Kademian et al., 2015, for related findings). • Characteristics of the curriculum materials: The original curriculum materials included limited opportunities for students to construct scientific explanations; Year 1 teachers’ enactments, too, reflected few explanation opportunities (Bismack et al., 2014). • Characteristics of the teachers: Teachers did not always understand the scientific practices or the rationale for the practices (Bismack et al., 2014). For example, one teacher in Year 1 did not distinguish between prediction and inference, and her interviews indicated an inaccurate understanding of the role of a control in an experiment. • Teachers’ perceptions of students: Year 3 teachers adjusted how they had students plan and conduct investigations based on their perceptions of the students’ capabilities (Kademian et al., 2015).

These findings, along with the literature reviewed briefly, lead us to our first design principle³:

Design Principle 1: Teachers will adapt curriculum materials. These adaptations are likely to be informed by teachers’ concerns about time and student capabilities and experiences. By anticipating these adaptations, educative features can facilitate principled and productive adaptations. Therefore, educative features should provide suggestions for adaptations of lessons that would take different amounts of time and meet a range of students’ needs while still meeting the intent of the reforms embedded in the curriculum materials. Examples of such educative features could include narratives describing choices that may reduce time needed while maintaining opportunities to learn.

Our findings highlight the need for educative curriculum materials to anticipate and support teachers’ adaptations to curriculum materials.

Uptake From Educative Features

Studies have demonstrated uptake of ideas in educative curriculum materials through characterizing teacher practice, teacher learning, or student outcomes (e.g., Beyer & Davis, 2009; Cervetti et al., 2015; Collopy, 2003; McNeill, 2009; Schneider & Krajcik, 2002). Studies have not, however, untangled which ideas teachers take up from specific kinds of educative features. We use tracers, as described previously, to discern the effects of specific educative features.

Students’ work, teachers’ comments on that work, and teachers’ practice all reflected teachers’ uptake of ideas, pointing to effects of specific educative features. Table 4 summarizes these and other findings associated with this research question. Educative features supported teachers in engaging students in some scientific practices, learning the content themselves, and teaching with scientific text.

Table 4

Findings Regarding Teachers’ Uptake of Ideas in Educative Features

Theme	Findings
Examples of effects of specific educative features	• The concept maps, content support boxes, and content storylines helped one Year 2 teacher choose what content she wanted to emphasize in each lesson (Arias, Bismack, Davis, & Palincsar, 2016). • Teachers tended to prefer (e.g., Arias, Bismack, et al., 2016; Bismack, Arias, Davis, & Palincsar, 2015) the educative features most directly grounded in their practice, such as the rubrics that provided examples of student work (which seemed to help teachers adopt higher expectations for their students) and the narratives that described a fictional teacher’s decision making. • Year 2 teachers used the practice overview pages—expository text defining the scientific practices—less extensively (Arias, Bismack, et al., 2016). • Year 2 teachers used the content charts (i.e., tabular depictions of examples of key scientific ideas) and rubrics directly with their students (Bismack et al., 2015) as teaching tools.
Examples of supporting scientific practices	• In Year 3, teachers with the educative features supported students in engaging in scientific practices, especially making and recording observations and justifying predictions, indicating clear uptake of ideas from the educative features. We found much more limited evidence of uptake of ideas about constructing evidence-based explanations (Arias, Davis, Marino, Kademian, & Palincsar, 2016). • Two Year 3 (treatment condition) case study teachers explained “predictions” clearly and drew directly from the educative curriculum materials; these teachers emphasized the justification of predictions, and their students justified their predictions. Students in the classroom of another treatment condition teacher, who did not take up the ideas, did not do so (nor did students in the comparison condition classrooms) (Arias, Smith, Davis, Marino, & Palincsar, 2017).
Examples of supporting content learning	• In Year 3, teachers who took up language from the educative features were more likely to use scientific language and use it accurately; on the other hand, one teacher with access to the educative features did not take up these ideas, and the language use in her classroom was similar to that in the comparison condition (Kademian, Arias, Davis, & Palincsar, 2017). • Similarly, a Year 2 teacher who used the content support features provided more accurate descriptions of phenomena than did a teacher who reported not using those features (Arias, Bismack, et al., 2016).
Examples of supporting teaching with scientific text	• Some Year 3 teachers, supported by the educative features, helped their students develop a text base that could guide their scientific observations (Arias, Palincsar, & Davis, 2015). • One Year 3 teacher from the treatment condition—who had used educative features for scientific practices such as prediction extensively (Arias, Davis, & Palincsar, 2014), as noted previously—made minimal use of the educative features supporting the use of text, saying that given her decades of experience as an elementary teacher, she did not need the literacy supports. In the same study, other teachers drew heavily on different educative features supporting the effective use of text (Arias et al., 2015).

Based on the findings summarized in Table 4, we extend the existing literature by identifying specific effects of specific educative features. Overall, patterns from this set of studies include: (a) the salience of highly situated educative features, including teaching tools, and (b) the need for multiple vectors for supporting content knowledge and scientific practices. These studies collectively also demonstrate the variation in how different teachers take up different educative features.

These are important contributions because they provide direct design guidance for curriculum developers, captured in Design Principles 2 and 3.

Design Principle 2: Educative features that provide representations of practice can support teachers’ uptake of the ideas in the features. Educative features that can be used as teaching tools can support concrete changes in teachers’ practice. Furthermore, sample student work can help teachers set higher expectations for their students than they might otherwise have. Therefore, educative features should be situated and grounded in teachers’ practice. Some features grounded in practice can be directly applied as teaching tools in the classroom. Examples of educative features that are situated in teachers’ practice include (a) rubrics that illustrate essential features of key ideas of reforms along with sample student work and possible teacher comments that reflect those key ideas and (b) narratives that describe teachers’ enactment of lessons in ways that demonstrate key ideas of reforms. Examples of teaching tools include rubrics, examples of key scientific ideas, and student-friendly definitions of terms.

Design Principle 2 reflects the situated nature of teacher learning (Putnam & Borko, 2000). This design principle is also bolstered by limited previous empirical findings indicating that preservice elementary teachers used ideas from narratives more readily than those that appeared in general, nonsituated⁴ educative features (Beyer & Davis, 2009). Collopy’s (2003) work also suggests the importance of tight connections between educative features and instruction. For example, one teacher from Collopy’s study did not shift in her fundamental beliefs about mathematics learning, but she made sustained changes to the kinds of tasks she provided to students to focus more on conceptual understanding and mathematical reasoning; these changes were consistent with her use of the educative features.

Design Principle 3 further highlights the forms of support designers might provide, specifically with respect to enhancing teachers’ subject matter knowledge.

Design Principle 3: As expectations for students and teachers change, demands on teachers’ subject matter knowledge grow. Multiple vectors may help teachers identify the “big ideas” to highlight in lessons. Therefore, designers should use multiple forms of support for highlighting important content. Examples include content storylines, student-friendly definitions, and graphics.

Hill and Charalambous’s (2012b) study of teachers’ use of educative mathematics curriculum materials highlights the importance of educative features that can bolster teachers’ own content knowledge for teaching. These authors compared a teacher’s prior incorrect understanding of a mathematical concept to her ability to support her students in understanding that concept after using the curriculum materials, which provided support for the concept. Our empirical work demonstrating uptake suggests specific ways of providing such support.

Findings across Research Questions 1 and 2 lead us to Design Principles 4 and 5.

Design Principle 4: Different teachers will need and take up different kinds of educative features (in terms of substance and form). Teachers’ variable uptake will be based on the needs they perceive in themselves (e.g., their knowledge of content, assessment, or reading strategies) and their students (e.g., their typical content struggles). Therefore, designers should develop a constellation of educative features that have the potential to meet these various needs. Designers also should help teachers recognize how the recommendations differ from their current practice, in part through emphasizing the rationales for the recommendations. A constellation could be constituted purposefully using different forms of educative features (e.g., narratives and call-out boxes) with different foci (e.g., subject matter knowledge and pedagogical content knowledge). An example of an educative feature that would highlight how recommendations differ from current practice could include a reading guide that signals changes from typical literacy practices.

Providing a range of educative features and calling attention to how these features enable practice that may differ from the status quo is consistent with the field’s understanding of the nature of learning and the role of tools in supporting learning (e.g., Putnam & Borko, 2000).

Our fifth design principle is science-specific.

Design Principle 5: Teachers take up the practice of scientific explanation in a limited way. Therefore, educative features should help teachers (a) appreciate the definition, intention, and value of constructing scientific explanations and (b) learn how to support students in engaging in explanation construction and argumentation. Examples include narratives, expository text, capstone questions, and rubrics that synergistically define, illustrate, and guide explanation construction and argumentation in the classroom.

Our curriculum use studies highlighted the uneven nature of teachers’ understanding of scientific practices. Our uptake studies and curriculum use studies revealed the challenges teachers had specific to supporting scientific explanation. McNeill’s (e.g., 2009) work on the challenges of scientific explanation and argumentation provides further support for this design principle. Students struggled with explanation construction and developing arguments. While supporting a claim with evidence is not straightforward for students, providing reasoning—that is, explicating how their data count as evidence to support a claim and what scientific principles connect a claim and evidence—tends to be particularly difficult. Furthermore, teachers’ instructional practices helped shape students’ engagement in these practices in that a clear definition of the practices seemed to support students’ engagement in them, while an oversimplification seemed to undermine students’ engagement. Because explanation and argumentation are key scientific practices, inherent in student sensemaking, yet present significant challenges for teachers and students, providing extensive support through educative science curriculum materials—and other venues—seems crucial.

Impact on Teachers and Students

Our third research question relates to the impact of educative curriculum materials on teachers and students. Some studies have found effects of educative curriculum materials on student learning (e.g., McNeill et al., 2013), and others suggest possible relationships with learning (e.g., Cervetti et al., 2015). We were interested in whether we would find similar effects.

Our project culminated in Year 3 with an efficacy study intended to discern effects of educative curriculum materials on teacher and/or student learning (for details, see Smith & Smith, 2014). Table 5 summarizes these findings. The primary outcome measures for teachers were multiple-choice assessments of science content knowledge. The lack of difference between conditions shown in Table 5 is not surprising given what we learned about how teachers took up the features, as described previously. Although several features were designed to support content knowledge, teachers used them unevenly; furthermore, some supports for content knowledge were provided as unit front matter; hence, they were less situated in the day-to-day teaching of the unit than our findings suggest would have been optimal. The findings for student content knowledge outcomes were similar to those for teacher outcomes.

Table 5

Findings Related to the Impact on Teachers and Students

Theme	Findings
Impact on teachers’ content knowledge	With regard to both units, treatment and comparison teachers in Year 3 showed statistically significant gains across timepoints, with substantial effect sizes. However, teachers with access to the educative features did not gain more than those without access (Smith & Smith, 2014).
Impact on students’ content knowledge	The measures of student content knowledge were multiple-choice assessments developed for the study and closely aligned to the content of the two units (Trygstad, Smith, Davis, & Palincsar, 2012). They were administered to all students immediately before and after the relevant unit. In both content areas, there was a statistically significant difference across time, with post-unit means significantly and substantially higher than pre-unit means, but there was no significant difference between conditions (Smith & Smith, 2014).
Impact on students’ scientific practice	In both units, students in treatment classes made significantly greater gains than students in comparison classes from pre-unit to post-unit on an item involving justification of predictions (hierarchical linear modeling, p < .05; effect size = .27 standard deviations in each unit), suggesting that teachers’ use of the educative features positively impacted students’ ability to justify predictions (Arias, Smith, Davis, Marino, & Palincsar, 2017).

Finally, the student assessments included one constructed-response question asking students to construct and justify a prediction related to the content of the unit. (Due in part to constraints on project funding, we included only a single item to assess science practices.) These were scored for the extent and quality of justification students included. On this out-come measure, we did detect a difference between students in treatment and comparison groups (Arias, Smith, Davis, Marino, & Palincsar, 2017).

In sum, our efficacy study did not, for the most part, reveal significant effects on teacher or student knowledge. The exception was that students in the treatment condition did perform better on an item requiring the justification of a prediction. We interpret these findings in light of our findings on teachers’ uptake of educative features, which indicated extensive variability, and in light of the many factors that matter in supporting student learning, of which teacher subject matter knowledge is just one.

Our final design principle, which is also science-specific, is grounded in the findings across all three research questions, about teachers’ use of curriculum materials, teachers’ uptake of ideas from educative curriculum materials, and the impact studies.

Design Principle 6: Certain scientific practices, including making and recording observations and making and justifying predictions, were taken up effectively by most teachers. Therefore, educative features should support easier-to-enact scientific practices, with the idea of moving incrementally toward more ambitious science teaching in elementary classrooms. Designers should connect to teachers’ existing teaching practice to create leverage points while helping teachers recognize salient differences. Examples of such educative features could include narratives and how-and-why support for prediction that reinforce the need for justification, point the path toward argumentation, and connect to ways teachers may already use prediction (e.g., in English language arts).

Our findings indicated that some scientific practices were more accessible for teachers to take up than others. The NRC’s (2012) Framework for K–12 Science Education emphasizes a range of scientific practices but provides little guidance about how to support teachers to include these in their instruction. Star’s (2015) recommendation for incremental change toward reform-based teaching and Janssen, Westbroek, and van Driel’s (2014) bridging progression to support incremental change might help designers conceptualize how to move toward implementing these practices in typical classrooms. Rather than expecting a wholesale change in lesson design, these models focus on small, more manageable changes. Starting with more straightforward practices and using them as entry points to the more complex practices could be a version of such incremental change. For example, prediction, as a practice that involves justification based on evidence and/or reasoning, may serve as an “entrée” for explanation and argumentation by providing a lower stakes need for justification. Carefully recorded observations can provide evidence for explanation and argumentation and thus make engagement in those more sophisticated practices possible.

In summary, Table 6 presents all six design principles.

Table 6

Design Principles for Educative Curriculum Materials

Design Principles
Design Principle 1: Teachers will adapt curriculum materials. These adaptations are likely to be informed by teachers’ concerns about time and student capabilities and experiences. By anticipating these adaptations, educative features can facilitate principled and productive adaptations. Therefore, educative features should provide suggestions for adaptations of lessons that would take different amounts of time and meet a range of students’ needs while still meeting the intent of the reforms embedded in the curriculum materials. Examples of such educative features could include narratives describing choices that may reduce time needed while maintaining opportunities to learn.
Design Principle 2: Educative features that provide representations of practice can support teachers’ uptake of the ideas in the features. Educative features that can be used as teaching tools can support concrete changes in teachers’ practice. Furthermore, sample student work can help teachers set higher expectations for their students than they might otherwise have. Therefore, educative features should be situated and grounded in teachers’ practice. Some features grounded in practice can be directly applied as teaching tools in the classroom. Examples of educative features that are situated in teachers’ practice include (a) rubrics that illustrate essential features of key ideas of reforms along with sample student work and possible teacher comments that reflect those key ideas and (b) narratives that describe teachers’ enactment of lessons in ways that demonstrate key ideas of reforms. Examples of teaching tools include rubrics, examples of key scientific ideas, and student-friendly definitions of terms.
Design Principle 3: As expectations for students and teachers change, demands on teachers’ subject matter knowledge grow. Multiple vectors may help teachers to identify the “big ideas” to highlight in lessons. Therefore, designers should use multiple forms of support for highlighting important content. Examples include content storylines, student-friendly definitions, and graphics.
Design Principle 4: Different teachers will need and take up different kinds of educative features (in terms of substance and form). Teachers’ variable uptake will be based on the needs they perceive in themselves (e.g., their knowledge of content, assessment, or reading strategies) and their students (e.g., their typical content struggles). Therefore, designers should develop a constellation of educative features that have the potential to meet these various needs. Designers also should help teachers recognize how the recommendations differ from their current practice, in part through emphasizing the rationales for the recommendations. A constellation could be constituted purposefully using different forms of educative features (e.g., narratives and call-out boxes) with different foci (e.g., subject matter knowledge and pedagogical content knowledge). An example of an educative feature that would highlight how recommendations differ from current practice could include a reading guide that signals changes from typical literacy practices.
Design Principle 5: Teachers take up the practice of scientific explanation in a limited way. Therefore, educative features should help teachers (a) appreciate the definition, intention, and value of constructing scientific explanations and (b) learn how to support students in engaging in explanation construction and argumentation. Examples include narratives, expository text, capstone questions, and rubrics that synergistically define, illustrate, and guide explanation construction and argumentation in the classroom.
Design Principle 6: Certain scientific practices, including making and recording observations and making and justifying predictions, were taken up effectively by most teachers. Therefore, educative features should support easier-to-enact scientific practices, with the idea of moving incrementally toward more ambitious science teaching in elementary classrooms. Designers should connect to teachers’ existing teaching practice to create leverage points while helping teachers recognize salient differences. Examples of such educative features could include narratives and how-and-why support for prediction that reinforce the need for justification, point the path toward argumentation, and connect to ways teachers may already use prediction (e.g., in English language arts).

Implications for Studying Educative Curriculum Materials

This program of research, like any such program, has limitations. Many of our results stem from rich but small case studies of a few teachers. We had 50 teachers in our quasi-experimental study but uneven numbers in each condition. We had a single item to measure scientific practice despite evidence that in retrospect suggests teachers’ uptake of support for scientific practice was more clear than their uptake of support for content.

With the hindsight afforded by a completed project, we can glean insights about methodological lessons learned for studying educative curriculum materials. Here we first discuss the tradeoffs we considered and the rationales for our decisions in three areas: the curriculum materials themselves, the specification and development of outcome measures, and the timing of the efficacy study.

The first group of research design decisions centered on the curriculum materials. We decided on a 6- to 8-week instructional unit to allow for sustained opportunity for students to engage with a coherent set of disciplinary concepts, but we stipulated two units to allow for the possibility of generalizing claims across content areas. A second critical decision was whether to create these instructional materials de novo, along with the educative features, or to overlay the features on existing materials. We chose to develop features and overlay them on highly respected, commercially available materials. This decision saved time and money as we were able to build on a strong foundation. It enhanced ecological validity and eased the process of creating a comparison condition. As noted, we systematically built tracers into the educative features so we could study teachers’ uptake and the influence on student learning opportunities. This turned out to be an important way to identify uptake even when we did not see large-scale impact; other researchers should consider incorporating tracers into their materials as well.

The second group of decisions centered on outcome measures. The ultimate dependent variable for the efficacy study was a measure of student learning of the targeted science concepts (see Smith & Smith, 2014). We created our own measures, aligned closely with the science content in the two units and with good evidence of validity and reliability. We opted not to focus on science practices in the teacher or student assessments, with the exception of the single practice question in the student assessments, due to limitations in time and resources. We recognized this as a design tradeoff with implications about the claims we would be able to make.⁵ In retrospect, we should have divided assessment resources more evenly. Evidence from our constructed response item supports our claim that the features positively shaped student learning, but we had limited opportunity to collect this evidence due to our assessment development decisions. A pilot efficacy study might have helped us better align the assessments with the anticipated outcomes; other researchers should consider incorporating such a study.

Deciding when to measure the dependent variable represents a third study design decision. The instructional materials we chose, though highly regarded, are not widely used (Banilower et al., 2013). Typically, teachers need at least two iterations with materials before they begin to use them purposefully (Loucks-Horsley, Love, Stiles, Mundry, & Hewson, 2003). Thus, our teachers’ uptake of the educative features may have been diminished by their preoccupation with the instructional materials themselves. Our decision to use existing materials afforded the time needed for conducting the pilot work, which informed development and refinement of the educative features. A more ideal design, though not feasible within our funding constraints, would have allowed all teachers to become more familiar with the materials before initiating the efficacy study. Teachers’ familiarity with the materials might then have freed them to attend more purposefully to the supports—or given them a false sense of security and led them away from the supports (a question that itself bears study). This aspect of study design also highlights the importance of including a valid and reliable measure of the extent of teachers’ use of educative features. The educative features were used differently by different teachers. However, because teachers inconsistently completed the lesson logs that asked about use of each feature, we did not have a measure of curricular use from all teachers that could be included in the quantitative analysis—like the others mentioned here, a methodological issue likely to be relevant regardless of discipline. Other researchers should try to ensure the incorporation of such a measure of curricular use.

The previous discussion focuses on implications related to study design. Our work also points to implications for the focus of future research. Our evidence that the educative features impacted student learning is mixed. The field needs studies across contexts that examine whether and how specific aspects of educative curriculum materials affect student learning. Further, future research should continue to explore the tensions described by Davis and Krajcik (2005): finding the balance between adequate support and too much, navigating between prescription and autonomy, and determining the role of technology. Progress is being made particularly on the last of these issues (e.g., Marco-Bujosa, McNeill, González-Howard, & Loper, 2016), and further work could explore how advances in assessment could help tailor the educative features provided to teachers. Another area ripe for continued work is exploring how teachers learn to use educative features in curriculum materials (cf. Drake et al., 2014). Furthermore, educative curriculum materials are not a panacea and should be supported by strong professional development (e.g., Pringle, Mesa, & Hayes, 2017). Empirical research could investigate how professional development and educative curriculum materials work together to support teacher learning. Finally, of course, we hope that the design principles presented here provide guidance for the development of and research on new educative curriculum materials across a range of subject areas, grade levels, and purposes.

Conclusion

As demands on teachers and students increase, educative curriculum materials have potential for improving teaching and learning in elementary, middle school, and high school classrooms (cf. Krajcik & Delen, 2017). When used in conjunction with other professional learning opportunities, they are uniquely positioned to support such change. As we noted previously, a decade ago, the field lacked an empirical basis for design principles for educative curriculum materials, and Davis and Krajcik (2005) instead put forward a set of design heuristics. In the ensuing years, a stronger empirical foundation has been laid, allowing the development of design principles for curriculum development with an eye toward supporting teacher learning as well as student learning.

In brief, our domain-general design principles suggest that educative features should:

suggest adaptations of lessons that would take different amounts of time and meet a range of students’ needs,

be situated and grounded in teachers’ practice,

take multiple forms,

work together to meet a range of teacher needs.

Our domain-specific design principles suggest that educative science curriculum materials should provide purposeful support for scientific argumentation as well as “entry-level” scientific practices as a mechanism to effect change. Going forward, the recommendations made here can help shape an agenda for design-based research on educative curriculum materials.

Footnotes

Notes

Authors

ELIZABETH A. DAVIS, PhD, is a professor of science education at the University of Michigan, 610 East University Avenue, Ann Arbor, MI 48109-1259; betsyd@umich.edu . Her research focuses on how curriculum materials and teacher education experiences can support the learning of teachers, with a particular focus on elementary science teachers.

ANNEMARIE SULLIVAN PALINCSAR, PhD, is the Jean and Charles Walgreen, Jr., Chair of Reading and Literacy at the School of Education, University of Michigan, 610 East University Avenue, Ann Arbor, MI 48109-1259; annemari@umich.edu . Her research focuses on the intersection of language literacy and science literacy in the elementary grades.

P. SEAN SMITH, PhD, is the president of Horizon Research, Inc., 326 Cloister Court, Chapel Hill, NC 27514; ssmith62@horizon-research.com . His research focuses on pedagogical content knowledge in science.

ANNA MARIA ARIAS, PhD, is an assistant professor of science education at Illinois State University, 239 DeGarmo Hall, Campus Box 5330, Normal, IL 61790-5330; aarias4@ilstu.edu . Her research focuses on the development of teaching practice to support student sensemaking with emphasis on science practices, teachers’ use of curriculum materials, and preservice teacher education.

SYLVIE M. KADEMIAN, PhD, is a postdoctoral fellow at the University of Michigan, 610 East University Avenue, Room 3003, Ann Arbor, MI 48109-1259; smkademi@umich.edu . Her research focuses on using tools to support beginning science teachers learning to facilitate investigation-based discussions.

References

Abell

(2007). Research on science teacher knowledge. In Abell

Lederman

(Eds.), Handbook of research on science education (pp. 1105–1149). Mahwah, NJ: Lawrence Erlbaum Associates.

Arias

A. M.

Bismack

Davis

E. A.

Palincsar

A. S.

(2016). Interacting with a suite of educative features: Elementary science teachers’ use of educative curriculum materials. Journal of Research in Science Teaching, 53(3), 422–449.

Arias

A. M.

Davis

E. A.

Marino

J.-C.

Kademian

Palincsar

A. S.

(2016). Teachers’ use of educative curriculum materials to engage students in science practices. International Journal of Science Education, 38(9), 1504–1526. doi:10.1080/09500693.2016.1198059

Arias

A. M.

Davis

E. A.

Palincsar

A. S.

(2014). Using educative curriculum materials to support teachers in engaging students to justify predictions. In Polman

Kyza

O’Neill

Tabak

Penuel

Jurow

O’Connor

Lee

D’Amico

(Eds.), The International Conference of the Learning Sciences 2014 (Vol. 3, pp. 1429–1431). Boulder, CO: International Society of the Learning Sciences.

Arias

A. M.

Palincsar

A. S.

Davis

E. A.

(2015). The design and use of educative curricular supports for text-based discussions in science. Journal of Education, 195(1), 21–37.

Arias

A. M.

Smith

P. S.

Davis

E. A.

Marino

J.-C.

Palincsar

A. S.

(2017). Justifying predictions: Connecting use of educative curriculum materials to students’ engagement in science argumentation. Journal of Science Teacher Education, 28(1), 11–35.

Ball

D. L.

Cohen

D. K.

(1996). Reform by the book: What is—or might be—the role of curriculum materials in teacher learning and instructional reform? Educational Researcher, 25(9), 6–8, 14.

Banilower

Smith

P. S.

Weiss

Malzahn

Campbell

Weis

(2013, February). Report of the 2012 National Survey of Science and Mathematics Education. Chapel Hill, NC: Horizon Research, Inc.

Beyer

Davis

E. A.

(2009). Using educative curriculum materials to support preservice elementary teachers’ curricular planning: A comparison between two different forms of support. Curriculum Inquiry, 39(5), 679–703.

10.

Bismack

A. S.

Arias

A. M.

Davis

E. A.

Palincsar

A. S.

(2014). Connecting curriculum materials and teachers: Elementary science teachers’ enactment of a reform-based curricular unit. Journal of Science Teacher Education, 25(4), 489–512.

11.

Bismack

A. S.

Arias

Davis

E. A.

Palincsar

A. S.

(2015). Examining student work for evidence of teacher uptake of educative curriculum materials. Journal of Research in Science Teaching, 52(6), 816–846.

12.

Borko

Jacobs

Eiteljorg

Pittman

M. E.

(2008). Video as a tool for fostering productive discussions in mathematics professional development. Teaching and Teacher Education, 24, 417–436.

13.

Cervetti

Kulikowich

Bravo

(2015). The effects of educative curriculum materials on teachers’ use of instructional strategies for English language learners in science and on student learning. Contemporary Educational Psychology, 40, 86–98.

14.

Cheung

Slavin

R. E.

Kim

Lake

(2017). Effective secondary science programs: A best-evidence synthesis. Journal of Research in Science Teaching, 54(1), 58–81.

15.

Collopy

(2003). Curriculum materials as a professional development tool: How a mathematics textbook affected two teachers’ learning. The Elementary School Journal, 103(3), 227–311.

16.

Davis

E. A.

Janssen

Van Driel

(2016). Teachers and science curriculum materials: Where we are and where we need to go. Studies in Science Education, 52(2), 127–160. doi:10.1080/03057267.2016.1161701

17.

Davis

E. A.

Krajcik

(2005). Designing educative curriculum materials to promote teacher learning. Educational Researcher, 34(3), 3–14.

18.

Davis

E. A.

Palincsar

A. S.

Arias

Bismack

Marulis

Iwashyna

(2014). Designing educative curriculum materials: A theoretically and empirically driven process. Harvard Educational Review, 84(1), 24–52.

19.

Drake

Land

Tyminski

(2014). Using educative curriculum materials to support the development of prospective teachers’ knowledge. Educational Researcher, 43(3), 154–162.

20.

Duncan

Frymier

(1967). Explorations in the systematic study of curriculum. Theory Into Practice, 6(4), 180–199.

21.

Grossman

Thompson

(2008). Learning from curriculum materials: Scaffolds for teacher learning? Teaching and Teacher Education, 24(8), 2014–2026.

22.

Hill

Charalambous

(2012a). Teacher knowledge, curriculum materials, and quality of instruction: Lessons learned and open issues. Journal of Curriculum Studies, 44(4), 559–576.

23.

Hill

Charalambous

(2012b). Teaching (un)Connected Mathematics: Two teachers’ enactment of the pizza problem. Journal of Curriculum Studies, 44(4), 467–487.

24.

Institute of Education Sciences & National Science Foundation. (2013). Common guidelines for education research and development. Washington, DC: Author.

25.

Janssen

Westbroek

van Driel

(2014). How to make guided discovery learning practical for student teachers. Instructional Science, 42, 67–90.

26.

Kademian

Arias

Davis

E. A.

Palincsar

A. S.

(2015). Analyzing teacher adaptations of two investigation-based science units: Teaching with the goals of the NGSS in mind. Paper presented at the annual meeting of the NARST organization, Chicago.

27.

Kademian

Arias

Davis

E. A.

Palincsar

A. S.

(2017). Supporting the use of scientific language: Teachers’ use of content-foregrounded educative features. Journal of Science Teacher Education, 28, 146–168.

28.

Krajcik

(2017). The benefits and limitations of educative curriculum materials. Journal of Science Teacher Education, 28(1), 1–10.

29.

Loucks-Horsley

Love

Stiles

Mundry

Hewson

(2003). Designing professional development for teachers of science and mathematics (2nd ed.). Thousand Oaks, CA: Corwin Press.

30.

Marco-Bujosa

L. M.

McNeill

K. L.

González-Howard

Loper

(2016). An exploration of teacher learning from an educative reform-oriented science curriculum: Case studies of teacher curriculum use. Journal of Research in Science Teaching, 53(2), 261–290.

31.

McNeill

(2009). Teachers’ use of curriculum to support students in writing scientific arguments to explain phenomena. Science Education, 93(2), 233–268.

32.

McNeill

Pimentel

Strauss

(2013). The impact of high school science teachers’ beliefs, curricular enactments and experience on student learning during an inquiry-based urban ecology curriculum. International Journal of Science Education, 35(15), 2608–2644.

33.

National Academies of Sciences Engineering and Medicine. (2015). Science teachers’ learning: Enhancing opportunities, creating supportive contexts. Washington, DC: National Academies Press.

34.

National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: The National Academies Press.

35.

National Science Resources Center. (2004a). Science technology and children ecosystems (2nd ed.). Burlington, NC: Carolina Biological Supply Company.

36.

National Science Resources Center. (2004b). Science technology and children electric circuits (2nd ed.). Burlington, NC: Carolina Biological Supply Company.

37.

Pringle

Mesa

Hayes

(2017). Professional development for middle school science teachers: Does an educative curriculum make a difference? Journal of Science Teacher Education, 28(1), 57–72.

38.

Putnam

Borko

(2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 29(1), 4–15.

39.

Quintana

Reiser

Davis

E. A.

Krajcik

Fretz

Duncan

R.G.

. . . Soloway

. (2004). A scaffolding design framework for software to support science inquiry. Journal of the Learning Sciences, 13(3), 337–386.

40.

Remillard

J. T.

(2000). Can curriculum materials support teachers’ learning? Two fourth-grade teachers’ use of a new mathematics text. Elementary School Journal, 100(4), 331–350.

41.

Remillard

J. T.

(2005). Examining key concepts in research on teachers’ use of mathematics curricula. Review of Educational Research, 75(2), 211–246.

42.

Romance

N. R.

Vitale

M. R.

(2001). Implementing an in-depth expanded science model in elementary schools: Multi-year findings, research issues, and policy implications. International Journal of Science Education, 23(4), 373–404.

43.

Roseman

Herrmann-Abell

Koppal

(2017). Designing for the Next Generation Science Standards: Educative curriculum materials and measures of teacher knowledge. Journal of Science Teacher Education, 28(1), 111–141.

44.

Schneider

Krajcik

(2002). Supporting science teacher learning: The role of educative curriculum materials. Journal of Science Teacher Education, 13(3), 221–245.

45.

Smith

P. S.

Smith

A. A.

(2014). Investigating the impacts of educative curriculum materials: A quantitative perspective. Paper presented at the annual meeting of the NARST Annual International Conference, Pittsburgh, PA.

46.

Star

(2015). Improve math teaching with incremental improvements. Retrieved from http://www.kappancommoncore.org/improve-math-teaching-with-incremental-improvements/

47.

Stein

M. K.

Remillard

Smith

(2007). How curriculum influences student learning. In Lester

F. K.

(Ed.), Second handbook of research on mathematics teaching and learning (pp. 319–369). Greenwich, CT: Information Age Publishing.

48.

Trygstad

Smith

P. S.

Davis

E. A.

Palincsar

A. S.

(2012). Assessment tools for studying the effect of educative curriculum materials. Paper presented at the annual meeting of the National Association for Research in Science Teaching, Indianapolis, IN.