Abstract
In this article, the authors provide a response to the special issue The Science of Teacher Professional Development: Iterative Design Studies Across Content Areas. In doing so, they present the framework of professional development (PD) enactment developed by Mary Kennedy and apply it to the three studies highlighted in this special issue. They also present implications regarding the advancement of coherent theories of change within special education teacher education research.
The three studies highlighted in this special issue of Teacher Education and Special Education (TESE) showcase research funded by the U.S. Department of Education, Institute for Education Sciences (IES), under its Goal 2: Development and Innovation Competition. In Goal 2 studies, researchers design professional development (PD) innovations according to a theory of change that they ascribe to, and then determine, using data, if the PD innovation is feasible for teachers to implement, can be implemented with fidelity, and shows promise for improving teaching and student outcomes. Each study in this special issue (a) outlines a theory of change that reflects the researchers’ assumptions about those processes that are most likely to improve teachers’ knowledge and skills for educating students with disabilities, (b) designs a PD innovation based on that theory, and (c) tests the degree to which the innovation promotes intended changes. We analyze these studies to determine how the design and results might inform future teacher education research within special education and beyond, as well as provide directions for future research.
To further science on educational innovations, researchers need to articulate theories about how they expect people, whether they be adults or students, to learn in ways that will allow more expert performance to develop. To accomplish this goal, researchers need to articulate what underlies effective performance in an area and how can it be promoted; in other words, they need a clear theory of change that describes the ways in which more expert performance can be promoted. Furthermore, their methods for studying innovations should be well aligned with their theory of change. In special education teacher education, and in general education teacher education for that matter, researchers have not been clear about the theories of change underlying their innovations. In fact, often there does not appear to be any clear theory underlying the proposed innovation or the methods for studying it.
Kennedy Framework
To frame our analysis of Goal 2 studies in this special issue, we use Mary Kennedy’s (2016) categorization of PD research into four different mechanisms used to enact learning.
One, PD is used to prescribe a specific set of actions for teachers. This is the most didactic approach. PD providers “prescribe” the content and processes teachers should implement in their classrooms. The PD, therefore, is uniform and requires no teacher judgment. The underlying assumption is that PD providers are the experts and know the best ways to address problems of practice. Teachers, therefore, need only implement the PD content as intended. For example, teachers receiving PD on the implementation of a specific intervention are often taught the steps of the intervention and the rationale for implementing it with a high degree of fidelity. In these studies, PD providers often conduct follow-up observations to assess teachers’ fidelity of implementation. This approach to PD is likely one that is most familiar to special education, particularly when one considers the heavy emphasis our field places on helping teachers learn to implement evidence-based practices with fidelity.
Two, PD is used to assist teachers in making strategic decisions about their instruction; in other words, teachers are presented with multiple strategies, and then somehow aided in determining which strategies are best to employ given particular students’ needs. How to implement the strategies may be prescriptive, but teachers exert some level of independent judgment as to when to use a particular strategy. The underlying assumption is that teachers can address problems of practice when they fully understand an educational goal and have at their disposal a collection of strategies that can be flexibly applied to meet the goal. An example of this type of enactment is when teachers receive PD to meet the goal of increasing student engagement. In this example, teachers might be taught a series of strategies known to increase student engagement such as student choice, varied opportunities to respond, and behavior contracts. Teachers then choose which strategies to implement given their particular context.
Three, PD is used to help teachers gain insights into their instruction. Here, PD providers craft opportunities for teachers to have “aha!” moments about their practice and students’ learning (Kennedy, 2016, p. 955). PD that relies on an insight approach attempts to help teachers view situations from different perspectives, and then determine how to respond. The assumption is teachers will be compelled to exert independent judgment to alter their practice in response to the new insight. Take, for example, PD in which a teacher is asked to analyze students’ thinking about a mathematical concept and realizes that one of her students is developing some significant misconceptions. As a result, the teacher has newfound motivation to adopt a different approach to help the students develop more accurate conceptions of the material.
Four, PD strategies are used to help teachers accumulate a body of knowledge. Here, the underlying assumption is that with more knowledge, teachers will make better decisions about how to teach. In this approach, PD providers supply teachers with a coherent body of knowledge, much like a traditional university course (Kennedy, 2016). Teachers have complete autonomy in whether, what, or how they enact the knowledge. In this approach, there is no underlying assumption that teachers will necessarily carry out any particular action. One example of bodies of knowledge is when teachers attend a presentation at a national conference. The session presenter shares information related to a particular topic; attending teachers then make decisions about what they will do with the new information (if anything).
Kennedy then used these different categorizations of PD to analyze research on PD that examined the impact of various approaches on student achievement. She found that strategic and insight approaches were most likely to have an impact on student achievement; whereas, the other two approaches did not. Kennedy theorized that approaches that promoted teacher judgment and understanding might be more productive in terms of fostering teacher learning.
Analysis of Studies
Two of the special issue papers claim to use Desimone’s (2009) framework for conceptualizing PD (Griffin and colleagues; Lembke and colleagues). Desimone’s framework articulates the importance of designing PD according to specific features (e.g., active learning opportunities, content-focused learning opportunities, duration), but does not detail the underlying theory or theories of action as Kennedy (2016) suggested. Closer analysis of actual PD studies conducted by Desimone and colleagues, however, show that the framework seems to incorporate what Kennedy labels an insight and strategies approach, where emphasis is placed on helping teachers develop insights into the way students are learning content and then helping them implement instructional practices (e.g., active learning opportunities and collective participation) that would improve student learning. Thus, Desimone’s framework for PD seems most aligned with what Kennedy characterizes as insight and strategies approaches.
Kennedy and colleagues claim to use a cognitive apprenticeship approach to improving teacher learning. In this approach, PD providers assist teachers in developing expertise by modeling how they, as experts, engage in teaching tasks and make decisions about teaching and scaffolding teachers’ performance through feedback and support. The process of modeling and scaffolding involves a gradual reduction in expert guidance as teachers gain mastery of PD content and strategies. In the research on student learning, cognitive apprenticeship approaches have often been employed to teach P-12 students strategic approaches to their learning (e.g., Reciprocal Teaching). Thus, cognitive apprenticeship approaches would fit best within Mary Kennedy’s strategies approach to teacher learning, and possibly, insights approach if teachers were to gain better insight into their own teaching as they listened to coaches reason aloud while modeling effective instruction or when they were interacting with coaches as they assisted teachers in reflecting on their instruction (scaffolding).
For each study in this special issue, we analyze approaches used by the researchers to promote teacher learning according to Kennedy’s framework. In doing so, we move beyond an analysis of the studies in terms of their “program design features” (Kennedy, 2016, p. 971) to identify the underlying learning mechanisms inherent in these different PD designs and discuss their potential for promoting teacher learning. Specifically, we look at which of the four approaches identified by Kennedy are present across the studies included in the special issue and how they helped teachers translate new ideas into practice. We also discuss the degree to which researchers employed research designs and measures that captured changes in teachers that were anticipated by their underlying theories of change. Finally, we identified lingering questions about the research studies, particularly regarding relationships between study outcomes and specified theories of change. We use information generated from our analysis to develop recommendations for future research.
Lembke and Colleagues
Lembke and colleagues describe using features of Desimone’s (2009) framework (content, active learning opportunities, duration, and coherence) to improve teachers’ ability to implement data-based instruction (DBI) in writing. They address the content feature of Desimone’s framework by providing teachers with procedural knowledge of how to implement curriculum-based measures (CBMs) in writing and interpret resulting student data for the purpose of identifying research-based writing strategies that will improve students’ achievement on CBM of writing. They also provide teachers, through learning modules, with knowledge about how young children learn to write and strategies that can support acquisition of the writing process for students with disabilities. Primarily, they use coaching to help teachers implement a data-based decision-making process and evidence-based writing strategies, but it is unclear how they help teachers understand the ways in which students are developing their knowledge and skill in the writing process, a more diagnostic approach that definitely would reflect Desimone’s view of content-focused PD. They also assert coaching will provide teachers with the active learning opportunities they need to implement DBI and research-based writing strategies, further allowing teachers to align practices with the curriculum they are teaching. In Desimone’s framework, this is the coherence principle.
What is less clear in Lembke and colleagues’ theory of change are the ways in which they believe their learning modules combined with coaching will result in desired changes. Their use of coaching and their focus on implementation fidelity seem to assume that teachers will implement and adapt research-based writing strategies if they see their students are performing poorly on CBMs of writing, and if they have knowledge of and skill in using selected CBM and research-based writing strategies. Thus, the nature of their PD seems to ascribe to two approaches to teacher learning. One of those approaches is prescriptive; that is, teacher learning will occur when teachers can implement a specific set of skills with fidelity and with some fluency. It also seems, however, that Lembke and colleagues used a strategies approach, in which they facilitated teacher learning by providing teachers with knowledge and coaching in strategies they could select to improve student engagement and achievement based on data generated from CBMs. Student performance on CBMs were used to signal teachers to select strategies learned in the modules to meet students’ needs.
To determine whether their theory of change and resulting innovation showed promise, the Lembke and colleagues’ study used a combination of qualitative and quantitative measures. As is typical in IES Goal 2 studies, the researchers employed development and feasibility studies prior to piloting the innovation, and feasibility studies are concerned with implementation fidelity. These methods seemed well aligned with the more implicit theory of teacher learning to which they appear to ascribe, a prescriptive theory. Specifically, Lembke and colleagues, using knowledge and fidelity measures, assessed the degree to which teachers understood the components of data-based decision making and evidence-based writing strategies, and were able to apply what they learned with fidelity. In turn, by implementing these practices with fidelity, teachers were able to improve student learning. Thus, PD modules and active learning opportunities, such as coaching and collaborative discussions with teachers, do serve to help teachers implement evidence-based practices effectively, and doing so improves students’ achievement. Interviews with teachers also established that the prescribed strategies would align well with their curriculum materials and instructional context—Desimone’s coherence principle.
Less clear to us were the ways in which efficacy for teaching writing and writing orientation, content-focused not procedural knowledge, and strategic thinking figure into Lembke and colleagues’ theory of change or approach to teacher learning. By including teachers’ efficacy and orientations toward writing, they seemed to hypothesize that these would change as a result of the instructional modules and coaching, but they did not theorize why this might be the case. Research by Bandura (1982), Guskey (1986), and Tschannen-Moran, Hoy, and Hoy (1998) has shown that teachers (or students) are likely to change their self-efficacious beliefs and other beliefs when changes in their behavior result in some sort of improvement that is perceived as important. Furthermore, when teachers adopt a more self-efficacious stance, then they are more likely to use those practices. When it comes to beliefs about instruction, however, teachers are more likely to use practices that are aligned with their beliefs. In this case, changes in beliefs would result in changes in practice. In addition, it is difficult to know how changes in teachers’ analysis of data were linked to changes in their ability to select specific strategies, though we know their CBM data use and strategy use increased. In designing future research in this area, it might be important to further theorize exactly how the innovation contributes to changes in self-efficacy and beliefs about writing instruction; determine whether linkages can be made between the innovation, changes in teacher skills, and changes in teachers’ beliefs and self-efficacy; and track on how changes in teachers’ analysis of data result in changes in strategies they select.
Kennedy and Colleagues
Kennedy and colleagues report several elements of their Content Acquisition Podcast Professional Development (CAP-PD) intervention that aligned with elements of the cognitive apprenticeship approach. The CAP-PD included several components: (a) Researchers modeled evidence-based vocabulary strategies by creating Content Acquisition Podcasts; (b) separate CAPs were created that taught specific vocabulary using one or more of the evidence-based practices, these Content Acquisition Podcasts- Teacher Slides (CAP-TS) were for teachers to use during instruction; and (c) researchers provided teachers with feedback using the Classroom Teaching (CT) Scan. The modeling component of cognitive apprenticeship seems to be addressed in Content Acquisition Podcasts for Teachers with Embedded Modeling Videos (CAP-TV) demonstrations and CAP-TS materials, but it is not entirely clear how models presented helped teachers understand the ways in which experts think about vocabulary instruction. In addition, the feedback provided by the CT Scan seems to address the scaffolding strategy used in a cognitive apprenticeship approach, as teachers changed their practice in response. We are less clear about the ways in which researchers operationalized “gradual reduction in expert guidance,” a hallmark of cognitive apprenticeship approaches. We do know, however, from researchers’ anecdotal observations that teachers enhanced the CAP-TS, and one or two did so rather dramatically, suggesting that teachers were becoming more expert in providing vocabulary instruction and less in need of expert guidance.
In terms of Mary Kennedy’s (2016) framework for categorizing enactment of learning, the strategy approach was evident. Kennedy and colleagues presented teachers with five vocabulary instructional strategies (e.g., student-friendly definitions, examples, and nonexamples), and teachers were encouraged to use strategies. It was unclear, however, whether researchers provided a framework for when it was best to use particular strategies. Did teachers learn when and why it was appropriate to use particular strategies or how to determine whether they should select a different strategy based on how students were responding to instruction? Furthermore, even though a cognitive apprenticeship approach certainly could be aligned with an insight approach, we can only assume teachers gained more insights into their instruction as they demonstrated more effective teaching practice on the CT Scan, and anecdotally, they adapted CAP-TS in ways that improved them.
Kennedy and colleagues, however, also provided PD in ways that epitomized the prescriptive approach. Teachers were asked to watch and implement the vocabulary strategies with fidelity as presented in the CAP-TV and were also given coordinated curricular materials via the CAP-TSs. The use of coaching emails in response to data displays from the CT Scan is an interesting element because it simultaneously corresponds to an insight and prescriptive approach. Providing teachers with visual data on their instruction could lead to teacher insight. For example, a teacher might be surprised to learn that, for 40% of her instruction, students are passively working on seatwork. Teachers, however, also received direct feedback from coaches on their instructional strengths and areas to improve. This type of direct feedback aligns more with a prescriptive approach where coaches tell teachers what to do differently in the future. It is unclear to what degree the research team supported teachers in processing insights when they saw the results of their CT Scans. There is anecdotal evidence, however, that teachers did gain more insights as they adapted CAP-TS in ways that enhanced instruction on the vocabulary evidence-based strategies. Teacher interviews could be used in future work to better understand how teachers were coming to understand their instruction.
In evaluating their innovation, Kennedy and colleagues used quantitative and qualitative methods. For the iterative development process and assessing social validity, the research team conducted qualitative interviews. An interesting finding from early pilot work was that teachers wanted PD that went beyond simply having access to videos and sample teaching PowerPoint slides. Teachers seemed to want more prescriptive support. These findings led researchers to make deliberate changes to future iterations of the innovation. The fact that Kennedy and colleagues were able to develop instrumentation prior to implementing the intervention allowed them to systematically study how teacher performance improved as they redesigned the PD. This is a strength because it contributed to the larger body of evidence the research team amassed on what particular PD experiences led to differences in teacher performance. Quantitatively across all pilot studies, providing teachers with access to the vocabulary instructional materials (CAP-TSs) resulted in them implementing more frequently in their instruction, though not always with a high degree of fidelity. When teachers were included in the more comprehensive CAP-PD (i.e., CAP-TV, CAP-TS, and coaching), their time spent on vocabulary instruction was statistically significantly higher than teachers who only had access to curricular materials via the CAP-TSs. As noted earlier, this finding supports the idea that teachers’ instructional practice improves to a greater degree when they are given access to curricular materials, as well as video models for how to use them and feedback on implementing them. We do not know, however, whether the added positive effects are from the CAP-TV video models, the coaching emails, or both.
In terms of measures, Kennedy and colleagues used several teacher- and student-level measures that are well aligned with the intent of their PD, and this alignment was a notable strength of the study. Students’ science and vocabulary knowledge directly aligns with the content of the PD, and the CT scan provides data on the time teachers spent in vocabulary instruction (instructional quantity) and the particular vocabulary strategies implemented (instructional quality). The consistent results across studies provide evidence that when teachers are supplied with knowledge of vocabulary strategies, how to implement them, supportive curricular materials, and feedback on implementation, their vocabulary instruction improves both in quantity and quality. What we do not learn in Kennedy and colleagues’ study, that seems important for future research, is whether providing extensive, prescriptive curricular and implementation support allows teachers to generalize their learning to other aspects of their instruction. For instance, will teachers provide stronger vocabulary instruction with novel words that are not a part of the curriculum? Or, will they apply effective teaching practices that they are acquiring to other areas of their instruction? And, how does their thinking about instruction change?
Griffin and Colleagues
Griffin and colleagues incorporated several PD design features based on Desimone’s (2009) framework. In terms of content, Griffin and colleagues focused on mathematics and how students with disabilities learn mathematics. The coherence feature is seen in the PD’s grounding in the mathematics Common Core State Standards (CCSS-M; National Governors Association Center for Best Practices, Council of Chief State School Officers, 2010) and the National Council of Teachers of Mathematics (NCTM; 2000, 2006) standards. Active and collective participation was enacted through teacher inquiry projects and online discussions. Finally, Prime Online was extended in duration, lasting a full year.
Griffin and colleagues did not seem to employ a prescriptive approach to PD, as did Lembke and colleagues. Instead, they modeled a strategies approach to PD enactment by providing teachers with a series of mathematics instructional strategies to enhance students with disabilities’ learning in inclusive classrooms. We also see two instances of learning through insight. In Segment 2, teachers completed research cycles intended to expose students’ needs in mathematics. In Segment 3, teachers engaged in a teacher inquiry process, in which they pose a question and then collect and analyze data to determine an appropriate course of action. As part of this inquiry process, the teachers participated in an online discussion forum that provided opportunities for them to discuss their problems of practice, engage in joint problem solving, and share findings. Griffin and colleagues hypothesized the positive findings of Prime Online are due, in part, to these collective participation experiences. In terms of PD enactment through insight, however, it is unclear whether positive findings would have resulted simply by having teachers learn through insight, or whether a critical component of learning through insight is the opportunity to process insights with other colleagues.
In Prime Online, Griffin and colleagues used a preexperimental pre-/posttest design for the beliefs and practices measures, and a pre-/mid/posttest latent growth curve model for the Content Knowledge for Teaching Mathematics (CKT-M) subtests (Ball & Rowan, 2004; Hill, Ball, Blunk, Goffney, & Rowan, 2007). The measures used align well with the PD, but illustrate common methodological challenges special education teacher education researchers face. The CKT-M was well aligned with the content of the PD. Teachers’ beliefs and self-reported instructional practices changed in positive ways from pre- to posttest. However, as the authors stated, the participants’ beliefs at pretest already indicated a high degree of alignment with the PD. This supports the coherence principle, and, as the authors state, helps explain the positive findings. It is less clear, however, what the outcomes would have been if the participating teachers had entered the PD with divergent beliefs, or if having more aligned beliefs leads to different insights into instruction than unaligned beliefs. In terms of teacher practice, the Griffin and colleagues study presents an interesting tension in terms of online learning. A strength of Prime Online is that it was delivered entirely online, which has positive implications for its potential reach to a wide array of teacher populations. This, however, required the research team to rely on teacher self-report measures of practice, which may be subject to teachers’ over- or underreporting of certain practices. Furthermore, we do not know whether increased insights into student learning and instruction enabled teachers to better select strategies. In expanding the research on online learning, a next step might be to investigate how technologies that can provide remote observation and coaching might become core features. Finally, to measure student outcomes, Griffin and colleagues relied on the Florida Comprehsive Assessment Test (FCAT) statewide standardized assessment data, and did not find significantly different outcomes in students’ mathematics achievement. We posit a plausible explanation for this finding is not that Prime Online was ineffective, but rather the research team relied on mathematics data from the FCAT, which is likely not sensitive enough to detect growth for students with disabilities when teacher samples are small. These measures might also not be sufficiently aligned with the PD to detect changes.
Conclusion and Implications
Researchers involved in the studies of this special issue are to be commended for charting what has been largely uncharted territory in special education; that is, quantitative research focused on strategies for improving teacher learning whose efficacy can be documented through changes in teacher and student outcomes. Studies conducted by Lembke and colleagues and Kennedy and colleagues confirm previous findings from research on coaching and performance feedback in both the PD and initial preparation literature (Leko, Brownell, Sindelar, & Kiely, 2015), and Lembke and colleagues’ study draws into question Kennedy’s (2016) conclusions about effective PD. Specifically, these studies suggest teachers can learn to implement evidence-based strategies when they receive ongoing coaching support, particularly in the form of performance feedback. In addition, Kennedy and colleagues demonstrate that coaching feedback is more powerful in helping teachers implement effective strategies than simply having access to well-designed instructional materials that model effective implementation. In both studies, researchers, at least implicitly, ascribe to a prescriptive view of teacher learning. Thus, coaching, particularly when it uses performance feedback, seems to be a valuable strategy for improving instructional practice in PD studies that view teacher learning as a matter of acquiring a prescribed set of evidence-based practices. Furthermore, in the case of the Lembke and colleagues’ study, learning to implement a set of prescribed instructional practices can affect student achievement, at least on those assessments that are more proximal measures of achievement. Even though these authors did not correlate changes in teacher performance with changes in students’ CBM gains, we can assume that differences between control and treatment groups were due to the strategies used by treatment teachers.
At the same time, studies included in this special issue highlight an important need for future PD special education research, and according to Kennedy (2016), a need that is also present in general education PD research on teacher learning. Researchers in these studies and others need to be clearer about the underlying mechanism for teacher change—interestingly, a requirement of Goal 2 studies. As an illustration, PD innovations in the studies by Lembke and colleagues and Griffin and colleagues focused on duration, content, coherence, and active learning opportunities and teachers increased in their knowledge in both studies. Teacher knowledge, however, appeared to be conceptualized, promoted, and measured quite differently in these two studies, making it difficult to draw any conclusions about how PD might change teacher knowledge. Lembke and colleagues appear to assess what teachers know about implementing CBM and writing strategies. Lembke and colleagues seem to assume effective teaching results from implementing evidence-based assessment and instructional strategies. That is, when teachers know how to implement CBM and effective writing strategies, poor performance on CBM measures will signal teachers to use the writing strategies they know, and that using these strategies with fidelity will result in CBM gains. Learning how to diagnose students’ underlying writing problems and respond to them seems less emphasized.
In contrast, Griffin and colleagues assess what teachers understand about students’ mathematical thinking and how they might respond to students’ thinking. Thus, Griffin and colleagues, by using action research and helping teachers better understand misconceptions students develop and strategies they might use to respond to misconceptions, seem to demonstrate a belief that effective teaching results when teachers can effectively diagnose students’ misconceptions, gain insights on their instruction through analyzing it, respond appropriately by using strategies that have been shown to be effective in remediating those misconceptions, and evaluate the impact of their instruction on student learning. Both studies adopt a similar framework for change (Desimone), but upon closer examination, how the PD produces change is different, making it difficult to compare these approaches in terms of their impact. In addition, we were unclear in Griffin and colleagues’ and Lembke and colleagues’ studies about the ways in which beliefs about instruction figured into their theories of change. Did the researchers theorize that newly acquired knowledge about effective practice would result in changes in teachers’ beliefs about instruction? Or, did they believe that when teachers changed their practices and saw improvements in student achievement, they would subsequently change what they believed about effective instruction for these students? Theorizing these causal paths and assessing them will be important in determining whether a PD innovation works as intended.
Kennedy and colleagues, by contrast, avert the need to improve teachers’ knowledge, at least of science content and pedagogical practices, by creating CAP-TS for each vocabulary word in typical science curriculum and modeling each of the vocabulary strategies they want teachers to demonstrate. Instead, they focus their efforts on skilled implementation via coaching and feedback on the CT Scan. In making this move, Kennedy and colleagues assumed that teachers did not have adequate knowledge for selecting the appropriate vocabulary and strategies, did not have the time to identify appropriate vocabulary or were not motivated to do so, could learn to teach vocabulary more effectively if some of the planning burden was lifted, or some combination of these factors. What we do not come to understand in the Kennedy and colleagues’ study is the degree to which teachers gained better understandings or insights into what it meant to implement more effective vocabulary instruction. We know that researchers collected anecdotal information from observing teachers who suggested they did acquire insights—teachers adapted the CAP-TS slides in ways that seemed to enhance the quality of vocabulary instruction provided. We, however, do not fully understand what teachers learned about effective vocabulary instruction from implementing materials. Could they apply their knowledge to novel words? Could they provide more effective instruction in other aspects of their teaching given they received feedback on it? Going forward, Kennedy and colleagues and other researchers might theorize and test how PD innovations can support developmental differences in teachers’ knowledge and skills, and in turn, understand the impact differential support has on teachers’ knowledge and skills.
Differences in how Griffin and colleagues, Lembke and colleagues, and Kennedy and colleagues promote changes in teacher knowledge, beliefs, efficacy, and/or skills, and their development in their PD innovation highlight the need for future researchers to clarify their theories of action. If researchers do not articulate clearly how teacher learning is promoted, or what the underlying learning mechanism is, then we will be challenged to understand why PD approaches with similar features do not always yield similar findings, and we will continue to be challenged to identify, through a synthesis of the research, those PD innovations that are most effective.
Studies in this special issue also raise three major measurement and design concerns. One has to do with availability of high-quality teacher and student measurements for researchers to use, and the burden their creation places on researchers when they are simultaneously trying to develop and implement an innovation. Kennedy and colleagues’ study was the only study that was funded for 4 years and was able to use the first year to develop the measurement and PD innovation prior to implementing either. Doing so allowed Kennedy and colleagues to systematically study the development and refinement of his PD innovation over a 3-year period. In most Goal 2 studies of teachers, however, researchers are tasked with developing the innovation and then most, if not all, of the measures they need to assess the theory of change within a 3-year period. Having to simultaneously develop the measurement and innovation might compromise the iterative research process that is intended in Goal 2 studies. Furthermore, the financial and intellectual capacity required to develop simultaneously PD innovations and valid measures is enormous. Instead, researchers likely resort, in some cases, to selecting instrumentation that is already developed, but may not be well aligned with the content of their PD or their theory of change.
The second and third concerns have to do with student measurement and design of Goal 2 studies. For the most part, group-administered assessments of student achievement, which are the most feasible to employ in Goal 2 studies, are often distal measures of student achievement. Performance on these measures is often not well aligned with what teachers learned in the PD. In addition, students’ performance on group-administered state assessments becomes more and more dependent on factors that are not within teachers’ control (e.g., cognitive ability) as students age. Thus, researchers are in the position of having to implement a limited number of individually administered student assessments, typically CBMs, such as oral fluency. In addition, unlike in Goal 2 studies of student interventions, teacher education researchers have to secure adequate numbers of teachers to assess the impact of their innovations on teacher outcomes as well as assess achievement of their students. These measurement and design demands overwhelm researchers’ capacity for implementing PD research. Perhaps, expectations for Goal 2 studies could be reconsidered in the future, a point to which we return in future directions.
Future Directions for Research
Enactments of PD employed by research teams in this special issue point to several avenues for future research. In all three studies, positive results in terms of teacher outcomes, and in some cases, student outcomes occurred, but several unanswered questions remain. One question relates to long-term impact of particular approaches to teacher learning. For example, to what degree are teachers able to sustain and generalize a PD approach that is largely prescriptive? Will they continue to implement practices and strategies with fidelity when they no longer receive performance feedback? Will they be able to generalize the principles promoted in the PD to other instructional contents and contexts? Another unanswered question centers on coaching. The Lembke and colleagues’ and Kennedy and colleagues’ studies provided evidence for the positive effects of coaching, but it is unclear whether coaching is most effective when used to provide prescriptive performance feedback or when the coach serves as a mechanism through which teachers gain insights about their practice, or both. Similar questions could be raised about the approach Griffin and colleagues adopted. Does an insight approach allow teachers to recognize other aspects of their mathematics instruction that are problematic? Does the insight approach provide enough implementation support to change teachers’ instruction? A final question is that of when to use particular approaches to teacher learning and why. Is it possible certain approaches are better matched with particular outcomes of interest? Perhaps, a prescriptive approach is strongest when the outcome of PD is for teachers to acquire specific didactic knowledge; whereas, an insight approach is best when PD is designed to help teachers acquire an inquiry-oriented stance.
Finally, methodological challenges presented earlier lead us to believe a reconceptualization of the structure of IES Goal 2: Development and Innovation projects in personnel preparation for students with disabilities might be warranted. Perhaps, investigators whose Goal 2 studies show promise could receive guaranteed additional funding to study the pilot teachers for an additional year to determine the long-term effect of particular approaches to PD on teacher and student outcomes. We also assert Goal 2 studies in special education contexts should not mandate evaluating impacts on students, or make clear that researchers are not expected to use multilevel modeling to address effects. The budget for these projects is not large enough to address issues of power and nesting of students within classrooms and classrooms within schools. In other words, it seems unfeasible to expect multilevel designs for IES Goal 2 projects. IES Goal 3: Efficacy studies seem to be the appropriate place for this kind of work. Finally, in terms of measurement, some strategic requests for measurement proposals in teacher education could begin to address the dearth of validated measures.
Moving forward, it will be incumbent on special education researchers to consider carefully their theories of teacher learning and how those theories align with teacher and student measures selected to assess them. Ultimately, theories of teacher learning should articulate both what defines effective teaching and how it can be promoted. Making these theories explicit will help to unify the special education teacher preparation research base, so long-standing questions about what constitutes effective preparation can be answered with greater confidence.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.