The Effect of Content-Focused Coaching on the Quality of Classroom Text Discussions

Abstract

This study examines the effect of a comprehensive literacy-coaching program focused on enacting a discussion-based approach to reading comprehension instruction (content-focused coaching [CFC]) on the quality of classroom text discussions over 2 years. The study used a cluster-randomized trial in which schools were assigned to either CFC or standard practice in the district for literacy coaching. Observers rated classroom text discussions significantly higher in CFC schools. Teachers in the CFC schools participated more frequently in coaching activities that emphasized planning and reflecting on instruction, enacting lessons in their classrooms, building knowledge of the theory underlying effective pedagogy, and differentiating instruction than did the teachers in the comparison condition. Qualitative analyses of coach interviews identified substantive differences in the professional development support available to coaches, scope of coaches’ job responsibilities, and focus of coaching resources in the CFC schools and comparison schools.

Keywords

HLM (hierarchical linear modeling)literacy/reading teacher education urban teacher education

Classroom talk plays a critical role in the development of students’ literacy skills (Murphy, Wilkenson, Soter, Hennessey, Alexander, & Nystrand, 2009; Nystrand, 2006). Research links improved reading comprehension to interactive classroom discussions that encourage students to respond to and build on others’ contributions and that support students to apply higher level cognitive skills to interpret a text (Murphy, Wilkenson, Soter, Hennessey, & Alexander, 2009). These types of discussions benefit all children, including children from low-income families (Applebee, Langer, Nystrand, & Gamoran, 2003; Murphy et al., 2009; Nystrand, 2006) and language-minority backgrounds who are most at risk for reading problems (August & Shanahan, 2006; Goldenberg, 2010). However, research suggests that interactive and cognitively demanding discussions in classrooms are rare (Applebee et al., 2003). Such discussions represent a significant departure from traditional classroom discourse characterized by students providing brief answers to known-answer questions and are difficult for many teachers to implement (Cazden, 2001; Tharp & Gallimore, 1988).

Hiring school-based literacy coaches is one approach adopted by districts to improve the quality of classroom discussions and instruction generally (Neufeld & Roper, 2003). Literacy coaching has been supported by nearly every urban district in the country and included in educational reform policies implemented at the state (e.g., Florida) and federal levels (e.g., Reading First). Although there are many different types and approaches to coaching, such as cognitive coaching (Costa & Garmston, 1994) and peer coaching (Joyce & Showers, 1995), these approaches to coaches’ intended work with teachers share many commonalities: modeling instructional practices, reflecting on observed instruction, studying student work with teachers, lesson planning, and leading professional learning groups for teachers.

In spite of wide endorsement for literacy coaching, however, districts and schools vary greatly in their interpretation and implementation of coaching practice (Blarney, Meyer, & Walpole, 2009; Duessen, Coskie, Robinson, & Autio, 2007; Poglinco et al., 2003). Working with teachers to improve their instruction is only one of several responsibilities tasked to coaches, leaving considerable room for interpretation in how much time coaches should devote to each of their many duties (Bean, Draper, Hall, Vandermolen, & Zigmond, 2010). Duessen et al. (2007), for example, found that less than half of Reading First coaches devoted a large percentage of their time to working with teachers. A large percentage of Reading First coaches spent the majority of their time tutoring students or carrying out administrative tasks. Other research found that coaches can be spread thinly across grades and schools, further reducing the amount of time they have available to work with individual teachers (Rivera, Burley, & Sass, 2004). In addition, professional standards for coach qualifications have been developed only recently and are rarely enforced (Blarney et al., 2009). Coaches do not always have the pedagogical knowledge and skills necessary to coach effectively, and few professional development opportunities are available for coaches to develop their skills (Marsh et al., 2008; Neufeld & Roper, 2003).

The significant investment of resources allocated to hiring coaches notwithstanding, evidence of the effectiveness of literacy coaching for improving teaching quality is limited. The majority of research on literacy coaching is descriptive and does not support causal inferences about the effect of coaching on instruction (e.g., Neufeld & Roper, 2003). To address this gap, more rigorous designs are being implemented to examine the effectiveness of clearly specified coaching models on teaching (Garet et al., 2008; Neuman & Wright, 2010; Powell, Diamond, Burchinal, & Koehler, 2010; Sailors & Price, 2010). These studies have yielded mixed findings. Sailors and Price (2010), for example, compared the effectiveness of two professional development models on teaching quality in Grades 2 through 8: a 2-day professional development institute and the same institute with classroom-based coaching. Results showed a positive effect of the professional development institute plus coaching intervention on the quality of reading instruction. In contrast, Garet et al. (2008) found positive effects of two models of professional development—participation in a professional development institute and participation in the same institute with coaching—on teachers’ explicit reading instruction in second grade relative to a comparison sample. They found no added benefit of coaching on teachers’ instruction.

Studies conducted by Powell et al. (2010) and Neuman and Wright (2010) identified positive effects of coaching on the structural characteristics of the classroom environment, but not on the process characteristics (i.e., teacher–student interactions). In Powell et al. (2010), teachers in Head Start centers were randomly assigned to on-site or remote coaching conditions or to a comparison condition. Results indicated positive effects of the coaching interventions on the quality of the classroom environment and the literacy supports provided to children, but not on the specific quality of teaching interactions to support children’s language development. In Neuman and Wright (2010), teachers in child care centers and family day care settings participated in one of three conditions: traditional coursework, coursework with coaching, or to serve as a control sample. Teachers in the coursework with coaching group realized greater gains in the quality of the book area, writing area, and general literacy environment relative to the teachers in the other two conditions. No effect was detected, however, on the quality of teaching interactions or strategies.

In sum, research to date presents mixed results about the impact of coaching on the interactions between teachers and students. Moreover, most of the experimental research has focused on teaching in the preschool and primary grades. Research is needed that examines the effectiveness of coaching for inducing the types of discursive exchanges in classrooms associated with developing higher level reading comprehension skills in older students (Applebee et al., 2003; August & Shanahan, 2006; Goldenberg, 2010; Murphy et al., 2009; Nystrand, 2006).

Purpose of the Study

In this study, we investigate the effect of a comprehensive literacy-coaching program (content-focused coaching [CFC]) focused on implementing a discussion-based approach to comprehension instruction on the quality of classroom discourse. The study is a cluster-randomized trial. Specifically, we compare text discussion quality in schools that participated in the CFC intervention with a comparison sample of schools that continued with the literacy coaching that was standard practice in the district.

Our second purpose for this study is to investigate how the CFC program with its focus on implementing a discussion-based approach to comprehension instruction might achieve its desired effect on text discussion quality. To accomplish this goal, we examine the level of implementation of the CFC program and the specific ways literacy coaching differed in the CFC and comparison conditions. We pay special attention to four critical factors that can either help improve or constrain the quality of teachers’ learning opportunities (Allington, 2006; Blarney et al., 2009; Duessen et al., 2007): the amount and type of coaching received by teachers, coaches’ professional training and support received, coaches’ job responsibilities, and the focus of coaches’ work with teachers. Identifying differences between treatment and comparison conditions is key to understanding an intervention’s effect and is especially important in a study such as ours where the comparison condition of standard practice in a district is less well defined than what would be the case in a laboratory environment (Raudenbush, 2005).

CFC Program Intervention

The Content-Focused Coaching® program was originally developed in mathematics (Staub, West, & Bickel, 2003), but was adapted for use in elementary literacy by the University of Pittsburgh’s Institute for Learning (IFL). The design of CFC is based on the theory that substantive improvement in the quality of teaching and learning requires learning opportunities for personnel across multiple levels of the school system (Resnick & Spillane, 2006). The CFC program thus provides intensive professional development to literacy coaches aimed at increasing their pedagogical knowledge and skills at teaching a curricula domain, in addition to ensuring their ability to work effectively with teachers. The subject-matter explicitness of CFC distinguishes this approach from other coaching programs that focus on general coaching and teaching strategies (e.g., Costa & Garmston, 1994; Joyce & Showers, 1995). District leaders and principals also participate in the coach professional development to help create the organizational conditions in schools that support teacher learning and instructional improvement (Elmore, 2003). CFC-trained coaches, in turn, return to their school and work with teachers in grade-level teams (i.e., in professional learning groups) and individually in their classroom to improve the quality of instruction. In the following sections, we describe the substance and form of the professional development provided to CFC coaches and leaders to ensure coaching quality, and the substance and form of the professional development provided to teachers to improve the quality of their classroom text discussions.

Professional Development for CFC Coaches and Leaders

The substance of the training provided to the CFC coaches focused primarily on planning and enacting Questioning the Author (QtA; Beck & McKeown, 2006) lessons based in the district’s curricula (i.e., basal reader series). QtA is a discussion-based approach to reading comprehension instruction that encourages students to view authors as fallible. Students are taught to understand that difficulties understanding a text can be due to unclear writing as opposed to limitations in their reading skills To support comprehension, teachers develop queries that they pose to students throughout the reading of a text that are designed to focus students on key ideas and to build students’ conceptual understandings. QtA was chosen as the substance of the CFC intervention based on research linking this method to improved reading comprehension skills for students in the upper elementary grades (McKeown, Beck, & Blake, 2009), and based on the IFL’s special expertise in this method. The professional development also focused on building coaches’ skills at conferring with teachers and leading a professional learning group, in addition to educating leaders about how they could support their coaches’ work with teachers.

The form of the professional development provided to coaches to build their pedagogical skills draws on a cognitive apprenticeship model of instruction (Collins, Brown, & Holum, 1991). The IFL began their work with coaches by studying the theory underlying QtA and effective reading comprehension instruction. They then modeled QtA lessons for coaches and shared QtA lesson plans with the coaches that were rooted in the district’s curricula. The coaches then practiced teaching QtA lessons in teachers’ classrooms and coplanned QtA lessons with the IFL and their fellow coaches before moving on to independently model and plan QtA lessons with teachers. Following a similar pattern, coaches developed their coaching skills by first watching videotapes of IFL fellows conferring with individual teachers and leading teacher professional learning groups. The coaches then were observed conferring with teachers individually and leading professional learning groups (i.e., grade-level teams) and received feedback from the IFL and their coach colleagues. The coaches then moved on to independently coach teachers in their school.

To monitor the quality of the coaches’ pedagogical skills, coaches were observed twice teaching a QtA lesson in a teacher’s classroom (once by the IFL fellows and once by a coach colleague) in the 1st year of the study. To monitor the quality of coaches’ developing coaching skills, coaches also were observed twice (once by the IFL fellows and once by coach colleagues) leading a teacher learning group and conferring individually with teachers in the 2nd study year. CFC coaches also kept logs to track and analyze their use of time and created portfolios of their work in Year 2 of the study, demonstrating their efforts in four key areas: (a) teaching students as a model for teachers, (b) leading teacher learning groups, (c) conferring individually with teachers, and (d) meeting regularly with their principal.

Professional Development Provided to Teachers

The substance of the CFC coaches’ work that they then undertook with teachers focused primarily on planning and enacting QtA lessons. To support this instructional goal, coaches led weekly professional learning groups with fourth- and fifth-grade teachers to develop their knowledge of QtA and effective reading comprehension instruction. Coaches also met with teachers individually on a monthly basis to engage in a cycle of lesson planning, lesson enactment, and reflecting on QtA lessons with a special focus on student learning (e.g., what students did or did not understand and the next steps for instruction). Conceptual tools created by the program developers were used to assist coaches’ work with teachers. For example, coaches followed a protocol for lesson planning that guided coaches and teachers through a specific series of questions that include the following: What is the intended student learning? How will the teacher model clear expectations for students’ learning and activity promote rigorous thinking? How will instruction be adapted to students’ learning needs? Why is the lesson content appropriate to students’ learning needs and prior knowledge? Coaches began their lesson enactment work with teachers by first modeling QtA lessons. In addition to showing teachers what QtA looks like in practice, beginning work with teachers in this way helped establish coaches’ credibility as instructional specialists and a learning culture where everyone’s practice—beginning with the coach—is put forth for critique. Over time, coaches also observed teachers and cotaught QtA lessons with teachers during the lesson enactment phase of the coaching cycle. Concentrating coaching resources on two grades helps ensure that coaches have sufficient time to meet with teachers at these levels of frequency.

Method

Design of the CFC Trial

To examine the effectiveness of the CFC program, schools were assigned to either the CFC program or the comparison condition to be studied over a 3-year period. In the study reported here, we present results from the 2nd year of the trial to provide insight on the implementation of the program and to monitor potential interim effects. The natural clustering in schools, with students nested within teachers nested within schools, and implementation at the school level make a group-randomized design (GRT), the most appropriate design for studying program effectiveness (Bloom, 2005; Boruch & Foley, 2000). Schools in the comparison condition continued with the professional development resources considered standard practice for the district. No effort was made to influence or constrain teachers’ professional learning activities in these schools. After assigning schools to conditions, we found that the large majority of comparison schools (n = 11) hired literacy coaches, thus providing a unique opportunity for us to compare CFC literacy coaching more precisely with standard practice for literacy coaching in the district.

Sample

Schools

Title 1 schools serving high numbers of low-income, minority, and English language learning (ELL) students were recruited for this trial from an urban district in the Southwestern United States (15 intervention and 14 comparison schools). As mentioned above, 11 of the 14 comparison schools also had literacy coaches. The remaining comparison schools used their professional development resources in other ways, for example, to hire reading specialists who worked directly with students. In this study, we specifically report on these 11 schools with literacy coaches to compare CFC with literacy coaching that was standard in the district. Our final sample includes 26 schools. We compared school demographics for this subsample and found no differences between CFC and comparison schools on student characteristics (see Table 1). On average, the overwhelming majority of students were low income and Hispanic, and more than 40% of the students were English language learners.

Table 1.

Baseline School Characteristics of Student Demographics and Reading Achievement

	CFC (n = 15)		Comparison (n = 11)
School measure	%	M (SD)	%	M (SD)
Grade 4	54		55
Gender: Female	51		51
Ethnicity
African American	12		20
Latino	83		77
White	4		3
Other	1		0
Qualified for free lunch program	90		94
Immigrant status	4		3
Limited English proficient status	43		38
State standardized reading achievement (score)		2,198 (42)		2,172 (64)
Degrees of reading power (score)		36 (4)		35 (5)

Note: CFC = content-focused coaching. No differences were detected on any school-level measure.

Teachers

All of the fourth- and fifth-grade teachers in a school were recruited to participate in the trial. As previously mentioned, we report findings based on schools with literacy coaches and their teachers who remained in the trial for 2 years. At baseline, 192 teachers agreed to participate in the study. By the end of the 1st year, 95 teachers in the treatment group and 82 teachers in the comparison group remained in the study. We found no difference in attrition rates across conditions. At the conclusion of the 2nd study year, 93 teachers (59 CFC and 32 comparison) were still participating in the trial and teaching the fourth- and fifth grade. The loss of teachers from baseline to the conclusion of the 2nd year is high but is not unusual in urban schools (Allensworth, Ponisciak, & Mazzeo, 2009; Hanushek, Kain, & Rivkin, 2004). Six teachers left the trial because of departmentalization; they no longer taught language arts. Two teachers were on maternity leave and one teacher was on leave from her school for an unidentified reason. Two teachers left the trial because they no longer wanted to participate in the study. The rest of the teachers left because they transferred to a different school. Comparisons were made between the teachers who left the trial by the end of Year 2 and those who remained in the study for 2 years. Using survey reports of teachers’ education, number of years teaching, and language of instruction, no significant differences were detected at baseline. No differences were detected on quality of classroom text discussion outcomes at baseline and student reading achievement outcomes of the trial collected at the end of Year 1 (see descriptions of measures in sections below).

We also compared the CFC and comparison teachers who remained in the trial for the full 2 years (see Table 2). These comparisons were made to identify any systematic preexisting differences in teachers’ education and teaching experiences and to ensure that the groups were not associated with the condition assignment at baseline. The two groups reported similar levels of education, experience, and language of instruction. All indicated that they were full-time teachers. In all, 32% of the teachers had attained a master’s degree; 8% held temporary, provisional, or emergency certification; and 4% were certified through the National Board for Professional Teaching Standards (NBPT). The teachers reported a wide range of teaching experience, ranging from 0 to 31 years and averaging 10 years teaching generally and 9 years teaching reading.

Table 2.

Baseline Survey Reports of Teacher Background, Quality of Text Discussion Rating, and Student Reading Achievement for Teachers Remaining in the CFC Study from Fall 2006 to Spring 2008

	CFC (n = 59)		Comparison (n = 34)
Background measure	%	M (SD)	%	M (SD)
Teacher
Bachelor’s degree (%)	100		100
Master’s degree (%)	34		26
Certified (%)	98		96
Regular certificate (%)	88		83
Temporary certificate (%)	10		4
NBPT certificate (%)	2		9
Teaching a designated ELL class (%)	60		61
Teaching in general (years)	9		11
Teaching reading (years)	8		10
Student
State standardized reading achievement (score)		2,204 (64)		2,185 (72)
Degrees of reading power (score)		36 (7)		36 (10)

Note: CFC = content-focused coaching; ELL = English language learning. Survey responses on background information were provided at baseline by 73 of the 93 teachers remaining in the study. No background differences were detected between CFC and comparison teachers.

Literacy coaches

Comparisons of the teaching experience and education of literacy coaches in the CFC and comparison schools also revealed no significant differences. CFC coaches averaged 12 years of elementary teaching experience (ranging from 0 to 22 years) and 2 years of prior coaching experience (ranging from 0 to 9 years). One coach without prior teaching experience held a doctorate and worked previously at a research center developing professional development materials for teachers and conducting trainings for teachers. Four of the other coaches held a master’s degree. Literacy coaches in the comparison schools who were working with the upper elementary grade teachers (e.g., not Reading First coaches) averaged 15 years of prior teaching experience (ranging from 4 to 40 years) and 3 years of prior coaching experience (ranging from 1 to 6 years). Two of the coaches held a master’s degree and one held a doctorate.

Measures and Procedures

To assess the impact of the CFC program on the quality of classroom text discussions in this study, data were obtained from multiple sources: classroom observations, teacher survey reports, and literacy-coach interviews.

Quality of classroom text discussions

The Instructional Quality Assessment (IQA; Matsumura, Slater, Junker, Peterson, Boston, Steele, & Resnick, 2006; Matsumura, Garnier, Slater, & Boston, 2008) was used to measure the quality of text discussions observed in the classroom. The IQA focuses on key features of classroom talk that research has linked to improved student outcomes. These features include a high proportion of authentic and challenging questions, participant uptake of each other’s ideas, and participant use of evidence and reasoning to support claims (Michaels, O’Connor, & Resnick, 2007; Nystrand, 2006; Saunders & Goldenberg, 1999; Soter et al., 2008). All teachers were observed in the fall and spring of each year at times convenient for teachers. Observers requested in advance to visit teachers’ classrooms during the portion of their language arts block when the class held a discussion about a text. In the spring of each year, 89% to 93% of the teachers identified their observed lesson as “typical.”

Observers engaged in a 3-day training session that consisted of studying the IQA rubrics along with videotaped lessons and practicing rating and taking field notes from longer excerpts of samples of videotapes. Observers who successfully completed that portion of the training then observed four to six classroom lessons with the data collection supervisor. Observers who could not obtain more than 80% agreement with the data collection supervisor did not continue in the trial. Overall agreement between the observers and the data collection supervisor across all observations for each rater pair averaged 77% and 85% in the fall and spring of Year 1, and 86% and 87% in fall and spring of Year 2 of the trial. One of the observers in Year 1 and two of the observers in Year 2 were bilingual.

Seven items from the IQA were averaged to create one latent measure of text discussion quality (α = .78). Each item was assessed on a 4-point scale (coded 1 = poor, 2 = fair, 3 = good, 4 = excellent) with the exception of the item measuring text quality that was assessed on a 3-point scale (coded 1 = low, 2 = fair, 3 = high). The individual items were as follows:

Widespread student participation in the discussion

This item measures the percentage of students who participate in the discussion. To obtain the highest score of 4, more than three quarters of the students participate in the discussion at least once. To obtain a basic score of 2, fewer than half of the students participate in the discussion.

Teacher connects students’ contributions

This item measures the extent to which a teacher supports students in connecting ideas and positions to build coherence in the discussion. To obtain a 4, at least twice in the discussion a teacher revoices students’ contributions to highlight how their ideas relate to one another. To obtain a score of 2, a teacher repeats students’ comments but does not show how their ideas relate to one another.

Students connect to each other’s contributions

This item assesses the degree to which students connect to each other’s contributions in the discussion. To obtain a 4, at least twice in the discussion students connect their contributions to other students’ statements and shows how their ideas relate to one another. To obtain a 2, students refer to other students’ contributions in the discussion, but they do not show how their ideas relate to the ideas expressed by others.

Teacher presses for text-based evidence

This item assesses the extent to which a teacher presses students to provide text-based evidence for their contributions. To obtain a 4, at least twice in the discussion a teacher presses students to provide text-based evidence for their contributions or assertions. To obtain a 2, a teacher makes a superficial or formulaic effort to ask students to provide evidence.

Students provide text-based evidence for their contributions

This item assesses the degree to which students provide evidence for their contributions. To obtain a 4, students at least twice in the discussion provide accurate and elaborated evidence for their assertions. To obtain a 2, the evidence that students use to support their claim is inaccurate, vague, or unsubstantiated (e.g., a student states that the character will change, but not why).

Cognitive demand of the discussion

This item assesses the opportunity students have to analyze and interpret a text in the discussion. To obtain a 4, at least twice in the discussion a teacher asks questions that guide students to analyze the underlying meanings or literacy characteristics of a text. To obtain a 2, a teacher only asks questions that support students to identify the basic content or events in a text.

Grist of a text discussed

This item assesses the potential of a text to support a high-level discussion. Grist is evidenced in the complexity of a plot, characters, content, and/or writer’s craft (e.g., language, form, etc.). Examples of texts that receive the highest rating (a 3) on this scale include Esperanza Rising by Pam Munoz Ryan (2000) and Charlotte’s Web by E. B. White (1952). Texts that receive a 2 on this scale include most excerpts from basal readers and short articles from children’s magazines.

Teacher participation in coaching

At baseline and at the end of each trial year, teachers in both the treatment and comparison conditions completed surveys in which they described their interactions with coaches in activities and the content emphasized in those activities over the past year. Teachers reported how often they met with their coach in grade-level team meetings and individually, and how often their coach modeled lessons in their classroom, observed them teaching for at least 30 min, and cotaught a lesson with them. These items were coded 1 = never, 2 = 1 to 3 times/year, 3 = 4 to 6 times/year, 4 = monthly, and 5 = weekly.

The content emphasized in the coaching activities in which teachers participated was assessed with 16 items and averaged to create four dimensions:

Planning and reflecting on instruction (average of eight items, α = .88);

Enacting lessons in their classrooms (average of five items (α = .84);

Building knowledge of the theory underlying effective pedagogy (one item);

Differentiating instruction (average of two items, α = .80).

Each item was coded 1 = none, 2 = minor emphasis, 3 = moderate emphasis, and 4 = major emphasis. Cut points were then used to create variables measuring the four levels of emphasis from 1 = none to 4 = major emphasis.

Coach interviews

Coaches in the CFC and comparison schools were interviewed at the end of each study year using a structured protocol. The interviews ranged from 1 to 3 hr in length depending on the responder. The coach interview protocol focused on coaches’ professional development experiences, work with principals and teachers, and the factors at their school that supported or impeded their work. The interview transcripts were coded by a single researcher. A second researcher coded five interviews to check the reliability of the codes. The level of agreement between researchers was 88% overall across the different interview topics.

Analytic Methods

To account for the nested design of our trial, teachers within schools, and treatment being assigned at the school level, we use hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002). Teachers were assessed at baseline (fall 2006) and at three subsequent time points (spring 2006, fall 2007, spring 2008). We used a three-level HLM to model growth over time. Level 1 represents the repeated measures, Level 2 represents the teachers, and Level 3 represents the schools. Time was centered on spring 2008 to enable comparisons at the final time point between CFC and the comparison group. Initially, we included three teacher characteristics to improve the precision of our estimates and to compensate for any systematic biases due to preexisting differences in our sample. The covariates included years teaching reading, level of education, and regular teaching certificate. In preliminary analyses, none of the teacher-level covariates were significantly associated with the quality of text discussions outcomes and were omitted in favor of a more parsimonious model. We also examined a linear and a quadratic growth model, and the linear growth model provided the best model fit. The final combined model is as follows:

Y_{t i j} = γ_{000} + γ_{001} (CFC) + γ_{100} (time) + γ_{101} (time \times CFC) + u_{00 j} + u_{10 j} (time) + r_{0 i j} + r_{1 i j} (t i m e) + e_{t i j}

where Y_tij is the rating of quality of text discussion at time t for teacher i in school j, γ₀₀₀ is the mean rating of quality of text discussion at spring 2008, γ₀₀₁ is the difference between CFC and comparison on mean ratings of quality of text discussions at spring 2008, γ₁₀₀ is the linear growth rate of quality of text discussions, γ₁₀₁ is the difference between CFC and comparison on growth rate of quality of text discussions, γ₁₀₁ is the difference between CFC and comparison on growth rate of quality of text discussions, u_00j is the random effect associated with mean spring 2008 ratings of quality of text discussions across schools, and u_10j is the random effect associated with mean growth rates across schools, r_0ij and r_1ij are teacher-level random effects associated with the mean and growth rate, and e_tij are Level 1 random effects. Of primary interest in the model is the effect of the CFC program on (a) teachers’ mean quality of text discussions at spring 2008 (γ₀₀₁) and (b) difference between CFC and comparison on mean growth rate in the quality of text discussions (γ₁₀₁).

Cross-tabulations with chi-square tests were used to compare the amount and type of coaching received by teachers in the CFC and comparison (standard practice) conditions.

Finally, coach interviews were analyzed to further understand the results of the statistical analyses. Coach interviews were transcribed and analyzed using QSR N6, the sixth version of a product once known as NUD*IST (Non-numerical Unstructured Data Indexing Searching and Theorizing). Following Strauss and Corbin (1990), a process of open coding was used to identify the major ideas contained within each response from each coach to each question. An inductive process then was used to assign labels or codes to the different ideas, and a process of axial coding was used to reexamine and compare individual codes and associated text passages. Codes were then refined or combined as appropriate to characterize a particular action or idea.

Results

Effect of CFC on the Quality of Classroom Text Discussions

The average ratings of the quality of classroom text discussions observed at baseline and again in spring 2007, fall 2007, and spring 2008 are presented in Table 3. To measure the effect of the CFC program on growth in the average quality of classroom text discussions, we use the multilevel model described in the previous section.

Table 3.

Means and Standard Deviations of Quality of Text Discussion Ratings in CFC and Comparison Schools, Fall 2006 to Spring 2008

	Fall 2006	Spring 2007	Fall 2007	Spring 2008
Quality of text discussions rating	M (SD)	M (SD)	M (SD)	M (SD)
Quality of text discussions in CFC schools (n = 59 teachers)	2.14 (.46)	2.40 (.53)	2.57 (.55)	2.56 (.49)
Quality of text discussions in comparison schools (n = 34 teachers)	2.15 (.49)	2.21 (.50)	2.09 (.65)	2.15 (.58)

Note: CFC = content-focused coaching.

As shown in Table 4, CFC had a positive and significant effect on the quality of text discussion in spring 2008 (effect size [ES] = 0.89).¹ That is, CFC teachers’ ratings of quality of text discussion were 0.89 standard deviations, on average, higher than comparison teachers at the end of the 2 years. In addition, the growth rates for the two groups differed significantly; the average linear growth rate for CFC teachers was 0.18 standard deviations greater than the comparison teachers. Whereas the average quality of text discussions in CFC schools improved over 0.02 rating points per month, there was no significant growth seen in the comparison schools. Our ES for the CFC program on the average quality of text discussion is substantial and somewhat larger than those identified by Lipsey and Wilson (1993) in their meta-analyses of educational inservice professional development programs for teachers. Those ESs ranged from 0.47 to 0.80. In the context of other randomized trials of coaching programs, our ES of 0.89 is similar to those identified by Sailors and Price (2010) with regard to the influence of literacy coaching on reading comprehension instruction (0.64) and the opportunities teachers provide for students to engage in cognitive reading strategies (0.78). Our ES also is somewhat smaller than those identified by Powell et al. (2010; 0.92 and 0.99) and similar, on average, to Neuman and Wright (2010; 0.36-1.02) with regard to the impact of coaching on the structural features of the classroom literacy environment.

Table 4.

Results of HLM Growth Curve Analysis: Quality of Text Discussions in CFC and Comparison Schools^a, Fall 2006 to Spring 2008

Final fixed effects	Coefficient	SE	p value
Mean text discussion quality at Spring 2008, γ₀₀₀	2.390	.064	.000
CFC effect on mean text discussion quality at Spring 2008, γ₀₀₁	0.470^b	.125	.001
Mean growth rate, γ₁₀₀	0.011	.005	.032
CFC effect on mean growth rate, γ₁₀₁	0.024^c	.009	.018
Final random effects	Variance component	χ²	p value
Level 1 effect, e_tij	.198
Mean text discussion quality, r_0ij	.083	115.114	.000
Growth rate (teacher level), r_1ij	.000	65.026	>.500
Mean text discussion quality, u_00j	.047	42.481	.011
Growth rate (school level), u_10j	.000	50.249	.002

Note: HLM = hierarchical linear modeling; CFC = content-focused coaching; ES = effect size.

n = 1,706 students, n = 93 teachers, n = 26 schools (n = 15 CFC schools, n = 11 comparison schools with coaches).

ES = 0.89.

ES = 0.18.

Fidelity of Implementation of the CFC Intervention

To be able to interpret our results and understand what facilitated the changes in instructional quality, we turned to evidence of treatment fidelity using descriptions of coaching activities and content provided by teacher reports. Table 5 identifies the percentage of CFC and comparison teachers participating at different levels in individual coaching activities over the past year. Results show that very few teachers in the CFC schools participated in coaching at the levels intended by the program developers. Just 26% of the teachers in the CFC schools reported that they met with their coach in grade-level teams on a weekly basis, and only 39% of the teachers reported that they met individually with their coach once a month or more. Only 24% of the teachers reported being observed by their coach for at least 30 min on a monthly basis or more. Fewer teachers said their coach modeled a lesson in their classroom or cotaught a lesson with a coach on at least a monthly basis (19% and 14%, respectively). At most, 16 teachers participated in all of the coaching activities at the level intended by the CFC program developers.

Table 5.

Percentage of Teachers Participating at Different Levels in Literacy-Coaching Activities, Spring 2008

	CFC (n = 54)					Comparison (n = 21)
	Never	1 to 3 times	4 to 6 times	Monthly	Weekly	Never	1 to 3 times	4 to 6 times	Monthly	Weekly
Literacy-coaching activity	%	%	%	%	%	%	%	%	%	%
Coach met with me and other teachers in grade-level meetings	2	13	19	41	26	8	8	8	46	30
Coach met with me individually	2	35	25	12	27	27	54	4	11	4
Coach observed me teaching 30 or more minutes	9	48	19	15	9	50	39	8	0	4
Coach taught a model lesson for me	11	59	11	11	8	81	19	0	0	0
Coach taught a lesson with me in my classroom	33	46	8	10	4	96	4	0	0	0

Note: CFC = content-focused coaching.

Professional Development Support Provided to Teachers

We examined the amount and type of coaching received by teachers across conditions to identify differences in teachers’ professional development experiences that could help account for CFC’s effect on discussion quality. Results showed that greater percentages of teachers in CFC schools reported participating at higher levels in four activities than teachers in the comparison condition: meeting with their coach individually, χ²(4) = 16.77, p < .01, having their coach model lessons in their classroom, χ²(4) = 28.91, p < .001, observe them teaching for at least 30 min, χ²(4) = 23.65, p < .001, and coteach a lesson with them, χ²(4) = 22.49, p < .001. CFC and comparison teachers met with their coach in grade-level meetings with similar frequencies, most often monthly. In all, 39% of the teachers in the CFC schools reported that they met individually with their coach on either a weekly or monthly basis whereas just 15% of the teachers in the comparison schools did so. Furthermore, coaches never observed their teachers for at least 30 min or modeled a lesson for them in the classroom in 50% to 81% of comparison schools compared with only 10% of CFC schools.

Likewise, we compared teachers on how much coaches emphasized different content in their coaching activities. Table 6 presents the percentage of teachers reporting no, minor, moderate, and major emphasis coaches placed on each content area over the past year. Larger percentages of teachers in CFC schools reported greater emphasis on all content areas targeted by the CFC program: planning and reflecting on instruction, χ²(3) = 22.41, p < .001, enacting lessons in their classrooms, χ²(3) = 18.47, p < .001, building knowledge of the theory underlying effective pedagogy, χ²(3) = 17.65, p < .001, and differentiating instruction, χ²(3) = 19.56, p < .001, compared with the coaching received in comparison schools.

Table 6.

Percentage of Teachers Reporting Emphasis of Focus in Literacy-Coaching Activities, Spring 2008

	CFC (n = 54)				Comparison (n = 21)
	None	Minor	Moderate	Major	None	Minor	Moderate	Major
Focus of activity	%	%	%	%	%	%	%	%
Emphasis on planning and reflecting on instruction	4	14	62	20	4	74	19	4
Emphasis on help during lesson enactment	4	18	50	28	4	33	56	7
Emphasis on building theory underlying pedagogy	16	26	40	18	30	52	11	7
Emphasis on differentiating instruction	10	32	42	16	19	67	15	0

Note: CFC = content-focused coaching.

Compared with 82% of the CFC teachers, 23% of the comparison teachers reported moderate or strong emphasis on planning and reflecting on instruction in the coaching they received. Likewise, 58% of the teachers in the CFC schools but only 15% to 18% of teachers in the comparison schools reported moderate or strong emphasis on differentiating instruction and building theory underlying pedagogy. However, most teachers in comparison and CFC schools reported at least moderate emphasis on lesson enactment (63% and 78%, respectively). How much this stronger emphasis may have affected the quality of instruction must be considered in the context of the type of coaching received by teachers such as modeling lessons for teachers. Merely 4% to 19% of teachers in the comparison schools reported ever having a model lesson taught for them or with them in their classrooms compared with 67% to 89% of CFC teachers.

Professional Development Support Provided to Coaches

Substantive differences in the level of training and support provided to coaches in the CFC and comparison schools to build their pedagogical and coaching skills emerged in the qualitative analysis of the coach interviews. CFC schools provided coaches with greater professional development support, and only CFC schools involved principals in coach training.

CFC coach professional development

The CFC coaches participated in professional development led by the IFL for 3 days each month. All of the coaches reported that the training they received was very helpful for supporting their work in schools, especially the opportunity to collaborate and practice skills with other coaches in advance of working with teachers. As one coach described,

Conferring with another coach . . . really helps you clarify. Planning, for example, [a coach colleague] would ask me questions that I hadn’t thought of, and it helped me get my thoughts straight before I went and talked to the teacher . . . So when I [met with the teacher] I had gone through all those questions that the teacher might have been thinking, but might not have said something.

The CFC coaches described several other aspects of training as supporting their work with teachers: (a) the opportunity to apply theory to analyze instruction; (b) being provided with protocols for developing lessons, conferring with teachers, and approaching resistant teachers; and (c) studying models of instruction provided by the IFL trainers.

Comparison coach professional development

In the comparison schools, coaches reported that although the district had provided professional development in the previous year, the district temporarily halted the professional development support during the 2nd year of the CFC trial—the focus of our current study—to be resumed during the 3rd year. Notably, however, the district had not provided professional development to their coaches prior to the CFC intervention. This lack of support is commensurate with other research on literacy coaching showing that ongoing professional development targeted to coaches rarely occurs in districts (Blarney et al., 2009; Marsh et al., 2008; Neufeld & Roper, 2003). The training received by coaches in the 1st year of the intervention differed from the CFC professional development in that it occurred less frequently, a half day a month, and focused on multiple domains of the literacy curricula, for example, fluency in second grade and writing in fourth grade. At least two sessions also focused on the state accountability test or, as one coach described, “helping the teachers look at the [accountability] test and looking at the questions and seeing what objectives the question was referring to, and training teachers on materials they can implement on their classroom.” All of the comparison coaches were positive about the professional development they received from the district, especially the opportunity to meet regularly with other coaches.

Principal participation in coach professional development

Another key difference in the training and support received by the CFC coaches and the coaching in the comparison condition concerns principal involvement. In both conditions, district leaders participated in the literacy-coach training—as providers of the coach professional development in the comparison condition and as participants in the CFC condition. Only in the CFC condition, however, were principals included in the coach professional development activities. Principals attended the CFC training one day a month to foster their support of their CFC coaches’ work with teachers by, for example, occasionally observing their coach model a lesson in a teacher’s classroom and attending some grade-level team meetings led by the coach. From what we could surmise from the coach interviews, comparison schools did not expect principals to participate in coaching activities.

In line with the CFC program developers’ assumption that principal support would be critical to the implementation of CFC in schools, all of the CFC coaches reported that principal support played an important role in their ability to engage teachers in coaching activities. As one CFC coach explained,

I really think that it boils down to the support of the principal. So it’s whatever the principal is supporting. I think it sends a clear message if the principal says this is important. How are we to make time to do it than I think it’s more apt to get done.

Notably, however, the CFC coaches were mixed in their perceptions of how successful the CFC training was for ensuring principal support (see Matsumura, Sartoris, Bickel, & Garnier, 2009). Only half of the CFC coaches reported that the professional development and related assignments successfully ensured principal support even though all of the coaches reported that their principals attended at least most of the sessions and completed the required assignments.

Coaches’ Work in Schools

Job responsibilities

Coach interviews also revealed differences in the scope of coaches’ job responsibilities across conditions. The IFL job description for the CFC coaches clearly specified that coaches were intended to spend their time that was funded by the district working with fourth- and fifth-grade teachers to improve their teaching (e.g., meeting with teachers in grade-level teams and individually, planning lessons, modeling instructional strategies for teachers, and observing teachers in their classroom). Similarly, the comparison coaches included working with teachers to improve their practice as part of their job responsibilities. Aligned with other research on literacy coaching, however, the comparison coaches described additional job responsibilities that were not emphasized in the CFC model (Blarney et al., 2009; Duessen et al., 2007). These additional responsibilities included tutoring students and performing a range of administrative tasks such as coordinating testing and analyzing test data. As one comparison coach explained when asked to describe her job responsibilities at her school,

My role requires flexibility. At the beginning of the year it was mainly working with our new teachers and modeling a lot for them and coplanning and coteaching with them, but now, since about December or so I haven’t been able to do as much of that because now I am working with [student] groups all day. Reading and writing groups . . . So now my role has kind of changed to more working with students.

Focus of coaches’ work

Coaches described another key difference in the focus of their work with teachers. First, the CFC program concentrated coaches’ time on two grades (fourth and fifth) and all teachers in those grades. Working with all teachers in a grade—and not prioritizing new teachers—was intended to promote a culture of professional reflection and learning in schools that was inclusive of all teachers regardless of experience and level of skill. In other words, the CFC program aims to promote the view that coaching is desirable for all teachers, not just inexperienced and/or struggling teachers. That said, however, all CFC coaches reported that new teachers were especially receptive to working with them in contrast to experienced teachers who were mixed in their willingness to be coached. In contrast, the coaches in the comparison schools were expected to work with teachers across all grades, and specifically prioritized their work with new teachers to help familiarize them with the district’s curricula and help them establish their classroom routines.

Second, the instructional, or subject matter, focus of coaches’ work with teachers further distinguished CFC from standard practice for literacy coaching. The CFC model concentrated coaches’ efforts on reading comprehension, specifically teachers’ skills at leading text discussions. The instructional focus of the CFC intervention was determined in advance of placing coaches in schools and in collaboration with the school district that identified declining reading scores for low-income students in the upper elementary grades as a significant problem in their schools and an important district-wide reform priority. The comparison coaches, however, focused on improving instruction in all areas of the literacy curricula (e.g., writing, decoding, fluency), and in other content areas besides literacy. The focus of their work was reportedly determined by the perceived needs of individual teachers, as well as by a school’s need to meet specific accountability goals in the grades and subjects assessed on the state achievement test. As one comparison coach explained,

We work on the subjects that are tested on [the state test]; those are the ones that are important. So reading comprehension is important, math is important, fifth-grade science is important, and writing in fourth grade is important.

Discussion

The difficulty of breaking from traditional patterns of classroom discourse and enacting high-quality interactive discussions is a constant barrier to instructional reform. Our study supports the use of well-designed, comprehensive literacy-coaching programs as a strategy for increasing interactive and cognitively demanding discussions in classrooms. Although comparison schools in our study also provided coaching programs, the quality of their classroom text discussions remained unchanged at the end of the 2 years and was lower than the quality seen in CFC schools.

How did CFC achieve its effect on classroom discussion quality? Several key factors distinguished CFC from standard practice for literacy coaching in the district and likely contributed to CFC’s success: teacher participation, concentration of coaching resources, coach training focused on improving pedagogical expertise and coaching skills, engagement of school leadership, a clear definition of the coaching job, and QtA. The CFC program encouraged teachers to participate more frequently in coaching over the course of the year, and the coaching they received focused more strongly on pedagogy (e.g., lesson planning, enacting lessons, differentiating instruction) relative to teachers in the comparison schools. The CFC program also concentrated coaching resources on teaching reading comprehension lessons based in the district’s curricula. This likely created a more intensive, deeper learning experience for teachers than participating in coaching that touched lightly on more areas of the curricula. Other research has linked instructional improvement to professional development activities that are sustained, connected to one another over time, and aligned with the curricula to be taught (Cohen & Hill, 2001; Guskey & Yoon, 2009).

Coaches were better trained and supported in the CFC schools relative to coaches in the comparison schools. The professional development the CFC coaches received was highly practice based and collaborative, thus providing a foundation for effectively delivering higher quality coaching to teachers (Ball & Cohen, 1999; McLaughlin & Talbert, 2006). Based on research showing that school leaders are important for creating the conditions in schools that support instructional improvement (Quint, Akey, Rappaport, & Willner, 2007), it is possible that including principals and district administrators in the CFC coach professional development also may have contributed to teachers’ greater coaching participation and improved practice. As described earlier, however, coaches were mixed in their reports of principal support and involvement in their work.

The CFC program clearly defined the coaching job, focused exclusively on improving teachers’ instruction and concentrating coaching resources on specific grades to achieve this end. Clear boundaries on the coaches’ responsibilities likely increased the amount of available time for coaches to work with teachers. Other research shows that being tasked with administrative tasks and other noncoaching-related work, as was the case in the comparison schools, is a barrier to coaches working with teachers to improve their instruction (Blarney et al., 2009).

QtA implemented with coaching support also was critical to the program’s success. Other research has demonstrated that QtA produces positive effects on teachers’ text discussion quality (McKeown & Beck, 2004; McKeown et al., 2009). This research was based on small samples of teachers (e.g., two to six) who received extensive coaching support from the program’s developers to implement QtA techniques. This support included modeling lessons for teachers, observing and providing feedback to teachers as they implemented QtA, and meeting with teachers to discuss specific problems of implementation (described in McKeown & Beck, 2004). As with other discussion-based approaches to instruction (e.g., instructional conversations), QtA is challenging to implement. The findings from our study indicate that CFC, a well-designed coaching program, can be effective for supporting teachers to enact QtA in relatively large numbers of classrooms.

Notably, CFC achieved its effects despite lower-than-intended levels of teacher participation. Aligned with other research (e.g., Marsh et al., 2008), CFC coaches reported that teachers were too busy to meet with them as frequently as desired. Teachers were not required to participate in coaching as a condition of their school receiving a CFC-trained coach, nor were they released from other responsibilities or compensated for participating in coaching. Coaches believed that many teachers were overwhelmed by the plethora of reforms and instructional activities in which they were expected to engage, and some teachers, especially experienced teachers, simply would not work with coaches despite their best efforts. Added to these barriers to implementation is the current policy climate stressing test-based accountability. Nearly all of the CFC coaches reported that pressure to meet accountability targets constrained their work with teachers. Teachers had less time to participate in CFC coaching after the winter holidays when test preparation activities in most schools began, and some teachers and principals did not see the program, with its emphasis on discussion quality, as central to improving students’ scores on the state accountability test. As shown in other research, many factors impeded the implementation of the CFC program in schools (Duessen et al., 2007; Marsh et al., 2008). We consider this issue further when we discuss the implications of our results for designing coaching programs and implementing coaching programs in large numbers of schools.

Limitations

Our study is limited by the following risk factors: attrition and response bias, less-than-intended program implementation, and instrumentation. We discuss these limitations in the following sections.

Attrition and response bias

Nearly half of the teachers left their school by the end of the 1st year of the trial. Schools serving large numbers of low-income and minority students typically experience high rates of teacher mobility relative to schools that serve more privileged students (Hanushek et al., 2004). Research conducted in the Chicago public schools, for example, describes schools serving low-income and primarily African American and Latino students as typically losing at least a quarter of their teachers each year (Allensworth et al., 2009). However, in our trial, teacher attrition rates did not differ across conditions and were unrelated to demographic characteristics, teaching practices, quality of text discussions, and student achievement. Furthermore, when we added to our analyses baseline teacher measures that are relevant for quality of classroom text discussions (e.g., years of experience teaching reading), none of these measures were associated with our outcomes.

We acknowledge, however, that teachers who left their school may have differed in other systematic ways that were not captured by our measures (e.g., in their commitment to working with low-income students, cooperativeness, motivation to participate in professional development). In addition, 21% of the teachers remaining in the trial failed to complete surveys. We identified no differences between responders and nonresponders on their treatment condition, quality of text discussions, or students’ reading achievement, but again acknowledge that the remaining teachers who completed surveys may have differed in other unmeasured ways.

Implementation of the CFC program

In field trials, differences in interventions as planned and interventions as implemented are not uncommon (McMillan, 2007). We believe that the program effects we observed are consistent with the theory underlying the CFC program and the operational definitions, but we do not know what differences CFC could have achieved if implemented at the intended levels.

Measures of implementation

Finally, collecting additional and more precise measures of coaching activities and content presented in the activities would provide a more detailed and clear picture of the program’s implementation. Relying solely on teachers’ end of year survey reports and coach interviews limited our results. The information obtained from annual data collections can contain measurement error given the cognitive demand of retrospectively describing practice over an extended time period (Sudman, Bradburn, & Schwarzs, 1996). Information from additional sources, such as logs (see Rowan, Camburn, & Correnti, 2004), would yield rich, precise information about the amount and type of coaching received by teachers and how coaches spent their time in schools. Lacking observational data on coaching quality also limited our study. Although the CFC coach professional development provided by the IFL monitored coaching quality during training, we cannot directly link coaching quality to teacher outcomes in our analyses.

Implications and Conclusion

The quality of interactive classroom discussions is a key factor in improving students’ higher level reading comprehension skills but remains difficult to change. Districts and state efforts to implement literacy-coaching policies have shown little effect on the quality of teaching and students’ reading comprehension skills. Our findings provide evidence that a well-designed literacy-coaching program focused on implementing a discussion-based approach to comprehension instruction can induce positive changes in discussion quality. In the following sections, we discuss the potential implications of our findings for helping districts and states design coaching program and allocate coaching resources more effectively.

Provide Ongoing, Practice-Based Professional Development for Coaches and School Leaders

Our study supports the importance of ongoing, practice-based professional learning opportunities for coaches and school leaders that are focused on implementing research-based instructional models in the context of the curricula to be taught. Such professional development could ensure the pedagogical expertise coaches need to model new forms of practice and provide support for teachers to implement these practices in their classroom. Including school leaders in the coaching program could help establish a clear understanding of the CFC coaches’ role and continued support for coaches’ work.

Focus Coaching Resources

We suggest that concentrating coaching resources by narrowing coach job responsibilities, targeting specific populations of teachers to participate in coaching, and focusing on a particular instructional model and goal will create deeper learning experiences for teachers. This approach may be a more effective use of coaching resources than requiring coaches to perform multiple tasks (e.g., administrative work, tutoring students, support all teachers, and work across curricula domains and content areas). Even when presented with evidence of an effective program, accountability pressure and breadth of teacher and student learning needs in low-performing schools make this a formidable strategy to implement.

Assess a School’s Readiness for a Coaching Intervention

Providing professional development to coaches and concentrating coaching resources may not be sufficient for ensuring full principal support and teacher participation. Additional strategies likely are needed to promote the implementation of coaching programs. One strategy to consider is assessing a school’s readiness for a coaching intervention in advance of committing resources toward hiring and training literacy coaches. Assessing the readiness of a community to adopt an intervention is a common practice in the social sciences (see Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). The literature on assessing a community’s readiness for change emphasizes the importance of gaining unambiguous buy-in from stakeholders, ensuring that organizational structures are in place to support the intervention, and clarifying how an intervention fits within a community’s priorities for change. Applied to implementing a literacy-coaching program, program leaders would obtain teachers’ commitment to participate in coaching and principals’ commitment to support the program in advance of placing literacy coaches in schools. This strategy would help ensure that coaches did not waste time attempting to win over resistant principals and teachers. Obtaining a principal’s commitment to the program also would ensure that organizational structures in schools, such as time in teachers’ schedules to meet with coaches and release from other professional duties, would be in place to support the intended implementation of the coaching program. Obtaining teacher and principal commitment also would ensure buy-in of the instructional goals promoted through a literacy-coaching intervention.

In conclusion, the findings from our study contribute to a growing body of evidence from experimental and quasiexperimental research showing a positive effect of well-designed literacy-coaching programs on teaching quality (e.g., Sailors & Price, 2010). Implementing a comprehensive coaching program such as CFC would require districts to allocate their literacy-coaching resources in new and unfamiliar ways. Districts would need to direct resources toward coach and principal professional development, allocate coaching resources to a limited number of grades, and pursue more focused goals for reform rather than multiple goals. These changes would present a challenge to districts. We assert that accepting this challenge to foster high-quality classroom discussions would be worth the effort.

Footnotes

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work described here was supported by a Teacher Quality Grant from the Institute for Educational Sciences (R305W060027), Washington, DC, USA. We are appreciative of the support provided by IES. All errors of fact, omission and/or interpretation are solely the responsibility of the authors.

Notes

References

Allensworth

Ponisciak

Mazzeo

(2009). The schools teachers leave: Teacher mobility in Chicago public schools. Chicago, IL: Consortium on Chicago School Research.

Allington

R. L.

(2006). President’s message: Reading specialists, reading teachers, reading coaches: A question of credentials. Reading Today, 23(4), 16-17.

Applebee

Langer

Nystrand

Gamoran

(2003). Discussion-based approaches to developing understanding: Classroom instruction and student performance in middle and high school English. American Education Research Journal, 40, 685-730.

August

Shanahan

(2006). Developing reading and writing in second-language learners: Lessons from the report of the National Literacy Panel on Language-Minority Children and Youth. New York, NY: Routledge.

Ball

D. L.

Cohen

D. K.

(1999). Developing practice, developing practitioners: Toward a practice-based theory of professional education. In Sykes

Darling-Hammond

(Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 3-32). San Francisco, CA: Jossey-Bass.

Bean

R. M.

Draper

J. A.

Hall

Vandermolen

Zigmond

(2010). Coaches and coaching in Reading First schools: A reality check. Elementary School Journal, 111(1), 87-114.

Beck

I. L.

McKeown

M. B.

(2006). Improving comprehension with Questioning the Author: A fresh and expanded view of a powerful approach. New York, NY: Scholastic.

Blarney

K. L.

Meyer

C. K.

Walpole

(2009). Middle and high school literacy coaches: A national survey. Journal of Adolescent & Adult Literacy, 52(4), 310-323.

Bloom

H. S.

(Ed.). (2005). Learning more from social experiments: Evolving analytic approaches. New York, NY: Russell Sage.

10.

Boruch

R. F.

Foley

(2000). The honestly experimental society. In Bickman

(Ed.), Validity and social experiments: Donald Campbell’s legacy (pp. 193-239). Thousand Oaks, CA: SAGE.

11.

Cazden

(2001). Classroom discourse: The language of teaching and learning. Portsmouth, NH: Heinemann.

12.

Cohen

D. K.

Hill

H. C.

(2001). Learning policy: When state education reform works. New Haven, CT: Yale University Press.

13.

Collins

Brown

J. S.

Holum

(1991). Cognitive apprenticeship: Making thinking visible. American Educator, 6(11), 38-46.

14.

Costa

A. L.

Garmston

R. J.

(1994). Cognitive coaching: A foundation for Renaissance Schools. Norwood, MA: Christopher-Cordon.

15.

Duessen

Coskie

Robinson

Autio

(2007). “Coach” can mean many things: Five categories of literacy coaches in Reading First (REL Northwest [REL 2007-No. 005]). Washington, DC: Institute of Education Sciences, U.S. Department of Education.

16.

Elmore

(2003). School reform from the inside out. Cambridge, MA: Harvard University Press.

17.

Feingold

(2009). Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychological Methods, 14, 43-53.

18.

Fixsen

D. L.

Naoom

S. F.

Blasé

K. A.

Friedman

R. M.

Wallace

(2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI #231).

19.

Garet

M. S.

Cronen

Eaton

Kurki

Ludwig

Jones

. . . Sztejnberg

(2008). The impact of two professional development interventions on early reading instruction and achievement (NCEE Report No. 2008-4031). Washington, DC: National Center for Educational Evaluation and Regional Assistance, Institute of Education Sciences.

20.

Goldenberg

(2010). Reading instruction for English language learners. In Kamil

Pearson

P. D.

Moje

Afflerbach

(Eds.), Handbook of reading research (Vol. IV, pp. 684-710). Newark, DE: International Reading Association.

21.

Guskey

T. R.

Yoon

K. S.

(2009, March). What works in professional development? Phi Delta Kappan, 90(7), 495-500.

22.

Hanushek

E. A.

Kain

J. F.

Rivkin

(2004). Why public schools lose teachers. Journal of Human Resources, 39(2), 326-354.

23.

Joyce

Showers

(1995). Student achievement through staff development: Fundamentals of school renewal (2nd ed.). London, England: Longman.

24.

Lipsey

Wilson

(1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181-1209.

25.

Marsh

J. A.

McCombs

J. S.

Lockwood

J. R.

Martorell

Gershwin

Naftel

. . . Crego

(2008). Supporting literacy across the sunshine state: A study of Florida middle school reading coaches. Santa Monica, CA: RAND.

26.

Matsumura

L. C.

Garnier

Slater

S. C.

Boston

M. B.

(2008). Measuring instructional interactions ‘at-scale’, Educational Assessment, 13(4), 267-300.

27.

Matsumura

L. C.

Sartoris

Bickel

Garnier

(2009). Leadership for literacy coaching: The principal’s role in launching a new coaching program. Educational Administration Quarterly, 45(5), 655-93.

28.

Matsumura

L. C.

Slater

S. C.

Junker

Peterson

Boston

Steele

Resnick

(2006). Measuring reading comprehension and mathematics instruction in urban middle schools: A pilot study of the Instructional Quality Assessment. (CSE Technical Report #681). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

29.

McKeown

M. G.

Beck

I. L.

(2004). Transforming knowledge into professional development: Six teachers implement a model of teaching for understanding text. Elementary School Journal, 104(5), 391-408.

30.

McKeown

M. G.

Beck

I. L.

Blake

R. G. K.

(2009). Rethinking reading comprehension instruction: A comparison of approaches for strategies and content approaches. Reading Research Quarterly, 44(3), 218-253.

31.

McLaughlin

M. W.

Talbert

J. E.

(2006). Building school-based teacher learning communities: Professional strategies to improve student achievement. New York, NY: Teachers College Press.

32.

McMillan

James H.

(2007). Randomized field trials and internal validity: Not so fast my friend. Practical Assessment Research & Evaluation, 12(15). Retrieved from http://pareonline.net/getvn.asp?v=12&n=15

33.

Michaels

O’Connor

Resnick

L. B.

(2007). Deliberative discourse idealized and realized: Accountable talk in the classroom and civic life. Studies in Philosophy and Education, 27(4), 283-297.

34.

Murphy

P. K.

Wilkenson

I. A. G.

Soter

A. O.

Hennessey

M. N.

Alexander

J. F.

(2009). Examining the effects of classroom discussion on students’ comprehension of text: A meta-analysis. Journal of Educational Psychology, 101, 740-764.

35.

Neufeld

Roper

(2003). Coaching: A strategy for developing instructional capacity—Promises and practicalities. Washington, DC: Aspen Institute.

36.

Neuman

S. B.

Wright

T. S.

(2010). Promoting language and literacy development for early childhood educators: A mixed-methods study of coursework and coaching. Elementary School Journal, 111(1), 63-86.

37.

Nystrand

(2006). Research on the role of classroom discourse as it affects reading comprehension. Research in the Teaching of English, 40, 392-412.

38.

Poglinco

Bach

Hovde

Rosenblum

Saunders

Supovitz

(2003). The heart of the matter: The coaching model in America’s Choice schools. Philadelphia, PA: Consortium for Policy Research in Education, University of Pennsylvania.

39.

Powell

D. R.

Diamond

K. E.

Burchinal

M. R.

Koehler

M. J.

(2010). Effects of an early literacy professional development intervention on Head Start teachers and children. Journal of Educational Psychology, 102(2), 299-312.

40.

Quint

J. C.

Akey

T. M.

Rappaport

Willner

C. J.

(2007). Instructional leadership, teaching quality, and student achievement: Suggestive evidence from three urban school districts. New York, NY: Montreal Diabetes Research Center. Retrieved from http://www.mdrc.org/publications/470/overview.html

41.

Raudenbush

S. W.

(1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173-185.

42.

Raudenbush

S. W.

(2005). Learning from attempts to improve schooling: The contribution of methodological diversity. Educational Researcher, 34, 25-32.

43.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: SAGE.

44.

Raudenbush

S. W.

Liu

(2001). Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychological Methods, 6, 387-401.

45.

Resnick

L. B.

Spillane

J. P.

(2006). From individual learning to organizational designs for learning. In Verschaffel

Dochy

Boekaerts

Vosniadou

(Eds.), Instructional psychology: Past, present and future trends: Sixteen essays in honour of Erik De Corte (Advances in Learning and Instruction series), Oxford: Pergamon.

46.

Rivera

Burley

Sass

(2004, February). Evaluation of school-based professional development (2002-03, Planning, Assessment and Research Division Publication No. 187.). Los Angeles Unified School District, Los Angeles, CA.

47.

Rowan

Camburn

Correnti

(2004). Using teacher logs to measure the enacted curriculum in large-scale surveys: Insights from the study of instructional improvement. Elementary School Journal, 105, 75-102.

48.

Ryan

P. M.

(2000). Esperanaza Rising. New York, NY: Scholastic.

49.

Sailors

Price

(2010). Professional development for cognitive reading strategy instruction. Elementary School Journal, 110, 301-323.

50.

Saunders

W. M.

Goldenberg

(1999). Effects of instructional conversations and literature logs on limited- and fluent-English proficient students’ story comprehension and thematic understanding. Elementary School Journal, 99, 277-301.

51.

Soter

A. O.

Wilkinson

I. A.

Murphy

P. K.

Rudge

Reninger

Edwards

(2008). What the discourse tells us: Talk and indicators of high-level comprehension. International Journal of Educational Research, 47, 372-391.

52.

Staub

F. C.

West

Bickel

D. D.

(2003). What is content-focused coaching? In West

Staub

F. C.

(Eds.), Content-focused coaching: Transforming mathematics lessons (pp. 1-17). Portsmouth, NH: Heinemann.

53.

Strauss

Corbin

(1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: SAGE Publications.

54.

Sudman

Bradburn

Schwarzs

(1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass.

55.

Tharp

R. G.

Gallimore

(1988). Rousing minds to life: Teaching, learning and schooling in social context. New York, NY: Cambridge University Press.

56.

What Works Clearinghouse. (2008). WWC procedures and version 2 standards handbook. Retrieved from http://ies.ed.gov/ncee/wwc/references/idocviewer/doc.aspx?docid=19&tocid=1

57.

White

E. B.

(1952). Charlotte’s Web. New York, NY: HarperCollins.