Teachers’ Support of Students’ Vocabulary Learning During Literacy Instruction in High Poverty Elementary Schools

Abstract

The purpose of this study was to examine third-grade teachers’ support for students’ vocabulary learning in high poverty schools characterized by underachievement in reading. We examined the prevalence and nature of discourse actions teachers used to support vocabulary learning in different literacy lessons (e.g., phonics); these actions varied in the cognitive demands placed on the students. Results showed that teachers rarely engaged students in cognitively challenging work on word meanings. Various lesson features and student and teacher characteristics were associated with teachers’ support for students’ vocabulary learning (e.g., teachers’ knowledge about reading). A major finding was that the extent of teachers’ support of their students’ vocabulary learning was significantly related to gains in reading comprehension across the year.

Keywords

vocabulary knowledge literacy observation of instruction early elementary

There is general consensus among researchers that elementary school students who begin formal schooling with limitations in their vocabulary knowledge are at risk for significant underachievement in reading comprehension and school learning (e.g., Biemiller, 2003; Graves, 2006; Nagy, 2005). Regardless of the reason for the limited vocabulary knowledge (e.g., learning English as a second language, family poverty), such students need extensive support to address gaps in their vocabulary knowledge if the goal is significant improvement in their reading comprehension. According to researchers (e.g., Graves, 2006; Nagy, 2005; Stahl, 2005), explicit instruction during planned vocabulary or text-reading lessons is not sufficient to offset early gaps in vocabulary learning and reading comprehension for students significantly underachieving in vocabulary and reading. Also critical is “as-needed” support for learning unfamiliar words in all types of literacy lessons.

Recent studies of upper elementary and middle school teachers’ vocabulary instruction (e.g., Scott, Jamieson-Noel, & Asselin, 2003; Watts, 1995) have documented the frequency of use of specific methods and features of instruction. However, because of the critical importance of addressing underachievement in vocabulary and reading comprehension in the early elementary years (e.g., Biemiller, 2003; U.S. Department of Education, 2002), we need similar studies of vocabulary instruction in earlier grades. One contribution these could make would be to advance our knowledge of the nature and extent to which teachers support their students’ vocabulary learning, particularly in high poverty schools and classrooms with a large proportion of students underachieving in early literacy. We also need to understand the characteristics of teachers’ support for vocabulary learning that are associated with their students’ gains in reading comprehension over the course of a year.

Our study focuses on these issues within the context of the Reading First (RF) initiative. Schools were selected to participate in this federal program because of high levels of poverty and underachievement in reading; the intent of the law was to improve early elementary teachers’ reading instruction and reading achievement by the end of third grade. In a context such as this one, we have an opportunity to examine the extent to which variation in teachers’ support for students’ vocabulary learning is associated with gains in reading comprehension.

Early Influences on Students’ Vocabulary Knowledge

By the time they start formal schooling, children vary enormously in the number of words they know. Both the amount and nature of talk they are exposed to affect their language and literacy development. There are marked individual differences in the rate of word learning among children, starting in early childhood, in large measure attributable to the nature and extent of their exposure to language (Hart & Risley, 1995; Huttenlocher, Haight, Bryk, Seltzer, & Lyons, 1991; Weizman & Snow, 2001). The nature of parents’ and teachers’ support is related to children’s comprehension of oral and written discourse (Dickinson, 2011; NICHD Early Child Care Research Network, 2002). Weizman and Snow (2001) found that the density of sophisticated words that mothers used in talking with their preschool children contributed to their vocabulary development and reading comprehension in kindergarten and second grade.

Studies have shown that the socioeconomic and educational status of the family is associated with vocabulary development (NICHD Early Child Care Research Network, 2002; Weizman & Snow, 2001). The gap between the vocabulary of more and less advantaged children does not go away when they start school. While Nagy and Herman (1987) estimated that children learn about 3,000 words a year, the figures vary considerably for children of different family backgrounds. Graves (2006) estimated that linguistically advantaged third graders know on average 10,000 words, whereas linguistically disadvantaged third graders know about 5,000 words. Further, White, Graves, and Slater (1990) found that Grade 1 through 4 students in urban, semi-rural, and suburban schools differed significantly in the extent of their vocabulary development, with the students in the suburban school outperforming their peers in the other two schools. This finding raises questions about the quality of teachers’ support for vocabulary learning in high poverty and/or urban schools and classrooms.

Language ability (including vocabulary) at ages 3 and 4 predicts later reading comprehension through high school (Dickinson, 2011). The significant impact that limited vocabulary has on reading comprehension development is a major reason researchers are so interested in learning what teachers can do to increase students’ vocabulary (e.g., Beck & McKeown, 2007; Carlo, August, Snow, Lively, & White, 2004; Lesaux, Kieffer, Faller, & Kelley, 2010). In theory, the close association of vocabulary knowledge and reading comprehension suggests that improvements in vocabulary should contribute to improvements in comprehension of texts (Anderson & Freebody, 1981; Nagy, 2005; Stanovich, 1986; Tannenbaum, Torgesen, & Wagner, 2006). As Biemiller (2003) stated: “There is evidence that vocabulary differences affect comprehension, and that when the rate of vocabulary acquisition is increased, concomitant gains in reading comprehension are also obtained” (p. 326).

In fact, Stahl and Fairbanks’ (1986) meta-analysis of the relation of vocabulary instruction and reading comprehension showed that vocabulary instruction had a significant effect on text comprehension. Explicit vocabulary instruction is particularly likely to affect comprehension of texts that includes the target words (e.g., Beck & McKeown, 2007; Coyne, McCoach, & Kapp, 2007). Vocabulary instruction contributes not only to students’ knowledge of specific words that appear in their school work but also to their fund of knowledge about the world and their word consciousness; these have the potential to affect comprehension of texts (Anderson & Freebody, 1981; Scott & Nagy, 2004).

However, explicit instruction specifically during vocabulary and text-reading lessons (e.g., explaining word meanings) is not likely to be sufficient to offset disadvantages of limited vocabulary on comprehension in the early school years. Of the 3,000 words students learn (on average) in a year, most are learned through incidental encounters, not through explicit instruction. Incidental word learning occurs through language activities and discourse in the classroom. Teachers need to be sensitive to students’ gaps in vocabulary and provide support as needed throughout the school day. Independent reading is certainly valuable (Cunningham, 2005) but by itself will not address significant gaps in vocabulary because students underachieving in reading tend to read simpler texts and have fewer opportunities to learn new words through reading (Stanovich, 1986). Further, basal reading programs target typically developing children (Crowe, Connor, & Pretscher, 2009), so that in classrooms with students underachieving in vocabulary and reading comprehension, teachers need to scaffold students’ learning of many unfamiliar words in the reading selections. Without such support, the students are likely to remain behind their peers in vocabulary knowledge and reading comprehension throughout their school years (Biemiller, 2003; Graves, 2006).

Teachers’ Instructional Support for Vocabulary Learning

Two recent observational studies of vocabulary instruction in the upper elementary or middle school years (Scott et al., 2003; Watts, 1995) provide examination and description of teachers’ vocabulary instruction. A comparison with researchers’ recommendations for effective vocabulary instruction suggests gaps between what is and what ought to be done. In both studies, analysis of results led the researchers to suggest the need for further study of the extent and quality of teachers’ actions in teaching vocabulary.

In both studies, teachers typically focused on words in texts the students were reading. Instruction generally involved work on definitions and the use of words in context with little use of other approaches. While research findings show that definition and context approaches are effective (Stahl & Fairbanks, 1986), Graves (2006) pointed out that these approaches do not necessarily engage students in active analysis of word meanings and uses. Thus, it is worrisome that in both studies teachers’ vocabulary instruction was described as quite superficial or shallow (e.g., looking up definitions of word in a dictionary). Deep or rich instruction (i.e., instruction that involves active processing and that challenges students’ thinking about words) is needed to support students’ vocabulary learning and contribute to improvements in reading comprehension.

With regard to the extent of teachers’ support teachers for vocabulary learning, Scott et al. (2003) and Watts (1995) observed little attention to vocabulary outside of vocabulary and reading lessons. This runs counter to experts’ views that teachers need to provide support for vocabulary learning both in planned lessons and in incidental encounters with unfamiliar words throughout the day (e.g., Graves, 2006; Stahl, 2005). Such support is particularly critical for students with limited vocabulary knowledge, as they tend to have difficulties inferring the meanings of words from context on their own (McKeown, 1985).

Thus, two major gaps between observed teachers’ instruction and researchers’ recommendations for effective instruction are (1) the extent to which teachers provide sufficient support for students’ vocabulary learning and (2) the extent to which their support is of a kind to facilitate substantial gains in vocabulary knowledge. If teachers’ support for vocabulary learning in third grade is similar to that found in Scott et al.’s (2003) and Watts’ (1995) classrooms, this would suggest a critical educational problem because of the likely effect on students’ reading comprehension.

The contributions of Scott et al.’s (2003) and Watts’ (1995) studies could be extended by undertaking analysis of teachers’ discourse actions during vocabulary instruction. This would involve collecting systematic data on the frequency and quality of teachers’ vocabulary instructional actions. Frequency can be determined by examining support that teachers provide both during vocabulary and comprehension lessons and during literacy lessons with other purposes (e.g., writing). Quality can be determined by examining the level of cognitive challenge the teacher presents to the students through his or her choice of instructional actions. Take, for example, the common focus on definitions. Telling students the meaning of a word does not require the students to do much, if any, thinking, whereas asking them to provide a definition themselves should get them to ponder the defining features of words and/or ways to express the meaning of that word. Thus, we set out to observe and code the discourse actions third-grade teachers used to support students’ vocabulary learning. We coded five different instructional actions that varied in the levels of engagement and depth of processing asked of the students; these five instructional actions were coded in lessons not only on vocabulary and comprehension but also on writing, fluency, phonics, and guided reading so that direct comparisons could be made of the extent and nature of support teachers provided in planned vocabulary and comprehension lessons and in incidental encounters with unfamiliar words in other contexts.

The study was designed to examine the level of teachers’ support in different lessons during the literacy block on 4 days across the school year. We expected to find marked variation in the nature and extent of support provided by teachers in lessons taught for different literacy purposes. This is because teachers must decide about the importance of spending time working on vocabulary. Thus, we would expect more density in the instructional actions directed toward vocabulary learning in lessons on vocabulary than in lessons on phonics. Stahl (2005) pointed out both the advantages of and constraints on as-needed work on vocabulary, which he referred to as “point of contact” instruction. During a lesson (e.g., a writing lesson) the teacher might notice that students are unfamiliar with a word and simply give students a synonym or definition (e.g., “crimson means red,” p. 101). This teacher’s instructional action might be brief so as not to interfere with the flow of the lesson. While giving a definition by itself is not likely to lead to a deep and lasting memory of that word, such instruction has the potential value of making it easier for students to understand the content they are working on.

An example of richer instruction is the routine Beck and McKeown (2001, 2007) use in their program, Text Talk. The recommendation is that teachers use three different instructional actions during a book-reading lesson. The teacher gives the students the meaning of a word in the text; he or she provides examples of ways that the word can be used; then, he or she asks students to use the word in a sentence of their own. Beck and McKeown (2007) have shown that students whose teachers engaged in this routine made significant gains in vocabulary, as compared to students whose teachers simply read the books aloud. In theory, the use of several different instruction actions focused on the meaning of a word should contribute to deeper processing than any one action used alone. Rich instruction might be preplanned, as in Beck and McKeown’s study, but it might also arise when teachers feel that students would benefit from active analysis of word meanings. In both cases (i.e., planned and incidental work on an unfamiliar word), teachers would most likely start by using shallower instructional actions (e.g., giving the students a definition) before engaging students in more cognitively challenging discussion (Watts, 1995).

Variation in Teachers’ Capability to Teach Vocabulary Well

Researchers have expressed various concerns about the status of vocabulary instruction in the elementary years. Some consider the amount of time teachers spend on vocabulary instruction inadequate, especially for students disadvantaged in vocabulary (Scott et al., 2003). Others have expressed concern about the cursory nature of instruction; for example, Scott et al. (2003) reported that most instruction entailed mentioning meanings of words or assigning words to be learned. These researchers also found variation in the amount and kind of vocabulary instruction provided by different teachers, leading us to wonder what factors might explain this variation.

Stahl and Nagy (2006) noted that there are many reasons that vocabulary instruction is particularly challenging, including the need for teachers to understand the nature of words and their meanings, to be knowledgeable about guidelines for selecting words for instruction, and to know different approaches for supporting students’ vocabulary learning. Snow, Griffin, and Burns (2005) argued that teachers’ knowledge about language and literacy is a critical factor in the quality of their literacy instruction, but that this knowledge needs to be linked to their understanding of students’ development of reading skill (and associated problems). In short, teachers’ knowledge about literacy might affect the quality of their vocabulary instruction. Some studies have reported significant effects of teachers’ knowledge about reading on teachers’ practices and students’ achievement (e.g., Carlisle, Kelcey, Rowan, & Phelps, 2011). Given the challenge of providing sufficient high quality support for students’ vocabulary learning, we wondered whether more knowledgeable teachers would be particularly likely to understand and use instructional practices that might lead to improvements in students’ vocabulary and reading comprehension.

Research Questions

To summarize, extensive vocabulary instruction is critical if the goal is to improve reading comprehension of students disadvantaged in vocabulary knowledge (e.g., Nagy, 2005). In theory, given the extent of underachievement in reading in RF schools, just teaching words prior to reading passages or just asking students to read on their own is not likely to be sufficient to bring about the kinds of improvement in vocabulary knowledge that would contribute to improved comprehension of texts (e.g., Graves, 2006). Because of the importance of high quality and extensive instruction in vocabulary for students underachieving in reading, we designed a study to document the nature of teachers’ support for vocabulary learning. Through coding instructional actions that varied in cognitive challenge during lessons for different literacy purposes, we were able to examine individual teachers’ commitment to supporting their students’ vocabulary learning across lessons on a given day and across 4 days in the school year.

We viewed analysis of the context effects on teachers’ support for vocabulary learning as critical to our understanding of their vocabulary instruction and so treated context as a multidimensional construct that includes features of the lesson (e.g., duration), the aggregated sociodemographic characteristics of students in the class (e.g., percentage qualifying for subsidized lunch), and characteristics of the teacher (e.g., educational attainment). We expected that teachers’ knowledge about reading instruction would be associated with their support for students’ vocabulary learning and that the nature and extent of teachers’ support for vocabulary learning would be associated with their students’ gains in reading comprehension. Our research questions were as follows:

Research Question 1: What was the relation of teachers’ support for vocabulary learning and the context in which this support was provided?

Research Question 2: Were there significant associations of the teachers’ support for vocabulary learning and gains in reading across the year?

Method

Participants

Participants were part of a larger study investigating second- and third-grade instruction in RF schools in Michigan. The eligibility for RF funding included a high level of poverty and chronic underachievement in reading. In 2007–2008, 41% of the third-grade students were below the 25th percentile on Iowa Tests of Basic Skills (ITBS) reading comprehension and 72% fell below the 50th percentile (based on national norms). We recruited 19 schools across six districts to participate in a study of classroom practices. For this study, participants were the 44 third-grade teachers in the larger study; of these, 91% were female, and 21% were from minority racial/ethnic backgrounds (that is, excluding White); 52% had a master’s degree. See Table 1 for teacher and classroom characteristics. On average, these teachers had over 13 years teaching experience (range, 1–39 years). Nine of the teachers were new to their RF school in the year of the study. We observed the literacy block of participating teachers four times across the 2007–2008 school year, spacing out the observations as evenly as possible.

Table 1

Descriptive Characteristics of Teachers and Classrooms

	M	SD
New to Reading First teacher (proportion)	0.18	0.38
Ethnic/racial minority teacher (proportion)	0.26	0.44
Teacher with master’s degree (proportion)	0.52	0.50
Teachers’ years of experience	15.38	11.43
Teachers’ reading knowledge	−0.01	0.78
Proportion eligible for free/reduced price lunch	0.71	0.28
Proportion of class with minority status	0.38	0.40
Proportion of class with limited English proficiency	0.18	0.26
Proportion of class labeled special education	0.13	0.10
Fall 2007 oral reading fluency (DIBELS)^a	72.48	14.26
Average ITBS reading comprehension in spring 2008^b	181.41	20.88
Average ITBS reading comprehension, spring 2007^b	169.15	17.90
Average ITBS reading comprehension, spring 2006^b	149.49	11.73
Average ITBS vocabulary, spring 2008^b	180.82	19.40
Average ITBS vocabulary, spring 2007^b	165.65	19.00
Average ITBS vocabulary, spring 2006^b	147.57	15.61

Dynamic Indicators of Basic Literacy (DIBELS) oral reading fluency is the number of words read correctly.

Iowa Tests of Basic Skills (ITBS) reading comprehension represents standard scores; for the standardization sample, the median for the end of third grade is 185, end of second grade is 168, and end of first grade is 150.

In the 44 teachers’ classrooms, the number of students ranged from 13 to 28 with an average of 22. Characteristics of the third-grade students came from the Single Record Student Database maintained by the state. In the average classroom, 36% of the students were identified as racial/ethnic minorities (this category included all students who were not White) and 73% were eligible for free or reduced price lunch (FRL); 19% of the students were of limited English proficiency, and 12% qualified for special education.

We encountered a small amount of missing data at both the teacher and student levels (background data only; about 2% at the teacher level and 12% at the student level). Our missing data analyses suggested that there was no relation between missing data and either achievement or instructional practice. Rather than remove cases with incomplete data, we employed the missing at random (Rubin, 1976) assumption and used multiple imputation to address missing data (e.g., Raghunathan, Lepkowski, Van Hoewyk, & Solenberger, 2001).

Data Sources

Classroom Observations

Automated Classroom Observation System for Reading (ACOS-R) is a system designed to code the presence of features of early elementary reading instruction using a PC tablet with a number of automatic coding and data management features. Coding was carried out in 5-minute intervals; during each interval, the observer recorded the following fields: purpose of the lesson, grouping arrangement (e.g., whole class), type of materials used during the lesson (e.g., anthology, trade book), literacy instructional actions (e.g., giving directions, modeling), word meaning discourse actions, and percentage of students actively engaged in the lesson. Observers entered notes about the lesson in a textbox; typically they described the activities in a given lesson (e.g., reading sentences). Observers designated a change in activity to indicate the start of a new lesson within a 5-minute interval.

Observers attended 2 days of training; they had a number of opportunities to compare and discuss their coding of videotaped and live lessons. Then, on an initial visit to a classroom and one other time during the year, two observers independently coded the instruction in the same classroom for the entire literacy block (typically lasting 90 to 120 minutes). We assessed interobserver reliability in two ways. The first was agreement on the number of lessons and the purpose of each lesson. Across six pairs of observers (1 of the 11 observers was a member of two observer pairs), overall agreement was 88%. Second, we compared the agreement of code assignment across all fields and all options within each field. Overall agreement was 87.2% with a range of 80% to 96% agreement.

The literacy block was divided into lessons identified by literacy purpose (e.g., phonics) and grouping (i.e., when guided reading lessons were taught to more than one group, each was considered a separate lesson). Lesson duration was calculated by summing over intervals and subintervals as needed; mean lesson duration was 13 minutes. In the third-grade observations, there were 1,068 total instructional lessons (excluding lessons coded as non-literacy or assessment). About 13% of the lessons focused on vocabulary, 27% on comprehension, and 30% on guided reading/small group; the remaining 30% focused on writing, fluency, or phonics. The average lesson duration was 12.7 minutes (SD = 11.7). On average there were 6.9 lessons in each literacy block.

In the Word Meaning field, observers coded any of the five discourse actions observed in a lesson, given the following options: The teacher: (1) defines word or word parts (e.g., “A hoe is a garden tool”), (2) states or reads a sentence/sentences to examine the meaning or use of a word in context (e.g., “Let’s read the sentence to see how the word hoe is used and what it might mean”), (3) asks students to explain a word’s meaning (e.g., “Can anyone tell me what a silo is?”), (4) asks students to use a word in a sentence (e.g., “Can you use the word silo in a sentence?”), (5) fosters discussion of word meaning (e.g., “The passage we read mentions hoe, rake, and shovel. Let’s talk about how these tools have different purposes”).

In some of the third-grade classrooms, we videotaped the literacy block at the same time that the live observation was carried out. Using videotaped lessons, we identified two lessons to illustrate the five discourse actions. Transcriptions of excerpts from the two lessons are given in the Appendix. Part A shows discourse actions during a vocabulary lesson, and Part B shows discourse actions during a comprehension lesson.

Information About Teachers

Teachers completed questionnaires that provided information about their personal and professional background (as described previously). They also completed an assessment of their knowledge about reading (Teachers’ Knowledge About Reading and Reading Practices, or TKRRP) administered in the spring; the 22 items in this measure are presented in other reports (Carlisle, Johnson, Phelps, & Rowan, 2008; Carlisle, Kelcey, Rowan, et al., 2011). The measure assesses knowledge about different areas of reading, including word reading, vocabulary, and comprehension. Performance on TKRRP has been found to account for third-grade teachers’ engagement in instructional actions used in teaching reading comprehension (Carlisle, Kelcey, Berebitsky, & Phelps, 2011). The scale has an item response theory (IRT) reliability of .76.

Students’ Reading Achievement

The Iowa Tests of Basic Skills reading comprehension standard score was used as a measure of reading comprehension achievement. This test requires students to select responses to questions that follow short passages. Test reliability for the reading comprehension subtest in Grade 3 was .91 (computed with Kuder-Richardson Formula 20; Hoover, Dunbar, & Frisbee, 2003). Three measures were used to control for prior reading achievement: the ITBS reading comprehension subtest from each of the previous 2 years (end of first and second grade) and the fall of third grade Oral Reading Fluency (ORF) score from Dynamic Indicators of Basic Literacy (DIBELS) (https://dibels.uoregon.edu). At the end of third grade, the average standard score for ITBS reading comprehension was 181.4 (SD = 20.8) and for vocabulary was 180.8 (SD = 19.4). These figures indicate that the third graders performed somewhat below the median for the standardization sample (185). The correlation of performance on the reading comprehension and vocabulary subtests was .75.

Data Analyses

Measurement Model

To describe teachers’ support for vocabulary learning and to draw inferences about its association with key teacher and classroom characteristics, we developed an explanatory multilevel item response model (De Boeck & Wilson, 2004; Wang & Liu, 2007). Our application of the model was organized around several salient features that connected our framework and observation system with our measurement model. First, codes reported by observers simply recorded the presence or absence of the targeted actions. As a result, simple approaches that treat the codes as continuous and linear measures were not appropriate for describing teachers’ use of the actions across lessons. Our approach addressed the categorical nature of our observations by relating teachers’ observed use of discourse actions to a continuous latent measure of teachers’ vocabulary support through a Rasch item response model.

Second, the reported use of discourse actions by teachers in each lesson was likely influenced by aspects of the observational environment beyond teachers’ use of discourse actions (Kennedy, 2010). As a result, a significant problem with teachers’ reported use of the actions was that they confound construct-irrelevant variation with teachers’ vocabulary support (i.e., observed scores were not independent of the characteristics of a specific observation). Left untreated, construct-irrelevant variance has the potential to unfairly describe teachers’ instruction and undermine the reliability and validity of classroom observations (Messick, 1989).

In our classroom observation system, variation in teachers’ uses of the measured discourse actions stemmed from five primary sources: differences among teachers in terms of their consistent vocabulary support, differences within a teacher across days, differences within a teacher and within a day across lessons, differences among observers, and differences among measured discourse actions. We target the teacher component, which represents the part of vocabulary support that is common to a teacher across all days and lessons—the stable or consistent component. Variation that is uniquely attributable to the teacher represents the construct-relevant variation or the object of measurement. In contrast, the remaining facets represent construct-irrelevant sources of variation and introduce measurement error. Variation attributable to observers arises from the differences among observers in terms of the severity or leniency with which they code features. Even with extensive training, observers may inadvertently adhere to more lenient or strict interpretations of what constitutes the presence of actions (Hill, Charlambous, & Kraft, 2012). Day-to-day variation within a teacher occurs when a teacher’s support for vocabulary growth differs across days (e.g., particularly high support across all lessons on one day). Lesson-to-lesson variation manifests when teachers’ support for vocabulary learning varies across lessons within a day (e.g., short-term instability that decreases support because of an unplanned and disruptive situation). Finally, variation in teachers’ uses of targeted actions may also originate from the underlying differences in their difficulty or cognitive demand (Smith, 2001).

For these reasons, our analysis extended conventional applications of item response theory by developing a cross-classified multilevel item response model. Through our analytic approach, we first indexed teachers’ support for vocabulary growth in each lesson using an item response model. We concurrently linked this measurement model with a cross-classified multilevel structure to decompose teachers’ lesson-specific support for vocabulary growth, θ_ldtr, into four components: teachers’ consistent vocabulary support (T), day-to-day variation within teachers (D), lesson-to-lesson variation within teachers and within a day (L), and rater variation (R).

Let the log-odds of teacher t employing action i in lesson l on day d rated by rater r be described as

\log [\frac{P (Y_{i l d t r} = 1)}{1 - P (Y_{i l d t r} = 1)}] = θ_{l d t r} - b_{i} .

Here Y_ildtr is 1 if teacher t employed action i in lesson l on day d rated by rater r and 0 otherwise; let θ_ldtr be the level of vocabulary support for lesson l on day d for teacher t rated by rater r; and let b_i represent the difficulty of action i. Under this representation the measured discourse actions are viewed as items, with b_i describing action i’s cognitive demand or difficulty. As a result, the hierarchy of action/item difficulties can provide evidence for or against our theoretical ordering (Smith, 2001). Using the estimated difficulty parameters, b_i, we can empirically examine the extent to which teachers’ use of actions align with the theoretical expectations concerning the cognitive demand actions require. If the frequencies with which actions are used are related to the cognitive demand they require, results should demonstrate that actions requiring higher cognitive demand are more difficult.

Next, we concurrently linked this measurement model to a cross-classified multilevel structure to decompose the vocabulary support in a lesson, θ_ldtr, into independent components representing each of the aforementioned sources of variation

θ_{l d t r} = T_{t} + D_{d t} + L_{l d t} + R_{r} .

Here, each term represents an independent and normally distributed random effect with a mean of 0 (except T) and with an estimated variance. We used T_t to denote teacher t’s consistent level of vocabulary support across all days and lessons, D_dt to represent the day-specific deflection for day d for teacher t, L_ldt to indicate the lesson-specific deflection for lesson l on day d for teacher t, and R_r to describe rater r’s specific leniency level.

By introducing random effects through a multilevel structure, we attempted to separate the component of the observed variation uniquely attributable to teachers’ consistent support for vocabulary growth from the construct-irrelevant variance introduced by classroom observations

That is, T describes the differences among teachers in terms of their consistent support for vocabulary growth whereas L, D, and R describe construct-irrelevant variance driven by the lessons, days, and raters. Additional variation potentially arises from the respective interactions among these facets. Take, for instance, the rater by teacher two-way interaction. The presence of such an interaction would suggest that a rater’s leniency differs depending on which teacher the rater is evaluating (e.g., a rater evaluates female teachers more severely than male teachers). The nature of classroom observations and the limitations of our partially crossed design largely confounded estimation of such interactions, and we therefore were forced to assume that their contribution is negligible.

Finally, because our framework underscores the importance of context, we further expanded our analyses to explore explanatory variables. Similar to including variables to explain variation at the cluster level in a multilevel model, sources of variation (i.e., T, D, L, and R) can be associated with a set of explanatory variables through a latent regression framework. For instance, consistent differences among teachers in their support for vocabulary learning might, in part, be driven by teachers’ perceptions of students’ needs. Similarly, variation attributable to lessons within a teacher might be due to changes in lesson purpose (e.g., Connor, Morrison, & Underwood, 2007). To explore how background and context was associated with the outlined facets (sources of variation), our approach concurrently links each facet in Equation 2 of our model to explanatory variables by expanding Equation 2 to

F^{(p)} = \sum_{c = 1}^{C} β_{p c} X_{p c} + ε^{(p)}

(De Boeck & Wilson, 2004). Here, we let F^(p) denote facet p in Equation 2 (i.e., p is T, D, L, or R), X_pc are the covariates associated with facet p with corresponding coefficients β_pc, and ε^(p) is the normally distributed residual. Maximum likelihood estimates of the parameters were obtained using the Laplace approximation to the log-likelihood (Bates & DebRoy, 2004). By jointly estimating Equations 1 through 3, we were able to use a latent regression approach so as to address the potential attenuation bias and measurement error problems associated with two-step approaches that analyze these equations separately (e.g., Briggs, 2008).

Our analyses considered multiple explanatory variables across the teacher, classroom, day, and lesson levels. At the teacher/classroom level, our model included the following: teacher’s knowledge of reading as determined by the TKRRP, teachers new to RF status, teacher’s racial/ethnic minority status (non-White), and the proportion of students in the class who were eligible for free/reduced price lunch and were of racial/ethnic minority; also included was the average prior achievement score on the ITBS reading comprehension subtest for students in the class. Associated with the lesson-level facet, we considered the length of the lesson, the materials used (e.g., trade books, anthology), and the purpose of the lesson (e.g., phonics, fluency). Our study did not include specific observer characteristics and limited day characteristics to the day of the week the observation took place.

Achievement Model

The second research question focused on whether teachers’ levels of vocabulary support accounted for significant variation in students’ reading comprehension when taking into account central student- and teacher-level variables. Because engagement in vocabulary instruction might be particularly beneficial for students underachieving in vocabulary and comprehension development, we examined differences in achievement gains by classrooms in which achievement on the prior year’s ITBS reading comprehension fell above or below the median achievement level.

To estimate the association of teachers’ support for students’ vocabulary learning and student achievement within prior achievement subgroups, we modeled student performance on ITBS reading comprehension using a hierarchical linear model (Raudenbush & Bryk, 2002). At Level 1, we included three student covariates: the spring 2007 ITBS reading comprehension score (end of second grade), the spring 2006 ITBS reading comprehension score (end of first grade), and the fall 2007 ORF score (start of third grade). The general form of the model at Level 1 was

Y_{i j} = π_{0 j} + \sum_{p = 1}^{n = 3} π_{p} X_{p i j} + ε_{i j},

where Y_ij is the 2008 ITBS reading comprehension score for student i in classroom j, π_0j is the average student score adjusted for the student variables, X, and π_p are the variable’s corresponding coefficients while ϵ_ij has a normal distribution with mean 0 and variance σ².

At Level 2, we modeled the adjusted average, π_0j, using teacher and classroom variables. We included variables describing whether the class had above average prior achievement, the teachers’ reading knowledge and its interaction with the aforementioned average prior achievement, whether the teacher was an ethnic/racial minority, and the proportion of the class with limited English proficiency. In addition, we included our estimates of teachers’ consistent support for vocabulary growth (T_t) derived from the aforementioned measurement model. Because our estimates of teachers’ vocabulary support levels obtained from the measurement model carry some uncertainty or imprecision, we adopted a plausible value approach and used five draws from the conditional posterior distribution of teachers’ support levels (Mislevy, Johnson, & Muraki, 1992). To reflect the uncertainty of the coefficients and their standard error, we re-estimated the regression model for each set of plausible values. Subsequent analyses combined the estimated coefficients and their standard errors from these analyses to draw inferences (Rubin, 1987).

Our analyses additionally examined other classroom and teacher descriptive variables such as teacher new to RF status, teacher degrees and credentials, and student descriptors such as eligibility for free/reduced price lunch, racial/ethnic minority status, and special education status. Our final model retained significant predictors as well as those that address our specific research questions. The form of the model at Level 2 was

π_{0 j} = β_{00} + \sum_{q = 1}^{Q} β_{q} V_{q j} + δ W_{j} + r_{0 j},

where β₀₀ is the average adjusted achievement, β_q is average association of covariate (or interaction) V_qj, and W_j is the estimated teacher vocabulary support level with coefficient δ. Finally, r_0j is the random effect of teacher j and has a normal distribution with mean 0 and variance τ.

Results

Properties of Word Meaning Discourse Actions

The amenability of the measurement of instruction to the assumptions of a multilevel Rasch measurement model rested on several key assumptions, and we explored the tenability of these assumptions. Our framework and application of the Rasch model assumed that discourse actions differed in the difficulty or cognitive demand they required. To examine this assumption, we first explored the frequency with which the five discourse actions were used within and across lessons taught for different purposes. One or more of the discourse actions appeared in 50.3% of the 1,068 observed lessons. As Table 2 shows, the presence of one or more actions was more common in vocabulary (83.1%) and comprehension (60.3%) than lessons for other purposes.

Table 2

Percentages of Six Types of Lessons in Which the Five Discourse Actions Were Observed

The Teacher:	All Lessons	Phonics	Fluency	Writing	Comprehension	Vocabulary	Small Group Reading Lessons
1. Defines a word or word part	26.2	26.4	7.0	15.0	31.0	46.5	24.5
2. Examines word in context	24.1	9.7	9.6	13.5	25.8	41.5	27.6
4. Asks students to use/read word in sentence	31.6	23.6	11.3	14.3	44.6	64.1	21.6
3. Asks students to give meaning of word	9.5	2.8	1.7	4.5	2.8	16.9	18.5
5. Fosters discussion	7.6	4.2	1.7	3.0	7.0	16.9	8.8
Any discourse action	50.3	38.9	18.3	32.3	60.3	83.1	48.3

Note. The five discourse actions are not in their original order; they have been reordered to distinguish the more common actions (1, 2, and 4) from the less common actions (3 and 5).

The frequency of each of the five vocabulary discourse actions varied widely. The most frequently used were No. 4, the teacher asks students to use the target word in a sentence (31.6% over all lessons); No. 1, the teacher defines the word (26.2%); and No. 2, the teacher uses a sentence to examine the meaning of a word in context (24.1%). Asking the students to define a word themselves (No. 3) and fostering discussing of a word (No. 5) were rare (9.5% and 7.6%, respectively). However, as Table 2 shows, use of the different discourse actions varied by lesson purpose. For example, teachers asked students to give the meaning of one or more words in 17% of the vocabulary lessons but in only about 3% of the phonics and comprehension lessons.

The result of our measurement model also suggested a similar ordering regarding the difficulties of actions (see Table 3). We had not expected the fourth action (teacher asks students to use target words in sentences) to be used so frequently in perfunctory activities that did not require analysis of word meanings and uses. Formulating a sentence using a target word is regarded as a cognitively challenging task (e.g., Beck & McKeown, 2007). However, when we examined the notes about lesson activities that observers wrote in the textboxes as they marked the fourth discourse action, we found that the activities did not demand much reflection or analysis. Some examples from notes in the textboxes are as follows: “review of vocabulary selection using workbook page,” “teacher calling on students to read from workbook pages,” “students asked to independently fill in workbook page,” “teacher and students review vocabulary words and definitions using an overhead transparency from the core program,” “worksheet has a story with blanks for them to fill in the correct word from the list on the overhead.” As a result, while we retained the numbering of the discourse actions (that is, in the order of what we expected to be least to most cognitively challenging), we conceptually grouped the fourth discourse action with the two other less challenging actions (Nos. 1 and 2) because the activities did not require students to engage in deep processing or analysis. Furthermore, we found support for this decision from analyses shown in Table 3, which suggested that the difficulty of the No. 4 discourse action was similar in difficulty to the other two shallower discourse actions.

Table 3

Difficulty of Teachers’ Discourse Actions

Items	Difficulty (SE)
Defines word	0.89 (0.06)
Examines word in context	0.99 (0.06)
Asks students to give meaning of word	1.85 (0.09)
Asks students to use word in sentence	0.67 (0.06)
Fosters discussion	2.03 (0.10)

We also found that when teachers employed rarer actions (e.g., fosters discussion), they generally had a higher probability of also employing the more common actions (e.g., giving students a definition of a word). Similarly, our analyses suggested that the model constraining the discrimination parameters to be equal across items fit well; it was not substantially improved upon by a two-parameter model. Our investigation suggested that the measured discourse actions might form a scale whereby the likelihood of using a more difficult or cognitively demanding action improves with a higher underlying aptitude to support vocabulary growth. An example of this process can be seen in the excerpt from the vocabulary lesson (Appendix, Part A), wherein the teacher asks students about the meaning of the word pace, but then initiates a discussion, apparently concerned about their grasp of the underlying concept.

Application of item response theory to the measurement of teaching also assumes teachers’ uses of the actions are independent conditional upon the latent variables (i.e., local independence assumption). Although it is unreasonable to assume teachers’ choices of actions are independent within a lesson, our inclusion of lesson, day, and rater random effects relaxes the local independence assumption so that actions need to be independent given teacher effects and lesson, day, and rater effects.

We investigated the tenability of this assumption by examining the residual local dependence of discourse actions (Yen, 1984). We first estimated the dispersion of the responses and found little evidence of residual dependency (e.g., Raudenbush, Johnson, & Sampson, 2003). Next, we examined the correlation of residuals across actions using the Q₃ statistic as a measure of the residual local discourse dependence (Yen, 1984). With only a teacher facet (i.e., L, D, and R set to zero Equation 1), the Q₃ index indicated substantial positive local dependence, as hypothesized. In other words, by ignoring the fact that, for example, instruction is situated within lessons, there was substantial residual dependence among teachers’ use of discourse actions (see Appendix Part A, for example). However, upon adding a latent variable for lessons, and to a much lesser extent for observers and days, the Q₃ index indicated virtually no remaining dependency among discourse actions. As a result, our analyses indicated that, conditional in the facets we included, there was no evidence of residual local action dependency.

Taken as a whole, the consistency of our theory with the empirical measurement results suggested that the actions give rise to a cognitive engagement scale such that infrequent actions are more cognitively challenging.

Factors That Influence Teachers’ Support for Vocabulary Learning

In addressing the first research question, we first assessed the variance attributable to each of the facets (Table 4). Our estimates suggested that variation in vocabulary instruction largely stemmed from the differences among lessons within teachers and the consistent differences among teacher in the amount of support they provide. The decomposition of observed variation also suggested that a moderate portion of the variance in vocabulary instruction stemmed from the day the teacher was observed as well as the differences among raters.

Table 4

Measurement Model Variance Components

Component	Variance
Lessons (L)	2.15
Observation days (D)	0.09
Teachers (T)	0.35
Observers (R)	0.11

With regard to the first research question, the results of the multilevel explanatory measurement model offered interesting insights about variables associated with supportive vocabulary instruction (Table 5). Before describing the results, we note the small size and exploratory nature of our explanatory model. Notwithstanding these limitations, there were three significant and interesting findings at the teacher level. One was that teachers who were new to their Reading First school provided less consistent support for their students’ vocabulary learning. The second was that teachers’ performance on the knowledge measure (TKRRP) was positively related to their support for vocabulary learning—that is, more knowledgeable teachers consistently provided more support for students’ vocabulary learning than less knowledgeable teachers. However, because so much of the support that all students received was not particularly cognitively demanding, we wondered if there were differences in the kind of discourse actions (i.e., cognitively challenging or not) initiated by more and less knowledgeable teachers. A follow-up analysis showed that when compared to teachers below the median on TKRRP, teachers with knowledge scores above the median used 5% more shallow actions but 33% more cognitively challenging actions.

Table 5

Relationships Between Variables and Levels of Vocabulary Support

Fixed Effects	Coefficient (SE)
Observation day characteristics
Monday	0.12 (0.26)
Tuesday	0.17 (0.26)
Wednesday	−0.08 (0.26)
Thursday	−0.15 (0.25)
Lesson characteristics
Lesson length	0.06* (0.00)
Fluency lessons	−0.93* (0.33)
Writing lessons	−0.27 (0.29)
Comprehension lessons	0.38 (0.26)
Vocabulary lessons	1.91* (0.26)
Small group lessons	0.23* (0.26)
Used anthology in lesson	0.31* (0.15)
Used trade/leveled/big book in lesson	0.31* (0.14)
Teacher characteristics
Teacher’s reading knowledge	0.35* (0.11)
Teacher new to Reading First	−0.52* (0.26)
Ethnic/racial minority teacher	0.63* (0.23)
Classroom characteristics
Proportion eligible for free/reduced price lunch	−0.98^ (0.58)
Class average Iowa Tests of Basic Skills (ITBS) reading comprehension in spring 2007	−0.04* (0.02)
Proportion of class with ethnic/racial minority status	−0.90* (0.39)

^p < .10. *p < .05.

The third important finding was that racial/ethnic minority teachers consistently engaged in significantly more support for students’ vocabulary learning than their White colleagues. We carried out a follow-up analysis to determine how extensive the difference was in terms of simple frequencies. We found that racial/ethnic minority teachers used 14.5% more discourse actions to support students’ vocabulary learning across all lessons than White teachers.

We also examined classroom and lesson characteristics as a way to understand variation in instructional practice. With regard to students in the classroom, significantly predictive of teachers’ consistent vocabulary support were students’ prior reading achievement, the proportion eligible for free/reduced price lunch, and the proportion of minority students. We also found significant associations between vocabulary instruction and a number of lesson characteristics (e.g., purpose, materials). More vocabulary support was observed on Mondays and Tuesdays than on other days, but these differences were not significantly different than that on Fridays. In contrast, the use of the anthology and of trade books was associated with the amount of teachers’ support for their students’ vocabulary learning; there was more talk about words in lessons in which texts were used than those where they were not.

Relation of Teachers’ Support for Vocabulary Learning and Students’ Reading Comprehension

Our second research question focused on the extent to which our measure of teachers’ support for students’ vocabulary learning was associated with students’ improvement in reading comprehension. The 44 classrooms were ordered, based on the students’ average prior achievement score on the 2007 ITBS reading comprehension subtest and then separated into two groups—those above and those below the median classroom. Table 6 provides characteristics for the two groups of classrooms. On average, the below-median classrooms had higher percentages of free/reduced price lunch students, minority students, and special education students. Teachers in the below-median classrooms tended to have less experience teaching and were more likely to be new to RF and to be a racial/ethnic minority. These teachers had a slightly lower propensity for using the vocabulary discourse actions.

Table 6

Descriptive Characteristics by Above/Below Median Subgroup (Proportions/Means)

	Below Median	Above Median
New to Reading First teacher	0.28	0.08
Ethnic/racial minority teacher	0.34	0.17
Teacher with master’s degree	0.26	0.78
Teachers’ years of experience	11.24	19.52
Teachers’ reading knowledge	0.16	−0.17
Proportion eligible for free/reduced price lunch	0.81	0.60
Proportion of class with ethnic/racial minority status	0.56	0.19
Proportion of class with limited English proficiency	0.18	0.18
Proportion of class labeled special education	0.15	0.10
Fall 2007 oral reading fluency (Dynamic Indicators of Basic Literacy)	63.23	81.73
Average Iowa Tests of Basic Skills (ITBS) reading comprehension, spring 2008	176.43	186.38
Average ITBS reading comprehension, spring 2007	164.56	173.73
Average ITBS reading comprehension, spring 2006	147.55	151.43
Support for vocabulary learning	−0.08	0.09

Note. Below/above median refers to those classrooms that had below/above the median 2007 ITBS comprehension scores.

Partitioning the variance in achievement attributable to differences among students versus differences among classrooms, we found that approximately 17% of the variation was attributable to classrooms (Table 7). After adjusting for baseline differences through the three lagged measures of prior achievement, our results suggested that teachers’ support for students’ vocabulary learning, as measured by their discourse actions, was significantly associated with gains in reading comprehension, as Table 8 shows. Specifically, a 1 SD increase in support for vocabulary learning is associated with 0.11 SD unit increase in reading comprehension. Further, this effect seemed to be similar for both high and low performing classes. Results also showed that the direct effect of teachers’ knowledge about reading (TKRRP) was marginally significant in above-median classrooms but not in below-median classrooms.

Table 7

Variance Components for Achievement Model

	Unconditional	Conditional
Residual	.83	.42
Classroom	.17	.02

Table 8

Reading Achievement Model

	Coefficient (SE)
Student characteristics
Intercept	−6.11* (0.34)
Iowa Tests of Basic Skills (ITBS) reading comprehension in spring 2007	0.02* (0.00)
ITBS reading comprehension in spring 2006	0.01* (0.00)
Fall 2007 oral reading fluency (Dynamic Indicators of Basic Literacy)	0.01* (0.00)
Teacher/classroom characteristics
Teachers’ reading knowledge	−0.13 (0.08)
Ethnic/racial minority teacher	−0.18* (0.09)
Support for vocabulary learning	0.11* (0.05)
Support for vocabulary learning interaction with high performing class	0.04 (0.08)
Teachers’ reading knowledge interaction with high performing class	0.14^ (0.08)
Proportion of class with limited English proficiency	−0.32* (0.12)
Above-median classroom	0.05 (0.07)

Note. High performance class refers to those classrooms that had above the median 2007 ITBS comprehension scores.

^p < .10. *p < .05.

Discussion

The purpose of this study was to examine the extent to which third-grade teachers in high poverty schools provided the extent and kind of support for students’ vocabulary learning that experts believe is necessary to contribute to meaningful gains in reading comprehension. We examined the teachers’ support for vocabulary learning during the literacy block in their classroom on four occasions across the year. Observers coded teachers’ use of five discourse actions that placed different cognitive demands on the students. Our analyses indicated that teachers’ use of these discourse actions formed a latent construct, showing variation in their support for students’ vocabulary learning in lessons taught for different literacy purposes. Further, the results showed ways in which the context of the lesson was related to the amount and kind of support teachers provided. While there was less support for vocabulary learning than one might hope to observe, this support still contributed to students’ gains in reading comprehension—a finding with possibly important implications for educators. In what follows, we discuss these results, considering contextual factors that influenced the extent and level of support for students’ vocabulary learning.

When and How Teachers Support Vocabulary Learning

We were particularly interested in the extent to which teachers supported vocabulary learning not only in lessons on vocabulary and text reading but also in lessons focused on other areas of literacy. To start with, we found that one or more of the five discourse actions was observed in 50.3% of all literacy lessons. This is similar to Scott and her colleagues’ (2003) report that vocabulary instruction was observed in 52% of the upper elementary literacy lessons.

Our observations showed that the extent and kind of support for vocabulary learning varied with the purpose of lessons. Using phonics lessons as a point of reference, we found that teachers were significantly more likely to use vocabulary discourse actions during lessons on vocabulary and significantly less likely to use them in writing and fluency. The percentage of lessons in which any of the five discourse actions was observed was lowest in fluency lessons (18.3%) and highest in vocabulary lessons (83.1%). Similarly, Scott et al. (2003) observed much less vocabulary instruction during spelling and writing than during reading of novels.

Further, the particular discourse actions teachers used varied by lesson purpose. For example, consider teachers’ focus on definitions—common in our observations and those of Watts (1995) and Scott et al. (2003). In our study, teachers asked students to give the meaning of a word in 17% of the vocabulary lessons but in only 3% of the reading comprehension lessons.

The relative frequency of use of shallower discourse actions is noteworthy—specifically, defining words for students (in 26.2% of all lessons), examining words in context (in 24.1% of all lessons), or using words in a sentence (in 31.6% of all lessons). Particularly in phonics, writing, and fluency lessons, these might reflect teachers’ use of “point of contact” instruction to provide a small amount of information about a word’s meaning in the middle of a lesson (Stahl, 2005). We note that teachers were far more likely to simply define a word for the students than to ask students to define the words themselves (9.5% of all lessons).

Given that helping students with unfamiliar vocabulary might lead to digressions from planned lessons, we would expect there to be more use of the cognitively challenging activities when the purpose of the lesson focused on meanings of words and texts. This is what we found. Teachers initiated discussion of word meanings in 16.9% of the vocabulary lessons but only 4.2% of the phonics lessons and 3.0% of the writing lessons. However, the fact that we observed discussion in only 16.9% of the vocabulary lessons is itself worrisome. We are not alone in expressing this concern. Watts (1995) found that after teachers raised a question about students’ familiarity with a given word, only in a few instances did they then facilitate discussion of the word’s meaning. Because deep processing is critical if instruction is to have an impact on students’ vocabulary knowledge and reading comprehension (e.g., Biemiller, 2003; Graves, 2006), the scarcity of teachers’ efforts to engage students in cognitively challenging activities (e.g., discussing different definitions of a word) is a concern in RF classrooms, where large percentages of the students are underachieving in literacy.

We also found that there were very few instances in our observation data of teachers facilitating discussion of a word without first using one or more of the other discourse actions. The results suggest that it is likely that teachers initiated discussion after first probing students’ familiarity with a word, and in doing so realizing that students needed further clarification of its meaning. Such a pattern is shown in the Appendix, Part A, where the teacher is persistent in probing students’ understanding of the word pace. She started by asking students about the meaning of pace; she acknowledged one student’s use of the word as related to fluency of reading and then initiated discussion by asking the students if pace can be slow. In the exchange that ensued, students came up with examples of pace as referring to speed, whether fast or slow.

In vocabulary and reading comprehension lessons, the infrequent use of the more challenging discourse actions is of particular concern. It is certainly not easy to know which words to teach or how to engage students in learning specific words, especially since there is likely to be variability in knowledge of specific words in a classroom of students (Stahl & Nagy, 2006). However, the relatively small percentage of lessons in comprehension that involved asking students to define words (2.8%) or engaging them in a discussion (7%) is striking. Vocabulary is critical in understanding texts.

In the example in the Appendix, Part A, the teacher was introducing unfamiliar words prior to reading a trade book. In our study of contextual factors, we found that use of both the reading anthology and trade books (or little books) was positively associated with support for vocabulary learning. Other researchers have found that vocabulary instruction commonly occurs when teachers are working with students on reading texts (e.g., Biemiller & Boote, 2006; Scott et al., 2003). However, we don’t know whether teachers’ discourse actions were spontaneous or part of a lesson plan. For example, some of the lessons might have come from the teacher’s guide. The excerpt from the comprehension lesson in the Appendix, Part B, suggests that use of a basal program does not mean that the teacher’s questions came from the teacher’s guide. This teacher used the anthology to introduce graphic displays, but when she asked the students what they thought active means in the phrase active volcano, the videotape of the lesson shows that she was not using the teacher’s guide. Her interaction with the students was spontaneous.

Context Effects: Teachers and Classrooms

The results of the measurement model indicated that certain teacher and classroom characteristics were related to the teachers’ support of their students’ vocabulary learning. One important finding was that teachers’ knowledge about reading (as shown on the TKRRP) was positively associated with their use of support for students’ learning of vocabulary. This conforms to our theoretical framework: Knowledge about practice should be related to teachers’ actual practice (see also Carlisle, Kelcey, Berebitsky, et al., 2011). More extensive knowledge of reading and reading instruction may have contributed to a greater understanding of the need to improve students’ vocabulary in order to support the development of their reading comprehension. Interestingly, the more knowledgeable teachers used somewhat more shallow actions than less knowledgeable teachers, but they used dramatically more cognitively challenging actions (33%). We also found that teachers who were in their first year in a RF school engaged in less support of their students’ vocabulary learning than their peers. This might suggest less familiarity with instructional practices teachers were encouraged to use in RF schools.

A second important finding was that racial/ethnic minority teachers provided significantly more support for vocabulary learning than their White colleagues. Follow-up analyses showed that across all lesson types, minority teachers used 14.5% more discourse actions—clearly, a meaningful difference in the extent of support for students’ vocabulary learning. Further research is needed to help us understand this intriguing finding.

Some classroom characteristics (that is, aggregated student characteristics) were related to the latent variable of support for students’ vocabulary learning. In classrooms with lower prior achievement, higher proportions of students who qualified for FRL, or with higher proportions of racial/ethnic minority students, teachers employed the discourse actions less often to support students’ vocabulary learning. In classrooms with a high proportion of students who would have reasons to be struggling readers, teachers face particular challenges in meeting their literacy needs (in particular, limitations in their vocabulary).

Support for Vocabulary Learning Contributes to Reading Comprehension

The results of this study contribute new insights into the extent to which teachers’ support for vocabulary learning is associated with improvements in students’ reading comprehension (e.g., Carlo et al., 2004). Previous studies have indicated that students with stronger vocabulary and comprehension benefit more from instruction than their lower achieving peers (e.g., Carlisle, Fleming, & Gudbrandsen, 2000; Stanovich, 1986). As a result, we examined the relation of support for vocabulary learning in classrooms that were above and below the group’s reading comprehension median at the start of the year. Initial examination of the above and below median reading comprehension classrooms showed some potentially important differences among the teachers and students. The below median classrooms were more likely to have a higher percentage of students who qualified for FRL, were members of a minority, or qualified for special education. The teachers in below median classrooms were less experienced, less likely to hold a master’s degree, more likely to be new to their RF school, and more likely to be a racial/ethnic minority. These student and teacher characteristics of the below median classrooms represent persistent challenges, ones that require extensive instructional support to overcome gaps in vocabulary knowledge and reading comprehension (e.g., Lesaux et al., 2010).

The results indicated that students’ gains in reading comprehension were associated with the latent variable from the measurement model, representing each teacher’s support for students’ vocabulary learning. The amount of support for students’ vocabulary learning was positively associated with gains in reading comprehension. Overall, a 1 standard deviation increase in teachers’ support for vocabulary learning had an impact similar to that of about an additional 6 to 7 weeks of schooling.

The multilevel analysis contributed further to our understanding of the influence of teachers’ support for vocabulary learning on students’ reading comprehension. Teachers provided more support for vocabulary learning in above median classes, and their knowledge about reading was more influential in the above median classrooms. We speculate that the measure of teachers’ knowledge captured something that worked differently for classes below and above the median on reading comprehension. For example, perhaps knowledge about effective instruction is difficult to enact in lower performing classrooms because of other constraints on the teachers (e.g., a need to use instructional time for other purposes).

Limitations, Future Research, and Final Thoughts

There are several limitations that need to be kept in mind in interpreting the results of this study. First, we have focused only on third-grade teachers and only on teachers in RF schools in Michigan. Second, while we have many hours of observation in the 44 teachers’ classrooms, this is a relatively small sample for the kinds of statistical analyses we carried out in an effort to understand how teachers, students, and lessons mediated the effects of teachers’ support for their students’ vocabulary learning. Third, we chose to code just five discourse actions; these differed in the extent to which they actively engaged students’ thinking about words without presenting observers with too long a list of actions to code reliably. While these actions were selected to provide valuable insights about variation in teachers’ vocabulary instruction, there may well be others that should be explored in future studies.

A number of other issues warrant further study. First, it is important to determine whether our findings hold up in high poverty schools that did not participate in the RF initiative. Second, further study is needed to determine whether teachers are aware of the need to provide not only explicit instruction during vocabulary and comprehension lessons but also support for students’ vocabulary learning during lessons with other literacy purposes. Third, studies are needed to examine more fully the influence of the literacy curriculum and reading materials on the nature and quality of their vocabulary discourse actions. Fourth, a study of student-teacher discourse would provide a better sense of students’ engagement in word learning. Analysis of the effects of teachers’ choice of words on students’ responses and engagement would be valuable (e.g., Ryder & Graves, 1994).

Concluding Comments

This study has broken new ground by examining the nature and extent of teachers’ support for students’ vocabulary learning in lessons focused on different literacy areas. The results suggest that the third-grade teachers’ support for students’ vocabulary learning tended to focus students’ attention to unfamiliar words but not engage them actively in understanding word meaning and uses. There wasn’t as much support as we might hope to see across the literacy block, and the kinds of support tended not to be cognitively engaging. Even so, we found a significant relation between the extensiveness and depth of vocabulary support and students’ gains in reading comprehension across the year. The results did suggest that teachers’ knowledge about reading and reading instruction was related to the quality of support they provided for their students’ vocabulary learning. This suggests the possibility that efforts to improve teachers’ knowledge of students’ acquisition of reading would lead to improved vocabulary and comprehension instruction for their students.

Footnotes

Appendix: Annotated Transcripts of Parts of Two Reading Lessons

Notes

Joanne F. Carlisle, PhD, is professor emerita in the School of Education, University of Michigan, 610 E. University Avenue, Ann Arbor, MI 48109-1259; e-mail: jfcarl@umich.edu . Her research interests focus on effective literacy instruction in language with a special interest in children for whom literacy acquisition presents unusual challenges and professional development for teachers of reading.

Ben Kelcey, PhD, is an assistant professor in the College of Education, Criminal Justice, and Human Services at the University of Cincinnati. His research interests include the development and application of measurement models and quantitative research methods to understand effective teaching and teachers.

Dan Berebitsky, PhD, is an assistant professor in the Educational Policy and Leadership Department in the Simmons School of Education and Human Development at Southern Methodist University. His research focuses on school culture, education policy, school social networks, and instructional leadership.

References

Anderson

R. C.

Freebody

(1981). Vocabulary knowledge. In Guthrie

J. T.

(Ed.), Comprehension and teaching: Research reviews (pp. 77–117). Newark, DE: International Reading Association.

Bates

DebRoy

(2004). Linear mixed models and penalized least squares. Journal of Multivariate Analysis, 91, 1–17.

Beck

I. L.

McKeown

M. G.

(2001). Text Talk: Capturing the benefits of read-aloud experiences for young children. The Reading Teacher, 55(1), 10–20.

Beck

I. Y.

McKeown

M. G.

(2007). Increasing young low-income children’s oral vocabulary repertoires through rich and focused instruction. Elementary School Journal, 107, 251–271.

Biemiller

(2003). Vocabulary: Needed if more children are to read well. Reading Psychology, 24, 323–335.

Biemiller

Boote

(2006). An effective method for building meaning vocabulary in primary grades. Journal of Educational Psychology, 98, 44–62.

Briggs

(2008). Using explanatory item response models to analyze group differences in science achievement. Applied Measurement in Education, 21, 89–118.

Carlisle

J. F.

Fleming

Gudbrandsen

(2000). Incidental word learning in science classes. Contemporary Educational Psychology, 25, 184–211.

Carlisle

J. F.

Johnson

D. J.

Phelps

Rowan

(2008). Teachers’ knowledge about reading and reading practices (Technical report No. 08-02. Assessment of Teachers’ Knowledge About Reading). Ann Arbor, MI: University of Michigan.

10.

Carlisle

J. F.

Kelcey

Berebitsky

Phelps

(2011). Embracing the complexity of instruction: A study of effects of teachers’ instruction on students’ reading comprehension. Scientific Studies of Reading, 15, 409–439.

11.

Carlisle

J. F.

Kelcey

Rowan

Phelps

(2011). Teachers’ knowledge about early reading: Effects on students’ gains in reading achievement. Journal of Research on Educational Effectiveness, 4, 289–321.

12.

Carlo

M. S.

August

Snow

C. E.

Lively

T. J.

White

C. E.

(2004). Closing the gap: Addressing the vocabulary needs of English-language learners in bilingual and mainstream classrooms. Reading Research Quarterly, 39, 188–215.

13.

Connor

C. M.

Morrison

F. J.

Underwood

P. S.

(2007). A second chance in second grade: The independent and cumulative impact of first-and second-grade reading instruction and students’ letter-word reading skill growth. Scientific Studies of Reading, 11, 199–233.

14.

Coyne

M. D.

McCoach

D. B.

Kapp

(2007). Vocabulary intervention for kindergarten students: Comparing extended instruction to embedded instruction and incidental exposure. Learning Disability Quarterly, 30, 74–88.

15.

Crowe

E. C.

Connor

C. M.

Pretscher

(2009). Examining the core: Relations among reading curricula, poverty, and first through third grade reading achievement. Journal of School Psychology, 47, 187–214.

16.

Cunningham

A. E.

(2005). Vocabulary growth through independent reading and reading aloud to children. In Hiebert

E. H.

Kamil

M. L.

(Eds.), Teaching and learning vocabulary: Bringing research to practice (pp. 45–68). New York, NY: Routledge.

17.

De Boeck

Wilson

(2004). Explanatory item response models. New York, NY: Springer.

18.

Dickinson

D. K.

(2011). Teachers’ language practices and academic outcomes of preschool children. Science, 333, 964–967.

19.

Graves

M. F.

(2006). The vocabulary book: Learning and instruction. Newark, DE: International Reading Association.

20.

Hart

Risley

T. R.

(1995). Meaningful differences in the everyday experiences of young American children. Baltimore, MD: Paul H. Brookes.

21.

Hill

Charlambous

Kraft

(2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher, 41(2), 56–64.

22.

Hoover

H. D.

Dunbar

S. B.

Frisbee

D. A.

(2003). Iowa Test of Basic Skills: Guide to research and development. Ithaca, IL: Riverside Publishing.

23.

Huttenlocher

Haight

Bryk

Seltzer

Lyons

(1991). Early vocabulary growth: Relation to language input and gender. Developmental Psychology, 27, 236–248.

24.

Kennedy

M. M.

(2010). Attribution error and the quest for teaching quality. Educational Researcher, 39, 591–598.

25.

Lesaux

N. K.

Kieffer

M. J.

Faller

S. E.

Kelley

J. G.

(2010). The effectiveness and ease of implementation of an academic vocabulary intervention for linguistically diverse students in urban middle schools. Reading Research Quarterly, 45, 196–228.

26.

Linacre

J. M.

(1989). Many-facet Rasch measurement. Chicago, IL: MESA Press.

27.

McKeown

M. G.

(1985). The acquisition of word meaning from context by children of high and low ability. Reading Research Quarterly, 20, 482–496.

28.

Mislevy

Johnson

Muraki

(1992). Scaling procedures in NAEP: Special issue: National Assessment of Educational Progress. Journal of Educational Statistics, 17(2), 131–154.

29.

Messick

(1989). Validity. In Linn

R. L.

(Ed.), Educational measurement (3rd ed., pp. 13–104). New York, NY: American Council on Education and Macmillan.

30.

Nagy

W. E.

(2005). Why vocabulary instruction needs to be long-term and comprehensive. In Hiebert

E. H.

Kamil

M. L.

(Eds.). Teaching and learning vocabulary: Bringing research to practice (pp. 27–44). New York, NY: Routledge.

31.

Nagy

W. E.

Herman

P. A.

(1987). Breadth and depth of vocabulary knowledge: Implications for acquisition and instruction. In McKeown

M. G.

Curtis

M. E.

(Eds.), The nature of vocabulary acquisition (pp. 19–35). Hillsdale, NJ: Erlbaum.

32.

NICHD Early Child Care Research Network. (2002). Early child care and children’s development prior to school entry: Results from the NICHD study of early child care. American Educational Research Journal, 39, 133–164.

33.

Raghunathan

T. E.

Lepkowski

J. M.

Van Hoewyk

Solenberger

(2001). A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodology, 27, 85–96.

34.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Application and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.

35.

Raudenbush

Johnson

Sampson

(2003). A multivariate, multilevel Rasch model with application to self-reported criminal behavior. Sociological Methodology, 33, 169–211.

36.

Rubin

D. B.

(1976). Inference and missing data. Biometrika, 63, 581–592.

37.

Rubin

D. B.

(1987). Multiple imputation for nonresponse in surveys. New York, NY: John Wiley & Sons.

38.

Ryder

R. J.

Graves

M. F.

(1994). Vocabulary instruction presented prior to reading in two basal readers. Elementary School Journal, 95, 139–153.

39.

Scott

J. A.

Jamieson-Noel

Asselin

(2003). Vocabulary instruction throughout the day in twenty-three Canadian upper elementary classrooms. Elementary School Journal, 103, 269–286.

40.

Scott

J. A.

Nagy

W. E.

(2004). Developing word consciousness. In Baumann

J. F.

Kame’enui

E. J.

(Eds.), Vocabulary instruction: Research to practice (pp. 201–217). New York, NY: Guilford Press.

41.

Smith

E. V.

Jr. (2001). Evidence for the reliability of measures and the validity of measure interpretation: A Rasch measurement perspective. Journal of Applied Measurement, 2, 281–311.

42.

Snow

C. E.

Griffin

Burns

M. S.

(2005). Knowledge to support the teaching of reading: Preparing teachers for a changing world. San Francisco, CA: Jossey-Bass.

43.

Stahl

S. A.

(2005). Four problems with teaching word meanings (and what to do to make vocabulary an integral part of instruction). In Hiebert

E. H.

Kamil

M. L.

(Eds.), Teaching and learning vocabulary: Bringing research to practice (pp. 95–114). New York, NY: Routledge.

44.

Stahl

S. A.

Fairbanks

M. M.

(1986). The effects of vocabulary instruction: A model-based meta-analysis. Review of Educational Research, 56, 72–110.

45.

Stahl

S. A.

Nagy

W. E.

(2006). Teaching word meanings. Mahwah, NJ: Erlbaum.

46.

Stanovich

K. E.

(1986). Matthew effects in reading: Some consequences in individual differences in the acquisition of literacy. Reading Research Quarterly, 21, 360–407.

47.

Tannenbaum

K. R.

Torgesen

J. K.

Wagner

R. K.

(2006). Relationships between word knowledge and reading comprehension in third-grade children. Scientific Studies of Reading, 10, 381–398.

48.

U.S. Department of Education. (2002). Guidance for the Reading First program. Retrieved from http://www2.ed.gov/programs/readingfirst/guidance.pdf.

49.

Wang

Liu

(2007). Formulation and application of the generalized multilevel facets model. Educational and Psychological Measurement, 67, 583–605.

50.

Watts

(1995). Vocabulary instruction during reading lessons in six classrooms. Journal of Reading Behavior, 27, 399–424.

51.

Weizman

Z. O.

Snow

C. E.

(2001). Lexical input as related to children’s vocabulary acquisition: Effects of sophisticated exposure and support for meaning. Developmental Psychology, 37, 265–279.

52.

White

T. G.

Graves

M. F.

Slater

W. H.

(1990). Growth of reading vocabulary in diverse elementary schools: Decoding and word meaning. Journal of Educational Psychology, 82, 281–290.

53.

Yen

(1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145.