Ability of Early Literacy Measures to Predict Future State Assessment Performance

Abstract

The purpose of this study was to determine the extent to which early literacy measures administered in kindergarten and Oral Reading Fluency (ORF) measures administered in Grade 1 are related to and predict future state reading assessment performances up to 7 years later. Results indicated that early literacy and ORF performances were significantly and moderately related to performances in Grades 3, 5, and 7. Grade 3 achievement was best predicted by ORF, followed by Phoneme Segmentation Fluency (PSF), and then Initial Sound Fluency (ISF). After controlling for the effects of previous state assessment scores in Grade 3, additional significant variance in Grade 5 performance was accounted for by ORF. Finally, after controlling for the effects of Grades 5 and 3 state assessment performances, early literacy and ORF measures did not significantly predict Grade 7 achievement. Discussion focuses on the implications of these findings for theory and practice, as well as limitations and directions for future research.

Keywords

curriculum-based assessment education assessment validity measurement literacy testing language and literacy educational testing

Many researchers have demonstrated the crucial importance of specific emergent literacy skills for later successful reading development (Catts, Compton, Tomblin, & Bridges, 2012; Catts, Fey, Tomblin, & Zhang, 2002; Scarborough, 1998; Snowling, Bishop, & Stothard, 2000). For example, Spira, Bracken, and Fischel (2005) found among a sample of low-income elementary school students that 70% of students who lacked these basic literacy skills in first grade continued to experience significant reading problems in fourth grade. Furthermore, those students with more severe early literacy deficits fared worse as time progressed. To help students with emergent literacy deficits catch up to typically developing peers, empirically validated early instruction and intervention approaches are necessary.

In a widely disseminated study, Whitehurst and Lonigan (1998) identified a set of early skills that are associated with later word reading and reading comprehension competency. Among these early literacy skills are phonological awareness, phonological processing, receptive and expressive language, and general print knowledge. The significance of these early literacy skills was subsequently reinforced by the National Reading Panel (2000) who identified five areas crucial to successful reading development, including phonemic awareness, phonics (alphabetic principle), fluency, vocabulary, and comprehension. These skills have since become a primary focus of reading research, instruction, and assessment measures (Chard & Kame’enui, 2000; Fletcher, Lyon, Fuchs, & Barnes, 2007; Rayner, Foorman, Perfetti, Pesetsky, & Seidenberg, 2001; Walker & Shinn, 2002).

With respect to the evaluation of these constructs, general outcome measures (GOMs) have been identified that can be used to progress monitor the development of early literacy and general reading skills in response to instruction and intervention (Fuchs & Deno, 1991). GOMs are general indicators of basic skill success that are collected on an ongoing basis and have important implications for later performance. In the area of emergent literacy, the Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good & Kaminski, 2002) includes the early literacy measures Initial Sound Fluency (ISF), Phoneme Segmentation Fluency (PSF), Letter Naming Fluency (LNF), and Nonsense Word Fluency (NWF). Each of these measures can be utilized to measure kindergarten students’ skill development against normative sample scores for kindergarteners in winter and spring. Beginning in the winter of Grade 1, Oral Reading Fluency (ORF) probes and norms are available to evaluate the ORF competency of students.

The increased accountability of schools to produce competent readers has evoked great interest in firmly establishing the relationships between reading-related GOMs, like DIBELS ORF, and state-mandated test scores (Reschly, Busch, Betts, Deno, & Long, 2009; Shapiro, Keller, Lutz, Santoro, & Hintze, 2006; Yeo, 2010). Most studies have investigated the relationship between ORF and state assessment performances collected in the same year (e.g., Fuchs, Fuchs, Hosp, & Jenkins, 2001; Fuchs, Fuchs, & Maxwell, 1998; McGlinchey & Hixson, 2004; Roehrig, Petscher, Nettles, Hudson, & Torgesen, 2008; Shapiro et al., 2006; Silberglitt, Burns, Madyun, & Lail, 2006; Stage & Jacobsen, 2001). There is consensus in the literature that within school year, ORF and state reading assessment performances are highly correlated. In fact, Yeo (2010) conducted a recent meta-analysis of 27 studies and identified a correlation of .69 between ORF and state assessment performances, providing robust evidence of the predictive validity of ORF.

Other studies have investigated the extent to which ORF may be used to predict state reading assessment performance beyond the school year. For example, Hintze and Silberglitt (2005) found in a large-scale study that winter Grade 1 ORF performance was moderately correlated (r = .49) with state assessment reading scores two years later in Grade 3. Likewise, Keller-Margulis, Shapiro, and Hintze (2008) conducted a cross-grade exploration and found that state reading achievement remained moderately correlated with ORF performance after 1 year and 2 years. Baker et al. (2008) found that considering ORF slope across benchmark periods in addition to ORF in year 1 increases the predictive year 2 value of ORF measures. These studies contribute evidence in support of the long-term predictive validity of reading ORF.

As research involving reading GOMs and future state reading achievement has largely focused on ORF skills (see Reschly et al., 2009; Yeo, 2010, for meta-analytic reviews), fewer studies have explored the predictive validity of early literacy measures like the DIBELS ISF, LNF, PSF, and NWF. Longitudinal studies assessing early literacy skills of kindergarten children have shown that these skills are predictive of performance on individually administered standardized reading measures in first and second grades (e.g., Burke, Hagan-Burke, Kwok, & Parker, 2009; Petersen, Allen, & Spencer, 2016). Petersen et al. (2014) found that ISF, LNF, PSF, NWF kindergarten DIBELS and ORF DIBELS administered in Grade 1 all had moderate to high correlations with a standardized reading comprehension measure administered in Grade 2. Fien et al. (2008) established that NWF administered in kindergarten remains moderately correlated with group-administered reading achievement test scores 2 years later. These findings indicate that early literacy performance continues to be related to later reading performance. More specifically, DIBELS early literacy measures seem to be reliable indicators of reading achievement in subsequent years.

Fewer investigations still have related early literacy measures to later state reading assessment performance. Goffreda, DiPerna, and Pedersen (2009) conducted a study involving 67 first-grade students who were administered DIBELS LNF, PSF, NWF, and ORF measures in Grade 1 and then engaged in a state reading assessment in Grade 3. Consistent with previous research, Grade 1 ORF was most related to later state assessment performance (r = .54) and ORF was the lone predictor of future achievement. Notably, Grade 1 NWF was found to have a small but significant correlation with the state assessment two years later (r = .35).

Some researchers have suggested that the role and strength of specific early literacy measures can change over time. Kirby, Parrila, and Pfeiffer (2003) measured how phonological awareness and naming speed predicted reading development in Grades 1 through 5. These researchers found that, while both were significant predictors of first- and second-grade reading achievement measures, phonological awareness became a weaker predictor as the grades progressed and naming speed became a stronger predictor. Similar to these findings, other researchers have found that, as reading competency develops, alphabetic principle measures (like LNF and NWF) tend to retain predictive value, whereas phonological awareness measures (like PSF) lose predictive strength (Kame’enui, Simmons, Good, & Harn, 2001; Schilling, Carlisle, Scott, & Zeng, 2007; Speece, Mills, Ritchey, & Hillman, 2002).

Although these studies and others (e.g., National Early Literacy Panel, 2008; Schatschneider et al., 2004) have demonstrated that early literacy measures are predictive of later reading achievement, it is unknown whether these early measures have additional explanatory value beyond measures of reading performance recorded temporally closer to the future reading measures. To forecast seventh-grade state test performance, one would intuitively examine fifth-grade state test performance before examining earlier measures of reading achievement. However, presently unknown is if early DIBELS measures explain unique variance in distal state test performance while controlling for the effects of prior state test performance. This is an important issue as it may inform the extent to which educators should consider various previous sources of information in predicting later reading achievement. The current study attempts to fill this gap in the literature.

In sum, the ability of ORF to predict state assessment performance beyond 2 years and the ability of kindergarten early literacy measures to add unique information when predicting future state test performance are unaddressed in the peer-reviewed literature. As a result, this study explored the relationships among kindergarten DIBELS ISF, PSF, LNF, and NWF, Grade 1 ORF, and state reading assessment performance in Grades 3, 5, and 7. This investigation also sought to contribute to the extant literature by determining the extent to which the early literacy measures administered at these points in time predict future state assessment scores up to 7 years later (i.e., Grades 3, 5, and 7), and whether the measures predict future achievement while controlling for previous state test performance.

Method

Participants

Participants of this study were enrolled in a suburban school district in southwestern Pennsylvania. To accomplish the goal of exploring the relationships among the early literacy measures, ORF, and future state assessment performance, the study examined data for a district-wide cohort of kindergarten students. Only participants with each of the following data points were included in the study: kindergarten early literacy DIBELS, Grade 1 ORF, and Grades 3, 5, and 7 state assessment scores. Of the 185 students in the cohort who participated in fall kindergarten benchmark testing, 130 students were still enrolled in the school district 7 years later and, as a result, were included in analyses. Mean age of participants at the winter kindergarten DIBELS administration was 5 years 7 months.

Only de-identified data were available for analyses, so the descriptive characteristics of the 130 participants are unknown. However, the school district in which data were collected historically reports 25% of the students as economically disadvantaged (receiving free or reduced lunch), 15% of the population as receiving special education services (excluding gifted), and less than 1% of students as having limited English proficiency. The racial/ethnic background of students within the district historically includes 95% Caucasian, 2% Black (non-Hispanic), 1% Hispanic, and less than 1% Asian/Pacific Islander students.

Measures

DIBELS (Good & Kaminski, 2002) is a widely used assessment system for measuring early literacy and ORF skills with strong evidence regarding reliability and validity of the measures. The score associated with DIBELS measures is generally the number of correct responses per standardized time period; hence, DIBELS measures are considered fluency measures. With respect to the winter kindergarten benchmark period, four probe types are available for administration. LNF requires a student to correctly identify the letter name of randomly presented uppercase and lowercase letters. ISF involves a student’s ability to identify and produce the first sound of a word. PSF measures a student’s ability to segment three- and four-phoneme words into individual phonemes, or the smallest units of sound within a word. NWF requires a student to read aloud nonsense words (e.g., vug, lut, ov). The ORF measure is based on the work of Deno (1985) and Shinn (1989) and is introduced at the Grade 1 winter benchmark. Students are required to read grade-level controlled passages for 1 min while the examiner records the number of omissions, substitutions, and hesitations of more than 3 s as errors. The score metric associated with ORF is words read correct per minute.

Reliability and validity information regarding these probes were obtained from the DIBELS technical manual (Dynamic Measurement Group, 2008). Multiple probe reliability values were reported as the following: .89 to .95 for ISF, .98 to .98 for LNF, .90 for PSF, .95 to .98 for NWF, and .96 to .99 for Grade 1 ORF. Mean concurrent and predictive validity values comparing DIBELS probes with other criterion measures included the following: 0.47 and 0.39 for ISF, 0.53 and 0.63 for LNF, 0.45 and 0.49 for PSF, and 0.52 and 0.64 for NWF (Dynamic Measurement Group, 2008). Grade 1 ORF mean concurrent and predictive validity coefficients were .78 and .63, respectively.

The Pennsylvania System of School Assessment (PSSA) is the state criterion-referenced measure created for educational accountability purposes in Pennsylvania (Pennsylvania Department of Education, 2002). Student performance is assessed in the areas of reading, writing, mathematics, and science at specific grade levels mandated by the state. In brief, the reading assessment requires students to read passages and answer multiple-choice questions to demonstrate comprehension of text. A scaled score is calculated and performance is classified into four categories including Advanced, Proficient, Basic, and Below Basic. Regarding Grades 3, 5, and 7 reading performance, respectively, scores between 1,235 and 1,441, 1,275 and 1,496, and 1,279 and 1,469 fell within the proficient range. Test–retest reliability for the reading assessment across criterion grades is approximately .90 (Data Recognition Corporation, 2010).

Procedure

This study utilized existing data available as a result of the universal screening efforts of the participating school district. Certified school staff members were trained to administer and score the DIBELS measures consistent with standardized DIBELS administration procedures. Refresher trainings were also provided across the school year. If a question regarding administration or scoring arose during a screening period, the staff member was instructed to consult an identified student data manager to reach a resolution and ensure integrity of the student data. All DIBELS data were collected on three separate occasions (i.e., fall, winter, and spring) each school year. Three probes of each DIBELS measure were administered at each benchmark period. Median performance is used by the district’s problem-solving team for decision making, and this performance was obtained for analysis. For the purposes of this study, only median winter benchmark period DIBELS data were used in analyses, as data from this time period are known to have greater predictive validity as compared with fall or spring benchmark data (Shapiro et al., 2006). All students were administered the PSSA Reading tests in the spring (March or April) of Grades 3, 4, 5, 6, and 7. Grades 3, 5, and 7 PSSA Reading scaled scores were obtained for analysis in this study. The primary author obtained these de-identified data in the form of an electronic spreadsheet.

Results

Preliminary analyses were conducted to identify outliers and establish that the assumptions of correlational and multiple regression analyses, including normality and homoscedasticity, were upheld. Table 1 presents the means and standard deviations associated with this study’s variables, as well as benchmark (i.e., lowest score obtainable that would not target a child for intervention) data regarding ORF, the early literacy measures, and the respective PSSA proficiency ranges. Regarding ORF in Grade 1, the mean student performance was above the winter benchmark but well below the spring benchmark. Mean NWF performance was a bit above benchmark, whereas mean LNF and, in particular, PSF performances were below the winter benchmark. Mean student achievement across administrations of the PSSA fell within the respective “proficient” range.

Table 1.

Descriptive Statistics Regarding Study Variables.

Measure	M	SD	Winter benchmark	Proficiency range
ISF-WK	24.95	14.26	Not provided
LNF-WK	32.78	16.06	34
PSF-WK	16.79	12.17	33
NWF-WK	15.67	13.33	19
ORF-W1	36.39	29.05	19^a
Grade 3 PSSA	1,427.96	190.17		1,235-1,441
Grade 5 PSSA	1,380.25	177.59		1,275-1,496
Grade 7 PSSA	1,425.12	189.87		1,279-1,469

Spring ORF benchmark = 47.

With respect to the relationships among the dependent variables, Table 2 provides the Pearson correlation coefficients among median winter kindergarten performance on ISF, LNF, PSF, and NWF; winter Grade 1 ORF performance; and PSSA reading performances in Grades 3, 5, and 7. All subskill mastery measures and ORF measures and future reading performance indicators were found to have significant and generally moderate positive correlations at the p < .01 level. Winter kindergarten PSF and Grade 7 PSSA Reading performance resulted in the lowest correlation (r = .27), and winter Grade 1 ORF and Grade 5 PSSA Reading performance resulted in the greatest correlation (r = .52). The strongest correlation among early literacy probes was found between LNF and NWF (r = .77). Among PSSA Reading performances, the strongest correlation was discovered between Grades 5 and 7 (r = .77).

Table 2.

Pearson Correlations Among General Outcome Measures and State Reading Assessment Performance.

Measure	1	2	3	4	5	6	7	8
1. ISF-WK	1
2. LNF-WK	.38	1
3. PSF-WK	.43	.38	1
4. NWF-WK	.40	.77	.30	1
5. ORF-W1	.33	.71	.25	.74	1
6. PSSA3	41	.40	.43	.38	.46	1
7. PSSA5	.29	.47	.31	.41	.52	.73	1
8. PSSA7	.33	.48	.27	.41	.45	.67	.77	1

Note. ISF-WK = Initial Sound Fluency–Winter Kindergarten; LNF-WK = Letter Naming Fluency–Winter Kindergarten; PSF-WK= Phoneme Segmentation Fluency–Winter Kindergarten; NWF-WK = Nonsense Word Fluency–Winter Kindergarten; ORF-W1= Oral Reading Fluency–Winter Grade 1; PSSA = Pennsylvania State System of Assessment; PSSA3 = Grade 3 Reading PSSA; PSSA5 = Grade 5 Reading PSSA; PSSA7 = Grade 7 Reading PSSA. All correlations were significant at the p < .01 level.

To explore the future predictive value of the early literacy measures and ORF, three multiple regression analyses were conducted to determine which measures accounted for significant variance in Grades 3, 5, and 7 PSSA reading performances, respectively. Regarding the Grade 3 regression analysis, the five predictor variables were entered into the respective regression analysis one at a time in the order of ORF, NWF, LNF, PSF, and ISF. ORF was entered first into the regression analysis as an exhaustive literature base is present to suggest that as the most complex task, ORF would be expected to be the most robust predictor of later reading performance. Furthermore, the ORF measure might be expected to hold greatest explanatory value as it was obtained closest in time to the criterion measures (i.e., ORF was obtained in Grade 1 whereas the emergent literacy measures were obtained in kindergarten). Next, the emergent literacy measures were entered into each regression analysis one at a time to determine the extent to which the emergent literacy measures added explanatory value beyond ORF.

The order of entry of the emergent literacy measures was determined after considering Ehri’s theoretical framework regarding the development of fluent reading skills (Ehri, 1995, 2005; Ehri & McCormick, 1998) and how that theory has been implemented in contemporary literature (e.g., Burke et al., 2009). Consistent with Ehri’s descriptions of the prealphabetic, partial alphabetic, full alphabetic, and consolidated alphabetic phases of reading development leading to automatic reading, ORF, as stated earlier, was entered into each regression analysis first. Then, NWF, LNF, PSF, and ISF were entered one at a time in order of decreasing theoretical complexity and reverse order of development because more complex tasks would be expected to hold greater predictive value. Regarding Grade 5 analyses, Grade 3 PSSA was entered into the regression analysis prior to ORF and the early literacy measures in the order described above to control for previous test performance. With respect to Grade 7 analyses, Grade 5 PSSA was entered first, followed by Grade 3 PSSA, ORF, and then the early literacy measures in descending order of complexity.

Preliminary analyses were conducted prior to each multiple regression analysis to verify that all variance inflation factors (i.e., values below 10; Hair, Anderson, Tatham, & Black, 1995) and tolerances (i.e., values above 0.10; Tabachnick & Fidell, 2001) were within acceptable limits and multicollinearity concerns were not present. Regarding the first multiple regression, Table 3 presents the five resulting models and associated statistics. ORF was found to explain a significant 21.3% of the variance in Grade 3 PSSA performance, F(1, 128) = 34.56, p < .001. The inclusion of NWF and then LNF did not explain further significant variance. The insertion of PSF explained an additional significant 9.3% of variance, F(1, 125) = 17.12, p < .001, and the subsequent inclusion of ISF explained a further significant 2.3% of the variance, F(1, 124) = 5.28, p < .05, in performance. With respect to state reading testing 3 years later, PSF and ISF predicted reading performance beyond ORF.

Table 3.

Summary of Regression Analyses for Variables Predicting Grade 3 PSSA Reading Performance.

	B	SE B	β	t	p	R ²	ΔR²
Model 1						.213	.213***
ORF-W1	3.02	0.51	.46	5.88	.000
Model 2						.216	.003
ORF-W1	2.60	0.76	.40	3.43	.001
NWF-WK	1.23	1.65	.09	0.75	.458
Model 3						.224	.008
ORF-W1	2.29	0.81	.35	2.85	.005
NWF-WK	0.11	1.92	.01	0.06	.955
LNF-WK	1.75	1.54	.15	1.14	.256
Model 4						.318	.093***
ORF-W1	2.42	0.76	.37	3.19	.002
NWF-WK	−0.11	1.81	−.01	−0.06	.952
LNF-WK	0.23	1.49	.02	0.15	.879
PSF-WK	5.18	1.25	.33	4.14	.000
Model 5						.345	.028*
ORF-W1	2.35	0.75	.36	3.15	.002
NWF-WK	−0.74	1.80	−.05	−0.41	.682
LNF-WK	0.15	1.47	.01	0.10	.919
PSF-WK	4.16	1.31	.27	3.18	.002
ISF-WK	2.60	1.13	.20	2.30	.023

Note. PSSA = Pennsylvania State System of Assessment; ORF-W1= Oral Reading Fluency–Winter Grade 1; NWF-WK = Nonsense Word Fluency–Winter Kindergarten; LNF-WK = Letter Naming Fluency–Winter Kindergarten; PSF-WK= Phoneme Segmentation Fluency–Winter Kindergarten; ISF-WK = Initial Sound Fluency–Winter Kindergarten.

p < .05. ***p < .001.

Table 4 presents the six resulting models and associated statistics regarding Grade 5 PSSA reading performance. Grade 3 PSSA was found to explain a significant 53.5% of the variance in Grade 5 PSSA performance, F(1, 128) = 147.52, p < .001. Then, the inclusion of ORF explained an additional significant 4.4% of variance, F(1, 127) = 13.40, p < .001. No early literacy measures accounted for additional significant variance. Table 5 presents the seven resulting models and associated statistics regarding prediction of PSSA reading performance 7 years later. Grade 5 PSSA was found to explain a significant 59.9% of the variance in Grade 7 PSSA performance, F(1, 128) = 190.82, p < .001. The subsequent inclusion of Grade 3 PSSA explained an additional and significant 6.2% of variance, F(1, 127) = 7.17, p < .01, in performance. Neither ORF nor emergent literacy measures contributed additional significant variance in achievement 7 years later.

Table 4.

Summary of Regression Analyses for Variables Predicting Grade 5 PSSA Reading Performance.

	B	SE B	β	t	p	R ²	ΔR²
Model 1						.535	.535***
PSSA3	0.68	0.06	.73	12.15	.000
Model 2						.580	.044***
PSSA3	0.58	0.06	.62	9.60	.000
ORF-W1	1.45	0.40	.24	3.66	.000
Model 3						.580	.000
PSSA3	0.58	0.06	.62	9.54	.000
ORF-W1	1.44	0.55	.24	2.64	.009
NWF-WK	0.03	1.14	.00	0.03	.977
Model 4						.586	.006
PSSA3	0.57	0.06	.61	9.38	.000
ORF-W1	1.21	0.57	.20	2.12	.036
NWF-WK	−0.88	1.32	−.07	−0.67	.504
LNF-WK	1.45	1.06	.13	1.37	.174
Model 5						.588	.002
PSSA3	0.59	0.07	.63	9.04	.000
ORF-W1	1.15	0.58	.19	2.00	.047
NWF-WK	−0.86	1.32	−.06	−0.65	.518
LNF-WK	1.63	1.09	.15	1.50	.137
PSF-WK	−0.72	0.97	−.05	−0.74	.461
Model 6						.591	.003
PSSA3	0.60	0.07	.65	9.04	.000
ORF-W1	1.14	0.58	.19	1.99	.049
NWF-WK	−0.66	1.34	−.05	−0.49	.624
LNF-WK	1.65	1.09	.15	1.52	.132
PSF-WK	−0.47	1.01	−.03	−0.47	.643
ISF-WK	−0.81	0.86	−.07	−0.94	.348

Note. PSSA3 = Grade 3 Reading Pennsylvania State System of Assessment; ORF-W1 = Oral Reading Fluency–Winter Grade 1; NWF-WK = Nonsense Word Fluency–Winter Kindergarten; LNF-WK = Letter Naming Fluency–Winter Kindergarten; PSF-WK= Phoneme Segmentation Fluency–Winter Kindergarten; ISF-WK = Initial Sound Fluency–Winter Kindergarten.

***

p < .001.

Table 5.

Summary of Regression Analyses for Variables Predicting Grade 7 PSSA Reading Performance.

	B	SE B	β	t	p	R ²	ΔR²
Model 1						.599	.599***
PSSA5	0.83	0.06	.77	13.81	.000
Model 2						.620	.021**
PSSA5	0.66	0.09	.62	7.68	.000
PSSA3	0.22	0.08	.22	2.68	.008
Model 3						.621	.001
PSSA5	0.64	0.09	.60	7.10	.000
PSSA 3	0.21	0.08	.21	2.57	.011
ORF-W1	0.25	0.43	.04	0.59	.556
Model 4						.627	.006
PSSA5	0.64	0.09	.60	7.13	.000
PSSA3	0.20	0.08	.20	2.51	.014
ORF-W1	−0.28	0.57	−.04	−0.50	.616
NWF-WK	1.64	1.15	.12	1.42	.157
Model 5						.638	.010
PSSA5	0.62	0.09	.58	6.92	.000
PSSA3	0.20	0.08	.20	2.53	.013
ORF-W1	−0.58	0.58	−.09	−1.00	.320
NWF-WK	0.35	1.33	.03	0.27	.790
LNF-WK	2.03	1.07	.17	1.90	.061
Model 6						.640	.003
PSSA5	0.62	0.09	.58	6.84	.000
PSSA3	0.23	0.08	.23	2.70	.008
ORF-W1	−0.65	0.59	−.10	−1.10	.273
NWF-WK	0.39	1.33	.03	0.29	.772
LNF-WK	2.27	1.10	.19	2.06	.042
PSF-WK	−0.91	0.98	−.06	−0.93	.352
Model 7						.643	.003
PSSA5	0.62	0.09	.58	6.91	.000
PSSA3	0.21	0.09	.21	2.42	.017
ORF-W1	−0.65	0.59	−.10	−1.10	.273
NWF-WK	0.17	1.34	.01	0.13	.901
LNF-WK	2.23	1.10	.19	2.02	.045
PSF-WK	−1.19	1.01	−.08	−1.18	.241
ISF-WK	0.92	0.86	.07	1.07	.288

Note. PSSA5 = Grade 5 Reading Pennsylvania State System of Assessment; PSSA3 = Grade 3 Reading Pennsylvania State System of Assessment; ORF-W1= Oral Reading Fluency–Winter Grade 1; NWF-WK = Nonsense Word Fluency–Winter Kindergarten; LNF-WK = Letter Naming Fluency–Winter Kindergarten; PSF-WK= Phoneme Segmentation Fluency–Winter Kindergarten; ISF-WK = Initial Sound Fluency–Winter Kindergarten.

p < .01. ***p < .001.

Discussion

This investigation contributes to the extant literature by exploring the extent to which early literacy measures administered in kindergarten and ORF administered in Grade 1 predict future state assessment scores up to 7 years later (i.e., in Grades 3, 5, and 7) and while controlling for previous state test performance.

Consistent with previous literature relating early literacy measures and ORF to future state test achievement (e.g., Hintze & Silberglitt, 2005; Keller-Margulis et al., 2008), ORF was a significant predictor of state reading performance in Grade 3. This is similar to Goffreda et al. (2009) who found that ORF was the best predictor of state reading performance 2 years later. However, unlike that study, the present investigation found that early literacy DIBELS measures accounted for additional significant variance in state assessment performance. For example, Grade 3 state reading performance was significantly predicted by Grade 1 ORF, but kindergarten PSF and ISF also accounted for additional significant variance beyond ORF.

Of note, the inclusion of Grade 1 ORF explained an additional and significant 4.4% of variance indicating that ORF in Grade 1 can be used to understand state assessment performance 2 years later. Thus, the aforementioned early literacy kindergarten measures PSF and ISF may be considered by a school-based team aiming to forecast state reading assessment performance up to 3 years into the future and ORF administered in Grade 1 up to 2 years into the future.

This finding is consistent with previous research that found kindergarten early literacy measures could be used to predict performance on a standardized reading comprehension test 2 years later (Burke et al., 2009; Petersen et al., 2014). In addition, these data add further evidence of the predictive validity of early literacy DIBELS measures not only 2 years into the future (Burke et al., 2009; Goffreda et al., 2009) but well beyond. Furthermore, these results indicate that basic skills, such as those measured by PSF and ISF, are important to the development of successful readings skills during early elementary school.

The current results further demonstrated that ORF significantly predicted performance, above and beyond previous state assessments, on state standardized assessments up to 5 years later. However, findings indicated that Grade 5 and then Grade 3 state reading test scores significantly predicted performance on the Grade 7 state assessment with no early literacy measure contributing additional significant variance. This finding fills a gap in the predictive reading performance literature and is of value because it may influence the sources of information to which educators look to predict future test performance.

Despite not adding significant variance above previous state test performance, the predictive validity of LNF should be further investigated as it approached significance, and with a larger sample size and more power, LNF may have significantly predicted performance on Grade 7 state test scores beyond scores on Grades 3 and 5 tests. This notion would be consistent with previous findings indicating rapid retrieval of alphabetic knowledge, such as LNF, may retain predictive strength across time (Kame’enui et al., 2001; Kirby et al., 2003; Schilling et al., 2007; Speece et al., 2002). Future research should attempt to replicate this study with a larger sample size.

Limitations and Future Research

One limitation of the present investigation was the modest sample size of participants, all enrolled in a single school district. Although many GOM/state criterion test studies also included findings from one school district, the generalizability of these findings would be enhanced if participant data were obtained across school districts. Second, the present data also pertain to the prediction of Pennsylvania’s criterion test, so inferences to other state criterion tests should be made with caution. Third, the participants in this study were students who remained within the school district for at least 7 years, so the present results may not be reflective of more transitory populations of students. Finally, although descriptive data regarding the actual participants were unavailable in the data set, the participants of this study are likely homogeneous with respect to racial/ethnic background and primary language status. Future studies should explore the relationships among these variables in diverse populations of students, including English language learners.

The emergent literacy and ORF constructs measured in the present study only accounted for 21.3%, 9.3%, and 2.8% of the total variance in state reading assessment performance 3 years later. With the knowledge that the criterion of state reading assessments is essentially comprehension, future studies should continue to investigate the extent to which other cognitive and environmental constructs relate to future reading performance. Consensus exists that a broad array of cognitive skills are required to glean meaning from text, especially as the passages become more conceptually complex, vocabulary demands increase, topics are less familiar, and the length of passage increases. For example, McGrew and Wendling (2010) concluded, after a review of the empirical literature, that crystallized, language-related skills and background knowledge are significantly related to reading comprehension and perhaps take on greater importance than fluency indicators across time. Furthermore, researchers have consistently pointed to the importance of listening comprehension, working memory, vocabulary, and higher order processes like inferencing and comprehension monitoring skills as contributors to reading comprehension (Fletcher et al., 2007; Shaywitz, 2003). Including these constructs in future studies may help researchers understand the relationships between other early skill sets and future reading ability.

The present findings have considerable utility for school-based teams attempting to identify students at risk for failure on later state reading assessments. The present data provide additional concurrent and predictive validity data regarding DIBELS ORF and early literacy measures. Practitioners should be aware that early literacy measures—even when obtained in kindergarten—may be predictive of performance on later administrations of the state reading assessments. However, the present results also revealed that past state reading assessment scores best predicted future state assessment scores, and early literacy measures may not contribute additional predictive value. To forecast upcoming state assessment performance, problem-solving teams should focus on the most proximal measures of reading test performance.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Baker

S. K.

Smolkowski

Katz

Fien

Seeley

J. R.

Kame’enui

E. J.

Beck

C. T.

(2008). Reading fluency as a predictor of reading proficiency in low-performing, high-poverty schools. School Psychology Review, 37, 18-37.

Burke

M. D.

Hagan-Burke

Kwok

Parker

(2009). Predictive validity of early literacy indicators from the middle of kindergarten to second grade. The Journal of Special Education, 41, 209-226.

Catts

H. W.

Compton

Tomblin

J. B.

Bridges

M. S.

(2012). Prevalence and nature of late-emerging poor readers. Journal of Educational Psychology, 104, 166-181.

Catts

H. W.

Fey

M. E.

Tomblin

J. B.

Zhang

(2002). A longitudinal investigation of reading outcomes in children with language impairments. Journal of Speech, Language, and Hearing Research, 45, 1142-1157.

Chard

D. J.

Kame’enui

E. J.

(2000). Struggling first-grade readers: The frequency and progress of their reading. The Journal of Special Education, 34, 28-38.

Data Recognition Corporation. (2010). Technical report for the 2010 Pennsylvania System of School Assessment. Retrieved from http://www.portal.state.pa.us/portal/server.pt/community/technical_analysis/7447

Deno

(1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219-232.

Dynamic Measurement Group. (2008). DIBELS 6th edition technical adequacy information (Technical Report No. 6). Eugene, OR: Author. Retrieved from http://dibels.org/pubs.html

Ehri

L. C.

(1995). Phases of development in learning to read words by sight. Journal of Research in Reading, 18, 116-125.

10.

Ehri

L. C.

(2005). Learning to read words: Theory, findings, and issues. Scientific Studies of Reading, 9, 167-188.

11.

Ehri

L. C.

McCormick

(1998). Phases of word learning: Implications for instruction with delayed and disabled readers. Reading and Writing Quarterly, 12, 135-164.

12.

Fien

Baker

S. K.

Smolkowski

Smith

J. L. M.

Kame’enui

E. J.

Beck

C. T.

(2008). Using nonsense word fluency to predict reading proficiency in kindergarten through second grade for English learners and native English speakers. School Psychology Review, 37, 391-408.

13.

Fletcher

J. M.

Lyon

G. R.

Fuchs

L. S.

Barnes

M. A.

(2007). Learning disabilities: From identification to intervention. New York, NY: Guilford Press.

14.

Fuchs

L. S.

Deno

S. L.

(1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57, 488-499.

15.

Fuchs

L. S.

Fuchs

Hosp

M. K.

Jenkins

(2001). Oral reading fluency as an indicator of reading competence: A theoretical, empirical, and historical analysis. Scientific Studies of Reading, 5, 239-256.

16.

Fuchs

L. S.

Fuchs

Maxwell

(1998). The validity of informal reading comprehension measures. Remedial and Special Education, 9, 20-28.

17.

Goffreda

C. T.

Diperna

J. C.

Pedersen

J. A.

(2009). Preventive screening for early readers: Predictive validity of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS). Psychology in the Schools, 46, 539-552.

18.

Good

Kaminski

(2002). Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement.

19.

Hair

J. F.

Jr. Anderson

R. E.

Tatham

R. L.

Black

W. C.

(1995). Multivariate data analysis (3rd ed.). New York, NY: Macmillan.

20.

Hintze

J. M.

Silberglitt

(2005). A longitudinal examination of the diagnostic accuracy and predictive validity of R-CBM and high-stakes testing. School Psychology Review, 34, 372-386.

21.

Kame’enui

E. J.

Simmons

D. C.

Good

R. H.

Harn

B. A.

(2001). The use of fluency-based measures in early identification and evaluation of intervention efficacy in schools. In Wolf

(Ed.), Dyslexia, fluency, and the brain (pp. 307-331). Timonium, MD: York.

22.

Keller-Margulis

M. A.

Shapiro

E. S.

Hintze

J. M.

(2008). Long-term accuracy of curriculum based measures in reading and mathematics. School Psychology Review, 37, 374-390.

23.

Kirby

J. R.

Parrila

R. K.

Pfeiffer

S. L.

(2003). Naming speed and phonological awareness as predictors of reading development. Journal of Educational Psychology, 95, 453-464.

24.

McGlinchey

M. T.

Hixson

M. D.

(2004). Contemporary research on curriculum-based measurement to predict performance on state assessments in reading. School Psychology Review, 33, 193-204.

25.

McGrew

K. S.

Wendling

B. J.

(2010). Cattell–Horn–Carroll cognitive achievement relations: What have we learned from the past 20 years of research. Psychology in the Schools, 47, 651-675.

26.

National Early Literacy Panel. (2008). Developing early literacy: Report of the National Early Literacy Panel. Washington, DC: National Institute for Literacy.

27.

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of scientific research literature on reading and its implications for reading instruction (NIH Pub. No. 00-4769). Washington, DC: National Institute of Child Health and Human Development.

28.

Pennsylvania Department of Education. (2002). The Pennsylvania System of School Assessment: Handbook for report interpretation 2002 PSSA mathematics and reading assessments. Harrisburg: Author.

29.

Petersen

D. B.

Allen

M. M.

Spencer

T. D.

(2016). Predicting reading difficulty in first grade using dynamic assessment of decoding in early kindergarten: A large-scale longitudinal study. Journal of Learning Disabilities, 49, 200-215. doi:10.1177/0022219414538518

30.

Rayner

Foorman

B. R.

Perfetti

C. A.

Pesetsky

Seidenberg

M. S.

(2001). How psychological science informs the teaching of reading. Psychological Science in the Public Interest, 2, 31-73.

31.

Reschly

A. L.

Busch

T. W.

Betts

Deno

S. L.

Long

J. D.

(2009). Curriculum-based measurement oral reading as an indicator of reading achievement: A meta-analysis of the correlational evidence. Journal of School Psychology, 47, 427-469.

32.

Roehrig

A. D.

Petscher

Nettles

S. M.

Hudson

R. F.

Torgesen

J. K.

(2008). Accuracy of the DIBELS Oral Reading Fluency measure of predicting third grade reading comprehension outcomes. Journal of School Psychology, 46, 343-366.

33.

Scarborough

H. S.

(1998). Early identification of children at risk for reading disabilities: Phonological Awareness and some other promising predictors. In Shapiro

B. K.

Accardo

P. J.

Capute

A. J.

(Eds.), Specific reading disability: A view of the spectrum (pp. 75-119). Timonium, MD: York.

34.

Schatschneider

Fletcher

J. M.

Francis

D. J.

Carlson

C. D.

Foorman

B. R.

(2004). Kindergarten prediction of reading skills: A longitudinal comparative analysis. Journal of Educational Psychology, 96, 265-282.

35.

Schilling

S. G.

Carlisle

J. F.

Scott

S. E.

Zeng

(2007). Are fluency measures accurate predictors of reading achievement? The Elementary School Journal, 107, 429-448.

36.

Shapiro

E. S.

Keller

M. A.

Lutz

J. G.

Santoro

L. E.

Hintze

J. M.

(2006). Curriculum-based measures and performance on state assessment and standardized tests: Reading and math performance in Pennsylvania. Journal of Psychoeducational Assessment, 24, 19-35.

37.

Shaywitz

(2003). Overcoming dyslexia: A new and complete science-based program for reading problems at any level. New York, NY: Knopf.

38.

Shinn

(1989). Curriculum-based measurement: Assessing special children. New York, NY: Guilford Press.

39.

Silberglitt

Burns

M. K.

Madyun

N. H.

Lail

(2006). Relationship of reading fluency assessment data with state accountability test scores: A longitudinal comparison of grade levels. Psychology in the Schools, 43, 527-535.

40.

Snowling

Bishop

D. V. M.

Stothard

S. E.

(2000). Is preschool language impairment a risk factor for dyslexia in adolescence? Journal of Child Psychology and Psychiatry, 41, 587-600.

41.

Speece

D. L.

Mills

Ritchey

K. D.

Hillman

(2002). Initial evidence that letter fluency tasks are valid indicators of early reading skill. Journal of Special Education, 36, 223-233.

42.

Spira

E. G.

Bracken

S. S.

Fischel

J. E.

(2005). Predicting improvement after first grade reading difficulties: The effects of early language, emergent literacy, and behavior skills. Developmental Psychology, 41, 225-234.

43.

Stage

S. A.

Jacobsen

M. D.

(2001). Predicting student success on a state-mandated performance-based assessment using oral reading fluency. School Psychology Review, 30, 407-420.

44.

Tabachnick

B. G.

Fidell

L. S.

(2001). Using multivariate statistics fourth edition. Boston, MA: Allyn & Bacon.

45.

Walker

H. M.

Shinn

M. R.

(2002). Structuring school-based interventions to achieve integrated primary, secondary, and tertiary prevention goals for safe and effective schools. In Shinn

M. R.

Stoner

Walker

H. M.

(Eds.), Interventions for academic and behavior problems: Preventive and remedial approaches (pp. 1-25). Silver Spring, MD: National Association of School Psychologists.

46.

Whitehurst

G. J.

Lonigan

C. J.

(1998). Child development and emergent literacy. Child Development, 69, 848-872.

47.

Yeo

(2010). Predicting performance on state achievement tests using curriculum-based measurement in reading: A multilevel meta-analysis. Remedial and Special Education, 31, 412-422.