Abstract
This study investigated the relationship between specific cognitive patterns of strengths and weaknesses (PSWs) and the errors children make in reading, writing, and spelling tests from the Kaufman Test of Educational Achievement–Third Edition (KTEA-3). Participants were selected from the KTEA-3 standardization sample based on five cognitive profiles: High Crystallized Ability paired with Low Processing Speed and Long-Term Retrieval (High Gc), Low Crystallized Ability paired with High Processing Speed and Long-Term Retrieval (High Gs/Glr), Low Orthographic Processing (Low OP), Low Phonological Processing (Low PP), and Low Phonological Processing paired with Low Orthographic Processing (Low PP_OP). Error factor scores for all five groups were compared on Reading Comprehension and Written Expression; the first four groups were compared on Letter & Word Recognition, Nonsense Word Decoding, and Spelling, and the first three groups were compared on Phonological Processing. Significant differences were noted among the patterns of errors demonstrated by the five groups. Findings support the notion that students with diverse cognitive PSWs display different patterns of errors on tests of academic achievement.
Language acquisition and development is an important topic in many fields of study, including research on academic achievement, literacy, and cognitive functioning. The types of errors students make in the academic areas of reading, writing, and spelling have been studied to gain insight into the learning processes involved in developing these language-based academic skills. Many of these prior studies have used error analysis to identify the types of errors students make as well as to understand developmental trends. Error analysis provides an effective means for developing students’ language-based academic skills and informing educational interventions.
Error Analysis and Reading
Since the 1970s, numerous studies in the published literature have promoted the importance of error analysis as a means to provide insight into how individuals learn to read and the strategies upon which they rely (Leu, 1982; see Greenberg, Ehri, & Perin, 2002). Error analysis has helped researchers learn about how written information is processed and how literacy skills develop (Leu, 1982). Greenberg et al. (2002) noted that the analysis of children’s reading errors has been the basis for the development of major theories of literacy (e.g., Ehri, 1986; Frith, 1985) and has played an important role in understanding and diagnosing dyslexia.
In an interesting error analysis study, Greenberg et al. (2002) found that adults who were learning to read in literacy courses differed from children who were learning to read in school, both quantitatively in their error rates and qualitatively in their error types. Children relied primarily on phonological decoding, whereas adults favored visual memory–cognitive differences which translate to different kinds of recommended interventions for children versus adults.
Another study used error analysis to identify individual differences in children’s reading and spelling strategies, and to identify the cognitive skills underlying their strategy use (McGeown, Medford, & Moxon, 2013). Through analyzing children’s errors, they found that different cognitive skills predicted reliance upon different strategies. For example, if children had more well-developed decoding skills, they were more likely to rely upon a phonological strategy to read new words and less likely to reply upon an orthographic (visual) strategy. According to McGeown et al. (2013), understanding students’ reading strategies is valuable for planning appropriate reading instruction and intervention.
Error Analysis and Writing and Spelling
Error analysis has also been widely utilized in the study of written expression and spelling. Writing is a complex task that requires the integration of multiple cognitive, linguistic, and motor abilities (Mather, Wendling, & Roberts, 2009). Writing involves both basic-level transcription skills (handwriting, spelling, punctuation, capitalization, and grammar) and higher level composition skills (planning, content, organization, and revision) (Graham & Harris, 2005).
Poor spelling can limit both the quality and quantity of writing (Gregg & Mather, 2002). Spelling error analysis research has shown that there are stages of spelling development (Bear & Templeton, 1998; Moats, 1995; Read, 1975), and the progression of spelling knowledge is not rigid or entirely predictable. In fact, elements from later stages can be seen in the early years of spelling and literacy development (Sharp, Sinatra, & Reynolds, 2008; Treiman & Bourassa, 2000; Treiman, Cassar, & Zukowski, 1994).
In a study using both words and pseudowords which contained double letters, Wright and Ehri (2007) built on Treiman’s (1993) work by showing that students in kindergarten and first grade who are exposed to more and more print begin to conform to legal orthography (e.g., ll in the word “bell”, or ck in the word “back”) in their spelling. Specifically, students who had been exposed to more print tended to place double letters in the middle and at the end of words, as opposed to at the beginning. This skill reflects increasing knowledge of orthographic patterns because double letters in the initial position of a word are uncommon in the English language. According to Wright and Ehri, even when children cannot read, they gain awareness that certain conventions exist in written language through visual exposure to print.
A study conducted by Gajar (1989) utilized error analysis to compare the compositions written by university students with and without learning disabilities. This research identified a three-factor structure of vocabulary/fluency, syntactic maturity, and vocabulary/diversity based on 17 variables associated with written expression. Results revealed that students with learning disabilities differed significantly on the factors of vocabulary/fluency and syntactic maturity. Specifically, students with learning disabilities were found to be not as fluent in word production and in the number of different words used in their compositions as their normally achieving peers. The findings of this study provide important information for understanding the differences in the instructional needs of students identified with learning disabilities with respect to further developing their written expression skills.
Error Analysis Summary
According to McCloskey, Kaufman, Kaufman, and McCloskey (1985), the information provided by analyzing a student’s correct and incorrect response patterns can be highly beneficial to the clinician who desires to carry the diagnostic assessment process beyond the interpretation of standard scores. Diagnosticians can use error analysis information to identify the specific skill deficit areas in which intervention should be planned or those areas for which further diagnostic testing should be carried out. This information may also help guide the development of specific, measurable, and short-term instructional objectives for classroom intervention as well as evidence-based Individualized Education Program (IEP) objectives for special education students. Furthermore, and of special relevance for this study, error analysis information may assist educators in better understanding the relationship between children’s patterns of strengths and weaknesses (PSWs) in their cognitive profiles and the types of errors they may be more likely to make in reading or writing. Based on a better understanding of how cognitive PSWs relate to students’ patterns of errors, a PSW approach may be shown empirically to help educators better link their assessment findings with appropriate interventions—a notion that is foreign to some researchers and writers (see, for example, Burns, 2016).
The Relationship Between Cognitive Skills and Academic Errors in Reading and Writing
Although there is no universally accepted theory of cognitive abilities, the Cattell–Horn–Carroll (CHC) theory is perhaps the most widely accepted, unified empirical theory of cognition to date (Schneider & McGrew, 2012). The CHC theory is a three-level model of human cognitive abilities that includes general intelligence (g; Stratum III), nine or more broad cognitive abilities (Stratum II), and more than 100 narrow cognitive abilities (Stratum I; Flanagan, Ortiz, & Alfonso, 2013).
According to a study conducted by Evans, Floyd, McGrew, and Leforgee (2002), the following CHC cognitive abilities demonstrated a relationship with reading achievement across childhood and adolescence: comprehension-knowledge (Gc), short-term memory (Gsm), auditory processing (Ga), long-term retrieval (Glr), and processing speed (Gs). Glr abilities are particularly important with respect to the ability to store and retrieve sound–symbol relations and efficiently retrieve lexical and general knowledge, skills which are critical to early reading development (Perfetti, 2007; Shaywitz, Morris, & Shaywitz, 2008; Vellutino, Scanlon, & Zhang, 2007).
Results of another study (Floyd, McGrew, & Evans, 2008) suggest that the CHC cognitive abilities that are most related to written expression include the broad abilities of Ga, Glr, Gs, Gsm, crystallized intelligence (Gc), and fluid reasoning (Gf). Relationships between specific cognitive abilities and various domains of achievement have been found by a variety of prior studies.
Long-Term Retrieval (Glr)
In her work with school-age children, Ismailer (2015) found that the CHC abilities of Glr, Gsm, and Ga were important to the development and prediction of reading skills. Results showed that Glr was most strongly related to basic reading skills, whereas Ga and Gsm demonstrated moderate relationships to basic reading. In addition, Glr showed the strongest relationship with reading comprehension, whereas Gc, visual processing (Gv), and Gsm displayed a moderate relationship (Ismailer, 2015).
Research has also shown that when Glr abilities are weak, this area of weakness is associated with reading difficulty. According to Mather, Vogel, Spodak, and McGrew (1991), students who perform poorly on Glr tests may have difficulty with paired association tasks such as learning the names of the letters of the alphabet. More recent research has also supported the association between reading difficulties and weak Glr abilities with respect to the key skills of accessing background knowledge while reading, accessing phonological representations during decoding, and retelling or paraphrasing what one has read (Flanagan & Alfonso, 2010).
Processing Speed (Gs)
Processing speed (Gs) is defined as the ability to fluently perform relatively simple cognitive tasks automatically, especially when under pressure to maintain focused attention and concentration (Schneider & McGrew, 2012). The narrow ability within Gs that explains variance in basic reading skills appears to be perceptual speed (Gs-P). The importance of Gs-P is seen in the relationship between perceptual speed, speed of processing, and the need for automaticity in integrating phonological and orthographic codes in word reading (e.g., Barker, Torgesen, & Wagner, 1992; Berninger, 1990; Hale & Fiorello, 2004; Joshi & Aaron, 2000; Urso, 2008). The speed with which children name familiar stimuli is a strong predictor of reading skill (Norton & Wolf, 2012), including word decoding, comprehension, and oral reading. Finally, researchers (Berninger, Abbott, Thomson, & Raskind, 2001; Kintsch & Rawson, 2005; Shaywitz et al., 2008) have emphasized the importance of a variety of speed or fluency constructs in early reading skill acquisition (e.g., rapid automatized naming, naming speed, speed of semantic or lexical access, verbal efficiency, automaticity). Research also indicates that having a slower reading speed is associated with lower Gs, a weakness that can interfere with comprehension and require the rereading of material for understanding (Flanagan & Alfonso, 2010). Conversely, higher levels of Gs are associated with higher reading speed.
Crystallized Ability or Verbal Comprehension/Knowledge (Gc)
Crystallized intelligence (Gc) is defined as the verbal knowledge, often gained from school and acculturation, and its application. Gc contributes to oral expression and listening comprehension, and it is a fundamental component of accessing information from school and society. Researchers have found that Gc predicts reading comprehension beyond other broad cognitive abilities in first- and second-grade students (McGrew, Flanagan, Keith, & Vanderwood, 1997). Gc also has strong relations with decoding and word recognition skills (McGrew & Wendling, 2010). Similarly, Ismailer (2015) found that Gc was one of the best predictors of scores on a passage comprehension task for students in Grades K-8. Previous research has shown that Gc abilities become increasingly more important with age. Benson (2008) found that Gc had a direct effect on basic reading skills and reading comprehension; although this effect was not strong for students in kindergarten through third grade, it was considerably stronger for students in Grades 4 through 12.
Orthographic Processing
CHC theory does not currently include orthographic processing as a narrow ability under the broad ability of visual processing (Gv), but it is acknowledged as an important contributor to reading and spelling development (Flanagan et al., 2013). Orthography is the system of marks that make up a printed language (Wagner & Barker, 1994). Orthographic processing is defined as “the ability to form, store, and access orthographic representations” (Stanovich & West, 1989, p. 404). Although visual memory may contribute to orthographic processing, these are two distinct abilities (Aaron, 1994; Kilpatrick, 2015). McCallum et al. (2006) found significant correlations between orthographic awareness and both reading and spelling ability across a sample of typically developing school-age children.
A longitudinal study also demonstrated that preschool orthographic skills were more predictive of reading in later stages than in early stages. Badian (1995) found that, while preschool orthographic awareness did not show a significant relationship with first-grade reading comprehension, preschool orthographic awareness predicted reading comprehension and spelling at higher grade levels.
Some research suggests that orthographic processing contributes to the development of spelling and written transcription skills. For example, the findings from a case study of a student with developmental hyperlexia are consistent with a meaningful relationship between an orthographic deficit and low performance in “writing to dictation” (Glosser, Grugan, & Friedman, 1997). However, the relationship between orthographic processing and writing skills, in particular, is sparse in the research literature, and continued research is needed.
Phonological Processing
Phonological processing refers to the use of phonological information (i.e., the sounds of one’s language) in processing oral and written language (Wagner & Torgesen, 1987). Phonological processing has been used as an early predictor of the acquisition and development of literacy ability, and poor phonological processing skills have been shown to contribute to reading difficulty and dyslexia among school-age children (Stanovich, 1988). Phonological processing includes phonological awareness, which is the ability to sense and manipulate the phonology (sound structure) of oral language (Mattingly, 1972).
Phonological awareness has received considerable research attention. A study by Nelson, Linstrom, Lindstrom, and Denis (2012) suggested that word reading skills at the kindergarten and first-grade levels are best predicted by a phonological awareness composite alone. Research (e.g., Engen & Høien, 2002; Snowling & Nation, 1997) also suggests that phonological awareness contributes to reading comprehension. In a study looking at other aspects of reading (Del Campo, Buchanan, Abbott, & Berninger, 2015), phonological processing (phonology for whole words and phonemes) explained unique variance in fourth and sixth graders’ reading levels.
Substantial evidence also exists for the essential role of phonological processing on handwriting, spelling, and composing. Berninger’s (2009) research suggests that an orthographic processor perceives letters in text, whereas a phonological processor maps letters onto the spoken equivalents. The importance of establishing automatic orthographic–phonological connections has been stressed by several researchers (Adams & Bruck, 1993; Ehri, 1992).
Study Purpose
The present study was conducted to discern whether there is a relationship between students’ PSW profile and the kinds of errors they make in reading and writing tests from the Kaufman Test of Educational Achievement–Third Edition (KTEA-3; Kaufman & Kaufman, 2014). Five PSW profiles were of interest which represent the students’ patterns of cognitive strengths and weaknesses based on their KTEA-3 scores on subtests that measure Gc, Gs/Glr, phonological processing, and orthographic processing: High Gc paired with Low Gs/Glr (abbreviated as High Gc), Low Gc paired with High Gs/Glr (abbreviated as High Gs/Glr), Low Phonological Processing (Low PP), Low Orthographic Processing (Low OP), and Low Phonological Processing paired with Low Orthographic Processing (Low PP_OP). The methods for identifying students with each profile are explained in the “Procedure” section.
The following research questions were investigated in this study:
Method
Participants
The participants in this study were students tested during the standardization of the KTEA-3 between August 2012 and July 2013. They represent a subsample of the standardization and validation sample for the KTEA-3.
Demographic data for the entire standardization and validation samples are provided in the KTEA-3 Technical & Interpretive Manual (Kaufman, Kaufman, & Breaux, 2014). About half of the sample was tested on KTEA-3 Form A and half on KTEA-3 Form B. The total standardization sample (N = 3,842) included students in grades prekindergarten through 12 who ranged in age from 4 to 19 years (M age = 10.4, SD = 3.9). The sample was stratified by grade, sex, race/ethnicity, parent education level, and geographic region, and representative of the U.S. population according to the U.S. Census Bureau’s American Community Survey 2012 1-year period estimates (Ruggles et al., 2010).
The sample that includes the three KTEA-3 subtests that were used within this item error analysis study, LWR, NWD, and Spelling (SP), is a subset of the larger KTEA-3 standardization sample. The LWR, NWD, and SP sample included a stratified KTEA-3 error analysis normative sample (n = 1,400), 108 participants from the larger standardization sample who were not part of the stratified error analysis sample, and 273 participants included in KTEA-3 cognitive validity studies. According to the KTEA-3 Technical & Interpretive Manual, trained scorers classified the errors made by students on the LWR, NWD, and SP subtests, and then the total number of errors by category was calculated for each student. The total sample (n = 1,781) included 911 females and 870 males in grades prekindergarten through 12 (median grade = 6) who ranged in age from 5 to 19 years (M age = 11.5, SD = 3.5). The sample was 56.8% White, 20.2% Hispanic, 13.6% African American, 4.3% Asian, and 5.1% “Other” (e.g., Native American). Parents’ education (used as an estimate of socioeconomic status) was 30.9% with less than 12 years of schooling, 32.7% with high school diplomas or a general education diploma (GED), 33.6% with 1 to 3 years of college or technical school, and 2.8% with 4-year college degrees or more. All participants lived in the United States with 23.1% residing in the midwest, 14.2% in the northeast, 41.4% in the south, and 21.4% in the west. The LWR, NWD, and SP samples closely resemble the percentages in each category as reported by the U.S. Census Bureau’s American Community Survey 2012 1-year period estimates, which are reported in the KTEA-3 Technical & Interpretive Manual (Kaufman et al., 2014).
Study Sample
From the larger KTEA-3 standardization sample, subsets of participants were selected if they met the following patterns of cognitive strengths and weaknesses: High Crystallized Ability paired with Low Processing Speed and Long-Term Retrieval (High Gc, n = 304), Low Crystallized Ability paired with High Processing Speed and Long-Term Retrieval (High Gs/Glr, n = 288), Low OP (n = 150), Low PP (n = 198), and Low PP_OP (n = 37). The rules for selecting PSW cases are explained in the “Procedure” section. All students in the present study were in kindergarten through Grade 12 and between the ages of 5 and 18. Refer to Table 1 for detailed demographic information.
Sample Demographics.
Note. Low OP = Low Orthographic Processing; Low PP = Low Phonological Processing; Low PP_OP = Low Phonological Processing paired with Low Orthographic Processing.
Measures
KTEA-3
The KTEA-3 (Kaufman & Kaufman, 2014) is an individually administered measure of academic achievement for individuals ages 4 through 25 and provides information regarding a student’s academic strengths and weaknesses (Kaufman et al., 2014). The KTEA-3 utilizes a unique error analysis methodology based on the specific subskills measured by a given subtest that includes the categorization of reading and writing errors; norm-referenced cutoff points for determining strong, average, and weak skill areas; and guidance for developing interventions to remediate these errors. The descriptive labels of strength, average, and weakness are called the skill status.
The KTEA-3 Oral Language composite score, which includes the Associational Fluency, Listening Comprehension, and Oral Expression subtests, was used as an estimate of global ability (g). An estimate of Gc was created by averaging standard scores on the KTEA-3 Listening Comprehension and Oral Expression subtests. Gs/Glr scores were generated by taking an average of the standard scores from the Associational Fluency, Letter Naming Facility, and Object Naming Facility subtests. Based on expert review (D. Flanagan, personal communication, August 12, 2015), the KTEA-3 rapid automatized naming subtests, Object Naming Facility and Letter Naming Facility, were classified as Gs/Glr because they not only involve naming automaticity and lexical access (Glr), but they also load most strongly on Gs because they are speeded. In addition to the classification as Gs/Glr, Letter Naming Facility was also considered a measure of orthographic processing. As Bowers and Ishaik (2003) explained, the process underlying rapid automatized naming contributes to the speed of orthographic processing (i.e., decoding the structure of words) by requiring the integration of verbal and visual information.
The present study utilized subtest and composite scores measuring reading, writing, and spelling. Table 2 provides the mean scores and standard deviations for subtests and composite scores of interest by group.
Means and Standard Deviations on Subtest and Composite Scores of Interest.
Note. Low OP = Low Orthographic Processing; Low PP = Low Phonological Processing; Low PP_OP = Low Phonological Processing paired with Low Orthographic Processing.
Table 3 provides the mean error scores for error factors of interest by group. Factor scores were originally created using z scores; however, they were converted to scaled scores with a mean of 10 and a standard deviation of 3 for ease of interpretation. The description of each error factor is provided in the notes of Table 3.
Group Means and Standard Deviations on Error Factors for Each Subtest.
Note. Factors z scores were converted to scaled scores with a M of 10 and SD of 3. Low OP = Low Orthographic Processing; Low PP = Low Phonological Processing; Low PP_OP = Low Phonological Processing paired with Low Orthographic Processing.
Procedure
A multistep process was used to investigate the relationship between students’ cognitive profiles on KTEA-3 cognitive tasks and their corresponding KTEA-3 error scores in reading, writing, and spelling.
Derivation of factor scores
The three skill statuses based on a normative comparison, weakness (bottom 25%), average (middle 50%), and strength (top 25%), were dichotomized into an error score of 0 (weakness) or 1 (average or strength). Students received multiple skill status error scores within each subtest. To facilitate the use of these skill status error scores in further analyses, exploratory factor analysis (EFA) and principal components analysis (PCA) were used to create a reduced error score variable set. The derivation of factor scores for reading and spelling subtests was based on EFA (O’Brien et al., 2017), and the derivation of factor scores for tests of Reading Comprehension, Written Expression, and Phonological Processing was based on PCA (Choi et al., 2017; Hatcher et al., 2017).
To create the factor scores, polychoric correlation matrices were generated for each subtest with the exception of Reading Comprehension and Written Expression subtests. Because each of these subtests has a smaller number of error scores that, in general, are the same across the subtest type (comprehension or expression), one polychoric correlation matrix was generated for each subtest type (comprehension or expression). An EFA using unweighted least square extraction was conducted for each of the subtests excluding Reading Comprehension, Written Expression, and Phonological Processing because these subtests include a small number of error scores; therefore, PCA was used to extract the factors for these subtests.
Regardless of the factor extraction technique, a combination of parallel analysis (PA; Horn, 1965), a visual inspection of the scree plot (Cattell, 1966), and content review of the factor structure were used to determine the number of factors to extract. For the subtests related to the current study, four factors were extracted for the Comprehension and Expression subtests, three factors for LWR, and two factors for the NWD, SP, and Phonological Processing subtests. R Version 3.2.3 was used to generate Bartlett factor scores (Bartlett, 1950) for each of the extracted factors.
PSWs
The next analytic step involved the classification of a subset of 977 students into four distinct groups that represent the students’ patterns of cognitive strengths and weaknesses based on their KTEA-3 profile of scores. The High Gc profile included a Gc score greater than 90 and at least 15 points higher than the Gs/Glr score. The High Gs/Glr profile included a Gc score less than 110 and at least 15 points lower than the Gs/Glr score. Note that based on our criteria, it is technically possible for a student with Gc = 91 to be in the High Gc group and for another student with Gc = 109 to be in the High Gs/Glr (i.e., Low Gc) group. We were not concerned with that possibility because the study was focused on individual patterns of strengths and weaknesses (on measures of Gc and Gs/Glr), not on level of ability. Thus, “High Gc” really should be interpreted as “Relatively High Gc.”
To identify students for the Low PP group, we relied on their standard score on the KTEA-3 Phonological Processing subtest. To choose students for the Low OP group, we used the KTEA-3 Orthographic Processing Composite (composed of the KTEA-3 Spelling, Word Reading Fluency, and Letter Naming Facility subtests). For both groups, we required target scores (on Phonological Processing or Orthographic Processing) that were less than 90 and at least 15 points lower than an estimate of g (the Oral Language composite score, composed of the KTEA-3 Listening Comprehension and Oral Language subtests). We also identified a small group of students who scored low (<90) on the measures of both phonological and orthographic processing, who scored at least 15 points higher on the measure of g (Low PP_OP).
No overlapping cases occurred between the two Gc groups (High Gc vs. Low Gs/Glr and High Gs/Glr vs. Low Gc). After cases were identified for these profiles, overlapping cases were eliminated between the three Orthographic Processing/Phonological Processing profiles and the two Gc/Glr profiles by placing cases in the group with the lowest score (e.g., if a participant qualified for Low PP and High Gs/Glr group, and the Phonological Processing score was lower than Gc, this participant was placed in Low PP group only). In addition, the Low PP, Low OP, and Low PP_OP samples did not overlap.
Data analysis
The final analytic step was to investigate whether students with different cognitive PSWs had different mean error factor scores on the Reading and Writing subtests. Six one-way MANOVAs were conducted with subtest error factor scores as dependent variables and the cognitive PSWs as the independent variable. Prior to conducting the analyses, each set of subtest factor scores was examined for univariate normality issues and outliers. Any extreme cases were analyzed to verify their impact on the distributional properties of each subtest. Using a criteria of |2| skewness and |6| kurtosis (Lix, Keselman, & Keselman, 1996), no violations of normality were observed. To examine the assumption of homogeneity of within-group covariance matrixes, a two-step analysis process was utilized (Huberty & Petoskey, 2000). First, for each analysis, the Box F test was calculated. For each subtest, the Box F test was significant. However, as noted by Huberty and Petoskey (2000), the Box F test is an extremely powerful test. Therefore, as a follow-up analysis, the natural log of the determinant of the covariance matrix for each level of the independent variable was compared with the natural log determinant of the pooled matrix (Huberty & Petoskey, 2000; Olejnik, 2010) for each subtest. In the judgment of the researchers, the differences were relatively close, with the largest difference between a given group and the pooled natural log determinant equal to −2.00.
Results
Differences Between PSW and KTEA-3 Reading Error Factors
In this analysis, we examined whether there are differences among the mean error factor scores obtained by different cognitive profiles or PSWs on the KTEA-3 reading domain. To examine this hypothesis, two one-way MANOVAs were conducted with the PSW serving as the independent variables and the error factor scores for each of the reading subtests (e.g., LWR, NWD) serving as the dependent variables. Because the PCA for comprehension produced two error factor scores related to reading, a third one-way MANOVA was conducted with these two scores used as dependent variables. Mean error scores and standard deviations on each error factor by cognitive groups are listed in Table 3, and MANOVA summary results for factor analysis by error factors are listed in Table 4. All error factors were named and interpreted based on expert reviews (D. Kilpatrick, personal communication, April 2, 2016; N. Mather, personal communication, March 26, 2016; J. Willis & R. Dumont, personal communication, March 26, 2016).
Factor Analysis Results Summary.
Note. Mean difference equals the mean for the first group listed minus the mean for the second group (e.g., in the first column of differences, positive values indicate higher scores for the High Gc group than the High Gs/Glr group). Differences are expressed in standard score units. Low PP = Low Phonological Processing; Low OP = Low Orthographic Processing.
p < .05. **p < .01. ***p < .001.
LWR
Three error factor scores were generated for the LWR subtest. The three factors were named Contextual Vowel Pronunciation, Intermediate Letter–Sound Knowledge, and Consonant Pattern Knowledge (D. Kilpatrick, personal communication, April 2, 2016). Contextual Vowel Pronunciation is knowledge and skills that influence the correct pronunciation of vowels within specific contextual elements of phonetically regular and irregular words such as open and closed syllables, stressed or nonstressed syllables, and silent vowels. Intermediate Letter–Sound Knowledge is the knowledge of consonant and vowel digraph blends, which is acquired after the mastery of single letter–sound awareness. Consonant Pattern Knowledge is the ability to correctly pronounce digraphs when knowledge of patterns of letters and orthographic context is required.
Across these three dependent variables, students with different PSW profiles exhibited different mean error scores on the dependent variables (see Table 3), Wilks’s λ of .806, F(900.63) = 9.29, p < .001. Furthermore, 19% (η2 = 1 − λ) of the variation on these dependent variables can be explained by the differences in the PSW profiles.
Thus, the overall model indicated that error scores on LWR were different for students with different PSW profiles. Subsequent ANOVAs revealed significant F values for each of the three LWR error factors (Table 4). Bonferroni-corrected comparisons (p < .05) between the different PSW groups (also shown in Table 4) show that the groups tend to make different kinds of errors on the LWR factors. For example, the High Gc group earned significantly higher error scores (i.e., made fewer errors) than the High Gs/Glr group on the Contextual Vowel Pronunciation and Consonant Pattern Knowledge factors, but not on the Intermediate Letter–Sound Knowledge. Similarly, the High Gs/Glr group made significantly fewer errors than the Low OP group on two of the three factors but did not differ significantly from the Low PP group on any of the three LWR error factors.
NWD
Two error factor scores were generated for the NWD subtest. The two factors were named Letter–Sound Knowledge and Basic Phonic Decoding (D. Kilpatrick, personal communication, April 2, 2016). Letter–Sound Knowledge is the knowledge of letter–sound correspondence, which indicates awareness of the relationship between specific letters and sounds. Basic Phonic Decoding is the ability to produce, discriminate, and manipulate the phonological structure of language.
Students with different PSW profiles exhibited different mean error scores on the dependent variables: Wilks’s λ of .876, F(632) = 7.19, p < .001. Furthermore, 12% (η2 = 1 − λ) of the variation in the dependent variables can be explained by the differences in the PSW profiles. The significant F values for both error scores in the subsequent ANOVAs and the post hoc pairwise comparisons are shown in Table 4. As for NWD, the differences between the factor scores for the various PSW samples are notable. The High Gc sample made significantly fewer errors than both the High Gs/Glr and Low PP samples on the Letter–Sound Knowledge factor, but did not differ significantly from either of these groups on the Basic Phonic Decoding factor. The high Gs/Glr sample made fewer errors than the Low OP sample on both error factors (not surprising in view of the fact that poor word reading fluency and spelling were criteria for identifying the Low OP sample), but did not differ significantly from the Low PP sample on either factor.
Reading comprehension
Two error factor scores were generated for the comprehension tasks related to reading: Expository–Literal and Narrative–Inferential (N. Mather, personal communication, March 26, 2016; J. Willis & R. Dumont, personal communication, March 26, 2016). Expository–Literal is the ability to comprehend concrete and factual concepts from written information. Narrative–Inferential is the ability to comprehend inferred ideas and details as well as context cues from auditory information.
Students with different PSW profiles exhibited different mean error scores on these factors (see Table 3), Wilks’s λ of .913, F(1650) = 9.6, p < .001. Furthermore, 9% (η2 = 1 − λ) of the variation in these error factors was explained by the differences in the PSW profiles. Table 4 shows the results of the ANOVAs (significant for both reading comprehension factors) and the Bonferroni comparisons. Overall, the High Gc group made significantly fewer errors than the High Gs/Glr group and the Low OP group on both factors, but did not differ from the Low PP group on either factor. Interestingly, performing poorly on phonological processing affected errors on the “reading decoding” factors (the High Gc group outscored the Low PP sample on four of the five error factors for LWR and NWD) but not on the reading comprehension factors (see Table 4).
Differences between PSW and KTEA-3 written language error factors
We also examined whether there are differences among the mean error factor scores obtained by different PSW profiles in the KTEA-3 written language domain. To examine this hypothesis, two one-way MANOVAs were conducted with the PSW profile serving as the independent variables and the error factor scores for each of the written language subtests (Spelling, Written Expression) serving as the dependent variables.
Spelling
Sound to Letter Mapping and Phonological Awareness were the two error factors generated for the Spelling subtest (D. Kilpatrick, personal communication, April 2, 2016; J. Willis & R. Dumont, personal communication, March 26, 2016). Sound to Letter Mapping is the ability to determine the type of letter that appropriately represents the sounds in the pronunciation of a given word. Students with different PSW profiles exhibited different mean error scores on these factors, Wilks’s λ of .861, F(582) = 7.54, p < .001. Also, 14% (η2 = 1 − λ) of the variation in the dependent variables can be explained by the differences in the profiles.
Table 4 indicates that the separate ANOVAs yielded significant F values for both factors. However, the PSW samples tended to differ significantly on only one of the two factors. For example, the High Gc group made fewer errors than both the High Gs/Glr and Low OP samples on the Sound to Letter Mapping factor, but not on the Phonological Awareness factor.
Written expression
General Written Expression and Mechanics are the two error factors that were generated for the Written Expression subtest (J. Willis & R. Dumont, personal communication, March 26, 2016). General Written Expression is defined as the ability to communicate effectively in writing. Mechanics are abilities unique to written language such as punctuation, capitalization, and paragraphs. Different PSW profiles exhibited different mean error scores on these error factors, Wilks’s λ of .894, F(1800) = 13.02, p < .001, with 11% (η2 = 1 − λ) of their variation being explained by the differences in the PSWs. Both factors yielded significant F values in the subsequent ANOVAs (Table 4). The post hoc comparisons shown in Table 4 indicate that the High Gc group made fewer writing errors—both on the General Written Expression factor and on the Mechanics factor—than the High Gs/Glr and the Low OP sample, but did not differ significantly from the Low PP sample. However, when Low PP is combined with Low Orthographic Processing (Low PP_OP), that sample scored significantly lower than all other PSW groups on both error factors. These data are not shown in Table 4 because the Low PP_OP group was only large enough (n = 37; see Table 1) to be included in the reading comprehension and written expression analyses. However, Table 3 provides mean error factor scores for this sample on the two reading comprehension and two written expression factors. The sample earned low mean scores on both General Written Expression and Mechanics (about 8) but scored in the average range (about 10) on both reading comprehension factors.
Phonological Processing
Basic Phonological Awareness (sound awareness: the ability to identify and distinguish sounds in words) and Advanced Phonological Processing (phonological skills that allow an individual to hold phonological information temporarily and decompose or manipulate it) are the two error factors identified for the Phonological Processing subtest (D. Kilpatrick, personal communication, April 2, 2016).
Students with different PSW profiles exhibited different mean error scores on these error factors, Wilks’s λ of .903, F(1296) = 17.02, p < .001, and explained 10% (η2 = 1 − λ) of the variation. Separate ANOVAs indicated significance for both the basic and advanced error factors, with the High Gc group outscoring the High Gs/Glr and Low OP group on both factors (Table 4). The Low PP sample was eliminated from this analysis because the students’ standard score on the KTEA-3 Phonological Processing subtest was used to identify the sample.
Discussion
The results of the current study support the hypothesis that different cognitive PSWs predict different error patterns among various reading and writing factors. The High Gc group outperformed the other groups on the phonological processing, word reading, and decoding error factors with few exceptions. First, no significant differences were noted between the High Gc and High Gs/Glr group on the LWR Intermediate Letter–Sound Knowledge error factor. One possible explanation for this exception is that the errors might be due to instructional effects (D. Kilpatrick, personal communication, April 2, 2016). Second, the High Gc group did not differ significantly from the High Gs/Glr or Low PP groups on the NWD Basic Phonic Decoding error factor. Interestingly, the High Gs/Glr and Low PP groups had similar LWR error scores, and the Low OP group scored significantly lower than the High Gs/Glr group on two of the three error factors. Overall, these results suggest that Gc is an important broad ability for facilitating basic reading skills. The High Gc group made fewer word reading errors; however, the High Gs/Glr group did not make more errors compared with the other groups. The High Gc group also outperformed the High Gs/Glr and Low OP groups on all reading comprehension and Written Expression error factors, which suggests that strong crystallized ability facilitates performance in these academic areas as well.
Excluding the small Low PP_OP sample, the lowest error score observed was for the Low OP group on the LWR Contextual Vowel Pronunciation factor (see Table 3). This finding suggests that students with poor letter naming facility may have particular difficulty with vowels and affixes during word reading. The Low OP group also scored significantly lower than all other groups on the Spelling Sound to Letter Mapping error factor, but that finding is contaminated by the fact that low scores on the Spelling subtest were used as part of the definition of Low OP in this study. Poor error scores were also observed for the Low PP_OP group on the two written expression error factors. This combination of processing weaknesses in both phonological and orthographic processing has an apparent extreme effect on students’ writing mechanics and general written expression skills, but does not seem to have much of an impact on errors in reading comprehension (Table 3).
Table 4 provides the best overview of the different kinds of error patterns observed for the different PSW groups. Clearly, the High Gc sample made significantly fewer errors on virtually every error factor when compared with students with High Gs/Glr. This finding is not surprising given that the High Gc group (with Low Gs/Glr) was—by definition—more intelligent than the High Gs/Glr group (which, correspondingly, had Low Gc). Nevertheless, despite the difference in g, the High Gc group did not differ from the High Gs/Glr sample on the LWR Intermediate Letter–Sound Knowledge factor, the NWD Basic Phonic Decoding factor, or the Phonological Awareness factor (on the Spelling subtest). Furthermore, effect sizes ranged from small on several reading decoding error factors to moderate (differences of .51-.59) on a diverse set of error factors such as Advanced Phonological Processing, General Written Expression, and Expository–Literal reading comprehension (Table 4). Also of interest is the finding that the High Gc group made significantly fewer errors than the Low PP group on only five of the 11 factors summarized in Table 4, and the fact that the Low PP sample significantly outscored the High Gs/Glr sample on the reading comprehension Expository–Literal factor and on the General Written Expression factor.
The results of the present study provide empirical data to support the link between specific cognitive PSWs and academic achievement, notably the kinds of errors that students make on tests of Reading Decoding, Reading Comprehension, spelling, and Written Expression. Much of the previous empirical literature in support of using cognitive PSWs to inform interventions is derived from correlational studies—typically, the correlations between assorted CHC broad and narrow abilities and specific areas of academic achievement (e.g., Evans et al., 2002; Flanagan et al., 2013; Floyd et al., 2008; McGeown et al., 2013). Numerous researchers, for example, have found that Gc is an especially good predictor of reading achievement, notably reading comprehension (Ismailer, 2015; McGrew et al., 1997) as well as decoding and word recognition skills (McGrew & Wendling, 2010). Others have listed an array of abilities that are associated with written expression, notably Glr, Gs, Gc, Gsm, and Gf (Floyd et al., 2008). This line of research is vital because it provides empirical data to support the importance of a variety of CHC abilities to success in distinct aspects of academic achievement. The present research extends these findings by studying CHC abilities in relation to each other (e.g., High Gs/Glr paired with Low Gc), and essential research component to support the use of a PSW approach during the diagnostic and intervention process. Moreover, the present study provides data based on individual students’ cognitive profiles, not just on aggregated data from groups of children and adolescents.
Burns (2016) and others have challenged the advocates of PSW to show these doubters the data. This study, when combined with data from a related PSW study that examined math errors (Koriakin et al., 2017), presents data to support the notion that students with different cognitive PSWs differ in the kinds of errors they make on tests of academic achievement. Naturally, further research is needed to extend these empirical findings and explore implications for instruction and educational interventions.
Limitations and Future Directions
These results must be interpreted in light of the limitations of the present study, which may direct future research in this area. First, the conclusions of our study are based on a wide grade range, and the sample size did not allow developmental effects to be analyzed. For example, some of these cognitive profiles, such as Low PP, may have had a greater or different impact on reading errors at lower grade levels relative to higher grades. Similarly, previous research suggests that Gc is increasingly important in reading as age increases; therefore, our results might be different if broken into smaller age groups. Although the present samples were large, it was necessary to analyze all age groups together to ensure appropriate power in the statistical analyses that were conducted. Future research should aim to compare whether error patterns differ between these cognitive profiles at different age/grade levels.
Another limitation is that the small sample size for the Low PP_OP group prevented comparisons with the other groups on LWR, NWD, and Spelling error factors. Future research should explore the academic implications for students with weaknesses in both phonological and orthographic processing. The present findings with the small sample suggested that this combination of processing weaknesses may affect written expression more than reading comprehension. This hypothesis should be tested with a larger sample. In addition, the relations between poor letter naming facility and word reading difficulties, and the tendency that students in Low OP groups have difficulty with basic spelling skills, are worthy of further exploration. It benefits educators to understand developmental trends and find proper educational interventions for poor readers.
Another limitation of this study is that the Low OP group was identified by their low scores on three KTEA-3 subtests, Spelling, Word Reading Fluency, and Letter Naming Facility. Therefore, this sample (and the Low PP_OP sample) would be expected to make more reading and, especially, spelling errors than other samples. The reason this group was included in the study was because it was the pattern of differences on the error factors that was of greatest interest, not simply which group made more errors than another. From Table 4, it is evident that the Low OP group made significantly more errors than the High Gc group on the Spelling factor Sound to Letter Mapping (difference = .88), but not on the Spelling factor Phonological Awareness (difference = .21). That pattern is of interest, although future studies should not use measures of reading or spelling to identify students with Low OP.
Footnotes
Acknowledgements
The authors wish to thank NCS Pearson for providing the standardization and validation data for the Kaufman Test of Educational Achievement–Third Edition (KTEA-3). Copyrights by NCS Pearson, Inc. used with permission. They are also grateful to Nadeen Kaufman for her contributions to this study.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Alan and Nadeen Kaufman received royalties on the sale of the KTEA-3.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.