Abstract
Test Description
The Comprehensive Test of Phonological Processing–Second Edition (CTOPP-2; Wagner, Torgesen, Rashotte, & Pearson, 2013) is a norm-referenced test that measures phonological processing skills related to reading for individuals aged 4 to 24. According to its authors, the CTOPP-2 may be used to identify individuals who are markedly below their peers in phonological abilities, determine individuals’ relative strengths and weaknesses in phonological abilities, document individuals’ progress in specific intervention programs, and serve as a research tool in studies investigating phonological processing. This second edition differs from its predecessor in that it has new norms, ceilings have been increased by adding more difficult items, and a new phonological awareness subtest has been added (i.e., Phoneme Isolation). Floor effects present in the 5- and 6-year-old version of the first edition were successfully addressed by adding easier items, extending the use of the new edition to children as young as 4. However, floor effects were apparent on selected subtests for children in the lowest end of the extended age range (i.e., 4-year-olds). Measures of symbolic and nonsymbolic naming are now included in the 4- to 6-year-old form.
General Description
The structure of the CTOPP-2 was based on a theoretical model of phonological processing developed by Wagner and colleagues (e.g., Wagner & McBride-Chang, 1996; Wagner, Torgesen, & Rashotte, 1994) comprised of phonological awareness, phonological memory, and rapid naming. The first edition of the CTOPP was appropriate to administer to children as young as 5 years of age. However, the CTOPP-2 has broadened its age range and now includes test versions for both 4- to 6-year-olds and 7- to 24-year-olds. The 4- to 6-year-old version contains nine core subtests and one supplemental subtest, whereas the 7- to 24-year-old version contains seven core subtests and two supplemental tests. Each of the constructs in the above theoretical model is represented by one or more composite scores. For 4- to 6-year-olds, the Phonological Awareness (PA) Composite is comprised of the Elision, Blending Words, and Sound Matching subtests; however, for 7- to 24-year-olds, the Sound Matching subtest is replaced by Phoneme Isolation. The Phonological Memory (PM) Composite is comprised of the Memory for Digits and Nonword Repetition subtests for all age ranges. The Rapid Symbolic Naming (RSN) Composite consists of the Rapid Digit Naming and Rapid Color Naming subtests for all age ranges. A fourth composite is available for each age group. For the 4- to 6-year-olds, the Rapid Nonsymbolic Naming (RNSN) Composite consists of the Rapid Color Naming and Rapid Object Naming subtests. The 7- to 24-year-olds may be administered an Alternate Phonological Awareness (APA) Composite, a supplemental composite comprised of the Blending Nonwords and Segmenting Nonwords subtests. Descriptions of the CTOPP-2 subtests appear below:
Elision (PA, all ages): The examinee listens to a word, repeats that word, and then is asked to say the word without a designated sound (e.g., “Say cat without the /c/”).
Blending Words (PA, all ages): The examinee listens to a series of audio-recorded separate sounds and then is asked to blend the sounds together to make a whole word (e.g., “What do these sounds make: b-oi?” The correct response is “boy.”).
Sound Matching (PA, 4-6 years): The examiner says a word, pauses, and then says three other words while pointing to drawings that represent all four words. For the first 13 items, the examinee is asked to point to the picture that has the same initial sound as the word the examiner provides (e.g., “Which word starts with the same sound as pig? Pan, cat, or dog?” The correct response is “pan.”). For the remaining 13 items, the examinee is asked to point to the picture that has the same final sound as the word the examiner provides.
Phoneme Isolation (PA, 7- to 24-year-olds): The examinee is asked to identify the first and last sounds of words (e.g., “what is the first sound in the word mop?”). Subsequent items require the examinee to identify middle sounds and then to identify other parts of words made up of more sounds (e.g., “what is the third sound in the word stop?”).
Memory for Digits (PM, all ages): The examinee listens to an audio recording of numbers and is asked to repeat them.
Nonword Repetition (PM, all ages): The examinee listens to an audio recording of nonwords and is asked to repeat them.
Rapid Digit Naming (RSN, all ages): The examinee is asked to quickly name the numbers on each row from left to right until all the numbers have been named.
Rapid Letter Naming (RSN, all ages): The examinee is asked to quickly name the letters on each row from left to right until all the letters have been named.
Rapid Color Naming (RNSN, 4- to 6-year-olds): The examinee is asked to quickly name the colors on each row from left to right until all the colors have been named.
Rapid Object Naming (RNSN, 4- to 6-year-olds): The examinee is asked to quickly name drawings of objects (e.g., star, key) on each row from left to right until all the objects have been named.
Blending Nonwords (APA, 7- to 24-year-olds): The examinee listens to a series of audio-recorded separate sounds and then is asked to blend the sounds together to make a nonword.
Segmenting Nonwords (APA, 7- to 24-year-olds): The examinee listens to an audio-recorded nonword, repeats the nonword, and then says it one sound at a time.
The CTOPP-2 represents an important assessment tool for school psychologists, clinical psychologists, speech and language pathologists, reading specialists, and other educational professionals. Of particular importance is the use of the CTOPP-2 for identifying children who are at risk of future reading problems. Consensus exists that phonological processing represents a core deficit for children who struggle with learning to read, including those identified as reading disabled (Melby-Lervåg, Lyster, & Hulme, 2012). Floor problems associated with academic achievement measures in reading often complicate the process of identifying reading disabilities in young students. However, the CTOPP-2 may be used to identify kindergarten and first graders with phonological processing deficiencies who may be at risk of developing future reading difficulties, thereby facilitating the provision of early intervention or prevention services. Some states (e.g., Georgia) require evidence of a deficit in a basic psychological process contributing to the area of academic weakness when determining eligibility for a specific learning disability. Whether or not such information is required by state eligibility criteria, assessment of an individual’s phonological processing skills may also aid case conceptualization and guide intervention efforts by providing a potential explanation for the observed reading difficulties. In sum, assessment of phonological processing is an important aspect of a comprehensive psychoeducational assessment when a reading disability is suspected.
Once identified as at risk or as reading disabled, the CTOPP-2 may be used to monitor a student’s response to intervention services. In particular, effective interventions for promoting early reading skills should promote growth in phonological awareness (rapid naming speed and phonological memory appear less amenable to training). The CTOPP-2 may also provide valuable information regarding an individual’s relative strengths and weaknesses in phonological processing, contrasting features of phonological awareness, phonological memory, and rapid naming skills. Finally, the wide age span and comprehensiveness of the CTOPP-2 make it unique as a clinical or research tool. Consequently, the CTOPP-2 may be used to address a broad array of research questions.
Test Construction/Format
Overall, the CTOPP-2 test materials are well designed and user friendly. Abbreviated instructions along with practice items and rules for establishing ceilings are provided in the examiner record booklets, making the examiner’s manual unnecessary during test administration. Few additional items (e.g., a CD player) are needed for the relatively straightforward and quick CTOPP-2 administration. The examiner’s manual is written in a readily understandable manner and provides clear and specific administration and scoring instructions for each subtest and composite. Clear information is also provided regarding the technical adequacy of this instrument (i.e., reliability, validity, normative data, and methods for reporting scores).
Administration
Administration of the entire core battery for each age group takes approximately 30 min. The CTOPP-2 was designed to be individually administered by examiners with extensive formal training in assessment. The CTOPP-2 should not be administered to children younger than 4 years or adults older than 24 years 11 months. Materials needed for administration include the picture book, examiner record booklet (separate booklets are available for ages 4 to 6 and for ages 7 to 24), the CD, a CD player, a stopwatch, and a sharpened pencil with an eraser.
The examiner’s manual indicates that the administration of the CTOPP-2 is a rather challenging experience for both examiner and examinee because many of the subtests include tasks that may be unfamiliar. For example, it is unlikely that individuals have experience listening to a series of separate sounds and then combining them together to create a nonword, as is required in the Blending Nonwords subtest. Feedback is given to the examinee during the administration of the initial items to ensure that the examinee has mastered the task. The administration of all untimed subtests (i.e., Elision, Blending Words, Sound Matching, Phoneme Isolation, Blending Nonwords, Segmenting Nonwords, Memory for Digits, and Nonword Repetition) begins with Item 1 for all examinees, regardless of age. The examinee receives 1 point for each correct answer and 0 points for each incorrect answer until the ceiling is reached (3 consecutive incorrect responses). For untimed subtests, testing is discontinued and no score is recorded if the examinee cannot answer one of the first three items correctly after receiving feedback from the examiner. For the timed subtests (i.e., Rapid Digit Naming, Rapid Letter Naming, Rapid Object Naming, and Rapid Color Naming), the examinee’s score is dependent on the number of seconds it takes to complete the task. The examinee is prompted to go to the next item if he or she hesitates for more than 2 s on any item. On the timed subtests, testing is discontinued and no score is recorded if the examinee cannot name all of the items (i.e., the letters, numbers, colors, or objects) correctly after error correction during the practice session or if the examinee makes four or more naming errors during administration of the subtest. Mastery of items during the practice session is essential, as the purpose of the timed subtests is to assess the speed with which the examinee can name the symbolic or nonsymbolic items rather than their knowledge of those items.
Test Scores and Interpretation
The CTOPP-2 yields six types of normative scores: age equivalents, grade equivalents, percentile ranks, subtest scaled scores, composite indexes, and developmental scores. The subtest scaled scores have a mean of 10 and a standard deviation of 3. The composite score indexes have a mean of 100 and a standard deviation of 15. A change from the first edition to the CTOPP-2 is a new Rasch-based developmental score available for the untimed subtests, which provides an equal-interval scale for examining changes in performance over time. The developmental scores associated with each raw score have approximately equal intervals, suggesting that the scaling was successful. Although age and grade equivalents are available, which show the relative standing of individuals’ scores, the test authors recommend that examiners use scaled scores, composite scores, and percentile ranks when reporting results, which is consistent with professional standards. In addition, the authors provide clear descriptions for determining statistically significant discrepancies at a .05 level of confidence between subtest and composite scores. The authors encourage the use of composite scores for making decisions regarding diagnosis and placement, but subtest scores may be used to determine individuals’ strengths and weaknesses. According to the CTOPP-2 examiner’s manual, test scores may contribute to selecting long-term educational goals, but scores should not be used for planning daily instruction programs.
Technical Adequacy
Standardization
The CTOPP-2 norming sample included 1,900 individuals aged 4 to 24 years in six states. The characteristics of the sample were compared with U.S. census data from 2010, and according to the examiner’s manual, the normative sample is representative of the nation as a whole based on this comparison. To further demonstrate how the sample was representative of the nation as a whole, the demographic data were stratified by age. The standardization process appeared to generate a nationally representative norming sample suitable for general use in the United States.
Scale/Item Characteristics
It should be noted that several of the CTOPP-2 subtests have insufficient floors during the earliest age levels and insufficient ceilings during the later age levels. Test items should measure performance at least two standard deviations from the mean for each age range to allow for differentiation at the lowest and highest levels of functioning (Bracken, 1987). Specifically, the subtests comprising the PA Composite and APA Composite demonstrate insufficient floors for very young children (i.e., less than age 5). For example, a child aged 4 years 0 months who obtains a raw score of 1 on each subtest will earn a scaled score of 9 on the Elisions subtest and an 8 on the Blending Words, Sound Matching, and Blending Nonwords subtests. Additional items were added to these subtests during the revision process of this second edition to address floor effects, but the age range for administration of this test was also lowered from 5 years 0 months to 4 years 0 months. Items from the Elision and Phoneme Isolation subtests have insufficient ceilings for older individuals. For example, an individual aged 15 years 0 months through 24 years 0 months who answers all possible items correctly for each subtest will earn a 13 on the Elision subtest and a 15 on the Phoneme Isolation subtest. However, ceiling effects for these subtests may be expected as the development of PA asymptotes with age.
Reliability
Reliability data are provided for content sampling (internal consistency), time sampling (test–retest), and inter-rater agreement. Generally, coefficients of .80 or higher are considered adequate with values exceeding .90 to be desirable. With regard to content sampling, coefficient alphas were computed for the untimed subtests, and alternate-form (immediate administration) reliability correlations were computed for the timed subtests. Average internal consistency coefficients presented for the CTOPP-2 exceed the minimal standard of .80 for all subtests, except Nonword Repetition, with an average alpha of .77. The average internal consistency coefficients for the composites were all .85 or higher. The average alternate-form reliability coefficients met or exceeded .85 for the timed subtests and exceeded .90 for the timed composite scores. Furthermore, coefficient alphas and alternate-form (immediate administration) reliability coefficients for the CTOPP-2 were strong across all subgroups included in the sample (i.e., race/ethnicity, gender, learning disabled). Test–retest correlations for the core subtests ranged from .75 to .92, those for the composites ranged from .76 to .86, and the supplemental subtests were .73 and .75. Inter-rater reliability coefficients ranged from .96 to .98 for a subset of the standardization sample that was scored by two independent raters affiliated with Pro-Ed, indicating that testers generally agreed in their ratings on the CTOPP-2.
Validity Evidence
The examiner’s manual provides a detailed description of the three types of validity evidence for interpreting the scores produced by the CTOPP-2, including content-description validity, criterion-prediction validity, and construct-identification validity. The manual offers three sources of content-description validity evidence, including the following: (a) the rationale for the formats and items for each subtest, (b) the results of conventional item analysis used during test construction to identify good versus bad test items, and (c) the results of differential item functioning analysis that were used to demonstrate the absence of bias in test items. The evidence provided in support of the content-description validity of the CTOPP-2 appears to be adequately developed.
With regard to concurrent criterion-prediction validity, correlations between the CTOPP or CTOPP-2 subtests and composites and a variety of other tests assessing phonological processing were reviewed across a total of 33 studies. Twenty-eight studies examining the CTOPP were drawn from the research literature and 5 studies examining the CTOPP-2 were conducted by the test publisher. The 28 studies conducted by independent researchers on the CTOPP included small to large samples (i.e., N = 28 to N = 276) comprised of normal, at risk, or reading disabled individuals ranging from 3 to 43 years of age, and represented a variety of racial and ethnic groups and geographical regions within the United States or other countries. The 5 studies conducted by Wagner and colleagues included modest to large sample sizes (i.e., N = 70 to N = 384) of normal readers representing a wide age range (i.e., 4-20 years of age) and geographical regions (e.g., California, North Dakota, New York, and Florida), who were relatively diverse in terms of racial or ethnic group membership. The averaged coefficients for the CTOPP or CTOPP-2 subtests with the phonological criterion tests range from .49 to .84; those for the composites range from .65 to .76. The manual provides ample evidence regarding the CTOPP and CTOPP-2’s concurrent criterion-prediction validity.
Six sources of evidence were provided regarding the construct-identification validity of the CTOPP-2. First, a confirmatory factor analysis was conducted to confirm the relationship of the CTOPP-2 subtests to the constructs of interest. This analysis was especially appropriate given that the CTOPP and CTOPP-2 were based on a theoretical model of phonological processing depicting three distinct but related abilities (e.g., phonological awareness, phonological memory, and rapid symbolic naming). This model was found to demonstrate excellent fit for the 7- to 24-year-old norming sample. The 4- to 6-year-old model included a fourth factor, rapid nonsymbolic naming, and demonstrated comparable fit. Second, correlation coefficients between the CTOPP and CTOPP-2 composite scores and a variety of reading criterion tests were provided. The average correlation coefficient for the reported criterion reading measures to phonological awareness was .48, those to phonological memory was .34, and those to rapid symbolic naming was .45. The authors noted that these moderate relations between phonological processing and reading skills are comparable with those reported by meta-analyses in the research literature (e.g., Hammill, 2004).
Third, all subtests showed considerable correlations with age during the early developmental period, with the exception of Segmenting Nonwords. The authors suggested that the small correlation with age for the Segmenting Nonwords subtest might be attributable to the difficulty of this task for young children. Fourth, CTOPP and CTOPP-2 scores were found to differentiate between normative groups and those who have reading and/or learning difficulties. African American individuals were found to perform slightly below average on almost all tests of phonological awareness, although their performance was comparable with other ethnic/racial groups on measures of memory and rapid naming speed. African Americans are disproportionately represented in low socioeconomic status groups, and may have home language experiences that differ from standard English environments. It was argued that these differences reflect that phonological awareness is more amenable to environmental, language, or training experiences than are memory and rapid naming skills. Fifth, across intervention studies designed to improve phonological processing and reading, the average effect size was .70 for subtests and .60 for composites, suggesting that the CTOPP-2 may be a useful measure of response to intervention. Finally, correlations between the items within each subtest and the total score of that subtest demonstrated adequate evidence of item validity.
Commentary
Overall, the CTOPP-2 appears to be a reliable and valid measure of phonological processing for individuals aged 4 to 24. However, due to apparent floor and ceiling problems, some caution may be warranted in interpreting the scores for very young children or for individuals nearing the upper limit of the age range on selected phonological awareness subtests. Overall, the test materials are well constructed, making the CTOPP-2 easy to administer, score, and interpret. The test manual was not organized around the purported uses of the test, but ample validity evidence for interpreting the scores produced by the CTOPP-2 was provided. To our knowledge, the CTOPP-2 is the only standardized and norm-referenced, comprehensive test of phonological processing that covers such a broad age range. Given the importance of phonological variables to the development of basic reading skills (Melby-Lervåg et al., 2012), the CTOPP-2 is likely to yield clinically and educationally relevant information. In particular, the CTOPP-2 may be useful to school psychologists and other educational professionals who wish to identify young children who are at risk of reading difficulties to facilitate the provision of early intervention services, to evaluate the effectiveness of intervention efforts, and to provide a comprehensive depiction of the phonological processing skills that underlie reading. In sum, the CTOPP-2 appears to be a useful assessment tool for psychologists, educators, and researchers who wish to gain in depth information regarding an individual’s skills in phonological awareness, phonological memory, and rapid naming.