Academic Achievement Test Use and Assessment Practices: A National Survey of Special Education Administrators

Abstract

This study examined current academic achievement test use and assessment practices in the United States. Special education administrators from 725 US school districts were surveyed using an online platform to determine current trends in the use of norm-referenced academic achievement tests (NRTs) and curriculum-based measures (CBMs). Administrators reported that school psychologists most commonly administered the NRTs used to help determine special education eligibility, although special education teachers also fill this role frequently, particularly in Western states. General and special educators are the most likely staff to administer CBMs. A detailed accounting of academic achievement assessment practices is discussed including which tests are used and the differences in assessment roles by region and district locale. Implications of these findings for the training of educators and for research are considered.

Keywords

academic achievement testing assessment roles eligibility

Assessment has long been a cornerstone of the special education system. Individualized education evaluations are typically conducted to determine if a student has a disability and to delineate the services they should receive (Individuals with Disabilities Education Act [IDEA], 2004). Eligibility determination is a multidisciplinary team process, involving a special educator, general educator, evaluator (e.g., school psychologist), parent, and school administrator (Heward et al., 2017). Various team members may be responsible for collecting, interpreting, and using assessment results to inform eligibility decisions. In many districts, when measures of cognitive ability, academic achievement, and socio-emotional functioning are required, school psychologists administer and interpret those tests (Benson et al., 2019; Walcott & Hyson, 2018) due to their specialized training with standardized assessment (Lockwood & Farmer, 2019). However, special education teachers or other team members may be qualified to engage in academic achievement assessment (Qualifications Policy, n.d.). While a team approach to identification is required by IDEA, there is a paucity of literature examining who administers the academic achievement tests used when determining special education eligibility across the United States. Additionally, there are no data regarding which academic achievement measures are used across professionals. These gaps in the literature are glaring as these assessments are conducted to help make high-stakes decisions such as determining the presence of a disability and whether students will have access to the services and protections provided under the Americans with Disabilities Act (1990) and IDEA (2004) (Guerriero et al., 2020). In order to fill these gaps, research examining academic achievement test use and assessment practices is needed to determine what tests are being used, and who administers these measures.

Roles

School Psychologists

Surveys of school psychologists dating back to the early 1980s indicate that assessment is one of the primary roles of these professionals. The first such study of 274 school psychologists by Goh et al., (1981) found that 47% of their time was devoted to testing-related activities, and this trend was consistent across geographic regions. Goh et al. also found that roughly a quarter of a school psychologist’s assessment time is spent administering academic achievement tests. Studies over the past four decades (e.g., Hutton et al., 1992; Stinnett et al., 1994; Walcott & Hyson, 2018) have confirmed Goh et al.’s findings, suggesting that school psychologists consistently spend roughly half of their time assessing constructs such as cognitive ability, academic achievement, and behavior. Most recently, in 2017, 85% of school psychologists reported administering at least one academic achievement test in the past year. While school psychologists are frequently involved in academic assessment, less than 30% reported administering curriculum-based measures (CBMs) during the same time frame (Benson et al., 2019); instead, their focus appears to be on norm-referenced testing. For this reason, Benson et al. hypothesized that other school personnel such as teachers and support staff may also be administering CBMs and suggested that further research is needed to determine who fills this role.

Special Education Teachers

With the exception of school psychologists, little is known regarding the roles that other team members play in academic achievement assessment for special education eligibility across the United States. However, two likely test administrators include special education teachers and educational diagnosticians. The Council for Exceptional Children (CEC) notes that special education teachers must be knowledgeable in using, interpreting, and discussing test data (McLeskey, 2017). Additionally, special education teachers are called to “understand the different types of assessment tools available to them, and how to use those tools and the information generated from them to help the educational team design, implement, evaluate, and revise programs that meet the individual needs of students with disabilities” (McLeskey, 2017, p. 44). While the CEC acknowledges the need for assessment literacy, we were unable to find any literature describing the role that special education teachers typically fill in the administration of these measures, what measures they use, and how test administration roles may vary by state in the United States. However, Lockwood et al. (2019) suggested that it is unlikely that special education teachers are tasked with engaging in norm-referenced academic achievement test (NRT) administration more than six times per year. Additionally, these authors argued that while little is known regarding the training that special education teachers receive in NRT administration, they likely do not have as extensive of training in assessment as other professionals (i.e., school psychologists).

Educational Diagnosticians

Educational diagnosticians also act as test administrators in multiple states. Educational diagnosticians are professionals who have several years’ experience as classroom teachers, hold an advanced degree in special education or a related field, and have training that allows them to conduct academic achievement tests (Sutton et al., 2009, Sutton et al., 2014). However, Sutton et al., (2009) asserted that training, licensing, and practice among these professionals varies significantly across the 18 states that issue certificates to educational diagnosticians. Guerra et al., (2017) noted that “research regarding the specific tasks and role of educational diagnosticians in special education is limited” (p. 709) and is focused on Texas practitioners who administer academic achievement tests to facilitate special education eligibility determinations. Like school psychologists, there is a critical shortage of these practitioners (Guerra et al., 2017).

School Administrators

Administrators, in their role as district representatives, play a significant part in the special education evaluation process. Administrators such as principals are generally required to attend eligibility meetings, and those who fail to take leadership roles on multidisciplinary evaluation teams can be removed from their position (McElhinny & Pellegrin, 2014). Special education administrators are especially crucial as they focus on student outcomes and legal compliance, both of which are important components of special education evaluations (Bellamy & Iwaszuk, 2017). School administrators are responsible for observing and evaluating special education teachers (Rodl et al., 2018) and often promulgate assessment and special education eligibility practices to school psychologists (Boccio et al., 2016). Administrators also act as team leaders that determine how frequently progress monitoring will occur and who should administer these measures (Tackett et al., 2009). Therefore, while administrators do not generally fill the role of test administrator, they have intimate knowledge of district test use and assessment practices.

NRTs

The two most common types of academic achievement measures used when making a special education eligibility determination are standardized norm-referenced tests and CBMs. NRTs¹ are often a key component in determining eligibility and services and are generally included in a comprehensive psychoeducational evaluation (Mather & Urso, 2011). NRTs are typically administered in a one-on-one setting and are designed to measure the academic skill areas corresponding to a specific learning disability, such as reading fluency, reading comprehension, written expression, and mathematics calculation (Breaux & Lichtenberger, 2016). NRTs should be administered by individuals who have training in the clinical use of the instruments in order to reduce the risk of invalid test scores and interpretation (Breaux & Lichtenberger, 2016). Current research examining the use of NRTs by nonschool psychology team members is limited (Harrison et al., 2018; Lockwood et al., 2019). Google Scholar and EBSCOhost searches revealed no studies documenting the frequency of administration, interpretation, and use of NRTs across educational professionals (e.g., teachers). However, Benson and colleagues’ 2017 survey (published in 2019) of 1317 US school psychologists provided an up-to-date accounting of NRT administration by these professionals. According to these researchers, the Kaufman Test of Educational Achievement, Third Edition (KTEA-3; Kaufman & Kaufman, 2014) was used by 62% of respondents, the Woodcock–Johnson IV Tests of Achievement (WJ IV-ACH; Schrank et al., 2014) was used by 49% of respondents, and the Wechsler Individual Achievement Test, Third Edition (WIAT-III; Wechsler, 2009) was used by 47% of respondents in any given month. However, no information regarding who primarily administers these measures, or if any other educators filled this role, was reported.

CBMs

CBMs are brief, standardized tools that are easy to administer and offer a reliable measure of students’ academic achievement over time (Shinn, 2008). These measures are often used to monitor a student’s progress in the areas of reading, spelling, written expression, and mathematics. CBMs should be used for benchmark testing for all students on a quarterly or trimester schedule. Students who score significantly below their same-grade peers during benchmark testing should be monitored with greater frequency (Shinn, 2008). In a RTI framework, CBMs may be used to inform and monitor interventions, as well as identify students who are eligible for special education services (Burns et al., 2017), although recent data suggest CBMs may not be used for data-based decision-making as often as anticipated (Silva et al., 2020).

Limited research regarding CBM practices exists and has generally focused on school psychologists. For example, Silva et al. (2020) surveyed 387 school psychologists and found that roughly 20% reported using Dynamic Indicators of Basic Early Literacy Skills (DIBELS), 19% AIMSweb, 10% Excel, 7% STAR, 4% MAP, 3% easyCBM, 2% iReady, and 2% mCLASS. However, these researchers used snowball sampling, and their sample was restricted to practitioners from 13 states that were largely (50%) from the Northeast (Silva et al., 2020), which likely impacts the generalizability of these findings (Atkinson & Delamont, 2010). Benson et al. (2019) survey of 1317 school psychologists indicated that 18% of their participants reported the specific CBMs they used. Of these participants, 48% reported using AIMSweb, 36% DIBELS, 23% easyCBM, 14% Renaissance Star, 11% Northwest Evaluation Association (NWEA), 9% FastBridge, and 9% iReady. Additionally, the authors indicate that CBMs were used to assess reading fluency by 29% of school psychologists, comprehension by 30%, early literacy by 27%, math computation by 28%, math concepts and applications by 27%, math early numeracy by 23%, and spelling by 10% in any given month. However, both of these studies focused solely on school psychologists, who likely are not the primary administrators of CBMs (Benson et al., 2019).

The few published studies that have examined CBM practices by nonschool psychologists’ have relied on limited samples (Clemens et al., 2016; Jenkins et al., 2013). Specifically, Jenkins et al. (2013) surveyed 62 schools, Mellard et al., (2009) 41 schools, Tackett et al. (2009) five schools, and Mahdavi and Beebe-Frankenberger (2009) two schools regarding their CBM use. Additionally, none of these studies surveyed schools from all U.S. demographic areas and all relied on convenience sampling, which significantly impacts the external validity of their findings (Olsen et al., 2013). For these reasons, the findings of these studies were not reviewed.

However, one research group has examined RTI practices, a system that relies upon CBMs, by districts across the United States. Spectrum K-12 et al. (2010) surveyed 1101 districts and found that 49% use RTI for all students, 61% use RTI to identify student in need of early intervention, and 41% use RTI to determine special education eligibility. Most commonly, reading and math were the primary areas assessed as part of RTI-related assessment. The Spectrum K-12 researchers found that 90% of elementary schools, 67% of middle schools, 43% of secondary schools, and 52% of pre-K schools implemented RTI in reading. For math, 59% of elementary schools, 48% of middles schools, 34% of secondary schools, and 28% of pre-K schools implemented RTI practices. Only 19% of districts reported full implementation at that time. The most common initiators of RTI in these districts were school administrators, who did so in 76% of districts. However, this study also has limitations including an overreliance on Southern states (50% of the sample), no reporting of response rates, and a 69% completion rate. Furthermore, this survey is 10 years old, and practices may have changed since that time. In sum, our understanding of CBM practices is incomplete.

Current Study

The purpose of this study was to examine special education academic assessment practices across professions throughout the United States. For this study, we defined “assessment practices” as the measures that are used and the professional who administers and interprets these measures. Furthermore, as CBMs are used for more than special education eligibility, we extended the definition of “assessment practices” to include the areas that CBMs are used (e.g., reading fluency, reading comprehension, and math computation), the purpose of their use (e.g., special education eligibility and progress monitoring), and the grades for which they are most commonly administered. Because national research has largely focused solely on school psychologists (e.g., Benson et al., 2019) or is outdated (e.g., Spectrum K-12 et al., 2010), examining test use and assessment practices across professions (e.g., teachers and diagnosticians) is needed to provide a clearer understanding of which academic achievement measures are used. Additionally, we hoped to explore which variables influence assessment roles. Due to the shortage of school psychologists, especially in rural districts (National Association of School Psychologists (NASP), 2017) and Western states (American Association for Employment in Education (AAEE), 2016), we were interested in exploring if region and district locale shifted the onus of test administration from school psychologists to other staff. Furthermore, we wanted to determine if NRT administration roles shifted after initial eligibility (e.g., re-evaluations) to mitigate these shortages. This knowledge is important because NRTs are high-stakes and difficult to administer with fidelity (Lockwood et al., 2019; Styck & Walsh, 2016). Therefore, highly trained and practiced professionals (e.g., school psychologists) may obtain more valid test scores resulting in improved special education classification/placement decisions and service provision than special education teachers. Due to the lack of literature on this topic, all of our research questions were exploratory in nature. Specifically, we hoped to answer the following questions:

What NRT practices are used across professionals?

What CBM practices are used across professionals?

Do assessment roles vary depending on:

Region?

Locale?

Method

This research was approved by the lead author’s university Institutional Review Board. Participation was voluntary, and consent was provided by clicking an “agree” button on a web-based consent form. Eysenbach’s (2004) methodology for reporting the results of internet surveys was used in drafting the results section. We report our sample size, all data exclusions, and all manipulations utilized in the study.

Survey Design

The survey was created to gather information regarding participant demographics as well as data regarding district-level (a) NRT and (b) CBM practices. Survey items were generated based on recent peer-reviewed articles (e.g., Lockwood & Farmer, 2019) and textbooks (e.g., Breaux & Lichtenberger, 2016; Waugh & Gronlund, 2013). The survey was reviewed by nine content experts. Of these content experts, five were school psychology faculty members, each of whom have taught assessment courses and have peer-review publications in the area evidence-based assessment. One expert was a special education trainer who teaches courses on academic achievement assessment and has published in this area. Three current special education administrators also acted as content experts providing feedback regarding the clarity of items from a nonschool psychologist’s perspective. Items were added and revised based on the experts’ feedback. The survey was administered using the Qualtrics software platform. The survey consisted of 37 items that focused on demographics, test use, and assessment practices. Multiple item formats were utilized including 17 single answer, 12 multiple answer, three Likert-type, three text entry, and two drop-down list items. Adaptive questioning (i.e., “skip logic”) was used to minimize completion time. A PDF version of the survey is available at (https://osf.io/wcuvz/) and is open access.

Participant Recruitment

A recruitment email with a link to the survey was sent to the special education director (or equivalent) of all U.S. districts with a student population of 3000 or greater as listed on the National Center for Education Statistics website for the 2018–2019 school year (National Center for Education Statistics, 2019). This minimum enrollment threshold was based on the criteria of Levenson (2012) addressing the untenability of surveying all 18,407 US school districts and the logic that the majority (approximately 71%) of US children are served in districts with 3000 or more students. Administrators were chosen for this study as we wanted to gather data about achievement assessment practices across professionals (not just school psychologists) and because of their knowledge of district test use and assessment practices. In total, the survey link was emailed to 3401 participants from April 23 to May 28, 2019 using the Tailored Design Method, which uses evidence-based approaches for writing survey questions and reducing response reluctance to maximize the response rates (Dillman et al., 2014). One initial and two follow-up requests were sent. Participants were instructed to forward the survey request to the proper person in their district if they were not an administrator with “knowledge of the special education eligibility process” in their district.

A total of 790 participants clicked on the survey link. Of these respondents, 19 (2.4%) declined to provide consent, four (.5%) provided consent but did not begin the survey, and three (.5%) clicked “no” in response to the inclusionary criterion which read “Are you a director of special education, superintendent, lead school psychologist, principal, or otherwise have knowledge of the special education eligibility process in your district?” Additionally, 26 participants (3.3%) did not attempt a single item beyond the inclusionary criterion, and 13 (1.6%) completed 60% or less of the survey. All of the above participants were deleted leaving a sample of 725 or 21.3% of contacted districts. Median completion time for the survey was seven minutes 21 seconds.

Participants

Responses from 725 administrators representing 48 states were used. The vast majority of participants indicated that they were white (88%), women (80%), with a job title of Director of Special Education or equivalent (92%) and worked in traditional school districts (94%). Most participants reported their highest obtained degree was a masters (44%), followed by a doctorate (27%), and a specialist degree (24%). Participants reported a mean age of 49 years that they have been working in education for an average of 24 years and have been employed in their current district for an average of 12 years. Eight percent of participants indicated that their district is urban, 31% suburban, 22% large town, 33% town, and 6% rural. Of note, district locales were determined by US census categories and were operationally defined on the survey as follows: urban (city with population greater than 250,000), suburban (included in an urban metro area but not within city limits), large town (population greater than 25,000 and not part of a greater metro area), town (population between 2500 and 24,999), and rural or town with a population of less than 2500.

Analysis

Data were downloaded from Qualtrics on June 6, 2019 and imported to SPSS v. 25 for analysis. Descriptive statistics were calculated to gather general information regarding test use, assessment practices, and demographics. To answer research question 3, six chi-square tests were used to identify differences in assessment roles across geographic regions and locales served. As a requirement of chi-square statistics is that no more than 20% of expected counts may be <5 (Moore et al., 1999), levels from two categorical variables were combined. Specifically, due to the large numbers of states (48) represented, responses to the variable state were coded into a new variable titled region corresponding to the geographical (i.e., West, Midwest, South, and Northeast) regions identified by National Center for Education Statistics (2020). Additionally, as CBMs were almost exclusively administered by teachers, all other response types (i.e., school psychologists, teachers’ aides, and “other”) were combined as “other.” This resulted in the following three levels of CBM administrators: special education teacher, general education teacher, and “other.” Post hoc analyses were conducted to evaluate the contribution of each cell to the statistically significant findings of the model utilizing residual analysis. As these post hoc analyses involved multiple tests, the criteria set forth by Garcia-Perez and Nunez-Anton (2003) were utilized. Specifically, significance values for region were set at .0031 (.05/16) and at .0025 (.05/20) for district locales when examining differences in NRTs. Significance values for region were set at .0042 (05/12) when examining regional differences CBM administration.

Results

Current Practices in the Use of Norm-referenced Tests of Academic Achievement

Participants were asked to identify all of the tests of academic achievement used in a standard special education eligibility evaluation in their district. Ninety-nine percent of participants indicated that their district utilized at least one norm-referenced measure for this purpose. The most reported NRT was the WJ IV-ACH, which 86% of participants indicated their district used. This was followed closely by the WIAT-III (84%), the KTEA-3 (61%), and then by the Wide Range Achievement Test-Fifth Edition (Wilkinson & Robertson, 2017; 25%). Three percent of participants indicated older versions of these tests were used, 2% acknowledged that they were not sure what tests were used in their district, and 11% indicated that “other” NRTs were used. These “other” measures included NRTs that were not listed on the survey (e.g., KeyMath-3 Diagnostic Achievement Test; Connolly, 2007; 3%), tests of IQ (2%), tests of processing (1%), behavioral checklists (1%), and CBMs (1%); 1% of participants checked “other” but did not provide clarification in the text entry.

Tables 1–4 provide detailed information regarding who administers these tests. School psychologists are the most frequent administrators of test of academic achievement, filling this role the majority of the time in 50% of districts’ initial and 43% of districts’ triennial evaluations. This is followed by special educations teachers, who administer the majority of these tests in 27% of districts’ initial and 34% of districts’ triennial evaluations. Additionally, 50% of participants indicated that special education teachers administer these tests at least occasionally. Ninety-one percent of participants indicated that the person who conducts the tests used in a standard special education eligibility determination evaluation also interprets these results, while 7% indicated that regardless of who administers the test, the school psychologists in their district interpret the test results.

Table 1.

Initial Evaluations by Region.

Region	Special Ed teacher	School psych	Ed diagnostician	Other	Total
	n (%)	n (%)	n (%)	n (%)	Total
	Adj. Res.	Adj. Res.	Adj. Res.	Adj. Res.	N
Overall	197 (27.2)	361 (49.9)	109 (15.1)	57 (7.9)	724
West	77 (59.7)	43 (33.3)	4 (3.1)	5 (3.9)	129
West	9.1*	−4.1*	−4.2*	−1.9
Midwest	43 (23.4)	113 (61.4)	18 (9.8)	10 (5.4)	184
Midwest	−1.4	3.6*	−2.3	−1.4
South	23 (8.7)	130 (49.2)	80 (30.3)	31 (11.7)	264
South	−8.5*	−.3	8.7*	2.9
Northeast	54 (36.7)	75 (51.0)	7 (4.8)	11 (7.5)	147
Northeast	2.9	.3	−3.9*	−.2

Note. χ²(9, n = 724) = 176.98, p < .001, and Cramer’s V = .29. *Significant after Bonferroni correction.

Table 2.

Triennial Evaluations by Region.

Region	Special Ed teacher	School psych	Ed diagnostician	Other	Total
	n (%)	n (%)	n (%)	n (%)	Total
	Adj. Res.	Adj. Res.	Adj. Res.	Adj. Res.	N
Overall	246 (34.0)	310 (42.8)	109 (15.1)	59 (8.1)	724
West	84 (65.1)	38 (29.5)	4 (3.1)	3 (2.3)	129
West	8.2*	−3.4*	−4.2*	−2.7	129
Midwest	57 (31.0)	98 (53.3)	17 (9.2)	12 (6.5)	184
Midwest	−1.0	3.3*	−2.6	−.9	184
South	32 (12.1)	118 (44.7)	82(31.1)	32 (12.1)	264
South	−9.4*	.8	9.1*	3.0*	264
Northeast	73 (49.7)	56 (38.1)	6 (4.1)	12 (8.2)	147
Northeast	4.5*	−1.3	−4.2*	0	147

Note. χ²(9, n = 724) = 180.24, p < .001, and Cramer’s V = .29. *Significant after Bonferroni correction.

Table 3.

Initial Evaluations by Locale.

Locale	Special Ed teacher	School psych	Ed diagnostician	Other	Total
	n (%)	n (%)	n (%)	n (%)	Total
	Adj. Res.	Adj. Res.	Adj. Res.	Adj. Res.	N
Overall	195 (27.0)	360 (49.9)	109 (15.1)	57 (7.9)	724
Urban	18 (30.5)	33 (55.9)	4 (6.8)	4 (6.8)	59
Urban	.6	1.0	−1.9	−.3	59
Suburban	60 (26.8)	120 (53.6)	30 (13.4)	14 (6.3)	224
Suburban	−.1	1.3	−.9	−1.1	224
Large town	45 (28.1)	85 (53.1)	19 (11.9)	11 (6.9)	160
Large town	.3	.9	−1.3	−.5	160
Town	59 (25.1)	109 (46.4)	44 (18.7)	23 (9.8)	235
Town	−.8	−1.3	1.9	1.3	235
Rural	13 (30.2)	13 (30.2)	12 (27.9)	5 (11.6)	43
Rural	.5	−2.7	2.4	.9	43

Note. χ²(12, n = 721) = 19.94, p = .068, and Cramer’s V = .10.

Table 4.

Triennial Evaluations by Locale.

Locale	Special Ed teacher	School psych	Ed diagnostician	Other	Total
	n (%)	n (%)	n (%)	n (%)	Total
	Adj. Res.	Adj. Res.	Adj. Res.	Adj. Res.	N
Overall	245 (34.0)	308 (42.7)	109 (15.1)	59 (8.2)	721
Urban	23 (39.0)	29 (49.2)	3 (5.1)	4 (6.8)	59
Urban	.08	1.0	−2.2	−.4	59
Suburban	83 (37.1)	97 (43.3)	28 (12.5)	16 (7.1)	224
Suburban	1.2	.2	−1.3	−.7	224
Large town	58 (36.3)	73 (45.6)	20 (12.5)	9 (5.6)	160
Large town	.7	.8	−1.0	−1.3	160
Town	66 (28.1)	101 (43.0)	45 (19.1)	23 (9.8)	235
Town	−2.3	.1	2.1	1.1	235
Rural	15 (34.9)	8 (18.6)	13 (30.2)	7 (16.3)	43
Rural	.1	−3.3*	2.9	2.0	43

Note. χ²(12, n = 721) = 31.07, p < .01, and Cramer’s V = .12. *Significant after Bonferroni correction.

Differences in NRT Administration

A McNemar–Bowker test was used to determine whether there were significant changes in the percentage of professionals that administered NRTs between initial and triennial evaluations as this analysis allows for repeated measure designs that are not possible using chi-square tests (Pallant, 2010). Results suggested that administration roles shifted significantly from initial to triennial evaluations χ²_MB = 48.17, df = 6, p < .001. Post hoc analyses revealed one significant finding suggesting a shift in the test administration roles by school psychologists to special education teachers from initial to triennial evaluations χ²_MB = 29.82, df = 63, p < .001. No other significant shifts were noted.

Chi-square statistics suggested that there were significant differences in NRT roles across regions for both initial (χ² (9, N = 724) = 176.98, p < .001, Cramer’s V = .29) and triennial ( $χ$ ² (9, N = 724) = 180.24, p < .001, Cramer’s V = .29) evaluations. Additionally, significant differences in roles across district locales were not found for initial ( $χ$ ² (12, N = 721) = 19.94, p = .068, Cramer’s V = .10) but were noted for triennial (χ² (12, N = 721) = 31.07, p < .01, Cramer’s V = .12) evaluations.

Post hoc analyses suggested that special education teachers were significantly more likely to administer NRTs for both initial and triennial evaluations in Western states and significantly less likely in Southern states evaluations. Special education teachers were also significantly more likely to administer these tests in the Northeast but only for triennial evaluations. School psychologists were significantly more likely to fill this role for both initial and triennial evaluations in Midwestern states and significantly less likely in Western states. Educational diagnosticians were significantly more likely to fill this role for both initial and triennial evaluations in Southern states and significantly less likely to do so in Western and Northeastern states. The likelihood that some other educator filled this role did not significantly differ based on location for initial evaluations but was significantly more likely in Southern states for triennial evaluations. Although overall significant differences based on district locales were found for triennial evaluations, post hoc analyses revealed only one meaningful finding; school psychologists were significantly less likely to fill this role in rural settings. More details regarding these findings may be found in Tables 1–4.

Current Practices in the Use of CBM

Approximately 82% of participants reported that their district uses standardized CBMs, and 3% reported that they are uncertain as to whether their district does or does not use CBMs. The grades in which CBMs were most frequently utilized were grades K-5, which were endorsed by 70–80% of participants. Additionally, participants indicated that CBMs were used in greater than half of middle schools and in roughly a third of high schools across districts. Supplementary Table details the percentage of participants who report CBM usage by grade.

CBMs were most frequently used to measure progress in reading, particularly fluency (75%), comprehension (75%), and phonemic awareness (73%). The most frequently used measures were DIBELS (42%), AIMSweb (30%), easyCBM (22%), FastBridge Learning (8%), and NWEA (6%). Additionally, 28% of participants reported using another standardized CBM.

Supplementary Table details both the proportion of districts using CBMs by grade as well as the areas for which CBMs are used. Of note, only 34% indicated that CBMs were used to aid in special education identification and were more frequently used to measure academic progress (76%) and identify students in need of more intense interventions (70%). Of the participants that indicated using standardized CBMs, 62% indicated that the majority of these measures were administered by general education teachers, 25% by special education teachers, 2% by school psychologists, 2% by teachers’ aides, and the remaining 9% by other staff or a combination of staff (responses included: “RTI interventionist” and “all hands on deck”).

Differences in CBM Administration

Chi-square statistics suggested significant differences in CBM administration roles across regions (χ² (6, N = 588) = 22.58, p < .01, Cramer’s V = .14). No significant differences in roles across locales (χ² (8, N = 586) = 7.59, p = .474, Cramer’s V = .08) were found. Post hoc analyses suggest that general education teachers were significantly less likely to fill this role in Midwestern states than in other regions. Additionally, the likelihood that some other educator filled this role was significantly greater in Midwestern states and significantly less likely in Southern states. Additional details can be found in Tables 5 and 6.

Table 5.

Curriculum-based Measure Administrator by Region.

Region	Special Ed teacher	General Ed teacher	Other	Total
	n (%)	n (%)	n (%)	Total
	Adj. Res.	Adj. Res.	Adj. Res.	N
Overall	151 (25.7)	358 (60.9)	79 (13.4)	588
West	17 (17.0)	66 (66.0)	17 (17.0)	100
West	−2.2	1.2	1.1	100
Midwest	43 (28.3)	77 (50.7)	32 (21.1)	152
Midwest	.9	−3.0*	3.2*	152
South	60 (28.2)	139 (65.3)	14 (6.6)	213
South	1.0	1.6	−3.7*	213
Northeast	31 (25.2)	76 (61.8)	16 (13.0)	123
Northeast	−.1	.2	−.2	123

Note. χ²(6, n = 588) = 22.58, p < .01, and Cramer’s V = .14. *Significant after Bonferroni correction.

Table 6.

Curriculum-based Measure Administrator by Locale.

Locale	Special Ed teacher	General Ed teacher	Other	Total
	n (%)	n (%)	n (%)	Total
	Adj. Res.	Adj. Res.	Adj. Res.	N
Overall	149 (25.4)	358 (61.1)	79 (13.5)	586
Urban	6 (15.0)	29 (72.5)	5 (12.5)	40
Urban	−1.6	1.5	−.2	40
Suburban	48 (26.7)	104 (57.8)	28 (15.6)	180
Suburban	.5	−1.1	1.0	180
Large town	34 (25.8)	79 (59.8)	19 (14.4)	132
Large town	.1	−.3	.3	132
Town	52 (26.3)	120 (60.6)	26 (13.1)	198
Town	.3	−.2	−.2	198
Rural	9 (25.0)	26 (72.2)	1 (2.8)	36
Rural	−.1	1.4	−1.9	36

Note. χ²(8, n = 586) = 7.59, p = .474, and Cramer’s V = .08.

Discussion

No clear understanding of special education academic achievement assessment practices across professionals and across the United States currently exists, even though IDEA (2004) clearly indicates that eligibility evaluation must be a team process. Additionally, other professionals beyond school psychologists, including special education teachers and educational diagnosticians, likely conduct these tests in some districts. The purpose of this study was to gain a detailed understanding of NRT and CBM use and assessment practices in US districts with 3000 or greater students to fill this gap in the literature.

NRTs such as the WJ IV-ACH and WIAT-III appear to be used by the overwhelming majority of districts with 3000 or more students. Usage rates of the WJ IV-ACH and WIAT-III were much higher in our sample than were obtained by Benson et al. (2019), although KTEA-3 use was almost identical. This difference may stem from how the questions were framed (i.e., we asked whether the test was used; whereas, Benson and colleagues reported usage per month) or may reflect that school personnel other than school psychologists are administering the assessment instruments. While school psychologists most typically administered NRTs, special education personnel appeared to be responsible for administering these tests roughly a third of the time in districts with 3000 or greater students. There also seemed to be a significant shift in this role, with special education teachers engaging in 26% more academic achievement testing and school psychologists conducting 14% less achievement testing when the team moved from initial to triennial evaluations. Additionally, half of participants indicated that special education teachers used these types of tests at least occasionally. Thus, while the differences in our data likely stem in part from how the questions were asked and reported, our data also reflect differences in the assessment practices of other district personnel that were not reflected by Benson and colleagues’ survey.

Demographic variables appeared to impact the likelihood of whether a given educator will administer NRTs in districts with 3000 or greater students. In Western states, special education teachers appeared to fill this role far more frequently than in other regions, especially the South, and appeared to be the primary NRT administrators in approximately two-thirds of evaluations. School psychologists were most likely to administer NRTs in the Midwest, doing so at least half of the time. Educational diagnosticians appeared to be most active in the South, filling this role roughly a third of the time. Additionally, school psychologists administered NRTs significantly less often in rural settings for triennial evaluations in districts with 3000 or more students.

Unlike NRTs, which appeared to often be the domain of school psychologists, most participants (97%) indicated that other staff, most notably general and special education teachers, administered the majority of CBMs in districts with 3000 or greater students. Like the Spectrum K-12 et al. (2010) survey, we found that CBMs were most commonly used in elementary schools to monitor reading progress and were used in the vast majority of districts. Furthermore, our results confirmed the Spectrum K-12 findings that CBMs are far more likely to be used to identify students in need of intervention than as part of special education eligibility determination process. We found that CBMs were used more than twice as frequently in all academic areas than was reported in Benson et al.’s (2019) study. Like Silva et al. (2020) and Benson and colleagues, we found that DIBELS and AIMSweb were the most commonly used CBMs, although at rates that were one and a half to two times higher than Silva et al. (rates could not be determined for Benson et al. due to their reporting method). Demographic variables did not appear to have as much of an impact on the likelihood that a given educator would administer CBMs, which were generally administered by teachers in districts with 3000 or more students.

Limitations and Future Directions

In this survey, we asked school administrators to report about their districts’ assessment practices. To identify test use and assessment practices, we relied on participants to self-report behavior, and, as such, our data may reflect a normative bias. It may also be the case that administrators are unaware of current district practices in the administration of academic achievement tests. Future research may examine objective measures of practice, including district academic achievement protocols, to see which educators are typically completing them.

Due to our sampling method, our study consisted mostly of traditional school districts, all of which had 3000 or more students. For this reason, our results may not generalize to other settings, such as charter schools or rural settings. To address this possible limitation, future research could focus on districts with student populations less than 3000 students. Another methodological limitation is the unknown impact of nonresponse bias, which is always a consideration when conducting survey research.

As noted in the analysis section, states were coded into regions and school psychologists, educational diagnosticians, and “other” were combined into a new variable other when examining CBM administration to improve analysis and interpretation. Had we not done so, random error may have led to erroneous conclusions. While coding into categories is not necessarily problematic (Pazzaglia et al., 2016), future researchers may wish to gather a larger sample to allow for comparison across each state and each professional type. Additionally, to avoid familywise error, we utilized stringent alpha corrections across null hypothesis tests (i.e., p < .0025, p < .0031, and p < .0042). While it is common practice in educational research to avoid this practice (Moote & Archer, 2018), we believe it was justified to avoid Type I error. However, multiple post hoc comparisons that failed to meet our criteria were significant at the .05, .01, and .001 level. Given our use of conservative criteria, these findings may justify further examination.

Implications for Practice and Research

There are multiple implications that can be inferred from this study. First, as teachers engage in high-stakes academic achievement assessment, in the areas of NRT and standardized CBM, it is imperative that teachers are experts in administering and interpreting the measures they use. To achieve and maintain competency, adequate pre-service and in-service training in the areas of standardized assessment, psychometrics, and test interpretation is necessary for educators who fill these roles. This is especially true in Western states where special education teachers administer the majority of NRTs, which are difficult to administer with fidelity, and errors are ubiquitous (Lockwood et al., 2019; Styck & Walsh, 2016). Second, the limited administrations by school psychologists in rural settings suggest a continuing need for recruitment in these locales. Additionally, this finding suggests that it may be prudent to provide additional training in rural locales where more nonschool psychologists are likely to fill this role. At a minimum, training should focus on the importance of accurate, standardized administration, common errors (see Lockwood et al., 2019), and on interpretation. Third, as a significant number of special educators and educational diagnosticians administer NRTs, future research should examine the training that these professionals receive in areas crucial for proper test administration, such as psychometrics, statistics, and standardized procedures (Lockwood & Farmer, 2019); relatedly, future research into the NRT administration fidelity of these practitioners would also appear crucial. Finally, only a third of participants indicated that their district uses CBM data to aid in special education identification. Team members should use the CBM data they have collected when making eligibility decisions instead of (presumably) relying solely on NRT scores and grade/file reviews to aid in this process. This is imperative as NRT administrations are often rife with errors that can impact the validity of their scores (Harrison et al., 2018; Lockwood et al., 2019), and CBMs can serve to confirm (or disconfirm) the results of NRTs. Inadequate training, a lack of continuing education, and insufficient fidelity checks of assessment practices may lead to misleading or invalid scores, which may lead to inaccurate educational decisions with long-lasting effects on students. Specifically, children may be denied the services that they are entitled to, may be mislabeled as having a disability when they do not, or may have their services prematurely terminated. Without adequate safeguards, it is unclear how well districts are able to prevent these harmful outcomes.

Conclusion

Educational placement decisions based on assessment data are of high importance, often following a student for 1–3 years, and even longer in some cases. The purpose of this study was to better understand test use and assessment practices across the United States and professionals. This study suggests that school personnel aside from school psychologists are often responsible for collecting data that may be used to inform decisions regarding the provision of special education services or changes in educational placement for students. Additionally, participants indicated that assessment practices varied greatly across regions. Trainers of educators across subfields (e.g., special education) should be conscious of these critical roles when planning programs of study and practical experience expectations of their pre-service and advanced education students, especially as these roles are often thought of as being the domain of school psychologists.

Supplemental Material

sj-pdf-1-jpa-10.1177_0734282920984290 – Supplemental Material for Academic Achievement Test Use and Assessment Practices: A National Survey of Special Education Administrators

Supplemental Material, sj-pdf-1-jpa-10.1177_0734282920984290 for Academic Achievement Test Use and Assessment Practices: A National Survey of Special Education Administrators by Adam B. Lockwood, Ryan L. Farmer, Kathy J. Bohan, Shannon Winans and Karen Sealander in Journal of Psychoeducational Assessment

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Adam B. Lockwood

Ryan L. Farmer

Supplemental Material

Supplemental material for this article is available online.

Note

References

American Association for Employment in Education (AAEE). (2016). Educator supply and demand report 2015–2016: Executive summary. AAEE.

Americans with Disabilities Act . (1990). Pub. L. No. 101-336, 104 Stat. 328. https://www.ada.gov/pubs/ada.htm

Atkinson

Delamont

(2010). Sage qualitative research methods. Sage.

Bellamy

Iwaszuk

(2017). Responding to the need for new local special education administrators: A case study. University of Washington. https://ceedar.education.ufl.edu/wp-content/uploads/2017/12/Case-Study-SPED-Admin-DRAFT.pdf

Benson

N. F.

Floyd

R. G.

Kranzler

J. H.

Eckert

T. L.

Fefer

S. A.

Morgan

G. B.

(2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 national survey. Journal of School Psychology, 72, 29-48. doi:10.1016/j.jsp.2018.12.004

Boccio

D. E.

Weisz

Lefkowitz

(2016). Administrative pressure to practice unethically and burnout within the profession of school psychology. Psychology in the Schools, 53(6), 659-672. doi:10.1002/pits.21931

Breaux

K. C.

Lichtenberger

E. O.

(2016). Essentials of KTEA-3 and WIAT-III assessment. John Wiley & Sons.

Burns

M. K.

Riley-Tillman

T. C.

Rathvon

(2017). Effective school interventions: Evidence-based strategies for improving student outcomes. Guilford Publications.

Clemens

N. H.

Keller-Margulis

M. A.

Scholten

Yoon

(2016). Screening assessment within a multi-tiered system of support: Current practices, advances, and next steps. In Jimerson

Burns

VanDerHeyden

(Eds.), Handbook of response to intervention (pp. 187-213). Springer. doi:10.1007/978-1-4899-7568-3_12

10.

Connolly

A. J.

(2007). Keymath-3 diagnostic assessment. Upper Saddle Creek, NJ: Pearson.

11.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

(2014). Internet, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley & Sons.

12.

Eysenbach

(2004). Improving the quality of web surveys: The checklist for reporting results of internet esurveys (CHERRIES). Journal of Medical Internet Research, 6(3), e34. doi:10.2196/jmir.6.3.e34

13.

García-pérez

M. A.

Núñez-antón

(2003). Cellwise residual analysis in two-way contingency tables. Educational and Psychological Measurement, 63(5), 825-839. doi:10.1177/0013164403251280

14.

Goh

D. S.

Teslow

Fuller

G. B.

(1981). The practice of psychological assessment among school psychologists. Professional Psychology, 12(6), 696. doi:10.1037/0735-7028.12.6.696

15.

Guerra

M. G.

Kupczynski

L. P.

Mundy

M. A.

Garza

(2017). The diagnostic competencies: Perceptions of educational diagnosticians, administrators, and classroom teachers. International Journal of Special Education, 32(4), 708-722. https://files.eric.ed.gov/fulltext/EJ1184166.pdf

16.

Guerriero

T. S.

Houser

M. A.

McGinley

V. A.

(2020). The special educators guide to assessment. Sage.

17.

Harrison

G. L.

Goegan

L. D.

Macoun

S. J.

(2018). Common examiner scoring errors on academic achievement measures. Canadian Journal of School Psychology, 34(2), 98-112. doi:10.1177/0829573518763484

18.

Heward

W. L.

Alber-Morgan

S. R.

Konrad

(2017). Exceptional children: An introduction to special education (11th ed.). Pearson.

19.

Hutton

J. B.

Dubes

Muir

(1992). Assessment practices of school psychologists: Ten years later. School Psychology Review, 21, 271-284.

20.

Individuals with Disabilities Education Improvement Act (IDEA) . (2004). Pub. L. No. 108-446,1118Stat. 2647; 2004 enacted H. R. 1350; 108 enacted H. R. 1350.

21.

Jenkins

J. R.

Schiller

Blackorby

Thayer

S. K.

Tilly

W. D.

(2013). Responsiveness to intervention in reading: Architecture and practices. Learning Disability Quarterly, 36(1), 36-46. doi:10.1177/0731948712464963

22.

Kaufman

A. S.

Kaufman

N. L.

(2014). Kaufman test of educational achievement (3rd ed.). NCS Pearson.

23.

Levenson

(2012). Boosting the quality and efficiency of special education. Thomas B. Fordham Institute.

24.

Lockwood

Farmer

(2020). The course on cognitive assessment: Two decades later. Psychology in the Schools, 57(2), 265-283. doi:10.1002/pits.22298

25.

Lockwood

Sealander

Gross

T. J.

Lanterman

(2019). Teacher trainees’ administration and scoring errors on the Kaufman test of educational achievement. Journal of Psychoeducational Assessment, 38, 551-563. doi:10.1177/0734282919871144

26.

Mahdavi

J. N.

Beebe-Frankenberger

M. E.

(2009). Pioneering RTI systems that work: Social validity, collaboration, and context. Teaching Exceptional Children, 42(2), 64-72. doi:10.1177/2F004005990904200208

27.

Mather

Urso

(2011). Assessment of academic achievement. In Learning and Attention Disorders in Adolescence and Adulthood. John Wiley & Sons. doi:10.1002/9781118093085.ch7

28.

McElhinny

E. T.

Pellegrin

R. D.

(2014). The principal’s role with IEP Teams. National Association of Elementary School Principals. https://www.naesp.org/communicator-october-2014/principal-s-role-iep-teams

29.

McLeskey

(2017). High-leverage practices in special education. Council for Exceptional Children.

30.

Mellard

D. F.

McKnight

Woods

(2009). Response to intervention screening and progress-monitoring practices in 41 local schools. Learning Disabilities Research & Practice, 24(4), 186-195. doi:10.1111/j.1540-5826.2009.00292.x

31.

Moore

D. S.

McCabe

G. P.

Yates

(1999). The practice of statistics: Advanced placement. W.H. Freeman & Co.

32.

Moote

Archer

(2018). Failing to deliver? Exploring the current status of career education provision in England. Research Papers in Education, 33(2), 187-215. doi:10.1080/02671522.2016.1271005

33.

National Association of School Psychologists (NASP). (2017). Shortages in school psychology: Challenges to meeting the growing needs of US students and schools. NASP.

34.

National Center for Education Statistics. (2019). The condition of education at a glance. https://nces.ed.gov/programs/coe/indicator_cgg.asp

35.

National Center for Education Statistics. (2020). National assessment of educational progress. Interpreting NAEP geography results. https://nces.ed.gov/nationsreportcard/geography/interpret_results.aspx

36.

Olsen

R. B.

Orr

L. L.

Bell

S. H.

Stuart

E. A.

(2013). External validity in policy evaluations that choose sites purposively. Journal of Policy Analysis and Management, 32(1), 107-121. doi:10.1002/pam.21660

37.

Pallant

(2010). SPSS survival manual: A step by step guide to data analysis using SPSS (4th ed.). McGraw Hill.

38.

Pazzaglia

A. M.

Stafford

E. T.

Rodriguez

S. M.

(2016). Survey methods for educators: Analysis and reporting of survey data (part 3 of 3) (REL 2016–164). Department of Education, Institute of Education Sciences. Retrieved from http://ies.ed.gov/ncee/edlabs

39.

Qualifications Policy . (n.d.). https://www.pearsonassessments.com/professional-assessments/ordering/how-to-order/qualifications/qualifications-policy.html

40.

Rodl

J. E.

Bonifay

Cruz

R. A.

Manchanda

(2018). A survey of school administrators’ training and support related to evaluating special education teachers. Journal of School Administration Research and Development, 3(1), 19-31. https://eric.ed.gov/?id=EJ1190961

41.

Schrank

F. A.

McGrew

K. S.

Mather

(2014). Woodcock-Johnson IV tests of cognitive abilities. Riverside.

42.

Shinn

M. R.

(2008). Best practices in using curriculum-based measurement in a problem-solving model. In Thomas

Grimes

(Eds). Best practices in school psychology V (pp. 243-261). National Association of School Psychologists.

43.

Silva

M. R.

Collier-Meek

M. A.

Codding

R. S.

Kleinert

W. L.

Feinberg

(2020). Data collection and analysis in response-to-intervention: A survey of school psychologists. Contemporary School Psychology, 1-18. doi:10.1007/s40688-020-00280-2

44.

Spectrum K-12, American Association of School Administrators, Council of Administrators of Special Education, National Association of State Directors of Special Education, & State Title 1 Directors. (2010). Response to intervention (RTI) adoption survey. http://sss.usf.edu/resources/presentations/2010/fasp_summer_inst2010/Resource_SLD/RTI/2010RTIAdoptionSurveyReport.pdf

45.

Stinnett

T. A.

Havey

J. M.

Oehler-Stinnett

(1994). Current test usage by practicing school psychologists: A national survey. Journal of Psychoeducational Assessment, 12, 331–350. doi:10.1177/073428299401200403

46.

Styck

K. M.

Walsh

S. M.

(2016). Evaluating the prevalence and impact of examiner errors on the Wechsler scales of intelligence: A meta-analysis. Psychological Assessment, 28(1), 3-17. doi:10.1037/pas0000157

47.

Sutton

J. P.

Frye

E. M.

Frawley

P. A.

(2009). Nationally certified educational diagnostician (NCED). Assessment for Effective Intervention, 35(1), 17-23. doi:10.1177/1534508408325579

48.

Sutton

J. P.

Montani

T. O.

Frawley

P. A.

McElroy

P. A.

(2014). Nationally certified educational diagnostician: A credential with value-added potential. Assessment for Effective Intervention, 39(4), 208-218. doi:10.1177/1534508413494407

49.

Tackett

K. K.

Roberts

Baker

Scammacca

(2009). Implementing response to intervention: Practices and perspectives from five schools--frequently asked questions. Center on Instruction. https://files.eric.ed.gov/fulltext/ED521572.pdf

50.

Walcott

C. M.

Hyson

(2018). Results from the NASP 2015 membership survey, part one: Demographics and employment conditions. National Association of School Psychologists.

51.

Waugh

K.C.

Gronlund

N. E.

(2013). Assessment of student achievement (10th ed.). Allyn & Bacon Publishing.

52.

Wechsler

(2009). Wechsler individual achievement test (3rd ed.). Pearson.

53.

Wilkinson

G. S.

Robertson

G. J.

(2017). Wide range achievement test—Fifth edition (WRAT-5). Pearson.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB