Abstract
Universal screening measures can be used to identify students at risk academically due to learning disabilities or other difficulties. Research and legislation support the use of screening measures early in students’ education to ensure they receive any supports necessary to bolster their academic achievement. When selecting a screening measure, school personnel should identify the purpose of the screening measures in their school and the resources available. The research literature should then be consulted on established screening measures, with careful attention paid to required resources, evaluation of the students as a whole through the consultation of varied sources of information, relative biases of screening measures, further assessments for low-scoring students, and test metrics. This article discusses research-based procedures to follow in selecting screening measures, specifies factors to consider in this process, and provides examples of existing measures for use in the universal screening process.
Universal screening is a process designed to identify students requiring extra scaffolding in order to succeed in the academic curriculum. This process can include the administration of standardized assessments, the application of rating scales, or the use of observations to look at the achievement of all students in a school or grade level. Within a response-to-intervention (RTI) framework, results of a screening measure can be used to identify students at risk for learning or other disabilities so that these students can be provided with supports in Tier 2 or 3 interventions to help them succeed academically (Gansle & Noell, 2008; Lazarus & Ortega, 2007; Scott & Delgado, 2003). Ideally, these students can be identified early in their education, as research has shown that students with learning difficulties who are given early interventions have a greater probability of improving their academic achievement (Gansle & Noell, 2008; Kettler, 2007; Kettler & Elliott, 2010; Lilles et al., 2009). Particular consideration must be applied in this selection process to ensure that the measure chosen is effective and valuable for the specific purpose and population intended, as failure to correctly identify students in need of further supports affects the development of competencies in academic, social, and motor ability domains as a student and, consequently, throughout one’s life (Lilles et al., 2009). This article presents research-based guidelines to select effective screening measures for use in general academic evaluations based on a three-step process: (a) consider school factors, (b) research screening measures, and (c) identify the best fit.
Importance of Screening
The administration of universal screening measures is both supported by prior research and required by the federal government in recent legislations. The No Child Left Behind Act (2001) and the Individuals With Disabilities Education Improvement Act (2004) require schools to screen all students and students with disabilities, respectively, to identify those at risk academically and to ensure that all students are achieving at the highest possible level (Gansle & Noell, 2008; Kettler, 2007; Lazarus & Ortega, 2007; Oakland & Rossen, 2005). Beyond the federal mandate, the established literature presents an abundance of support for the use of effective universal screening measures to identify a broad range of students early in order to provide support or more intensive interventions to improve these students’ later achievement (Gansle & Noell, 2008; Kane, Roy, & Medina, 2013; Kettler, 2007; Scott & Delgado, 2003). Lacking such support, students with learning disabilities or difficulties are at increased risk of school dropout and subsequent unemployment (Kane et al., 2013; Lilles et al., 2009). For this reason, students benefit from both earlier and more inclusive screening processes to minimize false negatives and to provide individualized interventions to as many at-risk students as possible.
The preponderance of research findings points to the need to employ some kind of preventative measure in order to identify students with learning disabilities or difficulties early in their education (Gansle & Noell, 2008; Kane et al., 2013; Kettler, 2007; Lilles et al., 2009; Scott & Delgado, 2003). Universal screening as an identification process has been specifically supported in the literature because of its relative strength, its ability to reach more students, and its reasonably unbiased assessment of students from various cultures, socioeconomic classes, and backgrounds, as compared to spontaneous teacher referrals, parent appointments, and other methods of identification (Gansle & Noell, 2008; Oakland & Rossen, 2005; Scott & Delgado, 2003, 2006; VanDerHeyden, 2013). However, there is little existing practical guidance for administrators and school psychologists to follow when deciding which screening measure to select.
The Process of Selecting Universal Screening Measures
When choosing a universal screening measure to utilize in a specific school or grade level, there are several factors that must be taken into consideration. Specifically, professionals should attend to the recommendation to conduct the screening process as early in the school year as possible, especially for students in younger grades (Gansle & Noell, 2008; Lazarus & Ortega, 2007; Lilles et al., 2009; Quirk & Furlong, 2014). By implementing screening measures early, students can be given necessary supports in a more timely fashion, improving their chances of academic success. In addition, careful attention should be applied to the process of selecting screening measures because they can be designed for various purposes and populations and may be in diverse stages of research or development. In adopting a measure, decisions must be made on the purpose of the screening process, the resources available, the effectiveness of measures to provide a comprehensive evaluation without bias, and the importance of various test metrics as determined by prior research (Gansle & Noell, 2008; Kettler, 2007; Scott & Delgado, 2003; VanDerHeyden, 2013). In completing this selection process, it may be advantageous to assemble a committee of school staff members for the purpose of picking a universal screening measure for the school.
Step 1: Consider School Factors
The initial step in selecting a screening measure involves contemplating the factors of the school and students to be evaluated. First, educators should determine the purpose of the screening process and decide what resources will be invested in these assessments. This first step specifically may be accomplished through the collaboration of several key school staff members who can offer insight on the academic abilities of the students and the budget needs of the school.
Identify the purpose
First and foremost, educators must identify the purpose of implementing a screening measure in their school. By doing so, measures that can collect the required data may be identified and selected. Too often, school personnel fail to consider the purpose of their screening measure prior to implementation and realize too late that they did not obtain the information needed (VanDerHeyden, 2013). Examples of these different purposes for screening might be (a) to identify students at risk for learning or other disabilities, (b) to individualize Tier 2 or 3 interventions, (c) to determine which students are high achievers, (d) to look at achievement in specific academic domains, or (e) to examine students’ performance in nonacademic areas. For instance, a school looking to identify students with deficits in basic skills areas could employ the System to Enhance Educational Performance screening measure for this target purpose, while professionals looking for a measure to determine which students have learning difficulties might use the preschool screening measure adapted by Scott and Delgado (Gansle & Noell, 2008; iSTEEP, 2009; Scott & Delgado, 2003, 2006; VanDerHeyden, 2013). In determining the purpose of the screening process, school professionals may want to consult particular staff members, such as the school psychologist, reading or mathematics specialists, or school counselors, to aid in identifying the specific academic needs of the students and, subsequently, the targeted purpose of the screening process.
Following the description of each substep outlined in this article, critical questions that should be considered by the selection committee and any consulted specialists will be listed to further direct professionals in how to successfully select an appropriate screening measure for their school. The critical questions for identifying the purpose of the screening measure are the following:
What specific skill sets or competencies do we want to evaluate?
With which grade level(s) will the measure be used?
The answers to these questions will immediately narrow the field of potential screening measures. The response to Critical Question 1 will allow screening measures outside the target purpose to be eliminated. For example, if the purpose of the screening process is to identify students who are at risk for learning difficulties or disabilities, any screening measure that does not advertise this function may be discarded. Similarly, the answer to the second critical question can lead professionals to focus on screening measures developed for the target grade levels, further concentrating the pool of potential measures. The answers to these two questions should be considered immediately and used as a means of identifying potential screening measures to research. For each of the remaining substeps within Steps 1 and 2 of this process, the responses to the critical questions should be meticulously recorded, as they will be reviewed in depth in Step 3.
Determine available resources
School personnel must also determine the resources available for the screening process. Screening measures can vary in financial cost, time requirements, and personal labor (Kettler, 2007; Lilles et al., 2009; VanDerHeyden, 2013). Measures should be selected that use the minimum quantity of resources while ensuring the maximum level of effectiveness. Therefore, schools should decide what resources are available by consulting budget or resource specialists within the school and then select the most efficient and accurate screening measure accessible with these resources.
Critical questions to determine what resources are available are the following:
What financial resources are available?
How much time do we have to devote to screening?
Which staff members are available to screen?
Do we have any other resources at our disposal or any other constraints to consider?
These critical questions, like those in the previous section, allow professionals to exclude various measures from the group under consideration. Specifically, the first question on available resources focuses the search on screening measures that will not exceed the defined financial limit. The selection process can be further concentrated with the second and third questions by eliminating any measures that require too much time or personnel for their administration. The fourth question in this segment will be helpful in identifying any other constraints that should be considered when identifying the best fit in Step 3.
Step 2: Research Screening Measures
Following reflection on the factors of the school and the students to be screened, educators should research various characteristics of potential screening measures. In particular, attention should be paid to (a) the required resources of the screening measures, (b) the consultation of multiple sources of information to look at students’ backgrounds and contexts, (c) the relative biases of measures, (d) how the measures suggest to conduct further assessments of low scoring students, and (e) findings about test metrics of the screening measures. School staff members conducting this research should consider the critical questions following each substep when gathering necessary information on various screening measures from both the publishers of the measures themselves and from external evaluators. This information should be organized to be presented to the professional or committee leading the selection process.
Research required resources
After deciding what resources are available for the screening process, schools should research the resources needed for the implementation of existing screening measures. Ideally, schools should select the most efficient screening measure accessible with the available resources. One method of conducting universal screening that requires relatively low levels of financial resources and professional training for implementation is the teacher rating scale, which is employed by several established screening measures. Studies have also consistently found support for the accuracy of teacher rating scales to identify at-risk students (Gansle & Noell, 2008; Gray, McCallum, & Bain, 2009; Kettler, 2007; Kettler & Elliott, 2010). Teachers have the opportunity to interact with students on a daily basis and have the knowledge to compare students’ achievement levels to age- and grade-level norms (Kettler, 2007; Kettler & Elliott, 2010). They can also consider the whole child when rating him or her, outside of the narrow range of academic skills (Gray et al., 2009). As compared to spontaneous teacher referrals, these rating scales offer a much more structured platform through which teachers can evaluate their students’ performance and achievement. Often, screening measures that employ teacher rating scales as either the primary assessment or a subcomponent of an evaluation, as with the Academic Competency Evaluation Scales, are especially effective for use with younger students (DiPerna & Elliott, 2000; Kettler & Elliott, 2010). On the other hand, teacher rating scales require relatively high levels of labor resources, as teachers must complete an individual rating scale for each of their students, rendering these less useful for certain schools. However, there is a broad selection of measures that may be more applicable for schools with different types and quantities of resources available for the implementation of the screening measures.
Critical questions to investigate necessary resources are the following:
What financial resources are needed?
How much time does this measure require?
Which and how many staff members are needed to administer or complete the screening measure?
Are any other resources needed?
Professionals should record the answers to each critical question for future consideration in selecting a screening measure. The school staff should document the financial cost, the time requirement, the type and quantity of staff that will be needed to administer each of the measures, and any other resources necessary to implement the measures. At this point in the process, the only action required is the documentation of these responses, as the consideration and decision portion will be accomplished later in the procedure.
Look at multiple sources of information to explore students’ context
Another consideration when selecting a universal screening measure is the established effectiveness of gathering information from multiple sources. Though this, too, may require more resources in exchange for greater accuracy, it is something that should be specifically considered as it has been consistently supported in the literature. Studies have shown that accuracy can be increased by the use of both multiple procedures—such as standardized assessment, rating scales, and observations—and multiple informants—such as parents, teachers, other professionals, and the students themselves (Compton et al., 2010; Gray et al., 2009; Lilles et al., 2009).
Each screening method and each informant can comment only on portions of the student. By combining several of each, the school can gain a more complete picture of the student as a whole. Standardized, norm-referenced assessments gather information only on specific skills sets for comparison with age- or grade-level peers. This procedure fails to consider the child’s varied and nuanced strengths, weaknesses, and life experiences (Gansle & Noell, 2008; VanDerHeyden, 2013). For instance, though two students may receive the same score on a screening measure, one student may be much more resilient, responsible, or persistent than the other, meaning that these two students will require different types of support to achieve academically (Oakland & Rossen, 2005). By gathering information about the child as a whole, the real problems and possible solutions can be identified. Similarly, observations of students can provide information only about a student’s performance in a specific environment on a particular day. The perspectives of individual informants are also often lacking in observation- or rating scale–based screening methods. For instance, although parents have seen their child develop, they lack the ability to compare their child’s achievement to grade-level norms, while teachers have contrasting capabilities (Gray et al., 2009; Oakland & Rossen, 2005).
Furthermore, students’ backgrounds can affect their performance in school, either academically or behaviorally. Students from particular localized geographic areas often share similar backgrounds. By gathering information about the local geographic area or standardizing assessment measures for this region, some of the students’ contexts can be taken into consideration in the screening process (Oakland & Rossen, 2005; Scott & Delgado, 2003).
Critical questions to consider in looking at the sources of information that screening measures assess and how these measure address students’ context are the following:
Does the screening measure use multiple procedures (i.e., standardized assessment, rating scales, observations) and multiple informants (i.e., parents, teachers, student, other professionals)?
Does the screening measure consider students’ backgrounds or contexts?
Is the screening measure standardized for localized geographic norms?
Again, the answers to these questions should be recorded by school staff members. Ideally, measures should employ multiple procedures and informants, consider students’ backgrounds, and be standardized for local geographic norms. However, various screening measures will possess different combinations of strengths and weaknesses that can be weighed in more depth in Step 3 of the screening measures selection process. The developers of some measures may not present the responses to some of the questions in this section and those in the following sections. In this case, professionals should attempt to locate these answers in external research or evaluations when possible. New measures may have undergone less research, resulting in insufficient information to answer these critical questions. The lack of information should be noted but should not necessarily be cause for elimination of the measure.
Examine relative biases of screening measures
School personnel should also consider the relative bias of possible screening measures. Such bias might be affected by variance in language proficiency, socioeconomic background, or previous education. Though some research has been conducted on the biases of various measures, this is still a difficulty for some established screening measures (Gray et al., 2009). In selecting screening measures for use in schools, professionals should consult the existing research on the biases of different measures. The simple presence of research on various biases for a screening measure may be evidence of its merit, as many measures lack any research on this topic. However, professionals should seek out all research on the matter and personally evaluate the according value of various screening measures. Some measures have been designed with the specific aim of reducing particular types of bias. For example, the Universal Nonverbal Intelligence Test–Gifted Screening Scale was developed to minimize test bias for children with communication difficulties or limited English language proficiency (Gray et al., 2009). Depending on school characteristics, the issue of bias may be more or less important to consider when selecting screening measures. Specifically, it is imperative to consider the quality of core instruction within a school so variance in prior education does not create bias in the evaluation of screening measures.
Critical questions to examine the relative biases of screening measures are the following:
Can this screening measure be administered in multiple languages?
Does this screening measure have an alternate form that can be administered nonverbally?
Has cultural bias of this measure been studied?
Professionals should obtain more than a simple confirmation or negation for critical questions in this section. For instance, if the response to the first question is found to be affirmative, school staff should then record the languages in which the measure can be administered as well as any research on the quality of the measure in the alternate languages. Similarly, professionals should explore how a nonverbal alternate form would be administered and any research on the alternate form. The determined level of cultural bias and how this was established should also be recorded. It may be helpful to retain citations of any references used in answering these questions to allow for consultation in the decision-making process to follow. Similar to the responses in the prior section, the answers to these questions should not be used as a vehicle for immediate elimination; rather, they should be carefully weighed in the decision-making process. Preferably, each of these questions will be answered positively and supported by sufficient and quality research, although measures that are found to be less than ideal in this category may still be the best fit for a particular school due to strengths or capabilities in other areas.
Think about further assessments
When looking at screening measures, schools should also think about further assessments or evaluations to employ with students who score poorly on initial measures in order to determine the necessary supports or interventions for the individual students. Research supports the use of a three-tiered RTI approach to screening and to intervention, in which assessment and intervention become increasingly intensive as a child progresses through the tiers (Kettler, 2007). Typically developing students do not require evaluation or support beyond the first tier, while students with greater academic difficulties or disabilities will progress on to Tiers 2 and 3 as necessary (Kettler, 2007). Specifically, students who are identified as at risk by the initial, universal screening measure can be further evaluated through teacher, parent, or student interviews; more specialized or intensive assessment measures; careful review of the student’s previous development; or additional formal progress monitoring measures (Gansle & Noell, 2008; VanDerHeyden, 2013). Some screening measures, such as the Brief Academic Competence Evaluation Screening System, incorporate assessments for Tiers 2 and 3 to be used when necessary (Kettler, 2007; Kettler & Elliott, 2010). Through this process, individual supports or plans can be designed to help each student achieve to his or her potential. When selecting a screening measure, it is important for school personnel to identify the next steps for low-scoring students prior to the implementation of the measure.
Critical questions to study how measures address further assessments for low-scoring students are the following:
Are further assessments included or recommended in this screening measure for low-scoring students?
Does this screening measure use a research-based, three-tiered model of assessment?
Often, the answers to these two questions can be drawn directly from the manual or from other publications written by the developers of the screening measures. If affirmative responses are found for either of these questions, professionals should record detailed descriptions of the further assessments or how the measures reflect three-tiered models, respectively. Measures with included further assessments for low-scoring students that are based on three-tiered models of assessment are preferred, although those offering specific external recommendations for further assessments are also valuable. Additional resources required for further assessments should be taken into account when weighing the measure’s costs and benefits.
Study test metrics
Finally, test metrics for potential screening measures should be carefully reviewed and considered when selecting a measure to implement in a school. Such metrics are often already present in the literature for professionals to research and utilize in comparing possible screening measures for use in their school. Several resources are also available to professionals in which these metrics for various measures have already been researched and compiled, such as the Center on Response to Intervention (2015) and Understood.org (2015). Important metrics to examine include tests of reliability, validity, sensitivity, specificity, and predictive power, which all affect the accuracy of screening measures. Reliability, for example, determines whether a screening measure consistently evaluates the same skill or construct. There are several types of reliability that schools can look for when selecting screening measures, such as item-total reliability, which is the correlation between individual items on a scale and the scale as a whole, and interrater reliability, which is the correlation between various raters’ or observers’ scores (Kettler, 2007). It is critical to look at reliability because if a screening measure does not dependably assess the same skill in the same way, then the results cannot be trusted.
Validity, on the other hand, is evidence of whether a screening measure actually evaluates what it sets out to evaluate. Types of validity to consider when selecting screening measures are construct validity, which is the extent to which a measure assesses what it intends to assess; concurrent validity, which is whether the results of the screening measure match up with results of other established sources; and predictive validity, which is the degree to which the screening measure matches up with a student’s later achievement or performance (Kettler, 2007; Scott & Delgado, 2003). Screening measures must also be valid for the target population. For instance, some measures may be valid only for specific grade levels or for certain language competencies. If screening measures are being selected for an elementary school composed of primarily English language learners, for example, the school personnel must examine the validity of measures for these specific populations.
Sensitivity and specificity are two metrics that measure how accurate a screening measure is in identifying the correct students. Sensitivity is a measure of the proportion of the students with learning disabilities or difficulties who are correctly identified by the screening measure, while specificity is a measure of how many children who are not at risk of academic difficulties are correctly identified by the screening measure (Scott & Delgado, 2003, 2006; VanDerHeyden, 2013). Once it is determined that reliability and validity levels are sufficiently high, sensitivity and specificity are the next metrics to check when selecting a screening measure. It is also important to know that as sensitivity increases, specificity decreases and vice versa, so the goal is to find the optimum balance between the two.
The final test metrics that will be discussed are the positive and negative predictive powers of screening measures. Although these measures are related to specificity and sensitivity, they are much more variable, depending on the participants, and measure probability of accuracy rather than the actual proportion of accurate identifications. Specifically, positive predictive power is a measure of the probability that a student identified by a screening measure as at risk academically actually does fall into this category, while negative predictive power is the probability that a student identified by the measure as typically developing is truly not at risk (VanDerHeyden, 2013). This metric would be one of the final measures to look at when selecting a screening measure because of its relative variance based on population and the dependence of its accuracy on the levels of reliability and validity of the measure. When selecting a screening measure, however, it is important to examine all metrics available to gain the greatest understanding possible before making a decision.
Critical questions in studying findings about test metrics are the following:
How reliable is this screening measure (i.e., item-total, interrater reliability)?
How valid is this screening measure (i.e., construct, concurrent, predictive validity)?
What measures of sensitivity and specificity have been found for this measure?
What levels of positive and negative predictive power does this screening measure have?
Responses to these questions are especially significant, as high values for these test metrics are necessary to ensure the function of a screening measure. Before answering the critical questions in this segment, professionals should review the meaning and importance of the various metrics. Further, it is important that the questions in this section are answered and considered in the order in which they are listed, as these values are influenced by those preceding. For instance, if the reliability is found to be poor in response to Question 1, it follows that the validity queried in Question 2 will also be lacking.
Step 3: Identify the Best Fit
Finally, school staff should consider all information gathered in Steps 1 and 2 to make an educated decision about which screening measure to implement in their school. Professionals may again assemble a committee to collaboratively decide on the ideal measure for their school, drawing on the individual expertise and knowledge of the various committee members. Specifically, educators should ensure that the screening measure fits well with the needs and capabilities of their school and that the measure has maximum benefits for minimum costs. While one school, for example, might determine the Dynamic Indicators for Basic Early Literacy Skills to be the optimal measure as a result of its high levels of predictive power, relative ease of use, and utility of assessing early literacy skills, another school may benefit more from the Brief Academic Competence Evaluation Screening System because of its more individualized assessment process and general academic focus, which could be more applicable to their screening purpose and student population (Good & Kaminski, 2002; Kettler, 2007; Kettler & Elliott, 2010). By carefully and intentionally selecting a screening measure supported by research that matches with the needs and characteristics of the school, professionals can maximize their students’ potential academic success (Lazarus & Ortega, 2007; Oakland & Rossen, 2005; VanDerHeyden, 2013).
Critical questions to address in finding the best screening measure for your school are the following:
Which screening measure best fits the needs and capabilities of our school?
Which screening measure provides the highest benefit-to-cost ratio for our purpose?
Answers to the final critical questions necessitate a review of the notes from each of the preceding sections. First, professionals should determine if the measures fit the needs and capabilities of their school by referencing the responses to questions on available and required resources. This process may allow the school staff to eliminate measures that require more resources than the school has available. When answering the second question, professionals should refer to the researched notes from the four sections on the sources of information gathered, the relative bias of measures, further assessments to use for low-scoring students, and the test metrics of the measures. Although this process may take different forms in various schools and cannot be completely prescribed, the ideal characteristics of screening measures as discussed in this article should be considered, as should the student population in the school. Specifically, a school with a large group of students from different linguistic or cultural backgrounds might need to place more value on measures with alternative languages or less bias than on measures with included further assessments. In addition, professionals should reflect on the quality and quantity of research backing each of the measures as part of their decision-making process. The strengths and weaknesses of each measure, as well as how these match with the student population and screening purpose, should be deliberated in depth to identify the best fit for each school.
Conclusion
As evidenced in the literature and through federal legislation, the implementation of universal screening measures in early education is imperative to ensure the greatest academic success for students (Individuals With Disabilities Education Improvement Act, 2004; Kettler, 2007; Lazarus & Ortega, 2007; Lilles et al., 2009; No Child Left Behind Act, 2001; Scott & Delgado, 2003). Identifying students who have academic difficulties or who are at risk of learning or other disabilities early allows for earlier interventions, which have been consistently shown to have greater effects on students’ later achievement and development than interventions implemented later in students’ education (Gansle & Noell, 2008; Kettler & Elliott, 2010; Scott & Delgado, 2006). Further, it has been shown that universal screening measures are more accurate and effective than spontaneous parent or teacher referrals, can reach a greater quantity of students, and can offer a relatively unbiased evaluation of students’ academic performance as compared to other identification processes (Gansle & Noell, 2008; Scott & Delgado, 2006; VanDerHeyden, 2013).
Though research is continually being conducted on individual screening measures, there is a great deal of information available for school personnel to access in the process of selecting a measure. Specific, concise steps to consider in this process of choosing a screening measure are to first consider school factors, then research screening measures, and finally, identify the best fit for the particular students in a school:
While the most appropriate screening measure to employ may vary from one school to the next, the administration of universal screening measures in every school is imperative. By following the steps outlined in this article, educators will ensure that each and every student receives the individual support needed to succeed to his or her greatest potential.
Footnotes
Acknowledgements
The author would like to thank Joshua Wilson, PhD, for his consultation in the completion of this article.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
