Abstract
A major tenet of both the Individuals with Disabilities Education Act and the No Child Left Behind Act is the identification and use of evidence-based practices, or those instructional techniques shown by research as most likely to improve student outcomes meaningfully. However, much confusion exists regarding the meaning and potential applications of evidence-based practices in special education. Evidence-based practices are traditionally supported by the findings of multiple, high-quality, experimental research studies. Rather than changing the nature of teaching or limiting teachers to following prescribed methods, prioritizing evidence-based practices will allow teachers to maximize the impact of their instructional efforts.
Keywords
It seems clear that the vast majority of teachers have the best interests of their students at heart and, working within the constraints imposed by the realities of schools (e.g., limited resources, extreme variability in student characteristics), frequently make valiant efforts to bring about desired outcomes for their students. Indeed, it is common to find special education teachers in their classrooms early in the morning, late into the night, and on weekends; purchasing instructional materials and pursuing professional development with their own money; and working closely with parents, administrators, and other specialists to ensure that they are meeting their students’ needs to the best of their abilities. In addition to these endeavors, most teachers put considerable thought into identifying and implementing best practices. The term best practice conjures up notions of instructional procedures that have been established as effective, merit one’s trust, and should be prioritized by educators wanting to optimize their students’ educational outcomes.
However, Peters and Heron (1993) noted that best practice became a ubiquitous term many years ago, used to describe a range of practices—including those championed on the basis of personal experience, opinion, and affinity in addition to scientific research. Because so many approaches have been referred to as best practices, including some that are not particularly effective, by such a multitude of sources (e.g., Web sites, textbooks, college instructors, in-service providers, colleagues), many teachers have developed a sense of confusion as to what practices actually are effective and who can be trusted to make such a determination. Due at least in part to this uncertainty, many well-intentioned teachers have been implementing ineffective practices, have not been using the most effective practices in their classrooms, or both (Kauffman, 1996).
The term evidence-based practice (EBP) (Odom et al., 2005) recently has been advanced to refer to practices that have been shown to be effective by credible research. Recent legislation has reflected the importance of utilizing EBPs in both general and special education. For example, one of the four pillars of the No Child Left Behind Act of 2001 (NCLB; 2002) is “proven education methods,” which emphasizes educational programs and practices that have been proven effective through rigorous scientific research (U.S. Department of Education, 2004). In fact, NCLB uses the term scientifically based research more than 100 times. Similarly, the Individuals with Disabilities Education Improvement Act of 2004 repeatedly highlights the need for teachers to be trained in scientifically based (i.e., evidence-based) instructional practices to improve the academic and functional performance of students with disabilities.
Despite the attention paid to EBPs in recent professional literature and legislation, the gap between research evidence and classroom practice persists in both general and special education (Cook & Schirmer, 2006). So, just how are teachers supposed to find out which practices are evidence based for students with disabilities? Criteria and standards for determining EBPs have been developed and applied in other fields such as general education (U.S. Department of Education, n.d.), school psychology (Kratochwill & Stoiber, 2002), psychology (Chambless et al., 1998), and medicine (Sackett, Richardson, Rosenberg, & Haynes, 1997), to name a few. Although the Council for Exceptional Children (CEC) is in the process of developing and applying standards for EBPs in special education (Bruno, 2007), the field has yet to establish an inventory of EBPs. When a list of EBPs in special education is developed, it seems that teachers will be more likely to hold those practices in high regard and make efforts to incorporate them into their daily teaching routines if they understand what evidence based means and just how a practice comes to be considered evidence based.
How Are Evidence-Based Practices Determined?
According to NCLB, the term scientifically based research refers to research that “involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs” (Title IX, Part A, Section 9101 [37]). To meet these requirements, research must be conducted that demonstrates that an educational practice is the cause of improved student learning. Only a few research designs allow one to draw reliable conclusions about whether a practice (e.g., classwide peer tutoring) improved student outcomes (e.g., improved reading fluency). Research designs that demonstrate experimental control—such as group experimental, quasi-experimental, and single-subject research—provide consumers with the greatest confidence that an instructional practice caused improved outcomes. The quality with which a research study is conducted (i.e., the methodological quality of the study) is another factor to consider when determining whether a practice is evidence based. Considering only high-quality research studies greatly increases one’s confidence in the collective findings of those studies. Moreover, the quantity of research supporting a practice should also be considered. Converging findings from multiple high-quality, experimental research studies provide the greatest confidence that a practice truly works. Taken together, an evidence-based practice can be identified when a sufficient quantity of high-quality research studies that demonstrate experimental control have been conducted and show that student outcomes are improved as a result of using the practice.
Experimental Control and Evidence-Based Practices
Different research designs result in specific types of information that answer different questions (Rumrill & Cook, 2001). For example, a qualitative research study might yield information regarding how specific teachers feel about implementing an instructional strategy or why they decide to maintain or discontinue their use of a teaching technique. Correlational research, on the other hand, can show the extent to which two variables (e.g., number of office referrals and academic test scores) are related. For example, correlational research might establish that attendance and achievement are positively related—that students who are high achievers tend to have high attendance rates. Although the research method can show the degree to which attendance and achievement are related, it cannot tell us whether attendance caused improved achievement or high achievement caused improved attendance. Qualitative and correlational studies provide important insights into teaching and learning, but they cannot determine meaningfully whether a practice is evidence based, that is, whether a practice causes desired change in student outcomes.
To claim that an educational practice causes a change in student outcomes, researchers must show that they have experimental control, that is, they have accounted for and ruled out alternative explanations for the change in student outcomes (i.e., explanations other than students’ receiving the educational practice being tested). Research designs that show experimental control can do so in two primary ways: (a) by systematically comparing the outcomes of a group who use the practice against a comparison (or control) group who does not use the practice or (b) by systematically comparing individuals’ performance with the practice in place against their performance when it is not used. Only research designs that demonstrate experimental control (i.e., group experiments, quasi-experiments, single-subject research, and derivations of these research designs—research designs collectively referred to as experimental research) allow one to confidently conclude that the use of a practice causes changes in student outcomes. Although experimental research cannot indicate why a teaching practice works or how teachers and students feel about an instructional approach—as qualitative research can—it is unique in that it demonstrates experimental control and can, therefore, reliably determine whether a practice is effective for improving student outcomes. For a more detailed discussion of experimental research and causality, see Creswell (2005) and Rumrill and Cook (2001).
Reflecting the importance of experimental control, the What Works Clearinghouse (U.S. Department of Education, n.d.) and the National Association of School Psychologists (NASP) (Kratochwill & Stoiber, 2002) consider only experimental research in determining which practices are evidence based for general education and for school psychologists, respectively. It is not surprising, then, that plans for determining evidence-based practices in special education propose to do so on the basis of the findings of these same research designs (Bruno, 2007; Gersten et al., 2005; Horner et al., 2005). Basing decisions of whether practices are evidence based on the findings of research designs that exhibit experimental control does not imply that these research designs are better than others or that other research designs (e.g., qualitative, correlational) should be thought of as less important. Rather, this is an example of how each research design is uniquely suited to answer particular types of questions. Whether a practice causes changes in student outcomes is a rather narrow question that is best addressed by experimental research.
Quality of Research
Concluding that a practice is evidence based requires that the research studies supporting it not only exhibit experimental control but also are of sufficiently high quality for the results to be trustworthy. If research is not conducted properly, the findings can be meaningless and misleading. For example, imagine a group experimental study in which an experienced teacher instructed the students in the experimental group (i.e., those receiving the intervention) while a novice teacher instructed students in the control group (i.e., those not receiving the intervention). If the performance of the experimental group is found to increase more than that of the control group throughout the course of the study, because teachers were not comparable across conditions, one has no way of knowing whether it was the intervention or the teacher that caused the higher gains in the experimental group. Although there is not, as of yet, a clear consensus as to the methodological characteristics needed to ensure that experimental research in special education is trustworthy, groups of prominent special education researchers recently proposed a number of quality indicators (i.e., characteristics found in high-quality research) for group experimental (Gersten et al., 2005) and single-subject (Horner et al., 2005) research in special education. When making decisions as to whether a study is of sufficient quality to be trustworthy, and therefore considered in determining whether a practice is evidence based, the field of special education will have to agree on and systematically apply a set of quality indicators such as those proposed by Gersten et al. (2005) and Horner et al. (2005).
Quantity of Research
Educational research cannot provide absolute proof that an intervention is effective. Instead, the findings of an experimental study can either add support to or weaken the hypothesis that an intervention causes meaningful changes in student outcomes. The more high-quality experimental studies that support a practice, the greater the confidence one has that it causes desired changes in student outcomes. Findings from one study, even if it is of very high quality, provide insufficient evidence for the field of special education to recognize a practice as evidence based. As opposed to clinical research conducted in tightly controlled laboratory settings, intervention research in special education takes place in schools and classrooms and therefore involves any number of contextual variables beyond the control of researchers. For example, even in a high-quality experimental investigation, the experimental group might, on average, have better lighted work areas, encounter less noise and fewer distractions, receive extra assistance from a particularly helpful paraprofessional, or differ from the control group on a host of other variables not measured or controlled for by the researcher. Accordingly, converging evidence from multiple high-quality studies is needed to recognize a practice as evidence based.
Gersten et al. (2005) and Horner et al. (2005) also provided recommendations for the number and type of high-quality group experimental and single-subject studies, respectively, needed to consider a practice evidence based. For example, Gersten et al. suggested that two high-quality group experimental studies supporting a practice are necessary to consider a technique evidence based. Horner et al. advised that for a practice to be considered evidence based on the basis of single-subject research, the technique must be supported by a minimum of five high-quality single-subject studies that (a) were conducted by at least three different researchers in at least three different locations and (b) included a total of at least 20 participants. Although these specific standards have not yet been widely applied or endorsed, they are consistent with the notion that multiple experimental studies of high quality are necessary for a practice to be deemed evidence based.
What Does Being an Evidence-Based Practice Mean?
The essential steps in determining whether a practice is evidence based seem to be (a) locating the high-quality, experimental research that examines the effectiveness of the practice and then (b) determining whether sufficient quantity of evidence showing that the practice causes improved learner outcomes exists. Once the field has identified EBPs through such a process, what does that mean and how will they be used in classrooms?
Are Evidence-Based Practices Guaranteed to Work?
Being evidence based does not mean that an instructional practice is guaranteed to work for every student in every situation. For example, the findings from a group experimental study might demonstrate that students who received an intervention achieved superior outcomes on average in comparison to similar students in the control group who did not receive the intervention. That does not mean that every student in the experimental group excelled following the implementation of the intervention. Indeed, in most studies investigating the effects of even the most effective interventions, the performance of a small number of nonresponders does not meaningfully increase despite receiving the intervention (McMaster, Fuchs, Fuchs, & Compton, 2002). Teachers should not, then, expect every student to respond favorably to EBPs even when appropriately applied.
Despite this caveat, when implemented as designed, educators can be confident that EBPs provide the highest likelihood of improving student outcomes. To use a gambling analogy, implementing an EBP is akin to placing a bet that is 95% likely to pay off, whereas implementing a practice shown by research not to be evidence based may have, for example, only a 50% chance of paying off. Even though a payoff is not guaranteed with either strategy, it is clear which is the best bet in the high-stakes game of a child’s education. Accordingly, EBPs should be the first option for responsible teachers, administrators, teacher–educators, and policy makers for generating improved student outcomes. The reality that no instructional practice, even those that are evidence based, is guaranteed to work with every student underscores the need for using formative assessment (Deno, 2006). By monitoring the ongoing performance of students, nonresponders can be readily identified and instruction modified as needed.
Will Practices That Are Not Evidence Based Be Prohibited?
It seems neither possible nor desirable to mandate that special educators use only a limited array of practices, even if those practices are evidence based. Given the unique and highly varied needs of students with disabilities, special educators will need to remain flexible in their instructional approaches and seek out innovative practices for some students who do not respond to EBPs. Moreover, any list of EBPs will be a work in progress that may change when additional research is conducted or examined. Indeed, rather than having been conclusively shown not to work, some teaching techniques and instructional approaches may not be recognized as EBPs because (a) a sufficient number of high-quality, experimental studies have not been conducted regarding their effectiveness or (b) researchers have not systematically reviewed the existing body of literature regarding the practice. In fact, at present, the special education community does not unequivocally endorse any practices as evidence based, because no established procedures are in place to determine EBPs in special education. Accordingly, any inventory of EBPs will be too limited for teachers to rely on exclusively when meeting the diverse needs of all students with disabilities.
Nonetheless, teachers aiming to maximize student outcomes should strive to use practices that have been demonstrated to be extremely likely to work (i.e., EBPs) whenever possible. Evidence-based practices can form the highest level of a hierarchy of teaching practices, which teachers prioritize when selecting instructional techniques. That is, whenever possible, teachers should implement EBPs, as these are most likely to result in desired student outcomes. Only in cases in which no relevant EBPs have been identified or progress monitoring has indicated that all relevant EBPs did not produce appropriate outcomes should non-EBPs be used. Among non-EBPs, teachers should prioritize techniques that are supported by (a) high-quality research (e.g., a single high-quality experimental study or multiple high-quality correlational and/or qualitative studies), (b) theory, and/or (c) other compelling evidence (e.g., objective data showing that the technique produced desired outcomes for the student or similar students in the past). When teachers implement non-EBPs, it is critically important that they closely monitor student outcomes to determine if the student is making adequate progress toward reaching his or her goal and, if not, to quickly identify when a change in instruction is needed.
Will Evidence-Based Practices Be Easily Adopted?
Knowing what works and doing what works are separate considerations. Just because a practice has been identified as effective does not necessarily mean that many teachers will use it as designed over time. Indeed, although it is important to determine which practices are evidence based, it is just as important that researchers determine whether teachers find EBPs acceptable, which aspects of EBPs teachers find problematic, and how teachers successfully adapt EBPs to work in their classroom and with their students. Surveys, observational research, and qualitative interviews with teachers are uniquely suited to provide this critical information.
Because the most effective practices may not be routinely used in contemporary classrooms (Cook & Schirmer, 2006), implementing EBPs may necessitate that many teachers change their instructional routines and adopt new techniques, a transformation many will find difficult. Research has found that teachers are more likely to adopt and sustain effective practices when supports include ongoing professional development, feedback on implementation of practice, collaborative support of others implementing the practice, and student outcome data to assess and demonstrate the impact of the practice (see Wanzek & Vaughn, 2006, for a review of this literature). Although some exceptional teachers may be able to adopt EBPs on their own, large-scale implementation of EBPs will almost certainly necessitate that teachers are provided with systemic supports.
Although a great deal of responsibility for implementing evidence-based practices lies with teachers, other stakeholders have critical roles to play as well (Tankersley, Landrum, & Cook, 2004). For example, universities should base teacher training on EBPs rather than provide preservice teachers with mixed messages about best practices. Researchers must also (a) conduct more high-quality, experimental research investigating the effectiveness of instructional practices for students with disabilities (Gersten, Baker, & Lloyd, 2000) and (b) synthesize existing experimental studies to determine whether there is sufficient evidence to consider practices evidence based (Lloyd, Pullen, Tankersley, & Lloyd, 2006).
Conclusion
Teachers make the ultimate determination regarding the instruction students receive. University professors, administrators, in-service providers, textbooks, and research articles recommending EBPs have no impact on student outcomes without teachers who believe that the practices really do work and are worth their time and effort to implement. Most individuals, including special educators, are more receptive to innovations when they understand the rationale behind them and their intended effect. Accordingly, we have attempted to provide a basic understanding of how practices come to be considered evidence based and how EBPs might be implemented in special education. See Table 1 for a summary of issues related to evidence-based practices in special education.
Summary of Issues Related to Evidence-Based Practices in Special Education.
Rather than limiting teacher freedom and effectiveness, EBPs can provide a liberating framework that enables teachers to maximize their students’ achievement. Evidence-based special education will in no way obviate the need for devoted and skilled special educators. Indeed, special educators will have to be as quick thinking and innovative as ever as they endeavor to implement EBPs. At the heart of special education is the notion of individualization. Rather than following prescribed techniques by rote, implementing EBPs will require teachers to skillfully apply their professional wisdom. For example, special educators will have to adapt EBPs to their own instructional strengths, to the environmental demands of their schools and classrooms, and to the unique needs of the students without compromising the core elements of the practices that make them effective (Cook, Tankersley, & Harjusola-Webb, 2008). Utilizing EBPs will not alter the essential roles and functions of special education teachers, but it can allow them to maximize the impact of their efforts on student outcomes.
Footnotes
Editors’ Note
The fifth decade of Intervention in School and Clinic represents a strong field just getting stronger. In this decade, special issues were published dealing with specific topics ranging from transition to math strategies, technology, diversity, and research for educators—all with a strong emphasis on the implementation of evidence-based practices. We were inspired by the work of authors such as Dr. Peggy King-Sears, Dr. Moira Konrad, Dr. Kristin Sayeski, Dr. Greg Conderman, Dr. Sharon Field, and Dr. Cari Dunn—just to name a few. Because this decade included articles we had edited, it was a pleasure to revisit the writing of the 8 years represented, and we look forward to the next 2 years that will round out the decade. It is exciting to read the work of new writers in our field and watch their work develop and become as influential as the research of the authors from the first decades of the journal. Because the focus of Intervention is on the translation of research into practice and the culmination of that translation is an evidence-based practice, we selected an article by Dr. Bryan Cook, Dr. Melody Tankersley, Dr. Lysandra Cook, and Dr. Timothy Landrum to represent this decade. The article provides an excellent overview for educators concerning evidence-based practices as well as clears up questions that may be asked.
This paper was originally published as: Cook, B. G., Tankersley, M., Cook, L., & Landrum, T. J. (2008). Evidence-based practices in special education: Some practical considerations. Intervention in School and Clinic, 44(2), 69–75.