Searching for Evidence-Based Practice

Abstract

In an environment of accountability, the development of evidence-based practices is expected. To demonstrate that a practice is evidence based, quality indicators of rigorous methodology should be present including indications that teachers implementing an intervention have done so with fidelity to its design. Because evidence-based practices assume fidelity of implementation (FOI), failure to establish FOI limits the conclusions that can be drawn from any outcome evaluation. This study surveys the gifted education literature to ascertain the degree to which FOI has been assessed, reported, and tied to measured outcomes in curriculum intervention efficacy studies and outcome evaluations. Of the 11 curriculum intervention studies included in this survey, each addressed FOI. The nature and quality of the methods used, however, to measure FOI, as well as the degree to which fidelity data were reported, varied widely among these studies, which suggests the need for increased methodological rigor in gifted curriculum research.

Keywords

fidelity of implementation gifted education curriculum evidence-based practices

Fidelity of implementation (FOI) is broadly defined as determination of how well an intervention is implemented in comparison with the original program design during an efficacy and/or effectiveness study (see O’Donnell, 2008, for a thorough review of the fidelity literature). In certain traditional fields of research, including public health, measuring FOI and establishing its relation to outcomes have been recognized as essential to demonstrating the effectiveness of an intervention. This is because it is only by gauging whether all elements of an intervention have been faithfully implemented that researchers and practitioners can understand whether it has contributed to intervention outcomes (Carroll et al., 2007; Keller-Margulis, 2012).

In contrast to its more robust consideration in the public health literature, FOI is a relatively nascent construct in K-12 curriculum research. In fact, surveys of reported large-scale studies examining the efficacy and effectiveness of K-12 curriculum interventions rarely report FOI, and even more rarely report how FOI enhances or limits the effects of the intervention on outcomes, thus limiting confidence that external and internal validity have been established (Mowbray, Holter, Teague, & Bybee, 2003). In light of these findings, educators, researchers, and policy makers increasingly expect educational researchers to measure and report FOI to K-12 curriculum intervention efficacy or effectiveness studies in the development and adoption of evidence-based practices (Cook & Cook, 2011; Jolly & Kettler, 2008; Slavin, 2002; Walsh, Kemp, Hodge, & Bowes, 2012). For example, the Institute of Education Sciences (IES) at the U.S. Department of Education requires researchers to evaluate and report FOI, at least to a basic extent, in funded curriculum intervention efficacy research to demonstrate that a practice is evidence based (U.S. Department of Education, 2003a).

O’Donnell’s (2008) review of the FOI literature addressed K-12 curriculum interventions broadly. It did not specifically delineate curriculum interventions for gifted learners. In addition, the way FOI is studied has developed and changed significantly in the past 6 years. Thus, the degree to which FOI has been measured and reported in curriculum intervention studies implemented in gifted classrooms is poorly understood and warrants exploration at this time.

FOI in Evidence-Based Practices

The importance of measuring and reporting FOI can be understood in the more general context of establishing that a curricular intervention constitutes an evidence-based practice. Broadly speaking, evidence-based practices consist of clearly described curricular interventions, programs, and instructional techniques with methodologically rigorous research bases supporting their effectiveness (Cook & Cook, 2011; Horner, Sugai, & Anderson, 2010; Walsh,et al., 2012). In order for an educational practice to be considered evidence based, certain quality indicators of rigorous methodology should be present (Cook, Tankersley, & Landrum, 2009; Gersten et al., 2005). These quality indicators for evaluating research typically address (a) the nature of the research design (e.g., group experimental, quasi-experimental), (b) the quality of the research study (e.g., adequate FOI shown, psychometrically sound outcome measures used), (c) the quantity of studies documenting an experimental effect (i.e., replicated findings), and (d) a consequential magnitude of effect on student outcomes.

When evaluating whether a K-12 curriculum intervention study satisfies the quality of research study criterion, there is growing recognition among educational researchers of the value of both measuring and reporting FOI, particularly since the O’Donnell (2008) review (Century, Rudnick, & Freeman, 2010). Because it refers to the extent to which delivery of a curricular intervention is consistent with the intervention as originally designed, FOI seeks to answer the question of whether teachers charged with implementing an intervention have done so with integrity to its curricular design and theoretical underpinnings (Azano et al., 2011). The failure to establish FOI in a curriculum intervention study limits the conclusions and generalizations that can be drawn from any outcome evaluation and limits any assertion of methodological rigor (Carroll et al., 2007; O’Donnell, 2008).

To determine whether a study adequately establishes FOI for purposes of documenting its methodological quality, it is important to assess not only whether FOI was measured but also how it was measured (Gersten et al., 2005; O’Donnell, 2008). At a minimum, researchers should explicitly identify the most central or “critical” components of an intervention, differentiate the intervention from business-as-usual conditions, and then regularly observe and record teachers utilizing those components over the course of the study (Century et al., 2010). Preferably, methods for fidelity assessment should be multidimensional (e.g., researcher observation checklists, teacher self-report instruments, student work-product) and multi-informant (e.g., teachers, students, researchers). Records generated by researchers through observations, as well as the methods used to record fidelity, should be described. Ideally, these records should provide a measure of interobserver reliability that is reported with the results of a study (Gersten et al., 2005). Researchers (Keller-Margulis, 2012) further recommend that fidelity observations should be scheduled to appear at both predictable and unpredictable times to capture actual teacher implementation. In addition, researchers are encouraged to both measure and report the quality with which teachers employ the techniques embedded within an intervention along with moderating variables that constrained implementation fidelity such as lack of time, lack of resources, and/or low teacher expectations about student ability to engage with an intervention (Little, Feng, VanTassel-Baska, Rogers, & Avery, 2007). Finally, using fidelity data to determine their relation to student outcomes has been recommended (Azano et al., 2011; Century et al., 2010; O’Donnell, 2008). While observing and recording teacher methods for FOI may or may not translate to effect on student outcomes, it is essential to collect these data to understand any such effects.

Evidence-Based Practices in Gifted Education

In the era of accountability, a principal contention in the field of gifted education is that gifted students require comprehensive differentiated curricula and instruction shown to effectively develop their abilities (Hertberg-Davis & Callahan, 2013). Thus, researchers in the field (VanTassel-Baska, Robinson, Coleman, Shore, & Subotnik, 2006; Walsh et al., 2012) urge the development and deployment of rigorously conducted curriculum intervention studies to demonstrate that recommended curricula and practices for gifted learners are evidence based. Additional specific recommendations are that curriculum evaluation studies should be designed to provide data that both demonstrate measurable academic growth for gifted learners (Gallagher, 2006; VanTassel-Baska & Brown, 2007) and allow teachers and educational decision makers to support their use in gifted education classrooms (Coleman, Gallagher, & Job, 2012; Klimis & VanTassel-Baska, 2013), thereby bridging the research to practice gap. Replication studies that provide documentation of a valued effect across multiple contexts and diverse groups have also been encouraged (Dai, Swanson, & Cheng, 2011; Makel & Plucker, 2014). Finally, several researchers have identified FOI as a critical indicator of evidence-based practices in gifted education (Callahan & Moon, 2007; VanTassel-Baska, 2013), and problems associated with inconsistent treatment fidelity in implementing curriculum interventions (such as teacher dropout and lack of time and resources) have been recognized (Ambrose, VanTassel-Baska, Coleman, & Cross, 2010; Azano et al., 2011; Feng, VanTassel-Baska, Quek, Bai, & O’Neill, 2007).

To understand the state of the field’s engagement in methodologically rigorous research, some researchers have surveyed the extant empiric literature. Generally, these surveys report the scarcity of research where quality indicators of evidence-based practices are present (Callahan & Moon, 2007; White, Fletcher-Campbell, & Ridley, 2003). For example, limited reporting of effect size estimates has been identified as a serious issue undermining the quality of empiric research in the gifted education literature (Gentry & Peters, 2009; Matthews et al., 2008). The limited number of studies conducted using experimental designs (Dai et al., 2011; Walsh et al., 2012) and the absence of a well-established empiric research base supported by replication studies (Jolly & Kettler, 2008; Makel & Plucker, 2014) have also been documented in the field. Collectively, these surveys have drawn attention to the state of evidence-based practices in gifted education research and the need to develop a more rigorous research base documenting the efficacy of curricular interventions for gifted students (Dai et al., 2011). However, while ease of implementation and teacher receptivity to curricular interventions have been described as critical to demonstrating effectiveness (VanTassel-Baska & Brown, 2007), the degree to which researchers in the field have systematically addressed, measured, and reported FOI data (or their relation to outcomes) in the context of curriculum interventions studies remains unclear, and no survey on this indicator of evidence-based practices in the gifted education literature has been undertaken.

The purpose of this article is to add to the literature addressing the use of quality indicators of evidence-based practices in gifted education research (Jolly & Kettler, 2008; Matthews et al., 2008). More specifically, we surveyed the gifted education literature to ascertain the degree to which FOI has been assessed and reported in curriculum intervention efficacy studies and outcome evaluations in the field of gifted education.

Method

Search Procedures

We engaged in an examination of the extant literature reporting studies evaluating the efficacy of experimental curriculum units, frameworks, and/or lessons designed for gifted students on learning outcomes. The review of the literature reported here first involved an examination of four journals: Gifted Child Quarterly (GCQ), Journal of Advanced Academics (JAA), Journal for the Education of the Gifted (JEG), and Roeper Review (RR). The rationale for using these databases was that most publications in them contain peer-reviewed journal articles that report original empirical studies (Dai et al., 2011). Moreover, these journals are often cited as leading peer-reviewed journals publishing primary research in the field of gifted education (Jolly & Kettler, 2008; Matthews et al., 2008; Walsh et al., 2012). We examined hard copies of all issues of these journals published from 2004 through 2013. The year 2004 was selected as it immediately followed the release of the Study Design and Implementation Assessment Device (DIAD; Valentine & Cooper, 2003) by the What Works Clearinghouse of the U.S. Department of Education. A major goal of the DIAD is to assist researchers in their ability to evaluate whether published research demonstrates sufficient quality to be considered an evidence-based practice. Thus, this survey addresses research in the decade since the release of the DIAD.

To locate additional empirical studies from 2004 through 2013 evaluating the efficacy of specific curriculum units and/or lessons designed for gifted learners reported outside of the leading gifted education journals, we also completed keyword and title searches of Academic Research Complete and Google Scholar using combinations of the following keywords: gifted, high ability, advanced learner, curriculum, lessons, experimental design, quasi-experimental design, fidelity, implementation, treatment fidelity, treatment integrity, intervention, efficacy, effectiveness, and adherence.

Inclusion Criteria

For inclusion in this survey, a publication had to meet several criteria reflecting the quality indicators described above. First, the publication had to be a primary research paper evaluating the outcomes of a curriculum intervention for gifted learners published in a peer-reviewed journal between 2004 and 2013. Second, the curriculum intervention study had to have utilized an experimental or quasi-experimental research design. Although randomized control designed studies are considered to be the “gold standard” in demonstrating that a practice is evidence based (Cook & Cook, 2011), the difficulties of conducting these studies in K-12 classrooms has been acknowledged. Consequently, reported studies utilizing either experimental or quasi-experimental designs were included in this survey as they are generally considered of sufficient quality in educational research to support a determination that a practice is evidence based (Cook et al., 2009). As a third inclusion criterion, the study had to report efficacy or effectiveness data on the curriculum intervention (Walsh et al., 2012).

All empiric studies from 2004 through 2013 that satisfied the inclusion criteria were incorporated into a matrix. In all, 11 curriculum intervention studies were located that met the specified criteria and were indicated in the matrix according to the relevant authors, titles, journals, and curriculum models.

Categories of FOI Assessment and Reporting

Next, we adapted the matrix by categorizing studies according to the method(s) of their FOI assessment and the degree to which the study reported FOI data (if at all; Century et al., 2010; Gersten et al., 2005; Keller-Margulis, 2012). More specifically, we first categorized studies according to whether the authors of a study indicated that FOI had been assessed. For example, researchers in one study included in this survey represented that “professional development staff visited each classroom once a week across the approximate 12 weeks of the intervention to document fidelity of implementation including adherence to the unit content and instructional strategies” (Gavin, Casa, Firmender, & Carroll, 2013, p. 77). Thus, a simple representation by the researchers of a published study that FOI had been considered sufficed for inclusion in this category.

Next, we again adapted the matrix to indicate whether a study identified the critical components of the intervention (“Critical Components”). Those studies addressing critical components provided specific descriptions of the theoretical and pedagogical underpinnings of an intervention, the curricular materials used, and the specific instructional strategies employed as they were intended to influence outcomes of an intervention. In addition, we delineated the matrix to indicate whether the authors differentiated the intervention from “business as usual” conditions in control classrooms by describing the curricula and instructional strategies used in them (“Program Differentiation”; Century et al., 2010).

Next, we further adapted the matrix according to whether the authors identified the method(s) used to measure FOI (“Method for FOI Assessment”) and, if so, how FOI was measured. Within the “Methods for FOI Assessment” category, we established codes to identify the variety of methods used by the researchers to assess FOI that have been recommended in the literature (Century et al., 2010; Gersten et al., 2005) as follows: researcher observation of experimental group teachers, researcher observation of control group teachers (ROCT), frequency of observation described (FO), observation protocol or fidelity instrument used (OP), observation protocol or fidelity instrument described (FID), researcher interview of experimental teacher (RIET), predictable and unpredictable times for observation used (P/U), intraobserver or interrater reliability of teacher implementation observation established (IR), teacher self-report of FOI (TS), moderating variables limiting implementation fidelity identified (MV), and quality of teacher implementation assessed (QI).

Finally, we adapted the matrix to indicate whether (and how) FOI data were reported (“FOI Reported”). For example, this category identified those studies where qualitative or quantitative fidelity data were reported or where FOI data were related to measured outcomes.

The matrix illustrating the development of these categories is reported in Table 1 and is discussed in the “Results” section.

Table 1.

Fidelity of Implementation (FOI) Categories.

Author(s) and journal	Curriculum model	Critical components/program differentiation	Method for FOI assessment	FOI data reported
Gavin, Casa, Firmender, and Carroll, (2013). The Impact of Advanced Geometry and Measurement Curriculum Units on the Mathematics Achievement of First-Grade students. GCQ	Project M2	Yes/No	ROET, FO (weekly)	None
Gavin, Casa, Adelson, and Firmender (2013). The impact of challenging geometry and measurement units on the achievement of Grade 2 students. Journal for Research in Mathematics Education	Project M2	Yes/Yes	ROET, FO (weekly), OP, TS (after every lesson)	None
Aljughaiman and Ayoub (2012). The Effect of an Enrichment Program on Developing Analytical, Creative, and Practical Abilities of Elementary Gifted Students. JEG	Oasis enrichment model	No/No	ROET	None
Azano et al. (2011). Exploring the Relationship Between Fidelity of Implementation and Academic Achievement in a Third-Grade Gifted Curriculum: A Mixed-Methods Study. JAA	CLEAR curriculum model	Yes/No	ROET, FO (three times), RIET (three times), OP, FID, ROCT, TS, MV, QI	Qualitative data from TS, ROET, and RIET; QI data related to measured student outcomes; MV data (teacher perceptions of time, teacher expectations, teacher perceptions of autonomy, teacher perception of professional expertise)
Kim et al. (2012). Project Clarion: Three Years of Science Instruction in Title I Schools Among K-Third Grade Students. Research in Science Education	Project Clarion; integrated curriculum model	Yes/No	ROET, OP, FID, IR, QI	Observation protocol content validity, internal consistency, and interrater reliability; QI data related to measured student outcomes
Reis, McCoach, Little, Muller, and Kaniskan (2011). The effects of differentiated instruction and enrichment pedagogy on reading achievement in five elementary schools. American Educational Research Journal	SEM reading framework	Yes/Yes	ROET, ROCT, FO (one to two times per month), P/U, OP, FID, RIET, MV, IR, QI	Average teacher implementation percentage scores computed from observation protocol; interrater reliability data for experimental and control group teacher observations; instructional strategies and curriculum materials used by control group teachers
Mofield and Chakraborti-Ghosh (2010). Addressing Multidimensional Perfectionism in Gifted Adolescents With Affective Curriculum. JEG	Searching for perfect balance (affective curriculum)	Yes/No	ROET, FO (at least once), OP, FID, TS (after every lesson), QI	Treatment integrity scores computed from fidelity protocols completed by researcher and teacher; interrater reliability data
Gavin, Casa, Adelson, Carroll, and Sheffield, (2009). Project M3: Mentoring Mathematical Minds—A Research-Based Curriculum for Talented Elementary Students. GCQ	Project M3: Mentoring Mathematical Minds	Yes/No	ROET, FO (weekly), TS (after every lesson)	None
VanTassel-Baska, Bracken, Feng, and Brown (2009). A Longitudinal Study of Enhancing Critical Thinking and Reading Comprehension in Title I Classrooms. JEG	Project Athena; integrated curriculum model	Yes/Yes	ROET, FO (twice), OP, TS (after every lesson), IR, FID, QI	Internal consistency reliability computed for the observation protocol
Gavin et al. (2007). Project M3; Mentoring Mathematical Minds—A Research-Based Curriculum for Talented Elementary Students. JAA	Project M3: Mentoring Mathematical Minds	Yes/No	ROET, FO (weekly)	None
Little, Feng, VanTassel-Baska, Rogers, and Avery (2007). A Study of Curriculum Effectiveness in Social Studies. GCQ	Project Phoenix; integrated curriculum model	Yes/No	ROET (twice), OP, IR, TS (twice), FID, QI, MV	Descriptive statistics for observed teacher classroom implementation behaviors, interrater reliability of observation protocol, content validity for observation protocol, MV data (teacher effectiveness in implementing packaged curriculum)

Note. ROET = researcher observation of experimental group teachers; FO = frequency of observation described; GCQ = Gifted Child Quarterly; OP = observation protocol or fidelity instrument used; TS = teacher self-report; JEG = Journal for the Education of the Gifted; RIET = researcher interview of experimental group teachers; FID = observation protocol or fidelity instrument described; ROCT = researcher observation of control group teachers; MV = moderating variables limiting implementation fidelity identified; QI = quality of teacher implementation assessed; JAA = Journal of Advanced Academics; IR = intraobserver or interrater reliability of teacher implementation observation established; P/U = predictable and unpredictable times for observation used; CLEAR = challenge leading to engagement, achievement, and results; SEM = schoolwide enrichment model.

Results

As indicated in Table 1, 11 experimental or quasi-experimental curriculum intervention studies developed for gifted learners were located that satisfied inclusion criteria for this survey. Each contained representations that the authors considered FOI in connection with their curriculum intervention research.¹

Of the 11 studies included in this survey, 10 described the critical components of the intervention. Of those, 3 provided specific information of the business as usual conditions in control group classrooms and indicated program differentiation.

Each of the studies included in this survey reported assessing FOI through researcher observations of experimental teachers during implementation of the intervention. Two studies additionally reported that control group teachers were observed. Eight reported the frequency of observations, and one reported using both predictable and unpredictable times for conducting observations. Seven studies reported the use of observation protocols, and six studies described the fidelity protocols or instruments used during observations. Six studies reported the use of multiple fidelity observers (teachers and researchers) as well as teacher self-report. Four studies established intraobserver reliability among observers in the use of fidelity instruments. Six studies assessed the quality with which the teachers implemented interventions. Two studies also identified variables that moderated the degree to which teachers implemented with fidelity. Six studies reported qualitative and/or quantitative data related to FOI, and two reported FOI data in relation to measured outcomes from the intervention.

Discussion

Results from this survey illuminate recent practices in measuring and reporting FOI in outcome evaluations of curricular interventions developed for gifted learners. This survey of the extant literature indicates that researchers engaged in gifted curriculum intervention studies broadly represent the occurrence of teacher FOI when reporting the results of curriculum intervention outcome evaluations. This result suggests researchers’ growing recognition of the importance of treatment fidelity in establishing the efficacy of a curriculum intervention and that FOI is an important feature of evidence-based practices (Azano et al., 2011; Century et al., 2010; O’Donnell, 2008). Because the studies included in this survey also employed quality research designs (experimental and quasi-experimental) and reported effect sizes, the results of this survey further suggest that the field is increasing its engagement in methodologically rigorous curriculum intervention research as recommended by experts in the field (VanTassel-Baska & Brown, 2007; White, Kim, Kingston, & Foster, 2014).

Nevertheless, the nature and quality of the methods used to measure FOI, the degree to which fidelity data were reported, if reported at all, as well as the degree to which fidelity data were tied to measured outcomes varied widely among the studies included in this survey. First, the majority of studies included described the critical components of the curriculum being evaluated by providing the theoretical underpinnings of the interventions, as well as the curriculum materials and instructional strategies used. Clearly, identifying the critical components of curriculum interventions provides important information to educators that is useful for identifying specific curricula and instructional strategies that can help bridge the research to practice gap. However, only three studies either observed control group classrooms or provided any information about business as usual curricula and instructional strategies utilized in control classrooms, which precludes the ability to differentiate between experimental and control group classroom conditions. Consequently, it is difficult to draw conclusions about which component(s) of the interventions, whether curricular or instructional, included in this survey actually affected student outcomes (Century et al., 2010). Thus, researchers are encouraged to more explicitly differentiate the critical components of an intervention from the business as usual conditions in control classrooms by describing both fully. This would enable evaluators and consumers of research to make more rigorous determinations of the relationships among specific elements of an intervention and student outcomes, and understand which components support student growth (Reis, McCoach, Little, Muller, & Kaniskan, 2011). This will also facilitate the implementation of replication studies to further deepen the understanding of evidence-based practices for gifted learners (Makel & Plucker, 2014).

The results of this survey also suggest that while many researchers recognize that fidelity should be monitored (as indicated by the representations that fidelity occurred), few researchers in the field systematically measure implementation fidelity as recommended by educational researchers (Century et al., 2010; O’Donnell, 2008), or they do not articulate these methods in reporting data. While researcher observation of experimental group teachers served as the primary method for assessing FOI, several studies neither used multidimensional methods for assessing FOI nor described fidelity instruments. Moreover, few reported any means of analysis related to assessing fidelity or measured the impact of FOI on measured outcomes. The lack of articulating the methods for measuring fidelity and the limited reporting of FOI data limit both the conclusions that can be drawn about the efficacy of reported curriculum interventions and the ability to generalize findings (Century et al., 2010; Keller-Margulis, 2012). Finally, few studies explored the variables that serve as barriers to implementation fidelity for teachers; possibly because FOI was not the focus of these studies. Understanding the contextual factors that might limit treatment integrity could provide direction for adapting curriculum materials that support the degree to which teachers successfully implement research-based practices in diverse settings and also guide professional development efforts (Achinstein & Ogawa, 2006; Azano et al., 2011; Fogelman, McNeill, & Krajcik, 2011; Foster, 2011).

Limitations and Implications

Before turning to the broader implications of this study, potential limitations should be noted. Members of the research team examined different journals and conducted independent database searches. We did not include dissertations, papers presented at research conferences, or studies reported outside of peer-reviewed journal databases or conducted outside of the United States, nor did we calculate the number of articles that did not satisfy the inclusion criteria. Moreover, we acknowledge that examining the extant gifted education literature across a broader date range may have illuminated trends in how researchers in the field have addressed FOI. Finally, this study’s focus was on the inclusion and reporting of FOI in efficacy and effectiveness studies to address validity, not on the determination of effectiveness itself.

Turning to the implications of this study, this survey is consistent with other reviews of the gifted education literature as it broadly demonstrates the need to deepen the quality of reported research in the field and the more specific need for systematic curriculum intervention research that moves the field forward in developing a well-established research base on which to build practice (Feng, VanTassel-Baska, Quek, Bai, & O’Neill, 2005; Jolly & Kettler, 2008). Thus, it is clear that establishing evidence-based practices in curriculum research remains a work in progress (Matthews et al., 2008). In an era of accountability where the use of evidence-based practices dominates the educational agenda, educators and parents of gifted children expect to have both knowledge of and access to curricula proven to be effective.

To determine which curricular interventions have a deep evidence base establishing their effectiveness and deliver the best outcomes for gifted learners, it remains essential that researchers conduct efficacy studies indicating methodological rigor and quality; establishing FOI is a key indicator of rigor (Cook et al., 2009; Foster, 2011; VanTassel-Baska, 2013). Explicitly differentiating the critical components of a curriculum intervention from the curriculum materials and instructional strategies commonly used in gifted classrooms is vital when assessing FOI. Researchers are encouraged to consistently describe with specificity the methods used to assess treatment fidelity and the FOI data collected, assess the relationship between fidelity and outcomes, and then report these methods and data so that conclusions as to their impact on outcomes can be determined. Doing so will facilitate scale-up and replication research, and also aid stakeholders in evaluating whether interventions were implemented as program developers intended and/or affected student outcomes (Century et al., 2010). In addition, the development of fidelity instruments that could be used across multiple curriculum efficacy studies is warranted as such instruments have the potential to serve as a foundation or framework for measuring FOI and for deepening the understanding of which specific components of interventions in gifted classroom are effective (Century et al., 2010; Reis et al., 2011; VanTassel-Baska et al., 2009).

We hope these recommendations will allow researchers to conduct replication studies and identify curricula for gifted students that are evidence based, which in turn will culminate in the consistent implementation of the most effective curricula and practices for gifted students and thereby bridge the research to practice gap. The failure to do so may “marginalize the field of gifted education from more rigorous educational research and limit the possible applications of powerful meta-analytic techniques to the study” of evidence-based curricular interventions and practices (Matthews et al., 2008, p. 64).

Footnotes

Authors’ Note

Lisa H. Foster is now at Development Services Group, Bethesda, Maryland.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

About the Authors

Tracy C. Missett, PhD, is an assistant professor in the Department of Education at Sweet Briar College, as well as an adjunct faculty member in the Curry School of Education at the University of Virginia and the College of Education at Marshall University. She holds a bachelor’s degree in rhetoric and communication studies from the University of Virginia, a law degree from the University of California, Hastings College of the Law, a master’s degree from Teachers College, Columbia University, and a PhD in educational psychology from the University of Virginia. Her research interests include twice-exceptional students, particularly those with complex social and emotional profiles, teacher expectations, and fidelity of implementation.

Lisa H. Foster, PhD, is now the coprincipal investigator on the What Works Clearinghouse postsecondary contract at Development Services Group. She completed a 3-year postdoctoral research fellowship for the READS (Reading Enhances Achievement During Summer) Summer Learning project at the Harvard Graduate School of Education. She received her PhD in higher education from the University of Virginia, BAs in international studies and political science at Virginia Tech, and an MEd in curriculum and instruction at Liberty University. Her research interests include fidelity of implementation and assessments during efficacy and effectiveness studies.

References

Achinstein

Ogawa

R. T.

(2006). (In)Fidelity: What resistance of new teachers reveals about professional principles and prescriptive educational policies. Harvard Educational Review, 76, 30-63.

Aljughaiman

A. M.

Ayoub

A. E. A.

(2012). The effect of an enrichment program on developing analytical, creative, and practical abilities of elementary gifted students. Journal for the Education of the Gifted, 35, 153-174. doi:10.1177/0162353212440616

Ambrose

VanTassel-Baska

Coleman

L. J.

Cross

T. L.

(2010). Unified, insular, firmly policed, or fractured, porous, contested, gifted education? Journal for the Education of the Gifted, 33, 453-478. doi:10.1177/016235321003300402

Azano

A. P.

Missett

T. C.

Callahan

C. M.

Foster

L. M.

Moon

(2011). Exploring the relationship between fidelity of implementation and academic achievement in a third-grade gifted curriculum: A mixed-methods study. Journal of Advanced Academics, 22, 693-719. doi:10.1177/1932202X11424878

Callahan

C. M.

Moon

T. R.

(2007). Sorting the wheat from the chaff: What makes for good evidence of effectiveness in the literature in gifted education? Gifted Child Quarterly, 51, 305-319. doi:10.1177/0016986207306317

Carroll

Patterson

Wood

Booth

Rick

Balain

(2007). A conceptual framework for implementation fidelity. Implementation Science, 2(1), 2-40. doi:10.1186/1748-5908-2-40

Century

Rudnick

Freeman

(2010). A framework for measuring fidelity of implementation: A foundation for shared language and accumulation of knowledge. American Journal of Evaluation, 31, 199-218. doi:10.1177/1098214010366173

Coleman

R. C.

Gallagher

J. J.

Job

(2012). Developing and sustaining professionalism within gifted education. Gifted Child Today, 35, 27-36.

Cook

B. G.

Cook

S. C.

(2011). Unraveling evidence-based practices in special education. Journal of Special Education, 47, 71-82. doi:10.1177/0022466911420877

10.

Cook

B. G.

Tankersley

Landrum

T. J.

(2009). Determining evidence-based practices in special education. Exceptional Children, 75, 365-383. doi: 10.1177/1053451208324504

11.

Dai

D. Y.

Swanson

J. A.

Cheng

(2011). State of research on giftedness and gifted education: A survey of empirical studies published during 1998-2010 (April). Gifted Child Quarterly, 55, 126-138. doi:10.1177/0016986210397831

12.

Feng

A. X.

VanTassel-Baska

Quek

Bai

O’Neill

(2005). A longitudinal assessment of gifted students’ learning using the integrated curriculum model (ICM): Impacts and perceptions of the William and Mary language arts and science curriculum. Roeper Review, 27, 78-83. doi:10.1080/02783190509554294

13.

Fogelman

McNeill

K. L.

Krajcik

(2011). Examining the effect teachers’ adaptations of a middle school science inquiry-oriented curriculum unit on student learning. Journal of Research in Science Teaching, 48, 149-169. doi:10.1002/tea.20399

14.

Foster

(2011). Fidelity: Snapshots of implementation of a curricular intervention (Doctoral dissertation) (ProQuest Document Identification No. 908430898). Retrieved from http://search.proquest.com//docview/908430898

15.

Gallagher

J. J.

(2006). How to shoot oneself in the foot with program evaluation. Roeper Review, 28, 122-124.

16.

Gavin

M. K.

Casa

T. M.

Adelson

J. L.

Carroll

S. R.

Sheffield

L. J.

(2009). Project M3: Mentoring mathematical minds—A research-based curriculum for talented elementary students. Gifted Child Quarterly, 53, 188-202. doi:10.1177/0016986209334964

17.

Gavin

M. K.

Casa

T. M.

Adelson

J. L.

Carroll

S. R.

Sheffield

L. J.

Spinelli

A. M.

(2007). Project M3: Mentoring mathematical minds—A research-based curriculum for talented elementary students. Journal of Advanced Academics, 18, 566-585. doi:10.4219/jaa-2007-552

18.

Gavin

M. K.

Casa

T. M.

Adelson

J. L.

Firmender

J. M.

(2013). The impact of challenging geometry and measurement units on the achievement of Grade 2 students. Journal for Research in Mathematics Education, 44, 478-509.

19.

Gavin

M. K.

Casa

T. M.

Firmender

J. M.

Carroll

(2013). The impact of advanced geometry and measurement curriculum units on the mathematics achievement of first-grade students. Gifted Child Quarterly, 57, 71-84. doi:10.1177/0016986213479564

20.

Gentry

Peters

S. J.

(2009). Effect sizes in gifted education research. Gifted Child Quarterly, 53, 219-222. doi:10.1177/0016986209334976

21.

Gersten

Fuchs

L. S.

Compton

Coyne

Greenwood

Innocenti

M. S.

(2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71, 149-164.

22.

Hertberg-Davis

H. L.

Callahan

C. M.

(2013). Defensible curriculum for gifted students: An introduction. In Callahan

C. M.

Hertberg-Davis

H. L.

(Eds.), Fundamentals of gifted education: Considering multiple perspectives (pp. 259-262). New York, NY: Routledge.

23.

Horner

R. H.

Sugai

Anderson

C. M.

(2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptional Children, 42, 1-13.

24.

Jolly

J. L.

Kettler

(2008). Gifted education research 1994-2003: A disconnect between priorities and practice. Journal for the Education of the Gifted, 31, 427-446. doi:1177/0162353212440610

25.

Keller-Margulis

M. A.

(2012). Fidelity of implementation framework: A critical need for response to intervention models. Psychology in the Schools, 49, 342-352. doi:10.1002/pits.21602

26.

Kim

K. H.

VanTassel-Baska

Bracken

B. A.

Feng

Stambaugh

Bland

(2012). Project Clarion: Three years of science instruction in title I schools among K-third grade students. Research in Science Education, 42, 813-829. doi:10.1007/s11165-011-9218-5

27.

Klimis

VanTassel-Baska

(2013). Designing self-contained middle schools for the gifted: A journey in program development. Gifted Child Today, 36, 172-178.

28.

Little

C. A.

Feng

A. X.

VanTassel-Baska

Rogers

K. B.

Avery

L. D.

(2007). A study of curriculum effectiveness in social studies. Gifted Child Quarterly, 51, 272-284. doi:10:1177/0016986207302722

29.

Makel

M. C.

Plucker

J. A.

(2014). Facts are more important than novelty: Replication in the education science. Educational Researcher, 43, 304-316. doi:10.3102/0013189X14545513

30.

Matthews

M. S.

Gentry

McCoach

D. B.

Worrell

F. C.

Matthews

Dixon

(2008). Evaluating the state of the field: Effect size reporting in gifted education. The Journal of Experimental Education, 77, 55-65.

31.

Mowbray

C. T.

Holter

M. C.

Teague

G. B.

Bybee

(2003). Fidelity criteria: Development, measurement, and validation. American Journal of Education, 24, 315-340.

32.

Mofield

E. L.

Chakraborti-Ghosh

(2010). Addressing multidimensional perfectionism in gifted adolescents with affective curriculum. Journal for the Education of the Gifted, 33, 479-513. doi:10.1177/016235321003300403

33.

O’Donnell

C. L.

(2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78, 33-84. doi:10.3102/0034654307313793

34.

Reis

S. M.

McCoach

D. B.

Little

C. A.

Muller

L. M.

Kaniskan

R. B.

(2011). The effects of differentiated instruction and enrichment pedagogy on reading achievement in five elementary schools. American Educational Research Journal, 48, 462-501. doi:10.3102/0002831210382891

35.

Slavin

R. E.

(2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31, 15-21. doi:10.3102/0013189X031007015

36.

U.S. Department of Education. (2003, December). Identifying and implementing educational practices supported by rigorous evidence: A user-friendly guide. Washington, DC: Institute of Education Sciences.

37.

Valentine

J. C.

Cooper

(2003). What Works Clearinghouse study design and implementation assessment device (Version 1.0). Washington, DC: U.S. Department of Education.

38.

VanTassel-Baska

(2013). Matching curriculum, instruction and assessment. In Plucker

J. A.

Callahan

C. M.

(Eds.), Critical issues and practices in gifted education (pp. 377-385). Waco, TX: Prufrock Press.

39.

VanTassel-Baska

Bracken

Feng

Brown

(2009). A longitudinal study of enhancing critical thinking and reading comprehension in title I classrooms. Journal for the Education of the Gifted, 33, 7-37.

40.

VanTassel-Baska

Brown

E. F.

(2007). Toward best practice: An analysis of the efficacy of curriculum models in gifted education. Gifted Child Quarterly, 51, 342-358. doi:10.1177/0016986207306323

41.

VanTassel-Baska

Robinson

N. M.

Coleman

L. J.

Shore

B. M.

Subotnik

R. F.

(2006). A report card on the state of research in the field of gifted education. Gifted Child Quarterly, 50, 339-355.

42.

Walsh

R. L.

Kemp

C. R.

Hodge

K. A.

Bowes

J. M.

(2012). Searching for evidence-based practice: A review of the research on educational interventions for intellectually gifted children in the early childhood years. Journal for the Education of the Gifted, 35, 103-138. doi:10.1177/0162353212440610

43.

White

Fletcher-Campbell

Ridley

(2003). What works for gifted and talented pupils: A review of recent research. Slough, Berkshire: National Foundation for Educational Research.

44.

White

Kim

Kingston

Foster

(2014). Replicating the effects of a teacher-scaffolded voluntary summer reading program: The role of poverty. Reading Research Quarterly, 49, 5-30. doi:10.1002/rrq.62