There is a growing recognition of the value of measuring fidelity of implementation (FOI) as a necessary part of evaluating interventions. However, evaluators do not have a shared conceptual understanding of what FOI is and how to measure it. Thus, the creation of FOI measures is typically a secondary focus and based on specific contexts and programs. This article describes a project that holds the development of FOI measures as its primary goal and has developed a suite of data collection tools designed to be used across multiple programs. It describes the foundation of the suite—a conceptual framework for clearly and specifically describing FOI and the need for the framework. It also describes where the framework resides in existing literature and how it can be used to support measurement of interventions in education and other fields.
Balfanz, R., Mac Iver, D.J., & Byrnes, V. ( 2006). The implementation and impact of evidence-based mathematics reforms in high-poverty middle schools: A multi-site, multi-year study. Journal for Research in Mathematics Education, 37, 33-64.
2.
Bauman, L.J., Stein, R.E.K., & Ireys, H.T. ( 1991). Reinventing fidelity: The transfer of social technology among settings. American Journal of Community Psychology, 19, 619-639.
3.
Bay, J.M. ( 1999). Middle school mathematics curriculum implementation: The dynamics of change as teachers introduce and use standards-based curricula (Unpublished doctoral dissertation). University of Missouri , Columbia.
4.
Becker, D.R., Smith, J., Tanzman, B., Drake, R.E., & Tremblay, T. ( 2001). Fidelity of supported employment programs and employment outcomes. Psychiatric Services, 52, 834-836.
5.
Blakely, C.H., Mayer, J.P., Gottschalk, R.G., Schmitt, N., Davidson, W.S., Roitman, D.B., & Emshoff, J.G. ( 1987). The fidelity-adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253-268.
6.
Bodilly, S.J., & Keltner, B. ( 1998). Lessons from new American schools’ scale-up phase: Prospects for bringing designs to multiple schools. Santa Monica, CA: RAND (MR-1777-NAS).
7.
Bodzin, A.M., Cates, W.M., & Price, B. ( 2003). Formative evaluation of the exploring life curriculum: Year two implementation fidelity findings. Paper presented at the meeting of the National Association for Research in Science Teaching (NARST), Philadelphia, PA.
8.
Bond, G.R., Becker, D.R., Drake, R.E., & Vogler, K.M. ( 1997). A fidelity scale for the Individual Placement and Support model of supported employment. Rehabilitation Counseling Bulletin , 40, 265-284.
9.
Bond, G., Williams, J., Evans, L., Salyers, M., Kim, H., Sharpe, H., & Leff, H.S. ( 2000). Psychiatric rehabilitation fidelity toolkit. Cambridge, MA: Human Services Research Institute.
10.
Bruns, E.J., Burchard, J.D., Suter, J.C., Leverentz-Brady, K., & Force, M.M. ( 2004). Assessing fidelity to a community-based treatment for youth: The Wraparound Fidelity Index. Journal of Emotional and Behavioral Disorders, 12, 79-89.
11.
Buston, K., Wight, D., Hart, G., & Scott, S. ( 2002). Implementation of a teacher-delivered sex education programme: Obstacles and facilitating factors. Health Education Research , 17, 59-72.
12.
Dane, A.V., & Schneider, B.H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control ? Clinical Psychology Review, 18, 23-45.
13.
Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W.B. ( 2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237-256.
14.
Emshoff, J.G., Blakely, C., Gottschalk, R., Mayer, J., Davidson, W.S., & Erickson, S. ( 1987). Innovation in education and criminal justice: Measuring fidelity of implementation and program effectiveness. Educational Evaluation and Policy Analysis, 9, 300-311.
15.
Fagan, P.J. ( 1996). 1995 implementation status of mathematics and science reform in Iowa: Based on teachers’ concerns, professional activity, and philosophical beliefs. Iowa Mathematics and Science Coalition, Cedar Falls. Retrieved from ERIC database. (ED399191).
16.
Flocks, J., Clarke, L., Albrecht, S., Bryant, C., Monaghan, P., & Baker, H. ( 2001). Implementing a community-based social marketing project to improve agricultural worker health. Environmental Health Perspectives, 109, 461-468.
17.
Fullan, M. ( 1983). Evaluating program implementation: What can be learned from follow through. Curriculum Inquiry, 13, 215-227.
18.
Fullan, M., & Pomfret, A. ( 1977). Research on curriculum and instruction implementation . Review of Educational Research, 47, 335-397.
19.
Gersten, R.M., & Carnine, D. ( 1980). Measuring implementation of the direct instruction model in an urban school district: An observational approach. Paper presented at the Annual Meeting of the American Educational Research Association, Boston, MA.
20.
Gresham, F.M., Gansle, K., & Noell, G.H. ( 1993). Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis, 26, 257-263.
21.
Hall, G.E., & Hord, S.M. ( 1987). Change in schools: Facilitating the process. State University of New York Press.
22.
Hall, G.E., & Loucks, S.F. ( 1977). A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal , 14, 263-276.
23.
Heaney, C.A., & Burke, A.C. ( 1995). Ideologies of care in community residential services: What do caregivers believe? Community Mental Health Journal , 31, 449-462.
24.
Huntley, M.A. ( 2005). Operationalizing the concept of ‘‘fidelity of implementation’’ for NSF-funded mathematics curricula. Unpublished manuscript.
25.
Lastica, J., & O’Donnell, C. (2007). Considering the role of fidelity of implementation in science education research: Fidelity as teacher and student adherence to structure. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL.
Lipsey, M.W., & Cordray, D.S. ( 2000). Evaluation methods for social intervention. Annual Review of Psychology, 51, 345-375.
28.
Lynch, S., & O’Donnell, C. (2005). The evolving definition, measurement, and conceptualization of fidelity of implementation in scale-up of highly rated science curriculum units in diverse middle schools. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Canada.
29.
McDonel, E.C., Bond, G.R., Salyers, M., Fekete, D., Chen, A., McGrew, J.H., & Miller, L. ( 1997). Implementing assertive community treatment programs in rural settings. Administration and Policy in Mental Health , 25, 153-173.
30.
Mowbray, C.T., Holter, M.C., Teague, G.B., & Bybee, D. ( 2003). Fidelity criteria: Development, measurement, and validation . The American Journal of Evaluation, 24, 315-340.
31.
Neale, D.C., Smith, D., & Johnson, V.G. ( 1990). Implementing conceptual change teaching in primary science . The Elementary School Journal, 91, 109-131.
32.
O’Donnell, C. ( 2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78, 33-84.
33.
Penuel, W.R., & Means, B. ( 2004). Implementation variation and fidelity in an inquiry science program: Analysis of GLOBE data reporting patterns. Journal of Research in Science Teaching, 41, 294-315.
34.
Rezmovic, E.L. ( 1982). Program implementation and evaluation results . Evaluation and Program Planning, 5, 111-118.
35.
Ross, J.A., McDougall, D., Hogaboam-Gray, A., & LeSage, A. (2003). A survey measuring elementary teachers’ implementation of standards-based mathematics teaching. Journal for Research in Mathematics Education, 34, 344-363.
36.
Ruiz-Primo, M.A. ( 2005). A multi-method and multi-source approach for studying fidelity of implementation. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Canada.
37.
Sabelli, N., & Dede, C. ( 2001). Integrating educational research and practice: reconceptualizing goals and policies: ‘‘How to make what works work for us?’’ Retrieved June 26, 2007 from http://www.virtual.gmu.edu/ss_research/cdpapers/policy.pdf
38.
Songer, N.B., & Gotwals, A.W. ( 2005). Fidelity of implementation in three sequential curricular units. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal , Canada.
39.
Wafa, M.A., & Yasin, M.M. ( 1998). A conceptual framework for effective implementation of JIT: An empirical investigation. International Journal of Operations & Production Management, 18, 1111-1124.
40.
Wang, M.C., Nojan, M., Strom, C.D., & Walberg, H.J. ( 1984). The utility of degree of implementation measures in program implementation and evaluation research. Curriculum Inquiry , 14, 249-286.
41.
Ysseldyke, J.E., Spicuzza, R., Kosciolek, S., Teelucksingh, E., Boys, C., & Lemkuil, A. ( 2003). Using a curriculum-based instructional management system to enhance math achievement in urban schools. Journal of Education for Students Placed at Risk, 8, 247-265.
