Evaluation of the Prevention Programs That Work Curriculum to Teach Use of Public Health Evidence to Community Practitioners

Abstract

Training on use of evidence may increase the adoption of evidence-based strategies in communities. The purpose of this study was to evaluate the Prevention Programs That Work training. This article summarizes the training modules and evaluation results from three trainings conducted in Spring 2009. The training teaches practitioners from community-based organizations to locate, choose, adopt or adapt, and evaluate an evidence-based program. Participants completed a pretest and posttest that covered competencies related to use of evidence, demographics, and feedback on training. A total of 47 participants attended three trainings. They represented coordinators, directors, nurses, and outreach workers. Participants showed the greatest increases in the following competencies: locate evidence-based resources, narrow search results of potential programs or strategies to reflect programmatic specifics, define steps in the adaptation process, and discuss factors during each phase of implementation. According to participants, the most useful aspects of the training content were the areas that covered program implementation, adaptation, and evaluation. A workshop can increase competencies to use evidence-based strategies among community practitioners. Building competencies is a critical initial step in promoting the dissemination of evidence-based strategies and helping practitioners use a systematic process to select an evidence-based intervention and implement the programs with fidelity.

Keywords

community organization health promotion program planning and evaluation training

Introduction and Literature Review

Widespread use of evidence-based strategies has been promoted to improve public health. A recent survey of public health practitioners found that 58% of state and local programs were considered evidence based (Dreisinger et al., 2008). This suggests that a large proportion is not evidence based, likely because of the barriers that exist in disseminating evidence-based interventions (EBIs) to communities. These may include practitioners being unaware of sources of evidence, a lack of community resources for adoption of interventions, lack of a support system for technical assistance (TA), and a need for support systems to sustain organizational interventions. Conflicting guidelines for certain evidence-based strategies, planners who want to develop their own program to match their audience, researchers who do not translate their findings into an understandable form for other audiences, reinvention of intervention protocols, and the early state of the science of translation are additional barriers (Hannon et al., 2010; Kerner et al., 2005). Other organizational and support factors have been identified as barriers, including turbulence within settings, complexity and variations in settings, funding issues, and lack of organizational support (Rohrbach, Donofrio, Backer, & Montgomery, 1996). Training in how to use evidence-based strategies may increase knowledge about what is evidence, where to find sources of evidence, how to implement a strategy based on implementation guidance or protocols, how to receive assistance in adopting or adapting a strategy, and gaining organizational support.

Active dissemination is critical to accelerate the translation of research into practice (Glasgow, Marcus, Bull, & Wilson, 2004; Kelly, Heckman, et al., 2000; Rohrbach et al., 1996). Training and TA are two of the most common active dissemination strategies. TA may range from passive to intensive consultations between program experts and program implementers (Harshbarger, Simmons, Coelho, Sloop, & Collins, 2006; Schoenwald, Sheidow, Letourneau, & Liao, 2003). Training on specific evidence-based strategies has been offered in a variety of formats, including workshops of various lengths and intensities, self-study, train-the-trainer programs, and booster sessions (Adams et al., 2000; Basen-Engquist et al., 1994; Eke, Neumann, Wilkes, & Jones, 2006; Harshbarger et al., 2006). The importance of TA and training in translation of EBIs into local communities has been reinforced in several studies (Elliott & Mihalic, 2004; Rohrbach et al., 1996). For example, one study found that AIDS service organizations that received a packaged intervention, training on intervention delivery, and TA had higher rates of adoption of evidence-based programs and higher frequency of its use than those who received a packaged program only (Kelly, Somlai, et al., 2000). Recent research calls for more practitioner-focused training on the use of EBIs among an array of local organizations like nonprofit organizations, hospitals, and work sites (Brownson, Fielding, & Maylahn, 2009).

Many resources describe evidence-based strategies and programs, such as the Community Guide to Preventive Services (Briss, Brownson, Fielding, & Zaza, 2004) and compendiums such as National Cancer Institute’s (NCI) Research-Tested Intervention Programs (NCI, 2011a), SAMHSA’s (Substance Abuse and Mental Health Services Administration) National Registry of Effective Programs and Practices (NREPP; SAMSHA, 2011), and Diffusion of Effective Behavioral Interventions (Lyles, Crepaz, Herbst, Kay, & HIV/AIDS Prevention Research Synthesis Team, 2006). Glossaries are also available to describe key terms in the use of evidence (Rabin, Brownson, Haire-Joshu, Kreuter, & Weaver, 2008; Rychetnik, Hawe, Waters, Barratt, & Frommer, 2004). However, few training curricula that promote the use of public health evidence have been developed and evaluated (Chinman, Imm, & Wandersman, 2004; Dreisinger et al., 2008; Lloyd, Rychetnik, Maxwell, & Nove, 2009; NCI, 2011b). The purpose of this article is to describe the Prevention Programs That Work training for community practitioners and evaluation results of the training.

Method

We developed a comprehensive training curriculum on how to find and use chronic disease and cancer evidence in program planning in Spring 2009. This work originated from our role as a Cancer Prevention and Control Research Network (CPCRN)-funded university through the Emory Prevention Research Center. The mission of the CPCRN is to promote the adoption, replication, implementation, and dissemination of successful programs in communities, and it addresses critical gaps in the cancer prevention and control evidence base (Harris et al., 2005). With this goal in mind, we created a training to help community organizations learn how to locate, to adopt the program as is or adapt it, and to evaluate an evidence-based program. The theoretical foundation for this training is the Interactive Systems Framework for Dissemination and Implementation (ISF), which explains the translation of innovations to practice. The ISF depicts key infrastructure and relationships involved in the translation of research to practice: (a) prevention synthesis and translation, (b) prevention support system, and (c) prevention delivery. The prevention synthesis and translation systems are responsible for the distillation of scientific advances for use through the creation of user-friendly products (Wandersman et al., 2008). This training serves as part of the prevention support system that supports practitioners who use innovations (the prevention delivery system) through the delivery of training or TA on how to use evidence to practitioners in the field. The intent of the training was to position community organizations to apply for a minigrant to strengthen their capacity to use evidence-based strategies and program. The training usually lasts about 7 hours and is typically delivered by two to four trainers through a face-to-face workshop.

The training content was based on the NCI’s training, Using What Works (NCI, 2011b). However, substantive content was added to address training topics such as organizational fit, core elements of a program, organizational capacity to implement a program, and program adaptation guidelines. In addition, other models of dissemination of evidence-based strategies such as the Replicating Effective Programs framework for health care interventions and the Map of the Adaptation Process helped inform content on selecting strategies based on fit of the intervention to the organization and steps in program adaptation (Kilbourne, Neumann, Pincus, Bauer, & Stall, 2007; McKleroy et al., 2006). We also employed evaluation data from a minigrant program that funds community organizations to use evidence-based programs to determine areas for training in how to use evidence such as identifying program core elements and consulting with program developers about intervention materials and implementation issues (Honeycutt, Carvalho, Daniel, Glanz, & Kegler, in press). The curriculum takes community practitioners through a systematic decision and implementation process in the use of evidence-based programs or strategies. The key objectives of the training are (a) to define the term evidence-based, (b) to describe the benefits of using evidence-based programs, (c) to describe resources and methods for collecting needs assessment data about their community, (d) to locate evidence-based strategies or programs that meet their program’s goals and objectives, (e) to assess program fit and organizational readiness, (f) to identify what elements of an intervention can and cannot be changed when adapting an evidence-based program, and (g) to conduct process and outcome evaluation of their program.

Description of the Training Modules

The modules are divided into seven sections (Table 1). Collectively they cover the sequential steps program planners should perform to locate, select, and implement an evidence-based program. The training is interactive in that key content is presented and then a variety of training techniques are used to engage the learners. These techniques include probing questions, worksheets, group discussions, and reports back to the class. We allow time for discussion and group work to grapple with issues such as selecting an evidence-based program or strategy and determining what adaptations are minimal. The training focused primarily on evidence-based programs because they are more likely than policy strategies to have core elements (components) outlined for planners to follow. The availability of this specific program information allows for discussion of fidelity, adaptation, and evaluation of those core elements.

Table 1

Designing Public Health Programs That Work Curriculum

Module	Key Concepts
1. Defining evidence	Definition of evidence based
	Model of translation/knowledge transfer
	Continuum of evidence (practice–research)
	Benefits of use of evidence-based strategies
2. Assessing needs for your audience	Determinants related to health issues
	Methods to assess community needs/assets
	Goal and objectives of a program
3. Finding an evidence-based strategy/program	Sources of evidence-based strategies (e.g., Community Guide to Preventive Services) and programs (SAMSHA NREPP, RTIPS)
4. Selecting a program that fits your community	Definitions of program and organizational fit
	Comparing programs or strategies for best fit
5. Adapting an evidence-based program	Definitions for adaptation, core elements, and fidelity
	Elements that can and cannot be adapted (stop–go continuum)
	Steps in the adaptation process
6. Implementing an evidence-based program	Definitions and components of organizational readiness/capacity to implement an evidence-based program
	Steps in the implementation process
	Implementation plan
7. Evaluating an evidence-based program	Types of evaluation
	Measuring fidelity and outcomes

NOTE: SAMSHA NREPP = Substance Abuse and Mental Health Services Administration National Registry of Effective Programs and Practices; RTIPS = Research-Tested Intervention Programs.

The first training module, Defining Evidence, provides an introduction to the term evidence-based. It also describes how evidence ranges from research-tested to practice-tested (evaluations), a model of translation (Wilson, Brady, Lesesne, & NCCDPHP Work Group on Translation, 2011), benefits of use of evidence-based strategies, and barriers to adoption of evidence-based strategies. The continuum of evidence that is presented includes systematic reviews, data and research evidence, program evaluation, and professional and stakeholder experience (Brownson et al., 2009; NCI, 2011b; O’Neall & Brownson, 2005). The second module, Assessing Needs for Your Audience, presents information about describing determinants related to health issues, discussing which factors programs may address, exploring methods to assess community needs and assets, and creating goals and objectives of a program based on determinants and community needs. In the Finding an Evidence-Based Program module, we present two options for evidence: broad interventions strategies (e.g., Community Guide) and packaged programs/EBIs (e.g., SAMSHA NREPP, Research-tested Intervention Programs). Evidence-based strategies are broad categories of interventions that are proven to work (e.g., use of client reminders to increase cancer screening, smoking bans or restrictions to reduce secondhand smoke). These broad strategies are defined in the Community Guide by overarching commonalities but may differ in details such as methods or implementation protocols. Evidence-based intervention programs are specific, packaged programs that may include a set of educational materials, implementation protocols, and/or defined processes. We provide screen captures of each website and describe how to locate information about a particular strategy or program and use that information to select an appropriate intervention. We also offer tips such as using the search features or signing up for listservs for updates to the evidence registries.

In the fourth module, Selecting a Program That Fits Your Community, the concepts of program and organizational fit are discussed. Program fit is defined as the potential match between the community’s needs, organizational resources, and capacity to implement a program with the requirements of the program (McKleroy et al., 2006). We present fit considerations for selecting the most appropriate program in terms of considering the program, audience, and the organizational and community characteristics. In groups, participants perform a “Program Comparison Tool” exercise (see Table 2 for selected training activities).

Table 2

Selected Training Activities

	Topic/Activity
Selecting a program	Program comparison tool
	Given a hypothetical community and intervention setting and are given 2 EBI options to compare for fit to their community.
	Read about each EBI, discuss how each factor (e.g., program goals, audience, intervention components, delivery method, length) of the EBI relates to the community and setting, and choose one that best “fits” them.
Adapting an EBI	Choosing core elements of a program activity
	Read an EBI description and list the core elements of the program.
	Green, yellow, and red light adaptation presentation
	Green light: minimal changes that do not affect fidelity very much and do not change the intent of the program. Examples might include updating statistics or tailoring materials to a particular audience in terms of language and culture.
	Yellow light: these adaptations should be conducted with caution to not affect fidelity (e.g. changing the sequence of activities or adding activities to reinforce concepts).
	Red light: Major adaptations that should be avoided. These include the following: deleting core elements or program strategies, adding many other program components that detract from core elements, or shifting the focus of the program from one health issue to others (Lesesne et al., 2007)
	Program fit and adaptation assessment tool
	Once a program is selected, participants should consider how to increase fit between the selected program with the intended audience and organization for a health promotion activity.
	They describe changes that they could make to the program or materials of the selected EBI to increase fit with the hypothetical community and intervention setting. They then assess if the changes can be made based on the adaptation continuum presented (e.g., yes with minor adaptation).
	Steps in the adaptation process
	Steps involve the following: determining the needs of the audience, reviewing the program and materials while considering fit to the audience, define the extent to which adaptation is needed and appropriate, develop the adapted intervention, work with experts to evaluate the adapted materials for relevance, accuracy and conformity to the original materials, and conduct process measures on the adapted intervention implementation.
	Materials adaptation exercise
	They assess some evidence-based program materials and describe how they could be adapted for a particular audience.

NOTE: EBI = evidence-based intervention.

The Adapting an Evidence-Based Program module is a critical module that expands on concepts that are not available in other courses about evidence-based strategies. We describe definitions for fidelity, core elements, and adaptation. Fidelity is the extent to which a program’s implementation matches the original intervention’s design (Valente, 2002). Core elements are the essential components, representing the theory or logic of that intervention (Eke et al., 2006), or those shown through research (McKleroy et al., 2006) that are believed to make an evidence-based program effective and should be retained to maintain effectiveness of the intervention. Adaptation is the process of changing a program to meet specific needs of the context in a particular community or organization. Adaptations may involve changes, additions, deletions, or substitutions of program elements (Lesesne et al., 2007). We emphasize that in making program adaptations, the core element/components must be maintained to remain true to the original intent of the program and increase likelihood of success. A continuum of levels of program adaptation is presented including modifications of materials/content; delivery mechanisms such as characteristics of program delivery, channel, or setting (Castro, Barrera, & Martinez, 2004) or other program elements (Table 2). We discuss that there may be an inverse relationship between adaptation and fidelity to the original program. When more adaptations are made, then the program is less likely to adhere to the original and requires more rigorous outcome evaluation. Finally, we present steps in the adaptation process derived from several resources on adaptation (NCI, 2009; Wingood & DiClemente, 2008).

In the sixth module, Implementing an Evidence-Based Program, we described concepts of organizational readiness/capacity to implement an evidence-based program and an implementation plan. Organizational capacity refers to the different types of resources needed for an organization to implement and possibly sustain a program (Lesesne et al., 2007). It includes human capacities (e.g., staff, leadership), technical expertise, fiscal capacities (budget), and formal linkages or partnerships. For the implementation plan, we describe steps in the implementation process from preimplementation to evaluation with a focus on the conduct of the core elements. We also emphasize the need to have a timeline and roles and responsibilities for delineated tasks (McKleroy et al., 2006). An “Organizational Readiness” checklist is shared as a tool to assess whether an organization has the ability and resources to implement the program. For the final module, Evaluating an Evidence-Based Program, we present information on types of evaluation to address the need to measure implementation (process) and program results (outcome) and methods for data collection through a case study example. We also discuss the need to document conduct of core elements, fidelity to the original program, and adaptations that occur (McKleroy et al., 2006). Throughout the process, we encourage participants to use available TA through the original EBI developer, funders, universities, and other entities to learn more about the EBI, how to adopt or adapt the intervention, and how to best evaluate its effectiveness. The training content is located on the Emory Prevention Research Center’s website, http://www.sph.emory.edu/eprc/minigrant/2010training.html.

Evaluation Methods

We administered a pretest and a posttest to all participants across 3 regional training events in Georgia. The pretest had one question of ratings of 18 competencies in which participants rated themselves on the ability to perform the training objectives on a scale of low to high (1-5) in a matrix format (Table 3). For example, competencies included defining and locating evidence-based resources and assessing the fit of organizational characteristics to potential program characteristics. The eight-item posttest included measures on general attitudes about the trainings, level of the training, and balance between lecture and group interaction. The posttest also included the same item on ratings of the training objectives found in the pretest. Additionally, participants provided feedback about what they like most about the training, how the training would be useful to their work, and improvements to the training. We were able to administer the survey to everyone at pretest; however, there was attrition at posttest because some participants left early and did not complete the survey. This study received a research exemption status from the Emory Institutional Review Board.

Table 3

Participants’ Self-Reported Ability to Meet Training Competencies Before and After Training

	Pretraining	Posttraining
	N = 47; Mean (SD)	N = 38; Mean (SD)	Difference
Describe what evidence based means	3.60 (1.21)	4.71 (0.61)	1.11***
Discuss the benefits of using evidence-based programs	3.55 (1.17)	4.78 (0.53)	1.23***
Describe how to assess your program’s audience	3.17 (1.12)	4.58 (0.60)	1.41***
Locate resources for secondary sources of data	3.28 (1.17)	4.66 (0.67)	1.38***
Determine methods to collect primary data	3.17 (1.12)	4.63 (0.63)	1.46***
Consider your audience, goals, and objectives to help you select a program or strategy	3.38 (1.21)	4.66 (0.58)	1.29***
Locate evidence-based resources like the Community Guide to Preventive Services, Cancer Control P.L.A.N.E.T., and Research-Tested Intervention Programs (RTIPs)	3.04 (1.23)	4.70 (0.57)	1.66***
Use search options to narrow your programs and to find out what programs will and will not work with your community	3.02 (1.07)	4.65 (0.59)	1.63***
Assess the fit of organizational characteristics to potential intervention program characteristics	3.23 (1.05)	4.64 (0.59)	1.40***
Select top candidate intervention categories/programs to narrow your program choices to a final choice	3.28 (1.04)	4.61 (0.60)	1.33***
Define adaptation	3.02 (1.14)	4.64 (0.59)	1.62***
Determine what needs to be changed in your evidence-based program through program planning and brainstorm activities	3.13 (1.05)	4.64 (0.59)	1.51***
Define steps in adaptation process	2.71 (1.29)	4.66 (0.59)	1.95***
Define phases of implementation	3.09 (1.26)	4.56 (0.66)	1.47***
Discuss important factors in each phase	2.96 (1.23)	4.64 (0.64)	1.68***
Define different types of evaluation	3.04 (1.06)	4.69 (0.58)	1.65***
Apply or use different types of evaluation	3.08 (1.05)	4.61 (0.60)	1.52***
Match your evaluation methods with your program objectives	3.13 (1.11)	4.64 (0.59)	1.51***

NOTE: P.L.A.N.E.T. = Plan, Link, Act, Network with Evidence-based Tools.

p < .05. **p < .01. ***p < .001.

We ran descriptive statistics for the quantitative items. We ran independent t tests to compare the difference in means between ability ratings for the training competencies between the pretest and posttest. For the open-ended questions, we coded major themes and compiled representative quotes.

Results

In total, 47 participants attended the Prevention Programs That Work Training in Albany, Rome, and Valdosta, Georgia trainings. Of the participants who completed the section on degree levels attained, 16 (55%) held graduate-level degrees (14 master’s and 2 PhDs). In addition, 11 reported nursing degrees and 2 had degrees in social work. More than 50% of the participants indicated that they were from a community agency or other nongovernmental organization, whereas the other 50% represented health care agencies, government agencies, academic institutions, or other types of organizations. Participants reported job titles, including coordinator/managers (n = 14), directors (n = 11), nurses (n = 5), and health education/outreach workers (n = 4). Of those in attendance, 88% were female. About 73% of participants completed both the pretest and posttest.

At pretest, the participants reported lower ability in locating evidence-based sources, defining adaptation, using search options to narrow program options, determining what programs will work with their community, and defining different types of evaluation (Table 3). There was a significant difference in all ratings of ability to perform each competency from pretest to posttest. The competency areas that showed the greatest increase were the ability of program participants to define steps in the adaptation process (Δ = 1.95), being able to locate evidence-based resources (Δ = 1.66), narrowing search results to reflect programmatic specifics (Δ = 1.63), being able to discuss important factors during each phase of implementation (Δ = 1.68), defining different types of evaluation (Δ = 1.65), and defining adaptation (Δ = 1.62; Table 3). Overall, the majority of participants (94.7%) agreed that the training was a good experience and was relevant. Most participants (97.3%) also said that the content was appropriate for their level of experience. Finally, there was a consensus that the balance between lecture and group interaction was “just right.”

Participants were asked what they liked most about the training and which part would be most useful in their work. Many said that the training materials and resources such as the organized training binder, forms, tools, and Internet resource list were very useful, helpful, and applicable to their work. The participants shared these comments:

The information was relevant to what our organization was trying to accomplish.

I have received a mental health grant focusing on substance abuse and evidence based programs . . . This is the type of training that they need to do for their recipients. I received great amounts of info that I could have benefited from a year ago.

They reported that the most useful aspects of the training content were the areas that covered evaluation, implementation, adaptation, and information related to a minigrants program application. The participants noted that the following topics were particularly useful:

How to navigate The Community Guide website as well as the other resources that we were introduced to

Evidence based information, core elements, strategies

Adaptation and how to identify what you can adapt and what has to stay the same. Very helpful.

Another participant commented that the training made them reflective and helped them think about barriers to implementing programs:

Forced to think of what I don’t know. Made me think about barriers to implementing program. Made me evaluate planning phase of new program.

The participants commented that the presenters were knowledgeable, were engaging, and used real-life personal situations to explain topics. Some of the improvements suggested by participants were to increase length of training to allow for more breaks, add fun activities, and provide tips on how to select a program. The participants planned to use the information to navigate the Community Guide and other resources for evidence-based strategies, adapt programs, advance current programs, and apply for grants.

This training formed the core of several trainings conducted nationally through conferences. We have copresented with other researchers and affiliated staff of universities in the CPCRN network. They also serve as expert consultants in reviewing the training content and sharing their training experiences with similar workshops for community-based organizations and public health agencies in their respective states.

Discussion

The results show that a comprehensive curriculum may build competencies in the use of evidence-based strategies among community practitioners. The training workshop addressed critical topics in using evidence such as describing sources of evidence-based strategies, assisted with selecting strategies with the best fit to the agency, shared a process for adaptation, and discussed adaptation and fidelity issues. There were significant improvements in self-ratings across all training competencies from baseline to posttest. Overall, this evaluation found that participants benefited from this training on how to locate, select, use, adapt, and evaluate evidence-based strategies.

The open-ended responses further confirmed that the training content, tools, and resources were relevant and helpful for work in their communities. It seemed important to participants to not only receive information on resources but also learn how to navigate and use the strategies in their work to meet their organizational mission. They clarified which content areas were most useful: program planning and implementation, adaptation guidelines about what can and cannot be changed, real-life examples, and evaluation.

These results are similar to other findings that training can increase general skills in using evidence among community organizations. Chinman, Tremain, and Imm (2009) found that an online planning process, interactive Getting to Outcomes (iGTO), which incorporates the use of evidence, also increased ratings of practitioners’ ability to conduct skills better than before in the following areas: locating and evaluating strategies that are effective, assessing capacity needed to implement strategies, developing a detailed implementation plan, and evaluating whether strategies are implemented according to plan. Similarly, Brownson et al. (2007) found significant changes in knowledge and skills, such as where to find EBIs or decide if an intervention is effective, after participants attended workshops on evidence-based decision making for physical activity.

Training is a critical active dissemination process to assist practitioners in choosing and implementing effective strategies in their local communities. General training on how to use evidence and training on specific evidence-based strategies may be critical to their successful implementation; this intervention-specific training should provide an overview of the intervention (goals, core elements) and present the skills and tasks to implement it (Adams et al., 2000).

Additional evaluation could be conducted to follow-up with community practitioners to learn about the actual use of the evidence-based process in their planning of health promotion programs and to assess the transfer of the key concepts from the training into their organizational decision-making and program-planning processes. Further research could explore what topics may require further booster training or assistance for participants as they plan to use an EBI in the field. Challenges to the use of evidence-based programs have been identified in the field such as acquiring an EBI or information about it, assessing fit between an EBI and an intended audience, preparing staff for implementation and adaptation, and time constraints to undertake all the steps in the use of EBI (Dreisinger et al., 2008; Veniegas, Kao, Rosales, & Arellanes, 2009). It would be helpful to know if training alone is sufficient to overcome these barriers or if additional support is needed. In addition, few long-term evaluations of evidence-based public health courses exist (Dreisinger et al., 2008).

Limitations

Several limitations exist in our study. All data were based on self-report, and it is possible that some participants may have provided socially desirable responses. We had some attrition at posttest. This may introduce selection bias if those who felt that they benefited from the training were more likely to respond. Last, there was no longitudinal follow-up to assess whether participants were able to apply their new competencies.

Conclusions

The Prevention Programs That Work curriculum is an active dissemination method to teach community practitioners how to locate, choose, adopt, or adapt an evidence-based strategy. Training like this one can help promote the dissemination of available research-tested and practice-tested strategies (evidence) and may also assist organizations in a systematic process to select an EBI and implement the programs with fidelity.

Footnotes

Acknowledgements

The authors would like to thank the members of the Emory Prevention Research Center’s Community Advisory Board for their many contributions to this project.

This research was supported by Cooperative Agreement Number 1 U48 DP001909-01-1 from the Centers for Disease Control and Prevention (CDC) and the National Cancer Institute (NCI). The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the CDC or NCI.

References

Adams

Terry

M. A.

Rebchook

G. M.

O’Donnell

Kelly

J. A

Leonard

N. R.

Neumann

M. S.

(2000). Orientation and training: Preparing agency administrators and staff to replicate an HIV prevention intervention. AIDS Education and Prevention, 12(5 Suppl.), 75-86.

Basen-Engquist

O’Hara-Tompkins

Lovato

C. Y.

Lewis

M. J.

Parcel

G. S.

Gingiss

(1994). The effect of two types of teacher training on implementation of Smart Choices: A tobacco prevention curriculum. Journal of School Health, 64, 334-339.

Briss

P. A.

Brownson

R. C.

Fielding

J. E.

Zaza

(2004). Developing and using the Guide to Community Preventive Services: Lessons learned about evidence-based public health. Annual Review of Public Health, 25, 281-302.

Brownson

R. C.

Ballew

Brown

K. L.

Elliott

M. B.

Haire-Joshu

Heath

G. W.

Kreuter

M. W.

(2007). The effect of disseminating evidence-based interventions that promote physical activity to health departments. American Journal of Public Health, 97, 1900-1907.

Brownson

R. C.

Fielding

J. E.

Maylahn

C. M.

(2009). Evidence-based public health: A fundamental concept for public health practice. Annual Review of Public Health, 30, 175-201.

Castro

F. G.

Barrera

Martinez

C. R.

(2004). The cultural adaptation of prevention interventions: Resolving tensions between fidelity and fit. Prevention Science, 5(1), 41-45.

Chinman

Imm

Wandersman

(2004). Getting To Outcomes™ 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation. Santa Monica, CA: RAND Corporation.

Chinman

Tremain

Imm

(2009). Strengthening prevention performance using technology: A formative evaluation of interactive Getting To Outcomes™. American Journal of Orthopsychiatry, 79, 469–481.

Dreisinger

Leet

T. L.

Baker

E. A.

Gillespie

K. N.

Haas

Brownson

R. C.

(2008). Improving the public health workforce: Evaluation of a training course to enhance evidence-based decision making. Journal of Public Health Management and Practice, 14, 138-143.

10.

Eke

A. N.

Neumann

M. S.

Wilkes

A. L.

Jones

P. L.

(2006). Preparing effective behavioral interventions to be used by prevention providers: the role of researchers during HIV Prevention Research Trials. AIDS Education and Prevention, 18(4 Suppl. A), 44-58.

11.

Elliott

D.S.

Mihalic

S. F.

(2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5, 47-53.

12.

Glasgow

R. E.

Marcus

A. C.

Bull

S. S.

Wilson

K. M.

(2004). Disseminating effective cancer screening interventions. Cancer, 101(5 Suppl.), 1239-1250.

13.

Hannon

P. A.

Fernandez

M. E.

Williams

R. S.

Mullen

P. D.

Escoffery

Kreuter

M.W.

. . . Bowen

D. J.

(2010). Cancer control planners’ perceptions and use of evidence-based programs. Journal of Public Health Management and Practice, 16(3), E1-E8.

14.

Harris

J. R.

Brown

P. K.

Coughlin

Fernandez

M. E.

Hebert

J. R.

Kerner

. . . Wilson

(2005). The cancer prevention and control research network. Preventing Chronic Disease, 2, A21.

15.

Harshbarger

Simmons

Coelho

Sloop

Collins

(2006). An empirical assessment of implementation, adaptation, and tailoring: The evaluation of CDC’s National Diffusion of VOICES/VOCES. AIDS Education and Prevention, 18(4 Suppl. A), 184-197.

16.

Honeycutt

Carvalho

Daniel

S. D.

Glanz

Kegler

M. C.

(in press). Research to reality: A process evaluation of a mini-grants program to disseminate evidence-based nutrition interventions to rural churches and worksites. Journal of Public Health Management and Practice.

17.

Kelly

J. A.

Heckman

T. G.

Stevenson

L. Y.

Williams

P. N.

Ertl

Hays

R. B.

. . . Neumann

M. S.

(2000). Transfer of research-based HIV prevention interventions to community service providers: Fidelity and adaptation. AIDS Education and Prevention, 12(5 Suppl.), 87-98.

18.

Kelly

J. A.

Somlai

A. M.

DiFranceisco

W. J.

Otto-Salaj

L. L.

McAuliffe

T. L.

Hackl

K. L.

. . . Rompa

(2000). Bridging the gap between the science and service of HIV prevention: transferring effective research-based HIV prevention interventions to community AIDS service providers. American Journal of Public Health, 90, 1082-1088.

19.

Kerner

Guirguis-Blake

Hennessy

K. D.

Brounstein

P. J.

Vinson

Schwartz

R. H.

. . . Briss

(2005). Translating research into improved outcomes in comprehensive cancer control. Cancer Causes & Control, 16(Suppl. 1), 27-40.

20.

Kilbourne

A. M.

Neumann

M. S.

Pincus

H. A

Bauer

M. S.

Stall

(2007). Implementing evidence-based interventions in health care: Application of the replicating effective programs framework. Implementation Science, 2, 42.

21.

Lesesne

C. A.

Lewis

K. M.

Moore

Fisher

Green

Wandersman

(2007). Promoting science-based approaches to teen pregnancy prevention using Getting to Outcomes: Draft June 2007. Santa Monica, CA: Rand Corporation.

22.

Lloyd

Rychetnik

Maxwell

Nove

(2009). Building capacity for evidence-based practice in the health promotion workforce: evaluation of a train-the-trainer initiative in NSW. Health Promotion Journal of Australia, 20, 151-154.

23.

Lyles

C. M.

Crepaz

Herbst

J. H.

Kay

L. S.

, & HIV/AIDS Prevention Research Synthesis Team. (2006). Evidence-based HIV behavioral prevention from the perspective of the CDC’s HIV/AIDS Prevention Research Synthesis Team. AIDS Education and Prevention, 18(4 Suppl. A), 21-31.

24.

McKleroy

V. S.

Galbraith

J. S.

Cummings

Jones

Harshbarger

Collins

, . . . ADAPT Team. (2006). Adapting evidence-based behavioral interventions for new settings and target populations. AIDS Education and Prevention, 18(4 Suppl. A), 59-73.

25.

National Cancer Institute. Adaptation guidelines. (2009). Retrieved from http://rtips.cancer.gov/rtips/reference/adaptation_guidelines.pdf

26.

National Cancer Institute. Research Tested Intervention Programs. (2011a). Retrieved from http://rtips.cancer.gov/rtips/

27.

National Cancer Institute. (2011b). Using What Works. Retrieved from http://cancercontrol.cancer.gov/use_what_works/start.htm

28.

O’Neall

Brownson

(2005). Teaching evidence-based public health to public health practitioners. Annals of Epidemiology, 15, 540-544.

29.

Rabin

B. A.

Brownson

R. C.

Haire-Joshu

Kreuter

M. W.

Weaver

N. L.

(2008). A glossary for dissemination and implementation research in health. Journal of Public Health Management and Practice, 14, 117-123.

30.

Rohrbach

Donofrio

Backer

Montgomery

(1996). Diffusion of School-based Substance Abuse Prevention Programs. American Behavioral Scientist, 39, 919-934.

31.

Rychetnik

Hawe

Waters

Barratt

Frommer

(2004). A glossary for evidence based public health. Journal of Epidemiology and Community Health, 58, 538-545.

32.

Schoenwald

S. K.

Sheidow

A. J.

Letourneau

E. J.

Liao

J. G.

(2003). Transportability of multisystemic therapy: Evidence for multilevel influences. Mental Health Services Research, 5, 223-239.

33.

Substance Abuse and Mental Health Services Administration. (2011). The National Registry of Evidence-based Programs and Practices (NREPP). Retrieved from http://www.nrepp.samhsa.gov/

34.

Valente

(2002). Evaluating health promotion programs. New York, NY: Oxford University Press.

35.

Veniegas

R. C.

Kao

U. H.

Rosales

Arellanes

(2009). HIV prevention technology transfer: Challenges and strategies in the real world. American Journal of Public Health, 99(Suppl. 1), S124-S130.

36.

Wandersman

Duffy

Flaspohler

Noonan

Lubell

Stillman

. . . Saul

(2008). Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology, 41, 171-181.

37.

Wilson

Brady

T. J.

Lesesne

, & NCCDPHP Work Group on Translation. (2011). An organizing framework for translation in public health: The knowlege to action framework. Preventing Chronic Disease, 8(2), A46.

38.

Wingood

G. M.

DiClemente

R. J.

(2008). The ADAPT-ITT model: A novel method of adapting evidence-based HIV interventions. Journal of Acquired Immune Deficiency Syndromes, 47(Suppl. 1), S40-S46.