Policy,Systems,and Environmental Change in the Mississippi Delta

Abstract

Community-level policy, systems, and environmental (PSE) change strategies may offer an economical and sustainable approach to chronic disease prevention. The rapidly growing number of untested but promising PSE strategies currently underway offers an exciting opportunity to establish practice-based evidence for this approach. This article presents lessons learned from an evaluation of a community-based PSE initiative targeting stroke and cardiovascular disease prevention in the Mississippi Delta. Its purpose is to describe one approach to evaluating this type of PSE initiative, to stimulate discussion about best practices for evaluating PSE strategies, and to inform future evaluation and research efforts to expand practice-based evidence. The evaluation used a descriptive mixed-methods design and focused on the second year of a multisectoral, multiyear initiative. Cross-sectional data were collected in the summer and fall of 2010 using four data collection instruments: a grantee interview guide (n = 32), a health council member survey (n = 256), an organizational survey (n = 60), and a grantee progress report (n = 26). Fifty-eight PSE changes were assessed across five sectors: health, faith, education, worksite, and community/city government. PSE strategies aligned with increased access to physical activity opportunities, healthy food and beverage options, quality health care, and reduced exposure to tobacco. Results showed that grantees were successful in completing a series of steps toward PSE change and that sector-specific initiatives resulted in a range of PSE changes that were completed or in progress. Considerations for designing evaluations of community-based PSE initiatives are discussed.

Keywords

cardiovascular disease community health promotion outcome evaluation process evaluation social ecology

Individual behavior plays a major role in the onset of chronic diseases and thus presents an important challenge to its prevention. Improvements to population health are in part dependent on individual-level behavior change. Applications of social-ecological concepts to influence health behavior focus on multiple levels of influence and offer sustainable support for individual change, as well as the breadth to affect population health (Lieberman, Golden, & Earp, 2013; McLeroy, Bibeau, Steckler, & Glanz, 1988; Sallis, Owen, & Fisher, 2008; Stokols, 1996). Communities are ideal settings to target behavioral determinants of health from a multilevel, multisectoral perspective (Bunnell et al., 2012; Green & Kreuter, 2005). In contrast, interventions that focus solely on individuals and small groups are resource-intensive and have limited sustainability (Frieden, 2010; Leeman et al., 2012). Policy, systems, and environmental (PSE) change strategies may offer an economical and sustainable approach to making healthful choices the convenient and most frequently selected choices (Frieden, 2010).

PSE strategies have gained prominence in community-based health promotion as of late, in large part due to funding priorities of the Centers for Disease Control and Prevention, the Robert Wood Johnson Foundation, and others (Bunnell et al., 2012; Leviton & Strunk, 2012; Nichols, Ussery-Hall, Griffin-Blake, & Easton, 2012; Pettibone, Friend, Nargiso, & Florin, 2013). Although a number of PSE-focused evaluation studies have examined behavioral and health outcomes (Cheadle et al., 2012; Chomitz et al., 2010; Chomitz et al., 2012; Cradock et al., 2011; Johns, Coady, Chan, Farley, & Kansagra, 2013; Tee & Bockle, 2012), more research is needed to establish a strong evidence base for PSE strategies (Brennan, Castro, Brownson, Claus, & Orleans, 2011; Golden & Earp, 2012; Institute of Medicine, 2010). A recent review highlights the relatively small number of research-tested PSE strategies in relation to the rapidly growing number of untested, but promising, practice-based strategies, with respect to obesity prevention (Brennan et al., 2011). Similarly, the Institute of Medicine (2010) recently recognized the dearth of evidence to support PSE strategies and recommended quality evaluation of policies and programs currently being implemented in practice settings.

Several frameworks have been developed to inform evaluations of PSE strategies, with an emphasis on both process and outcomes. Leeman et al. (2012) developed an evaluation framework for obesity-related policies that can easily be extended to other chronic disease risk factors. Their evaluation framework includes a useful logic model showing how they conceptualize the relationships between policy making and related outputs (e.g., reach) and outcomes (e.g., environmental change, health status). Leeman et al.’s (2012) logic model draws on multiple evaluation and policy-making frameworks. Inputs in the logic model include problems (e.g., assessment data), solutions (e.g., model policies), politics (e.g., political will), and other factors (e.g., staffing). The authors conceptualize policy making as a four-stage process: (1) formulation, (2) enactment, (3) implementation, and (4) maintenance/modification. Throughout this process, practitioners engage stakeholders, raise awareness, and advocate for change. These activities are expected to lead to a variety of outputs, including increased awareness and political will, policies enacted, reach to intended populations, adoption by settings/sectors, and enforcement. These outputs are critical early markers of potential population impact. The authors describe short-term outcomes as changes to the physical, economic, social, and/or communication environment resulting from the change in policy. Intermediate outcomes are viewed as changes in social norms and behaviors, and the long-term impact includes population-level health outcomes.

Other frameworks place greater emphasis on the process of policy change. Masters and Osborn (2010) developed a framework for assessing the movement toward environmental and policy change that involves five core elements: organizing an authentic base, developing leadership, creating and implementing vision and ideas, building alliances, and developing advocacy infrastructure. Similarly, Devlin-Foltz, Fagen, Reed, Medina, and Neiger (2012) suggested assessing multiple markers of advocacy impact, including increased interest among populations and policy makers. Although none of these frameworks highlight a multisectoral approach per se, they each mention the importance of new alliances and partnerships, as well as community and stakeholder engagement as key to policy change.

Recent PSE evaluations have focused on reach, community mobilization, community capacity, and barriers to the implementation process (Cheadle, Egger, LoGerfo, Schwartz, & Harris, 2009; Gantner & Olson, 2012; Townsend et al., 2011). While more evaluations of this nature are needed to understand the intermediate steps toward achieving PSE changes and to understand how policy adoption and implementation may vary by community context, the need for additional outcome evaluations is perhaps more critical. Challenges associated with generating this type of evidence are not new, but the limited empirical support for PSE initiatives is generating fresh dialogue about how best to evaluate this type of community intervention (Brownson, Brennan, Evenson, & Leviton, 2012; Leviton & Strunk, 2012; Pettibone et al., 2013).

The current article describes an evaluation of a community-based PSE initiative in seven Mississippi Delta counties to support cardiovascular disease and stroke prevention. The aim of this article is threefold: (1) to share evaluation methods and selected results, (2) to stimulate discussion regarding best practices for evaluating PSE strategies being implemented nationwide, and (3) to inform future evaluation and research efforts.

Method

Description of the Initiative

The initiative sought to reduce risk factors for cardiovascular disease and stroke morbidity and mortality through PSE changes in the Mississippi Delta that supported healthy behaviors. The initiative was guided by an Advisory Council and implemented by the Mississippi State Department of Health in collaboration with local community partners. Multisector, community-driven interventions were funded through community grants to federally qualified health centers, Mayor’s Offices, and nonprofit organizations. In the first 2 years of the initiative, the funded nonprofit organizations included community-based organizations with their own programming and intermediary organizations that collaborated with sector-based partner organizations (e.g., churches) to implement activities. Community grants were used to establish and support Mayor’s Health Councils, church gardens, congregational health nurse programs, worksite wellness councils, and a chronic disease management quality improvement initiative in federally qualified health centers. Table 1 lists selected PSE priorities by sector.

Table 1.

Example of Policy, Systems, and Environmental Change by Sector.

Sector	Physical activity	Nutrition	Tobacco	Chronic disease management	General health/wellness
Local government/community-wide	Access or improvements to places for physical activity	Promote healthy food/beverage options at local restaurants and food venues	Smoke-free policy for outdoor public places	None	Promote healthy environments or healthy lifestyles in local media
Worksites	Promote stairwell use	Promote healthy food/beverage options	Refer to smoking cessation	Access to preventive services	Evaluate health promotion programs
Health care	Physical activity counseling/education	Nutrition assessment	Tobacco counseling/education	System to monitor risk factors for heart disease and stroke	None
Churches	Access to physical activity through classes, church gardens, or an onsite fitness center	Promote healthy food/beverage options	Indoor smoke-free policy	Access to and training for congregational health nurse or other source for preventive health care or health education	Organizational support for wellness
Schools	Access to a broad range of competitive and noncompetitive physical activities	Improve the nutritional quality and appeal of school meals	Tobacco-free school policy	None	Train teachers to incorporate health education into school curriculum

Evaluation Framework and Design

The evaluation used a descriptive, mixed-methods design. Cross-sectional data were collected in the second year of the initiative (summer and fall of 2010) using four data collection instruments. The evaluation team, with members from the Emory Prevention Research Center, Jackson State University, and Mississippi State Department of Health, developed a conceptual framework that identified a theory of change, which begins with community mobilization through formation of coalitions and provision of community grants. The model suggests that community mobilization, combined with funding, training, and technical assistance, leads to increased community capacity for implementing PSE strategies. Capacity is operationalized as skills, leadership opportunities, new partnerships, and leveraging of additional resources. Theoretically, increased capacity supports targeted sectors’ ability to influence PSE changes that ultimately lead to individual-level behavior changes and, with sufficient reach, improved population health (Figure 1).

Figure 1.

Evaluation framework.

Evaluation questions covered the four major domains outlined in the evaluation framework (Table 2). Because of the large number of questions, we are focusing on only three related to PSE change in the current article: (1) What steps were taken toward PSE change as a result of the initiative? (2) How many and what types of PSE changes were made as a result of the initiative? (3) What facilitated and inhibited progress in cultivating PSE change in various community sectors?

Table 2.

Mississippi Delta Health Collaborative Evaluation Questions.

Evaluation framework domain	Evaluation questions
Community mobilization and activities	How many organizations were involved and in what ways?
	How many people were involved and in what ways (e.g., on Health Council)?
	Which community sectors were engaged and how?
	What types of activities were conducted as part of the initiative?
	What types of training and technical assistance did organizations receive from the initiative?
Community capacity	What new knowledge and skills were gained by community members and leaders as a result of participation?
	What new opportunities were created for leadership development in cardiovascular disease and stroke prevention as a result of the initiative?
	How many and what types of new partnerships and linkages were formed as a result of the initiative?
	What new resources were mobilized for cardiovascular disease and stroke prevention as a result of the initiative?
	To what extent was capacity for planning and evaluating cardiovascular disease and stroke prevention initiatives strengthened as a result of the initiative?
Policy and environmental change	What steps were taken toward policy and environmental change as a result of the initiative? To what extent were residents and local leaders trained, new collaborative structures created, action plans developed and implemented, and advocacy/community change actions undertaken?
	How many and what types of environmental and policy changes were made as a result of the initiative?
	What facilitated and inhibited progress in creating environmental and policy change in various community sectors?
	How likely are these environmental and policy changes to be sustained?
	Which sectors were more successful in environmental and policy change and why?
	What was the process for creating environmental and policy change in various sectors?
Evaluations of individual behavior and health outcome change	What individual behavior changes were specific initiatives/grantees able to document with technical assistance in evaluation?
	What evaluation systems were put in place to assess individual behavior change and health outcomes in future efforts?

Study Population

Evaluation data were collected from grantees and their partner organizations from worksite, faith, health care, school, and local government/general community sectors (Table 3). Participants included grantees (n = 32), council and advisory committee members (n = 256), and sector-based partner organizations (n = 60). Response rates were 100% for grantee interviews, 69.1% for the council and advisory committee member survey, 51.7% for the sector-specific organizational surveys, and 100% for the grantee progress reports.

Table 3.

MDHC Evaluation Data Sources.

Data source	Methods	Instrument	Participants
Grantee interviews	In-person interviews	Structured interview guide	Grantees (n = 32)
Organizational surveys	Survey hand-delivered	Self-administered written survey (primarily close-ended questions)	Partner organizations (n = 60):
	Follow-up with telephone reminders for nonrespondents		• Churches (n = 31)
	Follow-up with telephone reminders for nonrespondents		• Worksites (n = 28)
			• Schools (n = 1)
Health council surveys	Survey distributed at council meetings	Self-administered written survey (primarily close-ended questions)	Council/coalition members (n = 256): • Mayor’s Health Councils (n = 207) • Community health coalitions (n = 26) • MDHC Advisory Council (n = 23)
	Mail or hand-delivered follow-up for absent members
	Telephone reminders for nonrespondents
	Telephone reminders for nonrespondents
Grantee progress reports	Document review of progress reports completed and submitted by MDHC grantees	Progress report template (primarily open-ended questions)	Grantees/coordinators (n = 26)

Note. MDHC = Mississippi Delta Health Collaborative.

PSE Change Measures

The primary outcomes of the evaluation included PSE changes in five key sectors to promote physical activity, nutrition, tobacco prevention and cessation, and chronic disease self-management (Table 1). We developed a common set of measures for implementation of PSE changes, steps taken to promote PSE change, and contextual factors influencing PSE change. These measures were used in multiple data collection instruments.

Implementation of PSE Changes

Potential PSE changes were identified from the published literature and stakeholder input (Centers for Disease Control and Prevention, 2010; Prevention Institute, 2010). Across all sectors, 58 different PSE strategies were assessed. For each specific strategy, respondents were asked whether the change was (1) completed/in place, (2) in progress, or (3) not in place/not in progress. The evaluation team defined completed/in place as a completed action that necessitates ongoing or occasional monitoring or support, and in progress as an action that has been started but not fully implemented. If a change was completed/in place or in progress, the respondent was asked whether or not that change was supported by the initiative. Support was defined as the initiative providing financial assistance, training, technical support, other instruction, and/or facilitating networking and partnerships to support this action.

Steps to Promote Change

We reviewed both the published literature and practical guidance documents (i.e., Partnership for Prevention’s Action Guides, 2008) to identify steps grantees could have taken to promote PSE change (Table 4). With stakeholder input, we selected 12 steps, divided into two subcategories. Planning steps included the following: convene a group to help plan and implement a project, collect information or identify available resources, and develop an action plan or goals and objectives. Advocacy steps included the following: generate media coverage, and draft and share policy proposals.

Table 4.

Grantees Reporting Steps Toward PSE Change.

					Grantees collaborating with sector-based partners
	Steps toward PSE changes	All grantees (N = 32), n (% yes)	Mayor’s offices (n = 19), % yes	Health care (n = 7), % yes	Faith-based (n = 2), % yes	Worksites (n = 1), % yes	Schools (n = 1), % yes	Community (n = 2), % yes^a
Planning	1. Convene a group to help plan and implement project	31 (96.9)	100	85.7	100	100	100	100
	2. Collect information or identify available resources	31 (96.9)	94.7	100	100	100	100	100
	3. Develop an action plan or goals and objectives	32 (100)	100	100	100	100	100	100
	4. Identify people to work on specific tasks	31 (96.9)	100	85.7	100	100	100	100
	5. Train residents or local leaders	26 (81.3)	94.7	42.9	100	100	100	50
	6. Apply for additional funding	15 (46.9)	26.3	71.4	50	100	100	100
	7. Plan for sustainability	32 (100)	100	100	100	100	100	100
Advocacy	8. Conduct outreach or promotional activities	30 (93.7)	100	71.4	100	100	100	100
	9. Generate media coverage, letters to the editor	22 (68.8)	63.2	57.1	100	100	100	100
	10. Draft and share policy proposals	16 (50.0)	47.4	57.1	0	100	100	50
	11. Contact local legislators or decision makers	26 (81.3)	89.5	42.9	100	100	100	100
	12. Attend city council or county commissioner meetings	27 (84.4)	84.2	42.9	0	100	100	100

Note. PSE change = policy, systems, and environmental change.

One of the community-based grantees established coalitions in two counties, the other provided physical activity opportunities to youth.

Contextual Factors Influencing Change

The grantee interview guide, organizational survey, and health council survey included 17 contextual factor items that may have affected grantees’ efforts to implement PSE change. Questions pertained to organizational-level contextual factors (e.g., your organization’s mission) and community-level factors (e.g., economic conditions). For each item, respondents indicated whether the factor hindered, helped, or had no effect on the success of implementing PSE change (Kegler, Norton, & Aronson, 2008).

Data Collection Instruments

We developed four data collection instruments: a grantee interview guide, health council member survey, organizational survey, and grantee progress report. Across all instruments, we used the common measures described above to assess PSE change. The grantee interview guide covered community mobilization, community capacity, PSE change, evaluation activities, and respondent demographics. The interview guide was tailored for the five different sectors targeted by the initiative and for PSE changes specific to physical activity, nutrition, tobacco use prevention and control, chronic disease self-management, and general health and wellness.

The health council member survey included questions pertaining to council participation, activities, functioning (e.g., communication), PSE change, and respondent demographics (Kegler, Norton, & Aronson, 2007).

The organizational survey was tailored to three specific sectors including, schools, churches, and worksites. Questions addressed topics of organizational background, participation, PSE changes, and demographics. The PSE changes listed in the survey were those identified as priorities through the grantee interviews.

The grantee progress report assessed the progress toward grantee-generated objectives, individual and organizational involvement in the initiative, community capacity for PSE change, and site-specific program evaluation. The procedures used to collect data are outlined in Table 3.

Analysis

The overall evaluation involved mixed methods, with both quantitative and qualitative analyses. This article reports a subset of the findings for selected PSE-related outcomes and required only simple descriptive statistics of quantitative items.

Results

Steps Toward PSE Change

Table 4 shows the number and percentage of grantees reporting achievement of steps toward PSE change. Overall, five out of seven planning steps were reported by more than 90% of grantees: convene a group to help plan and implement the project, collect information or identify available resources, develop an action plan or goals and objectives, identify people to work on specific tasks, and plan for sustainability. Of the planning activities, the fewest grantees reported applying for funding. The most common advocacy step was conducting outreach or promotional activities (93.7%). A relatively high percentage of grantees had contacted local legislators or decision makers (81.3%) and attended city council or county commissioner meetings (84.4%). Less frequently reported steps included generating media coverage (68.8%) and drafting and sharing policy proposals (50%).

PSE Changes in the Faith Sector

Table 5 shows churches reporting various types of PSE change supported by the initiative. Similar results are available for the other sectors (data not shown). Of the 18 churches responding to the organizational survey, 61% (n = 11) provided access to or were actively working on providing access (33.3%, n = 6) to physical activity opportunities. Other PSE strategies completed or in place among 50% of the responding churches were providing access to and training for a congregational health nurse or other source for preventive health care or health education, and demonstrating organizational support for wellness. A high percentage of the churches were also working on instituting healthy food and beverage options at church-sponsored meetings and events, 8 (44.4%) reported this was completed or in place, and 9 (50%) reported this strategy as in progress. Of the six PSE changes reported by churches, all were either completed/in place or in progress in more than 70% of the churches.

Table 5.

Churches Reporting PSE Change (n = 18).

Policy or environmental systems change	Completed or in place, N	In progress, N	Total in place or in progress, N (%)
Provide access to physical activity through classes, church gardens, or an onsite fitness center	11	6	17 (94.4)
Institute healthy food and beverage options at church-sponsored meetings and events	8	9	17 (94.4)
Provide access to a congregational health nurse or other source for preventive health care or health education	9	6	15 (83.3)
Provide training to a congregational health nurse or other source for preventive health care or health education	9	4	13 (72.2)
Demonstrate organizational support for wellness, for example, by forming a wellness committee or health ministry	9	6	15 (83.3)
Provide recognition to congregational members participating in chronic disease prevention measures	3	11	14 (77.8)

Note. PSE change = policy, systems, and environmental change.

Contextual Factors Influencing PSE Implementation

Contextual factors that grantees identified as barriers or facilitators to implementing PSE strategies were also examined. The economic condition in the local community was the most commonly reported barrier, followed by challenges related to the demographics of the community. The top facilitator was organizational willingness to partner with other agencies. Among members of the Mayor’s Health Councils, the leadership and membership of the council and support from the initiative were also important facilitators. Other grantees noted the value of their own organization’s mission in facilitating PSE implementation, along with their own organization’s leadership and support from the initiative.

Discussion

A great deal can be learned from practice-based initiatives such as the one described here. Our evaluation documented that with adequate support and technical assistance, churches, worksites, schools, and community health centers were able to implement PSE change within a fairly short time frame (6-12 months). Churches, for example, were able to provide access to physical activity opportunities, offer healthy foods and beverages at meetings and events, and provide access to a congregational nurse. Although not reported here, similar changes occurred across the other priority sectors. We also learned that organizations took multiple steps to foster PSE change. Planning steps were likely to occur similarly across community sectors, whereas advocacy steps were more varied across sectors. This could be due to varying levels of comfort with these strategies (Gantner & Olson, 2012), or explained by the type of PSE strategy under consideration.

Another valuable set of findings from our evaluation focused on how organizational and community context affected the PSE change process. Organizational factors (e.g., organizational mission) were viewed as primary facilitators in the change process, which is consistent with literature that identifies the importance of the “inner setting” of an organization in successful implementation (Damschroder et al., 2009). In contrast, barriers tended to be external to the organization and focused on the community (e.g., economic conditions). These findings are consistent with prior research documenting how community context can influence the implementation of community-based initiatives (Kegler, Rigler, & Honeycutt, 2011).

The current evaluation, while useful to the primary stakeholder for documenting progress, did not examine the reach at the community or individual level, nor did it document policy implementation or enforcement. Similar to many other community-based PSE initiatives, it also did not document behavioral or health outcomes (Brownson et al., 2012; Leviton & Strunk, 2012; Nichols et al., 2012; Pettibone et al., 2013). Reasons for the latter included timing of the evaluation, infeasibility of control communities, and budget constraints. The challenges faced in this evaluation can be used to highlight issues that evaluators and practitioners must grapple with in attempting to contribute to practice-based evidence for this important and evolving approach to chronic disease prevention. Key challenges, described in more detail below, included prioritizing evaluation questions and balancing depth versus breadth in the evaluation; design constraints such as funding limitations, the timing of the evaluation, the lack of baseline data, and not having a comparison or control group; the difficulty of assessing a wide range of PSE change strategies implemented with great variety at the local level; and selecting appropriate evaluation participants given the reach and potential intensity of different strategies.

One of our first challenges was to strategically prioritize evaluation questions. Multisector, community-based PSE change initiatives can generate a vast array of evaluation questions. Funding limitations and feasibility influenced our decisions about the scope of the evaluation, as did the initiative’s early stage of development and timing. Stakeholders were particularly interested in outcomes but understood that PSE change can be a lengthy process and that behavioral outcomes were likely not observable during the evaluation time frame (6-12 months). For a more mature program, it may be feasible to examine long-term outcomes such as behavior change and health outcomes at the community level (e.g., smoking prevalence or proportion of the priority population with a healthy body mass index); however, it is unlikely that even the most successful community-based PSE initiative will yield community-level impact in the short-term. For a newer program, it may be more appropriate to assess short-term outcomes such as environmental changes (e.g., limits on where smoking is allowed, increased availability of healthful food options).

An overarching consideration in this evaluation was the need to balance depth versus breadth. We selected breadth over depth and prioritized documenting the number of targeted organizations within each sector that implemented PSE change and/or took steps toward PSE change. An alternative approach would have been to identify a sector and assess the reach of the intervention within all organizations in that sector (i.e., evaluating all churches in the Mississippi Delta). Another option would be to select a few exemplary organizations and study the context, change process, and impact on affiliated individuals, such as church members or employees. Each of these approaches would allow the evaluation to answer different types of questions, and all could be appropriate strategies for evaluating PSE change initiatives, depending on the program and stakeholders’ information needs.

Selecting the best study design was another challenge. The evaluation team was not established until several months after the community grants were awarded; thus, a true baseline was not feasible. The inability to collect baseline data combined with the lack of randomization affected our ability to attribute PSE change to the intervention. Given resource constraints and the targeting of the initiative in a high-needs region, collecting data from comparison counties was also infeasible. Our efforts to document attribution to the intervention were therefore limited to asking about PSE changes directly targeted by the initiative and asking respondents to indicate how the initiative supported that change.

One of the strengths of community-driven initiatives is that intervention strategies are adaptable to the context of specific communities; however, this complicates evaluation designs and requires flexibility. Each county in the current initiative had a different constellation of program activities, and each partner organization selected from a menu of PSE change options. This made it difficult for evaluators to identify specific and common elements of the intervention. The complexity of the initiative (e.g., 58 potential PSE changes) necessitated tailoring data collection instruments for each sector to list the appropriate menu of PSE options for that sector (e.g., churches vs. worksites). This approach, while useful for documenting the comprehensiveness of the initiative, made it impossible to examine specific PSEs in any depth. Additional flexibility was required when the initiative strengthened its health care emphasis while the evaluation was underway. This required the evaluation team to scale back its original plans for data collection at two time points to a cross-sectional design in order for future evaluation resources to focus more intensively on the health care sector.

The long and varied list of PSE strategies targeted within the initiative highlights the complexity of the change process and how little is known about best practices for this approach to chronic disease prevention. The decision to measure the steps toward PSE change was an initial attempt to identify possible core elements of this approach. The results of the current evaluation provide evidence that most of the planning steps were completed by the grantees, as were a high percentage of the advocacy steps. These findings suggest that a more nuanced, qualitative measure may be needed to learn about the implementation process in more depth, and ultimately to connect those features of the implementation process to outcomes.

In designing the evaluation, we grappled with the reach and potential intensity of each intervention and implications for the sampling frame. In other words, who should data be collected from—people directly affected by organizational change or a representative sample from the entire community? For most PSE interventions, with tobacco as the exception, there is very little evidence of a threshold for which PSE changes reliably result in individual behavior change (Golden & Earp, 2012). The strength of a PSE strategy and the dose delivered can be considered within an organization—for example, a church may have healthy foods as an option or may decide to serve only healthy foods at church events. One would expect the latter to have a greater impact. However, if only the church environment changes, is that sufficient to lead to individual behavior change outside of the church setting? Multisector, multicomponent interventions expect individual change to result from the synergistic effect of PSE change across multiple community settings. How much exposure to the culture fostered by PSE change is needed before behavior change can be expected to occur? Does it make sense to evaluate behavior change within members of a specific organization, or is it more logical to evaluate behavior change once there is substantive exposure to PSE changes across a range of community sectors? Given the answers to these questions are currently unknown, would sector-specific or community-wide data collection be most valuable to the field? When resources permit, a blend of the two would be ideal.

Implications for Practice

Capitalizing on the large number of PSE initiatives currently underway to generate practice-based evidence requires strong partnerships between practitioners and evaluators and/or academic researchers. Ideally, evaluators are involved during the planning and grant writing phases of an initiative. Early engagement of evaluators is necessary for collecting baseline data, which are required for creating empirical evidence. From a practitioner’s perspective, there are both benefits and detriments to engaging with external evaluators. External evaluators can allow for a more objective and rigorous evaluation, but they may lack familiarity with the local culture and the history and intricacies of the program. External evaluators can also face mistrust from stakeholders and community partners. The impact of these barriers can be lessened through a participatory evaluation process. The current evaluation paired an in-state evaluation team with an out-of-state evaluation team to mitigate some of these challenges. This approach was successful but required a concerted effort to maintain frequent, open communication among key partners. Biweekly calls ensured that all partners were kept abreast of progress and challenges and that the evaluation addressed programmatic needs.

Given the stage of the program and stakeholders’ information needs, the current evaluation focused on the implementation of PSE changes and did not explore behavioral or health outcomes. This evaluation met key objectives by providing stakeholders with evidence for the selection of PSE strategies that were appropriate for use in the Mississippi Delta and provided a blueprint for continued efforts in the region and a second round of funding for the initiative.

Footnotes

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The Contract between the State of Mississippi and Emory University gave the State the right to review manuscripts to be submitted for publication. Emory agreed to respond to concerns in good faith but retained the right to publish.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This publication was supported by Cooperative Agreement Numbers # 5U48DP001909 and 1U50DP001811 from the Centers for Disease Control and Prevention. The findings and conclusions in this journal article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. The Emory Prevention Research Center had an evaluation contract from the Mississippi State Department of Health for the evaluation reported in this article. Partial salary support was provided to Michelle C. Kegler, Sally Honeycutt, Carla Berg, and Emily Russell.

Supplement Issue Note

This article is part of a Health Education & Behavior supplement, “The Evidence for Policy and Environmental Approaches to Promoting Health,” which was supported by a grant to the Society for Public Health Education (SOPHE) from the Robert Wood Johnson Foundation. The entire supplemental issue is open access at .

References

Brennan

Castro

Brownson

R. C.

Claus

Orleans

C. T.

(2011). Accelerating evidence reviews and broadening evidence standards to identify effective, promising, and emerging policy and environmental strategies for prevention of childhood obesity. Annual Review of Public Health, 32, 199-223.

Brownson

R. C.

Brennan

L. K.

Evenson

K. R.

Leviton

L. C.

(2012). Lessons from a mixed-methods approach to evaluating Active Living by Design. American Journal of Preventive Medicine, 43(Suppl. 4), S271-S280.

Bunnell

O’Neil

Soler

Payne

Giles

W. H.

Collins

Bauer

(2012). Fifty communities putting prevention to work: Accelerating chronic disease prevention through policy, systems and environmental change. Journal of Community Health, 37, 1081-1090.

Centers for Disease Control and Prevention. (2010). Community Health Assessment and Group Evaluation (CHANGE). Retrieved from http://www.cdc.gov/healthycommunitiesprogram/tools/change.htm

Cheadle

Egger

LoGerfo

J. P.

Schwartz

Harris

J. R.

(2009). Promoting sustainable community change in support or older adult physical activity: Evaluation findings from the Southeast Seattle Senior Physical Activity Network (SESPAN). Journal of Urban Health, 87, 67-75.

Cheadle

Rauzon

Spring

Schwartz

P. M.

Gee

Gonzalez

. . . Williamson

(2012). Kaiser Permanente’s community health initiative in Northern California: Evaluation findings and lessons learned. American Journal of Health Promotion, 27, e59-e68.

Chomitz

V. R.

McDonald

J. C.

Aske

D. B.

Arsenault

L. N.

Rioles

N. A.

Brukilacchio

L. B.

. . . Cabral

H. J.

(2012). Evaluation results from an active living intervention in Somerville, Massachusetts. American Journal of Preventive Medicine, 43(Suppl. 4), S367-S378.

Chomitz

V. R.

McGowan

R. J.

Wendel

J. M.

Williams

S. A.

Cabral

H. J.

King

S. E.

. . . Hacker

K. A.

(2010). Healthly Living Cambridge Kids: A community-based participatory effort to promote healthy weight and fitness. Obesity, 18(Suppl. 1), S45-S53.

Cradock

A. L.

McHugh

Mont-Ferguson

Grant

Barrett

J. L.

Wang

Y. C.

Gortmaker

S. L.

(2011). Effect of school district policy change on consumption of sugar-sweetened beverages among high school students, Boston, Massachusetts, 2004-2006. Preventing Chronic Disease, 8(4), 1-9.

10.

Damschroder

L. J.

Aron

D. C.

Keith

R. E.

Kirsh

S. R.

Alexander

J. A.

Lowery

J. C.

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50.

11.

Devlin-Foltz

Fagen

M. C.

Reed

Medina

Neiger

B. L.

(2012). Advocacy evaluation: Challenges and emerging trends. Health Promotion Practice, 13, 581-586.

12.

Frieden

T. R.

(2010). A framework for public health action: The health impact pyramid. American Journal of Public Health, 100, 590-595.

13.

Gantner

L. A.

Olson

C. M.

(2012). Evaluation of public health professionals’ capacity to implement environmental changes supportive of healthy weight. Evaluation and Program Planning, 35, 407-416.

14.

Golden

S. D.

Earp

J. A.

(2012). Social ecological approaches to individuals and their contexts: Twenty years of health education & behavior health promotion interventions. Health Education & Behavior, 39, 364-372.

15.

Green

Kreuter

(2005). Health promotion planning: An educational and ecological approach (4th ed.). New York, NY: McGraw-Hill.

16.

Institute of Medicine. (2010). Bridging the evidence gap in obesity prevention: A framework to inform decision-making. Washington, DC: National Academies Press.

17.

Johns

Coady

M. H.

Chan

C. A.

Farley

S. M.

Kansagra

S. M.

(2013). Evaluating New York City’s smoke-free parks and beaches law: A critical multiplist approach to assessing behavioral impact. American Journal of Community Psychology, 51, 254-263.

18.

Kegler

M. C.

Norton

B. L.

Aronson

(2007). Skill improvement among coalition members in the California Healthy Cities and Communities program. Health Education Research, 22, 450-457.

19.

Kegler

M. C.

Norton

B. L.

Aronson

(2008). Achieving organizational change: Findings from case studies of 20 California Healthy Cities and Communities coalitions. Health Promotion International, 23, 109-118.

20.

Kegler

M. C.

Rigler

Honeycutt

(2011). The role of community context in planning and implementing community-based health promotion projects. Evaluation and Program Planning, 34, 246-253.

21.

Leeman

Sommers

Jernigan

Payne

Thompson

. . . Ammerman

(2012). An evaluation framework for obesity prevention policy interventions. Preventing Chronic Disease, 9, E120.

22.

Leviton

L. C.

Strunk

S. L.

(2012). Active living by design: The value of evaluation. American Journal of Preventive Medicine, 43(5 Suppl. 4), S267-S268.

23.

Lieberman

Golden

S. D.

Earp

J. A.

(2013). Structural approaches to health promotion: What do we need to know about policy and environmental change? Health Education & Behavior, 40, 520-525.

24.

Masters

Osborn

(2010). Social movements and philanthropy: How foundations can support movement building. Foundation Review, 2, 12-27.

25.

McLeroy

K. R.

Bibeau

Steckler

Glanz

(1988). An ecological perspective on health promotion programs. Health Education Quarterly, 15, 351-377.

26.

Nichols

Ussery-Hall

Griffin-Blake

Easton

(2012). The evolution of the steps program, 2003-2010: Transforming the federal public health practice of chronic disease prevention. Preventing Chronic Disease, 9, E50.

27.

Pettibone

K. G.

Friend

K. B.

Nargiso

J. E.

Florin

(2013). Evaluating environmental change strategies: Challenges and solutions. American Journal of Community Psychology, 51, 217-221.

28.

Partnership for Prevention. (2008). Action guides. Retrieved from http://www.prevent.org/Initiatives/Action-Guides.aspx

29.

Prevention Institute. (2010). Strategic alliance: ENACT. Retrieved from http://eatbettermovemore.org/sa/enact/members/index.php

30.

Sallis

Owen

Fisher

(2008). Ecological models of health behavior. In Glanz

Rimer

Viswanath

(Eds.), Health behavior and health education: Theory, research and practice (pp. 465-486). San Francisco, CA: Jossey-Bass.

31.

Stokols

(1996). Translating social ecological theory into guidelines for community health promotion. American Journal of Health Promotion, 10, 282-298.

32.

Tee

Bockle

(2012). Closing the gap-A partnership approach to community care education for long term conditions. Nurse Education Today, 32, 822-828.

33.

Townsend

J. S.

Pinkerton

McKenna

S. A.

Higgins

S. M.

Tai

Steele

C. B.

. . . Brown

(2011). Targeting children through school-based education and policy strategies: Comprehensive cancer control activities in melanoma prevention. Journal of the American Academy of Dermatology, 65, s104.e101-s.104.e111.