Capacity Building for and Implementation of Policy,Systems,and Environmental Change: Results from a Survey of the National Comprehensive Cancer Control Program

Abstract

Policy, systems, and environmental (PSE) approaches are commonly used to improve population health. Cancer-related examples include providing data and education to stakeholders about policies that support healthy living, or health systems changes such as universal reminders about recommended cancer screening. The National Comprehensive Cancer Control Program (NCCCP) funds health departments to form cancer coalitions that develop and implement cancer plans. NCCCP initiated a demonstration program in 13 of 65 funded grantees to determine whether skilled, dedicated staffing and using a strategic process to examine data, form a workgroup, and develop an agenda would enhance their capacity to implement PSE approaches, recruit new partners, and provide data and education to stakeholders. The objective of this study was to compare demonstration program grantees to other NCCCP grantees on their ability to develop and implement PSE strategies, and the short-term results that were achieved. Program directors (PDs) from each NCCCP-funded jurisdiction completed web surveys at 2 time points during implementation to assess changes in their capacity for PSE approaches, identify implementation activities, and document short-term outcomes. Responses from demonstration program PDs and other PDs at both time points were compared in a descriptive analysis. Demonstration program grantees experienced greater increases in skills and capacity to address PSE approaches, engaged in necessary implementation activities more often, and achieved greater improvements in stakeholder and decision maker awareness and support for PSE strategies, compared to nonparticipating NCCCP grantees. These findings support continued implementation of PSE approaches for sustainable cancer prevention and control.

Introduction

Cancer is the second leading cause of death in the United States,¹ and although incidence rates are decreasing for many cancer sites,² the absolute number of new cases continues to increase because of the aging US population.³ Some cancer risks can be mitigated by reducing tobacco and alcohol use, obesity, and exposure to select infectious and environmental agents.^4
–6 Early detection of breast, cervical, and colorectal cancer can improve outcomes, and is needed among underserved populations in particular.^7,8 For the growing number of cancer survivors, preventive care and support can reduce recurrence and new primary cancers, mitigate long-term and late effects of cancer treatment, and improve quality of life.^9

–14

Policy, systems, and environmental (PSE) approaches are promising strategies to address the nation's cancer burden because they have the broadest population impacts by changing the overall context for health-related decision making and facilitating long-lasting protective interventions.¹⁵ PSE approaches are a commonly used public health strategy to reduce disease burden.¹⁵ Some examples of PSE approaches in cancer include providing data to stakeholders and decision makers about the harms of indoor tanning among minors and its relationship with melanoma incidence to inform policy; educating about systems-level changes to increase early detection of cancer, such as expanding mammography clinic hours in a health system; or promoting changes to the built environment that support healthy living, such as the development of walking trails and accessible recreational areas to increase physical activity that can reduce cancer risk. Many PSE approaches occur at the local level through voluntary organizational policies or environmental changes in schools, worksites, health systems, and communities.

The National Comprehensive Cancer Control Program (NCCCP), funded by the Centers for Disease Control and Prevention (CDC), initiated a demonstration program to enhance grantee capacity to develop and implement these approaches. Comprehensive cancer control is a collaborative and strategic process that brings together various stakeholders in cancer control to develop and implement cancer control plans for their jurisdiction based on cancer burden and local factors, and it is based on a participatory innovation diffusion model.¹⁶ Traditionally, NCCCP grantees worked with their partners to implement interventions across the cancer control continuum.

Beginning in 2010, grantees were encouraged to prioritize interventions that address primary prevention, early detection and screening, and cancer survivorship.¹⁷ Although NCCCP grantees have been encouraged to include PSE strategies in their activities since the program's founding in 1998, a greater focus on using them became a cross-cutting programmatic priority in 2010.¹⁷ The intent of the demonstration program was to begin building a practice- and evidence-based framework to support PSE approaches among all NCCCP grantees.

The demonstration program was designed to enhance cancer control capacity by assisting grantees with skill building among program staff and partners, prioritizing a set of PSE approaches based on cancer surveillance data and local context to form a PSE agenda (plan), using appropriate messaging and framing to discuss the cancer burden and strategies, and by strengthening and expanding partnerships among all relevant stakeholders, including with nontraditional partners. Multiple theoretical frameworks, most notably the Component Model of Infrastructure and the Community Coalition Action Theory, were used to guide development of the program's conceptual model, which served as a framework to evaluate its short-and long-term outcomes.^18

–22

Thirteen NCCCP grantees were selected for this demonstration program in 2010 through a competitive application process that focused on organizational capacity to implement PSE change initiatives, ability to build on existing partnerships or expand partnerships, proposed activities and outcomes, and plans to measure performance. The 13 grantees included 1 tribal nation and 12 states that are diverse in geography and racial/ethnic composition.

This demonstration program included the addition of a dedicated policy analyst at the grantee level, and grantees received specialized technical assistance (TA) from CDC and its partners on how to develop and implement a data-driven, evidence-based policy agenda that is feasible based on local political context and cultural preferences. Trainings also focused on lobbying restrictions; building and expanding partnerships; creating compelling messages and developing a media/communications strategy; effectively working with PSE decision makers to provide data and educate them about evidence-based PSE strategies; and conducting evaluation. Peer-to-peer communication between grantees was common, and provided another platform for grantees to share effective practices with one other and with their partners and stakeholders.

Policy analysts were encouraged to have competencies in the following areas: framing public health issues, media utilization, message tailoring, policy analysis, and policy formulation. Policy analysts played key roles in reviewing cancer burden data and developing an environmental scan of existing PSE approaches; forming and maintaining the PSE workgroup; developing the policy agenda; creating effective messaging and developing a media plan; and providing data and information on evidence-based strategies when approached by stakeholders and decision makers.

The purpose of this study is to compare demonstration project grantees with other grantees funded by NCCCP to determine whether the addition of specialized TA and a dedicated policy analyst led to more changes in the capacity to develop and implement PSE strategies over time.

Methods

Survey description

The study team reviewed the literature, examined existing tools and instruments, and created an evaluation consultant group to inform development of a survey instrument that included measures to capture constructs around program capacity and infrastructure, and implementation of PSE change strategies. The team administered a web-based survey (Office of Management and Budget Control Number: 0920-1016) in the fourth and fifth year of the demonstration program's 5-year funding period (Time 1 [T1] and Time 2 [T2], respectively).

Program directors (PDs) in all 65 NCCCP-funded states, tribes, and territories (13 demonstration program grantees and the remaining 52 NCCCP grantees not participating in this demonstration program) received an email explaining the survey's purpose and requesting their participation. Up to 5 reminder emails were sent to nonrespondents on a weekly basis until the survey closed. PDs were surveyed because they were expected to be the most knowledgeable about program capacity and cancer control infrastructure in their jurisdictions. However, because staff roles and responsibilities vary across programs, PDs were allowed to designate a staff person who had a more direct role in the PSE-related activities to be the respondent for the survey. Only 1 respondent per grantee entity was allowed to complete the survey.

The survey addressed 2 overarching evaluation questions: (1) How did demonstration program grantees build the capacity required to develop an environmental scan, policy agenda, evaluation plan, and media plan? (2) What methods were used by demonstration program grantees to implement the policy agenda and media plan?

Survey domains – NCCCP program staff and PSE workgroup/coalition competencies and skills

To specifically assess capacity and infrastructure development and operationalize this concept, the survey first asked respondents to rate NCCCP program staff in 6 competency areas. Staff competency was measured on a 4-point Likert scale (ie, Not at all; Basic - staff have only general knowledge; Moderate - staff could do this; Expert - staff could teach this), with the option to mark each as Don't Know. Next, respondents indicated their level of agreement with statements about the skills of program staff and partners (ie, PSE workgroup/cancer coalition members) (Table 1), and the representation of appropriate sectors among cancer coalitions and PSE workgroup members. Extent of agreement with skill statements was measured on a 5-point Likert scale (ie, strongly disagree; somewhat disagree; neutral; somewhat agree; strongly agree), with the option to mark each as not applicable.

Table 1.

Program Directors/Designees Assessments of PSE Change Competencies and Skills Among NCCCP Program Staff and PSE Workgroup/Cancer Coalition

Capacity and infrastructure for PSE change	Time 1		Time 2
Capacity and infrastructure for PSE change	Demo PDs %	Other PDs %	Demo PDs %	Other PDs %
Competencies of NCCCP Staff^*	n = 12	n = 39	n = 10	n = 32
Building relationships with partners, stakeholders	92	95	100	94
Issue framing	83	69	100	77
PSE strategy formulation	67	56	100	69
PSE strategy analysis	67	62	82	63
Media utilization	50	67	73	31
Message framing	67	72	73	83
Skills of NCCCP Staff^**	n = 12	n = 39	n = 10	n = 32
Gathering data to show value of PSE strategies	100	82	100	88
Developing specific PSE goals	100	82	100	82
Assembling evidence to support PSE strategies	92	90	100	77
Convening a PSE workgroup with sector representation	83	80	100	80
Working with allies to support PSE change goals	92	87	100	88
Leading action planning for PSE change	83	74	90	80
Developing PSE change proposals	100	72	90	77
Providing data to decision makers	83	85	90	85
Technical assistance for implementation	83	74	90	82
Technical assistance for sustainability	83	67	90	68
Holding local forums to promote PSE change	67	67	80	71
Skills of PSE Workgroup or NCCCP Coalition^**	n = 12	n = 38	n = 10	n = 31
Educating decision makers	83	71	100	77
Holding stakeholder meetings	83	74	90	74
Reaching out to allies	67	79	90	90
Developing specific PSE goals	58	66	80	61
Providing input to an action plan	75	76	80	90
Assessment of political environment, context	67	53	70	71
Providing technical assistance	33	55	70	45
Developing PSE change proposals	42	61	60	65
Organizing media events	67	53	60	45
Using media strategies	58	47	60	48
Monitoring implementation quality	42	42	50	58

The total sample size may vary depending on the survey item because not all PDs/designees provided responses to every item. During each survey time point, 13 demo PDs and 52 other PDs were contacted to participate.

Percent reporting moderate or expert levels for each competency. Level of competency was measured on a 4-point Likert scale – Not at all; Basic - staff have only general knowledge; Moderate - staff could do this; Expert - staff could teach this – with the option to mark each as Don't know.

Percent reporting strong–moderate agreement with each statement. Extent of agreement with skill statements was measured on a 5-point Likert scale – strongly disagree, somewhat disagree, neutral, somewhat agree, strongly agree – with the option to mark each as not applicable.

NCCCP, National Comprehensive Cancer Control Program; PD, program directors/designees; PSE, policy, systems, and environmental.

Respondents also indicated (yes/no) which trainings or resources were most helpful in building capacity for key skill sets.

Survey domains – implementation of PSE change strategies

The next set of survey items addressed implementation of PSE strategies prioritized in the grantees' PSE agendas. Of the PSE strategies led by their internal cancer coalition (a diverse group of local cancer control practitioners and researchers who establish the cancer plan for the state, tribe, or territory) or PSE workgroup (either a subset of the cancer coalition or a group of health department staff and partners with interest and expertise in using PSE approaches), respondents identified 2 they considered to have the most importance and the highest potential for success. For each strategy, the survey asked a series of questions about: methods used for stakeholder outreach and decision maker education; interactions between PSE workgroup/cancer coalition members, allies (individuals or groups with a similar interest in the strategy), and opponents (individuals or groups who opposed the strategy); and key events that influenced the implementation of each strategy. Extent of agreement with each statement on level of awareness was measured on a 5-point Likert scale (ie, strongly disagree, somewhat disagree, neutral, somewhat agree, strongly agree) with the option to mark each as not applicable.

Survey domains – short-term outcomes (decision maker awareness and stakeholder support)

Finally, the survey addressed short-term outcomes: the level of awareness and knowledge of decision makers and stakeholders involved in the 2 identified PSE strategies; and decision maker position and stakeholder support for these strategies. Awareness was categorized as aware/accurately informed, aware but uninformed, aware but inaccurately informed, unaware, unsure/don't know. Response categories for support included visible and vocal program champion (stakeholders only), supportive, neutral, no position, opposed, and don't know.

Data analysis

Data analysis was descriptive, employing univariate statistical methods (Stata, version 13; StataCorp LLC, College Station, TX) with tables and graphs for display and summarization. The study team calculated response percentages for each group at each time point to facilitate comparisons between demonstration program respondents (demo PDs) and respondents not participating in the demonstration program (other PDs), and to examine changes over time. The team collapsed the competency categories as moderate/expert versus basic/not at all. Levels of agreement regarding skill level and interaction with allies and opponents were grouped as strongly/somewhat agree versus strongly disagree/somewhat disagree/neutral. The percentage aware was the combined percentages for the response categories aware but uninformed, aware but inaccurately informed, and aware/accurately informed, versus the other categories unaware, unsure/don't know. The percentage in support was the combined percentages for the response categories visible and vocal program champion (stakeholders only) and supportive, versus the other categories opposed, neutral, no position, don't know.

No inferential statistical tests were done, as the survey was planned as a census of the directors of all NCCCP-funded programs, and any final set of survey respondents would be a non-probabilistic sample of that population. Thus, the findings should be regarded as representative of survey respondents only, rather than all NCCCP-funded programs.

Results

Survey response was less than 100% at both time points, with a higher response at T1 than at T2. The overall response at T1 was 78%: 92% among demo PDs, and 75% among other PDs. The overall response at T2 was 65%: 77% among demo PDs, and 62% among other PDs.

Capacity and infrastructure for PSE change

Competencies of NCCCP staff

By T2, a higher percentage of demo PDs compared to other PDs reported that staff had either moderate or expert competency in building relationships, issue framing, PSE strategy formulation, and media utilization (Table 1). Ratings for most competencies increased between T1 at T2 for both groups of PDs, though to a greater degree for demo PDs, especially for PSE strategy formulation, issue framing, and media utilization.

Skills of NCCCP staff

At T2, the vast majority of demo PDs expressed confidence (somewhat or strongly agree) that staff had each of the listed skills needed to promote PSE strategies, with 100% endorsement for 5 of the 11 skills: gathering data to show value of PSE strategies, developing specific PSE goals, assembling evidence, convening a PSE workgroup with sector representation, and working with allies to support PSE change goals (Table 1). Although a majority of other PDs also expressed confidence for these skills, their percentages were lower than demo PDs by 5 to 23 percentage points at T2. Between T1 and T2, the percentage of demo PDs who endorsed staff skills increased or remained at 100% for 10 of the 11 skills, while the change in endorsements for other PDs was more variable.

Skills of PSE workgroup or NCCCP coalition

By T2, 90% of demo PDs had either created a PSE workgroup or had plans to do so in the coming year, compared to only 58% of other PDs (data not shown). Compared to other PDs, a higher percentage of demo PDs said that their PSE workgroup had the right sector representation to meet its PSE objectives (88% compared to 58%; data not shown). For both PD groups, between T1 and T2 there was little change in the percentage with a PSE workgroup or plans to form one, and sector representation also changed minimally.

At T2, the majority of demo PDs expressed confidence that their PSE workgroups and cancer coalition members had the necessary abilities for 10 of the 11 skills (Table 1). In contrast, other PDs' confidence in partner skills tended to lag behind the demo PDs. Between T1 and T2, the percentage of demo PDs who endorsed staff skills increased for 10 of the 11 skills, while the change in endorsements for other PDs was more variable. Notably, for the 3 skills for which demo PDs had lower percentages of endorsements at T1 than other PDs (reaching out to allies; developing specific PSE goals; providing technical assistance), by T2 demo PD endorsements either matched or exceeded other PDs.

Trainings and resources

Figure 1 shows the resources and training sources, per PSE skill set, selected by the highest percentages of demo and other PDs at T2 as the most helpful in building PSE capacity. More demo PDs selected standard trainings by CDC and national partners and/or tailored trainings by national partners as the most helpful sources for developing key messages for target audiences, working with various media, and PSE strategy formulation – the next highest sources for these 3 skill sets are trainings developed locally, and networking with other programs (both tied at 40%, 40%, and 50%, respectively). Demo PDs overwhelmingly selected networking with other programs as the most beneficial source for gaining stakeholder support and forming strategic alliances – the next highest source for those 2 skill sets is trainings developed locally (40% for both). Similarly, 70% of demo PDs selected guides, websites, tools as the most helpful for building skills to provide data and information to raise decision-maker awareness – the next highest source is networking with other programs at 40%.

FIG. 1.

Top selected resources and training sources most helpful for building policy, systems, and environmental change capacity at year 5 of the demonstration program, by skill areas and program director/designee group. Note: Resources and training sources included: standard trainings by CDC/national partners; tailored trainings by national partners; trainings developed locally; networking with other programs; guides, websites, tools; other resource; none. CDC, Centers for Disease Control and Prevention; NP, national partners; PSE, policy, systems, and environmental; Demo PDs, program directors in demonstration program; Other PDs, program directors not in demonstration program.

The sources most frequently selected by other PDs largely paralleled those selected by demo PDs, but at lower percentages given that their selections were distributed more evenly across multiple sources. However, other PDs most frequently indicated that they had no source for working with various media, though approximately a quarter also selected standard trainings by CDC and national partners (26%), networking with other programs (23%), and guides, websites, tools (23%). Other PDs also differed from demo PDs in the sources they tended to select for the providing data and information to raise decision maker awareness skill set: the most frequently selected source was networking with other programs, but not far behind are tailored trainings by national partners (26%), guides, websites, tools (26%), and standard trainings by CDC and national partners (23%).

Methods used to implement PSE change strategies

Stakeholder outreach methods

At T2, demo PDs reported using multiple outreach methods to increase awareness of PSE strategies among stakeholders (Table 2), such as holding meetings and community forums; engaging stakeholders in collecting evidence to show the value of PSE strategies; creating white papers, websites, or newsletters; and securing earned media. Other PDs reported a similar usage pattern for these methods, though at much lower frequencies. There was very little change between T1 and T2 for stakeholder outreach and the other implementation activities (data not shown).

Table 2.

Implementation of PSE Change Strategies at Year 5 of the Demonstration Program

Methods used to implement PSE change strategies	T2/year 5
Methods used to implement PSE change strategies	Strategies used by demonstration program grantees %	Strategies used by other NCCCP grantees %
Stakeholder Outreach Methods^*	n = 25	n = 85
Hold meetings, workshops, and community forums	84	69
Create white papers, websites, or newsletters	40	22
Secure earned media, public service announcements	24	14
Engage stakeholders in collecting evidence to show value of PSE change strategies	72	55
Decision Maker Education Methods^*	n = 16	n = 45
Provide data and other information to demonstrate value	81	67
Provide evidence that shows the health impact of PSE change strategies	44	58
Develop proposal for a specific PSE change strategies	56	47
Create white papers, websites, or newsletters	0	7
Create media, public service announcements	0	9
When requested, attend in-person meetings or briefings to educate decision makers	44	20
Provide data and information at board meetings and legislative hearings, upon request	31	27
Demonstrate support for PSE change strategies from stakeholders and constituents	19	20
Interactions with Allies^**	n = 22	n = 64
Identified Allies	96	80
Reached out to Allies	86	78
Had 1 or more meetings to discuss common interests	100	70
Regularly shared information and coordinated activities to achieve mutual goals	91	75
Interactions with Opponents^**	n = 22	n = 64
Identified Opponents	59	30
Reached out to Opponents	23	22
Identified strategies to address Opponent's concerns	27	28

PDs answered questions on their top 2 PSE change strategies.

Percent reporting “yes” for each method of stakeholder outreach and decision maker education.

Percent reporting either strongly agree or somewhat agree with each statement. Extent of agreement measured on a 5-point Likert scale – strongly disagree, somewhat disagree, neutral, somewhat agree, strongly agree – with the option to mark each as not applicable.

NCCCP, National Comprehensive Cancer Control Program; PD, program directors/designees; PSE, policy, systems, and environmental.

Decision maker education methods

At T2, demo PDs reported using multiple methods to educate decision makers about PSE strategies (Table 2): provide data and other information to demonstrate value, develop specific proposals for PSE strategies, attend in-person meetings or briefings with decision makers when requested, and provide evidence showing the health impact of PSE strategies. Other PDs selected the first 3 methods less frequently than demo PDs, but stated more often that they provide evidence showing the health impact of PSE strategies (58% compared to 44%). There was very little change between T1 and T2 (data not shown).

Interactions with allies

At T2, most respondents reported that their PSE workgroups/cancer coalition members had identified allies for the selected PSE strategies (Table 2), though demo PDs reported this more frequently than other PDs (96% and 80%, respectively). Similarly, at T2 a higher percentage of demo than other PDs reported that they: reached out to allies, had 1 or more meetings with allies to discuss common interests, and regularly shared information and coordinated activities to achieve mutual goals (Table 2).

Interactions with opponents

In general, all respondents reported interactions with opponents less frequently compared to interactions with allies (Table 2). However, at T2 demo PDs stated more often than other PDs that they identify opponents (59% compared to 30%). The 2 groups were similar in terms of reaching out to opponents to better understand their position or find common ground and to say they looked for ways to address opponents' concerns about the PSE approaches (Table 2).

Outcomes

Figure 2 provides an overview of reported awareness and support for selected PSE strategies among stakeholders and decision makers at both time points.

FIG. 2.

Changes in stakeholder and decision maker awareness of and support for policy, systems, and environmental change issues. % Aware is the combined percentages for the response categories aware but uninformed, aware but inaccurately informed, and aware/accurately informed, versus the other categories unaware, unsure/don't know. % Support is the combined percentages for the response categories visible and vocal program champion [stakeholders only] and supportive, versus the other categories opposed, neutral, no position, don't know. PD, program directors/degrees; Demo PDs, PDs in demonstration program; other PDs, PDs not in demonstration program.

Stakeholder awareness

Overall, respondents in both groups reported high levels of awareness among stakeholders at both time points, and similar magnitudes of change between the time points (8 percentage points). Demo PDs reported higher percentages of stakeholder awareness at both time points than other PDs. From T1 to T2, demo PDs reported an increase for aware/uninformed (14 to 24%), no change for aware/inaccurately informed (steady at 8%), and a slight decrease for aware/accurately informed (70% to 68%). From T1 to T2, the other PDs reported an increase for aware/uninformed (9% to 17%), an increase for aware/inaccurately informed (6% to 21%), and a substantial decrease for aware/accurately informed (65% to 51%).

Stakeholder support

At T1, the 2 PD groups reported similar percentages of support among stakeholders. But by T2, demo PDs reported an 11-point increase, while other PDs reported a 4-point decrease. This overall shift in support reflects a reduction in the percentage of demo PDs reporting either opposed, no position, or don't know by T2. In contrast, other PDs were reporting an increase in the number of opposed and don't know by T2.

Decision maker awareness

Overall, respondents in both groups reported high levels of awareness among decision makers at both time points (although lower than stakeholders), and similar magnitudes of change between the time points (2–3 percentage points). However, the 2 groups had different directions of change: demo PDs reported a slight increase in awareness by T2, while other PDs reported a slight decrease. From T1 to T2, demo PDs reported a decrease for unaware (6% to 0%), a slight decrease for aware/uninformed (22% to 19%), no change for aware/inaccurately informed (steady at 6%), and an increase for aware/accurately informed (44% to 50%). From T1 to T2, other PDs reported an increase for aware/uninformed (9% to 17%), an increase for aware/inaccurately informed (6% to 21%), and a substantial decrease for aware/accurately informed (65% to 51%). Both groups reported substantial percentages of unsure/don't know at both time points with small increases from T1 to T2: 22% to 25% for demo PDs; 18% to 20% for other PDs.

Decision maker support

At T1, demo PDs stated less often than other PDs that decision makers were supportive (67% and 74%, respectively) and stated more often that they were opposed to PSE approaches (11% and 2%, respectively). This gap closed quite a bit by T2 (69% and 71%, respectively), when fewer demo PDs reported opposition (6%), while other PDs held steady (2%).

Discussion

The results from the PD Survey showed that PDs who participated in the demonstration program reported greater increases in staff competencies over time compared to other PDs for most skills. Furthermore, demo PDs were more likely than other PDs to say that TA and trainings from CDC and national partners were the most helpful resources for building their capacity. Evaluations of other national initiatives to promote the use of PSE approaches in public health also have demonstrated the importance of TA/training for building capacity and contributing to successful implementation and outcomes.^23

–28 TA/training is also recognized as an essential component for improving translation of research findings into health promotion practice.^29,30 However, quality TA/training is key to building capacity.^30,31 Elements of the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework, particularly adoption and maintenance,³² may be helpful as well to measure impact and make refinements as needed to implementation activities so that NCCCP grantees adopt practices to build their capacity for PSE strategies and maintain them over time for effective PSE change.

Demo PDs reported having a PSE workgroup, reported sufficient sector representation within their workgroups, and expressed higher confidence in the workgroup members' PSE competencies more often when compared to other PDs. Collaboration, partnerships, and communication among key stakeholders, including the formation of dedicated workgroups, are important facilitating factors for achieving the goals of PSE strategies.^{24,26
–28,33
–35} Although a high proportion of both PD groups surveyed reached out to allies to discuss areas of mutual interest, share information, and collaborate, demo PDs had greater awareness of opponents. However, low levels of outreach to opponents were reported by both groups of PDs, as was looking for ways to address their concerns.

The overall percentage of PDs who said their stakeholders were aware of PSE approaches, whether informed or uninformed, was much higher among demo PDs compared to other PDs, a gap that grew over time. Similarly, the percentage of demo PDs reporting awareness among decision makers increased slightly, and those reporting opposition fell—while the percentage of other PDs reporting awareness and support among decision makers decreased. These changes may reflect the greater outreach and education efforts on the part of PSE workgroups within demonstration program grantees.

This survey has several limitations. Because the study team did not achieve a 100% census among all PDs, the findings cannot be considered representative of all NCCCP grantees. At baseline, demonstration grantees may have had better skills for PSE approaches because they were selected through a competitive application process to receive funding. The lower survey response rate at T2 limited the ability to use analytic approaches, such as analysis of covariance, to control for baseline differences between groups. During both survey time periods, numerous public health initiatives also were occurring to build capacity for PSE approaches. Therefore, these efforts also may have contributed to present study findings. Data collection approval for the first time point was delayed, resulting in a shortened period between the data collection time points. Therefore, some changes of interest that occurred earlier in the demonstration project may not have been detected. Some staff turnover occurred from T1 to T2, resulting in a handful of survey respondents who were not the original PD or designee at T2. This may have biased the results to a small extent if the new survey respondents were not familiar enough with their staff and partners' skills and competencies, or lacked knowledge about their stakeholders or decision makers. Finally, as the survey relies on self-reported information from PDs (or their designees), social desirability bias could affect some of the findings.

This survey also has several strengths. One strength is that survey measures were linked to a theory- and evidence-based conceptual model developed to guide the overall evaluation.²² Furthermore, this study adds a unique and novel aspect in that previous studies in related areas have not examined proactive engagement with stakeholders who may be opposed to PSE initiatives.^24,28,34 Additionally, the survey allowed for the examination of change in measures over time, and for comparison of demonstration grantees to the other NCCCP grantees.

Conclusion

NCCCP has consistently recognized the use of PSE strategies as an essential component of comprehensive cancer prevention and control.¹⁷ Since 2010, PSE strategies have been an explicit priority for the NCCCP and this is reflected in the programmatic requirements and resource investments. For example, with the funding cycle that started in 2012, the NCCCP required grantees to hire a half-time subject matter expert in PSE approaches. This trend also is reflected in CDC's priorities for chronic disease prevention and control in general.³⁶ The demonstration program examined whether additional resources and support could promote and enhance the use of PSE approaches to a large extent and within a relatively short period of time. The results presented here suggest that the demonstration program framework was successful in enhancing grantee capacity for using PSE strategies. The program framework, and the results of its evaluation, can be used to promote continued implementation of PSE strategies for sustainable cancer prevention and control. It also may be adapted for other chronic disease public health programs that desire to increase their capacity for using such strategies.

Footnotes

Acknowledgements

The authors would like to acknowledge CDC colleagues Monique Young and Anne Major for their valuable contributions to this evaluation. The authors would also like to thank Adrienne Cocci of Battelle for producing the tables and figures. We would also like to thank the study participants for sharing their experiences and insights with us.

Author Disclosure Statement

The authors declare that there are no conflicts of interest. The authors received no financial support for this article. The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

References

Heron

, Anderson

. Changes in the leading cause of death: recent patterns in heart disease and cancer mortality. NCHS Data Brief, 2016:1–8.

Jemal

, Ward

, Johnson

, et al. Annual report to the nation on the status of cancer, 1975–2014, featuring survival. J Natl Cancer Inst, 2017; 109. DOI: 10.1093/jnci/djx030.

Weir

, Thompson

, Soman

, Moller

, Leadbetter

. The past, present, and future of cancer incidence in the United States: 1975 through 2020. Cancer, 2015; 121:1827–1837.

De Flora

, Bonanni

. The prevention of infection-associated cancers. Carcinogenesis, 2011; 32:787–795.

Reuben

. 2008. –2009 Annual Report, President's Cancer Panel. Reducing environmental cancer risk :what we can do now. Bethesda, MD: US Department of Health and Human Services, National Institutes of Health, National Cancer Institute, 2010.

US Department of Health and Human Services. The health consequences of smoking: 50 years of progress. A report of the surgeon general. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, 2014.

Nelson

, Cantor

, Humphrey

, et al. Screening for breast cancer: a systematic review to update the 2009 U.S. Preventive Services Task Force Recommendation. AHRQ Publication No. 14-05201-EF-1. Rockville, MD: Agency for Healthcare Research and Quality, 2016.

Vesco

, Whitlock

, Eder

, et al. Screening for cervical cancer: a systematic evidence review for the U.S. Preventive Services Task Force. Report No.: 11-05156-EF-1. Rockville, MD: Agency for Healthcare Research and Quality, 2011.

Cohen

, LaMonte

, Erb

, et al. American Cancer Society head and neck cancer survivorship care guideline. CA Cancer J Clin, 2016; 66:203–239.

10.

El-Shami

, Oeffinger

, Erb

, et al. American Cancer Society colorectal cancer survivorship care guidelines. CA Cancer J Clin, 2015; 65:428–455.

11.

Runowicz

, Leach

, Henry

, et al. American Cancer Society/American Society of Clinical Oncology breast cancer survivorship care guideline. CA Cancer J Clin, 2016; 66:43–73.

12.

Skolarus

, Wolf

, Erb

, et al. American Cancer Society prostate cancer survivorship care guidelines. CA Cancer J Clin, 2014; 64:225–249.

13.

Travis

, Demark Wahnefried

, Allan

, Wood

, Ng

. Aetiology, genetics and prevention of secondary neoplasms in adult cancer survivors. Nat Rev Clin Oncol, 2013; 10:289–301.

14.

Wood

, Vogel

, Ng

, Foxhall

, Goodwin

, Travis

. Second malignant neoplasms: assessment and strategies for risk reduction. J Clin Oncol, 2012; 30:3734–3745.

15.

Frieden

. A framework for public health action: the health impact pyramid. Am J Public Health, 2010; 100:590–595.

16.

Abed

, Reilley

, Butler

, Kean

, Wong

, Hohman

. Comprehensive cancer control initiative of the Centers for Disease Control and Prevention: an example of participatory innovation diffusion. J Public Health Manag Pract, 2000; 6:79–92.

17.

Belle Isle

, Plescia

, La Porta

, Shepherd

. In conclusion: looking to the future of comprehensive cancer control. Cancer Causes Control, 2010; 21:2049–2057.

18.

Leeman

, Sommers

, Vu

, et al. An evaluation framework for obesity prevention policy interventions. Prev Chronic Dis, 2012; 9:E120.

19.

Kegler

, Swan

. An initial attempt at operationalizing and testing the community coalition action theory. Health Educ Behav, 2011; 38:261–270.

20.

Lavinghouze

, Snyder

, Rieker

. The component model of infrastructure: a practical approach to understanding public health program infrastructure. Am J Public Health, 2014; 104:e14–e24.

21.

Butterfoss

, Kegler

. The community coalition action theory. In: DiClemente RJ, Crosby RA, Kegler MC, eds. Emerging theories in health promotion practice and research, 2nd ed. San Franciso, CA: Jossey-Bass, 2009:237–276.

22.

Rohan

, Chovnick

, Rose

, Townsend

, Young

, Moore

. Prioritizing population approaches: results from a case study evaluation of policy, systems, and environmental change initiatives for cancer prevention and control. Popul Health Manag 2018 [Epub ahead of print]; DOI: 10.1089/pop.2018.0081.

23.

Bunnell

, O'Neil

, Soler

, et al. Fifty communities putting prevention to work: accelerating chronic disease prevention through policy, systems and environmental change. J Community Health, 2012; 37:1081–1090.

24.

Cheadle

, Cromp

, Krieger

, et al. Promoting policy, systems, and environment change to prevent chronic disease: lessons learned from the King County communities putting prevention to work initiative. J Public Health Manag Pract, 2016; 22:348–359.

25.

Hefelfinger

, Patty

, Ussery

, Young

. Technical assistance from state health departments for communities engaged in policy, systems, and environmental change: the ACHIEVE Program. Prev Chronic Dis, 2013; 10:E175.

26.

Kane

, Hinnant

, Day

, et al. Pathways to program success: a qualitative comparative analysis (QCA) of communities putting prevention to work case study programs. J Public Health Manag Pract, 2017; 23:104–111.

27.

Parekh

, Scott

, McMahon

, Teel

. Role of public-private partnerships in tackling the tobacco and obesity epidemics. Prev Chronic Dis, 2014; 11:E99.

28.

Walkinshaw

, Mason

, Allen

, et al. Process evaluation of a regional public health model to reduce chronic disease through policy and systems changes, Washington State, 2010–2014. Prev Chronic Dis, 2015; 12:E37.

29.

Wandersman

, Duffy

, Flaspohler

, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol, 2008; 41:171–181.

30.

Wandersman

, Chien

, Katz

. Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement. Am J Community Psychol, 2012; 50:445–459.

31.

Leeman

, Calancie

, Hartman

, et al. What strategies are used to build practitioners' capacity to implement community-based interventions and are they effective?: a systematic review. Implement Sci, 2015; 10:80.

32.

Glasgow

, Vogt

, Boles

. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health, 1999; 89:1322–1327.

33.

Burbage

, Gonzalez

, Dunning

, Simon

, Kuo

. Building mutually beneficial partnerships to improve physical activity opportunities through shared-use efforts in under-resourced communities in Los Angeles County. Prev Med, 2014; 67 suppl 1:S4–S9.

34.

Kegler

, Honeycutt

, Davis

, et al. Policy, systems, and environmental change in the Mississippi Delta: considerations for evaluation design. Health Educ Behav, 2015; 42(1 suppl):57S–66S.

35.

Lafleur

, Gonzalez

, Schwarte

, et al. Increasing physical activity in under-resourced communities through school-based, joint-use agreements, Los Angeles County, 2010–2012. Prev Chronic Dis, 2013; 10:E89.

36.

Bauer

, Briss

, Goodman

, Bowman

. Prevention of chronic disease in the 21st century: elimination of the leading preventable causes of premature death and disability in the USA. Lancet, 2014; 384:45–52.