Abstract
Participation in community-based self-management education and physical activity interventions has been demonstrated to improve quality of life for those who have arthritis and other chronic diseases. The Centers for Disease Control and Prevention Arthritis Program funded 21 state health departments to expand the reach (defined as the number of people who participate in interventions) of 10 evidence-based interventions in community settings. The Arthritis Centralized Evaluation assessed the strategies and tactics used by state health departments to expand the reach of these evidence-based interventions. The evaluation compared and contrasted processes used by the states to expand reach. Engaging multisite delivery system partners, prioritizing reach, embedding interventions within partners’ routine operations, and collaborating across chronic disease program areas were all dissemination strategies that were correlated with expanded intervention reach. However, states also encountered challenges that limited their ability to successfully engage delivery systems as partners. These barriers included difficulty identifying delivery system partners and the lengthy time periods partners needed to adopt and embed the interventions.
Keywords
Introduction
An estimated 52.5 million adults in the United States are living with doctor-diagnosed arthritis (Barbour et al., 2013). It is the most common cause of disability in U.S. adults (Hootman, Brault, Helmick, Theis, & Armour, 2009). There is strong evidence that community-based physical activity and self-management education intervention programs can improve quality of life for people living with arthritis and other chronic diseases (Brady, Jernick, Hootman, & Sniezek, 2009; Institute of Medicine, 2012), but only small numbers of individuals with chronic conditions participate in these interventions. For example, in the United States, during the period 2010-2012, approximately 100,000 people attended one of the most widely disseminated self-management interventions, the Chronic Disease Self-Management Program (Ory et al., 2013). Forty-one percent of them had arthritis, suggesting that this most widely available intervention reached less than 0.1% of people with arthritis during this period.
Though there is growing evidence on the effectiveness of “packaged” (ready-to-use) self-management education and physical activity interventions (Brady et al., 2009), we know little about the most effective ways to achieve widespread dissemination and use of these interventions (Der Ananian, Desai, Smith-Ray, Seymour, & Hughes, 2012; Owen, Glanz, Sallis, & Kelder, 2006). Recent literature has identified a number of pitfalls in efforts to scale up evidence-based interventions for population impact. Establishing “pay-to-play” partnerships, investing in unsustainable delivery models, and using nonstrategic growth strategies have been observed to limit expansion of interventions (Brady, Sniezek, & Ramsey, 2012). Institutional barriers such as lack of adequate funding can also prevent wide dissemination (Brownson et al., 2007). We were not able to locate any systematic collection of data from the field on the relative effectiveness of dissemination strategies used with packaged interventions.
The Centers for Disease Control and Prevention’s (CDC) Arthritis Program has a strong interest in identifying effective translation strategies because it focuses on improving the quality of life of people affected by arthritis by increasing the reach of evidence-based physical activity and self-management education interventions (Hootman, Helmick, & Brady, 2012). From 2008 to 2012, CDC funded 12 Arthritis Programs in state health departments (SHD-APs) to disseminate arthritis-appropriate interventions. SHD-APs were assigned a reach target (the number of people who participate in interventions) equal to 4% of the people in their state who had arthritis, with the target capped at 40,000 total participants. In addition, to explore the feasibility of disseminating arthritis-appropriate interventions through other state chronic disease programs, the National Association of Chronic Disease Directors (NACDD), working with the CDC, made $50,000 Arthritis Integrated Dissemination (AID) grants to nine other state health departments. AID grantees were charged to integrate an arthritis-appropriate intervention into a state chronic disease program and expand the intervention’s reach but were not given a specific reach target. SHD-AP and AID grantees selected from a menu of 10 packaged physical activity and self-management education interventions, and they were asked to follow a systems approach to expanding intervention reach (Brady et al., 2012).
To learn about effective strategies for disseminating evidence-based interventions, NACDD and the CDC Arthritis Program conducted the Arthritis Centralized Evaluation. The purpose of the evaluation was to systematically explore real-world efforts to expand the reach of interventions by comparing and contrasting dissemination strategies and tactics used by both SHD-APs and AID grantees. We anticipated that this evaluation would be useful to a broad range of professionals engaged in developing, implementing, or evaluating interventions as well as those interested in translating research-based interventions into widespread public health practice. It also contributes to the growing research-to-practice knowledge base for evidence-based interventions. The purpose of this article is to identify dissemination strategies and tactics associated with increased reach of evidence-based interventions through a multisite quantitative and qualitative evaluation. A secondary purpose was to highlight challenges and facilitators associated with these strategies and tactics.
Method
The evaluation design was guided by two organizational frameworks: (1) the RE-AIM framework for evaluating public health impact (Glasgow, Vogt, & Boles, 1999) outlining Reach, Efficacy, Adoption, Implementation, and Maintenance aspects of health promotion programs, which has been used in previous evaluations of dissemination efforts (Bopp et al., 2007), and (2) the National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Knowledge to Action framework developed by the NCCDPHP Work Group on Translation (Wilson, Brady, & Lesense, 2011), which highlights key translation processes. While the evaluation did not look at all dimensions of either framework, both were useful in identifying factors of interest related to dissemination and reach of the evidence-based interventions.
Data Collection
The evaluation used quantitative and qualitative data collection methods to capture information from the 21 funded states. Data sources included the following: (1) states’ semiannual progress reports that summarized a wide-range of state activities and budget allocations from July 2008 to June 2011, (2) reach data submitted by the states, (3) in-depth interviews and site visits, and (4) follow-up interviews (see Table 1 for data sources and topics).
Data Sources and Topics
Data from progress reports were extracted and coded into a database. For quality assurance, evaluation staff met regularly to review codes and establish consensus on coding decisions. In addition to progress reports, annual reach numbers from 2008 to 2011 were coded into the database.
The evaluation team conducted site visits in seven states selected to gain perspective from grantees with different funding levels and sources, levels of performance on program objectives, reach numbers, and regions. Site visits typically included interviews with the arthritis program coordinator, chronic disease director, bureau chief, multiple project staff within the state health department, and selected delivery system partners (organizations offering the intervention to their constituents at multiple sites). Arthritis program coordinators in the remaining 14 states participated in a 2-hour telephone interview. All grantees completed a preinterview questionnaire that identified delivery system partners they had “successfully” worked with to expand the reach of their interventions. Telephone interviews were conducted with selected partners identified in this questionnaire. Site visits and interviews were held from December 2010 to May 2011 (for data collection protocols, see Appendix A, available online at hpp.sagepub.com/supplemental).
Follow-up interviews with nine selected arthritis program coordinators were conducted during March-April 2012 to assess strategy changes or adjustments.
Qualitative data including strategies, characteristics, and observations from interviews and site visits were coded using an analytic rubric created to allow comparisons across the state grantees. Rubrics have proved a useful way to synthesize qualitative evaluation data and permit comparisons across multiple cases (Taggart, Nixon, & Wood, 1998). The rubric distinguished between states’ strategic perspectives (best overall approaches to meet reach goals) and strategic position (tactics states use to meet goals; Patton & Patrizi, 2010). This analysis included 32 variables from the rubric; four used a yes/no rating and 25 contained a three-option rating scale, for example, low (factor minimal or absent), moderate, high (stellar example of factor); the remainder were descriptive (i.e., award type, funding type, partner name; for a full description of variables, see Appendix B, available online at hpp.sagepub.com/supplemental). Rubric items were examined as individual variables in the analyses; no summary score was created because the limited number of grantees in the cross-site evaluation would not have permitted us to validate any ad hoc summary scales. Similar rubrics and rating scales have proven useful in assessing grant implementation at the state level in a number of multisite evaluations of prevention and health promotion (Center for Mental Health Services, 2006; Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration, 2010). Rubric categories included overall assessments, contextual factors, partners, strategies, and tactics used to recruit and support partners and to expand reach. Coding dimensions and anchors were developed in an iterative process, based on the interviews and document review from pilot site visits and then expanded and refined during subsequent site visits.
Data Analysis
To integrate quantitative and qualitative data, progress report, reach, and rubric data were merged into a SAS data file. The study’s dependent variable was Reach, calculated for each SHD-AP as the percentage of the assigned reach target achieved by summing annual numbers of participants across 2008-2011. Additionally, a categorical Reach variable was created by examining reach trends graphically over time; categorization was based on observed distribution of grantees on the reach outcome. While the middle two quartiles were a cluster of very similar reach totals, those in the highest and lowest quartiles were distinct from the middle quartiles. This categorical reach variable was used to illuminate findings by facilitating contrasts in the characteristics and practices of SHD-APs in the highest and lowest quartiles. Only the 12 SHD-APs had assigned reach targets, so this article focuses primarily on those 12 state programs, although selected qualitative data from the 9 AID states were incorporated where helpful.
Independent variables were state and partner characteristics, implementation activities, strategies, challenges, and solutions identified in the database and analytic rubric. Correlations were computed among these characteristics of grantee organizations and approaches and the continuous dependent variable. Additionally, cross-tabs were examined with the categorical Reach variable to identify common practices adopted by high-performing states, as well as to identify challenges or barriers disproportionately faced by low-performing states.
In some cases, quantitative correlations were not statistically significant at the 95% level of confidence, but qualitative data suggested a relationship. Because the evaluation was multimodal and designed to be exploratory, we used the term weakly correlated to refer to relationships with a chi-square probability of p < .20, and at least 3 of the 21 states strongly indicated that the activity contributed to their program effectiveness. Other relationships were considered “not correlated.” Descriptions of challenges to and facilitators of success also drew on qualitative data from all 21 participating states.
Results
Interventions
Each state selected from a menu of 10 evidence-based self-management education and physical activity interventions (Table 2). Some interventions are arthritis-specific; others are appropriate for participants with a variety of chronic conditions, including arthritis. As part of their funding, SHD-APs implemented at least one self-management education and one physical activity program. All SHD-APs chose to implement the Chronic Disease Self-Management Program (CDSMP) and most also offered the Arthritis Self-Management Program. For physical activity, most SHD-APs offered the Arthritis Foundation Exercise Program, EnhanceFitness (all described in Brady et al., 2009), and Walk With Ease (Callahan et al., 2011). Most AID grantees focused their efforts on CDSMP.
Evidence-Based Arthritis and Chronic Disease Interventions Adopted by State Arthritis Programs and Arthritis Integrated Dissemination Grantees
NOTE: Interventions (Brady et al., 2009): ALED = Active Living Every Day; ASMP = Arthritis Self-Management Program; AFEP = Arthritis Foundation Exercise Program; CDSMP = Chronic Disease Self-Management Program; EF = EnhanceFitness, F&S = Fit & Strong! Toolkit = The Arthritis Toolkit; WWE = Walk With Ease (Callahan et al., 2011).
Reach
Using a linear projection model based on trends in annual reach numbers from 2008-2011, we projected forward to estimate the likely reach through 2012. SHD-APs demonstrated yearly increases, but none of the 12 SHD-APs were projected to meet their reach target during the grant cycle. There was considerable variation in reach across SHD-APs; three States were projected to achieve more than half of their reach target while five States were unlikely to achieve as much as a third of their target (Figure 1).
Cumulative Percentage of Reach Target, by 12 State Arthritis Programs 2008-2011
Strategies for Expanding Reach
The evaluation team identified seven strategies SHD-APs used to expand the reach of their selected interventions (Table 3). Four strategies were associated with greater reach, and three other strategies were attempted but not associated with greater reach.
Strategies Used to Expand Reach and Their Correlation With Reach, State Arthritis Programs (n = 12)
Strategy had a chi-square probability of p < .20, and at least three states strongly indicated this strategy contributed to their program effectiveness. In this manner, qualitative findings were integrated with quantitative findings and were used to support statistically detected relationships.
Strategies Associated With Greater Reach
Partnering With Delivery Systems With Multiple Sites
As part of their grants, SHD-APs were encouraged to partner with multisite delivery systems. The intent was that these partner organizations, which had the capacity to deliver the intervention and multiple delivery sites, would adopt an intervention and deliver it using their own organizational resources and access to the population. This strategy would permit SHD-APs to invest their efforts in partnership development, technical assistance, and quality control. The correlation between level of rated SHD-AP efforts to work with multisite delivery systems and achieved reach was substantial (r = .66) and statistically significant (p = .020).
Though all SHD-APs attempted to work with delivery systems, SHD-AP commitment to the strategy varied, and they experienced varying levels of success applying the strategy. Interviews revealed that many SHD-APs encountered barriers that limited their ability to successfully engage multisite delivery systems. Some potential delivery systems, such as the YMCA or some senior housing organizations, were more decentralized than expected, with decision makers often located at the local or county level. Successful outreach to these partners required a site-by-site approach that was time-consuming and provided mixed results. Other delivery system partners, such as large hospitals, had complex organizational structures that proved challenging to navigate. Once an appropriate contact person was identified, it took time to learn about the organization’s mission and needs, educate him or her about the interventions, and gain buy-in within the organization. This process was further delayed or derailed if the identified point person left their position.
Analysis of budget allocations showed a weak correlation between SHD-APs financially supporting interventions (e.g., paying for program licenses and materials) or providing marketing materials (e.g., brochures, flyers, or other promotional materials) and increased reach. Financially supporting the intervention or providing marketing materials to partners were more effective ways to implement the delivery system strategy than making grants that financially supported partners’ day-to-day operations.
Among possible partner organizations, states most commonly identified hospitals and University Extensions as valued delivery system partners. Area Agencies on Aging were identified by several SHD-APs; however, other SHD-APs encountered varying levels of enthusiasm and commitment to the interventions from Area Agencies on Aging. Similarly, several SHD-APs identified local health departments as “successful partners,” but others had limited success with local health departments.
Prioritizing Expansion of Reach
Some states explicitly prioritized reach expansion among their program goals. Before every activity, these states asked themselves whether it would help them expand the reach of their interventions. The more focused a SHD-AP was on reach expansion, the more it expanded its reach (r = .83, p = .001).
Embedding Interventions Into Partner Operations
One strategy to promote long-term sustainability was to embed the interventions in the routine operations of the partner organization so that interventions would become institutionalized and delivery would persist. This strategy was weakly correlated with reach (r = .46, p = .128).
Interviews indicated SHD-APs did not share a common understanding of what constituted embedding. SHD-APs with higher levels of reach, per the categorical reach variable, described embedding in terms of organizational or policy change, while SHD-APs with lower levels of reach simply considered an intervention embedded when classes were held regularly or offerings were being expanded. Some SHD-APs considered an intervention embedded if the partner organization sought independent funding sources or obtained their own license for the intervention. SHD-APs encountered barriers that limited their ability to embed interventions in partner organizations. Contextual factors such as the economic recession beginning in 2008 were perceived as affecting an organization’s ability to commit to sustaining the intervention. There were some unique challenges posed by specific interventions. EnhanceFitness, for example, requires instructors with a physical activity background, and it was not always easy to identify people who could qualify to become instructors. For CDSMP, many partners—especially those in early stages of embedding—opted to train employees because it increased their familiarity with the program. However, training employees introduced a new set of implementation challenges, for example, some potential partners were reluctant to allow their employees the 4 days for CDSMP training. Additionally, partner staff often lacked work time to hold the interventions and sometimes had to lead classes in their spare time.
Certain embedding practices were associated with increased reach. SHD-APs that most increased reach (i.e., those in the highest quartile of the categorical reach variable) encouraged their partners to make organizational changes, such as revising job descriptions, revising a mission statement or Web page, or establishing a standard process for enrolling in a program.
Collaborating With Other Chronic Disease Programs
The grants asked SHD-APs to collaborate with other state chronic disease programs to build support for the interventions within the state health departments. Other programs could support an intervention by promoting it at events, referring potential participants, or contributing budget resources for trainings or materials. Efforts to collaborate with other chronic disease programs were weakly correlated with reach (r = .50, p = .102).
SHD-APs reported cultural, organizational, and individual challenges when trying to collaborate with other chronic disease programs. The most frequently mentioned challenge was other program coordinators prioritizing their own grant requirements over cross-program collaboration. When SHD-APs were able to forge collaborative bonds with other program areas, chronic disease leads and bureau chiefs had often encouraged collaboration by developing work plans that included collaboration or by discussing collaboration during regular staff meetings. Chronic disease leads sometimes identified specific collaboration opportunities for program coordinators. Collaboration was also more likely in workplaces that provided opportunity for informal information exchange such as colocating staff or shared lunch or break space.
State Units on Aging also emerged as important collaboration partners. Under the American Recovery and Reinvestment Act of 2009 (Recovery Act), 2-year grants were made available to states to expand CDSMP during the study period. Two thirds of the states in our study received Recovery Act grants, usually awarded to the state Unit on Aging or, in a few cases, the state Health Department. Though State Departments of Health and Units on Aging frequently recognized the collaboration opportunity, contractual barriers, differences in organizational culture, and philosophical differences made collaboration a challenge. Those that found ways to work together collaboratively had greater resources for intervention delivery and support.
Strategies That Did Not Affect Reach
The evaluation identified several strategies SHD-APs used that were not correlated with reach expansion. Several SHD-APs attempted to create regional collaboratives by connecting several organizations together to function as a delivery system to implement interventions. In a rejection of the delivery system strategy, some SHD-APs partnered with organizations to offer interventions at single sites. These partners may have used their internal capacity to host classes, yet only had one location. Finally, some SHD-APs attempted to expand reach by conducting trainings of leaders or master trainers, who in turn were responsible for finding sites to hold classes or workshops, as their growth strategy. None of these strategies were correlated with increased reach.
Discussion
This evaluation added to the knowledge base on strategic approaches to expanding the reach of evidence-based interventions. Prior to 2008, SHD-APs disseminated interventions site-by-site and through leader training, yet this evaluation showed that these strategies were not associated with increased reach. Rather, this evaluation found that by working with delivery system partners, prioritizing expansion of reach in the day-to-day operations of the SHD-AP, and embedding interventions within partner operations, SHD-APs achieved increases in reach. These results are consistent with other observations in the literature, such as Brownson, Fielding, and Maylahn (2009) who found that engaging community partners and using data to monitor and provide feedback on their progress was integral to effective public health efforts. Addressing both organizational and leadership characteristics as was done in this evaluation has been recognized as key components of implementation science frameworks (Damschroder et al., 2009) and taking interventions to scale (Rosenberg & Westmoreland, 2010; Weiss, 2010).
Because of the significant comorbidities that exist between chronic diseases, participation in physical activity and self-management education interventions can contribute to managing not only arthritis but also an array of chronic conditions (U.S. Department of Health & Human Services, 2010). Collaboration on these interventions across program areas leverages limited resources for shared benefits. The evaluation found that support from state chronic disease leadership was key in creating opportunities for collaboration between SHD-APs and other chronic disease programs, which is consistent with research on factors affecting dissemination of physical activity interventions by state health departments (Brownson et al., 2007).
Several limitations of this evaluation should be noted. First, reach numbers varied substantially in quality. States worked with dozens of partners to deliver interventions and encountered challenges in collecting accurate reach numbers from some of their partners. This suggests a need for increased education of delivery system partners on the value and methods of effective reach data collection. Second, changes in reach numbers did not always reflect only SHD-AP strategy or program efforts, limiting their validity as measures of program success. For example, a few SHD-APs worked with well-funded and highly organized partners that had adopted the interventions independent of CDC or NACDD funding. These SHD-APs benefitted by being able to count those partners’ numbers toward their reach targets, but these delivery system partners may have achieved high levels of reach regardless of SHD-AP’s involvement. Third, SHD-AP efforts could not always be separated from activities supported by other funds.
This study had several strengths that add to the robustness of the findings. It employed a systematic approach and integrated mixed research methods that allowed for in-depth inquiry with validation from multiple data sources. It explored real-world experiences and followed states’ efforts for 3 years. Finally, we drew from a diverse set of experiences by states with varying levels of funding and different interventions. Within each state, we sought diverse perspectives, including chronic disease leadership, program directors, and delivery system partners.
This evaluation also suggests directions for future research. In this grant cycle, most states focused on partnership development. Ongoing conversations about effective sustainability practices to ensure that partners continue the interventions will be necessary. We found that embedding was an effective strategy, but more research is needed to understand change processes that lead to the institutionalization of interventions within partner organizations. Further guidance could be developed using organizational change literature (Cohen, March, & Olsen, 1972; March, 1994).
Conclusion
Community-based evidence-based interventions can increase the quality of life for people with arthritis and other chronic diseases. The strategies identified in this study as effective for expanding the reach of these interventions—working with multisite delivery system partners, emphasizing reach expansion, embedding interventions in partners’ routine operations, and collaborating with other chronic disease programs—may be useful to any state or local health department trying to scale their interventions for broader public health impact. This evaluation provides a glimpse of how to effectively expand the reach of physical activity and self-management education interventions in order to make the interventions available to the many people who need them.
Footnotes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
