Abstract
Tobacco control (TC) networks (in which multiple agencies collaborate) are essential components within comprehensive TC efforts. The aim of this study was to assess the internal coalition outcomes hierarchy model (via the Internal Coalition Effectiveness [ICE] scale) in the present sample. Participants (members of a TC Network on the U.S.–México border; independent Waves 1 [N = 30] and 2 [N = 33; at a 1-year subsequent assessment]) completed a background questionnaire and an adapted version of the ICE scale. Mean values for ICE subscales suggested a strong enthusiasm of Network members and recognition of the importance of a cohesive social vision, employment of efficient practices, a need for improved and maintained knowledge/training, and stable social relationships among members. However, no significant differences were observed between data waves in the ICE subscales, multivariate analysis of variance: λ = .97, F(4, 43) = 0.31, p > .86. Considering a multifaceted assessment may enhance the understanding of the dynamics and strengths of the Network. Finally, including an assessment of the leadership’s perspective regarding internal coalition outcome hierarchy model constructs to compare them with members’ perspective is warranted.
Collaboration among agencies and among community leaders who focus on common health promotion or disease prevention goals has been recognized as an effective means to address complex public health problems (Butterfoss, Morrow, Webster, & Crews, 2003; Cramer, Atwood, & Stoner, 2006a; Gajda & Koliba, 2007). Multiple health outcomes have been approached through the use of agency collaboration, including health education, obesity/nutrition, teen pregnancy, family planning, physical activity, diabetes, alcohol abuse, and substance use in general (for reviews, see Gillies, 1998; Roussos & Fawcett, 2000). These strategic networks employ a range of communication and labor sharing as the primary techniques for achieving both short- and long-term goals that would otherwise not be attainable as entities working independently (Gajda, 2004). Networks may be able to work with communities in order to promote and implement programs and community interventions that may enhance their impact on public health outcomes (e.g., Cramer, Mueller, & Harrop, 2003). Specifically, the methods to achieve positive outcomes and behavior change at the population level require systematic strategizing that may include community interventions, policy change, and health education, among other efforts, in order to be considered effective networks (Gillies, 1998). Furthermore, effectively trained networks may have an impact on public health agendas by improving members’ knowledge, leadership skills, and sharing of resources (Butterfoss et al., 2003; Mueller, Luke, Herbers, & Montgomery, 2006; National Cancer Institute, 2007).
However, maintaining effective collaborative networks with common goals and objectives is not without challenges (Smith et al., 2009). Challenges may include a focus on process-related goals instead of health outcomes, and general difficulties in developing and conducting systematic assessments/evaluations (Smith et al., 2009). Importantly, the internal structure of a network (including members’ perceptions) has been found to be positively correlated with network effectiveness and to potentially translate into synergistic effects toward common goals (Lasker, Weiss, & Miller, 2001).
Tobacco Control Efforts
Specifically, tobacco control (TC) networks have promoted the advocacy of policy, community programs, and capacity building among its members and their organizations/agencies (Mueller et al., 2006; National Cancer Institute, 2007; Reinert, Carver, & Range, 2005; World Health Organization, 2004, 2008). As part of comprehensive initiatives, TC networks contribute to tobacco use prevention, smoking cessation/reduction, and reductions in exposure to environmental tobacco smoke primarily through collaboration with local health departments and public health experts (Centers for Disease Control and Prevention [CDC], 2007; Cramer et al., 2003; Mamudu & Glantz, 2009; Roussos & Fawcett, 2000). Similar to the approaches targeting other health outcomes, TC efforts include a wide range of interventions at the individual, community, and population levels. Population-level resources (e.g., QuitLine, online cessation services) include the expertise of media firms, health care providers, tobacco treatment specialists, and other experts within the TC arena to provide effective results (CDC, 2007). Regionally, along the U.S.–México border, systematic efforts made by stakeholders, community coalitions, the A Smoke Free Paso del Norte (a regional comprehensive TC initiative), and the implementation of a municipal clean indoor air ordinance have all likely contributed to declines in adult tobacco use after more than 10 years of the implementation of a coordinated and systematic TC strategy (Paso del Norte Health Foundation, 2007; Taylor et al., 2012).
Recent Potential TC Foci
TC networks can also contribute to the promotion and support of both clean indoor and outdoor air policies (CDC, 2007). Strategies include advocating for policies that ban the use of tobacco products in worksites, universities (Lee et al., 2010; Lee, Ranney, & Goldstein, 2013), public housing (Treiber, Acosta-Deprez, Kipke, Satterlund, & Araquel, 2012), bus stops, and parks (Hyland, Barnoya, & Corral, 2012). For example, in the United States, approximately 844 municipalities have implemented smoke-free park policies, with Oklahoma already having a statewide smoke-free park policy (American Nonsmokers’ Rights Foundation, 2013). Guided by national trends (while simultaneously supporting/promoting clean indoor air ordinances), the Paso del Norte Tobacco Control Network has focused attention on promoting outdoor tobacco-free policies as part of the primary objectives for the A Smoke Free Paso del Norte initiative (Taylor et al., 2012).
Network Engagement and Participation Dynamics
Networks can produce synergistic effects on public health outcomes (Jones & Barry, 2011); this synergy may be facilitated by a strong leadership that promotes a positive learning and organizational environment while working toward common goals (Cramer et al., 2006a). Nevertheless, a number of network dynamics are important considerations in network goal achievement (El Ansari & Phillips, 2001). First, it is important for networks to consider the costs and benefits to participation, members’ satisfaction from participation, commitment, and members’ contributions toward stated objectives (El Ansari & Phillips, 2001). A network leadership that focuses only on goal attainment and neglects members’ knowledge/skills acquisition may affect the collaborative environment (Cramer et al., 2003; Provan & Milward, 2001). Additionally, a network in which members’ participation and responsibilities are clearly defined may promote satisfying collaborative relationships (Cramer, Atwood, & Stoner, 2006b).
A network evaluation process is important to consider since networks undergo developmental phases similar to those identified in group process theory (Lantz, Viruell-Fuentes, Israel, Softley, & Guzman, 2001). Stages include formation, implementation, maintenance, and the accomplishment of clear goals (Butterfoss, Goodman, & Wandersman, 1996). During network formation, there is a focus on the monitoring of activities, members, and services; later, networks move into phases involving action/implementation, and finally into dissemination and policy impact (Lantz et al., 2001). Kegler, Steckler, Malek, and McLeroy (1998) suggested that networks’ needs depend on their level of formation, such that during the implementation stage, important factors to consider are a network’s ability to provide its own vision (i.e., preferably local), to have prepared staff with sufficient work time, productive communication, and cohesion among members.
Theoretical Model: The Internal Coalition Outcome Hierarchy Model
The internal coalition outcome hierarchy model (ICOH) provides a framework to assess relevant constructs that inform the evaluation of internal network structure and programming (Cramer et al., 2006a). As an adaptation of the targeting outcomes of programs model, which focuses on the assessment of coalition impact, the ICOH model includes seven constructs (social vision, efficient practices, knowledge and training, relationships, participation, activities, and resources; Cramer et al., 2003), which are assessed via the Internal Coalition Effectiveness (ICE) scale (Cramer et al., 2006b). The first three constructs (resources, activities, and participation) focus on the most basic aspects of a network that will promote its proper functioning. The next three constructs (relationships, knowledge and training, and efficient practices) focus on outcomes that are relevant to the internal structure of the network and perceived gains by members (e.g., improved relationships/networking and knowledge). Finally, the last construct (social vision) denotes the need for both network members and the leadership to have a shared vision with respect to the mission and goals of the collaborative (i.e., Network). The ICOH constructs are theorized to promote long-term sustainability by building capacity and by improving the benefits obtained by members as a result of their participation within the network (Cramer et al., 2006a). As such, guided by the ICOH model and the constructs assessed via the ICE instrument, this study will examine whether member participation promotes the strengthening of the Network’s internal structure/functioning.
The Paso del Norte Tobacco Control Network
The strengthening of local TC networks through the contribution of multiple agencies promotes improved community services (Cramer et al., 2006a). Although dynamic, the membership size of the Network has remained stable (median number of attendance = 38 individuals), including the time during which it was assessed for the purposes of this study. The Network had been formed for approximately 3 years by the time of this study (at the time of Wave 2), having ongoing monthly meetings, although for the previous 5 or more years, a smaller, less structured TC coalition had existed. The Paso del Norte Tobacco Control Network (located in El Paso, Texas) serves as a link for health providers, public health scientists, and other professionals from both sides of the U.S.–México border. The background of the Network members is varied and includes those who work at the clinical/individual level to public health scientists working at the macro level (see Table 1 for the type of agencies represented by participants). Similarly, the leadership of the Network is composed of individuals with expertise that represents the different levels of and realms within TC. The present study assessed the internal structure of a Network that provides a linkage between individuals and agencies (e.g., nonprofit, academics, and practitioners) that collaborate on the common goal of TC. Particularly, given the demographics of the region (a predominantly Hispanic population), these efforts are consistent with recommendations to involve ethnic minorities (including stakeholders from both sides of the border) within coalition/network activities (Northridge, 2004). Although the need to evaluate programmatic efforts and outcomes (Cramer et al., 2003) is essential, there also exists a need to focus on process evaluations, which may likely enhance a network’s internal functioning, as well as its effectiveness in addressing health-related goals. As such, this study will assess Network members’ perceptions within the ICOH model.
Descriptive Network Member Characteristics
Aim and Hypothesis
The aim of this study was to provide an assessment of the ICOH model (via ICE) within The Paso del Norte Tobacco Control Network.
The study hypothesis was that significant improvements would be observed between assessment points with respect to the social vision, efficient practices, knowledge and training, and relationships subscales.
Method
Participants
Participants (affiliated with organizations devoted to TC issues) from an existing, though dynamic, TC Network on the U.S.–México border completed a survey at two time periods (N = 30, Wave 1; N = 33, Wave 2, at a 1-year subsequent assessment). The median Network members’ attendance during the 2 years in which the assessments took place was 38 participants. As such, wave sample sizes are reflective of typical attendance. Participants included members who had shown a pattern of regular attendance or who recently became involved in the Network. See Table 1 for additional participant characteristics.
Measures
Participants completed the following paper-and-pencil measures:
A background questionnaire collected data about member agency affiliation (e.g., medical), agency size, and their status as Network members (e.g., newcomer, regular). Participants also reported approximately how many meetings they attended during the previous year, plans to attend the next meeting, and percentage of their work time devoted to TC. Last, participants rated the adequacy of activities within the Network (adequate/inadequate) with respect to eight TC aspects/domains (e.g., tobacco use prevention/cessation).
Network members completed an adapted version of the ICE scale (Cramer et al., 2006b). The adapted version (only four subscales used from the original six) was used to accommodate to the nature and current state of The Paso del Norte Tobacco Control Network. The subscales assessed included social vision (four items; e.g., “By working together members have a shared social vision”), efficient practices (eight items; e.g., “By working together members have strengthened each other’s advocacy efforts”), knowledge and training (six items; e.g., “By working together members have enriched each other’s abilities and skills in the issues”), and relationships (five items; e.g., “By working together members have established positive relationships and strong links with community members that the coalition wants to engage and mobilize”). Each construct was assessed using Likert-type scale items (1 = strongly agree to 5 = strongly disagree). In the original validation study (Cramer et al., 2006b), the internal consistency for the four subscales (from the original six subscales) assessed in this study ranged from α = .59 to .95; however, in this study, internal reliability ranged from α = .75 to .86 (see Table 2).
Internal Coalition Effectiveness Scale
NOTE: Range = 1-5; Wave 1, N = 30; Wave 2, N = 33. λ = .97, F(4, 43) = .31, p > .86.
Procedure
The study was approved by the Internal Review Board of The University of Texas at El Paso on the U.S.–México border. To maintain confidentiality, a number code based on the last four digits of members’ telephone number was used to identify participant data. Participants completed the informed consent process before completing the survey in a conference room used for monthly meetings. The survey was administered both at Wave 1 and, then 1 year later, at Wave 2.
Approach to Analyses
Descriptive statistics were generated (see Table 1). Significant differences between survey waves for meeting attendance in the past year were assessed via a z test. Changes between waves with respect to adequacy of activities were analyzed using a paired samples t test for those who were surveyed in both time periods.
The adapted ICE scale was examined within the context of four subscales of interest: social vision, efficient practices, knowledge and training, and relationships. The efficient practices, knowledge and training, and relationships subscales were examined simultaneously within a general linear modeling framework to assess for changes between survey wave responses by following the ICOH model (Cramer et al., 2003). Given that ICE subscales are conceptually related, Wilks’s lambda was used to test for changes between survey waves using a multivariate analysis of variance.
Results
Fluctuation in member attendance between survey waves was high, and only 7 individuals out of 63 were surveyed in both waves. For this reason, we chose to focus on differences in members who provided independent responses, rather than on the extremely small sample of 7 individuals from which very limited inference can be made in public health–related activities. Infrequently and sporadically, missing data were also present, although item-level missingness was thought to be related to some participants hurrying to complete the survey to attend to later activities, suggesting that sporadic item missingness is not informative. Perceived adequacy of activities was relatively high on average (M = 83%, SD = 20%), and no significant differences between data waves were observed. Common work-type settings for both waves are described in Table 1.
Mean values (all greater than the midpoint, 3; range 1-5) for ICE subscales (for both data waves) suggested a strong enthusiasm of Network members and recognition of the importance of a cohesive social vision (Mwave 1 = 4.02; Mwave 2 = 4.02), employment of efficient practices (Mwave 1 = 3.81; Mwave 2 = 3.96), a need for improved and maintained knowledge/training (Mwave 1 = 3.93; Mwave 2 = 4.01), and stable social relationships among members (Mwave 1 = 3.92; Mwave 2 = 4.04). However, no significant differences were observed between data waves in any of the ICE subscale means, multivariate analysis of variance: λ = .97, F(4, 43) = 0.31, p > .86. Finally, ratings of the adequacy of Network activities were positively associated with the knowledge and training, r = .34, p < .05, and with the relationships ICE subscales, r = .29, p < .05.
Discussion
Evaluation of community networks represents applied science (Reinert et al., 2005); thus, it is important to understand internal functioning with respect to efficient practices. The presence of new members likely attenuated potential significant changes between the two data waves. In part, our findings highlight the flux that networks may experience in terms of both new recruits and inconsistent attendance by other members due to other work pressures. This nonsignificant finding is relevant, since according to the ICOH model, networks should serve as learning organizations for members to promote best practices in their agencies (Cramer, Lazure, Morris, Valerio, & Morris, 2013). On average, members perceived a high adequacy of activities, and the perceived adequacy of activities was associated with the knowledge/training and the relationships ICE subscales.
ICE Constructs
Inconsistent with hypothesis, no significant changes in ICE subscales were observed, which may be partly explained by overall high mean scores at Wave 1 (and thus less room for change in Wave 2). Given that the Network likely was (at the time of the assessments) in a formative stage, it was hypothesized that a significant improvement on ICE subscales would be observed. However, the relatively high ratings/scores present at both waves of data collection indicated that members perceived an adequate strength of the Network with respect to ICE constructs (e.g., social vision, efficient practices). As such, it may be that potential future assessments may show/identify whether similar perceptions continue or change among Network members and how these may be affected by activities conducted within the Network.
A strong enthusiasm of members and recognition of the importance of a cohesive social vision, employment of effective practices, a need for improved and maintained training, and stable social relationships among members were reported. The assessment of ICOH/ICE constructs can be classified into three categories (Cramer et al., 2013): process (e.g., resources), outcome (e.g., relationships), and impact (i.e., social vision). Since each of these is considered a step toward a unified social vision (at the top), it may be that an emphasis on lower order constructs is needed before striving to reach higher order constructs (Cramer et al., 2013). Assessing constructs that may be more salient to relatively recently formed networks (e.g., participation) may be warranted (Cramer et al., 2006b). Finally, activities among members (e.g., increased support) may strengthen the Network and facilitate programmatic efforts (Cramer et al., 2003). Additionally, other researchers (Provan & Milward, 2001; Provan, Veazie, Staten, & Teufel-Shone, 2005) have suggested a more comprehensive evaluation model considering the community, network, and organization/participant levels. For example, by assessing the leadership’s attitudes/perceptions, and comparing them to members’ perceptions, adaptations to improve collaboration could be implemented (Butterfoss et al., 1996; Reinert et al., 2005). Potential activities conducted by the Network leadership targeting specific ICOH model constructs may be useful to potentially enhance the perceptions of collaboration. For example, the knowledge and training and the efficient practices ICOH constructs may have likely been enhanced by recent workshops conducted within the Network and by a continuous update on novel tobacco research by the Network leadership (e.g., attendance to national tobacco research conference).
Limitations and Strengths
The present study had several limitations. The nature of the study did not allow for the inclusion of a control group to effectively assess the impact of Network attendance and activities on relevant constructs. As such, given that the survey was not conducted from an external evaluator, it might be argued that some demand characteristics may have been present (although unlikely given the anonymity provided to study participants). Additionally, this study assessed internal coalition constructs from the members’ perspective only and not from the leadership as well (which could be regarded as a potential bias). The limited attendance overlap (participants present at both data waves) and the addition of members at Wave 2 limited an assessment of change over time for individual members. Last, having a 1-year follow-up assessment (between Waves 1 and 2) could have been either too short to observe significant differences or too long, such that it may have missed temporal changes (e.g., in perceptions, attendance). Yet the cross-sectional responses offered insight regarding Network dynamics among a wide span of professionals engaged in TC efforts in a large urban area. As such, given the dynamics of this Network, the lack of attendance overlap at the particular point in sampling did not necessarily reflect dropout from the Network.
Several other strengths are also noteworthy. Knowledge regarding the internal structure of TC networks that work primarily with (and are likely composed of) minority populations, particularly on the U.S.–México border, is limited; thus, the current study will likely contribute to the literature. Additionally, the internal structure of the Network was assessed by using a theoretical model. Finally, the assessment of members’ perceptions regarding ICOH constructs may allow the Network leadership to improve cohesiveness and structure and strengthen the impact of the Network as an agent within the ongoing comprehensive TC regional efforts.
Conclusion and Future Directions
The present study assessed the internal structure of a Network that provides a linkage between individuals and agencies that collaborate on the common goal of TC within a predominantly Hispanic population. Few differences/associations among measures were observed between assessments. Overall, members from The Paso del Norte Tobacco Control Network endorsed a high adequacy of TC activities. Based on the potential low power to detect significant differences between data waves, the potential, inconsistent (not in number of individuals but in individuals present at both assessment points) Network meeting attendance may have affected study results. However, study results offer insight into the dynamic nature of participation within public health networks.
Future directions include assessing Network members’ priorities regarding TC activities in relation to their perceived level of collaboration. Considering a multifaceted assessment may enhance the understanding of the dynamics and strengths of the Network. Finally, including an assessment of the leadership’s perspective regarding ICOH constructs and the Network’s collaboration level to compare it with members’ perspective is warranted.
Footnotes
Acknowledgements
The authors would like to acknowledge A Smoke Free Paso del Norte, members of the Prevention and Treatment in Clinical Health Lab, and the Centro San Vicente Family Health Center (especially Melissa Aguirre) for their assistance with this study.
The thoughts, views, and comments expressed in this article in no way represent the views of the Department of Veterans Affairs or the U.S. Government.
This work was supported by A Smoke Free Paso del Norte: An Initiative of the Paso del Norte Health Foundation (26-8113-97) awarded to Theodore V. Cooper.
The authors have no competing interests to declare.