Abstract
Although school wellness policies have the potential to transform school environments, relatively little has been written about postadoption policy implementation and evaluation (policy to practice). The authors report results of a research study that examined the implementation of school wellness policies in two school districts in northern New Mexico. Through nine key informant interviews with administrators and two focus groups with students, they found that physical activity and nutrition policies were implemented inconsistently in both districts. Study participants identified facilitating factors (e.g., champions, grant funding) and barriers (e.g., competitive food sales, lack of clarity about responsibility for policy enforcement) to policy implementation. Participants also provided recommendations to improve policy implementation, including wellness policy training for school personnel and parents, improving the taste, nutritional value of, and choices in cafeteria food; and involving the community health council to promote community understanding and support of the policies. This study underscores the need to identify and address factors involved in the successful implementation of school wellness policies, looking at schools in the larger context of their communities. It also serves as an example of the potential for communities, schools, and others to work together to address a locally identified health priority.
Keywords
Introduction
Successful implementation of school wellness policies depends on community support and involvement. Promoting healthy choices through public policy is seen increasingly as a joint commitment between communities and schools and a potentially effective tool for addressing the childhood obesity epidemic (Dietz, Bland, Gortmaker, Molloy, & Schmid, 2002). As part of nationwide efforts to address childhood overweight and obesity, the Child Nutrition and WIC (Women, Infants, and Children) Reauthorization Act of 2004 mandated that school districts with federally funded school meal programs develop and implement policies to support healthy nutrition and physical activity habits among children (Jack, 2005). School districts are in the relatively early stages of implementation of appropriate wellness policies that meet the federal mandate. In New Mexico, most school districts have adopted school wellness policies (New Mexico Public Education Department [NMPED], 2004) that followed federal standards.
In July 2008, one of New Mexico’s 32-funded community health councils joined with university faculty, two New Mexico Department of Health programs (the Diabetes Prevention and Control Program and the Office of Community Health Partnerships), to implement the statewide planning and evaluation framework (Sánchez, Carrillo, & Wallerstein, 2011). Through its community health assessment and planning process, the community health council prioritized diabetes and obesity, two interrelated conditions, in their County Health Profile and Plan (San Miguel Community Health Council, 2007). The council then further identified the school environment as a key setting for future intervention, given that students spend about a third of the day in school and eat at least one meal daily in schools (Frieden, Dietz, & Collins, 2010; Wojciki & Heyman, 2006). The collaborative study team, consisting of University of New Mexico, the Department of Health, and the community health council (which we called the Community Diabetes Collaborative or CDC2), received funding in 2009 to work with the two local school districts to study the implementation of school wellness policies.
Background/Literature Review
Although school wellness policies have the potential to transform school environments, relatively little has been written about policy implementation and evaluation. Most school districts do not require evaluation of wellness policy implementation or ongoing policy review and revision (Turnock, 2009).
Since the passage of The Child Nutrition and WIC Reauthorization Act of 2004, several studies have documented varying levels of success in policy adoption (Moag-Stahlberg, Howley, & Luscri, 2008; Probart, McDonnell, Weirich, Schilling, & Fekete, 2008; Sandoval, Chriqui, Hagin, Schneider, & Chaloupka, 2008) and policy implementation (Crepinsek, Gordon, McKinney, Condon, & Wilson, 2009; Fox, 2010; Fox, Gordon, Nogales, & Wilson, 2009; Kubik, Lytle, Farbakhsh, Moe, & Samuelson, 2009; Kubik et al., 2010; Nollen et al., 2009; Snelling & Kennard, 2009; Wenz, Thorius, Wendland, & Litchfield, 2009).
Some studies have assessed the relative strength of written school wellness policies, examining the extent to which the policies contained enforcement mechanisms, funding mechanisms, provision for evaluation, or guidelines for addressing the federal mandate (Belansky, Chriqui, & Schwartz, 2009). In a review of 55 studies examining school-based obesity prevention programs Waters et al. (2011) found that well-articulated school policies and programs contained (a) physical activity and nutritional components, (b) mechanisms for supporting teachers and staff in policy implementation, and (c) mechanisms for the inclusion of parents in achieving both nutritional and physical activity goals. Yet few studies have examined factors that affect the translation of policies (written statements of intent) into effective sustainable practices (organizational changes, procedures, or interventions), such as the perceptions and direct experiences of those administering the policies (Agron, Berends, Ellis, & Gonzalez, 2010; Belansky, Chriqui, et al., 2009; Lambert, Monroe, & Wolff, 2008) or the personal experiences of those most directly affected by the policy: students.
We report on a collaborative research effort to address the problem of childhood obesity by examining the implementation of school wellness policies in two school districts of a New Mexico community. The results of the study would inform the next stage of the community’s longer term commitment to reducing youth obesity.
Policy Analysis
Method
The aim was to examine school nutrition and physical activity policy implementation in two school districts in a northern New Mexico town. Our main question was this: What does the implementation of school wellness policy look like from the points of view of those implementing the policy and those most directly affected by it (e.g., students)? Because we wanted to identify next steps for the community/school collaboration, we also sought to answer the following question: How can the community health council and other partners support implementation of the wellness policies in the two districts?
The study site, a northern New Mexico town with a population of 20,074, is the government center for the county and the commercial center for surrounding rural areas. The county population is predominantly Hispanic (77% in 2007). The town has two public school districts: District A, with 8 school sites and 1,991 students, and District B, with 10 school sites and 1,773 students, excluding the Head Start program (NMPED, 2011). More than 25% of the student population falls under the federal poverty guidelines (NMPED, 2011). The 2009 New Mexico Youth Risk and Resiliency overweight/obesity rate among high school students is 28.5%; the rate for this county’s high school students is similar at 28.1% (New Mexico Department of Health, 2009)
Study team members included CDC2 members and a social work intern/research assistant. All completed the Collaborative Institutional Training Initiative. The University of New Mexico Human Review Research Committee, Health Sciences Center, approved the study.
We first compared state-level school wellness policy process evaluation measures with each district’s wellness policy components (NMPED, 2004). This quick scan allowed us to view the status of policy adoption in the districts, which guided the development of key informant interviews
Key informant (KI) interviews (n = 9)
We used a purposive sample of district-level and school-level administrators and conducted nine key informant interviews from both school districts. We chose administrators because of their knowledge about how policies are implemented at the district and school levels. The purpose of the interviews was to learn from these key stakeholders about facilitating factors and barriers in the implementation of physical activity and nutrition policies.
Focus groups (n = 2)
We conducted two focus groups: one with middle school students from District A (n = 6) and one with high school students from District B (n = 10). The purpose of the focus groups was to learn about students’ experiences with physical activity and nutrition policies.
The study team developed the interview and focus group guides (Appendix) and pilot tested them with individuals from other communities. Two Department of Health partners, both with extensive evaluation and training experience, trained all research team members to ensure comfort and consistency of data collection. The community research team conducted all the interviews and focus groups. One of the community researchers created verbatim transcripts of the recorded transcribed interviews and focus groups, and a university team member checked the transcriptions for accuracy. The study team used a qualitative data management program (QSR International, 2008) to analyze interview and focus group data. University researchers de-identified the interview and focus group data, which they shared with research team members, including community partners. The principal investigator (PI) conducted a 2-hour introductory training session on content analysis (to code qualitative data by key ideas) with the research team. Department of Health and community research team members read and coded a sample of interview data. Members submitted their coded text to the PI, who then compiled the coded results, which was used in finalizing the code book.
The PI used interview and focus group questions (e.g., barriers and facilitators) to develop a priori codes. Question-level coding allowed us to focus on key questions about barriers, facilitators, and recommendations across key informants and focus groups.
Two university research team members coded all individual interviews and focus group data. We coded all interviews and focus groups by district, participant type, and question. Each coder read and coded the transcripts using the a priori codes. Some themes were combined during the course of the coding process. The coders discussed and agreed on new codes as they emerged. When coding differences occurred, they used consensus to arrive at final coding. The full research team reviewed and discussed final coding results.
Confirmatory Analysis
We reported the preliminary results to each school district’s School Health Advisory Council (SHAC), the county community health council, and both school boards to confirm the study results (Merriam, 2009). Several key informants were members of these groups, allowing us to also assess credibility of our findings.
Results
We present the results, including facilitating factors and barriers to policy implementation, recommendations to improve policy implementation, and recommendations for further community/school collaboration to support policy implementation, common to both districts.
Facilitating Factors
We defined facilitating factors as organizational, community, and parent practices or norms that supported policy implementation. A universal facilitating factor among was the presence of one or more district-level champions, or advocates, who were explicitly committed to improving opportunities for physical activity and/or the availability of nutritional food choices to students. Another key factor in both districts was grant funding that supported school-based physical activity and nutrition activities. Key informants provided examples of funding that included federal after-school academic enrichment programs; programs to initiate, expand, or enhance physical education programs; and an evidence-based program to facilitate healthy eating and physical activity behaviors. The impact of these grants was summarized by key informants:
We have two schools right now that are funded through the New Mexico Public Education Department physical education grant. . . . we have one full-time teacher that provides physical education . . . a couple days a week. In the other elementary schools we do have one additional full-time teacher to provide [physical education] one day a week to the remaining elementary children. (KI, District A)
Facilitating factors related to physical activity were largely dependent on financial resources to implement programs. On the other hand, facilitating factors related to nutrition policies relied heavily on school and parental practices or compliance. Key informants noted compliance with policies about food available at schools, including the elimination of soft drink and candy-vending machines and the presence of free and reduced-cost meals. Compliance was summarized by a key informant:
The success is probably the administration support, principals, and of course the parents, even though you do have those couple of parents that are difficult to work with, but all in all everyone has been cooperating and everyone is try to abide with the policy we had set forth. (KI, District B)
Barriers to Policy Implementation
We defined barriers as organizational, community, and parent practices or norms that impeded policy implementation. Barriers across both school districts included lack of time for physical activity in the school day because of competing programs (e.g., No Child Left behind or graduation requirements) and insufficient understanding of written policies by staff and parents.
No Child Left Behind is not only affecting Physical Education but all other elective courses because in an environment such as ours where a student is required to take seven classes a day if he is testing below grade level in reading and math then we have to put these students in intervention classes. (KI, District A)
Although both school districts had SHACs (composed of school and community representatives), their roles and, specifically, the extent to which they actually reviewed and assessed policies were unclear to several key informants:
Can you tell me what SHAC is? [Interviewer: It is the School Health Advisory Health Council]. I have not really been made aware of the current SHAC committee, or who it consists of. I know when I was the assistant principal at the high school our principal was a member of that committee and I know they did meet once or twice but since then I do not know what the current status of the School SHAC is. (KI, District A)
A frequently cited physical activity barrier was the limited, formal physical education requirement (one middle and one high school physical education class). A key nutrition-related barrier in both districts was the presence of commercial vendors outside the school boundaries. Administrators pointed out that they had no jurisdiction over off-campus vendors, who regularly circumvented nutrition policy:
It [the presence of off-campus food vendors] is creating a competing interest for the district in which we can’t compete . . . because we can’t sell the sugar and the [soft drinks] and those kinds of food items. So that’s a big inhibitor to full implementation to this policy. Until we get support from the city there is nothing the district can do. (KI, District B)
Key informants also expressed concern about highly processed or fast food they see school children eating:
I am concerned also that we have a great number of parents at our middle school who everyday bring their children these meals from the burger joints and stuff around town and while we are trying to follow nutritional guidelines with at least 90% of the kids. Here is one kid sitting at the table with his fast-food hamburger and fries and soft drink. (KI, District B).
Both the key informants and the students shared challenges of adhering to the “50/50 rule” where 50% of school-sponsored fund-raisers follow healthy food guidelines. Middle school student focus group members stated what foods they preferred:
Student 2: Yeah, I like bake sales at the school. Kind of like really tempting.
Facilitator: What kind of things do they serve at bake sales?
Student 1: Candy, pickles, soft drinks, chips.
Student 2: All the good stuff.
Student 3: Donuts.
Facilitator: Are those quite often at your school?
Multiple students: Every day.
Both middle and high school focus groups stated that cafeteria food was visually unappetizing, lacked variety and choice, and was not as appealing as food sold by competing vendors. High school focus group participants responded to questions about outside food vendors:
Student 4: We have like three vendors: the pizza guy, the burrito guy, and the Chinese guy.
Facilitator: And how often do you eat at the vendors?
Student 4: All week every week, through the whole year.
Facilitator: What appeals to you about eating at the vendors?
Student 3: It’s better than the cafeteria.
Other students: Yeah.
Finally, middle school focus group members in District A perceived that cafeteria food quality contributes to skipping meals:
Facilitator: So how many eat at lunch every day?
Students 3, 4, 5: I don’t eat everyday.
Facilitator: So you’re skipping meals sometimes?
Students: Yeah.
Facilitator: Is there any specific reason why?
Student 3, 4, 5: The food [is not appetizing]
Accountability Barrier
Even though both districts had adopted written wellness policies, key informant data showed a widespread lack of specific policy knowledge among superintendents, principals, wellness coordinators, and SHAC representatives, with little shared understanding of who was responsible for carrying out and enforcing school health and wellness policies:
I believe that there is not sufficient enforcement of [the wellness policy]. It’s kind of left up to each administrator, left up to almost down to the individual teacher. Nobody comes over and says, “Are you doing this, this, and this? Are you meeting this and that goal?” We don’t have set goals that are set in place that we need to meet. (KI, District A) Where my concern is the accountability . . . they say we building principals are in charge, but are we really? Who are we held accountable for? Are we making sure that [the 50/50 rule is being enforced] . . . are we really adhering to that? Who is really checking that? (KI, District A)
Recommendations to Improve Accountability
Key informants emphasized the need to make the wellness policy a “working document” (i.e., included and referred to frequently in school conversations) and improving communication between the SHAC and the rest of the school community. Common recommendations to improve implementation of physical activity and nutrition policies from both districts emphasized in-service training for school personnel and community outreach to parents. They recommended integration of physical activity in classrooms and in after-school programming. Recommendations to improve nutrition focused on cafeteria selections; some participants noted the importance of reviewing vending policies and practices.
Recommendations for Community Health Council Involvement
An important intended outcome of this study was to identify ways in which the community health council could help mobilize the community to improve implementation of school health and wellness policies. Key informants suggested the health council could facilitate communication and increase community awareness about school wellness policies, physical activity, nutrition, and obesity prevention. They also thought the health council could help seek funding for wellness-related initiatives. These recommendations indicate the acceptance of a school/community partnership, paving the way for future school/community collaboration.
Discussion
School systems are part of a larger ecological system and therefore subject to multiple social and political pressures and imperatives. Key informants, for example, referred to the political difficulties and costs of limiting off-campus vendors as significant barriers to reducing the impacts of competitive foods. Because schools do not operate in a vacuum, it may be very important to strengthen the school/community connection through the policy-to-practice continuum (i.e., policy adoption, implementation, evaluation, and modification of school wellness policies).
Key informants’ recommendations about the health council’s multilevel involvement reflected a social-ecological understanding of community systems. These recommendations are consistent with other studies citing the importance of family and community involvement (Food Research and Action Center, 2006; Hammerschmidt, Tackett, Golzynski, & Golzynski, 2011). Community groups (e.g., health councils) can influence nonstructural factors through facilitating communication and outreach to support policy implementation.
Champions or Advocates
The idea of a program champion has been identified in the organizational change literature as a key element in successful interventions (Belansky, Cutforth, et al., 2009; Frieden et al., 2010; Goldman, 2003; Pluye, Potvin, & Denis, 2004; Khatapoush, Thaker, Hallfors, Sánchez, & Steckler, 2004). Our data confirmed the importance of champions in each district. However, the efforts of school champions in our study were related more to physical activity than to nutrition. Our observation is that in some communities, it may be easier to find vocal champions of physical activity than of dietary changes because of the complex nature of community norms surrounding food choice and change. Food plays an important role in social and cultural interactions, as is evident in social gatherings and seasonal celebrations in the study community; it is often associated with appreciation and sharing.
Accountability
The importance of accountability and follow-through emerged as a key finding. Once adopted, school policies require systematic approaches to implementation, with clear lines of authority and responsibility. Although each school district had adopted wellness policies, including those specific to nutrition and physical activity, we found inconsistency in key informants’ understanding of who was responsible for implementing and monitoring these policies. Belansky et al. (2009) reported similar results in their study of federally mandated school wellness policy. Principals noted wellness policies are among many that are handed out but not discussed. As one principal in their study stated, there are “. . . no ‘teeth’ involved; nobody’s watching . . .” (Belansky, Cutforth, et al., 2009, p. S156). In sum, it appears that although school boards adopt policies, they do not always have the resources and guidance to ensure implementation.
Barriers and Facilitators
Our study identified factors at the family, organizational, and community levels that influence policy implementation, similar to other studies. For example, our findings about the availability of competitive foods (through fund-raising or outside vendors) are consistent with other studies (French, Story, Fulkerson, & Gerlach, 2003; Nollen et al., 2009). Research suggests that students have better diets relative to U.S. Department of Agriculture guidelines when competitive foods are either limited or not available (Larson & Story, 2010).
The facilitators and barriers to physical activity implementation were largely structural (e.g., No Child Left Behind, graduation requirements). Such barriers, outside the control of teachers and school administrators, are not unusual in U.S. public schools (Belansky, Cutforth, et al., 2009; Frieden et al., 2010; Hammerschmidt et al., 2011).
Unlike barriers, however, policy implementation successes were often under the control of administrators and teachers. For example, key informants reported compliance with the 50/50 rule, cooperation from parents when sending food for class parties, and the elimination of high-calorie snacks and soft drinks from school vending machines.
Limitations
The wellness policy environment is complex. In our study, we were not able to assess implementation successes or challenges for each policy component (e.g., competitive foods, physical activity in each school). In addition, our study included a small convenience sample of administrators and students who may not be representative of the entire school community. However, we countered the sampling limitation through dissemination, where our findings resonated with various stakeholders (teachers, parents, school nurses, and counselors).
Conclusions/Implications for Practice
These results demonstrate that multiple facilitating factors and barriers influence policy implementation in these two school districts. They also underscore the need to address accountability issues and raise questions about the possible influence of community norms on policy acceptance and implementation. Our findings also raise the larger question regarding the potential impacts of policy on social and cultural norms and practices, which, in turn, influence individual behaviors related to food and physical activity. Social and cultural facilitators and barriers to improving nutrition and physical activity are potentially fertile ground for future investigations. This study also highlights the importance of community stakeholders (i.e. students, teachers, administrators) community health councils or other community organizations as supportive partners for school districts in reducing childhood obesity.
Footnotes
Appendix
Acknowledgements
We acknowledge the members of the community health council research team: Adam Metcalf, Lacey Houdek, and the student intern, Darla Tenorio. We thank all the school personnel and students who shared their experience, perceptions, and recommendations with us.
The CDC2 collaboration with the schools is continuing through a new grant to develop Photovoice interventions with each district’s high school to further assess food and nutrition practices within the schools and in the community.
This research was funded by the Robert Wood Johnson Foundation Center for Health Policy at the University of New Mexico.
