Abstract
This article describes the design, implementation, and evaluative findings of the HEROES (Healthy, Energetic, Ready, Outstanding, Enthusiastic, Schools) Initiative, a school-based multilevel childhood obesity prevention intervention. Based on the Centers for Disease Control and Prevention’s recommended coordinated school health approach, the HEROES Initiative works to alleviate the burden of childhood obesity in Southern Indiana, Northwestern Kentucky, and Southeastern Illinois in the United States. Process evaluation was conducted with the 17 participating schools in spring 2012 based on interviews with school personnel and observation of the school environment. Findings showed that despite some variability, schools were generally able to implement the intervention with fidelity. School-level outcome evaluation was also based on observation of the school environment, and revealed that schools had implemented a number of new practices to encourage physical activity and healthy eating. Assessment of student-level outcomes was based on professionally collected physiological measurements and self-reported behavioral data collected over an 18-month period of time, last collected in spring 2012. Findings demonstrated that the HEROES Initiative has been successful in reducing the percentage of overweight children in participating schools and healthfully modifying their dietary, physical activity, and sedentary behaviors. Strategies that have facilitated success and challenges related to the intervention are discussed.
Childhood obesity has become a problem of epidemic proportions. Over the past 30 years, the rate of childhood obesity has tripled and continues to increase (Ogden & Carroll, 2010). In 2008, 19.6% of children age 6 to 11 years and 18.1% of adolescents age 12 to 19 years were obese (Ogden, Carroll, Curtin, Lamb, & Flegal, 2010). Decreasing childhood obesity is a health promotion imperative, as obese youth have an increased risk of high cholesterol and high blood pressure (Freedman, Zuguo, Srinivasan, Berenson, & Dietz, 2007), are at a greater risk for several physical and mental health concerns (Daniels et al., 2005; U.S. Department of Health and Human Services [USDHHS], 2001), and are more likely to become overweight or obese adults who suffer from health consequences such as heart disease, type 2 diabetes, and stroke (USDHHS, 2001). Schools have the potential to play a pivotal role in improving obesity-associated behaviors among youth (Centers for Disease Control and Prevention [CDC], 2012b; Lobstein, Baur, & Uauy, 2004). Although more research is needed on the utility of school-based childhood obesity prevention interventions (Kropski, Keckley, & Jensen, 2008), previous studies have found that programs that institute multicomponent changes within the school environment can be effective at decreasing obesity in students (Gonzales-Suarez, Worley, Grimmer-Somers, & Dones, 2009; Khambalia, Dickinson, Hardy, Gill, & Baur, 2012) and improving learning outcomes (Murray, Low, Hollis, Cross, & Davis, 2007; Rosas, Case, & Tholstrup, 2009). However, few published intervention studies explicitly use and evaluate strategies based on the coordinated school health (CSH) approach. Furthermore, interventions under investigation are rarely thoroughly described (Gonzales-Suarez et al., 2009; Khambalia et al., 2012), posing a challenge for health promotion practitioners who wish to replicate and disseminate successful program elements (Baranowski, Cullen, Nicklas, Thompson, & Baranowski, 2002; Kumanyika, Jeffery, Morabia, Ritenbaugh, & Antipatis, 2002; Waters et al., 2011). Filling these gaps is central to the goals of the present study.
The HEROES Initiative (Healthy, Energetic, Ready, Outstanding, Enthusiastic, Schools) is a grant-funded multilevel and multiple-year obesity prevention intervention in Southern Indiana, Northwestern Kentucky, and Southeastern Illinois, states with some of the highest obesity rates in the United States (CDC, 2012a). The HEROES Initiative aims to facilitate change within individual schools by implementing the CSH approach (CDC, 2013a) and helps schools decrease childhood obesity and increase healthy lifestyle habits among students, their families, and school staff. The HEROES Initiative has a particularly innovative approach given several unique facets of the intervention. It provides a supportive infrastructure for participating schools that involves local oversight, substantial funding, and prescribed implementation strategies that can be tailored based on schools’ individual needs and identities. The intervention uses the CSH approach, implemented at the school level rather than the more typical district level (CDC, 2012c), which encourages individual schools to take ownership over needs assessment, planning, and implementation of strategies for school change. Finally, the intervention includes a rigorous evaluation framework, with an annual cycle of assessment and related opportunities for program enhancement to best meet students’ needs and increase the likelihood of the intervention’s success.
This article describes the HEROES Initiative, including its conceptual model and evaluation framework, and provides an overview of key findings and lessons learned for future implementation. It is hoped that the HEROES Initiative can add to the emerging body of literature on the CSH model and that through the dissemination of the initiative’s program strategies and assessment methods, health promotion practitioners and researchers may be able to garner ideas or refine and strengthen current and future CSH efforts, create positive changes within schools, and increase the practice of healthy behaviors among students, their families, and school staff.
Conceptual Framework
The HEROES Initiative is intended to help schools increase a culture of wellness by implementing the CSH approach, a multipronged strategy to improve students’ health and learning that is recommended by the CDC (2013a). Given the connection between health and learning and the substantial time students spend at school, the CSH approach aims to focus attention on health and to coordinate wellness efforts within schools by enhancing collaboration among school personnel and community agencies (CDC, 2013a). Situated within the social-ecological framework, CSH addresses both individual and organizational factors that interdependently influence behavior change (McLeroy, Bibeau, Steckler, & Glanz, 1988). There are eight components of CSH. The HEROES Initiative focuses on the five that most closely relate to the prevention of childhood obesity: health education, physical education, nutrition services, health promotion for staff, and family and community involvement (CDC, 2013b). The other three CSH components—health services, counseling, psychological, and social services, and healthy and safe school environment—are not a part of the HEROES Initiative.
Program Description
The HEROES Initiative was originally conceptualized by a private foundation in Southern Indiana with the mission of improving health in the tri-state area it serves. After piloting the program in one school and determining that the program was feasible and implementation was preferred at the school rather than the district level, a request for proposals was released in 2008 that has since been available annually to schools in the tri-state region. Schools must commit to 3 years of involvement in HEROES and must intend to sustain changes made to the school environment once the funding period has ended. Through the fall of 2012, 31 schools have been accepted into HEROES, 13 of which have completed the 3-year funding period. HEROES schools include elementary, middle, and high schools; both private and public schools; and schools in rural and urban communities.
The HEROES Initiative provides support for data-based school-level interventions intended to increase opportunities for physical activity and healthy eating among students and staff, to integrate health and wellness education into the overall academic curriculum, to engage parents and community-based organizations in enhancing the healthfulness of the school environment, and to empower the school to develop and implement policies that support healthy lifestyles for students, their families, and the school staff. Financial support for HEROES schools consists of a stipend for a part-time school wellness coordinator, program materials, and upgrades for physical activity equipment. Technical support is also provided by the funding agency in the form of monthly meetings for wellness coordinators, trainings related to program materials, and individual consultations with experienced CSH professionals. Within each school, the HEROES Initiative is led by the school wellness coordinator and a school wellness committee, which includes various school and community stakeholders.
The HEROES Initiative, as well as CSH, is grounded in community-level partnerships with organizations such as local health-related agencies, hospitals, and county departments of health. The HEROES Initiative is also based on data-based planning, and for this reason, each school wellness committee is responsible for conducting a needs assessment in their first year of funding using the School Health Index, a self-assessment and planning tool for school health and wellness policies (CDC, 2013d) that has been found to provide useful data and serve as a catalyst for change within schools (Sherwood-Puzzello, Miller, Lohrmann, & Gregory, 2007; Staten, et al., 2005). Each committee then uses the results of the needs assessment to develop a 3-year CSH-based HEROES plan. Because the CDC only recommends CSH components and not specific programmatic activities, HEROES requires the implementation of specific strategies related to each component. Table 1 shows the nine domains central to HEROES, how they represent each CSH component, and the strategies schools are expected to implement. It should be noted that the HEROES domains that pertain to all of the five included CSH components are aligned with the CDC’s recommendations for CSH program implementation (CDC, 2013c).
HEROES Initiative CSH-Based Strategies
NOTE: HEROES = Healthy, Energetic, Ready, Outstanding, Enthusiastic, Schools; CSH = coordinated school health; SWAG = Student Wellness Awareness Group.
The HEROES Initiative logic model is provided in Figure I, which depicts the resources necessary to implement the program, activities that the schools perform, and the short- and long-term outcomes that are anticipated and measured at the school and student levels.
HEROES Initiative Logic Model
Evaluation Framework
The HEROES Initiative is evaluated through an academic–community partnership using a three-tiered approach that addresses key questions related to the implementation of the CSH framework, how schools change policies and practices, and the extent to which students change their nutrition and physical activity behaviors and show changes in body mass index. Evaluation strategies are described in more detail below.
Process Evaluation
Fidelity to the intervention is measured annually through site visits to each school to assess four of the nine HEROES domains: Administrative Involvement, HEROES Coordination, District and School Wellness Policy, and SWAG (Student Wellness Awareness Group). These domains represent required elements of HEROES implementation and are consistent with CSH expectations outlined by the CDC. During the site visits, trained researchers conduct key stakeholder interviews with the school wellness coordinator, a school administrator (most often the principal), and the cafeteria manager based on a semistructured interview protocol. Site visitors also observe various elements of the school environment and materials provided by the wellness coordinator. Schools can receive a 4 (exists and exceeds standard), 3 (exists and meets standard), 2 (exists but does not meet standard), or 1 (does not exist) on several indicators associated with each domain. Indicator scores are averaged for each domain. Following each annual round of site visits, schools are provided with reports detailing their implementation fidelity levels. The aim of these reports is to provide formative feedback that encourages improvements in ongoing implementation.
School-Level Outcome Evaluation
The school-level outcome evaluation facilitates the assessment of systemic changes to prevent or reduce obesity occurring in each of the schools involved in the HEROES Initiative. Data collected for this element of the evaluation are acquired through the same site visit process described above for the process evaluation. The domains of the framework relevant to the school-level outcome evaluation include physical education/activity, nutrition education, food service, staff wellness, and family and community involvement. These five domains describe healthy changes in practice that schools are expected to make while participating in HEROES.
Student-Level Outcome Evaluation
The student-level outcome evaluation assesses changes in student behaviors and biometric indicators through the collection of online survey data and a standardized physiological data collection protocol. School wellness coordinators, nurses, and nursing students from local colleges measure students’ weight and height in the fall and spring of each school year. All personnel involved in the physiological data collection are trained to ensure measurement consistency. Students are asked to remove heavy outer clothing, shoes, and any heavy accessories. Weight is measured using a digital scale that measures to the nearest tenth of a pound, and height is measured using a stadiometer. Body mass index is computed by dividing kilograms by squared meters.
Students in Grades 4 and higher are also asked to complete the Student Health Assessment Questionnaire (SHAQ), an online survey developed for the initiative, in the fall and spring of each year of the school’s participation in HEROES. Two slightly different versions of the SHAQ are used for the evaluation, a 54-item version for students in Grades 9 to 12 and a slightly shorter, 40-item version for students in Grades 4 to 8. The SHAQ includes questions related to dietary habits, physical activity, and sedentary behaviors of individual students and is based on the validated School Physical Activity and Nutrition questionnaire (Hoelscher, Day, Kelder, & Ward, 2003). Consumption of fruits and vegetables, candy, fast food, soft drinks, and other foods is assessed through 24-hour recall questions. Vigorous physical activity is measured by a question asking whether or not students exercised or participated in physical activity (that made their heart beat fast and made them breathe hard) for at least 30 minutes on the day preceding the survey, such as basketball, soccer, running, swimming laps, fast bicycling, fast dancing, or similar activities. This question was taken directly from the School Physical Activity and Nutrition Survey, which was based on federal guidelines (USDHHS, 2013), although response categories were modified from 1 week to 1 day in order to assist with recall. Students also report the amount of “screen time” they engaged in on the day prior to the survey, which includes watching TV and playing video games.
Results
Because the HEROES Initiative evaluation is an ongoing effort, results related to the most recent wave of comprehensive data collection (spring 2012) are presented here. Process- and school-level outcome evaluation results include data collected from 17 schools participating in the 3-year initiative during the 2011-2012 school year. These data were collected in March through May of 2012 and include a summary of implementation and outcomes for all 17 schools. Student-level outcome evaluation results include changes in student behaviors and body composition from baseline to the end of the second year of HEROES implementation for 12 schools that received funding in fall 2009 or fall 2010. Pre- and postintervention results are summarized for all 12 schools and span an 18-month period from the fall of the program’s initiation to the spring of the second year of implementation. The research was approved by the authors’ institutional review board.
Process Evaluation Findings
Data collected during the 2012 spring site visits were used to assess implementation fidelity of programs across all years of the intervention. For the purposes of the analyses, scores of 4 (exists and exceeds standard) were converted to 3 (exists and meets standard), in order to identify the percentage of indicators for which implementation standards were met for each school. Implementation fidelity of HEROES was determined by a minimum score of 2.5 across all indicators included within the process-oriented domains of the HEROES framework. Although a score of 3.0 indicates that the school’s practice meets the standards defined for a specific indicator, an average score of 2.5 across all items in a domain was considered sufficient for fidelity in any given year of the program. A threshold of 2.5 indicates that schools are meeting the standard on at least half of all indicators within a given domain. Moreover, because HEROES is implemented across multiple years, fidelity is defined as progress toward standards in each year of the grant. The SWAG domain was not included because it was introduced as a component of the initiative in the middle of the school year, which did not provide schools with the time needed for implementation. For the HEROES Coordination and Administrative Involvement domains, 15 of the 17 schools (88.2%) were implementing with fidelity. For the District and School Wellness Policies domain, 6 schools (35.3%) were implementing with fidelity. Of the three domains, the mean number of indicators for which each school obtained a 2.5 or more on was 2.12 with a range of 0-3.
School-Level Outcome Evaluation Findings
School-level outcome analyses were conducted in the same way as the process evaluation analyses. Twelve of the 17 schools (70.6%) got a 2.5 or more on the Physical Education/Activity domain, 15 (88.2%) for the Nutrition Education domain, 6 (35.3%) for the Food Service domain, 7 (38.9%) for the Staff Wellness domain, and 15 (88.2%) for the Family/Community Involvement domain. Of the five domains, the mean number of indicators for which schools obtained a 2.5 or more on was 3.24 with a range of 2 to 5.
Student-Level Outcome Evaluation Findings
Data from 3,385 students were included in the analysis. Students’ mean age was 10.6 years (SD = 2.7), 51.6% were male, and 41.0% of students were eligible for free or reduced lunch. The majority (86.3%) were White/Caucasian. Z-score calculations performed on the height and weight data collected during the fall of each school’s first year and the spring of their second year of HEROES showed that the percentage of students who were overweight after 18 months of the intervention was smaller than the percentage of overweight students at the beginning of the intervention (p = .006; see Table 2). In addition, a greater proportion of students were of normal weight post–18 months compared to baseline levels (p < .001).
Pre– and Post–18 Months Weight Status of Students Attending HEROES Schools
NOTE: HEROES = Healthy, Energetic, Ready, Outstanding, Enthusiastic, Schools.
As shown in Table 3, self-reported behavioral data, measured at baseline and in the spring of the second year of the HEROES intervention, showed positive changes in several nutrition behaviors. Z-score calculations revealed that more students reported eating one to three servings of vegetables on the day prior to the survey following 18 months of the intervention (p < .001). However, there was a 1.3% decrease in the proportion of students who reported eating vegetables four or more times over the same time period (p < .001). This corresponded with a 2.7% decrease in the percentage of students who reported they consumed no vegetables on the day prior to the survey (p < .001). Fruit consumption showed a similar pattern of change, with a 3.5% decrease in the percentage of students consuming no fruit (p < .001) and a 0.7% increase in the percentage who consumed fruit four or more times on the day prior to the survey (p = .004).
Pre– and Post–18 Months Dietary and Physical Activity Behaviors of Student Attending HEROES Schools
NOTE: HEROES = Healthy, Energetic, Ready, Outstanding, Enthusiastic, Schools.
Unhealthy eating behaviors showed positive improvements over the 18-month period as well. Specifically, 54.3% of students reported consuming no soda on the day prior to the survey following 18 months of HEROES compared to 51.3% at baseline (p < .001). In addition, 4.1% of students reported consuming soda four or more times following the intervention, compared to 5.3% at baseline (p < .001). Similarly, the percentage of students reporting consuming fast food two or fewer times during the week prior to the survey increased from 80.1% to 83.3% (p < .001). The proportion of students reporting consuming fast food six or more times decreased from 3.1% to 1.7% (p < .001).
Results for physical activity-related behaviors also showed consistent positive trends. There was a 2.0% increase in the proportion of students reporting engaging in vigorous physical activity on the day prior to the survey (p < .001). In addition, 2.0% fewer students reported engaging in 3 to 6 hours of screen time post–18 months compared to baseline (p < .001). However, 1.2% more students reported more than 6 hours of screen time after 18 months of HEROES compared to baseline (p < .001).
Discussion
Process- and school-level evaluation results showed considerable variability between schools and domains assessed. The majority of the three process evaluation domains were well implemented, although schools struggled most with creating new or enhanced wellness policies. For the school-level outcomes, in general schools were able to effectively create changes to the school environment regarding physical education/activity, nutrition education, and family and community involvement. Schools had more of a challenge with modifying their food service and providing staff wellness opportunities. Future efforts should examine possible differences in schools or domain characteristics that may be responsible for the disparities identified.
Student-level outcome data showed small but significant changes in the proportion of overweight students between baseline and the first 18 months of the intervention. Specifically, fewer students were overweight and more students were of normal weight during the spring of the second year of the intervention compared to baseline. However, no changes were observed in the proportion of students who were obese. The lack of changes observed related to obese rather than overweight students suggests that this primary prevention strategy may be more effective for students who are not yet obese or only slightly overweight. For those students who are already obese, more targeted secondary prevention strategies may be required.
Behavioral data measured at baseline and in the spring of the second year of the HEROES intervention showed small but positive changes in both healthy (consumption of fruits and vegetables) and unhealthy eating behaviors (consumption of fast food and soda). More students also reported engaging in vigorous physical activity following the intervention compared to baseline. These changes in physical activity levels were also accompanied by similarly positive changes in screen time. Whereas systematic reviews and meta-analyses of school-based obesity prevention controlled trials have generally had inconsistent findings, interventions that combine nutrition and physical activity components (Brown & Summerbell, 2009; Gonzales-Suarez et al., 2009; Katz, O’Connell, Njike, Yeh, & Nawaz, 2008; Khambalia et al., 2012), in addition to family components (Katz et al., 2008; Khambalia et al., 2012), and that are implemented for more than 1 year (Gonzales-Suarez et al., 2009; Khambalia et al., 2012) have had the highest likelihood of success for weight reduction. These best practices are integral to the HEROES Initiative, in addition to the other CSH components that serve as the program’s foundation.
Although many of the findings reported may appear modest, small changes can make an important impact on population-level health outcomes (Johnson, Kremer, Swinburn, & de Silva-Sanigorski, 2012; Rose & Day, 1990). Furthermore, overweight and obese children are likely to maintain their weight status into adulthood (Freedman et al., 2005; Serdula et al., 1993), and the dietary behaviors (Craigie, Lake, Kelly, Adamson, & Mathers, 2011; Mikkila, Rasanen, Raiakari, Pietinen, & Viikari, 2004) and physical activity (Craigie et al., 2011) that young people engage in have been shown to set the stage for the behaviors they practice in adulthood. Given that inadequate physical activity and unhealthy dietary behaviors in adulthood are associated with obesity and related health consequences (USDHHS, 2001), changes made in childhood may have even more substantial long-term implications for health.
The HEROES program has not been immune from the challenges that come with implementing a large-scale initiative. Staff turnover within each school, particularly when it involved champions of the program or required key staff to take on additional responsibilities, caused some difficulties. Some food service staff members prioritized preparation for the new U.S. Department of Agriculture standards rather than even more expansive modifications to foods served required by HEROES. Given the costs associated with potential food waste, there was some resistance to altering cafeteria choices.
Although one of the strengths of the initiative is its ability to evolve with each round of data collection, this has also posed a challenge for consistent data collection efforts intended to compare each evaluation cycle to the next. Furthermore, although behaviors that occur during the school day are integral to children’s overall well-being, the school environment is only one part of children’s multifaceted lives. School-based programs have even more potential for success when they are part of larger scale interventions that involve other realms of children’s environments, such as their households, communities, and media. Consistent messaging that reinforces what children learn and do in diverse venues should further the effects of school-based programs.
Interpretation of these findings should consider the limitations to the data collection strategies employed. Although a rigorous evaluation framework was implemented, an experimental design was not used. Although the physiological data collection was done by trained professionals or paraprofessionals, the behavioral data were self-reported by students. These data were therefore subject to potential social desirability and recall issues. Furthermore, although the evaluation team obtained implementation information from various sources at each school to assist with the reliability of scoring, it is possible that interviewees may have had a positive response bias out of the desire impress the evaluation team or other school personnel. Also, although an 18-month follow up period certainly provides initial longitudinal data, additional future assessment would be helpful in determining if the changes identified are sustained for a longer term (Shaya, Flores, Gbarayor, & Wang, 2008). Given the current number of HEROES schools, the association between school-level changes and individual outcomes could not be assessed. This is worthy of future research.
The evaluation findings of the HEROES Initiative provide additional evidence for the CSH model. The results also provide some useful tools for practitioners who plan to develop CSH-based interventions. Programmatic strategies that coincide with each relevant CSH component are provided that can be replicated (e.g., taste testing and walking clubs). Understanding the CSH components that have been implemented with more or less ease can inform intervention expectations and areas that may need more emphasis during intervention planning and oversight. In addition, these results can help school-based practitioners prioritize limited resources to implement those CSH components that have been shown to be more successfully put in place. Despite challenges to both the implementation and evaluation of the initiative, HEROES has made an important public health impact. Through the provision of an empowering infrastructure and ongoing technical assistance at the school level, HEROES’ students and staff have taken ownership of innovative and data-driven programming that is effectively improving children’s health and well-being.
Footnotes
This work was supported by grants from the Welborn Baptist Foundation (WBF; Grant Nos. 4440311, 4443010, and 4443011).
Authors’ Note:
The content is solely the responsibility of the authors and does not necessarily represent the official views of the WBF. The following authors received support from these grants: Mindy H. King, Alyssa M. Lederer, Danielle Sovinski, Heidi M. Knoblock, and Dong-Chul Seo. Rhonda K. Meade is also an employee of the WBF.