Abstract
This article presents results from a mixed-method evaluation of a structured cooking and gardening program in Australian primary schools, focusing on program impacts on the social and learning environment of the school. In particular, we address the Stephanie Alexander Kitchen Garden Program objective of providing a pleasurable experience that has a positive impact on student engagement, social connections, and confidence within and beyond the school gates. Primary evidence for the research question came from qualitative data collected from students, parents, teachers, volunteers, school principals, and specialist staff through interviews, focus groups, and participant observations. This was supported by analyses of quantitative data on child quality of life, cooperative behaviors, teacher perceptions of the school environment, and school-level educational outcome and absenteeism data. Results showed that some of the program attributes valued most highly by study participants included increased student engagement and confidence, opportunities for experiential and integrated learning, teamwork, building social skills, and connections and links between schools and their communities. In this analysis, quantitative findings failed to support findings from the primary analysis. Limitations as well as benefits of a mixed-methods approach to evaluation of complex community interventions are discussed.
School gardening and garden-based nutrition programs have the potential to improve children’s nutritional and health status as well as have a positive impact on social and environmental behaviors. Despite the increasing popularity of community and school edible gardens, several researchers have noted the limited evidence base assessing their impacts and outcomes (Lautenschlager & Smith, 2007; Murphy, 2003; Ozer, 2007; Phibbs & Relf, 2005; Robinson-O’Brien, Story, & Heim, 2009; Skelly & Bradley, 2000). Most school garden research literature focuses on evaluation of short-term interventions, often at a single site, and evidence for the effects of school-based gardening programs integrated with a well-developed cooking component is lacking in the published literature.
Studies that do exist have examined a wide variety of outcomes for gardening interventions, with fewer studies reporting on the outcomes of school-based cooking programs. A recently published review of school gardening programs documented evidence that such programs had positive effects in the areas of science achievement and food behavior and qualitative evidence of positive effects on social and environmental behaviors (Robinson-O’Brien et al., 2009). Studies and reviews of garden-based nutrition interventions have stressed the limited evidence base for their effectiveness but suggested that they had the potential to increase willingness to taste and consume fruits and vegetables among young people (Ratcliffe, Merrigan, Rogers, & Goldberg, 2011; Robinson-O’Brien et al., 2009). Children’s perceptions of self-efficacy in cooking skills were found to increase through participation in nutrition education programs that incorporated food preparation and cooking, such as The Cookshop Program (Liquori, Koch, Contento, & Castle, 1998) and The Tooty Fruity Vegie Project (Newell et al., 2004).
Studies that consider the social impacts of gardening and nutrition interventions have shown that the tangible rewards as well as interactive and hands-on experiences of cooking and gardening programs are both engaging and enjoyable for participants (Alexander, North, & Hendren, 1995; Heim, Stang, & Ireland, 2009; Morris, Koumjian, Briggs, & Zidenberg-Cherr, 2002; Newell et al., 2004). The capacity of community gardens to increase psychosocial well-being has been noted by several researchers. In a survey of 13 school garden programs in Queensland, Australia, Somerset, Ball, Flett, and Rebecca (2005) commented on the consistent response of participating teachers who indicated the positive influence of school gardens in enhancing psychosocial well-being, specifically with regards to self-confidence and esteem (see also Murphy, 2003). Studies of gardening programs have also pointed to the potential for positive impacts on children’s self-esteem through improvements in “life-skills” (Robinson & Zajicek, 2005) and on the behavior of disruptive students (Somerset et al., 2005). The usefulness of gardens as multicurricular learning tools has been discussed, and Klemmer, Waliczek, Zajicek (2005) reported a significant increase in the science test scores of fifth-grade students participating in a year-long school garden program, providing evidence that school garden programs may have the capacity to improve academic performance. Gardening programs have also been found to promote connections between children and adults and between schools and communities (Morris, Neustadter, & Zidenberg-Cherr, 2001; Waliczek, Lineberger, Zajicek, & Bradley, 2000; Walters, 2008) as well as encourage interest in gardening among families of participating children (Alexander et al., 1995).
Recent reviews of this literature have concluded that although results for the effects of school gardening programs are promising, there remains a lack of rigorous peer-reviewed research providing sufficient evidence for their outcomes (Ozer, 2007; Robinson-O’Brien et al., 2009). One of the aims of the investigation reported here was to address this need for evaluative research.
The Stephanie Alexander Kitchen Garden Program
The Stephanie Alexander Kitchen Garden (SAKG) Program was initiated at a single pilot school in 2001 by Australian chef, restaurateur, and food writer Stephanie Alexander. With the support of the Victorian State Government, it was introduced into a further 46 primary schools across the state from 2005 to 2010 and is now being expanded nationally, reaching 180 schools across all Australian states and territories in 2011. The SAKG Program provides a seed to table experience, offering primary school children between the ages of 8 and 12 years, the opportunity to plant, nurture, harvest, prepare, and share fresh, nutritious, and seasonal food. The program model consists of all children in Grades 3 to 6 (aged 8-12 years) participating, at a minimum, in a 45-minute garden class and a 1.5-hour kitchen class every week as an ongoing part of the school curriculum. Children keep kitchen garden program journals in which they write about program activities, record new vocabulary, and write up related exercises. These might include science experiments performed or math activities, such as measuring and graphing plant growth. Garden and kitchen classes are planned and supervised by specialist staff assisted by the classroom teacher and adult volunteers. In most, although not all cases, specialist staff are employed who have prior qualifications and experience in horticulture and hospitality, respectively. Emphasis in garden classes is on learning and practicing organic methods of food production. Children are actively involved in all aspects, from garden design, preparing beds, planting seeds, transplanting seedlings, nurturing the growing plants (including weeding, watering, fertilizing with homemade compost and “worm juice,” and applying organic pest control), through to harvesting the yield. In the kitchen, children work in small groups to prepare sophisticated, multi-course meals based on seasonal produce from the garden, which they then sit down to share with staff and volunteers. A typical meal might start with homemade bread topped with herbs, a leafy green salad, or vegetable soup followed by a curry or homemade pasta with a broccoli sauce and finish with glazed fruit or an orange and cardamom cake. The SAKG Program aims to increase children’s appreciation and enjoyment of a wide range of fresh, seasonal foods and dishes while developing skills, knowledge, and confidence in the kitchen and garden. It also aims to develop understanding of environmental sustainability and create an enjoyable learning environment that promotes self-esteem and strengthens positive relationships with peers and adults.
Although core elements of the program are set out in a program manual, within this model, schools are able to adapt weekly program activities to suit their particular social and physical environments and constraints as well as to respond to local events and other aspects of the school curriculum. In practice, this results in some diversity between schools when it comes to program delivery.
The Evaluation
The authors were contracted to conduct an independent evaluation that comprised a quasi-experimental mixed-methods study of the processes, impacts, costs, and outcomes of the program. The evaluation design is a nonrandomized comparison of six schools receiving the program (intervention) and six control schools. All are government-funded primary schools in urban and rural areas of Victoria, Australia. A full description of the SAKG Program and a report summarizing the evaluation methods and key outcomes of the program related to children’s increased appreciation of diverse foods, knowledge, and confidence in the kitchen and garden, and determination of the feasibility, acceptability, costs, and sustainability of the program is available online (Block et al., 2009). This article will focus on the impact of the kitchen garden program on the broader social and learning environment of the school.
Although the program had been developed without specific reference to a theoretical model, the research team identified multiple aligned theoretical frameworks of relevance. The evaluation draws primarily on principles of ecological theory, which recognizes the interdependence between individuals and their physical and social environment (Bronfenbrenner, 1979; Hawe & Riley, 2005; McLaren & Hawe, 2005; Ozer, 2007) and principles of effective health promotion, which require the combined elements of policy, environment, community, personal, and service elements to be addressed (World Health Organization, 1986). The evaluation was also informed by models of experiential learning that emphasize the importance of hands-on learning activities (Parmer, Salisbury-Glennon, Shannon, & Struempler, 2009). A social–ecological conceptual model of the potential effects of school gardens, developed by Ozer (2007), provides a rationale for examining the effects of school gardens on multiple domains for individuals, schools, families, and communities, and distinguishes between proximal and distal effects of different program components (Figure 1). This conceptualization is useful for the analysis presented here of the social impacts of the program, many of which can be described as distal outcomes and which are also affected by a wide range of factors beyond the influence of the program. This point will be revisited in the Discussion section of this article, where the limitations as well as benefits of a mixed-methods approach to evaluation of complex community interventions are addressed.

Conceptual model of potential effects of school garden programs
Method
Baseline and follow-up quantitative data were collected over the period 2007 to 2009. At the beginning of the evaluation period, 20 schools had received state government funding to implement the SAKG Program and were therefore eligible for inclusion. These were stratified according to geographic location (metropolitan or rural), socioeconomic status of families attending school (less than 35% of families or more than 35% of families eligible for additional government support), and size (student numbers in Grades 3 to 6; low ≤99, medium 100-199, high ≥200). Six schools were then randomly selected from these groups to provide a range of these characteristics and the principals of all six agreed to participate. Six comparison schools, matched on the criteria of location, socioeconomic status, and size, were recruited from a pool of 16 identified as suitable by the education department. To allow for possible difficulties in recruiting comparison schools, two to three matched comparison schools were identified for each program school. Of these, four declined, two were not able to give a timely response, six accepted, and four were not contacted because another school in their category had accepted.
Local adaptations by schools of the program model resulted in a range of exposure to program activities from 1.25 hours to 2.5 hours per week (mean 1.8 hours) for each student. Because of a number of factors, including progressive release of evaluation funding, the period between baseline and follow-up data collection also varied by school with a range of 12 to 25 months. The total exposure to the program captured by the evaluation is summarized in Table 1.
Exposure to Program Activities Captured in the Evaluation
Apart from the principal interviews, which were conducted at all 12 schools, qualitative data were collected at the 6 program schools only. Ethical approval for the evaluation was provided by relevant university ethics committees and the Department of Education, Victoria. All staff and families of children from these schools (in Grades 3 to 5 at baseline) were sent information letters and consent forms via the school.
Qualitative Data
Qualitative data were collected through focus groups, individual interviews, and participant observation. Focus groups were conducted toward the end of the 2½-year evaluation period. All classroom teachers of participating year levels were invited to attend the focus groups via the school principal or administration staff. Parents and volunteers were invited to attend focus group discussions through notices placed in the school newsletter and letters distributed by the school. Teachers were asked to select children who already had parental consent to participate and who would represent a range of year levels and experiences of the program. A total of 26 teachers, 17 volunteers (9 of whom were also parents), and 20 other parents participated in focus group discussions at 4 program schools. Two mixed-gender child focus group discussions (one each for children in Grades 3 or 4 and Grades 5 or 6, respectively) were held at each of the 6 program schools, and 124 children (an average of 10 per group) participated in these. School principals at all the participating program and comparison schools were interviewed at the time of baseline and follow-up quantitative data collection, and kitchen and garden specialist staff were interviewed at the time of follow-up data collection at each of the program schools. These semistructured focus group discussions and interviews focused on participants’ expectations and experiences of the program, changes in the school and home environment, highlights, and areas for potential improvement, with findings relevant to the program’s impact on the school social and learning environment reported here. Focus group and interview transcripts were coded and analyzed using QSR NVivo8 computer software as a data management tool. Inductive thematic content analysis was performed in which data were coded to identify emerging themes and patterns that were then categorized and interpreted according to their relationship to research questions and theoretical perspectives.
Participant observation was conducted by one of the authors (CL) at four of the program schools at three time points over the course of the evaluation. Detailed field notes consisted of observations and description of children’s attitudes, behaviors, interactions, conversations, and changes to all these over time. Analysis included the researcher’s reflections on the meanings and explanations for what had been observed and were also informed by discussions with relevant adults (specialist staff, teachers, volunteers, and principals).
Quantitative Data
Of the 1,474 participants invited to participate, 764 children (475 program, 289 comparison) and 562 of their parents (326 program, 236 comparison) completed questionnaires at baseline. Of those with completed baseline data in the program group, 75.8% of children and 81.3% of parents participated in follow-up data collection. For the comparison group, 87.2% of children and 68.2% of parents participated in follow-up data collection. The final quantitative sample analyzed consisted of 592 children (352 program, 240 comparison) and 316 parents (186 program, 130 comparison). The duration between baseline and follow-up data collection was longer for program schools than for comparison schools (median 15 months vs. 10 months) because of delays experienced in recruiting the latter. The differences in duration were adjusted for in the analyses.
Several quantitative measures included in the study had the potential to capture program impacts on the social and learning environment of the school. Child well-being was assessed using the KIDSCREEN-10 measure (KIDSCREEN Group Europe, 2006). Child cooperative behaviors were reported by children, parents, and teachers using a modified section of Robinson and Zajicek’s (2005) scale. This measure was based on five items that used a 5-point Likert-type scale with responses ranging from never to always, totaling a maximum score of 20. Teacher questionnaires used a 4-point scale to measure responses (ranging from very negative/strongly disagree to very positive/strongly agree) to multiple statements concerning student attitudes and the school social and learning environment. Outcomes were compared between program and comparison groups using random effects linear regression models (Goldstein, 2005) or generalized estimating equations (GEE; Hanley, Negassa, Edwardes, & Forrester, 2003). Analyses were adjusted for baseline, school clustering, and child grade (for child and parent data). All analyses were carried out using Stata 10.1 software (StataCorp, 2000). All schools were asked to provide school-level data on absenteeism and academic achievement in literacy and numeracy.
Results
Student Engagement and Confidence
Increased student engagement was described as an outcome of the SAKG Program by all groups of interviewees. Children themselves frequently described how much they looked forward to “kitchen and garden days” and how school had changed since the program had been introduced. When asked what they liked about it, typical replies included “We love it!,” “Fun!,” “Awesome!,” “Exciting!,” “School’s not boring,” “Brilliant!,” “Educational,” and “You don’t want to be away!”
In addition to expressing general enthusiasm, children were sometimes more explicit about ways in which the program had increased their confidence at school. Learning to use “real” chef’s knives, for example, when preparing food, was particularly valued by many children as a symbol of their own capability and the trust being placed in them by adults and was contrasted with what they had previously been entrusted to do at home. At other times, their growing confidence was implied by the obvious pride and self-esteem with which they referred to their new skills, knowledge, and accomplishments in the kitchen and garden.
Participant observation also provided insights into the ways in which children’s confidence developed and was expressed in the kitchen and garden settings. During garden classes, with few exceptions, groups of children were able to self-direct, remain engaged in, and complete their allocated tasks without close supervision. Similarly, in the kitchen, children demonstrated their knowledge of the routine and would cooperatively complete preparation and cleaning tasks without instruction. Confidence was reinforced through a sense of achievement. Self-esteem was enhanced by student artwork and writing on display in and around the kitchens and gardens in all six program schools.
Teachers, parents, and volunteers all remarked on the growth in confidence, independence, and self-esteem they had witnessed in the children since they had been participating in the program. Watching the children safely wield “those big knives” as well as prepare a multicourse meal in a little over an hour had caused some adults to revise their assumptions about what the children were capable of achieving. Newfound confidence in other parts of children’s lives as well as a willingness to try new things was also attributed to participation in the kitchen garden program. One staff member stated that “the children have been empowered” and that “becoming more open to new foods should—must translate into other areas of life.” A parent described the benefit for her son as follows: “It has widened his world in more ways than one. That would be the best way I could explain it.”
Although kitchen activities were almost universally popular, reactions by children to garden classes were far more variable. Whereas for some children they were the best part of the program, others were much less enthusiastic. Teachers and volunteers too noted that some children were clearly more engrossed in garden classes than others. There appeared to be a positive relationship between the level of horticultural qualifications and expertise of the garden specialist and differing levels of engagement of the children. This view was supported by volunteer focus group comments suggesting that children’s observed level of engagement varied according to the sophistication of the content and concepts they were being taught in the garden. Reasons given by some children for not liking the garden were that it was “boring,” unpleasant when too hot or too cold, and that some children did not like to get dirty. As one child explained, “If you get dirty, you’ve got no clothes to change into and parents yell at you!”
On the other hand, some children expressed a great sense of achievement from their “hard work” in the garden. Children spoke of their satisfaction at having created a garden from “bare earth” at one school and from a “mud pile” at another. For some, the garden was also a special place, which made the school more “attractive and colorful.” It engendered a sense of ownership and connection to the school. One child announced proudly that, “it was my idea to put flowers around the shed.” For another, it functioned as a peaceful refuge: “If you want a little alone time . . . you can go in there and sit down.”
These qualitative findings suggesting a positive program impact on student engagement and confidence were not, however, reflected in the quantitative results. Child well-being assessed through self-reported health-related quality of life demonstrated no statistically significant difference between mean scores at program and comparison schools (p = .09; Table 2). Reported absenteeism also showed little change. Results from teacher questionnaires showed that the proportion of teachers who strongly agreed with statements, “students look forward to coming to school” and “student social behavior in this school is good” increased between baseline and follow-up measurement in both program and comparison schools (Table 2). When adjusted for baseline and clustering, there was little evidence of a true difference between program and comparison schools.
Results Related to School Social and Learning Environment
Note. CI = confidence interval; ICC = intraclass correlation coefficient; SD = standard deviation. Analysis adjusted for baseline and school clustering; where ICC = 0, not adjusted for school clustering. Child sample size ranged from 328 to 342 for Program and from 210 to 335 for Comparison. Parent sample size: 184 for Program and 125 for Comparison. Teacher sample size ranges from 26 to 28 for Program and is 24 for Comparison.
Also adjusts for grade. ADDIN
Impact on School Learning Environment
Although the SAKG Program was described as engaging students from across the academic spectrum, it was seen as particularly significant and valuable for connecting and integrating children potentially at risk of long-term disengagement. Children described as “nonacademic” or exhibiting “learning difficulties” and “challenging” behaviors were experiencing “success” at school with the SAKG Program, perhaps for the first time. Teachers would often harness this: choosing a “challenged” child to explain the knife safety rules to a new volunteer for example or having a rotating “head chef” take on a leadership role for each kitchen session. In addition, kitchen and garden tasks were tailored for children with specific disorders, such as autistic spectrum to provide them with experiences of inclusion and success. Classroom teachers at most schools described ways in which the whole experience of education had been transformed for individual children with complex needs. At one school, teachers spoke of a boy with poor social and academic skills, whom they had previously worried about as one who might “be lost.” Now he wanted to be a chef, and they were reassured by his new sense of purpose and engagement at school. A teacher at another school described,
One kid in particular who’s a bit of a social misfit is the “go to girl” in the kitchen because she can cut up the onions without tears . . . There’s another one who’s got a low academic base but she’s got fantastic facility in the kitchen . . . A child who struggled and had learning disabilities . . . her confidence and her ability to outshine other kids, who have strengths in other areas was just amazing . . . She knew exactly what she was doing, she was in control, she was [a star] while she was organising the other kids. The building of confidence was just amazing.
During one observation, a teacher pointed out two children in conversation as they washed dishes, noting that despite significant language disorders, both were completing the required task and communicating with each other. She described another girl who was cutting up pumpkin, and as engaged as any other class member, as being unable to stay on task for more than 3 minutes in the classroom. She noted that being in the kitchen class was “a big motivator for [another child] not to punch, not to throw things . . .” All the identified children were participating in a way that rendered them indistinguishable from those around them.
Participants in several of the parent focus groups credited the kitchen garden program with having such positive effects because it provided “a level playing field,” which lacked the competitive structure of academic and sporting activities and on which all children could participate as equal members of a team. Perhaps even more important, it incorporated an active, “hands-on” way of learning. Children were learning “practical life skills” while assimilating numeracy and literacy. Children learned fractions by measuring ingredients and cutting up fruit into portions. They were also keen to write about the program because they enjoyed it, and science was observed to be present in almost all activities, with described examples including learning about seed life cycles, nitrogen fixation, the role of insects, and how to tell whether an egg is fresh. Some parents and teachers saw this experiential learning as particularly useful for boys and it was also felt to be clearly effective at engaging “kids who don’t shine in academia.” Having volunteers present also meant that children could receive more individual attention than in the classroom as suggested here by one of the classroom teachers:
The children that struggle with concepts of maths and weight and measurement respond far better to a hands-on approach. And look, you just can’t have that all of the time in the classroom because it’s so hard to set up and you just don’t have the ratio of teachers to students.
A number of teachers reported having preliminary concerns that the program would interfere with teaching the core curriculum and add to their workload. Some parents also recalled that they held initial reservations about the program, feeling that it was being added to an already crowded curriculum and that they would have preferred schools to focus on core areas, such as literacy and numeracy. In almost all cases, at least among focus group participants, these concerns were allayed once the program was established, largely because kitchen and garden classes functioned as a natural site for curriculum integration. The children themselves were very conscious of how much they had learned while participating in the program as well as of the practical skills they had mastered. Children gave numerous examples of activities that had been incorporated into the SAKG Program: art in the garden and kitchen; information and communications technology, including making PowerPoint presentations and contributing to school websites; numerous science experiments; learning about the environment, sustainability, nutrition, health and hygiene, geography, and cultural diversity; and public speaking, where at one school a child would prepare a short speech to welcome visitors and thank the volunteers in the kitchen class each week. Children also showed us their kitchen garden journals, workbooks in which they recorded weekly descriptions of garden and kitchen class activities, such as measuring and graphing plant growth as well as practiced a variety of writing genres to capture their experiences. As one teacher enthused,
A 2-hour cooking session involves maths, social interaction, and oral development. It’s learning. It’s high-level learning and it’s in small group situations. If I had kids at school here I’d be absolutely rapt [thrilled]. I’d be ecstatic to see the learning experience they get and they get to eat something good at the end.
At one culturally diverse school, teachers felt that kitchen classes functioned as a natural environment in which to discuss and incorporate diversity. The tables were laid with chopsticks as well as knives and forks and children of all backgrounds enjoyed learning to use them as well as talking about how other cultures might traditionally eat with their hands. In kitchen classes they had also learned to consider issues such as religious dietary restrictions, celiac disease, allergies, and vegetarianism. For the many children at this school for whom English is a second language, the opportunities for social interaction were seen as particularly beneficial for oral language development.
The SAKG Program was described by several interviewees as creating a “learning community” that benefited other stakeholder groups as well as the children. Teachers, parents, and volunteers were also learning new skills, with some “confessing” they could now recognize and cook with fresh herbs for the first time. One school in an economically disadvantaged area with high unemployment had formalized the process of volunteer learning in partnership with a local adult-learning center. Hours spent volunteering at the school were formally recognized as training, and additional short courses offered at the school allowed participants to gain trade certificates in areas, such as food handling and first aid.
Focus group participants also drew attention to the knowledge that some volunteers brought to the school and imparted both to the children and to the kitchen and garden staff. One volunteer had taught the kitchen specialist much about preserving. Another remarked,
My mother-in-law helps in the kitchen and the older people too have a wealth of information about things we don’t cook anymore and I think everyone learns from them as well, which is a good thing.
It was observed that these features of the program offered a model of “lifelong learning” to the children, best summed up by one focus group participant as: “We all learn from one another, we work together, we share it all.”
Again however, quantitative results did not support qualitative evidence of changes in the learning environment. Although the percentage of teachers agreeing that “student attitude toward academic achievement is very positive” increased between baseline and follow-up at program schools and declined at comparison schools, there was little evidence of a true difference between the groups when adjusted for baseline and clustering (Table 2). All six of the program schools provided academic achievement scores for literacy and numeracy at baseline while five of the comparison schools provided data at baseline, and three provided data at follow-up. All scores at both baseline and follow-up were close to the Victorian state average.
Social Connections: Connecting Students, Teachers, School, and Communities
The value of working in groups was frequently mentioned as an important aspect of the SAKG Program. Children not only found it fun but also felt that they were learning and improving when it came to this valuable skill and expressed awareness that they were expanding their social networks beyond their immediate friendship groups. A teacher described the program as
Excellent for cooperation: Boys, girls, adults, friends are all engaged in the same task and work as a team to achieve it. A three course meal in one and a half hours!
Teachers appreciated their involvement in kitchen and garden classes, which are run by the specialist staff, as positive time spent with students. They enjoyed having less need to focus on discipline as well as learning together with the children and felt that this enhanced student–teacher relationships. Teachers and volunteers also noted and welcomed children’s newly acquired table manners:
Social skills and eating etiquette, to sit at a table, to learn how to set a table, it sounds silly doesn’t it? You don’t start until everyone is seated and you pass the food around. I think that’s a really important part . . . But remember when we used to go on camp; it would send you insane because they had no table skills . . .
Gardens had added considerably to the aesthetic appeal of school grounds, often enhanced with children’s artwork, such as mosaics and ornamental paving. They were seen as places of beauty for both play and quiet contemplation. The program was often reported to have raised a school’s profile and local reputation, giving it a unique and special local identity that attracted new families and many interested visitors to the school.
The SAKG Program was seen as highly effective at connecting schools with parents, grandparents, and the wider community. Although the need to fund-raise and attract volunteers to run the program was often described as a challenge, these activities were also responsible for developing many connections. Schools received donations of money, goods, and services from local businesses, farmers, and tradespeople as well as community groups who might, for example, come in and build a new chicken shed. Local families would also come in to check and water the garden over the school holidays.
For parents, volunteering, fund-raising, and working to establish gardens were opportunities to be involved in their children’s school and education beyond the early years (when parents often come in to help in classroom literacy programs). Parents from non-English speaking backgrounds, typically uncomfortable in classroom literacy programs, also volunteered in the kitchen and garden. This was highly valued by the school principal and teachers as an opportunity to engage this group of parents and by the parents as an occasion for cultural sharing. Parents also attributed to the program increased connections with each other, describing it as a good “conversation starter” at the school gate.
Grandparents too were motivated to volunteer in the program and, as already noted, often brought valuable knowledge and skills, while enjoying the opportunity to be involved and contribute to their grandchildren’s education and the community. A parent explained how in her family, the program had enhanced intergenerational communication:
[It closes] the generation gap. It is something they can talk about to grandma which has no age level and they can talk about gardening and they can talk about cooking. It’s the basis for conversation all of the time. Instead of just saying, “How is school going?”—especially with boys who won’t talk about school—but talking about cooking and gardening, the conversation just opens up, it flows and I like that. It is really good for the generations.
Many volunteers had no family connection or other prior connection to the school. This included local high school and university students, employees in local businesses and others, such as retirees with an interest in the program or a desire to help children. Children were appreciative of their contact with volunteers, describing their interactions as different from those with teachers. Several children commented that one of the big differences that the program had made to their school was that there were now lots of visitors. “Visitors” referred not only to volunteers but also to local media and other community members with an interest in observing the changes in the school and seeing the program in action. Children referred to the influx of visitors as a positive, both evoking pride in their school and adding interest to their time there. One child remarked,
The people from the Day Centre—that’s the old people—they came to have a look at the garden and they thought it looked nice . . . It’s also good for people just driving by because they always have a look and it always looks nice for them. It was once just a big dead grass patch and it didn’t look very good.
Sadly, some schools had suffered a degree of vandalism and theft affecting gardens and one kitchen. However, these incidents were also noted somewhat paradoxically to have led to a subsequent increase in community support. At one school, a teacher felt that local residents now monitored and “looked out for” the school and its children, while at another, the community had rallied to rebuild the garden and local businesses had donated new trees and plants. Many of the schools taking part in the evaluation had also entered food, plants, and artwork in local shows. Where this was the case, children frequently referred with pride to prizes that had been won by themselves or their peers for items such as pickles, relishes, decorated cupcakes, vegetables, and garden photography.
Links between schools in the same region were seen as both potentially and actually occurring because of the kitchen garden program. An unanticipated outcome of the program was the interest taken in it by teachers from local secondary school food technology classes. Having realized that they needed to adapt secondary school cooking programs to the more advanced knowledge and skills of children coming through the SAKG Program, these teachers were visiting the primary school kitchen and garden classes to inform their own teaching. In a smaller rural school, where the kitchen was only used for part of the week, staff felt that there was potential to share both facilities and specialist teachers with another school if it took on the program. Benefits would occur in terms of community building as well as cost saving.
I like the idea of bringing schools together because you’re creating more of a community. It’s always nice and the kids like coming together and then they’re going to high school together.
Again, quantitative results failed to support qualitative evidence of changes to school sense of community. The proportion of teachers who strongly agreed that their “school has a strong sense of ‘community’ or ‘family’” increased between baseline and follow-up measurement in both program and control schools while the proportion of teachers reporting that the community served by their school is “supportive of its goals and activities” was stable in program schools and rose in comparison schools. There was little evidence to support a true difference between program and comparison schools for these measures (Table 2). Although school principals, teachers, parents, volunteers, specialist staff, and children all described “teamwork” as a skill that was honed by participation in the SAKG Program, and the proportion of teachers who strongly agreed that “students cooperate well with other students” increased, there was little quantitative evidence of a true difference in child cooperative behavior scores between program and comparison groups (Table 2).
Discussion of Findings and Limitations
These results have demonstrated strong evidence based on extensive interviews, focus groups, and observations of kitchen and garden classes that participating in the SAKG Program has been an overwhelmingly positive experience for children and school communities. Positive social impacts include increases in children’s engagement and confidence at school, improvements to the school learning environment, and strengthening of social connections both within the school and between schools and their wider communities.
Existing studies indicate that community garden projects have the capacity to increase “beautification” activities and improve residents’ attitudes and sense of ownership toward their local neighborhood (Armstrong, 2000; Teig et al., 2009). In line with these and other previously published studies, the results presented here suggest that children’s enjoyable experiences in the kitchen and garden promote engagement and attachment to the school (Alexander et al., 1995; Waliczek, Bradley, & Zajicek, 2001) whereas achievements and experience of success build confidence and self-esteem, which “spill over” into other aspects of their lives (Alexander et al., 1995; Liquori et al., 1998; Newell et al., 2004; Robinson & Zajicek, 2005). An enhanced sense of relatedness is associated with academic success whereby students who have positive connections with teachers and fellow students have enhanced engagement and higher levels of academic achievement (Furrer & Skinner, 2003). Kitchen garden programs are characterized by experiential learning approaches that provide students with the opportunity to create meaning from their direct experience and hence optimize their learning outcomes. Previous studies have shown that primary school students who participate in experiential learning have enhanced academic outcomes, including learning and retention (Laney, 1989). The qualitative findings from this study have also shown that interviewees attribute improvements in interpersonal relationships among children and between children and adults as well as greater connectedness between schools and communities to involvement in the program. Such findings are consistent with previously published qualitative findings substantiating a positive correlation between participation in community gardens and strengthening community organization, mutual trust, social connections, and/or interpersonal relationships (Armstrong, 2000; Kingsley, Townsend, & Henderson-Wilson, 2009; Teig et al., 2009). These social effects of the program are valued as highly by evaluation participants as those related to its more immediate objectives concerning nutrition.
This study has also highlighted some of the benefits and limitations of mixed-methods research when applied to evaluation of complex community interventions. Despite remarkable consistency of qualitative findings across a diverse range of schools and stakeholder groups that included children, teachers, parents, volunteers, school principals, and specialist staff, for these social impacts, qualitative results were not supported by any statistically significant quantitative results. This divergence between qualitative and quantitative results invites further reflection as to whether it should be considered a failure of the quantitative evaluation or perhaps even an expected outcome (Moffatt, White, Mackintosh, & Howel, 2006).
Recruiting comparison schools for this study was challenging. Given that a number of potential comparison schools declined participation, it is likely that those which did agree to participate were different from nonparticipating schools. Such differences pose a threat to the validity of the study (Ji, DuBois, Flay, & Brechling, 2008). It became clear during the course of the evaluation that schools with a particular interest in the project had greater incentive to participate, with the result that at the beginning of the study, five of the six comparison schools had their own edible gardens and by the end, the sixth school had also established one. In most of these cases, gardening was offered on an ad hoc basis depending on the interests of children or teachers. However, one of the comparison schools moved, over the course of the evaluation, from using their garden produce for occasional cooking to developing their own structured cooking program in which all children participated once or twice per term. Thus, many of the children in the comparison schools participated in gardening (and some cooking) activities, although for substantially fewer hours and in a less structured way than in the program schools. This means that the evaluation needed to establish an impact of the SAKG Program model over and above any of these alternative activities rather than in comparison with schools with no cooking or gardening pursuits at all. On one level, where (as discussed further below) significant differences were detected on more proximal program outcomes, this could be interpreted as providing additional strength of evidence of the program’s worth. However, our evaluation may have failed to show quantitative evidence of effect for the outcomes discussed here because the difference between the program and comparison schools recruited to our evaluation was too small to be detected by our planned sample size.
The primary outcome being evaluated for the SAKG Program is its influence on children’s appreciation of a diverse range of healthy foods. Sample size calculations used in designing the evaluation were therefore performed to ensure sufficient child participant numbers to capture a mean effect size between groups for this outcome with 80% power at the 5% level of significance. Secondary outcomes include the impact of the SAKG Program on children’s knowledge, confidence, and enjoyment in the kitchen and garden as well as on the wider social impacts discussed in this article. For outcomes concerning children’s appreciation of new foods and for their kitchen and garden knowledge and confidence, qualitative findings have been supported by quantitative findings of a positive outcome (Block et al., 2009). Distal effects of the program on the wider social environment of the school are by their nature subject to a greater range of influencing factors than more proximal outcomes, such as attitudes and knowledge concerning food production, preparation, and consumption. Given the range of influencing factors, quantifiable impacts on such distal outcomes as absenteeism and literacy and numeracy test data, if they occurred, would be expected to be small and as these measures are available only at the school level, very large sample sizes would be required to statistically test for a policy-relevant difference in these measures. In addition, pedagogical theory would suggest that even if the program has succeeded in improving the school learning environment and perhaps even some individual academic performances, changes to aggregate academic outcomes would be likely to take longer to occur than the single year between baseline and follow-up testing that yielded data for this evaluation (Ozer, 2007). This latter point was anticipated by one of the school principals interviewed for this study:
[My] concern is pressure from the Department of Education to improve academic outcomes. If resources were spent on a literacy teacher this may happen, but that would take resources away from the Stephanie Alexander Kitchen Garden Program . . . Teachers know that the program is making a difference but this is difficult to demonstrate in terms of hard data [i.e., literacy and numeracy scores].
Other measures of interest when considering social impacts of the program include those concerning child cooperative behavior as well as overall child well-being. Given that learning to work as part of a team was frequently referred to by children themselves as well as school principals, teachers, specialist staff, volunteers, and parents as an important part of the program experience, it is perhaps a little more concerning that there was no change detected in the quantitative measure of child cooperative behaviors as reported by parents and children. Limitations of the teacher-reported data are discussed below. Child cooperative behavior scores reported by all groups may have been subject to the phenomenon known as the ceiling effect, where the high score recorded for this measure at baseline (approximately 17 out of 20 for program and comparison groups) left little room for improvement. Alternatively, it is possible that children in the comparison schools received similar benefits to those in program schools or that the 1-year period between baseline and follow-up evaluation may not have been long enough for any changes to cooperative behavior to be extended beyond the kitchen and garden classes and into general classroom activities.
Although not reaching statistical significance at the standard 5% level, the included measure of child quality of life using the KIDSCREEN-10 scale showed weak evidence of a small positive program effect, of 1.2 points in favor of the program group (p = .09; Erhart et al., 2009). However, the confidence interval (−0.2 to 2.7) includes zero, suggesting that further research with a larger sample size would be necessary to confirm whether or not this is a policy-relevant effect (Table 2). It should perhaps be noted here that gathering quantitative evidence for changes to subjective well-being can be difficult. Considerable debate exists over the extent to which quality-of-life measures are sensitive to change, with some scholars arguing that scores are maintained within a narrow range by “homeostatic” mechanisms (Cummins, 2000) and other studies finding that qualitative indications of enhanced quality of life were not supported by child self-reports and parent reports using a KIDSCREEN measure (Davis et al., 2009). Again, it must be emphasized that both well-being and child cooperative behavior are influenced by a multiplicity of factors beyond the impact of any school-based program.
The remaining quantitative measures of interest when considering the social impacts of the program were contained in the teacher questionnaire. Both program logic and qualitative findings suggest that we might expect to see improvements in teacher ratings of student social behavior and attitudes toward school as well as a school’s sense of community because of the SAKG Program. However, the small sample size for teacher-reported outcomes limits the interpretation of teacher questionnaire data. After some loss to follow-up through staff turnover, 52 teachers comprised the analyzed sample (24 from comparison schools and 28 from program schools). The wide confidence intervals around quantitative results on cooperative behavior and social and learning environment from the teacher questionnaire, consistent with a true intervention effect either strongly in favor or strongly against the SAKG Program (Table 2) indicate that this sample size was too small to yield a statistically significant finding.
This discussion of methodological limitations has so far emphasized the risk of making Type II errors or assuming that no changes have occurred because we have failed to demonstrate them statistically. However, by combining multiple sources of data as well as qualitative and quantitative methods we also maximize the potential for understanding why some of these results, which we intuitively expect, might be difficult to quantify. Parents, principals, teachers, and specialist staff at several schools all made the point that the program was particularly meaningful for individual children who were at risk of disengagement; and that for some, participating in the program had completely transformed their experience of school. A richly illustrative example of the implications of this for program evaluation occurred during a research visit to a participating school. One child had noticeably struggled to complete the child questionnaire, seeming uninterested in the process and having difficulty with the open-ended questions in particular. He later participated in a focus group but had little to contribute beyond saying he liked the program and wanted to be a chef one day. Later again, this particular child was referred to during an interview with the kitchen specialist. She extolled his enthusiasm, confidence, and prowess in kitchen classes as well as his ambition to become a chef while also noting his lack of success in the more “academic” environment of the classroom.
Here was a clear example of an important program impact that could not be measured through research techniques that are clearly more effective at capturing the experiences of the more literate and articulate. It is likely that many such children may even be unintentionally excluded from our quantitative evaluation sample either because they struggle to fill in written questionnaires or because they or their families may be less likely to return written consent. That this impact was captured here is testimony to the strength of including multiple data sources as well as multiple methods of data collection. This example also serves us well, when considering why mean scores of well-being, behaviors, or academic test data may remain unaffected by nonetheless highly significant program effects. If children at the “lower end” of the scale for these measures have the most to gain from this intervention, this has important implications for the potential of the SAKG Program to promote equity in our schools. Assuming we manage to include such children in our sample when measuring these effects, the distribution “tail” may well be shortened without any detectable shift in the mean. As many of these children are also likely to be those at risk of disengagement or suffering from socioeconomic disadvantage more broadly, this conclusion also has important implications for how we attempt to measure any such redistributive effects.
Conclusion and Summary of Implications for Policy, Practice, and Research
Some of the attributes of the program valued most highly by study participants went well beyond its immediate objectives. Opportunities for experiential and integrated learning, teamwork, building social skills, social connections, and links between schools and their communities were seen as critically important. Observed increases in student engagement and confidence were key themes and seen by many as particularly valuable for “nonacademic” or “challenging” children, some of whom had few other opportunities for experiencing success at school. These program effects were often listed among its most important outcomes and were a key motivating factor for school principals’ ongoing commitment to the program. The SAKG Program was seen as directly benefiting not only the children but also the other stakeholder groups by creating a “learning community” in which all could be involved. These findings suggest that policy makers should continue to support expansion and ongoing evaluation of the program, given that it has the potential not only to promote improved nutrition but also to provide additional benefits to individuals and communities through enhanced student engagement and confidence as well as community connectedness within and beyond the school. Specifically, funding to include kitchens in both new and existing schools is likely to reap dividends. In this study, evidence also emerged that children were more engaged in the gardening program in schools where the content reflected a higher level of sophistication and knowledge on the part of the educator. This finding suggests that it is better policy and practice to bring in specialist staff—trained in horticulture and cooking respectively—rather than expect classroom teachers to add gardening or food preparation to their repertoire.
This research goes some way toward addressing an identified need for rigorous evaluation of school gardening and kitchen garden programs. In terms of implications for practice, qualitative findings suggest that the kitchen component of the SAKG Program is particularly successful in engaging all participating students and adds substantially to the value of a garden program alone. These positive findings support a need for continued evaluation, including using randomized study designs, as the program is expanded. Further research should aim to clarify the benefits of a combined gardening and kitchen program over gardening or cooking programs alone. Reflections on the discordance between qualitative and quantitative evidence for these social impacts of the SAKG Program also have important implications for future research in this area. When evaluating complex interventions with a range of outcomes such as the SAKG Program, sample sizes and timeframes that are adequate for detecting proximal or primary outcomes may be insufficient when it comes to distal impacts of a program, which are influenced by other powerful factors. In addition, we need to pay careful attention to ensuring that our research methods do not inadvertently exclude those who have the most to gain from interventions addressing inequity.
Footnotes
Acknowledgements
The authors would like to thank the school principals, teachers, children, parents and volunteers for their generous support of this research. For their assistance in the early stages of this project, Elizabeth Waters, Helen Bolger-Harris, Monica Green, Lucy Westerman, Michele Bell, and Sing Kai Lo are also gratefully acknowledged.
Authors’ Note
All organizations involved in funding the study placed no restrictions or requirements on the publication of research findings.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:
Funding to conduct the evaluation was provided by the Stephanie Alexander Kitchen Garden Foundation. We also acknowledge the financial support the Foundation received for the evaluation from VicHealth, the Helen Macpherson Smith Trust, the Victorian Department of Education and Early Childhood Development, and Deakin University. Lisa Gibbs and Lisa Gold acknowledge the NHMRC Capacity Building Grant for Child and Adolescent Obesity Prevention and Lisa Gibbs acknowledges the Jack Brockhoff Child Health and Wellbeing Program for salary and operational funding support.
