Littering Reduction

Abstract

Littering continues to be a problem worldwide. The purpose of this article is to update earlier systematic reviews on littering and using a scorecard of seven social marketing components, assess the extent of social marketing use in identified littering programs. Following Center for Reviews and Dissemination guidelines, the search included all peer-reviewed studies published between 1995 and 2015 in the English language available through 10 databases. A total of 1,220 articles were initially identified and resulted in a final set of 17 that met the study criteria. The analysis revealed key insights including a lack of social marketing use in litter prevention efforts to date providing an opportunity for future research. Limitations of the current study and opportunities for future research are outlined.

Keywords

social marketing systematic literature review littering behavior change intervention

Background

Littering continues to be a problem across the globe. Littering is not only unattractive; it also threatens the environment (Huffman, Grossnickle, Cope, & Huffman, 1995). In this study, litter is considered as any piece of inappropriately placed waste matter (Schultz, Bator, Large, Bruni, & Tabanico, 2013). Discarded items may not be equally counted as litter, for example, a discarded cigarette butt versus an abandoned refrigerator. While both are unsightly, studies in this review focus on individual pieces of litter that can be held in the hand of an individual and can be deposited in a trash can and excludes matter such as abandoned motor vehicles and white goods including refrigerators and washing machines, which may require vastly different intervention approaches.

Litter that can be held in the hand and disposed of in a trash can has evolved from being viewed primarily as an aesthetic problem to a broader environmental issue and generally involves paper, bottles, and food packaging (Al-Khatib, 2009). Littering not only reduces the aesthetic appeal of public places including streets, parks, and waterways but can also degrade water quality, endanger and kill wildlife, and contribute to flooding by blocking drainage systems (Abu-Hilal & Al-Najjar, 2004; Al-Khatib, Arafat, Daoud, & Shwahneh, 2009; Chitotombe, 2014; Hartley, Thompson, & Pahl, 2015). Littering is a social behavior (Ong & Sovacool, 2012; Spacek, 2004; Stephen & James, 2014). Thus, designing and evaluating litter management from a social perspective is important (Ma & Hipel, 2016).

A variety of approaches have been employed to reduce littering including public policy (Ong & Sovacool, 2012), technology, educational and awareness programs (Hartley et al., 2015), infrastructure (Hoppe, Bressers, de Bruijn, & Franco-Garcia, 2013), persuasive messages (de Kort, McCalley, & Midden, 2008), and community development (Liu & Sibley, 2004). A distinct absence of social marketing interventions targeting litter in the literature has previously been observed. Social issues such as littering require multifaceted approaches to achieve behavior change which can include a variety of disciplines (Parkinson, Schuster, & Russell-Bennett, 2016). Each discipline has evolved with its own unique theories, tools and techniques, and a lack of connectedness between behavior change fields has been noted with the behavior change sector being criticized for operating in silos (Tapp & Rundle-Thiele, 2016), suggesting there is merit in understanding the extent that discipline understanding is applied across the behavior change sector.

Social marketing has been identified as a useful yet underused approach in environmental issues, particularly when targeting individual behavior (Takahashi, 2009). Social marketing, which involves the application of marketing thinking, tools, and techniques to achieve social change (Donovan & Henley, 2010), may provide an important and valued contribution to the reduction of littering, thereby enhancing the environment and public health domains. Centered on audience understanding social marketing may assist to deliver innovative solutions catering to heterogeneous populations (Tapp & Rundle-Thiele, 2016).

Systematic literature reviews allow essential benchmarking of development in a field while also informing direction for future research with respect to program effectiveness. Additionally, systematic literature reviews assist in identifying topical, theoretical, and methodological trends (Truong, 2014; Williams & Plouffe, 2007). There is a scarceness of rigorous reviews and meta-analyses of environmental interventions in general (Bates, 2010) and recent litter-oriented campaigns in particular. Therefore, the first aim of this article is to update earlier systematic literature reviews of littering by Dwyer, Leeming, Cobern, Porter, and Jackson (1993) and to provide a contemporary review of interventions aiming to reduce littering. Previous reviews of littering have not included a quality assessment (Dwyer, Leeming, Cobern, Porter, & Jackson, 1993; Huffman et al., 1995). Consideration of study quality assists by informing future intervention design, measurement, and reporting, which in turn may assist to enhance intervention effectiveness over time. Thus, this article also aims to assess the methodological quality of included studies.

To differentiate social marketing from other fields such as public health, Andreasen (2002) developed six components defining social marketing. Researchers have previously used Andreasen’s (2002) six components to determine the extent that social marketing has been applied in social marketing and other behavior change interventions (Carins & Rundle-Thiele, 2014; Gordon, McDermott, Stead, & Angus, 2006; Stead, Gordon, Angus, & McDermott, 2007). Recent research indicates that behavior change is more likely when more of Andreasen’s (2002) six components are applied (Carins & Rundle-Thiele, 2014). Theory inclusion in the development of behavior change programs has also been identified as an important element for successful programs (Truong, 2014) and the National Social Marketing Centre (NSMC; French & Blair-Stevens, 2005), who extended Andresean’s (2002) six social marketing components added theory as a component. Therefore, this study includes Andreasen’s (2002) six components along with the NSMC’s (French & Blair-Stevens, 2005) theory component extending the social marketing scorecard applied in earlier studies (e.g., see Carins & Rundle-Thiele, 2014; Kubacki, Rundle-Thiele, Pang, & Buyucek, 2015). A scorecard of recent interventions aiming to reduce littering may assist future researchers to understand how social marketing components can be applied and in turn how intervention effectiveness may be enhanced. Thus, the final aim of this article is to deliver a social marketing scorecard for identified studies. Overall, the study aims to update the litter reduction evidence base and to provide a contemporary understanding of the elements included in these programs to assist in the development of future behavior change programs.

Method

Study Design

The authors adopted steps outlined in the Center for Reviews and Dissemination guidelines (Tacconelli, 2010) to conduct the systematic review of littering interventions. The guidelines aim to avoid introducing bias. The steps followed involved developing the background of the study, the questions the review sought to answer, inclusion and exclusion criteria, and defining the outcomes of interest. A narrative technique was used to report findings, as meta-analysis was not possible due to study heterogeneity, including variation of data analysis approaches (i.e., both qualitative and quantitative) and the absence of a common statistical measure including reported effect sizes for interventions.

Search Method

Relevant literature published since the Dwyer et al. (1993) and Huffman, Grossnickle, Cope, and Huffman (1995) reviews were identified initially through key word searches in PubMed, PsycINFO, ScienceDirect, ProQuest, Sage, EBSCO, MEDLINE, Web of Science, Green FILE, ProQuest ERIC, and Taylor & Francis (see Table 1). Search terms included the following: “litter” or “waste” or “garbage” or “rubbish” or “trash” and “Intervention*” or “randomized controlled trial” or “evaluation” or “trial” or “campaign*” or “program*” or “study” or “studies” or “behavior change” and “Social marketing”. These terms were consistent with searches reported in previous systematic literature reviews in social marketing (see Carins & Rundle-Thiele, 2014; Kubacki et al., 2015) in the areas of healthy eating and problem alcohol use. The titles and abstracts of retrieved articles were reviewed and were discarded if they were not related to littering or did not meet the inclusion criteria. The full text of articles was obtained for further clarification on study measures and objectives. Further searches were conducted using reference lists and “cited by” searches on Google Scholar from papers identified in the initial review. The literature searches were conducted in May 2015. Searches were conducted by the lead author and two coauthors independently reviewed the articles for eligibility for inclusion.

Table 1.

Databases and Articles Retrieved in Initial Search.

Databases	Number of Articles Retrieved
ScienceDirect	153
ProQuest all databases	120
Web of Science	420
Sage	8
Emerald	61
EBSCO all databases	260
PubMed	12
Taylor & Francis	67
PsycINFO (Ovid)	88
ERIC	31
Total	1,220

Inclusion Criteria

Original quantitative research published in peer-reviewed journals between January 1995 and May 2015 that were written in English were included in the current study. Research was eligible if either self-reported or objective measures of littering behavior in an identified program, experiment, or intervention were included. For all studies, the inclusion criteria for outcome measures were littering behavior. Several potentially relevant articles identified during the literature search process did not meet the inclusion criteria as no program or intervention was identified. As only full-length articles were considered, a number of publications including editorials, abstracts, book reviews, practitioner profiles, research notes, and commentaries were excluded.

Search Outcomes

The initial literature search yielded 672 potential articles after removing duplicate records (see Figure 1). Of these, 17 met the inclusion criteria. Although all of the interventions described in this article targeted littering behavior, they were carried out in a diverse range of contexts and often included different outcome measures without effect size. This study focused on identifying whether positive, negative, or no change was observed, without attempting to determine the size of the effect.

Figure 1.

Systematic review process.

Quality Appraisal

Previous systematic literature reviews on littering (Dwyer et al., 1993; Huffman et al., 1995) have not undertaken quality assessments. The studies included in this review were assessed for their methodological quality using the Mixed Methods Appraisal Tool (MMAT) developed by Pace et al. (2012). This tool has been used previously in reviews of literature examining the physical environment (Joseph, Choi, & Quan, 2015). Using the MMAT allows the researcher to evaluate the quality of a study quickly and efficiently by responding to a set of questions dependent on the study type. A study quality score is calculated based on the criteria met. Two authors independently evaluated the quality of the 17 papers, and an interrater reliability score was calculated using Cohen’s κ (.81) indicating substantial agreement. Across the studies, the quality criteria were fulfilled differently, and no studies were excluded after the appraisal process. Interestingly, the papers which used only self-report behaviors (Daniels & Marion, 2005; Hartley et al., 2015) did not report the use of any validated scales in their papers, which was an apparent weakness of the studies. Two studies, Bateson, Callow, Holmes, Roche, and Nettle (2013) and Sussman and Gifford (2013) reported using an independent data point design to ensure the observations were likely to be independent of each other providing a rigorous approach to their observation experiments.

A variety of methods were used in the assessed studies. Of the 17 papers examined, 10 reported mixed methods (observation and survey), 4 reported observation alone, and 3 quantitative (survey). Within the studies, the most common method used was observations with 14 of 17 papers using this technique, which as Schultz and colleagues (2013) note is the most appropriate method in the littering space.

Data Abstraction and Synthesis

From the final 17 papers reporting on 16 studies the following data were extracted (see Table 2) including authors’ name and date journal published, study location, year, participants, study design, exposure(s), outcomes(s) needed, analytical methods, behavior measured, evaluation and theories if used, discipline field, outcome variables, results, and quality assessment. By “a study,” we mean all the published papers reporting on a single evaluation of a specific program. To understand which essential ingredients should be present in a littering intervention, the scorecard developed using Andreasen’s (2002) social marketing components and theory were used to assess interventions (see Table 3). In order to increase interrater reliability, all excerpts were reviewed, compared, and discussed by two social marketing researchers. If no evidence of a criterion was identified by any of the researchers, a “not reported” outcome was assigned to the intervention against the specific criterion.

Table 2.

Study Overview.

References	Location, Study Year, Participants	Study Design	Exposure(s)	Outcome(s)	Analytical Methods	Behavior Measured	Evaluation	Results	MMAT Quality Score
Bateson, Callow, Holmes, Roche, and Nettle (2013)	England Six bicycle racks on the campus of Newcastle University October to December 2012 Cyclists student and staff—439 males and 181 females (ages 18 and 40 years)	Experimental	Durable signs featuring a large pair of staring eyes preinstalled on the wall above the rack and manipulate the amount of litter present in the ground.	Behavior	Generalized linear models with binomial probability distribution and a logit link function.	Littering dropping decision	Observations	No evidence that images of watching eyes make behavior more normative (p = .060).	High
Brown, Ham, and Hughes (2010)	Tasmania Russell Falls Track in Mt Field National Park 2006–2007 Park visitor (18 years or older), 29 interviewed, 68 surveyed, and 571 observed and surveyed	Experimental	Different messages containing the logo of the park, reference to the location of the rubbish bin. A crushed aluminum can placed.	Beliefs, attitude, and behavior	One way ANOVA, t test, and χ²	Pick up other people’s litter	Interviews, survey, and observation	15–20% increase in litter pickup (p = .009 and .039)	High
de Kort, McCalley, and Midden (2008)	the Netherlands Shopping center Year not reported Study 1: Public—315 people surveyed (17–60 years old), 68% female and 32% male Study 2: 1,755 persons observed (20–40 years), 54% female and 46% male Study 3: 70 people (41 male and 30 female; age not reported)	Experimental	Persuasive trash-can design, verbal prompt. Messages formulated regarding personal, injunctive, and descriptive norms. Eight trash cans were designed with different arm gestures that communicate antilitter norms. Mirror over the trash can. Flyers (with information not relevant to the majority of people) were offered to all passersby while observers coded behavior. Sign placed over the trash can, large, bright red, wooden sign, 2 m high and 75 cm wide.	Behavior norms	Factor analysis, reliability analysis, ANOVA, χ²	Littering norm activation	Observations and survey	Control: 19% of flyers littered Treatment conditions: 10%, 11%, and 12% of flyers littered (χ² = 27.9, df = 6, p < .001).	High
de Lange, Debets, Ruitenburg, and Holland (2012)	the Netherlands Dutch trains Year not reported Participants (not reported)	Experimental	Fifty milliliters of perfume oil “capitaine” was added to every liter of cleaner in two compartments.	Behavior	ANOVA	Littering behavior change	Counts	The amount of littering was significantly reduced in train compartments where cleaner scent was dispersed compared to control compartment, F(1, 60) = 10.86, p < .01, n = .15	High
Ernest-Jones, Nettle, and Bateson (2011)	England Self-cleaning cafeteria Newcastle University 2007–2009 Student, staff, and visitors to campus	Experimental	Posters featuring images of pair of human eyes or of some flowers (control) color contrast equalized using Adobe Photoshop and verbal messages (congruent or incongruent) with littering behavior. Hung at eye-level around the walls of the cafeteria.	Behavior	Logistic regression	Clear one’s litter in university cafeteria	Observation	Significant effect of eye images, F(1, 29) = 19.86, p < .01, but not of message congruence, F(1, 29) = .87, ns, on the proportion of tables littering.	High
Hansmann and Scholz (2003)	Switzerland Ten Cinemas June 2001 4,329 Visitors	Experimental	Two step communication strategies. Step 1: Ambiguous message presented on the cinema screen to achieve high motivation for the cognitive processing of the subsequent antilittering information. Step 2: Picture projected on the screen immediately following the message of Step 1, displayed a common sign representing correct waste disposal, resolving the ambiguity.	Behavior	Not reported	Reduce littering behavior in cinema	Counts	Significant improvement 28.3% The effect of the size of the audience was not significant, F(1, 18) = .09, p = .77	Moderate
Hartley, Thompson, and Pahl (2015)	England Nine schools Year not reported 176 School children (8–13 years; 76 males and 99 females)	Intervention	Multiple techniques (posters and artwork, demonstrations, and mini-experiments). Activities (sea kayaking, beach conservation, and a tour of the aquarium).	Perception, concern, awareness, attitude, and behavior	Nonparametric statistical methods. Post hoc Wilcoxon matched-pair ranks tests (Z-score)	Litter reducing behavior, greater problem awareness, and concern about marine litter	Survey 176 responses Follow-up 1 week later	Children reported significantly greater levels of appropriate litter disposal action and encouraging family and friends to perform more litter-reducing behaviors after the intervention, p < .001	Moderate
Kallgren, Reno, and Cialdini (2000)	United States Stairwell Arizona State University Year not reported 296 Students (138 female and 158 male; age not reported) 149 Visitors to public urban hospital (98 female and 51 male), age median = 35.0 years. 107 student (37 female and 43 male)	Experimental	Arousal (focus on normative information) participants were led to a landing atop a stairwell. Confederate (focus on injunctive social norm against littering) exposes some participants to individual picking up a piece of litter in public parking garage, and the other participants were exposed to an individual who simply walked past + one or two large handbill tucked under windshield. Self-focusing techniques. Expose participants to a closed circuit TV picture of themselves or geometric shapes.	Behavior	Log-linear model/hierarchical model/χ²	Encourage prosocial behavior (littering)	Observation and survey	The least littering occurred in the high norm focus/two-handbill condition, χ² (1, N = 149) = 7.54, p < .01, 9.4% versus 34.2%. The high norm focus/two-handbill condition produced less littering (9.4%) than either the low norm focus/two-handbill condition (42.5%), χ² (1, N = 149) = 9.72, p < .01, or the high norm focus/one hand	High
Khawaja and Shah (2013)	Quaid-i-Azam University Islamabad October 2012 Randomly chosen participants Baseline and cost treatment, (were 40 participants) Undergraduate students (School of Economics)	Experimental	Examines whether the negative externality of littering can be internalized by associating a cost to the act by imposing a fine on the litterers.	Behavior	One-tailed Wilcoxon rank-sum test	Reduce litter behavior	Counts	The distribution of number of waste pieces outside the waste envelope in the baseline treatment is higher than the distribution of number of waste pieces outside the waste envelope in the cost treatment (p < .01).	High
Lindemann-Matthies, Bönigk, and Benkowitz (2012)	Germany Karlsruhe Forest March–April 2011 171 Children (80 girls and 91 boys) Grades 1–3 7–9 Years old	Experimental	Forest tour Verbal appeal (asked not to litter and pick up litter already in forest ground and put it in a bag). Modeling tour guide deliberately picked up litter and placed it in a bag.	Behavior	General linear models and linear regression	Not litter in forest school tour. Picking up litter, putting it in a bag	Observation, interview, and survey	A combination of verbal appeal and demonstration more positively influenced children’s litter behavior than the appeal alone, 8.55, p = .026	High
Liu and Sibley (2004)	New Zealand Quadrangle Victoria University of Wellington 1996–1999 2,397 students observed and surveyed (1,143 female and 1,186 male)	Intervention	Phase 1: Attitude salience interventions using prompts, modeling. Prompt using a banner hung in the Quad. Smaller notices with similar slogans were posted around the steps area—green and red stickers with slogan were offered. Model pick up five pieces of litter left in Quad. Structural interventions (e.g., changing the permanent physical characteristics). Feedback on the percentages of males and females who littered	Attitude and behavior	Inferential statistics (χ² test) and parallel analysis	Solve the problem of littering at university -Quad	Observation, survey, number of participants unstated 2 weeks later	Structural intervention (adding ashtrays and litter bins) reduced cigarette littering by 64% = 74.44 (p = .001) without changing attitudes toward littering. Posted feedback significantly reduced cigarette littering by 16.9% = 12.45 (p < .001)	High
Sibley and Liu (2003)	New Zealand The Victoria University of Wellington Quad 1996–1999 452 People (146 female and 302 male; age not reported)	Intervention	Feedback: Posted daily feedback on the percentage of men and women who littered. Feedback was provided through two mediums. First, an article was published in the student magazine. Second, a banner (1.3m × 5 m) was hung directly above the quad steps. Environmental design intervention: Additional litter receptacles and ashtrays were placed in the environment.	Behavior	Multinomial analysis of variance and Wald χ² test	Reduce active and passive littering behavior	Observation, number of participants not reported 2 days later	Posted feedback significantly reduced cigarette littering by 17% = 133.74 (p < .001) and no cigarette littering by 19% = 22.16 (p < .001)	High
Reams, Geaghan, and Gendron (1996)	Louisiana Baton Rouge, three selected neighborhoods December 1990–May 1991 Participants (age and sex not reported)	Field study	Recycle program—Education literature mailed (emphasis the environmental, economic and energy benefits). Structural interventions (bins were provided at no charge).	Behavior—correct recycling of litter (reduce recyclable litter), awareness, and understanding	Simple linear regression with ANCOVA	Litter less or/ and pick up more accumulated litter	Counts	Lower levels of recyclable litter in treatment areas than control (p = .0032). Using a recycling program to reduce littering behavior was not effective with the general litter, it was not statically significant (p = .4533)	Moderate
Sussman and Gifford (2013)	Canada Shopping center + restaurant Year not reported 562 Participants (47% female and 53% male variety of ages)	Intervention	Sign: Simple three-panel signs 8.5′′ × 11′′ color sheets of paper and folding them in thirds to form a triangle, placed on tabletops. Visual prompt: Above compost bin. Modeling: A total of 28 volunteers combined to form 23 pairs of confederates to act as models.	Behavior	Log linear	Encourage the use of public compost bins	Observation, interview, and survey	Models were more effective than signs (p < .001) Signs did not significantly influence behavior (p > .05).	High
Taylor, Curnow, Fletcher, and Lewis (2007)	Melbourne Suburb/Oak Park 2002–2004 81 Merchants and 342 public surveyed (age and sex not reported)	Education campaign	Merchants related elements: Brochure, one-to-one site visits, meetings with merchants, a cooperative cleanup event, a fact sheet, maintenance of infrastructure in the street (including bin-related infrastructure), posters in shop windows, windproof ashtrays and drain stenciling. Public-related elements: Posters in shop windows and a nearby railway station, stormwater drain stenciling, brochures distributed by merchants, and maintenance of the local environment (e.g., cleanup of dumped rubbish and improvement to local council-managed infrastructure).	Awareness, knowledge, attitude, and behavior	ANOVA test	Reduce littering in commercial areas.	Interview, survey, observation, counts, number of participants not reported Seven months later	Small positive change. This result was not, statistically significant (p = .34).	High
Uneputty, Evans, and Suyoso (1998)	Eastern Indonesia Ambon Bay November 1994–May 1995 Village community (age and sex not reported)	Education campaign	Clean up event, speech, seminar, and assistance government member for cleaning.	Behavior	Mann–Whitney U test	Reduce litter pollution on shores	Counts	There were significantly lower densities of litter after the cleanup event than before it (W = 153; p < .001; Mann–Whitney U test)	Moderate
Wever, Van Onselen, Silvester, and Boks (2010)	the Netherlands Coffee outlet canteen area Delft University of Technology 4,000 Consumers (age and sex not reported) Dutch National Rowing Championships in Amsterdam 898 consumers (age and sex not reported) Suburban shopping center	Experimental	Antilittering labels applied to disposable coffee cups in both conspicuous and inconspicuous forms.	Behavior	χ² test	Influence of the characteristic of an object on littering behavior	Counts, observation, number of participants not reported Several weeks later	The litter percentage was 11.2% with the labels applied, this dropped to 6.7%. In the after-test some weeks later, the percentage had increased again to 14.1%. This reduction was found to be significant (p < .001), but the labels do not seem to cause a memory effect.	Moderate

Note. ANOVA = analysis of variance; MMAT = Mixed Methods Appraisal Tool.

Table 3.

Scorecard Assessment.

No.	Authors	Behavior	Audience	Number of Scorecard Components	Behavioral Objective	Audience Segmentation	Formative Research	Exchange	Marketing Mix	Competition	Theory
1	Bateson, Callow, Holmes, Roche, and Nettle (2013)	Littering dropping decision	Cyclists (students + staff)	2	✓ (ns)	×	×	×	0	✓	×
2	Brown, Ham, and Hughes (2010)	Littering behavior	Park visitors	5	✓ (+)	×	✓	×	2 (promotion, product)	✓	Theory of planned behavior
3	de Kort, McCalley, and Midden (2008)	Littering norm activation	Public (20–60 years)	5	✓ (+)	×	✓	×	2 (promotion, product)	✓	Focus theory of normative conduct
4	de Lange, Debets, Ruitenburg, and Holland (2012)	Littering behavior	Commuters	2	✓ (+)	×	×	×	1 (product)	✓	×
5	Ernest-Jones, Nettle, and Bateson (2011)	Littering behavior	Students, staff, and visitors to campus	2	✓ (+)	×	×	×	1 (promotion)	✓	×
6	Hansmann and Scholz (2003)	Littering behavior	Visitors	3	✓ (+)	×	×	×	1 (promotion)	✓	Reputation and punishment models
7	Hartley, Thompson, and Pahl (2015)	Littering behavior	Schoolchildren (8–13 years)	4	✓ (+)	×	✓	×	2 (promotion, product)	✓	Social learning theory
8	Kallgren, Reno, and Cialdini (2000)	Littering behavior	Students	4	✓ (+)	×	✓	×	0	✓	Focus theory of normative conduct
9	Khawaja and Shah (2013)	Littering behavior	Undergraduate students	1	✓ (+)	×	×	×	1 (price)	×	×
10	Lindemann-Matthies, Bönigk, and Benkowitz (2012)	Littering behavior	Children Grades 1–3 (7–9 years)	2	✓ (+)	×	×	×	1 (product)	✓	×
11	Liu and Sibley (2004) and Sibley and Liu (2003)	Littering behavior	Students	5	✓ (+)	×	×	✓	3 (promotion, product, and place)	✓	Microeconomic theory
12	Reams, Geaghan, and Gendron (1996)	Littering behavior	Households	3	✓ (+)	×	×	×	4 (promotion, product, place, and price)	✓	×
13	Sussman and Gifford (2013)	Encourage the use of public compost bins	Community	4	✓ (+)	×	×	×	3 (promotion, product, and place)	✓	Subtheory of social space
14	Taylor, Curnow, Fletcher, and Lewis (2007)	Littering behavior	Merchants + public	4	✓ (ns)	×	×	×	3 (promotion, place, and product)	✓	Two-stage process model
15	Uneputty, Evans, and Suyoso (1998)	Littering behavior	Community	3	✓ (+)	×	×	×	1 (promotion)	✓	Schwartz Altruism model
16	Wever, Van Onselen, Silvester, and Boks (2010)	Littering behavior	Public	4	✓ (+)	×	×	×	2 (promotion, product)	✓	Focus theory of normative conduct

Results

First, an overview of the contexts of the included studies is presented. This is then followed by the scorecard assessment. The studies looked at a range of different littering environments (see Table 2). Five studies were undertaken in university settings, three were conducted in community settings (one in a beach community, one in a commercial precinct, and another in a local council area), two were conducted within shopping areas (one in a shopping center and the other an outdoor shopping strip), two took place in a national park or forest setting, two were conducted in public areas (one on a train and one in a cinema), and one study was conducted in a school setting.

Scorecard Assessment

The scorecard assessment found that most of the included studies used a range of social marketing components. Each scorecard component is briefly summarized in turn.

Behavior change

All 16 studies reported a specific littering behavior change goal and 5 of the 16 studies included additional change objectives. For example, Taylor, Curnow, Fletcher, and Lewis (2007) measured littering knowledge and attitudes and general stormwater management. The majority of studies reported objective littering measures (e.g., observed littering behavior [n = 5] or litter volume [n = 4]). Two studies used self-report behavior (Daniels & Marion, 2005; Hartley et al., 2015) as their outcome variable.

Fourteen studies reported litter behavior change. For example, Brown, Ham, and Hughes (2010) reported a 15–20% increase in litter pickup (p = .009) and Hartley, Thompson, and Pahl (2015) reported a significant reduction in children’s littering behavior (p < .001). Three studies reported moderate outcomes. Bateson et al. (2013) found weak evidence that an image of watching eyes makes behavior more normative (p = .060). Reams, Geaghan, and Gendron (1996) reported being partially successful with lower levels of recyclable litter found in treatment areas when compared to control areas (p = .0032), with limited spillover effect. Reams et al. (1996) applied a recycling program to change littering behavior suggesting the activation of social norms that lead to one desirable environmental behavior also has potential to lead to other desirable environmental behaviors, such as disposing of litter correctly. Finally, Taylor et al. (2007) reported no effect on reducing litter (p = .34).

Formative research

Formative research is crucial in the development of any behavior change intervention to understand what people in the target audience would value (Andreasen, 2002). Behavior change practitioners and researchers use formative research to gain insights into the motives, opportunities, barriers, and triggers that surround behavior changes (Tapp & Rundle-Thiele, 2016). Only four studies reported the use of formative research to inform their intervention. Brown et al. (2010) used interviews with visitors to a national park where the intervention was to take place to identify a set of salient beliefs that could be used to influence park visitors. A scenario study using a survey was used by de Kort, McCalley, and Midden (2008) to understand how various types of norms could be used to activate social judgments. Kallgren, Reno, and Cialdini (2000) drew upon an observation experimental study conducted by Cialdini, Reno, and Kallgren (1990), and Hartley et al. (2015) used a literature review on previous littering research focused upon children to develop their intervention. Limited reported use of formative research to understand the target audience offers considerable room for improvement.

Audience segmentation

Segmentation relies on dividing a total population targeted by an intervention into smaller groups using multivariate techniques such as cluster analysis. Following the identification of segments, marketing activities and materials tailored to the unique needs and wants of one or more unique groups are delivered to one or more target segments (Dietrich, Rundle-Thiele, & Kubackia, 2017). While segmentation is recognized as a key component of social marketing, a recent umbrella review of segmentation use in social marketing indicates that full application of the segmentation process is limited (Kubacki et al., 2017). Segmentation is a three-step process that in any large market comprises of segmenting, targeting, and positioning (Dietrich et al., 2015). While one study (Taylor, Curnow, Fletcher, & Lewis, 2007) developed different offerings for two distinct target audiences, segmentation was not evident as there were no attempts to understand whether distinct segment groups existed within both merchants and the public visiting the intervention precinct. Some studies targeted specific groups. For example, the Hartley et al. (2015) study developed a program specifically for children aged 8–13 years; however, there was only one program offered with no reported use of segmentation analysis prior to target selection. Given that segmentation seeks to develop accurate, robust, and meaningful understandings of population subsets that share common characteristics, and these segments are internally homogeneous meaning people in the segment are similar in, for example, their attitudes or beliefs, age, gender, physical location, and most importantly for social marketing, share similarities in their behaviors (French, 2017), we deem the Hartley et al. (2015) study provides evidence of targeting but not the full segmentation process.

Exchange

Stead, Gordon, Angus, and McDermott (2007) explain that in a behavior change context, an exchange occurs when individuals have to give something up in order to receive the proposed benefits offered by an intervention. Hence, the use of exchange (including incentives, disincentives, and an appropriate environment) is what distinguishes marketing from education (Takahashi, 2009). This indicates it is essential for social marketers and other behavior change disciplines to consider what would motivate people to engage voluntarily with the intervention or desired behavior and offer them something beneficial in return (Stead et al., 2007). Only one study in this review, the Victoria University Quad study, explicitly mentioned exchange (Liu & Sibley, 2004; Sibley & Liu, 2003), where students were rewarded for performing the desired behavior via incentives and punished for not performing the behavior. The incentive was to receive a green sticker for making the effort to dispose of litter correctly. The punishment was receiving a red sticker if litter was disposed of incorrectly (e.g., dropping litter on the ground or leaving it where they sat).

Marketing mix

Consistent with French and Blair-Stevens’s (2006) claim that to be classified as using a marketing mix, an intervention must use at least two elements of the marketing mix (product, price, place, or promotion) studies, in this review, were classified as using the marketing mix if they reported using at least two marketing mix elements. Of the studies reviewed in this article, only Reams et al. (1996) employed the full marketing mix. Two studies used three elements of the marketing mix, first, the Victoria University Quad study (Liu & Sibley, 2004; Sibley & Liu, 2003) reported the use of product, place, and promotion in their program. They investigated the differences between using only promotion and promotion plus infrastructure change by way of adding additional rubbish bins and cigarette ashtrays in the areas where high levels of littering occurred, thus addressing product and place of the marketing mix. There were significantly higher levels of litter disposed of correctly in the intervention using three elements of the marketing mix than the intervention that only used promotion. Second, Sussman and Gifford (2013) reported the use of compost bins which were conveniently placed next to the garbage and recycle bins in a community shopping center food court with tabletop signs used to promote the benefits of composting, this resulted in reduced levels of litter.

Ten interventions in this review reported using products, both tangible and intangible. Tangible products included, for example, extra litter bins and new ashtrays (Sibley & Liu, 2003) and compost bins (Reams, Geaghan, & Gendron, 1996; Sussman & Gifford, 2013). Intangible product offerings incorporated scent exposure on trains (de Lange, Debets, Ruitenburg, & Holland, 2012), services which included maintenance of infrastructure at Snell Grove commercial areas (Taylor et al., 2007), educational programs for school children on the harm of litter on marine life (Hartley et al., 2015), and a demonstration for school children on how to clean up litter in a forest (Lindemann-Matthies, Bönigk, & Benkowitz, 2012).

Evidence of promotion was identified in 11 interventions. The interventions used a wide range of promotional tools such as events (n = 2), brochures (n = 2), cinema advertising (n = 1), tabletop signs (n = 1), posters (n = 4), newsletters (n = 1), and banners or signs (n = 2) to raise awareness of the harms of littering, change beliefs and attitudes toward littering, and change littering behavior.

Place, the location where the target audience enter into an exchange was identified in three interventions, for example, in the Victoria University Quad study, extra litter bins and new ashtrays were placed in convenient locations where students gathered to eat and smoke cigarettes and left large amounts of litter (Liu & Sibley, 2004). Sussman and Gifford (2013) reported placing compost bins next to general litter bins and recycling bins in a shopping center food court to make it convenient for shoppers to place their food waste in compost bins and reduce the amount of compostable materials going into landfill. Finally, price was only identified in the intervention discussed in Reams et al. (1996), where dedicated recycling bins were provided at no charge to residents living in the trial area.

Competition

Competitive analysis means that program developers must consider the competition posed by alternate behaviors, in terms of time and attention, to the ones being targeted in the intervention and employ strategies which seek to minimize this competition (Stead et al., 2007). An intervention was classified as having met the competition component of social marketing if at least one form of competition was identified in the analyzed studies. Ten intervention studies recognized competition to their desired behavior of disposing of litter in bins (de Lange et al., 2012; Liu & Sibley, 2004; Sibley & Liu, 2003; Taylor et al., 2007), picking up litter (Brown, Ham, & Hughes, 2010; de Kort et al., 2008; Ernest-Jones, Nettle, & Bateson, 2011; Lindemann-Matthies et al., 2012; Uneputty, Evans, & Suyoso, 1998), not putting recycling into general litter bins (Reams et al., 1996), and composting (Sussman & Gifford, 2013).

Theories and models used in the programs

The use of theory in intervention design has long been argued to be important (Glanz & Bishop, 2010; Rice & Atkin, 2000). Stead et al. (2007) posit the use of a theoretical framework, combined with the use of formative research, helps translate theoretical constructs into persuasive and acceptable interventions, which is a significant prerequisite for success. Ten interventions were theoretically informed. The focus theory of normative conduct was used in three interventions (de Kort et al., 2008; Kallgren, Reno, & Cialdini, 2000; Sussman & Gifford, 2013) with significant positive effects on littering behavior observed in all three studies. Brown et al. (2010) used the theory of planned behavior to design their intervention and Hansman et al. (2003) used social learning theory, and both reported significant positive effects on littering behavior. Taken together, use of the focus theory of normative conduct, the theory of planned behavior, and social learning theory in intervention design and delivery may assist to reduce littering behavior.

Discussion

This article extends earlier systematic literature reviews in littering (Dwyer et al. (1993) and provides a contemporary review of interventions aiming to reduce littering. Additionally, this article applied a quality assessment of the included studies which has not been reported previously. Finally, this article introduces a scorecard which establishes the extent that social marketing components have been used to change littering behavior.

An absence of social marketing studies in the context of littering was evident, which is surprising, given Takahashi’s (2009) call for social marketing use to combat environmental issues. However, a broad range of approaches were observed in the reviewed studies offering some important lessons. There have been widespread efforts to change littering behavior ranging from images picturing watching eyes to the building of environmental infrastructure. Littering interventions and/or programs identified in the review have used both individual behavior change and infrastructure change to build environments to reduce littering. Assessing each study against the social marketing scorecard has demonstrated that there is room to improve intervention planning and design to center programs on the target audience’s needs and wants. Furthermore, the findings of this study provide additional evidence that there may be merit in using theory to develop programs aimed at changing behavior.

Extending on earlier reviews, summarizing more than 100 studies on littering (see, e.g., Dwyer et al., 1993; Huffman et al., 1995), this systematic review of the literature identified 17 peer-reviewed journal articles reporting on 16 littering studies published between 1995 and 2015 that met the study’s inclusion criteria. The quality of the studies was also examined with a mixed range of scores observed. The quality assessment identified a lack of consistent and validated measures for outcome assessment.

This review summarized the extent that the developed social marketing scorecard based on Andreasen’s (2002) six social marketing components with the addition of theory has been applied to change littering behavior. A total of seven components were assessed in the current review. This study, together with earlier reviews (Kubacki et al., 2015; Carins & Rundle-Thiele, 2014) in other contexts, provides an overview that researchers and practitioners alike can access to better understand how social marketing components have been applied in programs. Formative research and segmentation were not widely used in the current review and this represents a considerable opportunity to more closely orient programs to meet the needs and wants of the target audience(s), rather than applying a one-size-fits-all approach.

Conclusion, Limitations, and Future Research Directions

The results of this study present several opportunities for future research. First, to follow systematic review protocols and for quality assurance, only published academic peer-reviewed papers were included in this review. Thus, books, government reports and other gray literature, working papers, and reports were not considered in the current study. Future research is recommended to extend our understanding beyond the academic peer-reviewed literature. For example, the scorecard employed in this study could be used to review the gray literature to understand what other strategies and approaches have been successfully used to change littering behavior. Second, the current study was restricted to interventions and randomized control trials. Future research should also consider evaluations that do not include randomization, control groups or do not refer to themselves as programs or interventions to provide deeper understanding of the littering issue. Another important limitation is that the majority of evidence considered in this review is from studies conducted in developed countries. Future research is recommended to focus on applying a broad range of behavior change approaches in developing countries to extend our understanding beyond developed countries. In addition, more detail on sample recruitment, response rates, and matched control groups would improve the quality of reported studies. Finally, study heterogeneity, including different methodological approaches and the absence of a common statistical measure, prevented meta-analysis from being undertaken. To address the identified limitations in this study, future littering research should aim to use consistent outcome measures and research designs to enable rigorous and systematic literature reviews and meta-analyses to be undertaken.

Footnotes

Author Note

Author Yara Almosa is also affiliated to Ministry of education Saudi Arabia.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Abu-Hilal

A. H.

Al-Najjar

(2004). Litter pollution on the Jordanian shores of the Gulf of Aqaba (Red Sea). Marine Environmental Research, 58, 39–63. Retrieved fromhttps://dx-doi-org.web.bisu.edu.cn/10.1016/j.marenvres.2003.12.003

Al-Khatib

I. A.

(2009). Children’s perceptions and behavior with respect to glass littering in developing countries: A case study in Palestine’s Nablus district. Waste Management, 29, 1434–1437. Retrieved fromhttps://dx-doi-org.web.bisu.edu.cn/10.1016/j.wasman.2008.08.026

Al-Khatib

I. A.

Arafat

H. A.

Daoud

Shwahneh

(2009). Enhanced solid waste management by understanding the effects of gender, income, marital status, and religious convictions on attitudes and practices related to street littering in Nablus–Palestinian territory. Waste Management, 29, 449–455. doi:10.1016/j.wasman.2008.02.004

Andreasen

A. R.

(2002). Marketing social marketing in the social change marketplace. Journal of Public Policy & Marketing, 21, 3–13. Retrieved fromhttps://dx-doi-org.web.bisu.edu.cn/10.1509/jppm.21.1.3.17602

Bates

C. H.

(2010). Use of social marketing concepts to evaluate ocean sustainability campaigns. Social Marketing Quarterly, 16, 71–96. doi:10.1080/15245000903528357

Bateson

Callow

Holmes

J. R.

Roche

M. L. R.

Nettle

(2013). Do images of ‘watching eyes’ induce behaviour that is more pro-social or more normative? A field experiment on littering. PLoS One, 8, e82055. Retrieved fromhttps://dx-doi-org.web.bisu.edu.cn/10.1371/journal.pone.0082055

Brown

T. J.

Ham

S. H.

Hughes

(2010). Picking up litter: An application of theory-based communication to influence tourist behaviour in protected areas. Journal of Sustainable Tourism, 18, 879–900. doi:10.1080/09669581003721281

Carins

J. E.

Rundle-Thiele

S. R.

(2014). Eating for the better: A social marketing review (2000–2012). Public Health Nutrition, 17, 1628–1639. Retrieved fromhttps://dx-doi-org.web.bisu.edu.cn/10.1017/S1368980013001365

Chitotombe

J. W.

(2014). Interrogating factors associated with littering along road servitudes on Zimbabwean highways. Environmental Management and Sustainable Development, 3, 181–193. doi:10.5296/emsd.v3i1.5023

10.

Cialdini

R. B.

Reno

R. R.

Kallgren

C. A.

(1990). A focus theory of normative conduct: Recycling the concept of norms to reduce littering in public places. Journal of Personality and Social Psychology, 58, 1015. Retrieved fromhttps://dx-doi-org.web.bisu.edu.cn/10.1037/0022-3514.58.6.1015

11.

Daniels

M. L.

Marion

J. L.

(2005). Communicating leave no trace ethics and practices: Efficacy of two-day trainer courses. Journal of Park and Recreation Administration, 23, 1–19.

12.

de Kort

Y. A.

McCalley

L. T.

Midden

C. J.

(2008). Persuasive trash cans activation of littering norms by design. Environment and Behavior, 40, 870–891. doi:10.1177/0013916507311035

13.

de Lange

M. A.

Debets

L. W.

Ruitenburg

Holland

R. W.

(2012). Making less of a mess: Scent exposure as a tool for behavioral change. Social Influence, 7, 90–97. doi:10.1080/15534510.2012.659509

14.

Dietrich

Rundle-Thiele

Kubackia

(2017). Segmentation in social marketing: Methods and approaches. Singapore: Springer.

15.

Dietrich

Rundle-Thiele

Schuster

Drennan

Russell-Bennett

Leo

… Connor

J. P.

(2015). Differential segmentation responses to an alcohol social marketing program. Addictive Behaviors, 49, 68–77. doi:10.1016/j.addbeh.2015.05.010

16.

Donovan

Henley

(2010). Principles and practice of social marketing: An international perspective. Cambridge, MA: Cambridge University Press.

17.

Dwyer

W. O.

Leeming

F. C.

Cobern

M. K.

Porter

B. E.

Jackson

J. M.

(1993). Critical review of behavioral interventions to preserve the environment research since 1980. Environment and Behavior, 25, 275–321. doi:10.1177/0013916593255001

18.

Ernest-Jones

Nettle

Bateson

(2011). Effects of eye images on everyday cooperative behavior: A field experiment. Evolution and Human Behavior, 32, 172–178. doi:10.1016/j.evolhumbehav.2010.10.006

19.

French

Blair-Stevens

(2005). The big pocket guide to using social marketing for behaviour change. London, England: The National Social Marketing Centre. Retrieved February 21, 2015, fromwww.thensmc.com/sites/default/files/Big_pocket_guide_2011.pdf

20.

French

Blair-Stevens

(2006). Social marketing national benchmark criteria. London, England: The National Social Marketing Centre.

21.

French

(2017). The importance of segmentation in social marketing strategy. In Segmentation in Social Marketing (pp. 25–40). Singapore: Springer.

22.

Glanz

Bishop

D. B.

(2010). The role of behavioral science theory in development and implementation of public health interventions. Annual Review of Public Health, 31, 399–418. doi:10.1146/annurev.publhealth.012809.103604

23.

Gordon

McDermott

Stead

Angus

(2006). The effectiveness of social marketing interventions for health improvement: What’s the evidence?Public Health, 120, 1133–1139. doi:10.1016/j.puhe.2006.10.008

24.

Hansmann

Scholz

R. W.

(2003). A two-step informational strategy for reducing littering behavior in a cinema. Environment and Behavior, 35, 752–762. doi:10.1177/0013916503254755

25.

Hartley

B. L.

Thompson

R. C.

Pahl

(2015). Marine litter education boosts children’s understanding and self-reported actions. Marine Pollution Bulletin, 90, 209–217. doi:10.1016/j.marpolbul.2014.10.049

26.

Hoppe

Bressers

de Bruijn

Franco-Garcia

(2013). Street litter reduction programs in the Netherlands: Reflections on the implementation of the Dutch litter reduction program for 2007–2009. Lessons from a public private partnership in environmental policy. Environmental Engineering and Management Journal, 12, 1657–1668.

27.

Huffman

K. T.

Grossnickle

W. F.

Cope

J. G.

Huffman

K. P.

(1995). Litter reduction a review and integration of the literature. Environment and Behavior, 27, 153–183. doi:10.1177/0013916595272003

28.

Joseph

Choi

Y.-S.

Quan

(2015). Impact of the physical environment of residential health, care, and support facilities (RHCSF) on staff and residents a systematic review of the literature. Environment and Behavior, 1–39. doi:10.1177/0013916515597027

29.

Kallgren

C. A.

Reno

R. R.

Cialdini

R. B.

(2000). A focus theory of normative conduct: When norms do and do not affect behavior. Personality and Social Psychology Bulletin, 26, 1002–1012. doi:10.1177/01461672002610009

30.

Khawaja

F. S.

Shah

(2013). Determinants of littering: An experimental analysis. The Pakistan Development Review, 52, 157–168.

31.

Kubacki

Rundle-Thiele

Lahtinen

Parkinson

(2015). A systematic review assessing the extent of social marketing principle use in interventions targeting children (2000-2014). Young Consumers, 16, 141–158.

32.

Kubacki

Rundle-Thiele

Pang

Carins

Parkinson

Fujihira

Ronto

(2017). An umbrella review of the use of segmentation in social marketing interventions. In Dietrich

Kubacki

Rundle-Thiele

(Eds.), Segmentation in social marketing (pp. 9–23). Singapore: Springer.

33.

Lindemann-Matthies

Bönigk

Benkowitz

(2012). Can’t see the wood for the litter: Evaluation of litter behavior modification in a forest. Applied Environmental Education & Communication, 11, 108–116. doi:10.1080/1533015X.2012.751294

34.

Liu

Sibley

(2004). Attitudes and behavior in social space: Public good interventions based on shared representations and environmental influences. Journal of Environmental Psychology, 24, 373–384. doi:10.1016/j.jenvp.2003.12.003

35.

Hipel

K. W.

(2016). Exploring social dimensions of municipal solid waste management around the globe—A systematic literature review. Waste Management. doi:10.1016/j.wasman.2016.06.041

36.

Ong

I. B. L.

Sovacool

B. K.

(2012). A comparative study of littering and waste in Singapore and Japan. Resources, Conservation and Recycling, 61, 35–42. doi:10.1016/j.resconrec.2011.12.008

37.

Pace

Pluye

Bartlett

Macaulay

A. C.

Salsberg

Jagosh

Seller

(2012). Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. International Journal of Nursing Studies, 49, 47–53. doi:10.1016/j.ijnurstu.2011.07.002

38.

Parkinson

Schuster

Russell-Bennett

(2016). Insights into the complexity of behaviours: the MOAB framework. Journal of Social Marketing, 6, 412–427. doi:10.1108/JSOCM-10-2015-0071

39.

Reams

M. A.

Geaghan

J. P.

Gendron

R. C.

(1996). The link between recycling and litter a field study. Environment and Behavior, 28, 92–110. doi:10.1177/0013916596281005

40.

Rice

R. E.

Atkin

C. K.

(2000). Public communication campaigns (Vol. 3). Thousand Oaks, CA: Sage.

41.

Schultz

P. W.

Bator

R. J.

Large

L. B.

Bruni

C. M.

Tabanico

J. J.

(2013). Littering in context: Personal and environmental predictors of littering behavior. Environment and Behavior, 45, 35–59. doi:10.1177/0013916511412179

42.

Sibley

C. G.

Liu

J. H.

(2003). Differentiating active and passive littering a two-stage process model of littering behavior in public spaces. Environment and Behavior, 35, 415–433. doi:10.1177/0013916503035003006

43.

Spacek

(2004). DO MESS WITH IT!: A sociopolitical study of littering and the role of southern and nearby statesApplied Research Projects, Texas State University-San Marcos. Retrieved fromhttp://ecommons.txstate.edu/arp/27/

44.

Stead

Gordon

Angus

McDermott

(2007). A systematic review of social marketing effectiveness. Health Education, 107, 126–191. Retrieved fromhttps://dx-doi-org.web.bisu.edu.cn/10.1108/09654280710731548

45.

Stephen

James

(2014). Changing audience behaviour: Festival goers and throwaway tents. International Journal of Event and Festival Management, 5, 247–262. doi:10.1108/IJEFM-11-2013-0031

46.

Sussman

Gifford

(2013). Be the change you want to see modeling food composting in public places. Environment and Behavior, 45, 323–343. doi:10.1177/0013916511431274

47.

Tacconelli

(2010). Systematic reviews: CRD’s guidance for undertaking reviews in health care. The Lancet Infectious Diseases, 10, 226. Retrieved fromhttps://dx-doi-org.web.bisu.edu.cn/10.1016/S1473-3099(10)70065-7

48.

Takahashi

(2009). Social marketing for the environment: An assessment of theory and practice. Applied Environmental Education and Communication, 8, 135–145. doi:10.1080/15330150903135889

49.

Tapp

Rundle-Thiele

(2016). Social marketing and multidisciplinary behaviour change. In Spotswood

(Ed.), Beyond behaviour change: Key issues, interdisciplinary approaches and future directions (p. 135). Bristol, CT: Policy Press.

50.

Taylor

Curnow

Fletcher

Lewis

(2007). Education campaigns to reduce stormwater pollution in commercial areas: Do they work?Journal of Environmental Management, 84, 323–335. doi:10.1016/j.jenvman.2006.06.002

51.

Truong

V. D.

(2014). Social marketing a systematic review of research 1998–2012. Social Marketing Quarterly, 20, 15–34. doi:10.1177/1524500413517666

52.

Uneputty

Evans

Suyoso

(1998). The effectiveness of a community education programme in reducing litter pollution on shores of Ambon Bay (eastern Indonesia). Journal of Biological Education, 32, 143–147. doi:10.1080/00219266.1998.9655611

53.

Wever

Van Onselen

Silvester

Boks

(2010). Influence of packaging design on littering and waste behaviour. Packaging Technology and Science, 23, 239–252. doi:10.1002/pts.892

54.

Williams

B. C.

Plouffe

C. R.

(2007). Assessing the evolution of sales knowledge: A 20-year content analysis. Industrial Marketing Management, 36, 408–419. doi:10.1016/j.indmarman.2005.11.003