The Limits to Planning

Abstract

If planning is to matter for urban development and policy, it is not sufficient for plans to be implemented. Plans and planning must also have a causal role—they must lead to outcomes that would not be realized otherwise. In case studies of municipal climate action planning in California, I find little evidence for any causal impacts. Instead, cities are using climate plans to codify policies that were likely to happen anyway. The results call for a more nuanced view of when it makes sense to plan, what types of plans are most useful, and how to evaluate a plan’s effects.

Keywords

climate action plans climate planning plan evaluation plan implementation

Introduction

One of the most derogatory comments that can be made to a city planner is that the main tangible product of his or her work—the plan—simply “sits on a shelf.” Urban planning would seem to have limited value if the plans fail to be implemented, and are not dusted off and used by city staff, elected officials and developers. Indeed, scholars such as Talen (1996b) and Laurian et al. (2010) have explicitly addressed the question of urban plan implementation, decrying the lack of empirical research on the subject.

Plan implementation, however, is not a sufficient condition for the urban planning process and the adopted plan (which I refer to collectively as “planning” in this paper¹) to have an impact on development and policy choices. Impact implies a causal relationship between planning and outcomes, which implementation alone does not. After all, a plan may be implemented, but merely codify choices that had been made before planning began, or that would have been made anyway. While there has been limited empirical work on plan implementation, there has been even less research on the causal impact of planning.

Planning should not be judged solely on its ability to bring about change. Indeed, other benefits that accrue from planning include the generation of social capital, and the provision of information-related collective goods that may reduce the costs of urban infrastructure (Rydin and Pennington 2000; Hopkins 2001, 9; Waldner 2008, 696). However, I argue that planning should primarily be evaluated on the extent to which it changes outcomes such as the spatial pattern of development, expenditure decisions by local governments, and the transportation and housing choices of individuals. Moreover, implicit assumptions of causality are central to the legitimacy of planning, and also to much of planning theory—particularly in the realms of representation and power relations (Innes and Booher 1999; Flyvbjerg 2002). If the goal of planning is merely to gather information or chart a path toward an agreed-on goal, then concerns about the extent to which the plan reflects community consensus seem less critical. Alternatively, if the main goal is to develop social capital, then why go to the trouble of developing a plan?

In this paper, I use case studies of climate planning in California cities, coupled with statistical results reported in Millard-Ball (2012), to show that while the provisions of climate plans are often implemented, there is little evidence for any causal impact on policy and development outcomes. I show that pre-existing environmental preferences provide a more compelling explanation for cross-city variation in environmental metrics such as the number of green buildings. Moreover, devoting effort to climate planning may have detracted from doing, that is, taken attention away from the implementation of specific measures to reduce emissions.

Climate planning may well be a special case, not least because of its voluntary nature. Thus, other types of urban planning may well have a causal impact through mechanisms that are not applicable in the case of climate planning. In particular, statutory Comprehensive Plans might change outcomes through allowing different actors to coordinate interdependent decisions or through legal sanctions. But there is little empirical evidence to either support or rebut such a view. My results highlight the need for greater empirical work on how and when planning can have an impact.

The paper proceeds as follows. I first provide a conceptual framework for understanding the causal impacts of planning, setting out a typology of five causal pathways. In the methods section, I show how the combination of case studies and statistical results leads to greater confidence in the findings, and also allows for a systematic approach to case selection. After presenting the main empirical results, I conclude with a discussion of the implications for other types of planning.

How Plans Have Causal Impacts

Conformance, Performance, and Causality

The literature on planning implementation often distinguishes between conformance and performance (for recent reviews, see Kinzer 2010; Oliveira and Pinho 2010; Hopkins 2012). Conformance generally equates to implementation. That is, do development outcomes and regulatory decisions conform to the goals and policies expressed in the plan? A typical approach is to compare the actual spatial pattern of development or facility provision to that called for in the plan and attempt to explain any discrepancies (Alterman and Hill 1978; Talen 1996a; Brody and Highfield 2005).

Performance is interpreted in more diverse ways in the literature. Hopkins (2012) sees performance as synonymous with causality. The earlier formulation by Mastop and Faludi (1997) and Faludi (2000) sees the use of the plan as central—whether the plan plays a role in decision situations, and whether the plan was useful to, was consulted by, and influenced decision makers. In contrast, Berke et al. (2006, 589) equate performance with better decisions as judged by normative criteria. If, as I argue, the causal role of plans is the central basis on which to evaluate planning, then the utility of a broad conceptualization of performance is limited, as causality is only one of its components.

Instead of distinguishing between conformance and performance, it seems more useful to distinguish among a plan’s implementation, causal attribution, and causal pathways. Implementation addresses the extent to which measures and outcomes called for in the plan materialize in practice, regardless of whether these outcomes can be attributed to the plan. Causal attribution addresses whether the existence of a plan results in different outcomes than would have occurred in the absence of either plan adoption or the planning process. A focus on causal attribution also simplifies the process of planning evaluation. Rather than delving into controversies such as what constitutes success, whether particular outcomes are desirable, or which outcomes should be measured (Talen 1997; Brody and Highfield 2005, 160-61), one can simply ask whether planning has any impact on outcomes. The causal pathways category takes causal attribution a step further, asking not just whether planning has a causal role but identifying explicit pathways through which the impact occurs. Note that the three categories are not mutually exclusive: causal pathways cannot be identified if there is no causal attribution, and both causal attribution and causal pathways presuppose some degree of implementation, although there may be exceptions if planning has unintended consequences.

The distinction that I propose here cuts across the categories of conformance and performance (Table 1). Most conformance studies would fall under “implementation,” as they have not sought to establish a causal relationship (exceptions include Chapin, Deyle, and Baker 2008; Waldner 2008). On the performance side, some studies aspire to understand the causal impact of planning, but others would be better classed as implementation. Faludi’s (2000) study of performance is a case in point: it focuses on whether a plan was used by decision makers. But even if the final plan was never consulted, planning may still change outcomes through shaping the preferences of residents or decision makers. Conversely, causal impact does not automatically follow by virtue of a plan being used. Instead, the plan could be consulted in order to justify a decision post hoc, or to provide political cover to a decision maker.

Table 1.

Typologies of Plan Evaluation Studies.

	Implementation Only	Causal Attribution	Causal Pathways
Implementation	Yes	Usually	Usually
Causal role	No	Attribution of impacts to plan	Attribution of impacts to plan through explicit pathway(s)

Conformance	Do outcomes and regulatory decisions match what was called for in the plan?	Do outcomes and regulatory decisions match what was called for in the plan, and would they not have occurred in the absence of the planning process or plan adoption?	Do outcomes and regulatory decisions match what was called for in the plan, and can they be attributed to the planning process or plan adoption through an explicit causal pathway?
Conformance	Example: Alterman and Hill 1978	Example: Chapin, Deyle, and Baker 2008
Performance	Was the plan used by decision makers? Are “good” outcomes achieved?	Does the planning process or plan adoption lead to different outcomes compared to the “without plan” counterfactual, regardless of whether the outcomes conform to the plan?	Does the planning process or plan adoption lead to different outcomes compared to the “without plan” counterfactual, and can an explicit causal pathway be identified?
Performance	Examples: Faludi 2000; Berke et al. 2006	Example: Laurian et al. 2010

The desirability of evaluating the causal role of planning, through comparing outcomes in the presence of planning to a “no planning” counterfactual, has oft been recognized (Preece 1990; Chapin, Deyle, and Baker 2008; Laurian et al. 2010, 747; Hopkins 2012). But if there is a paucity of empirical research on planning implementation, there is an even greater dearth of studies that attempt to establish causal attribution. The limited amount of empirical work that does exist has sought to identify changes in land values following plan adoption, which signal changed expectations of future development patterns (Johnston, Schwartz, and Klinkner 1978; Knaap, Ding, and Hopkins 2001); used game theory to predict the responses of local governments and developers to a plan (Knaap, Hopkins, and Donaghy 1998); or compared “with plan” and “without plan” cases (Ingram 2009). Another approach by Laurian et al. (2010) relies on a combination of theory-driven plan logic (i.e., whether the plan is capable of having a causal impact) and expert judgment to assess causal relationships. The causal pathways category, meanwhile, is populated even more sparsely. While there is theoretical work that identifies plausible mechanisms (see especially Hopkins 2001, 2012), there is little empirical research that carefully isolates alternative pathways.

Some scholars (e.g., Talen 1996a, 83) have considered the establishment of causality to be an unrealistic empirical goal. But while there are certainly empirical challenges to identifying causal impacts, there are well-established methodological approaches, including the time-series cross-sectional designs advocated by Chapin, Deyle, and Baker (2008). Moreover, a large body of work in other disciplines discusses methods for estimating causal effects (for one review, see Winship and Morgan 1999).

Causal Pathways for Planning

This section develops a typology of five causal pathways (Figure 1), which I apply empirically later in the paper. It consists of a selective review of theoretical strands from planning and related disciplines. Rather than being comprehensive, I aim to develop a typology that directly addresses causality.

Figure 1.

Causal pathways for planning.

First, a plan document may serve as a coordination mechanism in the face of interdependent decisions or high transaction costs (Path A in Figure 1). Here, game theory provides a way of understanding the causal process (Knaap, Hopkins, and Donaghy 1998; Hopkins 2001, 27-28). While a developer and a city might prefer growth to occur in different areas, the worst outcome for both parties is that housing and infrastructure are built in different parts of the city. Thus, once a plan specifies the physical location of growth, there is no incentive for either developer or city to deviate from the plan, and there may be a causal impact on behavior. A similar coordination game applies if there are high transaction costs related to development decisions, as when delays in the development approval process are costly to all parties. Once parties have agreed on the coordination point elaborated through the plan, there is no incentive to deviate—adherence to the plan yields the highest payoff over the long run.

Second, planning may have a causal role through increasing the knowledge available to decision makers (Path B). Here, planning adds value through analytic reasoning and information gathering. Under the rational planning model, which has a long history in the planning literature, planning is a way to uncover new information about the consequences of particular courses of actions. But knowledge can be generated through multiple avenues, including debate as well as technical analysis. Thus, this second pathway includes the process of discovering the preferences of those affected by decisions (Faludi 2000, 304), and knowledge gathered through a collaborative planning process (Innes and Booher 1999, 418). This knowledge-gathering pathway is consistent with the “emergent web” view of planning, in which a web of possibly inconsistent plans influences decisions through the information they contain, but does not directly change authority or power (Donaghy and Hopkins 2006).

Third, planning may shape the preferences of decision makers or their constituents (Path C). In turn, changed preferences become the basis for decisions on both the plan itself, and subsequent policies and projects. This third pathway is implicit or explicit in much of the planning literature, and a classic example of preference shaping can be found in the business-funded Chicago Plan of 1909. The businesses spent more than twice as much money to promote the plan as on the plan itself (Hopkins 2001, 86)—signifying a concerted marketing effort to change preferences. Indeed, the classic quote by the plan’s author, Daniel Burnham, about how plans need to be grand and provide “magic to stir men’s blood” (quoted in Hall 1997, 174), can be interpreted as preference shaping.

Fourth, even if individual preferences remain fixed, the planning process may change how they are aggregated by decision makers (Path D). For example, individuals who participate in the planning process and attend public hearings may have their preferences weighted more highly. Alternatively, planning may provide a frame of reference or function as an agenda-setting device that focuses attention on specific issues (Mastop and Faludi 1997; Hoch 2007, 96). Or the planning process may generate agreements that are “less likely to produce unhappy stakeholders who might sabotage implementation” (Innes and Booher 1999, 414). Through ensuring procedural justice, planning may assuage opposition or allow decision makers to proceed in the face of continued resistance.

The fifth potential causal pathway relates to costs of noncompliance that are incurred because a plan exists. The pathway includes legal enforcement, but also reputational costs. Here, there is a parallel with the international relations literature, which provides evidence that reputational concerns deter nation-states from violating treaty provisions (Simmons 2000). Plans may function in the same way, if departures from plans are viewed unfavorably by a decision maker’s constituents (Path E). Thus, the adoption of a plan introduces a reputational cost to reversing decisions later on.

Climate Action Planning: Empirical Setting

The Development of City Climate Action Plans

The empirical work presented here examines city climate action planning in California. This example has several features that simplify the analysis. First, the plans have a single goal—reducing greenhouse gas emissions. Second, as the plans are voluntary, my sample can contrast “with plan” and “without plan” cities. Third, most climate action plans, at least in California, are similar in structure; cities usually join the Cities for Climate Protection (CCP) campaign and follow a series of five milestones, including an inventory of greenhouse gas emissions, adoption of an emission reduction target, and adoption of a climate action plan. California accounts for more than one-quarter of CCP members in the U.S. (ICLEI 2011), including some of the earliest adopters.² I concentrate on mitigation planning (i.e., planning to reduce greenhouse gas emissions), and ignore climate adaptation planning, which is in a more embryonic phase (Wheeler 2008).

Impacts of Climate Action Planning

I provide a more extensive review of the climate planning literature in Millard-Ball (2012). In summary, much work has used climate planning to illustrate broader theoretical interventions on transnational environmental networks or scales of environmental regulation (Betsill and Bulkeley 2007; While, Jonas, and Gibbs 2010). Other authors have asked why cities adopt a climate plan (Zahran et al. 2008), or assessed the plan documents (Wheeler 2008). The limited research on the impacts of climate planning (Bulkeley and Betsill 2003; Kousky and Schneider 2003; Burch 2010) has focused on implementation rather than on causal role, and qualitative work has lacked apparent purposeful case selection.

In a companion paper (Millard-Ball 2012), I address this gap in the literature using quantitative methods. I present data from all 478 California cities on eight indicators of climate action, including the number of green (LEED-registered) buildings in a city, the number of residential solar photovoltaic systems, and expenditure on bicycle and pedestrian facilities. Cities that have climate plans perform better on all eight of these indicators compared to cities that have not even joined CCP, suggesting that climate plans are at least to some extent being implemented.

However, my statistical results provide little robust evidence for a causal role from planning, as would be indicated by statistically significant coefficients attached to climate planning variables (joining CCP, preparing an inventory, adopting a target, and adopting a climate plan). Instead, environmental preferences—as measured by voting on environmental initiatives, membership in civic environmental groups, and employment in carbon-intensive industries—have greater explanatory power. In short, the decision to pursue a climate plan reflects existing environmental preferences, but those preferences also lead a city to adopt emission reduction measures independently of its planning efforts.

The advantage of this type of quantitative analysis over case studies is generalizability. However, regression does little to explain why climate planning has little causal impact. Moreover, these regression results assess the average impact of planning and are less suited to identifying heterogeneous impacts—for example, if planning has only been successful in some cities. In order to probe causal pathways and assess the validity of my statistical results through triangulation (Tashakkori and Teddlie 1998, chap. 3), I pursue a qualitative approach in this paper.

Case Selection

Mixed-methods research enables systematic case selection that mitigates potential bias (Geddes 1990) and increases the chances of learning something valuable from the cases. Careful case selection can enable cross-case analysis to support causal inferences, on top of the within-case analysis that traces causal processes, and is an inherent strength even of single-case research designs (Collier, Mahoney, and Seawright 2004).

I select cases using the regression residuals from one indicator of climate mitigation, in line with the “deviant case” approach (Seawright and Gerring 2008). Choosing deviant cases allows for cross-case comparison of causal mechanisms on two dimensions: (1) between two cities that perform above expectations on environmental indictors, where only one city has a climate plan, and (2) between two cities with climate plans, where only one performs above expectations. The choice of deviant cases also supports within-case analysis; deviant cases “often reveal more information because they activate more actors and more basic mechanisms in the situation studied” (Flyvbjerg 2006, 229).

The indicator I choose is the number of LEED-registered projects. Given that green building is central to almost all California climate plans, an increase in the number of LEED projects can be seen as an intermediate step between planning and actual emission reductions from green buildings.³ Figure 2 shows the actual number of LEED projects in a city against the prediction of the regression model in Millard-Ball (2012). The further a case from the 45-degree line, the worse the prediction and the more deviant the case. By using the regression residuals, rather than looking for extreme values on the dependent value, the selected cases are deviant even when controlling for environmental preferences and other covariates included in the regression model. In other words, I select cases that perform above or below expectations, rather than cities that perform well or poorly in absolute terms.

Figure 2.

Regression residuals and case selection.

I use the following procedure to select case studies. First, I eliminate cases where the predicted number of LEED projects differs from the actual number by fewer than four projects in absolute terms or 50 percent in relative terms (the shaded area in Figure 2). Second, I eliminate cases with extremes on other key variables such as population and employment. From the remaining cases, I choose those with the largest absolute residuals (difference between actual and predicted values), and where personal contacts could facilitate introductions to key informants. I choose one case from each of the three categories in Table 2. The cases are identified by pseudonyms to maintain the anonymity of interviewees; brief vignettes of each case follow in order to provide context for the empirical results.

Table 2.

Case Selection Strategy.

	Actual vs. Predicted Number of Green Buildings
Climate Planning Program?	Large Positive Residuals (Better-Than-Predicted Environmental Performance)	Small Residuals	Large Negative Residuals (Worse-Than-Predicted Environmental Performance)
Yes (with plan)	Suburbiton	–	Collegetown
No (without plan)	Greenacres	–	N/A

Note: For “With Plan” cases, I selected from cities that had adopted an emission reduction target by December 2008 and had completed at least a draft climate plan. “Without Plan” candidate cases were cities that had not joined CCP or begun a climate plan outside CCP by December 2009. I do not choose a city from the lower-right cell, because of the challenges in gathering data where neither a climate plan nor green buildings exist. Even if appropriate informants could be identified, it would be difficult to ask meaningful questions. Moreover, such cases are likely to be irrelevant for identifying causal processes (Mahoney and Goertz 2004).

Case Vignettes

Suburbiton

Suburbiton has an adopted climate action plan, and its environmental performance is considerably better than predicted by the regression model. Suburbiton is a “most likely” case that is well suited to falsification of propositions—if climate planning has no impact here, it is unlikely to have an impact anywhere. While the city does not mandate that new commercial buildings be LEED certified, many developers have pursued green buildings on a voluntary basis. The city partly serves as a bedroom community for San Francisco and other cities in the Bay Area, but also has an industrial base centered on the technology industry. From a statewide or national perspective, the city’s politics are left wing, but they are fairly mainstream by the standards of the Bay Area region.

Collegetown

Collegetown has also developed a climate action plan, but in contrast to Suburbiton has lower than predicted environmental performance. This makes the city a “least likely” case for planning to have an impact. Discovery of an impact in Collegetown would strongly suggest that climate planning does play some causal role. Lower than predicted environmental performance does not imply that the city is an environmental laggard—on the contrary, Collegetown has a statewide reputation for its green efforts, and its citizens overwhelmingly support environmental and other liberal causes. However, the city’s record on green buildings is lower than the regression model predicts taking into account voting patterns, rates of growth, and other variables. Lying outside the state’s major conurbations of the San Francisco and Los Angeles regions, Collegetown’s economy is dominated by the local university.

Greenacres

Greenacres does not have a climate plan, but its environmental performance is considerably better than predicted by the regression model. Interviews in Greenacres revealed that the city would like to develop a climate action plan in the future, but had not yet embarked on this process because of staff turnover and budgetary considerations. This case thus sheds light on other influences on green building that might confound inferences of the impact of climate planning, and approximates a counterfactual to suggest what Suburbiton and Collegetown might have looked like had they not pursued a plan. Greenacres is located outside California’s major metropolitan areas. Its economy is based on tourism and natural resources, and so environmental issues are of considerable local salience. However, concern has historically been focused on growth management and the preservation of habitat and public open space, rather than on global environmental issues.

Data Collection and Analysis

Initial data collection involved a search of local newspaper archives (via the Access World News database) and municipal websites using the keywords ICLEI, climate, and greenhouse. (ICLEI is a nonprofit organization that runs the Cities for Climate Protection campaign.⁴) I also reviewed climate action plans and emission inventories, and relevant staff reports for city council meetings. The newspaper articles and official documents formed the basis for identifying potential interviewees and asking informed questions that related to specific events or policies.

My primary source of qualitative data consists of interviews with city staff members, elected officials, environmental advocates, developers, and staff from regional agencies. I initially contacted staff with responsibility for climate planning or green buildings, and individuals who were quoted in newspaper articles or who had served on climate planning working groups. After each interview, informants were asked to recommend other potential interviewees as part of a “snowball” sample.

I conducted twenty-three formal, semistructured interviews with twenty-six individuals (see appendix) between August and December 2010, plus a dozen more informal, unstructured interviews at workshops or other events. Three interviews were conducted by telephone, and the remainder in person. Most interviews lasted about an hour (see appendix). A standard protocol, available from the author on request, was used to guide each interview, and in most cases the protocol was customized to focus specifically on the interviewee’s area of first-hand experience. I asked about motivations for undertaking climate planning; the planning and implementation process; and the impacts of the local climate plan and climate policy in general on specific policy and project decisions. Interviews were coded and analyzed using TAMS Analyzer software. I also observed City Council meetings in person or via webcast.

The Causal Role of City Climate Action Planning

When asked specifically about the impacts of climate planning, interviewees had mixed views. Some considered it a valuable tool to boost the profile of mitigation measures, while others suggested that climate planning was having little to no effect. However, these perceptions are of limited value in drawing conclusions. First, interviewees may not be able to imagine a counterfactual state of the world without climate action planning—a prerequisite for understanding causality. Second, many interviewees found it difficult to separate the impact of climate planning from the motivation provided by climate change more broadly, that is, the overall policy desire to reduce emissions.

Therefore, this section structures the empirical findings around how interviewees envision climate planning to be having an impact, using the typology of causal pathways presented earlier. Interviewees were not asked directly about each pathway—not least, because they are somewhat abstract. Instead, the pathways are used as a lens to organize the responses. Table 3 shows the types of evidence that I interpret as being consistent with a particular pathway.

Table 3.

Operationalization of Causal Pathways.

Pathway	Types of Evidence Consistent with Pathway
A: Coordination of interdependent decisions	Climate planning helps different actors or agencies coordinate efforts Interdependence of climate mitigation and other policies or regulations High transaction costs in changing or not implementing a measure
B: Knowledge gathering	Identification or prioritization of mitigation strategies Generation of policy ideas from across departments Availability of dedicated staff to transmit information Research on best practices in other cities
C: Shaping preferences	Staff, elected officials, or residents change their opinions on climate change, or on the feasibility of local mitigation action Increased urgency or motivation to act on climate change
D: Reaggregating preferences	City council or staff pay greater attention to those calling for local climate action Climate change rises up the local policy agenda
E: Costs of noncompliance	Calls to implement measures primarily because they are “in the plan” Staff or elected officials shamed by failure to implement plan

Path A: Coordination of Interdependent Decisions

Interviewees did not mention anything that could be construed as related to the first pathway—coordination in the face of interdependent decisions or high transaction costs. This is unsurprising, as climate plans typically do not address interdependent decisions in the same way as physical land-use plans. Nor are there likely to be high transaction costs from revisiting policy decisions. For example, even if a climate plan includes a measure to “adopt a mandatory green building ordinance,” this policy would be reconsidered by a city council when approving the detailed ordinance.

Path B: Knowledge Gathering

The second pathway—a knowledge-gathering process characterized by cognitive effort, fact finding, and mutual learning—is clearly applicable to climate action plans. It is the ostensible way in which the plans are implemented under the CCP campaign’s “Five Milestone” framework through which a local government can uncover which courses of action will achieve a desired emissions goal.

This knowledge-gathering pathway was by far the most common way in which interviewees suggested climate planning was having an impact. In some cases, interviewees considered the actual plan and related documents to be the main influence. Informants believed that the plan provided a roadmap (Suburbiton); helped develop budget priorities (Suburbiton); provided a way to track implementation over time (Collegetown); or helped justify staff time allocated to greenhouse gas reduction (Collegetown). The process of conducting an emissions inventory and developing a climate plan also helped to focus attention on larger sources of emissions such as the transportation sector. According to one City Council member in Collegetown, by highlighting the sectors that contribute most to emissions, the emissions inventory counterbalances the tendency to focus on smaller-scale environmental efforts such as recycling ink cartridges (Interview 1).

Other interviewees considered the planning process to be more important in knowledge gathering than the plan itself. Above all, the process allowed staff to get out of their offices and learn from colleagues in other departments and communities. This was a particular boon for those outside of large metropolitan areas, as in Collegetown, where informants talked of the value of conferences and networking groups in exposing staff to best practices that could be implemented locally.

Interdepartmental working groups were another way in which the climate planning process allowed staff to collaborate and learn from each other. An informant in Suburbiton related how climate planning triggered “green teams” that brought staff together from multiple departments (Interview 15). In Collegetown, meanwhile, emission inventories promoted similar types of collaboration:

It was almost immaterial or irrelevant what their final inventory numbers were. It was the process of going through the steps of gathering the information that required somebody to leave their office, go down the hall, and talk to someone they would never normally talk to. And then that exchange of information opened other opportunities for communication and collaboration within the city. (Interview 7)

In these ways, the interviews provide evidence consistent with the knowledge-gathering pathway. However, three lines of analysis discussed in turn below—triangulation, temporal sequencing, and the approximate counterfactual—suggest that much of the impact would have occurred even in the absence of climate planning. In particular, some interviewees may conflate action due to general concern over climate change with action due to climate planning—for them, this did not seem to be a meaningful distinction.

Triangulation involves comparing the stated opinions of informants to those of other interviewees or documentary evidence. One example is the claim that climate planning in Collegetown helped to prioritize mitigation in the transportation sector, once the emissions inventory revealed the sector’s large share of emissions. Yet six interviewees, from all three case study cities, singled out transportation as a difficult place to make progress. Along with recycling, transportation is the area where climate planning has had the least impact, according to a staff member in Collegetown (Interview 5). “I don’t really see climate having a big impact yet in the way that the council makes [transportation policy] decisions. I don’t think it’s been factored in as much as it ought to be,” added a council member in Suburbiton (Interview 11).

Temporal sequencing asks whether changes attributed to climate planning occurred before or after the start of the planning process. In Suburbiton, interviewees credited much of the impact of climate planning to their city’s environmental staff person, who helped make connections to energy efficiency resources beyond the city, and expanded recycling programs. “[He] helped us go from ‘want to do’ to ‘know how to do,’” said a developer (Interview 14). “I see him as the driving force behind a lot of what you’re talking about. He’s a very enlightened, well-informed and effective person,” added a City Council member (Interview 11). However, the staffer had been hired well before the advent of the climate planning process, and so it is more appropriate to attribute his efforts to the environmental concern that led the city to hire him in the first place. Indeed, while interviewees in Suburbiton talked extensively about the progress the staff member had made on implementing greenhouse gas reduction measures, when asked specifically what measures were due to the plan, the answer was: “We haven’t had a whole lot . . . basically the plan has been on hold [since adoption]” (Interview 12).

The approximate counterfactual provided by the “without plan” case of Greenacres also helps assess the extent to which the initiatives cited by informants would have happened in the absence of climate planning. In Greenacres, an environmental staff person performed a similar role to his counterpart in Suburbiton, and interviewees made similar comments about the importance of his work—further evidence that it is difficult to attribute the work of an environmental staff person to a climate plan. Moreover, Greenacres formed an interdepartmental environmental working group analogous to those in Collegetown and Suburbiton, and a staff member described its work in a comparable manner:

We sat around and brainstormed, came up with some ideas, gave people action items, they walked away, they got things done, you come back to report. And that’s one of the positive things about a committee with a lot of people doing action items . . . there’s a little bit of accountability. (Interview 19)

This is not to question the value of forming such an interdepartmental working group, or the hiring of an environmental staff person. Rather, the fact that these initiatives take place and are described in the same way in both “with plan” and “without plan” cities raises questions about whether they can be causally attributed to climate planning.

Path C: Shaping Preferences

The third pathway—shaping the preferences of decision makers and their constituents—is another potential way for climate action planning to have an impact. Taking action to reduce greenhouse gas emissions—at least to a substantial level—involves significant costs, while the benefits are diffused globally. It would seem that climate planning would lead to increased mitigation if it were to shape preferences in an altruistic direction.

In both Suburbiton and Collegetown, the plan was developed with the help of community and stakeholder participation at public meetings, or through a working group involving business leaders, residents, city staff, and elected officials. Interviewees in both cities highlighted the importance of the planning process in creating a vision that inspired staff and elected officials, and helped them to see the global importance of reducing greenhouse gas emissions.

Neither case study, however, provides strong evidence that the climate planning process has significantly affected preferences, and none of the types of evidence listed in Table 3 were mentioned by informants. First, this is because preferences in both cities were already strongly in favor of reducing emissions before plan-making began. Interviewees in both Suburbiton and Collegetown talked of a supportive City Council, of motivated staff, and of residents who valued climate planning work. And if climate planning’s impact occurs through awareness raising, then this impact is likely to be negligible in relation to the effect of media coverage, peer effects, and other channels. In other words, a resident is likely to hear far more about climate change, and form his or her opinions, from the news media and from family and friends than from a climate planning process.

Second, the lack of an impact through preference shaping appears to be because climate plans have steered clear of controversial or costly measures that would secure deeper emission reductions. Thus, planning has not enabled preferences to be shaped through argumentation and debate, as it has in contexts such as planning and environmental review for dam removal and forest management (Taylor 1984). Climate planning in the case study cities does not appear to have affected preferences over simpler, low-cost mitigation actions, as there was broad consensus over these in the first place.

One developer noted that the plan in Suburbiton focused on goals and ideas rather than potentially more controversial mandates for green buildings. The plan “didn’t have a lot of mandates so there wasn’t a lot of controversy,” he said, adding that there was little disagreement within the committee that developed the climate action plan (Interview 14). In similar vein in the same city, a council member suggested many of the measures in the climate plan could easily be on the consent calendar, a mechanism that allows routine items to be approved by Council without discussion (Interview 11).

There is certainly the possibility that climate planning affected the overall prioritization of emission reduction projects, as the council member went on to suggest. But both the temporal sequencing and the approximate counterfactual suggest otherwise. The measures that she referred to—primarily green buildings for municipal facilities—were already covered by a more detailed City Council policy to “build green” that predated the climate planning process. When taking decisions to fund specific green building and renewable energy projects, staff and council members in Suburbiton invoked both the green building policy and the climate plan, along with a more general desire for the city to show environmental leadership. Thus, while climate planning may have added some support at a later date, particularly in the face of fiscal constraints, it seems likely that individual green building projects would have gone forward regardless. Moreover, similar municipal buildings in the “without plan” Greenacres were built with rooftop solar and other green features, and staff explained these decisions using similar language to interviewees in Suburbiton.

Staff has also tried to avoid opposition by presenting climate plans as a restatement of existing city policy, rather than a foray into new and potentially controversial policy areas. “This is not a plan that establishes policy,” but instead is a plan that reiterates policies already adopted by the city, was how a staff member presented the city’s draft climate plan to Collegetown City Council. As well as smoothing the way for council adoption, such a strategy of avoiding controversy may help ensure buy-in from staff in other city departments—again, at the expense of allowing a preference-shaping process to occur. At the extreme, such buy-in is achieved through ensuring that the plan consists of measures that potentially recalcitrant departments planned to pursue anyway. One staff member in Collegetown explained how she encouraged the transportation department, which was not interested in reducing emissions, to reframe its existing priorities in climate policy terms:

I keep telling them . . . stop saying you’re not going to do [climate change], and start figuring out which things that you are going to do will benefit climate as well. . . . Next time they come and talk to you about [climate], say what we’re going to do is congestion relief projects, we’re going to do signal timing projects . . . and all those things are going to do climate relief because they’re going to reduce congestion. . . . That’s the kind of thing I try to subtly do, is say you’re not going to get away from this stuff [climate change] so find some part of it that makes sense to do and do it. (Interview 5)

In this example, there is again a possibility that climate planning has affected project prioritization in subtle ways. But the broader implication is that environmental staff—who one might expect to be the biggest advocates for change—has encouraged reluctant colleagues to simply rebrand already-planned measures as climate mitigation. City environmental staff, at least in Collegetown, has generally opted to take the path of least resistance, which streamlines the planning process but shies away from the conflict and debate that provide a forum for preferences to be shaped.

Path D: Reaggregating Preferences

The fourth pathway—changing how decision makers aggregate the preferences of their constituents—provides another plausible way for climate planning to have a causal impact. The existence of a physical document may allow climate action proponents to weigh in during municipal budget and policy deliberations, through figuratively or literally waving the climate plan around. Public hearings and other meetings may allow the proponents to set the agenda in favor of greenhouse gas reductions, and to have their opinions clearly heard by decision makers.

Certainly, several interviewees have seen climate planning as a way to advance a particular agenda, and have become involved for that reason. Recycling and public transportation were cited as two issues where proponents saw value in joining the climate-change “bandwagon.” However, climate change is already on the agenda in all three case study cities, and advocates have not needed to invoke the plan when calling for the implementation of specific policies. As one staff member in Collegetown says:

Very rarely have I needed to get out the books and say, hey, here’s where it says [in the plan, that we need to implement a measure], because not only has Council been so supportive but the community has been informed, they’ve been involved and they’ve been invested, so it takes a lot of the covering-my-tail work out of it. (Interview 3)

Similar sentiments came from a planning commissioner in Suburbiton. “No. I’ve never seen that,” he replied when asked whether the plan had ever been invoked before City Council to provide political support for greenhouse gas reduction measures (Interview 13). The possible exception is the green buildings example discussed above under Path C, where both the climate plan and the city’s green building policy were cited by staff and council members, but even here it was a preexisting city policy that ensured that the item was on the council’s agenda.

While climate planning has the potential to force climate change onto the agenda, in practice city decision makers already see climate change as a priority. As discussed below, city councils in all three case study cities, and particularly Suburbiton and Collegetown, were strongly supportive of greenhouse gas reduction measures, whether or not the measure was in the climate plan.

Path E: Costs of Noncompliance

Climate plans are voluntary, and no legal action has been taken to date to enforce their provisions. And while reputational costs for failing to implement a plan are a plausible way in which climate plans might have an impact, they have not been a factor in practice. First, climate plans have not had a high profile; if few people are aware of them, then reputational costs will be correspondingly low.

Second, the nature of climate planning makes it difficult to assess whether a plan has lived up to its promises. Commitments in a climate plan are rarely of a binary nature, where the measure is either implemented or not implemented. Instead, more typical of climate plans are measures such as “encourage mixed-use, infill and higher density development” and “identify opportunities for wind energy generation.”⁵ While the emissions reduction target provides a quantitative benchmark against which a city’s performance can be measured, the target years usually lie ten to twenty years or more in the future. In addition, whether a city reaches a target or not will depend greatly on factors outside of its control, such as changes to the electricity generation mix and vehicle fuel economy improvements. This “noise” will make it difficult to ascertain whether a city has lived up to its own commitments (for a detailed discussion, see DeShazo and Matute 2012).

Impact of Preferences

To claim that climate planning has little impact on city decision-making and environmental outcomes does not imply that cities are irrelevant from a climate policy perspective. Rather, it appears that the environmental preferences of elected officials, city staff, residents and business owners go some way to explaining the costly actions that cities are taking to reduce greenhouse gas emissions. In this section, I use the example of green buildings to illustrate my findings, but a similar argument can be made with respect to transportation, energy, or waste policy.

In the context of development decisions, a planning commissioner in Suburbiton explained how the city’s approach to green buildings owed more to his colleagues’ preferences than to climate planning:

I don’t think it’s the [climate] plan that’s having an impact. I think it’s having knowledgeable planning commissioners that embrace the vision of the plan. It’s more the vision of the plan than the climate action plan that’s driving things. (Interview 13)

Interviewees in both “with plan” and “without plan” case study cities made similar comments about the importance of environmental preferences in promoting green buildings. The number of green buildings is partly a product of market interest—in other words, environmental preferences among potential home purchasers and commercial tenants. In a progressive community, green building certification can also help to assuage neighbors and elected officials concerned about traffic and similar nuisances. “It does help with the neighbor complaints,” said a developer in Collegetown, adding with laughter: “For some reason solar makes people stop complaining about things . . . ‘Oh, you’re putting solar up, oh you guys are OK’” (Interview 9).

In Greenacres, one staff person explained the benefits of LEED certification in similar terms:

It [LEED] actually helped with the neighbors, [and] it helped the Planning Commission and everyone to accept the project and embrace it and say, “We’re going to have another example project that we can say is more efficient and better for the community.” (Interview 20)

Local developers also pursue green buildings because of their own values. A developer in Suburbiton spoke of how pressure from his children pushed him to incorporate more green features:

My middle school child came home last night and told me that her carbon footprint was 5.4, which meant we needed 5.4 planets to support her lifestyle, and that was really scary and what was I going to do about it? When parents get asked those questions in their home, they take that question to the business . . . maybe you should change your business practices. (Interview 14)

Thus, preferences translate into green buildings through at least three avenues that are independent of climate planning—market demand, assuaging neighborhood opposition to development, and the values of the developers themselves (or at least of their children). More broadly, the types of cities that are developing climate plans are the same types of cities that are pursuing specific measures to reduce emissions. In Collegetown, the City Council was eager to adopt almost any environmental program that the staff proposed:

Council is always interested. People always ask me: “How do you get your Council to do that?” I often say, it’s a matter of getting them [Council] to not do things that we [staff] really aren’t ready to do yet. They’re very eager. They’ll do whatever. If we go in and say, “Here’s this great environmental program, we should do this,” they’ll do it. Whether they have money or staff or not, they’ll do it. And honestly it’s a bit of a problem sometimes because they’ll say we want this and we’re like, “We can’t really do that.” (Interview 5)

In other words, city decision-making takes place in the context of residents, staff, and elected officials who are eager to contribute to greenhouse gas reduction—regardless of whether a city has gone through a planning process or adopted a plan that assembles discrete policies and projects into an overall package.

Conclusions

Given the centrality of causation to claims about what planning can do, it is surprising that there is little work drawing together the theoretical mechanisms by which planning can have causal impacts (Hopkins 2001 being one of the main exceptions), and even less empirical evidence as to whether it does. In the case of municipal climate planning, the preferences of residents, business owners, city staff, and elected officials over environmental policies appear to offer a more convincing causal explanation than does planning for the implementation of specific measures to reduce greenhouse gas emissions.

In this paper, I use a typology of five potential causal pathways as a lens to interpret interview data, as summarized in Table 4. Path A is implausible in the case of climate planning. Path E is more promising, but the low profile and long time horizons of climate plans limit the extent to which reputational costs are incurred from nonimplementation. The remaining pathways are plausible—for example, knowledge gathering can be fostered through analysis of emissions data and learning from colleagues in other departments and other cities. But there are two primary reasons to question the causal role of climate planning in this regard: temporal sequencing and the approximate counterfactual. Regarding temporal sequencing, actions or efforts that were attributed by interviewees to climate planning, such as a green buildings policy or hiring of environmental staff, had often taken place well before the climate planning process had started. Regarding the approximate counterfactual, the “without plan” case of Greenacres suggests that benefits attributed to climate planning—such as exposure to best practice in other cities, and the creation of new bureaucratic processes—may well have been obtained anyway.

Table 4.

Evidence for the Impact of Climate Planning.

Pathway	Evidence
A: Coordination of interdependent decisions	No evidence. Plans have not addressed interdependent decisions.
B: Knowledge gathering	Most likely pathway for planning to have an impact, e.g., through exposure to best practice and interdepartmental working. But knowledge-gathering mechanisms preceded the planning process, and/or also exist in the “without plan” case of Greenacres.
C: Shaping preferences	No evidence. Preferences were already strongly in favor of climate change, before the climate planning process began, and planning has steered clear of controversial issues.
D: Reaggregating preferences	No evidence. Climate planning has not performed an agenda-setting role, as decision makers in case study cities already see climate change as a policy priority.
E: Costs of noncompliance	No evidence. Reputational costs are limited by the low profile and long time horizon of climate plans.

Fundamentally, it is concern over climate change, rather than climate planning, that is behind municipal policy decisions. Many measures in climate plans have been implemented, but were sometimes attributed post hoc to the plan even if they had been in place before work on the plan had begun. Indeed, interviewees sometimes had difficulty in disentangling the impact of planning from the climate change policy motivation. Moreover, city staff and elected officials often talked of climate planning in terms of gaining recognition from local residents or state officials for their city’s existing climate policy efforts. If the motivation for climate planning is primarily to compare favorably to neighboring cities, to showcase a city’s existing efforts, and to defuse political pressure to do more on climate change, it is perhaps unsurprising that there is limited evidence that planning has a causal impact. The climate plan becomes more of a marketing device than a template for action, helping a city to gain a (probably deserved) “green” reputation for action that predated or occurred independently of climate planning.

Suburbiton is a “most likely” case—a city where regression analysis suggests climate planning is most likely to have had an impact. But even here, interviewees explicitly stated that the plan “has been on hold” since it was adopted, or was “an academic exercise.” Again, this evidence is not definitive, and it is impossible to observe the counterfactual of what would have happened without climate planning (even if Greenacres does provide some indications). But the lack of robust evidence for impacts in the “most likely” case does not augur well for the effectiveness of climate planning elsewhere.

The conclusion that climate action planning in its current form has little or no impact does not imply that it can never do so. A long-term evaluation is beyond the scope of this paper, and it is certainly possible that impacts will eventually occur. A possible analogy is provided by the National Environmental Policy Act, which in the long term allowed a generation of organizational “analyst advocate” entrepreneurs to shape projects to incorporate environmental concerns (Taylor 1984). Moreover, as climate planning is not currently mandated, it is impossible to observe its effects in recalcitrant cities that neither develop a plan nor undertake mitigation measures on a voluntary basis. It is conceivable that climate planning might spur climate action in places that would otherwise do little or nothing in this regard. And climate plans may still be worth making—even if the impacts are very modest, this is not a problem if the costs are minimal as well. But climate planning should not be taken as an indication of a new wave of local government action to reduce emissions; of a “Plan B” in the face of federal inertia on climate policy; or that cities have overcome the “commons” or collective action problem in dealing with climate change (as claimed by Lutsey and Sperling 2008, 682-83).

What are the implications for other types of plans typically developed by local governments, such as neighborhood plans or comprehensive plans? Is it the idiosyncratic features of climate planning—in particular, their voluntary nature, the difficulty in assessing progress toward emission reduction targets ten or more years in the future, and the collective action problem that hinders climate change mitigation at all levels of government—that limit its utility, or do similar challenges arise with other types of planning? While the results of this article do not generalize beyond climate action planning, it can be instructive to consider the circumstances under which other types of planning may have an impact. While causal impacts are only one potential benefit of planning, the idea that communities can use planning to shape their physical and policy future is central to the legitimacy of the planning enterprise.

There is a clear gain from planning if it can solve the coordination problem resulting from the interdependence of decisions. This suggests that Comprehensive Plans and other physical land-use plans are likely to have a causal impact, although even here plans may to some extent simply reflect zoning decisions that have already been made, or codify decisions that would have been made on a parcel-by-parcel basis (Pogodzinski and Sass 1994). Future research might explore whether, as hypothesized, the causal impact of a plan is affected by the interdependence and irreversibility of the decisions that it seeks to affect. Comprehensive plans that focus on physical change and are legally binding are a “most likely” class of plans where causality could be tested—if planning has no impact here, it is unlikely to do so with other types of plans.

Meanwhile, if the role of planning is to increase the supply of information, then it is open to question whether the creation of a plan is the best way for this to occur. Analysis of emissions impacts and searches for best practices can be done on a more informal basis by staff or in the context of legislative action, simply skipping the initial step of drawing up a plan. Planning is only one activity that planners do, and their analytical, consensus building, and implementation work can often proceed outside of the framework of developing a plan.

Planning and preferences are closely intertwined, but there is little empirical work to identify how planning shapes as well as reflects preferences, and future work should explore whether the causal impact of planning is related to the quantity or heatedness of civic debate. One might speculate that controversial planning processes—for example, a transportation plan that includes congestion pricing, or a neighborhood plan that substantially raises height limits—might be more likely to affect outcomes through this pathway. If preference shaping is the main way through which planning has an impact, then the rhetorical power of the 1909 Chicago Plan may be a better model than the more prosaic prose of climate action plans. And if plans are a commitment device analogous to international treaties, then the status of plans needs to be elevated and implementation progress tracked closely, so that reputational costs are a deterrent to deviating from the plan.

The results presented in this paper call for a more nuanced view of both planning evaluation and the role of planning—what planning can do, when it makes sense to plan, and what types of plans are useful. An emphasis on empirical research that uses careful statistical design and purposeful case selection in order to identify not only the causal role of planning, but the mechanisms through which any causal impact occurs, can help broaden understanding of how to make and use plans well.

Footnotes

Appendix

List of Interviewees

Number	Case	Role	Interview Location	Length (minutes)
1	Collegetown	City council member	Collegetown	61
2	Collegetown	City staff	Collegetown	72
3	Collegetown	City staff	Collegetown (joint with no. 4)	62
4	Collegetown	City staff	Collegetown (joint with no. 3)	62
5	Collegetown	City staff	Collegetown	78
6	Collegetown	Environmental advocate	Collegetown (joint with no. 7)	56
7	Collegetown	Environmental advocate	Collegetown (joint with no. 6)	56
8	Collegetown	Consultant	Collegetown	47
9	Collegetown	Developer	Collegetown	43
10	Suburbiton	City council member	Telephone	22
11	Suburbiton	City council member	Neighboring city	61
12	Suburbiton	City staff	Telephone	29
13	Suburbiton	Planning commissioner	Suburbiton	59
14	Suburbiton	Developer	Neighboring city	33
15	Suburbiton	Regional agency	Oakland (joint with no. 16)	53
16	Suburbiton	Regional agency	Oakland (joint with no. 15)	53
17	Greenacres	City council member	Greenacres	64
18	Greenacres	Planning commissioner	Greenacres	65
19	Greenacres	City staff	Greenacres	60
20	Greenacres	City staff	Greenacres	53
21	Greenacres	City staff	Greenacres	38
22	Greenacres	Environmental advocate	Greenacres (joint with no. 23)	52
23	Greenacres	Environmental advocate	Greenacres (joint with no. 22)	52
24	Greenacres	Regional agency	Greenacres	74
25	Multiple	Environmental advocate	Withheld*	65
26	Multiple	Environmental advocate	Telephone	49

Note: City council member may refer to a current or former council member.

In order to help maintain the anonymity of informants, I sometimes change genders in the main text.

Withheld in order to maintain anonymity

Acknowledgements

I am grateful to the many people in the case study cities for generously sharing their time and experiences. This research would not have been possible without their assistance. I also thank Leonard Ortolano, Michael Wara, Andrew Perlstein, Michelle Stahlhut, Mark Massoud, and three anonymous reviewers, along with Bruce Stiftel and other participants at the 2011 JPER Writing Workshop for New Scholars, for their helpful comments and assistance.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The author acknowledges support from a U.S. Department of Transportation Eisenhower Graduate Fellowship, a Landreth Fellowship, and a David and Lucille Packard Foundation Stanford Graduate Fellowship.

Notes

Author Biography

Adam Millard-Ball was an assistant professor in the Department of Geography and McGill School of Environment, McGill University at the time this work was completed, and is now an assistant professor in the Environmental Studies Department at the University of California, Santa Cruz. His research interests are interdisciplinary and span the fields of urban geography, environmental economics, and urban planning, with a particular emphasis on transportation policy and planning.

References

Alterman

Hill

1978. Implementation of urban land-use plans. Journal of the American Institute of Planners 44 (3): 274-85.

Berke

Backhurst

Day

Ericksen

Laurian

Crawford

Dixon

2006. What makes plan implementation successful? An evaluation of local plans and implementation practices in New Zealand. Environment & Planning B: Planning & Design 33 (4): 581-600.

Betsill

Bulkeley

2007. Looking back and thinking ahead: A decade of cities and climate change research. Local Environment 12 (5): 447-56.

Brody

S. D.

Highfield

W. E.

2005. Does planning work? Testing the implementation of local environmental planning in Florida. Journal of the American Planning Association 71 (2): 159-75.

Bulkeley

Betsill

M. M.

2003. Cities and climate change: Urban sustainability and global environmental governance. London: Routledge.

Burch

2010. Transforming barriers into enablers of action on climate change: Insights from three municipal case studies in British Columbia, Canada. Global Environmental Change Part A: Human & Policy Dimensions 20 (2): 287-97.

Chapin

T. S.

Deyle

R. E.

Baker

E. J.

2008. A parcel-based GIS method for evaluating conformance of local land-use planning with a state mandate to reduce exposure to hurricane flooding. Environment and Planning B: Planning and Design 35 (2): 261-79.

Collier

Mahoney

Seawright

2004. Claiming too much: Warnings about selection bias. In Rethinking social inquiry: Diverse tools, shared standards, edited by Brady

H. E.

Collier

, 85-102. Plymouth, MA: Rowman & Littlefield.

DeShazo

J. R.

Matute

2012. The local regulation of climate change. In Oxford handbook of urban planning, edited by Weber

Crane

, 455-76. New York: Oxford University Press.

10.

Donaghy

K. P.

Hopkins

L. D.

2006. Coherentist theories of planning are possible and useful. Planning Theory 5 (2): 173-202.

11.

Faludi

2000. The performance of spatial planning. Planning Practice & Research 15 (4): 299-318.

12.

Flyvbjerg

2002. Bringing power to planning research. Journal of Planning Education and Research 21 (4): 353.

13.

Flyvbjerg

2006. Five misunderstandings about case-study research. Qualitative Inquiry 12 (2): 219-45.

14.

Geddes

1990. How the cases you choose affect the answers you get: Selection bias in comparative politics. Political Analysis 2: 131-50.

15.

Hall

1997. Cities of tomorrow. Oxford, UK: Blackwell.

16.

Hoch

2007. How plan mandates work. Affordable housing in Illinois. Journal of the American Planning Association 73 (1): 86-99.

17.

Hopkins

L. D.

2001. Urban development: The logic of making plans. Washington, DC: Island Press.

18.

Hopkins

L. D.

2012. Plan assessment: Making and using plans well. In Oxford handbook of urban planning, edited by Weber

Crane

, 803-22. New York: Oxford University Press.

19.

ICLEI. 2011. Member list. http://www.icleiusa.org/about-iclei/members/ (accessed December 8, 2011).

20.

Ingram

G. K.

, ed. 2009. Smart growth policies: An evaluation of programs and outcomes. Cambridge, MA: Lincoln Institute for Land Policy.

21.

Innes

J. E.

Booher

D. E.

1999. Consensus building and complex adaptive systems—A framework for evaluating collaborative planning. Journal of the American Planning Association 65 (4): 412-23.

22.

Johnston

R. A.

Schwartz

S. I.

Klinkner

T. F.

1978. Successful plan implementation. Growth phasing program of Sacramento County. Journal of the American Institute of Planners 44 (4): 412-23.

23.

Kinzer

2010. Public participation and planning implementation: Ships passing in the night or tugboat and tanker? Paper presented at Association of Collegiate Schools of Planning Annual Conference, Minneapolis, October 7-10, 2010.

24.

Knaap

G. J.

Hopkins

L. D.

Donaghy

K. P.

1998. Do plans matter? A game-theoretic model for examining the logic and effects of land use planning. Journal of Planning Education and Research 18 (1): 25-34.

25.

Knaap

G. J.

Ding

C. R.

Hopkins

L. D.

2001. Do plans matter? The effects of light rail plans on land values in station areas. Journal of Planning Education and Research 21 (1): 32-39.

26.

Kousky

Schneider

S. H.

2003. Global climate policy: Will cities lead the way? Climate Policy 3 (4): 359-72.

27.

Laurian

Crawford

Day

Kouwenhoven

Mason

Ericksen

Beattie

2010. Evaluating the outcomes of plans: Theory, practice, and methodology. Environment and Planning B: Planning and Design 37 (4): 740-57.

28.

Lutsey

Sperling

2008. America’s bottom-up climate change mitigation policy. Energy Policy 36 (2): 673-85.

29.

Mahoney

Goertz

2004. The possibility principle: Choosing negative cases in comparative research. American Political Science Review 98 (4): 653-69.

30.

Mastop

Faludi

1997. Evaluation of strategic plans: The performance principle. Environment and Planning B: Planning and Design 24 (6): 815-32.

31.

Millard-Ball

2012. Do city climate plans reduce emissions? Journal of Urban Economics 71 (3): 289-311.

32.

Oliveira

Pinho

2010. Evaluation in urban planning: Advances and prospects. Journal of Planning Literature 24 (4): 343-61.

33.

Pogodzinski

J. M.

Sass

T. R.

1994. The theory and estimation of endogenous zoning. Regional Science and Urban Economics 24 (5): 601-30.

34.

Preece

1990. Development control studies: Scientific method and policy analysis. Town Planning Review 61 (1): 59-74.

35.

Rydin

Pennington

2000. Public participation and local environmental planning: The collective action problem and the potential of social capital. Local Environment 5 (2): 153-69.

36.

Seawright

Gerring

2008. Case selection techniques in case study research—A menu of qualitative and quantitative options. Political Research Quarterly 61 (2): 294-308.

37.

Simmons

B. A.

2000. International law and state behavior: Commitment and compliance in international monetary affairs. American Political Science Review 94 (4): 819-35.

38.

Talen

1996a. After the plans: Methods to evaluate the implementation success of plans. Journal of Planning Education and Research 16 (2): 79-91.

39.

Talen

1996b. Do plans get implemented? A review of evaluation in planning. Journal of Planning Literature 10 (3): 248-59.

40.

Talen

1997. Success, failure, and conformance: An alternative approach to planning evaluation. Environment and Planning B: Planning and Design 24: 573-87.

41.

Tashakkori

Teddlie

1998. Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage.

42.

Taylor

1984. Making bureaucracies think. The environmental impact statement strategy of administrative reform. Stanford: Stanford University Press.

43.

Waldner

2008. Regional plans, local fates? How spatially restrictive regional policies influence county policy and regulations. Environment & Planning B: Planning & Design 35: 679-700.

44.

Wheeler

S. M.

2008. State and municipal climate change plans: The first generation. Journal of the American Planning Association 74 (4): 481-96.

45.

While

Jonas

A. E. G.

Gibbs

2010. From sustainable development to carbon control: Eco-state restructuring and the politics of urban and regional development. Transactions of the Institute of British Geographers 35 (1): 76-93.

46.

Winship

Morgan

S. L.

1999. The estimation of causal effects from observational data. Annual Review of Sociology 25 (1): 659-706.

47.

Zahran

Grover

Brody

S. D.

Vedlitz

2008. Risk, stress, and capacity: Explaining metropolitan commitment to climate protection. Urban Affairs Review 43 (4): 447-74.