When plans are used to no effect: Considering implementation performance of greater Brisbane’s compact activity centre policies

Abstract

Evaluations of plan implementation are typically conceived in terms of plan conformance (the degree to which outcomes align with planning intent) or plan performance (the degree to which decision makers apply the plan). In this research, we consider the relationship between performance–conformance by evaluating the implementation of compact activity centre policy in greater Brisbane. We examine two decades of changes to local land use planning regulations using content analysis, quantifications of permitted development intensity, and comparisons of actual land use changes with planned intent, to identify evidence of the use of metropolitan scale activity centre policy. We find that the activity centre policy performed well and was used as intended across the metropolitan area. However, despite strong performance, the policy conformed poorly in terms of achieving the expected outcomes. These findings highlight the importance of understanding plan implementation from both performance and conformance perspectives. While conformance evaluations are essential to connect planning intent to physical outcomes, understanding plan performance is also necessary to explain the results of conformance evaluations and whether conformance successes or failures were due to deficiencies with the plan itself or how the plan was used by key actors charged with its implementation.

Keywords

Activity centres compact city plan evaluation plan performance land use planning

Introduction

Objectives to develop compact cities have become a common feature of planning policies across the Global North and aim to realise a range of urban sustainability benefits (OECD, 2012). Such policies often combine compact city ideals with aspirations for polycentric spatial patterns, with planning authorities across Europe, North America and Australia utilising metropolitan level plans to pursue urban forms featuring an intense urban core, supported by a network of compact activity centres throughout the middle and outer rings of the broader conurbation (Filion, 2009; Forster, 2006; OECD, 2012; Schmitt, 2013). However a growing body of research from North America and Australia shows that these attempts to cluster residential and commercial development around sub-centres is conforming poorly with the planned intent, particularly in car-dependent ‘dispersed suburban’ areas where further compactness could potentially yield the greatest benefits (Brewer and Grant, 2015; Chhetri et al., 2013; Filion, 2015; Filion and Saboonian, 2019; Newton and Glackin, 2014; Phan et al., 2009; Limb et al., 2020a).

Although the work of planning increasingly focusses on sustainability issues with associated strategies for more compact cities and the associated design and guidelines to shape city forms, the realisation of these urban forms and their supposed benefits ‘…is still more in the realm of beliefs than in theoretical arguments confirmed by practice’ (Oliveira and Pinho, 2010a: 357). As such, there is a need for empirical evaluation of the practical application of urban form policies, such as policies attempting to implement the compact city (Oliveira and Pinho, 2010a).

Previous research suggests that planning is struggling to deliver these results and although it provides some explanations for this poor conformance, we argue that a necessary yet often ignored step in explaining plan outcomes requires an evaluation of plan performance to determine whether poor policy conformance is due to issues associated with the nature of the policy and its implementation mechanisms or instead the result of how the plan was used (or not) by key actors. In this paper, we undertake a performance evaluation of the implementation of activity centre plans in Brisbane Australia to build on our previous conformance-based research (Limb et al., 2020a). Through a detailed examination of changes to two decades of planning policy, we add additional empirical evidence to highlight the important role of plan performance evaluation in explaining plan implementation success or failure.

Evaluating plan implementation

Although plan evaluation is a well-established component of the planning process, it continues to be poorly applied in practice (Guyadeen and Seasons, 2018). Studies of plan implementation focus on two fundamentally different concepts of how ‘successful’ plan implementation should be defined: plan ‘performance’ and ‘conformance’ (Guyadeen and Seasons, 2016; Laurian et al., 2010; Loh, 2011; Oliveira and Pinho, 2009, 2010a; Talen, 1997). Plan performance evaluations consider a plan successful if the concepts are taken up by decision makers who have the means to implement plan objectives (Alexander and Faludi, 1989; Faludi, 2000). In contrast, plan conformance evaluation seeks to directly connect plan objectives to changes in the physical world (Alexander and Faludi, 1989; Loh, 2011; Oliveira and Pinho, 2009; Talen, 1997).

Faludi (2006) illustrates this concept in a study that examines the patchy implementation of the European Spatial Development Perspective in terms of how it is used by decision makers in various EU member states. Although the plan was not always followed, Faludi (2006) concludes the strategy was successful as it was consistently referenced in decisions by the countries involved and therefore its ideas had been ‘absorbed’. Conversely, Altes (2006) describes a situation where a plan achieved the initially intended results; however, it was not used in making the decisions necessary to adapt to changing circumstances. The performance-based approach to plan evaluation was therefore justified on the basis that although the plan achieved results in conformance with the initial plan goals, the plan was unsuccessful as these results were no longer those required (Altes, 2006).

Under the performance view, a plan is therefore considered as a ‘message’ to decisions makers and the plan is successful when it is invoked appropriately in decision making (Faludi and Altes, 1994). However, critics hold that decision makers are let ‘off the hook’ when plan conformance is not evaluated, amounting to ‘evaluation avoidance’ (Talen, 1996). The literature indicates that there are benefits to evaluating plan implementation from both performance and conformance perspectives. Some combined approaches to plan evaluation include studies examining plan conformance and plan quality (Brody and Highfield, 2005; Burby, 2003) and, more recently, a number of studies have incorporated measures of both plan performance and plan conformance (Altes, 2006; Berke et al., 2006; Feitelson et al., 2017; Lyles et al., 2015; Zhong et al., 2014).

Plan performance evaluations are typically undertaken either by interviewing (Oliveira and Pinho, 2010b) and/or surveying key policy actors (Feitelson et al., 2017; Lyles et al., 2015), or through document analysis (Altes, 2006; Faludi, 2006; Zhong et al., 2014). With either approach, evaluating planning performance needs to identify three key aspects (Faludi and Altes, 1994: 414–415):

‘… the decisions on which the plan should have had a bearing.’

‘… the commitments which decisions carry, together with the arenas for, and the critics of, their justification.’

… if the ‘plan has … helped in shaping the codes used in justifying subsequent decisions, and [whether] this improved the quality of the justification of decisions in terms of taking account of the wider field of choice.’

While there has been some research that evaluates the conformance of activity centre policies (see Introduction), there is an absence of empirical evaluations of these policies which combine conformance and performance evaluation as suggested by plan evaluation theory.

The case of greater Brisbane

Brisbane is Australia’s third largest city and represents a typical case of Australian metropolitan policy for compact activity centres. The policy has been in place via a series of regional scale plans for more than 20 years. Brisbane’s activity centre policy attempts to manage growth to create more polycentric built forms, based around a strong hierarchy of sub-centres and a dominant central business district. This is a relatively common policy response and approach and shared by several cities globally including Stockholm, Helsinki, Warsaw, Toronto, Portland and Vancouver¹ (Filion, 2009; Schmitt, 2013).

In terms of governance and implementation, however, Brisbane shares more in common with North American examples where activity centre policy is driven by a ‘powerful’ metropolitan level authority (e.g. the state government) and implemented largely through changes to the regulatory land use planning system. In Brisbane, from 2005 onwards, local governments have been required to incorporate the general principles of the activity centres into their land use regulations, which typically consist of mapped zoning ordinances and the like. Prior to this (from 1996 to 2005), local governments undertook such changes voluntarily as in common with many European metropolitan planning governance systems (Schmitt, 2013). Our study covers both of these periods (as discussed further in the Results), throughout which local governments have maintained significant levels of discretion in how they may interpret and apply activity centre policy to their land use regulations, with this discretion being subject to final approval by the state government.

Under this system, local governments are also delegated powers from the state government to make decisions on the application of land use regulations. In most instances, compliance is determined by local governments either through granting of a development permit for a specific proposed use or through the creation of regulations that automatically permit certain forms of development provided they meet specific criteria. Individual development proposals are assessed against qualitative statements of planning intent rather than purely against prescriptive rules using a ‘performance-based’ approach as also often used in Great Britain and sporadically attempted in the United State and Europe (Baker et al., 2016).

Following the Faludi and Altes (1994) framework outlined above, the case of Brisbane allows for a plan performance evaluation by considering changes to local government land use regulations. Such changes require significant commitments to a future land use pattern as outlined in the metropolitan level activity centre policy. Zoning decisions give rights in land, are legally binding and reversing direction through the subsequent removals of rights can be politically and economically costly. Queensland planning legislation also requires public consultation to be included in the process of rezoning land, giving voice to a range of potential critics or supporters of proposed changes. Regulatory land use plans also formalise the framework for subsequent decision making on individual development applications for land use change.

The activity centre policy would therefore be performing when local governments use the plan to make regulatory land use decisions on sites in the activity centres. We therefore search for evidence of decisions that enable the types of uses proposed by the activity centre policy. Where evidence can be found, we describe the result as being positively performing, i.e. decisions were made in accordance with the planned intent.

With two exceptions,² we selected all the principal and major regional activity centres nominated in the South East Queensland Regional Plans (SEQRPs) for the greater Brisbane area as shown in Supp Fig 1.³ The extent of each centre is defined as all properties within a 1200-m walkable catchment from each centre’s key public transport node.⁴ Our previous research demonstrated that activity centre policy had poor conformance in greater Brisbane (Limb et al., 2020a). Understanding whether the plan was used as intended when making planning decisions adds richness to these results, begins to address why outcomes did not conform to the plan, and demonstrates the value of evaluation of both performance and conformance to explain plan implementation. In the case of greater Brisbane, we find strong performance with poor conformance, which indicates planners need to reconsider the policy’s selected implementation mechanisms.

Methods

Content analysis of planning documents

We first examine local government land use regulations for the presence of terminology and references to metropolitan planning policy. The regulations in force across a 20-year period in 1996, 2006 and 2016⁵ were examined to determine if regional planning policy for activity centres had been incorporated into the subordinate local plans over time. In total, 26 regulatory documents were reviewed to determine if they specifically referenced the relevant regional plan for the time period or if they incorporated regional planning terminology. Relevant planning document references to regional policies for activity centres were assigned one of four categories:

No reference to regional policy.

Reference to regional policy, but not in relation to activity centres.

Regional policy for activity centres is described at a strategic level; however, the centre is not specifically nominated (i.e. the centres are not listed or named directly as regional activity centres).

Regional policy for activity centres is specifically discussed and the centre is specifically nominated (the individual centres are named directly as regional activity centres).

Where regional planning terminology for activity centres is specifically referenced in the regulations, the regional plan is considered to have been used in decision making for the development and adoption of the regulations.

Quantification of development intensity

The compact activity centre policy intends to create a network of centres characterised by higher density residential uses, a greater diversity of housing types and mixed clusters of uses that generate employment and provide localised services (Limb et al., 2020a). As the principal implementation mechanism for this policy, land use regulations would be expected to reflect these objectives if used as intended. Where regulatory plans exhibit these forms of change over time, combined with the incorporation of activity centre terminology, we consider them to be positively performing, i.e. that decisions were made to incorporate regional policy into subordinate plans.

We measured changes in the intensity of development permitted by the land use regulations effecting the centre extents by assessing land use regulations over a 20-year period. The regulatory provisions are often contained in a series of overlapping layers (such as zones, local plans or developments plans) and each part of the component layers was examined and assigned Development Intensity Scores (DISs) based on DIS criteria derived from the terminologies and types of use classifications observed in planning schemes themselves (Supp Table 2). The various layers were then combined in accordance with their relative importance as described by the regulatory documents to create an overall layer that captured the relevant regulations applicable to all properties in each centre in 1996, 2006 and 2016. A total of 1322 planning scheme parts were initially coded with DISs and applied to a total of 44,063 property parcels. A re-test comparison using Cohen’s κ demonstrated excellent agreement between tests.⁶

Changes to the proportion of land assigned to different development types are used to give an indication as to whether local governments have decided to incorporate policies for compact activity centres in their regulatory documents. The land areas for each DIS were summed by centre to quantify these changes and the centres were classified according to their degree of performance. Centres in which regulations that changed in-line with activity centre policy intent were positively performing. Centres were considered to have marginal positive performance when only a minor change to regulations was observed (change of <5% of centre area⁷), or where the regulatory changes did not show a clear picture of overall intensification/reduction. Centres that saw regulatory changes contrary to activity centre policy were recorded as policy exceptions.

Conformance of land use change to planning regulations

Local governments also make regular decisions on the application of land use regulations to individual development proposals to intensify or change land use. To address this aspect, the conformance of land use and development to planning regulations was also considered. A deductive approach is used to link conformance to performance. This assumes that if development decisions are occurring in accordance with the plan, then the plan is being used in decision making and is therefore also performing.

We combined a detailed, point-based land use dataset developed through Google Street View and historical aerial image observations of all properties in the study area (Limb et al., 2020a) with the DISs to test conformance of not only use types but also relative use intensity, while allowing for regulations that are permissive of a range of different uses on a single site (see Loh (2011) for a discussion of the limitations of binary classifications of conformance/non-conformance). Conforming areas are sites where a land use aligns with a DIS that reflects the intended use. Under-developed areas are sites where the scale of the existing land use is less than the scale of use intended by the land use regulations. Exceeding areas are sites where the existing land use is of a type or scale that conflicts with or exceeds the land use regulations. It should be noted that the classification only applies to the observed use and does not consider the potential for mixed use development on a single site. For example, most regulations for ‘big box’ shopping centre sites (commercial DIS 5 and above) will also permit high density residential uses on the same site. Where a big box centre is developed without residences, the lack of conformance with the residential component is not considered as the observed use conforms with one of the intended uses for the site.

Considering changes based on individual development sites better reflects the actual processes of land use planning decisions that are made on discrete proposals for development. Using our previously developed land use database, we consider land use change in terms of the number of planning decisions rather than just in terms of land area. This is an important distinction as land area calculations can distort results in situations where a use change occurs over very large or small areas.

Where a use changed between 1996 and 2006, the new use was classified for conformance using the 1996 DIS. Where a use changed between 2006 and 2016, it was classified using the 2006 DIS. This tests the conformance of a use change against the plan regulations that were most likely to have been in force at the time of the change.⁸ The number of conforming, under-developed or exceeding sites for each centre was then counted for comparison, and to calculate the proportions of conforming and non-conforming uses for each centre. This approach also enables the calculation of the proportion of sites that were intended to change, but which did/did not change, as an analysis relating to the types of uses that changed/conformed/did not conform. Each changed use is reflective of a local government planning decision to enable the change. If a high degree of conformance is found, it therefore suggests that the local governments are using the regulations as they are intended. Further analysis of non-conforming cases then considers the nature of these changed uses and the implications of these issues of non-conformance to broader objectives to create more compact activity centres to deduce the extent of policy use.

Results

Our analysis of plan performance demonstrates that compact activity centre policy is performing across the evaluation measures. All the centres adopted direct references to activity centre policy during the study period, and the regulations were changed to be more permissive of uses aligned with compact activity centre principles. This indicates that the policy has been well integrated in local government land use regulations both at the strategic and code level. Where land uses changed, new uses are typically in accordance with regulations. This demonstrates that local governments are applying land use regulations as intended by making decisions on individual developments that reflect regulatory intent.

However, despite the apparent successful adoption of the centre strategies, the overall goals of the policy have largely failed to materialise (Limb et al., 2020a). This indicates that evaluating planning policy in terms of performance alone can be misleading by potentially generating an impression of success that is not matched by what materialises on the ground.

Content analysis of planning documents

Based on the 1996 era documents, most local governments were quick to include centre policy into their planning regulations (Supp Fig 2). Regulations in 12 of the 19 centres included a reference to regional compact centre policy. Only three of the centres were specifically mentioned as being centres as defined in the regional policy. Regardless of regional policy, all local governments were undertaking some form of centre-based planning for these locations and made use of a hierarchy of centre types. This planning was usually based around planning for commercial and industrial uses unless the centre was in the fourth category.

From the release of the first SEQRP in 2005, local governments were required to incorporate regional planning objectives when creating or amending their planning schemes.⁹ The early adoption of the regional centre policy concepts and terminology shows that Redlands, Ipswich, Pine Rivers and Brisbane councils were referencing regional policy in their land use planning prior to this date and were therefore undertaking these changes voluntarily. Aside from Redlands and Brisbane, these councils were applying the policy differently across their centres, suggesting that the policy was being used and that councils were choosing to not just adopt the policy in a general sense but were prepared and motivated to adapt it to local circumstances, which is, in itself, an indicator of performance success. Gold Coast City Council also voluntarily adopted the regional plan by specifically designating Beenleigh as a regional activity centre in their 2003 planning scheme. The case for voluntary adoption in Logan and Redcliffe councils is less clear as these local governments did not include the regional activity centre policies in their planning documents until after the release of the SEQRP.

Local government land use plans are the principal mechanism for implementing compact activity centre policies in the state of Queensland, and indeed throughout the rest of Australia. The direct inclusion of terminology, principles and references to the regional plans themselves indicates that the policy ‘messages’ were received by local governments and, that over time, decisions had been made to include these messages in their own plans. This suggests that the regional policy was used in decision making when creating more local land use regulations – as intended by the regional plans, and that the regional policy is therefore performing well.

Quantification of development intensity

These results were further confirmed by an analysis of the amount of development intensity permitted by the planning regulations using DISs as summarised in Supp Table 3.¹⁰

The results show that in most instances, land use regulations are changing to match regional policy intent. Positive performance is particularly evident in the residential category, where all centres saw changes that permitted higher density residential development. Changes to regulations for commercial development also reflected positive performance with 15 of the 19 centres recording positive changes, and the remaining four centres showing marginal positive change. The results for each category are described and discussed in further detail below.

Residential changes

The residential category shows particularly strong positive performance with regional policy intent, with all centres showing regulatory change to be more permissive of higher density residential development. Figure 1 shows the differences in the DIS for residential uses between 1996 and 2016 for each location in the study area. These differences are expressed in terms of their percentage of the total land area for each centre. The residential DIS is a scale of the intensity of permitted residential use ranging from 1 (no residential development permitted) through 7 (apartment buildings seven stories and above).

Figure 1.

Differences in land area of residential DIS between 1996 and 2016, as a percentage of total land area, by inner, middle and outer suburbs.

Theses charts shows a clear picture of land use regulations changing to be more permissive of higher density residential uses, particularly those in DISs 6 and 7 (which permit four to six storey and seven plus storey apartment development, respectively). The results also show that over time, the degree of permissiveness has increased, where changes between 1996 and 2006 were more likely to be in the form of increases to DISs 5 and 6, and changes between 2006 and 2016 saw a larger increase in DIS 7. The changes show a consistent increase of permitted residential intensity across all centres. Overall, the outer centres show greater proportional increases in land zoned for higher densities compared to middle ring centres.

Commercial changes

Changes in the commercial category also showed mostly positive performance, where there was an overall increase in the intensity of permitted commercial development, and only four of the outer centres with marginal changes. Figure 2 shows the changes to the commercial DIS as a percentage of the total land area of inner, middle and outer ring centres.¹¹ The most significant changes in all locations were increases in DIS 6, which indicates allowances for large scale retail and office complexes in structures five storeys and above. Overall, the change to commercial regulations affected a relatively small proportion of each centre’s area compared to the residential changes (12% average change vs. 27%). The general trend across centres was to rezone existing commercial uses to more intensive uses rather than create new commercial sites, with only five of the centres showing greater than 5% of their areas changing from no or limited commercial uses.

Figure 2.

Differences in land area of commercial DIS between 1996 and 2016, as a percentage of total land area, by location.

Inner and middle centres drew mostly from sites already zoned for intensive commercial use, to permit increased areas for high-rise (five storey and above) commercial development. These centres also reduced limited commercial development opportunity (COM 1 and 2) areas to a greater extent than outer areas. There was a greater variation in commercial regulatory changes in outer centres; however, the overall trend was for outer centres to convert existing small-scale centre zoned areas (DIS 3) to permit more intensive commercial uses. This difference between locations however reflects the amount of land in the DIS 3 category in 1996, with outer centres having a far higher proportion of DIS 3 land compared to middle centres (12% of centre area vs. 1%).

Based on the above analysis, we conclude that local governments are generally changing their land use regulations to align with activity centre policy although it is also common to see local governments adjust regulations differently for different centres. Despite this level of local variation, all centres demonstrated positive performance in the key categories of residential and commercial intensification. Overall, local governments are making decisions on land use regulations both in terms of strategic intent and in terms of the regulations themselves, which are generally well aligned with regional compact activity centre policy. This analysis is based on the outputs of regulatory decision making, rather than on the processes of decision making itself. As such, how regional policy was considered when making regulatory decisions is not known. Changes to the regulations themselves, however, imply a strong degree of policy performance, with activity centre policy being well reflected in changes to regulations. Land use regulations are the key implementation mechanism for activity centre policy. Based on these changes, it can be concluded that the regional policy is performing well, with local governments using activity centre policy to inform regulatory change.

Despite this apparent success the critical question remains, i.e. whether actual land uses and land use changes respond to these regulations?

Conformance of land use changes to planning regulations

The study area consisted of 24,789 unique sites in 2016. Most of these sites (22,455 – 91%) did not change use in the preceding 20 years. Of course, land use regulations do not propose changes on all sites in the study area. The most likely sites for changes are locations where a site’s land use has regulations that expect further or different types of development (i.e. areas that are either under or over performing). However, only a small proportion of these types of sites saw any change. A total of 4901 sites were classified as under or exceeding the expected level of development intensity for the entire 20-year study period, and only 728 of these sites (approximately 15%) changed use.

Where the land use did change, the change was classified using the previously described categories of conforming, under-developed and exceeding. In all locations, development had a high degree of conformance with land use regulations with an average rate of conformance across all centres of 85%. Figure 3 shows the rates of development conformance by location, with inner and outer centres showing greater rates of under development compared to the middle ring centres.¹²

Figure 3.

Development conformance with land use regulations by location, 1996–2016.

The overwhelming majority of use change has been in the form of residential development. However, the type of residential use change varies by location, with inner residential changes primarily composed of higher density dwellings, while middle and outer areas showed larger proportions of low-density dwellings (Figure 4). Outer areas also differed by having a far greater proportion of changes to non-residential uses.

Figure 4.

Type and proportion of use change by location, 1996–2016.

The relatively high proportion of low-density development sites may perhaps seem initially concerning in terms of conformance to compact city ideals. However, 85% of the low-density development is occurring in locations that are regulated for this type of housing, making it highly conformant to land use regulations. Regional policy for activity centres does not dictate that all low-density housing areas are to be replaced, and local governments are free to select the areas they deem most suitable for higher density housing. As shown in the previous section, all centres are positively performing in this respect. The remaining 14% of low-density development that was categorised as under-developed, made up more than 73% of all the under-developed sites in the study area. This is potentially problematic for the implementation of compact activity centre policy, as newly constructed houses on sites identified for more intensive uses effectively block further development for the lifespan of the house, a period typically of at least two decades.

Sites where development exceeded what was envisioned by land use regulations were very rare, representing approximately 4% of changed sites. This type of non-conformance may be reflective of Queensland’s ‘performance-based’ planning system, which enables development to be approved even if it is contrary to land use regulations based on a qualitative assessment of its merits. Such an assessment includes aspects related to whether the use conforms to broader strategic intent such as regional level policies. In inner and middle locations, exceeding sites were almost entirely residential (100% for inner and 83% for middle). Residential non-conformances result in greater amounts of dwellings than would otherwise have been provided and are therefore not considered contrary to overall policy aims as increasing overall population density is a core objective of activity centre policy.

In summary, activity centre policy in greater Brisbane has performed well in influencing subordinate plans and land use planning decisions with most local governments taking up the concepts voluntarily and the overall framework being adopted quite quickly. In addition, an examination of the data related to potential development intensity using high density residential and commercial land uses as examples demonstrates that the policy has had success in terms of creating a regulatory environment that is permissive of the desired greater levels of development. The only thing that does not seem to have happened is the development itself. The policy has been adopted, the environment to encourage more intense land uses has been created, but the hoped-for development has largely failed to materialise. The final section of this paper will examine this gap between policy and implementation and discuss the implications for comprehensive evaluations of planning policy.

Discussion and conclusion

In this research, we provide a novel method of quantifying and analysing changes to regulatory planning systems and apply this approach to highlight fundamental limitations with market led growth management activity centre policies, and the importance and potential of plan performance evaluation.

In greater Brisbane, we found that local governments are taking up the substantive aspects of activity centre policy and incorporating them into their land use regulations as intended. Local governments began to adopt compact activity centre planning in 1996 and by 2016, all nominated centres were referenced in local government compact activity centre policy. These references proved to be more than lip-service to regional policy, with the regulatory controls themselves supporting activity centre policy, especially in terms of intensive residential and commercial use types. Moreover, local governments are using the regulations as intended to make development decisions to permit uses that align with the regulations of activity centre policy. Where uses did change, they proved to be highly conformant to land use regulations and regional activity centre policy.

While local government decisions show strong evidence of performance, the aspects that local government typically do not directly control – namely the actual development of land – failed to deliver compact activity centres in most instances (Limb et al., 2020a). The overall situation is therefore one of strong performance of metropolitan policy implementation but mixed to poor conformance with the intended outcomes. These actions therefore are clearly insufficient to deliver compact centres, at least in Brisbane. It also highlights that measuring performance by itself is inadequate as a means of evaluating policy implementation. In this case, the glowing record of performance belies a much less impressive capacity for actual change.

Proponents of the plan performance approach to evaluation consider implementation successful when a plan is used to inform decisions as intended, including decisions to not implement the plan (Alexander and Faludi, 1989; Faludi, 2000). The common critique of plans ‘gathering dust’, sitting unused on a shelf would of course be considered a planning failure. And we concur that the example described by Faludi (2006) represents a planning success, with decision makers using a plan (sometimes accepting its messages, sometimes rejecting them) as best fits individual circumstance. Likewise, it is reasonable to consider the dogmatic application of planning to implement a solution that is no longer appropriate as described by Altes (2006) as a failure of plans to perform.

However, the case of greater Brisbane presents a different situation that is immediately recognisable to planners attempting to alter existing urban forms via common methods of land use planning. Here, a planning policy to establish activity centres has been in place continuously for more than two decades. This policy has enjoyed both popular and technical support, with changes in policy calling for ever more intensive development to invigorate the nominated locations. Yet despite the plan being used by the intended decision makers as shown in this research, the plan has not been dignified by the change it intended to make in the physical world. Where plans are designed to be effectuating devices (Talen, 1996) and represent appropriate planning responses to a given situation, it suggests significant shortcomings in the plan’s implementation mechanisms and a need for reconsideration in future iterations of the plan.

The activity of land use planning naturally involves proposals to alter the physical nature of a geographic area. For a land use plan to be implemented in terms of conformance, at some point decisions will need to be made to mobilise the necessary resources to create a material difference in the world. The nature of these decisions will also have an impact on what is physically delivered. For such plans, evaluation can therefore be considered in terms of a matrix that plots plan failure and success on a combined spectrum of both performance and conformance (Figure 5).

Figure 5.

Matrix of conformance vs. performance in evaluative approaches (authors).

Considered in these terms, an evaluation of both performance and conformance is necessary to explain plan implementation successes or failures. Measuring performance absent measures of conformance lacks real world context to plan intent. Performance evaluation, however, is also important as it enables a differentiation between whether poor conformance is due to deficiencies with the plan itself and how the plan was used by key actors charged with its implementation.

This evaluation of greater Brisbane’s activity centre policies also reveals the fundamental limitations of relying on market led development in achieving normative strategic planning objectives. Where there is market demand for the type of development envisioned by the plan, regulatory changes shape some parts of the urban form by restricting or permitting the scale and type of use. But where no such demand exists, the regulations change without a corresponding change to the physical world (Limb et al., 2020b). Absent was an understanding of whether the plan was being used to actually inform decision making as intended in the regulatory land use planning system, so we could not have explained whether the lack of conformance was due to the nature of the plan’s implementation mechanisms or a lack of use. Evaluating plan performance is therefore vital to understanding the nature and processes of plan implementation.

Presented in this manner, we consider performance evaluations as a logical necessity to holistically understand plan implementation and believe this example from greater Brisbane provides support to this position. The deductive approach used in this research design was appropriate in this instance as the high levels of support for the policy from local governments meant evidence of use of the policy was likely to be found in subordinate plans and policies. However, there are two key areas of future inquiry which would require different research designs to measure plan performance.

First, are situations where a policy is likely to not be invoked, or invoked and then not followed. These situations may not leave documentary evidence of whether the policy was used and therefore require more qualitative methods that directly engage with the actors involved. Second, are research designs that seek to understand why policy makers insist on particular approaches and how they consider plan implementation when creating and adopting policy. In these instances, plan performance evaluation has more intrinsic value to understand the considerations that inform decision making in planning. Such approaches represent an important direction for future plan performance-based evaluation research. For example, some argue that planning policy formation is locked into a form of path dependency based on its historical approach (Bunker, 2012) or from the expectations associated with established urban forms (Filion, 2015). To empirically address such questions, the deductive approach to performance evaluation we have used here is less appropriate. Such institutional and sociological considerations involved in plan implementation are currently poorly understood and lack empirical evidence. Research evaluating plan performance from such perspectives would offer new insights into how plans are used when developing and implementing policy.

Supplemental Material

sj-pdf-1-epb-10.1177_2399808320951583 - Supplemental material for When plans are used to no effect: Considering implementation performance of greater Brisbane’s compact activity centre policies

Supplemental material, sj-pdf-1-epb-10.1177_2399808320951583 for When plans are used to no effect: Considering implementation performance of greater Brisbane’s compact activity centre policies by Mark Limb, Carl Grodach, Paul Donehue and Severine Mayere in Environment and Planning B: Urban Analytics and City Science

Footnotes

Acknowledgements

The authors would like to thank our anonymous peer reviewers for their time and effort in considering our paper, and for their constructive advice which led to its improvement.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was undertaken with the financial support of a Queensland University of Technology Civil Engineering and Built Environment Scholarship.

ORCID iD

Mark Limb

Supplemental material

Supplemental material for this article is available online.

Notes

Mark Limb is a lecturer in Urban and Regional Planning at the Queensland University of Technology, Brisbane. His research interests include policy evaluation and land use planning.

Carl Grodach is foundation professor and director of Urban Planning & Design at Monash University. He is author of the book Urban Revitalization: Remaking Cities in a Changing World (Routledge, 2015) and editor of The Politics of Urban Cultural Policy: Global Perspectives (Routledge, 2013).

Severine Mayere is an associate professor of Urban and Regional Planning at Queensland University of Technology, Brisbane, Australia. Her research interests include urban consolidation, stakeholder engagement and governance.

Paul Donehue lectures in Urban and Regional Planning at Queensland University of Technology. His current research interests include Australia’s network of rural and regional airports, the resilience of infrastructure and development control.

References

Alexander

Faludi

(1989) Planning and plan implementation: Notes on evaluation criteria. Environment and Planning B: Planning and Design 16(2): 127–140.

Altes

WKK

(2006) Stagnation in housing production: Another success in the Dutch ‘planner’s paradise’? Environment and Planning B: Planning and Design 33(1): 97–114.

Baker

Sipe

Gleeson

(2016) Performance-based planning. Journal of Planning Education and Research 25(4): 396–409.

Berke

Backhurst

Day

, et al. (2006) What makes plan implementation successful? An evaluation of local plans and implementation practices in New Zealand. Environment and Planning B: Planning and Design 33(4): 581–600.

Brewer

Grant

(2015) Seeking density and mix in the suburbs: Challenges for mid-sized cities. Planning Theory & Practice 16(2): 151–168.

Brody

Highfield

(2005) Does planning work?: testing the implementation of local environmental planning in Florida. Journal of the American Planning Association 71(2): 159–175.

Bunker

(2012) Reviewing the path dependency in Australian metropolitan planning. Urban Policy and Research 30(4): 443–452.

Burby

(2003) Making plans that matter: Citizen involvement and government action. Journal of the American Planning Association 69(1): 33–49.

Chhetri

Han

Chandra

, et al. (2013) Mapping urban residential density patterns: Compact city model in Melbourne, Australia. City, Culture and Society 4(2): 77–85.

10.

Faludi

(2000) The performance of spatial planning. Planning Practice and Research 15(4): 299–318.

11.

Faludi

(2006) Evaluating plans: The application of the European spatial development perspective. In: Alexander

(ed.) Evaluation in Planning: Evolution and Prospects. Hampshire, UK: Ashgate, pp.119–143.

12.

Faludi

Altes

(1994) Evaluating communicative planning: A revised design for performance research. European Planning Studies 2(4): 403–418.

13.

Feitelson

Felsenstein

Razin

, et al. (2017) Assessing land use plan implementation: Bridging the performance–conformance divide. Land Use Policy 61: 251–264.

14.

Filion

(2009) The mixed success of nodes as a smart growth planning policy. Environment and Planning B: Planning and Design 36(3): 505–521.

15.

Filion

(2015) Suburban inertia: The entrenchment of dispersed suburbanism. International Journal of Urban and Regional Research 39(3): 633–640.

16.

Filion

Saboonian

(2019) Large multifunctional suburban centres and the transformation of the suburban realm. Journal of Urbanism: International Research on Placemaking and Urban Sustainability 12(3): 296–319.

17.

Forster

(2006) The challenge of change: Australian cities and urban planning in the new millennium. Geographical Research 44(2): 173–182.

18.

Guyadeen

Seasons

(2016) Plan evaluation: Challenges and directions for future research. Planning Practice & Research 31(2): 215–228.

19.

Guyadeen

Seasons

(2018) Evaluation theory and practice: Comparing program evaluation and evaluation in planning. Journal of Planning Education and Research 38(1): 98–110.

20.

Laurian

Crawford

Day

, et al. (2010) Evaluating the outcomes of plans: Theory, practice, and methodology. Environment and Planning B: Planning and Design 37(4): 740–757.

21.

Limb

Grodach

Mayere

(2018) Where is the compact city? A method to monitor the implementation of compact activity centre policy. In: State of Australian cities conference, Adelaide, Australia.

22.

Limb M, Grodach C, Mayere S and Donehue P (2020a) Policy intent meets reality: The conformance of 20 years of metropolitan compact activity centre policy in greater Brisbane, Australia. OSF Preprints. DOI:10.31219/osf.io/5epxr.

23.

Limb M, Grodach C, Mayere S, et al. (2020b) Rethinking the Implementation of the Compact City: Factors Affecting Compact Activity Centre Policy Conformance in Greater Brisbane, 1996 to 2016. Urban Policy and Research. DOI: 10.1080/08111146.2020.1792284.

24.

Loh

(2011) Assessing and interpreting non-conformance in land-use planning implementation. Planning Practice & Research 26(3): 271–287.

25.

Lyles

Berke

Smith

(2015) Local plan implementation: Assessing conformance and influence of local plans in the United States. Environment and Planning B: Planning and Design 43(2): 381–400.

26.

Newton

Glackin

(2014) Understanding infill: Towards new policy and practice for urban regeneration in the established suburbs of Australia’s cities. Urban Policy and Research 32(2): 121–143.

27.

OECD (2012) Compact City Policies: A Comparative Assessment. Paris: OECD Publishing.

28.

Oliveira

Pinho

(2009) Evaluating plans, processes and results. Planning Theory & Practice 10(1): 35–63.

29.

Oliveira

Pinho

(2010a) Evaluation in urban planning: Advances and prospects. Journal of Planning Literature 24(4): 343–361.

30.

Oliveira

Pinho

(2010b) Measuring success in planning: Developing and testing a methodology for planning evaluation. Town Planning Review 81(3): 307–332.

31.

Phan

Peterson

Chandra

(2009) Residential intensification in a suburban fringe local government area, Casey, Melbourne metropolitan area, Australia. Australasian Journal of Regional Studies 15(1): 81–100.

32.

Schmitt

(2013) Planning for polycentricity in European metropolitan areas—challenges, expectations and practices. Planning Practice and Research 28(4): 400–419.

33.

Talen

(1996) After the plans: Methods to evaluate the implementation success of plans. Journal of Planning Education and Research 16(2): 79–91.

34.

Talen

(1997) Success, failure, and conformance: An alternative approach to planning evaluation. Environment and Planning B: Planning and Design 24(4): 573–587.

35.

Zhong

Mitchell

Huang

(2014) Success or failure: Evaluating the implementation of China’s national general land use plan (1997–2010). Habitat International 44: 93–101.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.91 MB