Sustainable and inclusive – Evaluating urban sustainability indicators’ suitability for measuring progress towards SDG-11

Abstract

The adoption of the sustainable development goals marks a transition in the global sustainability discourse to a growing focus on equity, with urban areas’ role in achieving sustainable and inclusive growth more explicit in sustainable development goal-11. Within this discourse, urban sustainability indicators could be used to monitor environmental quality and equity within individual cities, while promising to deepen our understanding of how urban areas contribute to global environmental sustainability. We examine 484 indicators of urban and regional environmental sustainability sourced from 40 indexes and online data repositories to determine their suitability for measuring both urban environmental performance and equity. Despite the large number of existing indicators related to urban environmental monitoring, we find that they are inadequate as tools for evaluating progress towards sustainable development goal-11’s integrated goal of sustainable and inclusive (i.e. equitable) urban areas, due to a lack of benchmarks, targets, and explicit measurement of equity considerations. Future research should emphasize data collection that can be disaggregated geographically to make it possible to measure distributional equity and establish locally appropriate benchmarks and realistic targets for urban sustainability indicators. Lastly, we argue that utilizing large-scale, high-resolution datasets has the potential to help overcome these data collection challenges.

Keywords

Equity indicators sustainable development goal 11 urban analytics urban sustainability measurement

Introduction

Urban sustainability has been identified as a key component of the global sustainability agenda, most notably in sustainable development goal-11 (SDG-11), which sets a goal for cities to be inclusive and sustainable by 2030. While the 2030 Agenda for Sustainable Development includes 17 global SDGs and around 230 indicators to track their implementation (UN Statistics, 2018), they are primarily focused at the national level. SDG-11 is “path breaking” in the UN system because it introduces a standalone urban goal for the first time, recognizing the critical role cities play in advancing progress towards global sustainable development (Parnell, 2016). It brings social inclusion and environmental sustainability together under a “balanced and integrated approach to urban sustainability monitoring” (Zinkernagel et al., 2018). Most SDG indicators include a link to cities and human settlements, and approximately one-third of SDG indicators are measured at the local level (UN-Habitat, 2018).

Despite SDG-11’s aim to “substantially increase” cities’ policies towards “inclusive urbanization” (Table S1) or, more generally, “inclusion” (Targets 11.3 and 11.B), the term “inclusion” is not explicitly defined. Several targets and indicators identify the term’s relationship, however, to the equity and universality dimensions encapsulated in the SDGs. Three out of the 10 indicators for SDG-11 feature provisions to “ensure” or “provide” access (Targets 11.1, 11.2, 11.7) to environmental goods and services, including basic housing, sustainable transport, and urban green spaces, “in particular for women and children, older persons and persons with disabilities” (Target 11.7) or “with special attention to the needs of those in vulnerable situations” (Target 11.2).

While SDG-11’s unique holistic framing of environmental sustainability and inclusion could help spur needed reforms in urban governance, this structure also creates difficult monitoring challenges (Klopp and Petretta, 2017; Simon et al., 2015). While researchers, government agencies, and non-profit institutions have created systems of indicators that synthesize and compare data across different urban contexts for a range of different applications and audiences, whether they can be applied to measure cities’ progress towards SDG-11 is the primary research question we aim to tackle in this paper. What does the landscape of urban sustainability indicators suggest about the challenge of tracking progress towards SDG-11, particularly its equity and inclusion dimension?

In this paper, we review a range of urban indexes to determine their suitability to measuring progress towards SDG-11’s environmental sustainability and inclusion dimensions. We evaluate 40 urban sustainability indexes and frameworks produced by academic or non-government organizations (NGOs), governments, and private sector/consulting groups that include a total of 484 indicators. They range from the Global Liveable Cities Index, which evaluates 64 cities on five dimensions of urban quality of life, including environmental sustainability, to the Sustainable Cities Index, which is a private-sector effort to assess 100 cities’ environmental sustainability performance. We organize our evaluation as follows: we first provide a background discussion of the emergence of urban indicators and how we consider equity in the context of urban sustainability. We then discuss our methods for coding and assessment, followed by our findings. We conclude by proposing directions for future research on urban sustainability measurement and monitoring SDG-11.

Background and literature review

Challenges of urban indicators

Indicators are defined in a myriad of ways, but they represent the “past or projected performance of different units,” and are generated through a “process that simplifies raw data about a complex social phenomenon” (Davis et al., 2015: 4). They have been applied in diverse contexts to measure environmental performance and sustainability, and are typically aggregated from multiple data sources using statistical methods to form indices, or complex indicators (Ness et al., 2007). Indicators can provide a powerful basis for political decision-making, building public awareness, and assessing the ways in which fundamentally environmental problems are conceptualized and their solutions determined (Davis et al., 2015). In urban areas, indicator systems have increasingly resonated with decision-makers and managers, in large part due to what Kitchin et al. (2015) refer to as the rise of “new managerialism” within cities. This trend emphasizes the instrumental nature of indicators that are often used to guide practices, provide evidence of performance, or inform policies or decision-making (Hezri and Dovers, 2006; Kitchin et al., 2015). This type of “dashboard governance” allows urban managers to collect and integrate statistical information, including traffic control, weather conditions, and municipal and utility services (Mattern and Mattern, 2015).

Indicators in cities are now a commonplace means of tracking performance and guiding policy formation, to the point where they have become “normalized as a de facto civic epistemology through which a public administration is measured and performance is communicated” (Kitchin et al., 2015: 7). One study suggests that there are now 200 city benchmarking initiatives that aim to compare hundreds of cities globally (Moonen and Clark, 2013). These include indices that strive to rank the economic competitiveness of cities, such as the Global City Index (Kearney, 2019) and the Global City Power Index (Mori Memorial Foundation, 2018), to others that are more specifically focused on environmental performance, including the Siemens Green City Index (EIU, 2012). These indices typically rank different aspects of cities that are weighted and then aggregated into a single composite score. Despite the growth in indicators and indices in urban areas, meaningful and comparable indicators are difficult to achieve due to the complexities embedded in defining sustainability (Bohringer and Jochem, 2007 Cooper, 1999) within the urban boundary (Alberti, 1999; Bagliani et al., 2008; UNHSP, 2004). Mori and Christodoulou (2012: 95) suggest that existing indexes do not allow for the ability to “assess and compare cities’ sustainability performances for understanding the local and global impact of cities on the environment and human life.”

Another challenge facing urban indicators, and in particular SDG-11, is with respect to their generalizability or comparability across, at times, vastly different contexts. While all SDGs face monitoring challenges, SDG-11’s subnational and spatial focus adds complexity because residents’ experience of SDG-11 indicators can vary significantly within the city, and are often shaped by neighborhood-specific socio-spatial factors (Ulbrich et al., 2019). Many cities that do attempt to collect data for the SDGs, by leveraging their existing monitoring systems or applying SDG indicators to local contexts, capture the information most relevant to local priorities and politics, which does not necessarily allow for comparability across cities (Pfeffer et al., 2019). In Germany, for instance, Koch and Krellenberg (2018) assessed three initiatives focused on translating and tracking SDG-11 and found limited comparability between each initiative’s indicators; difficulties in data disaggregation; and a lack of sustainability indicators by German cities themselves. Similarly, Simon et al. (2015) assessed pilot SDG-11 indicators in Bangalore, Cape Town, Gothenburg, Greater Manchester, and Kisumu. Each city struggled to access sufficient data to track the indicators, and all proposed changes that would make them more relevant to their unique context. In other words, it is difficult not only to track a city’s progress towards SDG-11, but to compare and assess progress towards this goal at a global scale.

Reflecting environmental equity in urban sustainability measurement

SDG-11’s focus on the distribution of environmental harms (e.g. exposure to air pollutants) and access to environmental benefits (e.g. urban green spaces and transportation) across different demographic categories suggests that this goal primarily measures the distributive dimensions of environmental equity. Distributive environmental equity emphasizes the allocation of social, economic and political goods, costs, and privileges among demographic groups, including groups defined by age, income, race, education, gender, employment status, or social groups and districts (Agyeman and Evans, 2004; Rawls, 2009; Schlosberg, 2009: 12–13).

Researchers also often emphasize the importance of two additional dimensions of environmental equity – procedural equity, or the fair access of different demographic groups to the decision-making processes (Agyeman and Evans, 2004; Reckien et al., 2017; Schlosberg, 2009) and justice as recognition (Reckien et al., 2017; Young, 1990), the influence of pre-existing political, economic, and social conditions on the crafting and implementation of environmental policy. Like SDG-11, however, many existing urban environmental indexes primarily consider the distributive dimension of environmental equity, likely because it is often easiest to quantitatively measure (Reckien et al., 2017). We focus on our analysis on this dimension of environmental equity, while noting procedural equity and justice as recognition are also important elements of understanding environmental in/equity in urban areas (see, for instance, Kopnina (2016) for a critical discussion of whether the SDGs successfully engage with global unsustainability challenges).

Although attempts to measure distributive equity are more common, they carry their own challenges and complications. The notion of distributive equity, by definition, implies that environmental benefits and burdens should be placed equally among individuals. In practice, however, any marginal exposure to an environmental harm (i.e. pollution) for a vulnerable population could be detrimental (Maguire and Sheriff, 2011). For example, low-income populations often have less access to medical care, alternative water sources, or housing options that can enable individuals to avoid or reduce the impact of exposure to environmental harms (Maguire and Sheriff, 2011). Distributing environmental burdens equally across all communities may still disproportionately impact marginal populations. Fainstein (2014), in her comparison of spatial justice in the highly unequal cities of New York, Amsterdam, and Paris, argues that inherited advantages mean that a just distribution of public benefits should favor the disadvantaged. While we note these caveats around the application and measurement of distributional equity, we adopt this concept as our primary way of evaluating if and how existing urban environmental indicators account for equity in ways that could be operationalized to track progress towards SDG-11.

Methods

Index and indicator selection

To examine the landscape of existing urban sustainability indicators and the extent to which they could be applied to assess equity and consequently SDG-11, we reviewed 41 indexes (Table S2) – that is, composite urban sustainability frameworks that include multiple indicators – that contained a combined total of 484 indicators. We conducted a “horizon-scanning” (e.g. Joss, 2011: 270) exercise to identify and characterize indexes. We used keywords such as “urban sustainability index” and “urban environment index” in Google searches to identify candidate indexes. We conducted the search between January – July 2016 and repeated it in January 2018, searching for urban indexes available in English that included indicators of environmental assessment or sustainability. This analysis is not intended to be a comprehensive review; instead, we stopped collecting indexes when we felt we had reached a wide diversity of concepts covered in the indicators. The selected indexes assess geographic areas ranging from neighborhoods to mixed urban-rural regions. They were created by organizations including: NGOs, academic institutions, governments, development agencies, and consulting companies (see Table S2 for a full list of the indexes reviewed).

The goal of reviewing these indexes is ultimately to understand what individual urban sustainability indicators measure in terms of environmental issues; if they address equity and at what scale; and the data sources they rely upon. Urban sustainability indexes cover a wide range of topics and often include economic and social indicators alongside environmental or ecological indicators. The analysis focuses on individual indicators included in the composite indexes, rather than the indexes as a whole. In cases where indexes included a mix of environmental, social, and economic indicators, we selected indicators that measure environmental quality (e.g. air pollution), or human activities that affect environmental quality (e.g. vehicle miles traveled) in urban areas. Social and economic indicators, such as population density and unemployment rate, that did not directly measure an environmentally related dimension of SDG-11 were thus excluded from our final analysis. However, we did consider social and economic indicators when deciding whether environmental indicators are “able to assess equity” or not, even if the index authors did not explicitly calculate equity distributions. This process is further discussed in the section on qualitative coding below. We further aimed to identify indexes with adequate metadata (e.g. information or data about the data itself) to allow us to analyze the characteristics (e.g. target type, presence or absence of baseline) of the sampled urban sustainability indicators.

Qualitative coding

We applied qualitative coding to identify key aspects of each indicator, including: the origin and geographic scope, an indicator’s environmental focus issue, the target type, whether the indicator related to equity, the unit of analysis, and the type of data source. Four different researchers evaluated indicators across these categories and each of their resulting lists were compared. Discrepancies between the lists were resolved by group consensus.

Geographic origin and coverage

We identified the country of origin and the geographic coverage for each index. We coded each country as “developed,” “developing,” “in transition,” and “least developed” based on their Gross National Income (GNI) and the United Nations’ 2016 classifications for their World Economic Situation and Prospects report (UN, 2016). We also coded each index creator (e.g. a non-governmental organization (NGO), government or private sector/consulting company) and conducted a network analysis to visualize the geographic connections between index creators and their sector and focus regions of analysis.

Environmental issue area

Because many indexes included similar indicators, such as “Annual mean concentration of fine particulate matter” or “Level of pollution concentration: SOx, NOx, CO, or Ozone,” we sought to identify commonalities in the thematic topics and subject areas evaluated. We identified 13 common environmental issue areas: air quality, green economy, environmental health, water and sanitation, climate and energy, land use and the built environment, transportation, waste management, environmental governance and planning (capacity), biodiversity and habitat, natural hazard vulnerability, urban food and agriculture, and perceptions and participation. Table S5 provides keywords that were used to categorize indicators into issue areas. We also mapped these issue areas to any relevant SDG-11 target or indicator. This process identified indicators that could be used to measure progress towards SDG-11 targets or indicators, but did not require that indicators exactly matched the SDG-11 targets or timelines.

Targets and baselines

We identified the following targets and baselines in the indicators (Table S3): no target and no baseline; no target with a baseline (which could make it possible to assess the direction of change over time); directional target (a goal to increase or decrease an indicator from a baseline or the current level without specifying by how much); and specific targets (an aspirational absolute level of the indicator or specific percent change from the baseline, both within a particular time frame).

Table S3 provides examples of how these different target types were articulated in practice. Baselines were particularly difficult to observe from metadata, but often a date would be given as a baseline year. In some cases, the indicators included original calculations from existing data. We considered indicators that did not identify a baseline or starting point for measurement as having “no baseline.” While the earliest year that the indicator was published could serve as a baseline for future measurement of change, these index developers did not make this specification.

Equity

According to the distributive dimension of equity, we identified any indicator as an “equity indicator” if it was defined according to different demographic groups or reported in a way that corresponds to spatially disaggregated and comparable demographic groups. Any indicator that differentiated by social identity – such as gender, race, reproductive status, class, income, employment or age – was considered an “equity indicator” because it can indicate disproportionate burdens or benefits to different groups. We used this strategy because many indexes do not explicitly identify these indicators as being related to equity. We interpreted the inclusion of language referring to demographic groups as an attempt to disaggregate data and show differences among these groups, in line with the definition of distributive equity. Table S5 captures several examples of these indicators. For instance, CalEnviroScreen measures the annual mean concentration of PM_2.5 air pollution, analyzed by sensitive populations and sociodemographic factors such as housing burden, linguistic isolation, poverty, and unemployment. Table S4 lists keywords from the equity indicators' metadata – which range from specific mentions of “at-risk” or “vulnerable” residents to specific populations, such as “pregnant women” and “First Nation” and “Hispanic” communities – to capture the focus of different equity indicators.

Spatial scale

The spatial distribution of marginalized groups identified in Table S4, such as residents living in areas with higher levels of poverty, housing burdens, unemployment, and linguistic isolation, is generally clustered in different parts of the city (Bravo et al., 2016). As a result, data with a relatively high level of spatial resolution or disaggregation can serve as a proxy for demographically disaggregated data that is often needed to measure distributive equity. We therefore also evaluated each indicator’s unit of analysis, and coded them accordingly:

Site-specific (the smallest unit, including a single parcel, building, urban real estate development, or complex);

Neighborhood (areas within a city, such as wards or neighborhoods);

City limit (one central city or municipal corporation - the “city proper”);

Metro region (the contiguous urbanized area spanning multiple urban government boundaries);

Regional (large areas spanning at least on central urban area and rural surroundings); and

National (aggregated urban and rural areas within the same country).

Data sources

Data gaps pose a significant challenge to creating urban sustainability indexes. Particularly for large comparative indexes, for which data collection may be prohibitively expensive, data availability usually constrains index methodologies and overall indicator selection. The UN-Habitat Urban Indicators Guidelines draws a distinction between Cluster A and Cluster B data sources (UNHSP, 2004). Cluster A sources (referred to as “official government data”) include “indicators to be obtained from Census, Demographic and Health Surveys, Multiple Indicators Cluster Surveys and national households surveys” (UNHSP, 2004: 7). Cluster B indicators represent all other data sources, including “official record and published studies of Government institutions, housing boards and agencies, service parastatals, finance institutions, police, NGOs as well as using informed estimates made by small groups of experts on specific issues” (UNHSP, 2004: 7). These could include remotely sensed and citizen science data. We reviewed each indicator’s data sources to determine whether it relied on official government data (Cluster A) or alternative or “open” (Cluster B) data sources.

Results

Geographic origin and coverage

Figure 1 maps the corresponding country of the index creators, as well as the country where the cities assessed by these indexes are located. The points on the map are shaded according to the United Nations’ 2016 classifications for their World Economic Situation and Prospects report (UN, 2016), based on 2014 GNI (these dates were chosen to correspond with the timing of our data collection). Only 24 of the 40 indexes publish the countries where they have been applied, and these applications are run by organizations located in just 9 high-income countries. These include multiple indexes from the United States (14), Canada (3), and one each from Australia, Belgium, Korea, Netherlands, Sweden, United Arab Emirates, and the United Kingdom. These indexes cover 114 countries, primarily developing countries (47 countries included), followed by developed countries (34), least developed (23), and countries in transition (10). The most prevalent types of index creators vary by location: government indexes often measure urban sustainability in Europe; consulting firms more often include developed country cities in their indexes in Europe, Latin America and Asia; and NGOs include cities from Africa and Southeast Asia. As Figure 1 shows, urban sustainability in developed countries and countries in transition is often assessed using indicators developed by a mix of governments, consulting companies, and NGOs. Our sample did not include any government-created indexes from developing and least developed countries; instead, NGOs and consulting companies primarily track urban sustainability in these regions.

Figure 1.

Country classifications for the index creators. The size of each node (i.e. degree) represents the number of connections between countries, with larger nodes representing those with more connections. The direction of the arrows indicates the countries included in indexes, while the origin is the headquarters country of the index organization. The color of the edge between two points indicates the sector of the index organization.

Environmental issue areas

Our analysis found that several core issues occurred most frequently across the reviewed indicators. Figure 2, which shows the frequency of indicator issue areas in the sample, demonstrates that six issue areas are observed much more frequently than others. We label these core issues for urban sustainability indexes: water and sanitation, climate and energy, land use and built environment, transportation, waste management, and air quality. The least frequent core issue area – air quality – was still nearly twice as frequent in our sample compared to the next most frequent issue area, urban food, and agriculture. For some of the analyses described below, we focused only on the core issue areas.

Figure 2.

Indicators by issue classification and index creators. The most prevalent categories include: water and sanitation (n = 82), climate and energy (n = 77), land use and built environment (n = 60), transportation (n = 56), waste management (n = 40), and air quality (n = 37).

We also reviewed the extent to which each indicator could measure one of the SDG-11 targets or indicators, summarized in Figure 3. Out of 484 indicators we surveyed, about 40% (194) could potentially be leveraged to measure an SDG-11 target or indicator. The indicators were most relevant to Target 11.6, “By 2030, reduce the adverse per capita environmental impact of cities, including by paying special attention to air quality and municipal and other waste management” (77 indicators) and Target 11.2,

Figure 3.

Frequency of SDG indicators and targets as represented in reviewed indicators. (see UN DESA n.d.) for full indicator and target names.).

“By 2030, provide access to safe, affordable, accessible and sustainable transport systems for all, improving road safety, notably by expanding public transport, with special attention to the needs of those in vulnerable situations, women, children, persons with disabilities and older persons” (47 relevant indicators).

The indicators were also more applicable to SDG-11 targets (157) than to the more detailed and specific SDG-11 indicators (37).

Targets and baselines

We use targets and baselines, which are often used for performance management and goal-setting, to assess the degree to which urban sustainability indicators could be used for management. Only 43% of reviewed indicators had either specific or directional targets, leaving the remaining 57% with ambiguous management objectives. Figure 4 shows how the presence of different kinds of targets and baselines varies across different issue areas. Core issue area indicators have a similar pattern: many indicators with no target and no baseline, or indicators with only a directional target. Water and sanitation (8), land use (7), and air quality (6) indicators have the largest number of targets with a baseline. The perceptions and participation category has the largest percentage and absolute number of indicators that fall into the “Has Baseline Only,” category. For example, the Central Texas Sustainability Indicators project includes a series of indicators that gauge citizens’ knowledge about water consumption, including indicators such as, “percent of people who ‘definitely know’ the source of their drinking water” and report a baseline percentage of survey respondents who answered positively. The index, however, does not specify what a target is, whether 100% of respondents answering positively is a goal, or otherwise (see Table S2 for additional examples of indicator targets and baselines).

Figure 4.

Comparing the frequency of target type by issue area reveals that many of the indicators reviewed do not include a target. A visual examination of the table suggests that there are different patterns for some of the issue areas.

Equity indicators

Of the 484 indicators we evaluated, only 61 (13%) attempted to assess distributional equity, the only form of equity we assessed. Figure 5 illustrates the proportion of indicators in each issue area that was determined to be an equity-related indicator because it either disaggregated the indicator by a demographic group, or set a goal to ensure equity in distributional impacts for all populations (e.g. clean air quality for all). The environmental health category has the greatest proportion of equity indicators; 64% of the assessed environmental health indicators also relate to equity. Air quality (28%) and water and sanitation (18%) had the second and third highest proportion of equity-related indicators. The climate and energy, biodiversity and habitat, and natural hazard vulnerability categories had the fewest equity-related indicators.

Figure 5.

Percentage (count) of equity indicators by environmental issue area.

Of the equity indicators that specify a unit of analysis, 40% relate to the neighborhood level, making neighborhoods the most common unit of analysis for equity indicators. Twenty-four percent (23) of neighborhood-focused indicators relate to equity. Equity indicators tend to have a smaller unit of analysis than other indicators that are not related to equity, supporting the conclusion that spatial disaggregation enables measuring equity.

Data sources

Very few indicators included clear information about their underlying data sources. Many of the data sources for the indicators we could assess (67%, 114) were collected and published by government agencies (Cluster A indicators), while 33% (56) could be identified as using alternative methods (Cluster B indicators). The remainder (314) could not be assessed regarding the type of source data (Cluster A or B). Of the indicators we reviewed, 90 (19%) used open data sources. Regardless of Cluster A or B sources, publicly available data (i.e. open data) are more likely the source for equity indicators. Over half of the equity indicators were derived from open data sets, even though open data account for only a quarter of the indicators.

Discussion

Our review of existing urban environmental indicators suggests that they fall short of filling the data gaps in measuring progress towards SDG-11. Although many focus on relevant SDG-related environmental issues – water and sanitation, climate and energy, land use and built environment, transportation, waste management, and air quality, only roughly half of the indicators could be operationalized to assess progress towards an SDG-11 target or indicator. Transportation, air pollution, and waste were the most prominent themes in existing urban sustainability indicators, which could apply to SDG-11’s targets and indicators focusing on affordable and sustainable transit, reduction in environmental impact, and waste management. These results may reflect the high priority of these issues in both local and national agendas; indicators tracking these core issue areas had far higher levels of government-created indexes than other indicators. The large and often directly observable impact of air pollution, traffic, and waste management on city dwellers’ quality of life could also contribute to their prioritization by environmental management agencies.

Most indicators also originate from the Global North, which could bias both the construction of the indicators themselves and the locations they cover (Nagendra et al., 2018). While the coverage of the indexes we evaluated are relatively balanced between developed and developing countries (Figure S1), the coverage for economies in transition and least-developed countries is much lower. This gap is concerning, considering the expected growth in urbanization in these areas, with 55% of global urban growth anticipated in China and India and a 590% increase over 2000 levels of urbanization in the African continent (Seto et al., 2012). Additionally, given the requirement for SDGs to be universally applied, a challenge is determining whether existing urban sustainability and equity indicators are even applicable to these rapidly developing contexts, which could reveal different priorities in environmental measurement. Knowledge production systems developed in the Global North risk overlooking the differences in contexts and drivers of urbanization in the Global South (Marchetti et al., 2019; Nagendra et al., 2018), such as the vital role played by informal services (Katomero and Georgiadou, 2018; Pfeffer et al., 2019) or the importance of understanding the environmental conditions in informal settlements (Kuffer et al., 2018).

From a practical perspective, many indices reviewed did not include clear metrics that could be operationalized to track progress towards SDG-11. Most indicators lacked quantitative elements, such as targets and baselines, making progress on these indicators and urban environmental sustainability difficult to define or measure. The majority of the indicators fail to meet recommendations to be “outcome focused” (UN Statistical Division, 2015: 5), as well as “specific,” “measurable,” and “timebound” (Hák et al., 2016) – potentially making them more difficult to integrate into policymaking. In the City Prosperity Index, for example, some benchmarks, such as the Gini coefficient, poverty rates, and slum households, use the “best” observed value as the target for high performance, without setting higher aspirations in situations where there are poor scores across the board (UNHSP, 2017). Unambitious benchmarks may lead to a “race to the bottom” (Vogel, 1997) where needed progress is not achieved. The lack of benchmarks and targets pose practical challenges to performance management. An alternative is to adopt normative values; for example, the CPI’s gender indicators are benchmarked to 50% representation for women, which reflects a view of equal representation as equity.

Our results also reveal a gap between the focus on equity in the urban sustainability discourse and the attributes of urban sustainability indicators themselves. The ability to measure the distributional dimension of environmental equity – and therefore, to track SDG 11’s call for “inclusive, safe, resilient and sustainable” cities – varied across different issue areas. In total, only 13% of our reviewed urban sustainability indicators are capable of measuring distributional equity, and only 5% at the neighborhood level. The greatest number of indicators suitable for measuring equity was found in the environmental health issue domain, which may reflect a greater awareness of this topic’s link to environmental justice issues. It may also reflect the ability to access detailed spatial data. Air quality indicators can often be disaggregated at lower spatial scales due to recent advances in applying satellite measurements to estimate ground-based exposure to fine particulate pollution (PM_2.5) (Van Donkelaar et al., 2010).

Looking ahead

Despite the rapid increase of urban indexes, it is clear that greater investment in data collection for SDG-11 will be needed for cities to track progress towards SDG-11. Our findings are consistent with Klopp and Petretta (2017)’s conclusions that (1) the poor availability of standardized, open, and comparable data; (2) the lack of strong data collection institutions to support monitoring at the city scale; and (3) the challenges of “localizing,” the SDGs in widely different cities all hinder monitoring for SDG-11. At least 10 out of the 15 SDG-11 indicators require new monitoring approaches and tools for collecting, analyzing, and using information (UN Habitat, 2018), and even for cities in the Global North, capacity for data collection is low (Simon et al., 2015). Cities are also likely to face considerable challenges collecting data disaggregated by age, sex, and ability, as proposed in SDG-11, and acquiring more detailed datasets may be unrealistic given the current state of data collection and management administration. This challenge is especially true in developing countries, where institutional capacity and poor transportation and information technology infrastructure, among other factors, limit data collection (Simon et al., 2015). Spatially-explicit, disaggregated data will be particularly critical to locate spatial patterns and inequalities and identify policy priorities that can be more effective in targeting specific areas and social groups (Pfeffer et al., 2019; Ulbrich et al., 2019).

Where not available through official government sources, sub-city or neighborhood-level data could be developed from alternative data sources, such as citizen science, crowdsourcing, low-cost sensors, or commercial/private sector data (Fritz et al., 2019; Hsu et al., 2014). Open datasets such as Open Street Maps and earth observation data represent promising sources of spatially-explicit information that could be adapted to measure urban-scale environmental performance, though they require technical skill and capacity to obtain and analyze (Andries et al., 2019; Kuffer et al., 2018). Technology-based citizen engagement efforts, however, also need to carefully weigh potential ethical and privacy considerations (MacFeely, 2019). The Urban Environmental and Social Inclusion Index (Hsu et al., 2018), for instance, represents emerging research to measure progress on SDG-11’s urban environment and equity dimensions in a spatially-explicit, neighborhood-disaggregated way using open data sources and satellite remote sensing.

Conclusion

A promising shift in the global sustainability discourse has put equity and urban areas at the center of the sustainable development agenda. Through a review of 458 urban sustainability indicators, we reveal that existing metrics fall short of measuring progress towards SDG-11. While some urban-focused indicators address aspects of environmental performance, many existing indexes struggle to measure environmental equity. With a decade left to achieve the 2030 Sustainable Development Agenda, time is running out to demonstrate progress towards these goals. The UN’s own assessments reveal that the world is failing to make significant progress in most of the 17 goals, including SDG-11 (UNSD, 2019). Data and indicators to identify where progress is occurring, which policies and interventions are having an effect, and where challenges remain are urgently needed.

Supplemental Material

sj-pdf-1-epb-10.1177_2399808320975404 - Supplemental material for Sustainable and inclusive – Evaluating urban sustainability indicators’ suitability for measuring progress towards SDG-11

Supplemental material, sj-pdf-1-epb-10.1177_2399808320975404 for Sustainable and inclusive – Evaluating urban sustainability indicators’ suitability for measuring progress towards SDG-11 by Ryan Thomas, Angel Hsu and Amy Weinfurter in Environment and Planning B: Urban Analytics and City Science

Footnotes

Acknowledgements

The authors thank Nikola Alexandre (Yale School of Forestry of Environmental Studies/School of Management ‘18), Kavya Gopal (Yale-NUS College ‘18), Subhas Nair (Yale-NUS College ‘17), Jeffrey Tong (Yale-NUS College ‘18), Odelle Tan (Yale-NUS College ‘22), and Hannah Melville-Rea (NYU-Abu Dhabi ‘19) for assistance in data collection and coding.

Data availability statement

The coded data and all of the scripts used to produce the figures in this paper are available on Harvard Dataverse (Thomas et al., 2020; ).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding was provided by the Samuel Centre for Social Connectedness (Grant number: AWDR14157).

ORCID iD

Angel Hsu

Supplemental material

Supplemental material for this article is available online.

Ryan Thomas is a PhD candidate in the Department of Urban and Regional Planning at Cornell University.

Angel Hsu, PhD, is an Assistant Professor in Public Policy and Energy, Environment and Ecology at the University of North Carolina-Chapel Hill and the Director/Founder of the Data-Driven Environmental Policy Lab (Data-Driven Lab).

Amy Weinfurter is a Research Associate at the Data-Driven Lab.

References

Agyeman

Evans

(2004) ‘Just sustainability’: The emerging discourse of environmental justice in Britain? The Geographical Journal 170(2): 155–164.

Alberti

(1999) Urban patterns and environmental performance: What do We know? Journal of Planning Education and Research 19(2): 151–163.

Andries

Morse

Murphy

, et al. (2019) Translation of earth observation data into sustainable development indicators: An analytical framework. Sustainable Development 27(3): 366–376.

Bagliani

Bravo

Dalmazzone

(2008) A consumption-based approach to environmental kuznets curves using the ecological footprint indicator. Ecological Economics 65(3): 650–661.

Böhringer

Jochem

(2007) Measuring the immeasurable – A survey of sustainability indices. Ecological Economics 63(1): 1–8.

Bravo

Anthopolos

Bell

, et al. (2016) Racial isolation and exposure to airborne particulate matter and ozone in understudied US populations: Environmental justice applications of downscaled numerical model output. Environment International 92(Supplement C): 247–255.

Cooper

(1999) Which focus for building assessment methods – Environmental performance or sustainability? Building Research & Information 27(4-5): 321–331.

Davis

Kingsbury

Merry

(2015) Introduction: The local-global life of indicators: Law, power, and resistance. In: Merry, S., Davis, K., & Kingsbury, B. (eds) The Quiet Power of Indicators: Measuring Governance, Corruption, and Rule of Law. Cambridge, UK: Cambridge University Press, pp.1–24.

Economist Intelligence Unit (EIU) (2012) Siemens Green City Index. Available at: https://apps.espon.eu/etms/rankings/2012_European_Green_City_Index_sum_report.pdf (accessed 11 November 2020).

10.

Fainstein

(2014) The just city. International Journal of Urban Sciences 18(1): 1–18.

11.

Fritz

See

Carlson

, et al. (2019) Citizen science and the united nations sustainable development goals. Nature Sustainability 2(10): 922–930.

12.

Hák

Janoušková

Moldan

(2016) Sustainable development goals: A need for relevant indicators. Ecological Indicators 60: 565–573.

13.

Hezri

Dovers

(2006) Sustainability indicators, policy and governance: Issues for ecological economics. Ecological Economics 60(1): 86–99.

14.

Hsu

, et al. (2018) The Urban Environment and Social Inclusion Index. New Haven, CT: Yale University.

15.

Hsu

Malik

Johnson

, et al. (2014) Development: Mobilize citizens to track sustainability. Nature News 508(7494): 33.

16.

Joss

(2011) Eco-cities: The mainstreaming of urban sustainability – Key characteristics and driving factors. International Journal of Sustainable Development and Planning 6(3): 268–285.

17.

Katomero

Georgiadou

(2018) The elephant in the room: Informality in Tanzania’s rural waterscape. ISPRS International Journal of Geo-Information 7: 437.

18.

Kearney

(2019) The 2019 Global Cities Report. bit.ly/2019-Global-Cities. https://www.kearney.com/documents/20152/4977718/A+Question+of+Talent%E2%80%942019+Global+Cities+Report.pdf

19.

Kitchin

Lauriault

McArdle

(2015) Knowing and governing cities through urban indicators, city benchmarking and real-time dashboards. Regional Studies, Regional Science 2(1): 6–28.

20.

Klopp

Petretta

(2017) The urban sustainable development goal: Indicators, complexity and the politics of measuring cities. Cities 63: 92–97.

21.

Koch

Krellenberg

(2018) How to contextualize SDG 11? Looking at indicators for sustainable urban development in Germany. ISPRS International Journal of Geo-Information 7(12): 464.

22.

Kopnina

(2016) The victims of unsustainability: A challenge to sustainable development goals. International Journal of Sustainable Development & World Ecology 23(2): 113–121.

23.

Kuffer

Wang

Nagenborg

, et al. (2018) The scope of earth-observation to improve the consistency of the SDG slum indicator. ISPRS International Journal of Geo-Information 7(11): 428.

24.

MacFeely

(2019) The big (data) bang: Opportunities and challenges for compiling SDG indicators. Global Policy 10: 121–133.

25.

Maguire

Sheriff

(2011) Comparing distributions of environmental outcomes for regulatory environmental justice analysis. International Journal of Environmental Research and Public Health 8(5): 1707–1726.

26.

Marchetti

Oliveira

Figueira

(2019) Are global North smart city models capable to assess Latin American cities? a model and indicators for a new context. Cities 92: 197–207.

27.

Mattern

(2015) Mission control: A history of the urban dashboard. Places Journal, March 2015. Available at:https://placesjournal.org/article/mission-control-a-history-of-the-urban-dashboard/ ▪. (accessed 16 May 2019).

28.

Moonen

Clark

(2013) The business of cities 2013: What do 150 city indexes and benchmarking studies tell us about the urban world in 2013? Jones Lang LaSalle. Available at: http://docplayer.net/16590812-The-business-of-cities-2013.html (accessed 11 November 2020).

29.

Mori

Christodoulou

(2012) Review of sustainability indices and indicators: Towards a new city sustainability index (CSI). Environmental Impact Assessment Review 32(1): 94–106.

30.

Mori Memorial Foundation (2018) 2018 global cities power index yearbook. Available at: http://mori-m-foundation.or.jp/english/ius2/gpci2/index.shtml (accessed 11 November 2020).

31.

Nagendra

Bai

Brondizio

, et al. (2018) The urban South and the predicament of global sustainability. Nature Sustainability 1(7): 341–349.

32.

Ness

Urbel-Piirsalu

Anderberg

, et al. (2007) Categorising tools for sustainability assessment. Ecological Economics 60(3): 498–508.

33.

Parnell

(2016) Defining a global urban development agenda. World Development 78: 529–540.

34.

Pfeffer

Georgiadou

(2019) Global ambitions, local contexts: alternative ways of knowing the world. International Journal of Geo-Information ▪: ▪.

35.

Rawls

(2009) A Theory of Justice: Revised Edition. Cambridge, MA: Harvard University Press.

36.

Reckien

, et al. (2017) Climate change, equity and the sustainable development goals: An urban perspective. Environment and Urbanization 29(1): 159–182.

37.

Schlosberg

(2009) Defining Environmental Justice: Theories, Movements, and Nature. Oxford: Oxford University Press.

38.

Seto

Güneralp

Hutyra

(2012) Global forecasts of urban expansion to 2030 and direct impacts on biodiversity and carbon pools. Proceedings of the National Academy of Sciences of the United States of America 109(40): 16083–16088.

39.

Simon

, et al. (2015) Developing and testing the urban sustainable development goals targets and indicators – a five-city study. Environment and Urbanization 28: 49–63.

40.

Thomas

Hsu

Weinfurter

(2020) Replication data for: Sustainable and Inclusive – Evaluating urban sustainability indicators’ suitability for measuring progress towards SDG-11. Harvard Dataverse. Available at: https://doi.org/10.7910/DVN/30FLEB.

41.

Ulbrich

Porto de Albuquerque

Coaffee

(2019) The impact of urban inequalities on monitoring progress towards the sustainable development goals: Methodological considerations. ISPRS International Journal of Geo-Information 8(1): 6.

42.

UN Human Settlements Program (UNHSP) (2004) Urban indicators guidelines: Monitoring the habitat agenda and the millennium development goals. Available at: http://unhabitat.org/urban-indicators-guidelines-monitoring-the-habitat-agenda-and-the-millennium-development-goals/ (accessed 11 November 2020).

43.

UNHSP (2017) Measuring the new urban agenda and sustainable development goals: The city prosperity initiative. Presented at the UN habitat III, New Urban Agenda, Quito, Ecuador, October 2016.

44.

United Nations (2016) Global Indicator Framework adopted by the General Assembly (A/RES/71/313). New York, NY: Author.

45.

United Nations Statistical Division (UNSD) (2015) Discussion Paper on Principles of Using Quantification to Operationalize the SDGs and Criteria for Indicator Selection. [ESA/STAT/441/2/58A/14]. New York, NY: UN Statistical Division.

46.

UNSD (2019) The Sustainable Development Goals Report. Available at: https://unstats.un.org/sdgs/report/2019/The-Sustainable-Development-Goals-Report-2019.pdf (accessed 11 November 2020).

47.

UN-Habitat (2018) Tracking Progress towards Inclusive, Safe, Resilient and Sustainable Cities and Human Settlements: SDG 11 Synthesis Report – High Level Political Forum 2018. Nairobi, Kenya: UN-Habitat., Available at: https://doi.org/10.18356/36ff830e-en.

48.

United Nations Human Settlement Programme (UN-Habitat) (2018) Accelerating SDG-11 Achievement: Policy Brief in Support of the First SDG-11 Review at the UN High-Level Political Forum 2018. Nairobi, Kenya: Author.

49.

Van Donkelaar

Martin

Brauer

, et al. (2010) Global estimates of ambient fine particulate matter concentrations from satellite-based aerosol optical depth: Development and application. Environmental Health Perspectives 118(6): 847–855.

50.

Vogel

(1997) Trading up and governing across: Transnational governance and environmental protection. Journal of European Public Policy 4(4): 556–571.

51.

Young

(1990) The scaling of bodies and the politics of identity. Justice and the Politics of Difference 122–155.

52.

Zinkernagel

Evans

Neij

(2018) Applying the SDGs to cities: Business as usual or a new dawn? Sustainability 10(9): 3201.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.22 MB