Roadmap for the development of a resilience indicator system for protection and civil defense organizations: A literature review of the evidence

Abstract

BACKGROUND:

Disasters are the result of adverse events that cause human, material, environmental, and economic and social damage. To deal with disaster management, prevention, response, and recovery organizations need a system of indicators to measure their resilience.

OBJECTIVE:

To develop a road map to select indicators of organizational, institutional and governmental resilience to be applied to evaluate the resilience of public Protection and Civil Defense Organizations (PCDOs) of developing countries.

METHOD:

A literature review on resilience indicators for disaster management using Scopus database, identifying and classifying the resilience indicators available in the scientific literature, to discuss the possibilities of their application in PCDOs.

RESULTS:

Resilience indicators for disaster management available in the literature have many diverse classifications and they were developed for the evaluation of communities’ resilience. The literature review results also indicated that there is a lack of indicators to evaluate PCDOs’ resilience.

CONCLUSIONS:

Indicators of the institutional, organizational and governmental categories identified in the review, originally developed for the evaluation of communities’ resilience, can be used to compose a hybrid system of resilience indicators able to measure the resilience of PCDOs.

Keywords

Resilience engineering organizational resilience emergency management natural disasters

1 Introduction

Disasters, whether of human or natural origin, are the result of adverse events that cause human, material, environmental, economic and social damage. These phenomena have become increasingly a global concern because they pose great dangers to people’s lives, health and well-being, and compromise the normal daily routines of organizations and society [1].

Since the adoption of the Hyogo Framework for Action in 2005, progress has been made on disaster risk reduction (DRR) initiatives in several countries. However, disasters still have a major impact on societies around the world. In the last years about 700,000 people lost their lives, more than 1.4 million were injured, around 23 million were made homeless and more than 1.5 billion people were affected by disasters in various ways [2].

Disaster reduction strategies include vulnerability and risk assessment, as well as a range of institutional and operational capacities. The activities that guide disaster risk management proposed by the United Nations International Strategy for Disaster Reduction (UNISDR) are based on the preparation, response and recovery phases and involve the joint participation of organizations, governments and communities. The preparation phase refers to the knowledge and skills developed by governments, response and recovery organizations, professionals, communities and individuals, so as to effectively anticipate, respond to and recover from the impacts of events and conditions of probable, imminent or current risks. The response phase concerns the provision of emergency services and public assistance during or immediately following a disaster, in order to save lives, reduce impacts on people’s health, public safety and meet basic subsistence needs of individuals. In the recovery phase, measures are taken to restore and improve, where appropriate, the facilities, livelihoods and living conditions of communities affected by disasters, including efforts to reduce risk factors [3, 4].

To contribute to disaster risk reduction, as well as to minimize its effects, several studies, in different contexts, have been identified in the literature (Jülich [5]; Zhai et al. [6]; Xie et al. [7]; Vitoriano et al. [8]). Furthermore, governments, institutions, and related companies are increasingly interested in measuring their resilience, as well as the resilience of communities in risk situations. The aim is to assess the potential of institutions and communities to cope with disaster situations and to outline strategies for action. It was also observed that the scientific community has been interested in this subject and there are several tools to evaluate the resilience of communities (Cutter et al. [9]; Jordan and Javernick-Will [10]; Lam et al. [11]), as well as studies on resilience in organizations in various contexts (Grecco et al. [12]; Huber et al. [13]; Lee et al. [14]; Chan et al. [15]; Carvalho et al. [16]; Witmer and Mellinger, [17]). However, there is not much research on resilience indicators for public institutions or organizations that deal with disaster management issues. It is important to note that there are slightly different views about the meaning of resilience in an organization, in a public institution, or in a community. Therefore there is no comprehensive set of resilience indicators to be used by disaster management institutions or organizations such as Protection and Civil Defense Organizations (PCDOs).

The UNISDR [18] defines resilience as: “The ability of a system, community or society exposed to hazards to resist, absorb, accommodate to and recover from the effects of a hazard in a timely and efficient manner, including through the preservation and restoration of its essential basic structures and functions.” For Cutter et al. [19], resilience is like a set of capacities that can be promoted through interventions and policies, which, in turn, help to build and improve the community’s ability to respond to and to recover from disasters. For Hale and Heijer [20], resilience in an organization is about a management characteristic that focuses on anticipating and circumventing the threats. It appears in the ability to manage severe pressures and conflicts between safety and the organization’s primary production or performance goals. In engineering, Hollnagel et al. [21] define resilience as the intrinsic ability a system has to regulate its operation before, during or after changes and disturbances, so that it can maintain the required operations under both expected and unexpected conditions. Omidvar et al. [22] report that Resilience Engineering can be an alternative option to traditional risk assessment techniques, to predict and manage the security conditions of modern socio-technical organizations. It is perceived that in public organizations that deal with disaster risk management issues, the evaluation of their resilience through indicators would assist such organizations to know the factors that may contribute to their own resilience, which are internal to the institution (e.g., a team’s capacity to respond to threats) and external (e.g., the response capacity of the community affected by the disaster, collaboration of the population, change of the city’s traffic routes according to the disaster site, and so forth).

According to Lee et al. [14], the indicators serve to know the resilience of current situations and to monitor the behavior of resilience after the situation undergoes political or project interventions. Rose and Krausmann [23] reiterate that resilience indicators can be very useful in assessing the levels of various types of resilience and improving them over time. These indicators can also be a useful first step in assessing spending on resilience as part of the overall risk management process.

In Latin America and particularly in Brazil, the characteristics of socioeconomic development often lead to a disorderly growth of cities and, consequently, occupation and irregular construction in risk areas, causing problems such as waterproof soil, urban desertification, and insufficiency of the drainage, sewer, electricity and telephony systems [24].

Therefore, natural disasters, particularly those triggered by heavy rainfall, are causing major damage and deaths [25]. Due to the practical impossibility of removing people from risk areas, the Brazilian civil defense organizations are responsible for dealing with disaster situations, from early warning systems, training the population through simulations, improving people’s safety culture, and making the cities more resilient in the face of disasters [16]. For example, in the city of Natal-RN, Brazil, the municipal public agency for disaster risk management has faced many difficulties in reducing the risks of disasters, as already shown in the MãeLuíza neighborhood disaster [26]. This agency lacks a judicious evaluation system that allows an evaluation of its own resilience. Without such a system it is not possible to identify the issues that are negatively impacting its resilience and outline new resilience improvement strategies for disaster risk reduction and, consequently, to improve the city’s resilience to hazards and disasters.

This situation has resulted in the premise that for a city to become resilient to risks and disasters, it is necessary that the public agency responsible for disaster risk management should also become resilient. To do so, it is important that the agencies involved in disaster response and management have a system of indicators to evaluate their resilience.

This premise has triggered a systematic review in the available scientific literature on indicators (and systems of indicators) of organizational resilience to disasters, with the objective of identifying indicators (and systems of indicators) of resilience to disasters that could be applied in the Natal public agency and other agencies in Brazil, in order to assess their resilience.

This article is a literature review of the classifications of indicators (and indicator systems) of resilience available in the scientific literature to produce a roadmap for the development of a resilience indicator system for organizations that deal with disaster management. The specific objectives are to characterize the existing approaches, models, typologies, scope and applications on resilience measures in disaster organizations. The aim is to analyze and select those indicators and systems of indicators of resilience that can be used to measure and evaluate the resilience of a public agency of disaster risk management.

The overall objective of the roadmap is to provide easy ways to develop a system of organizational resilience indicators that will allow the organization to continuously monitor its resilience in dealing with risks and disasters, and thereby establish new guidelines and actions to continuously improve its performance, aiming at improving the city’s resilience in the face of risks and disasters.

2 Methods

A systematic review of the scientific literature on disaster resilience indicators was carried out initially in databases containing Brazilian journals in the Portuguese language (Scielo) and in other databases that contained international journals (Science Direct, Web of Science, Scopus). After an initial search and due to the objective of this study - to identify indicators and indicator systems of disaster resilience that could be applied in a public disaster risk management agency in order to assess their resilience –we realized that the Scopus database included most of the articles of importance for the research. Therefore the systematic literature review uses only the Scopus database.

In this literature review we collected, classified, and analyzed recent work related to the topic of resilience indicators for disaster management. We have highlighted scientific evidence on the efforts that have been made to design indicator systems and measure resilience in disaster situations. Articles were searched for in the Scopus database by combining the following keywords: resilience indicators, resilience measurement, and resilience index with the keyword disasters, using the search feature “And” as follows: resilience indicators and disasters (first search), resilience measurement and disasters (second search), and resilience index and disasters (third search). We always use the word disasters to avoid the large number of articles related to resilience indicators in situations other than disaster contexts.

The initial result of the first search was 170 documents. A filter was then applied by document type, in this case “Article”, resulting in 122 articles. For the second search, 83 documents were initially obtained, resulting in 54 articles after the application of the filter by document type, “Article”. In the third search, repeating the same procedures as before, 175 documents were initially obtained, which resulted finally in 123 articles. In all, the three searches resulted in 299 (122 + 54 + 123) articles. Therefore, the titles and abstracts of the 299 articles were read and the selection of articles was started based on the following criteria:

Articles on indicators for assessing the resilience of communities inserted in a disaster context;

Articles on indicators for evaluating the resilience of organizations inserted in a disaster or crisis context.

After considering the criteria defined, 27 articles remained. Subsequently, a reading of the 27 articles was carried out to identify the categories and dimensions of the indicator systems presented. Promptly, a new refinement of the research was made, considering the focus of this work in the category of so-called organizational, institutional and governmental indicators. In other words, indexes composed of indicators that were used to assess the resilience of communities, but which did not contain the category of organizational, institutional and governmental resilience, and the other indexes that did not have indicators applicable to a municipal/public management organization that deals with disaster management, were discarded.

This entire procedure resulted in the selection of nine papers, on which a detailed review was carried out. They are: Bruneau et al. [27]; Cutter et al. [9]; Cutter et al. [19]; Chan et al. [15]; Engle et al. [28]; Lee et al. [14]; Cutter et al. [29]; Dasgupta and Shaw [30] and Yoon et al. [31]. The contents of the selected articles were analyzed for the exploration and treatment of the results, through content analysis [32].

For this work, the content analysis was defined in three phases. In the initial phase the work plan was established, an initial analysis of the articles was carried out, and the first readings were made. In the second phase a systematized reading and recordings of the nine articles were carried out. In the third phase a matrix-synthesis (Table 1) was organized with the types of resilience indicators found in the articles selected in this bibliographic review.

Table 1
Typology of classification of the resilience indicators systems

Author Indicators system’s name Classification Types of classification Application areas Model nature

Bruneau et al. [27] No name Dimension Technical, organizational, social, and economic Communities, infrastructure and emergency systems Theoretical

Cutter et al. [9] DROP - Disaster Resilience of Place Dimension Ecological, social, economic, institutional, infrastructure, and community competence Communities Theoretical

Cutter et al. [19] BRIC - Baseline Resilience Index for Communities Dimension Social, economic, institutional, infrastructure and community capital Communities Applied

Engle et al. [28] No name Category Governance and security, natural resource systems, social systems, economic systems and built environment/infrastructure Development agency, organizations, government institutions Theoretical

Lee et al. [14] ROR - Relative Overall Resilience Factor Adaptive capacity and planning Organizations Theoretical

Cutter et al. [29] BRIC - Baseline Resilience Indicators for Communities Category Social, economic, community capital, institutional, housing/infrastructural and environmental Communities Applied

Chan et al. [15] No name Dimension Science and technical, built environment, organizations and institutes, social economic, natural environment Communities, organizations, governments Applied

Dasgupta and Shaw [30] No name Dimension Socio-economic, physical, institutional, coastal zone management and environmental/natural Communities Applied

Yoon et al. [31] CDRI - Community Disaster Resilience Index Dimension Human, social, economic, institutional, physical, environmental Communities Applied

Author	Indicators system’s name	Classification	Types of classification	Application areas	Model nature
Bruneau et al. [27]	No name	Dimension	Technical, organizational, social, and economic	Communities, infrastructure and emergency systems	Theoretical
Cutter et al. [9]	DROP - Disaster Resilience of Place	Dimension	Ecological, social, economic, institutional, infrastructure, and community competence	Communities	Theoretical
Cutter et al. [19]	BRIC - Baseline Resilience Index for Communities	Dimension	Social, economic, institutional, infrastructure and community capital	Communities	Applied
Engle et al. [28]	No name	Category	Governance and security, natural resource systems, social systems, economic systems and built environment/infrastructure	Development agency, organizations, government institutions	Theoretical
Lee et al. [14]	ROR - Relative Overall Resilience	Factor	Adaptive capacity and planning	Organizations	Theoretical
Cutter et al. [29]	BRIC - Baseline Resilience Indicators for Communities	Category	Social, economic, community capital, institutional, housing/infrastructural and environmental	Communities	Applied
Chan et al. [15]	No name	Dimension	Science and technical, built environment, organizations and institutes, social economic, natural environment	Communities, organizations, governments	Applied
Dasgupta and Shaw [30]	No name	Dimension	Socio-economic, physical, institutional, coastal zone management and environmental/natural	Communities	Applied
Yoon et al. [31]	CDRI - Community Disaster Resilience Index	Dimension	Human, social, economic, institutional, physical, environmental	Communities	Applied

This matrix is formed by nine rows and six columns, the lines being constituted by the articles selected in the bibliographic search and the columns constituted by the following elements referring to the articles: authors (first column); name assigned by the author to the system of indicators developed (second column); classification as to dimension, category or factor (third column); types of indicators or resilience indicator systems (fourth column); scope (place or scope where it was or should be applied) (fifth column); nature of the indicator model (if it is theoretical - is not proposed to be applied or has not yet been applied, or is applied) (sixth column).

It is important to note that dimension, category, or factors were terms used by the respective authors to classify the indicators of resilience in their manuscripts. That is, the indicators were classified according to the category or the factor. There is no universal terminology to classify the indicators, and each author uses the one that best suits him/her, without presenting an explanation of the choice. The typology of resilience and indicators of resilience is also not universal. There is, for example, an author who uses the term ecological resilience while another uses environmental resilience for the same purposes.

Next, the indicators of resilience identified in the institutional, organizational and governmental categories were systematized (Table 2) and discussed, showing their possibilities of application and their limitations.

Table 2

Indicators of organizational, institutional and governmental category of resilience

Author	Category	Indicators	Initials
Bruneau et al. [27]	Organizational	Robustness (continued ability to carry out designated functions); Redundancy (back-up/duplicate systems, equipment and supplies); Resourcefulness (plans and resources to cope with damage and disruption); Rapidity (minimize time needed to restore services and perform key response tasks)	-
Cutter et al. [9]	Institutional	Participation in hazard reduction programs (NFIP Storm Ready); Hazard mitigation plans; Emergency services; Zoning and building standards; Emergency response plans; Interoperable communications and continuity operations plans	DROP
Cutter et al. [19]	Institutional	Mitigation; Flood coverage; Municipal services; Political fragmentation; Previous disaster experience; Mitigation and social connectivity	BRIC
Engle et al. [28]	Governance and security	Disaster prevention plan; National governance indicators; Drought preparedness; River basin preparedness	-
Lee et al. [14]	Organizational	Minimization of silos; Internal resources; Staff engagement and involvement; Information and knowledge; Leadership; Innovation and creativity; Decision making; Situation monitoring and reporting; Planning strategies; Participation in exercises; Proactive posture; External resources and recovery priorities	ROR
Cutter et al. [29]	Institutional	Mitigation spending; Flood insurance coverage; Jurisdictional coordination; Disaster aid experience; Local disasters training; Performance regimes-state capital; Performance regimes-nearest metro area; Population stability; Nuclear plant accident planning and crop insurance coverage	BRIC
Chan et al. [15]	Organizational and institutional	Disaster prevention plans; River basin management organizations; Resource distribution capability
Dasgupta and Shaw [30]	Institutional	Laws and policy; Coordination; Emergency response; Adaptive action and governance	-
Yoon et al. [31]	Institutional	Rate of structural disaster mitigation capacity; Rate of non-structural disaster mitigation capacity; Preparedness of hazard mitigation plan; Preparedness of rainwater outflow reduction plan and preparedness of detention facilities plan	CDRI

3 Classification of resilience indicators

From the analysis of the articles that make up this bibliographical review, it was evidenced that the systems of indicators receive the most diverse classifications and nomenclatures, as observed in Table 1.

Bruneau et al. [27] proposed a conceptual framework to define the resilience of communities to seismic disasters. The framework developed by the authors integrates measures in four dimensions of community resilience defined by them (TOSE - Technical, Organizational, Social and Economic). The dimensions are explained below.

The technical dimension of resilience concerns the ability of physical systems to perform their functions at acceptable and/or desired levels when subjected to earthquake forces.

The organizational dimension of resilience refers to the ability of the organizations that manage critical facilities and are responsible for performing critical functions related to disaster, to make decisions and take actions that contribute to the attainment of resilience properties (robustness, redundancy, resourcefulness and speed).

The social dimension consists of measures that are designed to minimize the extent to which earthquake-hit communities and government jurisdictions suffer from negative consequences due to the loss of essential services due to the earthquakes.

The economic dimension refers to the ability to reduce direct and indirect economic losses resulting from earthquakes.

Bruneau et al. [27] relate the dimensions of resilience measures to resilience properties, the latter defined by the authors as follows:

Robustness: refers to the strength or capacity of elements, systems and other units of analysis to support a certain level of stress or demand without suffering degradation or loss of function;

Redundancy: characterized by the existence of elements, systems or other units of analysis that are replaceable, that is, capable of satisfying the functional requirements in the case of rupture, degradation or loss of functionality;

Development: consists of the ability to identify problems, define priorities and mobilize resources when there are conditions that threaten to reach some element, system or other unit of analysis;

Speed: consists of the ability to meet priorities and achieve objectives in a timely manner in order to contain losses and avoid future disruptions.

Bruneau et al. [27] proposed a set of 80 illustrative resilience measures presented in five tables. The resilience measures are related to four dimensions of resilience (technical, organizational, social and economic), to the four resilience properties (robustness, redundancy, resourcefulness and speed) and to five systems (global, electricity, water, hospital, response and recovery systems).

The authors report that the proposed framework allows an assessment of the contribution of resilience to earthquakes of various activities, whether components, systems or organizations, with applications ranging from essential systems such as electricity, water supply and telephone, as well as construction systems, and organizations providing these critical services. However, the model proposed by the authors is theoretical and they do not present any application that proves its validation.

Cutter et al. [9] based their work on previous studies and undertook an effort to systematize the dimensions of resilience disseminated in the scientific literature to understand and measure community resilience in relation to natural hazards through the DROP-Disaster Resilience of Place model.

These authors, when developing the resilience measurement model at community level, refer to the indicator dimensions for community resilience, classifying these indicators according to the same typologies used to classify resilience: ecological, social, economic, organizational, or institutional, infrastructure and community competence. Cutter et al. [9] defined these indicators according to the following dimensions:

Ecological or ecological systems - relates to biodiversity factors, redundancies, diversity of response, spatiality, and management and governance plans that influence resilience;

Social - can be enhanced through improvements in communications, risk awareness and preparedness, and also through the development and implementation of disaster plans, insurance policy and information sharing to assist in the recovery process;

Economic - relates to losses of property and the impacts caused by the interruption of post-event business;

Organizational or institutional - encompasses institutions and organizations. It consists of assessments of physical properties, such as the number of members, communications technology and the number of emergency assets (vehicles, hospital beds etc.). It also includes elements that measure how organizations manage or respond to disasters, such as organizational structure, capacity, leadership, training and experience;

Infrastructure - involves the physical systems themselves, as well as their dependence or interdependence with each other;

Community competence - measures how well the community functions before and after the disaster, including community senses and ideals, as well as the desire to be in the same place and to preserve cultural norms.

Cutter et al. [9] report that the DROP model presents the resilience capacity as a dynamic process dependent on the antecedent conditions, the severity of the disaster, the time between the events and the influence of exogenous factors. The model is theoretical and they suggest additional research on measures of resilience in order to operationalize the model.

Thus, in a new paper, Cutter et al. [19] use the DROP model to develop indicators to measure key community characteristics that promote resilience. The model was titled BRIC - Baseline Resilience Index for Communities. The BRIC set of indicators adopts the dimensions of resilience proposed by Cutter et al. [9] in the DROP model, with the exception of the ecological resilience dimension. The authors justify the exclusion because of inconsistent data on the resilience of ecological systems when applied to large and diverse study areas, in this case in regions far away from the coast or in wetlands and dunes.

In Cutter et al. [19], the dimension “community competence” defined in Cutter et al. (2008) is titled this time as “community capital”. In this way, the BRIC is composed of the following dimensions: social, economic, institutional, infrastructure and community capital.

Unlike the DROP model in Cutter et al. [9], the BRIC model was applied in comparative studies among 736 municipalities within the Federal Emergency Management Agency (FEMA). As a result, the authors concluded that there are variations in the level of resilience to disasters and these variations are more evident in rural areas, noting that communities in urban areas have higher rates of resilience. The authors suggest that the BRIC can help to trigger initial interest in the research, allowing community discussions, and attracting the public and stakeholders to promote communities resilient to disasters.

In a new study, Cutter et al. [29], after conducting an analysis of indicators published in various scientific papers, used cities in the United States as a unit of study and assessed the disaster resilience and geographic variability applied at specific sites using the BRIC model. In this work, the authors consider the term community as a locality, rather than the relational interactions between populations, organizations and other institutions. In order to demonstrate the method, Cutter et al. [29] used a common set of variables to measure the inherent resilience of cities in the United States.

In this context, the authors define inherent resilience as the qualities of a community, arising from daily processes, which can improve or impair their capacity to prepare, respond, recover and mitigate environmental events. The indicators are classified by Cutter et al. [29] in six types of resilience categories, already advocated by these authors in Cutter et al. [9] and Cutter et al. [19] and the previously designated dimensions: social, economic, housing/infrastructure, institutional and community capital. The sixth category in this study by Cutter et al. [29] is the environmental category, which is related to the quality of the environment, which increases the capacity of wave absorption, deals with flooding, among others. The authors also include indicators that estimate the efficiency of the community using natural resources.

Cutter et al. [29] concluded with the BRIC application that municipalities with lower inherent resilience were due to low rankings of indicators in housing, infrastructure, institutional, community capital, and environmental resilience categories. The authors suggest that the BRIC may assist decision making by highlighting where certain types of initiatives or programs can improve the resilience of communities to disasters.

Engle et al. [28], based on several methods described in the literature, propose a hybrid approach for the development of indicators, to guide decision making and adaptation to climate change. The authors suggest a framework for the development of indicators in which the analysts can select the most suitable structure for specific sites or sectors. Based on the analysis of previous research efforts on resilience and vulnerability, the authors identified five categories of resilience indicators. They point out that these categories may need modification in specific cases and other categories may be added. Engle et al. [28] describe the five categories of resilience indicators that guide their approach, as follows:

Administration and safety - concerns the role that government organizations play in the ability to cope with and adapt to climate change. They reflect aspects of resilience governance, identifying structures, processes and mechanisms that could facilitate and improve coping with immediate disasters and long-term adaptations;

Natural resource systems - consist of assessing the conditions of natural resources and the ways in which people use and depend on them. This helps to characterize ways in which populations can be vulnerable to the impacts of climate change;

Social systems - the systems that merge larger communities and societies. Demographic indicators tell stories about how people live and are connected and therefore how they may respond together in a new climate regime;

Economic systems - refers to indicators such as the per capita measure of Gross Domestic Product (GDP), changes in the labor market and labor trends, which can be evaluated for their potential to enable short-term/long-term adaptation;

Built environment and infrastructure - the environments that can protect against short-term climate impacts and gradual long-term climate change. Systems that have this potential include, for example, the energy industry, transport, water infrastructure, commercial and residential buildings, and energy/communication infrastructure.

Engle et al. [28] set out some principles for applying the hybrid approach proposed by them, including: adopting an approach that transparently discloses definitions, variables and data sources; investing in continuous iterations with stakeholder groups; starting with and adapting indicators already in use; considering the feasibility of implementation and the extent to which the proposed indicators are understandable for decision makers, among others. The authors add that a decision maker can use the hybrid framework to analyze resilience as they have described it. However, together with their theoretical model, Engle et al. [28] recommend that a number of unresolved practical, methodological and theoretical questions should be considered as a priority for research and development.

Lee et al. [14] proposed a tool called Relative Overall Resilience (ROR) to measure and compare the resilience of organizations. The tool can be used to identify the strengths and weaknesses of resilience and help organizations understand their current resilience, so that they can develop improvement strategies. The relative overall resilience (ROR) model of McManus [33], on which Lee et al. [14] based their study, presents three factors of organizational resilience, which are defined by McManus [33] as follows:

Situation awareness - is defined as a measure of the organization’s understanding and perception of its entire operating environment. This includes the ability to anticipate opportunities, potential crises and the ability to accurately identify crises and their consequences;

Management of keystone vulnerabilities - refers to the operational and managerial aspects of an organization that have the potential to have significant negative impacts in a crisis situation. There are two main aspects to identifying keystone vulnerabilities. The first is the speed that a component failure entails in a negative, fast or insidious impact, and the second is the required number of component failures to have a significant negative impact on the organization;

Adaptive capacity - a measure of the culture and dynamics of an organization that allows it to make decisions in a timely and appropriate manner, both day-to-day in normal business and also in crises. Adaptive capacity considers aspects of an organization that may include but are not limited to: leadership structures and decision-making, the acquisition, dissemination and retention of information and knowledge; and the degree of creativity and flexibility that the organization promotes or tolerates.

Lee et al. [14] verified the McManus ROR model indicators through a literature review and workshops with managers and staff from four organizations that were part of the original McManus [33] survey in order to identify potential gaps and update the model template if necessary. This verification resulted in a second organizational metrics model, and each item was tested using a Likert scale. The authors aimed to verify the usability of the tool and check the validity of the items.

Then, the indicators defined by McManus [33] and proposed additional indicators were reorganized and incorporated into a new model that operationalizes resilience as a function of two factors: factor 1 was called adaptive capacity and factor 2 was called planning. Lee et al. [14] report that organizations can use this model to measure their resilience, enabling them to obtain information about their resilience strengths and weaknesses to indicate how resilient they are, if their resilience levels meet their expectations and those of their stakeholders and, consequently, what they can do to improve their resilience. Considering the theoretical character of the model, Lee et al. [14] emphasize some limitations and suggest that studies for new model tests should be done using factorial analysis. They also report the need to verify whether the model is unique or whether it is applicable in any organization.

Chan et al. [15] combined the Fuzzy Delphi Method (FDM) and the Analytical Network Process (ANP) in order to establish a set of disaster resilience indicators for the reconstruction of the urban area in Tan-Sui, Taiwan. The authors classified the five-dimensional resilience indicators, which are described below:

Science and technical - regarding rescue and preparedness resources;

Built environment - targets life support systems and infrastructure capacity. In the context of the work developed by the authors, the built environment concerns the spatial structure of urban and regional areas in the Tan-Sui river basin;

Organizations and institutions - refers to developed hardware and software plans in order to avoid and cope with disasters. They also involve management capacity and institutions to deal with catastrophe outbreaks;

Socioeconomic - considers the social and economic capacity for control of emergency situations and recovery works. Includes the government’s financial capacity to deal with disasters such as typhoons and floods;

Natural environment - focuses on the high potential of landslides, floods and other hazards. It also includes water conservation areas and slope areas for conservation plans.

The work of Chan et al. [15] presents an interdependence solution of indicators based on FDM and ANP techniques, through interviews with group specialists. The authors used the Tan-Sui River Basin (Taiwan) as an example and demonstrated the effectiveness of the methodology proposed in the prioritization of disaster resilience indicators.

Chan et al. [15] claim that the model can be used as a basis for evaluation, development and planning of indicators for disaster resilience. The results of the study can also be used as a reference for the development of government policies. Furthermore, the integrated research model can be used directly as an objective and systematic evaluation tool in the field of disaster resilience research. It can also be extended to solve relevant problems of multiple criteria in the management, evaluation and decision-making in other fields of urban science.

Dasgupta and Shaw [30] developed a set of indicators to assess the resilience to disasters of rural communities off the coast of Asian mega deltas. The authors present a framework of indicators for assessing coastal resilience in five dimensions: socioeconomic, physical (structural), institutional, coastal/ecological, and environmental/natural zone management. Dasgupta and Shaw [30] describe each of these dimensions of indicators as follows:

The socio-economic dimension of the resilience of coastal rural communities consists of community competence for the sustainable use of resources. Indicators and variables used to measure socioeconomic resilience include demography, livelihoods, health, education and awareness;

The physical (structural) dimension corresponds to the indicators related to transport, residential infrastructure, electricity, telecommunication, water infrastructure and sanitation. Dasgupta and Shaw (2015) report that these indicators are derived from the studies of Cutter et al. [9] and Joerin and Shaw [34];

Institutional resilience indicators aim to measure the institutionalization of disaster risk reduction, adaptation to climate change, as well as local government response mechanisms to existing risks. The indicators incorporate measures of the capacity of institutions to deal with crisis situations and also incorporate specific variables to measure lack of financial capacity, corruption and lack of coordination;

The management dimension of the coastal zone concerns an integrated process combining a complex set of social, economic and environmental information for the sustainable development of coastal regions. Among the five main indicators proposed in the table, four have ecological significance, such as biodiversity conservation and pollution control. In addition, all the variables included in the management dimension of coastal zones have some relevance for the ecological performance of Sundarbans mangroves;

The natural or environmental resilience dimension corresponds to the exposure of the region to specific coastal and terrestrial risks, including erosion, rising sea level, salinity etc. In addition, it contemplates the action of the environmental guard in the fight against environmental threats due to natural, biogeochemical and geophysical factors.

Dasgupta and Shaw [30] concluded that community resilience in Indian Sundarban follows an inversely proportional relationship with the region’s exposure to the coast, that is, resilience tends to decrease with the proximity of the sea. This situation is aggravated by increasing deficits, livelihoods, institutional arrangements and inadequate management of coastal areas. The authors add that in order to improve the existing level of community resilience, a number of structural and non-structural measures are required. These measures may differ depending on geographic location. However, in addition to the structural measures, there is a strong requirement for “adaptation” at community level, which must be incorporated through gradual institutional interventions.

Yoon et al. [31],assessing the disaster resilience of communities in Korea, constructed a community-to-disaster resilience index called CDRI - Community Disaster Resilience Index. The indicator system assesses resilience in the human, social, economic, institutional and physical/environmental dimensions. These dimensions are defined by Yoon et al. [31] as follows:

Human dimension - includes demographic attributes of a community in terms of age, gender, level of education, family characteristics and occupation. Based on several authors, Yoon et al. [31] report that communities with older people, more women and less educated families are likely to demonstrate greater vulnerability and lower levels of resilience to disasters compared to others composed of younger people;

Social dimension - relates to the social capital of the community, that is, the function of trust, norms and social networks, among other characteristics that guide the coexistence and the relationship between people. Yoon et al. [31] used volunteering as a substitute for social capital in their study;

Economic dimension - refers to the percentage of the population’s social security beneficiaries, the local government security budget and the per capita disaster relief fund. Thus, communities with fewer low-income residents and larger budgets to spend on disaster recovery are likely to have a greater capacity for absorption, response and recovery to emergencies;

Institutional dimension - relates to local governments in terms of performance and preparedness to mitigate the negative impact of disasters. Indicators include the level of structural and non-structural performance mitigation measures and whether the local government has adopted plans for disaster reduction. Structural mitigation measures include improving the capacity of facilities for disaster reduction, including the maintenance of flooded rivers/plains, water/sewage facilities, and rainwater holding facilities. Non-structural mitigation measures include the designation of hazardous areas through risk assessment, adequate levels of coordination and networks among government organizations related to disaster response and recovery, adequate investment levels for disaster mitigation projects, and training and education in response to disasters;

Physical dimension - includes measures on the percentage of impermeable surfaces, number of building permits, number of dams and urban areas;

Environmental dimension - includes the total green area, such as parks, open spaces and wetlands, average number of rainy days etc.

Yoon et al. [31] confirmed that all the resilience indices have statistically significant impacts on the reduction of property damage and human losses, and that these impacts are different for each municipality. Thus, each locality has a distinct character in terms of disaster resilience and must develop its own strategy that reflects local conditions for increasing community resilience.

4 Selection of resilience indicators

Following the classifications of the resilience indicators of several systems cited in this review, Table 2 summarizes the indicators related to organizational, institutional and governmental resilience.

The organizational indicators proposed by Bruneau et al. [27], although formulated in a theoretical model and developed in the context of seismic disasters, are useful and can be adapted to other types of disasters because they portray some of the necessary capacities that are reflected in the resilience of organizations responsible for infrastructure services and emergency control.

The institutional indicators of the DROP model of Cutter et al. [9] were developed in the context of natural disasters. Although Cutter et al. [9] did not validate their model, the indicators reflect key factors that government and emergency institutions should consider in order to achieve resilience.

The institutional category of the BRIC model of Cutter et al. [19] presents indicators that are specific to the context in which they were developed, but these indicators portray the linkage of community resilience with the resilience of organizations that are part of disaster risk management, including the issue of awareness that these organizations must have on the preparation of the community to face these phenomena.

The application of these indicators in developing countries is made more limited by the lack of effective public policies for disaster risk reduction, but the proposed metrics may serve as insights for the development of indicators that seek to represent the interconnection between the resilience of communities and organizations that deal with disaster management.

Engle et al. [28], referring to the governance category of their model, proposed metrics that are of great value for the development of other indicators in several scenarios of climate risks and for the evaluation of the resilience of governmental institutions. However, some of the suggested metrics are limited in their application in developing countries, where public policies for disaster risk reduction are minimal and social inequalities and political conflicts make some of the proposed metrics impractical.

In the ROR model of Lee et al. [14], the proposed indicators are consistent with the organizational practices that should be adopted by organizations in order to achieve resilience. In fact, the indicators reflect an organization at the level of excellence in its internal and external operations. Although the metrics have been proposed in a crisis context, they can be adapted to the context of disasters and to assess the resilience of organizations dealing with disaster prevention, response and recovery.

Cutter et al. [29], when addressing the institutional category of the BRIC model, proposed specific indicators limited to the context in which they were applied. However, the indicators reflect some of the necessary skills of government institutions to build more resilient communities, and can be used to generate other indicators in similar contexts.

The indicators proposed by Chan et al. [15] in the organizational category are not explicit, but other important indicators may be originated from the proposed metrics, in particular, by means of the capacity management and disaster coping metric. From this perspective, indicators can be developed to assess the capacities of disaster mitigation, response and recovery actions. In addition, new indicators for the capacity to distribute other essential resources, other than water, could be incorporated into a new index.

Dasgupta and Shaw [30] have proposed, in the institutional category, consistent indicators to evaluate the resilience of public institutions that are directly linked to disaster risk management. Even if they are defined in a context of risks in coastal zones, the indicators can be adapted for assessments in other geographical locations. The proposed metrics may be useful in developing other indicators that are better suited to the context of government institutions in developing countries.

The institutional indicators proposed by Yoon et al. [31] in the CDRI model are cohesive and reflect the resilience measures that translate the capacity of government institutions to be prepared for disaster response. The metrics presented are global and can be fragmented in order to generate new indicators that can also measure the disaster response and recovery capacity of governmental organizations in different parts of the world.

4.1 Initial version of indicators system

Subsequently, bibliographical research was also used in documents related to UN World Conferences on Disaster Risk Reduction (DRR) and National Civil Protection and Protection System (SINPDEC) legislation [42]. Also included were indicators of resilience that were not classified by resilience dimensions/categories/factors: the disaster recovery indicators proposed by Horney et al. [35] and indicators to assess the performance and effectiveness of risk management proposed by Carreño et al. [36].

From the in-depth analysis of the indicators from the literature review and bibliographic research, the first version composed of 46 indicators was created. These indicators were classified according to the 4 phases of the disaster management cycle: Phase I - Prevention and Mitigation; Phase II - Preparation; Phase III - Response; Phase IV - Recovery. Table 3 presents the 46 indicators and the respective paper, and is the first version of the indicator system.

Table 3
System of indicators from literature review

Phase Indicator Author

I –Prevention and Mitigation 1.1 - Survey and update of information on vulnerability, hazards or threats and risks of the city Carreño et al.[36]; Horney et al. [35]

1.2 - Population covered Cutter et al. [19]

1.3 - Prior withdrawal of people from risk areas Carreño et al. [36]

1.4 - Public information Horney et al. [35]

1.5 - Alarm and alert systems Carreño et al. [36]; Chan et al. [15]; Engle et al. [28]

1.6 - Database Carreño et al. [36]

1.7 - Investment allocated to DRM Dasgupta and Shaw [30]

1.8 - Reserve funds for institutional strengthening Carreño et al. [36]

1.9 - Public meetings Horney et al. [35]

1.10 - Disaster response experience Cutter et al. [9]

1.11 - Use of technology Chan et al. [15]

1.12 – Interinstitutional relationship, multiagency Carreño et al. [36]; Dasgupta and Shaw [30]

1.13 - Stimulating and supporting innovation and creativity initiatives Engle et al. [28]

1.14 - Decision making delegated to agents Engle et al. [28]

1.15 - Strong leadership and good management Engle et al. [28]

1.16 - Continuous assessment of strategies and work plans Engle et al. [28]

1.17 - Standardization of processes Cutter et al. [9]

1.18 - Job stability Engle et al. [28]

1.19 – Proactivity Engle et al. [28]

1.20 - Projects and works SINPDEC [42]

1.21- Relationship with agencies linked to SINPDEC at the municipal level SINPDEC [42]

1.22 - Provision of information to CEMADEM SINPDEC [42]

1.23 - Estimated time to start response actions Bruneau et al. [27]

1.24 - Risk communication Dasgupta and Shaw [30]

1.25 - Articulation with vulnerable communities Cutter et al. [29]; Carreño et al. [36]

1.26 - Contingency plan Horney et al. [35]; SINPDEC [42]

1.27 - Provision for emergency resources Dasgupta and Shaw [30]

1.28 - Guidelines for recovery or recovery plans Horney et al. [35]; Carreño et al. [36]

II –Preparation 2.1 - Numbers of protection and civil defense agents Cutter et al. [9]

2.2 - Empowering vulnerable communities Cutter et al. [9]; Cutter et al. [29]; Carreño et al. [36]

2.3 - Training of school, hospital and health unit communities Cutter et al. [9]; Carreño et al. [36]

2.4 - Training of agents Cutter et al. [9]; Engle et al. [28]

2.5 - Simulated training Cutter et al. [9]; Engle et al. [28]; Dasgupta and Shaw [30]

2.6 -Team arrival time at the disaster site Bruneau et al. [27]

2.7 - Planning and organization of simulated trainings Engle et al. [28]

3.1 - Human and material resources to maintain actions Bruneau et al. [27]

3.2 - Mobilization of other agents as needed Chan et al. [15]

III - Response 3.3 - Systems or technologies Chan et al. [15]; Carreño et al. [36];

4.1 - Emergency building standards Cutter et al. [9]

4.2 - Emergency funds Dasgupta and Shaw [30]

IV –Recovery 4.3 - Continuity of operations/recovery plans Cutter et al. [9]

4.4 - Compensation for disaster victims Cutter et al. [9]; Engle et al. [28]; Cutter et al. [29]

4.5 - Recovery priorities Engle et al. [35]

4.6 - Number of public meetings Horney et al. [35]

4.7 - Updating recovery plans Horney et al. [35]

4.8 - Monitoring the progress of actions Horney et al. [35]

A roadmap for resilience indicators provides ways to align all stakeholders involved in a project around the same sequential steps towards the integral construction of the indicator system, leaving everyone involved aware of the evolution process and what variables are involved in each path. This approach follows the basic concepts of system/participatory ergonomics of Wilson [37], involving scholars, experts and PCDOs agents as much as possible in successive validation steps during the development of the indicators system. As indicated by Kleiner [38], it is important to involve workers in planning and controlling their own work activities to reach desirable goals, and the indicator system is a high-level tool that influences how work activities should be done.

Because of the importance of the validation process, we recommend the content validation. According to Wynd et al. [39], content validation is an essential step for the development of new empirical measurement tools, because the validation process enables the association of abstract concepts with observable and measurable indicators. DeVon et al. [40] add that in the development of any instrument, the results obtained through bibliographic research need to be validated with scholars/researchers in the domain and representatives of the population of interest. Polit and Beck [41] define content validation as a judgment process, arguing the need for evaluation of the instrument through the analysis of experts in the domain of interest, or multiprofessional validation.

Based on the recommendations cited, the roadmap for the development of resilience indicators was divided into 3 phases, as shown in Fig. 1. The theoretical construction phase includes the systemic literature review shown in this study. The next steps to be taken are: the evaluation by scholars in a first validation step; then, the multiprofessional validation, which includes an evaluation by PCDOs professionals. The result should be an indicator system customized according to the characteristics of each PCDO. The last step included in the roadmap is the situated validation, which is a pilot test by means of which managers of PCDOs will evaluate to what extent their organizations comply (or not) with the resilience indicators proposed.

Fig. 1

Roadmap for the development of the resilience indicator system.

5 Conclusions

The objectives proposed in this article were to identify and describe the categories of resilience indicators and to systematize the indicators according to the respective categories/dimensions, focusing on the organizational, institutional and governmental categories/dimensions of resilience, to produce a roadmap for the development of resilience indicator systems to be applied in government organizations that deal with disaster risk management.

This review showed that there are many classifications for resilience indicators and some indicators receive different nomenclatures, although they are destined to the same end. A few indicators developed for specific disaster contexts are difficult to apply in different situations regarding the type of the disaster or the country/region development level. However, most of the proposed metrics and indicators are adaptable to other circumstances beyond those for which they were developed.

As an alternative to the small number of systems of organizational indicators found in this literature review, it is assumed that the use of the institutional, organizational and governmental categories in the indicator systems for the evaluation of community resilience reflect well the interconnection of government actions, disaster/risk management institutions, and essential infrastructure such as emergency rescue, water and electricity distribution in producing community resilience. It becomes clear that, being resilient, these organizations contribute directly to the quick recovery of the communities.

It can be concluded that the indicators of resilience obtained in the scientific literature, classified in the institutional, organizational and governmental categories, may form part of a possible hybrid system of indicators of resilience, capable of measuring the resilience of essential service organizations and government disaster management institutions.

Therefore, and based on the existing gap in the scientific literature, the need is emphasized to construct a system of indicators for the evaluation of organizational resilience, intended for government institutions, such as Civil Protection and Defense, especially in developing countries such as Brazil, because these institutions are responsible for municipal disaster prevention, preparedness, response and recovery actions.

Building resilient cities in developing countries relies on the resilience of civil protection and defense agencies and vulnerable communities. Continuous evaluations of the resilience of these organizations, as well as of the community, help in the formulation and execution of public policies aimed at reducing disaster risks.

The roadmap presented in this article provides the first step in building a system of resilience indicators for agencies dealing with disaster risk management, especially Civil Protection and Defense organizations in Brazil. It also indicates, according to the nature, characteristics and metrics of such indicator system, that the participatory and situated development approach will have better chances of effectiveness.

Conflict of interest

None to report.

Footnotes

Acknowledgments

The authors gratefully acknowledge the support of Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) and Conselho Nacional de Pesquisas (CNPq).

References

Lindell

. Disaster studies. Current Sociology Review. 2013;61:797–825.

UNISDR. Sendai Framework for Disaster Risk Reduction. UNISDR, Geneva. 2015. Available from http://www.preventionweb.net/.

UNISDR. Living with a Risk: A Global Review of Disasters Reduction Initiatives. International Strategy for Disaster Reduction. 2004;1-457.

UNISDR. Disaster Resilience Measurements: Stocktaking of Ongoing Efforts in Developing Systems for Measuring Resilience. United Nations Development Programme. 2014;1–59.

Jülich

. Development of a Composite Index with Quantitative Indicators for Drought Disaster Risk Analysis at the Micro Level. Human and Ecological Risk Assessment: An International Journal. 2014;21:37–66.

Zhai

, Li

, Chen

. Reducing Urban Disaster Risk by Improving Resilience in China from a Planning Perspective. Human and Ecological Risk Assessment. 2015;21:1206–1217.

Xie

, Li

, Wu

, Hao

. Disaster Risk Decision: A Dynamic Computable General Equilibrium Analysis of Regional Mitigation Investment. Human and Ecological Risk Assessment. 2015;21:81–99.

Vitoriano

, Rodríguez

, Tirado

, Martín-Campo

. Intelligent Decision-Making Models for Disaster Management. Human and Ecological Risk Assessment: An International Journal. 2015;21(5):1341–1360.

Cutter

, Barnes

, Berry

, Burton

, Evans

, Tate

, et al. A place-based model for understanding community resilience to natural disasters. Glob. Environ. Chang. 2008;18(4):598–606.

10.

Jordan

, Javernick-Will

. Indicators of Community Recovery: Content Analysis and Delphi Approach. Natural Hazards Review. 2013;14:21–28.

11.

Lam

NSN

, Reams

, Li

et al. Measuring Community Resilience to Coastal Hazards along the Northern Gulf of Mexico. Natural Hazards Review. 2015;17:1–12.

12.

Grecco

CHS

, Vidal

MCR

, Santos

IJAL

, Carvalho

PVR

. A method to assess safety and resilience in radiopharmaceuticals production process. Work. 2012;41:5839–43.

13.

Huber

, Gomes

, De Carvalho

PVR

. A program to support the construction and evaluation of resilience indicators. Work. 2012;41:2810–6.

14.

Lee

, Vargo

, Seville

. Developing a Tool to Measure and Compare Organizations’ Resilience. Nat. Hazards Rev. 2013;14(1):29–41.

15.

Chan

, Wey

, Chang

. Establishing Disaster Resilience Indicators for Tan-sui River Basin in Taiwan. Social Indicators Research. 2014;115:387–418.

16.

De Carvalho

PVR

, Grecco

CHS

, Souza

. A fuzzy model to assess disaster risk reduction maturity level based on the Hyogo Framework for Action. Natural Hazards. 2016;83:309–326.

17.

Witmer

, Mellinger

. Organizational resilience: Nonprofit organizations’ response to change. Work. 2016;54:255–265.

18.

UNISDR. Terminology on Disaster Risk Reduction. International Strategy for Disaster Reduction. 2009;1-35.

19.

Cutter

, Burton

, Emrich

. Disaster Resilience Indicators for Benchmarking Baseline Conditions. J.Homel. Secur. Emerg.Manag. [Internet]. 2010;7(1). Available from: https://www.degruyter.com/view/j/jhsem.2010.7.1/jhsem.2010.7.1.1732/jhsem.2010.7.1.1732.xml.

20.

Hale

, Heijer

. Defining Resilience. In: Hollnagel E, Woods DD, Leveson N, editors. Resilience engineering, concepts and precepts. Aldershot: Ashgate; 2006. p. 35-40.

21.

Hollnagel

, Paries

, Woods

, Wreathall

. Resilience engineering in practice: A guidebook. London, Farnham: Ashgate; 2011.

22.

Omidvar

, Mazloumi

, Mohammad Fam

, Nirumand

. Development of a framework for resilience measurement: Suggestion of fuzzy Resilience Grade (RG) and fuzzy Resilience Early Warning Grade (REWG). Work. 2017;56:463–474.

23.

Rose

, Krausmann

. An economic framework for the development of a resilience index for business recovery. International Journal of Disaster Risk Reduction. 2013;5:73–83.

24.

De Carvalho

RJM

. ERGOPOLIS: An ergonomics approach applied to a city. Work. 2012;41:6071–8.

25.

Dolif

, Engelbrecht

, Jatobá

, Silva

AJD

, Gomes

, Borges

MRS

, Nobre

, Carvalho

PVR

. Resilience and brittleness in the ALERTA RIO system: a field study about the decision-making of forecasters. Natural Hazards. 2013;65:1831–1847.

26.

Da Silva

JCS

, De Carvalho

RJM

, Pimenta

AF da S

, De Carvalho

PVR

. The Meetings of Disaster Victims as a Space for Developing Community Resilience. Procedia Manufacturing. 2015;3:1825–1831.

27.

Bruneau

, Eeri

, Chang

, Ronald

, Lee

, Rourke

TDO

, et al. A framework to quantitatively assess and enhance the seismic resilience of communities. Earthq. Spectra. 2003;19(4):733–52.

28.

Engle

, de Bremond

, Malone

, Moss

. Towards a resilience indicator framework for making climate-change adaptation decisions. Mitigation and Adaptation Strategies for Global Change 2013;19:1295–1312.

29.

Cutter

, Ash

, Emrich

. The geographies of community disaster resilience. Glob. Environ. Change [Internet]. 2014;29:65–77. Available from: http://dx.doi.org/10.1016/j.gloenvcha.2014.08.005

30.

Dasgupta

, Shaw

. An indicatorbased approach to assess coastal communities’ resilience against climate related disasters in Indian Sundarbans. Journal Coast. Conserv. 2015;19(1):85–101.

31.

Yoon

, Kang

, Brody

. A measurement of community disaster resilience in Korea. Journal of Environmental Planning and Management. 2016;59(3):436–460.

32.

Krippendorff

. Content Analysis: An introduction to its methodology. Newbury Park, California: Sage Publications;1980.

33.

McManus

, Seville

, Vargo

, Brunsdon

. Facilitated Process for Improving Organizational Resilience. Natural Hazards Review. 2008;9:81–90.

34.

Joerin

, Shaw

. Chapter 3 Mapping climate and disaster resilience in cities. In: Shaw R, Sharma A, editors. Climate and disaster resilience in cities (Community, Environment and Disaster Risk Management, Vol. 6). UK: Emerald Publications; 2011. p. 47–61.

35.

Horney

, Dwyer

, Aminto

, Berke

, Smith

. Developing indicators to measure post-disaster community recovery in the United States. Disasters. 2017;41(1):124–49.

36.

Carreño

, Cardona

, Barbat

. A disaster risk management performance index. Nat. Hazards. 2007;41(1):1–20.

37.

Wilson

. Fundamentals of ergonomics in theory and practice, Applied Ergonomics. 2000;31:557–67.

38.

Kleiner

. Macroergonomics: Analysis and design of work systems, Applied Ergonomics. 2006;37:81–89.

39.

Wynd

, Schmidt

, Schaefer

. Two quantitative approaches for estimating content validity. Western Journal of Nursing Research. 2003;25:508–518.

40.

DeVon

, Block

, Moyle-Wright

, Ernst

, Hayden

, Lazzara

, Savoy

, Kostas-Polston

. A psychometric toolbox for testing validity and reliability [Electronic Version]. Journal of Nursing Scholarship. 2007;39:155–164.

41.

Polit

, Beck

. The Content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Research in Nursing & Health. 2006;31:489–497.

42.

Brasil

. 12.608. 2012. Presidência da república (in Portuguese). Available from: http://www.planalto.gov.br/ccivil_03/Ato2011-2014/2012/Lei/L12608htm

Phase	Indicator	Author
I –Prevention and Mitigation	1.1 - Survey and update of information on vulnerability, hazards or threats and risks of the city	Carreño et al.[36]; Horney et al. [35]
	1.2 - Population covered	Cutter et al. [19]
	1.3 - Prior withdrawal of people from risk areas	Carreño et al. [36]
	1.4 - Public information	Horney et al. [35]
	1.5 - Alarm and alert systems	Carreño et al. [36]; Chan et al. [15]; Engle et al. [28]
	1.6 - Database	Carreño et al. [36]
	1.7 - Investment allocated to DRM	Dasgupta and Shaw [30]
	1.8 - Reserve funds for institutional strengthening	Carreño et al. [36]
	1.9 - Public meetings	Horney et al. [35]
	1.10 - Disaster response experience	Cutter et al. [9]
	1.11 - Use of technology	Chan et al. [15]
	1.12 – Interinstitutional relationship, multiagency	Carreño et al. [36]; Dasgupta and Shaw [30]
	1.13 - Stimulating and supporting innovation and creativity initiatives	Engle et al. [28]
	1.14 - Decision making delegated to agents	Engle et al. [28]
	1.15 - Strong leadership and good management	Engle et al. [28]
	1.16 - Continuous assessment of strategies and work plans	Engle et al. [28]
	1.17 - Standardization of processes	Cutter et al. [9]
	1.18 - Job stability	Engle et al. [28]
	1.19 – Proactivity	Engle et al. [28]
	1.20 - Projects and works	SINPDEC [42]
	1.21- Relationship with agencies linked to SINPDEC at the municipal level	SINPDEC [42]
	1.22 - Provision of information to CEMADEM	SINPDEC [42]
	1.23 - Estimated time to start response actions	Bruneau et al. [27]
	1.24 - Risk communication	Dasgupta and Shaw [30]
	1.25 - Articulation with vulnerable communities	Cutter et al. [29]; Carreño et al. [36]
	1.26 - Contingency plan	Horney et al. [35]; SINPDEC [42]
	1.27 - Provision for emergency resources	Dasgupta and Shaw [30]
	1.28 - Guidelines for recovery or recovery plans	Horney et al. [35]; Carreño et al. [36]
II –Preparation	2.1 - Numbers of protection and civil defense agents	Cutter et al. [9]
	2.2 - Empowering vulnerable communities	Cutter et al. [9]; Cutter et al. [29]; Carreño et al. [36]
	2.3 - Training of school, hospital and health unit communities	Cutter et al. [9]; Carreño et al. [36]
	2.4 - Training of agents	Cutter et al. [9]; Engle et al. [28]
	2.5 - Simulated training	Cutter et al. [9]; Engle et al. [28]; Dasgupta and Shaw [30]
	2.6 -Team arrival time at the disaster site	Bruneau et al. [27]
	2.7 - Planning and organization of simulated trainings	Engle et al. [28]
	3.1 - Human and material resources to maintain actions	Bruneau et al. [27]
	3.2 - Mobilization of other agents as needed	Chan et al. [15]
III - Response	3.3 - Systems or technologies	Chan et al. [15]; Carreño et al. [36];
	4.1 - Emergency building standards	Cutter et al. [9]
	4.2 - Emergency funds	Dasgupta and Shaw [30]
IV –Recovery	4.3 - Continuity of operations/recovery plans	Cutter et al. [9]
	4.4 - Compensation for disaster victims	Cutter et al. [9]; Engle et al. [28]; Cutter et al. [29]
	4.5 - Recovery priorities	Engle et al. [35]
	4.6 - Number of public meetings	Horney et al. [35]
	4.7 - Updating recovery plans	Horney et al. [35]
	4.8 - Monitoring the progress of actions	Horney et al. [35]