Comparative study of resilience engineering in solid waste treatment plants: A case study in two European cities

Abstract

BACKGROUND:

Municipal solid waste treatment plants are industrial facilities with important occupational health and safety issues. Hence, a risk assessment system would be very useful to help workers to cope successfully with complexity when they are under pressure situations, such as loss of control or failures of the system safety. In recent years, Resilience Engineering has come up as a new proactive approach to improve and keep safety the complex systems. To evaluate Resilience Engineering the methods at our disposal are mostly qualitative, which are complex and difficult to compare due to their external validity.

OBJECTIVE:

The present research proposes a method for the quantitative assessment of Resilience Engineering in the municipal solid waste treatment sector.

METHODS:

The study was carried out as part of an EU SAFERA project in two European cities. The data were obtained from a survey of a sample of 328 workers of treatment plants.

RESULTS:

The results indicated that priority must be given to improving top-level commitment, culture, preparedness and learning culture, while awareness and opacity emerges as positive results. Significant differences can be seen in the Resilience Engineering evaluation for different posts.

CONCLUSION:

These findings allow practitioners and management with a view to implementing appropriate corrective measures to achieve high Resilience Engineering in the plant.

Keywords

Resilience engineering quantitative occupational health and safety prevention risk municipal solid waste

1 Introduction

The growing complexity of social and technical systems has highlighted the importance of Resilience Engineering (RE). Resilience Engineering is a paradigm based on the ideas that rather than focusing on failures it is preferable to learn from normal successful functioning and that rather than restricting variability we should reinforce it when it has positive outcomes. In essence it tries to help workers to cope successfully with complexity when they are under pressure. Resilience Engineering has been mostly studied in the context of complex systems of high risk, such as in the aviation, process and petro-chemical industries, and nuclear power industries [1] but its concepts also tend to be beneficial for other industries that have not been studied. Municipal Solid Waste (MSW) treatment is an important sector all over the world, and even its accident levels are not so high, in comparison with other industrial sectors it is defined as a risky sector, taking into account the severity of some accidents [2]. Therefore, the aim of the study is to propose a method for the quantitative assessment of RE in the MSW treatment sector. To that end, the quantitative evaluation method developed by Shirali et al. [3] for an Iranian chemical producer it was validated, via its application to the MSW treatment sector in two European cities.

1.1 Resilience engineering

Resilience Engineering was defined as follow: “Resilience engineering is a paradigm for safety management that focuses on how to help people cope with complexity under pressure to achieve success [1].

The application of Resilience Engineering (RE) to occupational health and safety dates back to 2007, when its main concepts were published [1].

Some authors have focused on RE in sectors such as building, where the number of accidents is particularly high [4, 5]. Erik Hollnagel, David Woods and others based their work on Cognitive Systems Engineering (CSE). CSE is a specialty discipline of systems development that addresses the design of socio-technical systems. A sociotechnical system is one in which humans provide essential functionality related to deciding, planning, collaborating and managing. CSE, as a forerunner of RE, instead of seeing the man-machine interface as a system of mechanical principles, treats it as a system whose functions adapt by using knowledge of itself and the environment and by planning and modifying actions. The principles of CSE also focus on helping people to cope successfully with complexity when under pressure [4].

However, the concept of Resilience Engineering (RE) is not easy to define and has evolved over time [6]. The central idea of RE could be synthesised as the need to learn from normal functioning, facilitate variability, design limits that are tolerant of tangible and visible errors, and constantly monitor performance proactively with a view to detecting disruption sufficiently in advance [7].

1.2 Methods for assessing RE

Few methods have been developed to assess Resilience Engineering (RE) so further research is called for [3]. Indeed, the development of such methods is a priority area for research in RE [8]. In this regard, Grecco et al. [9] published a qualitative model for assessing RE. Costella et al. [10] have also published a qualitative model called “Method for assessing health and safety management systems “ (onwards MAHS), based on the specifications for OHSAS 18001 and ILO-OSH 2001. Saurin et al. [11] proposed a method for evaluating health and safety management systems based on RE. Woods et al. [12] defined a method for selecting indicators which would point to coincidences and overlaps in RE indicators. Huber et al. [13] analysed the usefulness of RE indicators, based on an audit of the health and safety management system in a chemical plant. Shirali et al. [3] devised a quantitative method based on Principal Component Analysis, using the 6 principles proposed by Wreathall [14], extending the number of items in the qualitative model designed by Grecco et al. [9] to 61 and applying it to processing industry.

Based on the principles of CSE, Wreathall [14] defined 6 principles for RE, which Grecco et al. [9] subsequently developed into 43 qualitative actions. Shirali et al. [3] extended these individual measures to 61 and developed a model for quantitative assessment based on Principal Component Analysis (PCA).

Fig. 1

Methodology flowchart.

The 6 principles had the objectives listed below [3]:

Top management commitment.

This section endeavours to manifest how much top management devotes to RE and safety.

Just culture.

The aim of this section was identification of the potential obstacles to achieving a Just culture. Just culture is defined as an atmosphere of trust in which employees are encouraged to report essential safety-related issues.

Learning culture.

The objective of these actions is to understand how much the plant tries to learn from incidents, near misses and mishaps.

Awareness and opacity.

Awareness and opacity are critical for assessment of sacrifice judgments and also anticipation of future changes in the environment because those may affect the system’s ability to function. The principle is based on the fact that employees should be aware both of their current state and the current status of the defences in the plant. They should also be aware of systems’ boundaries and know how close it is to their edge.

Flexibility.

This section sought how much the plant actively anticipates problems and prepares for them.

Preparedness.

The aim of this section was to understand that the plant can restructure itself in response to changes or pressures, and also that its work system design is error-tolerant against human error, and that the employees are able to make critical decisions on their own without having to wait for their boss.

In this respect, qualitative methods are of limited use because they are more complex to validate and to compare the results. With quantitative methods one can compare performance in different jobs and rank them, planning action which focuses on particular principles, posts and areas of activity.

2 Methodology and data

The aim of the study was to propose a method for the quantitative assessment of Resilience Engineering (RE) in the municipal solid waste (MSW) treatment sector, working empirically with two companies in the sector, in Spain and Italy. This research was developed as part of a SAFERA project. The project got the objective to facilitate the simulation of management decisions of health and safety based on their impact on the RE in MSW Treatment and Collection Companies. By selecting cities in two different countries it was hoped that any bias could be reduced arising from a local focus. Methodological process was described in Fig. 1.

First, a deep literature review was developed using the main academic sources as Web of Science (WOS), Science Direct, Scopus, Google Scholar, Wiley Online and others. In this step, methodology proposed by Shirali et al. [3] was found as an appropriate methodology aligned with the research’s objectives. Secondly, the questionnaire proposed in Shirali’s methodology was checked and adapted. After that, questionnaires were distributed in Italy and Spain, collected, and tested their adequacy. Finally, results were analysed.

The starting point was the 6 principle model devised by Wreathal et al. [14] and the 61 measures for the RE principles defined by Shirali et al. [3] in their quantitative method.

First, the questions were reformulated in the questionnaire, translating them into Spanish and Italian and adapting them to the sector, because some of them were not easy to understand after the initial translation, and some modifications for a better understanding of the real sense of some items were necessary.

The questionnaire was revised on 4 successive occasions with 4 experts on occupational health and safety and on MSW treatment, familiar with the principles and objectives of RE. During this process, if minor doubts arose regarding the exact meaning of the items, we contacted one of the authors, Shirali, to ensure that the sense of the questionnaire was not distorted. Some doubts in the positive or negative interpretation of some items was contrasted and checked with the support of the original author.

The questionnaire used a Likert scale ranging from 1 (strongly disagree) to 5 (in complete agreement). This can be seen in Shirali et al. [3].

Table 1
Results of validity tests for PCA of TP positions in Málaga and Naples

Position KMO Chi-square Significance

Málaga Naples Málaga Naples Málaga Naples

Driver 0.830 0.817 44.922 124.515 0 0.000

Machine operator, first class 0.692 0.691 31.045 39.128 0.09 0.001

Bascule operator − 0.691 − 15.653 − 0.041

Machine operator, second class 0.34 − 17.546 − 0.287 −

Maintenance and cleaning staff 0.721 0.519 33.541 13.286 0.004 0.058

Composting operator 0.646 − 75.93 − 0 −

Maintenance of plant staff 0.511 0.617 32.149 14.349 0.006 0.049

Leachate and biogas plant operator 0.604 0.877 23.95 82.363 0.066 0.000

Packaging plant staff 0.59 − 47.275 − 0 −

Debris plant staff − 0.578 − 13.757 − 0.054

Supervisors 0.383 − 22.369 − 0.098 −

Position	KMO	Chi-square	Significance
Driver	0.830	0.817	44.922	124.515	0	0.000
Machine operator, first class	0.692	0.691	31.045	39.128	0.09	0.001
Bascule operator	−	0.691	−	15.653	−	0.041
Machine operator, second class	0.34	−	17.546	−	0.287	−
Maintenance and cleaning staff	0.721	0.519	33.541	13.286	0.004	0.058
Composting operator	0.646	−	75.93	−	0	−
Maintenance of plant staff	0.511	0.617	32.149	14.349	0.006	0.049
Leachate and biogas plant operator	0.604	0.877	23.95	82.363	0.066	0.000
Packaging plant staff	0.59	−	47.275	−	0	−
Debris plant staff	−	0.578	−	13.757	−	0.054
Supervisors	0.383	−	22.369	−	0.098	−

In Spain the questionnaire was given to the staff working at the Municipal Solid Waste Treatment Plant (TP) in Málaga, which deals with municipal solid waste (MSW) from a population of 568,030 [15]. The questionnaire was administered in the period May to December 2014 and the responses were anonymous. A total amount of 121 questionnaires were distributed, of which 84 were correctly completed. In Italy the questionnaire was given to TP staff in Naples, which has a population of 978,399 [16]. The questionnaire was administered in the period November 2014 to March 2015 and the responses were all anonymous. 276 questionnaires were distributed, of which 244 were correctly completed.

The positions studied were as follows: drivers, machine operators (first class), bascule operators, machine operators (second class), discharge control operators, maintenance and cleaning staff, maintenance of plant staff, leachate (liquid drained from solid waste) and biogas plant operators, packaging plant operators, debris plant operators, and supervisors.

Principal component analysis (PCA) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. There will be a new reduced set of variables such the each new variable in this set is named a principal component. To ensure correct (PCA) it was checked that the number of questionnaires was sufficient to apply Bartlett’s sphericity test and Kaiser-Meyer-Olkin (KMO) sampling adequacy analysis. In essence, it is looking for a relationship between the variable it is hope to analyse, i.e. the correlations do not indicate an identity matrix [17].

In the Málaga TP the machine operators was excluded, second class (KMO = 0.34, significance, 0.287, Chi-square = 17.546) and the supervisors (KMO = 0.383, significance, 0.098, Chi-square = 22.369), where KMO values were below 0.5. In the case of these positions it can be seen that, after applying Bartlett’s sphericity test, significance is less than 0.1, making it too high for correct statistical analysis (see Table 1).

2.1 Questionnaire limitations

As they were only a limited number of questionnaires for some positions in Málaga it was tried to regroup them with others which were similar. This was done after consultation with company staff and managers. Consequently the debris plant operative was considered together with the compost plant operative, as in practice they both worked in either of the two areas. The 3 questionnaires from the bascule operators had to be discarded because they were not correctly completed, and so did the only questionnaire from a discharge control operative. The number of questionnaires finally used for each job can be seen in Table 2. The KMO analysis for the Naples TP is shown in Table 1 and the number of questionnaires in Table 2. It was not necessary to exclude any job.

Table 2
Valid questionnaires per position in Málaga and Naples TPs

Position Number of valid questionnaires

Málaga Naples

Driver 11 59

Machine operator, first class 10 21

Bascule operator − 9

Maintenance and cleaning staff 8 10

Composting plant operative 23 −

Maintenance of plant staff 10 54

Leachate and biogas plant operator 7 22

Packaging plant staff 15 −

Debris plant staff − 69

Total 84 244

Position	Number of valid questionnaires
Driver	11	59
Machine operator, first class	10	21
Bascule operator	−	9
Maintenance and cleaning staff	8	10
Composting plant operative	23	−
Maintenance of plant staff	10	54
Leachate and biogas plant operator	7	22
Packaging plant staff	15	−
Debris plant staff	−	69
Total	84	244

3 Results

Mean, standard deviation and Cronbach alpha values were calculated for each principle and activity in the Málaga and Naples Treatment Plants. The internal consistency of the responses was evaluated with the Cronbach alpha test for each scale (Table 3).

Table 3
Cronbach coefficient, mean and standard deviation for Málaga and Naples TPs

Resilience principles α Mean Standard deviation

Málaga Naples Málaga Naples Málaga Naples

Top-level commitment 0.857 0.818 3.635 3.039 1.251 1.193

Culture 0.844 0.705 3.225 2.86 1.222 1.216

Learning 0.851 0.621 3.336 3.01 1.149 1.197

Awareness and opacity 0.671 0.523 3.481 3.20 1.119 1.186

Preparedness 0.710 0.660 3.642 3.04 1.086 1.202

Flexibility 0.587 0.837 3.155 3.15 1.187 1.241

Resilience principles	α	Mean	Standard deviation
Top-level commitment	0.857	0.818	3.635	3.039	1.251	1.193
Culture	0.844	0.705	3.225	2.86	1.222	1.216
Learning	0.851	0.621	3.336	3.01	1.149	1.197
Awareness and opacity	0.671	0.523	3.481	3.20	1.119	1.186
Preparedness	0.710	0.660	3.642	3.04	1.086	1.202
Flexibility	0.587	0.837	3.155	3.15	1.187	1.241

In the Málaga Treatment Plant (TP) it can be seen that the reliability of four of the six Resilience Engineering (RE) principles is acceptable as the Cronbach coefficient is greater than 0.7 [18]. The Cronbach alpha result is lower for the principles of Flexibility (0.587) and Awareness and opacity (0.671). In Naples results below 0.7 were recorded for Learning (0.621), Awareness and opacity (0.523) and Preparedness (0.660). As can be seen, the values below 0.7 do not fall far short of this minimum and in some cases come quite close to it. Moreover, the fact that the results are less than 0.7 does not necessarily mean that one must reject the scale, as the Cronbach coefficient is not a statistic including a p-value on the basis of which we can reject the hypothesis of the reliability of the scale. It was therefore decided that all the principles initially contemplated should be retained.

3.1 PCA results

Based on the average values obtained in valid questionnaires for each RE principle, by post and activity, a correlation matrix “R” was calculated. In view of the high number of tables that would be involved if information for all the posts were included, it will take as an example the post of composting operative in the Málaga Treatment Plant (TP). After calculating the correlation matrix “R”, we calculated the eigenvalues “λ”(or main components) and eigenvectors “v”, using the equations (1) and (2) shown below. (1) is used to calculate “λ” and subsequently (2) is used to calculate “v”. $| R - λ I | = 0$ (1) $(R - λ_{i} I) ν = 0$ (2)

Table 4 shows the eigenvalues for each eigenvector or principal component (PCi), and the level of explained variance and cumulative variance for the post of composting operative at the Málaga TP. It will be seen that the first 4 components account for 95.8% of variance. Eigenvectors for each Resilience Engineering (RE) principle and the post in question are also shown.

Table 4

Eigenvalues for each component, % of explained variance accounted for by the component, cumulative % of variance accounted for by components, and eigenvectors or main components (PCi) for each principle for the composting operatives at Málaga TP

Variable	PC1	PC2	PC3	PC4	PC5	PC6
Eigenvalues (λ)	3.271	1.069	0.908	0.504	0.183	0.065
Proportion (% variance)	54.517	17.822	15.128	8.399	3.051	1.083
Cumulative (% variance)	54.517	72.339	87.467	95.866	98.917	100
Top-level commitment	0.4598	0.2816	0.206	0.4459	0.6791	0.0821
Just culture	0.5165	0.1513	0.165	−0.005	−0.5353	0.6298
Learning culture	0.5285	−0.0714	0.002	0.2017	−0.3728	−0.7321
Awareness and opacity	0.4307	−0.1671	0.0376	−0.8257	0.3158	−0.061
Preparedness	0.0612	−0.9119	0.2692	0.2451	0.0744	0.1635
Flexibility	0.2309	−0.1825	−0.9254	0.1366	0.0894	0.1741

The Z values for the principle of top-level commitment are shown in Equation 3.

$\begin{matrix} Z_{Top - level commitment} = 0.54517 * 0.4598 \\ + 0.17822 * 0.2816 + 0.15128 * 0.206 \\ + 0.08399 * \end{matrix}$ (3)

After calculating Z values for each post and principle, we calculated the S vector for each post, with a component for each RE principle. Each component of the S vector by post is calculated for each principle. To do this we added the values obtained from each respondent for each of the questions related to each principle. Subsequently, it was calculated the mean value of the above sums for all respondents (see Appendix A).

Table 5 shows calculations of the value of the components of the S vector for the composting post.

Table 5

S vector for the post of composting operative at the Málaga TP

Top-level commitment	Just culture	Learning culture	Awareness and opacity	Preparedness	Flexibility
28.65	30.04	44.65	36.48	32.00	16.70

The value of the final PCA score is obtained with equation 4. $Funal PCA score = S \cdot Z$ (4)

Expression shows the final PCA value for the composting post. $\begin{matrix} Final PCA score (Composting) = 28.65 * 0.3695 nonumber \\ + 30.04 * 0.3331 + 44.65 * \\ 0.2926 + 36.48 * 0.1414 + 32 * (- 0.0678) nonumber \\ + 16.70 * (- 0.0352) = 36.059 \end{matrix}$ (5)

The results of the Z scores obtained for all the Treatment Plant (TP) posts and all the RE principles, and the PCA values and their ranking, after carrying out the PCA analysis, are shown in Appendix B (for the Málaga TP and the Naples TP).

4 Discussion

When the mean values for Málaga are analyzed (Appendix B), the principle with the lowest score was “Preparedness” (0.1239), with all the scores falling between this level and the score for “Awareness and opacity” (0.2472).

Main improvements were classified according to areas with low scores:

“Preparedness” should therefore be considered a priority, which will involve improving mechanisms for providing information in unforeseen situations and what to do in these situations, sharing knowledge of potential hazards, incidents and the way to deal with them, the dissemination of guidelines for decision making, holding group meetings for this purpose, etc.

“Top-level commitment”. The second lowest value was recorded for “Top-level commitment” (0.1914) and this commitment must therefore be strengthened, communicated and made visible, appropriate resources being made available, positive feedback being encouraged and every effort being made to ensure that when there is a conflict between safety and production, safety is not prejudiced [5].

“Just culture” (0.1942) recorded scores which were almost the same as those for “Top-level commitment”, thus revealing a weakness in the organisation’s ability to achieve worker participation and in its motivation to communicate regarding health and safety, possibly reflecting the degree of commitment this involves and lack of access by employees to their managers in connection with safety issues.

Better results were recorded for the principles of “Flexibility”, “Learning culture” and “Awareness and opacity”, which achieved similar scores, with figures much like those obtained by Shirali et al. [19].

Results in Italy were not fully comparable with those for Spain. We have posts for which the questionnaires gave unusable or insufficient data. An analysis of the mean values for the principles (the last line in Appendix B) shows that, of the 3 principles with the lowest scores in Spain, 2 also appear among the 3 lowest scoring principles in Italy: those of “Just culture” and “Top-level commitment”. In Italy the lowest score was for “Just culture” (0.0496), suggesting very low levels of participation and motivation and serious reservations about the communication of incidents and malfunctions.

If we turn to the results by post, in the case of the Spanish Treatment Plant (TP), the first point we would emphasise is that only “Packaging staff” display negative Z values for nearly all the principles, except “Flexibility”, situating this post in last position, with -41.369. It is clear that this post requires the greatest attention with regard to increasing the potential for Resilience Engineering (RE) in the Spanish plant (Saurin & Junior, 2011). For “Composting staff”, the group ranked next, much higher values are recorded (36.059), although there are still negative scores for the principles of “Preparedness” ((–0.068) and “Flexibility” (–0.035).

The top four posts in the Spanish ranking, with scores between 65.121 and 71.812 points, are “Maintenance and cleaning”, “Driver”, “Leachate and biogas plant operative” and “Machine operator first class”, in this order. A result of particular interest is that for “Flexibility” in the case of the highest ranking post (“Maintenance and cleaning”); at 0.165 points it was clearly below the average of 0.2369. Although “Maintenance and cleaning” staff record the highest scores for RE, “Maintenance of plant” staff are in fifth place out of the seven groups listed. The explanation for this difference may be that, despite the similarity in their job titles, the groups are assigned very different tasks. “Maintenance and cleaning” staff are responsible for vehicle maintenance and cleaning, changing oil in motor vehicles, etc. while “Maintenance of plant” staff carry out electrical and mechanical maintenance work on the premises. Given the importance of all posts related to maintenance with regard to occupational health and safety and to the safety of the system itself, and the extent to which the work of these posts can affect many others, it would seem very important to improve their results for RE.

Overall the figures for Spain reflect a difference between two types of post: more specialised positions, such as packaging and composting operatives, and those which are less specialised but which account for a greater volume of work, such as maintenance, machine operators and drivers.

In Italy the worst results were for “Maintenance of plant” (–11.620) and “Leachate and biogas plant operative” (15.674). Of those cases that can be compared, information being available for both countries, “Maintenance of plant” is also among the three lowest scoring groups in Spain. The results obtained for “Maintenance of plant” are especially negative, four of the six principles record negative scores while all of the principles show below average results. The position of “Driver” also obtained a significantly positive result in Italy (second with 52.443 points), very similar to that in Spain.

In Italy the post of “Debris plant operator” was in first place, with 62.485 points, but this result cannot be compared with Spain, where there are no figures for this position.

The figures for other jobs are more difficult to interpret because were not found any reasonable explanations for differences in the results between countries. For example, “Leachate and biogas plant operator” (3rd out of 6 in Spain and 5th out of 6 in Italy) and “Machine operator, first class” (4th in Spain and 2nd in Italy).

5 Conclusions

The aim of this study is to propose a method for the quantitative assessment of Resilience Engineering (RE) in the municipal solid was (MSW) treatment sector. To this end, we empirically validate the method used by Shirali et al. to assess RE, in a production process which is different from that for which it was designed and in a different cultural context [3]. The main contribution of the study is the validation of the model proposed and the identification of the criteria and posts where improvements could be made in municipal solid waste (MSW) treatment plants, a sector not hitherto studied. A first assessment of the sector has thus been made, using the cases of two European cities, Málaga and Naples. The results are quantified and ranked, allowing one to identify priorities for improving RE.

Regarding the RE principles, the results show that in both plants priority must be given to improving “Top-level commitment” and “Just culture”, to which we must add “Preparedness” in Málaga and “Learning culture” in Naples. These findings should be borne in mind by sector practitioners and management with a view to implementing appropriate corrective measures.

The RE scores recorded for individual posts differ significantly from each other and between the two plants. “Maintenance of plant” staff recorded one of the worst PCA figures in the Treatment Plant (TP), a particularly significant figure because of the impact on the occupational health and safety of operatives and the impact on safety in the plant as a whole. The responsibilities and working conditions of this post should be studied in depth with a view to developing its RE potential. Although the poor results for positions such as packaging plant operator and composting plant operator in Málaga or leachate and biogas plant operator in Naples may be affected by their limited volume of work, the low scores for these groups are not easy to explain.

5.1 Impact on the industry

Research findings allow practitioners and management with a view to implementing appropriate corrective measures to achieve high Resilience Engineering in the plant.

5.2 Limitations

The study has been limited by the impossibility of assessing certain posts, such as bascule operator, discharge control operator, packaging plant operator or debris plant operator in one city or the other because an insufficient number of questionnaires were available, making it impossible to compare results for these groups. This is an area it would be highly desirable to tackle in future research. Moreover, as the study was confined to the cases of companies in two countries, Italy and Spain, it would also be interesting to extend research to other countries with different cultures.

Conflict of interest

None to report.

Footnotes

Appendices

Acknowledgments

This research forms part of the CICE SAFERA Eranet 2013 project “Design and development of a simulation tool for decision making in the management of health and safety based on RE to promote a safety culture change process in MSW treatment companies (Asses-Re-Tool)”. This work was funded by the Government of Andalusia, awarded by the Department of Economy, Innovation and Science through the Agency of Innovation and Development of Andalusia. We would like to thank Professor Saeed Shirali at the Ahvez Jundishapur University, the Spanish and Italian workers who took part in the study, LIMASA, which facilitated our participation in the project, and Universita Degli Studi Federico II di Napoli. We would also like to express our gratitude for the field work carried out by Daniel Sánchez García.

References

Hollnagel

, Woods

, Leveson

. Resilience Engineering - Concepts and Precepts. Resilience Engineering: Concepts and Precepts. Surrey, England: Ashgate Publishing, Ltd, 2007.

Consejería de Empleo, Condiciones de trabajo y gestión preventiva en las empresas de residuos sólidos urbanos en Andalucía. Instituto Andaluz de Prevención de Riesgos Laborales, 2011. Available from: http://www.juntadeandalucia.es/export/drupaljda/1__informe_rsu.pdf .

Shirali

, Mohammadfam

, Ebrahimipour

. A new method for quantitative assessment of resilience engineering by PCA and NT approach: A case study in a process industry. Reliab Eng Syst Saf. 2013;119:88–94.

Saurin

, Formoso

, Cambraia

. An analysis of construction safety best practices from a cognitive systems engineering perspective. Saf Sci. 2008;46(8):1169–83.

Saurin

, Carim

Jr. . Evaluation and improvement of a method for assessing HSMS from the resilience engineering perspective: A case study of an electricity distributor. Saf Sci. 2011;49(2):355–68.

Woods

. Four concepts for resilience and the implications for the future of resilience engineering. Reliab Eng Syst Saf. 2015;141:5–9.

Le Coze

J-C

. New models for new times. An anti-dualist move. Saf Sci [Internet]. 2015;141:142–52. Available from: https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0951832015000654%5Cnhttps://dx-doi-org.web.bisu.edu.cn/10.1016/j.ress.2015.03.007

Righi

, Abreu

, Wachs

, Saurin

, Wachs

, A systematic literature review of resilience engineering: Research areas and a research agenda proposal. Reliab Eng Syst Saf [Internet]. 2015;141:142–52. Available from: https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0951832015000654%5Cnhttps://dx-doi-org.web.bisu.edu.cn/10.1016/j.ress.2015.03.007.

Grecco

CHS

, Vidal

MCR

, Santos

, Carvalho

PVR

. A method to assess safety and resilience in radiopharmaceuticals production process. Work. 2012;4(Suppl 1):5839–43 10.3233/WOR-2012-0969-5839.

10.

Costella

, Saurin

, de Macedo Guimarães

. A method for assessing health and safety management systems from the resilience engineering perspective. Saf Sci. 2009;47(8):1056–67.

11.

Saurin

, Formoso

, Fama

. Criteria for assessing safety performance measurement systems: Insights from resilience engineering In: Nemeth

, Hollnagel

, editors. Resilience Engineering in Practice, Volume 2: Becoming Resilient. 1st Edition London: CRC Press 2014, 63–78.

12.

Woods

, Herrera

, Woltjer

, Identifying Imbalances in a Portfolio of Safety Metrics: The Q4-Balance Framework for Economy-Safety Tradeoffs, In: 5th Resilience Engineering Symposium [Internet], 2013, pp. 149–54. Available from http://hdl.handle.net/1811/60454.

13.

Huber

, van Wijgerden

, de Witt

, Dekker

SWA

, Learning from organizational incidents: Resilience engineering for high-risk process environments. Process Saf Prog. 2009;28(1):90–5.

14.

Wreathall, J, Properties of Resilient Organizations: An Initial View, In: Resilience engineering: Concepts and precepts 2006, pp, 275–88.

15.

Instituto Nacional de Estadística, Estadísticas del Padrón Continuo, Available from: http://www.ine.es/jaxi/Tabla.htm?path=/t20/e245/p05/a2015/10/&file=00029006.px&L=0.

16.

Istituto Nazionale di Statistica, Statistiche demografiche, 2015. Available from: http://demo.istat.it/pop/index1_e.html.

17.

De la Garza García

, Serrano

BNM

, Cavazos

BAG

, Análisis estadístico multivariante, McGraw-Hill, 2013.

18.

Reiman

, Oedewald

. Measuring maintenance culture and maintenance core task with CULTURE-questionnaire - A case study in the power industry. Saf Sci. 2004;42(9):859–89.

19.

Shirali

GHA

, Motamedzade

, Mohammadfam

, Ebrahimipour

, Moghimbeigi

, Challenges in building resilience engineering (RE) and adaptive capacity: A field study in a chemical plant. Process Saf Environ Prot [Internet]. 2012;90(2):83–90. Available from: 10.1016/j.psep.2011.08.003.

Position	KMO	Chi-square				Significance
	Málaga	Naples	Málaga	Naples	Málaga	Naples
Driver	0.830	0.817	44.922	124.515	0	0.000
Machine operator, first class	0.692	0.691	31.045	39.128	0.09	0.001
Bascule operator	−	0.691	−	15.653	−	0.041
Machine operator, second class	0.34	−	17.546	−	0.287	−
Maintenance and cleaning staff	0.721	0.519	33.541	13.286	0.004	0.058
Composting operator	0.646	−	75.93	−	0	−
Maintenance of plant staff	0.511	0.617	32.149	14.349	0.006	0.049
Leachate and biogas plant operator	0.604	0.877	23.95	82.363	0.066	0.000
Packaging plant staff	0.59	−	47.275	−	0	−
Debris plant staff	−	0.578	−	13.757	−	0.054
Supervisors	0.383	−	22.369	−	0.098	−