Comparative study of resilience engineering in solid waste treatment plants: A case study in two European cities

1 Introduction

The growing complexity of social and technical systems has highlighted the importance of Resilience Engineering (RE). Resilience Engineering is a paradigm based on the ideas that rather than focusing on failures it is preferable to learn from normal successful functioning and that rather than restricting variability we should reinforce it when it has positive outcomes. In essence it tries to help workers to cope successfully with complexity when they are under pressure. Resilience Engineering has been mostly studied in the context of complex systems of high risk, such as in the aviation, process and petro-chemical industries, and nuclear power industries [1] but its concepts also tend to be beneficial for other industries that have not been studied. Municipal Solid Waste (MSW) treatment is an important sector all over the world, and even its accident levels are not so high, in comparison with other industrial sectors it is defined as a risky sector, taking into account the severity of some accidents [2]. Therefore, the aim of the study is to propose a method for the quantitative assessment of RE in the MSW treatment sector. To that end, the quantitative evaluation method developed by Shirali et al. [3] for an Iranian chemical producer it was validated, via its application to the MSW treatment sector in two European cities.

1.2 Methods for assessing RE

Few methods have been developed to assess Resilience Engineering (RE) so further research is called for [3]. Indeed, the development of such methods is a priority area for research in RE [8]. In this regard, Grecco et al. [9] published a qualitative model for assessing RE. Costella et al. [10] have also published a qualitative model called “Method for assessing health and safety management systems “ (onwards MAHS), based on the specifications for OHSAS 18001 and ILO-OSH 2001. Saurin et al. [11] proposed a method for evaluating health and safety management systems based on RE. Woods et al. [12] defined a method for selecting indicators which would point to coincidences and overlaps in RE indicators. Huber et al. [13] analysed the usefulness of RE indicators, based on an audit of the health and safety management system in a chemical plant. Shirali et al. [3] devised a quantitative method based on Principal Component Analysis, using the 6 principles proposed by Wreathall [14], extending the number of items in the qualitative model designed by Grecco et al. [9] to 61 and applying it to processing industry.

Based on the principles of CSE, Wreathall [14] defined 6 principles for RE, which Grecco et al. [9] subsequently developed into 43 qualitative actions. Shirali et al. [3] extended these individual measures to 61 and developed a model for quantitative assessment based on Principal Component Analysis (PCA).

OPEN IN VIEWER

Fig. 1

Methodology flowchart.

The 6 principles had the objectives listed below [3]:

Top management commitment.

This section endeavours to manifest how much top management devotes to RE and safety.

Just culture.

The aim of this section was identification of the potential obstacles to achieving a Just culture. Just culture is defined as an atmosphere of trust in which employees are encouraged to report essential safety-related issues.

Learning culture.

The objective of these actions is to understand how much the plant tries to learn from incidents, near misses and mishaps.

Awareness and opacity.

Awareness and opacity are critical for assessment of sacrifice judgments and also anticipation of future changes in the environment because those may affect the system’s ability to function. The principle is based on the fact that employees should be aware both of their current state and the current status of the defences in the plant. They should also be aware of systems’ boundaries and know how close it is to their edge.

Flexibility.

This section sought how much the plant actively anticipates problems and prepares for them.

Preparedness.

The aim of this section was to understand that the plant can restructure itself in response to changes or pressures, and also that its work system design is error-tolerant against human error, and that the employees are able to make critical decisions on their own without having to wait for their boss.

In this respect, qualitative methods are of limited use because they are more complex to validate and to compare the results. With quantitative methods one can compare performance in different jobs and rank them, planning action which focuses on particular principles, posts and areas of activity.

The aim of the study was to propose a method for the quantitative assessment of Resilience Engineering (RE) in the municipal solid waste (MSW) treatment sector, working empirically with two companies in the sector, in Spain and Italy. This research was developed as part of a SAFERA project. The project got the objective to facilitate the simulation of management decisions of health and safety based on their impact on the RE in MSW Treatment and Collection Companies. By selecting cities in two different countries it was hoped that any bias could be reduced arising from a local focus. Methodological process was described in Fig. 1.

First, a deep literature review was developed using the main academic sources as Web of Science (WOS), Science Direct, Scopus, Google Scholar, Wiley Online and others. In this step, methodology proposed by Shirali et al. [3] was found as an appropriate methodology aligned with the research’s objectives. Secondly, the questionnaire proposed in Shirali’s methodology was checked and adapted. After that, questionnaires were distributed in Italy and Spain, collected, and tested their adequacy. Finally, results were analysed.

The starting point was the 6 principle model devised by Wreathal et al. [14] and the 61 measures for the RE principles defined by Shirali et al. [3] in their quantitative method.

First, the questions were reformulated in the questionnaire, translating them into Spanish and Italian and adapting them to the sector, because some of them were not easy to understand after the initial translation, and some modifications for a better understanding of the real sense of some items were necessary.

The questionnaire was revised on 4 successive occasions with 4 experts on occupational health and safety and on MSW treatment, familiar with the principles and objectives of RE. During this process, if minor doubts arose regarding the exact meaning of the items, we contacted one of the authors, Shirali, to ensure that the sense of the questionnaire was not distorted. Some doubts in the positive or negative interpretation of some items was contrasted and checked with the support of the original author.

The questionnaire used a Likert scale ranging from 1 (strongly disagree) to 5 (in complete agreement). This can be seen in Shirali et al. [3].

In Spain the questionnaire was given to the staff working at the Municipal Solid Waste Treatment Plant (TP) in Málaga, which deals with municipal solid waste (MSW) from a population of 568,030 [15]. The questionnaire was administered in the period May to December 2014 and the responses were anonymous. A total amount of 121 questionnaires were distributed, of which 84 were correctly completed. In Italy the questionnaire was given to TP staff in Naples, which has a population of 978,399 [16]. The questionnaire was administered in the period November 2014 to March 2015 and the responses were all anonymous. 276 questionnaires were distributed, of which 244 were correctly completed.

The positions studied were as follows: drivers, machine operators (first class), bascule operators, machine operators (second class), discharge control operators, maintenance and cleaning staff, maintenance of plant staff, leachate (liquid drained from solid waste) and biogas plant operators, packaging plant operators, debris plant operators, and supervisors.

Principal component analysis (PCA) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. There will be a new reduced set of variables such the each new variable in this set is named a principal component. To ensure correct (PCA) it was checked that the number of questionnaires was sufficient to apply Bartlett’s sphericity test and Kaiser-Meyer-Olkin (KMO) sampling adequacy analysis. In essence, it is looking for a relationship between the variable it is hope to analyse, i.e. the correlations do not indicate an identity matrix [17].

In the Málaga TP the machine operators was excluded, second class (KMO = 0.34, significance, 0.287, Chi-square = 17.546) and the supervisors (KMO = 0.383, significance, 0.098, Chi-square = 22.369), where KMO values were below 0.5. In the case of these positions it can be seen that, after applying Bartlett’s sphericity test, significance is less than 0.1, making it too high for correct statistical analysis (see Table 1).

Mean, standard deviation and Cronbach alpha values were calculated for each principle and activity in the Málaga and Naples Treatment Plants. The internal consistency of the responses was evaluated with the Cronbach alpha test for each scale (Table 3).

In the Málaga Treatment Plant (TP) it can be seen that the reliability of four of the six Resilience Engineering (RE) principles is acceptable as the Cronbach coefficient is greater than 0.7 [18]. The Cronbach alpha result is lower for the principles of Flexibility (0.587) and Awareness and opacity (0.671). In Naples results below 0.7 were recorded for Learning (0.621), Awareness and opacity (0.523) and Preparedness (0.660). As can be seen, the values below 0.7 do not fall far short of this minimum and in some cases come quite close to it. Moreover, the fact that the results are less than 0.7 does not necessarily mean that one must reject the scale, as the Cronbach coefficient is not a statistic including a p-value on the basis of which we can reject the hypothesis of the reliability of the scale. It was therefore decided that all the principles initially contemplated should be retained.

When the mean values for Málaga are analyzed (Appendix B), the principle with the lowest score was “Preparedness” (0.1239), with all the scores falling between this level and the score for “Awareness and opacity” (0.2472).

Main improvements were classified according to areas with low scores:

“Preparedness” should therefore be considered a priority, which will involve improving mechanisms for providing information in unforeseen situations and what to do in these situations, sharing knowledge of potential hazards, incidents and the way to deal with them, the dissemination of guidelines for decision making, holding group meetings for this purpose, etc.

Better results were recorded for the principles of “Flexibility”, “Learning culture” and “Awareness and opacity”, which achieved similar scores, with figures much like those obtained by Shirali et al. [19].

Results in Italy were not fully comparable with those for Spain. We have posts for which the questionnaires gave unusable or insufficient data. An analysis of the mean values for the principles (the last line in Appendix B) shows that, of the 3 principles with the lowest scores in Spain, 2 also appear among the 3 lowest scoring principles in Italy: those of “Just culture” and “Top-level commitment”. In Italy the lowest score was for “Just culture” (0.0496), suggesting very low levels of participation and motivation and serious reservations about the communication of incidents and malfunctions.

If we turn to the results by post, in the case of the Spanish Treatment Plant (TP), the first point we would emphasise is that only “Packaging staff” display negative Z values for nearly all the principles, except “Flexibility”, situating this post in last position, with -41.369. It is clear that this post requires the greatest attention with regard to increasing the potential for Resilience Engineering (RE) in the Spanish plant (Saurin & Junior, 2011). For “Composting staff”, the group ranked next, much higher values are recorded (36.059), although there are still negative scores for the principles of “Preparedness” ((–0.068) and “Flexibility” (–0.035).

The top four posts in the Spanish ranking, with scores between 65.121 and 71.812 points, are “Maintenance and cleaning”, “Driver”, “Leachate and biogas plant operative” and “Machine operator first class”, in this order. A result of particular interest is that for “Flexibility” in the case of the highest ranking post (“Maintenance and cleaning”); at 0.165 points it was clearly below the average of 0.2369. Although “Maintenance and cleaning” staff record the highest scores for RE, “Maintenance of plant” staff are in fifth place out of the seven groups listed. The explanation for this difference may be that, despite the similarity in their job titles, the groups are assigned very different tasks. “Maintenance and cleaning” staff are responsible for vehicle maintenance and cleaning, changing oil in motor vehicles, etc. while “Maintenance of plant” staff carry out electrical and mechanical maintenance work on the premises. Given the importance of all posts related to maintenance with regard to occupational health and safety and to the safety of the system itself, and the extent to which the work of these posts can affect many others, it would seem very important to improve their results for RE.

Overall the figures for Spain reflect a difference between two types of post: more specialised positions, such as packaging and composting operatives, and those which are less specialised but which account for a greater volume of work, such as maintenance, machine operators and drivers.

In Italy the worst results were for “Maintenance of plant” (–11.620) and “Leachate and biogas plant operative” (15.674). Of those cases that can be compared, information being available for both countries, “Maintenance of plant” is also among the three lowest scoring groups in Spain. The results obtained for “Maintenance of plant” are especially negative, four of the six principles record negative scores while all of the principles show below average results. The position of “Driver” also obtained a significantly positive result in Italy (second with 52.443 points), very similar to that in Spain.

In Italy the post of “Debris plant operator” was in first place, with 62.485 points, but this result cannot be compared with Spain, where there are no figures for this position.

The figures for other jobs are more difficult to interpret because were not found any reasonable explanations for differences in the results between countries. For example, “Leachate and biogas plant operator” (3rd out of 6 in Spain and 5th out of 6 in Italy) and “Machine operator, first class” (4th in Spain and 2nd in Italy).

5 Conclusions

The aim of this study is to propose a method for the quantitative assessment of Resilience Engineering (RE) in the municipal solid was (MSW) treatment sector. To this end, we empirically validate the method used by Shirali et al. to assess RE, in a production process which is different from that for which it was designed and in a different cultural context [3]. The main contribution of the study is the validation of the model proposed and the identification of the criteria and posts where improvements could be made in municipal solid waste (MSW) treatment plants, a sector not hitherto studied. A first assessment of the sector has thus been made, using the cases of two European cities, Málaga and Naples. The results are quantified and ranked, allowing one to identify priorities for improving RE.

Regarding the RE principles, the results show that in both plants priority must be given to improving “Top-level commitment” and “Just culture”, to which we must add “Preparedness” in Málaga and “Learning culture” in Naples. These findings should be borne in mind by sector practitioners and management with a view to implementing appropriate corrective measures.

The RE scores recorded for individual posts differ significantly from each other and between the two plants. “Maintenance of plant” staff recorded one of the worst PCA figures in the Treatment Plant (TP), a particularly significant figure because of the impact on the occupational health and safety of operatives and the impact on safety in the plant as a whole. The responsibilities and working conditions of this post should be studied in depth with a view to developing its RE potential. Although the poor results for positions such as packaging plant operator and composting plant operator in Málaga or leachate and biogas plant operator in Naples may be affected by their limited volume of work, the low scores for these groups are not easy to explain.

Conflict of interest

None to report.