Evaluation and empirical research on the operation effect of the risk precontrol management system of safety in coal mines based on OSWM-TOPSIS

Abstract

BACKGROUND:

Most studies have focused on the establishment and application of the risk precontrol management system for safety in coal mines and have seldom considered the evaluation of the system operation effect.

OBJECTIVE:

This study aimed to evaluate the operation effect of risk precontrol management system of safety in coal mines and propose policy suggestions to improve the risk precontrol management level of safety.

METHODS:

This study applied the Objective and Subjective Weighting Method (OSWM) combined with the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) to conduct evaluation and empirical research on the operation effect of the risk precontrol management system of safety in coal mines.

RESULTS:

First, the evaluation index system is mainly composed of six first-level indicators and 30 subordinate secondary indicators. Second, the OSWM combined with TOPSIS is an effective method for operation effect evaluation, which yields accurate and undistorted evaluation results. Third, the calculation reference value of the operation effect in the Gengcun coal mine is 57.34, and its corresponding effect level is level III, which is basically effective. Moreover, the calculation reference values of production equipment management (P4) and inspection, audit and review (P6) are the lowest, while the calculation reference values of risk precontrol management (P1) and auxiliary management (P5) reach the critical value corresponding to effect level I, which indicates a good operation effect.

CONCLUSIONS:

Corresponding policy suggestions to improve the risk precontrol management level in the Gengcun coal mine are proposed based on the above evaluation results.

Keywords

Coal mine enterprises risk management system evaluation policy suggestions

1 Introduction

The coal mining industry is a typical industry with three highs, namely, a high labour intensity, high employment risk, and high accident rate [1 –3]. Therefore, the problem of coal mine safety is a focus of Chinese political circles, and it remains an important topic worthy of study in academic circles [4]. In recent years, risk-based governance has rapidly developed in the main coal-producing countries worldwide, and it has gradually replaced the prescriptive mode of coal mine safety governance. Compared to traditional prescriptive coal mine safety governance, risk-based governance is more desirable and preferable, and it is the development trend of coal mine safety management in the future [5], which has also attracted much attention from the Chinese government and coal mine enterprises and has been widely investigated. On July 12, 2011, the former State Administration of Work Safety issued the risk precontrol management system of safety in coal mines specification (AQ/T 1093-2011), which was implemented on December 1, 2011, marking the beginning of the era of the risk precontrol management system of safety in Chinese coal mines [5, 6]. AQ/T 1093-2011 has been rapidly promoted across the coal industry, which also marks the transformation of the Chinese coal mine safety management mode from traditional experiential management and institutional management to higher-level risk precontrol management.

The implementation of the risk precontrol management system for safety in coal mines has reduced the number of coal mine accidents and casualties. However, the control of coal mine accidents has not achieved the expected effect due to the existing unscientific or unsystematic problems in the implementation of the system. Compared to the main coal-producing countries globally, the casualty and death rates per million tons in Chinese coal mines remain very high [8]. Therefore, it is of major theoretical and practical importance to conduct evaluation and empirical research on the operation effect of the risk precontrol management system of safety in coal mines, resolve the current problems in the operating process of the risk precontrol system, and effectively improve the safety management level of coal mine enterprises.

2 Literature review

The foreign coal mine safety management theory has experienced a transformation process in terms of machine-, human- and management-centred aspects. At present, both theoretical and empirical studies on safety management are relatively mature, mainly focusing on the development status and effectiveness evaluation of the safety management system [9]. On the basis of the development of the safety management system theory, various safety management systems and methods have successively emerged from the internationally well-known Occupational Health and Safety Assessment Series (OHSAS) management system to the Health, Safety and Environment (HSE) management system in the petrochemical industry, from the comprehensive management system in the United States to the NOSA safety management system in South Africa [10, 11].

With the development of the theoretical system of coal mine safety management, foreign scholars focused on the effect of risk precontrol management, mainly reflected in the impact on the accident rate [12], the degree of injuries in the workplace [13 –15], and the production efficiency of coal mining enterprises [16, 17], Robson evaluated the effectiveness of the intervention of the occupational health and safety management system (OHSMS) on enterprise safety management [12]. Asadzadeh comprehensively analysed and assessed HSE and health, safety, environment and ergonomics (HSEE) factors through a fuzzy cognitive map (FCM) [12]. Haas and Yorio constructed and evaluated OHSMS performance indicators [16]. Stolzer et al. evaluated the effectiveness of the safety management system based on the evolution law of big data [17].

In recent years, domestic scholars have also focused on research in the field of risk precontrol management of safety in coal mines from different perspectives, such as Li et al., Zhang and Yan, Meng et al., He et al., Yang, and Liu et al. who studied the relevant theories and methods of risk precontrol management of safety in coal mines [18 –23], and Chen, Hao and Song, Hao, Meng and Li, Li, and Qiao and Xie, who studied the construction and implementation of the risk precontrol management system of safety in coal mines [6 , 24–27]. With the development of the theoretical system of coal mine safety management, Chinese scholars have studied the disadvantages of the current risk precontrol management system of safety in coal mines and relevant improvement measures from different perspectives, e.g., Yang and Gao and Liu examined how to improve the risk precontrol effect of safety in coal mines from the perspective of institutions. In addition to the above viewpoints [28, 29], Fu and Zhang pointed out that the content of risk classification control is to identify hazards, assess hazards, and control risks [30]. Fu et al. further determined the relationship between hazards and accidents and found that hazards are the root cause of accidents [31]. Wang proposed the organic integration of the risk precontrol management system and post-standard operation process in coal mines and reported that the post-standard operation process is an effective measure to realize risk control [32]. Wang and Wu developed a new method of system safety management based on the best safety-related information, which attains a high correctness and effectiveness in theory [33]. In addition, a few scholars have investigated the operation effect of the risk precontrol management system of safety in coal mines in recent years, e.g., Jiang and Li proposed an evaluation model of the risk precontrol management system of safety in coal mines based on fuzzy data envelopment analysis (DEA) [34]. Sun conducted an empirical study on the operation effect of a risk precontrol system in the Shenhua Group and suggested targeted improvement countermeasures [35, 36].

In summary, relevant experts in China and abroad have performed much research and examination of the risk precontrol management system of safety in coal mines, and foreign research has transitioned from the stage of establishing risk management systems to the stage of improving and upgrading risk management systems, but most Chinese studies focus on the stage of system establishment and application and seldom involve the evaluation of the system operation effect and corresponding empirical research. Therefore, based on the above research background and status, this paper selects the operation effect evaluation process of the risk precontrol management system of safety in coal mines as the research object, applies the Objective and Subjective Weighting Method (OSWM) combined with the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) to conduct evaluation and empirical research on the operation effect of the risk precontrol management system of safety in coal mines and proposes policy suggestions to improve the risk precontrol management level of safety in coal mines based on the obtained evaluation results.

3 Construction of the evaluation index system

The establishment of an evaluation index system according to the actual conditions of coal mining enterprises is the premise and basis of the evaluation of risk precontrol management systems of safety in coal mines [37]. In this study, the steps of constructing the evaluation index system of the risk precontrol management system are shown in Fig. 1.

Fig. 1

Steps of constructing the evaluation index system.

3.1 Evaluation index acquisition and analysis

First, this paper analyzes 177 serious accidents and extraordinarily serious accidents in coal mines from 2014 to 2018, as listed in Tables 1 2, traces the causes of these accidents, and extracts the reasons influencing the effectiveness of the risk precontrol management system of safety in coal mines. The data are mainly obtained from the National Coal Mine Safety Administration, China Coal Network, and Coal Mine Safety Net. Second, based on the literature and theoretical analysis, this paper adopts Citespace combined with pivot tables in Excel to establish the PivotChart method and then extracts 33 influencing factors of the implementation of the risk precontrol management system, as summarized in Table 3.

Table 1
Number of serious and extraordinarily serious accidents and death toll from 2014–2018

Year 2014 2015 2016 2017 2018

Number of serious and extraordinarily serious accidents 46 40 29 32 30

Death toll 302 219 245 173 117

Year	2014	2015	2016	2017	2018
Number of serious and extraordinarily serious accidents	46	40	29	32	30
Death toll	302	219	245	173	117

Table 2

Type of serious and extraordinarily serious accidents and their numbers and death toll from 2014–2018

Accident type	Number of accidents	Death toll	Accident type	Number of accidents	Death toll
Gas explosion accidents	26	192	Carbon monoxide poisoning accidents	17	89
Water inrush accidents	27	184	Blasting accidents	7	46
Roof fall accidents	28	103	Coal and gas outburst accidents	32	176
Fire accidents	13	82	Accidents caused by natural disasters	3	8
Collapse accidents	7	42	Other accidents	11	79
Transportation accidents	3	41

Table 3

Literature distribution of the influencing factors of the implementation of the risk precontrol management system

Influencing factors	Literature number	Proportion (%)
1 Hazard identification	122	3.7983%
2 Risk assessment	51	1.5878%
3 Risk warning	77	2.3973%
4 Risk control	48	1.4944%
5 Monitoring and review	126	3.9228%
6 Organizational guarantee	45	1.401%
7 Institutional guarantee	140	4.3587%
8 Technology input	120	3.736%
9 Capital input	115	3.5803%
10 Safety education and training	130	4.0473%
11 Physical and mental states of employees	92	2.8643%
12 Cultural background of employees	101	3.1445%
13 Standardization of operation	82	2.5529%
14 Staff training	134	4.1719%
15 Employee behaviour supervision	118	3.6737%
16 Safety awareness and attitude of employees	127	3.9539%
17 Intact rate of the one ventilation and three prevention equipment system	163	5.0747%
18 Intact rate of mining equipment	134	4.1719%
19 Intact rate of blasting equipment	96	2.9888%
20 Intact rate of geological survey equipment	42	1.3076%
21 Intact rate of drainage equipment	97	3.0199%
22 Intact rate of transportation equipment	76	2.3661%
23 Intact rate of lifting equipment	139	4.3275%
24 Intact rate of other equipment	121	3.7671%
25 Environmental management	68	2.1171%
26 Emergency and accident management capability	148	4.6077%
27 Fire management	89	2.7709%
28 Occupational health management	70	2.1793%
29 Logo management	59	1.8369%
30 Operational management	68	2.1171%
31 Safety inspection institution	121	3.7671%
32 System audit institution	48	1.4944%
33 Management review institution	45	1.4010%

3.2 Integration of the evaluation indexes

Based on the above typical case analysis and literature research and referring to the risk precontrol management system of safety in coal mines specification (AQ/T 1093-2011), the evaluation indexes of the risk precontrol management system are summarized into six aspects: risk precontrol management, safety assurance management, personnel safety management, production equipment management, auxiliary management, and inspection, audit and review. Therefore, the evaluation index system is preliminarily constructed, including 6 first-class indicators and 33 second-class indicators.

3.3 Screening of the evaluation indexes

The preliminary evaluation indexes are modified by designing and analysing questionnaires combined with the correlation coefficient analysis method. Finally, the above six first-class indicators and 30 second-class indicators are screened as the evaluation indexes of the risk precontrol management system of safety in coal mines, as listed in Table 4.

Table 4
Evaluation indexes of the risk precontrol management system of safety in coal mines

First-class indicators Second-class indicators

Risk precontrol management (P₁) Hazard identification (P₁₁)

Risk assessment (P₁₂)

Risk warning (P₁₃)

Risk control (P₁₄)

Monitoring and review (P₁₅)

Safety assurance management (P2) Organizational guarantee (P₂₁)

Institutional guarantee (P₂₂)

Technology input (P₂₃)

Capital input (P₂₄)

Safety education and training (P₂₅)

Personnel safety management (P3) Physical and mental states of employees (P₃₁)

Staff training (P₃₂)

Employee behaviour supervision (P₃₃)

Safety awareness and attitude of employees (P₃₄)

Production equipment management (P4) Intact rate of the one ventilation and three prevention equipment system (P₄₁)

Intact rate of mining equipment (P₄₂)

Intact rate of blasting equipment (P₄₃)

Intact rate of geological survey equipment (P₄₄)

Intact rate of drainage equipment (P₄₅)

Intact rate of lifting equipment (P₄₆)

Intact rate of other equipment (P₄₇)

Auxiliary management (P5) Environmental management (P₅₁)

Emergency and accident management capability (P₅₂)

Fire management (P₅₃)

Occupational health management (P₅₄)

Logo management (P₅₅)

Operational management (P₅₆)

Inspection, audit and review (P6) Safety inspection institution (P₆₁)

System audit institution (P₆₂)

Management review institution (P₆₃)

First-class indicators	Second-class indicators
Risk precontrol management (P₁)	Hazard identification (P₁₁)
	Risk assessment (P₁₂)
	Risk warning (P₁₃)
	Risk control (P₁₄)
	Monitoring and review (P₁₅)
Safety assurance management (P2)	Organizational guarantee (P₂₁)
	Institutional guarantee (P₂₂)
	Technology input (P₂₃)
	Capital input (P₂₄)
	Safety education and training (P₂₅)
Personnel safety management (P3)	Physical and mental states of employees (P₃₁)
	Staff training (P₃₂)
	Employee behaviour supervision (P₃₃)
	Safety awareness and attitude of employees (P₃₄)
Production equipment management (P4)	Intact rate of the one ventilation and three prevention equipment system (P₄₁)
	Intact rate of mining equipment (P₄₂)
	Intact rate of blasting equipment (P₄₃)
	Intact rate of geological survey equipment (P₄₄)
	Intact rate of drainage equipment (P₄₅)
	Intact rate of lifting equipment (P₄₆)
	Intact rate of other equipment (P₄₇)
Auxiliary management (P5)	Environmental management (P₅₁)
	Emergency and accident management capability (P₅₂)
	Fire management (P₅₃)
	Occupational health management (P₅₄)
	Logo management (P₅₅)
	Operational management (P₅₆)
Inspection, audit and review (P6)	Safety inspection institution (P₆₁)
	System audit institution (P₆₂)
	Management review institution (P₆₃)

4 Evaluation and empirical research

4.1 Evaluation method

The commonly applied weighting methods mainly include the subjective weighting method, which is based on subjective experiential knowledge, and the objective weighting method, which is based on objective data using mathematical statistics. These two methods have their own advantages and disadvantages. First, the subjective weighting method computes qualitative indicators to obtain weights mainly based on the knowledge and experience of experts or scholars, which suffers a certain subjectivity. Moreover, the limitations of the educational background and research field of the experts may generate certain differences in the evaluation results. Second, the objective weighting method relies on mathematical statistics to calculate the index weight based on certain objective data, which objectively reflects the reality of the indexes but does not reflect the attributes among the indicators and may be inconsistent with the reality. Therefore, this paper applies the analytic hierarchy process (AHP) and entropy methods to obtain the independent weights and then comprehensively implements the OSWM to obtain the combined weights [38 –40].

A coal mine system is a dynamic and complex system composed of multiple factors in time and space, and its safety status is influenced by many factors. TOPSIS ranks the relative advantages and disadvantages of the existing multiple indexes. Its core idea is to calculate the distance between the evaluation object and the optimal and worst solutions according to the solution steps based on the obtained indexes, and it adopts the absolute distance to measure the advantages and disadvantages of various schemes [41]. Therefore, after applying the OSWM to obtain the combined weights, TOPSIS is implemented to determine the closeness degree on the basis of the ideal solution and Euclidean distance. Furthermore, the comprehensive evaluation score of the system operation effect is calculated by multiplying the closeness degree and combined weight. The combination of OSWM and TOPSIS gives full play to the advantages of these two methods, ensures accurate and undistorted evaluation results and accurately and logically achieves the evaluation of the system operation effect.

4.2 Empirical analysis

It has been seven years since the Gengcun coal mine of the Yimei Group in Henan Province started to construct and implement the risk precontrol management system of safety. Therefore, on the basis of the above evaluation index system, this paper chooses the Gengcun coal mine as an example to conduct an empirical study on the operational effect of the risk precontrol management system of safety by comprehensively applying the OSWM and TOPSIS and proposes policy suggestions to enhance the risk precontrol management level according to the evaluation results.

4.2.1 Determination of the subjective weights using AHP

With the use of the expert scoring method, 25 highly experienced experts were selected, including 8 professors engaged in coal mine safety management research at universities for many years, and the rest consisted of middle-level leaders or above of the management, technology and production departments of coal mine enterprises. Furthermore, the statistics, analysis, and summary of the expert opinions were anonymous obtained, and the judgement matrix was then constructed by comparing the indexes in pairs according to the expert experiences. MATLAB was adopted to determine the judgement matrix to obtain the weights at all levels of the indicators, whereby the weights of the first-level indicators were as follows: the weight of P₁ is 0.3794, that of P₂ is 0.1605, that of P₃ is 0.2488, that of P₄ is 0.1024, that of P₅ is 0.0435, and that of P₆ is 0.0655. The weights of the second-level indicators are listed in Table 5.

Table 5
Weights of the evaluation index system calculated by the AHP, entropy method and OSWM

Indexes Weights (AHP) Weights (entropy method) Weights (OSWM) Indexes Weights (AHP) Weights (entropy method) Weights (OSWM)

P₁₁ 0.158 0 0.032 9 0.171 0 P₄₂ 0.038 2 0.031 8 0.040 0

P₁₂ 0.037 0 0.026 9 0.032 7 P₄₃ 0.017 4 0.033 5 0.019 2

P₁₃ 0.061 0 0.039 9 0.080 1 P₄₄ 0.010 0 0.033 1 0.010 9

P₁₄ 0.099 0 0.027 0 0.087 9 P₄₅ 0.006 1 0.039 2 0.007 9

P₁₅ 0.024 0 0.034 4 0.027 2 P₄₆ 0.007 4 0.033 7 0.008 2

P₂₁ 0.041 0 0.041 6 0.056 1 P₄₇ 0.002 4 0.038 2 0.003 0

P₂₂ 0.066 0 0.025 9 0.056 2 P₅₁ 0.016 0 0.034 5 0.018 2

P₂₃ 0.017 0 0.029 3 0.016 4 P₅₂ 0.011 0 0.030 2 0.010 9

P₂₄ 0.027 0 0.026 1 0.023 2 P₅₃ 0.008 0 0.034 2 0.009 0

P₂₅ 0.009 0 0.042 5 0.012 6 P₅₄ 0.003 0 0.032 8 0.003 2

P₃₁ 0.123 6 0.033 8 0.137 5 P₅₅ 0.002 0 0.031 0 0.002 0

P₃₂ 0.021 9 0.035 9 0.025 9 P₅₆ 0.004 0 0.038 8 0.005 1

P₃₃ 0.023 2 0.037 2 0.028 4 P₆₁ 0.019 0 0.032 4 0.020 3

P₃₄ 0.013 9 0.027 2 0.012 4 P₆₂ 0.011 0 0.029 3 0.010 6

P₄₁ 0.023 5 0.034 1 0.026 4 P₆₃ 0.035 0 0.032 6 0.037 5

Indexes	Weights (AHP)	Weights (entropy method)	Weights (OSWM)	Indexes	Weights (AHP)	Weights (entropy method)	Weights (OSWM)
P₁₁	0.158 0	0.032 9	0.171 0	P₄₂	0.038 2	0.031 8	0.040 0
P₁₂	0.037 0	0.026 9	0.032 7	P₄₃	0.017 4	0.033 5	0.019 2
P₁₃	0.061 0	0.039 9	0.080 1	P₄₄	0.010 0	0.033 1	0.010 9
P₁₄	0.099 0	0.027 0	0.087 9	P₄₅	0.006 1	0.039 2	0.007 9
P₁₅	0.024 0	0.034 4	0.027 2	P₄₆	0.007 4	0.033 7	0.008 2
P₂₁	0.041 0	0.041 6	0.056 1	P₄₇	0.002 4	0.038 2	0.003 0
P₂₂	0.066 0	0.025 9	0.056 2	P₅₁	0.016 0	0.034 5	0.018 2
P₂₃	0.017 0	0.029 3	0.016 4	P₅₂	0.011 0	0.030 2	0.010 9
P₂₄	0.027 0	0.026 1	0.023 2	P₅₃	0.008 0	0.034 2	0.009 0
P₂₅	0.009 0	0.042 5	0.012 6	P₅₄	0.003 0	0.032 8	0.003 2
P₃₁	0.123 6	0.033 8	0.137 5	P₅₅	0.002 0	0.031 0	0.002 0
P₃₂	0.021 9	0.035 9	0.025 9	P₅₆	0.004 0	0.038 8	0.005 1
P₃₃	0.023 2	0.037 2	0.028 4	P₆₁	0.019 0	0.032 4	0.020 3
P₃₄	0.013 9	0.027 2	0.012 4	P₆₂	0.011 0	0.029 3	0.010 6
P₄₁	0.023 5	0.034 1	0.026 4	P₆₃	0.035 0	0.032 6	0.037 5

4.2.2 Determination of the objective weight using the entropy method

The entropy method facilitates an objective and fair comprehensive system evaluation as an objective weighting method. This paper selects the corresponding data of the Gengcun coal mine from 2014 to 2018 according to AQ/T 1093-2011 and the actual situation to evaluate the scores of each indicator using the entropy method. The weights at all levels of the indicators are determined. Specifically, the weights of the first-level indicators are as follows: the weight of P₁ is 0.1611, that of P₂ is 0.1655, that of P₃ is 0.1341, that of P₄ is 0.2436, that of P₅ is 0.2015, and that of P₆ is 0.0943. In addition, the weights of the second-level indicators are summarized in Table 5.

4.2.3 Determination of the combined weight using OSWM

After obtaining the independent weights by the AHP and entropy methods, the combined weights (Equation 1) are obtained by using the OSWM. Specifically, the weights of the first-level indicators are as follows: the weight of P₁ is 0.1720, that of P₂ is 0.1680, that of P₃ is 0.1860, that of P₄ is 0.1840, that of P₅ is 0.1180, and that of P₆ is 0.1720. Moreover, the weights of the second-level indicators are provided in Table 5. The combined use of the AHP and entropy methods, i.e., applying the OSWM to obtain the final weights of the indicators, not only makes use of the rich knowledge and experience of experts but also integrates the objective reality of actual data and overcomes the shortcomings of a single method to a certain extent. $W_{i}^{*} = \frac{W_{i} W_{I}^{'}}{\sum_{i = 1}^{m} W_{i} W_{I}^{'}}$ (1)

where:

W_i^* is the combined weight of the ith index calculated by the OSWM, W_i is the weight calculated by the AHP, $W_{i}^{'}$ is the weight calculated by the entropy method, and m is the number of indexes.

4.2.4 Establishment of the TOPSIS evaluation model

Based on the above literature research, most scholars have focused on the classification of the coal mine safety status, which is divided into five levels, namely, level I (effective), level II (relatively effective), level III (basically effective), level IV (basically ineffective), and level V (ineffective). The critical value of each level is defined according to the relevant provisions of the coal mine industry. With the Gengcun coal mine as an example, the reference value of the system operation state under the second-class indexes is obtained according to the actual conditions of the mine, and the reference value data (using the percentage system) are adopted as standards to evaluate the coal mine safety status, as listed in Table 6.

Table 6
Critical value division of each effective level in the Gengcun coal mine

Second -class indexes I II III IV V Reference value Second-class indexes I II III IV V Reference value

P₁₁ > 95 > 90 > 70 > 50 ≤50 76 P₄₂ > 95 > 90 > 85 > 80 ≤80 93

P₁₂ > 95 > 90 > 70 > 50 ≤50 92 P₄₃ > 95 > 90 > 85 > 80 ≤80 85

P₁₃ > 95 > 90 > 70 > 50 ≤50 63 P₄₄ > 95 > 90 > 85 > 80 ≤80 89

P₁₄ > 95 > 90 > 70 > 50 ≤50 85 P₄₅ > 95 > 90 > 85 > 80 ≤80 77

P₁₅ > 95 > 90 > 70 > 50 ≤50 79 P₄₆ > 95 > 90 > 85 > 80 ≤80 87

P₂₁ > 90 > 85 > 80 > 50 ≤50 77 P₄₇ > 95 > 90 > 85 > 80 ≤80 79

P₂₂ > 90 > 85 > 80 > 50 ≤50 85 P₅₁ > 90 > 80 > 70 > 60 ≤60 82

P₂₃ > 90 > 85 > 80 > 50 ≤50 84 P₅₂ > 90 > 80 > 70 > 60 ≤60 79

P₂₄ > 90 > 85 > 80 > 50 ≤50 92 P₅₃ > 90 > 80 > 70 > 60 ≤60 81

P₂₅ > 90 > 85 > 80 > 50 ≤50 65 P₅₄ > 90 > 80 > 70 > 60 ≤60 89

P₃₁ > 95 > 90 > 80 > 70 ≤70 92 P₅₅ > 90 > 80 > 70 > 60 ≤60 92

P₃₂ > 95 > 90 > 80 > 70 ≤70 89 P₅₆ > 90 > 80 > 70 > 60 ≤60 93

P₃₃ > 95 > 90 > 80 > 70 ≤70 95 P₆₁ > 95 > 90 > 80 > 70 ≤70 92

P₃₄ > 95 > 90 > 80 > 70 ≤70 96 P₆₂ > 95 > 90 > 80 > 70 ≤70 95

P₄₁ > 95 > 90 > 85 > 80 ≤80 95 P₆₃ > 95 > 90 > 80 > 70 ≤70 89

Second -class indexes	I	II	III	IV	V	Reference value	Second-class indexes	I	II	III	IV	V	Reference value
P₁₁	> 95	> 90	> 70	> 50	≤50	76	P₄₂	> 95	> 90	> 85	> 80	≤80	93
P₁₂	> 95	> 90	> 70	> 50	≤50	92	P₄₃	> 95	> 90	> 85	> 80	≤80	85
P₁₃	> 95	> 90	> 70	> 50	≤50	63	P₄₄	> 95	> 90	> 85	> 80	≤80	89
P₁₄	> 95	> 90	> 70	> 50	≤50	85	P₄₅	> 95	> 90	> 85	> 80	≤80	77
P₁₅	> 95	> 90	> 70	> 50	≤50	79	P₄₆	> 95	> 90	> 85	> 80	≤80	87
P₂₁	> 90	> 85	> 80	> 50	≤50	77	P₄₇	> 95	> 90	> 85	> 80	≤80	79
P₂₂	> 90	> 85	> 80	> 50	≤50	85	P₅₁	> 90	> 80	> 70	> 60	≤60	82
P₂₃	> 90	> 85	> 80	> 50	≤50	84	P₅₂	> 90	> 80	> 70	> 60	≤60	79
P₂₄	> 90	> 85	> 80	> 50	≤50	92	P₅₃	> 90	> 80	> 70	> 60	≤60	81
P₂₅	> 90	> 85	> 80	> 50	≤50	65	P₅₄	> 90	> 80	> 70	> 60	≤60	89
P₃₁	> 95	> 90	> 80	> 70	≤70	92	P₅₅	> 90	> 80	> 70	> 60	≤60	92
P₃₂	> 95	> 90	> 80	> 70	≤70	89	P₅₆	> 90	> 80	> 70	> 60	≤60	93
P₃₃	> 95	> 90	> 80	> 70	≤70	95	P₆₁	> 95	> 90	> 80	> 70	≤70	92
P₃₄	> 95	> 90	> 80	> 70	≤70	96	P₆₂	> 95	> 90	> 80	> 70	≤70	95
P₄₁	> 95	> 90	> 85	> 80	≤80	95	P₆₃	> 95	> 90	> 80	> 70	≤70	89

The initial judgement matrix (Q₁) is established according to Table 6. In Table 6, each row contains the second-class indexes of P₁₁, P₁₂, P₁₃, P₁₄, and P₁₅, the first four columns are the quantitative values of the operating state of the Gengcun coal mine, which are obtained through expert scoring according to the actual situation of the Gengcun coal mine, and the fifth column provides the reference value. Then, the initial judgement matrix (Q₁) is first standardized and multiplied by the above corresponding combined weights to obtain the weighting matrix ( $Q_{1}^{'}$ ). Furthermore, the positive ideal solution (V⁺) and negative ideal solution (V^–) are calculated. Finally, the Euclidean distances to the positive and negative ideal solutions, namely, D⁺ and D^–, respectively, are calculated, and the relative closeness degree (E_ij) is determined.

$\begin{matrix} Q_{1} = (\begin{matrix} 93 & 94 & 83 & 63 & 76 \\ 96 & 92 & 80 & 69 & 92 \\ 97 & 90 & 77 & 68 & 63 \\ 99 & 91 & 84 & 54 & 85 \\ 95 & 90 & 79 & 52 & 79 \end{matrix}), \\ Q_{1}^{''} = (\begin{matrix} 0.0941 & 0.0717 & 0.0885 & 0.0723 & 0.0833 \\ 0.0165 & 0.0164 & 0.0164 & 0.0185 & 0.0191 \\ 0.0274 & 0.0265 & 0.0260 & 0.0301 & 0.0277 \\ 0.0466 & 0.0435 & 0.0461 & 0.0586 & 0.0473 \\ 0.0106 & 0.0111 & 0.0105 & 0.0091 & 0.0131 \end{matrix}) \end{matrix}$ $\begin{matrix} V_{1}^{+} = (0.094, 0.019, 0.030, 0.059, 0.013), \\ V_{1}^{-} = (0.072, 0.016, 0.026, 0.043, 0.009) \end{matrix}$

$\begin{matrix} D_{11}^{+} = 0.013, D_{11}^{-} = 0.023, D_{12}^{+} = 0.027, \\ D_{12}^{-} = 0.068, D_{13}^{+} = 0.015, \\ D_{13}^{-} = 0.085, D_{14}^{+} = 0.022, D_{14}^{-} = 0.070, \\ D_{15}^{+} = 0.016, D_{15}^{-} = 0.082 \end{matrix}$

$\begin{matrix} E_{11} = 0.638, E_{12} = 0.713, E_{13} = 0.852, \\ E_{14} = 0.760, and E_{15} = 0.837 \end{matrix}$

Similarly, the closeness degrees of the remaining 25 second-class indicators, which are subordinate to the other five first-class indicators, are obtained. $\begin{matrix} E_{21} = 0.634, E_{22} = 0.218, E_{23} = 0.284, \\ E_{24} = 0.642, E_{25} = 0.519 \end{matrix}$

$\begin{matrix} E_{31} = 0.711, E_{32} = 0.661, E_{33} = 0.816, \\ E_{34} = 0.194, E_{35} = 0.640 \end{matrix}$

$\begin{matrix} E_{41} = 0.5139, E_{42} = 0.5893, E_{43} = 0.5140, \\ E_{44} = 0.5461, E_{45} = 0.3639 \end{matrix}$

$\begin{matrix} E_{51} = 0.504, E_{52} = 0.537, E_{53} = 0.529, \\ E_{54} = 0.199, E_{55} = 0.786 \end{matrix}$ $\begin{matrix} E_{61} = 0.676, E_{62} = 0.673, E_{63} = 0.704, \\ E_{64} = 0.35, E_{65} = 0.368 \end{matrix}$

Therefore, the closeness degree matrix (E) is constructed based on the above calculation of the closeness degrees of the various indicators. Furthermore, the weight matrix (α) is established based on the above obtained weights of the first-level indicators using the OSWM, namely, 0.172, 0.168, 0.186, 0.184, 0.118, and 0.172. Finally, the standardized and quantitative values of the operation effect level of the risk precontrol management system (F) are computed. $α = (0.172 0.168 0.186 0.184 0.118 0.172)$ $E = (\begin{matrix} 0.638 & 0.713 & 0.852 & 0.760 & 0.837 \\ 0.634 & 0.218 & 0.284 & 0.642 & 0.519 \\ 0.711 & 0.661 & 0.816 & 0.194 & 0.640 \\ 0.514 & 0.589 & 0.514 & 0.546 & 0.364 \\ 0.504 & 0.537 & 0.529 & 0.199 & 0.786 \\ 0.676 & 0.673 & 0.704 & 0.350 & 0.368 \end{matrix})$ $\begin{matrix} F & = α \times E = (0.172 0.168 0.186 0.184 0.118 0.172) \\ \times (\begin{matrix} 0.638 & 0.713 & 0.852 & 0.760 & 0.837 \\ 0.634 & 0.218 & 0.284 & 0.642 & 0.519 \\ 0.711 & 0.661 & 0.816 & 0.194 & 0.640 \\ 0.514 & 0.589 & 0.514 & 0.546 & 0.364 \\ 0.504 & 0.537 & 0.529 & 0.199 & 0.786 \\ 0.676 & 0.673 & 0.704 & 0.350 & 0.368 \end{matrix}) \\ = (0.6188 0.5697 0.6240 0.4588 0.5734) \end{matrix}$

4.2.5 Result analysis and policy suggestions

The first four columns of F contain the standardized and quantitative values of the operation effect level of the risk precontrol management system, and the last column provides the reference value of the operation effect level of the coal mine, which are converted into percentages to grade the values according to I > 62.4, II > 61.87, III > 56.97, IV > 45.88, and V≤45.88, while the reference value is 57.34. The operation effect level of the Gengcun coal mine is at level III, which is basically effective and is consistent with the actual environment of the coal mine. This indicates that the coal mine can continue to implement the risk precontrol management system according to the actual situation and should strengthen any remedial measures targeting existing defects in a timely manner. Moreover, after the closeness degree matrix (E) is converted into the percentage matrix, the first four columns contain the critical values of each operation effect level, and the fifth column provides the reference values of the six first-level indicators. Specifically, the reference value of risk precontrol management (P₁) is 83.7, which reveals that the operation effect level of risk precontrol management (P₁) is level II (relatively effective). The reference value of safety assurance management (P2) is 51.9, which indicates that the operation effect level of safety assurance management (P2) is level III (basically effective). The reference value of personnel safety management (P3) is 64.0, which demonstrates that the operation effect level of personnel safety management (P3) is level IV (basically ineffective). The reference value of production equipment management (P4) is 36.4, which shows that the operation effect level of production equipment management (P4) is level V (ineffective). The reference value of auxiliary management (P5) is 78.6, which reveals that the operation effect level of auxiliary management (P5) is level I (effective), and the reference value of inspection, audit and review (P6) is 36.8, which indicates that the operation effect level of inspection, audit and review (P6) is basically ineffective. Therefore, policy suggestions to enhance the level of risk precontrol management in the Gengcun coal mine are proposed based on the above evaluation results.

First, it is determined from the evaluation results that the calculated reference values of production equipment management (P4) and inspection, audit and review (P6) are the lowest and lower than the critical value corresponding to operation effect level V, which comprise the short list of enterprises. Therefore, the enterprise should focus on strengthening the following two aspects. The troubleshooting and maintenance processes of ventilation equipment, mining equipment, blasting equipment, ground survey equipment, fire prevention equipment, and dustproof equipment in the production system should be enhanced, and relevant equipment should be modified if necessary. Second, the aspects of the revision and improvement of inspection, audit and review should be improved, specifically: (1) the evaluation standards reflecting the performance of the risk precontrol management system in the enterprise should be revised and improved; (2) regular or irregular inspection should be conducted according to the evaluation standards, emphasizing the hazards that may cause unacceptable risks, and the inspection records must be true, accurate and traceable; (3) the causes of the nonconforming items identified during the inspection should be analysed and corrected according to the treatment procedures relevant to the nonconformity, and corresponding preventive measures should be re-audited or re-formulated to prevent recurrence; (4) system audit procedures should be properly revised and maintained, and audits of the risk precontrol management system should be regularly performed to determine whether the conformity evaluation between the audit situation and system effectively meets the enterprise’s policies and objectives; and (5) the system should be reviewed according to specified time intervals to ensure the continuous suitability, sufficiency and effectiveness of the system.

In addition, the reference value of safety assurance management (P₂) is 51.9, corresponding to operation effect level IV, which should also be strengthened and enhanced from the perspective of organizational, institutional, technical, capital and safety culture guarantees. Specifically, (1) in terms of the organizational guarantee, a risk precontrol management organization should be established and improved; (2) in terms of the institutional guarantee, the management institutions related to the risk precontrol management system of safety should be developed and enhanced; (3) in terms of the technical guarantee, safety technology management and control procedures should be regularly revised and maintained to control major hazards; (4) in terms of the capital guarantee, safety investment management and control procedures should be suitably updated and maintained to provide the necessary financial support for the implementation and improvement of the risk precontrol management system; and (5) in terms of the safety culture guarantee, safety culture construction management procedures should be revised and supported to give full play to the guidance, incentive and cohesion functions of the safety culture.

Finally, the calculation reference values of risk precontrol management (P₁) and auxiliary management (P₅) reach the critical value corresponding to operation effect level I, which indicates a good operation effect. The coal mining enterprise can continue existing working methods and focus on tracking the operation effect of relevant indicators to ensure that the operation effect of these indicators remains suitable and stable.

5 Conclusion

This paper chooses the evaluation of the system operation effect as the research object. An evaluation index system is first constructed, after which the OSWM combined with TOPSIS is applied to conduct evaluation and empirical research on the operation effect of the risk precontrol management system of safety in coal mines. Moreover, corresponding policy suggestions to improve the level of risk precontrol management in coal mines are proposed based on the evaluation results. The main research results are summarized below.

First, the influencing factors of coal mine accidents are extracted and summarized by analysing 177 serious and extraordinarily serious accidents from 2014 to 2018, and these influencing factors are then integrated and screened by using questionnaire surveys and the correlation coefficient method. Six first-class indicators, including risk precontrol management (P₁), safety assurance management (P₂), personnel safety management (P₃), production equipment management (P₄), auxiliary management (P₅), and inspection, audit and review (P₆), and 30 subordinate second-class indicators are screened as the evaluation indicators of the risk precontrol management system of safety in coal mines.

Second, the OSWM combined with TOPSIS is an effective approach to evaluate the operation effect of the risk precontrol management system of safety in coal mines, which includes the following five steps: determination of the subjective weight using AHP, determination of the objective weight using the entropy method, determination of the combined weight using the OSWM, establishment of the TOPSIS evaluation model, and result analysis. The combination of the OSWM and TOPSIS gives full play to the advantages of these two methods, ensures accurate and undistorted evaluation results and accurately and logically achieves the evaluation of the system operation effect.

Third, this paper selects the Gengcun coal mine as an example to conduct an empirical study, and the results reveal that the calculation reference value of the operation effect of the Gengcun coal mine is 57.34, and its corresponding operation effect level is level III, which is basically effective. Moreover, the calculation reference values of production equipment management (P₄) and inspection, audit and review (P₆) are the lowest and lower than the critical value corresponding to operation effect level V, which comprise the short list of enterprises, and the calculation reference values of risk precontrol management (P₁) and auxiliary management (P₅) reach the critical value corresponding to operation effect level I, which indicates a good operation effect.

Footnotes

Acknowledgments

This work was supported by the the National Social Science Foundation of China (Grant no. 21BGL297).

Conflict of interest

The authors declare that they do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

References

Liu

, Li

. Modeling and evaluation of the safety control capability of coal mine based on system safety. Journal of Cleaner Production. 2014;84:797–802.

Meng

, Liu

, Luo

, Zhou

. Risk assessment of human’s unsafe behavior in fatal gas explosion accidents in China’s underground coal mines. Journal of Cleaner Production. 2019;210:970–976.

Liu

, Li

, Meng

. Effectiveness research on the multi-player evolutionary game of coal-mine safety regulation in China based on System Dynamics. Safety Science. 2019;111:224–233.

Liu

, Meng

, Hassall

, Li

. Accident-causing mechanism in coal mine based on hazards and polarized management. Safety Science. 2016;85:276–281.

Gunningham

. Mine safety: lawregulation policy. Sydney: The Federation Press. 2007;12–13.

Meng

, Li

. The construction and implementation guide on risk precontrol management system of safety in coal mines. Xuzhou: China University of mining and Technology Press 2012.

Hao

. Risk precontrol management system of safety in coal mines. Beijing: Coal Industry Press 2012.

Liu

. Systematic evolutionary game analysis and control scenarios of coal mine safety inspection and regulation in China. Xuzhou, China University of mining and technology: Doctoral Dissertation 2016.

Aliabadi

, Darvishi

, Shahidi

, Ghasemi

, Mahdinia

. Explanation and prediction of accidents using the path analysis approach in industrial units: the effect of safety performance and climate. WORK: A Journal of Prevention, Assessment & Rehabilitation. 2020;66(3):617–624.

10.

Adams

. Accident causation and the management system. Professional Safety. 1976;21(10):26–29.

11.

Saari

. Accidents, and disturbances in the flow of information. Journal of Occupational Accidents.. 1984;6(1-3):91–105.

12.

Robson

, Clarke

, Cullen

, et al. , The effectiveness of occupational health and safety management system interventions: A systematic review. Safety Science. 2007;45(3):329–353.

13.

Asadzadeh

, Azadeh

, Negahban

, et al. , Assessment and improvement of integrated HSE and macro-ergonomics factors by fuzzy cognitive maps: The case of a large gas refinery. Journal of Loss Prevention in the Process Industries. 2013;26(6):1015–1026.

14.

Ghahramani

. Factors that influence the maintenance and improvement of OHSAS 1 in adopting companies: A qualitative study. Journal of Cleaner Production. 2016;137:283–290.

15.

Rothmore

, Gray

. Using the Work Ability Index to Identify Workplace Hazards. WORK: A Journal of Prevention, Assessment & Rehabilitation. 2019;62(2):251–259.

16.

Haas

, Yorio

. Exploring the state of health and safety management system performance measurement in mining organizations. Safety Science. 2016;83:48–58.

17.

Stolzer

, Friend

, Truong

, et al. , Measuring and evaluating safety management system effectiveness using data envelopment analysis. Safety Science. 2018;104:55–69.

18.

, Ma

, Li

. Analysis of coal mine safety management based on risk precontrol, Coal Engineering. 2010;42(4):118–119.

19.

Zhang

, Yan

. Gray prediction model for prevention and control of coal mine safety risk system application, Journal of System Science 2010;18(2):84–87.

20.

Meng

, Song

, Zhang

. Study on risk pre-control continuum theory in coal mine, China Safety Science Journal 2011;21(8):90–94.

21.

, Yang

, Li

. Practical application of risk pre-control technology to coal mines in China, China Coal 2011;37(8):89–72.

22.

Yang

. Research on safety risk pre-control management mode of coal mine, Coal Mining Technology. 2012;17(4):102–104.

23.

Liu

, Meng

, Li

, Luo

. Risk precontrol continuum and risk gradient control in underground coal mining, Process Safety and Environmental Protection. 2019;129:210–219.

24.

Chen

. Coal mine intrinsic safety management based on risk pre-control, China Safety Science Journal. 2007;17(7):59–62.

25.

Hao

, Song

. Coal mine intrinsic safety management, Xuzhou: China University of mining and Technology Press 2008.

26.

. Application of coal mine safety risk pre-control management system in Shangwan coal mine. Safety in Coal Mines. 2013;44(6):227–229.

27.

Qiao

, Xie

. Methods and measures of practicing safety risk pre control management system in Yimei group, Coal Engineering 2013;45(10):134–136.

28.

Yang

. Study on the system of coal mine safety risk prevention and control based on internal control management, China Coal 2015;41(7):120–123.

29.

Gao

, Liu

. Application of coal mine post standard operation procedure in lean and risk pre-control management system. Safety in Coal Mines. 2017;48(S1):122–127.

30.

, Suo

, Wang

. Study on the systematic characteristics of 24 Model. Systems Engineering –Theory & Practice. 2018;38(1):263–272.

31.

, Zhang

. Analysis of direct cause of accident and relationship between hazard and hidden danger. China Safety Science Journal. 2018;28(5):2–5.

32.

Wang

. The merging of coal mine post standard operating procedure and risk pre-control management system of safety in coal mine. Coal Engineering. 2019;51(4):152–156.

33.

Wang

, Wu

. A new approach of system safety management based on the evidence and risk: ERBS. Journal of Intelligence. 2018;37(9):142–147.

34.

Jiang

, Li

. Assessment model for coal mine safety risk pre-control management system based on fuzzy DEA. Safety in Coal Mines. 2015;46(11):238–244.

35.

Sun Q

. Empirical study on comprehensive evaluation of running of risk precontrol system in coal mines. China Coal. 2015;41(1):108–113.

36.

Sun

. Empirical study on the weightiness of evaluation indices of the RPMSCS operational performance. China Mining Magazine. 2016;25(1):38–42.

37.

Barbosa

, Azevedo

, Rodrigues

. Occupational safety and health performance indicators in SMEs: A literature review. WORK: A Journal of Prevention, Assessment & Rehabilitation. 2019;64(2):217–227.

38.

, Chen

, You

. Evaluation of circular economy of the urban agglomeration of Changsha, Zhuzhou and Xiangtan based on the subjective and objective weighting. Systems Engineering. 2010;28(1):113–117.

39.

Zhang

, Liu

, Zhang

. Situation assessment modeling for CGF based on the subjective and objective integrated weight. Systems Engineering and Electronics. 2013;35(1):85–90.

40.

Song

, Liu

, Shen

, Shi

, Qi

, Feng

. Multiple objective and attribute decision making based on the subjective and objective weighting. Journal of Shandong University (Engineering Science). 2015;45(4):1–9.

41.

Luo

, Peng

. Research on the vendor evaluation and selection based on AHP and TOPSIS in green supply chain. Soft Science. 2011;25(2):53–56.

Evaluation and empirical research on the operation effect of the risk precontrol management system of safety in coal mines based on OSWM-TOPSIS

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSIONS:

Keywords

1 Introduction

2 Literature review

3 Construction of the evaluation index system

Table 1 Number of serious and extraordinarily serious accidents and death toll from 2014–2018 Year 2014 2015 2016 2017 2018 Number of serious and extraordinarily serious accidents 46 40 29 32 30 Death toll 302 219 245 173 117

3.3 Screening of the evaluation indexes

4.1 Evaluation method

4.2 Empirical analysis

4.2.1 Determination of the subjective weights using AHP

4.2.3 Determination of the combined weight using OSWM

5 Conclusion

Footnotes

Acknowledgments

Conflict of interest

References

Table 1
Number of serious and extraordinarily serious accidents and death toll from 2014–2018

Year 2014 2015 2016 2017 2018

Number of serious and extraordinarily serious accidents 46 40 29 32 30

Death toll 302 219 245 173 117