Safety warning of coal mining face based on big data association rule mining

4.1.1 Structured processing

The data of daily maintenance, maintenance and related maintenance equipment, materials, accident processing and management personnel of coal mining face are mostly stored in text form. These unstructured texts store abundant valuable information. The data mining model is difficult to directly process these unstructured information. It is necessary to conduct structural processing on these texts to form structured data so as to meet the requirements of data mining. This paper adopts the unstructured or semi-structured data preprocessing in the big data platform based on association rules, the main steps are as follows: logical structure extraction, document preprocessing, data extraction and data organization.

1) Logical structure extraction

The unstructured text logical structure can be expressed as Eq. (1):

𝑇𝑎𝑏𝑙𝑒𝑆𝑡𝑟𝑢 = { 𝐼𝐷 , 𝑇𝑢𝑏𝑙𝑒𝐼𝐷 , 𝑇𝑖𝑡𝑙𝑒 , 𝑇𝑦𝑝𝑒 , 𝑃𝑎𝑟𝑒𝑛𝑡 }(1)

where ID is the identity of the document, TupleID is the identity of tuples, Title is Title area, Type is title area Type {Single, Multiple}, where Single means “Single value area” and Multiple means “many value area”, and Parent is Parent header area for a header area.

The input extracted by logical structure is an unstructured table document that does not contain any actual data. By matching the data dictionary, the table’s header areas are extracted, and the logical relationship is established. Finally, the extracted results are saved in the logical structure library [51].

2) Data extraction

Data extraction is to extract data areas in the table and establish semantic relations with the header area according to the structural features of documents and tables and data flow features. Data extraction in single-value regions and multi-value regions, pre-defined extraction rules, and according to data characteristics, the corresponding algorithm is adopted to generate multi-group structured data sets.

3) Data organization and storage

The data extraction is completed to extract the header area and data area from the unstructured data flow. The task of data organization is to organize the data according to certain rules and transform it into structured data. Finally, structured data text documents are moved to Hadoop’s HDFS distributed file system.

The structured data model can be represented by multiple groups, as shown in Eq. (2):

𝐷𝑎𝑡𝑎𝑆𝑡𝑟𝑢 = { 𝐼𝐷 , 𝑇𝑢𝑝𝑙𝑒𝐼𝐷 , 𝑇𝑖𝑡𝑙𝑒𝐼𝐴𝑟𝑒𝑎 , 𝐷𝑎𝑡𝑎𝐴𝑟𝑒𝑎 , 𝑅𝑒𝑙𝑎𝑡𝑖𝑜𝑛 , 𝐹𝑜𝑟𝑒𝑖𝑔𝑛 }(2)

where ID is unique identification of the document, TupleID is unique identity of tuples, TitleArea is title area value, DataArea is the value of a data area, which can be a header area string or a DataArea string; Where the Relation value is 1 or 0, when the value is 1, it means that the DataArea is the subtitle of TitleArea, when the value is 0, it means that the DataArea is the DataArea corresponding to TitleArea, and Foreign is the parent section of the header section.

According to the above treatment methods, combined with the hidden danger data attribute Table 1, the text hidden danger data of coal mining face can be transformed into the quantity expression form.

There is usually a large amount of abnormal data in the raw data collected, such as the deviation of the expected value of key indicators, missing key attribute information, error type data, etc., which will seriously affect the quality of the information if it is not cleaned up. The data is processed by smoothing noise data, filling in missing and missing data, removing abnormal data, and standardizing the data structure so that different data are integrated into a set.

The following issues need to be considered during data integration [52]:

1) Pattern integration problem

Entity setting problem, that is, the entities of multiple data sources match each other, such as whether the initial database shearer _ID of the coal face is the same entity as the coal miner _number in the run database. Solving this problem can be done through metadata contained in a database or data warehouse to avoid errors during schema integration.

2) Redundancy

In data integration, redundancy issues often arise. If an attribute can be inferred from other attributes, then the attribute is redundant, and inconsistent naming of attributes leads to redundancy. Correlation analysis can be used to identify the correlation between attributes, such as the correlation between attributes X and Y , which can be obtained according to the Eq. (3).

r X , Y = ∑ ( X - X ¯ ) ⁢ ( Y - Y ¯ ) ( n - 1 ) ⁢ δ X ⁢ δ Y(3)

where X ⁢ ( - ) and Y ⁢ ( - ) are the average values of X and Y respectively; the δ X and the δ Y are the standard deviation of X and Y respectively.

If $r_{X,Y}>0$ , then the attributes X and Y are positively correlated; The higher the r X , Y , the stronger the positive correlation; If r X , Y = 0 , then the attributes X and Y are independent of each other;

If 0, then then the attributes X and Y are negatively correlated, the greater the absolute value of r X , Y , the stronger the negative correlation.

When X and Y are positively correlated, and the value of r X , Y is large, X or Y are eliminated as redundant terms. In addition, duplicate data with more than two tuples is removed.

3) Data value conflict detection and elimination

For the same entity, the attribute values from different data sources may be different. For example, in the roadway of the coal mining face transportation along the coal chute, the fracture of the hanging net led to the sheet gang, the security officer on duty recorded the area of the falling off of the net, and the mine safety supervisor calculated the volume of the falling off of the sheet gang. In this way, the same hidden danger comes from different records and descriptions in the coal mining area and the mine safety monitoring station, resulting in data value conflict.

Data selection is based on business requirements, data information is selected, then data formatting is conducted, and unified coding is adopted to form a security warning database for data mining.

Data transformation refers to transforming or merging data into a form suitable for data mining, which main tasks are smoothing, aggregation, data generalization, normalization and attribute construction.

The normalization of data is to drop the data of related attributes into a relatively small range, in order to eliminate the deviation of mining results caused by different sizes of numeric attributes. In this paper, the maximum-minimum normalization method is adopted. It is assumed that min Y and max Y are the minimum and maximum values of the attribute Y , respectively, and the equation of the maximum-minimum normalization is in Eq. (4):

v ′ = v - min Y max Y - min Y ⁢ ( 𝑛𝑒𝑤 ⁢ _ ⁢ max Y - 𝑛𝑒𝑤 ⁢ _ ⁢ min Y ) + 𝑛𝑒𝑤 ⁢ _ ⁢ min Y(4)

In the above equation, v is a value of the attribute Y , and v ∈ [ min Y , max Y ] ; is the value mapped to v , and v ′ ∈ [ 𝑛𝑒𝑤 ⁢ _ ⁢ min Y , 𝑛𝑒𝑤 ⁢ _ ⁢ max Y ] .

4.2.1 Mining of association rules

The study of association rules is helpful for discovering the potential association between different items in the data set, which is represented by rules. After deduction and accumulation, the relationship model is obtained.

Setting the itemset is I = { i 1 , i 2 , … ⁢ i ⁢ n } , TR is the set of things; 𝑇𝑅 ⊂ I , and TR is a set of attributes that has a value between 0 and 1. If X is a set of items, and X ⊆ 𝑇𝑅 , the form of association rules is expressed as X ⇒ Y , and X ⊆ I , Y ⊆ I , X ∩ Y = φ , in this way, X is called the condition (antecedent) of the rule, and Y is the result (consequent) of the rule [25].

Association rules have two important attributes: support degree and confidence degree. The association rules that meet the minimum support threshold and minimum trust threshold are called strong rules.

1) Support

The support of association rules is defined as the ratio of the number of transactions that contain both X and Y of things to the number of all transactions, which is described as the follows.

sup 𝑝𝑜𝑟𝑡 ( X ⇒ Y ) = P ( X ∪ Y )(5)

It reflects the probability that the item set X and Y appear simultaneously in the object set TR. Eliminate meaningless rules that appear less frequently by setting a minimum threshold (minsup, minimum Support).

2) Confidence

The association rule’s support degree (confidence) is defined as the ratio of the number of transactions containing object X and Y to the number of transactions containing X , which is described as the follows.

𝑐𝑜𝑛𝑓𝑖𝑑𝑒𝑛𝑐𝑒 ( X ⇒ Y ) = P ( Y | X ) = P ⁢ ( X , Y ) P ⁢ ( X )(6)

It reflects the conditional probability of occurrence of item set X and occurrence of item set Y in the data set. The association rules that meet the need are further screened out by setting the minimum confidence threshold (mincon, minimum Confidence).

When the support and confidence of the mining association rules are satisfied (minsup, mincon), the rule can be considered valid, otherwise it is an invalid rule [52].

3) Lift

To verify the reliability of the rule, the lifting degree, as a complementary indicator of confidence, is described as:

𝑙𝑖𝑓𝑡 = 𝑐𝑜𝑛𝑓𝑖𝑑𝑒𝑛𝑐𝑒 ⁢ ( X , Y ) P ⁢ ( Y )(7)

When lift value is 1, it means that X and Y are independent of each other, and X does not improve the possibility of Y appearing. When lift value is greater than 1, and X has a greater effect on the probability of Y appearing, indicating stronger correlation.

Data mining association rules mainly include the following two steps, the first step is to select all the item sets that meet the minsup minimum threshold, the frequent item sets, and the second step is to find all rules that meet the mincon minimum confidence threshold from the frequent item set.

Through the above process, the information in the security warning database of the work face is transformed into different association rules. By using these rules, the safety manager can predict the possibility of other factors based on known factors and realize the security warning.

The commonly used algorithms for data mining association rules include Apriori algorithm, multi-level association rule algorithm and multi-value attribute association rule algorithm [25].

As mentioned above, the safety hidden danger data of coal mining face is in multi-dimensional space, which is more suitable for multi-level rule mining algorithm. The multi-level association rule mining is to find the association rules at each concept level. It can adopt a top-down strategy, starting from the first concept level and descending to the lower and more specific concept level, and calculate the frequent item sets at each concept level until the frequent item sets can no longer be found [53, 54, 55], each level can adopt Apriori algorithm. Taking some data of the hidden danger record for the safety inspection of coal mining face as an example, this paper explains the mining process using the multi-level rule algorithm.

The Fig. 2 is the hierarchical concept map corresponding to the above Table 1 (hidden danger description dimensions and attribute categories of coal mining face).

OPEN IN VIEWER

The root node of the conceptual hierarchy tree is encoded with continuous integer encoding, and then the node of each tree is coded layer by layer. The code of the child node is the combination of the code of the parent node and the serial number in the subtree. The coded concept hierarchy tree is shown in Fig. 3.

After the hierarchical tree coding, the data of security hidden dangers are coded, such as the coded things database classified according to the professional dimension (denoted as pro-dim) specified in Table 1, as shown in Table 2.

OPEN IN VIEWER

5.1.1 Data acquisition

Taking the data of the safety monitoring system of the mining face in January 2018 as the research object, five parameters including gas, carbon monoxide, wind speed, temperature and daily output were selected based on the analysis of the geological structure and coal seam characteristics of the mine. The original monitoring data is shown in Table 3.

For the convenience and validity of data mining, the original data is first processed. The attributes of gas concentration, carbon monoxide, temperature, wind speed and daily output are U , V , W , X and Y , respectively. According to the actual situation of gas concentration, it can be divided into three groups (0–0.05), (0.05–0.10] and (0.10– ∼ ), which are labeled as U 1, U 2 and U 3 respectively. According to the actual situation of carbon monoxide, it can be divided into three groups (0–0.10], (0.10–0.15]) and (0.15– ∼ ), respectively labeled as V 1, V 2 and V 3. According to the actual temperature situation, it can be divided into three groups (0–14], (14–20], (20– ∼ ), which are marked as W 1, W 2 and W 3 respectively. According to the actual wind speed, it can be divided into three groups (0–1.20], (1.20–1.35], (1.35– ∼ ), respectively marked as X 1, X 2 and X 3. According to the actual situation of coal daily output, it can be divided into three groups (0–900], (900–1000) and (1000– ∼ ), which are labeled as Y 1, Y 2 and Y 3 respectively. The data preprocessing is shown in Table 4.

By applying Apriori algorithm, the maximum item set is found according to preset minsup = 25%, and strong rules are obtained according to mincon = 70%. Select the following 5 groups of strong rule analysis:

Table 5 is described as follows:

1) U ⁢ 1 ⇒ V ⁢ 2 , support = 0.429, confidence = 0.751, lift = 1 gas concentration in grade 1, the value interval is (0- 0.05]); carbon monoxide concentration in grade 2, the value interval is (0.10–0.15];

2) V ⁢ 2 ⇒ W ⁢ 2 , support = 0.429, confidence = 1, lift = 1.4, gas concentration at level 2, the value interval i (0.05–0.10]; and temperature at level 2, the value interval is (14–20];

3) V ⁢ 2 ⇒ X ⁢ 3 , support = 0.429, confidence = 0.867, lift = 1.01. The gas concentration is at level 2, the value interval is (0.05–0.10], and the wind speed is at level 3, the value interval is (1.35– ∼ ].

4) , support = 0.286, confidence = 1, lift = 3.5, gas concentration in grade 1, the value interval is (0–0.05]; carbon monoxide concentration in grade 2, the value interval is (0.10–0.15]; coal daily output in grade 1,the value interval is (0–900];

5) , support = 0.286, confidence = 0.667, lift = 2.3, gas concentration at level 1, the value interval is (0–0.05); carbon monoxide concentration at level 2, the value interval is (0.10–0.15]; temperature at level 2, the value interval is (14–20]; ventilation rate at level 3, the value interval is (1.35– ∼ ); coal daily output at level 1, the value interval is (0–900).

The above association rules are further explained and analyzed as follows.

1) Rules (1) gas, carbon monoxide correlation between them, the gas concentration in the first level is lower, that face ventilation should be better, but carbon monoxide concentrations in second grade, specification on the high side, can be interpreted as the coal mining process continuously produce carbon monoxide oxidation, but lift = 1, theoretically U 1, V 2 are independent of each other, not related with the association rules and mine production does not conform to actual situation. The reason is analyzed. The selected object is a low gas mine. The concentration of gas collected by the monitoring system is very low. Combined with the analysis of other rules, the correlation between gas concentration and carbon monoxide concentration is higher, and measures can be taken to prevent and control the increase of gas and carbon monoxide concentration simultaneously.

2) Rules (2) and (3) indicate the relationship between carbon monoxide concentration and the temperature and wind speed of the working face. If the temperature of the working face is too high, the carbon monoxide concentration will also be affected. Therefore, ventilation is better at the working face, reducing the temperature of the working face and reducing the generation and agglomeration of carbon monoxide.

3) Rules (4) and (5) indicate the relationship between gas concentration, carbon monoxide concentration, wind speed and coal daily output in the working face. The safety production of coal mine is required to be determined by wind, which means that sufficient air supply must be ensured in the production process of coal mine. The wind speed of the working face is at level 3 (1.35– ∼ ]), and the ventilation condition is good, while the daily output is at level 1 (0–900), indicating that the output matches the ventilation and conforms to the safety production regulations. At the same time, the ventilation condition is good, the temperature of the working face is low, the concentration of gas and carbon monoxide is low, and the working face is in a safe state.