Outburst prediction and influencing factors analysis based on Boruta-Apriori and BO-SVM algorithms

3.1.1 Boruta

The basic idea of Boruta [15] is a wrapper constructed by a random forest classifier. The random forest classification algorithm can run without adjusting the parameters, and gives the estimation of feature importance. Random forest is an integration method, which performs classification by voting on a plurality of unbiased weak classifiers such as decision trees. The trees are independently constructed on different bagging samples of the training set, and the classification accuracy caused by the random arrangement of attribute values between objects is taken as the importance measure of attributes. Z-score is used instead of average accuracy loss and standard deviation to measure the recognition performance of a single feature because it can take into account the volatility of accuracy loss between different decision trees in the forest. The wrapper function identifies relevant parameters by performing multiple runs of the provided classification factors that test the performance of different subsets of the input parameters. The RF function can be used without extensive parameter adjustments and returns an estimate of the importance of the feature (Z-score). Z-score can not be directly used to measure the importance of features because they do not necessarily follow a standard normal distribution in general.

The Boruta algorithm solves this problem by constructing the corresponding shadow feature for each single feature and then measuring the importance with Z-score. The way of iterative is mainly used to evaluate the importance of each feature. In the process of evaluation, it is necessary to set shadow features for the original features and compare the importance of the two in each iteration. If the importance of the original feature is significantly higher than the importance of the shadow feature, the original feature is considered as important. And if the importance of the original feature is significantly lower than the importance of the shadow feature, the original feature is then considered as unimportant. The original feature is one that needs to be selected, and the shadow feature is generated based on the original feature. And the generation rule is as follows: First, adding a random interference term into the original feature can generate an extended feature. Second, sample from the extended features to generate shadow features, which are generated during each iteration. The process of the algorithm is shown as follows:

The extended data set is obtained by adding random components (also called shadow attributes) of all attributes of the original data set, and then running a random forest classifier on the extended data set to obtain the Z score for evaluating the importance of features.

The influencing factors of coal and gas outburst contain redundant and irrelevant features, which reduces the classification accuracy. In order to better select the features closely related to outburst, feature selection is required. Boruta feature selection method based on RF is used to select the optimal feature set, RF is used as classifier, and the influence of single feature and multiple feature combinations on outburst classification is considered. In Boruta algorithm RF classification algorithm is used to guide the feature selection process, and the recognition performance of features is determined by measuring the loss of classification accuracy. Each single feature constructs a decision tree in the forest, and then the tree is used to recognize the target category independently and calculate the average and standard deviation of the classification accuracy loss.

The association rules [16] are to find the correlation between different items that appear in the same transaction, and to find out all subsets and their relevance of items or attributes that occur frequently in the transaction. Association rules mining are a rule-based unsupervised machine learning algorithm, and the main purpose is to explore the association between the internal structural features of data. In mathematics, the association rules are defined as: there exist two disjoint non-empty sets x, y. If x and y have a logical implication relationship such as x⟶y, x⟶y is called an association rule, where x is called the rule left term, y is called the right term. x and y are both item sets. The purpose of association rules mining is to export associations between items hidden in the project set and export the relations between them. The correlation between items can be described by association rules, which can intuitively reflect the potential connections between data item sets. The validity of rules is often measured by Support and Confidence. Support is an indicator of how often an item set appears in a data set, which measures the universality of the application of simple association rules and is defined as the probability that item sets x and y appear simultaneously. The calculation formula of support is shown in Equation (2). Confidence is a measure of the authenticity of a simple association rule, which is defined as the probability that an transaction containing item set X also includes item set Y, and is calculated as shown in Equation (3).

The rule A → B has a support of S, which means that S is the percentage of A ∪ B contained in transaction D, i.e. the joint probability P(A ∪ B), expressed as: Support ( A → B ) = P ( A ∪ B ) = ( Number of transactions including A and B / Total number of transactions ) * 100 %(2)

The rule A → B has confidence C, indicating that C contains an item set B as well as A, i.e. conditional probability p (A|B), expressed as: Confidence ( A → B ) = P ( B | A ) = ( Number of transactions including A and B / Number of transactions including A ) * 100 %(3)

Association rule mining is to find association rules whose support and confidence are greater than the minimum support (Minsup) and the minimum confidence (Minconf) given by the user respectively. When the confidence and support of a rule are greater than Minsup and Minconf respectively, the association rule is considered as valid, which is called a strong association rule. The mining of association rules can be divided into two steps. The first step is to find all the frequent item sets that meet the minimum support given by the user from the transaction data.the second step is to generate all the Association rules that meet the minimum confidence given by the user on the basis of the frequent item sets. The higher the support and self-confidence are, the stronger and the more effective the rules are. In the process of association rules mining, if the threshold of support is too small, a large number of weak association rules will be generated and the mining time will be prolonged; otherwise, valuable association rules may be overlooked. Therefore, it is necessary to carry out interactive mining by modifying the threshold to determine the optimal support threshold. The minimum support is the threshold to measure support, indicating the lowest importance of item sets in the statistical sense. Minimum confidence is the threshold to measure confidence, which indicates the lowest reliability of association rules. Strong association rules can satisfy both the minimum support threshold and the minimum confidence threshold.

The Apriori algorithm [27] clustering algorithm is a classic association rule mining algorithm. And the core idea is to find frequent item sets through candidate set generation and downward closed detection. That is, the “K item set” is searched by using the iterative method of layer-by-layer search and the “K-1 item set”. The specific steps are as follows:

let C be the candidate item set and L be the frequent item set. Through iteration, all frequent item sets in the transaction database with a support not lower than the threshold set by the user are retrieved, and all the items appearing in the transaction set are taken as candidate 1-item sets. Count the support of candidate item sets, delete all items with support below the threshold, and generate frequent 1-item sets L1.

Due to the difficulty of sampling data of coal and gas outbursts, there may be errors in sample data and class labels, and the formation of noise data affects the classification effect, which is not conducive to outburst diagnosis. Through the analysis of Apriori association rules, we can find the relevant rules in the sample data and the wrong sample data labels. Then we can correct them, and form a new data set to improve the classification accuracy, the relevant flow chart is shown in Fig. 1.

OPEN IN VIEWER

Fig. 1

Flowchart of Apriori algorithm.

Coal and gas outburst is affected by many factors which are difficult to determine or estimate quantitatively. Therefore, it remains uncertain which factors play a major role in the outburst process and what the mechanism of action is. Apriori association rules can not only mine the association between coal and gas outburst and index features, it can also can get effective coal and gas outburst sample data. However, the relation between coal and gas outburst impact factors and the sample data are complex, Apriori association rules cannot remove the redundancy and irrelevance of feature attributes and cannot be used as outburst recognition knowledge and rules. The noise data affect the classification effect. Feature selection is the process of selecting some of the most effective features from the original feature set to reduce the dimension of the dataset. It saves the physical explanatory meaning of the original attribute, and gives it an advantage in functional readability and interpretation. By deleting irrelevant, redundant or noisy attributes etc., we select a smaller subset of related attributes from the original attributes and then identify a plurality of feature selections. Usually an optimal feature subset that optimizes a certain evaluation criterion is selected from the feature set. Boruta is a fully correlated encapsulated feature selection method, which tries to find all the features that carry the information that can be used for prediction, instead of finding the feature subset that produces the smallest error on the classifier like most traditional encapsulation algorithms. Boruta will find all relevant features regardless of the correlation between features and decision variables, which makes it very suitable for determining the best feature subset. Boruta algorithm can find all the features related to the category in the candidate features, so as to determine the number of features directly. And through Boruta feature selection, the correlation characteristics between coal and gas outburst and influencing factors can be found in order to determine the relationship between the main influencing factors of coal and gas outburst.

Boruta can find all the relevant features, which can solve the problem that Apriori algorithm cannot automatically give the best feature subset. Therefore, we consider building a hybrid model that integrates Apriori and Boruta. In the hybrid model, Apriori method is used to determine the correlation between coal and gas outburst sample data and feature attributes, and then it will find effective feature attributes and sample data as strong correlation rules for coal and gas outburst judgment. The Boruta encapsulation algorithm can remove the irrelevant features in the rules, and obtain the feature subset with higher classification performance, which improves the running efficiency and accuracy, so the characteristics of the two methods are well complementary. Therefore, this section proposes a hybrid model based on Apriori association rules mining and Boruta feature selection, and designs an efficient feature selection algorithm from the perspective of feature and sample selection to select a group of feature datasets with small redundancy and high-quality. This is helpful to find the significant factors affecting coal and gas outburst and improve the classification accuracy. The corresponding flowchart is shown in Fig. 2.

OPEN IN VIEWER

Fig. 2

Boruta-Apriori based feature selection and association analysis.

3.2.1 The idea of BO algorithm

The basic idea of BO algorithm [29] is to estimate the posterior distribution of the objective function f using the Bayesian Equation (4) based on the data, and then select the super-parameter combination of the next sample according to the distribution. BO makes full use of the information of the previous sampling point, and the working mode of optimization is to learn the shape of the objective function and find the parameters that make the result of Equation (5) lift to the global maximum. Bayesian optimization learns the shape of the objective function by assuming the acquisition function according to the prior distribution. Each time a new sampling point is used to test the objective function, it uses this information to update the prior distribution of the objective function, and then the algorithm tests the most probable point given by the posterior distribution. BO uses prior knowledge to approximate the posterior distribution of the unknown objective function f, so as to adjust the super-parameters. BO is extremely powerful in the case of unknown objective function and high computational complexity and has the advantages of combining local optimization with global optimization. p ( f | D 1 : t ) = p ( D 1 : t | f ) p ( f ) p ( D 1 : t )(4) where f represents an unknown objective function, D(1 : t) = { x₁, y₁, x₂, y₂, … .(x _t , y _t ) }} represents an observed set, x_t represents a decision variable, and y_t = f(x _t ) + ɛ _t represents an observed value, ɛ_t represents an observation error, p(f|D_1:t) represents a likelihood distribution, because the observed value has errors and noise. p(f) represents the prior probability distribution of f and the assumption on the state of the objective function, p(D_1:t) represents the marginal likelihood function of the marginalized f, p(D_1:t|f) represents the posterior probability distribution of f and the confidence degree of the unknown objective function after the prior is corrected via the observed data. x * = argmin x ɛ β ∈ R d f ( x )(5) where, x^* represents the dimension decision variable, β represents the decision space, x represents the parameter, f represents the objective function, and f(x) represents the quality of the result.

The core of the BO framework is a probability proxy model and an acquisition function (AC). Bayesian optimization uses the probability model to represent the complex black box function. The prior knowledge of the target to be optimized is introduced into the probability model to make the model more accurate, the following are the steps as shown in algorithm 1.

The probabilistic proxy model is used to represent the unknown objective function, starting from the assumption of a priori, by iteratively increasing the amount of information and modifying the priori, we can get more accurate results probabilistic surrogate model can be divided into parametric model and nonparametric model according to whether the number of parameters is fixed. Probability proxy model comprises a prior probability model and an observation model. The prior probability is p(f). The observation model describes the mechanism of observation data generation, i.e. likelihood distribution p(D_1:t|f). Updating the probability proxy model means that a posterior probability distribution p(f|D_1:t) containing more data information is obtained according to the Equation (4).

When using Bayesian method to solve specific problems, it is necessary to select the appropriate probability model to represent the black box function according to the problem background and expert knowledge. It is even more important to select the appropriate probability proxy model for Bayesian optimization than to select the appropriate acquisition function. In this paper, according to the characteristics of coal and gas outburst sample data, we choose the convenient and powerful Gaussian process optimization modeling algorithm to solve the problem. The process of finding optimal parameters is a Gaussian process. In the process of Bayesian parameter adjustment, we assume that a group of super parameters combination is x = x1, x2,..., xn (xn represents the value of a super parameter), and there is a functional relationship between this group of super parameters and the loss function that we need to optimize finally. The final evaluation result is y. we assume that f (x), y = f (x) is the best x through which we can get the optimal y. This is a graph of a Gaussian process, as shown in Fig. 3.

OPEN IN VIEWER

Fig. 3

Curve of Gaussian process.

In order to satisfy the behavior of black box function effectively and reduce unnecessary sampling, Bayesian optimization uses active selection strategy to determine the next step, compared with the random jump or neighborhood search strategy of the model free optimization method, the active selection strategy utilizes historical information and uncertainty. Qualitatively, by maximizing the acquisition function constructed according to the posterior distribution of the model, it can effectively balance the width search (explore the uncertain area to obtain more undecided information). In order to reduce unnecessary evaluation of objective function, the relationship between knowledge search and deep search is analyzed. The so-called acquisition function is from the input space, observation space and super parameter space. The function is defined by the observed data set D1, the posterior distribution is obtained by x _t : t is constructed, and the next evaluation point x_t+1: t is selected by maximizing the posterior distribution Equation (6):

x t + 1 = max x ɛ X a t ( x ; D 1 : t )(6)

The acquisition function is constructed according to the posterior probability distribution, and the next most potential evaluation point is selected by maximizing the acquisition function, and the effective acquisition function can ensure that the selected evaluation point sequence of the total loss l is minimized Equation (7), l = | y * - y t |(7)

Where, y^* is the optimal solution, y _t is the target function value.

For BO algorithm, once it finds a local maximum or minimum, it keeps sampling in this area, so it can easily fall into a local maximum or minimum. To alleviate this problem, Bayesian optimization can find out a balance between exploration and exploitation. Exploration is the acquisition of sampling points in areas that have not yet been sampled. Exploitation is to sample in the area where the global maximum value is most likely to occur according to the posterior distribution. The next selected point (x) should have a relatively large mean (exploitation) and a higher variance (exploration). These two standards are contradictory, so how to find a balance between them is the main challenge for function extraction. The choice of acquisition function may affect the effect of super parameter optimization model, there are three common acquisition functions: EI, PI and UCB.

The purpose of PI method is to maximize the probability that the new sampling can improve the maximum value. Suppose the maximum value is f(x⁺), then the acquisition function is as following Equation (8): PI ( x ) = p ( f ( x ) ⩾ f ( x + ) ) = δ ( μ ( x ) - f ( x + ) σ ( x ) )(8)

Where μ(x) represents exploitation, σ(x) represents exploration, δ (·) represents the normal cumulative distribution function, which is also called MPI (maximum probability of improvement), or PI algorithm. This will prevent a very slight increase near f(x⁺). PI is the first acquisition function proposed,different from the other ones, it can maximize the probability that the next sampling point can produce non-zero improvement. This function will tend to exploit rather than explore, so it is likely to converge close to f(x⁺). To solve this problem, we can add a trade-off coefficient ɛ in Equation (9).

PI ( x ) = p ( f ( x ) ⩾ f ( x + ) + ɛ ) = δ ( μ ( x ) - f ( x + ) - ɛ σ ( x ) )(9)

This will prevent a very slight increase near f(x⁺). Here, the coefficient ξ can be defined by itself, which can be adjusted dynamically to control whether it is biased towards exploration or exploitation. It is worth noting that Pi uses a greedy search strategy, so it is more like local search to some extent.

Using EI as acquisition function is a good choice to balance exploration or exploitation. When exploring, we should choose those points with large variance, while when exploiting, we should give priority to those points with large mean value, and the introduction of formulas are as following:

E ( I ) = ∫ I = 0 I = ∞ I 1 2 π σ ( x ) exp ( ( u ( x ) - f ( x + ) - I ) 2 σ 2 ( x ) ) dI = σ ( x ) [ u ( x ) - f ( x + ) σ ( x ) φ ( u ( x ) - f ( x + ) σ ( x ) ) + φ ( u ( x ) - f ( x + ) σ ( x ) ) ](10)

EI ( x ) = { ( u ( x ) - f ( x + ) ) φ ( Z ) + σ ( x ) φ ( Z ) , if σ ( x ) > 0 0 , if σ ( x ) = 0(11) Z = u ( x ) - f ( x + ) σ ( x )(12)

In the formulas, the parameter I can usually be fixed to 0.01, it is worth noting that the experiment shows that the dynamic parameter I cannot improve the effect, even it is worse sometimes.

In addition to EI and PI, UCB is a simpler idea to directly compare the maximum values in the confidence space. This method compares the maximum value within the confidence interval. UCB is typical described in terms of maximizing f rather than minimizing f, in this paper, according to the characteristics of coal and gas outburst sample data,we use UCB as AC,and in the context of minimization, AC takes the following Equations (13–14): UCB ( x ) = μ ( x ) + k σ ( x )(13) LCB ( x ) = μ ( x ) - k σ ( x )(14)

Where k is a trade-off parameter, although it looks simple, the actual effect is surprisingly good, the following Fig. 4 is curve of Gaussian process.

OPEN IN VIEWER

Fig. 4

Curve of Gaussian process.

BO framework is an iterative process, mainly including three steps:

First, according to the maximization of the acquisition function, we select the next most potential point.

Second, according to the selected evaluation point x _t , obtain the evaluation objective function value y(t) = f(x _t  +  ɛ ).

Third, the new input observation values are matched to {x _t , y _t }, add to historical observation set D1.

SVM [18, 23] is a supervised learning algorithm, which is based on VC theory and structural risk minimization criterion in statistical learning theory. Its basic idea is to construct an optimal hyperplane in the sample input space or feature space, so that the distance between the hyperplane and the two types of sample sets can be maximized, and the best generalization ability can be obtained. In this paper, according to the characteristics of the sample data, it is mainly nonlinear support vector machine classification. The learning process is to find the optimal separation hyperplane to maximize the segmentation of the two classes. The principle is as follows: for a binary classification problem, given the training sample set ( x i , y i ) i = 1 m , the x_i ∈ Rⁿ is input information, y_i∈ { ± 1 } is the output target information, and i is the number of training sample. The classification hyperplane is sought in the new feature space, and assuming that the separated hyperplane is linear, the linear estimation function is defined as Equation (15). f ( x ) = w T ∅ ( x ) + b = 0(15)

In the above formula, ∅(X) is a non-linear mapping function from the input space x to the high-dimensional feature space, w is the weight vector, b is the bias value, and the classification hyperplane should meet the following constraints Equation (16): y i f ( x i ) = y i ( w * x i + b ) ⩾ 1 , i = 1 , 2 , … , m(16)

For each sample x_i, a non-negative relaxation variable ɛ_i needs to be introduced to measure the extent to which w and b violate the constraints. c ∑ i = 1 m ɛ i is the loss function, c is a penalty parameter. Minimize 1 2 ∥ w ∥ 2 + c ∑ i = 1 m ɛ i i = 1 , 2 , 3 … m subject to { y i ( w * x + b ) ⪢ 1 - ɛ i ɛ i ⪢ 0 }(17)

By introducing lagrange multiplication α_i, and α _i ɛ [0, c], the above optimization problem can be transformed into a quadratic programming optimization problem, namely: Maximize L ( α ) = ∑ i = 1 m α i - 1 2 ∑ j = 1 m α i α j y i y j ( x i , x j ) subject to ∑ i = 1 m α i y i = 0 , α i ⩾ 0 , i = 1 , 2 , … , m(18)

Such a linear decision function can be defined by the following dual optimization problem: f ( x ) = sign ( ∑ ij = 1 m α i y i ( x i , y i ) + b )(19)

SVM has a better generalization ability to test samples that have never been seen before. Its kernel method makes the linear non-separable problem linearly separable in high dimensional space by mapping the original data to high dimensional space. At the same time, it avoids solving the problem in high dimensional space. The kernel method represented by SVM has achieved great success in solving nonlinear pattern analysis problems. The nonlinear mapping function can map the original data to a linearly separable high dimensional feature space. The nonlinear decision function is as follows: f ( x ) = sign ( ∑ ij = 1 m α i y i k ( x i , y i ) + b )(20) where k(x _i , y _i ) is the kernel function and k(x _i , y _i ) = ∅(x _i ) ∅(x _j ),

The selection of kernel function and the determination of its parameters have a significant impact on the classification performance, and there is no perfect theoretical basis so far. The parameter combination of kernel function includes regularization factor and kernel parameters. The regularization factor is structural risk and sample error. The value of kernel function bandwidth is related to the range and width of input data space, and there are different methods to optimize these parameters. The BO algorithm is simple, globally optimal and highly efficient, and its performance is stable and reliable. It can be used for optimization problem of kernel. The BO algorithm is used to optimize the basic parameters to obtain the best parameter combination. And its effectiveness is verified by function fitting and relevant parameter optimization in coal and gas outburst prediction. Therefore, this paper proposes a support vector machine parameter optimization algorithm based on BO and 10-fold cross verification.

Based on SVM framework and existing BO algorithm, this paper proposes an improved algorithm, the super parameters of SVM are optimized by BO algorithm. This method is highly efficient. The improved Markov chain Monte Carlo method is used to model the parameters algorithm to accelerate training. The training of Gaussian process model is to calculate the super parameter of Gaussian surrogate model. The UCB sampling method is used as the acquisition function to obtain the next sampling point. The loss value is calculated and then merged into the historical observation set. This process iterates until set of super parameters with good performance is obtained. This method can be used in different super parameter spaces Super parameter optimization is carried out under SVM framework. Using coal and gas outbursts dataset, the test set is used to test the performance of the algorithm, which shows the super performance of the proposed method. The parameters are better than those obtained by TPE, GPEI, MCMC and other similar optimization algorithms, the performance are more stable. The flow chart is shown in Fig. 5.

OPEN IN VIEWER

Fig. 5

SVM classifier based on BO algorithm.

4.2.1 Comparison of different feature selection methods

From Tables 1–3, we can see the comparison of feature number and performance indexes of different feature selection methods on different classifiers. In the feature selection of coal and gas outbursts data, the effect of feature subset formed by Boruta is the highest with the three classifiers compared to other feature data sets. The accuracy in SVM, NB and KNN are 92%, 87% and 83% respectively. The highest accuracy obtained by feature selection method CFS on the three different classifiers is 83%, 86% and 81% respectively, which is close to the Boruta feature selection method, but the number of features is high. The feature number of feature selection method FCBF is only 2, but the accuracy on different classifiers is 83%, 79% and 79% respectively, which are lower than Boruta and CFS.

Boruta is a fully correlated encapsulated feature selection method, which tries to find all the features that carry the information that can be used for prediction, instead of finding the feature subset that produces the smallest error on the classifier like most traditional encapsulation algorithms. Boruta will find all relevant features regardless of the correlation between features and decision variables, which makes it very suitable for determining the best feature subset. Boruta algorithm can find all the features related to the category in the candidate features, so as to determine the number of features directly. Through the above results, we can conclude that the feature selection Boruta can achieve the goal, by screening out the optimal index characteristics, so we select the Boruta feature selection method to obtain the highly relation features from the outbursts influencing factors and the prediction effects are obviously improved.

Since Apriori requires that the attributes of the sample to be non-numerical, the six attribute indicators should have discrete rules according to expert experience and relevant references. The continuous attribute values in the database are made discrete and are encoded, and the preprocessing and partial coding results are shown in Table 4. Rule extraction is carried out on the data samples after feature selection through Apriori association rules. Table 4 obtains the performance comparison of different indexes by setting different confidences and supports. Through association rule mining, we can easily obtain outstanding association rules: in the experiment, we set the minimum confidence as 10% and the minimum support as 60%, and the obtained results are shown in Table 5. From the above mined association rules, it can be seen that the initial velocity of gas output, the initial velocity of gas emission and the complexity of fault structure are closely related to coal and gas outburst. Considering the interaction, the influences of the main factors on coal and gas outburst can also be obtained. Through the above analysis of the relevant association rules, the influence degree and association degree between each influencing factor and prominence can be accurately judged. This not only helps the decision-maker to judge the probability that a certain influencing factor and another influencing factor may lead to prominence, and it can also help them understand the acceptance degree of a certain influencing factor to other influencing factors. Therefore, it can not only help improve the accuracy of the selection of influencing factors, and it can also carry out burst prediction activities in a targeted way and further improve the accuracy and efficiency of prediction. In order to verify that effectiveness of our proposed model, our experimental steps are as follows. First, we tested a single model; in other words, we created a classifier-based single model (Apriori and Boruta) to compare the predictability of fluctuations between individual models. Secondly, we construct a hybrid model combining Apriori and Boruta. This paper aims to evaluate the superiority of the Apriori + Boruta Model over the existing models.

Tables 6–8 show the comparison of classification performance of different classifiers combined with different feature selection methods, it can be seen from the table that the classification accuracy of the individual association rule mining algorithm is 88% with SVM,85% with KNN and 75% with NB respectively, and the feature size is 6. The classification accuracy of the individual Boruta algorithm is 92% with SVM,87% with KNN and 82% with NB respectively, and the feature size is 3, the dimension is lower than association rule mining algorithm. This is because the rules extracted by the independent association rule algorithm will produce redundancy, so the recognition effect is not high. The simple feature selection method only considers the effect of feature attribute comparison on the prominent target, but it ignores the noise data in the sample data, and the classification effect is not high. However, the combination of Apriori association rules mining and Boruta can significantly improve the classification accuracy compared with the feature selection and association rules mining alone, and the overall classification accuracy can reach 93% with SVM, 92% with KNN, and 83% with NB. The classification accuracy of proposed method is improved by 2 percent compared with the simple Apriori association rules mining and Boruta, and the feature size is only 3. The reason is that the combination of association rules mining and feature selection can make full use of the advantages of feature selection and association rules methods, it can remove redundant features and noise sample data of coal and gas outburst and improves the effectiveness of coal and gas outburst prediction feature selection, so it further verifies the universality of the method of combining Apriori association rules mining and Boruta feature selection.

In order to verify the effectiveness of acquisition functions in BO algorithm in the parameters optimization of SVM, we evaluate UCB, EI and PI functions, from Table 9, we can see that different AC can obtain different effects in accuracy and efficiency, the accuracy are 93%, 90%, 93% respectively, among them the accuracy of UCB and EI are close, which are 93%, Pi is poor; For runtime, we also seen that UCB is highest, is 4.74 s, which is lower than EI and PI. Comprehensive results conclude that UCB method is very effective in BO algorithm, and it can obtain the tradeoff between exploit and explore, so we select it as AC.

At the same time we verify the effectiveness of BO algorithm in the parameters optimization of SVM, we evaluate and compare with other optimization algorithms such as GS, RS, GA and PSO in optimizing the parameters of SVM. From the simulation results in Table 10, we can see that the BO algorithm has good accuracy and convergence speed, when its iterations is 14, the accuracy reach the highest value, and remain unchanging, and it is superior to other optimization algorithms in efficiency and accuracy. Among all kinds of indexes of prediction effect, BO algorithm can achieve the best results, and the prediction accuracy obtained by different algorithms are all 92% expect that RS. In terms of execution efficiency, BO algorithm performs the best, the runtime is 2 seconds, the runtime of GS, RS, PSO and GA algorithm are 45,36,33 and 33 seconds, respectively, which are far more than that of the BO algorithm. Based on above the statistical results, the BO algorithm is found to be the most effective among these methods.

The reason is that the parameter adjustment of BO algorithm adopts Gaussian process, considers the previous parameter information, constantly updates the prior, and adopts the active optimization strategy, so it omits many useless sampling point operations. On the premise of selecting the correct probability agent model and acquisition function, the BO algorithm finds a group of slightly better parameters than others after only a few iterations are performed, and considers the computing cost of the dataset in the parameters of the algorithm, so the optimization efficiency of the model parameters is significantly improved, then it is very suitable for finding a group of very good parameters in a short time. Moreover, the BO algorithm uses the information feedback mechanism of the individual to search, and realizes the balance between the local development ability and the global searching ability of the population, so it can find the best parameters combination. For the same dataset, the other algorithms can converge after more iterations are performed, with more iterations and slow running speed. The GS and RS algorithms are very easy to lead to dimension explosion when there are many parameters, and the prior parameter information is not considered in parameter optimization, so it is easy to get local optimization. PSO and GA algorithms are local or global optimization, and their optimization effects are very limited, which are lower than the optimization effect and execution efficiency of BO algorithm, it is seen that BO algorithm has more potential compared to the other optimization algorithms, so we choose BO algorithm to optimize the parameters of SVM.

In order to verify the effectiveness of our proposed model, RF is selected as the classifier to select the optimal feature subset. SVM is used as the classifier, and the prediction results of the method are compared with other prediction-based models mentioned in the literature. From Table 11, we can see that the accuracy, feature dimension of the model are better than those of the feature selection models in the current literature. The accuracy and precision of the model proposed in this paper are 93% and 96%, respectively, and the feature dimension is 3. At the same time, we can also see that different feature selection and classifiers produce different effects, and proves the effectiveness of the combination of association rule mining and feature selection gas outbursts prediction.

We should choose the appropriate method according to the characteristics of data distribution. From Table 11, we can see the comparison of experimental results of different feature selection. we can see the comparative analysis of this method and the feature selection methods in the literature, including FCBF, MIFS, NMIFS, MRMR, MIFS-ND. The results show that the classification performance of the proposed algorithm is higher than those in the literature. By verifying the feature selection results in the classifier, we find that the accuracy of these feature selection methods is between 79% and 83%. Compared with FCBF, MIFS, NMIFS, MRMR and MIFS-ND, the classification accuracy of this method is improved by 0.5-1 on average, but the feature dimension is less than these methods. This shows that the feature selection method in this paper can improve the prediction performance of each classifier. The main reason is that the feature selection methods of FCBF, MIFS, NMIFS, MRMR and MIFS-ND in judging the correlation between features and class markers and redundancy between features, and overestimate the importance of some features, while ignoring the joint effect between combined features and incomplete measurement of correlation information between features, resulting in inaccurate classification. The FCBF algorithm eliminates the redundancy feature one by one through the approximate Markov blanket, and the MRMR algorithm has relatively high performance and stable performance due to the addition of redundancy elimination function. The MIFS and MIFS-ND algorithms make the selected set more compact and efficient by adding redundancy elimination function, but it scatters the interdependent combinations. However, as previously described, FCBF and MRMR algorithms treat the correlation as redundancy and ignore the role of interaction in recognition. In proposed hybrid model, Apriori is used to determine the correlation between coal and gas outburst sample data and feature attributes, and then it will find effective feature attributes and sample data as strong correlation rules for coal and gas outburst judgment. Apriori has the following defects: the extracted association rules have universal redundancy, many association rules can provide the same information, resulting in the efficiency and accuracy of classifier greatly reduced, so it is necessary to carry out effective feature selection on the rules obtained by Apriori. The Boruta encapsulation algorithm can remove the irrelevant features in the rules, and obtain the feature subset with higher classification performance, which improves the running efficiency and accuracy, so the characteristics of the two methods are well complementary. This paper makes full use of the advantages of association rule algorithm in mining effective sample data and feature selection to select all relevant features. It considers various relationships between the influencing factors of coal and gas outbursts and provides a new idea for the feature selection of coal and gas outbursts. In the selection of classification model and optimization algorithm, we make full of the advantages of SVM classifier’s adaptability according to the distribution characteristics of small sample data of coal and gas outbursts. And we also consider Bayesian optimization’s execution efficiency, and good local and global optimization ability. High prediction accuracy on coal and gas outburst data set is obtained by this combined comprehensive model. Therefore, this section proposes a hybrid model based on Apriori association rules mining and Boruta and designs an efficient feature selection algorithm from the perspective of feature and sample selection to select a group of feature data sets with small redundancy and high-quality feature data sets. This is helpful to find the significant factors affecting coal and gas outburst and improve the classification accuracy. Therefore, the classification accuracy of these methods in the literature is not very high on the data set of this paper.

At the same time we also compare their performances in terms of computation complexity, assuming that the number of samples in a given dataset is n and the feature dimension is m, the number of candidate feature subset is k, the time complexity of Boruta feature selection is O(mn), the time complexity of Apriori is O(n⁴), Our proposed feature selection is composed of two parts: Boruta and Apriori, in conclusion, the time complexity of the our proposed feature selection is O(mn) + O(n⁴);For the time complexity of optimization algorithm and classifier are O(n) and O(m²n²), so the complexity of proposed methods is O(mn) + O(n⁴) + O(n) + O(m²n²). Therefore, our model is completely suitable for the needs of coal and gas outburst prediction. In a word, compared with the comprehensive model of coal and gas outbursts feature extraction and classification model in the literature, the model proposed in this paper has better recognition effect.