Big data attribute selection method in distributed network fault diagnosis database

Abstract

The selection of big data attributes plays a positive role in the development of the network. At present, the attribute selection for big data is completed by detecting the attribute of data, which can not guarantee the accuracy of the selection. In this paper, a big data attribute selection method based on support vector machine (SVM) is proposed for distributed network fault diagnosis database. The method is used to mine big data in the distributed network fault diagnosis database, and calculate its attribute weights according to which complete attribute classification, so as to complete the selection if big data attributes. Experiments show that the proposed method improves the efficiency of big data attribute selection, and has certain practical value.

Keywords

Distributed network fault diagnosis database big data attribute selection support vector machine

1 Introduction

The development of computer and Internet technology has brought convenience to people’s work and life, and the rapid development of the network has also brought about the problem of network security [17, 18]. Distributed network fault diagnosis is an effect means to prevent virus intrusion [15, 21]. The speed and accuracy of the selection of big data attribute in the database directly affect the effect of the distributed network fault diagnosis [5, 6]. There are many research methods for the real-time nature of distributed network fault diagnosis. Related researchers have selected large data attribute selection methods for distributed network fault diagnosis databases based on rough sets [19]. This method converts a distributed network fault diagnosis database into a sequence of objects, randomly extracts data objects from the database without returning, and determines whether the current candidate reduction can be used to distinguish between objects and objects in the current sequence of objects [20, 26]. If can be distinguished, the object is placed in the current object sequence, otherwise the appropriate attribute is added to the candidate reduction until all the data objects are distinguished [1, 4]. The method of attribute selection is to select the current candidate reduction as a big data reduction in the distributed network fault diagnosis database to complete its attribute selection. Although the speed is fast, the accuracy needs to be improved. This method has become the focus of discussion among relevant experts and scholars, and its research has gradually entered the scope of experts and scholars [28, 30]. With the deepening of the research content, lots of research results have been produced [23, 27].

The related literature proposes a method for selecting big data attributes in a distributed network fault diagnosis database based on local time-scale feature extraction of a decision tree. By constructing a distributed network fault diagnosis data transmission model, a big data attribute selection decision tree model database is established [22]. Then the paper uses the signal processing method to complete the big data attribute extraction in the database, according to which the big data attribute selection in distributed network fault diagnosis database can be completed. The drawback of this method is that the selection of attributes is quit slow. Literature [16] proposed a big data attribute selection method for distributed network fault diagnosis database based on IPv6. By analyzing the bottleneck in the process of distributed network fault diagnosis, a distributed network device driver based on NAPI (New Application Program Interface) optimized for multi-core processor is proposed. Then, it analyzes the necessity of IP fragmentation reorganization in IPv6 network behavior management system, the business process of big data attribute selection in distributed network fault diagnosis database, and the characteristics of IPv6 session flow, and proposes the session flow maintenance scheme based on IPv6 five tuples. The method searches the session flow to complete the big data attribute selection in the distributed network fault diagnosis database. However, this method has a small range of applications, and in the case of a large distributed network fault diagnosis database, it can increase the load for big data attribute selection. In the literature [2], a big data attribute selection method based on depth representation of the distributed network fault diagnosis database is proposed. First, the artificial neural network algorithm is analyzed and implemented. On the basis of it, the big data attribute is detected by the feature depth of the learning. And then DRBM (Dynamic Random Access Memory) expansion structure is adopted to construct big data attribute detection model, thus the big data attribute selection in the distributed network fault diagnosis database is completed. But this method is complicated and difficult to generalize.

Aiming at the above problems, a method for selecting big data attributes of distributed network fault diagnosis database based on support vector machine is proposed. First, the decision tree method is used to mine and calculate the big data in the distributed network fault diagnosis database, and the attribute of the big data is obtained. Then, the big data attribute classification is completed through a subset of assessment, stop criteria and result validity verification generated by big data attribute subset in distributed network fault diagnosis database. According to the similarity degree of data attribute space, the calculation method of attributes similarity and weight are obtained. The loss function is analyzed to improve the feature selection algorithm of big data attribute and calculate the weight of big data attribute. The gradient rise method is used to solve the saddle point, and furthermore to realize the large data attribute selection in the distributed network fault diagnosis database. The proposed method can effectively improve the accuracy of big data attribute selection in distributed network fault diagnosis database, reduce the calculation process, energy and time consumption, and has good practical value.

2 Research on big data attribute selection method in distributed network fault diagnosis database

Big data selection is the basis of classification, mining and processing of big data in distributed network fault diagnosis database [3]. With the development of application of big data mining processing technology [6, 9], the problem of big data attribute selection in distributed network fault diagnosis database has gradually become the focus of relevant experts and scholars [13].

2.1 Collection and analysis of big data attribute selection method in distributed network fault diagnosis database

2.1.1 Collection of big data attribute selection method in distributed network fault diagnosis database

In order to realize the selection of big data attributes in the distributed network fault diagnosis database, it is necessary to mine data in the distributed network fault diagnosis database and then calculate its attributes to realize big data attribute analysis.

There are many methods of data mining. Among them, the decision tree method is the most commonly used method [14, 29]. By using the tree structure to show the result of data mining [7], the method is simple and intuitive, and therefore it is suitable for this paper [25]. The specific process is shown in Fig. 1.

Fig. 1

Dig data attribute analysis process in distributed network fault diagnosis database.

U is the big data set in distributed network fault diagnosis database, F₁ and F₂ are two big data attributes on node N of the decision tree. The information gain of F₁ is greater than that of F₂, so the big data attribute F₁ on node N is selected as a classification attribute.

Assume E₁ and E₂ are the information entropy of F₁ and F₂ respectively, we get $\begin{matrix} gain (F_{1}) \geq gain (F_{2}) \Leftrightarrow I (p, n) - \\ E (F_{1}) \geq I (p, n) - E (F_{2}) \Leftrightarrow E (F_{1}) \leq \\ E (F_{2}) \Leftrightarrow E_{1} \leq E_{2} \end{matrix}$ (1) wherein, gain (F₁), gain (F₂) are the increased coefficients of F₁ and F₂.

Let M be a line recording the reduction of big data in distributed network fault diagnosis database, which belongs to the range of attribute j on node N. When the record is not reduced, the information entropy of the node attribute can be described as $\begin{matrix} E = \sum_{i = 1}^{m} \frac{p_{i} + n_{i}}{p + n} (\begin{matrix} - \frac{p_{i}}{p_{i} + n_{i}} log (\frac{p_{i}}{p_{i} + n_{i}}) - \\ \frac{n_{i}}{p_{i} + n_{i}} log (\frac{n_{i}}{p_{i} + n_{i}}) \end{matrix}) \\ = \frac{ɛ}{p + n} \end{matrix}$ (2)

In Equation (2), m represents the range count of big data attribute in given distributed network fault diagnosis database, n_i and p_i are the information entropy of big data attribute i value segment in the database [8], and ɛ represents the value segment of a big data attribute in a given database [24].

By reducing the centralized record of the dig data in distributed network fault diagnosis database, we can get attribute information entropy of the big data attribute node in new database as following [11]: $\begin{matrix} E^{'} = - \frac{1}{p + n - 1} {\underset{i \neq j}{\sum_{i = 1}}}^{m} (p_{i} log (\frac{p_{i}}{p_{i} + n_{i}}) + n_{i} log (\frac{n_{i}}{p_{i} + n_{i}})) + \\ - \frac{1}{p + n - 1} (- (p_{j} - 1) log (\frac{p_{j} - 1}{p_{j} + n_{j} - 1}) - n_{j} log (\frac{n_{j}}{p_{j} + n_{j} - 1})) \end{matrix}$ (3)

According to Equations (2) and (3), we obtain: $\begin{matrix} ɛ = - \sum_{i = 1}^{m} (p_{i} log (\frac{p_{i}}{p_{i} + n_{i}}) + n_{i} log (\frac{n_{i}}{p_{i} + n_{i}})), \\ ɛ \geq 0 \end{matrix}$ (4) $\begin{matrix} A = p_{j} log (\frac{p_{j}}{p_{j} + n_{j}}) + n_{j} log (\frac{n_{j}}{p_{j} + n_{j}}) - \\ (p_{j} - 1) log (\frac{p_{j} - 1}{p_{j} + n_{j} - 1}) - n_{j} log (\frac{n_{j}}{p_{j} + n_{j} - 1}) \end{matrix}$ (5)

A represents a finite nonempty set of all the attributes of the big data in the database. Let x = p_j, y = n_j, and subtract Equation (2) from Equation (3), we get Equation (6) $\begin{matrix} Δ E = E^{'} - E = \frac{ɛ + A}{p + n - 1} - \frac{ɛ}{p + n} = \frac{A}{p + n - 1} \\ + (\frac{ɛ}{p + n - 1} - \frac{ɛ}{p + n}) = \frac{A}{p + n - 1} \\ + \frac{ɛ}{(p + n - 1) (p + n)} \end{matrix}$ (6) wherein, A (x, y) represents a function of x, y, the big data attribute variables, in the database.

Depending on the nature of A (x, y), if y = 0, we conclude that A (x, 0) =0, x ∈ [1, + ∞], and: $\begin{matrix} A (1, y) & = y log y - (1 + y) log (1 + y) \\ = - [y log (1 + y) - y log y + log (1 + y)] \\ = - [y log (\frac{1 + y}{y}) + log (1 + y)] \end{matrix}$ (7)

In the above formula, A (1, y) ≥ - ɛ - log(1 + y) since $1 < y log (\frac{1 + y}{y}) < ɛ$ .

Let $\begin{matrix} A (x, y) = x log x + (x + y - 1) log (x + y - 1) \\ - (x + y) log (x + y) - (x - 1) log (x - 1) \end{matrix}$ , then: $A (x, y) \geq 1 - log ɛ - log (x + y - 1)$ (8)

According to ɛ ≤ p + n, and substituting Equations (7) and (8) into Equation (6), we calculate the maximum and minimum value of ΔE as follows:

The minimum ΔE can be calculated if A = 0 and ɛ = p + n $Δ E_{min} = \frac{1 - log ɛ - log (x + y - 1)}{p + n - 1}$ (9)

The maximum ΔE can be calculated if A = 1 - log ɛ - log(x + y - 1) and ɛ ≠ 0 $Δ E_{max} = \frac{1}{p + n - 1}$ (10)

Through the above method, the attribute collection of big data in distributed network database based on decision tree is completed.

2.1.2 Analysis of big data attribute selection method in distributed network fault diagnosis database

The attribute selection of big data in the distributed network fault database is to select m attributes subset from N big data attributes, while the m subset can accurately distinguish each data object in the database as well as the original N attributes. In the general distributed network fault diagnosis database [12], the analysis of big data attributes can be divided into four steps: big data attribute subset generation, big data attribute subset evaluation, stop criterion and result validity verification.

The subset generation is a search process that produces a subset of attributes for evaluation. N represents the number of bid data attribute in the distributed network fault diagnosis database, therefore 2^N represents the number of all candidate subsets. Even if the number of original dataset attributes N is small, the number of its subsets is substantial. Therefore, it is impossible to complete the exhaustive search for the big data attribute space in the entire distributed network fault diagnosis database.

Rough set theory is a mathematical tool, usually used to characterize data incompleteness and uncertainty, so as to analyze and process imprecise, inconsistent, incomplete information, discover the implied knowledge, and reveal the underlying laws [10, 28]. The rough set theory is applied in the selection of big data attributes in distributed network fault diagnosis database, and the specific process is shown in below.

In database I =〈 U, A, V, f 〉, U represents a finite nonempty set of big data in the database and U = {1, 2, ⋯ , n}, n indicates the number of big data, A represents a finite nonempty set of all the attributes of the big data in the database and A = C ∪ D, wherein C, D respectively represent the set of conditional attributes and the set of decision attributes. C = {a₁, a₂, ⋯ , a_m}, m indicates the number of conditional attributes. Let D = {d}, where d represents the decision attribute, V represents a collection of any possible values for all attributes of big data in distributed network fault diagnosis database V = ∪ _vi. f represents a distributed network fault diagnosis database.

Assume R represents the binary relation on U, which mean that R has reflexivity, transitivity and symmetry. x, y are any objects in U, and R (x) is a set of y satisfiedxRy . When meets all requirements of ∀x ∈ U, ∃ xRx, ∀x, y ∈ U, ∃ xRy ⇒ yRx and∀x, y, z ∈ U, ∃ xRy, yRz ⇒ xRz. The big data attribute selection error cloud formula is expressed as $E = \sum_{i = 1}^{k} {\sum_{p \in C_{i}} | p - o_{i} |}^{2}$ (11)

In the above function, E represents the sum of squared errors for all big data attributes, p represents an object of the dataset, o_i is mean of class C_i, C_i is the distributed network fault diagnosis database, and N_i indicates the number of data object in class C_i. Use formula (12) to calculate the distance from each p in the data set to k cluster center: $dist (x, y) = \sqrt{\sum_{i = 1}^{d} (x_{i} - y_{i})^{2}}$ (12)

And then the property extraction is completed through the attribute similarity. H_i, H_j ∈ R^D are two object spaces, where R^D represents a distributed network fault diagnosis database, d (H_i, H_j) represents the distance between two object spaces, and d (H_ik, H_jk) represents the spatial distance of the k -th dimension of the two object spaces. $d (H_{i}, H_{j}) = max_{k = 1, 2, \dots, D} {d (H_{ik}, H_{jk})}$ (13)

According to the calculation, the above formula is converted into: $d (H_{i}, H_{j}) = {\begin{matrix} 0, if (H_{ik} \subseteq H_{jk} {orH}_{ik} \supseteq H_{jk}) \\ \frac{E_{X_{ik}} - E_{X_{jk}}}{3 (σ_{ik} + σ_{jk})}, if (H_{ik} ⊄ H_{jk} {orH}_{ik} ⊄ H_{jk}) \end{matrix}$ (14)

In the above equation: E_{X
_ik}, E_{X
_jk} respectively represent the spatial center of the k -th dimension of the two object spaces, σ_ik and σ_jk represent the entropy of the two attribute spaces, and $\frac{1}{3 (σ_{ik} + σ_{jk})}$ is one-third of the sum of the two attribute spaces.

Assume s (H_ik, H_jk) is the similarity of the k -th dimension of the two object spaces, we can get $s (H_{ik}, H_{jk}) = {\begin{matrix} 1, & d (H_{ik}, H_{jk}) = 0 \\ 1 - d (H_{ik}, H_{jk}), & 0 < d (H_{ik}, H_{jk}) < 1 \\ 0, & d (H_{ik}, H_{jk}) = 1 \end{matrix}$ (15) wherein, d (H_ik, H_jk) represents the role of the k -dimensional attribute space for the classification of two object spaces. When d (H_ik, H_jk) =0, the k -dimensional attribute space has no effect on the classification of two object spaces H_i, H_j; when d (H_ik, H_jk) ≥1, H_i and H_j are disjointed in the object space; when 0 < d (H_ik, H_jk) <1, H_i and H_j are intersecting the object space.

Let the distance of m (m > 2) D dimensional object space is $d (H_{1}, \dots, H_{m}) = min_{i = 1, \dots m, j \neq i} {d (H_{i}, H_{j})}$ (16)

The distance of its k -dimensional attribute space is $d_{K} = min_{i = 1, \dots m, j \neq i} {d (H_{i}, H_{j})}$ (17)

For the classification of several object spaces with D attributes, the attribute with the smallest similarity degree has the greatest contribution to the classification, and the similarity of the attribute can be calculated to calculate the weight of the attribute in the classification [31, 32]. Through the classification of each attribute, and the similarity of data object space, we can get the calculation method of attribute similarity and attribute weight.

2.2 Selection big data attribute in distributed network fault diagnosis database based on support vector machine

There are n training samples in m class applications classification problem. The ith data attribute x_i (i = 1, 2, ⋯ , n) is a p dimensional vector x_i = (x_i1, x_i2, x_ip). y_i indicates its corresponding category mark, and y_i ∈ {1, 2, ⋯ , m}. Let f (x) = (f₁ (x) , ⋯ , f_m (x)) represents the decision vector of big data attributes, wherein f_c (x) , c = 1, 2, ⋯ , m represents the attribute discriminant function, and f_c (x) = xβ_c + β_0c. β_c represents the weight of the c-class data of the distributed network fault diagnosis database, and β_0c represents the offset of c class data. The final attribute discrimination rule is $Φ_{f} (x) = arg max_{c} f_{c} (x)$ .

According to the above discussion, the loss function of the big data attribute in the distributed network fault diagnosis database is: $V (y_{i} f (x_{i})) = L (y_{i}) \cdot (f (x_{i}) - y_{i})_{+}$ (18)

In the above equation, (f (x_i) - y_i) ₊ = ((f₁ (x_i) - y_i1) ₊, ⋯ , (f_m (x_i) - y_im) ₊). Class mark y_i is encoded to y_i = (y_i1, ⋯ , y_im), an m-dimensional vector. Assume that the corresponding large data attribute diagnostic type is j, the j-th component of y_i is 1, and the remaining components are denoted by -1/(m - 1). L (y_i) is also an m-dimensional vector, with 0 as its j-th component and 1 as its remaining components. For better representation, the expression of the loss function can be rewritten to formula (19) $V (y_{i} f (x_{1})) = \sum_{y_{i} \neq c} (x_{i} β_{c} + β_{0 c} + \frac{1}{(m - 1)})$ (19)

In order to ensure that each attribute belongs to only a certain category, f_c (x) need to meet the conditions: $\sum_{c = 1}^{m} f_{c} (x) = 0$ (20)

Since the above condition is satisfied for any of the data attributes x, it can be converted into $\sum_{c = 1}^{m} β_{c} = 0, \sum_{c = 1}^{m} β_{0 c} = 0$ (21)

Combining the formula (20) and the formula (21), we get that SVM-based supervised big data attribute feature selection algorithm is equivalent to optimization problem:

$\begin{matrix} min_{β_{c}, β_{0 c}, c = 1, \dots, m} C \sum_{c = 1}^{m} \sum_{i = 1, y_{i} \neq c}^{n_{c}} (β_{0 c} + x_{i} β_{c} + \frac{1}{m - 1}) + \\ λ_{1} \sum_{c = 1}^{m} {∥ β_{c} ∥}_{1} + \frac{λ_{2}}{2} \sum_{c = 1}^{m} {∥ β_{c} ∥}_{2}^{2} s . t . {\begin{matrix} \sum_{c = 1}^{m} β_{c} = 0 \\ \sum_{c = 1}^{m} β_{0 c} = 0 \end{matrix} \end{matrix}$ (22) wherein, C is the penalty parameter of the big data attribute in the distributed network fault diagnosis database, λ₁and λ₂are adjustment parameters. n_cindicates the number of data that does not belong to the c-th big data attribute. By solving the above equation, we get the weight β_ci of each attribute in the c-th big data, i = 1, 2, ⋯ , p which is also an important measure of the i th characteristic of the data attribute, so as to complete the determination of the big data attribute in the distributed network fault diagnosis database.

Assume that $({\hat{β}}_{c}, {\hat{β}}_{0 c})$ is the optimal solution of Equation (22), if the loss function satisfies the Lipschitz continuous condition, $({\hat{β}}_{c}, {\hat{β}}_{0 c})$ satisfied the following conditions for any 〈j, k〉: $\begin{matrix} \sum_{c = 1}^{m} {| {\hat{β}}_{cj} - {\hat{β}}_{ck} |}^{2} \leq \frac{2 M}{λ_{2}} \sum_{c = 1}^{m} | {\hat{β}}_{cj} - {\hat{β}}_{ck} | \\ \sum_{i = 1, y_{i} \neq c}^{n_{c}} | {\hat{β}}_{ij} - {\hat{β}}_{ik} | \end{matrix}$ (23)

In the above function, M represents a positive number, j and k represents the features of the big data attribute in the distributed network fault diagnosis database. The normalization of big data attribute eigenvalues is: $\begin{matrix} \sum_{c = 1}^{m} {| {\hat{β}}_{cj} - {\hat{β}}_{ck} |}^{2} \leq \frac{2 M}{λ_{2}} \sum_{c = 1}^{m} | {\hat{β}}_{cj} - {\hat{β}}_{ck} | \\ \sqrt{2 n_{c} (1 - ρ_{c})} \end{matrix}$ (24) wherein, ρ_c represents the correlation coefficient between feature j and feature k that does not belong to the big data attribute in the c -th distributed network fault diagnosis database, and $ρ_{c} = \sum_{i = 1}^{n} (x_{ij} - {\bar{x}}_{j}) (x_{ik} - {\bar{x}}_{k}) / \sqrt{\sum_{i = 1}^{n} {(x_{ij} - {\bar{x}}_{j})}^{2} \sum_{i = 1}^{n} {(x_{ik} - {\bar{x}}_{k})}^{2}}$ , x_i∈ {x|y_i ≠ c, i = 1, 2, ⋯ , n} , ${\bar{x}}_{j} = \frac{1}{n} \sum_{i = 1}^{n} x_{ij}$ , ${\bar{x}}_{k} = \frac{1}{n} \sum_{i = 1}^{n} x_{ik}$ . Then we get a set of weights $(β_{c}^{*}, β_{0 c}^{*})$ :

$(β_{c}^{*}, β_{0 c}^{*}) = {\begin{matrix} ({\hat{β}}_{cj} + {\hat{β}}_{ck}) / 2, j = k \\ {\hat{β}}_{cj}^{*}, otherwise \end{matrix}$ (25)

Let $X = (x_{ij})_{i = 1, j = 1}^{n, p}$ , X_c = {x|y_i ≠ c, i = 1, ⋯ , n}, c = 1, ⋯ , m, the formula (22) can be converted into: $\begin{matrix} min_{β_{c}, β_{0}} C \sum_{c = 1}^{m} \sum_{i = 1, y_{i} \neq c}^{n_{c}} (a_{ci})_{+} \\ + λ_{1} \sum_{c = 1}^{m} {∥ t_{c} ∥}_{1} + \frac{λ_{2}}{2} \sum_{c = 1}^{m} {∥ β_{c} ∥}_{2}^{2} \\ s . t . {\begin{matrix} \sum_{c = 1}^{m} β_{c} = 0 \\ \sum_{c = 1}^{m} β_{0 c} = 0 \\ a_{c} = \frac{1}{m - 1} 1_{c} + X_{c} β_{c} + β_{0 c} 1_{c} \\ t_{c} = β_{c} c = 1, \dots, m \end{matrix} \end{matrix}$ (26)

In the above equation, a_c = (a_ci, ⋯ , a_{cn
_c}), and 1_c represents the n_c-dimensional vector with all big data attributes equals to 1. The Lagrangian function of the above equation is $\begin{matrix} l (γ) = min_{β, β_{0}} \sum_{c = 1}^{m} (\begin{matrix} C \sum_{i = 1, y_{i} \neq c}^{n_{c}} (a_{ci})_{+} + λ_{1} \sum_{c = 1}^{m} {∥ t_{c} ∥}_{1} + \frac{λ_{2}}{2} \sum_{c = 1}^{m} {∥ β_{c} ∥}_{2}^{2} \\ + u_{c}^{T} (\frac{1}{m - 1} 1_{c} + X_{c} β_{c} + β_{0 c} 1_{c} - a_{c}) \end{matrix}) \\ + \sum_{c = 1}^{m} v_{c}^{T} (β_{c} - t_{c}) {+ p}^{T} \sum_{c = 1}^{m} β_{c} + q \sum_{c = 1}^{m} β_{0 c} \end{matrix}$ (27)l (γ) represents the vector of the big data to be optimized in the distributed network fault diagnosis database, l (γ) = {β_c, β_0c, a_c, t_c, u_c, v_c, p, q, c = 1, 2, ⋯ , m}, and u_c, v_c, p, q denote dual variables corresponding to the four constraints of Equation (26). Introduce penalties into the above equation, we can get: $\begin{matrix} L (γ) = l (γ) \\ + \sum_{c = 1}^{m} \frac{u_{ic}}{2} {∥ \frac{1}{m - 1} 1_{c} + X_{c} β_{c} + β_{0 c} 1_{c} - a_{c} ∥}_{2}^{2} + \\ \frac{μ_{2 c}}{2} {∥ β_{c} - t_{c} ∥}_{2}^{2} + \frac{μ_{3}}{2} {∥ \sum_{c = 1}^{m} β_{c} ∥}_{2}^{2} + \frac{μ_{4}}{2} {∥ \sum_{c = 1}^{m} β_{0 c} ∥}_{2}^{2} \end{matrix}$ (28)

In the formula, u_ic, μ_2c, μ₃, and μ₄ represent the big data attribute parameters in the distributed network fault diagnosis database. One saddle point $(β_{c}^{*}, β_{0 c}^{*}, a_{c}^{*}, t_{c}^{*}, u_{c}^{*}, v_{c}^{*}, p^{*}, q^{*}, c = 1, 2, \dots, m)$ of L (γ) can be calculated, and β_c, β_0c, a_c, t_c, u_c, v_c, p, q are satisfied the following equation: $\begin{matrix} L (β_{c}^{*}, β_{0 c}^{*}, a_{c}^{*}, t_{c}^{*} u_{c}, v_{c}, p, q) \\ \leq L (β_{c}^{*}, β_{0 c}^{*}, a_{c}^{*}, t_{c}^{*}, u_{c}^{*}, v_{c}^{*}, p^{*}, q^{*}) \\ \leq L (β_{c}, β_{0 c}, a_{c}, t_{c}, u_{c}^{*}, v_{c}^{*}, p^{*}, q^{*}) \end{matrix}$ (29)

In order to solve the problem of the saddle point, the gradient rise method is used to solve the dual problem: $max_{u_{c}, v_{c}, p, q} min_{β_{c}, β_{0 c}, a_{c}, t_{c}} L (β_{c}, β_{0 c}, a_{c}, t_{c}, u_{c}, v_{c}, p, q)$ (30)

Let $Q (u_{c}, v_{c}, p, q) = min_{β_{c}, β_{0 c}, a_{c}, t_{c}}$ L (β_c, β_0c, a_c, t_c, u_c, v_c, p, q), ∇Q (u_c, v_c, p, q) can be calculated by:

$\nabla Q (u_{c}, v_{c}, p, q) = (\begin{matrix} \frac{1}{m - 1} 1_{c} + X_{c} β_{c} + β_{0 c} 1_{c} - a_{c} \\ β_{c} - t_{c} \\ \sum_{c = 1}^{m} β_{c} \\ \sum_{c = 1}^{m} β_{0 c} \end{matrix})$ (31) (β_c, β_0c, a_c, t_c) can be solved by the following equation: $\begin{matrix} (β_{c}, β_{0 c}, a_{c}, t_{c}) = arg min_{β_{c}, β_{0 c}, a_{c}, t_{c}} \\ L (β_{c}, β_{0 c}, a_{c}, t_{c}, u_{c}, v_{c}, p, q) \end{matrix}$ (32) For the dual problem (30), according to the formula (31) and (32), we can get the multiplier method of formula (26): ${\begin{matrix} (β_{c}^{k + 1}, β_{0 c}^{k + 1}, a_{c}^{k + 1}, t_{c}^{k + 1}) = arg min_{β_{c}, β_{0 c}, a_{c}, t_{c}} \\ L (β_{c}, β_{0 c}, a_{c}, t_{c}, u_{c}^{k}, v_{c}^{k}, p^{k}, q^{k}) \\ u_{c}^{k + 1} = u_{c}^{k} + μ_{1 c} \\ (\frac{1}{m - 1} 1_{c} + X_{c} β_{c}^{k + 1} + β_{0 c}^{k + 1} 1_{c} - a_{c}^{k + 1}) \\ v_{c}^{k + 1} = v_{c}^{k} + μ_{2 c} (β_{c}^{k + 1} - t_{c}^{k + 1}) \\ p^{k + 1} = p^{k} + μ_{3} \sum_{c = 1}^{m} β_{c}^{k + 1} \\ q^{k + 1} = q^{k} + μ_{4} \sum_{c = 1}^{m} β_{0 c}^{k + 1} \end{matrix}$ (33)

For the above equation, it is necessary to pass through multiple iterations until the convergence condition is satisfied, and finally the solution (β_c, β_0c, a_c, t_c) is obtained. Because the objective function of the large data attribute problem in the distributed network fault diagnosis database is derivative, it can be transformed into a linear system of equations: $\begin{matrix} \begin{matrix} (\begin{matrix} (λ_{2} + μ_{2 c} + μ_{3}) I + μ_{1 c} X_{c}^{T} X_{c} & μ_{1 c} X_{c}^{T} 1_{c} \\ μ_{1 c} 1_{c}^{T} X_{c} & μ_{1 c} n_{c} + μ_{3} \end{matrix}) & (\begin{matrix} β_{c}^{k + 1} \\ β_{0 c}^{k + 1} \end{matrix}) \end{matrix} = \\ (\begin{matrix} μ_{2 c} t_{c}^{k} + μ_{1 c} X_{c}^{T} (a_{c}^{k} - \frac{1}{m - 1} 1_{c}) - X_{c}^{T} u_{c}^{k} - v_{c}^{k} - p^{k} - μ_{3} \sum_{i = 1, i \neq c}^{m} β_{i}^{k + 1} \\ μ_{1 c} 1_{c}^{T} a_{c}^{k} - 1_{c}^{T} u_{c}^{k} - q^{k} - \frac{μ_{1 c} n_{c}}{m - 1} - μ_{4} \sum_{i = 1, i \neq c}^{m} β_{0 i}^{k + 1} \end{matrix}) \end{matrix}$ (34)

In the above formula, I represents the unit p × p matrix, and $X_{c}^{T} X_{c}$ represents the lower order symmetry coefficient matrix.

In the above discussion, the big data attribute selection algorithm is improved by calculating the loss function, the weight of the big data attribute is calculated, and the saddle point is solved by the gradient rising method, so as to realize the selection process of big data attribute in distributed network fault diagnosis database based on support vector machine. The process is shown in Fig. 2.

Fig. 2

Big data attribute selection process in distributed network fault diagnosis database.

3 Experimental results and analysis

3.1 Experimental environment

In order to prove the validity of the big data attribute selection method in the distributed network fault diagnosis database based on support vector machine, we use MATLAB2008a as the platform and Intel P4 2 G processor to perform the simulation experiment. The main frequency is Dual P2.16, the memory is 3 G, the hard disk asks 250 G, and the operating system is windows 7.

3.2 Results and analysis

The experiment selects three data sets in the network database to analyze the data attributes of three different experimental data sets, and compares the time consuming of the three data sets. In the first data set, the method proposed in this paper is compared with the data attribute selection method proposed in [7] and [8], and the comparison result is shown below.

First, the time consumed (min), calculated through formula (35), in the three methods for big data attribute selection is compared. $T = \frac{E log L (γ)}{\sum_{i = 1}^{m} f (x)} \cdot σ$ (35)

In the above equation, σ is the response time parameter when the big data attribute is selected, and the average response time of the big data attribute selection is obtained according to the above three methods. The comparison results are shown in Table 1.

Table 1

Time-consuming comparison of three methods

Numbers of experiments/times	The proposed method/min	The method proposed in literature [7]/min	The method proposed in literature [8]/min
50	18	25	23
100	35	47	44
150	50	71	66
200	67	89	85
250	83	112	109
350	115	159	148
500	129	218	213

According to the formula (36), the average time-consuming comparison of the three methods in the second data set is calculated. In order to ensure the accuracy of the experiment, 500 experiments were carried out, with 50 experimental data as a set of data, so as to complete the average time calculation, the time unit is seconds (s), the formula is: $T_{0} = \frac{T - T^{'}}{50}$ (36)

In the above formula, T′ is the time spent in other work in the experiment. The average time of three big data attribute selection methods was calculated. The comparison results are shown in Table 2.

Table 2

Average time-consuming comparison of three methods for large data attribute selection

Numbers of experiments/times	The proposed method/min	The method proposed in literature [7]/min	The method proposed in literature [8]/min
50	14.8	22.8	19.1
100	14.4	22.1	17.8
150	14.1	21.9	21.6
200	15.1	22.0	17.9
250	15.2	22.5	17.6
300	15.1	22.9	20.9
350	15.5	22.6	19.5
400	15.2	21.8	19.7
450	15.3	22.3	20.2
500	15.0	22.1	20.8

In the third data set, the average time consumption of the three methods is compared, and the results are shown in Fig. 3. The dots on the straight line in Fig. 3 represent the theoretical time consuming of the three methods. With the increase of the number of experiments, the corresponding time of the method in this paper is less than 16 s, the method in reference 7 is greater than 22 s, and the method in reference 8 is between 17–22 s. It can be seen that difference between actual and theoretical time-consuming of the proposed method is less than that of the literature [7] and the literature [8]. The average time-consuming polyline of the proposed method is close to a straight line and the fluctuation is small, which indicates that the proposed method is stable in the big data attribute selection.

Fig. 3

The average time-consuming comparison of the three methods.

Then compare the energy consumption of three methods, we assume N as the energy consumption unit, $N = \sum \sum_{i = 1}^{m} β_{c} \cdot ɛ$ (37)

According to the above formula, the energy consumption of the three selection methods are compared, and the results are shown in Fig. 4.

It can be seen from Fig. 4 that with the increase of the running time, the energy consumption of the three methods also increases, and the energy consumption fluctuation of the proposed method in big data attribute selection is smaller than that of literature [7] and the literature [8]. When the running time is 50 h, the method of this paper is 227 N, the method of reference [7] is 359 N, and the method of reference [8] is 562 N. So the proposed method can effectively reduce the energy consumption in big data attribute selection process, which indicates that the proposed method is stable in the big data attribute selection.

Fig. 4

The energy consumption comparison results of the three methods.

Finally, three methods are used to select the data attributes of any seven databases in the network, and the accuracy of these three methods in the process of selecting big data attributes is compared. Since the number of attributes in the database is large, accuracy indicates that the number of big data attributes can be selected correctly, and error indicates that the number of big data attributes cannot be selected accurately. The results are shown in Table 3.

Table 3

The accuracy of three method for big data attribute selection

Number of attributes	The proposed method		The method proposed in literature [7]		The method proposed in literature [8]
	Accuracy	Error	Accuracy	Error	Accuracy	Error
132	123	9	109	23	113	19
210	203	7	187	23	196	14
218	207	11	201	17	289	29
345	339	6	319	26	327	18
426	413	13	401	25	407	19
457	443	14	426	31	431	26
543	522	21	507	36	516	27

The accuracy ratio is the ratio of the exact quantity to the total quantity. The error rate is the ratio of the number of errors to the total quantity. The formula is as follows (38). ${\begin{matrix} η = \frac{Exact number}{Total quantity} \times 100 \\ λ = \frac{Error number}{Total quantity} \times 100 \end{matrix}$ (38) In the formula, η and λ indicate the accuracy and error rate. Using the above table information, the accuracy of the three methods for big data attribute selection are compared, the results are shown in Figs. 5, 6.

Fig. 5

Comparison of accuracy of three methods.

Fig. 6

Comparison of error rate of three methods.

Through Figs. 5 and 6 we can see that the method proposed in this article has the highest accuracy and the lowest error rate.

In summary, the method proposed in this paper can effectively reduce the energy consumption and cost of big data attribute selection in the distributed network fault diagnosis database, improve the efficiency of big data attribute selection in the distributed network fault diagnosis database, ensure the real-time of network fault diagnosis, and has great practical value.

4 Conclusion

Reducing the consumption of big data attribute selection time can improve the fault diagnosis capability of distributed networks. The big data attribute method based on support vector machine proposed in this paper can effectively reduce the energy consumption in the process of big data attribute selection. And the energy consumption fluctuation of the proposed method in big data attribute selection is smaller than traditional methods, the corresponding time of the method is less than 16 s, and the average time-consuming polyline of the proposed method is close to a straight line and the fluctuation is small, and the method proposed in this article has the lowest error rate, which indicates that the proposed method is stable in the big data attribute selection, and it has good practical value. In future work, we will work to further reduce the bit error rate, improve the stability of the proposed method, and try to apply the proposed method to other types of data selection.

References

Cai

, Liu

, Fan

, Zhang

and Liu

, Multi-source information fusion based fault diagnosis of ground-source heat pump using Bayesian network, Applied Energy114(2) (2014), 1–9.

, Xu

and Yang

Y.C.

, Research and application of decision tree classification algorithm based on electronic design engineering, Electronic Design Engineering24(18) (2016), 1–3.

Xue

, Jing

L.I.

, Wang

and Niu

, Effects of target and distractor saturations on the cognitive performance of an integrated display interface, Chinese Journal of Mechanical Engineering28(1) (2015), 208–216.

Liu

, Liu

, Jin

, Jia

and Wang

, Research on workshop-based positioning technology based on internet of things in big data background, Complexity (2018), 7875460.

, You

and Liu

, Support vector machine (SVM) based prestack AVO inversion and its applications, Journal of Applied Geophysics120 (2015), 60–68.

Anagnostopoulos

, Zeadally

and Exposito

, Handling big data: research challenges and future directions, The Journal of Supercomputing72(4) (2016), 1494–1516.

Kim

and Hwan

, Real-time stream data mining based on Can Tree and Gtree, Information Sciences367 (2016), 512–528.

Qian

, Lv

, Yue

, Liu

and Jing

, Hierarchical attribute reduction algorithms for big data using Map Reduce, Knowledge-Based Systems73 (2015), 18–31.

Ferreira

J.C.

, Almeida

J.D.

and Silva

A.R.D.

, The impact of driving styles on fuel consumption: a data-warehouse-and-data-mining-based discovery process, IEEE Transactions on Intelligent Transportation Systems16(5) (2015), 1–10.

10.

Popovic

J.R.

, Distributed data networks: a blueprint for Big Data sharing and healthcare analytics, Annals of the New York Academy of Sciences1387(1) (2017), 105–111.

11.

Cui

, Yu

F.R.

and Yan

, When big data meets software-defined networking: SDN for big data and big data for SDN, IEEE Network30(1) (2016), 58–65.

12.

Wang

, Wang

and Chang

, Feature selection methods for big data bioinformatics: a survey from the search perspective, Methods111 (2016), 21–31.

13.

Askarian

, Zarghami

, Jalali-Farahani

and Mostoufi

, Fault diagnosis of chemical processes considering fault frequency via Bayesian network, Canadian Journal of Chemical Engineering94(12) (2016), 2315–2325.

14.

Berger

and Tutz

, tree-structured clustering in fixed effects models, Statistics17 (2015), 1–31.

15.

Panda

and Khilar

P.M.

, Distributed self fault diagnosis algorithm for large scale wireless sensor networks using modified three sigma edit test, Ad Hoc Networks25(4) (2015), 170–184.

16.

Unal

, Onat

, Demetgul

and Kucuk

, Fault diagnosis of rolling bearings using a genetic algorithm optimized neural network, Measurement58 (2014), 187–196.

17.

Karabadji

N.E.I.

, Seridi

, Khelf

, Azizi

and Boulkroune

, Improved decision tree construction based on attribute selection and data sampling for fault diagnosis in rotating machines, Engineering Applications of Artificial Intelligence35(35) (2014), 71–83.

18.

Arain

Q.A.

, Memon

M.H.

, Shaikh

R.A.

and Mangi

F.A.

, Intelligent travel information platform based on location base services to predict user travel behavior from user-generated GPS traces, International Journal of Computers & Application39(3) (2017), 1–4.

19.

Zhang

Q.H.

, Hu

, Sun

, Si

and Qin

, Concurrent fault diagnosis for rotating machinery based on vibration sensors, International Journal of Distributed Sensor Network13(1) (2013), 59–72.

20.

Islam

, Khan

S.A.

and Kim

J.M.

, Discriminant feature distribution analysis-based hybrid feature selection for online bearing fault diagnosis in induction motors, Journal of Sensors25(2) (2016), 1–16.

21.

Jin

, Cui

, Jin

and Wang

, AF-DHNN: fuzzy clustering and inference-based node fault diagnosis method for fire detection, Sensors15(7) (2015), 17366–17396.

22.

Kelkar

and Kamal

, Adaptive fault diagnosis algorithm for controller area network, IEEE Transactions on Industrial Electronics61(10) (2014), 5527–5537.

23.

Wang

, Sun

and Li

, Wind turbine gearbox fault diagnosis method based on riemannian manifold, Mathematical Problems in Engineering20(4) (2005), 1–10.

24.

Río

S.D.

, López

, Benítez

J.M.

and Herrera

, A mapreduce approach to address big data classification problems based on the fusion of linguistic fuzzy rules, International Journal of Computational Intelligence Systems8(3) (2015), 422–437.

25.

Yun

and Kim

, A fast perturbation algorithm using tree structure for privacy preserving utility mining, Expert Systems with Applications42(3) (2015), 1149–1165.

26.

Reppa

, Polycarpou

M.M.

and Panayiotou

C.G.

, Distributed sensor fault diagnosis for a network of interconnected cyberphysical systems, IEEE Transactions on Control of Network Systems2(1) (2015), 11–23.

27.

Jin

, Chow

T.W.S.

, Sun

, Shan

and Lau

B.C.P.

, Kuiper test and autoregressive model-based approach for wireless sensor network fault diagnosis, Wireless Networks21(3) (2015), 829–839.

28.

Gao

, Yang

, Tian

and Chen

, Entropy based test point evaluation and selection method for analog circuit fault diagnosis, Mathematical Problems in Engineering24(6) (2014), 1–16.

29.

Han

, Li

, Yang

, Liu

and Zhang

, Dynamic prediction research of silicon content in hot metal driven by big data in blast furnace smelting process under hadoop cloud platform, Complexity (2018), 8079697.

30.

Lei

, Jia

, Lin

, Xing

and Ding

S.X.

, An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data, IEEE Transactions on Industrial Electronics63(5) (2016), 3137–3147.

31.

Qin

, Lou

, Zhao

and Zhang

, Research on relationship between tourism income and economic growth based on meta-analysis, Applied Mathematics & Nonlinear Sciences3 (2018), 105–114.

32.

Sagna

, Multidimensional BSDE with Poisson jumps of Osgood type, Applied Mathematics and Nonlinear Sciences4(2) (2019), 387–394.