A Relief-PGS algorithm for feature selection and data classification

Abstract

As a supervised learning algorithm, Support Vector Machine (SVM) is very popularly used for classification. However, the traditional SVM is error-prone because of easy to fall into local optimal solution. To overcome the problem, a new SVM algorithm based on Relief algorithm and particle swarm optimization-genetic algorithm (Relief-PGS) is proposed for feature selection and data classification, where the penalty factor and kernel function of SVM and the extracted feature of Relief algorithm are encoded as the particles of particle swarm optimization-genetic algorithm (PSO-GA) and optimized by iteratively searching for optimal subset of features. To evaluate the quality of features, Relief algorithm is used to screen the feature set to reduce the irrelevant features and effectively select the feature subset from multiple attributes. The advantage of Relief-PGS algorithm is that it can optimize both feature subset selection and SVM parameters including the penalty factor and the kernel parameter simultaneously. Numerical experimental results indicated that the classification accuracy and efficiency of Relief-PGS are superior to those of other algorithms including traditional SVM, PSO-GA-SVM, Relief-SVM, ACO-SVM, etc.

Keywords

Support Vector Machine particle swarm optimization genetic algorithm Relief algorithm feature selection data classification

1. Introduction

Data classification is crucial in data mining, cluster analysis and intelligent information processing. The target of classification is to establish a model based on the feature of the number dataset, which can predict unknown samples categories to one of the given categories [1]. As a fundamental supervised learning method, the support vector machines (SVM) is widely used in text recognition, image recognition etc [2, 3, 4]. The principle of SVM is to maximize the difference between different types of data by constructing a classification hyperplane as a decision surface [5], which is considered to be an effective method to avoid local optimum and has unique advantages in dealing with complex problems such as limited samples, high dimensional and nonlinear data.

As a popular machine learning method, the performance of SVM in convergence, speed, and accuracy of training and classification is is affected by the selection of the penalty factor $C$ and the kernel parameter $\sigma$ . The value of the penalty factor $C$ weighs the empirical risk and the structural risk, whereas the kernel parameter $\sigma$ of SVM is related to the fineness of the sample division. However, inappropriate penalty factor $C$ and kernel parameter for SVM can lead to over-fitting or under-fitting problems [6]. It is necessary to optimize the penalty factor and kernel parameter of SVM, which can reduce the time complexity of classification and improve the accuracy of classification [7]. The optimization algorithms of penalty factor and kernel parameter for SVM including gravitational search algorithm [8], grid-search method [9], the gradient descent method [10], etc. Gravitational search algorithm based on the law of Newtonian gravity is time-consuming to obtain the solution since it has the problem of local optimization and the global optimal solution may not be found. Grid search algorithm has high learning accuracy and global optimization ability. On the contrary, it consumes much time to calculate the parameters of the algorithm. Gradient descent method based on a convex function is easy to fall into local optimization solution in high-dimensional space. To overcome these difficulties, various intelligent algorithms based on social intelligent behavior emerge, including neural network (NN) [11], genetic algorithm (GA) [12], particle swarm optimization (PSO) [13], firefly algorithm (FA) [14] and whale optimization algorithm (WOA) [15], etc. Ant colony algorithm based on the behavior of ants searching for food can search the classification features globally, but it is difficult to balance between population diversity and convergence speed [16]. As a popular and adaptable neural network architecture for machine learning, neural network algorithms have few and intuitive parameters but classification is relatively slow. GA algorithm based on a natural selection process that mimics biological evolution can evaluate multiple solutions and estimate the convergence speed and optimize the global search ability by the crossover and mutation [17]. However, the stability of GA algorithm is slightly poor and has a long calculation time [18]. PSO algorithm updates the position and speed of particles to realize the information interaction in the group. However, it has some problems such as premature convergence and easy to fall into local optimization [19]. FA algorithm based on its luminous characteristics has many problems, such as high complexity, difficult to optimize high-dimensional functions, easy to fall into local minima, etc. WOA algorithm is a meta-heuristic optimization algorithm based on the behavior of the humpback whales [20]. Nevertheless, it lacks global search ability and slow convergence speed. These intelligent algorithms can identify the best possible building design by searching through a number of possible iterations. However, the widespread application of optimization algorithms remains restricted due to low efficiency the optimization of the penalty factor $C$ and the kernel parameter $\sigma$ .

Due to the limitations of individual intelligent algorithms, the combination intelligent algorithms to accelerate the optimization process of SVM is presented in recent years, such as PSO-GA [21], GA-WOA [22], FA-PSO [23], NN-GA [24], etc. Among the nature-inspired algorithms, PSO-GA is popular algorithms. Liu et al. [25] proposed an SVM algorithm combining genetic algorithm and particle swarm algorithm to optimize the parameters of the five-parameter model. Huang et al. [26] introduced three algorithms including PSO, GA and grid-search algorithm to obtain the parameters of Radial Basis Function (RBF) kernels of SVM. Cui et al. [27] presented a novel method coupling a modified PSO algorithm with GA-SVM algorithm to optimize the kernel parameters of SVM. Bonah et al. [28] optimized SVM parameters based on genetic algorithms (GA), grid search (GS) algorithms and particle swarm optimization (PSO) to improve classification accuracy of spectral data. Optimizing the performance of the SVM can take into account optimizing subset of features, which can improve the robustness of the model and the speed of convergence.

Since the original feature contains a large amount of redundant information, it is necessary to reduce irrelevant features during training can accelerate classification process of SVM [29]. Relief algorithm is a multivariate filtering feature weighting and selection algorithm [30], which calculates the weight of the feature vector based on sample learning [31, 32]. The Relief-SVM consists of feature selection based on Relief algorithm and SVM classification method. The advantage of Relief-SVM method lies in that it can identify optimal features and construct optimal classifiers, which results in the increase of the classification accuracy and the decrease of the classification time [33]. However, it can not completely remove irrelevant and redundant features, which may not contribute to the performance for feature extraction problems.

To solve the problem that the existing methods are not suitable for optimizing feature subset selection and SVM parameters at the same time, a new SVM algorithm based on Relief algorithm and particle swarm optimization-genetic algorithm (Relief-PGS) is proposed to improve the classification potential of SVM. The main contributions of this paper include as follows. (1) A new effective hybrid method based on particle swarm optimization and genetic algorithm (PSO-GA) is proposed for feature selection and parameter optimization of SVM. (2) The feature subset screened by Relief algorithm and SVM parameters were encoded into the PGS algorithm for quadratic feature filtering to optimize the number of features and improve the classification accuracy of SVM. (3) The Relief-PGS algorithm can optimize both feature subset selection and SVM parameters simultaneously. (4) The efficiency and effectiveness of the proposed Relief-PGS are demonstrated by experimental results.

The remainder of the paper is organized as follow. Related works about the SVM algorithms are introduced in Section 2. In Section 3, some basic theories about Relief, PSO and GA algorithms are introduced. In Section 4, the proposed Relief-PGS algorithm is introduced. In Section 5, various kinds of experiments are conducted to evaluate the performance of the proposed algorithm under varying datasets. Finally, the conclusions are made in Section 6.

2. Related works

Support vector machine (SVM) is a classification method based on the principle of structural risk minimization [34, 35]. It is an effective method to avoid local optimum and has unique advantages in dealing with complex problems such as limited samples, high dimensional and nonlinear data. The performance of SVM is highly related to its kernel parameters and penalty factor, and the key to improve the classification accuracy is to select the appropriate parameters. SVM are sensitive to parameter conditioning and the choice of sum functions, and suitable for handling binary classification problems if left unmodified. Using intelligent algorithms to optimize the penalty factor C and kernel parameter $\sigma$ of support vector machines can increase the speed of classification and improve the accuracy of classification. At present, there are a lot of parameters optimization methods. Raman et al. [36] presented an adaptive and robust intrusion detection technology based on hypergraph genetic algorithm for parameter setting and feature selection in SVM. Saiparvathi et al. [37] proposed a new method for SVM parameter optimization by back-end GA algorithm. Tharwat et al. [38] introduced a quantum-behaved particle swarm optimization (QPSO) algorithm to optimize the parameters of SVM and improve the accuracy of classification. Hamid et al. [39] presented a feature selection method of set filter combining particle swarm optimization algorithm and SVM classification. Vieira et al. [40] proposed a binary particle swarm optimization algorithm for feature selection and parameter optimization of SVM. Qaraad et al. [41] presented a hybrid feature selection Elastic Net-SVM (ENSVM) model for classification.

Most of the methods can only individually optimize either feature subset selection or SVM parameters, greatly limiting the classification potential of SVM. In recent years, many scholars have proposed new algorithms for simultaneous optimization of feature selection and data classification [42]. Mehdi et al. [43] introduced a new intrusion detection method, which uses genetic algorithm to optimize SVM and particle swarm optimization algorithm to select the most influential features to learn classification model. Dinesh et al. [44] proposed the KPCA-GA-SVM feature classification optimization model, which improved the classification accuracy based on the selected relevant features. Bi et al. [45] presented a novel hybrid genetic algorithm-particle swarm optimization (GA-PSO) method to optimize the SVM model. The optimization process and result demonstrated that the GA-PSO-SVM method was more accurate and time-saving than the classical GA and PSO method. Compared with the classical Grid-search SVM, the combined GA-PSO-SVM model appeared to be more applicable for the property prediction.

In general, redundant features complicate the operation and reduce the processing speed, which may lead to a reduction in classification accuracy. Thus, it is necessary to select features that have a greater role in classification recognition. In order to improve the accuracy of the classification, Wang [46] introduced a fast Relief algorithm to compute the training sample weight values for the optimization of SVM kernel function in content-based image retrieval systems. Zhang et al. [47] proposed to use denoising features of Relief algorithm combined with mixed kernel function to optimize SVM model. Choi et al. [48] presented a relieve-based de-noising feature to extract relevant features from rough tablet surfaces. Dou et al. [49] introduced a Relief-SVM method, which is used to recognize coal and gangue based on image analysis. The Relief-SVM method presented in the study is proven to be effective in the aforementioned types of complex situations.

Based on the existing research work, we propose an Relief-PGS algorithm to solve the feature selection and data classification problem in terms of both feature subset and parameter optimization, and shows its superiority over other algorithms.

3. Preliminaries

3.1 SVM model

SVM is a classification algorithm based on statistical learning theory. The principle of SVM is to construct an optimal hyperplane as a decision surface by a small set of vectors near boundary and divide the data points of different categories in the vector space. When the problem is the linearly separable, the optimal decision surface needs to satisfy the condition that the samples can be separated and the classification interval is maximized. When the problem is linear inseparable, it is need to search for a multi-dimensional hyperplane to separate the samples. Thus, SVM solves the problems of traditional learning methods, such as nonlinearity, over learning, high dimension, local minima, etc.

A theoretical assumption for SVM algorithm is that there are data set $({x_{1},y_{1}}),({x_{2},y_{2}}),\cdots({x_{N},y_{N}})$ , $x\in R^{n},y\in({-1,1})$ , and a nonlinear regression function is used to fit the sample data set in the form.

$\displaystyle\left\{{{\begin{array}[]{l}{\min\frac{1}{2}w^{T}\times w+C\sum% \limits_{i=1}^{N}{\xi_{i}}}\\ {s.t.y_{i}({w^{T}x_{i}+b})>1-\xi_{i},\xi_{i}\geqslant 0,i=1,2,\cdots N}\\ \end{array}}}\right.$ (1)

where $C$ is penalty factor, $\xi_{i}$ is slack variable in the linearly inseparable data, $w$ is the weight and $x_{i}$ is the sample.

For some data that cannot be separated in linear space, SVM maps the data into high-dimensional linear space through the nonlinear equation of the kernel function to make the data separable. The function $f(x)$ of the data can be expressed in the original dimensional feature space in the form.

$\displaystyle f(x)=\text{sgn}\left({\sum\limits_{i=1}^{n}{\alpha_{i}y_{i}K({x_% {i},x})}+b}\right)$ (2)

where $K({x_{i},x})$ is kernel parameters, $\alpha_{i}({i=1,2,\cdots n})$ is Lagrange multiplier, $y_{i}$ is the label of sample and $b$ is threshold.

The selection of kernel parameters is very important for SVM classifiers because it has significant impact on the learning ability of the SVM methods. The kernel parameters commonly used in SVM include linear, ploy kernel function, RBF and sigmoid functions, etc. In this paper, the radial basis kernel parameters are selected as the kernel parameters of SVM. The RBF kernel parameters of SVM are given as

$\displaystyle K({x_{i},x})=\exp\left\{{-\frac{\|{x-x_{i}}\|^{2}}{2\sigma^{2}}}\right\}$ (3)

where $\sigma$ is the standard deviation parameter of the kernel function. Penalty factor $C$ and kernel parameter $\sigma$ are important parameters in the classification results. If penalty factor $C$ is too large, the fitting accuracy of training samples is very high, but the generalization ability of the model is very poor; if penalty factor $C$ is too small, the search time is very long and the ability of the model generalization is very low. The values of kernel parameter $\sigma$ also greatly affect the learning and generalization capability of the SVM method [50]. In order to obtain higher SVM classification performance, it is critical to select suitable $C$ and $\sigma$ to optimize the performance of SVM model.

3.2 PSO algorithm

Particle swarm optimization (PSO) is an evolutionary computing technology inspired by the population behavior of birds, where each bird in the flocks is considered as one of the particles and each individual particle represents a possible potential optimal solution.

The basic principle of the PSO algorithm is described as follows. First, it can be assumed that a population consists of m particles in an S-dimensional target search space, where the i-th particle represents an s-dimensional vector $\overline{x_{i}}=({x_{i1},x_{i2},\ldots,x_{iS}})$ , $i=1,2,\cdots m$ , where each particle’s position is a solution. Then, $\overline{x_{i}}$ is brought into an objective function, and its fitness is calculated, and the degree of the solution is judged according to the degree of fitness. The flight speed of the particle is an S-dimensional vector, denoted as $\overline{V_{i}}=({V_{i1},V_{i2},\ldots,V_{iS}})$ . The optimal position searched by the i-th particle is $\overline{P_{i}}=({P_{i1},P_{i2},\ldots,P_{iS}})$ , and then the optimal position searched for the entire particle swarm is $\overline{P_{gS}}=({P_{gS},P_{gS},\ldots,P_{gS}})$ . If $f(x)$ is the minimized objective function, then the best position of particle $i$ can be determined by Eq. (4).

$\displaystyle p_{i}({t+1})=\left\{{{\begin{array}[]{l}{p_{i}\to f({x_{i}({t+1}% )})\geqslant f({p_{i}(t)})}\\ {X_{i}({t+1})\to f({x_{i}({t+1})})<f({p_{i}(t)})}\\ \end{array}}}\right.$ (4)

The particles of PSO algorithm can be manipulated by

$\displaystyle v_{is}^{t+1}=wv_{is}^{t}+c_{1}r_{1s}^{t}({p_{is}^{t}-x_{is}^{t}}% )+c_{2}r_{2s}^{t}({p_{gs}^{t}-x_{is}^{t}})$ (5) $\displaystyle x_{is}^{t+1}=x_{is}^{t}+v_{is}^{t+1}$ (6)

where $\omega$ is the inertia weight, $v_{is}$ is velocity, $x_{is}$ is position, $i\in[{1,m}]$ , $s\in[{1,S}]$ , $c_{1}$ and $c_{2}$ are acceleration factors, $r_{1}$ and $r_{2}$ are independent pseudo-random numbers on the interval [0, 1].

3.3 Genetic algorithm

Genetic Algorithm (GA) is an intelligent computational method, which is designed by the laws of biological evolution and genetic mechanisms of organisms in nature. GA consists of selection, crossover and variation. GA does not need to derive the structural object and limit the continuity of the function. It can perform a series of operations directly on the structural object, so that the better global optimization ability can be obtained.

The core idea of GA is given as follows. First, a parent population is randomly obtained and the fitness value of individual in the parent population is calculated. Then, $n$ individuals with the maximum or minimum fitness are found from all individuals to determine the evolutionary direction of the population by sorting the value of fitness. Thereafter, genetic operation is performed on the chromosomes in the individual to obtain a new progeny population. Finally, the approximate optimal value is obtained when the final stop iteration condition is reached in limited time.

3.4 Relief algorithm

Relief algorithm is a feature weight selection algorithm, which determines the retaining of the feature as the weight of each feature. Generally, it is suitable for big data samples. The Relief algorithm is a feature filtering method by calculating feature weights. Its principle is to assign different weights to each feature on the correlation between every feature and category and delete the feature when the weight of the feature is smaller than the threshold. The Hypothesis-Margin is used to judge the classification precision using feature dimensions. It refers to the distance that the classifier can move without changing the classification results of any sample points, which can be defined as

$\displaystyle\theta=\frac{1}{2}({\|{x_{i}-M(x)}\|-\|{x_{i}-H(x)}\|})$ (7)

where $\theta$ is distance, $H(x)$ is the nearest neighbor sample point of same class and $M(x)$ is the nearest neighbor sample point of different class.

For the Relief algorithm, insufficient parameter features can lead to misclassification of sample categories and redundant parameters can lead to wasted computational effort. To solve this problem, the Relief algorithm associates data features with categories, and selects some features related for classification.

The calculation of the weight for Relief algorithm is

$\displaystyle W_{f}^{i}=W_{f}^{i-1}+\textit{diff}_{f}({x,M(x)})/m-\textit{diff% }_{f}({x,H(x)})/m$ (8)

where $f$ is the feature, $i$ is the randomly selected instance, $m$ is the number of sample, and $\textit{diff}()$ is the distance between samples.

The procedure of Relief algorithm is given as follows.

Step 1: initialize all feature weights of the sample and set it to 0.

Step 2: randomly select a sample $x_{i}$ from the sample set, choose the nearest neighbor sample $H$ from the same sample set of the category $x_{i}$ , and find the nearest neighbor sample $M$ from the different sample sets of the category $x_{i}$ .

Step 3: calculate the distance $\textit{dist}(x_{i},H)$ between $x_{i}$ and $H$ , and the distance $\textit{dist}(x_{i},M)$ between $x_{i}$ and $M$ . The sample feature $x_{i}$ can be distinguished according to judging condition $\textit{dist}(x_{i},H)<dist(x_{i},M)$ .

Step 4: repeat step 2 for $m$ times, and obtain $n$ feature weights.

Step 5: Sort the features according to the weights and select several dimensional features with larger weights.

4. Relief-PGS algorithm

4.1 PSO-GA optimization method

PSO and GA are swarm intelligence algorithms, which are widely used in various optimization problems. When PSO or GA individually optimize the parameters of SVM, it is found that the classification accuracy is not satisfactory and the classification accuracy is instability. For PSO algorithm, the flight speed and direction can be obtained by comparing the optimal position of the particle in the flight history. However, the lack of non-linear factor adjustment causes the problem that it is easy to fall into the local optimal solution in the latter part of the iteration and the global optimal solution cannot be obtained. For GA method, the global search and population diversity [51] is excellent. However, the coding and decoding process increases the complexity of calculation. Meanwhile, the GA needs to complete three operations in a loop, which results in the fact that the computational efficiency and convergence speed of GA algorithm are limited. Therefore, it is necessary to improve the efficiency of optimization process for GA algorithm.

To improve the classification accuracy of SVM, a hybrid PSO-GA optimization method is employed for the parameter selection of SVM. The PSO-GA algorithm uses the PSO algorithm to replace the GA algorithm’s selection operator, which accelerates the convergence speed of the PSO-GA algorithm and ensures the diversity of the GA [52]. The key step in the PSO-GA algorithm is that the process of selecting particles in GA is replaced by PSO, which means that appropriate individual particles have been searched in PSO and the GA employs the crossover and mutation to improve the ability of parameter optimization.

The procedure of PSO-GA algorithm is presented as follows.

Step 1: Initialize all parameters of PSO-GA algorithm, including acceleration factor, weights, number of iterations, etc.

Step 2: Update particle position and velocity in PSO.

Step 3: Calculate the fitness value of each particle and sort the individual as the fitness value.

Step 4: Select $n$ particles in PSO according to the fitness value from high to low and reproduce the $n$ individual particles.

Step 5: Cross and mutate individual particles to obtain the diversity of individual.

Step 6: Synthetic the new particle swarm. Update the optimal individual and group until the maximum number of iterations are reached.

The flowchart of PSO-GA algorithm is shown in Fig. 1.

Figure 1.

PSO-GA algorithm.

Figure 2.

Relief-PGS algorithm.

4.2 A Relief-PGS algorithm

The efficiency and effectiveness of SVM are determined by the selected feature numbers and the parameter of SVM including penalty factor and kernel parameters. The increase of related features can accelerate the classification speed, and appropriate kernel function can improve the ability of SVM to process nonlinear characteristic data of high-dimensional feature space in classification problems. Since the penalty factor and the kernel function are crucial for the classification accuracy and generalization ability of the SVM algorithm the radial basis function (RBF) kernel is selected as kernel function the generalization ability of SVM.

Generally, there is a coupling relationship between the number of input feature subsets and the optimal parameters of SVM. If the number of the input feature subset is changed, it will affect the optimization of the parameters of SVM. If the parameters of SVM is changed, it will also affect the number of the input feature subset. A Relief-PGS algorithm is proposed to reduce the input feature subset and optimize the kernel parameter. In order to speed up training process, the Relief algorithm is used to delete the irrelated features and reduce feature dimension. Meanwhile, the penalty factor and kernel parameters of SVM and the selected feature subsets are encoded into the chromosome of PSO-GA. Since the penalty coefficient C and the kernel parameter $\sigma$ of the SVM are both real-coded, the secondary screening of the feature is performed by binary coding. Meantime, 0 indicates that the feature is not selected and 1 means that the feature is selected. Then, the selected features are input into the SVM model to obtain the classification results and the fitness function is constructed based on the classification accuracy of SVM. In order to obtain the high classification results, PSO algorithm is employed to optimize individuals according to the fitness of the population. The optimized particles are sent into GA to complete the crossover operator. Last, determine whether to stop the iteration based on the fitness of the new population. The proposed algorithm uses Relief algorithm to accelerate the training speed and PSO-GA optimization algorithm to improve the classification accuracy.

The fitness function is uniformly defined as follow:

$\displaystyle\textit{fitness}=we_{1}\times\left({\frac{m_{1}}{\textit{nsv}_{1}% }}\right)+we_{2}\times\left({\frac{m_{2}}{\textit{nsv}_{2}}}\right)$ (9) $\displaystyle we_{1}+we_{2}=1$ (10)

where $we_{1}$ presents the weight of the result obtained after the training sample is identified, $we_{2}$ is the weight of the result obtained after the test sample is identified [53], $m_{1}$ is the total number of training samples, and $m_{2}$ is the total number of test samples, $\textit{nsv}_{1}$ indicates the correct result of the training sample classification and $\textit{nsv}_{2}$ indicates the correct result of the test sample classification. The smaller the fitness, the higher the accuracy of the classification. The weight is $we_{1}=0.15$ in experiments.

The procedure of Relief-PGS algorithm is presented as follows.

Step 1: Perform a feature extraction on the data features using the Relief algorithm to obtain a feature matrix.

Step 2: Initialize the parameters of PSO-GA algorithm including the size of the population and the number of iterations. Set the termination condition of Relief-PGS algorithm as follows:

$\displaystyle\frac{|{\textit{fitness}_{L+1}-\textit{fitness}_{L}}|}{\textit{% fitness}_{L}}<1\%$ (11)

where $L$ is the number of iterations.

Step 3: Generate the first-generation population. The penalty coefficient C and the kernel parameter $\sigma$ the SVM are encoded using real numbers as the first two bits of the individual; the feature matrix is secondarily filtered using binary encoding as the last N bits of the individual.

$\displaystyle\delta=\frac{U_{\max}-U_{\min}}{2^{\lambda}-1}$ (12)

where $\delta$ is the feature vector weight of decimal representation, $U_{\max}$ is the maximum value of eigenvector, $U_{\min}$ is the minimum value of eigenvector, $\lambda$ is the length of gene.

Step 4: Run Step 5 to Step 11 until the termination condition is satisfied, and output the parameters and feature numbers of SVM.

Step 5: Add the first N digits of the individual into the sample for secondary screening of features.

Step 6: Add the last two digits of the individual into the SVM model and determine the SVM classification model considering the training samples of the secondary screening.

Step 7: Add the second screened testing samples into the SVM classification model to obtain the classification result.

Step 8: Calculate the fitness value of the first-generation population.

Step 9: Update the optimal individual and the optimal group.

Step 10: Update the speed and position of the individual according to the fitness of the first-generation population. Add a constraint factor $\chi$ in velocity iteration calculation equation to speed up the convergence of the velocity factor.

$\displaystyle v_{is}^{t+1}=\chi\cdot({wv_{is}^{t}+c_{1}r_{1s}^{t}({p_{is}^{t}-% x_{is}^{t}})+c_{2}r_{2s}^{t}({p_{gs}^{t}-x_{is}^{t}})})$ (13) $\displaystyle x_{is}^{t+1}=x_{is}^{t}+v_{is}^{t+1}$ (14)

Step 11: Add the population updated by the PSO algorithm into GA algorithm Obtain a new population after a crossover and a mutation of GA algorithm, then go to Step 5. The crossover probability of GA algorithm can be defined as

$\displaystyle P_{c}=\frac{M_{c}}{M}$ (15)

where $P_{c}$ is selected in the range 0.4–0.99, $M$ is the total number of all genes, $M_{c}$ is the number of altered genes. The mutation probability of GA algorithm can be calculated by

$\displaystyle P_{m}=\frac{B}{M\cdot\lambda}$ (16)

where $P_{m}$ is set in the range 0.0001–0.1 $B$ is the number of mutant genes.

The flowchart of Relief-PGS algorithm is presented in Fig. 2.

The pseudo code is is illustrated in the following steps.

Algorithm: Relief-PGS
Initialization: Given sample set $D=\{{({x_{i},y_{i}})}\}_{i=1}^{N}$ , set $W_{f}^{i}=0({1\leqslant i\leqslant N})$ , the number of sample $m$ ; the size of the population is 30, the number of iterations is 50, the termination condition of the algorithm is the absolute value of the difference in fitness between the current generation and the previous generation is less than 1% of the fitness of the previous generation.
Step A: Randomly select a pattern $x$ from $D$ ;
Find the nearest hit $H(x)$ and miss $M(x)$ of $x$ ;
for $f=1$ to the number of the feature do
Compute: $W_{f}^{i}=W_{f}^{i-1}+\textit{diff}_{f}({x,M(x)})/m-\textit{diff}_{f}({x,H(x)}% )/m$ ;
end for
Step B: Generate the first-generation population: $\delta=\frac{U_{\max}-U_{\min}}{2^{\lambda}-1}$
while (the iteration stopping condition is satisfied) do:
Step C: Add the last N digits of the individual into the sample for secondary screening of features;
Step D: Add the first two digits of the individual into the SVM model and determine the SVM classification model considering the training samples of the secondary screening;
Step E: Add the second screened testing samples into the SVM classification model to obtain the classification result;
Step F: Calculate the fitness of the generation population: $\displaystyle\textit{fitness}=we_{1}\times\left({\frac{m_{1}}{\textit{nsv}_{1}% }}\right)+we_{2}\times\left({\frac{m_{2}}{\textit{nsv}_{2}}}\right)$ Step G: Update the optimal individual and the optimal group;
Update the speed and position of the individual: $\displaystyle v_{is}^{t+1}=\chi\{{wv_{is}^{t}+c_{1}r_{1s}^{t}({p_{is}^{t}-x_{% is}^{t}})+c_{2}r_{2s}^{t}({g_{is}^{t}-x_{gs}^{t}})}\}$ $\displaystyle x_{is}^{t+1}=x_{is}^{t}+v_{is}^{t+1}$ Step H: Add the population updated by the PSO algorithm into GA algorithm. Crossover and mutation operations are performed on the selected particles to obtain a new population in the form of $\displaystyle P_{c}=\frac{M_{c}}{M}$ $\displaystyle P_{m}=\frac{B}{M\cdot\lambda}$ end while
Return penalty factor ${C}$ and kernel function ${\sigma}$ .

5. Numerical experiments

5.1 Experimental environment and parameter settings

In order to verify the validity of the algorithm, a common UCI database was used in the experiments [54]. The basic information of the experimental data set is illustrated in Table 1. All the classifiers are implemented in MATLAB R2015a on a PC with Inter (R) Core (TM) CPU and 4GB RAM.

Table 1
The setup of the experimental data set

Serial number	Dataset	Sample size	Original features	Classes
1	Balance Scale Weight	690	4	2
2	WPBC	198	33	2
3	Breast-cancer-wisconsin	683	9	2
4	Climate Model	540	18	2
5	Dermatology	366	33	2
6	ILPD	583	9	2
7	Spectf heart	267	44	2
8	Breast	277	13	2
9	Heart	1000	24	2
10	German	303	13	2
11	Voet	435	16	2

To improve the classification accuracy of SVM, the selected data samples were required to be normalized, and then the data were subdivided into training sample sets and test sample sets. In order to compare the classification results of SVM parameters, the initial condition of GA, PSO and PSO-GA are given as: the population size is 30, the length of the population is 2, the iterative times is 50. The crossover probability and mutation probability of GA are adjusted according to the fitness.

5.2 Validation of PSO-SVM, GA-SVM and PSO-GA-SVM algorithm

For PSO-SVM and GA-SVM algorithm, the population size is 30, the length of the population is 2 and the iterative number is 500. The crossover probability and mutation probability of GA are adjusted by the fitness. The weight parameter $\omega$ of PSO algorithm and the crossover probability and mutation probability of GA are determined the fitness. For the PSO algorithm, the acceleration factor is set to be $c_{1}=c_{2}=2$ .

Table 2
The experimental results of PSO-SVM, GA-SVM and PSO-GA-SVM

Serial number	Dataset	Algorithm	Classifiers accuracy	Penalty coefficient	Kernel parameter
1	Balance Scale Weight	PSO-SVM	90.11	476.7258	0.1186
		GA-SVM	86.7	548.3324	0.1831
		PSO-GA-SVM	90.85	554.8213	4.5433
2	WPBC	PSO-SVM	74.0351	365.6563	0.2381
		GA-SVM	69.661	801.0146	0.2338
		PSO-GA-SVM	79.66	361.0533	0.1727
3	Breast-cancer-wisconsin	PSO-SVM	93.23	716.3562	0.0844
		GA-SVM	91.38	818.5383	0.6639
		PSO-GA-SVM	93.33	599.5846	4.4026
4	Climate Model	PSO-SVM	91.67	464.9397	3.2491
		GA-SVM	92.42	418.1111	0.3345
		PSO-GA-SVM	95.45	554.3982	3.89999
5	Dermatology	PSO-SVM	89.01	813.5264	0.1557
		GA-SVM	87.91	409.1771	1.3782
		PSO-GA-SVM	91.21	475.4944	4.1197
6	ILPD	PSO-SVM	72.92	959.8002	6.1929
		GA-SVM	72.92	480.3414	1.6728
		PSO-GA-SVM	74.31	579.5612	3.9008
7	Spectf heart	PSO-SVM	91.07	951.2125	5.3237
		GA-SVM	82.14	575.9592	0.945
		PSO-GA-SVM	92.86	214.5975	1.7027
8	Breast	PSO-SVM	69.88	496.5674	0.6341
		GA-SVM	66.26	941.8178	0.1374
		PSO-GA-SVM	76.88	596.9801	0.8739
9	Heart	PSO-SVM	76.67	710.901	1.4043
		GA-SVM	69.66	190.1246	1.9317
		PSO-GA-SVM	80.67	758.7312	0.6668
10	German	PSO-SVM	78.66	522.6689	0.3907
		GA-SVM	75.55	171.1211	0.2476
		PSO-GA-SVM	82.67	170.4883	0.7397
11	Voet	PSO-SVM	78.441	201.5461	1.2689
		GA-SVM	70.3077	126.4999	0.7720
		PSO-GA-SVM	86.77	568.9606	0.3573

Table 2 illustrates that the classification accuracy of PGS algorithm is higher than that of GA-SVM and PSO-SVM. It is because PSO-GA algorithm combines fast convergence speed of PSO and strong global research ability of GA to optimize the penalty factor and kernel coefficient of the SVM.

5.3 Validation of Relief-PGS algorithm

To verify the validity of the Relief-PGS algorithm, the classification results of the Relief-PGS, SVM, PSO-GA-SVM (PGS), Relief-SVM and ant colony optimization-SVM (ACO-SVM) algorithms are compared. The sample size and the number of feature subsets are shown in Table 3. The iteration number is set to be 500. For PGS-SVM, the PSO-GA algorithm is operated to obtain SVM hyperparameters to improve classification accuracy. The Relief algorithm is used to reduce features with small impact factors to improve the classification. In the ACO-SVM algorithm, the ACO algorithm is used to optimize SVM hyperparameters to improve classification accuracy.

Table 4 and and Fig. 3 compare classification accuracy of different algorithms. It can be observed that the classification accuracy of Relief-PGS is the highest for various experimental datasets, which displays the feasibility and effectiveness of the proposed algorithm. This is because the weights of relevant features are selected by Relief algorithm in the initial stage and the hyperparameters of SVM as particles are encoded into the PSO-GA algorithm for simultaneous optimization of the feature subsets and the parameters of the traditional SVM.

Table 3
The number of optimized feature subsets

Dataset	Sample size	Features
Balance Scale Weight	690	3
WPBC	198	18
Breast-cancer-wisconsin	683	6
Climate Model	540	12
Dermatology	366	23
ILPD	583	6
Spectf heart	267	30
Breast	277	9
Heart	1000	18
German	303	10
Voet	435	14

Table 4

Classification accuracy of different algorithms

Serial number	Dataset	SVM (%)	PGS (%)	ACO-SVM (%)	Relief-SVM (%)	Relief-PGS (%)
1	Balance Scale Weight	49.76	90.85	87.62	88.24	92.958
2	WPBC	67.8866	79.661	78.5344	83.2611	85.8841
3	Breast-cancer-wisconsin	94.12	93.33	94.21	94.12	94.828
4	Climate Model	92.59	95.45	93.5804	87.04	95.667
5	Dermatology	76.14	91.21	86.1514	76.15	91.209
6	ILPD	69.54	74.31	73.4574	59.38	76.667
7	Spectf heart	78.75	92.86	85.319	71.25	92.857
8	Breast	64.2122	76.8821	72.8468	78.0122	81.2168
9	Heart	68.1337	80.6742	79.8291	83.5823	86.6466
10	German	71.532	82.6788	81.7043	78.8832	87.6843
11	Voet	68.111	86.77	83.5812	84.9812	88.8433
Average accuracy		72.7978	85.8796	83.3485	80.4455	88.5873

The training time of different algorithms for different datasets is compared in Table 5. As can be seen from Table 4, the average classification accuracy of parameters optimization based on the traditional SVM is 72.7987%. The average classification accuracy using the PGS optimization algorithm is 13.08% better than that of traditional SVM. The average classification accuracy using Relief-SVM optimization algorithm is 7.65% higher than that of traditional SVM. The highest classification accuracy is 88.5873%, which is obtained by the Relief-PGS algorithm. Compared with traditional SVM, the training time of Relief-SVM is reduced due to the selection of relevant feature subset by Relief algorithm. Compared with PGS and ACO-SVM, the Relief-PGS algorithm shortens the training time and speed up iteration. The training time of traditional SVM and Relief-SVM is superior to that of Relief-PGS, which owes to the multiple optimization of PSO-GA algorithm.

Table 5

The training time of different algorithms (unit: second)

Serial number	Dataset	SVM	PGS	ACO-SVM	Relief-SVM	Relief-PGS
1	Balance Scale Weight	3.152	10.685	12.268	2.956	8.256
2	WPBC	1.072	4.300	5.105	0.985	3.768
3	Breast-cancer	3.655	13.684	15.433	3.363	12.635
4	Climate Model	2.894	11.265	13.785	2.659	9.865
5	Dermatology	2.689	8.625	10.025	2.036	8.286
6	ILPD	3.125	12.511	14.885	1.812	10.982
7	Spectf heart	1.415	5.269	6.815	1.315	5.032
8	Breast	1.458	5.426	6.998	1.566	5.236
9	Heart	5.230	19.203	19.388	4.223	16.248
10	German	1.624	6.514	6.889	1.492	5.705
11	Voet	2.332	9.352	9.103	2.166	7.188

Figure 3.

Classification accuracy of different algorithms.

Figure 4.

Optimal individual fitness of different algorithms in training process.

The optimal individual fitness of ACO-SVM, GA-SVM, PSO-SVM, PGS and Relief-PGS algorithms in the training process are shown in Fig. 4. The iteration number is 250, the ACO converges. The iteration number is 200, the GA converges. The number of iterations is 175, the PSO converges. It can be seen that the convergence rate of PSO is faster than GA and ACO in the early stage of algorithm iteration. The number of iterations is 100, the Relief-PGS algorithm converges. It can be concluded that the convergence rate of Relief-PGS algorithm is faster and the searched optimal solution is better than the other four algorithms.

5.4 Algorithm complexity analysis

Relief algorithm shortens the time of data selection process by decreasing the quantity of irrelevant features. The time complexity of the Relief algorithm to select features is $O(\log_{2}^{N})$ , where $N$ is the total number of sample collection. The time complexity of the PSO algorithm is $T_{1}=O({M\times n\times T_{t}})$ , $M$ is the particles number, $n$ is the iteration time and $T_{t}$ is the time required for one iteration. The time complexity of the GA algorithm is $T_{2}=O({K\times m\times T_{m}})$ , where $K$ is the number of particles which is selected from the PSO algorithm with larger fitness value, and $K<<M$ . Then, the selected particles in PSO directly replace the selected operator in GA, $K$ is the number of chromosomes, $T_{m}$ represents the time required for one iteration. The time complexity of the SVM algorithm is $O({N_{S}^{3}})$ , where $N_{S}$ is the number of support vectors. The time complexity of the entire training process for Relief-PGS algorithm is $T=O({\log_{2}^{N}})+O({T_{1}\times T_{2}})+O({N_{S}^{3}})$ .

6. Conclusions

A Relief-PGS algorithm is put forword based on the feature selection of Relief algorithm and parameter optimization of SVM to select the input feature subset and kernel parameter, which encodes the feature subsets and the parameters of SVM into the PSO-GA as chromosome. The feature subset and the parameters of the SVM are optimized synchronously. Relief-PGS algorithm combines Relief algorithm, PSO-GA algorithm and SVM for feature selection and classification of datasets. The proposed algorithm of feature selection and parameter optimization can select the optimal feature subset and improve the classification accuracy by obtaining the optimal parameters and achieving better classification performance. Numerical experiments indicates that Relief-PGS algorithm obtains the better classification precision than other algorithms including traditional SVM, PSO-GA-SVM, ACO-SVM and Relief-SVM. It is promising that the proposed Relief-PGS algorithm can deal with the problem of complicated data classification, such as image detection, signal identification, etc.

Footnotes

Acknowledgments

This paper is supported by National Natural Science Foundation of China (Grant No. 51875457, 52105274), the Key Research and Development Program of Shaanxi Province of China (2022SF-259), and Xi’an science and technology plan project (22GXFW0128).

References

Xie

Z.X.

and Hu

Q.H.

, Uncertain data classification with additive kernel support vector machine, Data & Knowledge Engineering 117(9) (2018), 87–97.

Liu

Wen

and Gao

, SVM based multi-label learning with missing labels for image annotation, Pattern Recognition 126(1) (2018), 586–595.

Rojo-Álvarez

J.L.

et al., A unified SVM framework for signal estimation, Digital Signal Processing 26(2) (2014), 1–20.

Bhuvaneswari

, Novel object detection and recognition system based on points of interest selection and SVM classification, Cognitive Systems Research 52(5) (2018), 985–994.

Liu

Guo

et al., Meteorological pattern analysis assisted daily PM2.5 grades prediction using SVM optimized by PSO algorithm, Atmospheric Pollution Research 10(9) (2019), 1482–149.

Grama

Tuns

and Rusu

, On the Optimization of SVM Kernel Parameters for Improving Audio Classification Accuracy, in: International Conference on Engineering of Modern Electric Systems, EMES 2017(14th), pp. 224–227.

Ning

Zhang

and Zhang

, A best-path-updating information-guided ant colony optimization algorithm, Information Sciences 433/434(4) (2018), 142–162.

C.S.

X.L.

and Li

R.H.

, A chaos embedded GSA-SVM hybrid system for classification, Neural Comput & Applic 26(7) (2015), 713–721.

Zhang

P.Y.

Shu

and Zhou

M.C.

, An Online Fault Detection Model and Strategies Based on SVM-Grid in Clouds, IEEE/CAA Journal of Automatica Sinica 5(02) (2018), 60–71.

10.

Wang

Shao

Y.H.

Bai

C.N.

Liu

L.M.

and Deng

N.Y.

, Insensitive stochastic gradient twin support vector machines for large scale problems, Information Sciences 462(9) (2018), 114–131.

11.

and Cui

, Digital image recognition based on Fractional-order-PCA-SVM coupling algorithm, Measurement 145(2) (2019), 150–159.

12.

Wang

and Wu

J.J.

, Using GA-SVM for defect inspection of flip chips based on vibration signals, Microelectronics Reliability 81(2) (2018), 159–166.

13.

Yan

et al., A particle swarm optimization-based flexible convolutional autoencoder for image classification, IEEE Transactions on Neural Networks and Learning Systems 30(8) 2018, pp. 2295–2309.

14.

and Fan

T.H.

, Object tracking with improved firefly algorithm, International Journal of Computing Science & Mathematics Ijcsm 3(9) (2018), 219–231.

15.

Zheng

and Wang

, A novel hybrid algorithm for feature selection based on whale optimization algorithm, IEEE Access 7(11) (2019), 14908–14923.

16.

Akinyelu

A.A.

Ezugwu

A.E.

and Adewumi

A.O.

, Ant colony optimization edge selection for support vector machine speed optimization, Neural Computing and Applications 4(1) (2019), 1–33.

17.

Chen

and Wang

, Rapid and efficient screening of human papillomavirus by Raman spectroscopy based on GA-SVM, Optik 210(1) (2020), 164514164524.

18.

Zhang

W.J.

Wen

et al., Estimating PM2.5 concentration using the machine learning GA-SVM method to improve the land use regression model in Shaanxi, Ecotoxicology and Environmental Safety 225(12) (2021), 112772–112772.

19.

Z.J.

Dong

Y.R.

Liu

H.L.

et al., Method of Forecasting Non-Equal Interval Track Irregularity Based on Improved Grey Model and PSO-SVM, IEEE Access 6(6) (2018), 34812–34818.

20.

Fan

Q.C.

and Xuan

, Transformer fault diagnosis method based on improved whale optimization algorithm to optimize support vector machine, Energy Reports 7(Supplement 7) (2021), 856–866.

21.

Moradi

Vosoughi

A.R.

and Anjabin

, Maximum buckling load of stiffened laminated composite panel by an improved hybrid PSO-GA optimization technique, Thin-Walled Structures 160(3) (2021), 107382–107393.

22.

Sanaj

M.S.

and Prathap

P.M.J.

, An efficient approach to the map-reduce framework and genetic algorithm based whale optimization algorithm for task scheduling in cloud computing environment, Materials Today: Proceedings 37(2) (2021), 3199–3208.

23.

Hammid

A.T.

and BinSulaiman

M.H.

, Series division method based on PSO and FA to optimize Long-Term Hydro Generation Scheduling, Sustainable Energy Technologies and Assessments 29(10) (2018), 106–118.

24.

Liu

Zhao

et al., An integrated building energy performance evaluation method: From parametric modeling to GA-NN based energy consumption prediction modeling, Journal of Building Engineering 45(1) (2022), 103571–103581.

25.

Liu

Y.Y.

Dai

J.J.

Zhao

S.S.

et al., Optimization of five-parameter BRDF model based on hybrid GAPSO algorithm, OptikInternational Journal for Light and Electron Optics 219(10) (2020), 164978–164984.

26.

Huang

W.C.

Liu

H.Y.

Zhang

et al., Railway dangerous goods transportation system risk identification: Comparisons among SVM, PSO-SVM, GA-SVM and GS-SVM, Applied Soft Computing 109(9) (2021), 107541–107542.

27.

Cui

Chen

Q.G.

Y.X.

and Tang

, A new model of flavonoids affinity towards P-glycoprotein: Genetic algorithm-support vector machine with features selected by a modified particle swarm optimization algorithm, Archives of Pharmacal Research 40(12) (2017), 214–230.

28.

Bonah

Huang

X.Y.

et al., Vis-NIR hyperspectral imaging for the classification of bacterial foodborne pathogens based on pixel-wise analysis and a novel CARS-PSO-SVM model, Infrared Physics & Technology 105(3) (2020), 103220–103228.

29.

Hoseininejad

F.S.

Forghani

and Ehsani

, A fast algorithm for local feature selection in data classification, Expert Systems 38(6) (2019), 1217–1227.

30.

Urbanowicz

R.J.

Meeker

et al., Relief-based feature selection: Introduction and review, Journal of Biomedical Informatics 85(9) (2018), 189–203.

31.

Toğaçar

Ergen

and Cömert

, Classification of flower species by using features extracted from the intersection of feature selection methods in convolutional neural network models, Measurement 158(1) (2020), 107703–107715.

32.

Abut

Akay

M.F.

and George

, Developing new VO2max prediction models from maximal, submaximal and questionnaire variables using support vector machines combined with feature selection, Computers in Biology and Medicine 79(12) (2016), 182–192.

33.

Gunduz

, An efficient dimensionality reduction method using filter-based feature selection and variational autoencoders on Parkinson’s disease classification, Biomedical Signal Processing and Control 66(4) (2021), 102452–102462.

34.

Shen

Zhang

J.K.

and Zhang

Z.G.

, Support vector machine based on analysis of factors influencing medical expenses in single disease, Health Econ. 31(1) (2012), 89–91.

35.

Zhan

L.J.

Liu

H.R.

et al., Application of the support vector machine model in the analysis of impact factors for hospitalization expenses, Hosp 18(1) (2014), 30–32.

36.

Gauthama Raman

M.R.

et al., An efficient intrusion detection system based on hypergraph – Genetic algorithm for parameter optimization and feature selection in support vector machine, Knowle dge-Base d Systems 134(15) (2017), 1–12.

37.

Saiparvathi

and Swetha

V.V.K.

, Impact of using Backend Genetic Algorithm to Optimize Parameters with the use of Support Vector, International Journal of Engineering Research & Technology (IJERT) 9(5) (2020), 1254–1258.

38.

Tharwat

et al., Quantum-behaved particle swarm optimization for parameter optimization of support vector machine, Journal of Classification 36(3) (2019), 576–598.

39.

Hamid

T.M.T.A.

et al., Ensemble based filter feature selection with harmonize particle swarm optimization and support vector machine for optimal cancer classification, Machine Learning with Applications 5(15) (2021), 100054.

40.

Vieira

S.M.

Mendonc

L.F.

et al., Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients, Applied Soft Computing 13(8) (2013), 3494–3504.

41.

Qaraad

Amjad

et al., A hybrid feature selection optimization model for high dimension data classification, IEEE Access 9(7) (2021), 42884–42895.

42.

Zhang

X.L.

Chen

Wang

B.J.

et al., Intelligent fault diagnosis of rotating machinery using support vector machine with ant colony algorithm for synchronous feature selection and parameter optimization, Neurocomputing 167(11) (2015), 260–279.

43.

Moukhafi

et al., A novel hybrid GA and SVM with PSO feature selection for intrusion detection system, International Journal of Advances in Scientific Research and Engineering 4(5) (2018), 129–134.

44.

Dinesh

M.G.

and Prabha

, Diabetes Mellitus Prediction System Using Hybrid KPCA-GA-SVM Feature Selection Techniques, Journal of Physics Conference Series 1767(1) (2020), 012001–012010.

45.

and Qiu

, An intelligent SVM modeling process for crude oil properties prediction based on a hybrid GA-PSO method, Journal of Chemical Engineering 27(1) (2019), 1888–1894.

46.

Wang

X.Y.

Liang

L.L.

W.Y.

D.M.

and Yang

H.Y.

, A new SVM-based relevance feedback image retrieval using probabilistic feature and weighted kernel function, Journal of Visual Communication and Image Representation 38(6) (2016), 256–275.

47.

Zhang

et al., Relief feature selection and parameter optimization for support vector machine based on mixed kernel function, International Journal of Performability Engineering 14(2) (2018), 280–289.

48.

Choi

Y.C.

Murtala

et al., Relief Extraction From a Rough Stele Surface Using SVM-Based Relief Segment Selection, IEEE Access 9(12) (2020), 4973–4982.

49.

Dou

D.Y.

W.Z.

Yang

J.G.

and Zhang

, Classification of coal and gangue under multiple surface conditions via machine vision and relief-SVM, Powder Technology 356(11) (2019), 1024–1028.

50.

Zhang

Ogren

R.M.

and Kong

S.C.

, A comparative study of biodiesel engine performance optimization using enhanced hybrid PSO-GA and basic GA, Applied Energy 165(3) (2016), 676–684.

51.

Wang

Cai

and Wang

, Optimization of a hybrid ejector air conditioning system with PSOGA, Applied Thermal Engineering 112(2) (2017), 1474–1486.

52.

Zhai

Liu

H.T.

Yang

Y.P.

and Wu

, Optimization of a heliostat field layout using hybrid PSO-GA algorithm, Applied Thermal Engineering 128(1) (2018), 33–41.

53.

Zhao

Wang

and Yong

, GA-SVM based feature selection and parameter optimization in hospitalization expense modeling, Applied Soft Computing 75(2) (2019), 323–332.

54.

UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences, http://archive.ics.uci.edu/ml/index.php, 2007.

A Relief-PGS algorithm for feature selection and data classification

Abstract

Keywords

1. Introduction

2. Related works

3. Preliminaries

3.1 SVM model

3.4 Relief algorithm

4.1 PSO-GA optimization method

5.1 Experimental environment and parameter settings

Table 1 The setup of the experimental data set

Table 2 The experimental results of PSO-SVM, GA-SVM and PSO-GA-SVM

Table 3 The number of optimized feature subsets

6. Conclusions

Footnotes

Acknowledgments

References

Table 1
The setup of the experimental data set

Table 2
The experimental results of PSO-SVM, GA-SVM and PSO-GA-SVM

Table 3
The number of optimized feature subsets