Classification of high-dimensional imbalanced biomedical data based on spectral clustering SMOTE and marine predators algorithm

Abstract

The research of biomedical data is crucial for disease diagnosis, health management, and medicine development. However, biomedical data are usually characterized by high dimensionality and class imbalance, which increase computational cost and affect the classification performance of minority class, making accurate classification difficult. In this paper, we propose a biomedical data classification method based on feature selection and data resampling. First, use the minimal-redundancy maximal-relevance (mRMR) method to select biomedical data features, reduce the feature dimension, reduce the computational cost, and improve the generalization ability; then, a new SMOTE oversampling method (Spectral-SMOTE) is proposed, which solves the noise sensitivity problem of SMOTE by an improved spectral clustering method; finally, the marine predators algorithm is improved using piecewise linear chaotic maps and random opposition-based learning strategy to improve the algorithm’s optimization seeking ability and convergence speed, and the key parameters of the spectral-SMOTE are optimized using the improved marine predators algorithm, which effectively improves the performance of the over-sampling approach. In this paper, five real biomedical datasets are selected to test and evaluate the proposed method using four classifiers, and three evaluation metrics are used to compare with seven data resampling methods. The experimental results show that the method effectively improves the classification performance of biomedical data. Statistical test results also show that the proposed PRMPA-Spectral-SMOTE method outperforms other data resampling methods.

Keywords

Biomedical data mRMR spectral clustering SMOTE marine predators algorithm

1 Introduction

In recent years, with the continuous progress and development of science and technology, the quantity and quality of biomedical data have been significantly improved. Effective analysis of biomedical data can help researchers better understand the mechanism of disease occurrence and help doctors more accurately diagnose and treat diseases, so how to analyze biomedical data scientifically and accurately has become an urgent problem. The complexity of biomedical data makes it impossible to analyze accurately by traditional methods, and machine learning and data mining have become important means of biomedical data analysis [1].

In biomedical data, the dimensionality of features is usually much higher than the number of samples [2], and the dimensionality of features may reach thousands or even tens of thousands, while the number of samples is usually small, which will lead to the “curse of dimensionality” problem [3, 4]. The “curse of dimensionality” problem leads to a decrease in the predictive performance of machine learning algorithms and an increase in the risk of algorithm overfitting [5]. Feature selection can remove redundant and noisy information from high-dimensional data, select the most relevant features, improve accuracy, reduce computational cost, and avoid overfitting, so feature selection has become an important method for biomedical data research [6, 7].

In addition to the high-dimensional problem, biomedical data usually suffers from the class imbalance problem, for example, when biomedical data is utilized for disease diagnosis, the number of diseased and healthy samples may differ greatly, and the distribution of categories shows an extreme imbalance [4]. Class imbalance can lead to classification algorithms that are more biased toward samples of the majority class and have a reduced classification performance for samples of the minority class, while in real life accurate identification of the minority class is often the goal that needs to be achieved [8]. Data resampling is an effective method to solve class imbalance by processing imbalanced data into balanced data through oversampling or undersampling methods [9]. Oversampling mainly achieves data balancing by increasing the minority class samples, and the most popular oversampling method is Synthetic Minority Over-sampling Technique (SMOTE), which generates new samples based on the minority class samples and their nearest neighbor samples [10]. Undersampling methods mainly balance the samples by reducing the number of majority class samples, Edited Nearest Neighbours [11] and Tomek Links [12] are classical undersampling methods. In previous studies of biomedical and bioinformatics data, most of the studies only consider the high-dimensional problem and often ignore the class imbalance problem [13]. The undersampling method will make information lost, which is not suitable for small-sample data research and generally not applicable in biomedical data, so this paper focuses on the application of SMOTE oversampling method in biomedical data. SMOTE method has also been widely used in biomedical data, Nakamura et al. [14] used learning vector quantization to improve the code set obtained by SMOTE method. Li et al. [15] proposed an oversampling method called adaptive swarm cluster dynamic multi-objective SMOTE for solving the problem of biomedical data class imbalance. Xu et al. [16] proposed a hybrid sampling method combining misclassification-oriented SMOTE and edited nearset neighbor based on Random forest for biomedical data.

In order to improve the performance of biomedical data classification, this paper takes into account the high dimensionality and class imbalance characteristics of biomedical data, and combines feature selection with data resampling methods to construct a hybrid classification model for biomedical data, with the following main innovations:

Feature selection of biomedical data using minimal-redundancy maximal-relevance method to solve the high dimensionality problem of biomedical data, reduce the computational cost;

Multi-kernel learning is introduced into spectral clustering to improve the clustering performance, and improvement of SMOTE methods using multi-kernel spectral clustering to address noise sensitivity, improve the quality of SMOTE-generated samples, and address class imbalance in biomedical data;

Marine predators algorithm combining piecewise linear chaotic maps and random opposition-based learning strategy (PRMPA) is proposed to avoid the algorithm from falling into a local optimum and to improve the algorithm’s optimization searching ability, and PRMPA is utilized to adaptively select the key parameters of the spectral-SMOTE, which further improves the classification performance of biomedical data;

Experiments are carried out on five real biomedical datasets to validate the performance of the proposed model.

The rest of the paper is organized as follows, Section 2 presents the related work, Section 3 describes the proposed model in detail, Section 4 evaluates the performance of the proposed model with respect to other comparative models using five real biomedical datasets and gives the results and analysis, and Section 5 concludes the paper.

2 Related work

In this paper, we mainly apply feature selection and SMOTE methods to the problem of high dimensionality and class imbalance in biomedical data, so this section will briefly summarize the above two aspects.

2.1 Feature selection

Through feature selection, redundant and unnecessary features can be eliminated, thus improving the learning ability and generalization of the model. Feature selection can be divided into three main categories: filter, wrapper and embedded [17]. Filter methods mainly evaluate the relevance of features based on their intrinsic properties [18], and features are evaluated based on variance, correlation coefficient, statistical tests, and information theory, whereas evaluation criteria based on information theory are more common. Battiti uses the mutual information criterion to evaluate candidate features [19], and since univariate filter methods only consider the correlation between but a single feature variable and the target variable. The univariate filtering approach only considers the correlation between a single feature variable and the target variable, and does not fully consider the correlation between features, which leads to the possibility of large redundancy among the subset of selected features. Peng et al. [20] proposed minimal-redundancy maximal-relevance (mRMR), which takes into account the redundancy among the features when performing feature selection, and effectively improves the performance of feature selection. Van et al. [21] applied a threshold-based feature selection method on bioinformatics data, which effectively improved the performance of feature selection. Lyu et al. [22] proposed a filter feature selection method based on improved maximum information coefficients (MICs) and used it for biomedical data. El-Manzalawy et al. [23] extended mRMR by proposing a multi-view mRMR to predict the survival time of ovarian cancer. Xiong et al. [24] used uncertainty indices to assess the correlation between features and class labels, removed redundant features. Wang et al. [25] used mutual information to screen gastric cancer genes to improve classification accuracy. Wrapper methods evaluate a selected subset of features by training a learning model [26]. Correa et al. [27] used discrete particle swarm optimization algorithm for bioinformatics dataset to maximize classification accuracy while minimizing the number of features. Li et al. [28] proposed a binary Wolf Search Algorithm with an elite mechanism, which effectively reduces the bioinformatics data The computation time of feature selection. Mafarja et al. [29] introduced multiple strategies into the grey wolf optimization to solve the problem of premature convergence of the algorithm and enhance the local search ability, and used the multi-strategy grey wolf optimization in feature selection of bioinformatics data. The embedded method uses the ability of learning algorithm itself for feature selection [30]. Nie et al. [31] introduced L1, L2 norms into machine learning algorithms to realize an efficient and robust feature selection method, which effectively eliminates noisy features in bioinformatics data. Guo et al. [32] used the embedded feature selection method to process multilabel bioinformatics data, and the results showed that the proposed algorithm has a more significant advantages than other algorithms. Wrapper and embedded are both related to learning algorithms, which usually can obtain higher accuracy than filter, but also lead to higher computational cost, while filter methods are independent of the model, which can effectively avoid the overfitting problem and have lower computational cost, so filter feature selection methods are more widely used in biomedical fields [22]. This paper uses the minimal-redundancy maximal-relevance methods to select features for biomedical data. The mRMR method not only reduces the redundancy between features during the feature selection process, but also selects the features that are most relevant to the target variable, which can be effective reduce the dimensionality of high-dimensional biomedical data, retain important features, reduce the “curse of dimensionality” in the disease classification process, and reduce computational complexity, thus facilitating subsequent research.

2.2 SMOTE

SMOTE [10] is the most classical oversampling method, which is widely used in the class imbalance problem, but it still has many problems, such as easy to introduce noise, can’t deal with sample overlap. Scholars have suggested many enhancement strategies to address the issues associated with SMOTE. Han et al. [33] proposed the Borderline-SMOTE, which synthesizes the new samples by using only a few class samples on the boundary samples, thus improving the distribution of samples. Bunkhumpornpat et al. [34] proposed Safe-Level-SMOTE method, which synthesizes new samples at a larger safety level based on the safety level of the minority class samples. Nguyen et al. [35] proposed SVM-based SMOTE method, which utilizes support vectors to generate new samples and effectively solves the problem of identifying boundary samples. He et al. [36] proposed the ADASYN method, which assigns different weights to different minority class samples and reduces the influence of noisy data. Maciejewski and Stefanowski [37] taking into account the local information of the minority class samples, proposed the LN-SMOTE, which effectively improves the quality of generated samples. S $\overset{´}{a}$ ez et al. [38] improved SMOTE based on the iterative integrated noise filtering method, which effectively overcomes the problem of noise and boundary sample generation. Ma et al. [39] proposed a clustering-based SMOTE method, which clusters a small number of classes of samples and removes noise and outliers, effectively avoiding noise generation. Douzas et al. [40] proposed an oversampling method based on K-means and SMOTE, which avoids introducing noise and generates new samples with high diversity. Although K-means SMOTE overcomes the problems of traditional SMOTE methods, the clustering performance of K-means clustering on high-dimensional sparse data decreases. Jadwal et al. [41] utilized spectral clustering as a preprocessing technique and employed SMOTE for oversampling avoid the noise problem. In spectral clustering, the construction of similarity matrix has a greater impact on the performance of the model. However, this method does not fully address the challenge of constructing a similarity matrix in spectral clustering. Moreover, there remain several hyperparameters in this method, and determining the optimal values for these hyperparameters remains an unresolved concern. Blagus and Lusa [42] conducted an in-depth study, and the results showed that the performance of SMOTE on high-dimensional data is significantly inferior to that on low-dimensional data. Maldonado et al. [43] proposed a new distance metric for calculating the neighbors of a few classes of samples for the problem of classifying high-dimensional imbalanced data. Shanab and Khoshgoftaar [13] deeply investigated the biomedical dataset class imbalance problem on classification performance and showed that models using both feature selection and data resampling perform better than models using only feature selection. Pes [4] used feature selection in conjunction with sampling-based methods and cost-sensitive categorization to delve into the most effective strategies for solving high-dimensional imbalanced biomedical data. Although the problem of class imbalance significantly affects the classification performance of biomedical data, there is still very little research on this topic.

In view of the high-dimensional and imbalanced characteristics of biomedical data, which makes it difficult for traditional methods to accurately classify, it is proposed that a hybrid method based on feature selection and oversampling. First, mRMR method is utilized to select features for biomedical data to reduce redundant features, improve generalization ability and reduce computational cost; then, for the problem that K-means SMOTE cannot handle high-dimensional sparse data efficiently, the multi-kernel spectral clustering method is used to improve SMOTE, improve the oversampling accuracy, and solve the noise-sensitive problem effectively; finally, the improved marine predators algorithm is used to optimize the selection of the key parameters of Spectral-SMOTE, which effectively improves the classification performance of biomedical data.

3 Methods

In this section, we will briefly describe the main methods. The mRMR feature selection method is introduced in Section 3.1. Multi-kernel spectral clustering and SMOTE methods are introduced in Sections 3.2 and 3.3 respectively. Section 3.4 describes the improved marine predators algorithm. Finally, the overall framework of the method proposed in this paper is described in detail in Section 3.5.

3.1 mRMR

The mRMR [20] is a filter feature selection method based on mutual information, whose main objective is to find a subset of features that have the highest correlation with the target variable while minimizing the redundancy among the features. The measure of correlation is mainly done through mutual information, which can be used to represent the common parts between two variables and measure the correlation between two variables. Given two random variables y and z, with probability density functions p (y) and p (z), respectively, p (y, z) is the joint probability density function of y and z, the mutual information of s y and z can be expressed as Equation (1): $I (y; z) = \iint p (y, z) log \frac{p (y, z)}{p (y) p (z)} dydz$ (1)

Minimum redundancy and maximum relevance are calculated by Equations (2) and (3), respectively: $\begin{matrix} min R (S), & R = \frac{1}{{| S |}^{2}} \end{matrix} \sum_{x_{i}, x_{j} \in S} I (x_{i}, x_{j})$ (2) $\begin{matrix} max D (S, c), & D = \frac{1}{| S |} \end{matrix} \sum_{x_{i} \in S} I (x_{i}; c)$ (3) where S represents the feature set, c represents the target variable, and |S| is the dimension of the feature. I (x_i, x_j) is the mutual information that measures the correlation between the features x_i and x_j, I (x_i, c) is the mutual information that measures the correlation between the features x_i and the target variable c. mRMR algorithms usually consider a simple form such as Equation (4) as an evaluation criterion for a subset of features: $\begin{matrix} max Φ (D, R) & Φ = D - R \end{matrix}$ (4) where Φ (D, R) is optimized with both D and R in mind.

3.2 Spectral clustering based on multi-kernel learning

Spectral clustering [44] is a clustering method based on graph theory, mainly by clustering the eigenvectors of the Laplace matrix of the original data. Spectral clustering is more adaptable to data distribution and can converge to the global optimal solution, so it is widely used. Spectral clustering first calculates the similarity between the original data and constructs the similarity matrix using the similarity; then constructs the corresponding Laplace matrix based on the similarity matrix and finds the eigenvectors corresponding to the first eigenvalues of the Laplace matrix to form a new solution space; finally, clustering is performed in the new solution space.

Spectral clustering only needs to calculate the similarity matrix between data, which is more conducive to dealing with sparse data than traditional clustering methods such as K-means, but its clustering effect is also highly dependent on the similarity matrix. The most common way to generate similarity matrix is to use RBF kernel function, but RBF kernel is a classic local kernel function with strong learning ability but weak generalization ability. Therefore, multi-kernel learning is used in the construction of similarity matrix in this paper. The Sigmoid kernel is a commonly used global kernel function known for its strong generalization ability but limited learning ability. By combining the Sigmoid kernel and the RBF kernel, a multi-kernel function can be constructed to enhance both learning ability and generalization ability [45]. The proposed multi-kernel function is represented by Equation (5): $κ_{mul} = w κ_{RBF} + (1 - w) κ_{Sigmoid}$ (5) Where w is the weight of the kernel function, κ_RBF and κ_Sigmoid are the RBF kernel function and Sigmoid kernel function, respectively, with the expressions shown in Equations (6) and (7): $κ_{RBF} = \exp (\frac{- {∥ x_{i} - x_{j} ∥}^{2}}{2 σ^{2}})$ (6) $κ_{Sigmoid} = \tanh ({ax}_{i}^{T} \cdot x_{j} + c)$ (7) where σ and a represent the kernel parameters of the RBF kernel and Sigmoid kernel, respectively.

3.3 SMOTE

In imbalanced data classification, the majority class refers to the class with a larger number of samples in the data set, while the minority class refers to the class with a smaller number of samples. SMOTE [10] is a classical oversampling method to solve the class imbalance problem, SMOTE is an improvement for the random oversampling method, random oversampling is only a simple replication of the minority class samples, which can easily lead to the model overfitting problem, whereas SMOTE effectively reduces the overfitting problem of the model by generating new minority class samples, and thus is widely used in imbalanced data. The core idea of SMOTE involves creating synthetic samples through linear interpolation between existing minority class samples and their neighboring samples of the same class., which has the following two main steps:

Given a minority sample x, compute its distance from other minority samples and find the k nearest neighbor of x;

In the near-neighborhood samples randomly selected a sample x_k, in the minority class samples x and x_k between the linear interpolation, constructed to generate a new minority class samples, the specific sample generation formula as shown in Equation (8): $x_{new} = x + rand (0, 1) \times (x_{k} - x)$ (8) where x_new is a newly generated minority class sample and rand (0, 1) is a random number between (0,1).

3.4 Improved marine predators algorithm

3.4.1 Marine predators algorithm (MPA)

Marine predators algorithm (MPA) [46] is a new swarm intelligence optimization algorithm proposed by Faramarzi et al. in 2020, which mainly simulates the process of survival of the fittest organisms in the ocean. The population is first initialized as shown below: $X_{ij} = X_{\min} + rand (X_{\max} - X_{\min})$ (9) where X_min, X_max is the range of the search space and rand is a random number between [0,1]. Then the elite matrix and the corresponding prey matrix are constructed according to the survival of the fittest theory. $Elite = {[\begin{matrix} X_{1, 1}^{I} & X_{1, 2}^{I} & \dots & X_{1, d}^{I} \\ X_{2, 1}^{I} & X_{2, 2}^{I} & \dots & X_{2, d}^{I} \\ ⋮ & ⋮ & ⋮ & ⋮ \\ X_{n, 1}^{I} & X_{n, 2}^{I} & \dots & X_{n, d}^{I} \end{matrix}]}_{n \times d}$ (10) $Prey = {[\begin{matrix} X_{1, 1} & X_{1, 2} & \dots & X_{1, d} \\ X_{2, 1} & X_{2, 2} & \dots & X_{2, d} \\ ⋮ & ⋮ & ⋮ & ⋮ \\ X_{n, 1} & X_{n, 2} & \dots & X_{n, d} \end{matrix}]}_{n \times d}$ (11)

The Elite matrix is constructed by replicating the top predator vector X^I times n, n and d are the number and dimensions of search agents, respectively. The MPA optimization process is divided into three main phases, where different speed ratios are considered in the simulation of the whole predator and prey life. The MPA optimization process is divided into three main steps as shown below:

At the beginning of the iteration, MPA mainly realizes the global exploration process of the search space, when the optimal selection strategy of the predator is to remain motionless at the original position: $\begin{matrix} I t e r < \frac{1}{3} M a x_I t e r \\ {\begin{matrix} s_{i} = R_{B} \otimes (E l i t e_{i} - R_{B} \otimes P r e y_{i}) \\ P r e y_{i} = P r e y_{i} + P • R \otimes s_{i} \end{matrix} i = 1, 2, \dots, n . \end{matrix}$ (12) where s_i is the moving step size, R_B denotes a normally distributed random vector based on Brownian motion, P = 0.5 is a constant number, and R is a vector of random numbers in [0,1], Iter is the current iteration number, and Max _ Iter is the maximum iteration number. In the middle of the iteration, when both prey and predator are in motion, the MPA shifts from the exploration phase to the exploitation phase: $\begin{matrix} \frac{1}{3} M ax_I t e r < I t e r < \frac{2}{3} M a x_I t e r \\ {\begin{matrix} s_{i} = R_{L} \otimes (E l i t e_{i} - R_{L} \otimes P r e y_{i}) \\ P r e y_{i} = P r e y_{i} + P • R \otimes s_{i} \end{matrix} i = 1, \dots, 2 / n . \end{matrix}$ (13) ${\begin{matrix} s_{i} = R_{B} \otimes (R_{B} \otimes E l i t e_{i} - P r e y_{i}) \\ P r e y_{i} = E l i t e_{i} + P • C F \otimes s_{i} \end{matrix} i = n / 2, \dots, n .$ (14) where R_L is a random vector based on the L $\overset{´}{e}$ vy distribution and $CF = {(1 - \frac{Iter}{Max_Iter})}^{(\frac{2 Iter}{Max_Iter})}$ is considered as an adaptive parameter to control the step size for predator movement.

At the end of the iteration, MPA is mainly realized as a process of exploitation of the search space, when the best strategy for the predator is to perform a L $\overset{´}{e}$ vy movement: $\begin{matrix} \frac{2}{3} Max_Iter < Iter < Max_Iter \\ {\begin{matrix} s_{i} = R_{L} \otimes (R_{L} \otimes {Elite}_{i} - {Prey}_{i}) \\ {Prey}_{i} = {Elite}_{i} + P • CF \otimes s_{i} \end{matrix} \begin{matrix} i = 1, \dots, n . \end{matrix} \end{matrix}$ (15)

Eddy formation or Fish Aggregating Devices (FADs) usually cause changes in the behavior of marine organisms, which help MPA to jump out of local optima during the optimization search process and prevent the algorithm from converging prematurely.

$P r e y_{i} = {\begin{matrix} P r e y_{i} + C F [X_{m i n} + R \otimes (X_{m a x} - X_{m i n})] \otimes U i f r \leq F A D s \\ P r e y_{i} + [F A D s (1 - r) + r] (P r e y_{r 1} - P r e y_{r 2}) i f r > F A D s \end{matrix}$ (16) where FADs = 0.2, U is a binary vector containing 0 and 1, r is a uniform random number between 0 and 1. r1 and r2 are random indexes in the prey matrix.

3.4.2 Piecewise linear chaotic map

The traditional MPA algorithm initializes the population by applying a random way in the search space, which will lead to the poor difference and diversity of the population, and the excellent initialized population can effectively improve the algorithm’s speed of searching for the optimal [47], so in this paper, we adopt the piecewise linear chaotic map (PWLCM) for the initialization of the population, and PWLCM has the traversal and randomness, and it can effectively improve the diversity of the initial solution, and the definition of the PWLCM is shown in Equation (17): $X (t + 1) = {\begin{matrix} \frac{X (t)}{a}, \begin{matrix} \begin{matrix} \end{matrix} & \begin{matrix} 0 ⩽ X (t) < a \end{matrix} \end{matrix} \\ \frac{X (t) - a}{0.5 - a}, \begin{matrix} \begin{matrix} \end{matrix} & a ⩽ X (t) < 0.5 \end{matrix} \\ \frac{1 - a - X (t)}{0.5 - a}, \begin{matrix} 0.5 ⩽ X (t) < 1 - a \end{matrix} \\ \frac{1 - X (t)}{a}, \begin{matrix} \begin{matrix} 1 - a ⩽ X (t) < 1 \end{matrix} \end{matrix} \end{matrix}$ (17) where a is the segmentation control factor, and according to [48], it works best at a = 0.3. Therefore, in this paper, a is taken as 0.3.

3.4.3 Random opposition-based learning strategy

Opposition learning strategy [49] is an improved strategy for swarm intelligence optimization proposed by Tizhoosh in 2005, and its main idea lies in generating an opposition solution that is opposite to the current solution, adding it to the current population, comparing the advantages and disadvantages of the current solution and the opposition solution, and selecting the best to enter the next iteration. However, due to the lack of randomness of the opposition solution generated by opposition learning, it cannot generate a population with diversity, so this paper adopts random opposition-based learning (ROBL) proposed by Long et al. [50], ROBL introduces random numbers into the opposition learning strategy learning strategy, which helps to generate a population with randomness and enhance the population diversity, to avoid the algorithm falling into local optimum, the formula of ROBL is shown in Equation (18): $X_{ROBL} = X_{\max} + X_{\min} - r \times X$ (18) where X_ROBL is a random opposition solution, X ∈ [X_min, X_max], r is random numbers between [0, 1].

3.5 Flow of the proposed method

Aiming at the high dimensional and class imbalance characteristics of biomedical data, this paper combines the feature selection method and data oversampling method to solve the high dimensional problem of biomedical data by using the mRMR feature selection method, and to solve the class imbalance problem by using the improved SMOTE oversampling method which effectively solves the current problems of biomedical data classification. The method proposed in this paper mainly includes the following four steps:

Step 1: Feature selection of biomedical data using mRMR

Aiming at the high dimensionality of biomedical data, which can easily lead to the problem of “curse of dimensionality”, this paper firstly uses the mRMR method to select the features of the data, to obtain the subset of features that are the most relevant to the target variables and have the smallest redundancy among the features, so as to reduce the dimensionality of the data, to improve the generalization ability of the model, and to reduce the cost of the subsequent calculations.

Step 2: Data oversampling of the feature-selected dataset using the Spectral-SMOTE

To address the class imbalance problem in biomedicine data, this paper proposes a new Spectral-SMOTE method for oversampling data. Spectral-SMOTE effectively solves the noise sensitivity problem of SMOTE, and at the same time, it solves the problem that K-means SMOTE can not deal with high-dimensional sparse data effectively. For the problem of constructing similarity matrix in spectral clustering, this paper uses the multi-kernel function to construct the similarity matrix with excellent learning ability and generalization ability. The specific process and pseudo-code of Spectral-SMOTE are shown in Algorithm 1.

Algorithm 1: Spectral-SMOTE
Input:X (feature matrix after mRMR)
y (class labels)
n (number of samples)
k (number of clusters)
irt (imbalance ratio threshold)
knn (number of K-nearest neighbors in SMOTE)
de (exponent used for computation of density)
w (weights of the kernel function)
Output:G_samples (new samples generated)
//Step1: Spectral cluster
clus ← spectralcluster (X, k, w)
//Step2: Filter and compute the sampling weight
fc ← φ
forc ∈ clusdo:
_{ir ←(majcount (c) +1)/(min count (c) +1)}
ifir < irtthen:
fc ← fc ∪ {c}
end if
end for
forf ∈ fcdo:
_{desity (f) ←min count (f)/mean (edclideandis (f)) ^de}
_{sparsity (f) ←1/density (f)}
_end for
_{sw (f) ←sparsity (f)/_∑f ∈fcsparsity (f)}
//Step3: Generate new samples with SMOTE
G _ samples ← φ
forf ∈ fcdo:
_{_nnew ←n ×sw (f)}
_{G _ samples ←G _ samples ∪{SMOTE (f,_nnew,knn)}}
end for
return G _ samples

Step 3: Optimize the selection of key parameters for Spectral-SMOTE using PRMPA

The parameters in Spectral-SMOTE are crucial to the performance of sampling, and optimizing these parameters is a complex mathematical optimization problem, so in this paper, we use PRMPA to optimize these parameters. PRMPA combines PWLCM and random opposition learning strategy, and firstly, PWLCM is used to generate the initialized population with diversity, which lays a good foundation for the subsequent algorithm to find the optimum. Then ROBL is used to generate the opposition solution of the current solution in each iteration to prevent the algorithm from falling into a local optimum. Since this paper mainly focuses on the problem of imbalanced biomedical data classification, the traditional accuracy rate effectively measures the goodness of the model, so this paper chooses F-measure as the fitness function. The parameters to be optimized in Spectral-SMOTE are shown in Table 1, the pseudo-code of PRMPA is shown in Algorithm 2, and the pseudo-code of PRMPA-Spectral-SMOTE is shown in Algorithm 3.

Table 1
Description of optimization parameters

Parameter Parameter description Optimization range

k(integer) number of clusters [1, n]

Irt(real) imbalance ratio threshold [0.1, 10]

knn(integer) number of K-nearest neighbors in SMOTE [3, 20]

de(real) exponent used for computation of density [0, 20]

w(real) weights of the kernel function [0.01, 0.99]

Parameter	Parameter description	Optimization range
k(integer)	number of clusters	[1, n]
Irt(real)	imbalance ratio threshold	[0.1, 10]
knn(integer)	number of K-nearest neighbors in SMOTE	[3, 20]
de(real)	exponent used for computation of density	[0, 20]
w(real)	weights of the kernel function	[0.01, 0.99]

Algorithm 2: PRMPA
Initialize algorithm parameters Max _ Iter, N, P, FADs.
Generate initialized populations according to PWLCM
WhileIter < Max _ Iterdo:
fori < Ndo:
Calculate the fitness function value.
Generate random opposition solutions according to Equation (18) and combine the opposition solutions with the original solutions to generate a new population NP.
Calculate the fitness function value of population NP.
The best performing individual is selected to move to the next iteration.
end for
ifIter < Max _ Iter/3 then:
Update the prey according to Equation (12)
else ifMax _ Iter/3 < Iter < 2 * Max _ Iter/3 then:
Update the first half of the prey according to Equation (13)
Update the other half of the prey according to Equation (14)
else ifIter > 2 * Max _ Iter/3 then:
Update the prey according to Equation (15)
end if
Accomplish memory saving and update Elite
Applying FADs effect and update according to Equation (16)
end while

Algorithm 3: PRMPA-Spectral-SMOTE
Input:Data (imbalanced data)
Output:Data_new (Optimized balanced data)
Initialize: k, irt, knn, de, w
//Step1: Selecting the fitness function
Split the original data into a training set and a test set.
Generate new samples on the training set using Spectral-SMOTE and construct a classification model.
Validating model classification performance with test set.
Based on the classification results, F-measure is calculated and the F-measure is used as the fitness function.
//Step2: Optimizing the hyperparameters of Spectral-SMOTE
While the termination condition is not met do:
Calculate the fitness function
(k, irt, knn, de, w) ← PRMPA
end while
//Step3: Generate optimized balanced data
(k^, irt^, knn^, de^, w^*)← Optimal parameters obtained by PRMPA
G _ samples ← Spectral - SMOTE (Data, k^, irt^, knn^, de^, w^*)
Data_new ← Data ∪ G _ samples
return Data _new

Step 4: Classify biomedical data using multiple classifiers

The dataset after feature selection is divided into a training set and a test set, on the training set a balanced training set is obtained by oversampling using PRMPA-Spectral-SMOTE, on the balanced training set a classification model is built using KNN, SVM, RF, and LR, and then a test is performed on the test set, and the final evaluation metrics are calculated based on the classification results.

4 Experiments, results and discussions

4.1 Datasets

To evaluate the performance of mRMR and PRMPA-Spectral-SMOTE, this paper selects five real high-dimensional imbalanced biomedical datasets, which are mainly used for various disease diagnosis and contain thousands of gene expression biological samples. Includes two binary classification datasets and three multi-category datasets. Colon, GLI_85, TOX_171, and GLIOMA datasets are obtained from https://jundongl. github.io/scikit-feature/datasets.html. The MLL dataset was obtained from https://csse.szu.edu.cn/staff/zhuzx/Datasets. html. In real life, identifying specific classifications may be the main purpose of biomedical data classification [51], so for multiclass classification datasets firstly convert them into binary datasets, in the MLL dataset, MLL is treated as one class, and the remaining two classes as one class; in the TOX-171 dataset, class 3, which has the least number of classes, is regarded as one class, and the remaining three classes are treated as one class; in the GLOIMA dataset, cancer oligodendrogliomas was considered as one class and the remaining three classes as one class. Table 2 lists the basic information of the selected dataset, IR is the imbalance ratio of the dataset. It can be seen from Table 2 that the number of features of the selected data sets ranges from 2000 to 22283, IR ranges from 1.82 to 6.14, and the total number of samples ranges from 50 to 171, indicating that biomedical data is typical high-dimensional imbalanced small sample data, and accurate classification of biomedical data is a very difficult learning task.

Table 2
Description of the datasets

Datasets Features Classes Samples Minority Majority IR

Colon 2000 2 62 22 40 1.82

GLI_85 22283 2 85 26 59 2.27

MLL 12583 3 72 20 52 2.60

TOX_171 5748 4 171 39 132 3.38

GLIOMA 4434 4 50 7 43 6.14

Datasets	Features	Classes	Samples	Minority	Majority	IR
Colon	2000	2	62	22	40	1.82
GLI_85	22283	2	85	26	59	2.27
MLL	12583	3	72	20	52	2.60
TOX_171	5748	4	171	39	132	3.38
GLIOMA	4434	4	50	7	43	6.14

4.2 Metrics

In this paper, the accuracy rate (Acc), F-measure and area under the curve (AUC) [51, 52] are used to evaluate the performance of the biomedical data classification model. First, the confusion matrix of the classification results is established (as shown in Table 3), and then the following evaluation indicators can be defined according to the confusion matrix: $Acc = \frac{TP + TN}{TP + TN + FP + FN}$ (19) $Recall = \frac{TP}{TP + FN}$ (20) $Precision = \frac{TP}{TP + FP}$ (21) $F - measure = \frac{2 \times Recall \times Precision}{Recall + Precision}$ (22)

Table 3

Confusion matrix

Actual value	Predicted results
	Positive	Negative
Positive	TP	FN
Negative	FP	TN

AUC represents the area under the ROC curve. The ROC curve takes TP/(TP + FN) as the ordinate and FP/(TN + FP) as the abscissa, and draws the entire curve by traversing all thresholds. When the data is imbalanced, F-measure and AUC have great reference value. The larger the Acc, F-measure and AUC, the better the classification performance of the model.

4.3 Experimental setups

All experiments in this paper adopt five-fold cross-validation, and each data set is repeated 5 times. The final result is the average result of the above process, which can effectively reduce the deviation caused by dividing the data set. In this paper, four classifiers, KNN, SVM, RF and LR, are used to classify and evaluate the data set after feature selection and data resampling. All experiments were carried out on a computer with Intel(R) Core(TM) i7-10700, CPU 2.90 GHz, 16.0 GB memory and Windows 10 operating system, using Python 3.7 for programming.

This paper mainly carried out three experiments, which are briefly introduced as follows:

Experiment 1: Compare the classification performance of the model after mRMR feature selection with the model using all features to verify the effectiveness of feature selection;

Experiment 2: Resample the feature-selected data using different SMOTE-based data resampling methods, and compare these methods with PRMPA-Spectral-SMOTE to verify the effectiveness of PRMPA-Spectral-SMOTE;

Experiment 3: The performance of different data resampling methods is evaluated using non-parametric statistical tests to test the difference between PRMPA-Spectral-SMOTE and other algorithms.

4.4 Comparison of results and discussions

4.4.1 Experiment 1: Comparison of feature selection results

In this paper, four classifiers, KNN, SVM, RF and LR, are first used to classify the original dataset and the dataset after mRMR feature selection, in which k in KNN is set to 3, and the parameters of the rest of the classifiers are set to default parameters. The experimental results are shown in Table 4 and Fig. 1.

Table 4
Comparison of feature selection results

Data method F-measure Acc AUC

Colon KNN 0.6401 0.7323 0.7235

SVM 0.5489 0.7628 0.6925

RF 0.7772 0.8432 0.8250

LR 0.7196 0.8159 0.7860

mRMR-KNN 0.8275 0.8708 0.8685

mRMR-SVM 0.8152 0.8733 0.8625

mRMR-RF 0.8071 0.8618 0.8540

mRMR-LR 0.8161 0.8705 0.8635

GLI_85 KNN 0.7075 0.8329 0.7936

SVM 0.6464 0.8471 0.7630

RF 0.7295 0.8729 0.8070

LR 0.8000 0.8965 0.8464

mRMR-KNN 0.8778 0.9341 0.9035

mRMR-SVM 0.9251 0.9553 0.9484

mRMR-RF 0.8555 0.9200 0.8930

mRMR-LR 0.9135 0.9506 0.9405

MLL KNN 0.7450 0.8383 0.8389

SVM 0.2400 0.7640 0.5750

RF 0.5474 0.8326 0.7090

LR 0.8073 0.9166 0.8592

mRMR-KNN 0.9714 0.9859 0.9750

mRMR-SVM 0.9638 0.9832 0.9700

mRMR-RF 0.9017 0.9533 0.9214

mRMR-LR 0.9429 0.9720 0.9500

TOX_171 KNN 0.7992 0.9204 0.8571

SVM 0.7296 0.9066 0.7964

RF 0.5811 0.8701 0.7150

LR 0.8991 0.9602 0.9184

mRMR-KNN 0.9263 0.9686 0.9485

mRMR-SVM 0.9456 0.9777 0.9535

mRMR-RF 0.8297 0.9369 0.8642

mRMR-LR 0.9336 0.9719 0.9431

GLIOMA KNN 0.7493 0.9340 0.8940

SVM 0.1133 0.8720 0.5550

RF 0.4220 0.8900 0.7056

LR 0.5060 0.9060 0.7463

mRMR-KNN 0.9600 0.9880 0.9700

mRMR-SVM 0.8533 0.9800 0.9200

mRMR-RF 0.6800 0.9600 0.8300

mRMR-LR 0.7867 0.9680 0.8900

Data	method	F-measure	Acc	AUC
Colon	KNN	0.6401	0.7323	0.7235
	SVM	0.5489	0.7628	0.6925
	RF	0.7772	0.8432	0.8250
	LR	0.7196	0.8159	0.7860
	mRMR-KNN	0.8275	0.8708	0.8685
	mRMR-SVM	0.8152	0.8733	0.8625
	mRMR-RF	0.8071	0.8618	0.8540
	mRMR-LR	0.8161	0.8705	0.8635
GLI_85	KNN	0.7075	0.8329	0.7936
SVM	0.6464	0.8471	0.7630
	RF	0.7295	0.8729	0.8070
	LR	0.8000	0.8965	0.8464
	mRMR-KNN	0.8778	0.9341	0.9035
	mRMR-SVM	0.9251	0.9553	0.9484
	mRMR-RF	0.8555	0.9200	0.8930
	mRMR-LR	0.9135	0.9506	0.9405
MLL	KNN	0.7450	0.8383	0.8389
	SVM	0.2400	0.7640	0.5750
	RF	0.5474	0.8326	0.7090
	LR	0.8073	0.9166	0.8592
	mRMR-KNN	0.9714	0.9859	0.9750
	mRMR-SVM	0.9638	0.9832	0.9700
	mRMR-RF	0.9017	0.9533	0.9214
	mRMR-LR	0.9429	0.9720	0.9500
TOX_171	KNN	0.7992	0.9204	0.8571
	SVM	0.7296	0.9066	0.7964
	RF	0.5811	0.8701	0.7150
	LR	0.8991	0.9602	0.9184
	mRMR-KNN	0.9263	0.9686	0.9485
	mRMR-SVM	0.9456	0.9777	0.9535
	mRMR-RF	0.8297	0.9369	0.8642
	mRMR-LR	0.9336	0.9719	0.9431
GLIOMA	KNN	0.7493	0.9340	0.8940
	SVM	0.1133	0.8720	0.5550
	RF	0.4220	0.8900	0.7056
	LR	0.5060	0.9060	0.7463
	mRMR-KNN	0.9600	0.9880	0.9700
	mRMR-SVM	0.8533	0.9800	0.9200
	mRMR-RF	0.6800	0.9600	0.8300
	mRMR-LR	0.7867	0.9680	0.8900

By analyzing Table 4, the classification performance after mRMR feature selection is significantly better than the unfeatured one on the four classifiers, and the F-measure, Acc and AUC values are greatly improved. When classifying the unfeatured selection dataset, KNN, RF and LR perform better and SVM performs the worst, while when classifying on the mRMR feature selected dataset, KNN and SVM are better, which indicates that the performance of different classifiers is not the same on different datasets, and that no single classifier can effectively and accurately classify all datasets, so it makes sense to select multiple classifiers for model evaluation is meaningful. The poor classification performance of SVM on the original dataset indicates that SVM is sensitive to dimensionality and may not be able to handle high-dimensional data effectively, but the dataset after mRMR feature selection presents good performance, and the F-measure even improves by 0.74 on the GLIOMA dataset, which further illustrates that the mRMR feature selection method can effectively improve the performance of biomedical data classification.

Figure 1 shows the visualization of the feature selection results, from which it can be more intuitively seen that on the five datasets, the classification performance of KNN, SVM, RF, and LR on the datasets processed by mRMR feature selection are all greatly improved.

Fig. 1

Comparison of feature selection result.

4.4.2 Experiment 2: Comparison of results from different SMOTE methods

In Experiment 2, the method proposed in this paper is compared with seven SMOTE-based data resampling methods: SMOTE [10], ADASYN [36], Borderline SMOTE (B-SMOTE) [30], SVMSMOTE (S-SMOTE) [35], K-means SMOTE (K-SMOTE) [40], SMOTE-ENN (SMOTE-E) [53], SMOTE-Tomek (SMOTE-T) [54]. These methods have been widely used in imbalanced data problems in previous research and are considered reliable baseline methods [51 , 56]. In conducting the experiments, all the datasets were mRMR feature selected, and all the above SMOTE-based data methods use the default parameters, and the data resampling methods are still evaluated using four classifiers, namely, KNN, SVM, RF, and LR, in order to eliminate the influence of the classifiers on the evaluation of the data resampling performance. Experiment 2 carried out model comparison analysis from two aspects, firstly, the performance evaluation of different data resampling methods using the same classifiers (as shown in Tables 5–8) to verify the effectiveness of the resampling methods proposed in this paper; and then the performance of different classifiers after data resampling from the same dataset was visualized to compare the performance differences of the different classifiers (as shown in Figs. 2–6).

Table 5
Comparison results of different data resampling methods on KNN

SMOTE ADASYN B-SMOTE S-SMOTE K-SMOTE SMOTE-E SMOTE-T Proposed

Colon

F-measure 0.8338 0.8001 0.7914 0.8037 0.8324 0.8343 0.8239 0.8995

Acc 0.8623 0.8259 0.8118 0.8292 0.8654 0.8646 0.8554 0.9195

AUC 0.8760 0.8555 0.8455 0.8560 0.8750 0.8765 0.8725 0.9305

GLI_85

F-measure 0.9235 0.9362 0.9276 0.9252 0.9315 0.9244 0.9210 0.9655

Acc 0.9576 0.9624 0.9576 0.9576 0.9600 0.9576 0.9553 0.9765

AUC 0.9423 0.9518 0.9462 0.9415 0.9487 0.9408 0.9382 0.9745

MLL

F-measure 0.9771 0.9784 0.9841 0.9829 0.9829 0.9714 0.9771 1.0000

Acc 0.9886 0.9888 0.9914 0.9918 0.9916 0.9859 0.9891 1.0000

AUC 0.9800 0.9832 0.9880 0.9850 0.9850 0.9750 0.9800 1.0000

TOX_171

F-measure 0.9539 0.9354 0.9328 0.9308 0.9451 0.9146 0.9339 0.9882

Acc 0.9766 0.9672 0.9649 0.9638 0.9732 0.9557 0.9673 0.9942

AUC 0.9848 0.9787 0.9773 0.9766 0.9791 0.9713 0.9771 0.9962

GLIOMA

F-measure 0.9387 0.9627 0.9520 0.9573 0.9467 0.9600 0.9493 1.0000

Acc 0.9800 0.9840 0.9840 0.9840 0.9840 0.9880 0.9800 1.0000

AUC 0.9886 0.9903 0.9908 0.9906 0.9911 0.9933 0.9881 1.0000

	SMOTE	ADASYN	B-SMOTE	S-SMOTE	K-SMOTE	SMOTE-E	SMOTE-T	Proposed
Colon
F-measure	0.8338	0.8001	0.7914	0.8037	0.8324	0.8343	0.8239	0.8995
Acc	0.8623	0.8259	0.8118	0.8292	0.8654	0.8646	0.8554	0.9195
AUC	0.8760	0.8555	0.8455	0.8560	0.8750	0.8765	0.8725	0.9305
GLI_85
F-measure	0.9235	0.9362	0.9276	0.9252	0.9315	0.9244	0.9210	0.9655
Acc	0.9576	0.9624	0.9576	0.9576	0.9600	0.9576	0.9553	0.9765
AUC	0.9423	0.9518	0.9462	0.9415	0.9487	0.9408	0.9382	0.9745
MLL
F-measure	0.9771	0.9784	0.9841	0.9829	0.9829	0.9714	0.9771	1.0000
Acc	0.9886	0.9888	0.9914	0.9918	0.9916	0.9859	0.9891	1.0000
AUC	0.9800	0.9832	0.9880	0.9850	0.9850	0.9750	0.9800	1.0000
TOX_171
F-measure	0.9539	0.9354	0.9328	0.9308	0.9451	0.9146	0.9339	0.9882
Acc	0.9766	0.9672	0.9649	0.9638	0.9732	0.9557	0.9673	0.9942
AUC	0.9848	0.9787	0.9773	0.9766	0.9791	0.9713	0.9771	0.9962
GLIOMA
F-measure	0.9387	0.9627	0.9520	0.9573	0.9467	0.9600	0.9493	1.0000
Acc	0.9800	0.9840	0.9840	0.9840	0.9840	0.9880	0.9800	1.0000
AUC	0.9886	0.9903	0.9908	0.9906	0.9911	0.9933	0.9881	1.0000

Table 6

Comparison results of different data resampling methods on SVM

	SMOTE	ADASYN	B-SMOTE	S-SMOTE	K-SMOTE	SMOTE-E	SMOTE-T	Proposed
Colon
F-measure	0.8194	0.8251	0.8247	0.8215	0.8191	0.8395	0.8280	0.8894
Acc	0.8677	0.8708	0.8738	0.8713	0.8746	0.8772	0.8762	0.9146
AUC	0.8605	0.8695	0.8680	0.8645	0.8650	0.8718	0.8735	0.9135
GLI_85
F-measure	0.9270	0.9337	0.9364	0.9437	0.9335	0.9255	0.9278	0.9607
Acc	0.9553	0.9600	0.9600	0.9647	0.9600	0.9553	0.9553	0.9741
AUC	0.9497	0.9565	0.9554	0.9627	0.9552	0.9485	0.9489	0.9715
MLL
F-measure	0.9657	0.9638	0.9771	0.9657	0.9695	0.9657	0.9581	1.0000
Acc	0.9834	0.9829	0.9888	0.9836	0.9861	0.9830	0.9810	1.0000
AUC	0.9700	0.9700	0.9800	0.9700	0.9750	0.9700	0.9650	1.0000
TOX_171
F-measure	0.9743	0.9751	0.9735	0.9583	0.9832	0.9612	0.9720	0.9840
Acc	0.9895	0.9930	0.9895	0.9824	0.9931	0.9824	0.9894	0.9941
AUC	0.9792	0.9832	0.9789	0.9719	0.9846	0.9774	0.9768	0.9861
GLIOMA
F-measure	0.9467	0.9867	1.0000	0.9600	0.8800	0.9733	0.9200	1.0000
Acc	0.9880	0.9960	1.0000	0.9960	0.9800	0.9920	0.9840	1.0000
AUC	0.9700	0.9900	1.0000	0.9800	0.9300	0.9800	0.9600	1.0000

Table 7

Comparison results of different data resampling methods on RF

	SMOTE	ADASYN	B-SMOTE	S-SMOTE	K-SMOTE	SMOTE-E	SMOTE-T	Proposed
Colon
F-measure	0.8142	0.7936	0.8175	0.8228	0.8204	0.8272	0.8210	0.8931
Acc	0.8718	0.8551	0.8662	0.8682	0.8715	0.8644	0.8682	0.9203
AUC	0.8660	0.8450	0.8650	0.8645	0.8600	0.8705	0.8645	0.9175
GLI_85
F-measure	0.8763	0.8531	0.8477	0.8697	0.8696	0.8342	0.8537	0.9343
Acc	0.9271	0.9176	0.9153	0.9200	0.9294	0.9106	0.9153	0.9624
AUC	0.9064	0.8944	0.8867	0.9071	0.9058	0.8768	0.8927	0.9468
MLL
F-measure	0.9329	0.9060	0.9314	0.9397	0.9286	0.9119	0.9206	0.9886
Acc	0.9670	0.9549	0.9695	0.9695	0.9638	0.9587	0.9610	0.9943
AUC	0.9432	0.9260	0.9450	0.9512	0.9412	0.9282	0.9360	0.9900
TOX_171
F-measure	0.8835	0.8763	0.8703	0.8987	0.8601	0.8604	0.8826	0.9556
Acc	0.9521	0.9508	0.9497	0.9567	0.9440	0.9368	0.9509	0.9812
AUC	0.9121	0.9075	0.8956	0.9318	0.8918	0.9156	0.9103	0.9617
GLIOMA
F-measure	0.7467	0.6907	0.7653	0.7067	0.7667	0.8133	0.7507	0.8800
Acc	0.9360	0.9400	0.9480	0.9400	0.9360	0.9560	0.9440	0.9720
AUC	0.8667	0.8539	0.8786	0.8461	0.8764	0.8933	0.8678	0.9278

Table 8

Comparison results of different data resampling methods on LR

	SMOTE	ADASYN	B-SMOTE	S-SMOTE	K-SMOTE	SMOTE-E	SMOTE-T	Proposed
Colon
F-measure	0.8200	0.8325	0.8377	0.8321	0.8226	0.8422	0.8420	0.8954
Acc	0.8677	0.8738	0.8805	0.8800	0.8690	0.8769	0.8867	0.9177
AUC	0.8645	0.8710	0.8820	0.8770	0.8675	0.8840	0.8865	0.9195
GLI_85
F-measure	0.9257	0.9311	0.9401	0.9406	0.9252	0.9297	0.9383	0.9661
Acc	0.9553	0.9553	0.9624	0.9624	0.9529	0.9576	0.9624	0.9765
AUC	0.9503	0.9535	0.9628	0.9607	0.9495	0.9510	0.9574	0.9744
MLL
F-measure	0.9562	0.9531	0.9562	0.9600	0.9600	0.9543	0.9524	0.9943
Acc	0.9806	0.9806	0.9802	0.9806	0.9804	0.9777	0.9775	0.9973
AUC	0.9650	0.9650	0.9650	0.9650	0.9650	0.9600	0.9600	0.9950
TOX_171
F-measure	0.9521	0.9357	0.9424	0.9527	0.9432	0.9414	0.9443	0.9906
Acc	0.9778	0.9695	0.9731	0.9778	0.9731	0.9707	0.9742	0.9953
AUC	0.9748	0.9692	0.9718	0.9787	0.9704	0.9757	0.9722	0.9969
GLIOMA
F-measure	0.9653	0.9787	0.9920	1.0000	0.9733	0.9840	0.9573	1.0000
Acc	0.9880	0.9920	0.9960	1.0000	0.9920	0.9920	0.9840	1.0000
AUC	0.9931	0.9953	0.9975	1.0000	0.9956	0.9950	0.9906	1.0000

Fig. 2

Comparison of results from different classification models on the Colon dataset.

Fig. 3

Comparison of results from different classification models on the GLI_85 dataset.

Fig. 4

Comparison of results from different classification models on the MLL dataset.

Fig. 5

Comparison of results from different classification models on the TOX_171 dataset.

Fig. 6

Comparison of results from different classification models on the GLIOMA dataset.

By analyzing the results in Tables 5–8, the following conclusions can be obtained:

Regardless of which classifier among KNN, SVM, RF and LR is used, the PRMPA-Spectral-SMOTE method proposed in this paper outperforms other data resampling methods in all evaluation indexes, the GLIOMA dataset is classified completely and correctly on KNN, SVM and LR, and the MLL dataset is classified completely and correctly on KNN and SVM. The evaluation indexes are all 1, indicating that the PRMPA-Spectral-SMOTE method is more suitable for imbalanced data classification and helps to improve the model classification performance;

Compared with the models without data resampling in Table 4, the performance of most of the classification models built after data resampling is improved to different degrees, indicating that data resampling can effectively alleviate the imbalance of biomedical data and improve the performance of biomedical data classification;

In the same classifier, the performance of the same data resampling method on different datasets shows large differences, such as in the KNN model, SMOTE-E performs better on the GLIOMA dataset, but performs worst on the TOX_171 dataset;

The performance of the same data resampling method on different classifiers is also different, for example, the S-SMOTE method performs better on the LR classifier but worse on other classifiers, while PRMPA-Spectral-SMOTE can achieve the best performance on all classifiers, which further illustrates the PRMPA-Spectral-SMOTE superiority.

By analyzing the results in Figs. 2–6, the following conclusions can be obtained:

On different datasets, the performance of PRMPA-Spectral-SMOTE is significantly better than the other comparison models, and the evaluation indexes are improved compared with the comparison models, but the degree of improvement is different on different classifiers, for example, on the GLI_85 dataset, the performance improvement of RF is significantly better than other classifiers;

There are also large differences in the performance of different classifiers on the same dataset, for example, in the TOX_171 and GLIOMA datasets, the performance of RF is significantly worse than that of KNN, SVM, and LR, which suggests that RF may not be suitable to be used as a classifier for these datasets;

From the GLIOMA dataset, it can be seen that the larger the imbalance rate of the dataset is, the more obvious the improvement effect of the PRMPA-Spectral-SMOTE method is, indicating that the method proposed in this paper can well solve the classification problem of highly imbalanced data.

4.4.3 Experiment 3: Non-parametric statistical tests

To evaluate the performance variations among various data resampling methods, this paper employs the Friedman test [57] and Holm’s test [58] for further statistical analysis. The main objective of the Friedman test and Holm’s test is to compare the average performance of multiple methods across multiple datasets. The main idea is to rank and categorize the results obtained to determine whether there is a significant difference in the average performance of the various methods on multiple datasets, and to comprehensively assess the differences in the performance of different methods on different datasets. This statistical test is also widely used in work related to imbalanced data classification [55, 59]. The original hypothesis is “there is no significant difference between the methods”. The statistical tests in this paper are implemented in the Python library provided by [60]. The results of the experiments are shown in Fig. 7 and Table 9–10:

Fig. 7

Mean rank of different data resampling methods using different classifiers.

Table 9

Result of Friedman test

Metric	KNN	SVM	RF	LR
F-measure	0.0244^*	0.0139^*	0.0029^*	0.0031^*
Acc	0.0105^*	0.0085^*	0.0047^*	0.0031^*
AUC	0.0050^*	0.0074^*	0.0051^*	0.0044^*

Table 10

Result of Holm’s test on data with tne proposed model as the control method

		SMOTE	ADASYN	B-SMOTE	S-SMOTE	K-SMOTE	SMOTE-E	SMOTE-T
KNN	F-measure	0.0271^*	0.0908	0.0325^*	0.0325^*	0.0908	0.0271^*	0.0010^*
	Acc	0.0325^*	0.0325^*	0.0179^*	0.0325^*	0.1752	0.0276^*	0.0087^*
	AUC	0.0567	0.0567	0.0567	0.0221^*	0.1376	0.0225^*	0.0016^*
SVM	F-measure	0.0145^*	0.0908	0.1556	0.0407^*	0.0407^*	0.0407^*	0.0109^*
	Acc	0.0075^*	0.0567	0.1376	0.0716^*	0.0908	0.0225^*	0.0070^*
	AUC	0.0117^*	0.1866	0.1866	0.0276^*	0.0508	0.0276^*	0.0070^*
RF	F-measure	0.0777	0.0008^*	0.0271^*	0.1213	0.0393^*	0.0335^*	0.0425^*
	Acc	0.1224	0.0037^*	0.0491^*	0.1224	0.0491^*	0.0034^*	0.0491^*
	AUC	0.1056	0.0013^*	0.0567	0.1056	0.0179^*	0.0846	0.0184^*
LR	F-measure	0.0117^*	0.0109^*	0.1056	0.2725	0.0149^*	0.0355^*	0.0220^*
	Acc	0.0220^*	0.0149^*	0.1056	0.2725	0.0087^*	0.0145^*	0.0355^*
	AUC	0.0109^*	0.0184^*	0.1627	0.1967	0.0109^*	0.0295^*	0.0268^*

Figure 7 shows the average rank results of different oversampling methods on different classifiers, the smaller the average rank, the better the performance of the method, and it can be seen that the PRMPA-Spectral-SMOTE is ranked the first in all the evaluation indexes. From each classifier, PRMPA-Spectral-SMOTE outperforms the other compared models in terms of F-measure, Acc and AUC. The above results also illustrate the superiority of PRMPA-Spectral-SMOTE.

Table 9 shows the Friedman test results of multiple data resampling methods on different classifiers, in this paper, the significance level α = 0.05 is set, and the bold in the table indicates that the original hypothesis can be rejected when the significance level is α. It is considered that there is a significant difference between the methods.

As can be seen from Table 9, on all classifiers, the p-value of F-measure, Acc and AUC is less than the significance level, and the original hypothesis can be rejected that there is a significant difference between multiple data resampling methods. However, Friedman Test cannot test the difference between the two methods, so Holm’s test is used to see if there is a significant difference between the proposed PRMPA-Spectral-SMOTE and other data resampling methods, and the experimental results are shown in Table 10.

From Holm’s test results in Table 10, it can be seen that most of the significant differences between PRMPA-Spectral-SMOTE and the other compared methods indicate that the proposed method in this paper outperforms the other compared methods in most of the cases. The most significant differences are between PRMPA-Spectral-SMOTE and SMOTE-E and SMOTE-T, followed by SMOTE, ADASYN, and K-SMOTE. Different data resampling methods also present different performances on different classifiers, for example, PRMPA-Spectral-SMOTE performs similarly to ADASYN and B-SMOTE in the SVM classifier, while their performances in the RF classifier show large differences, which is because the choice of classifiers also has a large impact on the final classification results.

In summary, the PRMPA-Spectral-SMOTE method proposed in this paper presents better performance compared with other data resampling methods, which can better solve the data imbalance problem and effectively improve the performance of biomedical data classification.

5 Conclusion

In order to address the problem of high dimensionality and class imbalance of biomedical data, this paper proposes a new method that combines feature selection and data resampling. Firstly, feature selection is performed on biomedical data using mRMR to reduce the feature dimensions and lower the computational cost; then the proposed Spectral-SMOTE is used to oversample the feature-selected data to obtain a balanced dataset; finally, the key parameters of Spectral-SMOTE are selected using the marine predators algorithm that incorporates PWLCM and ROBL, in order to enhance the performance of oversampling. Extensive experiments are conducted on five biomedical datasets, and the results show that the method proposed in this paper provides significant improvements in F-measure, Acc and AUC. The statistical test results show that the proposed PRMPA-Spectral-SMOTE is significantly different from other data resampling methods, which is highly superior and is an effective method to deal with the problem of class imbalance in biomedical data. The experimental results in this paper also illustrate that both feature selection and data resampling can significantly improve the classification performance of biomedical data, and thus, in the future, we can further explore the in-depth combination of these two aspects to more effectively solve the high-dimensional and class imbalance problems of biomedical data. In addition, extending and applying the proposed method to multi-classification problems is also an important direction for future research.

Statements & declarations

Authors contributions

Xiwen Qin: Conceptualization, Methodology, Supervision, Writing –original draft, Writing –review & editing. Siqi Zhang: Conceptualization, Methodology, Software, Formal analysis, Writing –original draft. Xiaogang Dong: Methodology, Conceptualization, Writing –review & editing. Hongyu shi: Software, Formal analysis, Writing –original draft. Liping Yuan: Software, Validation, Writing –review & editing.

Funding

This work was supported by Department of Science and Technology of Jilin Province project (20210101149JC, 20200403182SF); Education Department of Jilin Province project (JJKH20220662KJ); National Natural Science Foundation of China (12026430).

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

References

Tanwani

A.K.

, Afridi

, Shafiq

M.Z.

, Farooq

Guidelines to select machine learning scheme for classification of biomedical datasets, in: Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics: 7th European Conference (2009), 128–139.

Mirza

, Wang

, Choi

, Chung

N.C.

and Ping

, Machine learning and integrative analysis of biomedical big data, Genes 10(2) (2019), 87.

Peng

, Wu

and Jiang

, A novel feature selection approach for biomedical data classification, Journal of Biomedical Informatics 43(1) (2010), 15–23.

Pes

Handling Class Imbalance in High-Dimensional Biomedical Datasets, 2019 IEEE 28th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE) (2019), 150–155.

Drotar

, Gazda

and Smekal

, An experimental comparison of feature selection methods on two-class biomedical datasets, Computers in Biology and Medicine 66 (2015), 1–10.

Wang

, Ni

, Yang

, Pappu

, Fenn

M.B.

and Pardalos

P.M.

, Feature selection based on meta-heuristics for biomedicine, Optimization Methods and Software 29(4) (2014), 703–719.

Zhang

and Cao

, Classification of high dimensional biomedical data based on feature selection using redundant removal, PLoS One 14(4) (2019).

and Garcia

E.A.

, Learning from Imbalanced Data, IEEE Transactions on Knowledge and Data Engineering 21(9) (2009), 1263–1284.

Xie

, Qiu

, Zhang

, Peng

and Chen

, Gaussian distribution based oversampling for imbalanced data classification, IEEE Transactions on Knowledge and Data Engineering 34(2) (2020), 667–679.

10.

Chawla

N.V.

, Bowyer

K.W.

, Hall

L.O.

and Kegelmeyer

W.P.

, SMOTE: synthetic minority over-sampling technique, Journal of Artificial Intelligence Research 16 (2002), 321–357.

11.

Tomek

, Two modifications of CNN, Cybern 6(11) (1976), 769–772.

12.

Tomek

, An experiment with the edited nearest-neighbor rule, IEEE Trans Syst Man Cybern 6 (1976), 448–452.

13.

Shanab

A.A.

, Khoshgoftaar

T.M.

Is gene selection enough for imbalanced bioinformatics data? in: 2018 IEEE International Conference on Information Reuse and Integration (IRI), IEEE (2018, July) 346–355

14.

Nakamura

, Kajiwara

, Otsuka

and Kimura

, Lvq-smote–learning vector quantization based synthetic minority over–sampling technique for biomedical data, BioData Mining 6(1) (2013), 1–10.

15.

, Fong

, Sung

, Cho

, Wong

and Wong

K.K.

, Adaptive swarm cluster-based dynamic multi-objective synthetic minority oversampling technique algorithm for tackling binary imbalanced datasets in biomedical data classification, BioData Mining 9(1) (2016), 1–15.

16.

, Shen

, Nie

and Kou

, A hybrid sampling algorithm combining M-SMOTE and ENN based on Random forest for medical imbalanced data,, Journal of Biomedical Informatics 107 (2020), 103465.

17.

Saeys

, Inza

and Larrañaga

, A review of feature selection techniques in bioinformatics, Bioinformatics 23(19) (2007), 2507–2517.

18.

Lazar

, Taminau

, Meganck

, Steenhoff

, Coletta

, Molter

, Schaetzen

V.D.

, Duque

, Bersini

and Nowe

, A survey on filter techniques for feature selection in gene expression microarray analysis, IEEE/ACM Transactions on Computational Biology and Bioinformatics 9(4) (2012), 1106–1119.

19.

Battiti

, Using mutual information for selecting features in supervised neural net learning, IEEE Transactions on Neural Networks 5(4) (1994), 537–550.

20.

Peng

, Long

and Ding

, Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy, IEEE Transactions on Pattern Analysis and Machine Intelligence 27(8) (2005), 1226–1238.

21.

Van Hulse

, Khoshgoftaar

T.M.

, Napolitano

and Wald

, Threshold-based feature selection techniques for high-dimensional bioinformatics data, Network Modeling Analysis in Health Informatics and Bioinformatics 1 (2012), 47–61.

22.

Lyu

, Wan

, Han

, Liu

and Wang

, A filter feature selection method based on the Maximal Information Coefficient and Gram-Schmidt Orthogonalization for biomedical data mining, Computers in Biology and Medicine 89 (2017), 264–274.

23.

El-Manzalawy

, Hsieh

T.Y.

, Shivakumar

, Kim

and Honavar

, Min-redundancy and max-relevance multi-view feature selection for predicting ovarian cancer survival using multi-omics data, BMC Medical Genomics 11(3) (2018), 19–31.

24.

Xiong

, Li

, Wang

and Ye

, Informative gene selection based on cost-sensitive fast correlation-based filter feature selection, Current Bioinformatics 16(8) (2021), 1060–1068.

25.

Wang

, Wang

, Hu

, Shangguan

, Song

, Xu

, Wang

, Xue

, Wang

and Zhang

, Identification of key biomarkers for STAD using filter feature selection approaches, Scientific Reports 12(1) (2022), 19854.

26.

Solorio-Fernandez

, Carrasco-Ochoa

J.A.

and Martínez-Trinidad

J.F.

, A survey on feature selection methods for mixed data, Artificial Intelligence Review 55 (2021), 2821–2846.

27.

Correa

E.S.

, Freitas

A.A.

, Johnson

C.G.

A new discrete particle swarm algorithm applied to attribute selection in a bioinformatics data set, in: Proceedings of the 8th annual conference on Genetic and evolutionary computation (2006, July) 35–42.

28.

, Fong

, Wong

R.K.

, Millham

and Wong

K.K.

, Elitist binary wolf search algorithm for heuristic feature selection in high-dimensional bioinformatics datasets, Scientific Reports 7(1) (2017), 4354.

29.

Mafarja

, Thaher

, Too

, Chantar

, Turabieh

, Houssein

E.H.

and Emam

M.M.

, An efficient high-dimensional feature selection approach driven by enhanced multi-strategy grey wolf optimizer for biological data classification, Neural Computing and Applications 35(2) (2023), 1749–1775.

30.

Liu

, Zhou

and Liu

, An embedded feature selection method for imbalanced data classification, IEEE/CAA Journal of Automatica Sinica 6(3) (2019), 703–715.

31.

Nie

, Huang

, Cai

and Ding

, Efficient and robust feature selection via joint ℓ2, 1-norms minimization, Advances in Neural Information Processing Systems 23 (2010).

32.

Guo

, Chung

F.L.

, Li

and Zhang

, Multi-label bioinformatics data classification with ensemble embedded feature selection, IEEE Access 7 (2019), 103863–103875.

33.

Han

, Wang

W.Y.

, Mao

B.H.

Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning, in: International conference on intelligent computing, Berlin, Heidelberg: Springer Berlin Heidelberg (2005, August) 878–887.

34.

Bunkhumpornpat

, Sinapiromsaran

, Lursinsap

Safe-level-smote: Safe-level-synthetic minority oversampling technique for handling the class imbalanced problem, in: Advances in Knowledge Discovery and Data Mining: 13th Pacific-Asia Conference, PAKDD 2009 Bangkok, Thailand, April 27–30, 2009 Proceedings 13, Springer Berlin Heidelberg (2009), 475–482.

35.

Nguyen

H.M.

, Cooper

E.W.

and Kamei

, Borderline over-sampling for imbalanced data classification, International Journal of Knowledge Engineering and Soft Data Paradigms 3(1) (2011), 4–21.

36.

, Bai

, Garcia

E.A.

, Li

ADASYN: Adaptive synthetic sampling approach for imbalanced learning, in: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), Ieee (2008, June) 1322–1328.

37.

Maciejewski

, Stefanowski

Local neighbourhood extension of SMOTE for mining imbalanced data, in: 2011 IEEE symposium on computational intelligence and data mining (CIDM), IEEE (2011, April) 104–111.

38.

Saez

J.A.

, Luengo

, Stefanowski

and Herrera

, SMOTE–IPF: Addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering, Information Sciences 291 (2015), 184–203.

39.

and Fan

, CURE-SMOTE algorithm and hybrid algorithm for feature selection and parameter optimization based on random forests, BMC Bioinformatics 18(1) (2017), 1–18.

40.

Douzas

, Bacao

and Last

, Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE, Information Sciences 465 (2018), 1–20.

41.

Jadwal

P.K.

, Jain

, Pathak

and Agarwal

, Improved resampling algorithm through a modified oversampling approach based on spectral clustering and SMOTE, Microsystem Technologies 28(12) (2022), 2669–2677.

42.

Blagus

and Lusa

, SMOTE for high-dimensional class-imbalanced data, BMC Bioinformatics 14 (2013), 1–16.

43.

Maldonado

, López

and Vairetti

, An alternative SMOTE oversampling strategy for high-dimensional datasets, Applied Soft Computing 76 (2019), 380–389.

44.

Von Luxburg

, A tutorial on spectral clustering, Statistics and Computing 17 (2007), 395–416.

45.

Cao

, Wang

, Fernande

Multi-kernel support vector regression optimization model and indirect health factor extraction strategy for the accurate lithium-ion battery remaining useful life prediction, Journal of Solid State Electrochemistry (2023), 1–14.

46.

Faramarzi

, Heidarinejad

, Mirjalili

and Gandomi

A.H.

, Marine Predators Algorithm: A nature-inspired metaheuristic, Expert Systems with Applications 152 (2020), 113377.

47.

Han

, Du

, Zhu

, Li

, Yuan

and Zhu

, Golden-Sine dynamic marine predator algorithm for addressing engineering design optimization, Expert Systems with Applications 210 (2022), 118460.

48.

Wang

, Huang

X.Y.

, Zhu

, Yan

S.Q.

, Li

, Guo

Learning sparrow search algorithm that hybrids boundary processing mechanisms, Journal of Beijing University of Aeronautics and Astronautics (2022), 1–16. (in Chinese).

49.

Tizhoosh

H.R.

Opposition-based learning: a new scheme for machine intelligence, in: International conference on computational intelligence for modelling, control and automation and international conference on intelligent agents, web technologies and internet commerce (CIMCAIAWTIC’ 06), IEEE, 1 (2005, November) 695–701.

50.

Long

, Jiao

, Liang

, Cai

and Xu

, A random opposition-based learning grey wolf optimizer, IEEE Access 7 (2019), 113810–113825.

51.

Maldonado

, López

and Vairetti

, An alternative SMOTE oversampling strategy for high-dimensional datasets, Applied Soft Computing 76 (2019), 380–389.

52.

, Zhu

, Wu

and Zhu

, A novel oversampling technique for class-imbalanced learning based on SMOTE and natural neighbors, Information Sciences 565 (2021), 438–455.

53.

Batista

G.E.

, Prati

R.C.

and Monard

M.C.

, A study of the behavior of several methods for balancing machine learning training data, ACM SIGKDD Explorations Newsletter 6(1) (2004), 20–29.

54.

Batista

G.E.

, Bazzan

A.L.

and Monard

M.C.

, Balancing training data for automated annotation of keywords: a case study, Wob 3 (2003), 10–18.

55.

Chen

, Xia

, Chen

, Wang

and Wang

, RSMOTE: A self-adaptive robust SMOTE for imbalanced problems with label noise, Information Sciences 553 (2021), 397–428.

56.

Wei

, Mu

, Song

and Dou

, An improved and random synthetic minority oversampling technique for imbalanced data, Knowledge-Based Systems 248 (2022), 108839.

57.

Friedman

, The use of ranks to avoid the assumption of normality implicit in the analysis of variance, Journal of the American Statistical Association 32(200) (1937), 675–701.

58.

Holm

, A simple sequentially rejective multiple test procedure, Scand J Stat 6(2) (1979), 65–70.

59.

El Moutaouakil

, Roudani

and El Ouissari

, Optimal entropy genetic fuzzy-C-means SMOTE (OEGFCM-SMOTE), {Knowledge-Based Systems 262 (2023), 110235.

60.

Rodríguez-Fdez

, Canosa

, Mucientes

, Bugarín

STAC: a web platform for the comparison of algorithms using statistical tests, in: 2015 IEEE international conference on fuzzy systems (FUZZ-IEEE), IEEE (2015, August) 1–8.

Classification of high-dimensional imbalanced biomedical data based on spectral clustering SMOTE and marine predators algorithm

Abstract

Keywords

1 Introduction

2 Related work

2.1 Feature selection

2.2 SMOTE

3 Methods

3.1 mRMR

3.4.1 Marine predators algorithm (MPA)

4.1 Datasets

Table 2 Description of the datasets Datasets Features Classes Samples Minority Majority IR Colon 2000 2 62 22 40 1.82 GLI_85 22283 2 85 26 59 2.27 MLL 12583 3 72 20 52 2.60 TOX_171 5748 4 171 39 132 3.38 GLIOMA 4434 4 50 7 43 6.14

4.4 Comparison of results and discussions

4.4.1 Experiment 1: Comparison of feature selection results

Statements & declarations

Authors contributions

Funding

Competing interests

References

Table 2
Description of the datasets

Datasets Features Classes Samples Minority Majority IR

Colon 2000 2 62 22 40 1.82

GLI_85 22283 2 85 26 59 2.27

MLL 12583 3 72 20 52 2.60

TOX_171 5748 4 171 39 132 3.38

GLIOMA 4434 4 50 7 43 6.14