Interaction effect between data discretization and data resampling for class-imbalanced medical datasets

Abstract

Background

Data discretization is an important preprocessing step in data mining for the transfer of continuous feature values to discrete ones, which allows some specific data mining algorithms to construct more effective models and facilitates the data mining process. Because many medical domain datasets are class imbalanced, data resampling methods, including oversampling, undersampling, and hybrid sampling methods, have been widely applied to rebalance the training set, facilitating effective differentiation between majority and minority classes.

Objective

Herein, we examine the effect of incorporating both data discretization and data resampling as steps in the analytical process on the classifier performance for class-imbalanced medical datasets. The order in which these two steps are carried out is compared in the experiments.

Methods

Two experimental studies were conducted, one based on 11 two-class imbalanced medical datasets and the other using 3 multiclass imbalanced medical datasets. In addition, the two discretization algorithms employed are ChiMerge and minimum description length principle (MDLP). On the other hand, the data resampling algorithms chosen for performance comparison are Tomek links undersampling, synthetic minority oversampling technique (SMOTE) oversampling, and SMOTE–Tomek hybrid sampling algorithms. Moreover, the support vector machine (SVM), C4.5 decision tree, and random forest (RF) techniques were used to examine the classification performances of the different approaches.

Results

The results show that on average, the combination approaches can allow the classifiers to provide higher area under the ROC curve (AUC) rates than the best baseline approach at approximately 0.8%–3.5% and 0.9%–2.5% for twoclass and multiclass imbalanced medical datasets, respectively. Particularly, the optimal results for two-class imbalanced datasets are obtained by performing the MDLP method first for data discretization and SMOTE second for oversampling, providing the highest AUC rate and requiring the least computational cost. For multiclass imbalanced datasets, performing SMOTE or SMOTE–Tomek first for data resampling and ChiMerge second for data discretization offers the best performances.

Conclusions

Classifiers with oversampling can provide better performances than the baseline method without oversampling. In contrast, performing data discretization does not necessarily make the classifiers outperform the baselines. On average, the combination approaches have potential to allow the classifiers to provide higher AUC rates than the best baseline approach.

Keywords

class imbalance data mining data resampling discretization machine learning

1 Introduction

Data discretization is an important data preprocessing step in data mining aimed at transferring numerical (or continuous) feature values into categorical (or discrete) ones with finite intervals.^1,2 Many domain problem datasets, such as the temporal and time-series data used in the medical^3,4 and financial fields,^5,6 require data discretization for more effective data mining.

Generally, there are several advantages for implementing data discretization during the data mining procedure. First, some specific data mining algorithms, such as the C4.5 decision tree, naïve Bayes, and association rule techniques, can only deal with categorial values. The performances of some classification algorithms for specific domain problem datasets can be improved through data discretization.^7–9 Second, the mining process becomes more computationally efficient after discretization because the original data are reduced and simplified. Third, the mining results may become more compact and easier to interpret.^10,11

However, although the continuous type of dataset can be handled by data discretization when necessary, many real-world medical datasets containing continuous data possess the characteristic of unequal class distribution. This means that there are different numbers of data samples in different diagnostic classes, sometimes extremely different. This is the so-called class-imbalanced dataset problem.^12–15

In the literature, both data and algorithm level solutions have been used to deal with class-imbalanced datasets. Data-level solutions focus on rebalancing the datasets by either undersampling the majority class or oversampling the minority class. Meanwhile, the algorithm level solutions utilize higher misclassification costs or employ ensemble learning techniques to bias classifier learning toward the minority class.^16–18 Among these solutions, the data resampling approaches, especially oversampling, are the most widely considered because they can be applied independent of the classifier used.^17,19,20

Therefore, when the collected datasets are class imbalanced and contain some continuous features, data resampling or data discretization is used in the preprocessing step. This raises an interesting question that has never been discussed in the literature: what is the interaction effect if both data discretization and data resampling are applied on class-imbalanced datasets?

To answer this question, herein, we implement two different procedures including both data preprocessing steps. In the first procedure, data discretization of the continuous features in the original imbalanced datasets is performed first, followed by rebalancing of the discretized datasets. In the second procedure, rebalancing of the original imbalanced datasets is carried out first, followed by discretization of the continuous features of the rebalanced datasets. The objective of this study is to examine the classification performance of the two different procedures on various class-imbalanced domain problem datasets.

The contributions of this study are twofold. First, comparison of the effectiveness of both novel data discretization and data resampling combinations on two-class and multiclass medical imbalanced datasets is performed. Second, the best procedural order and combined algorithms are identified and can be regarded as a representative baseline approach for future research on class-imbalanced learning.

The rest of this paper is organized as follows. Section 2 provides an overview of the relevant literature on data discretization and data resampling techniques. Section 3 describes the research methodology, and Section 4 presents the experimental results. Finally, Section 5 concludes the paper.

2 Literature review

2.1 Data discretization

Data discretization focuses on transferring continuous variables into discrete ones with a finite set of intervals. Generally, the data discretization process can be divided into several steps. First, for a specific feature to be discretized, its continuous values are sorted. Second, a cut point is evaluated for splitting or adjacent intervals are evaluated for merging. Third, the intervals of continuous values are split or merged based on some predefined criterion. Finally, the process is stopped at some point.

Definition: Assume a dataset S consisting of N examples, M features, and C target classes. A discretization algorithm or discretizer produces a discretization scheme D for the continuous attribute $A \in M$ , which partitions attribute A into k discrete and disjoint intervals: $[d_{0}, d_{1}], [d_{1}, d_{2}], \dots [d_{k - 1}, d_{k}]$ , where $d_{0}$ is the minimal value and $d_{k}$ is the maximal value, and $P = d_{1}, d_{2}, \dots, d_{k - 1}$ is the set of cut points of attribute A in ascending order.

Many discretizers have been proposed in the literature, but not all function equally well. Garcia et al.¹ conducted an empirical study comparing the classification performance of 30 well-known discretizers. They found that the supervised learning-based discretizers performed better than the unsupervised learning ones. Therefore, herein, two supervised learning-based discretizers, namely, ChiMerge and minimum description length principle (MDLP), are used. Both perform relatively well in terms of obtaining a satisfactory tradeoff between the number of intervals produced and the accuracy for all types of classifiers. Both techniques are described below.

2.1.1 Chimerge

The ChiMerge algorithm proceeds as follows: Each distinct value of a numerical attribute is considered as one interval. Then, $x^{2}$ is used as the statistical measure. The value of $x^{2}$ for every pair of adjacent intervals is calculated using

\sum_{i = 1}^{m} \sum_{j = 1}^{k} \frac{{(A_{i j} - E_{i j})}^{2}}{E_{i j}}

(1)

where m is 2 (the two intervals being compared); k is the number of classes;

A_{i j}

is the number of examples in the i-th interval of the j-th class;

R_{j}

is the number of examples in the i-th interval, which is =

\sum_{j = 1}^{k} A_{i j}

;

C_{j}

is the number of examples in the j-th class, which is =

\sum_{i = 1}^{m} A_{i j}

; N is the total number of examples, which is =

\sum_{j = 1}^{k} C_{j}

; and

E_{i j}

is the expected frequency of

A_{i j} = R_{i} \times C_{j} / N

Next, the pair of adjacent intervals with the lowest $x^{2}$ values are merged together. This merging process proceeds recursively until the $x^{2}$ values of all adjacent pairs exceed a threshold, which is determined by the significant level and degrees of freedom.²¹

2.1.2 MDLP

MDLP is an entropy-based supervised discretization method where the feature values are initially sorted and each midpoint between two consecutive values is considered as a new interval. Then, the interval is recursively selected to produce the minimum class information entropy until the stopping criterion is met.²²

Definition: Given a set S of instances, an attribute A, and a cut point T, the class information entropy of the partition induced by T denoted as E(A, T; S) is defined as

E (A, T; S) = \frac{| S_{1} |}{| S |} E n t (S_{1}) + \frac{| S_{2} |}{| S |} E n t (S_{2})

(2)

where

E n t (S_{i})

is the class entropy of the subset

S_{i}

obtained by

E (S_{i}) = - \sum_{j = 1}^{k} P (C_{j}, S_{j}) log (P (C_{j}, S_{j})),

(3)

where there are k classes

C_{1}, \dots, C_{k}

and

P (C_{j}, S_{j})

is the proportion of examples of

S_{j}

that have class

C_{j}

Subsequently, a binary discretization for A is determined by selecting the cut point T for which E(A, T; S) is minimal among all the candidate cut points.

2.2 Data resampling

Data resampling is aimed at balancing the class distribution of the original class-imbalanced datasets. Generally, data resampling algorithms can be divided into undersampling, oversampling, and hybrid sampling categories. For undersampling algorithms, the majority class is reduced, resulting in the same number of examples as in the minority class. In contrast, oversampling algorithms focus on generating synthetic minority class examples to make the majority and minority classes have the same size. Hybrid sampling algorithms perform both reduction of the majority class and enlargement of the minority class.^16,17

Herein, from various undersampling, oversampling, and hybrid sampling algorithms, three well-known algorithms belonging to each category are chosen for performance comparison, namely, Tomek links undersampling, SMOTE oversampling, and SMOTE–Tomek hybrid sampling algorithms. They are described below.

2.2.1 Tomek links

The simplest undersampling method is based on randomly selecting the same number of majority class examples as that of minority class examples. However, this randomness is likely to produce biased results for the reduced majority class. Some systematic techniques have been proposed. One representative technique is the Tomek links procedure, which focuses on finding and removing the majority class examples with the lowest Euclidean distance to the minority class examples.

A Tomek link is defined as follows: Assume an instance pair $(x_{i}, x_{j})$ where $x_{i} \in S_{m i n}, x_{j} \in S_{m a x}$ and $d (x_{i}, x_{j})$ is the distance between $x_{i}$ and $x_{j}$ . This pair $(x_{i}, x_{j})$ is called a Tomek link if there is no instance $x_{k}$ such that $d (x_{i}, x_{k}) < d (x_{i}, x_{j})$ or $d (x_{j}, x_{k}) < d (x_{i}, x_{j})$ . That is, when two instances form a Tomek link, either one of these instances is regarded as a noisy sample or both instances are near the border.²³

2.2.2 Synthetic minority oversampling technique (SMOTE)

SMOTE is one of the most representative oversampling algorithms. It utilizes the k-nearest neighbor algorithm to create synthetic data. First, a random example from the minority class is chosen. Then, the k-nearest neighbors from the chosen data are found (typically k = 5). Finally, synthetic data are generated by interpolation between the random data and the selected k-nearest neighbors.²⁴

There are many variants of SMOTE, for example, Borderline-SMOTE²⁵ and Polynom-fit-SMOTE,²⁶ and this method has been successfully adapted for various domain problem datasets. It is considered as a standard benchmark for learning from class-imbalanced datasets.^19,20

2.2.3 SMOTE–tomek

Because oversampling of the minority class in the process of generating synthetic data examples could lead to an overfitting problem during the classifier training stage, data cleaning (or instance selection) methods must be employed to remove noisy examples from both the majority and minority classes.^27,28

SMOTE–Tomek is one of the hybrid data resampling algorithms. In SMOTE–Tomek, the first step is to apply SMOTE oversampling of the minority class, followed by executing Tomek links for data cleaning over the rebalanced training dataset. That is, the majority class examples of the rebalanced training dataset matching the Tomek links definition are identified and removed.²⁹

2.3 Discussion of related works

Most studies in the literature are related to data discretization and resampling and focus only on one or the other of the two data preprocessing steps. In other words, very few studies simultaneously consider both steps for data preprocessing. Related works that consider data discretization for class-imbalanced datasets are described below.

Tahan and Asadi³⁰ proposed a novel evolutionary multiobjective discretization algorithm for application to two-class imbalanced datasets. Its first objective function uses the area under the ROC curve (AUC) to choose better cut points, while the second objective function reduces the number of cut points. The third objective function is designed to minimize information loss during the data discretization by selecting low-frequency cut points. They assessed the performance of this discretization algorithm using 25 two-class imbalanced datasets. Unfortunately, the effect of performing data resampling on the proposed multiobjective discretization algorithm is unknown.

Zhang et al.³¹ used a quantile discretization method to transfer continuous variables into discrete ones on imbalanced credit scoring datasets, a two-class classification problem. They found that a logistic regression model with proper discretized variables performs better than a cost-sensitive logistic regression model without variable discretization. This work focused on combining data discretization with an algorithm level solution for class-imbalanced datasets rather than finding a data-level solution.

In their work, Jishan et al.³² combined both data discretization and resampling steps to improve the performance of the students’ final grade prediction model for a particular course, a five-class classification problem. Specifically, they applied SMOTE first to rebalance the original imbalanced training set before using the optimal equal-width binning method for data discretization.

Meanwhile, Orhobor et al.³³ and Cano et al.³⁴ employed data discretization first and resampling second. Cano et al.³⁴ introduced a novel discretization algorithm meant to improve the class-attribute interdependence maximization (CAIM) discretization method, which they named ur-CAIM. This method can not only generate more flexible discretization schemes while producing a smaller number of intervals but also improve the quality of the intervals based on the data class distribution. After data discretization, SMOTE is used to rebalance the discretized training set. Similarly, Orhobor et al.³³ performed k-means clustering for data discretization and SMOTE for data resampling over multiclass medical datasets.

Based on the above studies, there are two major limitations. First, although both data discretization and resampling have been applied in different orders and studied individually, their performance has not been fully compared. Second, past combination methods have only been applied to examine specific domain problem datasets for two-class or multiclass classification problems. Therefore, herein, the research objective is to assess the performance of the order in which data discretization and resampling are combined over different two-class and multiclass imbalanced datasets.

3 Research methodology

3.1 Combined data discretization and data resampling procedures

3.1.1 Performing data discretization first and data resampling second

In the first procedural combination, data discretization is performed first and data resampling second. Figure 1 shows the architecture of combining data discretization and data resampling, which is composed of six steps described below.

Figure 1.

Architecture of combining data discretization and data resampling. Step 1: divide the class-imbalanced dataset into training and testing sets. Step 2: perform data discretization over the training set. Step 3: perform data resampling over the discretized training set. Step 4: develop a classifier based on the discretized and balanced training sets. Step 5: perform data discretization over the testing set. Step 6: test the performance of the classifier based on the discretized testing set.

This combination procedure is defined as follows: Given a class-imbalanced dataset D, which is divided into a training set, TR, and a testing set, TE, and each example is represented by F continuous features. TR is composed of M majority class examples and N minority class examples. Then, the F continuous features are transferred into F discrete features individually based on a specific discretizer. The output of this step is a discretized imbalanced training set, denoted as $T R_{d i s c r e t i z e d}$ .

Next, some data resampling algorithm is applied to rebalance $T R_{d i s c r e t i z e d}$ . For oversampling, M–N synthetic examples are generated for the minority. Therefore, a new balanced training set is produced, which is composed of 2 M examples, denoted as $T R_{d i s c r e t i z e d_o v e r - s a m p l e d}$ . In particular, three different preprocessed training sets are produced by combining a specific discretizer with the three different types of data sampling algorithms that are applied over the chosen training set. They are $T R_{d i s c r e t i z e d_u n d e r - s a m p l e d}$ , $T R_{d i s c r e t i z e d_o v e r - s a m p l e d}$ , and $T R_{d i s c r e t i z e d_h y b r i d - s a m p l e d}$ .

Finally, the three different preprocessed training sets are used to construct their individual corresponding classifiers. For classifier testing, the testing set, TE, is discretized based on the cut points identified in $T R_{d i s c r e t i z e d}$ . As a result, the discretized testing set is produced, denoted as $T E_{d i s c r e t i z e d}$ , which is used to test the constructed classifiers.

Figure 2 shows the pseudocode of the procedure to perform data discretization first and data resampling second.

Figure 2.

Pseudocode for performing data discretization first and data resampling second.

3.1.2 Performing data resampling first and data discretization second

In the other procedure, data resampling is performed first and data discretization second. Figure 3 shows the architecture of combining data resampling and data discretization, which is composed of six steps described below.

Figure 3.

Architecture of combining data resampling and data discretization. Step 1: divide the class-imbalanced dataset into training and testing sets. Step 2: perform data resampling over the training set. Step 3: perform data discretization over the balanced training set. Step 4: develop a classifier based on the balanced and discretized training sets. Step 5: perform data discretization over the testing set. Step 6: test the performance of the classifier based on the discretized testing set.

This combination procedure is defined as follows: Some data resampling algorithm is used to rebalance TR first. For undersampling, M–N majority class examples are removed. This results in a new balanced training set, denoted as $T R_{u n d e r - s a m p l e d}$ . Similarly, three balanced training sets are produced based on the three different types of data resampling algorithms for the first step, which are denoted as $T R_{u n d e r - s a m p l e d}$ , $T R_{o v e r - s a m p l e d}$ , and $T R_{h y b r i d - s a m p l e d}$ .

Next, the continuous features of each of the three balanced training sets are transferred into discrete features by the chosen discretizer. The three discretized and balanced training sets obtained can be denoted as $T R_{u n d e r - s a m p l e d_d i s c r e t i z e d}$ , $T R_{o v e r - s a m p l e d_d i s c r e t i z e d}$ , and $T R_{h y b r i d - s a m p l e d_d i s c r e t i z e d}$ .

Finally, these three discretized and balanced training sets are used individually to construct three different classifiers. Then, TE is discretized based on the cut points identified in $T R_{u n d e r - s a m p l e d_d i s c r e t i z e d}$ , $T R_{o v e r - s a m p l e d_d i s c r e t i z e d}$ , and $T R_{h y b r i d - s a m p l e d_d i s c r e t i z e d}$ . As a result, three testing sets are produced, denoted as $T E_{d i s c r e t i z e d_b y_u n d e r}$ , $T E_{d i s c r e t i z e d_b y_o v e r}$ , and $T E_{d i s c r e t i z e d_b y_h y b r i d}$ . For classifier testing, these three testing sets are used to test their corresponding classifiers.

Figure 4 shows the pseudocode of the procedure to perform data resampling first and data discretization second.

Figure 4.

Pseudocode for performing data resampling first and data discretization second.

3.2 Experimental setup

3.2.1 Datasets and experimental environment

There were two experimental studies conducted, one based on 11 two-class imbalanced medical datasets collected from the KEEL dataset repository (https://sci2s.ugr.es/keel/datasets.php) and the other using 3 multiclass imbalanced medical datasets collected from the KEEL and UCI Machine Learning Repository (https://archive.ics.uci.edu/). The relevant information on these datasets is listed in Tables 1 and 2, respectively. Note that the datasets are ordered by their imbalance ratios (IRs).

Table 1.
Related information on the two-class imbalanced medical datasets.

Name IR No. of features (integer/continuous) No. of examples (majority/minority classes) No. of training/testing examples

wisconsin 1.86 9/0 444/239 546/137

Pima diabetes 1.87 6/2 500/268 614/154

yeast1 2.46 0/8 1055/429 1187/297

haberman 2.78 3/0 225/81 245/61

new-thyroid1 5.14 ¼ 180/35 172/43

new-thyroid2 5.14 ¼ 180/35 172/43

yeast3 8.1 0/8 1321/163 1187/297

dermatology-6 16.9 34/0 338/20 286/72

yeast4 28.1 0/8 1433/51 1187/297

yeast5 32.73 0/8 1440/44 1187/297

yeast6 41.4 0/8 1449/35 1187/297

Name	IR	No. of features (integer/continuous)	No. of examples (majority/minority classes)	No. of training/testing examples
wisconsin	1.86	9/0	444/239	546/137
Pima diabetes	1.87	6/2	500/268	614/154
yeast1	2.46	0/8	1055/429	1187/297
haberman	2.78	3/0	225/81	245/61
new-thyroid1	5.14	¼	180/35	172/43
new-thyroid2	5.14	¼	180/35	172/43
yeast3	8.1	0/8	1321/163	1187/297
dermatology-6	16.9	34/0	338/20	286/72
yeast4	28.1	0/8	1433/51	1187/297
yeast5	32.73	0/8	1440/44	1187/297
yeast6	41.4	0/8	1449/35	1187/297

Table 2.

Related information on the multiclass imbalanced medical datasets.

Name (no. of classes)	IR	No. of features (discrete/continuous)	No. of examples (majority/minority classes)	No. of training/testing examples
new-thyroid (3)	4.84	1/4	150/30	144/36
dermatology (6)	5.55	34/0	112/20	106/26
thyroid (3)	36.94	1/5	666/17	546/137

Each dataset is divided into 80% training and 20% testing sets using the fivefold cross validation method. In particular, the training set in each fold has the same IR as the original dataset.

Table 3 shows the hardware equipment and software environments for the following experiments.

Table 3.

Experimental environments.

Hardware	Equipment	Specification
	CPU	11th Gen Intel(R) Core(TM) i5-11300H @ 3.10 GHz and 3.11 GHz
	RAM	16.0 GB
	Graphics card	Intel(R) Iris(R) Xe Graphics NVIDIA GeForce RTX 3060 Laptop GPU
Software	Environment	Description
	Operating system	Windows 11/64-bit
	Development environment	Anaconda & Spyder
	Programming language	Python 3.9.7 & IPython 8.1.1

3.2.2 Data discretization and resampling algorithms

The two discretization algorithms employed are ChiMerge and MDLP. They were implemented using the Scorecard-bundle package from Python and the Orange data mining software, respectively.

The data resampling algorithms chosen for performance comparison are Tomek links undersampling, SMOTE oversampling, and SMOTE–Tomek hybrid sampling algorithms. They were implemented using the Sklearn.Imblearn package from Python.

Related parameters to implement the two discretization and three data resampling algorithms are shown in Table 4.

Table 4.
Related parameters for the two discretization and three data resampling algorithms.

Parameter names Values

ChiMerge min_intervals 5

max_intervals 10

confidence_level 0.9

decimal 3

output_dataframe True

MDLP method EntropyMDL

force True

SMOTE sampling_strategy not majority or minority

random_state 42

k_neighbors 5

Tomek links sampling_strategy 42

SMOTE + Tomek links sampling_strategy not majority

random_state 42

k_neighbors 5

	Parameter names	Values
ChiMerge	min_intervals	5
max_intervals	10
confidence_level	0.9
decimal	3
output_dataframe	True
MDLP	method	EntropyMDL
force	True
SMOTE	sampling_strategy	not majority or minority
random_state	42
k_neighbors	5
Tomek links	sampling_strategy	42
SMOTE + Tomek links	sampling_strategy	not majority
random_state	42
k_neighbors	5

3.2.3 Classifier construction and evaluation

The three classification techniques used to examine the classification performance of the different approaches are the support vector machine (SVM), C4.5 decision tree, and random forest (RF) techniques. In addition, the evaluation metric is based on AUC, which is the most widely used metric in class-imbalance learning studies.^17,27,35 They were implemented using the Sklearn and Chefboost packages from Python. The related parameters for SVM and RF are listed in Table 5. Note that C4.5 is constructed based on the entropy and gain ratio metrics.

Table 5.
Related parameters for SVM, C4.5, and RF.

Parameter names Values

SVM kernel rbf

random_state 42

probability True

C [0.01, 0.1, 1, 10, 100]

gamma [0.01, 0.1, 1, 10, 100]

RF tree algorithm C4.5

n_estimators 100

random_state 42

	Parameter names	Values
SVM	kernel	rbf
random_state	42
probability	True
C	[0.01, 0.1, 1, 10, 100]
gamma	[0.01, 0.1, 1, 10, 100]
RF	tree algorithm	C4.5
n_estimators	100
random_state	42

4 Experimental results

4.1 Results for two-class imbalanced datasets

4.1.1 Baselines

Table 6 shows the average AUC rates of SVM, C4.5, and RF obtained with the baseline approaches over the 35 two-class imbalanced datasets, including the classifiers without any preprocessing, those with SMOTE oversampling, and those with data discretization.

Table 6.
Average AUC rates for SVM, C4.5, and RF obtained through the baseline approaches.

Discretization

w/o preprocessing SMOTE ChiMerge MDLP

SVM 0.632 0.696 0.625 0.613

C4.5 0.686 0.756 0.719 0.713

RF 0.722 0.780 0.719 0.725

Avg 0.680 0.744 0.688 0.684

			Discretization
SVM	0.632	0.696	0.625	0.613
C4.5	0.686	0.756	0.719	0.713
RF	0.722	0.780	0.719	0.725
Avg	0.680	0.744	0.688	0.684

AUC rates obtained for class-imbalanced datasets when SMOTE oversampling was performed first were higher than those obtained for the ones without preprocessing. On average, the improvement in performance was approximately 6.4%. However, performing data discretization, whether ChiMerge or MDLP, did not always improve the performance of the classifiers. That is, performing data discretization without preprocessing did not make a significant level of difference in the performance of the classifier. In particular, only C4.5 could take advantage of data discretization. In summary, the top three approaches are RF by SMTOE (0.780), RF by MDLP (0.725), and RF without preprocessing (0.722).

4.1.2 Combinations of data discretization and oversampling

Table 7 shows the average AUC rates for SVM, C4.5, and RF for both orders of ChiMerge/MDLP and SMOTE combinations. Note that SMTOE–discretization means performing the procedure of SMOTE oversampling first and discretization second, whereas discretization–SMOTE indicates the opposite procedure.

Table 7.
Average AUC rates for SVM, C4.5, and RF obtained by combining ChiMerge–MDLP and SMOTE.

SMOTE–Discretization Discretization–SMOTE

ChiMerge MDLP ChiMerge MDLP

SVM 0.722 0.735 0.710 0.777

C4.5 0.781 0.770 0.772 0.774

RF 0.784 0.773 0.774 0.786

Avg 0.762 0.759 0.752 0.779

	SMOTE–Discretization	Discretization–SMOTE
SVM	0.722	0.735	0.710	0.777
C4.5	0.781	0.770	0.772	0.774
RF	0.784	0.773	0.774	0.786
Avg	0.762	0.759	0.752	0.779

As can be seen in Table 7, the best performing approach is based on RF with MDLP–SMOTE (0.786), whereas the second best one results from RF with SMOTE–ChiMerge (0.784). It is interesting that although the data discretization and oversampling combinations are different, the differences in performance between these two approaches are not significant. Moreover, the combined discretizers are different. This indicates that the choice of algorithms for discretization and oversampling as well as the classifier will affect the choice of the combination order.

Furthermore, Table 8 shows the average F-scores of the approaches. The better performances are based on the RF classifiers, in which the top two approaches are SMOTE–ChiMerge (0.773) and ChiMerge–SMOTE (0.772).

Table 8.

Average F-scores for SVM, C4.5, and RF obtained by combining ChiMerge/MDLP and SMOTE.

	SMOTE–Discretization		Discretization–SMOTE
	ChiMerge	MDLP	ChiMerge	MDLP
SVM	0.702	0.717	0.682	0.71
C4.5	0.713	0.709	0.709	0.724
RF	0.773	0.758	0.772	0.735
Avg	0.729	0.728	0.721	0.723
Average AUC rates for SVM, C4.5, and RF obtained by the baseline approaches.
		Resampling			Discretization
	w/o preprocessing	Tomek Links	SMOTE	SMOTE–Tomek	ChiMerge	MDLP
SVM	0.730	0.736	0.768	0.764	0.836	0.721
C4.5	0.799	0.799	0.829	0.827	0.816	0.767
RF	0.889	0.880	0.900	0.897	0.877	0.769
Avg	0.806	0.805	0.832	0.829	0.843	0.752

4.2 Results for multiclass imbalanced datasets

4.2.1 Baselines

Table 8 shows the average AUC rates for SVM, C4.5, and RF obtained using the baseline approaches over the ten multiclass imbalanced datasets. AUC rates for all classifiers using SMOTE and SMOTE–Tomek for data sampling were higher than those obtained for the ones without preprocessing. In addition, SMOTE slightly outperformed SMOTE–Tomek, while the performance of Tomek links for undersampling was slightly worse than that of the baseline without preprocessing.

Regarding data discretization, ChiMerge was the better choice, making SVM and C4.5 provide higher AUC rates than without preprocessing. However, MDLP performed significantly worse than the baseline approach without preprocessing.

Collectively, the best performing approach was based on SMOTE with RF, and the second best was SMOTE–Tomek with RF. On average, performing only data resampling by itself with either SMOTE or SMOTE–Tomek or data only discretization by ChiMerge is the recommended baseline approaches for multiclass imbalanced datasets.

4.2.2 Data discretization and oversampling combinations

Table 9 shows the average AUC rates for SVM, C4.5, and RF obtained for both ChiMerge/MDLP and Tomek Links/SMOTE/SMOTE–Tomek combinational orders. In the table, CM represents ChiMerge and TL, S, and S-T represent Tomek links, SMOTE, and SMOTE–Tomek, respectively.

Table 9.
Average AUC rates for SVM, C4.5, and RF obtained by combining different discretizers and resampling algorithms.

Discretization–Resampling Resampling–Discretization

CM MDLP TL S S-T

TL S S-T TL S S-T CM MDLP CM MDLP CM MDLP

SVM 0.823 0.866 0.856 0.724 0.787 0.788 0.840 0.763 0.860 0.857 0.870 0.847

C4.5 0.805 0.822 0.824 0.765 0.785 0.785 0.809 0.834 0.827 0.836 0.832 0.831

RF 0.875 0.889 0.889 0.770 0.793 0.793 0.872 0.854 0.901 0.879 0.902 0.879

Avg 0.834 0.859 0.856 0.753 0.788 0.789 0.840 0.817 0.863 0.857 0.868 0.852

	Discretization–Resampling	Resampling–Discretization
SVM	0.823	0.866	0.856	0.724	0.787	0.788	0.840	0.763	0.860	0.857	0.870	0.847
C4.5	0.805	0.822	0.824	0.765	0.785	0.785	0.809	0.834	0.827	0.836	0.832	0.831
RF	0.875	0.889	0.889	0.770	0.793	0.793	0.872	0.854	0.901	0.879	0.902	0.879
Avg	0.834	0.859	0.856	0.753	0.788	0.789	0.840	0.817	0.863	0.857	0.868	0.852

For the procedure where data discretization is performed first and data resampling second, it is better to employ ChiMerge for discretization and SMOTE or SMOTE–Tomek for resampling. That is, there were no significant differences in performance between ChiMerge–SMOTE and ChiMerge–SMOTE–Tomek. Similar to the previous results, the RF classifier using these two approaches outperformed the other two classifiers.

When data resampling is performed first and data discretization second, the best results were obtained when SMOTE–Tomek was employed first and ChiMerge second, and SMOTE–ChiMerge was the second best approach.

Table 10 shows the average F-scores of the approaches. The result is similar to the AUC result obtained on performing data discretization first and data resampling second. The better approaches are based on ChiMerge–SMOTE and ChiMerge–SMOTE–Tomek, whereas SMOTE–ChiMerge and SMOTE–Tomek–ChiMerge for the other procedure.

Table 10.

Average F-scores of the approaches.

	Discretization–Resampling						Resampling–Discretization
	CM			MDLP			TL		S		S-T
	TL	S	S-T	TL	S	S-T	CM	MDLP	CM	MDLP	CM	MDLP
SVM	0.729	0.798	0.778	0.571	0.635	0.635	0.754	0.639	0.782	0.774	0.798	0.759
C4.5	0.706	0.709	0.713	0.639	0.627	0.628	0.704	0.747	0.717	0.726	0.721	0.72
RF	0.807	0.822	0.823	0.651	0.643	0.643	0.804	0.773	0.836	0.807	0.834	0.804
Avg	0.747	0.776	0.771	0.62	0.635	0.635	0.754	0.72	0.778	0.769	0.784	0.761

On average, no significant difference was observed in the level of performance for the top two algorithms in these two procedures. However, when the RF classifier is used, the best choice is to perform SMOTE–Tomek or SMOTE first and ChiMerge second. These combinations significantly outperformed the other approaches and algorithms.

In addition, the findings were different from those obtained for two-class imbalanced datasets, where the optimal procedure was to perform MDLP first and SMOTE second. In contrast, for multiclass imbalanced datasets, first rebalancing multiclass datasets with SMOTE–Tomek or SMOTE in the first stage before data discretization by ChiMerge is advantageous.

4.3 Discussion

Regarding the experimental results of two-class imbalanced datasets, the baseline approach based on performing data resampling by SMOTE makes the classifiers significantly outperform the baselines by performing data discretization by ChiMerge and MDLP alone (p < 0.05) (Statistical analysis is based on the Wilcoxon signed-rank test). Particularly, the average AUC rate of the classifiers by SMOTE is 0.744, and the best baseline is based on the RF classifier, with an average AUC rate of 0.78.

For the combination of data discretization and data resampling, all combination approaches provide higher AUC rates than performing SMOTE alone, with performance improvements of approximately 0.8%–3.5%. Specifically, the top three combination approaches that outperform the best baseline are RF with MDLP–SMOTE (0.786), RF with SMOTE–ChiMerge (0.784), and C4.5 with SMOTE–ChiMerge (0.781), in which their performance differences are not significant.

For the experimental results of multiclass imbalanced datasets, on average, the top three baseline approaches that significantly outperform the other baselines are ChiMerge (0.843), SMOTE (0.832), and SMOTE–Tomek (0.829) alone (p < 0.05). In particular, the RF classifier based on SMOTE provides the highest AUC rate, i.e., 0.9.

For the combination approaches, on average, better performances than the best baseline approach, i.e., ChiMerge, are based on SMOTE–Tomek–ChiMerge (0.868), SMOTE–ChiMerge (0.863), ChiMerge–SMOTE (0.859), SMOTE–MDLP (0.857), ChiMerge–SMOTE–Tomek (0.856), and SMOTE–Tomek–MDLP (0.852). The performance improvements are approximately 0.9%–2.5%. Specifically, the RF classifiers based on SMOTE–Tomek–ChiMerge (0.902) and SMOTE–ChiMerge (0.901) perform the best, which are better than RF with SMOTE.

In addition to examining the classification performances of the approaches, Figure 5 shows the average processing times for each of the four combination approaches.

Figure 5.

Average processing times (s) of the combination approaches.

The results show that ChiMerge requires a significant amount of computational time for data discretization. Therefore, for two-class imbalanced datasets, performing MDLP first and SMOTE second is the optimal procedure. However, for multiclass imbalanced datasets, to obtain relatively higher classification accuracies by the two best combination approaches, i.e., the RF classifier based on SMOTE–Tomek–ChiMerge and SMOTE–ChiMerge, higher computational complexities are required.

5 Conclusion

Herein, we focus on applying both data discretization and data resampling on class-imbalanced medical datasets and examine their effects. Two experimental studies using two-class and multiclass imbalanced medical datasets were performed.

For two-class imbalanced datasets, two discretizers, namely, ChiMerge and MDLP, and the SMOTE oversampling algorithm were used in different combinations. The results showed that classifiers with oversampling can provide better performances than the baseline method without oversampling. In contrast, performing data discretization does not necessarily make the classifiers outperform the baselines.

For the combinations of different data discretization and data resampling algorithms, it was found that most of the different ChiMerge/MDLP and SMOTE combinations outperformed the procedures using only SMOTE or only ChiMerge/MDLP. On average, the combination approaches can allow the classifiers to provide higher AUC rates than the best baseline approach at approximately 0.8%–3.5%. Particularly, the top two approaches are based on performing MDLP first and SMOTE second, i.e., MDLP–SMOTE, and SMOTE first and ChiMerge second, i.e., SMOTE–ChiMerge. However, because performing ChiMerge requires a much larger computational cost, MDLP–SMOTE is recommended for two-class imbalanced datasets.

For multiclass imbalanced datasets, the SMOTE oversampling and SMOTE–Tomek hybrid sampling procedures performed better than the baseline approaches with Tomek links undersampling and without data resampling. However, similar to the results for two-class imbalanced datasets, the application of ChiMerge and MDLP for data discretization did not always lead to better classifier performance than the case without data discretization.

For the combinations of data discretization and data resampling, on average, several combination approaches made the classifiers outperform the best baseline approach with performance improvements of approximately 0.9%–2.5%. Specifically, both procedures using ChiMerge for data discretization and SMOTE or SMOTE–Tomek for data resampling showed better performance. However, when an RF classifier was used, the combination where SMOTE or SMOTE–Tomek was executed first and ChiMerge second significantly outperformed the other classifiers using the opposite procedure combining ChiMerge and SMOTE/SMOTE–Tomek.

There are some limitations of this work that can be regarded as future research areas. First, the chosen experimental datasets do not contain very high feature dimensions. However, for some specific domain datasets, such as cancer-related data based on microarray gene expressions composed of several thousands of attributes, feature selection is an important step to filter unrepresentative features.^36–39 Therefore, it would be worth examining the feature selection effect on the combination of data discretization and data resampling. Second, for data resampling, we only rebalanced the original training datasets based on the imbalance ratio of 1:1. However, this may not be the best setting for different domain datasets. Therefore, it would be interesting to examine the data discretization effect using different IR settings. Third, because the choice of classification techniques is an important factor affecting the final classification performance as well as the identification of the best combination of data discretization and data resampling algorithms, other advanced techniques such as ensemble classifiers and deep learning models⁴⁰ can be constructed for further performance comparison.

Footnotes

Acknowledgments

The work was supported in part by the Ministry of Science and Technology of Taiwan under Grant MOST 111-2410-H-008-027-MY3 and MOST 111-2410-H-182-015-MY3 and in part by the Chang Gung Memorial Hospital at Linkou under Grant BMRPH13.

ORCID iD

Wei-Chao Lin

Funding

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Garcia

Luengo

Saez

, et al. A survey of discretization techniques: taxonomy and empirical analysis in supervised learning. IEEE Trans Knowl Data Eng 2013; 25: 734–750.

Ramirez-Gallego

Garcia

Mourino-Talin

, et al. Data discretization: taxonomy and big data challenge. WIRES Data Min Knowl Discov 2016; 6: 5–21.

Aristodimou

Diavastos

Pattichis

. A fast supervised density-based discretization algorithm for classification tasks in the medical domain. Health Informatics J 2022; 28. doi:https://doi.org/10.1177/14604582211065397

Jann

Vera-Licona

. Benchmarking time-series data discretization on inference methods. Bioinformatics 2019; 35: 3102–3109.

Pal

Kar

. Time series forecasting for stock market prediction through data discretization by fuzzistics and rule generation by rough set theory. Math Comput Simul 2019; 162: 18–30.

Silva

DMB

Pereira

GHA

Magalhaes

. A class of categorization methods for credit scoring models. Eur J Oper Res 2022; 296: 323–331.

Bay

. Multivariate discretization for set mining. Knowl Inf Syst 2001; 3: 491–512.

Hayashi

Setiono

Azcarraga

. Neural network training and rule extraction with augmented discretized input. Neurocomputing 2016; 207: 610–622.

Rahman

Islam

. Discretization of continuous attributes through low frequency numerical values and attribute interdependency. Expert Syst Appl 2016; 45: 410–423.

10.

Liu

Hussain

Lim

, et al. Discretization: an enabling technique. Data Min Knowl Discov 2002; 6: 393–423.

11.

Tsai

C-F

Chen

Y-C

. The optimal combination of feature selection and data discretization: an empirical study. Inf Sci (Ny) 2019; 505: 282–293.

12.

Araf

Idri

Chairi

. Cost-sensitive learning for imbalanced medical data: a review. Artif Intell Rev 2024; 57: 80.

13.

Liu

, et al. Solving the class imbalance problem using ensemble algorithm: application of screening for aortic dissection. BMC Med Inform Decis Mak 2022; 22: 82.

14.

Suresh

Brijet

Subha

. Imbalanced medical disease dataset classification using enhanced generative adversarial network. Comput Methods Biomech Biomed Engin 2023; 26: 1702–1718.

15.

Welvaars

Oosterhoff

JHF

van den Bekerom

MPJ

, et al. Implications of resampling data to address the class imbalance problem (IRCIP): an evaluation of impact on performance between classification algorithms in medical data. JAMIA Open 2023; 6: ooad033.

16.

Das

Datta

Chaudhuri

. Handling data irregularities in classification: foundations, trends, and future challenges. Pattern Recognit 2018; 81: 674–693.

17.

Galar

Fernandez

Barrenechea

, et al. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans Syst May Cybern: Part C – Appl Rev 2012; 42: 463–484.

18.

Khairy

Mahmoud

Abd El-Hafeez

. The effect of rebalancing techniques on the classification performance in cyberbullying datasets. Neural Comput Appl 2023; 36: 1049–1065.

19.

Fernandez

Garcia

Herrera

, et al. SMOTE For learning from imbalanced data: progress and challenges, marking the 15-year anniversary. J Artif Intell Res 2018; 61: 863–905.

20.

Kovacs

. An empirical comparison and evaluation of minority oversampling techniques on a large number of imbalanced datasets. Appl Soft Comput 2019; 83: 105662.

21.

Kerber

. Chimerge: discretization of numeric attributes. In: Proceedings of the 10th national conference on artificial intelligence, 1992, pp.123–128.

22.

Fayyad

Irani

. Multi-interval discretization of continuous-valued attributes for classification learning. In: International joint conference on artificial intelligence, pp.1022–1029.

23.

Tomek

. Two modifications of CNN. IEEE Trans Syst Man Cybern 1976; 6: 769–772.

24.

Chawla

Bowyer

Hall

, et al. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 2002; 16: 31–357.

25.

Han

Wang

W-Y

Mao

B-H

. Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: De-Shuang Huang, Xiao-Ping Zhang, and Guang-Bin Huanget al. (eds) Advanced in intelligent computing. Berlin, Heidelberg: Springer, 2005, pp.878–887.

26.

Gazzah

Amara

NEB

. New oversampling approaches based on polynomial fitting for imbalanced data sets. In: IAPR International workshop on document analysis systems, 2008, pp.677–684.

27.

Lin

Tsai

C-F

Lin

W-C

. Towards hybrid over- and under-sampling combination methods for class imbalanced datasets: an experimental study. Artif Intell Rev 2023; 56: 845–863.

28.

Yang

Wang

Sun

, et al. A hybrid sampling algorithm combining synthetic minority over-sampling technique and edited nearest neighbor for missed abortion diagnosis. BMC Med Inform Decis Mak 2022; 22: 344.

29.

Batista

GEAPA

Prati

Monard

. A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explor Newslett 2004; 6: 20–29.

30.

Tahan

Asadi

. EMDID: evolutionary multi-objective discretization for imbalanced datasets. Inf Sci (Ny) 2018; 432: 442–461.

31.

Zhang

Ray

Priestley

, et al. A descriptive study of variable discretization and cost-sensitive logistic regression on imbalanced credit data. J Appl Stat 2020; 47: 568–581.

32.

Jishan

Rashu

Haque

, et al. Improving accuracy of students’ final grade prediction model using optimal equal width binning and synthetic minority over-sampling technique. Decis Anal 2015; 2: 1.

33.

Orhobor

Grinberg

Soldatova

, et al. Imbalanced regression using regressor-classifier ensembles. Mach Learn 2023; 112: 1365–1387.

34.

Cano

Nguyen

Ventura

, et al. ur-CAIM: improved CAIM discretization for unbalanced and balanced data. Soft Comput 2016; 20: 173–188.

35.

Branco

Torgo

Ribeiro

. A survey of predictive modeling on imbalanced domains. ACM Comput Surv 2016; 49: 31.

36.

Farghaly

Ali

Abd El-Hafeez

. Developing an efficient method for automatic threshold detection based on hybrid feature selection approach. Proc 9th Comput Sci On-line Conf 2020; 2: 56–72.

37.

Farghaly

Abd El-Hafeez

. A new feature selection method based on frequent and associated itemsets for text classification. Concurr Comput: Pract Exp 2022; 34: e7258.

38.

Farghaly

Abd El-Hafeez

. A high-quality feature selection method based on frequent and correlated items for text classification. Soft Comput 2023; 27: 11259–11274.

39.

Mostafa

Mahmoud

Abd El-Hafeez

, et al. Feature reduction for hepatocellular carcinoma prediction using machine learning algorithms. J Big Data 2024; 11: 88.

40.

El Koshiry

Eliwa

EHI

Abd El-Hafeez

, et al. Detecting cyberbullying using deep learning techniques: a pre-trained glove and focal loss technique. Peer J Comput Sci 2024; 10: e1961.

Interaction effect between data discretization and data resampling for class-imbalanced medical datasets

Abstract

Background

Objective

Methods

Results

Conclusions

Keywords

1 Introduction

2 Literature review

2.1 Data discretization

2.1.1 Chimerge

2.2.1 Tomek links

2.2.2 Synthetic minority oversampling technique (SMOTE)

2.2.3 SMOTE–tomek

2.3 Discussion of related works

3 Research methodology

3.1 Combined data discretization and data resampling procedures

3.1.1 Performing data discretization first and data resampling second

3.2.1 Datasets and experimental environment

Table 5. Related parameters for SVM, C4.5, and RF. Parameter names Values SVM kernel rbf random_state 42 probability True C [0.01, 0.1, 1, 10, 100] gamma [0.01, 0.1, 1, 10, 100] RF tree algorithm C4.5 n_estimators 100 random_state 42

4.1 Results for two-class imbalanced datasets

4.1.1 Baselines

Table 6. Average AUC rates for SVM, C4.5, and RF obtained through the baseline approaches. Discretization w/o preprocessing SMOTE ChiMerge MDLP SVM 0.632 0.696 0.625 0.613 C4.5 0.686 0.756 0.719 0.713 RF 0.722 0.780 0.719 0.725 Avg 0.680 0.744 0.688 0.684

Table 7. Average AUC rates for SVM, C4.5, and RF obtained by combining ChiMerge–MDLP and SMOTE. SMOTE–Discretization Discretization–SMOTE ChiMerge MDLP ChiMerge MDLP SVM 0.722 0.735 0.710 0.777 C4.5 0.781 0.770 0.772 0.774 RF 0.784 0.773 0.774 0.786 Avg 0.762 0.759 0.752 0.779

4.2.1 Baselines

4.2.2 Data discretization and oversampling combinations

Footnotes

Acknowledgments

ORCID iD

Funding

Declaration of conflicting interests

References

Table 5.
Related parameters for SVM, C4.5, and RF.

Parameter names Values

SVM kernel rbf

random_state 42

probability True

C [0.01, 0.1, 1, 10, 100]

gamma [0.01, 0.1, 1, 10, 100]

RF tree algorithm C4.5

n_estimators 100

random_state 42

Table 6.
Average AUC rates for SVM, C4.5, and RF obtained through the baseline approaches.

Discretization

w/o preprocessing SMOTE ChiMerge MDLP

SVM 0.632 0.696 0.625 0.613

C4.5 0.686 0.756 0.719 0.713

RF 0.722 0.780 0.719 0.725

Avg 0.680 0.744 0.688 0.684

Table 7.
Average AUC rates for SVM, C4.5, and RF obtained by combining ChiMerge–MDLP and SMOTE.

SMOTE–Discretization Discretization–SMOTE

ChiMerge MDLP ChiMerge MDLP

SVM 0.722 0.735 0.710 0.777

C4.5 0.781 0.770 0.772 0.774

RF 0.784 0.773 0.774 0.786

Avg 0.762 0.759 0.752 0.779