Hybrid Machine Learning Approach for Early Diagnosis of Polycystic Ovary Syndrome with Stable Features

Abstract

Polycystic Ovary Syndrome (PCOS) is a hormonal condition that typically affects female during the time of their reproduction. It is identified by the disruptions in hormonal balance, particularly an increase in levels of androgen (male hormone) in the female body. PCOS can lead to various symptoms and health complications including irregular menstrual cycles, ovarian cysts, fertility issues, insulin resistance, weight gain, acne, and excess hair growth. The real-world PCOS detection is a challenging task whilst PCOS specific cause is unknown and its symptoms are unclear. Thus, accurate and timely diagnosis of PCOS is crucial for effective management and prevention of long-term complications. In such cases, Machine learning based PCOS prediction model support diagnostic process, address potential errors and time constraints. Machine learning algorithms can analyze large set of patient data, including medical history, hormonal profiles, and imaging results, to assist in the diagnosis of PCOS. In particular, the performance of data analysis chore and prediction model is improved by ensemble feature selection strategies. These methods concentrate on selecting a subset of pertinent features from a broader range of features. The unstable nature of the outcome of feature selection algorithm is a frequent issue in practical applications, when it is applied multiple times on similar dataset or with slight modifications in the data. Thus, evaluating the robustness of feature selection algorithm is most important. To address these issues and quantify the robustness, this study uses Jenson-Shannon divergence, an information theoretic approach with ensemble feature selection method to handle the various findings, such as complete ranking, half ranking and top-k lists (without ranking). Furthermore, this article proposes a hybrid machine learning classifier with SMOTE – SVM for the prompt detection of PCOS and the performance of the model is compared with a number of other individual classifiers including KNN (K-Nearest Neighbour), Support Vector Machine (SVM), AdaBoost, LR –Logistic Regression, NB –Nave Bayes, RF –Random Forest, Decision Tree. The proposed SWISS-AdaBoost classifier surpassed other models with 97.81% of accuracy and AUC of 99.08%.

Keywords

Polycystic ovary syndrome (PCOS)Jenson-shannon divergence SVM (Support Vector Machine)K-nearest neighbour logistic regression decision tree naive bayes and AdaBoost

1 Introduction

PCOS stands as a prevalent endocrine condition impacting women in their childbearing years, distinguished by persistent irregularities in hormonal secretion. Numerous cysts develop inside the ovaries as a result of these abnormalities. This condition is widespread, affecting women of diverse ethnicities and geographic regions. It not only affects fertility but also causes high sleep apnea, cholesterol levels, type II diabetes, anxiety, high bp (blood pressure) and etc., between 5% and 20% of women of reproductive age around the world have PCOS [6, 16]. The global prevalence of PCOS disease is influenced by various factors, like genetic predisposition, lifestyle and environmental factors, as well as shifts in diagnostic standards and public awareness. It’s also crucial that PCOS may not be properly or fully diagnosed. Machine Learning has emerged as a transformative force in healthcare, offering significant advancements in disease diagnosis. Due to its capacity to analyses complicated data and spot patterns that may be challenging for human clinicians to recognize, it is crucial in the detection and diagnosis of PCOS. Machine learning models increased accuracy can result in earlier and more precise diagnosis, which will benefit PCOS patients overall [11]. Healthcare providers can identify PCOS more accurately and quickly by utilizing the analytical capabilities of machine learning algorithms. The prognosis and quality of life of PCOS sufferers are eventually improved by the early detection, timely intervention, and specialized management measures.

The accuracy, stability, and dependability of feature selection procedures have been improved with the ensemble feature selection methods in various areas, including healthcare, finance, and image analysis. These techniques offer a strong and complete framework for choosing the most instructive features and enhancing overall predictive modeling by utilizing the collective wisdom of several methodologies [19, 20]. The instability of the results when using feature selection algorithms repeatedly on similar dataset or with minor data adjustments is a prevalent problem with this approach. Stability evaluation tests must be included in feature selection procedures in order to guarantee their stability. These tests assist in determining the performance of ensemble feature selection techniques when it is used frequently on comparable dataset or when the data undergoes slight modifications.

A hybrid machine learning approach combines the prediction of multiple machine learning algorithms to solve a specific problem. The ability to leverage the strengths of different algorithms and techniques makes this approach as powerful tools for solving complex problems and achieving superior performance compared to using a single algorithm alone.

The following are the key contribution of this article:

1. 1.
Propose an Ensemble feature selection technique to find most important features and increase the model’s accuracy
2.
Employs Jenson Shannon approach, an information theoretic strategy with three alternative results (complete ranked list, half ranked list, and top –k list) for the feature selection strategies stability evaluation.
3.
Implement a hybrid machine learning model to quickly identify PCOS. The performance of the suggested model is compared to that of conventional machine learning classifiers, such as Decision Tree, NB (Naive Bayes), Support Vector Machine, AdaBoost, KNN (K-Nearest Neighbour), Logistic Regression and Random Forest.

This article is organized as follows. Section 2 illustrates the different proposed approach for stability assessment of feature selection methods and also it discusses the various Machine Learning techniques used for PCOS Detection. Methods and Methodologies used in this article are discussed in section 3. Proposed model is explained in section 4. Dataset description and preprocessing is discussed in section 5. Performance of the proposed model is presented in section 6 and section 7 concludes the research work.
2 Literature Survey

2.1 Feature selection methods and stability analysis

Deepak et al., [3] proposed RSt (Rank stability) a frequency-based stability measure to evaluate the ensemble FS (feature selection) algorithm according to feature ranking and feature subsets. Also, the proposed RSt satisfies the essential properties such as fully defined, strict monotonicity, minimum and maximum bounds and correction for change. They have performed the experiments on 11 different real-world dataset with ensemble-based feature selection methods and employed AdaBoost and SVM classifiers for testing the accuracy over traditional feature selection techniques. Utkarsh et al., [8] Analysis of the various feature selection methods (filter, wrapper, and embedding), as well as several stability metrics, such as –stability by index, stability by rank, and stability by weight, are presented. The authors also addressed the solutions to address the instability of feature selection methods. Rocío et al., [14] In order to assess the stability of feature selection techniques for different outputs (Full Ranked, partial and Top –k list), Jenson Shannon divergence method, based on information theory, has been introduced. To assess the reliability of feature selection approach outcomes, the authors used food quality dataset and compared the results with two different stability analysis methods Spearman’s rank correlation and KI (Kunchava’s index) and proved that Jenson Shannon approach is suitable for testing the different outcomes with feature ranking. Barbara [15] proposed ensemble ranking approach with bootstrap aggregation for feature extraction. Stability of each feature selection technique has been evaluated with different permutation levels of data. The authors used two different models includes such as Support Vector Machine (SVM) and RF –Random Forest over 18 different datasets and evaluated the performance.

2.2 Hybrid based machine learning techniques

Ritika et al., [1] proposed SMOTE with stacked hybrid model for early diagnosis of PCOS. To build a hybrid model, used six different classifiers as base learner including AdaBoost, Support Vector Machine (SVM), Naïve Bayes, LR –Logistic Regression, Decision Tree (DT), Random Forest and performed meta level implementation. For feature extraction, a backward feature elimination technique is employed. The authors compared the outcomes of the suggested stacking techniques with those of a single classifier. Among all the stacking classifier, Stack –Ada boost method outperformed with 90.24% accuracy. Hela et al., [2] To address the class imbalance problem, the authors employed SMOTE and Edited Nearest Neighbor (ENN). To extract the necessary features and reduce the dimensionality, Recursive Feature Elimination –RFE and Random Forest is used. The experimentation was performed with two different training and testing ratio of 70 : 30 and 80 : 20 and the proposed stacking classifier with Random Forest produces the highest result of 100 % than other models. Sayma et al., [4] proposed stacking ensemble model with one meta model as boosting model and considered five traditional models as base learner. Principle Component Analysis (PCA), Chi –Square method and RFE –Recursive Feature Elimination is used as technique for feature selection. The proposed system output is compared with the results of the conventional machine learning model. Stacking –Gradient Boosting model produced 95.7 % accuracy than other models. Laith Alzubaidi et al., [5] addressed the issue of limited training data (data scarcity) in various applications and provided the different solutions including GAN (Generative Adversarial Network), DeepSMOTE –Deep Synthetic Minority Oversampling Technique etc.,. Homay et al., [10] used SBS –sequential backward selection, Pearson correlation and Random Forest to explore the important features. The effectiveness of the classifier with fewer attribute is assessed, and the outcomes are compared with Ensemble RF (Random Forest), ET (Extra Tree), AdaBoost and MLP (Multilayer Perceptron). Amsy Denny et al., [17] used PCA (Principle Component Analysis) to identify the most relevant features (8 features) and employed different traditional machine learning approaches includes CART –Classification and Regression Tree, Support Vector Machine, KNN (K Nearest Neighbor) and (RF) Random Forest. The authors achieved 89.02% accuracy with Random Forest than other approaches.

Previous studies have shown a tendency to find the best subset of features from a given dataset using feature reduction techniques. These reduced feature subsets are then utilized for machine learning classification. However, only a small amount of analysis has been done on various reduced feature subsets that include various combinations and amounts of features from the total dataset. Additionally, there hasn’t been considerable study on how different machine learning classification strategies perform when employing different feature subsets obtained from various feature reduction methods. Furthermore, the robustness of feature selection methodologies in managing a variety of outcomes has not been properly evaluated.

This research aims to bridge the existing research gaps by focusing on the effective and efficient PCOS –Polycystic Ovary Syndrome detection through the optimal feature selection. To accomplish this, the proposed work employs Jenson Shannon Divergence to handle different outcomes produced by ensemble feature selection methods and evaluate its stability and also hybrid stacking classifier is implemented for the classification purposes. The developed stacking classifier is trained and evaluated to ensure its effectiveness.

3 Methods & Methodology

This section presents the ensemble feature selection methods and its stability assessment. Also discusses the hybrid machine learning technique, where diverse traditional models are combined in a novel way to enhance predictive performance.

3.1 Ensemble feature selection techniques

Feature selection is an extremely effective technique for increasing the level of accuracy of predictive algorithms. It ensures increased model performance with reduced computational times by diligently choosing the most important elements. Therefore, an exhaustive selection method is used to extract the best set of features from the PCOS dataset using ensemble filter and ensemble wrapper feature selection techniques.

3.1.1 Ensemble filter methods

Ensemble feature selection combines multiple individual filter methods to collectively determine the importance of features in a dataset. These techniques are used to exploit the diverse perspectives of different filter methods to identify the most relevant and informative features for a given machine learning task. It overcomes the limitations of individual filter methods and provides a more robust feature selection process [7, 20]. In this article, high performed individual filter methods for the provided dataset includes chi –square, Relief, fisher score and mRmR has been combined to determine the important features.

3.1.2 Ensemble wrapper method

Ensemble wrapper methods for feature selection are the combination of outputs of multiple individual feature selection techniques with the predictive power of a machine learning model. It integrates the feature selection process into the model’s training and evaluation, using the model’s performance as the criteria for selecting the best features. In order to determine the highly important features, the proposed model used Recursive Feature Elimination (RFE) with a variety of different classifier models. RFE-SVM (Support Vector Machine), RFE-RF (Random Forest), RFE-ET (Extra Tree), and RFE-LR (Linear Regression) are among the best performing classifier models for the provided dataset.

3.2 Stability assessment –feature selection method

The stability of a feature selection technique reflects its capacity to consistently recognize pertinent features and maintain their order of importance across diverse samples. A stable feature selection method is characterized by producing consistent outcomes when applied to various data subsets, enhancing its reliability and trustworthiness. To ensure the robustness of feature selection algorithm, this article employs Jenson –Shannon divergence, an information theoretic based approach with ensemble feature selection methods to handle the different outcomes such as complete ranked, half ranked and top –k list. This metric possesses the desired properties includes upper/lower bound, correction for change and maximum. The following subsection explains the stability assessment of different results produced by ensemble feature selection techniques.

3.2.1 Complete ranked list –Jenson shannon divergence

For the stability assessment, the outcome of the feature selection process is converted into a probability distribution [14]. Feature ranking approaches assigns the highest weight to the attributes at the top of the list. As their rank is established by the outcomes of the feature ranking algorithm, the likelihood reduces gradually in accordance with that ranking. This ensures that the most important features are given greater emphasis while those lower in rank receive comparatively lower weights. Thus, following [14], for the complete ranked list, rv = (r ₁, r ₂, r ₃, … … … r _n) the ranking vector, is mapped into pv = (p ₁, p ₂, p ₃, … … … p _n) the probability vector with $p v_{i} = \frac{1}{2 t} (1 + \sum_{j = 0}^{t - r_{i}} {(r_{i} + j)}^{- 1})$ (1)

Where, $\sum_{x = 1}^{t} p v_{i} = 1$ . Here, r_i is the rank assigned to feature i in run j, and t denotes the total number of features.

The degree of similarity between the distributions produced by several runs of a feature ranking algorithm can be assessed using the Jensen-Shannon divergence. Given a set of ‘n’ distributions (p ₁, p ₂, … … … p _n), each of which represents a run of a particular feature ranking algorithm represented as $D_{S} (p_{1}, p_{2}, \dots \dots \dots p_{n}) = \frac{1}{n} \sum_{j = 1}^{n} \sum_{i = 1}^{t} p_{ij} log \frac{p_{ij}}{{\bar{p}}_{i}}$ (2)

The feature i’s average probability is represented by ${\bar{p}}_{i}$ and the ranking output j’s feature and i’s probability is represented by p_ij .

Stability metric SJ_s for stability analysis is expressed as: $S J_{s} (p_{1}, p_{2}, \dots \dots \dots p_{n}) = 1 - \frac{D_{S} (p_{1}, p_{2}, \dots \dots \dots p_{n})}{{D_{S}}^{*} (p_{1}, p_{2}, \dots \dots \dots p_{n})} .$ (3)

Where, D_S –represents Jenson Shannon Divergence with ‘n’ ranking output and the divergence value for a completely random ranking generation is represented by $D_{S}^{*}$ and it is written as ${D_{S}}^{*} (p_{1}, p_{2}, \dots \dots \dots p_{n}) = \sum_{i = 1}^{t} p_{i} log (p_{i} t)$ (4)

Where p_i indicates features probability with rank.

3.2.2 Half ranked list –Jenson shannon divergence for the stability

For the half ranked list, converting the feature ranking’s output into a probability distribution is done by, $p v_{i} = {\begin{matrix} \frac{1}{2 n} (1 + \sum_{j = 0}^{n = r_{i}} {(r_{i} + j)}^{- 1}) & if r_{i} \leq k \\ 0 & otherwise \end{matrix}$ (5)

SJs is calculated using Equation (3), D_S , $D_{S}^{*}$ is calculated using Equations (2) and (4).

3.2.3 Top K –Jenson shannon divergence

For the feature subset with top –k list, converting the feature ranking’s output into a probability distribution is done by, $p v_{i} = {\begin{matrix} \frac{1}{n} & if r_{i} \leq n \\ 0 & otherwise \end{matrix}$ (6)

And $D_{S}^{*}$ is calculated using, ${D_{S}}^{*} (p_{1}, p_{2}, \dots \dots \dots p_{n}) = log (\frac{t}{n})$ (7)

SJs is calculated using Equation (3) and D_S is calculated using Equation (2).

3.3 Hybrid stacking machine learning model

Hybrid stacking model is an approach for combining base level classifier predictions and providing them as new input to the meta level classifier. In this article, K-Nearest Neighbor, Support Vector Machine, Naïve Bayes, Random Forest, Logistic Regression, Ada Boost and Decision Tree are considered as the base level classifier and one of these models are selected as meta level classifier.

4 Proposed Model

PCOS is a disease marked by the presence of excess androgen and ovarian dysfunction. Multiple ovarian cysts, irregular menstrual periods, elevated testosterone levels, and insulin resistance are the symptoms of Polycystic Ovary Syndrome (PCOS). It is caused by a confluence of genetic and environmental factors, while the specific origin is yet unknown. PCOS is associated with a number of health issues, including infertility, irregular menstruation, obesity, diabetes, and cardiovascular disease [6]. For enhanced outcomes and the prevention of complications, early PCOS diagnosis is essential. Leveraging machine learning models can be very helpful in medical diagnostics, assisting medical practitioners in detecting patients with PCOS [9]. To establish an effective treatment approach, the proposed work focused on two main phases (Fig. 1).

1. 1.
Ensemble feature selection technique with Stability Assessment
2.
Hybrid stacking machine learning model

Fig. 1
Proposed framework for PCOS detection.
4.1 Ensemble feature selection techniques with stability assessment

The effectiveness of data analysis tasks and prediction models is greatly enhanced by ensemble feature selection methods. These methods emphasize on extracting a smaller number of pertinent features from a bigger pool of available features. In this article, both Ensemble filter and Ensemble wrapper method are implemented. The method with highest stability is considered for the model training. Algorithm 1 explains the procedure of Ensemble Filter Method, it is a feature selection procedure that enhances the robustness and efficacy of feature selection by combining the results of various distinct filter methods and it also used to reduce feature dimensionality for better model performance. To produce a more accurate feature ranking, it makes use of the many perspectives offered by various filtering techniques. According to the ranked lists relevancy ratings, the significant features are chosen using proposed ensemble filter approach, which combines the results of best performed individual filters such as Chi-Square, Relief, Fisher Score, Minimum Redundancy and Maximum Relevance.

Algorithm 1 – Ensemble Filter Method
Input data: Training dataset D (v _m, w _n) // v instances
and w features
A set of t individual filter methods (Relief
(Rf), Chi –Square (χ²), Fisher Score (FS)
and mRmR)
Output: A ranked list of features based on their
Importance Scores (IS) obtained from the
Ensemble Filter Method
Calculate the Feature Importance Scores (FIS)
1. for t ← 1 to T do
Apply t to D (v _m, w _n) and calculate FIS
Normalize the FIS (NFIS) of range [0, 1]
2. Create an Ensemble Feature Importance Matrix
(EFIM) w.r.t (v × w)
3. for each instance i in D (v _m, w _n):
Initialize Feature Importance Vector FIV_i
w.r.t w // representing the IS of all features
for instance i
For each individual filter method j
Retrieve the NFIS for instance i from
the j-th filter method:
Rf: V_i = V_i - diff (s_i - nearHit_i) ² + diff (s_i - nearMiss_i) ²
$χ^{2} : χ^{2} = \sum \frac{{(X_{i} - W_{i})}^{2}}{W_{i}}$
FS: $F = \frac{\sum_{i = 1}^{j} a_{i}^{X} {(μ_{i} - μ)}^{2}}{\sum_{i = 1}^{j} y_{i} {σ_{i}}^{2}}$
mRmR: mR(Max Relevance):
$max i (Y_{I}, Y_{I}) = \frac{1}{\| Z \|} \sum_{k \in Z} I (f, k)$
mR(Min Redundancy):
$min i (Q_{I}, Q_{I}) = \frac{1}{\| Z \|^{2}} \sum_{k, j \in Z} I (f, k)$
4. Add the IS of FIV_i w.r.t each element
5. Determine the final FIS from intersection of IS for each set of feature for all methods
6. Rank all the features

Algorithm 2 explains the procedure of Ensemble wrapper method; it generates various feature subsets, trains distinct machine learning models for each subset, and evaluates the performance of each model. The optimal feature subset of the top-performing model is then chosen as the final set of features to be utilized for training the machine learning model. Recursive Feature Elimination (RFE) is used in this proposed work to discover significant features using a variety of different classifier models. RFE-SVM (Support Vector Machine), RFE-RF (Random Forest), RFE-ET (Extra Tree), and RFE-LR (Linear Regression) are the best performed classifier models that were used. RFE is used iteratively with each classifier to choose the most pertinent features, improving the dataset comprehension and interpretability for the individual models.

Algorithm 2 – Ensemble Wrapper Method
Input: X: The input feature matrix with n samples and m features represented as X $\in ℝ^{\land}$ (n×m)
Y: The corresponding target vector represented as Y $\in ℝ^{\land} n$
M: The number of models to be generated
Output: F_best: The final selected subset of features represented as F_best ⊆ {1, 2,..., m}
1. Initialize an empty set F_best = {} to store the final selected features
2. For i = 1 to M:
a. Randomly select a subset of features F_i ⊆ {1, 2,..., m} with size z, where z is the number of features to include in each subset. // create F_i using Recursive Feature Elimination (RFE)
b. Train the model using feature subset F_i: Model_i = Train_Model (X[:, F_i], y) // X[:, F_i] denotes the feature matrix with only the columns corresponding to the features in the subset F_i
c. Evaluate the performance of the model using stability_metrics:
Performance_i = Evaluate_Model (Model_i, X[:, F_i], y)
3. Identify the model with the best performance j_best = argmax (Performance_i)
4. Set F_best to the feature subset corresponding to the best performing model: F_best = F_j_best

4.1.1 Influence of Ensemble Feature selection on stability

The stability of the chosen features can be significantly influenced by ensemble feature selection techniques. When an algorithm is repeated on the same dataset or other dataset that are similar, stability refers to how consistently the results of feature selection are produced. In order to ensure that the chosen features are not significantly influenced by the specific random initialization or data perturbations, a stable feature selection approach should yield similar or identical feature subsets across different runs. The proposed method used three primary approaches to ensure the selected features robustness.

Approach 1: Dataset D has been used to create training dataset X and testing dataset T. Multiple training samples, X1, X2, Xm, are generated by bootstrapping from X.

Approach 2: To create k distinct training data samples, k-fold cross validation is employed. To evaluate the stability of the proposed ensemble feature selection technique, 5 –fold validation is used in this proposed work to produce 5 different sets of data samples with their ranking. One of the five folds is used as the testing dataset T for each cross-validation, and the other four folds are pooled to form the training dataset X. Five distinct training samples, X1, X2, X3, X4, and five equivalent testing samples, T1, T2, T3, and T5, are obtained after all iterations.

Approach 3: The stability of the feature selection method is assessed using the Jenson Shannon divergence approach when it is applied to several sub-data sets with ranking and various outputs, such as complete ranked, half ranked and top K (without ratings). The proof for this can be found in [14]. As a result, using three separate results, the Jenson Shannon technique is used to assess the stability of the suggested ensemble feature selection methods.

4.2 Hybrid stacking model

To improve the overall model performance, Hybrid stacking model is used to combine the individual classifier prediction at the base level and assign that as a new input to the meta level. The steps involved in hybrid stacking model are explained in Algorithm 3. In this proposed model K-Nearest Neighbor, Support Vector Machine, Naïve Bayes, Random Forest, Logistic Regression, Ada Boost and Decision Tree is implemented as the base level and one of these models are selected as meta level classifier. Prediction of all the base level classifier is combined and assigned as a new input to the meta level classifier.

Algorithm 3 – Hybrid Stacking Model
Input: Let D be the dataset.
Split D into training set D_train and validation set D_val
Step 1: Train base model:
Choose N diverse base models: M₁, M₂ \dots M_N
For i = 1 to N:
Train base model M_i on D_train
Pi ← predictions of M_i on D_val
Step 2: Build a Meta model:
Combine the base model predictions as input features for meta-model Meta
X_meta = [P₁, P₂, \dots , P_N] concatenated predictions from all base models
Step 3: Train the meta-model Meta on X_meta and the true target values Y_val from D_val
f_meta:X_meta⟶Y_val
Step 4: Final Prediction of the hybrid model:
For new, unseen data D_test:
For i = 1 to N:
P_test,i ← predictions of M_i on D_test
Concatenate P_test1,P_test2, \dots ,P_testN to form X_meta,test
Use Meta to make final predictions on X_meta,test: Y_pred=f_meta(X_meta,_test)

5 Dataset

PCOS dataset from the Kaggle Repository is used in this article, which comprises all the clinical and physical characteristics of PCOS patients. There are 541 instances and 43 attributes in the original dataset. SI. No and Patient File No. have been eliminated from the PCOS dataset since they have no impact on the output feature. Then, an attempt was made to convert the DataFrame to float data type and fill the remaining missing values using the mean of the relevant columns. The PCOS dataset is imbalanced; the target feature of the dataset is PCOS (Y/N) which consists of 541 rows and 43 columns. Out of 541, total rows of 177 have PCOS/YES, and 364 contain PCOS/No. To balance the dataset, SMOTE –SVM oversampling approach is used.

6 Experiments and Result Discussion

6.1 Stability assessment –feature selection method

In this article, the proposed method employed two different feature selection techniques, including Ensemble filter and Ensemble wrapper approach. A Jenson Shannon Divergence is used to evaluate the stability of both approaches on different outcomes such as complete ranked, half ranked, and top-k lists. The method with highest feature stability score is considered the model training. Table 1 represents the stability score of complete ranked list for individual filter method with different threshold values 16, 23, 30, and 35 and the corresponding ensemble features list of 9, 17, 26, and 34 using Jenson Shannon Divergence. It is clear from the results that the proposed ensemble filter method produces the highest score of 91.75% with the threshold value of 16 than individual methods.

Table 1
Jenson Shannon Stability Assessment –Individual vs proposed Ensemble Filter Method (complete ranked list)

Methods Threshold=16 Threshold=23 Threshold=30 Threshold=35

Chi-Square 89.42 86.79 84.39 82.74

Relief 90.75 89.62 87.17 86.32

Fisher score 90.63 89.19 87.54 86.73

mRmR 90.40 88.47 86.49 86.51

Proposed (Ensemble) 91.75 89.38 86.12 81.98

Methods	Threshold=16	Threshold=23	Threshold=30	Threshold=35
Chi-Square	89.42	86.79	84.39	82.74
Relief	90.75	89.62	87.17	86.32
Fisher score	90.63	89.19	87.54	86.73
mRmR	90.40	88.47	86.49	86.51
Proposed (Ensemble)	91.75	89.38	86.12	81.98

Table 2 depicts the stability score of individual wrapper methods with different threshold values 16, 23, 30 and 35 and the corresponding ensemble feature list of 9, 17, 24 and 34 using Jenson Shannon Divergence. From the table, it is clear that the proposed ensemble wrapper method produces the highest score 92.39% with the threshold of 16 than individual methods. While comparing the stability score of both approaches, ensemble wrapper method is more stable than ensemble filter method at all threshold level (Table 2). Therefore, feature list produced by the proposed SWISS (ensemble wrapper with stability assessment) is considered for model training and the experimentation is conducted at different threshold values. The results are shown in Fig. 2.

Table 2

Jenson Shannon Stability Assessment –Individual vs proposed Ensemble Wrapper Method (complete ranked list)

Methods	Threshold = 16	Threshold = 23	Threshold = 30	Threshold = 35
RFE –RF	90.92	88.53	86.13	84.05
RFE –LR	90.97	88.69	86.35	84.38
RFE –Extra tree	90.26	88.63	86.70	85.11
RFE –SVM	91.10	89.53	86.85	83.58
Proposed (Ensemble)	92.39	89.64	87.04	82.17

Fig. 2

Stacking model performance analysis with proposed SWISS method at different threshold values.

Among all, the features suggested by proposed SWISS with threshold 23 produces the highest accuracy of 97.81% and hybrid model is trained using those stable features.

Figure 3 shows the stability evaluation of the top-k (without ranking) list and the half-ranked list. It is noted that the stability score of top-k (without ranking) outcomes of feature selection methods are less than half ranked list. It shows the importance of ranking in the process of feature selection. Additionally, the stability of the feature selection methods decreases with an increase in the number of features. Therefore, selecting robust features that improve model performance is more crucial.

Fig. 3

Stability Assessment of Half ranked list vs Top-k list at different threshold values.

6.2 Individual classifier –result discussion

Performance analysis of individual classifiers with and without feature selection is presented in Tables 3 and 4. The individual classifiers used in this article includes K-Nearest Neighbor, (SVM) Support vector machine, (NB) Naïve Bayes, RF-Random Forest, LR (Logistic Regression), AdaBoost and DT (Decision Tree).

Table 3
Performance analysis of individual classifier without FS & oversampling

Classifier Accuracy Precision Recall F1 score AUC score

KNN 81.45 78.71 59.66 67.24 84.39

SVM 70.87 95.00 11.36 20.19 90.23

NB-Naïve Bayes 88.49 79.08 88.09 83.30 95.27

RF-Random Forest 90.15 87.50 78.96 83.90 95.48

LR-Logistic Regression 91.27 87.93 85.28 86.38 95.39

AdaBoost 88.49 84.60 79.00 81.59 94.16

Decision Tree 86.08 77.54 73.30 76.83 82.95

Classifier	Accuracy	Precision	Recall	F1 score	AUC score
KNN	81.45	78.71	59.66	67.24	84.39
SVM	70.87	95.00	11.36	20.19	90.23
NB-Naïve Bayes	88.49	79.08	88.09	83.30	95.27
RF-Random Forest	90.15	87.50	78.96	83.90	95.48
LR-Logistic Regression	91.27	87.93	85.28	86.38	95.39
AdaBoost	88.49	84.60	79.00	81.59	94.16
Decision Tree	86.08	77.54	73.30	76.83	82.95

Table 4

Performance analysis of individual classifier with FS & oversampling

Classifier	Accuracy	Precision	Recall	F1 score	AUC score
KNN	92.05	91.49	100.0	95.55	87.69
SVM	89.86	89.85	99.35	94.36	91.82
NB-Naïve Bayes	89.86	94.49	93.58	94.04	92.56
RF-Random Forest	93.69	93.65	99.35	96.42	95.72
LR-Logistic Regression	90.41	92.35	96.79	94.52	89.95
AdaBoost	92.05	93.53	97.43	95.44	93.97
Decision Tree	93.69	95.29	97.43	96.35	84.56

Table 3 shows that without feature selection and oversampling, Logistic Regression performed well than other models with 91.27% accuracy, 87.93% precision, 85.28% Recall, 86.38% f1 score and AUC score of 95.39%. Table 4 shows that with feature selection and oversampling, Random Forest outperformed with 93.69% accuracy, 93.65% Precision, 99.35% Recall, 96.42% f1 score and AUC score of 95.72%. It shows that, the performance of Random Forest classifier has improved with feature selection. Figure 4. Shows the ROC curve of individual classifier with feature selection and it was shown that the true positive rate of Random Forest is higher than other models.

Fig. 4

ROC Plot of individual classifier with feature selection.

6.3 Hybrid stacking classifier –result discussion

Table 5 represents the performance analysis of stacking hybrid model without feature selection and oversampling. The result shows that stacking AdaBoost produces 88.89% as highest accuracy, 78.57% as precision and recall, 93.19% as AUC score.

Table 5
Performance analysis of stacking classifier –without Feature selection and oversampling

Model Accuracy Precision Recall F1 score AUC score

Stacking-KNN 84.26 70.37 67.86 69.09 88.17

Stacking-SVM 87.04 79.17 67.86 73.08 88.97

Stacking -NB 84.26 67.74 75.00 71.19 92.72

Stacking-RF 87.04 76.92 71.43 74.07 93.95

Stacking-LR 87.96 77.78 75.00 76.36 94.42

Stacking-AdaBoost 88.89 78.57 78.57 78.57 93.19

Stacking-DT 83.33 66.67 71.43 68.97 79.46

Model	Accuracy	Precision	Recall	F1 score	AUC score
Stacking-KNN	84.26	70.37	67.86	69.09	88.17
Stacking-SVM	87.04	79.17	67.86	73.08	88.97
Stacking -NB	84.26	67.74	75.00	71.19	92.72
Stacking-RF	87.04	76.92	71.43	74.07	93.95
Stacking-LR	87.96	77.78	75.00	76.36	94.42
Stacking-AdaBoost	88.89	78.57	78.57	78.57	93.19
Stacking-DT	83.33	66.67	71.43	68.97	79.46

Table 6 represents the performance analysis of stacking classifier with stable features produced by the proposed SWISS model and oversampling. The result shows that SWISS with Stacking AdaBoost classifier produces highest accuracy of 97.81%, 97.80% of precision and 99.68% of recall, f1 score of 98.73% and AUC score as 99.08%. It is noticed from the results that the proposed stacking model with stable features produced by SWISS and SMOTE-SVM outperformed than individual models. Figure 5. Shows the ROC curve of the proposed model and it was shown that, true positive rate of SWISS-AdaBoost is higher than other meta models. The result comparison of proposed work and existing work is shown in Table 7. Ritika et al., [1] used Backward Feature Elimination (BFE) as feature selection technique. The authors used SMOTE for oversampling and implemented six stacking learners –LR, SVM, DT, RF, NB and AdaBoost with 30 features and obtained 90.24% of accuracy. Sayma et.al., [4] implemented five stacking classifiers such as GradBoost, XGBoost, AdaBoost, CATBoost, Random Forest with 25 features produced by principle component analysis (PCA) and attained 95.70% of accuracy. The proposed model used 17 features produced by SWISS method. SMOTE –SVM is used to oversampling the given dataset and implemented seven base and meta learners including KNN, LR, SVM, DT, RF, NB, AdaBoost and obtained 97.81% of accuracy and 99.08% of AUC, which is higher than the existing work.

Fig. 5

ROC plot for proposed model with stable features.

Table 6

Performance analysis of stacking classifier –with Feature selection and oversampling

Model	Accuracy	Precision	Recall	F1 score	AUC score
SWISS-KNN	96.44	96.87	99.04	97.94	95.58
SWISS-SVM	96.44	96.57	99.36	97.95	94.74
SWISS-NB	93.97	97.70	95.19	96.43	96.67
SWISS-RF	96.99	97.18	99.36	98.26	97.62
SWISS-LR	96.71	96.58	99.68	98.11	98.06
SWISS-AdaBoost	97.81	97.80	99.68	98.73	99.08
SWISS-DT	93.70	96.46	96.15	96.31	87.70

Table 7

Result comparison of proposed vs existing work

Source	Model	Feature selection method	No.of features used	Accuracy %
Ritika et al. [1] 2023	Six stacking learner: LR, SVM, DT, RF, NB, AdaB	BFE	30	90.24
Sayma et al. [4] 2023	Five stacking learner: Grad Boost, XGBoost, AdaBoost, CATBoost, Random Forest	PCA	25	95.70
Proposed model (SWISS)	Seven meta learners: KNN, LR, SVM, DT, RF, NB, AdaBoost	Ensemble wrapper method	17	97.81

7 Conclusion and Future Work

In order to properly treat and manage PCOS in women, this article describes data-driven early diagnosis of the disorder. This investigation has made it possible to identify a few key characteristics that would serve as the main indicators for diagnosing PCOS patients, help medical professionals quickly and accurately identify diseases, and be affordable for patients who underwent a number of tests to diagnose their illnesses. The PCOS dataset being used is imbalanced. SMOTE-SVM is therefore applied to the training set to balance the data. A Jenson Shannon Divergence was used in this proposed study to assess the features obtained by the individual and ensemble feature selection approaches. As the result, highly stable features produced by the proposed ensemble wrapper method with the stability score of 92.39% is considered for model training. The proposed hybrid stacking model with seven base learners: K-Nearest Neighbor, Support Vector Machine, Naïve Bayes, Random Forest, Logistic Regression, AdaBooost and Decision Tree are trained and implemented the meta learner. The result of the proposed model is compared with individual classifiers with and without feature selection. From Table 6, it is observed that reduced and more stable feature list with balanced dataset performed well with SWISS –AdaBoost and produced the accuracy of 97.81% and AUC of 99.08%. The stability measure presented in this research is applicable only to the feature selection algorithm that selects features based on ranking. Therefore, in future studies, we plan to examine stability measure with optimization-based feature selection algorithms like PSO (Particle Swarm Optimization) and Genetic Algorithm (GA) based feature selection to extract information on individual features and examine the results with different classifiers.

Financial support

This project was funded by RPS (Research Promotion Scheme) of AICTE –(All India Council for Technical Education) –File. No: 8- 98/FDC/RPS/POLICY-1/2021-22 (AQIS ID: 1-9331521061).

Declaration of competing interest

The authors have no conflicts of interest to declare.

References

Ritika Kumari Jaspreeti Singh Anjana Gosain , SmS:SMOTE-stacked hybrid model for diagnosis of polycystic ovary syndrome using feature selection method, Expert Systems with Applications 2023, 0957–4174.

Hela Elmannai Nora El-Rashidy Ibrahim Mashal Manal Abdullah Alohali Shaker El-Sappagh Hager Saleh , Polycystic ovary syndrome detection machine learning model based on optimized feature selection and explainable artificial intelligence, Diagnostics, MDPI 2023

Deepak Kumar Rakesh Raj Anwit Prasanta Jana

, A new ranking-based stability measure for feature selection algorithms, Soft Computing 27 (2023), 5377–5396, Springer Nature

Sayma Alam Suha Muhammad Nazrul Islam Exploring the dominant features and data-driven detection of polycystic ovary syndrome through modified stacking ensemble machine learning technique, Heliyon Diagnostics (2023), e14518.

Laith Alzubaidi Jinshuai Bai Aiman Al-Sabaawi Jose Santamaría, Albahri

A.S.

Bashar Sami Nayyef Al-dabbagh Mohammed Fadhel

Mohamed Manoufali Jinglan Zhang Ali Al-Timemy

Ye Duan Amjed Abdullah Laith Farhan Yi Lu Ashish Gupta Felix Albu Amin Abbosh Yuantong Gu , A survey on deep learning tools dealing with data scarcity: Defnitions, challenges, solutions, tips, and applications, Journal of Big Data, 2023

Shivani Aggarwal Kavita Pandey , Early identification of PCOS with commonly known diseases: Obesity, diabetes, high nd heart disease using machine learning techniques, Expert Systems with Applications (2023), 0957–4174.

Reem Salman Ayman Alzaatreh Hana Sulieman , The stability of different aggregation techniques in ensemble feature selection, Journal of Big Data, Springer 2022.

Utkarsh Mahadeo Khaire Dhanalakshmi

, Stability of feature selection algorithm: A review, Journal of King Saud University –Computer and Information Sciences (2022), 1060–1073.

Suriya

, Reka

, Elakkiya

, Early Diagnosis of PolCystic Ovary Syndrome (PCOS) in young women: A Machine Learning Approach, IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), DOI 10.1109, 2022.

10.

Homay Danaei Mehr Huseyin Polat , Diagnosis of polycystic ovary syndrome through different machine learning and feature selection techniques, 137–150, 2022, Health and Technology, 2022

11.

Shazia Nasim Mubarak Almutairi Kashif Munir Ali Raza Faizan Younas , A novel approach for polycystic ovary syndrome prediction using machine learning in bio informatics, IEEE Access, 2022.

12.

Maheswari

, Baranidharan

, Karthik

, Sumathi

Modelling of F3I based feature selection approach for PCOS classification and prediction, Journal of Ambient Intelligence and Humanized Computing 12 (2021)1349–1362.

13.

Muhammad Sakib Khan Inan Rubaiath Ulfath

Fahim Irfan Alam Fateha Khanam Bappee Rizwan Hasan , Improved Sampling and Feature Selection to Support Extreme Gradient Boosting For PCOS Diagnosis, IEEE Xplore, 2021.

14.

Rocío Alaiz-Rodríguez Andrew Parnell

, An information theoretic approach to quantify the stability of feature selection and ranking algorithms, Knowledge-Based Systems, 2020.

15.

Barbara Pes , Ensemble feature selection for highdimensional data: a stability analysis across multiple domains, Neural Computing and Applications, Springer 2020.

16.

Satish Nandipati Chew XinYing Khaw Khai Wah , Polycystic Ovarian Syndrome (PCOS) Classification and Feature Selection by Machine Learning Techniques, Applied Mathematics and Computational Intelligence, 2020

17.

Amsy Denny Anita Raj Ashi Ashok Maneesh Ram

Remya George , i-HOPE: Detection and Prediction System for Poly cystic Ovary Syndrome (PCOS) using Machine Learning Techniques, 978-1-7281-1895-6, IEEE Region 10 Conference, 2019.

18.

Jesús González Julio Ortegaa Miguel Damas Pedro Martín-Smitha John Gan

, A new multi-objective wrapper method for feature selection –Accuracy and stability analysis for BCI, Neurocomputing, 2019, 0925–2312.

19.

Sankalap Aroraa Priyanka Anand Nazrul Hoque , Binary butterfly optimization approaches for feature selection, Expert Systems With Applications (2019), 147–160.

20.

Nazrul Hoque Mihir Singh Dhruba Bhattacharyya , EFS-MI: an ensemble feature selection method for classification, Complex Intell. Syst, Springer 2018.