Accelerating the performance of machine learning classifiers using bacterial colony optimization for heart disease prediction

Chemotaxis and communication

In BCO, chemotaxis is combined with communication throughout the optimization process. The chemotactic behavior of bacteria over their lifetime can generally be classified into two modes: tumbling and swimming. In the tumbling phase, the bacterium introduces a stochastic disturbance to its orientation, enabling random turbulence to impact the optimal search direction. As a result, the position of each bacterium is updated according to Equation (1), where both the turbulent director and the optimal search director jointly determine the movement. ⁵⁵ In contrast, the bacterium moves without turbulence and smoothly in the direction of optimal exploitation during the swimming phase, updating its position according to Equation (2). The BCO employs an adaptive chemotaxis step, developed to guide bacterial movement during the search process, as defined by Equation (3).

Position i T = Position i T − 1 + C ( i ) f i * G best − Position i T − 1 + ( 1 − f i ) P best , i − Position i T − 1 + turbulent j

(1)

Position i T = Position i T − 1 + C ( i ) f i G best − Position i T − 1 + ( 1 − f i ) P best , i − Position i T − 1

(2)

C ( i ) = C min + iter max − iter j iter max n ⋅ ( C max − C min )

(3)

This formulation balances stochastic exploration and directed convergence, with tumbling promoting randomness to escape local optima and swimming guiding bacteria toward promising regions (P_best and G_best).

Elimination and reproduction

Each bacterium is assigned an energy level reflecting its search capability and fitness in heart disease detection. The decision rules for elimination and reproduction, which determine the removal of less fit bacteria and the reproduction of fitter ones, are defined by Equation (4), ensuring the colony progressively converges toward optimal solutions. ⁵⁵

i ∈ Candidate repr , if L i > L given and i ∈ healthy Candidate eli , if L i < L given and i ∈ healthy Candidate eli , if i ∈ unhealthy

(4)

Migration

In BCO, bacterial migration moves individuals to new random positions when specific conditions—such as energy level, similarity among bacteria, or chemotaxis efficiency—are met, as described by Equation (5). ⁵⁵ This process helps to avoid local optima, strengthens global search ability, and ensures efficient exploration.

Position i ( T ) = rand × ( u b − l b ) + l b

(5)

Hypothesized advantages of BCO in hyperparameter tuning

In contrast to traditional optimization algorithms that employ independent or velocity driven parameter updates, BCO is hypothesized to navigate the intricate, nonconvex search spaces associated with the heterogeneous classifiers studied in this literature. The communication mechanism in BCO plays a critical role in mitigating premature convergence. Tuning parameters for the classifiers such as SVM, RF, the search space includes numerous local optima that misleads to the region of high accuracy not representing the global optimum. Bacterial agents exchange information on nutrient concentration, which is respective to model accuracy for the classifier, and orientation.^39,40 This collaborating communication phase enables the population to discriminate local and global optimal areas and prevents the stagnation in suboptimal hyperparameter spaces, which is commonly observed in Particle Swarm Optimization. ³⁸ Additionally, while tumbling and swimming, communication helps the chemotaxis process by directing the swarm away from low performing areas of the search arena. In high dimensional classifiers, where random exploration can result in needless computing expenditure, this tailored direction enhances search efficiency. ⁵⁷

Support Vector Machine

In this research, the SVM was applied as a classifier with the aim of detecting the presence of cardiac disease. SVM derives an optimal hyperplane to separate two classes (patients with and without heart disease). Based on the training dataset, D = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , … , ( x n , y n ) } , x i ∈ R m , y i ∈ { − 1 , + 1 }

The classifier learns a separating function:

f ( x ) = sign ( w T x + b )

(6)

min w , b , ξ 1 / 2 ‖ w ‖ 2 + C ∑ i = 1 n ξ i , s.t. y i ( w T x i + b ) ≥ 1 − ξ i , ξ i ≥ 0

Since nonlinear correlations are frequently observed in patient health data, ⁵⁸ kernel functions are employed. The resulting decision function is:

f ( x ) = sign ∑ i = 1 n α i y i K ( x i , x ) + b

(7)

SVM performance, determined by C and γ, was optimized using BCO, where candidate pairs are estimated by accuracy and the colony iteratively prefers high performing candidates.

Logistic regression

LR is a widely used and reliable supervised method for addressing binary classification problems in medical research, including the prediction of heart disease. Linear regression predicts continuous values, whereas LR predicts the probability of a category. This characteristic makes LR especially valuable in medical diagnostics, where the primary objective is to determine whether a disease is present or absent. The logistic regression model predicts the probability of an outcome (e.g., the existence of heart disease) based on a linear combination of input features (age, blood pressure, cholesterol, electrocardiogram (ECG) results, blood sugar levels, etc.) weighted by coefficients ⁵⁹ :

z = β 0 + β 1 x 1 + β 2 x 2 + ⋯ + β n x n

(8)

This linear predictor is passed through the sigmoid function to produce a probability ranging from 0 to 1, and then a decision threshold of 0.5 is applied to classify the class label.

P ( y = 1 | x ) = 1 1 + e − z = 1 1 + e − ( β 0 + β 1 x 1 + ⋯ + β n x n )

(9)

K-Nearest Neighbors

The KNN algorithm is employed for heart disease detection by classifying patients in two groups: positive and negative cases, depending on their medical data.^60,61 In this context, clinical attributes like age, blood pressure, cholesterol levels, resting ECG, and various types of chest pain are represented as a feature vector:

X = ( x 1 , x 2 , x 3 , … , x n )

The affinity between two records, X and Y, is calculated based on the Euclidean distance:

d ( X , Y ) = ∑ i = 1 n ( x i − y i ) 2

(10)

In heart disease datasets, the feature ranges differ: for example, cholesterol spans 100 to 190, while age ranges from 40 to 80, which potentially biases distance calculations. To ensure that all features contribute equally to the distance metric, continuous attributes are standardized. ⁶²

The class of an unknown patient is derived from the labels of its closest neighbors. The neighborhood size k and distance metric have an impact on the k-NN’s performance. To optimize these parameters, BCO is utilized, which iteratively estimates candidate (k, d) solutions based on classification accuracy to determine the optimal configuration (k*, d*) for a reliable prediction of heart disease.

Multilayer Perceptron

The MLP, a variant of artificial neural networks, studies heart disease factors such as age, blood pressure, cholesterol level, blood sugar, and electrocardiographic observations through multiple layers of interconnected neurons to capture nonlinear dependencies and provide accurate estimations of heart disease risk. It is structured as a feedforward neural network composed of an input layer, multiple hidden layers, and an output layer. ⁶³

For a neuron j in layer l, the output is computed as:

o j ( l ) = f ∑ i w i j ( l ) o i ( l − 1 ) + b j ( l )

(11)

The output layer generates the predicted class label for a patient.

The MLP’s performance relies on hyperparameters such as learning rate, hidden neuron numbers, and activation functions. The optimal configuration (θ*) for a reliable heart disease prediction is determined by employing BCO, which analyzes candidate configurations (θ) based on classification accuracy.

Naïve Bayes

NB is a supervised learning algorithm based on Bayes’ theorem, because of its simplicity, efficiency in computation, and consistent performance even with smaller datasets. ⁶⁴ The model considers conditional independence among patient attributes, allowing the multivariate joint distribution to be represented as the product of individual probabilities.

For a patient with a feature vector, X = (x₁, x₂, …, x _n ), the posterior probability of class C _k (e.g., heart disease existence) is defined as:

P ( C k | X ) = P ( C k ) ∏ i = 1 n P ( x i | C k ) P ( X ) ,

(12)

As P (X) is constant across all classes, the classification simplifies to:

C ^ = arg max C k P ( C k ) ∏ i = 1 n P ( x i | C k )

(13)

To improve performance, BCO is utilized to choose the best parameters, such as feature subsets, smoothing factors, and variance estimations. Individual candidate configuration is evaluated as determined by classification accuracy, enhancing generalization while maintaining computational efficiency.

Random Forest

RF is a resilient method for ensemble learning that generates multiple decision trees, where each tree is trained on a bootstrapped sample of patient records and a randomly selected subset of clinical features. Random selection of data and features generates a variety of decision trees, mitigating overfitting and enhancing generalization. ⁶⁵ Each tree estimates heart disease, and the final classification is aggregated using majority voting, boosting diagnostic accuracy and stability over a single decision tree. Figure 2 illustrates how the RF classifier operates, where the predictions of multiple decision trees are combined through majority voting to produce the final class. In the context of heart disease detection, the dataset is expressed as:

X = { x 1 , x 2 , … , x n } , Y = { y 1 , y 2 , … , y n } , y i ∈ { 0,1 } ,

OPEN IN VIEWER

Figure 2.

Random Forest process to form a reliable ensemble from multiple trees using majority voting.

Each decision tree is built by determining the best splits depending on the Gini Index:

Gini ( D ) = 1 − ∑ k = 1 K p k 2

(14)

Majority voting is used to infer the final prediction for an unseen record x:

y ^ = arg max c ∈ { 0,1 } ∑ j = 1 M I ( h j ( x ) = c )

(15)

To improve prediction performance, BCO is utilized to optimize RF hyperparameters like the maximum depth and number of trees based on classification performance such as accuracy or F1-score.

Decision tree

A DT is a supervised classification method that uses recursive splitting of patient data, using clinical attributes to perform classification. At each decision node, the algorithm identifies the feature that maximizes class separation based on an impurity measure, ⁶⁶ typically the Gini Index, as defined in Equation (14). The decision tree grows recursively until predefined stopping criteria, such as maximum tree depth or minimum samples per node, are satisfied, at which point the leaf nodes denote the predicted class (presence or absence of heart disease). At the leaf node, the final prediction for an unseen record x is determined on the basis of the majority class.

y ^ = arg max c ∈ { 0,1 } p ( c | leaf )

(16)

Similar to RF, BCO is employed to optimize key DT hyperparameters, including maximum depth and minimum samples per split, enriching DT performance based on classification metrics.

Extreme Gradient Boosting

XGBoost is a robust ensemble learning method that sequentially develops a number of weak classifiers, such as decision trees, with each successive tree trained to minimize the residual errors produced by the preceding trees. By repeatedly focusing on misclassified samples, XGBoost minimizes bias and increases predictive accuracy. ⁶⁷ It also employs regularization to prevent overfitting and a learning rate to control each tree’s contribution, ⁶⁸ which makes it effective for the detection of heart disease using clinical attributes.

Figure 3 demonstrates that the working procedure of XGBoost involves sequentially constructing decision trees, optimizing residual errors, and applying regularization to enhance predictive performance. XGBoost constructs decision trees that follow a structure parallel to the DT model as described earlier in the same section. Consequently, the node-splitting in XGBoost also relies on impurity measures such as the Gini Index as previously defined in Equation (14).OPEN IN VIEWER

The final prediction is obtained by combining the outputs of all trees in the ensemble, weighted by their learning rates:

y ^ = σ ∑ t = 1 T η h t ( x )

(17)

Similar to its application in DT and RF, BCO is applied to XGBoost to optimize hyperparameters, such as the learning rate, the maximum depth of the tree, and the number of trees, improving the performance of the model based on classification metrics.

Light Gradient Boosting Machine

LGBM is a tree-based gradient-boosted ensemble method similar to XGBoost, but it differs in its tree construction. It employs a leaf-wise tree growth strategy with depth constraints, ⁶⁹ selecting the leaf that maximizes loss reduction at each step, rather than the level-wise growth used in XGBoost. As a result, LGBM captures complex feature interactions more effectively, trains faster with lower memory consumption, and achieves enhanced accuracy on large-scale, high-dimensional datasets. Similar to XGBoost models, LGBM utilizes clinical features, specifying splits based on impurity measures.

To further improve diagnostic performance, BCO is utilized to tune key hyperparameters such as learning rate, maximum depth, and number of trees.

Adaptive Boosting

AdaBoost is an ensemble learning technique that combines multiple weak classifiers, commonly shallow decision trees, to construct a strong classifier. In contrast to RF, which constructs trees separately and combines their votes, AdaBoost trains decision trees in sequential order, where each successive classifier assigns higher weight to misclassified patient samples, ⁷⁰ forcing the model to address difficult samples. Figure 4 illustrates the AdaBoost procedure for generating a strong classifier from multiple weak learners.OPEN IN VIEWER

For the training data (x _i , y _i ), where y _i ∈ {−1, + 1}, AdaBoost assigns initial equal weights: w _i = 1/N. At each iteration t, a weak classifier h _t (x) is trained, and its weighted error is computed as:

ϵ t = ∑ i = 1 N w i 1 h t ( x i ) ≠ y i

(18)

The weight of the classifier is then computed by:

α t = 1 2 ln 1 − ϵ t ϵ t

(19)

Finally, the strong classifier is derived as:

H ( x ) = sign ∑ t = 1 T α t h t ( x )

(20)

Classifiers with lower error are assigned higher weights, enabling the model to focus on samples that are hard to classify and iteratively improve accuracy.

Like DT, RF, XGBoost, and LGBM, BCO is also used with AdaBoost to adjust key hyperparameters, such as the number of weak classifiers and the learning rate, thus enhancing the robustness and classification performance in the detection of heart disease.