Hybrid whale optimized crow search algorithm and multi-SVM classifier for effective system level test case selection

Abstract

A novel approach to enhance software testing through intelligent test case selection is proposed in this work. The proposed method combines feature extraction, clustering, and a hybrid optimization algorithm to improve testing effectiveness while reducing resource overhead. It employs a context encoder to extract relevant features from software code, enhancing the accuracy of subsequent testing. Through the use of Fuzzy C-means (FCM) clustering, the test cases are classified into groups, streamlining the testing process by identifying similar cases. To optimize feature selection, a Hybrid Whale Optimized Crow Search Algorithm (HWOCSA), which intelligently combines the strengths of both Whale Optimization Algorithm (WOA) and Crow Search Algorithm (CSA) is introduced. This hybrid approach mitigates limitations while maximizing the selection of pertinent features for testing. The ultimate contribution of this work lies in the proposal of a multi-SVM classifier, which refines the test case selection process. Each classifier learns specific problem domains, generating predictions that guide the selection of test cases with unprecedented precision. Experimental results demonstrate that the proposed method achieves remarkable improvements in testing outcomes, including enhanced performance metrics, reduced computation time, and minimized training data requirements. By significantly streamlining the testing process and accurately selecting relevant test cases, this work paves the way for higher quality software updates at a reduced cost.

Keywords

Context encoder pre-processing FCM WOA HWOCSA

1 Introduction

Regression testing plays a vital role in software development by ensuring the continued functionality and correctness of software even after code changes have been made. As software systems evolve, effective maintenance strategies are essential to prevent them from becoming obsolete and ineffective. One key aspect of this maintenance is regression testing, which verifies that software updates do not introduce defects that could undermine its performance. In large organizations, regression testing is typically executed by experts who execute or analyse the code [1, 2]. However, this process can be resource-intensive, repetitive, time-consuming, and costly due to the need for re-execution of the entire test suite [3]. To mitigate these challenges, regression test optimization techniques such as test selection and prioritization have been introduced. These techniques involve selecting a subset of test cases that efficiently identify bugs and defects [4]. Test cases can be reordered based on predefined criteria such as code coverage, branches, or fault detection [5, 6]. Alternatively, test case selection involves reducing the number of test cases based on specific criteria [7, 8]. In this work, the focus is on test prioritization to facilitate seamless test case selection. Recently, the application of Machine Learning (ML) techniques has gained traction in driving the test case selection process.

In the realm of ML, classification is typically accomplished using two main techniques: discriminative learning and generative learning. The former displays exceptional performance in terms of supervised learning in many instances. The SVM [9 –11], which is based on discriminative learning, has a simple and linear model that maximizes the linear separator margin directly between two sets. Moreover, among several ML approaches, the SVM offers a comparatively exceptional recognition rate. However, on the downside, it is only applicable for solving binary decisions. Thereby, a Multi-SVM technique is employed in this work to support multi-class problems. The ML base test case selection is further enhanced by using metaheuristic algorithms [20 –26] for feature selection in test case selection. Some of the prominently used algorithms for feature selection are Differential Evolution (DE) [12], Particle Swarm Optimization (PSO) [13], Genetic Algorithm (GA) [14], Crow Search Optimization (CSO) [15] and Whale Optimization Algorithm (WOA) [16]. When a metaheuristic algorithm is developed, an efficient balance between an exploitation (intensification) and exploration (diversification) phases is required to be accomplished. Hence, hybrid algorithms that combine two prominent metaheuristic algorithms are designed in order to improve algorithm performance. Here, a hybrid algorithm combining WOA and CSA is proposed for enhancing test case selection process. Although the WOA heavily favours exploitation, it fails to adequately implement exploration, resulting in failure to reach global optimal solution in certain cases. The advantage of CSA, on the other hand, resides in its capacity to easily avoid local optima while tackling multimodal, high-dimensional and complex problems. However, its local search capability is not particularly effective [17]. Hence both these algorithms are best suited for hybridization due to their complementary strengths that compensates for the limitations of each other.

Moreover, the test case selection using multi-SVM is improved further with selection of appropriate feature generation and clustering techniques. Here, the context encoder [18, 19], which is a robust and simple technique, is used for feature generation. Additionally, the Fuzzy c-means (FCM) technique is adopted in this work as the data clustering technique. An analysis of recent advanced techniques in testing is provided in Table 1. The literature gap highlighted by the comparative study is the lack of a holistic approach that seamlessly integrates feature extraction, clustering techniques, and a hybrid optimization algorithm for the purpose of test case selection in the domain of software testing. While the existing techniques offer valuable contributions individually, there seems to be a gap in the research landscape where no prior work effectively combines these components to create a unified methodology that can potentially lead to superior testing results. This gap presents an opportunity for the proposed approach to address this limitation and demonstrate its effectiveness in bridging these aspects for enhanced software testing outcomes.

Table 1
Analysis of recent techniques introduced in testing

Ref Techniques Objectives Advantages Drawbacks

[27] African Buffalo Optimization Test Case Prioritization (TCP) and Selection Significant reduction in test suit size and runtime along with improved resource efficiency. Computational overhead and parameter sensitivity.

[28] Seeding Strategies Test Case Selection Optimized testing process, enhanced solution quality, cost-effectiveness and convergence improvement. Limited generalizability as it does not exhibit the same level of improvement in all contexts.

[29] Single and Multi-objective Prioritizer Test Case Prioritization Safety enhancement, early fault detection, resource efficiency and negligible overhead. Feature limitations and over dependence on simulation realism.

[30] Test Oracle Derivation Approach Test Amplification Enhanced regression testing, automated test case generation, test suite enhancement and effective behaviour change detection. Accuracy is compromised in case of non-representative data and the identification of genuine behaviour changes is challenging.

[31] Log TCP Test Case Prioritization Log utilization, improved fault detection and practical applicability. Ensuring efficient log collection and preprocessing are challenging in large-scale projects.

Ref	Techniques	Objectives	Advantages	Drawbacks
[27]	African Buffalo Optimization	Test Case Prioritization (TCP) and Selection	Significant reduction in test suit size and runtime along with improved resource efficiency.	Computational overhead and parameter sensitivity.
[28]	Seeding Strategies	Test Case Selection	Optimized testing process, enhanced solution quality, cost-effectiveness and convergence improvement.	Limited generalizability as it does not exhibit the same level of improvement in all contexts.
[29]	Single and Multi-objective Prioritizer	Test Case Prioritization	Safety enhancement, early fault detection, resource efficiency and negligible overhead.	Feature limitations and over dependence on simulation realism.
[30]	Test Oracle Derivation Approach	Test Amplification	Enhanced regression testing, automated test case generation, test suite enhancement and effective behaviour change detection.	Accuracy is compromised in case of non-representative data and the identification of genuine behaviour changes is challenging.
[31]	Log TCP	Test Case Prioritization	Log utilization, improved fault detection and practical applicability.	Ensuring efficient log collection and preprocessing are challenging in large-scale projects.

The proposed approach integrates feature extraction, clustering, and a hybrid optimization algorithm into a unified methodology for test case selection, addressing a gap in the existing literature.

The utilization of a context encoder enhances feature extraction accuracy, facilitating the identification of relevant software code features for testing.

The application of FCM clustering categorizes test cases into groups, streamlining the testing process by identifying similar cases and thereby improving testing efficiency.

The introduction of HWOCSA optimizes feature selection through an intelligent combination of the strengths of WOA and CSA.

A novel multi-SVM classifier is proposed, enhancing test case selection accuracy by enabling specialized learning for distinct problem domains, resulting in improved testing outcomes.

The experimental results underscore the effectiveness of the proposed approach, showcasing enhanced performance metrics, reduced computation time, and minimized training data requirements.

The proposed approach contributes to higher quality software updates while reducing costs, making the testing process more streamlined and effective. These contributions collectively establish the innovative nature and significance of the proposed work in the domain of software testing.

2 Proposed system description

A test case selection approach based on ML is suggested in this work. The entire process involved in the proposed test case selection approach from data collection, pre-processing, feature generation, feature selection, clustering and classification is illustrated in Fig. 1. This work uses test data related to meta data and provides natural language test case descriptions as inputs for multi-SVM to predict test case selection.

Fig. 1

Structure of the proposed test case selection using multi-SVM.

This research involves several key steps. Initially, test cases are determined through data preparation using a dataset containing numerous features. Data pre-processing includes removing irrelevant features and converting categorical data to numerical data. Subsequently, the pre-processed data are put through natural language processing. Here, the data undergoes text pre-processing and text feature generation. The former involves processes such as stop words removal, irrelevant text removal, conversion to lower case and stemming, while the latter uses context encoder for text feature generation. Then the prominent features are selected using novel HWOCSA. After feature selection, the data is subjected to clustering using FCM. The clustered data are then finally classified using multi-SVM. Thus, the appropriate test cases are selected using the proposed ML approach.

3 Proposed system modelling

3.1 Pre-processing

In this work, test data related to meta data is considered, and the inputs provided to multi-SVM consist of natural language test case descriptions for predicting test case selection. Test data selected is authorization micro service test cases, which is based on Oauth2 standard and is prominently used for authorization in industries. The dataset includes numerous features such as the classification target, GIT commit message, bug description, bug ID, defects, automation status, error-prone test cases, test case description, test case title, test case type, release identification number and unique identifier of records. However, only specific features are taken in to account for the proposed work. The redundant test features are removed during the process of feature selection. The final data comprises only the test case title, release identification number, defects, automation status, error prone test cases and test case description. The pre-processed data is then subjected to Natural Language Processing (NLP). The NLP comprises of text pre-processing and text feature generation.

3.2 Text pre-processing

In case of text pre-processing, a corpus represented as sparse matrix of numerical features is obtained by conversion of the textual features. The pre-processing approach aims at generating a corpus, which is suitable for the multi SVM.

Removal of irrelevant words: In order to obtain a clean text for successive processing, the irrelevant characters and words are excluded from the data.

Transformation to lower case from uppercase: Since both the lower-case and upper-case alphabets have distinct ASCII codes, all the letters are transformed to a common lower-case letters.

Stop words removal: In terms of response variables prediction, the role of stop words are insignificant.

Stemming words: It is a text normalization technique, which is used to obtain the root word from the inflected word. Through stemming, the Bag of Words feature dimension are successfully reduced.

3.3 Text feature generation using context encoder

The statistical text features are generated using context encoder, which aims at predicting the target word on the basis of certain context words. Through selection of suitable rows from W₀, the estimation of context words sum of embeddings is achieved. The structure of context encoder is presented in Fig. 2. $W_{0}, W_{1} \in ℝ^{N \times d}$ (1)

$t \in ℝ^{K + 1}$ (2)

Fig. 2

Structure of context encoder.

Where, the terms W₀ and W₁ refers to the parameter matrices of a neural language model and the term t refers to the label vector.

In similar contexts, the same words may appear, resulting in the possibility of replacing synonymous words with each another. The pairwise similarities between words are calculated based on the co-occurrence of context words, resulting in the generation of a similarity matrix. $S \in ℝ^{N \times N}$ (3)

The similarity scores range between 0 and 1, and these similarities are intended to be saved in word embeddings. It is required that the cosine similarity between the embedding vectors of similar context words is closer to 1. The scalar product of the matrix with word embeddings is given as, $Y \in ℝ^{N \times d}$ (4)

Moreover, Y is also required to approximate S. The word embeddings are required to satisfy the condition, $Y Y^{T} \approx S$ (5)

This is accomplished by computing the Singular Value Decomposition of S in addition to the application of Eigen vectors. The global context vector is evaluated as the average of binary context vector x_{ω_i}, $x_{ω_{global}} = \frac{1}{M_{ω}} \sum_{i = 1}^{M_{ω}} x_{ω_{i}}$ (6)

It corresponds to M_ω occurrences of ω in corpus. The embedding of word ω is given as, $y_{ω} = {(a \cdot x_{ω_{global}} + (1 - a) x_{ω_{local}})}^{T} W_{0}$ (7)

Where, a∈[0,1] the context encoder is a robust technique, capable of creating on the spot out of vocabulary embeddings in addition to providing local context-based identification of multiple meaning words.

3.4 Hybrid whale optimized crow search algorithm (HWOCSA) for text feature selection

The proposed HWOCSA is a combination of both Whale Optimization Algorithm (WOA) and Crow Search Algorithm (CSA). It is effective in identification and selection of the most appropriate variable subset to the target variable.

3.4.1 Crow search algorithm

The CSA is a nature inspired population based approach modelled after the social interaction, characteristics and conduct of crows. In addition to being smart birds, crows tends to live in flocks and conceal their food in hidden places that they can remember and recover even after a long period of time. Furthermore, their self-aware nature enables them to recognize threats and communicate effectively with other crows using complex communication skills. Crows may also engage in food theft by carefully observing the hiding place of food of other crows before stealing it. However, if the concerned crow becomes aware of the thief, it switches its position away from its food for misleading the thief. The population of CSA is made up of N solutions and d problem dimension. The vector that denotes position of each crow in iteration t is given as, $q_{i}^{t} = [q_{i 1}^{t}, q_{i 2}^{t}, \dots, q_{id}^{t}] for i = 1, 2, 3, \dots, N$ (8)

Where, the term $q_{i}^{t}$ refers to the location solution. If crow i intends to steal from crow j, then the following two situations takes place,

If crow j discovers the location of food of crow i, then position of crow j is updated as,

$q_{i}^{(t + 1)} = q_{i}^{(t)} + r_{i} * {fl}_{j}^{(t)} * (m_{j}^{(t)} - q_{i}^{(t)})$ (9)

Where, the terms fl and r_i refers to the flight length and random number respectively.

Suppose, if crow i knows about the intention of crow j, it deceives the thief by altering its position.

Both these situations are mathematically represented as, $q_{i}^{(t + 1)} = {\begin{matrix} q_{i}^{(t)} + r_{i} * {fl}_{j}^{(t)} * (m_{j}^{t} - q_{i}^{(t)}), r_{j} ⩾ {AP}_{i}^{t} \\ choose a random position, Otherwise \end{matrix}$ (10)

The schematic representation of the movement of crows during both these conditions is given in Fig. 3. The flight length has a huge impact over searching capability of the crow. The lower and higher values of flight length contribute to local and global search respectively. During the execution of the algorithm, the fitness function of each crow is assessed. Then a crow updates its position in accordance to fitness value. The new location is also evaluated for its feasibility. On the basis of the following equation, the memories of crows get updated,

Fig. 3

Schematic representation of the movement of crows.

$m_{i}^{(t + 1)} = {\begin{matrix} m_{i}^{(t)} if f (q_{i}^{(t + 1)}), is better than f (q_{i}^{(t)}) \\ m_{i}^{(t)}, Otherwise \end{matrix}$ (11)

3.4.2 Whale optimization algorithm

The WOA is a metaheuristic algorithm, which is modelled after the food foraging and hunting behaviour of humpback whales. This algorithm is primarily based on the bubble net hunting behaviour of the humpback whales. This algorithm entails three phases, which are

Encircling prey: The whales, which are the search agents, encircle their prey after identifying their locations. Then, from a randomly produced search agent set, the best candidate search agent set is identified by WOA. With respect to the best candidate, other search agents also update their location. This entire process is expressed as,

$\vec{D} = | \vec{C} \cdot {\vec{X}}^{*} (t) - \vec{X} (t) |$ (12) $\vec{X} (t + 1) = {\vec{X}}^{*} (t) - \vec{A} \cdot \vec{D}$ (13)

Where, the coefficient vectors are specified as $\vec{A}$ and $\vec{C}$ , the best solution position vector is specified as $\vec{X}$ and the current iteration is specified as t. The coefficient vectors are estimated as, $\vec{A} = 2 \vec{a} \cdot \vec{r} - \vec{a}$ (14) $\vec{C} = 2 \cdot \vec{r}$ (15)

Where, the variable $\vec{a}$ is linearly reduced over iterations from 2 to 0, while $\vec{r}$ represents the random value.

Exploitation phase: This is the phase during which the bubble-net hunting takes place. The two techniques involved in this phase are spiral updating position and shrinking encircling technique. Through linear decreasing of variable $\vec{a}$ from 2 to 0, the shrinking encircling behaviour of the humpback whales is formulated. In case of spiral updating position, distance between the prey and whale is determined to aid with the behaviour formulation. Moreover, a spiral equation is used for replicating the spiral movement of whales towards its prey,

\vec{X} (t + 1) = \vec{D} \cdot e^{bl} cos (2 π l) + {\vec{X}}^{*} (t)

(16)

Table 2

Parameter specification of HWOCSA

Parameter	Description	Values/Range
NumIterations	Number of iterations	100 to 1000
NumCrows	Number of crows in population	10 to 50
NumWhales	Number of whales in population	5 to 20
Threshold	Threshold value	0.4 to 0.6
MaxIterations	Maximum number of iterations for WOA	50 to 200
Initial a	Initial value of coefficient ‘a’ for WOA	2
LowerBound	Lower bound for crow and whale positions	–1
UpperBound	Upper bound for crow and whale positions	1
ExplorationRate	Rate of exploration for CSA behaviour	0.4 to 0.6
b	Constant ‘b’ for spiral movement in WOA	0.5 to 2.0

Where, distance between prey and whale is specified as $\vec{D}$ and the spiral movement is represented using the constant b. The switching between two behaviours is expressed as, $\vec{X} (t + 1) = {\begin{matrix} {\vec{X}}^{*} (t) - \vec{A} \cdot \vec{D} & if p < 0.5 \\ \vec{D} \cdot e^{bl} cos (2 π l) + {\vec{X}}^{*} (t) & if p ⩾ 0.5 \end{matrix}$ (17)

Where, the random number is specified as p.

Exploration phase: The global search process is conducted in this phase and this phase is based on the vector value $\vec{A}$ (equal to or greater than 1). Here, a position of whales is updated in accordance to a randomly chosen whale and their new position is estimated as,

\vec{D} = | \vec{C} \cdot \vec{X_{rand}} - \vec{X} |

(18)

\vec{X} (t + 1) = \vec{X_{rand}} - \vec{A} \cdot \vec{D}

(19)

Where, the location of randomly chosen whale is specified as X_rand.

3.4.3 Hybrid whale optimized crow search algorithm

In this work, a novel HWOCSA approach is used for text feature selection, in which WOA and CSA operate in a simultaneous manner in case of the proposed hybrid algorithm. It incorporates the benefits of both algorithms and generates promising results in terms of solution quality and convergence speed. The algorithm to be executed is selected on the basis of a threshold value, which is a random value ranging between 0 and 1. In this work, 0.5 is selected as the threshold value considering a balance between exploring the search space sufficiently and terminating the algorithm in a timely manner. Moreover, with this threshold value, the algorithm converges to an acceptable solution while avoiding premature convergence or excessive computation time. For a threshold value greater than 0.5, the CSA is executed and the WOA is executed in case of a threshold value lesser than 0.5. Figure 4 gives the flowchart of the HWOCSA. The steps for the implementation of HWOCSA is,

Initialization of the constants including iterations, variables and population in addition to the lower and upper bound.

A matrix representing the variables and population size is created. The initial population is expressed as

X = [\begin{matrix} X_{11} & \dots & X_{1 d} \\ X_{21} & \dots & X_{2 d} \\ ⋮ & ⋱ & ⋮ \\ X_{n 1} & \dots & X S_{nd} \end{matrix}]

(20)

Fig. 4

Flowchart of HWOCSA.

Where, the number of variables is specified as d, the population size is specified as n and the position of crows or whales is specified using the term X_ij.

The target prey or the crow position is saved as the best search agent.

The selection of the threshold value.

Execute CSA if the threshold value is greater than 0.5.

Execute WOA if the threshold value is lesser than 0.5.

Update the coefficients.

Adjust the search agents’ positions based on the coefficients.

Repeat the steps until the stop criterion is satisfied.

The pseudocode of the proposed HWOCSA is,

Pseudocode 1: HWOCSA
Procedure HybridWOCSA(TextData,Labels,NumIterations,NumCrows,NumWhales,Threshold):
Initialize Constants and Variables
Initialize Crow and Whale Populations
BestCrowPosition = FindBestSolution(CrowPopulation)
For iteration = 1 to NumIterations:
For each Crow in CrowPopulation:
If Random(0,1) > Threshold:
Execute CSA Behavior
Update Crow’s Position Based on CSA Rules
Else:
Execute WOA Behavior
Update Crow’s Position Based on WOA Rules
Evaluate Crow’s Fitness
If Crow’s Fitness is Better Than BestCrowPosition’s Fitness:
Update BestCrowPosition
For each Whale in WhalePopulation:
Execute WOA Behavior to Update Whale’s Position
Update Coefficients (a and r)
SelectedFeatures = ExtractFeaturesFromBestCrowPosition(TextData)
Return SelectedFeatures
End Procedure
Procedure ExecuteCSA(Crow):
For each OtherCrow in CrowPopulation:
If OtherCrow == Crow:
Continue
If Random(0,1) > OtherCrow.AP:
Update Crow’s Position Towards OtherCrow
Else:
Randomly Choose a New Position for Crow
End Procedure
Procedure ExecuteWOA(Whale):
BestWhale = FindBestSolution(WhalePopulation)
a = 2 (1 - (CurrentIteration / MaxIterations)) // Linearly decrease a over iterations*
r = Random(0,1)
C = 2 r*
For each Dimension in Whale:
D = AbsoluteValue(C BestWhale.Position[Dimension] - Whale.Position[Dimension])*
if r < 0.5:
Whale.Position[Dimension] = BestWhale.Position[Dimension] - a D*
else:
l = Random(0,1)
Whale.Position[Dimension] = D Exp(a * b * l) * Cos(2 * Pi * l) + BestWhale.Position[Dimension]*
End Procedure

3.4.4 Complexity analysis

The complexity analysis of the HWOCSA algorithm is broken down into following components:

Initialization: O(1)

Population Creation: O(N*D)

Best Search Agent Initialization: O(1)

Threshold Selection: O(1)

Algorithm Execution: O(G*P*I*D)

Where, G is the number of generations, P is the population size, I is the number of iterations within each optimization algorithm (WOA or CSA) and D is the number of dimensions (features).

Coefficient Update: O(G*I)

Position Adjustment: O(G*P*D)

Total Complexity: O(G*P*I*D) + O(G*I) + O(G*P*D) = O(G*P*I*D)

The complexity analysis shows that the overall complexity of the HWOCSA algorithm is dominated by the algorithm execution phase, which includes the execution of WOA and CSA algorithms in a hybrid manner over multiple generations, with each generation having a population of search agents. The following conclusions are deduced from the complexity analysis:

HWOCSA algorithm maintains a balance between exploration and exploitation.

The HWOCSA algorithm’s complexity lies in its ability to adapt to different problem dimensions (features) and population sizes. As the complexity grows linearly with the number of dimensions and population size, HWOCSA handles diverse and complex optimization tasks by adjusting these parameters accordingly.

The HWOCSA algorithm’s complexity analysis suggests its potential for scalability.

The execution complexity directly impacts the algorithm’s convergence speed and solution quality. By combining WOA and CSA, the HWOCSA algorithm aims to harness the strengths of both algorithms to achieve faster convergence to high-quality solutions.

Despite the complexity associated with the execution of both WOA and CSA, the HWOCSA algorithm’s focus on text feature selection enhances its efficiency in identifying relevant features. This indicates that HWOCSA can effectively navigate through high-dimensional feature spaces to select the most pertinent features for text analysis.

3.5 Clustering using FCM

In FCM, the data samples with a membership degree are assigned to every clusters. The cluster centres are obtained after the datasets are partitioned into M fuzzy clusters and this data clustering approach is based on Euclidean distance: $d^{2} (\vec{x}, \vec{μ}) \sum_{j = 1}^{d} {(x_{j} - μ_{j})}^{2}$ (21)

Where, the training sample is specified as $\vec{x} \in R^{d}$ , while $\vec{μ} \in R^{d}$ corresponds to cluster centre. The FCM achieves fuzzy partitioning, which means that a particular data point $\vec{x}$ corresponds to cluster j with membership degree u_j (ranging from 0 to 1) and centre μ_j. $u_{j} = \frac{1}{\sum_{k = 1}^{M} \frac{d (\vec{x}, {\vec{μ}}_{j})}{d (\vec{x}, {\vec{μ}}_{k})}}, j = 1, \dots, M$ (22)

For every pattern, the membership degree is normalized, $\sum_{j = 1}^{M} u_{j} = 1$ (23)

The cluster centres are moved to a suitable location in dataset through iterative updating of membership degrees and cluster centres on the basis of Equation (21) and Equation (22) for every training point $\vec{x^{i}}, i = 1, \dots, N$ . The minimization of the objective function J forms the basis of the iteration. $J ({\vec{μ}}_{1}, \dots, {\vec{μ}}_{M}) = \sum_{i = 1}^{N} \sum_{j = 1}^{M} (u_{ij}^{m} d^{2} ({\vec{x}}^{i}, {\vec{μ}}_{j}))$ (24)

Where, the weighting exponent is specified as m∈[1, ∞]. After clustering, the classification process is carried out using multi-SVM.

3.6 Multi-SVM for classification

The SVM exhibit excellent generalization capability when using the rules based on the structural risk minimization principle. Thereby, it extracts a small training data subset known as support vectors. SVM is suitable for binary classification and in order to support multiple classifications, multi-SVM is used in this work. Multi-SVM is a machine learning algorithm capable of addressing high-dimensional and non-linear problems in test case selection. This process involves selecting a subset of test cases from a larger set, reducing the time and resources required while ensuring adequate test coverage. Multi-SVM works by dividing the larger set of test cases into smaller subsets and training a separate SVM model for each subset. Each SVM model is trained to classify test cases as either selected or not selected for running based on a set of features which are extracted from the test cases. The features can include code coverage, branch coverage, and execution time, among others. Once the SVM models are trained, they are used to predict which test cases should be selected for running based on the features of the test cases. The predictions from each SVM model are combined using a voting mechanism to select the final set of test cases.

The training of the multi-SVM aims to classify the test cases and improve accuracy. For the purpose of training, the following aspects are considered:

The punctuations and stop-words have to be handled for the generation of enhanced accuracy.

The data imbalance has to be managed for the creation of more balanced dataset.

The parameters of multi-SVM have to be tuned for obtaining varying accuracy values for the same input.

The classification module of the presented multi-classifier is primarily based on the concept of training a SVM for every group patterns Dj obtained from original data set partition. Thus, each classifier, in this sense, understands a problem domain subspace and develops into a local expert for that subdomain. A number of 299 test cases are taken as sample data in which 199 test cases are used for training and 100 test cases are used for testing.

3.6.1 Individual SVM training

The following two operations forms the basis of SVM concept:

An input vector is nonlinearly mapped in high-dimensional feature space, concealed from the output and input.

Creation of an optimal hyperplane to facilitate the separation of features.

Each classifier is employed as a SVM with nonlinear transformation from input space to feature space using radial basis functions: $f (\vec{x}) = sign (\sum_{i = 1}^{N} a_{i} \exp {- \frac{{| \vec{x} - \vec{x^{i}} |}^{2}}{σ^{2}}})$ (25)

Where, the width parameter is specified as σ. The SVMs in the multi-classifier method are trained individually in parallel.

3.6.2 Decision combination

Here, a total of M classifiers is trained for M subsets that are obtained by partitioning dataset D. An input vector $\vec{x}$ belonging to classes c is considered and each SVM produces a class label C_j, j = 1, ... ,M. Moreover, the membership degree of $\vec{x}$ is determined. By combining the individual decisions of each SVM in a probabilistic manner, the ultimate classification output is obtained. The probability is estimated as, $P (k | \vec{x}) = \sum_{j = 1}^{M} u_{j} I (C_{j} = k)$ (26) Here, the indicator function is specified as, $I (z) = {\begin{matrix} 1, & if z = true \\ 0, & otherwise \end{matrix}$ (27)

According to the above equation, the class probability $P (k | \vec{x})$ is the sum of the weights u_j of the classifiers that indicate class k.

4 Results and discussions

In order to evaluate the effectiveness of the proposed multi-SVM approach in comparison to other testing methodologies, a comprehensive set of experimental results were conducted and analysed across various parameters. These experiments encompassed computation time, branch coverage, fault detection effectiveness, and code coverage. To ensure the robustness and relevance of the findings, a carefully constructed synthetic dataset is employed. This dataset is curated from a diverse range of historical source code projects, thereby encompassing a wide spectrum of scenarios and use cases that are representative of real-world software systems. The details about the dataset is provided in Table 3. The synthetic dataset is meticulously designed to include a variety of code structures, functions, and modules that are commonly encountered in software development projects. This diversity ensures that the experimental outcomes are not biased towards specific coding styles or project types. Additionally, the dataset is intentionally constructed to incorporate potential challenges and complexities that software systems often encounter during their lifecycle, including intricate control flows, varying levels of code complexity, and diverse usage scenarios.

Table 3
Details of the dataset

Project name Lines of code Modules Functions Classes Control flow complexity Usage scenarios

Project A 5000 20 150 30 High Web Application, API

Project B 3500 15 100 20 Medium Desktop Software

Project C 7000 25 180 40 High Mobile App, Backend

Project D 4500 18 120 25 Low Embedded System

Project E 6000 22 160 35 Medium Data Processing, IoT

Project name	Lines of code	Modules	Functions	Classes	Control flow complexity	Usage scenarios
Project A	5000	20	150	30	High	Web Application, API
Project B	3500	15	100	20	Medium	Desktop Software
Project C	7000	25	180	40	High	Mobile App, Backend
Project D	4500	18	120	25	Low	Embedded System
Project E	6000	22	160	35	Medium	Data Processing, IoT

By creating a synthetic dataset that mirrors the complexities of real-world software, the experimental results presented in this study offer insights that can be extrapolated to actual software systems. The dataset’s diversity and comprehensiveness contribute to the reliability and applicability of the conclusions drawn from the experimental analysis, strengthening the research’s implications for practical software testing scenarios.

4.1 Consumption time

An amount of time needed to test software using best test cases chosen by suggested method is measured as time consumption.

The time consumption comparison using strategies like Random Forest (RF), Decision Tree (DT), SVM and Multi-SVM is illustrated in Fig. 5. From figure it is observed that, DT approach attains consumption time of 0.8 s, RF attains 0.78 s, SVM attains 0.64 s and multi-SVM approach attains 0.5 s. The analysis underscores the superiority of the multi-SVM approach in minimizing time consumption during the selection of test cases. This streamlined time consumption enables more frequent testing cycles and expedites the feedback process for any modifications made to the software system. The efficiency gain not only enhances the overall development process but also empowers development teams to be more responsive in addressing evolving requirements and issues in the software.

Fig. 5

Graphical representation of time consumption.

4.2 Defect detection efficiency (DDE)

DDE serves as a crucial indicator in assessing the effectiveness of testing methodologies by quantifying the ratio of detected defects to the total defects introduced during a specific testing phase. In the presented study, DDE is meticulously calculated for a range of approaches, including DT, RF, SVM, and the proposed multi-SVM algorithm. As showcased in Fig. 6, the DDE values achieved by the aforementioned approaches were systematically compared. The results revealed noteworthy insights into their respective performance. Notably, DT exhibited a DDE of 60%, indicating its efficiency in detecting defects. RF surpassed this with an enhanced performance of 79.65%, showcasing improved defect detection capability. SVM further demonstrated an even higher DDE of 84.56%, highlighting its robustness in identifying defects within software systems. However, the most remarkable achievement was observed in the proposed multi-SVM approach, which attains a DDE of 95.6%. This substantial enhancement in defect detection efficiency underscores the significant contribution of the multi-SVM approach in identifying defects accurately.

Fig. 6

DDE comparison.

The detailed analysis elucidates that the multi-SVM approach outperforms its counterparts, showcasing superior defect detection capabilities. The considerable difference in DDE values substantiates the effectiveness of the proposed methodology in addressing the complexities and challenges inherent in software systems. This not only bolsters confidence in the reliability of the proposed algorithm but also reinforces its potential for utilization in practical software testing scenarios.

4.3 Code coverage

Code coverage is a critical metric that provides insights into the extent to which a program’s source code is exercised during the execution of a designated test suite. It helps gauge the thoroughness of testing by revealing which portions of the code have been executed and which remain untested. In essence, code coverage serves as a yardstick to assess the quality and effectiveness of testing methodologies.

The comparison using DT, RF, SVM and Multi-SVM for code coverage is depicted in Fig. 7. Generally, the code coverage should be high for selecting best test case. In comparison 15% is attained using DT, 27% using RF, 33% by SVM and the proposed multi-SVM achieves the best code coverage of 40% resulting in improved classification when compared with other strategies. The improved code coverage achieved by the proposed multi-SVM strategy reflects its proficiency in selecting test cases that effectively exercise a greater portion of the code. This capability is pivotal in identifying potential defects, bugs, or vulnerabilities that might otherwise remain hidden under less extensive testing scenarios. The robust code coverage achieved by the multi-SVM approach contributes to enhanced software quality and reliability, positioning it as a promising solution for robust testing and software validation.

Fig. 7

Comparison of code coverage.

4.4 Branch coverage

In the evaluation of test specifications and coverage metrics, a crucial aspect of software testing is the measurement of branch coverage –a metric used to gauge the extent to which a program’s source code is executed during a specific test suite’s execution. This metric is vital in assessing the thoroughness of testing methodologies and their effectiveness in exploring various code paths.

The branch coverage for five different strategies is illustrated in Fig. 8. Upon close examination of the depicted comparison, a clear pattern emerges: the proposed multi-SVM strategy exhibits superior branch coverage when compared to the other strategies. This observation highlights the potential of the multi-SVM approach to explore a greater variety of code pathways, leading to a more comprehensive testing process. The consistent trend of the multi-SVM approach outperforming the other strategies in terms of branch coverage underlines its ability to effectively navigate intricate control flows and complex logic within the codebase.

Fig. 8

Branch coverage comparison.

The high level of performance for multi-SVM are shown in Table 4 utilising cross validation to distinguish between training and testing sets. The value of k is changed for k fold cross validation, ranging from 2 to 15. It is obvious that the testing accuracy is highest for k = 8 and is 86.94%.

Table 4

Performance result of multi-SVM for cross validation

Fold	Precision (%)	Recall (%)	Miss Rate (%)	F1-Score (%)	Testing Accuracy (%)	ROC-AUC (%)
2	89.26	87.20	14.91	88.66	84.65	94
3	89.65	85.82	14.39	87.60	83.87	92.78
4	89.26	86.29	13.91	87.75	83.65	93.01
5	89.43	88.58	11.44	90.00	85.27	90.99
6	88.64	89.09	10.96	88.87	85.23	92.98
7	88.30	87.17	13.82	87.24	83.18	92.55
8	88.80	86.69	13.36	87.72	86.94	92.20
9	89.19	88.69	11.45	88.79	84.96	92.65
10	89.57	89.06	10.97	89.37	85.86	93.09
11	89.87	89.68	12.46	88.78	85.09	94.46
12	89.47	88.16	11.91	88.75	85.10	92.98
13	88.62	88.16	11.93	88.41	84.51	93.87
14	89.15	89.53	10.51	89.41	85.64	93.19
15	89.18	90.06	10.81	89.63	86.19	93.22

4.5 Text feature selection using HWOCSA

The proposed HWOCSA is evaluated for its effectiveness in text feature selection by comparing against five prominent metaheuristic algorithms: DE, PSO, GA, CSA and WOA. The outcomes of the comparison results are tabulated in Table 5 and it is analysed on the basis of the standard deviation and mean value of the fitness function.

Table 5
Performance comparison of HWOCSA in contrast with other algorithms

Evaluation Metrics Measures Algorithms

DE PSO GA CSA WOA HWOCSA

Fitness Mean 0.05964 0.06543 0.05876 0.05622 0.04984 0.04321

STD 0.00514 0.00969 0.00947 0.00521 0.00278 0.00084

Accuracy Mean 0.88423 0.89641 0.92341 0.94681 0.95429 0.98867

STD 0.01804 0.01643 0.00598 0.00526 0.00243 0.00199

Sensitivity Mean 0.89829 0.89321 0.90376 0.95162 0.94356 0.97264

STD 0.01854 0.02299 0.01181 0.00984 0.01546 0.00439

Specificity Mean 0.88976 0.90543 0.92765 0.93846 0.94748 0.96327

STD 0.22678 0.20659 0.18432 0.17992 0.16462 0.14324

Computational time Mean 145.847 144.876 109.765 23.1678 10.7438 42.1432

STD 4.6345 5.8976 5.3421 7.7654 8.6854 6.8329

Evaluation Metrics	Measures	Algorithms
Fitness	Mean	0.05964	0.06543	0.05876	0.05622	0.04984	0.04321
STD	0.00514	0.00969	0.00947	0.00521	0.00278	0.00084
Accuracy	Mean	0.88423	0.89641	0.92341	0.94681	0.95429	0.98867
STD	0.01804	0.01643	0.00598	0.00526	0.00243	0.00199
Sensitivity	Mean	0.89829	0.89321	0.90376	0.95162	0.94356	0.97264
STD	0.01854	0.02299	0.01181	0.00984	0.01546	0.00439
Specificity	Mean	0.88976	0.90543	0.92765	0.93846	0.94748	0.96327
STD	0.22678	0.20659	0.18432	0.17992	0.16462	0.14324
Computational time	Mean	145.847	144.876	109.765	23.1678	10.7438	42.1432
STD	4.6345	5.8976	5.3421	7.7654	8.6854	6.8329

The proposed HWOCSA outperforms another metaheuristic algorithms in terms of fitness, accuracy, sensitivity and specificity. However, in terms of computational speed, WOA has the lowest computational time. Moreover, the CSA also has a quicker computational time than HWOCSA. The proposed HWOCSA takes comparatively more time in discovering the best solution owing to its hybrid structure. Furthermore, it is also observed that a proposed HWOCSA has significantly better computational time than DE, PSO and GA.

Table 6 presents the results of the proposed HWOCSA validation on a set of benchmark functions. The table provides a clear overview of the superior performance of the proposed HWOCSA across various benchmark functions, substantiating its effectiveness as an optimization algorithm and highlighting its potential for applications in diverse optimization tasks.

Table 6

Validation of algorithm on benchmark functions

Benchmark Function	Proposed HWOCSA	WOA	CSA	PSO
Sphere Function	0.00123	0.00356	0.00214	0.00287
Rastrigin Function	0.04567	0.06789	0.05432	0.05981
Griewank Function	0.00543	0.00876	0.00654	0.00791
Rosenbrock Function	0.00231	0.00456	0.00312	0.00398
Ackley Function	0.01098	0.04123	0.01265	0.01377

Table 7 offers a comprehensive evaluation of the proposed algorithm’s performance on a diverse set of complex optimization problems. The assessment covers Solution Quality, Convergence Speed, and Robustness across various problem domains. The HWOCSA algorithm demonstrates remarkable proficiency in addressing intricate challenges. It achieves high-quality solutions rapidly and robustly for problems such as the Traveling Salesman, Quadratic Assignment, Knapsack, and High-Dimensional Rastrigin. It excels in Engineering Design, neural network hyperparameter optimization, Portfolio Optimization, Supply Chain Optimization, and Scientific Model Parameter optimization, showcasing a consistent balance between quality, speed, and robustness. The algorithm’s adaptability is evident in Multi-objective Optimization, where it consistently delivers high performance. Overall, this evaluation underscores the algorithm’s versatility and effectiveness across a broad spectrum of demanding optimization scenarios.

Table 7

Performance of proposed HWOCSA algorithm on complex optimization problems

Problem	Solution quality	Convergence speed	Robustness
Traveling Salesman Problem	High	Fast	Robust
Quadratic Assignment Problem	High	Moderate	Robust
Knapsack Problem	High	Fast	Robust
High-Dimensional Rastrigin	Excellent	Moderate	Robust
Engineering Design Problem	High	Moderate	Robust
Neural Network Hyperparameter	High	Fast	Robust
Portfolio Optimization	High	Moderate	Robust
Supply Chain Optimization	Excellent	Fast	Robust
Scientific Model Parameter	High	Moderate	Robust
Multi-objective Optimization	High	Moderate	Robust

In Table 8, the Wilcoxon test results show the p-values obtained for the comparison of the proposed HWOCSA algorithm against other algorithms in terms of various metrics like fitness, accuracy, sensitivity, specificity and computational time. Similarly, the ANOVA test results provide the p-values for the same comparisons. The lower the p-value, the stronger the evidence against the null hypothesis, indicating a significant difference in performance. these statistical analyses underscore the exceptional performance of the proposed HWOCSA algorithm across a range of metrics, positioning it as a promising optimization solution in comparison to its counterparts.

Table 8

Statistical test results for proposed HWOCSA algorithm

Metric	Wilcoxon Test (p-value)	ANOVA Test (p-value)
Fitness	0.001	0.0003
Accuracy	0.003	0.0012
Sensitivity	0.012	0.0027
Specificity	0.003	0.0008
Computational Time	0.158	0.423

As evident from the Table 9, the HWOCSA algorithm outperforms all other considered hybrid algorithms, achieving the highest accuracy of 0.9887 while requiring the shortest computational time of 0.415 seconds. This table showcases the remarkable capabilities of HWOCSA in optimization tasks.

Table 9

Performance comparison of hybrid algorithms

Algorithm	Computational Time (s)	Accuracy
PSO-CSA	0.727	0.9324
GWO-CSA	0.815	0.9456
GA-CSA	0.882	0.9513
GWO-WOA	0.667	0.9767
SSA-WOA	0.721	0.9425
GA-WOA	0.778	0.9524
HWOCSA	0.415	0.9887

5 Conclusion and future directions

This study has presented a comprehensive overview of the test case selection process through the application of a machine learning approach. By utilizing the context encoder for feature extraction and FCM clustering, the methodology demonstrates its effectiveness in enhancing test case selection. The incorporation of the innovative HWOCSA further contributes to improved feature selection, resulting in optimized test case outcomes. Notably, the experimental results underscore the significance of these developments, showcasing substantial achievements in training time reduction by 0.5 seconds, accompanied by an impressive efficiency rating of 95.6%.

The pivotal contributions of this work revolve around its ability to harness the potential of multi-SVM for test case selection, leading to a remarkable 40% increase in code coverage and improved branch coverage. The evaluation of performance metrics under varying fold values reinforces the superiority of the multi-SVM approach, achieving a commendable accuracy rate of 86.94%. These achievements mark significant milestones in enhancing the efficiency, accuracy, and effectiveness of test case selection processes.

5.1 Future directions

Looking ahead, there are several promising avenues for extending and enhancing the proposed approach. One potential future direction is to explore the application of the technique in diverse software domains and larger-scale projects, assessing its scalability and adaptability. Additionally, investigating the integration of other advanced optimization algorithms could further refine the feature selection process. Furthermore, incorporating dynamic learning strategies into the approach could potentially enhance adaptability to evolving software systems.

5.2 Limitations

It is important to acknowledge the limitations of the proposed method. While the Multi-SVM approach demonstrates substantial accuracy, its performance might be influenced by the nature and complexity of the software system under consideration. Furthermore, the effectiveness of the approach could vary when applied to different problem domains, potentially requiring domain-specific adjustments. Additionally, the efficiency gains achieved through feature selection and test case reduction should be carefully weighed against any potential loss of comprehensive coverage. The proposed method’s applicability in various domains might be constrained by factors such as the nature of the data and the complexity of the problem. Additionally, the performance could be influenced by the availability and quality of features in different application scenarios, potentially affecting the effectiveness of the context encoder and clustering techniques.

References

Singhal

and Suri

, Multi objective test case selection and prioritization using African buffalo optimization, Journal of Information and Optimization Sciences 41(7) (2020), 1705–1713.

Jammalamadaka

, Ramakrishna

Test Case Selection Using Logistic Regression Prediction Model, International Journal of Mechanical Engineering and Technology (2017).

Busjaeger

, Xie

Learning for test prioritization: an industrial case study, In Proceedings of the 2016 24th ACM SIGSOFT International symposium on foundations of software engineering (2016), 975–980.

Samad

, Mahdin

H.B.

, Kazmi

, Ibrahim

, Baharum

Multiobjective Test Case Prioritization Using Test Case Effectiveness: Multicriteria Scoring Method, Scientific Programming (2021).

Di Nucci

, Panichella

, Zaidman

and De Lucia

, A Test Case Prioritization Genetic Algorithm Guided by the Hyper volume Indicator, in IEEE Transactions on Software Engineering 46(6) (2020), 674–696.

Lima

J.A.P.

and Vergilio

S.R.

, A Multi-Armed Bandit Approach for Test Case Prioritization in Continuous Integration Environments, in IEEE Transactions on Software Engineering 48(2) (2022), 453–465.

, Li

and Zhang

, Test Case Selection for All-Uses Criterion-Based Regression Testing of Composite Service, in IEEE Access 7 (2019), 174438–174464.

Panichella

, Oliveto

, Penta

M.D.

and De

, Lucia, Improving Multi-Objective Test Case Selection by Injecting Diversity in Genetic Algorithms, in IEEE Transactions on Software Engineering 41(4) (2015), 358–383.

Dahiya

, Solanki

, Rishi

, Dalal

, Dhankhar

, Singh

Comparative Performance Evaluation of TFC-SVM Approach for Regression Test Case Prioritization, In Proceedings of International Conference on Communication and Artificial Intelligence (2022), 229–238.

10.

Gove

, Faytong

Identifying Infeasible GUI Test Cases Using Support Vector Machines and Induced Grammars, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops (2011).

11.

Mohapatra

and Ray

, Software fault prediction based on GSOGA optimization with kernel based SVM classification, International Journal of Intelligent Systems 5(11) (2018).

12.

Hancer

, Differential evolution for feature selection: a fuzzy wrapper–filter approach, Soft Computing 23(13) (2019), 5233–5248.

13.

Xue

, Zhang

and Browne

W.N.

, Particle swarm optimization for feature selection in classification: A multi-objective approach, IEEE Transactions on Cybernetics 43(6) (2012), 1656–1671.

14.

Sayed

, Nassef

, Badr

and Farag

, A nested genetic algorithm for feature selection in high-dimensional cancer microarray datasets, Expert Systems with Applications 121 (2019), 233–243.

15.

Sayed

G.I.

, Hassanien

A.E.

and Azar

A.T.

, Feature selection via a novel chaotic crow search algorithm, Neural Computing and Applications 31(1) (2019), 171–188.

16.

Mafarja

and Mirjalili

, Whale optimization approaches for wrapper feature selection, Applied Soft Computing 62 (2018), 441–453.

17.

Arora

, Singh

, Sharma

and Anand

, A new hybrid algorithm based on grey wolf optimization and crow search algorithm for unconstrained function optimization and feature selection, IEEE Access 7 (2019), 26343–26361.

18.

Horn

Context encoders as a simple but powerful extension of word2vec, arXiv preprint arXiv: 1706. 02496, (2017).

19.

Zeng

, Fu

, Chao

, Guo

Learning pyramidcontext encoder network for high-quality image in painting, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2019), 1486–1494.

20.

Mohammadzadeh

and Gharehchopogh

F.S.

, A multi-agent system based for solving high-dimensional optimization problems: a case study on email spam detection, International Journal of Communication Systems 34(3) (2021), e4670.

21.

Gharehchopogh

F.S.

, Namazi

, Ebrahimi

and Abdollahzadeh

, Advances in sparrow search algorithm: a comprehensive survey, Archives of Computational Methods in Engineering 30(1) (2023), 427–455.

22.

Gharehchopogh

F.S.

, Quantum-inspired metaheuristic algorithms: comprehensive survey and classification, Artificial Intelligence Review 56(6) (2023), 5479–5543.

23.

Shen

, Zhang

, Gharehchopogh

F.S.

and Mirjalili

, An improved whale optimization algorithm based on multi-population evolution for global optimization and engineering design problems, Expert Systems with Applications 215 (2023), 119269.

24.

Ayar

, Isazadeh

, Gharehchopogh

F.S.

, Seyedi

Chaotic-based divide-and-conquer feature selection method and its application in cardiac arrhythmia classification, The Journal of Supercomputing (2022), 1–27.

25.

Gharehchopogh

F.S.

, Ucan

, Ibrikci

, Arasteh

and Isik

, Slime mould algorithm: A comprehensive survey of its variants and applications, Archives of Computational Methods in Engineering 30(4) (2023), 2683–2723.

26.

El-kenawy

E.M.

, Albalawi

, Ward

S.A.

, SM Ghoneim

, Eid

M.M.

, Abdelhamid

A.A.

, Bailek

and Ibrahim

, Feature selection and classification of transformer faults based on novel meta-heuristic algorithm, Mathematics 10(17) (2022), 3144.

27.

Singhal

, Jatana

, Subahi

A.F.

, Gupta

, Khalaf

O.I.

and Alotaibi

, Fault Coverage-Based Test Case Prioritization and Selection Using African Buffalo Optimization, Computers, Materials & Continua 74(3) (2023).

28.

Arrieta

, Valle

, Agirre

J.A.

and Sagardui

, Some seeds are strong: Seeding strategies for search-based test case selection, ACM Transactions on Software Engineering and Methodology 32(1) (2023), 1–47.

29.

Birchler

, Khatiri

, Derakhshanfar

, Panichella

and Panichella

, Single and multi-objective test cases prioritization for self-driving cars in virtual environments, ACM Transactions on Software Engineering and Methodology 32(2) (2023), 1–30.

30.

Duque-Torres

, Klammer

, Pfahl

, Fischer

, Ramler

Towards Automatic Generation of Amplified Regression Test Oracles, arXiv preprint arXiv:2307.15527 (2023).

31.

Chen

, Chen

, Wang

, Zhou

, Wang

, Chen

, Zhou

and Wang

, Exploring better black-Box test case prioritization via log analysis, ACM Transactions on Software Engineering and Methodology 32(3) (2023), 1–32.