Comparative analysis of optimization algorithm-based training algorithms for a few shot continual learning

Abstract

Incremental learning relies on the availability of ample training data for novel classes, a requirement that is often unfeasible in various application scenarios, particularly when new classes are rare groups that are pricey or challenging to attain. The main focus of incremental learning is on the tricky task of continuously learning to classify new classes in incoming data with no erasing knowledge of old classes. The research intends to develop a comparative analysis of optimization algorithms in training few-shot continual learning models to conquer catastrophic forgetting. The presented mechanism integrates various steps: pre-processing and classification. Images are initially pre-processed through contrast enhancement to elevate their quality. Pre-processed outputs are then classified by employing Continually Evolved Classifiers, generated to address a matter of catastrophic forgetting. Furthermore, to further enhance performance, Serial Exponential Sand Cat Swarm optimization algorithm (SE-SCSO) is employed and compared against ten other algorithms, containing Grey Wolf Optimization (GWO) algorithm, Moth flame optimization (MFO), cuckoo Search Optimization Algorithm (CSOA), Elephant Search Algorithm (ESA), Whale Optimization Algorithm (WOA), Artificial Algae Algorithm (AAA), Cat Swarm Optimization (CSO), Fish Swarm Algorithm (FSA), Genetic Bee Colony (GBC) Algorithm, and Particle swarm optimization (PSO). From the experiment results, SE-SCSO had attained the maximum performance with an accuracy of 89.6%, specificity of 86%, precision of 83%, recall of 92.3% and f-measure of 87.4%.

Keywords

Few shot continual learning classifier optimization algorithm learning

1. .Introduction

In real-life situations, learning often happens gradually from continuous data. Even though, traditional object detection modes lack this ability. They usually assume a fixed or unchanging data distribution. Continually learning innovative ideas from original data is crucial. A mechanism, called increased mental learning enables this without forgetting previously gained details. This theme has attained significant interest recently as it has real-life purposes. For original classes, incremental learning presumes perfect training data, which is often impractical, especially when novel classes are rare or expensive to gather [1]. This motivates the examination of incremental few-shot learning – a complex scheme that aims to continuously attain innovative tasks with only a few instances. Incremental learning focuses on the challenge of identifying original classes in new data without forgetting past classes, which is the core of continual learning. Multi-class and multi-task incremental learning have commonly been part of previous research [2]. To conquer the issue of catastrophic forgetting, some approaches have applied additional restrictions on scheme parameters by penalising changes in those parameters [3]. Other attempts have tried to impose restraints on exemplars of past classes by penalizing deviations in embedding angles or restricting output logits. Structural learning models have been achieved through incremental learning of DAEs using network loss and hidden unit integration [4].

Based on whether task individuality is clearly provided or needs to be inferred, IL is separated into three kinds: class/domain/task IL. Straightly fine-tuning a scheme according to freshly added data may significantly decrease its presentation of past data, which is called catastrophic forgetting. Catastrophic forgetting is the chief challenge for IL/CL tasks. Catastrophic forgetting is a regular problem in ML and AI, where a mechanism trained on multiple tasks tends to forget formerly learned data when uncovered to new tasks [5]. This issue emerges due to the shortage of efficient mechanisms to separate and preserve the learned knowledge from diverse tasks [4].

FSCIL is an ML challenge that integrates teaching new classes from a small count of labeled samples lacking forgetting lastly learned information. FSCIL needs DNN schemes to gradually attain new tasks from a lesser count of labelled examples while retaining knowledge of old classes. FSIL is an ML mechanism that focuses on gradually enhancing a model’s performance by learning from limited data samples. This strategy is useful when getting a greater count of labeled data is difficult or time-consuming. Incremental learning lets a mechanism adapt and learn from new data without having to retrain the entire model again [6]. This is useful when dealing with changing environments or tasks that evolve over time. This machine learning system can easily face major problems like forgetting what it has learned before and overfitting the current data, which can seriously impact its performance [7]. FSCIL tries to create algorithms that can continuously learn new ideas from just a few examples, without losing the knowledge it had about older concepts [4].

The research aims to compare various optimization models used for FSCL in order to address catastrophic forgetting. The proposed mechanism involves several steps: pre-processing and classification. At first, the images experience pre-processing through contrast enhancement to improve their quality. Pre-processed outputs are then classified by utilizing Continually Evolved Classifiers, which were designed to mitigate matters of catastrophic forgetting. Furthermore, to further enhance performance, an optimization algorithm named SE-SCSO is utilized alongside ten other optimization algorithms, including PSO, MFO, CSOA, ESA, AAA, CSO, WOA, FSA, GWO and GBC.

The main aim of the suggested strategy is as follows:

–
To analyze and compare the performance of training algorithms by employing optimization algorithms, namely SE-SCOA, GWO, PSO, MFO, CSOA, ESA, AAA, CSO, WOA, FSA, and GBC, in a few shot incremental learning.
–
To assess the effectiveness of the presented strategy utilizing metrics such as f-measure, recall, specificity, precision, and accuracy.

Work is arranged as stated: the initial part introduces the topic. The second section reviews related works, evaluating previous studies and research. The third part presents the mechanism. The fourth part discusses findings and evaluations, and the fifth part presents the conclusion.
2. .Literature survey

The section details the thorough review based on past studies and research regarding the optimization algorithm-based training algorithms for few-shot continual learning.

Matthias Perkonigg et al. [8] developed a continual learning model to address domain shifts occurring at unpredictable time points. Furthermore, various approaches were tailored to handle deviations in a continuous data flow while also mitigating CF. Rehearsal was considered by dynamically storing a subset of training data in memory to prevent forgetting when expanding to additional domains. Memory was arranged by identifying pseudo-domains, signifying diverse manner in the data stream. Prakhar Kaushik et al. [9] examined RMNs, which was inspired by the best-fit hypothesis. The importance of weights was reflected in mappings for tasks at hand, with large weights assigned to important parameters. Additionally, the RMNs learned an optimized representational overlap, which surpasses twin issues of catastrophic forgetting and remembering. RMN model achieved traditional performance on various CL datasets, even outshining data replay strategies without violating restrictions for an ideal continual learning scheme.

Chenze Shao and Yang Feng [10] noted catastrophic forgetting can impact both CL and traditional static training. NN, such as those used in neural machine translation strategies, are vulnerable to catastrophic forgetting when they attain from a static training set. The main issue is that the training process tends to pay more attention to the most recently presented samples, neglecting the earlier ones. This imbalanced training leads to a lack of balanced exposure to all the training data during each model update. To address this problem, the researchers presented COKD. This approach employs dynamically updated teacher schemes, trained on specific data orders, to iteratively give complementary information to student technique. By doing so, the student model can learn a more balanced representation of the training data, reducing the impact of catastrophic forgetting. Jiaji Luo et al. [11] developed a lifelong learning system that uses Net2Net knowledge transfer and few-shot learning to recognise radar signals. Schemes adapt to variations in the dataset, allowing it to learn new classes, transfer knowledge, and apply few-shot learning. When tested on a dataset of eight radar signal types, the technique achieved higher recognition accuracy compared to a dataset with twelve signal types.

Mengxue Kang et al. [4] investigated a strategy that dynamically combines both feature information and semantics with consideration of within-class consistency and between-class discriminativeness on a Transformer-based detector. Distillation preserves the class discriminativeness by distilling class-level feature detachment, semantic and other categories amongst them, while within-class consistency is conserved by distilling feature details within each category as well as semantic information at the instance level. On both the Pascal VOC dataset as well as MS COCO benchmark datasets extensive experiments are conducted. The scheme outperforms all past CNN-based state-of-the-art mechanisms below several investigational strategies, with a notable mAP enhancement under a one-step IOD task. Xiang Song et al. [7] presented an architecture, named APPLE, which contains three key components. Initially, they have initiated an adaptive pseudo-label strategy to address the issue of missing labels. This mechanism leverages the existing strategy to label the old classes for new samples. Secondly, scholars have presented a clustered sampling strategy to obtain more diverse samples, which helps exterminate the crisis of catastrophic forgetting in the MLCIL setting. Finally, they have designed a class attention decoder to mitigate the object feature dilution problem in multi-label samples. The extensive experiments executed on the PASCAL VOC 2007 and MS-COCO datasets have defined that the presented APPLE mechanism significantly outperforms other state-of-the-art CIL strategies.

2.1. .Challenges

The challenges from the previous research regarding the optimization algorithm-based training algorithms for few-shot continual learning are assured in the segment, and research gaps established are below:

The mechanism employed in [4] needs to be further enhanced to attain better performance in accuracy and efficiency. RNMs [9] only require the learned weights of a CL network. This is gained by creating diverse sub-network mappings in each structure. Henceforth, this approach elevates the total count of parameters in the structure. Thus, to enhance performance in Few-Shot Incremental Learning, the research focuses on enhancing the training using optimization algorithms.

3. .Few shot continual learning with optimization algorithm-based training

The work intends to generate a strategy for few-shot continual learning with a new training algorithm to conquer catastrophic forgetting. At first, images are pre-processed through contrast adjustment to elevate their quality. Pre-processed images are classified by employing Continually Evolved Classifiers, which are designed to overcome catastrophic forgetting. Furthermore, an optimization algorithm, SE-SCSO, is used along with ten other optimization techniques, like PSO, GBC Algorithm, MFO, CSOA, ESA, WOA, AAA, CSO, FSOA, and GWO algorithm. The architectural representation of the presented strategy is stated in Fig. 1.

Figure 1.

Diagrammatic representation of few-shot incremental learning.

3.1. .Pre-processing by CLAHE

CLAHE is a mechanism that directs to diminish the noise and enhance the contrast of an image. CLAHE is an elevated mechanism of the previous strategy called AHE. CLAHE defines the kernel metrics and tries to rearrange the intensity rate of each and every pixel of the image input with an average value of the kernel weighting each pixel and neighbouring pixel. During the noise-reducing procedure, the kernel sizes are also impacted to attain better outcomes. For contrast enhancement, the CLAHE image provides the boundary value on a histogram. The limit value is named clip limit and the clip limit on the histogram is calculated as follows;

\begin{aligned} ω = \frac{a}{b} (1 + \frac{ε}{100} (c_{max} - 1)) \end{aligned}

(1)

where,

a

states the size of area,

b

states the value of grayscale, and

ε

is the clip factor that states the histogram among 0 to 100.

3.2. .Continually evolved classifier-based few-shot incremental learning

The few-shot part of this strategy refers to the scheme’s capability to learn from a small count of examples, which helps it better understand new, unseen data.

Initial training using CNN model: CNN structures are made up of diverse components, including pooling, fully connected, and convolution layers. A typical structure consists of an array of pooling and convolutional layers followed by additional fully connected layers. During forward propagation, input data passes through pooling and convolutional layers to generate an output [12].

Pseudo incremental learning phase: To update the weights of the classifier, a continually evolving classifier is employed, which integrates a phase for classifier adaptation. It is trained in each entity assembly according to the global context of the preceding session [13]. GAT is utilized to establish connections among these prototype vectors and transmit context data.

To enhance the learning performance, algorithms, such as PSO, GBC, FSOA, CSO, WOA, AAA, ESA, CSOA, MFO, GWO and SE-SCSO are used to learn the weights of the classifier.

3.2.1. .Training algorithms

The following section provides the training algorithms employed in this study. The algorithms used for the training are PSO, GBC, FSOA, CSO, WOA, AAA, ESA, CSOA, MFO, GWO and SE-SCSO. Here, the weight matrix is constantly adjusted by following the presented algorithms.

a) Solution encoding

A solution encoding scheme acts as an examination of a prime group of weight matrices for a classifier. Furthermore, a fitness function is employed to check if the group matches the optimization goal. In this case, the fitness of a solution is defined by investigating the accuracy of the strategy. Figure 3 states the solution encoding, which denotes the weights selected optimally using the eleven optimization algorithms.

Figure 2.

Solution representation.

Figure 3.

Comparative evaluation of proposed method with 5-way 5-shot for data set 1, (a) accuracy, (b) specificity, (c) precision, (d) recall, (e) f-measure.

3.2.2. .SE-SCSO

The SExpSCSO is a novel generated optimization mechanism that incorporates the plan of EWMA [14] from the SCSO model [15]. SCSO mechanism is inspired by the behavior of sand cats, where the capability to quickly identify and catch prey is a key feature. Sand cats have the remarkable capability to identify even the faintest sounds, which allows them to quickly locate and catch their prey. EWMA is a helpful statistical scheme that commonly makes use of making predictions, employing equal weights for all observations. By incorporating EWMA into the SCSO scheme, it can converge faster and give an optimal solution. Furthermore, the low memory requirement of EWMA is an added advantage when implementing it in an optimization mechanism. This scheme is a model based on population, and sand cats’ approach to prey searching relies on emitting minimal frequency noise. The prey can identify the predator based on the capability of their ears. The distance between the optimal solution and the current location $x$ of the sand cat is measured as shown in Eq. (3.2.2), in order to accurately arrange the stage of attack in SCSO. In this strategy, a hypothetical sensitivity range of the sand cat is stated as a circle; the route of progress is defined by a random angle ( $θ$ ) on circle.

\begin{aligned} x^{t + 1} & = x_{A}^{t} (1 - U * \cos ϕ * r a n d (0, 1) + U * \cos ϕ) * \\ {\frac{1}{ι} [x_{e}^{t} - (1 - ι) * (ι * x^{t - 1} + (1 - ι) * (ι * x^{t - 2} + (1 - ι) * x_{E}^{t - 3}))]} \end{aligned}

(2)

where,

ι

represents the optimal weight,

U

reveals the sensitivity range of each cat,

x_{A}^{t}

is the current location, a sensitivity range of the sand cat is hypothetical as a circle group way is determined by a random angle (

ϕ

) on a circle and

x_{e}^{t}

is the updated cat’s locations in a phase of exploitation.

3.2.3. .PSO

The PSO [16] is a stochastic optimization scheme that is enthused by swarm behavior in nature, like insects, herds, birds, and fish. These swarms cooperate to find food by changing their search patterns based on their own learning experiences and those of other members. The PSO algorithm comprises a population (swarm) of candidate solutions (particles) that travel in search space by employing simple formulas. Particles’ movements are influenced by their well-known position and the entire swarm’s well-known position. As suitable positions are identified, they guide the swarm’s movements. This procedure is repeated with the hope, but not guarantee, of eventually finding a satisfactory solution. Algorithm1 represents the pseudo-code of PSO model.

where, $A_{j}$ denotes the position, $B_{j}$ denotes the current velocity of the particle, $j$ denotes number of particle, $P_{B E S T}$ denotes the distance of each particle to the food source, $G_{B E S T}$ denotes the closest distance the swarm has achieved to the food source during iteration.

3.2.4. .GBC

The GBC [17] algorithm is a novel hybrid strategy that merges two natural algorithms: ABC and GA. The primary purpose of any metaheuristic algorithm is to discover an optimal possible solution. Gaining this needs a balance between exploitation and exploration. The original ABC algorithm excels at the exploration procedure to uncover new solutions in the optimization search space. Even though, it struggles with solution exploitation and necessitates extensive computational time to converge and identify optimal solutions. On the other hand, the GA is proficient at exploitation through operations like mutation and crossover. Nevertheless, it lacks the ability to effectively explore optimization search space. The GBC algorithm employs genetic algorithm operators during the onlooker phase to elevate information exchange among onlooker and employee bees, aiding in the discovery of the best solution. This combination approach seeks to gain a harmonious balance between exploitation and exploration while addressing the drawbacks of early convergence and long computational times commonly linked with naturally inspired metaheuristic evolutionary algorithms. Algorithm 2 represents the pseudo code of GBC model.

3.2.5. .FSOA

The FSOA [18] is a strategy that emulates the behavior of fish when searching for food. From an optimization standpoint, this behavior can be likened to “learning,” as it guides the fish swarm in exploring new regions and locating fresh food resources within the design space. When exhibiting arbitrary behavior, fish swim around aimlessly in search of food and other fish. In searching behavior, upon discovering an area with abundant food, a fish will rapidly swim straight towards that region. Swarming behavior leads the swimming fish to naturally form a swarm to avoid danger. In pursue behavior, when one fish in the swarm finds food, others follow it to food. If fish become trapped in an area, leaping behavior prompts them to leap and search for food in other regions. This algorithm employs the natural behaviors of fish to effectually explore and optimize a design space, similar to how fish search for the best feeding grounds. Algorithm 3 represents the pseudo code of FSA model.

3.2.6. .CSO

The CSO [19] mimics the behaviors of cats and operates in two main modes: tracing mode and seeking mode. In looking for mode, cats carefully assess their atmosphere without moving, to determine the best next move. In tracing mode, cats actively pursue their target with a specific velocity. A CSO-based solution thinks about both the cost of data transmission amongst execution costs and the dependent resources, reducing the number of iterations needed to find a preparation strategy and ensuring fair load balancing across obtainable resources. The process begins with the original population of cats, some in seeking mode and others in tracing mode. During the seeking mode, most cats engage in global exploration, while the remaining cats stay stationary and intelligently update their positions. This method involves the utilization of two components: SMP, which determines the number of duplications for each cat, and CDC, which indicates the modifications to be made slowly. The tracing mode identifies the swiftly moving cats and their local exploration as they move towards the next optimal position. Every cat symbolizes a task-resource mapping that is adjusted based on its current mode. Evaluating the cats’ fitness helps to identify the mapping with the lowest cost. In each cycle, a novel group of cats is selected to be in tracing mode. The final resolution yields optimal mapping with minimal cost.

\begin{aligned} u_{p, q} = u_{p, q} + m_{1} \times C_{1} \times (x_{b s t, q} - x_{p, q}) \end{aligned}

(3)

where,

q = 1, 2, \dots, M

x_{b s t, q}

is the cat’s position, which has the best fitness value,

x_{p, q}

is a position of cat

p

C_{1}

is a constant and

m_{1}

is an arbitrary number ranging from 0 to 1. Algorithm 4 represents the pseudo code of CSO model.

where, ${S e l f}_{P C}$ denotes the self-position consideration.

3.2.7. .WOA

WOA [20] is a meta-heuristics algorithm that mimics bubble-net hunting and social behavior of humpback whales in oceans to optimize numerical problems. The WOA involves three phases: exploration (prey searching), surrounding prey, and exploitation (assailing prey by bubble-net scheme). In the stage of exploration, the search agent (humpback whale) randomly seeks the best solution according to each agent’s position. Then, during hunting, whales encircle prey and regard the current best candidate solution as the optimal one. In the stage of exploitation, the whales use a bubble net to attack prey. A scheme of surrounding nature is employed to update the spot of whales to the best search agent. The bubble-net strategy combines a spiral updating position mechanism and a shrinking encircling mechanism. Algorithm 5 represents the pseudo code of WOA model.

where, $S^{∙}$ denotes the best search agent, $u$ denotes the current iteration number, $u_{M a x}$ denotes the maximum number of iteration, $g$ denotes the random number between [0,1], the elements of $c$ are linearly decreased from 2 to 0 and $D$ denotes the coefficient vector.

3.2.8. .AAA

The AAA [21] is a meta-heuristic method that draws inspiration from behaviors of microalgae. ‘Algae’ refers to a range of photosynthetic organisms with a distinct chlorophyll and nucleus, enabling them to produce their own food from carbon dioxide and water. Artificial algae are designed to replicate the characteristics of real algae. Like real algae, artificial algae can budge to light sources and undergo photosynthesis through a helical swimming motion. They can also adjust to their surroundings, change overriding species, and replicate through mitotic diversion. The AAA consists of three main elements: the evolutionary procedure, alteration, and helical progression. Under favourable nutrient conditions, if an algal colony gains sufficient light, it flourishes and regenerates, producing two new algal cells through mitotic division, the same as real-life algae. Insufficiently illuminated algal colonies can survive for a period, but eventually perish. Conversely, well-performing algal colonies, offering cost-effective and optimal solutions, thrive due to their high nutrient intake. As part of the evolutionary process, the demise of a small algal colony cell leads to the replication of a cell from the largest algal community. Adaptation occurs as an underdeveloped algal colony strives to emulate the largest community in its environment, altering the algorithm’s starvation level. The energy of an algal cell at a time is directly proportional to the nutrients it absorbs at that time. Thus, cells closer to the surface possess more energy and greater opportunity to move within a liquid. Algorithm 6 represents the pseudo code of AAA model.

3.2.9. .ESA

The ESA [22] is a contemporary metaheuristic search algorithm. It is motivated by behavior of elephant herds. The algorithm separates search agents into two gender groups, each stating a different search pattern – local and global. The ESA combines an evolutionary mechanism and a balance between exploitation and exploration. Evolutionary mechanisms make sure the solution is enhanced over time. Exploitation focuses on the search in the local area, while exploration looks for better solutions in the broader search space. The goal is to discover the globally best solution while evading getting stuck in local optima. Algorithm 7 represents the pseudo code of PSO model.

3.2.10. .CSOA

The CSOA [23] is a newly generated meta-heuristic optimization algorithm utilized for resolving optimization issues. It is encouraged by brood parasitism of certain cuckoo races and incorporates Levy flight’s random walks. An algorithm is presented by the procreation tactic of cuckoo species and the Lévy flight behavior of certain birds. In this strategy, a few cuckoo birds lay eggs in carrier bird nests, but the carrier birds may recognize these eggs as not their own and may either destroy or abandon them. Each nest contains an egg, on behalf of a potential solution, with the cuckoo egg representing a new and promising option. The resulting answer is a fresh option derived from the existing one, with some characteristics modified. The Cuckoo is a popular bird known for its sounds and aggressive reproduction strategy. Each nest contains a single cuckoo egg at its most fundamental level, while nests with multiple eggs symbolize a range of potential solutions. The cuckoo search algorithm draws inspiration from this distinctive breeding behavior. Algorithm 8 represents the pseudo code of CSO model.

where, $C$ denotes the current iteration and $C_{M A X}$ indicates the maximum iteration.

3.2.11. .MFO

Moths, similar to butterflies, are elegant insects with a remarkable ability to navigate in the dark. They have evolved to utilize moonlight for their night flights through a strategy known as transverse orientation, where they maintain a fixed angle concerning the moon. It allows them to voyage long distances on a straight trail. MFO [24] algorithm treats candidate resolutions as moths and the problem’s elements as spots of moths in space. Both moths and flames are considered solutions, with moths actively searching the area and flames representing the best positions discovered by the moths. Essentially, the flames serve as markers that moths leave behind while exploring the search space. In simpler terms, the flames act as flags or markers that the moths leave behind as they navigate search space. Every moth explores close to a flag (flame) and revises it if it discovers a superior solution. This ensures that the moths always maintain awareness of their best solution and do not misplace it. Algorithm1 represents the pseudo code of MFO model.

3.2.12. .GWO

Grey wolves are apex predators that live in packs. Their pack hierarchy has four roles: alpha, omega, beta, and delta. GWO [3] algorithm mimics this pack structure and hunting behaviour. Grey wolves work together to search, surround, and attack prey. They have a keen sense of prey location and can effectually encircle their targets. This algorithm simulates the leadership and hunting dynamics observed in natural grey wolf populations. The leader, or alpha, habitually leads the hunting procedure. The delta and beta wolves may also occasionally participate in the hunt. Even though, in an abstract search environment, it has no previous data on the location of the optimal solution (the prey). GWO algorithm permits its search agents to update their spots as per locations of alpha, beta, and delta wolves, and move towards prey. Unfortunately, the GWO algorithm is susceptible to stagnation in local solutions due to these operators. While the encircling mechanism proposed does demonstrate some degree of exploration, the GWO algorithm would benefit from additional operators to further emphasize exploration. Algorithm 10 represents the pseudo code of GWO model.

where, $G$ are linearly decreased from 2 to 0 over the course of iterations, $R$ and $S$ are coefficient vectors, $Q$ indicates the current iteration and $Q_{M a x}$ indicates the maximum iteration.

4. .Results

In this section, the presented mechanism is experimented with conventional strategies. The proposed model is implemented using PYTHON tool and the dataset “CIFAR-100 Python” is employed for experimentation. The performance of the suggested algorithm is analyzed for estimation metrics including accuracy, specificity, precision, recall and f-measure and compared with the mentioned algorithms. Table 1 represents the hyper parameters of algorithms.

Table 1
Hyper parameters of Algorithms.

Methods Number of population Maximum number of iterations

PSO 20 50

GBC 20 100

FSOA 20 100

CSO 25 100

WOA 50 100

AAA 30 100

ESA 20 100

CSOA 20 100

MFO 20 100

GWO 30 100

SE-SCSO 50 100

Methods	Number of population	Maximum number of iterations
PSO	20	50
GBC	20	100
FSOA	20	100
CSO	25	100
WOA	50	100
AAA	30	100
ESA	20	100
CSOA	20	100
MFO	20	100
GWO	30	100
SE-SCSO	50	100

4.1. .Performance metrics

4.1.1. .Accuracy

Accuracy refers to a level of precision and is expressed as,

\begin{aligned} a c c = \frac{P_{P} + P_{N}}{P_{P} + P_{N} + N_{P} + N_{N}} \end{aligned}

(4)

here, True positive stated,

P_{P}

True negative stated

P_{N}

, False positive stated

N_{P}

, and False negative stated

N_{N}

4.1.2. .Specificity

It is a representation of true negatives,

\begin{aligned} s p e = \frac{P_{N}}{P_{P} + N_{P}} \end{aligned}

(5)

4.1.3. .Precision

It is denoted as the simultaneous existence of exceeding two dimensions with respect to each other, as shown in Eq. (6).

\begin{aligned} p r e = \frac{P_{P}}{P_{P} + N_{P}} \end{aligned}

(6)

4.1.4. .Recall

Recall assesses the accuracy of positive predictions.

\begin{aligned} r c l = \frac{P_{P}}{P_{P} + N_{N}} \end{aligned}

(7)

4.1.5. .F1-score

The F1 Score combines precision and recall into a single measure.

\begin{aligned} F 1 S = \frac{2 \times p r e \times r c l}{p r e + r c l} \end{aligned}

(8)

4.2. .Dataset description

Figure 4.

(a) Performance dropping rate, (b) five class test accuracy for data set 1.

Dataset 1: “CIFAR-100 Python”: CIFAR100 is a classification dataset that consumes 60,000 images, each measuring 32 by 32 pixels and in full color. There are 100 classes, with 500 training images and 100 testing images per class. The dataset is divided into 60 base classes and 40 new classes. Eight new incremental sessions are added, each involving a five-way, five-shot classification task [25].

Dataset 2: “The CIFAR-10 dataset”: These datasets consist of 60,000 32 $\times$ 32 color images spread across 10 classes, with 6,000 images per class. It is divided into 50,000 training images and 10,000 test images. For training, the dataset is segmented into 5 batches, each comprised of 10,000 images. The test batch involves accurately 1,000 arbitrarily selected images from each class. As for the training batches, they hold the residual images in an arbitrary sequence, with some batches potentially having more images from one class than others. However, jointly, the training batches enclose accurately 5,000 images from every class [26].

4.3. .Comparative evaluation

The presented mechanism was compared to other traditional mechanism, like PSO [16], GBC [17], FSOA [18], CSO [19], WOA [20], AAA [21], ESA [22], CSOA [23], MFO [24], and GWO [3]. The experiment analysis was executed in 5-way 1-shot and 5-way 5-shot.

4.3.1. .Dataset 1

a) Comparative evaluation (5-way 5-shot) for data set 1

Figure 3 comprises a performance evaluation of the proposed strategy with other conventional approaches, like PSO, GBC, FSOA, WOA, CSO, ESA, AAA, MFO, GWO, and CSOA in a 5-way 5-shot evaluation. 5-way 5-shot requires gathering data from 5 classes, with each category containing 5 images. Figure 3(a) illustrates the accuracy of the suggested mechanism. For the fifth session, the SE-SCSO attained an accuracy of 0.676, whereas the FSOA, AAA, ESA, and MFO attained an accuracy of 0.448, 0.537, 0.567, and 0.626. Figure 3(b) states the specificity of the proposed model. SE-SCSO attained a specificity of 0.625 for the fourth session; however, the AAA and MFO attained 0.525 and 0.579. Figure 3(c) illustrates the precision of the proposed model. The SE-SCSO attained a precision of 0.830 for Session 1, whereas PSO, GBC and WOA attained a precision of 0.445, 0.486, and 0.608. Figure 3(d) states recall regarding the suggested scheme. For Session 2 the SE-SCSO attained a recall of 0.883, whereas the ESA, CSOA, and GWO attained a recall of 0.737, 0.797, and 0.856. Figure 3(e) illustrates the f-measure regarding the proposed model. The SE-SCSO gained an f-measure of 0.874 for Session 1, whereas MFO and GWO gained an f-measure of 0.820, and 0.852.

Figure 5.

Comparative evaluation of proposed method with 1 way 5-shot for data set 1, (a) accuracy, (b) specificity, (c) precision, (d) recall, (e) f-measure.

Figure 4 illustrates performance dropping rate and five class test accuracy. Figure 4(a) illustrates performance dropping rate. Performance dropping rate refers to the rate at which the performance of an algorithm, system, or process decreases over time or as the scale or complexity of the task was elevated. Performance dropping rate of PSO is 0.228, GBC attained 0.225, FSOA attained 0.225, CSO is 0.224, WOA is 0.223, AAA is 0.223, ESA is 0.221, CSOA gained 0.222, MFO attained 0.222, GWO attained 0.220, and SE-SCSO attained a performance dropping rate of 0.219. Figure 4(b) illustrates five class test accuracy. Here five class test accuracy for SE-SCSO strategy is 0.737, whereas FSOA is 0.544, ESA is 0.642, and CSOA gained 0.667 for Session 1.

Figure 6.

(a) Performance dropping rate, (b) five class test accuracy.

Table 2

Comparative evaluation for 5-way 5-shot regarding data set 2.

Sessions	PSO	GBC	FSOA	CSO	WOA	AAA	ESA	CSOA	MFO	GWO	SE-SCSO
Accuracy
1	0.733	0.750	0.767	0.784	0.80	0.817	0.834	0.851	0.867	0.884	0.926
2	0.628	0.652	0.674	0.697	0.721	0.744	0.767	0.790	0.813	0.836	0.878
3	0.596	0.612	0.628	0.644	0.660	0.676	0.693	0.725	0.756	0.788	0.817
4	0.503	0.535	0.566	0.598	0.630	0.661	0.692	0.708	0.724	0.740	0.788
5	0.468	0.493	0.518	0.543	0.568	0.592	0.617	0.642	0.667	0.692	0.757
Specificity
1	0.562	0.596	0.630	0.665	0.699	0.733	0.788	0.811	0.833	0.856	0.879
2	0.544	0.572	0.599	0.627	0.655	0.683	0.711	0.738	0.766	0.793	0.815
3	0.527	0.548	0.569	0.591	0.611	0.633	0.655	0.674	0.696	0.717	0.757
4	0.466	0.487	0.508	0.529	0.551	0.572	0.593	0.614	0.635	0.656	0.686
5	0.387	0.415	0.442	0.469	0.497	0.525	0.553	0.581	0.608	0.635	0.657
Precision
1	0.571	0.594	0.616	0.639	0.662	0.672	0.712	0.759	0.798	0.827	0.857
2	0.547	0.567	0.586	0.606	0.626	0.645	0.665	0.697	0.736	0.776	0.807
3	0.523	0.539	0.556	0.573	0.590	0.619	0.658	0.685	0.705	0.724	0.747
4	0.443	0.464	0.502	0.541	0.579	0.607	0.624	0.641	0.658	0.675	0.706
5	0.423	0.463	0.485	0.507	0.528	0.549	0.571	0.592	0.613	0.635	0.646
Recall
1	0.735	0.757	0.779	0.802	0.824	0.846	0.868	0.891	0.913	0.935	0.954
2	0.697	0.719	0.741	0.764	0.786	0.808	0.830	0.852	0.875	0.897	0.913
3	0.602	0.631	0.660	0.689	0.718	0.748	0.777	0.806	0.835	0.865	0.876
4	0.556	0.584	0.613	0.642	0.670	0.699	0.727	0.756	0.785	0.813	0.835
5	0.482	0.509	0.537	0.566	0.594	0.622	0.649	0.677	0.706	0.734	0.768
F-measure
1	0.643	0.666	0.688	0.711	0.734	0.749	0.783	0.820	0.852	0.878	0.903
2	0.613	0.634	0.655	0.676	0.697	0.718	0.738	0.767	0.799	0.832	0.857
3	0.559	0.582	0.604	0.626	0.648	0.677	0.713	0.741	0.764	0.788	0.807
4	0.493	0.517	0.552	0.587	0.622	0.649	0.672	0.694	0.715	0.737	0.765
5	0.450	0.485	0.511	0.534	0.558	0.583	0.607	0.632	0.656	0.680	0.702

b) Comparative evaluation (5-way 1-shot) for data set 1

Table 3

Comparative evaluation for 5-way 1-shot regarding data set 2.

Sessions	PSO	GBC	FSOA	CSO	WOA	AAA	ESA	CSOA	MFO	GWO	SE-SCSO
Accuracy
1	0.691	0.711	0.732	0.753	0.774	0.795	0.815	0.837	0.857	0.878	0.907
2	0.577	0.604	0.632	0.659	0.687	0.714	0.742	0.769	0.797	0.824	0.857
3	0.555	0.595	0.615	0.635	0.655	0.675	0.695	0.715	0.735	0.755	0.795
4	0.474	0.502	0.529	0.557	0.585	0.613	0.641	0.668	0.696	0.724	0.776
5	0.425	0.454	0.483	0.512	0.541	0.569	0.598	0.627	0.656	0.685	0.735
Specificity
1	0.579	0.609	0.639	0.669	0.698	0.728	0.758	0.788	0.818	0.848	0.865
2	0.453	0.489	0.527	0.564	0.601	0.637	0.675	0.712	0.749	0.786	0.806
3	0.416	0.448	0.481	0.513	0.545	0.577	0.609	0.642	0.675	0.707	0.735
4	0.373	0.401	0.428	0.457	0.485	0.513	0.541	0.568	0.597	0.625	0.657
5	0.320	0.352	0.384	0.416	0.447	0.479	0.511	0.543	0.575	0.607	0.625
Precision
1	0.583	0.608	0.633	0.659	0.685	0.710	0.736	0.761	0.787	0.813	0.835
2	0.513	0.542	0.571	0.601	0.629	0.658	0.687	0.716	0.746	0.775	0.799
3	0.423	0.462	0.501	0.541	0.579	0.618	0.657	0.689	0.697	0.705	0.724
4	0.362	0.396	0.429	0.464	0.498	0.532	0.566	0.601	0.635	0.669	0.697
5	0.346	0.373	0.421	0.449	0.476	0.503	0.531	0.558	0.586	0.613	0.635
Recall
1	0.726	0.746	0.766	0.786	0.806	0.826	0.846	0.866	0.876	0.906	0.934
2	0.655	0.678	0.703	0.727	0.751	0.775	0.798	0.823	0.847	0.871	0.899
3	0.568	0.596	0.625	0.654	0.682	0.711	0.738	0.767	0.795	0.824	0.857
4	0.506	0.534	0.562	0.590	0.618	0.646	0.674	0.702	0.729	0.757	0.788
5	0.473	0.500	0.527	0.554	0.580	0.607	0.634	0.660	0.687	0.714	0.746
F-measure
1	0.646	0.671	0.694	0.717	0.740	0.764	0.787	0.810	0.828	0.856	0.882
2	0.575	0.603	0.630	0.658	0.685	0.712	0.739	0.766	0.793	0.819	0.847
3	0.485	0.521	0.556	0.591	0.626	0.661	0.696	0.726	0.743	0.759	0.785
4	0.422	0.455	0.487	0.519	0.552	0.584	0.615	0.647	0.678	0.711	0.739
5	0.399	0.427	0.468	0.495	0.523	0.551	0.577	0.605	0.632	0.659	0.686

Table 4

Dropping rate regarding data set 2.

Sessions	PSO	GBC	FSOA	CSO	WOA	AAA	ESA	CSOA	MFO	GWO	SE-SCSO
1	0.725	0.743	0.761	0.778	0.796	0.821	0.849	0.878	0.907	0.936	0.969
2	0.677	0.706	0.735	0.763	0.792	0.815	0.832	0.851	0.868	0.886	0.906
3	0.642	0.663	0.679	0.703	0.724	0.746	0.768	0.791	0.813	0.835	0.856
4	0.634	0.657	0.676	0.693	0.711	0.727	0.744	0.762	0.778	0.796	0.825
5	0.559	0.580	0.601	0.622	0.643	0.664	0.684	0.705	0.726	0.747	0.768

Table 5

Accuracy regarding data set 2.

Sessions	PSO	GBC	FSOA	CSO	WOA	AAA	ESA	CSOA	MFO	GWO	SE-SCSO
1	0.768	0.785	0.801	0.816	0.832	0.848	0.864	0.879	0.896	0.912	0.957
2	0.662	0.686	0.711	0.735	0.759	0.783	0.807	0.832	0.856	0.879	0.896
3	0.535	0.565	0.595	0.625	0.655	0.685	0.715	0.745	0.775	0.805	0.825
4	0.509	0.538	0.569	0.601	0.631	0.662	0.693	0.724	0.755	0.785	0.806
5	0.484	0.510	0.537	0.564	0.590	0.617	0.643	0.670	0.697	0.724	0.747

Figure 5 comprises performance evaluation of proposed strategy with other conventional approaches like PSO, GBC, FSOA, CSO, WOA, AAA, ESA, CSOA, MFO, and GWO in a 5-way 1-shot manner. 5-Way 1-shot requires gathering data from 5 classes, with each containing 1 image. Figure 5(a) states accuracy of the presented scheme. For the fifth session, the SE-SCSO attained an accuracy of 0.646, whereas the FSOA, AAA, ESA, and MFO attained an accuracy of 0.399, 0.477, 0.503, and 0.555. Figure 5(b) states specificity of the proposed model. The SE-SCSO attained specificity of 0.613 for the fourth session, however, the AAA and MFO attained 0.466 and 0.556 for Session 4. Figure 5(c) illustrates the precision of the proposed model. The SE-SCSO attained a precision of 0.812 for Session 1, whereas PSO, GBC and WOA attained a precision of 0.438, 0.452, and 0.556. Figure 5(d) states recall regarding the presented scheme. For Session 2 the SE-SCSO attained a recall of 0.875, whereas the ESA, CSOA, and GWO attained recall of 0.745, 0.778, and 0.845. Figure 5(e) illustrates the f-measure regarding the proposed model. The SE-SCSO gained an f-measure of 0.859 for Session 1, whereas MFO and GWO attained an f-measure of 0.800, and 0.838.

Figure 6 illustrates performance dropping rate and five class test accuracy. Figure 6(a) illustrates The performance dropping rate of PSO is 0.248, GBC attained 0.246, FSOA attained 0.246, CSO is 0.246, WOA is 0.244, AAA is 0.247, ESA is 0.243, CSOA gained 0.242, MFO attained 0.241, GWO attained 0.240, and SE-SCSO attained a performance dropping rate of 0.236. Figure 6(b) illustrates five class test accuracy. Here five class test accuracy for SE-SCSO strategy is 0.934, whereas FSOA is 0.687, ESA is 0.807, CSOA gained 0.837 for Session 1.

4.3.2. .Dataset 2

This section demonstrates the clear description of the dataset 2. Table 2 depicts the comparative evaluation for 5-way and 5-shot regarding dataset 2. Here, the analysis is based upon the accuracy, specificity, precision, recall and F-measure. The overall analysis states that the SE-SCSO model performs better than the other models. For Sessions 5, the accuracy of the SE-SCSO model was 0.757, which was better than other models.

Table 3 demonstrates the comparative evaluation for 1-way and 5-shot regarding dataset 2. Here, the analysis is based upon the accuracy, specificity, precision, recall and F-measure. The overall analysis states that the SE-SCSO model performs better than the other models. For Sessions 5, the accuracy of the SE-SCSO model was 0.735, which was better than other models.

Tables 4 and 5 states the dropping rate and accuracy regarding the dataset 2. Here, the analysis states that SE-SCSO model performs better than the other models.

Tables 6 and 7 exhibited the computational time for dataset 1 and dataset 2. Regarding Table 5 the computational time of SE-SCSO model was 847.7, whereas the PSO, GBC, FSOA were 2038.4, 1915.8, and 1793.3 for Sessions 1. Regarding Table 6 the computational time of SE-SCSO model was 75.8, whereas the PSO, GBC, FSOA were 2038.4, 1915.8, and 1793.3 for Session 1.

Table 8 demonstrates the $p$ -value test. The $P$ value indicates probability, assuming there’s no effect or difference (null hypothesis), of getting an outcome as extreme as or more excessive than the one observed. Here, the analysis states that for both the 5-way 5-shot and 5-way 1-shot. Moreover, the analysis states that the $P$ -value of SE-SCSO model was than the other models.

Figure 7 demonstrates the analysis on average accuracy for both the dataset 1 and 2 regarding the 5-way and 1-shot. Figure 7a) demonstrates the analysis on average accuracy for 5-way. Here, the maximal average accuracy is attained by the SE-SCSO with the values of 0.790 for dataset 1, and 0.7834 for dataset 2. Figure 7b) illustrates the analysis on average accuracy for 1-shot. Here, the maximal average accuracy is attained by the SE-SCSO with the values of 0.769 for dataset 1, and 0.814 for dataset 2. Table 9 demonstrates the T-value test. For the 5-way 5 shot, the T-value test for dataset 1 was 0.0471, and dataset 2 was 0.0193. For the 5-way 1 shot, the $T$ -value test for dataset 1 was 0.013, and dataset 2 was 0.011.

Table 10 demonstrates the statistical test of the models regarding the 5-way 5 shot and 5 way 1-shot for dataset 1 and 2. Here, the statistical test is done by calculating the best, mean and variance. For 5 way 5 shot, the best value of SE-SCSO was 0.896 regarding dataset 1, and for 5 way 1 shot, the best value of SE-SCSO was 0.907 regarding dataset 2.

5. .Conclusion

The study aims to compare optimization algorithms for training few-shot continual learning models in order to address catastrophic forgetting. The process includes pre-processing and classification steps. At first, the pictures undergo contrast enhancement as a way to enhance the quality of the images. Pre-processed outcomes are input into classification utilizing

Table 6

Computation time for dataset 1.

Sessions	PSO	GBC	FSOA	CSO	WOA	AAA	ESA	CSOA	MFO	GWO	SE-SCSO
Computation time (s)
1	2038.4	1915.8	1793.3	1670.7	1548.1	1425.5	1302.9	1180.4	1057.8	935.3	847.7
2	2494.6	2157.1	2019.6	1882.1	1744.5	1606.9	1469.5	1331.9	1194.4	1056.8	982.8
3	2750.9	2398.4	2245.9	2093.4	1940.9	1788.4	1635.9	1483.4	1330.9	1178.4	1117.9
4	3178.2	2939.7	2572.3	2304.8	2137.3	1969.8	1802.4	1634.9	1467.5	1299.9	1253.1
5	3563.4	3281.1	2998.6	2516.2	2333.7	2151.3	1968.8	1786.4	1603.9	1421.5	1388.1

Table 7

Computation time for dataset 2.

Sessions	PSO	GBC	FSOA	CSO	WOA	AAA	ESA	CSOA	MFO	GWO	SE-SCSO
Computation time (s)
1	2012.1	1886.8	1760.5	1634.6	1508.8	1382.9	1257.1	1131.3	1005.5	879.6	753.8
2	2213.9	2082.1	2082.1	1818.2	1686.3	1554.4	1422.5	1290.6	1158.7	1026.8	894.9
3	2444.8	2303.9	2163.1	2022.2	1881.3	1740.5	1599.6	1458.7	1317.8	1176.9	1036.1
4	2741.1	2584.6	2428.3	2271.9	2115.5	1959.1	1802.7	1646.3	1489.9	1333.6	1177.2
5	3320.7	3120.4	2920.2	2719.9	2519.7	2319.5	2119.3	1919.0	1718.8	1518.5	1318.3

Table 8

$P$ -value test.

	PSO	GBC	FSOA	CSO	WOA	AAA	ESA	CSOA	MFO	GWO	SE-SCSO
5-way 5-shot
Dataset1	4.04e^{- 19}	3.11 e^{- 19}	2.84 e^{- 19}	2.48 e^{- 19}	2.13e^{- 20}	1.77e^{- 21}	1.42e^{- 22}	1.06e^{- 23}	7.09e^{- 24}	3.55e^{- 25}	1.47e^{- 28}
Dataset2	0.090	0.070	0.050	0.036	0.027	0.019	0.017	0.013	0.010	0.009	0.007
5-way 1-shot
Dataset1	1.35e^{- 25}	1.21e^{- 25}	7.08e^{- 26}	3.44e^{- 26}	9.09e^{- 27}	6.75e^{- 27}	5.40e^{- 27}	4.05e^{- 27}	2.70e^{- 27}	1.35e^{- 27}	6.42e^{- 30}
Dataset2	2.70e^{- 01}	2.13e^{- 01}	1.26e^{- 01}	8.00e^{- 02}	4.43e^{- 02}	2.86e^{- 02}	2.09e^{- 02}	8.73e^{- 03}	5.16e^{- 03}	5.92e^{- 04}	2.49e^{- 05}

Figure 7.

Analysis on average accuracy a) 5-way shot; b) 1-way shot.

Table 9

T-value test.

Dataset	SE-SCSO
5-way 5-shot
Dataset1	0.0471
Dataset2	0.0193
5-way 1-shot
Dataset1	0.013
Dataset2	0.011

Table 10

Statistical test.

	PSO	GBC	FSOA	CSO	WOA	AAA	ESA	CSOA	MFO	GWO	SE-SCSO
Dataset1
5-way 5-shot
Best	0.617	0.644	0.673	0.702	0.732	0.761	0.789	0.819	0.850	0.878	0.896
Mean	0.459	0.493	0.528	0.563	0.597	0.632	0.667	0.702	0.736	0.771	0.790
Variance	0.158	0.151	0.145	0.140	0.134	0.129	0.122	0.118	0.114	0.107	0.107
5-way 1-shot
Best	0.596	0.621	0.646	0.672	0.696	0.725	0.747	0.772	0.797	0.865	0.883
Mean	0.417	0.451	0.487	0.522	0.556	0.591	0.626	0.661	0.696	0.748	0.769
Variance	0.179	0.169	0.160	0.150	0.140	0.133	0.121	0.111	0.101	0.117	0.114
Dataset2
5-way 5-shot
Best	0.734	0.750	0.767	0.784	0.801	0.817	0.834	0.851	0.868	0.885	0.926
Mean	0.586	0.609	0.631	0.654	0.676	0.699	0.721	0.744	0.767	0.789	0.834
Variance	0.148	0.142	0.136	0.130	0.125	0.119	0.113	0.107	0.102	0.096	0.093
5-way 1-shot
Best	0.690	0.711	0.732	0.753	0.774	0.795	0.816	0.837	0.858	0.879	0.907
Mean	0.545	0.573	0.598	0.623	0.648	0.673	0.698	0.723	0.748	0.773	0.814
Variance	0.146	0.138	0.134	0.130	0.126	0.122	0.118	0.113	0.109	0.105	0.093

Continually Evolved Classifiers, which were designed to address the problem of catastrophic forgetting. Also, in order to enhance performance even more, the SE-SCSO is utilized and its outcomes are contrasted with ten other algorithms including PSO, GBC Algorithm, CSO, FSA, AAA, WOA, CSOA, MFO, GWO, and ESA algorithm. The experiment results show that among all the algorithms compared, SE-SCSO attained the maximum performance with an accuracy of 89.6%, specificity of 86%, precision of 83%, and recall of 92.3% and f-measure of 87.4%.

Footnotes

Abbreviation

ORCID iDs

Rupatai Lichode

Swapnili Karmore

References

Ashfahani

Pratama

. Autonomous deep learning: Continual learning approach for dynamic environments. In Proceedings of the SIAM international conference on data mining. 2019; pp. 666-674.

Zhang

Zuo

Zhen

. Learning to adapt with memory for probabilistic few-shot learning. IEEE Transactions on Circuits and Systems for Video Technology. 2021; 31(11): 4283-92.

Mirjalili

Lewis

. Grey wolf optimizer. Advances in Engineering Software. 2014; 69: 46-61.

Song

Shu

Dong

Cheng

Wei

Gong

. Overcoming Catastrophic Forgetting for Multi-Label Class-Incremental Learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024; pp. 2389-2398.

Robins

. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science. 1995; 7(2): 123-46.

Kirkpatrick

Pascanu

Rabinowitz

Veness

Desjardins

Rusu

Milan

Quan

Ramalho

Grabska-Barwinska

Hassabis

. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences. 2017; 114(13): 3521-6.

Kang

Zhang

Wang

Chen

Huang

. Alleviating catastrophic forgetting of incremental object detection via within-class and between-class knowledge distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023; pp. 18894-18904.

Perkonigg

Hofmanninger

Herold

Brink

Pianykh

Prosch

Langs

. Dynamic memory to alleviate catastrophic forgetting in continual learning with medical imaging. Nature Communications. 2021; 12(1): 5678.

Kaushik

Gain

Kortylewski

Yuille

. Understanding catastrophic forgetting and remembering in continual learning with optimal relevance mapping. arXiv preprint arXiv2102.11343. 2021 Feb 22.

10.

Shao

Feng

. Overcoming catastrophic forgetting beyond continual learning: Balanced training for neural machine translation. arXiv preprint arXiv2203.03910. 2022 Mar 8.

11.

Luo

Deng

. New classes inference, few-shot learning and continual learning for radar signal recognition. IET Radar, Sonar & Navigation. 2022; 16(10): 1641-55.

12.

Liu

Yang

Peng

Zhou

. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems. 2021; 33(12): 6999-7019.

13.

Veličković

Cucurull

Casanova

Romero

Lio

Bengio

. Graph attention networks. arXiv preprint arXiv1710. 10903. 2017.

14.

Lucas

Saccucci

. Exponentially weighted moving average control schemes: properties and enhancements. Technometrics. 1990; 32(1): 1-2.

15.

Seyyedabbasi

Kiani

. Sand Cat swarm optimization: A nature-inspired algorithm to solve global optimization problems. Engineering with Computers. 2023; 39(4): 2627-51.

16.

Wang

Tan

Liu

. Particle swarm optimization algorithm: an overview. Soft Computing. 2018; 22(2): 387-408.

17.

Alshamlan

Badr

Alohali

. Genetic Bee Colony (GBC) algorithm: A new gene selection method for microarray cancer classification. Computational Biology and Chemistry. 2015; 56: 49-60.

18.

Lobato

Steffen

Jr. Fish swarm optimization algorithm applied to engineering system design. Latin American Journal of Solids and Structures. 2014; 11: 143-56.

19.

Chu

Tsai

Pan

. Cat swarm optimization. In PRICAI 2006: Trends in Artificial Intelligence: 9th Pacific Rim International Conference on Artificial Intelligence Guilin, China, August 7–11, 2006 Proceedings. 2006; 9: 854-858.

20.

Mirjalili

Lewis

. The whale optimization algorithm. Advances in Engineering Software. 2016; 95: 51-67.

21.

Uymaz

Tezel

Yel

. Artificial algae algorithm (AAA) for nonlinear global optimization. Applied Soft Computing. 2015; 31: 153-71.

22.

Deb

Fong

Tian

. Elephant search algorithm for optimization problems. In 2015 tenth international conference on digital information management (ICDIM). 2015. pp. 249-255.

23.

Chitara

Niazi

Swarnkar

Gupta

. Cuckoo search optimization algorithm for designing of a multimachine power system stabilizer. IEEE Transactions on Industry Applications. 2018; 54(4): 3056-65.

24.

Mirjalili

. Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm. Knowledge-Based Systems. 2015; 89: 228-49.

25.

CIFAR-100 Python, https://www.kaggle.com/datasets/fedesoriano/cifar100, accessed on April 2024.

26.

The CIFAR-10 dataset, https://www.cs.toronto.edu/∼kriz/cifar.html, accessed on May 2024.

Comparative analysis of optimization algorithm-based training algorithms for a few shot continual learning

Abstract

Keywords

1. .Introduction

2.1. .Challenges

3. .Few shot continual learning with optimization algorithm-based training

3.2.1. .Training algorithms

a) Solution encoding

3.2.4. .GBC

3.2.5. .FSOA

3.2.6. .CSO

3.2.8. .AAA

3.2.9. .ESA

3.2.10. .CSOA

3.2.11. .MFO

3.2.12. .GWO

4. .Results

Table 1 Hyper parameters of Algorithms. Methods Number of population Maximum number of iterations PSO 20 50 GBC 20 100 FSOA 20 100 CSO 25 100 WOA 50 100 AAA 30 100 ESA 20 100 CSOA 20 100 MFO 20 100 GWO 30 100 SE-SCSO 50 100

4.1.1. .Accuracy

4.3.1. .Dataset 1

a) Comparative evaluation (5-way 5-shot) for data set 1

b) Comparative evaluation (5-way 1-shot) for data set 1

5. .Conclusion

Footnotes

Abbreviation

ORCID iDs

References

Table 1
Hyper parameters of Algorithms.

Methods Number of population Maximum number of iterations

PSO 20 50

GBC 20 100

FSOA 20 100

CSO 25 100

WOA 50 100

AAA 30 100

ESA 20 100

CSOA 20 100

MFO 20 100

GWO 30 100

SE-SCSO 50 100