Abstract
The Gravitational Search Algorithm (GSA) has been proposed for solving continues problems based on the law of gravity. In this paper, we propose a Cognitive Discrete GSA (called CDGSA) for solving 0-1 knapsack problem. The GSA has used a function of time to determine the number of the best particles for attracting others in each time, while our main idea is based on attracting each particle with two cognitive and social components. The cognitive component contains the best position of the particles up to now, while the social component contains the particle with the best position in the whole of the system at the current time and the particles with the best position in the neighborhood. In the other words, the cognitive component is such an appropriate actuator for embedding in the intelligent agents like robots. Such intelligent agent or robot is guided in the right direction with the help of its best previous position. Finally, by introducing discrete version of this idea, the efficiency of the proposed algorithm is measured for 0-1 knapsack problem. Experimental results on some benchmark and high dimensional problems illustrate that the proposed algorithm has gained the better accuracy in comparison of the other similar methods.
Keywords
Introduction
The 0-1 knapsack problem is one of the combinatorial optimization problems. It contains a set of N items, each with a weight and a value which want to select a subset of items; so that the total weight is less than or equal to the boundary of the knapsack and the total value is as large as possible [19, 37].
Suppose, W is the boundary of the knapsack (W > 0), vectors w = (w 1, w 2, …, w N ) and v = (v 1, v 2, …, v N ) stand for the weight and the value of items, where w i > 0, v i > 0, 1 ≤ i ≤ N.
The aim is to find a vector x = (x
i
, x
2, …, x
N
), where x
i
= 0 or 1 (1 ≤ i ≤ N) which satisfies the constraints of Equations (1) and (2):
If x i = 1 then the ith item put into the knapsack; if x i = 0 then the ith item does not put into the knapsack [19, 37].
Knapsack problem has a numerous applications in theory and real world [9] such as: capital budgeting problems [12], loading problems [18], resource allocation [33] and project selection problems [17], also it can be found as a sub problem of the other more general models [7].
Many exact and approximate methods have been developed to solve knapsack problem: exact methods such as dynamic programming [4, 35], branch-and-bound approach [24]; approximate methods such as ant colony optimization [22, 31], genetic algorithm [30, 34], particle swarm optimization [15, 16], simulated annealing [3], harmony search algorithm [11], amoeboid organism algorithm [39], a schema-guiding evolutionary algorithm [40], a soccer league algorithm [32] and a shuffled frog leaping algorithm [26].
Since some new, more difficult 0-1 knapsack problems with high dimensions are hidden in the real world; in addition the available algorithms may lose their efficiency in solving them due to their constraints, therefore the research on this topic is still important.
The Gravitational Search Algorithm (GSA) is one of the meta-heuristic algorithms which has been investigated for solving continues optimization problems based on the law of gravity and mass interactions [14]. Also, it was employed in many other fields such as clustering [2], classification [1] and etc. In the GSA, all particles attract each other with the gravitational force law and all particles move toward the particles with heavier masses. While the heavy particles which correspond to good solutions move more slowly than lighter ones.
In this paper, we propose a new approach for solving 0-1 knapsack problem with introducing a Cognitive Discrete GSA (CDGSA). Our main focus is how to interactions between the particles; in other words, just two components apply the force to each particle instead of that the particle is attracted by all the other particles. Two components are obtained according to the following criteria: the cognitive component (the best position of the each particle up to now), the social component (the best particle in the system at the current time, two of the best neighbors in term of similarity and distance). Furthermore, we introduce an approach for updating the position of the particles, discretely. Experimental results show that the CDGSA is successful in solving 0-1 knapsack problem.
The paper is organized as follows. Section 2 briefly introduces the mathematical model of the GSA and a discrete version of it. Section 3 introduces a cognitive discrete version of the GSA for solving 0-1 knapsack problem. Evaluating the efficiency of the proposed algorithm on some knapsack problems is presented in Section 4. Finally, the paper is concluded by a summary in Section 5.
In this section the GSA and a discrete version of it are introduced.
Gravitational search algorithm (GSA)
The GSA has been proposed to solve the optimization problems based on the Newtonian gravity law. The GSA has been considered as an artificial system of the particles (in the literature of the GSA [14] is named the masses) which they obey from the Newtonian gravity law. In the GSA, each particle has four attributes: position, inertial mass, active gravitational mass (a measure of the strength of the gravitational field) and passive gravitational mass (measure of the strength of a particle’s interaction with the gravitational field). The position of the particle determines a solution of the problems as well as its gravitational and the inertial masses are indicated by using a fitness function (Note: the GSA assumes that the value of the gravitational and the inertial masses are equal). In each time, the heaviest particle presents an optimum solution and the other particles will be attracted byit [14].
The GSA is formulated in a system with N particles as follows:
In the GSA, the position of the ith particle is shown with Equation (3):
The details of the components of Equation (4) is as follows:
1. G (t) is the gravitational constant at time t. It is a descending function of time and an initial value (G 0).
2. M
pi
and M
aj
are the passive gravitational mass of the ith particle and the active gravitational mass of the jth particle, iteratively. The equality of the gravitational and the inertial masses has been assumed by the GSA and they are updated by the following equations:
4. R
ij
(t) is the Euclidean distance between two particles at time t and it is computed by Equation (10):
In order to achieve a random characteristic to the GSA, the total applied force on the ith particle in the dth dimension is randomly weighted sum of the dth dimension of the forces that are applied from the other particles:
To receive a good compromise between exploration and exploitation, the GSA has just selected a K
best
number of particles with bigger mass for applying their force to the others. In order to avoid of trapping in a local optimum, the algorithm must use the exploration at the beginning with a large value for K
best
. K
best
is a function of time with an initial value (K
0) that decreases with time; in other words, at the beginning all the particles attract the others, then at over time, K
best
decreases linearly and at the end there will be just one particle for applying force to the others. Therefore, Equation (11) can be rewritten as follows:
Also, the next velocity of a particle is considered as a fraction of its current velocity added to its acceleration. So, the velocity of the particle could be updated by Equation (14):
Finally, updating the particle’s position is formulated by Equation (15) as follow:
To do combinatorial optimization, a discrete or binary optimization algorithm is required. In recent years, many researchers have tried to present a discrete GSA [13, 36]. Other optimization methods in continues spaces are converted to discrete one. For example a binary Cuckoo search algorithm is introduced in [10] for feature selection which is a binary optimization problem. Some of the methods use Sigmoid function for binarization or discretization of the algorithms [27].
A quantum-inspired gravitational search algorithm is proposed in [21] for binary encoded optimization problems. Another binary version of the GSA is introduced in [13]. Since the main difference between the continues and the discrete (or binary) GSA is in the relation of the position updating, so the Discrete GSA (DGSA) has 0 or 1 values in each position’s dimension and it updates the position with switching between 0 and 1 values (or vice versa) based on the particle’s velocity. The main idea for updating particle’s position is as follows:
A large absolute value of the velocity in each dimension shows that particle’s position in that dimension is not proper; while a small absolute value of the velocity in each dimension shows that the particle’s position in that dimension is proper. It should be noted that a zero value of the velocity shows that the particle’s position must not be changed and is good.
Based on the above mentioned concepts, the DGSA (or BGSA) uses the Equation (16) (presented in the Table 1) for updating the position; where in Equation (16), rand is a random number in the interval [0,1] and C denotes the complement function.
Proposed method
The GSA and a discrete version of it were introduced in the previous section. Reducing the number of interactions between the particles with using a function of time and an initial value (using Equation (12) instead of Equation (11)) was a good way to create a compromise between exploration and exploitation. Indeed, in Equation (12) the K best number of the particles attract others instead of that all particles attract others.
In our proposed algorithm, the situation of each particle is changed by two type of components: cognitive component (the best position of the same particle that has been achieved so far) and social components (the position of the best particle in the system, the position of the best particles in term of similarity and distance). In other words, the social components have taken from the behavior of the other particles in the system while the cognitive component is like the effect of an actuator of robot or an intelligent agent that selects an appropriate path by using its previous knowledge and the current situation.
On the other hands, in the GSA the position of a particle represents a solution of the problems. The continues problems have a continues relation for updating the position of the particles based on their velocity (Equation (15)). Therefore, to introduce a discrete version of the above idea, we apply a discrete relation for updating the position.
Also in the GSA, the gravitational constant is a function of time. In this paper, we employ two proper functions for updating G value during the algorithm.
Details of the proposed method have been presented at the next subsections:
Cognitive gravitational search algorithm (CGSA)
In the GSA, the total gravitational force on the ith particle in the dth dimension is calculated according to Equation (12). As regards to perform a good compromise between exploitation and exploration, all particles do not apply the force to others and just the K
best
numbers of the best particles just apply the force on the other particles in each time. K
best
is a function of time with an initial value (K
0), so it is considered as the parameter that directly affects on the solutions. Therefore, to increase the speed of the algorithm convergence, we consider four forces apply on the each particle in each time. Although in the GSA only the masses and the gravitational forces between particles are considered for optimization, in our proposed algorithm we consider the particles so that they can memorize their best situation of their self and their neighbors. These particles can be robots with memory and actuators or particles with embedded intelligent software agents. These forces are applied from the cognitive and the social components:
Social components:
The particle with the best position in the whole of the system (G
best
) The particle with the best position in the neighborhood of the each particle, in term of: similarity (S
best
) distance (D
best
)
In the following, the more detailed description of the above concepts will be mentioned.
Cognitive component of the ith particle determines the best position of it until the current time, according to Equation (17) (presented in Table 1); where in Equation (17), X ij represents the position of the ith particle at specific time j and fit i (t) represents the fitness value of the ith particle at time t.
Also, detecting a particle with the best position at the current time in the system can be calculated by Equation (18) (presented in Table 1).
In order to determine the best particle in the neighborhood of each particle, the size of the neighborhood (s N ) is considered as an essential parameter. In our experiments we investigate random sizes for the neighborhood.
Therefore, the necessary equation for finding a particle with the best position in the neighborhood of the ith particle (in term of the similarity) is considered as Equation (19) (presented in Table 1); where in the Equation (19), s N is the size of neighborhood around the ith particle and D denotes the dimension size.
In the literature of this paper, the meaning of the similarity neighborhood is as follows: the j th particle is a similar neighbor of the i th particle when their position is similar in the most of the dimensions.
As well as, a particle with the best position based on the distance in the neighborhood of the ith particle is expressed by Equation (20) (presented in Table 1); where in the Equation (20), ∥X i (t) , X j (t) ∥ 2 determines the Euclidean distance between two particles.
Finally, based on the above comments, Equation (12) can be rewritten as Equation (21) (presented in Table 1).
As a final point, if the applied forces lead to decline the position of the particle in term of fitness then these forces will be ineffective and the particle maintains its situations (such as position, velocity, acceleration and mass). Otherwise, the particle moves toward the better position.
Cognitive Discrete GSA (CDGSA) for solving 0-1 knapsack problem
In the GSA, the positions of the particles represent the solutions of the problem. Based on its representation (Equation (3)), the value of the particle’s position in each dimension () is dependent to the nature of the problem (continues or discrete).
So, to generalize to the discrete case, the following points should be noted: The position of the particles in each dimension should be included 0 or 1 values. The relation for updating the position should be discrete (switching 0 to 1 or vice versa). A large absolute value of the velocity determines a high probability of changing the position of the particle from 0 to 1 or vice versa. A small absolute value of the velocity determines a low probability of changing the position. It should be noted that a zero value of the velocity indicates that the position must not be changed.
Also, based on the concepts of the GSA, the following notes are essential to update the position:
Therefore, based on the above points, we employ a function (arc tan(x)) to update the position (Equation (22) instead of Equation (15)) which it illustrates in the Fig. 1 and satisfies all of the above mentioned requirements. Equation (22) is presented in Table 1; where in Equation (22), α is a random number in the interval [0, π /2] and C denotes the complement function.
Updating gravitational constant
The gravitational constant (G) is a function of the time and an initial value (G 0). At the beginning, G has large value then it will be reduced to control the search accuracy. Therefore, we employ two following functions (exponential, linear) for updating the gravitational constant in our method as:
and
The schema of Equation (23) and (24) are illustrated in Fig. 2.
At this point, we propose a fitness function in order to evaluate the population. We know that the 0-1 knapsack problem has a constraint that the weight of the selected items must be less than or equal to the boundary of the knapsack, while the total value of them must be as large as possible. Therefore, we employ Equation (25) (presented in Table 1) for evaluating the population. It calculates the fitness value of the ith particle at time t; where in Equation (25), D denotes the dimension size and α is named a penalty coefficient with a large value to guarantee that the fitness value of the best infeasible solution is lower than of that the worst feasible solution. We noted that in the GSA, the position of the particles represents the solutions.
At the end, the block diagram of the proposed algorithm for solving 0-1 knapsack problem is shown in Fig. 3.
Experimental results
In this section, the performance of the CDGSA is studied by different experiments. All the computational experiments are conducted with Matlab R2010b in Intel ® CoreTM2 Duo CPU T6500 @ 2.10 GHz with 4 GB RAM system. At first, we consider the ten small 0-1 knapsack problems which their details (contain item’s weight w, item’s value v, knapsack’s boundary W and optimum solution) have been presented in Table 2.
It should be noted that many authors are employed these problems for evaluating their method. Problems P 1 and P 2 are solved by an improved ant colony algorithm [22] while it does not received the optimum solution of them. Problem P 3 is used with a sequential combination tree algorithm [6] and problem P 4 is examined by a greedy-policy-based algorithm [38]. Problem P 5 is analyzed with the information of search-space landscape to search the optimum solution [23] and problem P 6 is solved in a manner similar to the shrinking boundary method [8]. Problems P 7 and P 8 are considered with a nonlinear dimensionality reduction method [25]. Problem P 9 is used by a DNA algorithm [41] and in the end problem P 10 is adopted from the literature [5]. It should be noted that problems P i where i = 3, 4, …, 10, have reached to optimum solution in the mentioned paper.
As a first test, the performance of the three algorithms, BGSA [13], CDGSA and DGSA (the proposed approach, but lacks the cognitive component) on the ten test samples (in Table 2) are reported in Table 3. The optimum solution and the best, worst, average and median solution, the standard deviation (std.dev) and average total time (avg.time) during 20 independent runs are presented in Table 3. In the literature of these experiments, the solution is the total value of the selected items in the knapsack; for example, when we say the best solution, it means the total value of the selected items for the particle with the best position. In all the experiments in Table 3, the initial parameters are as follows: maximum iteration number (Max_iteration) is 200, size of the population (pop) is 50 and initial gravitational constant (G 0) is set to 1000.
From Table 3, it is clear that the CDGSA has a
faster convergence for the large problems, P i where
i = 1, 2, 5, 6, 8, 10, in the comparison of the BGSA and DGSA. It is due to reduce the number of interactions between particles based on the two components. While the BGSA has sufficient to local optimum solution for many problems (such as P i , i = 2, 5, 8, 10). Also the DGSA has achieved a similar behavior in many problems (such as P i , i = 2, 8, 10).
Also, we execute the CDGSA on the other eight knapsack problems to evaluate its performance on the high dimension problems. In these problems, the number of items (N) is as 100, 200, 300, 500, 800, 1000, 1200 and 1500, respectively; while corresponding knapsack’s boundary is according to 1100, 1500, 1700, 2000, 5000, 10000, 14000 and 16000, respectively. Other parameters (the weight of items (w), the value of items (v)) are randomly generated as:
v
i
(i = 1, …, N) ∼ rand (50, 100)
w
i
(i = 1, …, N) ∼ rand (5, 20)
All results of this experiment have been gained during 10 independent runs. The best, worst, average and median solution and the standard deviation (std.dev) are presented in Table 4.
From Table 4, it is clear that the proposed method (CDGSA) in comparison of the DGSA (CDGSA without the cognitive component) has gained better results to solve the high dimensional 0-1 knapsack problems. This argument is obtained based on the best, worst, average and median solutions and standard deviation.
Also, we offered two different functions for updating the gravitational constant (G). Therefore, we have selected two medium and large problems (P 2 from Table 3 and P 11 from Table 4) for evaluating the performance of the CDGSA with Equations (23) and (24). We have reported the convergence diagram of this experiment in Fig. 4.
From Fig. 4, it seems that the using Equation (24) (linear relation) for updating the gravitational constant is quickly converged to the optimum solution. While using Equation (23) for updating the gravitational constant provide later convergence.
As a final point, we can conclude that the CDGSA with Equation (24) for updating the gravitational constant has gained good results for solving 0-1 knapsack problems with medium and large size. One of the most manifest features of the CDGSA is that it reaches to optimum solution of the problems in an appropriate time. The remarkable thing is that the performance of the CDGSA improve with the increasing in the dimensions of the problem due to its cognitive and social components. These components reduce the interaction between the particles, also they cause to decrease the total time of solving problems.
Conclusion
0-1 knapsack problem is an algorithm of NP-hard problems. To this end a number of algorithms have been developed for solving it. A Gravitational Search algorithm (GSA) is an optimization algorithm based on the law of gravity and mass interactions. In this paper, we introduced a Cognitive Gravitational Search Algorithm (CGSA). We employed a cognitive and social components for each particle in the system. Our original idea was to allocate a memory to each particle to maintain his former best situation. Also, we defined a social component that contains the particles with the best position in the whole of the system and in the neighborhood. Finally, by proposing a discrete version of the our idea (called CDGSA), we tried to solve 0-1 knapsack problem. Experimental results show the efficiency and the accuracy of the proposed method to solve 0-1 knapsack problem.
Footnotes
Acknowledgments
This work has been supported by a Grant from the University of Tehran (No. 29999/01/01).
