Comparison of particle swarm optimization variants with fuzzy dynamic parameter adaptation for modular granular neural networks for human recognition

Abstract

In this paper dynamic parameter adjustment in particle swarm optimization (PSO) for modular neural network (MNN) design using granular computing and fuzzy logic (FL) is proposed. Nowadays, there are a plethora of optimization techniques, but their implementations require having knowledge about these techniques in order to establish their parameters, because the performance and final results of a particular technique depend on the optimal parameter values. For this reason, in this paper the fuzzy adjustment of parameters during the execution is proposed, and this proposal allows to adjust the parameters depending on current PSO behavior in each iteration. The proposed method performs modular neural network optimization applied to human recognition using benchmark ear, iris and face databases. Two fuzzy inference systems are proposed to perform this dynamic adjustment, comparisons against a PSO without this dynamic adjustment (simple PSO) are performed to verify if the proposed adjustment using a fuzzy system is better improving recognition rate and execution time. The PSO variants presented in this paper are aimed at performing MNNs optimization. This optimization consists on finding optimal parameters, such as: the number of modules (or sub granules), percentage of data for the training phase, learning algorithm, goal error, number of hidden layers and their number of neurons.

Keywords

Modular neural networks granular computing particle swarm optimization fuzzy adaptation human recognition ear recognition iris recognition face recognition pattern recognition

1 Introduction

Optimization techniques have provided a lot of advantages in solving real world problems; finding optimal parameters, architectures or solutions. These techniques have emerged from several years ago, for example: genetic algorithm (GA) [21, 31], ant colony system (ACO) [15] and particle swarm optimization (PSO) [16], as they are pioneering optimization methods that successfully demonstrated their advantages in areas of application. Recently other methods have also emerged based on different behaviors of nature, such as the firefly algorithm (FA) [56], grey wolf optimizer (GWO) [35], gravitational search algorithm (GSA) [40] or Cuckoo Optimization Algorithm (COA) [39] just to mention a few. All optimization techniques have parameters and of these depend on the final performance of the algorithm, some parameters can be easily adjusted using prior knowledge, but that can take a long time and we do not know in advance if they are the best parameters for a certain application or area.

The optimization methods mentioned above have been combined with other important intelligent techniques creating hybrid intelligence systems, such as artificial neural networks (ANNs) [23, 24], fuzzy logic (FL) [58], granular computing (GrC) [57, 59] and robotics [14], expert system (ES) [4] among others [3 , 26]. There are quite a number of relevant works, where hybrid methods have been proposed, some are mentioned below. In [5], a new alpha level set optimization approach using PSO is applied, where hybrid random variables are simulated using fuzzy numbers. A hybrid modeling approach using response surface model (RSM) and support vector regression (SVR) is proposed in [30] and its results are compared with conventional techniques such as ANN, RSM and SVR are used to predict shear capacity of steel fiber-reinforced concrete beams (SFRCB). In [29], a fuzzy conjugate relaxed-finite step size method (fuzzy CRS) to improve the instability of the fuzzy first-order reliability is proposed. The method consists of two analyzer loops; the inner loop is established using the first-order reliability method (FORM) based conjugate search direction and relaxed approach and the outer loop is constructed using the genetic operator.

Artificial neural networks have been applied to human recognition, classification problems or time series prediction and their architectures have been improved using different optimization techniques [27], but conventional artificial neural networks have certain limitations to learn a large amount of information, for this reason, modular neural networks (MNNs) emerge to cover this limitation by creating experts modules which learn specific subtasks [47, 48]. Granular computing allows defining granules as sub granules or subsets allowing building computational models when a large amount of information is used [7]. Other intelligence technique capable to perform a human mind imitation is fuzzy logic, this technique employs modes of reasoning that are approximate rather than exact. A fuzzy inference system (FIS) has three components: fuzzy if-then rules, membership functions and an inference procedure performed by a reasoning mechanism [2].

In the literature, there are many works where of PSO and FL have been used, for example in image restoration [8], time series prediction [20, 34], control problems [6, 55], and other benchmark problems [42]. Fuzzy dynamic adaptation was already proposed and successfully applied to mathematical functions [37 , 54], control problems [51, 52] and classification problems [33, 38]. In this paper, modular granular neural networks, fuzzy logic, granular computing and particle swarm optimization are combined to create a hybrid intelligence system applied to human recognition, performing a dynamic adjustment of PSO parameters using a fuzzy inference system to prove the effectiveness of this kind of adjustment combined with the intelligence techniques above mentioned. This work aims at achieving a better recognition rate, but also seeks to compare execution times against PSO variants and with other algorithms. The optimization techniques applied to human recognition usually have a slower convergence than when they are applied to other applications. Therefore, this work also aims at finding which technique allows a faster convergence.

This paper is organized as follows. A background is presented in Section 2. The proposed fuzzy parameter adaptation is described in detail in Section 3. The results obtained using the PSO variants are presented and explained in Section 4. Statistical comparisons of PSO results are presented in Section 5. Finally, in Section 6, the conclusions and future work are shown.

2 Related works

In this section some related works will be explained.

2.1 Modular granular neural networks

This type of artificial neural network and its optimization using a hierarchical genetic algorithm (HGA) were proposed in [46], and in that work the granulation of a database is proposed, where a main granule represents a whole database that contains images of persons. This granule can be divided into sub granules of different sizes that represent images of a number of persons. Each sub granule is divided into images sets for training and testing. Each image is divided into 3 parts, in the case of images for training each part is learned by a sub module. The set of images for testing is simulated in each sub module, and the responses of the sub modules are combined using “the winner takes all” method. The number of sub granules and sub modules architectures are optimized using an optimization technique. In Fig. 1, the granulation and optimization of a database using a MGNN is illustrated.

Fig. 1

The general architecture of the MGNN granulation and optimization.

2.2 Particle swarm optimization

Particle swarm optimization was proposed in [16, 28] by R. C. Eberhart and J. Kennedy. This optimization technique is based on the social behaviors of fish schooling or birds flocking. A swarm contains particles, where each particle represents a solution and its next position in the search space is determined by Equation (1):

$x_{id} (t + 1) = x_{i} (t) + v_{i} (t + 1)$ (1) where x _id (t) denotes a current position of the particle i, dimension d, at time t. To calculate the next position a velocity v _i (t + 1) is assigned.

In this work, the star topology is used. In this variant, the neighborhood of each particle is the entire swarm. The PSO authors were improving the original algorithm, for this reason in this paper, the PSO with inertia weight (w) [17, 50] is used. A big inertia weight allows a global exploration to search new areas and a small inertia weight allows a local exploration, its value decreases linearly, this allows the exploration when PSO starts and in later iterations allows local explorations (exploitation). The particle velocity is determined by the Equation (2):

$\begin{matrix} v_{id} (t + 1) = & {wv}_{id} {(t) + c}_{1} r_{1 d} {(t) * [y}_{id} (t) - x_{id} (t)] \\ + c_{2} r_{2 d} {(t) * [\hat{y}}_{d} {(t) - x}_{id} (t)] \end{matrix}$ (2) where r₁ and r₂ are random values from 0 to 1, w is the inertia weight with a decrease value. Cognitive and social components are represents by c₁ and c₂ (their values are fixed during all the execution), the best position of a particle i in dimension d (pBest) is represented by y_id (t) and the best position of the swarm (gBest) in d dimension is denoted by $\hat{y}$ _d (t). In Fig. 2, the pseudo code of the PSO is presented.

Fig. 2

Pseudo code of the particle swarm optimization.

3 Proposed method

In this section the proposed PSO with fuzzy parameter adaptation and its application is described. The optimization technique with its fuzzy dynamic parameter adaptation designs the modular granular neural networks architectures applied to human recognition. In Fig. 3, the granulation and optimization of a database using a PSO variant is illustrated.

Fig. 3

The general architecture of the MGNN granulation and optimization with a PSO variant.

3.1 Description of the particle swarm optimization with fuzzy dynamic parameter adaptation

In this work, two fuzzy inference systems are proposed to compare which of them is better to find the parameters of the PSO. Both fuzzy inference systems have 2 input variables and 3 output variables. All variables use 3 triangular membership functions and their linguistic labels are respectively “Low”, “Medium” and “High”. Figure 4 shows the structure of the fuzzy inference systems.

Fig. 4

Fuzzy Inference System for dynamic adaptation.

The inputs variables are:

Iteration: This input variable gets the value of an internal variable that saves during PSO execution how many iterations the best particle’s objective function (see Equation (3)) remains its value without changing. This variable is restarted when its value achieved to 5 or 10 (depending on the fuzzy inference system).

CurrentW: This input variable gets the current value of w used in the PSO.

The outputs variables are:

C₁: The output value of this variable allows updating C₁, this new value will be used by the PSO.

C₂: The output value of this variable allows updating C₂, this new value will be used by the PSO.

UpdatedW: The output value of this variable allows updating w, this new value will be used by the PSO.

The fuzzy if-then rules and type of the membership functions were obtained by experimental knowledge. The 9 fuzzy if-then rules used in the fuzzy inference systems are shown in Fig. 5. These fuzzy if-then rules are formulated with a goal; a balance among the output values c₁, c₂ and the update of w, this means that, if the value of the update of w is small, c₁ and c₂ will have high values and vice versa. For example, if the value of iterations is high and the current w is low, then the w update is allowed to remain low but c₁ and c₂ should be high. The ranges of the variables in each FIS are presented in Table 1. These ranges were obtained based on values suggested by specialists [18], ranges proposed by authors of this kind of adaptations [38, 52] and “trial and error”.

Fig. 5

Fuzzy Inference System to dynamic adaptation.

Table 1

Ranges of variables

Variable	Range
	FIS #1	FIS #2
Iteration	1 to 10	1 to 5
ActualW	0.2 to 1.2	0.1 to 1
C₁	1 to 2	0.5 to 2
C₂	1 to 2	0.5 to 2
UpdatedW	0.2 to 1.2	0.1 to 1

The objective function is to minimize the error of recognition and is given by Equation (3):

$f = \sum_{i = 1}^{m} ((\sum_{j = 1}^{n_{m}} X_{j}) / n_{m})$ (3) where m is the total number of modules, X_j is 0 if the module provides the correct result and 1 if not, and n _m is total number of data/images used for testing phase in the corresponding module.

Each dimension of the particle represents a parameter of the MGNN to be optimized. The total number of parameters (dimensions) is calculated by:

$Dimensions = 2 + (3 * m) + (m * h)$ (4) where m and h are respectively the maximum number of sub granules and hidden layers per sub module that the PSO can use to perform the optimization. In Fig. 6, the diagram of the proposed method is illustrated and in Fig. 7, the structure of the particle is presented, where the dimensions of each particle are shown, for example, first dimension represents the number of modules, the second dimension represents the percentage of data for training, the next dimensions block represents all the parameters of the first sub module, the next block represents all the parameters of the second sub module, and so on until block “m”.

Fig. 6

Diagram of the proposed method.

Fig. 7

Particle dimensions.

3.2 Proposed method applied to human recognition

In this work, multi-layer feed-forward neural networks are used. As learning method a supervised learning algorithm was selected, which is the backpropagation algorithm. In this algorithm, the input signal is propagated through the neural network layer-by-layer, and then an error is calculated using the inputs and the desired outputs, the synapses weights are modified, repeating these steps until convergence [44]. Due to its ability to handle large learning problems, it has been used in a wide research area, such as human recognition [43]. To perform the modular granular neural networks training, three backpropagation algorithms used in previous works were selected [46, 48].

This selection was performed because their performances are faster and their achieved results are better than other backpropagation algorithms:

radient descent with scaled conjugate gradient (SCG)

Gradient descent with adaptive learning and momentum (GDX)

Gradient descent with adaptive learning (GDA).

The search space was established using the minimum and maximum values also used in [48]. This particle swarm optimization has two stopping conditions: when the maximum number of iterations is achieved or when the objective function has a value equal to zero.

The initial parameters of the PSO algorithm are shown in Table 2. It is important to remember that these values (C₁, C₂ and w) are only the initial ones, during the execution of the algorithm the values will be continuously updated, but to compare results, a PSO without fuzzy dynamic adaptation is used (in this work called “simple PSO”).

Table 2
Initial parameters of the PSO

Parameter Value

Particles (n) 10

Maximum Iterations 30

C₁ 2

C₂ 2

w 0.8

Parameter	Value
Particles (n)	10
Maximum Iterations	30
C₁	2
C₂	2
w	0.8

3.2.1 Databases and pre-processing

The description and the pre-processing applied to the databases used are presented below.

Ear database. The ear database is from the University of Science & Technology Beijing (USTB) and contains 77 persons, where each person has 4 images of one ear. The image dimensions are 300 x 400, with BMP format [10]. A sample of the images of a person is shown in Fig. 8. The pre-processing for each image of this database is: the image is manually cut, the image is resized to 132 x 91 pixels and automatically the image is divided into three regions of interest (helix, shell and lobe) [22, 46]. In Fig. 9, the pre-processing process is shown.

Fig. 8

Sample of the Ear Database from USTB.

Fig. 9

Sample pre-processing for ear database.

Iris database. The iris database is from the Institute of Automation of the Chinese Academy of Sciences (CASIA) [11] and contains 77 persons, where each person has 14 images (7 for each eye). Each image has a dimension of 320×280, JPEG format. Figure 10 shows a sample of the images of this database.

Fig. 10

Sample of the Iris database from CASIA.

The pre-processing for this database was proposed by Masek and Kovesi [32]. This is used to obtain coordinates and radius of iris and pupil to perform a cut in the iris, the image is resized to 21×21 pixels and finally, each image is automatically divided into three parts. In Fig. 11, the pre-processing is shown.

Fig. 11

Sample pre-processing for iris database.

Face database. The face database is from the AT&T Laboratories in Cambridge [12] and contains 40 persons, where each person has 10 images. The image dimensions are 92×112 pixels, PGM format. Figure 12 shows a sample of the images of this database.

Fig. 12

Sample of the ORL database.

As pre-processing process each image is automatically divided into three regions of interest (front, eyes and mouth). Figure 13 shows the pre-processing process.

Fig. 13

Sample pre-processing for ORL database.

Selection method. As it is known, the artificial neural networks have a learning phase, and in this phase they learn information. Usually, when databases for human recognition are used, a 70 or 80 percent of the images of each person are used for the learning phase, and the rest of images are used to test the artificial neural network (testing phase). In [46], another method to perform the selection process was proposed, where the percentage of data for training can vary (and be optimized) and the images for each phase are randomly selected unlike the conventional method where the percentage is always fixed and the same images are always selected for each phase. In this work, the percentage of images for training phase is optimized by the particle swarm optimization. In Fig. 14, the selection methods are illustrated.

Fig. 14

Selection methods.

4 Optimization results

The results achieved by different variants of PSO (simple PSO, PSO using the FIS#1 and PSO using the FIS#2) applied to human recognition are presented in this section. For the ear database, 30 runs are performed using up to 80 and 50 percent of the data. For the iris database, 20 runs are performed using up to 80 and 50 percent of the data and for the ORL Database, 5 runs were performed using respectively 4 and 5 images for training, but only results using 4 images are shown in Section 4.3. Finally, a summary of results of all databases is shown in Section 4.4.

4.1 Ear results

The results achieved using the ear database are presented below. Only the best result of each PSO variant is shown.

4.1.1 Results using a percentage of data for training up to 80%

In this test, each PSO variant can use up to 80% of data for the training phase. In Table 3 the best result of each variant is shown. It can be observed that the 3 variants achieve the same result, but the PSO with the FIS#2 designed a MGNN with less number of modules.

Table 3
The best results (up to 80%, Ear)

Method Images Num. Hidden layers and Num. of neurons Persons per module Rec. Rate Error

Training Testing

PSO 77% 23% 3(24,57,235) Module #1(1 to 10) 100% 0

(1,2 and 3) (4) 1(95) Module #2(11 to 17)

3(138,196,137) Module #3(18 to 31)

3(98,147,200) Module #4(32 to 41)

2(150,247) Module #5(42 to 56)

2(45,200) Module #6(57 to 59)

1(133) Module #7(60 to 66)

5(108,99,99,237,44) Module #8(67 to 77)

PSO+FIS1 79% 21% 1(96) Module #1(1 to 9) 100% 0

(1,2 and 3) (4) 1(109) Module #2(10 to 32)

1(243) Module #3(33 to 48)

1(112) Module #4(49 to 53)

1(139) Module #5(54 to 55)

1(194) Module #6(56 to 77)

PSO+FIS2 76% 24% 4(239,106,150,140) Module #1(1 to 24) 100% 0

(2,3 and 4) (1) 1(75) Module #2(25 to 60)

4(121,250,197,173) Module #3(61 to 77)

Method	Images	Num. Hidden layers and Num. of neurons	Persons per module	Rec. Rate	Error
PSO	77%	23%	3(24,57,235)	Module #1(1 to 10)	100%	0
	(1,2 and 3)	(4)	1(95)	Module #2(11 to 17)
			3(138,196,137)	Module #3(18 to 31)
			3(98,147,200)	Module #4(32 to 41)
			2(150,247)	Module #5(42 to 56)
			2(45,200)	Module #6(57 to 59)
			1(133)	Module #7(60 to 66)
			5(108,99,99,237,44)	Module #8(67 to 77)
PSO+FIS1	79%	21%	1(96)	Module #1(1 to 9)	100%	0
	(1,2 and 3)	(4)	1(109)	Module #2(10 to 32)
			1(243)	Module #3(33 to 48)
			1(112)	Module #4(49 to 53)
			1(139)	Module #5(54 to 55)
			1(194)	Module #6(56 to 77)
PSO+FIS2	76%	24%	4(239,106,150,140)	Module #1(1 to 24)	100%	0
	(2,3 and 4)	(1)	1(75)	Module #2(25 to 60)
			4(121,250,197,173)	Module #3(61 to 77)

The behavior of run #1 of the simple PSO is presented in Fig. 15. This run was one of the fastest runs to obtain an error value equal to zero. In Fig. 16, the average of convergence of the 30 runs of each PSO variant is shown. Although in the first iterations the simple PSO showed less error, in the end, the other variants of PSO had a better behavior.

Fig. 15

Convergence of run #1 of the simple PSO.

Fig. 16

Average of convergence (Test #1, Ear).

4.1.2 Results using a percentage of data for training up to 50%

In this test, each PSO variant can use up to 50% of data for the training phase. In Table 4 the best result of each variant is shown. It can be observed that the variant of PSO with the FIS#1 achieves the best result and also uses less number of modules. The behavior of run #20 of the PSO with the FIS#1 is illustrated in Fig. 17.

Table 4
The best results (up to 50%, Ear)

Method Images Num. Hidden layers and Num. of neurons Persons per module Rec. Rate Error

Training Testing

PSO 42% 58% 1(125) Module #1(1 to 10) 96.75% 0.0325

Run #2 (2 and 3) (1 and 4) 1(232) Module #2(11 to 22)

1(232) Module #3(23 to 31)

1(183) Module #4(32 to 38)

1(116) Module #5(39 to 53)

1(173) Module #6(54 to 66)

1(127) Module #7(67 to 72)

1(77) Module #8(73 to 75)

1(133) Module #9(76 to 77)

PSO+FIS1 49% 51% 1(26) Module #1(1 to 11) 97.40% 0.0260

Run #20 (2 and 3) (1 and 4) 1(43) Module #2(12 to 29)

1(106) Module #3(30 to 46)

1(170) Module #4(47 to 54)

1(44) Module #5(55 to 66)

1(244) Module #6(67 to 77)

PSO+FIS2 47% 53% 1(175) Module #1(1 to 7) 96.75% 0.0325

Run #9 (2 and 3) (1 and 4) 1(209) Module #2(8 to 17)

1(152) Module #3(18 to 23)

1(212) Module #4(24 to 40)

1(142) Module #5(41 to 52)

1(153) Module #6(53 to 62)

1(145) Module #7(63 to 64)

1(223) Module #8(65 to 77)

Method	Images	Num. Hidden layers and Num. of neurons	Persons per module	Rec. Rate	Error
PSO	42%	58%	1(125)	Module #1(1 to 10)	96.75%	0.0325
Run #2	(2 and 3)	(1 and 4)	1(232)	Module #2(11 to 22)
			1(232)	Module #3(23 to 31)
			1(183)	Module #4(32 to 38)
			1(116)	Module #5(39 to 53)
			1(173)	Module #6(54 to 66)
			1(127)	Module #7(67 to 72)
			1(77)	Module #8(73 to 75)
			1(133)	Module #9(76 to 77)
PSO+FIS1	49%	51%	1(26)	Module #1(1 to 11)	97.40%	0.0260
Run #20	(2 and 3)	(1 and 4)	1(43)	Module #2(12 to 29)
			1(106)	Module #3(30 to 46)
			1(170)	Module #4(47 to 54)
			1(44)	Module #5(55 to 66)
			1(244)	Module #6(67 to 77)
PSO+FIS2	47%	53%	1(175)	Module #1(1 to 7)	96.75%	0.0325
Run #9	(2 and 3)	(1 and 4)	1(209)	Module #2(8 to 17)
			1(152)	Module #3(18 to 23)
			1(212)	Module #4(24 to 40)
			1(142)	Module #5(41 to 52)
			1(153)	Module #6(53 to 62)
			1(145)	Module #7(63 to 64)
			1(223)	Module #8(65 to 77)

Fig. 17

Convergence of run #20 of PSO+FIS1.

In Fig. 18, the average of convergence of the 30 runs of each PSO variant is presented. For this test, the PSO with the FIS#2 had a best behavior.

Fig. 18

Average of convergence (Test #2, Ear).

4.2 Iris results

The results achieved using the iris database are presented below. Only the best result of each PSO variant is presented.

4.2.1 Results using a percentage of data for training up to 80%

In this test, each PSO variant can also use up to 80% of data for the training phase. In Table 5, the best result of each variant is shown. It can be observed that the simple PSO and the PSO with the FIS#2 achieve an error of recognition equal to zero with almost the same number of modules.

Table 5
The best results (up to 80%, Iris)

Method Images Num. Hidden layers and Num. of neurons Persons per module Rec. Rate Error

Training Testing

PSO 76% 24% (123) Module #1(1 to 5) 100% 0

Run #15 (1,2,3,4, (7,10 and 11) (130,230,143) Module #2(6 to 17)

5,6,8,9, (95,161) Module #3(18 to 26)

12,13 and (140) Module #4(27 to 37)

14) (210,22,32) Module #5(38 to 40)

(201,54,110,96) Module #6(41 to 53)

(151,230,99) Module #7(54 to 70)

(210,26,67,193) Module #8(71 to 75)

(151,209,210) Module #9(76 to 77)

PSO+FIS1 74% 26% (174,141,235,113) Module #1(1 to 6) 99.68% 0.0032

Run #13 (1,2,3,4, (7,10,11 and 13) (116,200) Module #2(7 to 9)

5,6,8,9, (44,249,90,186,94) Module #3(10 to 15)

12 and 14) (99,36) Module #4(16 to 22)

(182,245) Module #5(23 to 36)

(99,180) Module #6(37 to 52)

(247,95,30,96) Module #7(53 to 62)

(26,47,112,28) Module #8(63 to 68)

(131,142,180,198,159) Module #9(69 to 77)

PSO+FIS2 78% 22% (237) Module #1(1 to 3) 100% 0

Run #11 (1,2,3,4, (7,10 and 12) (99,167) Module #2(4 to 5)

5,6,8,9, (129,161,191) Module #3(6 to 15)

11,13 and 14) (176,218) Module #4(16 to 31)

(70,164,131) Module #5(32 to 46)

(80,130,56) Module #6(47 to 59)

(68,119) Module #7(60 to 66)

(90,190,113,171) Module #8(67 to 77)

Method	Images	Num. Hidden layers and Num. of neurons	Persons per module	Rec. Rate	Error
PSO	76%	24%	(123)	Module #1(1 to 5)	100%	0
Run #15	(1,2,3,4,	(7,10 and 11)	(130,230,143)	Module #2(6 to 17)
	5,6,8,9,		(95,161)	Module #3(18 to 26)
	12,13 and		(140)	Module #4(27 to 37)
	14)		(210,22,32)	Module #5(38 to 40)
			(201,54,110,96)	Module #6(41 to 53)
			(151,230,99)	Module #7(54 to 70)
			(210,26,67,193)	Module #8(71 to 75)
			(151,209,210)	Module #9(76 to 77)
PSO+FIS1	74%	26%	(174,141,235,113)	Module #1(1 to 6)	99.68%	0.0032
Run #13	(1,2,3,4,	(7,10,11 and 13)	(116,200)	Module #2(7 to 9)
	5,6,8,9,		(44,249,90,186,94)	Module #3(10 to 15)
	12 and 14)		(99,36)	Module #4(16 to 22)
			(182,245)	Module #5(23 to 36)
			(99,180)	Module #6(37 to 52)
			(247,95,30,96)	Module #7(53 to 62)
			(26,47,112,28)	Module #8(63 to 68)
			(131,142,180,198,159)	Module #9(69 to 77)
PSO+FIS2	78%	22%	(237)	Module #1(1 to 3)	100%	0
Run #11	(1,2,3,4,	(7,10 and 12)	(99,167)	Module #2(4 to 5)
	5,6,8,9,		(129,161,191)	Module #3(6 to 15)
	11,13 and 14)		(176,218)	Module #4(16 to 31)
			(70,164,131)	Module #5(32 to 46)
			(80,130,56)	Module #6(47 to 59)
			(68,119)	Module #7(60 to 66)
			(90,190,113,171)	Module #8(67 to 77)

The behavior of run #11 of the PSO with the FIS#2 is presented in Fig. 19. This run achieved an error value equal to zero in the iteration number 11.

Fig. 19

Convergence of run #11 of PSO+FIS2.

In Fig. 20, the average of convergence of the 20 runs of each PSO variant is presented. For this test, the PSO with the FIS#2 also had a best behavior than the other variants.

Fig. 20

Average of convergence (Test #1, Iris).

4.2.2 Results using a percentage of data for training up to 50%

In this test for the iris database, each PSO variant can use up to 50% of data for the training phase. In Table 6 the best result of each variant is shown. It can be observed that the PSO with the FIS#1 and the PSO with the FIS#2 achieve the same error of recognition with almost the same number of modules. The behavior of run #3 of the PSO with the FIS#2 is presented in Fig. 21. This run obtained a 97.59 % of recognition rate, and error value equal to 0.0241. In Fig. 22, the average of convergence of the 20 runs of each PSO variant is shown. For this test, the PSO with the FIS#2 had also a best behavior than the other variants.

Table 6
The best results (up to 50%, Iris)

Method Images Num. Hidden layers and Num. of neurons Persons per module Rec. Rate Error

Training Testing

PSO 49% 51% 1(130) Module #1(1 to 17) 97.40% 0.0260

Run #13 (1,3,5,6 (2,4,7,8, 1(30) Module #2(18 to 23)

9,13 and 10,11 1(76) Module #3(24 to 26)

14) And 12) 1(87) Module #4(27 to 37)

1(38) Module #5(38 to 40)

1(182) Module #6(41 to 49)

1(83) Module #7(50 to 51)

1(51) Module #8(52 to 59)

1(231) Module #9(60 to 70)

1(102) Module #10(71 to 77)

PSO+FIS1 48% 52% 1(171) Module #1(1 to 13) 97.59% 0.0241

Run #5 (3,4,5,6, (1,7,8,9, 1(212) Module #2(14 to 25)

13 and 14) 10,11 and 1(202) Module #3(26 to 33)

12) 1(237) Module #4(34 to 42)

1(249) Module #5(43 to 50)

1(110) Module #6(51 to 53)

1(57) Module #7(54 to 64)

1(245) Module #8(65 to 71)

1(78) Module #9(72 to 77)

PSO+FIS2 49% 51% 4(102,238,118,86) Module #1(1 to 3) 97.59% 0.0241

Run #3 (1,2,3,4, (7,8,9,10, 4(217,97,164,140) Module #2(4 to 15)

5, 6 and 11,12 and 2(158,173) Module #3(16 to 23)

14) 13) 2(188,144) Module #4(24 to 30)

1(36) Module #5(31 to 36)

1(193) Module #6(37 to 56)

2(155,163) Module #7(57 to 61)

5(127,174,143,76,142) Module #8(62 to 66)

1(234) Module #9(67 to 68)

2(136,167) Module #10(69 to 77)

Method	Images	Num. Hidden layers and Num. of neurons	Persons per module	Rec. Rate	Error
PSO	49%	51%	1(130)	Module #1(1 to 17)	97.40%	0.0260
Run #13	(1,3,5,6	(2,4,7,8,	1(30)	Module #2(18 to 23)
	9,13 and	10,11	1(76)	Module #3(24 to 26)
	14)	And 12)	1(87)	Module #4(27 to 37)
			1(38)	Module #5(38 to 40)
			1(182)	Module #6(41 to 49)
			1(83)	Module #7(50 to 51)
			1(51)	Module #8(52 to 59)
			1(231)	Module #9(60 to 70)
			1(102)	Module #10(71 to 77)
PSO+FIS1	48%	52%	1(171)	Module #1(1 to 13)	97.59%	0.0241
Run #5	(3,4,5,6,	(1,7,8,9,	1(212)	Module #2(14 to 25)
	13 and 14)	10,11 and	1(202)	Module #3(26 to 33)
		12)	1(237)	Module #4(34 to 42)
			1(249)	Module #5(43 to 50)
			1(110)	Module #6(51 to 53)
			1(57)	Module #7(54 to 64)
			1(245)	Module #8(65 to 71)
			1(78)	Module #9(72 to 77)
PSO+FIS2	49%	51%	4(102,238,118,86)	Module #1(1 to 3)	97.59%	0.0241
Run #3	(1,2,3,4,	(7,8,9,10,	4(217,97,164,140)	Module #2(4 to 15)
	5, 6 and	11,12 and	2(158,173)	Module #3(16 to 23)
	14)	13)	2(188,144)	Module #4(24 to 30)
			1(36)	Module #5(31 to 36)
			1(193)	Module #6(37 to 56)
			2(155,163)	Module #7(57 to 61)
			5(127,174,143,76,142)	Module #8(62 to 66)
			1(234)	Module #9(67 to 68)
			2(136,167)	Module #10(69 to 77)

Fig. 21

Convergence of run #3 of PSO+FIS2.

Fig. 22

Average of convergence (Test #2, Iris).

4.3 Face recognition results

To test the effectiveness of the proposed method, a well-known benchmark database is used: the ORL database. The results of this database will be directly compared with other authors. The results achieved using this database, using 4 images for the training phase, are presented below. Only the best result of each PSO variant is shown. In Table 7, the best result of each variant is presented. It can be observed that the PSO with the FIS#1 and the PSO with the FIS#2 achieve an error of recognition equal to zero with the same number of modules. The behavior of run #1 of the PSO with the FIS#2 is presented in Fig. 23. This run achieved an error value equal to zero in the iteration number 13. In Fig. 24, the average of convergence of the 5 runs of each PSO variant is illustrated. The PSO with the FIS#2 had a faster convergence than the other variants.

Table 7
The best results (4 images, ORL Database)

Method Images Num. Hidden layers and Num. of neurons Persons per module Rec. Rate Error

Training Testing

PSO 40% 60% 5(86,147,185,197,161) Module #1(1 to 9) 99.58% 0.0042

Run #1 (3,5,8 and 9) (1,2,4,6,7 and 10) 1(149) Module #2(10 to 11)

2(69,217) Module #3(12 to 20)

5(230,211,191,58,229) Module #4(21 to 22)

1(120) Module #5(23 to 25)

4(149,204,182,195) Module #6(26 to 27)

3(156,68,228) Module #7(28 to 29)

5(186,125,237,41,187) Module #8(30 to 33)

5(221,217,235,218,135) Module #9(34 to 35)

5(166,27,87,224,30) Module #10(36 to 40)

PSO+FIS1 40% 60% 4(38,116,156,182) Module #1(1 to 3) 100% 0

Run #1 (1,2,4 and 8) (3,5,6,7,9 and 10) 1(195) Module #2(4 to 8)

2(68,109) Module #3(9 to 10)

1(80) Module #4(11 to 17)

2(39,182) Module #5(18 to 25)

4(134,240,203,123) Module #6(26 to 27)

3(180,160,102) Module #7(28 to 31)

5(81,72,175,131,96) Module #8(32 to 36)

4(203,196,161,61) Module #9(37 to 38)

4(119,136,30,216) Module #10(39 to 40)

PSO+FIS2 40% 60% 3(122,160,210) Module #1(1 to 3) 100% 0

Run #1 (6,8,9 and 10) (1,2,3,4,5 and 7) 1(153) Module #2(4 to 10)

2(139,217) Module #3(11 to 12)

5(232,210,182,50,117) Module #4(13 to 15)

1(146) Module #5(16 to 21)

4(128,211,165,196) Module #6(22 to 26)

3(172,86,81) Module #7(27 to 29)

5(187,116,68,67,188) Module #8(30 to 36)

5(37,227,236,220,131) Module #9(37 to 38)

5(175,30,249,223,48) Module #10(39 to 40)

Method	Images	Num. Hidden layers and Num. of neurons	Persons per module	Rec. Rate	Error
PSO	40%	60%	5(86,147,185,197,161)	Module #1(1 to 9)	99.58%	0.0042
Run #1	(3,5,8 and 9)	(1,2,4,6,7 and 10)	1(149)	Module #2(10 to 11)
			2(69,217)	Module #3(12 to 20)
			5(230,211,191,58,229)	Module #4(21 to 22)
			1(120)	Module #5(23 to 25)
			4(149,204,182,195)	Module #6(26 to 27)
			3(156,68,228)	Module #7(28 to 29)
			5(186,125,237,41,187)	Module #8(30 to 33)
			5(221,217,235,218,135)	Module #9(34 to 35)
			5(166,27,87,224,30)	Module #10(36 to 40)
PSO+FIS1	40%	60%	4(38,116,156,182)	Module #1(1 to 3)	100%	0
Run #1	(1,2,4 and 8)	(3,5,6,7,9 and 10)	1(195)	Module #2(4 to 8)
			2(68,109)	Module #3(9 to 10)
			1(80)	Module #4(11 to 17)
			2(39,182)	Module #5(18 to 25)
			4(134,240,203,123)	Module #6(26 to 27)
			3(180,160,102)	Module #7(28 to 31)
			5(81,72,175,131,96)	Module #8(32 to 36)
			4(203,196,161,61)	Module #9(37 to 38)
			4(119,136,30,216)	Module #10(39 to 40)
PSO+FIS2	40%	60%	3(122,160,210)	Module #1(1 to 3)	100%	0
Run #1	(6,8,9 and 10)	(1,2,3,4,5 and 7)	1(153)	Module #2(4 to 10)
			2(139,217)	Module #3(11 to 12)
			5(232,210,182,50,117)	Module #4(13 to 15)
			1(146)	Module #5(16 to 21)
			4(128,211,165,196)	Module #6(22 to 26)
			3(172,86,81)	Module #7(27 to 29)
			5(187,116,68,67,188)	Module #8(30 to 36)
			5(37,227,236,220,131)	Module #9(37 to 38)
			5(175,30,249,223,48)	Module #10(39 to 40)

Fig. 23

Convergence of run #1 of PSO+FIS2.

Fig. 24

Average of convergence (4 images, ORL database).

4.4 Summary results

A summary of results is shown below and also comparisons with respect to other works using the ear, iris and face databases are presented. All the optimization techniques shown in this Section performed the same number of function evaluations. The summary of results obtained with the PSO and the two fuzzy adaptation methods for the ear database is shown in Table 8.

Table 8
The summary of results (Ear Database)

Method Number of images for training Recognition Rate

Best Average Worst

PSO Up to 80% 100% 100% 100%

0 0 0

PSO+FIS1 100% 100% 100%

0 0 0

PSO+FIS2 100% 100% 100%

0 0 0

PSO Up to 50% 96.75% 94.48% 92.20%

0.0325 0.0552 0.0779

PSO+FIS1 97.40% 94.74% 92.21%

0.0260 0.0526 0.0779

PSO+FIS2 96.75% 95.17% 93.51%

0.0325 0.0483 0.0649

Method	Number of images for training	Recognition Rate
PSO	Up to 80%	100%	100%	100%
		0	0	0
PSO+FIS1		100%	100%	100%
		0	0	0
PSO+FIS2		100%	100%	100%
		0	0	0
PSO	Up to 50%	96.75%	94.48%	92.20%
		0.0325	0.0552	0.0779
PSO+FIS1		97.40%	94.74%	92.21%
		0.0260	0.0526	0.0779
PSO+FIS2		96.75%	95.17%	93.51%
		0.0325	0.0483	0.0649

In some of our previous works, the modular granular neural networks have been optimized using different optimization techniques such as a HGA [46], a FA [48] and a GWO [47]. Also, other authors have used the same database, in [22] modular neural networks with two different integration methods (Winner takes all and Sugeno Integration) were performed. In [41], ear feature extraction is proposed and an artificial neural network for classification is used, also in [36] artificial neural networks are used, but the feature extraction is performed using eigenvectors.

In Table 9, the comparison between the non-optimized and optimized results is shown. When up to 80% of images are used, all PSO variants and GWO obtained an average of 100 recognition rate. When up to 50% of images are used, the best result is obtained by HGA, but the best average is obtained by the FA, this one achieved better results than any of the PSO variants.

Table 9

Table of comparison of results (Ear Database)

Method	Number of images for training	Recognition Rate
		Best	Average	Worst
		(%)	(%)	(%)
Mohd Rahim M.S. [36] (Eigenvector+ANN)	Up to 80%	93.33%	–	–
USTB [41]		85%	–	–
Gutierrez L. [22] (WTA integration)		97.40%	94.48%	–
Gutierrez L. [22] (Sugeno integration)		100%	97.42%	–
S $\overset{´}{a}$ nchez D. [46] (HGA)		100%	99.69%	93.5%
S $\overset{´}{a}$ nchez D. [48] (FA)		100%	99.89%	98.05%
S $\overset{´}{a}$ nchez D. [47] (GWO)		100%	100%	100%
PSO		100%	100%	100%
		0	0	0
PSO+FIS #1			100%	100%	100%
		0	0	0
PSO+FIS #2		100%	100%	100%
S $\overset{´}{a}$ nchez D. [46] (HGA)	Up to 50%	98.05%	94.81%	79.65%
S $\overset{´}{a}$ nchez D. [48] (FA)		97.40%	96.82%	95.45%
S $\overset{´}{a}$ nchez D. [47] (GWO)		96.75%	96.15%	95.45%
PSO		96.75%	94.48%	92.20%
		0.0325	0.0552	0.0779
PSO+FIS1		97.40%	94.74%	92.21%
		0.0260	0.0526	0.0779
PSO+FIS2		96.75%	95.17%	93.51%
		0.0325	0.0483	0.0649

The summary of results obtained with the PSO and the two fuzzy adaptation methods to the iris database are shown in Table 10. Also for iris database some of our previous works used other optimization techniques such as a HGA [45], a FA [48] and a GWO [47]. For this database, other authors have also presented results. In [19], contour segmentation of the iris was performed and train modular neural networks using 99 persons (1386 images). In [13], Daugman used 756 images of 108 persons, it means less number of images for training and testing, using different kinds of techniques for features extraction.

Table 10

The summary of results (Iris Database)

Method	Number of images for training	Recognition rate
		Best	Average	Worst
PSO	Up to 80%	100%	98.99%	98.70%
		0	0.0101	0.0130
PSO+FIS1		99.68%	99.03%	98.70%
		0.0032	0.0097	0.0130
PSO+FIS2		100%	99.31%	98.70%
		0	0.0069	0.0130
PSO	Up to 50%	97.40%	95.97%	95.36%
		0.0260	0.4029	0.0464
PSO+FIS1		97.59%	96.38%	95.45%
		0.0241	0.0362	0.0455
PSO+FIS2		97.59%	96.79%	96.10%
		0.0241	0.0321	0.0390

In Table 11, the comparison of results for the iris database among the PSO variants and the other works is presented. When up to 80% of images are used, PSO+FIS #2 and GWO obtained the best average of recognition. In the case that 50% of images are used, the best result is obtained by HGA, but its worst recognition rate (94.64%) is lower than the PSO variants. The best average is obtained by the PSO+FIS #2 variant.

Table 11

Table of comparison of results (Iris Database)

Method	Number of images for training	Recognition rate
		Best	Average	Worst
		(%)	(%)	(%)
Daugman J. [13]	Up to 80%	99.90%	–	–
Gaxiola F. [19]		97.13%	91.83%	–
S $\overset{´}{a}$ nchez D. [45] (HGA)		99.68%	98.68%	97.40%
S $\overset{´}{a}$ nchez D. [48] (FA)		99.13%	98.22%	96.59%
S $\overset{´}{a}$ nchez D. [47] (GWO)		100%	99.31%	98.70%
PSO		100%	98.99%	98.70%
		0	0.0101	0.0130
PSO+FIS#1		99.68%	99.03%	98.70%
		0.0032	0.0097	0.0130
PSO+FIS#2		100%	99.31%	98.70%
		0	0.0069	0.0130
S $\overset{´}{a}$ nchez D. [45] (HGA)	Up to 50%	97.77%	96.48%	94.64%
S $\overset{´}{a}$ nchez D. [48] (FA)		–	–	–
S $\overset{´}{a}$ nchez D. [47] (GWO)		–	–	–
PSO		97.40%	95.97%	95.36%
		0.0260	0.4029	0.0464
PSO+FIS#1		97.59%	96.38%	95.45%
		0.0241	0.0362	0.0455
PSO+FIS#2		97.59%	96.79%	96.10%
		0.0241	0.0321	0.0390

In Table 12, the comparison for the ORL database among the PSO variants results and the other works is presented. In [1], an adaptive technique for obtaining centers of the hidden layer neurons of a radial basis function neural network (RBFNN) for face recognition using FA is proposed. In [25], a GWO with Linear Collaborative Discriminant Regression Classification is proposed. In [49], feature selection using GA for face recognition based on principal component analysis (PCA), Wavelet and Support vector machine (SVM) is presented. In [47] and [48], modular granular neural networks are optimized using respectively GWO and FA. When 5 images for the training phase are used, any PSO version is better than the other works, and when 4 images for the training phase, the PSO+FIS #2 has better performance.

Table 12

The summary of results (ORL Database)

Method	Number of images for training	Recognition rate
		Best	Average	Worst
S $\overset{´}{a}$ nchez D. [48] (FA)	5	99%	98.30%	98%
S $\overset{´}{a}$ nchez D. [47] (GWO)		99%	98.5%	98%
Hosgurmath S. [25] (GWO)		–	95.11%	–
Agarwal V. [1] (FA)		97.35%	96.96%	96.30%
PSO		100%	100%	100%
		0	0	0
PSO+FIS1		100%	100%	100%
		0	0	0
PSO+FIS2		100%	100%	100%
		0	0	0
Satone M. [49] (GA)	4	97.5%	96.33%	95.80%
Hosgurmath S. [25] (GWO)		–	93.55%	–
PSO		99.58%	99.58%	99.58%
		0.0042	0.0042	0.0042
PSO+FIS1		100%	99.75%	99.58%
		0	0.0025	0.0042
PSO+FIS2		100%	100%	100%
		0	0	0

4.4.1 Comparisons of execution time

Comparisons of execution times are presented in this section. In Table 13, averages of execution times are presented. In this table can be observed, for the ear, a faster convergence of the PSO+FIS#2 when up to 80% of data is used (achieved a 100% recognition rate). When 50% is used, also PSO+FIS#2 is faster than the others, but it is important to remember that FA achieved better results. For the iris, PSO+FIS#2 was faster than other algorithms when up to 80% are used, and when 50% is used the faster algorithm was PSO+FIS#1. For the ORL database, PSO+FIS#2 was faster than the PSO+FIS#1 and the simple PSO.

Table 13
Execution times of the optimization techniques (Average)

Method Ear Iris Face

80% 50% 80% 50% 4 ima-ges

S $\overset{´}{a}$ nchez D. [46] (HGA) 5 : 34 : 02 25 : 36 : 21 5 : 41 : 53 1 : 58 : 43 –

S $\overset{´}{a}$ nchez D. [48] (FA) 2 : 31 : 58 22 : 38 : 06 5 : 26 : 39 – –

S $\overset{´}{a}$ nchez D. [47] (GWO) 2 : 24 : 04 19 : 35 : 19 5 : 39 : 20 – –

PSO 3 : 15 : 52 18 : 57 : 12 5 : 32 : 10 2 : 11 : 23 17 : 12 : 12

PSO+FIS1 1 : 50 : 31 18 : 44 : 02 5 : 35 : 30 1 : 52 : 06 13 : 53 : 22

PSO+FIS2 1 : 37 : 31 18 : 43 : 17 5 : 25 : 49 2 : 01 : 36 8 : 20 : 22

Method	Ear	Iris	Face
S $\overset{´}{a}$ nchez D. [46] (HGA)	5 : 34 : 02	25 : 36 : 21	5 : 41 : 53	1 : 58 : 43	–
S $\overset{´}{a}$ nchez D. [48] (FA)	2 : 31 : 58	22 : 38 : 06	5 : 26 : 39	–	–
S $\overset{´}{a}$ nchez D. [47] (GWO)	2 : 24 : 04	19 : 35 : 19	5 : 39 : 20	–	–
PSO	3 : 15 : 52	18 : 57 : 12	5 : 32 : 10	2 : 11 : 23	17 : 12 : 12
PSO+FIS1	1 : 50 : 31	18 : 44 : 02	5 : 35 : 30	1 : 52 : 06	13 : 53 : 22
PSO+FIS2	1 : 37 : 31	18 : 43 : 17	5 : 25 : 49	2 : 01 : 36	8 : 20 : 22

In Figs. 25 and 26, the execution times of each run for the ear are shown respectively when up to 80% and 50% of data are used. In Figs. 27 and 28, the execution times of each run for the iris are respectively shown when up to 80% and 50% of data are used.

Fig. 25

Execution time (Ear, 80%).

Fig. 26

Execution time (Ear, 50%).

Fig. 27

Execution time (Iris, 80%).

Fig. 28

Execution time (Iris, 50%).

In Fig. 29, the execution times of each run for the face database are respectively shown when 4 images for the training phase are used.

Fig. 29

Execution time (ORL database, 4 images).

5 Statistical comparison

To verify which PSO variant is better, statistical t tests were performed to verify if there is sufficient statistical evidence to prove it. In this section the statistical tests for ear, iris and face database are shown.

5.1 Results for ear database

When the ear database is used with up to 80% of data for training, all PSO variants achieved the same results, for this reason t-tests are omitted when 80% is used. In Table 14, the t-values for the ear database using up to 50% for training phase are shown. As results show, with a t-value of 2.69, there is sufficient evidence to say that only PSO with the FIS#2 significantly improve results unlike the simple PSO. When the proposed method is compared with other optimization techniques such as: FA and GWO, these algorithms had a better recognition rate, although, any variant of PSO had a better execution time.

Table 14
Values of the ear database (up to 50%)

Method N Mean Standard deviation Error standard deviation of the mean Estimated difference t value P value Degrees of freedom

PSO 30 0.0552 0.0117 0.0021 0.00260 – 0.90 0.370 57

PSO+FIS1 30 0.0526 0.0106 0.0019

PSO 30 0.0552 0.0117 0.0021 0.00693 – 2.69 0.009 51

PSO+FIS2 30 0.0483 0.00794 0.0014

PSO+FIS1 30 0.0526 0.0106 0.0019 0.00433 – 1.79 0.0783 53

PSO+FIS2 30 0.0483 0.00794 0.0014

PSO+FIS2 30 0.04827 0.00794 0.0014 0.00354 – 0.55 0.588 32

HGA [45] 30 0.0518 0.0345 0.0063

PSO+FIS2 30 0.04827 0.00794 0.0014 0.01645 – 9.81 6.0052E-13 46

FA [48] 30 0.03182 0.00462 0.00084

PSO+FIS2 30 0.04827 0.00794 0.0014 0.00974 – 5.85 4.8863E-07 45

GWO [47] 30 0.03853 0.00449 0.00082

Method	N	Mean	Standard deviation	Error standard deviation of the mean	Estimated difference	t value	P value	Degrees of freedom
PSO	30	0.0552	0.0117	0.0021	0.00260	– 0.90	0.370	57
PSO+FIS1	30	0.0526	0.0106	0.0019
PSO	30	0.0552	0.0117	0.0021	0.00693	– 2.69	0.009	51
PSO+FIS2	30	0.0483	0.00794	0.0014
PSO+FIS1	30	0.0526	0.0106	0.0019	0.00433	– 1.79	0.0783	53
PSO+FIS2	30	0.0483	0.00794	0.0014
PSO+FIS2	30	0.04827	0.00794	0.0014	0.00354	– 0.55	0.588	32
HGA [45]	30	0.0518	0.0345	0.0063
PSO+FIS2	30	0.04827	0.00794	0.0014	0.01645	– 9.81	6.0052E-13	46
FA [48]	30	0.03182	0.00462	0.00084
PSO+FIS2	30	0.04827	0.00794	0.0014	0.00974	– 5.85	4.8863E-07	45
GWO [47]	30	0.03853	0.00449	0.00082

5.2 Results for iris database

In Table 15, the t-values for the iris database using up to 80% for training phase are shown. As results show, with t-values of 2.82 and 2.59, there is sufficient evidence to say that PSO with the FIS #2 significantly improves results unlike the simple PSO and PSO with the FIS#1. Comparing with other algorithms, PSO+FIS#2 achieves significant improvements on the results obtained by HGA and FA with t-values of 3.30 and 5.83, respectively.

Table 15
Values of the iris database (up to 80%)

Method N Mean Standard deviation Error standard deviation of the mean Estimated difference t value P value Degrees of freedom

PSO 20 98.987 0.381 0.085 – 0.039 – 0.34 0.738 37

PSO+FIS1 20 99.026 0.349 0.078

PSO 20 98.987 0.381 0.085 – 0.326 – 2.82 0.008 37

PSO+FIS2 20 99.313 0.350 0.078

PSO+FIS1 20 99.026 0.349 0.078 – 0.287 – 2.59 0.013 38

PSO+FIS2 20 99.313 0.350 0.078

PSO+FIS2 20 99.313 0.350 0.078 – 0.630 – 3.30 0.003 26

HGA [45] 20 98.683 0.779 0.17

PSO+FIS2 20 99.313 0.350 0.078 – 1.088 – 5.83 3.3528E-06 26

FA [48] 20 98.225 0.758 0.17

PSO+FIS2 20 99.313 0.350 0.078 – 0.005 – 0.05 0.964 37

GWO [47] 20 99.307 0.407 0.091

Method	N	Mean	Standard deviation	Error standard deviation of the mean	Estimated difference	t value	P value	Degrees of freedom
PSO	20	98.987	0.381	0.085	– 0.039	– 0.34	0.738	37
PSO+FIS1	20	99.026	0.349	0.078
PSO	20	98.987	0.381	0.085	– 0.326	– 2.82	0.008	37
PSO+FIS2	20	99.313	0.350	0.078
PSO+FIS1	20	99.026	0.349	0.078	– 0.287	– 2.59	0.013	38
PSO+FIS2	20	99.313	0.350	0.078
PSO+FIS2	20	99.313	0.350	0.078	– 0.630	– 3.30	0.003	26
HGA [45]	20	98.683	0.779	0.17
PSO+FIS2	20	99.313	0.350	0.078	– 1.088	– 5.83	3.3528E-06	26
FA [48]	20	98.225	0.758	0.17
PSO+FIS2	20	99.313	0.350	0.078	– 0.005	– 0.05	0.964	37
GWO [47]	20	99.307	0.407	0.091

In Table 16, the t-values for the iris database using up to 50% for training phase are shown. As results show, there is sufficient evidence to say that PSO with the FIS#1 and PSO with the FIS#2 significantly improve results unlike the simple PSO. When both adjustments are compared, the PSO with FIS#2 is better. Comparing with the HGA, the PSO+FIS#2 achieves a better recognition rate, but there is not enough evidence to prove it.

Table 16

Values of the iris database (up to 50%)

Method	N	Mean	Standard deviation	Error standard deviation of the mean	Estimated difference	t value	P value	Degrees of freedom
PSO	20	95.971	0.535	0.12	– 0.410	– 2.35	0.024	37
PSO+FIS1	20	96.381	0.571	0.13
PSO	20	95.971	0.535	0.12	– 0.819	– 5.54	3.09E-06	34
PSO+FIS2	20	96.789	0.387	0.087
PSO+FIS1	20	96.381	0.571	0.13	– 0.408	– 2.65	0.012	33
PSO+FIS2	20	96.789	0.387	0.087
PSO+FIS2	20	96.789	0.387	0.087	– 0.305	– 1.37	0.183	25
HGA [45]	20	96.484	0.919	0.21

5.3 Results for face database

In Table 17, the t-values for the face database using 4 images are shown. PSO+FIS#2 results are used to compare because this variant had better results and a faster convergence. As results show, with t-values of 9.12 and 12.87, there is sufficient evidence to say that PSO with the FIS #2 significantly improve results unlike the GA and FA proposed by other authors.

Table 17
Values of the ORL database

Method N Images Mean Standard deviation Error standard deviation of the mean Estimated difference t value P value Degrees of freedom

PSO+FIS2 5 4 100 0.00447 0.0020 3.673 09.12 0.0028 3

Satone M. [49] (GA) 4 4 96.33 0.806 0.40

PSO+FIS2 5 5 100 0.00447 0.0020 3.190 12.87 5.0401E-05 5

Agarwal V. [1] (FA) 6 5 96.80 0.607 0.25

Method	N	Images	Mean	Standard deviation	Error standard deviation of the mean	Estimated difference	t value	P value	Degrees of freedom
PSO+FIS2	5	4	100	0.00447	0.0020	3.673	09.12	0.0028	3
Satone M. [49] (GA)	4	4	96.33	0.806	0.40
PSO+FIS2	5	5	100	0.00447	0.0020	3.190	12.87	5.0401E-05	5
Agarwal V. [1] (FA)	6	5	96.80	0.607	0.25

5.4 Wilcoxon signed-rank tests

Wilcoxon signed-rank tests were also performed for ear and iris database. The critical values are shown in Table 18, where the different values of α are shown depending of the statistical significance and number of samples.

Table 18
Critical values

N α

0.01 0.02 0.05 0.10

20 037 043 052 060

30 109 120 137 152

N	α
20	037	043	052	060
30	109	120	137	152

In Tables 19 and 20, the results of the Wilcoxon tests for the ear and iris databases are respectively shown. To compare the results achieved with 5% level of significance, the result in the column named “ W ” must be less than or equal to the column named “ W₀ ” to reject the null hypothesis.

Table 19

Wilcoxon test results (Ear)

	Method	Negative sum	Positive sum	Test statist	DF (m)	W₀ =Wα,m
		(W-)	(W+)	(W)
Ear 50%
	PSO PSO + FIS#1	168	241	168	30	137
	PSO PSO + FIS#2	101	323	101	30	137
	PSO+FIS#1 PSO + FIS#2	120	303	120	30	137
	PSO+FIS#2 HGA [45]	144	297	144	30	137
	PSO+FIS#2 FA [48]	0	410	0	30	137
	PSO+FIS#2 GWO [47]	25	358	25	30	137

Table 20

Wilcoxon test results (Iris)

	Method	Negative sum	Positive sum	Test statist	DF (m)	W₀ =Wα,m
		(W-)	(W+)	(W)
Iris 80%
	PSO PSO + FIS#1	070	096	70	20	52
	PSO PSO + FIS#2	033	147	33	20	52
	PSO+FIS#1 PSO + FIS#2	024	151	24	20	52
	PSO+FIS#2 HGA [45]	180	025	25	20	52
	PSO+FIS#2 FA [48]	209	000	00	20	52
	PSO+FIS#2 GWO [47]	086	088	86	20	52
Iris 50%
	PSO PSO + FIS#1	053	152	53	20	52
	PSO PSO + FIS#2	008	197	08	20	52
	PSO+FIS#1 PSO + FIS#2	031	175	31	20	52
	PSO+FIS#2 HGA [45]	130	077	77	20	60

When PSO and PSO+FIS#1 are compared, we fail to reject the null hypothesis for all the tests. This means that the best PSO variant is PSO+FIS #2. For ear database, we only fail to reject the null hypothesis when results are compared with HGA.

For the iris database, we only fail to reject the null hypothesis when results are compared with GWO using up to 80% of data and with HGA using up to 50% of data.

6 Conclusions

In this paper, a fuzzy dynamic adaptation approach is proposed to dynamically establish the PSO parameters, and this optimization technique is applied in designing modular granular networks architectures applied to human recognition seeking to minimize the recognition error. The proposed adaptation aims at dynamically tuning parameters of PSO during its execution, and this allows to improve its performance and to find better parameters depending on current behavior in each iteration. To perform a comparison, two fuzzy inference systems were proposed, and different runs with each fuzzy inference system were performed and comparisons with a simple PSO were made to evaluate which PSO variant is better. The proposed method is applied to human recognition using the ear, iris and face as biometric measurements, where different PSO variants design MGNNs architectures using up to 80 and 50 percent of the data for the training phase. This design implies parameters, such as the number of modules (sub granules), percentage of data for the training phase, goal error, learning algorithm, number of hidden layers and their respective number of neurons. Based on the results achieved by the PSO variants, better results were obtained when the fuzzy dynamic adaptation is implemented. If we compare the results of the proposed method against other optimization techniques, such as HGA, FA or GWO, the PSO results achieved the same or better results (except for ear database using up to 50 percent of data for training). If execution times are compared, PSO with dynamic parameter adjustment had a faster convergence than simple PSO and other optimization techniques. As a conclusion, this optimization technique can improve its performance if the parameters are correctly established depending on its behavior. As future works other fuzzy inference systems will be proposed using other variables, in order to increase the difference between this and the other methods, also a comparison among architectures will be performed in order to reduce their complexity.

References

Agarwal

and Bhanot

, “Firefly inspired feature selection for face recognition”, Contemporary Computing (IC3), 2015 Eighth International Conference, 2015, pp. 257–262.

Ajiatmo

and Robandi

, “A hybrid Fuzzy Logic Controller-Firefly Algorithm (FLC-FA) based for MPPT Photovoltaic (PV) system in solar car”, 2016 IEEE International Conference on Power and Renewable Energy (ICPRE), 2016, pp. 606–610.

Alpaydin

, Machine Learning: The New AI: The MIT Press, 2016.

Atis

and Ekren

, “Development of an outdoor lighting control system using expert system”, Energy and Buildings 130 (2016), 773–786.

Bagheri

, Miri

and Shabakhty

, “Fuzzy reliability analysis using a new alpha level set optimization approach based on particle swarm optimization”, Journal of Intelligent & Fuzzy Systems 30(1) (2016), 235–244.

Bingül

and Karahan

, “A Fuzzy Logic Controller tuned with PSO for 2 DOF robot trajectory control”, Expert Systems with Applications 38 (2011), 1017–1031.

Butenkova

, Zhukova

, Nagorovb

and Krivshac

, “Granular Computing Models and Methods Based on the Spatial Granulation”, Procedia Computer Science 103 (2017), 295–302.

Chang

B.-M.

, Tsai

H.-H.

and Shih

J.-S.

, “Using fuzzy logic and particle swarm optimization to design a decision-based filter for cDNA microarray image restoration”, Engineering Applications of Artificial Intelligence 36 (2014), 12–26.

Costea

, “Applying Fuzzy Logic and Machine Learning Techniques in Financial Performance Predictions”, Procedia Economics and Finance 10 (2014), 4–9.

10.

Database Ear Recognition Laboratory from the University of Science & Technology Beijing (USTB). [Online]. http://www.ustb.edu.cn/resb/en/index.htm

11.

Database of Face. AT&T Laboratories Cambridge, the ORL database of faces. [Online]. https://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html

12.

Database of Human Iris. Institute of Automation of Chinese Academy of Sciences (CASIA). [Online]. http://www.cbsr.ia.ac.cn/english/IrisDatabase.asp

13.

Daugman

, “Statistical Richness of Visual Phase Information: Update on Recognizing Persons by Iris Patterns”, International Journal of Computer Vision 45(1) (2001), 25–38.

14.

Dirican

, “The Impacts of Robotics, Artificial Intelligence On Business and Economics”,– , Social and Behavioral Sciences 195 (2015), 564–573.

15.

Dorigo

, “Optimization, learning and natural algorithms”, Politecnico di Milano, Italy, PhD Thesis, 1992.

16.

Eberhart

R.C.

and Kennedy

, “A New Optimizer using Particle Swarm”, Sixth International Symposium on Micro Machine and Human Science, 1995, pp. 39–43.

17.

Eberhart

R.C.

and Shi

, “Comparing Inertia Weights and Constriction Factors in Particle Swarm Optimization”, In Proceedings of the IEEE Congress on Evolutionary Computation 1 (2000), 84–88.

18.

Engelbrecht

, Fundamentals of Computational Swarm Intelligence: Wiley, 2005.

19.

Gaxiola

, Melin

and Lopez

M.A.

, “Modular Neural Networks for Person Recognition Using the Contour Segmentation of the Human Iris Biometric Measurement”, Soft Computing for Recognition Based on Biometrics 2010 (2010), 137–153.

20.

Gaxiola

, Melin

, Valdez

, Castro

J.R.

and Castillo

, “Optimization of type-2 fuzzy weights in backpropagation learning for neural networks using GAs and PSO”, Applied Soft Computing 38 (2016), 860–871.

21.

Goldberg

D.E.

, Genetic Algorithms in Search Optimization and Machine Learning: Addison-Wesley, 1989.

22.

Gutierrez

, Melin

and Lopez

M.A.

, “Modular neural network integrator for human recognition from ear images”, IJCNN, Barcelona, Spain, 2010, pp. 1–5.

23.

Hassoun

, Fundamentals of Artificial Neural Networks: A Bradford Book, 2003.

24.

Haykin

, Neural Networks:AComprehensive Foundation: Macmillan Coll Div, 1994.

25.

Hosgurmath

and Mallappa

V.V.

, “Grey Wolf Optimizer with Linear Collaborative Discriminant Regression Classification based Face Recognition”, International Journal of Intelligent Engineering and Systems 12(2) (2019), 202–210.

26.

Houssami

, Lee

C.I.

, Buist

D.S.M.

and Tao

, “Artificial intelligence for breast cancer screening: Opportunity or hype?”, The Breast 36 (2017), 31–33.

27.

Kaviani

and MirRokni

S.M.

, “Applying Genetic Algorithm in Architecture and Neural Network Training”, IJCSNS International Journal of Computer Science and Network Security 17(6) (2017), 118–124.

28.

Kennedy

and Eberhart

R.C.

, “Particle Swarm Optimization”, In Proceedings of the IEEE international Joint Conference on Neuronal Networks (1995), pp. 1942–1948.

29.

Keshtegar

and Bagheri

, “Fuzzy relaxed-finite step size method to enhance the instability of the fuzzy first-order reliability method using conjugate discrete map”, Nonlinear Dynamics 91(3) (2018), 1443–1459.

30.

Keshtegar

, Bagheri

and Yaseen

Z.M.

, “Shear strength of steel fiber-unconfined reinforced concrete beam simulation: Application of novel intelligent model”, Composite Structures 212 (2019), 230–242.

31.

Man

K.F.

, Tang

K.S.

and Kwong

, Genetic Algorithms: Concepts and Designs, Springer, Ed., 1999.

32.

Masek

and Kovesi

, “MATLAB Source Code for a Biometric Identification System Based on Iris Patterns”, The School of Computer Science and Software Engineering, The University of Western Australia, 2003.

33.

Melin

, Olivas

, Castillo

, Valdez

, Soria

and Valdez

, “Optimal design of fuzzy classification systems using PSO with dynamic parameter adaptation through fuzzy logic”, Expert Systems with Applications 40(1) (2013), 3196–3206.

34.

Melin

, Pulido

and Castillo

, “Ensemble Neural Network with Type-1 and Type-2 Fuzzy Integration for Time Series Prediction and Its Optimization with PSO”, Imprecision and Uncertainty in Information Representation and Processing, 2016, pp. 375–388.

35.

Mirjalili

, Mirjalili

S.M.

and Lewis

, “Grey Wolf Optimizer”, Advances in Engineering Software 69 (2014), 46–61.

36.

Mohd Rahim

M.S.

, Rehman

, Kurniawan

and Saba

, “Ear biometrics for human classification based on region features mining”, Biomedical Research 28(10) (2017), 4660–4664.

37.

Nobile

M.S.

, Cazzaniga

, Besozzi

, Colombo

, Mauri

and Pasi

, “Fuzzy Self-Tuning PSO: A settings-free algorithm for global optimization”, Swarm and Evolutionary Computation 39 (2018), 70–85.

38.

Olivas

, Valdez

and Castillo

, “Fuzzy Classification System Design Using PSO with Dynamic Parameter Adaptation Through Fuzzy Logic”, Fuzzy Logic Augmentation of Nature-Inspired Optimization Metaheuristics: Springer, 2014, pp. 29–47.

39.

Rajabioun

, “Cuckoo Optimization Algorithm”, Applied Soft Computing journal 11 (2011), 5508–5518.

40.

Rashedi

, Nezamabadi-Pour

and Saryazdi

, “GSA: a gravitational search algorithm”, Information sciences 179 (2009), 2232–2248.

41.

Research on Ear Recognition Laboratory from the University of Science & Technology Beijing (USTB). [Online]. http://www1.ustb.edu.cn/resb/en/subject/subject.htm

42.

Robati

, Barani

G.A.

, Abadi Pour

H.N.

, Fadaee

M.J.

and Pour Anaraki

J.R.

, “Balanced fuzzy particle swarm optimization”, Applied Mathematical Modelling 36 (2012), 2169–2177.

43.

Rojas

, Neural Networks: Springer-Verlag, 1996.

44.

Rumelhart

D.E.

, Hinton

G.E.

and Williams

R.J.

, “Learning representations by back-propagating errors”, Nature 323 (1986), 533–536.

45.

Sánchez

and Melin

, Hierarchical Modular Granular Neural Networks with Fuzzy Aggregation, 1st ed.: Springer, 2016.

46.

Sánchez

and Melin

, “Optimization of modular granular neural networks using hierarchical genetic algorithms for human recognition using the ear biometric measure”, Engineering Applications of Artificial Intelligence 27 (2014), 41–56.

47.

Sánchez

, Melin

and Castillo

, “A Grey Wolf Optimizer for Modular Granular Neural Networks for Human Recognition”, 2017 (2017), 4180510:1–4180510:26.

48.

Sánchez

, Melin

and Castillo

, “Optimization of modular granular neural networks using a firefly algorithm for human recognition”, Engineering Applications of Artificial Intelligence 64 (2017), 172–186.

49.

Satone

and Kharate

, “Feature Selection Using Genetic Algorithm for Face Recognition Based on PCA, Wavelet and SVM”, International Journal on Electrical Engineering and Informatics 6(1) (2014), 39–52.

50.

Shi

and Eberhart

, “Parameter selection in particle swarm optimization”, International Conference on Evolutionary Programming (1998), pp. 591–600.

51.

Valdez

, Melin

and Castillo

, “Fuzzy Control for Dynamical Parameter Adaptation in a Parallel Evolutionary Method Combining Particle Swarm Optimization and Genetic Algorithms”, Soft Computing for Intelligent Control and Mobile Robotics: Springer, 318 (2010), 161–178.

52.

Valdez

, Vázquez

J.C.

and Gaxiola

, “Fuzzy Dynamic Parameter Adaptation in ACO and PSO for Designing Fuzzy Controllers: The Cases of Water Level and Temperature Control”, Advances in Fuzzy Systems 2018 (2018), 1274969:1–1274969:19.

53.

Valdez

, Vázquez

J.C.

, Melin

and Castillo

, “Comparative study of the use of fuzzy logic in improving particle swarm optimization variants for mathematical functions using co-evolution”, Applied Soft Computing 52 (2017), 1070–1083.

54.

Vázquez

J.C.

and Valdez

, “Fuzzy logic for dynamic adaptation in PSO with multiple topologies”, IFSA/NAFIPS, 2013, pp. 1197–1202.

55.

Vijay

and Jena

, “PSO based neuro fuzzy sliding mode control for a robot manipulator”, Science Direct 4 (2017), 243–256.

56.

Yang

X.S.

, “Firefly algorithms for multimodal optimization”, Proc. 5th Symposium on Stochastic Algorithms, Foundations and Applications 5792 (2009), 169–178.

57.

Yao

Y.Y.

, “Perspectives of granular computing”, IEEE International Conference on granular computing (GrC), pp. 85–90, 2005.

58.

Zadeh

L.A.

, “Fuzzy Sets”, Information and Control 8 (1965), 338–353.

59.

Zadeh

L.A.

, “Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/intelligent systems”, Soft Computing 2 (1998), 23–25.

Comparison of particle swarm optimization variants with fuzzy dynamic parameter adaptation for modular granular neural networks for human recognition

Abstract

Keywords

1 Introduction

2 Related works

2.1 Modular granular neural networks

Table 2 Initial parameters of the PSO Parameter Value Particles (n) 10 Maximum Iterations 30 C1 2 C2 2 w 0.8

4.1 Ear results

4.1.1 Results using a percentage of data for training up to 80%

4.2.1 Results using a percentage of data for training up to 80%

5.1 Results for ear database

Table 18 Critical values N α 0.01 0.02 0.05 0.10 20 037 043 052 060 30 109 120 137 152

References

Table 2
Initial parameters of the PSO

Parameter Value

Particles (n) 10

Maximum Iterations 30

C₁ 2

C₂ 2

w 0.8

Table 18
Critical values

N α

0.01 0.02 0.05 0.10

20 037 043 052 060

30 109 120 137 152