An Efficient Hybrid Face Recognition Algorithm Using PCA and GABOR Wavelets

1. Introduction

Face recognition has received significant attention due to its wide range of applications. The application of face recognition have been expanded recently, not only to the public sector, such as security, surveillance and access control for offices, but also to personal devices such as digital cameras, service robots, smartphones and laptops. Numerous studies on face recognition have been reported, but the recognition performance of practical face recognition systems is largely influenced by variations in illumination conditions, viewing direction or poses, facial expression, aging and disguises [1, 2]. It is still regarded as a challenging task because of unresolved issues caused by these issues.

The existing face recognition algorithms are generally classified into three categories according to the type of features used:

Holistic methods: These methods represent the whole facial region as a high-dimensional vector that is used as an input to a classifier.

Face recognition based on the Eigenspace algorithm [3–6] - one of the holistic methods - was introduced in 1991 and achieved more noticeable recognition performance than other available algorithms at the time. As a result, holistic methods have received much attention and have been researched actively up to the present day. First and foremost, the characteristic aspect of the facial representation used in these holistic methods is that it contains overall information for a face, since the entire facial region is represented as a high-dimensional vector. This naturally leads to the fact that it holds the global characteristics of a face, such as the shape of the face and relationships among the facial components [7]. Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Independent Component Analysis (ICA) are well-known, popular and successful holistic methods. PCA is an effective feature extraction method in pattern recognition. It utilizes a set of the most expressive feature vectors, which maximizes the variances of the features [2, 4]. LDA uses the class information of features and finds a set of the most discriminating feature vectors that minimizes within-class scatter while maximizing between-class scatter [8]. ICA finds statistically-independent basis vectors and minimizes both second-order and higher-order dependencies in the input images [9].

Before holistic algorithms were developed in 1990s, local methods had been actively studied, employing features extracted from local facial regions [10, 11]. Recently, local methods came into focus again, partly due to their robustness against occlusion and variations in facial expressions. Local methods have two advantages compared to holistic methods [12, 13]. First, a face can be represented as a set of feature vectors extracted from local regions, resulting in lower-dimensional vectors. Thus, the “curse of dimensionality” problem could be alleviated. Second, local features might be useful when some part of the face is occluded. Bruneli and Possio [14] compared two classes of local methods - geometrical feature-based methods and template-based methods - in which the result showed that the template-based approach saw better performance. Ahonen et al. [15] applied a texture descriptor, called a ‘Local Binary Pattern’ (LBP), for face recognition. An LBP operator is applied to the face image, and then the output is partitioned into sub-regions. Histograms of the sub-regions are utilized for face recognition.

Among various local feature representations, Gabor wavelets have been recognized as one of the most successful local feature extraction methods due to their robustness against distortions caused by variations in illumination, scale, translation and rotation [16, 17]. Many approaches based on Gabor wavelets have been proposed for face recognition, such as Dynamic Link Architecture (DLA) [18], Elastic Bunch Graph Matching (EBGM) [19], Gabor-Fisher Classifier (GFC) [20], Fusion of Multiresolution Gabor Features [21], and Local Gabor Binary Pattern Histogram Sequence (LGBPHS) [22]. DLA and EGBM are face representation methods using elastic graph matching. DLA represents a face using a rectangular graph with local features extracted at deformable nodes using Gabor wavelets. Wiskott et al. [19] have expanded DLA to EGBM, employing an object-adapted graph and node, which refer to a specific facial landmark. Liu et al. [20] applied the enhanced Fisher linear discriminant model (EFM) to the Gabor feature vector for face recognition and showed that its result outperformed both PCA and LDA. Xu et al. [21] proposed a matching-score level with two fusion methods which obtains good performance on Gabor Features. LGBPHS describes a face by combining the LBP operator and the Gabor wavelet transforms.

With the hybrid methods, both holistic and local features are used to enhance the recognition performance by preserving their strengths and averting any weaknesses. Pentland et al. [23] applied the concept of eigenfaces to some facial components, such as the eyes, nose and mouth, which correspondingly produces ‘eigeneyes’, an ‘eigennose' and an ‘eigenmouth’. The work of Lanitis et al. [24] presents a flexible appearance model-based face recognition method which uses statistical shape information, an Active Shape Model (ASM) and grey-level information; both are constructed using PCA.

Both holistic and local features have their own merits in face recognition. Holistic features contain a statistical summary of the spatial layout properties, giving the global contours and textures of the image. On the other hand, local features may provide detailed regional properties. According to a psychological study of the brain's cognitive mechanisms underlying human scene recognition researched by Oliva and Torralba [25], both features are used in the human perception process, but they play different roles. The global precedent hypothesis [26] claims that holistic features are processed before analysing local features in human perception mechanisms. Holistic features contain a statistical summary of the spatial layout properties of a face, giving the coarse and global contours and textures of it. On the other hand, local features are known to provide a detailed local description.

In a previous study, we experimentally showed that the ground truth image - corresponding to a test image -tends to exist in the upper part when sorted by similarities [27]. In this paper, we propose a hybrid face recognition algorithm in which PCA is used in the first stage and LGBPHS - one of local methods - is used in the second stage.

The rest of this paper is organized as follows: related works concerning LGBPHS are described in the next section. In Section 3, the suggested hybrid method is described, along with the principles of its modules. Section 4 presents experimental results to verify the efficiency and advantages of our proposal. Conclusions are provided in the final section.

2. Local Gabor Binary Pattern Histogram Sequence (LGBPHS)

In this section, we introduce - briefly - the LGBPHS, proposed by Zhang et al. [22], a non-statistical-based face recognition approach which combines Gabor wavelets and LBP in enhancing the representational power of the spatial histogram. Gabor wavelet-based face representation has demonstrated its robustness against variations, such as illumination, scale, translation and rotation. LGBPHS employs Gabor Wavelets and LBPs to convert a face image into a sequence of histograms. An LGBPHS operator consists of the following three steps.

First, an original face image is convolved with Gabor filters to obtain the image, called a ‘Gabor Wavelet Representation’ (GWR) [22]. The GWR is obtained as follows:

G μ , v = f z * φ μ , v z

(1)

where f(·) denotes the image, * denotes the convolution operator, and φ denotes the Gabor filter with a frequency v ∊ {0, …, 4} and orientation μ ∊ {0, …, 7}.

We use 40 Gabor filters with five frequencies and eight orientations, as shown in Eq. (1).

Second, we apply an LBP operator to the magnitude part of the GWR to obtain a Local Gabor Binary Pattern (LGBP) image. The LBP is an operator which moves a 3×3 window in an image to extract the structural features which are known to be robust to the illumination and noise variations [15]. The LBP is defined as below:

Γ X = ⊗ ∀ p ∊ W ζ I − W , I p

(2)

where Γ(X) is the bit string representing the structural feature at a point X, ⊗ is a concatenation operation, W is a 3×3 window, Ī(W) is a mean intensity value of pixels in W, p is a pixel in W, I(p) is an intensity value of p, and ζ (Ī(W), I(p)) is a comparison operation that returns 1 if Ī(W) < I(p) and 0 otherwise. A typical example of an LBP operation is shown in Figure 1. The LBP exhibits relatively good performance in face detection and recognition, while its operation is simple [15, 28].

OPEN IN VIEWER

Figure 1.

Example of an LBP operation

Third, an LGBP image is partitioned into 15×15 pixel-sized sub-regions, whereby histograms of the sub-regions are obtained afterward. A 15×15 sub-region is selected because it provides an efficient recognition rate after having experimented with various alternative sizes of sub-regions, such as 5×5, 21×20, and 35×40. The reason why we divide the GBP images is that the dimension of the LGBP can be reduced by partitioning the entire face image into small local regions while detailed local information is preserved. Original facial images of 105×120 pixels are used throughout this paper. The number of histograms per image is 2,240, as it is obtained after the multiplication of 40 (the number of Gabor filters) by 56 (the number of sub-regions).

We compare the histograms obtained from the above procedure - LGBPHS - to measure the similarity between two facial images. The similarity between the histograms of a test image and a training image is measured by employing the Bhattacharyya distance shown below:

d B h a t t a c h a y y a H 1 , H 2 = 1 − ∑ I H 1 ( I ) ∙ H 2 ( I ) H 1 − H 2 − N 2

(3)

where H₁ and H₂ are histograms, N is the total number of histogram bins, and H ¯ k is defined as below:

H k − = 1 N ∑ j H K ( j )

(4)

The Bhattacharyya distance becomes 0 when two histograms are exactly the same, and it grows to 1 as the dissimilarity level is increased. Thus, one may select the training image which has the lowest Bhattacharyya distance value as the matching facial image according to the LGBPHS procedure.

We propose a hybrid classifier for face recognition, utilizing the advantages of both holistic and local features. The classifier, called ‘Dual-stage of PCA and LGBPHS’ (DPL), consists of two stages in a serial order. Figure 2 shows the flowchart of the classifier. In the first stage, PCA-based coarse recognition is performed, where the distances between a test image and the training images are used to measure the confidence level [27]. The confidence level measurement is performed as follows. Training images are sorted in ascending order according to their distance values between a test image and the training images. The ith sorted image is called rank i. If the ranks of 1, 2 and 3 belong to the same person or the identity of rank 1 exists three or more times among ranks 2 to 6, then it passes the confidence test. This specific test rule has been obtained from a number of experiments on PCA-based face recognition conducted in our previous work [27].

OPEN IN VIEWER

Figure 2.

Flowchart for the proposed DPL algorithm

When the confidence level is reliable enough (i.e., if it passes the confidence test), the recognition procedure is finished at the first stage. Otherwise, the second stage is initiated. In the second stage, the classifier reduces the number of candidate training images by selecting only the upper n training images after sorting them in ascending order according to their distance values between a test image and the training images. The retained n images are used for the second stage using the LGBPHS approach introduced in Section 2. In Section 4, we show the recognition rates with the corresponding computation times when we select the upper six and upper 25 training images. As the number of candidate images increases, we expect a lower probability of false rejection, but a higher computational load is unavoidable. We call the proposed dual-stage algorithm ‘DPL₆’ and ‘DPL25’ according to the number of candidate images supplied to the second stage.

In the experiment, we compared the cases of DPL6 and DPL25 to show the trade-off between the number of retained images and computational load. The main advantage of this coarse-to-fine strategy is the computational load reduction; only a small number of training images are retained from the former stage. This strategy is more efficient when the latter stage uses computationally heavy methods, like Gabor filtering.

The coarse-to-fine face recognition strategy has already been studied in other research [29, 30], but the criteria for the selecting the candidate images for the second stage was not clearly specified. We extend our previous work [27] to show the experimental grounds for the criteria. More importantly, the proposed recognition approach has a flexible structure that enables the finishing of the recognition procedure at the first stage using PCA. In experiments, the proposed approach showed higher recognition rates and required less time for computation under illumination variations than the PCA- and Gabor filter-based recognition algorithms.

This paper presented a hybrid face recognition method that employs dual-stage holistic- and local feature-based algorithms. The proposed DPL algorithm utilizes a PCA algorithm in the first stage and an LGBPHS in the second stage. The confidence test procedure was introduced to decide which test images were passed to the second stage for further classification.

By adopting the confidence test, we reduced the number of test images that were passed on to the more time consuming second stage. The average computation time per image was reduced, as it passed only the upper six or 25 training images when a test image failed the confidence test.

In the experiment, we also proved that the recognition rate has been improved in the proposed algorithm.