New Fuzzy-Based Retinex Method for the Illumination Normalization of Face Recognition

Abstract

We propose a new illumination normalization for face recognition which robust in relation to the illumination variations on mobile devices. This research is novel in the following five ways when compared to previous works: (i) a new fuzzy-based Retinex method is proposed for illumination normalization; (ii) the performance of face recognition is enhanced by determining the optimal parameter of Retinex filtering based on fuzzy logic; (iii) the output of the fuzzy membership function is adaptively determined based on the mean and standard deviations of the grey values of the detected face region; (iv) through the comparison of various defuzzification methods in terms of the accuracy of face recognition, one optimal method was selected; (v) we proved the validations of the proposed method by testing it with various face recognition methods. Experimental results showed that the accuracy of the face recognition with the proposed method was enhanced compared to previous ones.

Keywords

Face Recognition Illumination Normalization Fuzzy-based Retinex

1. Introduction

With the widespread use of mobile devices, such as smart phones and mobile computers, many applications have migrated to mobile systems. In order to use various services – which include content protection, mobile banking, shopping and trading via mobile systems – users should perform the authentication process using a personal identification number (PIN), password or universal serial bus (USB) keys. However, more reliable authentication methods are required since swindles and identity thefts easily happen with these methods. Biometric technologies constitute the best alternative for reliable authentication because it is difficult for biometric characteristics to be stolen or forged.

Biometric systems identify a person by using biometric characteristics such as a face, an iris, a fingerprint or finger-veins, etc. They have been employed in many applications, such as ATMs, immigration administration and door control. These systems have also been adopted on mobile devices. The face recognition system is one of the most popular biometric systems because a user is easily able to perform the recognition process [1, 2]. Although fingerprint, palm vein and iris recognition show better recognition accuracy than face recognition, they all have shortcomings in that they are difficult to apply to a small sized mobile device because they require an additional image capturing device. On the other hand, face recognition can be easily implemented using the built-in camera of a mobile device, thus providing a user with greater convenience. Face recognition systems for mobile devices were introduced by Kim et al. [3]. However, since mobile devices can be used in both indoor and outdoor environments, illumination variations occur, which causes the performance of the face recognition to deteriorate. To overcome this problem, we introduce a new fuzzy-based Retinex method for illumination normalization for face recognition on mobile devices.

There have been many previous studies into the illumination normalization of face images. Xie et al. proposed a local illumination normalization method based on the Lambertian model [4]. However, their method takes a great deal of processing time by using the CANDIDE model for local region segmentation. Qing et al. improved the accuracy of face recognition under varying lighting conditions by using the Gabor phase probabilistic model [5]. Shan et al. proposed gamma intensity correction (GIC) with histogram equalization (HE) and quotient illumination relighting (QIR) [6]. They used the GIC method to normalize the overall image intensity and eliminated the side light by using HE. In addition, the QIR method normalized illumination with the quotient illumination obtained by the bootstrap method from face images captured under various lighting conditions. Nam et al. adopted adaptive filters combining a Retinex filter, a histogram equalization filter and an ends-in contrast stretching filter [7]. Wang et al. proposed the self-quotient image (SQI) method to normalize illumination and eliminate shadows [8]. However, all these methods take too much processing time to be applied to mobile devices with low processing power. Nam et al. proposed a method combining the scores of two face recognition methods based on SVM. The two face recognition algorithms are Retinex filtering with PCA+LBP and histogram stretching with PCA+LBP [9]. However, they could not solve the problem of local variations of illumination in the face region since a Gaussian function with a fixed sigma value was used for conventional Retinex filtering [9]. As such, we propose a new fuzzy-based Retinex method which uses a Gaussian filter with an adaptive sigma value to deal with various illumination conditions.

The rest of this paper is organized as follows: In Section II, we explain the proposed fuzzy-based Retinex method for the illumination normalization. The experimental results and conclusions are described in Section III and Section IV, respectively.

2. Proposed Illumination Normalization Method for the Face Recognition System

2.1. An overview of the proposed method

An overview of the proposed method is depicted in Figure 1. First, the face region is detected by the Adaboost face detector [10] after capturing a facial image. For the alignment and the size normalizations of the facial region, the positions of the eyes are found with the Adaboost eye detector within the detected facial region [9]. This is because the size of the facial region can be altered along the Z distance between the face and a camera lens. As such, conventional face recognition needs to normalize the size of the facial region. In this research, the optimal size of the normalized face was determined as 32×32 pixels by experiments in terms of the accuracy of face recognition and processing time. Next, the optimal sigma value of the Retinex filter was determined based on the fuzzy logic algorithm in which the mean and standard deviation of the grey values of the normalized facial image are used as the inputs of the fuzzy membership function. After this, the illumination of the face region is normalized by the Retinex filter with an optimal sigma value. Finally, face recognition is performed.

Figure 1.

The flowchart of the proposed method

2.2. Conventional Retinex algorithm

In previous works, Nam et al. used the conventional Retinex algorithm for the illumination normalization of a facial image; they reported that it displayed better performance than histogram stretching and histogram equalization [9, 11]. The conventional Retinex algorithm is based on the notion that the image intensity (L(x, y)) is represented as the multiplication of the illumination (I(x, y)) and the reflectance ratio (R(x, y)), as shown in (1) [9, 11, 12]:

L (x, y) = I (x, y) \times R (x, y)

(1)

where I(x, y) is the amount of incident light at the (x, y) position and R(x, y) is the reflectance ratio of the objects to the incident light at the (x, y) position, which represents the inherent textures of the face region without any illumination variations. Based on (1), the illumination variations are reduced by eliminating the illumination (I(x, y)) from the image intensity (L(x, y)). For this, a logarithm operation is used, as shown in (2) and (3):

\log L (x, y) = \log (I (x, y) \times R (x, y))

(2)

\log R (x, y) = \log L (x, y) - \log I (x, y)

(3)

The illumination (I(x, y)) is estimated by the convolution operation of the Gaussian filter (F(x, y)) and an image (L(x, y)), as shown in (4) and (5) [7, 11, 12]:

\log R (x, y) = \log L (x, y) - \log (L (x, y) * F (x, y))

(4)

F (x, y) = \frac{1}{2 π σ^{2}} e^{- \frac{x^{2} + y^{2}}{2 σ^{2}}}

(5)

Consequently, log R(x, y) is called the Retinex output, which is the illumination normalized output [9, 11, 12]. Figure 2 shows the overall procedure for obtaining the Retinex output (log R(x, y)).

Figure 2.

The conventional Retinex algorithm

2.3. Fuzzy-based Retinex algorithm

It is often the case that both global and local variations of illumination exist in the face region. As shown in Figure 5 (c), the global variation is the shadow in the whole face region, such as being brighter in the left hand and darker in the right hand regions of face. On the other hand, the local variation is that in the local face region such as a darker area in the right corner of the left hand eye. As such, the single-scale Retinex (SSR) of (4) which produces the illumination image (I(x, y)) by using one Gaussian filter with a fixed sigma value about all the face images cannot reduce both global and local variations [9]. To overcome this problem, multi-scale Retinex (MSR) was introduced, which uses multiple SSRs with weights. However, it takes a great deal of processing to use multiple Retinex filters. To overcome this problem, we propose a new fuzzy-based Retinex algorithm. The procedure for the proposed algorithm is shown in Figure 3.

Figure 3.

The procedure of the fuzzy-based Retinex algorithm

First, the mean and standard deviation of the grey values of the input face image are calculated and then used as inputs values in the fuzzy logic system. Next, the optimal sigma value of the Gaussian filter in the Retinex algorithm is determined through the fuzzy logic system. From this, the illumination image (I(x, y)) is produced by the convolution of the Gaussian filter with the optimal sigma value. Finally, the Retinex output (log R(x, y)) is obtained. The proposed method can remove the global and local illumination variation due to the use of the Gaussian filter with the adaptive sigma value.

2.3.1. Definition of the membership function

In this section, the fuzzy logic system shown in Figure 3 is explained. The fuzzy logic system is used to determine the optimal sigma value (σ) of the Gaussian filter of the Retinex algorithm. We use the mean and standard deviation of the grey values of the facial region as the input values of the fuzzy logic system. Generally, the mean value of the facial region represents the level of global illumination and the standard deviation reflects the variation level of the local illumination. Figure 4 shows facial images with various global illuminations. Figure 4 (a) is the facial image which is captured without a light source in an indoor environment. Its level of global illumination and mean value are lower than those of the other images. Figures 4 (b) and (d) are the images which are captured outdoors with back sunlight or indoors with a fluorescent lamp, respectively. In addition, Figure 4 (c) is obtained outdoors with frontal sunlight. It should be noted that the mean value increases if the level of global illumination is augmented. Figure 5 shows the facial images with various local illuminations. Figure 5 (a) shows an image which is captured without any local illumination variation outdoors with frontal sunlight. Figures 5 (b) and (c) were acquired outdoors with right-side and left-side sunlight, respectively. As shown in Figures 5 (b) and (c), the local shades are generated in the corner of the eye or the side of the nose in the facial images captured in a sidelight environment. Therefore, we can see that the standard deviation of the grey value of the face region increases by the local shade if the variation level of the local illumination is augmented. From Figures 4 and 5, we witness the phenomena where the mean of the face region increases if the level of global illumination is augmented, and the standard deviation of the grey value of the face region increases by the local shade if the variation level of the local illumination is augmented. It is based on these phenomena that we design the fuzzy membership functions. To determine the optimal sigma value (σ) of the Gaussian filter for the Retinex algorithm, we classify the input values (mean and standard deviation) by using five memberships: very small (VS), small (S), medium (M), large (L) and very large (VL), as shown in Figure 6.

Figure 4.

Facial images with various global illuminations: the mean values of the face region are (a) 95.64, (b) 137.16, (c) 156.10, (d) 161.42

Figure 5.

Facial images at various variation levels of local illumination: the standard deviations of the face region are (a) 62.23, (b) 83.10, (c) 87.47

Figure 6.

The membership functions of the proposed fuzzy logic for the mean, standard deviation and sigma value, respectively

According to general fuzzy theory, we design the fuzzy membership functions by using triangular shapes – which are most widely used in fuzzy applications – as shown in Figure 6 [13]. Using the five membership functions, the sigma value of the Gaussian filter is obtained, as shown in Figure 6.

2.3.2. Determination of the fuzzy rules

As shown in Table 1, the five types of output values (the optimal sigma values of the Gaussian filter for the Retinex algorithm) are determined using 25 fuzzy rules. When both the measured mean value and the standard deviation of the face region are very small (VS), we can deduce that the brightness of input face image is very low (low saturation) and that the discriminations among the facial features such as the eyes, nose and mouth are diminished. This means that the I(x, y) of (3) and (4) includes a low level of global illumination (F(x, y) of (4) and (5)), having a very large (VL) sigma value (σ). In other words, the I(x, y) can be estimated by the convolution operation of the image (L(x, y)) and the Gaussian filter (F(x, y)), having low amplitude and a very large (VL) sigma value (σ). However, since the mean level of the illumination of normalized face images is adjusted based on the average grey level of face images in the later procedure of face recognition (section II. D), we can simply control the sigma value of the Gaussian filter for the simple design of the fuzzy logic system. As such, we assign a very large (VL) sigma value as the output of the fuzzy logic system, as shown in Table 1.

Table 1.

The fuzzy rules (Very Small (VS), Small (S), Medium (M), Large (L), Very Large (VL))

Input 1 (Mean)	Input 2 (Std.)	Output (Sigma)
VS
	VS	VL
	S	L
	M	L
	L	M
	VL	M
S
	VS	L
	S	L
	M	M
	L	M
	VL	S
	M
VS		M
S		M
M		S
L		S
VL		VS
L
	VS	L
	S	L
	M	M
	L	M
	VL	S
VL
	VS	VL
	S	L
	M	L
	L	M
	VL	M

When the mean value is very large (VL) and the standard deviation is very small (VS), we can deduce that the brightness of the input face image is very high (high saturation). In such a case, the discriminations among the facial features are also reduced. Accordingly, the I(x, y) of (3) and (4) can be also estimated by the convolution operation of the image (L(x, y)) and the Gaussian filter (F(x, y)), having great amplitude and a very large (VL) sigma value. As mentioned, since the mean level of illumination of normalized face images is adjusted based on the average grey level of face images in the later procedure of face recognition, we simply control the sigma value of the Gaussian filter. As such, we assign a very large (VL) sigma value as the output of the fuzzy logic system, as shown in Table 1.

In the case where the mean value is medium (M) and the standard deviation is very large (VL), the discriminations among the facial features are great. Thus, the I(x, y) of (3) and (4) can be estimated by the convolution operation of the image (L(x, y)) and the Gaussian filter (F(x, y)), having a very small (VS) sigma value. Accordingly, we assign a very small (VS) sigma value as the output of the fuzzy logic system, as shown in Table 1. A large standard deviation value means that there is a lot of local shade in the facial region, which can be normalized by using the Gaussian filter having a smaller sigma value.

2.3.3. Determination of the sigma value based on the defuzzification

Based on Figure 6 and Table 1, the detailed procedures for obtaining the sigma value of the Gaussian filter are as follows: First, we normalize the mean and standard deviation of the grey values of the face image as numbers from 0 to 1. The normalized values are used as the input values of the membership function of Figure 6. For example, if the normalized mean value is 0.3125, it has two output values which are 0.25 as M (Medium) and 0.75 as S (Small), as shown in Figure 7 (a). If the normalized standard deviation value is 0.7, it has 0.8 as L (Large) and 0.2 as M (Medium) by the membership function of Figure 7 (b). From the output values of the membership function of Figure 7, we obtain 4 pairs, such as (0.25(M), 0.8(L)), (0.25(M), 0.2(M)), (0.75(S), 0.8(L)) and (0.75(S), 0.2(M)). To combine the two output values of the membership function of Figure 7, we use the MIN-MAX method. The reason why we use the MIN-MAX method is as follows. In the case of the MIN method, the two output values are combined by using the AND operation and the smaller value among the two output values is selected. In this case, the fuzzy rule is “If the first condition is A and the second condition is B, then the result is C”. The rule is an AND operation in which both conditions need to be satisfied. In the case of the MAX method, the OR operation is used and the greater value is selected [13]. In this case, the fuzzy rule is “If the first condition is A or the second condition is B, then the result is C”. The rule is an OR operation in which the result is obtained if either A or B is satisfied. Based on this, in the case where multiple inputs are combined based on the AND rule when defining the fuzzy rules like in Table 1, the MIN method is adopted [13]. Since we consider both the mean and standard deviations of the grey value of the face image at the same time for obtaining the optimal sigma value of the Gaussian filter – as shown in the fuzzy rules of Table 1 – the mean and the standard deviations are combined by the AND rule. Consequently, we use the MIN method.

Figure 7.

Examples of the obtaining of output values using the membership functions: (a) mean, (b) standard deviation

For example, with (0.25(M), 0.8(L)) of Figure 7, the membership function (for the output Sigma value) of small (S) is selected, as shown in Table 1. Based on the MIN method, the smaller value of 0.25 among 0.25 and 0.8 is selected. Consequently, 0.25(S) is obtained from (0.25(M), 0.8(L)). By the same method, 0.2(S) is obtained from (0.25(M), 0.2(M)), 0.75(M) is obtained from (0.75(S), 0.8(L)) and 0.2(M) is obtained from (0.75(S), 0.2(M)). Here, 0.25(S), 0.2(S), 0.75(M) and 0.2(M) are called inference values (IV). These inference values are used as the inputs (the values of the vertical axis of Figure 8) for the membership function of Figure 8. Multiple sigma values (the values of the horizontal axis of Figure 8) are obtained from the four inference values, such as 0.25(S), 0.2(S), 0.75(M) and 0.2(M). In Figure 8, S₁ to S₈ represent the sigma values. For example, in the case of “IV 0.25(S)”, the value on the vertical axis of Figure 8 becomes 0.25 and the corresponding sigma values on the membership function (S) are 0.0625 (S₂ of Figure 8) and 0.4375 (S₅ of Figure 8). By the same method, in the case of “IV 0.2(S)”, the corresponding sigma values on the membership function (S) are 0.05 (S₁ of Figure 8) and 0.45 (S₆ of Figure 8). In the case of “IV 0.75(M)”, the corresponding sigma values of the membership function (M) are 0.4375 (S₄ of Figure 8) and 0.5625 (S₇ of Figure 8). In the case of “IV 0.2(M)”, the corresponding sigma values of the membership function (M) are 0.3 (S₃ of Figure 8) and 0.7 (S₈ of Figure 8). Since multiple sigma values (S₁ to S₈ of Figure 8) are obtained, we use a defuzzification method in order to determine one final sigma value. Defuzzification is the procedure used to obtain results as a quantifiable value [14]. In order to determine the optimal defuzzification method, we tested five known defuzzification methods: centre of gravity (COG), first of maxima (FOM), last of maxima (LOM), middle of maxima (MOM) and mean of maxima (MeOM) [14]. The FOM, LOM and MOM methods determine the final sigma value based on the sigma values which are obtained by using the maximum inference value (0.75(M) of Figure 8 (a)) among 0.25(S), 0.2(S), 0.75(M) and 0.2(M). From this, the FOM method selects S₄ as the final sigma value. The LOM method selects S₇ and the MOM selects the value of (S₄ + S₇)/2. The MeOM method selects the value of (S₁ + S₂ +… S₈)/8 as the final sigma value. The COG method obtains the final sigma value from the centre of gravity. This centre of gravity is the geometric centre of four gravity points (G₁, G₂, G₃ and G₄ in Figure 8 (b)) calculated from four quadrangles, such as Q₁, Q₂, Q₃ and Q₄, as seen in Figure 8 (b).

Figure 8.

The defuzzification methods: (a) FOM, LOM, MOM and MeOM, (b) COG

The experimental results showed that the accuracy of face recognition using the COG method was higher than those using the other defuzzification methods, such as FOM, LOM, MOM and MeOM. The accuracy was measured based on the equal error rate (EER). The EER is defined as the error rate when the false acceptance rate (FAR) and the false rejection rate (FRR) are the same minimum value. The FAR denotes the error rate in which an un-enrolled user is falsely accepted as an enrolled one, while the FRR denotes the error rate which an enrolled user is falsely rejected as an un-enrolled one. The EERs with COG, FOM, LOM, MOM and MeOM were 12.3%, 17.8%, 20.3%, 18.5% and 15.2%, respectively. The difference (D₁) between the lowest EER and the second lowest is 2.9% (15.2% – 12.3%), and the second difference (D₂) between the lowest EER and the highest is 8.0% (20.3% – 12.3%). Since the lowest EER is 12.3%, the ratio of the first difference (D₁) to the lowest EER is 23.6% (100 (2.9/12.3)) and the ratio of the second difference (D₂) to the lowest EER is 65.0% (100 (8.0/12.3)), which can be regarded as being statistically significant.

2.4. Face recognition methods

With the illumination normalized face image, we perform face recognition. There have been many face recognition methods advanced in previous works, such as principal component analysis (PCA) [15], local non-negative matrix factorization (LNMF) [16] and local binary pattern (LBP) [17], etc. We used various face recognition methods for testing the performance of the proposed fuzzy-based Retinex algorithm. The PCA method is good at representing and analysing the global information of a face, but it is very sensitive to illumination variations. We perform PCA-based face recognition with 140 eigenfaces in order to represent a face image of 32×32 pixels [9]. The optimal number of eigenfaces was experimentally determined in terms of the equal error rate (EER) of face recognition [9]. The non-negative matrix factorization (NMF) used non-negativity constraints in place of the orthogonality of the PCA for training. Generally, it trains a parts-based face representation [16]. However, the NMF method has a problem in that it is hard to make a factorization that includes local features. To overcome this problem, we used the LNMF method for making a factorization which included the local features of faces [9]. The LNMF is different from the NMF in terms of producing bases with orthogonality [16]. We perform the LNMF-based face recognition with 100 eigenfaces in order to represent a face image of 32×32 pixels. The optimal number of eigenfaces was experimentally determined in terms of the EER of face recognition [9]. The LBP method generally represents the binary patterns of the image by using a non-parametric (3×3) pixels kernel [17]. The binary patterns are decided by comparing the grey values of eight surrounding pixels with the centre pixel. If the grey value of the surrounding pixel is greater than that of the centre, then the binary value is “1”, otherwise the binary value is “0” [9]. The local face characteristics using the LBP method are better represented than those in other face recognition methods. The PCA + LBP and LNMF + LBP methods represent both the global and local information of a face with good performance. We combine them for representing a face with both global and local features. After extracting 1024 eigen-coefficients (from a 32 × 32 pixels image) as representative global features with the PCA or LNMF method, they are reconstructed into a 32 × 32 matrix. Next, the LBP method is performed on the 32 × 32 matrix and binary codes are extracted [9].

3. Experimental Results

For the experiments, we used a conventional mobile computer as our mobile device [9]. The mobile computer includes a 1.2 GHz CPU, 512 MB memory, a 30 GB HDD and a 4.5 inch LCD display. The built-in camera of the mobile computer is used for obtaining the face images and the proposed algorithm is tested on the mobile computer. The images were obtained under various environments: indoors without any light source, indoors under fluorescent lamps, and outdoors in frontal, left-side, right-side and back sunlight environments. They are collected without facial expression or pose variation, but there are grimace images due to the frontal sunlight. The spatial resolution of the face image is 640×480 pixels. For convenience, we will call it the ‘mobile computer-database’. From the mobile computer-database, 956 images of 44 persons were used for the experiment. 486 images were used for training and the remained 470 images were used for testing. Figures 9 and 10 show an original image and the corresponding results images following histogram equalization, histogram stretching, Retinex filtering and Fuzzy-Retinex filtering.

Figure 9.

The results with the images that do not include illumination variation: (a) original images, (b) after histogram equalization, (c) after histogram stretching, (d) after Retinex filtering, (e) after Fuzzy-Retinex filtering (the proposed method)

Figure 10.

The results with images that include local illumination variation: (a) original images, (b) after histogram equalization, (c) after histogram stretching, (d) after Retinex filtering, (e) after Fuzzy-Retinex filtering (the proposed method).

Since the illumination normalization is performed with the size normalized face region (32×32 pixels), as shown in Figure 1 and Section 2.1, Figures 9 and 10 show the illumination normalized images with the size normalized face regions. Although the actual size of the image is 32×32 pixels in Figures 9 and 10, each image is magnified for convenience in order to enhance the visibility. In the case including the illumination variation, as shown in Figure 10, the amount of local dark areas of (d) and (e) becomes smaller than that of other cases. When comparing Figures 10 (d) and (e), the local dark area remained in the right eye in case of Figure 10 (d). However, it is removed in Figure 10 (e). When comparing the images by the proposed method to those of others, both the dark areas and the saturated parts become smaller by the proposed method, as shown in Figure 10.

For a quantitative proof, we counted the average number of the saturated pixels (ANSP) in the processed images. The threshold of the saturated pixels was empirically determined as 230. The results with the all the images showed that the ANSP of the original images (without illumination normalization), that by Retinex filtering and that by the proposed method were 74, 66 and 59, respectively. It was only with the outdoor images when sunlight affected the side of the face (which causes the most severe variations of illumination because it increases both the degree of local dark areas and saturated pixels in the face) that the results showed that the ANSP of the original images (without illumination normalization), that by Retinex filtering and that by the proposed method were 226, 95 and 71, respectively. From these experiments, we can confirm that the saturated parts by the proposed method become smaller than those by Retinex filtering.

In the next experiment, we measured the recognition accuracy of each method without using the illumination normalization method and by using an illumination normalization method – such as histogram equalization (HE), histogram stretching (HS), conventional Retinex and the proposed fuzzy-based Retinex method – in order to compare the proposed method with the other, as shown in Table 2. Here, it can be seen that the accuracies of all the face recognition methods were enhanced when using the proposed fuzzy-based Retinex method. The recognition accuracy was best at 12.3%, when using the LBP method and the proposed fuzzy-based Retinex method. From Table 2, the optimal pairs of illumination normalization and face recognition methods are determined. For example, in the case of the proposed fuzzy-based Retinex method, the accuracy of LBP is best.

Table 2.

Face recognition errors (EER) according to the illumination normalization methods with the mobile computer-database (unit: (%))

	PCA	LNMF	LBP	PCA +LBP	LNMF+ LBP
Without Illumination Normalization	31.6	26.4	26.8	29.6	34.2
HE	31.8	29.6	27.1	29.1	35.9
HS	31.6	26.5	27.4	27.5	33.9
Conventional Retinex Filtering	16.9	21.2	21.1	16.3	23.1
Fuzzy-based Retinex (Proposed Method)	12.4	20.9	12.3	14.0	22.8

In the next experiments, we tested the proposed method with a public face database known as the AR face database [19]. The image resolution of the AR database is 768×576 pixels. The database consists of 1,783 images from 136 subjects (76 men and 60 women) in the first and second session. Since the goal of our research is to make the algorithm robust in relation to illumination variations, we exclude those face images used for testing expression variations in experiment. As such, we only used 1,008 facial images with neutral expressions (135 subjects × (4 ~ 8 images) in the first and second sessions). In order to check the interoperability of the proposed method irrespective of the kind of camera, 486 images (which were captured by the mobile computer camera) were used for training, as with the experiments of Table 2. The whole of the images of the AR database were used for testing. Table 3 showed that the face recognition error (EER) of PCA+LBP with the proposed fuzzy-based Retinex method was lowest at 14.4%, which was lower than the other methods. As can be seen in Figure 9, histogram stretching and histogram equalization are useful for enhancing the visibility of face images in case where there is uniform illumination in the face region. However, these two methods have a weakness in the case of face images with non-uniform illumination, such as left-side and right-side sunlight environments, as shown in Figure 10. In this case, the conventional Retinex filter has better performance, especially if the illumination conditions (directions) are similar in all the images. However, because it uses the fixed sigma value of Gaussian filtering, its performance is degraded in those cases where the illumination conditions (directions) are changed by sunlight in outdoor environments. As such, the proposed fuzzy-based Retinex method exhibited better recognition accuracy over the conventional Retinex filter since the proposed method can change the sigma value adaptively based on the image conditions (mean and standard deviation) of the facial region which are affected by illumination conditions (directions). Figure 11 gives some examples of the use of the proposed face recognition system on a mobile computer.

Table 3.

Face recognition errors (EER) according to the illumination normalization methods with the AR database (unit: (%))

	PCA	LNMF	LBP	PCA +LBP	LNMF+ LBP
Without Illumination Normalization	28.3	32.8	20.0	20.4	32.4
HE	33.2	36.3	19.8	36.9	37.0
HS	28.3	32.8	20.0	21.6	32.7
Conventional Retinex Filtering	15.4	29.4	16.3	14.8	25.2
Fuzzy-based Retinex (Proposed Method)	15.2	21.4	15.8	14.4	24.9

Figure 11.

Examples of using proposed face recognition system on mobile computer: (a) indoors, (b) outdoors.

4. Conclusion

In this paper, we propose a new fuzzy-based Retinex method for the illumination normalization of facial images on mobile devices. In addition, we see that the performance of face recognition was enhanced under various illumination conditions when using the fuzzy-based Retinex method. However, the EER of each face recognition method was still greater than 10%. This is because some local shade regions remained in facial images even with the fuzzy-based Retinex method. Accordingly, we plan to study an illumination normalization method combining the fuzzy algorithm, multi-scale Retinex algorithms and neural networks.

Footnotes

5. Acknowledgments

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (No. 2012R1A1A2038666), and in part by the Public welfare & Safety research program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (No. 2012-0006554).

References

Lee

Byun

(2003) A New Face Authentication System for Memory-Constrained Devices. IEEE Transactions on Consumer Electronics. 49: 1214–1222.

Atta

Ghanbari

(2010) Low-Memory Requirement and Efficient Face Recognition System based on DCT Pyramid. IEEE Transactions on Consumer Electronics. 56: 1542–1548.

Kim

D-J

Chung

K-W

Hong

K-S

(2010) Person Authentication Using Face, Teeth and Voice Modalities for Mobile Device Security. IEEE Transactions on Consumer Electronics. 56: 2678–2685.

Xie

Lam

K-M

(2006) An Efficient Illumination Normalization Method for Face Recognition. Pattern Recognition Letters. 27: 609–617.

Qing

Shan

Chen

Gao

(2006) Face Recognition under Varying Lighting based on the Probabilistic Model of Gabor Phase. Proceedings of the 18th International Conference on Pattern Recognition. 1139–1142.

Shan

Gao

Cao

Zhao

(2003) Illumination Normalization for Robust Face Recognition Against Varying Lighting Conditions. Proceedings of International Workshop on Analysis and Modeling of Faces and Gestures. 157–164.

Nam

M Y

Rhee

P K

(2004) An Efficient Face Recognition for Variant Illumination Condition. Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems. 111–115.

Wang

S Z

Wang

(2004) Face Recognition under Varying Lighting Conditions Using Self Quotient Image. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition. 819–824.

Nam

G P

Kang

B J

Park

K R

(2010) Robustness of Face Recognition to Variations of Illumination on Mobile Devices based on SVM. KSII Transactions on Internet Information Systems. 4: 25–44.

10.

Viola

Jones

M J

(2004) Robust Real-Time Face Detection. International Journal of Computer Vision. 57: 137–154.

11.

Rahman

Z-U

Jobson

D J

Woodell

G A

(2004) Retinex Processing for Automatic Image Enhancement. Journal of Electronic Imaging. 13: 100–110.

12.

Hines

G D

Rahman

Z-U

Jobson

D J

Woodell

G A

(2004) Single-scale Retinex Using Digital Signal Processors. Proceedings of Global Signal Processing Conference.

13.

Klir

G J

Yuan

(1995) Fuzzy Sets and Fuzzy Logic-Theory and Applications. Prentice-Hall.

14.

Leekwijck

W V

Kerre

E E

(1999) Defuzzification: Criteria and Classification. Fuzzy Sets and Systems. 108: 159–178.

15.

Turk

Pentland

(1991) Eigenfaces for Recognition. Journal of Cognitive Neuroscience. 3: 71–86.

16.

S Z

Hou

X W

Zhang

H J

Cheng

Q S

(2001) Learning Spatially Localized, Parts-based Representation. Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition. 1: 207–212.

17.

Rodriguez

Marcel

(2006) Face Authentication Using Adapted Local Binary Pattern Histograms. Lecture Notes in Computer Science. 3954: 321–332.

Input 1 (Mean)	Input 2 (Std.)	Output (Sigma)
VS
	VS	VL
	S	L
	M	L
	L	M
	VL	M
S
	VS	L
	S	L
	M	M
	L	M
	VL	S
	M
VS		M
S		M
M		S
L		S
VL		VS
L
	VS	L
	S	L
	M	M
	L	M
	VL	S
VL
	VS	VL
	S	L
	M	L
	L	M
	VL	M

Input 1 (Mean)	Input 2 (Std.)	Output (Sigma)
VS
	VS	VL
	S	L
	M	L
	L	M
	VL	M
S
	VS	L
	S	L
	M	M
	L	M
	VL	S
	M
VS		M
S		M
M		S
L		S
VL		VS
L
	VS	L
	S	L
	M	M
	L	M
	VL	S
VL
	VS	VL
	S	L
	M	L
	L	M
	VL	M

Input 1 (Mean)	Input 2 (Std.)	Output (Sigma)
VS
	VS	VL
	S	L
	M	L
	L	M
	VL	M
S
	VS	L
	S	L
	M	M
	L	M
	VL	S
	M
VS		M
S		M
M		S
L		S
VL		VS
L
	VS	L
	S	L
	M	M
	L	M
	VL	S
VL
	VS	VL
	S	L
	M	L
	L	M
	VL	M