An Illumination Insensitive Normalization Approach to Face Recognition Using Locality Sensitive Discriminant Analysis

Face recognition is an important identity verification technique employed in several applications and areas such as police verification, access control, video surveillance, and many others. It is an assignment to make a true recognition of a face in a photograph or video against an already accessible dataset of faces. Face recognition starts with detection, proceeds with separating the human faces from other objects within the image and then identify those detected faces.

In the last decade, a noteworthy development has been made in the field of face recognition. However, the effectuation of most intelligent face recognition algorithms is very sensitive to the variation of lighting conditions. These algorithms may not perform very well under the illumination variation. Many approaches have been proposed to get rid of the illumination variation effects in recent years [1]. Now the idea of development of the illumination-invariant normalization approaches for face recognition has been described briefly. To the best of our knowledge, almost each technique for face identification under variable lighting conditions, followed the initiative work of Horn [2], and utilized the fact that the luminance of an image is usually smoother than the reflectance. It was assumed that the most illumination variation lies in the low frequency part of the spectrum. But later on, Xie et al. [3] explored that some of the important features for face recognition may rest in the low frequency part and so not advisable to discard that part. In general, illumination normalization techniques can be categorized into three groups.

The first group adopts image preprocessing techniques for normalizing face images under varying illumination conditions. Lee et al. [4], the orientated local histogram equalization (OLHE) has been proposed to compensate illumination, while preserving rich information on the edge orientations. Zhang et al. [5], the lighting invariant representation of the image was obtained by taking the ratio of the image gradients. Their approach was based upon the fact that the lighting component changes slowly as compared to the reflection component of an image. Impressed by the benefits of the gradient domain, Tzimiropoulos et al. [6] have also described a novel technique of subspace learning from image gradient orientations instead of image intensities for recognizing face images. Wang et al. [7], the illumination invariant face images were obtained by taking the tangent inverse of the ratio of the reflectance change to the original reflectance. The algorithm considered the neighborhood of the image pixels based upon Weber’s law for this purpose. Ding et al. [8], a robust approach to obtain “Multi-Directional Multi-Level Dual-Cross Patterns” (MDML-DCPs) from the face images, has been proposed. The authors used the Gaussian operator and its first derivative to make the input images illumination insensitive and then proposed a new face image descriptor based upon the textural structure of human faces.

The second group handles the illumination problem by considering the 3-D representation of face images. Kumar et al. [9], the authors have proposed a 3-D face recognition algorithm by making use of the orthogonal tensor locality sensitive discriminant analysis (OLSDA). Their algorithm was inspired by the Locality Sensitive Discriminant Analysis (LSDA) [10, 11]. LSDA maximizes the margin between data points that belong to different classes and minimize the distance between data points that belong to the same class. LSDA represents an image in the form of a one-dimensional vector that makes it challenging to reconstruct 3-D face data. So, Kumar et al. [9] have used tensor OLSDA to overcome this problem by evaluating the mutually orthogonal basis functions in the iterative aspect using tensor data representation.

The third group extracts the invariant illumination information from the given intensity images. Chen et al. [12] have employed the scale-invariant property of natural images to construct a Wiener filter which extracts illumination invariant features from the images. The non-sub-sampled contourlet transform (NSCT) has been investigated for face image recognition in da Cunha et al. [13]. Their algorithm was based upon the decomposition of the image into various frequency subbands. The obtained coefficients of these subbands were used for face recognition directly. Their algorithm was extended by taking NSCT in the logarithmic domain to extort the illumination invariant facial features in Xie et al. [14]. They concluded that the decomposition process could not correctly extract the facial features, as the reflectance model non linearly characterizes an image. Nabatchian et al. [15], a maximum filter was used to get invariant illumination images, and then mutual information (MI) and image entropy were incorporated as a weight factor for classifiers in face recognition. Their algorithm is also robust to the face recognition system in case of image occlusion. Cao et al. [16] extracted the invariant illumination information from the face images using wavelet coefficients in the logarithmic domain.

Zhang and Xie [17] concluded that the combination of image preprocessing and illumination invariant extraction is the best way to attain the satisfied face recognition. Hu et al. [18], the classification of image sets is proposed based upon the sparse approximate nearest points (SANP) for face recognition. The optimization of SANP in their algorithm accounts for the minimization of the distance and the maximization of the sparsity of the nearest points. Naseem et al. [19] formulated their approach for identification of face images in terms of linear regression, giving their results in case of varying facial expression and occlusion also. Anvaripour and Ebrahimnezhad [20], the local shape descriptors were used to obtain the boundary fragment extraction using Poisson equation properties. Afterward, the Gaussian mixture model (GMM) was employed to find the relation between boundary fragments and then finally detect the object. Biswas and Ghose [21], an algorithm based upon orientation histogram of Hough Transform Peaks has been proposed. Tan and Triggs [22], a method using local texture feature sets has been presented for face recognition in case of uncontrolled lighting conditions.

Deep learning is a foremost technique in machine learning. Deep Neural Networks (DNN) find many applications in the field of face recognition, image classification and speech recognition. A nice demonstration has been given in Balaban [23] about state of the art (SOTA) regarding deep learning and face recognition. Elmahmudi and Ugail [24] performed experiments by taking individual parts of the face like nose, eyes and cheeks and also by joining them. They used two classifiers viz. cosine similarity and linear support vector machines to find the recognition rates. Bah and Ming [25], Local Binary Pattern (LBP) has been employed for face recognition after preprocessing of the face images using histogram equalization, contrast adjustment and bilateral filter. Jin et al. [26], face recognition approach has been demonstrated using neural learning techniques, which delivers voice messages to visually impaired persons so that they can navigate easily. Taigman et al. [27], a nine-layer deep neural network has been proposed for face identification. A combination of convolutional neural network (CNN) for feature extraction, support vector machine (SVM) as classifier and principal component analysis (PCA) for dimension reduction has been incorporated in Benkaddour and Bounoua [28] for face recognition. Again in Sun et al. [29], a deep learning approach has been used to extract in-depth identification verification features for face representations. Also, Ding and Tao [30], a comprehensive deep learning framework has been presented for face representation using multimodal information.

In this paper, we propose a novel method for the identification of images under varying illumination to get superior recognition rates. We are taking advantage of the merits of both first and third groups, i.e., using the combination of image preprocessing and illumination invariant extraction. In contrast to existing methods, the proposed method does not require multiple training images to obtain insensitive illumination images. The illumination normalized images are obtained by taking the ratio of the absolute value of the gradient of the image to the original intensity of the image. After preprocessing of the images, they are classified using Locality Sensitive Discriminant Analysis (LSDA) in the reduced dimensionality domain.

The organization of different sections in this paper is as follows: a brief review of the existing technique of LSDA is explained in section 2. Then the proposed method for image illumination invariant formulation is given in Section 3. Section 4 explores the experimental results, and finally, the paper is concluded in Section 5.

The proposed approach for face recognition can be accomplished in the following three phases:

–Preprocessing to get the illumination insensitive representation

–Dimensionality reduction using Locality Sensitive Discriminant Analysis

–Matching score generation using nearest neighborhood classifier and final decision

Here, we are going to explain all these steps in detail.

3.1 Illumination insensitive representation

The proposed procedure to obtain illumination insensitive representation of the images, includes two steps: a) homomorphic filtering of the original image; b) further gradient face computation using the filtered image. Basically, the homomorphic filter is used to attenuate the contribution made by the low frequencies (illumination) and amplify the contribution made by high frequencies (reflectance). The net result is sharpening of features and flattening of lighting variations in an image by the simultaneous dynamic range compression and contrast enhancement. Further, the second pre-processing step, performs photometric normalization of the filtered image using the Gradient face approach. It again makes the images, illumination invariant to a greater extent.

Let Ω be an open subset of $\Re^{2}$, a scalar function h defined on Ω be the depth of the shape, and (l_x, l_y, l_z) be the illuminant direction vector. According to Lambertian reflectance model, the image irradiance equation of the surface illuminated by a single distant light source and in the absence of self shadowing, is written as:

$I(x, y)=R(x, y) L(x, y)$  (9)

where, R(x, y) and L(x,y)=(l_x, l_y, l_z)are the reflectance and illuminance at a pixel point (x, y) of an image whose intensity value is I(x, y) at that particular pixel.

The illumination component of an image generally tends to vary slowly by spatial variations, where as the reflectance component changes precipitously. These characteristics lead to apply Fourier transform to the image, so as to associate the low frequencies with illumination and the high frequencies with reflectance. To reduce the impact of illumination, a homomorphic filter H(u, v) is used in our proposed algorithm. It is a slightly modified form of the Gaussian high pass filter [36]. The function H(u, v) is defined as:

$H(u, v)=\left(\gamma_{H-} \gamma_{L}\right)\left[1-e^{-c\left[\frac{D^{2}(u, v)}{D_{0}^{2}}\right]}\right]+\gamma_{L}$  (10)

where, D(u, v) is the distance from a point (u, v) in the frequency domain to the center of the frequency window. D₀ is the cutoff distance measured from the origin. The parameters of the filter, γ_L<1 and γ_H>1 are chosen so as to amplify the effect of reflectance and to attenuate the effect of illumination and c  is a constant to control the sharpness of the slope of the filter function as it transitions between γ_H and γ_L.

Now, take the natural logarithm of Eq. (9), as the Fourier transform of a product is not the product of transforms. Also add 1 to the original image to avoid ambiguity in considering logarithm of the image having zero intensity values. Suppose, we define

$I_{1}(x, y)=\ln (I(x, y))=\ln R(x, y)+\ln L(x, y)$  (11)

Then, after taking Fourier transform of Eq. (11)

$\mathcal{F}\left\{I_{1}(x, y)\right\}=\mathcal{F}\{\ln R(x, y)\}+\mathcal{F}\{\ln L(x, y)\}$

$\overline{\mathrm{I}_{1}(\mathrm{u}, \mathrm{v})}=\overline{R(u, v)}+\overline{L(u, v)}$  (12)

Afterwards, apply filter H(u, v), defined in Eq. (10), on $\mathrm{I}_{1}(\mathrm{u}, \mathrm{v})$, we get

$\mathrm{Z}(\mathrm{u}, \mathrm{v})=\mathrm{H}(\mathrm{u}, \mathrm{v}) \overline{\mathrm{I}_{1}(\mathrm{x}, \mathrm{y})}$  (13)

Then, taking the inverse Fourier transform of the filtered image, we obtain

$z(x, y)=\mathcal{F}^{-1}\{Z(u, v)\}$  (14)

Finally, we take the exponential of the filtered image in spatial domain in reverse order as z(x, y) was processed after taking the natural algorithm of the original image I(x, y).

$I^{\text {new }}(x, y)=e^{z(x, y)}$  (15)

Finally, we get homomorphic filtered image as I^new(x, y), which is further processed to neglect illumination effects upto a great extent. As we have discussed earlier, the value of L(x, y) depends upon the lighting source, therefore it is assumed that L(x, y) varies very slowly as compared to R(x, y). Let (i, j) be the position of any pixel in the considered image under varying illumination. Then L_i,j and L_i+1,j are approximately equal, where (i, j) and (i+1, j) are neighbouring points of an image. Again we consider the obtained filtered image as a function of illumination L¹(x, y) and reflectance R¹(x, y).

$I^{\text {new}}(x, y)=R^{1}(x, y) L^{1}(x, y)$  (16)

The filtered images are further processed in the gradient domain, so as to set up relationship with the neighbouring pixels also. The image gradients are evaluated using central difference operator in the interior of the image grid and appropriate forward and backward difference operator on the boundaries of the image. The formulae for calculating the image gradients are given as:

$p_{i, j}=\left\{\begin{array}{ll}\frac{I_{i+1, j}^{n e w}-I_{i-1, j}^{n e w}}{2} & \text { if }(i, j) \in \text { image interior } \\ I_{i+1, j}^{n e w}-I_{i, j}^{n e w} & \text { if }(i, j) \in \text { left boundary } \\ I_{i, j}^{\text {new }}-I_{i-1, j}^{\text {new }} & \text { if }(i, j) \in \text { right boundary }\end{array}\right.$

$q_{i, j}=\left\{\begin{array}{l}\frac{I_{i, j+1}^{n e w}-I_{i, j-1}^{n e w}}{2} \text { if }(i, j) \in \text { image interior } \\ I_{i, j+1}^{n e w}-I_{i, j}^{n e w} \text { if }(i, j) \in \text { upper boundary } \\ I_{i, j}^{n e w}-I_{i, j-1}^{n e w} \text { if }(i, j) \in \text { lower boundary }\end{array}\right.$

Using the above formulae of p_i,j and q_i,j in reflectance model defined in Eq. (16), we have

$p_{i, j}=\left(\frac{\partial R^{1}}{\partial x}\right)_{i, j} L_{i, j}^{1}$  (17)

$q_{i, j}=\left(\frac{\partial R^{1}}{\partial y}\right)_{i, j} L_{i, j}^{1}$  (18)

The absolute value of the gradient of the image I^new(x, y) is evaluated by taking the square root of the sum of the square of gradient of the image in x-direction and y-direction.

$\left|\nabla I^{n e w}(x, y)\right|=\sqrt{\left(p_{i, j}\right)^{2}+\left(q_{i, j}\right)^{2}}$$=\sqrt{\left(\left(\frac{\partial R^{1}}{\partial x}\right)_{i, j}\right)^{2}+\left(\left(\frac{\partial R^{1}}{\partial y}\right)_{i, j}\right)^{2}} \cdot L_{i, j}^{1}$  (19)

In order to attain illumination insensitive normalized image, the ratio of the absolute value of the gradient of the image to the original image intensity is computed in the following manner.

$\frac{\left|\nabla I^{n e w}(x, y)\right|}{I^{n e w}(x, y)}=\frac{\sqrt{\left(\left(\frac{\partial R^{1}}{\partial x}\right)_{i, j}\right)^{2}+\left(\left(\frac{\partial R^{1}}{\partial y}\right)_{i, j}\right)^{2}}}{R_{i, j}^{1}}$  (20)

Since the reflectance component R¹(x, y) is illumination insensitive and the right hand side of Eq. (20) is independent of the illumination component L¹(x, y), we have obtained illumination insensitive normalized image. In practical applications, the zero intensity value of the image in Eq. (20) gives infinite value to the illumination invariant measure. To avoid this type of ambiguity, we take the tangent inverse of the Eq. (20).

$I_{\text {final}}(x, y)=\tan ^{-1}\left(\frac{\left|\nabla I^{\text {new}}(x, y)\right|}{I^{\text {new}}(x, y)}\right)$  (21)

1.jpg

Figure 1. Concept map to generate illumination insensitive normalized images

2.jpg

Figure 2. Illumination sensitive images (Top-row), homomorphic filtered images (Middle-row), and gradient-based illumination insensitive normalized images (Last-row)

To avoid the computational difficulty for the pixels having zero value in I^new(x, y) as well as in their gradients, we set the intensity of these pixels by zero in the new Image I_final(x, y). The concept map of the whole process to generate illumination insensitive normalized images is given in Figure 1.

Some samples of the illumination insensitive normalized images from the Yale B database are shown in Figure 2 using the proposed algorithm.

3.2 Dimensionality reduction and matching

After getting the illumination insensitive normalized images, LSDA has been employed on the vector form of an image using the procedure explained in section 2.1. LSDA computes a projection matrix using training images along with their predefined class label information. Such a projection matrix projects the images into a low dimensional space where the images sharing same class label are closer and the images having different class label are far apart. This projection matrix is preserved for recognition phase. To identify the class of a probe image, the illumination insensitive normalized image is obtained first. This preprocessed image is projected into the same low dimensional space computed using gallery images (preserved previously). Finally, a nearest neighbourhood classifier is used to compute matching scores between probe image and gallery images. Euclidean distance measure is used as distance metric.