An Attention-Based Deep Regional Learning Model for Enhanced Finger Vein Identification

Personal identity recognition has demonstrated its validity as being among the most reliable and trustworthy technologies, as well as being rapid in expanding the science of establishing the identification of people [1]. A combination of measurements (measurements that enable a computer to identify individuals in terms of their physical and behavioral traits) is known as biometrics [2-6]. Using biometrics can be identified as how identification can be validated and developed as a method of electroencephalography (EEG) authentication [7], anthropometric identification [8], and facial recognition [9], which can also serve as distinct forms of biometrics, although there are specific options, such as eye iris recognition [10] and ear recognition [11], in which additional features are involved. Finger vein recognition has several advantages as a biometric identification technique: 1) Forging is extremely difficult. 2) It is suitable for use in a contactless environment or under weak contact measurement conditions. 3) Fingerprints are difficult to remove. Authentication using fingerprint sensors is used on laptops and mobile devices. They are embedded into discreet multi-use notebooks and smartphones that use budget fingerprint readers. One of the significant components of the biometric systems for individual identification is fingerprints. This type of identification has been utilized in individual identification as well as in forensic applications due to its stability and uniqueness from one individual to another, even from an individual's birth to death. The ridge of a fingerprint holds some unique points, defined as "minutiae points." The biometric technology of finger veins is less expensive and less intrusive compared to other biometric technologies such as face readers or retina scans. Finger vein recognition can be used in any type of environment. Most individuals discover that they cannot capture images by using video cameras or speaking into a microphone. Systems based on finger veins are easier to use. Moreover, it is considered a flexible alternative due to the ability to register multiple fingers. Finger vein technology has been tested and used for a long time compared to new technologies [12].

Nowadays, finger vein recognition and identification attract attention as a promising method of framework-based biometric identification. In recent years, the finger vein has received much attention from researchers. Finger vein images have very strong and unique patterns that make them very interesting in the biometric recognition field, but with some issues, such as the dryness of fingers, the surface makes accurate performance kind of hard to achieve. Sweatiness will decrease the output of the device, while dryness on the finger can clear the finger pattern. Advances in the quest for alternatives to the inadequacies of finger biometrics have come to the fore because of the increased awareness of their existence. This claim is true, every vein pattern is specific and proves to be important in personal verification. Changing the patterns of finger veins is tricky as it is an inner feature. The captured vein pattern value is difficult to manipulate through the skin, for example in the palm or finger veins. In addition, biometrics are more precise when compared with manual methods because they use the handprint feature in the finger vein scanners, which can be reduced in size by half. Compared to biometric technologies that use the hands, finger vein systems could have smaller devices [13].

Finger vein identification and recognition systems are still a big challenge since they mainly rely on the finger vein patterns beneath the human skin surface. The blood vessels, which are the main finger vein features, are used to extract the significant pattern of an individual, which is almost impossible to counterfeit. However, the blood vessels are invisible and require different techniques to extract the correct pattern, and it depends on the living individual. The main issue with the finger vein application is that there are still some challenges that affect the performance of this system, such as finger vein image quality and well-defined or extracted feature levels [14]. With the growth of the digital economy and safety systems for networks, it is now common for identity information to be verified. Identity verification is needed in everyday situations like using ATMs, unlocking cell phones, logging into personal online accounts, and shopping online. Traditional authentication methods like passwords and PINs [15] have some problems, like being less secure and easy to forget or lose. This makes it hard to meet the security needs of many applications. In recent years, biometric identification technology has gotten more and more attention because of its benefits, such as being able to identify living people, being hard to steal, and having unique features [16].

Traditional techniques for recognizing finger veins do not work well for a lot of people, so they are slowly being phased out. Image processing methods that are based on machine learning are the same as these traditional methods. Among them, deep learning techniques can be more accurate, but they often need a lot of training data, so they are not used very often right now. Different approaches for finger-vein image recognition and identification have been applied using machine learning and deep learning methods, but they still do not meet the challenges in this field, such as finger vein data localization, finger vein pattern extraction, and performance results [17]. For this reason, in this paper, we address those two factors and propose a new finger vein biometric identification and recognition system that solves the issue of image quality and achieves significant levels of feature extraction. Therefore, this paper can summarize some research problems that are being considered and try to build a strong approach and framework that can solve and manage them. Finger vein localization method in which background details and unnecessary information from finger vein images are removed using an unsupervised learning algorithm (optimized k-means clustering). Two stages of finger vein feature extraction are applied to extract some significant features that help the weak classifier achieve higher performance for the human finger vein identification approach. The main contributions of this paper are summarized:

· A new deep regional learning approach for finger vein recognition is proposed. The regional deep attention learning model is applied to the localized finger vein images, such as the original and colored, to significantly achieve the highest performance results on one of the most challenging finger vein datasets.

· The designed model is based on innovatively using attention mechanisms. The Regular Attention Mechanisms framework is used to fine-tune the backpropagated synthesis in a regular deep network. Instead of that, we used the attention mechanisms to fine-tune both feedforward and backward propagated synthesis in the same deep network by using a regional mask to obtain the ROI learning area.

· This study used k-means and fuzzy c-means to cluster, locate, and isolate the finger image from the entire finger vein image, instead of relying on the traditional methodology of global image thresholding.

· This study extracts significant features based on globalized feature pattern map indication (GFPMI).

The rest of this article is organized as follows: Section 2 presents a review of the literature for feature extraction and auto-classification. Section 3 explains the methodology of the study. Section 4 describes the details of the proposed method. Section 5 explains the experimental results.

3.1 Image acquisition

In this research, we utilized an available finger vein database named DUMLA-HMT. This dataset was constructed based on the computer designed by the Joint Lab for Wuhan University's Intelligent Computing and Systems, an ICT School lab within the WU that has gained a strong reputation. This dataset includes images of each person's hands: one has an index finger, the other has his middle finger, and the third hand is shown by the applicant. One information value is obtained six times to expand into a collection of six information images for each of the fingers, which works out to be six times more efficient. This database of finger veins consists of 6×6×106=3,816 images and each image has been stored in "BMP" format in 320×240 pixels size. Some sample images are shown in Figure 1 [28].

1.jpg

Figure 1. Sample images of finger vein dataset

3.2 Convolutional neural network (CNN)

A traditional approach to machine learning revolves around layer-by-layering, which is the idea of training successive layers of the network. Same as traditional machine learning, deep learning is a concept in machine learning that is focused on the notion of building successive layers of abstract representations. This type of hierarchical representation provides an opportunity for training data to be used for understanding and building concepts [28]. In practice, most methods of machine learning use two or three layers of neurons, whereas deep learning utilizes hundreds of layers. One of the most robust deep learning methods widely used in the literature is CNN. The CNNs are implemented to differentiate among various classes by producing such a vector of probabilities that are indicated by $\hat{y}=f(x)$ for all evaluated images in the dataset. If y is the true label of image x, the CNN performance of true label y of image x is measured by a loss function $l y(\hat{y}) \in R$ which assigns a penalty to classification errors.

3.2.1 Deep learning using convolutional neural network (CNN) mechanisms

The convolutional layer can convolve the outcome of the former layer with a group of earnable filters, where the convolution filter can be specified based on weights. This results in a more accurate filter classification per filter in the volume. Their volume height and width vary from filter to filter, producing a 2-dimensional activation chart. Similarly, the depth of the input is the same with both of the filters [29]. Three hyperparameters govern the size of the overall performance. The number of levels and the model's stride can be defined with the width, and zero padding can be adjusted separately.

Assume that we have some N×N square neuron layer which is followed by the convolutional layer of our layer. If we utilize an m×m filter with ω, the convolutional layer will be of size (N-m+1). To calculate the pre-nonlinearity input to some unit $x_{i j}^{\ell}$ our layer, we need to sum up the contributions (weighted by the filter components) from the former layer cells (See Eq. (1)):

$x_{i j}^{\ell}=\sum_{a=0}^{m-1} \sum_{b=0}^{n-1} w_{a b} y_{(i+a)(j+b)}^{l-1}$             (1)

After that, the convolutional layer implements its nonlinearity using Eq. (2):

$y_{i j}^{\ell}=\sigma\left(x_{i j}^{\ell}\right)$           (2)

The pooling layer works by reducing the size of the total input as well as the number of inputs, so multi-scale analysis is possible. Both operators, maxpooling and average pooling, are the two most commonly used pooling operators. Using these operators, the maximum or average value of a small block of data is found within a spatial entity. Pooling filters are considered the optimal filters based on their size 2×2 as well as their stride of 2. Eventually, all neurons of the former layer will be completely linked and bound to one another based on a fully connected layer. In addition, fully connected layers are usually used as the last classifier in a network, and they are often mostly used as decision classifiers.

3.2.2 Attention learning mechanisms

The attention keyword represents an action that guides directly to the object, has specific aims, and focuses attention. It can be known as giving need as it is the mind's ability to assign uneven consideration through the area of sensation. Furthermore, concentration confirms input to the center of attention while diminishing or ignoring others.

Technically, the attention given to the current activity has helped find the value of the system in the neural networks. The action of paying attention assists in terms of credit assignment. The key challenge of that action is ensuring that the range across which it applies is reliable and durable. A way of putting it is that the prediction has become stronger and more influenced by the other facts. The central Markov assumption is used to be employed in the attention network's Markov model to allow various likelihood values for different choices in the set of all possible states of the network (see Eq. (3)).

$P\left(w_1 w_2 \ldots w_n\right) \approx \prod_i P\left(w_i \mid w_{i-k} \ldots w_{i-1}\right)$            (3)

In other words, at a high level, the attention network has the capability of highlighting important information in the input and suppressing extraneous noise for the sake of the neural network's analysis. This means that it is concentrated on more relevant parts of the input instead of irrelevant parts during the prediction task. Although this attention network does make it possible to gather knowledge at the human level, in this situation, attention is required. Thus, the attention network described in this case may catch the nature of the input series in a single hidden state.

Here, the pre activation residual has been modified. The output of attention module H is given using Eq. (4):

$H_{i, c}=M_{i, c}(x) \times T_{i, c}(x)$                     (4)

where, ranges are the overall spatial locations and the channel index. It is possible to train the entire structure end-to-end. When it comes to backward propagation, the focus filter can not only act as a feature selector but can also be used as a part of the step-size control mechanism. In the soft mask section, the gradient of the mask for the input feature can be calculated using Eq. (5):

$\frac{\partial M(x, \theta) T(x, \theta)}{\partial \phi}=M(x, \theta) \frac{\partial T(x, \theta)}{\partial \phi}$                  (5)

where, the $\theta$ is the mask section parameters and the N is the trunk section parameters. Attention Modules could be powerful against noise labels due to this property. The branches of the mask can prevent incorrect gradients (from noisy labels) to update trunk parameters.

Similarly, if it can establish a soft mask unit with identical mapping, the performance should not be worse than that of its counterpart without attention, according to the opinion in residual learning. Therefore, the output of the attention module has been modified using Eq. (6).

$H_{i, c}(x)=\left(1+M_{i, c}(x)\right) \times F_{i, c}(x)$                (6)

where, M(x) ranges from [0, 1], with M(x) approximating 0, H(x) will approximate original features F(x). Thus, this approach can be named as attention to residual learning.

3.3 Proposed method

Many authors provided many methods for detecting and identifying the finger and palm veins, which have been described in Table 1. This work proposed a model of finger vein image identification and verification based on a deep regional learning (DRL) approach using an attention mechanisms (AM) model, shown in Figure 2. It mainly has three different major phases. The first phase is the globalized finger vein lines image construction based on the Globalized Finger Vein Lines Extraction Based (GMFPI) method. In this stage, we used our second model of double stages of feature extraction-based GFPMI for colored finger vein identification [30, 31] to automatically generate a perfect globalized finger vein line image that will be used for the next stage. The next stage is the feature extractor, which uses a convolutional neural network to extract the main feature map using the original finger vein images. The third stage uses the attention mechanisms by using the two generated maps (original and globalized line feature maps) to generate the final deep attention features map that will be used for the final stage, which is the classification stage.

2.png

Figure 2. Deep regional learning approach-based attention mechanisms model proposed system

3.3.1 Globalized finger vein lines image constriction-based (GMFPI)

This stage has been proposed in our second model of double phases of feature extraction using GFPMI for colored finger vein detection [31, 32].

3.png

Figure 3. (a) Original finger vein image, (b) localized finger vein image mask, (c) localized finger vein image, (d) gmpi map, (e) projected gmpi on the orginal finger vein image, (f) extracted finger vein lines, (g) localized grayscale featured finger vein image, (h) localized colored featured finger vein image

It has many steps; our first model proposed a method based on using unsupervised learning based on an optimized k-means clustering approach to generate a perfect localized binary image mask, which will be used later for the next step, as is shown in Figure 3 (b). Based on the localized binary mask, the localized finger vein image is generated, as shown in Figure 3 (c). The second step is the localized finger vein feature extraction. In this step, the finger vein lines are extracted based on the proposed globalized finger map as shown in Figure 3 (d), and they are projected onto the localized finger vein image as shown in Figure 3 (e). Finally, the finger vein feature extraction model based on the globalized feature orientation map is used to extract the finger vein lines, as shown in Figure 3 (f). The extracted finger vein lines were projected onto different finger vein image versions (grayscale and color) generated previously by our second model [31], as shown in Figures 3 (g) and (h), respectively.

3.3.2 Deep regional learning model

In our proposed model, we have eight differentiable layers of varying weights; the first five of these are considered convolutional, and the remaining layers are fully connected. The last completely connected layer of the network outputs values, which are fed to a 2-way softmax layer, which generates a probability allocation for the two-class labels. The statistical models we have developed make our network fully use the multinomial logistic regression goal, which is equal to increasing the average through cases of training of the log probability of the right label under the allocation of prediction. In the former, the kernels of the second, fourth, and fifth layers are not related to each other, and hence only the kernels of the next and subsequent layers are involved.

In the third and second convolutional layers, all kernel maps are connected. Moreover, neurons in fully connected layers and former layers are connected. The first and second layers of convolutional neural networks are followed by response-normalization layers. The max-pooling layer has been followed by both the fifth convolutional and response-normalization layers, which choose both response-normalized and pooled as input, and the first convolutional layer with filters based on 256 kernels in 5×5×48 sizes. Furthermore, the same thing was implemented on the third, fourth, and fifth layers. The nonlinearity of ReLU is implemented in the results produced by fully connected as well as convolutional layers. The first convolution layer performs a filtering operation with 96 convolution kernels of 11×11×3 with a stride of 4 pixels on the 227×227×3 input image. This is the number of neurons that are further apart from one another than they need to allow for more signal overlap to detect targets in a kernel. The second convolutional layer is connected to other convolutional layers without any intervention in both layers' pooling and normalization. The third convolutional layer consists of 384 kernels, and it is linked to the second convolutional layer outputs, which are based on normalized and pooled layers with a size of 3×3×256. The number of fourth and fifth convolutional layers is 384 and 256 kernels, with a size of 3×3×192 and 3×3×192, respectively. Finally, each fully connected layer has 4096 different neurons. The whole structure of our Deep Regional Learning Approach-based Attention Mechanisms Model is described in Table 2.

Table 2. Deep regional learning model structure description

The designed network has an input layer (I1) that has an input size of 227×227×3. The image normalization process is applied to each input. The second layer of our design is (C1), which is the first convolutional layer with a kernel size of 11×11×3 using 96 kernels with stride [4 4], padding [0 0 0 0]. The third layer is the R1, which is the ReLU. The fourth layer is the attention layer (A1). The fifth layer in our design is the normalization layer, followed by the sixth layer, which is P1. P1 is the first pooling layer in our design, with a 3x3 kernel size, stride [2 2], and padding [0 0 0 0]. The next layer (seventh layer) is C2, which is the second convolutional layer. This layer contains 256 kernels, each with a size of 5x5x48 and stride [1 1], as well as padding [2 2 2 2]. The next layer is R2, which is the second ReLU. The design is followed by A2, which is the ninth layer. The ninth layer is the second attention layer, followed by the normalization layer (N2) and max pooling (P2) with a 3x3 kernel size, stride [2 2], and padding [0 0 0 0]. C3, C4, and C5 are the next three convolutional layers, with 384, 384, and 256 kernels, respectively, using 3x3x256, 3x3x192, and 3x3x192 kernel sizes. Both are used with stride [1 1] and padding [2 2 2 2]. The convolutional and max-pooling layers are both followed by the ReLU and Attention Model layers. The last max-pooling layer is P6, which is used with a 3x3 kernel size, stride [2 2], and padding [0 0 0 0]. Finally, the fully connected layer with a feature size of 4096 is followed by the response layer (R7), response locality unit (ReLU), and attention layer (A8). A dropout of 50% is used with ReLU (D8). Another fully connected layer (F9) is used with the ReLU, Attention, and Dropout layers.

3.3.3 Deep attention model for globalized finger vein feature map extraction and classification

The stacked of residual learning differs from our attention residual learning [31]. The residual learning of original ResNet is performed using Eq. (7):

$H_{i, c}(x)=x+F_{i, c}(x)$                (7)

Approximates the residual function, in our formulation, F_i,c(x) represents the features that are extracted based on deep convolutional networks. The key lies in our mask branches M(x). They work as feature selectors that improve features and eliminate noises from trunk features.