Residential Access Control Facial Recognition Based on Improved ResNet Computing Model

With the rapid development of computer science and artificial intelligence, people's demand for life safety and convenience is increasing day by day. Under this trend, residential access control systems have received widespread attention as an important component of safe and convenient living. Facial recognition technology, known for its non-contact and high efficiency, has become an ideal choice for improving the performance of access control systems. However, current static facial recognition technology faces many challenges in practical applications, especially in terms of recognition accuracy in dynamic environments [1-3].

The main research challenge focuses on facial recognition in dynamic environments, such as multiple poses, changes in facial expressions, and recognition performance under different lighting conditions. These factors greatly increase the complexity of facial recognition in access control systems, affecting the accuracy and stability of recognition. For example, changes in profile or facial expressions may lead to recognition failure, reducing system efficiency and user experience. Additionally, recognition under different lighting conditions is also a technical challenge, as changes in light can significantly affect recognition results [4-6].

To address these challenges, a residential access control facial recognition method based on an improved ResNet computing model is proposed. This method initially introduces CNNs and ResNet, then enhances the loss function and incorporates joint loss to improve the model's performance in handling multiple poses and side face recognition. Finally, by refining the identity mapping of ResNet, the model's ability to recognize faces in complex environments is enhanced. The overall structure of the study includes four parts. The first part summarizes the relevant research achievements and shortcomings of neural networks and face recognition both domestically and internationally. In the second part, the study begins with the proposal of CNNs and ResNet, then goes on to improve the loss function and introduces joint losses based on this foundation. Subsequently, ResNet is further refined, and a ResNet identity mapping model is successfully constructed.

In the third part, the research experiment compares and analyzes the proposed improved ResNet. The fourth part summarizes the experimental results, highlights the shortcomings of the research, and proposes future research directions.

This study introduces a computational model utilizing CNNs and ResNet, enhancing the loss function through the incorporation of joint losses. Given that facial recognition is frequently compromised by variations in pose and facial expressions, the ResNet model has been further refined, culminating in the successful development of a ResNet identity mapping model.

3.1 Construction of computational models for CNNs and ResNet

The objective of this research is to adapt the ResNet computing model for use in facial recognition within residential access control systems. Dynamic facial recognition is critical in these systems, as it must efficiently process the continually changing facial expressions and angles captured in video streams. The deployment of deep learning techniques, particularly those founded on ResNet models, effectively meets these challenges.

The implementation of the system unfolds through several essential steps. Initially, facial detection and alignment are conducted to accurately extract facial images from video streams. Subsequently, feature extraction takes place. Regardless of the technology employed, the precise capture of facial features is imperative for the subsequent recognition phase. The process concludes with the confirmation of identity by comparing the similarity of facial features across various images. This comprehensive procedure is illustrated in Figure 1.

30-1.png

Figure 1. Residential access control system dynamic face recognition

The ongoing enhancements to CNN architecture have significantly advanced various recognition tasks. The structure of a CNN is reminiscent of the human visual system, making it exceptionally capable of processing two-dimensional and three-dimensional images, and adept at learning and extracting features [16, 17]. A CNN comprises a feature extractor and a classifier. The input data is processed layer by layer, through a series of convolutional and max pooling layers. The convolutional layer is tasked with feature extraction, while the max pooling layer reduces the dimensions of these features. Features are transmitted between network layers, and output feature maps are generated through either linear or nonlinear activation functions, as depicted in Eq. (1).

$x_j^l=f\left(\sum_{i \in M_j} x_i^{l-1} * k_{i j}^l+b_j^l\right)$     (1)

where, $x_j^l$ represents the input, $x_i^{l-1}$ represents the output of the previous layer, and $b_j^l$ represents the bias term. The pooling layer is used to reduce the size of the input feature map, thereby reducing the computational burden on subsequent layers of the network. By performing scanning operations on the input feature map, the pooling layer calculates the size reduction operation for each window, which may include taking the maximum value within the region or average pooling. These pooling operations help extract important features while reducing data dimensions, as shown in Eq. (2).

$\left\{\begin{array}{l}Z_j^l=\max \left\{y_i^{l-1}\right\}=\max _{i \in K_i} y_i^{l-1} \\ Z_j^l=\operatorname{mean}\left\{y_i^{l-1}\right\}=\frac{1}{\left|K_j\right|} \sum_{i \in K_j} y_i^{l-1}\end{array}\right.$       （2）

where, $K_i$ represents the selected $i$ th region. This area covers the $j$ th neuron in the first layer and its corresponding neuron set $y_i^{l-1}$ in the previous layer, which is the same size as the pooled nucleus. The backend of a network structure typically consists of fully connected layers, which convert extracted features into outputs for classification or regression tasks. To improve efficiency and performance, these dense layers are often replaced by average pooling layers, especially in parameter dense networks. In addition, the contrastive loss function plays a crucial role in complex tasks such as facial recognition, as it improves the recognition ability of the model by distinguishing positive and negative sample pairs. The specific formula is detailed in Eq. (3).

$\begin{gathered}L=y_{i j} \max \left(0,\left\|f\left(x_i\right)-f\left(x_j\right)\right\|_2-^{+}\right) \\ +\left(1-y_{i j}\right) \max \left(0^{-},-\left\|f\left(x_i\right)-f\left(x_j\right)\right\|_2\right)\end{gathered}$     (3)

The most commonly used loss function currently is Softmax loss, which is widely popular due to its simplicity and no need for additional hyperparameters. The loss function maps the extracted features to the range of 0 to 1 and performs recognition classification based on the obtained probability, as shown in Eq. (4).

$L_s=-\frac{1}{N} \sum_{i=1}^N \log \left(\frac{e^{f_{it}}}{\sum_{j=1}^c e^{f_j}}\right)$       (4)

where, C represents the total number of different categories, while N refers to the sample size for each batch of processing. Due to the limitations of Softmax in dealing with uneven distribution within classes and distinguishing similar samples, center loss is introduced to reduce intra class differences. By combining Softmax loss and center loss, samples can be trained simultaneously to achieve intra-class compactness and inter class dispersion, as shown in Eq. (5).

$\left\{\begin{array}{l}L_C=\frac{1}{2} \sum_{i=1}^N\left\|x_i-C y_i\right\|_2^2 \\ L=L_s+\lambda L_c=-\frac{1}{N} \sum_{i=}^N \log \left(\frac{e^{f_{yi}}}{\sum_{j=1}^C e^{f_j}}\right)+\frac{\lambda}{2} \sum_{i=1}^N\left\|x_i-C_{y_u}\right\|_2^2\end{array}\right.$   (5)

where, the center point $C_{y_i}$ of each category is adjusted with the update of the feature $x_i$ of each sample, to reduce the distance between samples within the class.

3.2 Construction of joint loss function model for ResNet

To adapt to dynamic facial recognition environments, a fusion loss function is further proposed, which combines Softmax loss, center loss, and marginal loss to reduce intra class differences and increase inter class separation. Improvements have been made on the ResNet34 architecture, such as reducing the size of the convolutional kernel to extract more facial information, and replacing the original activation function. In addition, ResNet identity mapping layer has been added to improve the recognition rate of lateral faces. In multi class classification, Softmax maps multiple outputs to the range of 0 to 1, as shown in Eq. (6).

$S_i=\frac{e^{V_i}}{\sum_j e^{V_j}}$   (6)

The Softmax layer structure process can be represented as shown in Figure 2.

30-2.png

Figure 2. Softmax structure

Figure 2 illustrates the Softmax operation, which converts the initial response of the network into a distribution probability. To fully understand Softmax loss, it is necessary to first understand the cross entropy function, which is a combination of Softmax function and cross entropy. Cross entropy evaluates the difference between the actual probability output q and the predicted probability output p, with lower values indicating higher similarity between the two [18]. The definition of cross entropy $H(p, q)$ for variable x is shown in Eq. (7).

$H(p, q)=\sum_x p(x) \log \frac{1}{q(x)}=-\sum_x p(x) \log q(x)$     (7)

30-3.png

Figure 3. Softmax loss classification flow chart

When performing image classification in CNNs, the training samples first pass through the network and output feature maps through fully connected layers. This feature map is multiplied by the weight matrix $W=\left\{W_1, W_2, W_3, \ldots, W_{K-1}, W_K\right\}$ of the classification layer to generate scores for each category. Next, these scores are processed using the Softmax function to generate normalized category probabilities, and the cross entropy loss function is calculated. Softmax is particularly skilled at amplifying small differences between categories, which is crucial in the optimization process. The specific workflow is shown in Figure 3.

In addition to precise classification of known categories, evaluating similarity is essential. Merely classifying known species accurately does not suffice to enhance generalization. It is crucial to ensure consistency within samples of the same type and discrimination between different types. However, these aspects are not directly optimized by Softmax loss. To address this, a new joint loss strategy combining center loss with Softmax loss has been developed, as detailed in Eq. (8).

$\left\{\begin{array}{l}L_C=\frac{1}{2} \sum_{i=1}^N\left\|f_i-c_{y i}\right\|_2^2 \\ L=L_S+\lambda L_C=-\frac{1}{N} \sum_{i=1}^N \log \frac{e^{W_{y i}^T f_i+b_{y i}}}{\sum_{j=1}^K e^{W_j^T f_i+b_j}}+\frac{\lambda}{2} \sum_{i=1}^N\left\|f_i-c_{y i}\right\|_2^2\end{array}\right.$      (8)

where, $C_{y i}$ represents the center point of class $y i$, while $\lambda$ is a hyperparameter that balances different loss types. After combining Softmax loss and center loss, intra-class consistency has been significantly improved. However, relying solely on Softmax loss to constrain inter class differences is insufficient, as it mainly promotes feature differentiation. Therefore, the study introduced a joint supervision method of marginal loss and Softmax loss, as shown in Eq. (9).

$\left\{\begin{array}{l}L=L_S+\lambda L_{M a r} \\ L_{M a r}=\frac{1}{N^2-N} \sum_{i, j, i \neq j}^N\left(\xi-y_{i j}\left(\left\|\frac{f_i}{\left\|f_i\right\|}-\frac{f_j}{\left\|f_j\right\|}\right\| \frac{2}{2}\right)\right)\end{array}\right.$   (9)

where, $f_i$ and $f_j$ represent the vectors of the ith and jth in the batch. $\xi$ represents the fault tolerance range beyond the classification decision boundary. Next, a fusion loss function is proposed based on the concepts of Softmax loss, center loss, and marginal loss. Softmax is responsible for classification accuracy, while joint loss adjusts inter class distance. It sets a standard for joint loss using the class centers determined by the center loss, and introduces penalty terms for these pairs to increase the total loss, as shown in Eq. (10).

$\left\{\begin{array}{l}L=L_S+a L_C+\beta L_J \\ L_J=\sum_{i, j=1}^K \max \left(\left\|c_i-c_j\right\|_2^2-M, 0\right)\end{array}\right.$     (10)

where, $a$ and $\beta$ are the key parameters for adjusting the center loss and joint loss, respectively. $c_i$ and $c_j$ represent the core points of $i$ and $j$ classes, while $M$ is the set minimum distance threshold.

3.3 Identity mapping model based on improved ResNet

Due to the influence of multiple poses and facial expressions on facial recognition, it is necessary to improve ResNet34 by removing the max pooling layer after the first convolutional layer to preserve more localization information. To stabilize the output distribution and accelerate the learning process, a batch normalization layer is added at the entrance of the residual unit, as shown in Figure 4. Batch normalization maintains the excitation values in the sensitive area of nonlinear functions by standardizing input values, avoiding gradient vanishing, and improving training speed.

30-4.png

Figure 4. Improvement of residual module

Before building a deep learning network, it is essential to initialize the weight parameters, W, of the neural network correctly. Training these models involves iterative updates to W. As the number of layers in the network increases, the risk of encountering vanishing or exploding gradients during the optimization process also rises, making proper weight initialization critical. While weights W and bias b in linear and logistic regression are typically initialized to zero, this approach can lead to failure in neural networks, particularly when hidden layers are involved, as noted in references [19, 20].

30-5.png

Figure 5. Neural network structure diagram

Figure 5 underscores the importance of proper weight initialization, illustrating why weights must not be set to zero. Moreover, traditional gradient descent optimization algorithms often fall short with large-scale datasets. In response, the Adam algorithm is introduced, which integrates the benefits of the momentum method and the RMS Prop algorithm to provide rapid and efficient optimization. Each iteration of Adam uses the exponential weighted moving average of the squared small batch random gradient in the momentum variable V and RMS Prop, as detailed in Eq. (11).

$\left\{\begin{array}{l}V_t \leftarrow \beta_1 V_{t-1}+\left(1-\beta_1\right) g_t \\ S_t \leftarrow \beta_2 S_{t-1}+\left(1-\beta_2\right) g_t \odot g_t\end{array}\right.$      (11)

Then bias correction on variables V and S is performed, as shown in Eq. (12).

$\left\{\begin{array}{l}\hat{V}_t \leftarrow \frac{V_t}{1-\beta_1^t} \\ \hat{S}_t \leftarrow \frac{S_2}{1-\beta_2^t} \\ g_t^{\prime} \leftarrow \frac{\eta \hat{V}_t}{\sqrt{\hat{S}_t+\epsilon}}\end{array}\right.$    (12)

where, $\eta$ represents the initial learning rate greater than zero, but $\in$ is a fixed constant. To avoid overfitting during training, normalization is applied to the loss function by introducing parameters representing model complexity, as shown in Eq. (13).

$\left\{\begin{array}{l}J=J_0+\lambda \sum_w|w| \\ J=J_0+\frac{\lambda}{2} \sum_w w^2\end{array}\right.$   (13)

It is difficult to learn deep facial representations with geometric stability under significant pose changes. Assuming there is a mapping relationship between the sides and the front, the differences between them in the feature space can be connected through consistency mapping, and it is possible to obtain the mapped front features through the side features, thereby improving the recognition ability of the sides. It sets the positive variable as $x_f$ and the lateral variable as $x_p$, treat CNN as a function, and the goal is to find the mapping $M_g$. The residual model is used to obtain Eq. (14).

$\begin{gathered}\phi\left(g X_p\right)=M_g \phi\left(X_p\right) \\ =\phi\left(x_p\right)+\gamma\left(X_p\right) R\left(\phi\left(X_p\right) \approx\left(X_f\right)\right.\end{gathered}$    (14)

At the basic feature output end of the network, a constant residual mapping section is introduced to transform the initial feature representation to achieve the final feature representation, as shown in Figure 6.

30-6.png

Figure 6. ResNet identity mapping example diagram

It can be observed that two red boxes configured with Mish activation functions represent fully connected layers trained independently of the main network. The training objective of this independent branch is to reduce the Euclidean distance between the side and the corresponding front, as shown in Eq. (15).

$\left.\min _{\Theta_R} E \| \phi(x)+y(x)+\Re(\phi(x)) ; \Theta_R\right)-\phi\left(x_f\right) \|_2^2$        (15)

The head rotation estimator is used to calculate the offset angle $y(x)$ between the image and the front. Firstly, key points are extracted through facial alignment, and then the EPNP algorithm is used to estimate the head rotation angle. The rotation angle is mapped to the range of $[0,1]$ through the function $\sigma\left(\frac{4}{\pi} y-1\right)$. The training method is to combine the trained identity mapping module with the main network, and conduct end-to-end overall training, specifically training the identity mapping module after end-to-end training.

A computational model combining CNNs and ResNet was proposed to address the impact of multiple poses and lighting changes on facial recognition technology in residential access control systems. Based on this, the loss function was further improved by introducing joint losses. This improvement led to the construction of the ResNet identity mapping model. The data showed that based on the improved ResNet-34 network model, the proposed model performed well on different datasets. Especially on the LFW, YTF, and SLLFW datasets, the proposed model achieved facial verification accuracy of 99.6% and 96%, respectively. This performance was superior to models that use traditional Softmax loss, center loss, marginal loss, and individual joint loss. In addition, in the experiments on the M-Face and F-Scrub datasets, the model also performed well in recognition and validation testing. In addition, compared with other models, the improved ResNet34 network achieved face recognition accuracy of 99.6% and 95.7% on the LFW and YTF datasets, respectively, and 0.922 and 0.945 on the IJBB and IJBC datasets, demonstrating its significant advantages. In video face recognition analysis, the model demonstrated strong performance in identifying characters in videos, even in situations where the face was heavily skewed. The real-time performance and FPS test results showed that the system met practical requirements, with a speed of 0.39 times that of the original video and an average FPS of 27-29 FPS. In summary, the improved ResNet model performed well in residential access control facial recognition, especially in handling multi pose and side face recognition. However, there is still room for improvement in the accuracy of the model under extreme lighting and certain deflection angles. Future research can focus on further improving the performance of models under these extreme conditions and exploring more efficient algorithms to reduce computational costs, thereby achieving wider applications.