Authentication in Liveness Detection Utilizing CNN and MobileViT Algorithm

Several researchers have undertaken numerous experiments looking at video injection attacks using deep learning. Taeb and Chi [9] presented a deepfake detection framework that uses two deep learning models, Xception and MobileNet, to classify fake films from the FaceForensics++ dataset. The models, trained on modified films created by four mainstream approaches (Deepfakes, Face2Face, FaceSwap, and NeuralTextures), achieved accuracies above 90% for the majority of datasets, however performance dipped for NeuralTextures (e.g., MobileNet: 88% accuracy). A voting method that aggregates forecasts from all models increases resilience. The work emphasizes model sensitivity to certain manipulation approaches and the necessity for larger datasets and additional variables, such as inter-frame correlations, for better detection.

Elsaeidy et al. [10] presented a CNN-based system for identifying replay attacks in smart cities based on multivariate time-series data from synthetic datasets sourced from Queanbeyan, Australia. The model was tested on soil management and environmental monitoring datasets (89,566 and 178,211 instances, respectively) and achieved 99.18% accuracy (precision: 99.31%, specificity: 99.26%, sensitivity: 99.11%) on the soil dataset and 98.47% accuracy (precision: 98.56%, specificity: 98.62%, sensitivity: 98.31%) on the environmental dataset. Compared to five cutting-edge approaches, including DRN and ESNC, the suggested model outscored competition across all measures. The work underlines the necessity of modeling the time dimension in attack detection and provides a smart city benchmark dataset for future research.

Zhang et al. [11] examined defense strategies for adversarial perturbations in deep neural networks (DNNs), classifying them as perturbation detection, input modification, stochastic defense, adversarial training, and certified robustness. Adversarial training showed remarkable robustness on datasets such as CIFAR10, but it needed large processing resources. JPEG compression and denoising autoencoders like MagNet obtained great detection rates. Perturbation detection approaches, such as Mahalanobis-based confidence scoring, demonstrated up to 96% accuracy against attacks such as CW and PGD, whereas stochastic defenses based on random noise or activation pruning increased robustness while reducing accuracy loss. The paper emphasizes the trade-offs between robustness and efficiency, recommending for the use of complementary strategies to achieve optimal defense.

Kelly et al. [12] proposed using morphed photos in the training phase to make face recognition systems (FRSs) more resilient against morphing attacks. They detecting discrepancies in identification attributes suggestive of morphing by training on real and augmented photos (morphs and authentic augmented pairings) using a VGG16-based architecture and triplet loss. By reducing the Morph Accept Rate at Equal Error Rate (MAREER) from 30.20% to 20.54% and improving the Differential Equal Error Rate (D-EER) from 6.70% to 5.61% on the FRGC test set, the experiments demonstrated increased resilience. On the ASML validation set, D-EER increased from 5.86% to 4.97%, whereas MAREER decreased from 24.94% to 16.48%. Nevertheless, difficulties with generalization on pose-variation datasets, as PUT, brought to light the necessity of realistic and varied morphing datasets to improve training.

Meena and Tyagi [13] present a deep learning-based approach to picture splicing detection that combines an SVM classifier, ResNet-50 as a feature extractor, and Noiseprint preprocessing to extract noise residuals. The technique outperforms current methods with an average detection accuracy of 97.24%, according to experiments conducted on the CUISDE dataset. The algorithm demonstrated its resilience in detecting spliced photos using camera-specific noise and deep feature extraction by properly classifying 178 out of 180 fabricated images and 175 out of 183 real images. Future research proposes expanding this technique to locate spliced areas in photos and identify splicing in films.

Arora et al. [14] proposed a robust deep learning framework for detecting face spoofing attacks, such as replay attacks, 3D mask attacks, and photo attacks, that employs convolutional autoencoders for dimensionality reduction and feature extraction, followed by classification using pre-trained encoder weights and a softmax classifier. When tested on three benchmark datasets (CASIA-FASD, Replay-Attack, and 3DMAD), the framework obtained high accuracy: 99.17% for CASIA-FASD, 99.03% for Replay-Attack, and 100% for 3DMAD, with a Half Total Error Rate (HTER) of 0%. Cross-database testing yielded good results, proving the framework's robustness and generalizability for detecting spoof faces in biometric systems.

The previous paper [15] proposes a method to enhance the security of liveness detection by extracting human physiological components from computational ghost imaging (CGI) signals. The method achieves a 96.0% correct rate against picture and mask attacks and is resolution-independent, working even at 32×32 pixels.

Meanwhile, authors [16] address the critical issue of securing facial recognition systems against spoofing attacks, which often involve the use of photos, videos, or masks to impersonate legitimate users. The author proposes robust and reliable liveness detection models designed to accurately distinguish between real faces and spoofed ones in various environmental conditions. The paper focuses on leveraging advanced techniques, including deep learning models and feature extraction methods, to enhance the performance of liveness detection. By incorporating dynamic facial cues, such as natural facial movements, and using multimodal approaches, the proposed models aim to improve the reliability of facial recognition systems. The study demonstrates the effectiveness of these models in real-world scenarios, achieving high detection accuracy and resilience to different spoofing techniques. The paper concludes that the proposed liveness detection models offer significant improvements in securing facial recognition systems, making them more resistant to sophisticated attack methods.

According to the papers previously discussed, the most recent developments in liveness detection encompass a variety of biometric modalities, such as iris, fingerprint, and facial recognition. Methodologies used in previous researched including computational ghost imaging, convolutional neural networks, multispectral imaging, and local contrast phase descriptors. The objectives of these methods are to improve accuracy, prevent sophisticated spoofing assaults, and enhance security, thereby demonstrating substantial advancements in the field. Meanwhile, this research aims to investigate alternative deep learning techniques, specifically with regard to liveness detection for authentication.

In contrast to machine learning, deep learning has its own mechanism [17-19]. Deep learning could perform classification without requiring feature extraction; hence, the approach illustrated in Figure 1 is utilized. The specifics of each step are explained in the subsequent subsections.

01.png

Figure 1. The flowchart of the process in detecting the fake faces using deep learning

2.1 Dataset and preprocessing

This research used the video dataset available on Kaggle [20], PARNEC [21] and synthetic. The dataset was processed by changing the video into an image with extension (png).

Pre-processing was done by involving decode to RGB channel the change of image, resize with the size of 224×224 pixel and normalize. Figures 2-4 show samples of every dataset, Table 1 presents the amount of data for each dataset, meanwhile Table 2 defines the information for each dataset.

02.png

Figure 2. Sample of images in dataset [20]

03.png

Figure 3. Sample of images in dataset [21]

04.png

Figure 4. Sample of images in synthetic dataset

Table 1. Datasets faces counts

2.2 Data splitting

The dataset was divided into two segments: 80% for training and 20% for testing, applicable to both augmented and non-augmented data.

2.3 Classification

In this study, a comparison among EfficientNetV2S, EfficientNetV2M, EfficientNetV2L, and MobileViT in the context of deep learning for image analysis was conducted for liveness detection.

2.3.1 EfficientNetV2

EfficientNetV2 [22] presented a new family of convolutional neural networks designed for improved training efficiency and parameter utilization. The models are developed through training-aware Neural Architecture Search (NAS), where optimizations are applied to accuracy, speed, and parameter efficiency. Techniques like progressive learning are introduced, allowing image size and regularization to be adaptively scaled during training, enabling faster convergence without sacrificing accuracy. EfficientNetV2 models are structured with Fused-MBConv layers to replace inefficient operations in early network stages, and a refined scaling strategy is employed to balance the network's complexity across layers.

The models are classified as EfficientNetV2-S, M, L, and XL, with different capacities and processing demands. EfficientNetV2-S is designed for smaller jobs, whereas EfficientNetV2-XL is intended for larger datasets. These models are intended to deliver considerable increases in training speed (up to 11 times faster) and parameter efficiency (up to 6.8 times smaller), as demonstrated on datasets such as ImageNet, CIFAR, and Flowers. Pretraining on larger datasets, such as ImageNet21k, is used to improve performance and achieve competitive accuracy with fewer computational resources. Illustration of EfficientNetV2 depicted in Figure 5.

05.png

Figure 5. Illustration of the architecture of EfficientNetV2

Table 2. Datasets information

2.4.2 MobileViT

MobileViT [23], a lightweight vision transformer designed for mobile devices, combines CNNs and transformers. MobileViT blocks combine global transformer processing with local convolutional representation learning, allowing for efficient visual interpretation while preserving spatial and patch-level order. The network uses fewer parameters and easier training procedures, making it ideal for mobile applications such as object detection and semantic segmentation. Three model variants are presented: MobileViT-XXS, XS, and S, with varying size and complexity. These models outperform classic lightweight CNNs and ViT-based approaches on benchmarks such as ImageNet-1K while being low latency and efficient on mobile devices. Illustration of the MobileViT architecture is presented in Figure 6.

06.png

Figure 6. Illustration of the architecture of MobileViT

2.5 System performance measurement

The performance of the deep learning model was evaluated using the confusion matrix presented in Table 3. The confusion matrix comprised several metrics, including accuracy, precision, recall, F1-score, False Negative Rate (FNR), False Positive Rate (FPR), and Half Total Error Rate (HTER), as described in Eqs. (1)-(7). Four components were utilized to create the matrices: True Positives (TP), False Positives (FP), True Negatives (TN), and False Negatives (FN). The matrices might benefit in predicting both the real and the fake images [24, 25].

Table 3. Confusion matrix

$Accuracy =\frac{(T P+T N)}{(T P+T N+F P+F N)}$       (1)

$Recall=\frac{T P}{(T P+F N)}$        (2)

Precision $=\frac{T P}{(T P+F P)}$           (3)

$F 1-Score =2 \times \frac{( { Precision } \times { Recall })}{( { Precision }+ { Recall })}$           (4)

$F N R=\frac{F N}{F N+T P}$          (5)

$F P R=\frac{F P}{F P+T N}$            (6)

$H T E R=\frac{F N R+F P R}{2}$              (7)

The model's accuracy and robustness in this paper are determined by analyzing the metrics of False Positive Rate (FPR), False Negative Rate (FNR), and Half Total Error Rate (HTER).

2.6 Data augmentation and testing scenario

Both the quantity and quality of training datasets are increased concurrently via data augmentation. This facilitates the improvement of deep learning models [26]. To enhance the diversity of data that utilize in model training, the data augmentation procedure employs ImageDataGenerator to apply a sequence of transformations to the dataset, which includes both real and fake images. This study generally entails data augmentation through several rules, including:

1. Flip: Horizontal, Vertical
2. Crop: 0% Minimum Zoom, 20% Maximum Zoom
3. Rotation: Between -15° and +15°
4. Hue: Between -15° and +15°
5. Saturation: Between -25% and +25%
6. Brightness: Between -15% and +15%
7. Exposure: Between -10% and +10%
8. Blur: Up to 2.5px
9. Noise: Up to 0.1% of pixels

This study investigates liveness detection through deep learning techniques. The employed deep learning algorithm is CNN, utilizing EfficientNetV2S, EfficientNetV2M, EfficientNetV2L, and MobileViT models, applied to the NUAA, Synthetic, and iBeta 1 datasets. The experimental results, as indicated by the parameters of the confusion matrix, error metric, and training duration, indicate that EfficientNetV2S is superior to other approaches among the three datasets. This is not surprising, considering that the advantages of the EfficientNetV2S algorithm are small model size, fast Inference Speed, and ease of Deployment Feasibility.

New datasets with various attack classes could be added for future research enhancements, in contrast to those used in this study. This aims to boost the detection of precision of new attacks that remain undetectable by this research.