Enhancing Deep Feature Learning and Visual Mining for Effective Detection of Suspicious Activities in Video Surveillance

Video surveillance has become one of the significant applications for public safety. The most challenging research problem in surveillance is the learning and classification of abnormal activities. In this process, classification accuracy and computational scalability are the most significant issues considered in the current video analytics research. Authors in literatures [12-16] attempted to detect or classify loitering, fighting, and abandoned objects based on the finding of suspicious behaviours. Most of these works are performed on offline video surveillance data; in real-time, the kind of post-detection of abnormalities or anomalies in videos may not be helpful. With this fact, the author in literature [17] developed the real-time blob-matching technique by finding the temporal features of blobs and activities. It can detect and classify fighting, theft, loitering, fainting, etc. Findings of the work and limitations are presented in Table 1. Long video event retrieval has been most attractive in real-time video surveillance systems. Text instruction (or caption) for long videos may also give the solution indirectly to help people know abnormal activities instantly. Lu and Grauman [18] implemented the algorithm of video summary generation that can be used to extract selected sub-videos from the long video based on the image quality factors for describing the events happening in the video. Another method is based on the key frame sequence, the event detection technique developed by Wolf [19]. It uses the frames rather than taking the video directly. It shows a significant reduction in video content and improved efficacy in video retrieval. Finding the specific moments from long video faces majorly in the above methods. Wan et al. [20] developed the superframe segmentation algorithm that removes the redundant frame in a video sequence to reduce the computational overheads when classifying video anomalies. Findings and approach details are mentioned below, along with the limitations of the work. The key literature and observations or limitations are presented in Table 1.

Table 1. Key literature of the work and observations

Dimensionality reduction methods playing the crucial role for handling the abnormalities in high-dimensional video surveillance data. Huang et. al. [21] proposed the method in study for reducing the computational complexities by transforming the high-dimensional video data into a low-dimensional notation. They proposed the variational-based subspace (VBS), which basically uses the principal component analysis (PCA) to prune the dimensions from N dimensional to required k-principal components. Central problem of determining k-components in PCA is resolved in VBS by estimation of covariance matrices for the different selections of k values. Finding covariance matrix is very expensive due to its computations are relevant to single value decomposition (SVD). Authors in study [22] faced the challenges of dimensionality reduction and used the methods, independent component analysis (ICA), non-negative matrix factorization (NMF), autoencoders (AE), variational auto encoders (VAE), and PCA. These methods were effectively applied in their work for reducing the dimensions complexities. The ICA transforms the high dimensional data in large scale repositories to linear combinations of statistically independent vectors. The limitation of ICA is unable to separate the Gaussian resources. Thus, NMF is another better option for the subspace learning to reduce the dimensions of large scale high-dimensional data. Benefit of NMF is to find the reduced data features and it also leads to automated the cluster tendency for the targeted high-dimensional data. The problem of NMF is does not yield a unique solution. In recent years, deep learning techniques are the most recommended for obtaining the optimized features. The autoencoders (AE) [23] greatly applied for dimensionality reduction and data reduction; therefore, reduced representation of features obtained that useful optimizing the requirements. Another nonlinear dimensionality reduction, namely, T-distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) [24], is widely used in large scale dimensionality reduction for video data. The t-SNE evaluates similarity in the original high-dimensional space by transforming distances between points into conditional probabilities. For every data point, the algorithm assigns a probability distribution over all other points, where close neighbors receive high probabilities and distant points receive extremely small values. This is done by centering a Gaussian kernel at each point and adjusting its width so that the effective number of neighbors (controlled by perplexity) remains consistent across the dataset. The complete set of these probability distributions forms the high-dimensional similarity structure that t-SNE attempts to preserve when projecting the data into a lower-dimensional space. UMAP and t-SNE offer considerable flexibility, allowing their configurations to be adapted to the characteristics of different datasets. The discussion [25] provides actionable insights to guide users in selecting the most appropriate dimensionality reduction approach and tuning its key parameters for their intended use case. These recommendations act as a practical framework for achieving an effective balance between performance, computational cost, and clarity of the resulting embeddings in a variety of video scenarios. Anomaly detection for suspicious activities is the most promising in video surveillance applications for saving social people. Surveillance cameras and their dynamic activity detections play a significant role in such applications. Some of the researchers [26-28] used unsupervised ideas for anomaly detection. It requires careful classification and handling approaches for abnormalities. Rather than unsupervised techniques, deep-based classification presents impressive abnormalities classifications. Sultani et al. [29] developed the 3D convolutional network (C3D), which extracts the Spatiotemporal features and computes the score of anomalies by a 3-layer fully connected network. This leads to one crucial problem: video segmentation scalability before feature extraction due to the massive size of the video frames. Zahid et al. [30] resolved the video segmentation problem with the ensemble technique of IBaggedFCNet, in which bagging imposes stringent segmentation and uses the Inception-v3 deep classifier technique for the video classification. With the broader availability of video data from surveillance cameras and social platforms, learning the importance of trained video has become tedious. Finding an efficient automated violence detection system concerned with the vital parameters of computational efficiency and accuracy is the most needed for real-time video surveillance applications. Violence detection for more extensive videos can be optimized in references [31]; here, the keyframe selection process plays a crucial role in selecting the optimal number of frames to reduce the computational overheads. It employs ensemble classification models using long short-term memory (LSTM), bidirectional-LSTM (Bi-LSTM), and gated recurrent unit (GRU) models to enable good video classification results. Manisha Mudgal et al. proposed a smart and intelligent real-time video monitoring system [32] for efficient detection of activities, slapping, hitting, punching, etc. It models the activities using the Gaussian Mixture Models (GMMs); It also develops the universal attribute models (UAM) for deriving the super action vector (SAV) that helps to improve the accuracy of classification. Thus, GMM and UAM are majorly defined for modeling the SAVs and making the classification using SVM. Xu [32]. Cong et al. [33] another intelligent video surveillance system using the deep learning approach was also proposed. Faster R-CNN with Inception ResNet V2 was discovered to achieve the best accuracies for classifying real-time activities. State-of-the-art techniques [34-36] use the trained video frames for extraction features and learning of one or more activities, including normal and abnormal activities. For testing data, suppose it matches any abnormal activities, then mark it as abnormal; otherwise, mark it as normal. These deep learning techniques are the most prominent in computer vision and other object detection and activity recognition applications. These works recommend the two-stage work for the video classification tasks. Sun et al. [37] proposed the deep one-class model (DOC), the end-to-end deep learning model. One-class support vector machine and deep CNN are integrated to optimize the accuracy and loss parameters. Lamani et al. [38] developed an efficient hybrid technique for human action recognition, in which lightweight residual 3D CNN was built for handling the computational hurdles and effective human action recognition. Existing deep learning techniques more beneficial for extraction of frames learned features. The real-time video capturing systems produce high-quality videos (i.e., massive frames). The selection of the frames and high dimensional deep features was initially extracted. High-dimensional features require the equivalent low-dimensional manifold subspace learning to reduce the complexities. The proposed work focuses on subspace learning techniques for the frame's deep features. In the high dimensional deep features, the data sparsity problem occurred, significantly impacting the classification rate. Thus, it needed to obtain the equivalent low-manifold subspace to reduce the effect of data sparsity problems. There are wider chances to improve the efficacy-related parameter values for the video classification results. Detailed procedural details and algorithm descriptions of the proposed work are presented in the following section.

The experiments are carried out on the 14 classes (Abuse, Arrest, Arson, Assault, Burglary, Explosion, Fighting, Normal Videos, Road Accidents, Robbery, Shooting, Shoplifting, Stealing, Vandalism) of benchmarked video dataset, UCF-Crime which publicly free available in reference [38]. After initiating the pre-processing, the video of each of the classes is divided into different frames (or images) and organized according to the train labels and test labels. The deep features are extracted using the Densenet, GoogLeNet, MobileNet, EfficientNetB0, Resnet, and VGG16 models. Further, these features are mapped into the lower-dimensional manifolds by deriving the principal components (or Eigenvectors) from the Eigen decomposition of the Laplacian Matrix. These reduced dimensional features are obtained in our proposed SDC-VM. The results of existing deep-based-classifier models (i.e., Deep Random Forest, Deep-SVC, and Deep-kNN) and proposed spectral-based-deep classifier video models (SDC-VM-Random Forest, SDC-VM-SVC, SDC-VM-kNN) are illustrated in Table 2. The evaluation parameters of accuracy, precision, recall, and f-score measure are depicted in reference [39].

Table 2. Performance comparison for the existing deep-based classifier models and proposed SDC-VM based classifier models

From these experimental values of performance scores, it was noted that SDC-VM achieved a reasonable classification rate compared to existing deep-based classifier models due to the reduction of the effect of the data sparsity problem in proposed spectral-based deep classifier models. The accuracy value improved at a rate of 10 to 12% in the proposed models compared to existing models. Similarly, the precision, recall, and f-score values improved at the rate of 6% to 10%, 5% to 11%, and 4% to 12%, respectively. These are comparative illustrations among the methods shown in Figure 1 to Figure 13.

The receiver operating characteristics (ROC) curves are used to depict the classifier accuracy for the multi-class video data. In the experimental work, 14 classes of the UCF-Crime classification data evaluated with the ROC curves. The value of the area under the curve (AUC) indicates the classifier's accuracy. The AUC values are between 0 and 1. Higher values indicate good classifier accuracy. Figure 14 to Figure 19 shows the ROC curves with estimated values of AUC for the proposed models of SDC-VM-KNN, SDC-VM-RF, and SDC-VM-SVC underlying the deep features extraction with the Densenet, GoogLeNet, MobileNet, EfficientNetB0, Resnet 50, and VGG16. The ROC analysis inference achieved good video anomaly classification results using the kNN underlying SDC-VM with all six variants of deep models. Mostly, the AUC scored above 0.5 in all variants of the proposed SDC-VM models.

tu_pian_2.png

Figure 2. Accuracy between Deep-KNN and SDC-VM-KNN

tu_pian_3.png

Figure 3. Accuracy between Deep-RF and SDC-VM-RF

tu_pian_4.png

Figure 4. Accuracy between Deep-SVC and SDC-VM-SVC

tu_pian_5.png

Figure 5. Precision between Deep-KNN and SDC-VM-KNN

tu_pian_6.png

Figure 6. Precision between Deep-RF and SDC-VM-RF

tu_pian_7.png

Figure 7. Precision between Deep-SVC and SDC-VM-SVC

tu_pian_8.png

Figure 8. Recall between Deep-KNN and SDC-VM-KNN

tu_pian_9.png

Figure 9. Recall between Deep-RF and SDC-VM-RF

tu_pian_10.png

Figure 10. Recall between Deep-SVC and SDC-VM-SVC

tu_pian_11.png

Figure 11. F-Score between Deep-KNN and SDC-VM-KNN

tu_pian_12.png

Figure 12. F-Score between Deep-RF and SDC-VM-RF

tu_pian_13.png

Figure 13. F-Score between Deep-SVC and SDC-VM-SVC

tu_pian_14.png

(a) SDC-VM-KNN ROC curve

tu_pian_15.png

(b) SDC-VM-RF ROC curve

tu_pian_16.png

(c) SDC-VM-SVC ROC curve

Figure 14. SDC-VM ROC curves using DenseNet-201 model

tu_pian_17.png

(a) SDC-VM-KNN ROC curve

tu_pian_18.png

(b) SDC-VM-RF ROC curve

tu_pian_19.png

(c) SDC-VM-SVC ROC curve

Figure 15. SDC-VM ROC curves using GoogLeNet (Inception-V3) model

tu_pian_20.png

(a) SDC-VM-KNN ROC curve

tu_pian_21.png

(b) SDC-VM-RF ROC curve

tu_pian_22.png

(c) SDC-VM-SVC ROC curve

Figure 16. SDC-VM ROC curves using MobileNet model

tu_pian_23.png

(a) SDC-VM-KNN ROC curve

tu_pian_24.png

(b) SDC-VM-RF ROC curve

tu_pian_25.png

(c) SDC-VM-SVC ROC curve

Figure 17. SDC-VM ROC curves using EfficientNetB0 model

tu_pian_26.png

(a) SDC-VM-KNN ROC curve

tu_pian_27.png

(b) SDC-VM-RF ROC curve

tu_pian_28.png

(c) SDC-VM-SVC ROC curve

Figure 18. SDC-VM ROC curves using ResNet-50 model

tu_pian_29.png

(a) SDC-VM-KNN ROC curve

tu_pian_30.png

(b) SDC-VM-RF ROC curve

tu_pian_31.png

(c) SDC-VM-SVC ROC curve

Figure 19. SDC-VM ROC Curves using VGG16 model

Figure 20 presents a comparative evaluation of SDC-VM against t-SNE and UMAP using KNN, Random Forest, and SVC classifiers on MobileNet deep features for the CamNuvem dataset. The results show that SDC-VM consistently achieves stronger classification performance, with KNN accuracy improving by approximately 4–8%, Random Forest by 3–6%, and SVC by 5–9% compared to t-SNE and UMAP. SDC-VM also provides more compact clusters, reflected in lower intra-class variance (10–15% reduction) and improved global–local structure preservation. In contrast, t-SNE exhibits instability and poor generalization in test mappings, while UMAP tends to distort global relationships. Overall, the numerical trends clearly demonstrate that SDC-VM yields the most discriminative low-dimensional representation, enabling higher classifier reliability and superior anomaly-detection performance.

tu_pian_32.png

KNN-PCA-ROC

tu_pian_33.png

KNN-TSNE-ROC

tu_pian_34.png

KNN-UMAP-ROC

tu_pian_35.png

RF-PCA-ROC

tu_pian_36.png

RF-TSNE-ROC

tu_pian_37.png

RF-UMAP-ROC

tu_pian_38.png

SVC-PCA-ROC

tu_pian_39.png

SVC-TSNE-ROC

tu_pian_40.png

SVC-UMAP-ROC

Figure 20. Combative analysis of proposed SDC-VM based classifier methods with other dimensionality methods (deep features extracted from MOBILENET model) for the CamNuvem dataset

tu_pian_41.png

KNN-PCA-ROC

tu_pian_42.png

KNN-TSNE-ROC

tu_pian_43.png

KNN-UMAP-ROC

tu_pian_44.png

RF-PCA-ROC

tu_pian_45.png

RF-TSNE-ROC

tu_pian_46.png

RF-UMAP-ROC

tu_pian_47.png

SVC-PCA-ROC

tu_pian_48.png

SVC-TSNE-ROC

tu_pian_49.png

SVC-UMAP-ROC

Figure 21. Combative analysis of proposed SDC-VM based classifier methods with other dimensionality methods (deep features extracted from DENSENET model) for the CamNuvem dataset

Figure 21 presents the experimental analysis comparing the proposed SDC-VM dimensionality-reduction technique with t-SNE and UMAP using KNN, Random Forest, and SVC classifiers on DenseNet deep features extracted from the CamNuvem dataset. The results demonstrate that SDC-VM consistently produces more discriminative low-dimensional embeddings, leading to noticeable improvements in classification performance. In particular, SDC-VM shows an average accuracy gain of 5–10% with KNN, 4–7% with Random Forest, and 6–11% with SVC when compared to t-SNE and UMAP. This improvement is attributed to SDC-VM’s superior preservation of both global and local structures, reflected in lower reconstruction error and better neighborhood retention, while t-SNE suffers from non-generalizable mappings and UMAP tends to distort global geometry. The stability and structural clarity of SDC-VM embeddings result in more compact class clusters, enabling the classifiers to form clearer decision boundaries. Overall, the experiments confirm that SDC-VM significantly enhances the performance of multiple classifiers for DenseNet features, further validating its robustness and effectiveness for video anomaly detection on the CamNuvem dataset.

The experimental results illustrated in Figures 22 and 23 demonstrate the consistent superiority of the proposed SDC-VM dimensionality-reduction framework over t-SNE and UMAP across multiple classifiers on the UCF-Crime dataset. Using deep features extracted from both MobileNet and DenseNet, SDC-VM shows significant improvements in classification performance, maintaining 5–12% higher accuracy with KNN, 4–9% with Random Forest, and 6–14% with SVC compared to the baseline methods. These gains highlight SDC-VM’s stronger ability to preserve intrinsic structure in high-dimensional video representations, enabling more compact and well-separated feature clusters that facilitate improved classifier decision-making. In contrast, t-SNE exhibits unstable embeddings and lacks a reliable transform for unseen data, while UMAP sometimes compromises global geometry, resulting in weaker inter-class boundaries. The observed reductions in reconstruction error and enhanced preservation of neighborhood relationships further confirm that SDC-VM generates more semantically meaningful embeddings tailored for anomaly detection.

tu_pian_50.png

KNN-PCA-ROC

tu_pian_51.png

KNN-TSNE-ROC

tu_pian_52.png

KNN-UMAP-ROC

tu_pian_53.png

RF-PCA-ROC

tu_pian_54.png

RF-TSNE-ROC

tu_pian_55.png

RF-UMAP-ROC

tu_pian_56.png

SVC-PCA-ROC

tu_pian_57.png

SVC-TSNE-ROC

tu_pian_58.png

SVC-UMAP-ROC

Figure 22. Combative analysis of proposed SDC-VM based classifier methods with other dimensionality methods (deep features extracted from MOBILENET model) for the UCF-Crime dataset

tu_pian_59.png

KNN-PCA-ROC

tu_pian_60.png

KNN-TSNE-ROC

tu_pian_61.png

KNN-UMAP-ROC

tu_pian_62.png

KNN-UMAP-ROC

tu_pian_63.png

RF-TSNE-ROC

tu_pian_64.png

RF-UMAP-ROC

tu_pian_65.png

SVC-PCA-ROC

tu_pian_66.png

STSNE-ROCVC-

tu_pian_67.png

SVC-UMAP-ROC

Figure 23. Combative analysis of proposed SDC-VM based classifier methods with other dimensionality methods (deep features extracted from DENSENET model) for the UCF-Crime dataset

tu_pian_68.png

tu_pian_69.png

tu_pian_70.png

Figure 24. Runtime comparison

tu_pian_71.png

tu_pian_71.png

tu_pian_73.png

Figure 25. Memory comparison

tu_pian_74.png

tu_pian_75.png

tu_pian_76.png

Figure 26. Reconstruction error, information preservation, and trustworthiness analysis

The experimental evaluation presented in Figures 24-26 offers a comprehensive analysis of the proposed SDC-VM framework in comparison with t-SNE and UMAP across multiple performance dimensions. The runtime comparison (Figure 24) shows that SDC-VM consistently achieves the lowest execution time, operating 25-40% faster than UMAP and 40-55% faster than t-SNE, which is critical for large-scale video-feature processing. Similarly, the memory comparison in Figure 25 presents that SDC-VM requires significantly fewer computational resources, consuming 20-35% less memory than UMAP and nearly half the memory footprint of t-SNE, making it highly suitable for real-time and resource-constrained environments. Beyond computational efficiency, Figure 26 demonstrates that SDC-VM also excels in representation quality, yielding the lowest reconstruction error, the highest information-preservation ratio, and superior trustworthiness scores, indicating stronger retention of local and global structures in reduced-dimensional embeddings. These advantages collectively highlight SDC-VM’s ability to generate compact, discriminative feature mappings while maintaining computational efficiency, establishing it as a robust and scalable alternative to conventional nonlinear embedding methods for video anomaly-detection tasks.