A Fully Supervised CNN Architecture for Precise Frame-Level Anomaly Detection in Surveillance Videos

Video anomaly detection is a crucial task in intelligent surveillance systems, aiming to automatically identify unusual or suspicious events in video feeds to ensure public safety [1]. Examples of anomalies include people walking on highways, bicycles or vehicles in pedestrian-only areas, or sudden crowd disturbances. Traditionally, most areas or sudden crowd disturbances. Traditionally, most unsupervised or semi-supervised manner due to the scarcity and unpredictable variety of anomalous events in real-world data [2]. These methods assume access only to "normal" training data and attempt to detect anomalies as deviations from learned normal patterns. While unsupervised methods have shown promise, they face several challenges. First, the definition of an anomaly is often implicit – anything that does not conform to normal training data is flagged, leading to ambiguous anomaly criteria and a tendency to produce false positives for benign novel events. Second, the lack of explicit anomaly examples during training can make it difficult for such models to explain or interpret what triggered an alert, as the decision arises from subtle reconstruction errors or latent feature deviations rather than transparent, learned discriminative features [3]. In recent years, deep learning-based unsupervised techniques have become the dominant research focus in video anomaly detection [4]. Notable approaches include convolutional autoencoder networks that learn to reconstruct normal frames, such that anomalies yield higher reconstruction errors; prediction-based models that learn to predict future frames and signal anomalies when predictions deviate from reality; and adversarial or memory-augmented models that improve the feature representation of normal patterns. As noted, these methods avoid the need for labelled anomalies and have achieved considerable success (frame-level AUC up to ~95% on UCSD Ped2 in the best cases) [5]. However, they often still suffer from some false alarms and missed detections because the boundary between normal and abnormal is learned indirectly. Furthermore, purely unsupervised models generally lack interpretability – it is not straightforward to understand what visual features caused an anomaly alarm, since the models do not output human-interpretable rationale beyond an anomaly score. In contrast, a fully supervised approach to video anomaly detection would utilize labelled examples of anomalies during training, thereby transforming the task into a straightforward binary classification problem at the frame level. This offers the potential for clearer decision boundaries and improved performance, at the cost of requiring annotated anomalies. Due to the impracticality of collecting exhaustive anomaly examples in most real-world scenarios, such supervised methods have been underexplored in literature. Weakly-supervised strategies have emerged as a compromise – for instance, using video-level labels indicating if a clip contains an anomaly (without frame localization) [6], who introduced a large-scale anomaly dataset with only video-level annotations and used multiple instances learning to infer anomalous segments. However, truly frame-specific supervised learning for anomaly detection remains relatively rare in published research.

Unlike traditional anomaly detection approaches on UCSD Ped2, which are usually used for unsupervised or weakly supervised learning, this work focuses on a fully supervised frame-level classification task. We exploit frame-level anomaly annotations to fit a binary classifier. This study is therefore less concerned with presenting a directly comparable anomaly detection approach than in estimating an empirical upper bound on achievable performance when dense supervision is available. The purpose of this setting is to investigate how normal and abnormal patterns in Ped2 can be separated under ideal supervision.

In this paper, we aim to fill this gap by demonstrating the effectiveness of a supervised frame-level convolutional neural network (CNN) for anomaly detection on the UCSD Ped2 benchmark dataset. UCSD Ped2 is a well-known surveillance video dataset consisting of pedestrian walkways, where anomalies (such as bicycles, skateboarders, carts, etc.) are clearly defined and annotated at the frame level [5]. This unique dataset offers an opportunity to train a supervised model, as the ground-truth labels for each frame (normal vs. anomaly) are available. We leverage this fact to train a CNN classifier that directly learns to recognise abnormal frames. By meticulously structuring the dataset and ensuring balanced training, we demonstrate that a relatively simple CNN architecture can achieve outstanding detection results, significantly surpassing the performance reported by unsupervised methods on the same dataset. Our approach emphasises precise labelling and model transparency. Because the model is supervised, we know exactly what constitutes an anomaly in training. Additionally, since the model is a straightforward CNN, its output is directly interpretable for each frame [6].

3.1 Dataset description

In this paper, the UCSD Ped2 dataset is applied, a common benchmark for video abnormality detection. The data set is separately divided into a train set containing only normal behavior and a test set that incorporates both normal and anomalous events. The training set has 16 video clips of pedestrians walking on the walkway with no unusual events to be seen. Of these video clips, the test set contains 12 (a total of 2,010 frames) in which a number of anomalous events occur -- for example, people riding bikes or skateboards across the field or vehicles crossing a frame. There is a radical difference from the previous frame-level approach: by dividing the data into consecutive codes of T frames, rather than treating each frame separately, greater dependence can be taken into account by assigning annotations at the frame level to the code level. We labelled each split with a binary tag based on the annotations given to the frame level: if a split contains at least one anomalous point within intervals specified in Table 1, it is declared as an anomalous clip (1). Otherwise, it is judged an orthodox one--that is to say, normal (0). To the human eye, the above paragraph may seem redundant. But it conveys some useful temporal information that would be lost if we treated each frame as entirely independent.

Table 1. Abnormal frame ranges for test videos

3.2 Data preparation and preprocessing

Each video frame was first transformed into grayscale as this would reduce computational intensity while preserving the fundamental structure of the signal. The frames were then resized to a fixed resolution of 200 × 200 pixels that satisfies all input requirements for our network. Direct resizing might distort the initial aspect ratio, and so we also evaluated aspect-ratio-preserving methods in an ablation study including padding and central cropping.

These choices are transparent for readers The frames were then normalized to further improve learning stability. Finally, the dataset was divided into training and test sets following a temporal split in order to prevent data leaks.

We assembled the processed frames into temporal volumes (tensors) of size (T, H, W, C) where T is the number of frames in each split [14, 15].

Labeled segments from the dataset were utilized during training to overestimate an upper bound. In order to avoid data leakage, the split has been performed at the video level; ensuring that all clips from a particular film are assigned exclusively to one of training and validation sets.

After preprocessing, we split our data into training and validation sets stratified by video segment. The original training set was joined with any correctly labeled splits from the test set, and the data was then split into a training set (80%) and validation (20%). This ensures that the normal class samples are correctly represented in both groups which prevents bias during evaluation by splitting at the segment level, rather than the frame level we ensure that our model is learning from continuous temporal patterns rather than being influenced by leakage of information between training and validation partitions see Figure 1.

Figure 1. Sample preprocessed frames (normal vs. anomaly)

We followed the general supervised learning split with 80% of all frames (normal and anomalous) for training, and 20% for testing in our experimental protocol. We re-emphasize that our approach is not meant to be directly compared with unsupervised baselines such as ConvAE or MemAE, which are generally trained using only normal sequences. Rather, our results provide a ceiling (upper-bound) for performance in supervised frame-level classification.

3.3 Convolutional neural network architecture

This study designed a custom CNN for binary classification of frames [16, 17]. The architecture is deliberately kept straightforward, both to demonstrate that a simple model can succeed with good data and to ease interpretability. Figure 2 summaries the CNN architecture, and we describe it below:

Figure 2. Convolutional neural network (CNN) architecture

This network consists of three convolutional layers, each increasing the number of feature maps while down sampling the spatial dimension, followed by a single hidden fully connected layer. The use of Batch Normalization in each convolutional block helps in faster convergence and better generalization, as it mitigates internal covariate shift [18-20]. Dropout in the penultimate layer provides regularization, which was important given the network's capacity relative to the dataset size – without dropout, we observed the model could memorize the training set too easily and potentially overfit. The overall model size (in terms of number of parameters) is modest, and the depth is shallow compared to very deep networks used in image recognition. We intentionally avoided an overly complex model, as the task at hand is relatively simple, and our goal was to see if even a simple CNN can excel with proper supervision [21, 22]. The architecture contains three convolutional blocks followed by a fully connected classification layer. The video frame after pre-processing is the network input Eq. (1). It will be represented by a tensor X.

$X\in {{R}^{H\text{*}W\text{*}c}}$         (1)

The input is described by the tensor X, which is the frame to be classified. Its dimensions are defined by H (the spatial height), W (the spatial width), and C (the number of channels). As the frames are converted to grayscale, the number of channels C is equal to 1. The frame size used after resizing is 200*200$ pixels

The CNNs feature extraction begins with convolutional operations Eq. (2). Each convolutional filter(k) calculates a weighted sum within its receptive field to create a pre-activation ${{Z}_{i,j,k}}$.

${{Z}_{i,j,k}}=\underset{c=1}{\overset{c}{\mathop \sum }}\,\underset{m=1}{\overset{m}{\mathop \sum }}\,\underset{n=1}{\overset{n}{\mathop \sum }}\,{{W}_{m,n,c,k}}*{{X}_{i+m,j+n,c+{{b}_{k}}}}$           (2)

The output of a filter at its position (i, j) is the pre-activation ${{Z}_{i,j,k}}$. This value is computed with the filter weights ${{W}_{m,n,c,k}}$ that correspond to the k-th filter. These are schoolbook parameters for the convolutional filter at kernel location (m, n) and input channel c. This weight is multiplied by the corresponding pixel value ${{X}_{i+m,j+n,c}}$ from frame's position which is offset. For any given filter, C input channels are summed over and the dimension M times N of its kernel is put into play. Finally, there is a bias term ${{b}_{k}}$ specific to that particular filter.

The neural network is designed as a simple binary classifier. The typical processes are: The flattened characteristics are fed to a fully connected (Dense) layer. The final output neuron then applies the Sigmoid activation function to make a prediction see Eq. (3):

$Output=\sigma \left( \underset{j}{\mathop \sum }\,{{w}_{j}}{{y}_{j}}+b \right)$        (3)

The output is the final result, a probability indicating how likely it is that the input frame is an anomaly. The calculation uses the Sigmoid function (σ), which inside itself calculates it by summing weighted inputs (features) ${{x}_{j}}$ that received from the previous layer--where they have landed after Dropout.

These weights are ${{w}_{j}}$ and b is the offset of the output neuron.

A network for training requires an objective function. The possible error in prediction needs to be measured. Binary CrossEntropyonEntropy function was used because it could naturally suit the Sigmoid output neuron.

This standard log-likelihood formulation is displayed in Eq. (4):

$\mathcal{L}=-\frac{1}{n}\underset{i=1}{\overset{n}{\mathop \sum }}\,[{{y}_{i}}\log \left( {{{\hat{y}}}_{i}} \right)+\left( 1-{{y}_{i}} \right)\text{log}\left( 1-{{{\hat{y}}}_{i}} \right)]$         (4)

The equation calculates average Loss ($\mathcal{L}$) over a mini batch of size N (which we set to 32 frames). The current frame i has its loss calculated. In the above is the true binary label (Ground Truth) for that frame (0 for normal, 1 for anomaly) and ${{\hat{y}}_{i}}$ from Eq. (3), for output probability predicted. The subtracted value is what penalizes our model when predicted probabilities ${{\hat{y}}_{i}}$ depart far from actual label ${{y}_{i}}$.

The network was trained as a binary classifier using the Adam optimizer, chosen for its adaptive learning rate adjustment and proven reliability in CNN applications [23, 24]. An initial step size of 1 × 10⁴ yielded stable convergence; exploratory sweeps showed that larger steps (1 × 10³) produced oscillatory loss, whereas smaller steps only prolonged optimization without tangible accuracy gains. Binary cross-entropy served as the objective function, penalizing misclassifications via the standard log-likelihood formulation and aligning naturally with the sigmoid output neuron. Training proceeded for a maximum of thirty epochs, using mini-batches of thirty-two frames to balance the quality of stochastic gradients against GPU memory demands. Overfitting mitigation relied on two Kera’s callbacks. Early stopping tracked the validation loss and terminated optimization after five stagnant epochs, preserving the weights that achieved the minimum loss. ReduceLROnPlateau halved the learning rate after three consecutive epochs without improvement; for instance, a plateau at epoch 4 reduced the rate from 10⁴ to 10⁵, enabling finer weight updates during the subsequent refinement phase [25, 26]. Experiments were run on a single NVIDIA Tesla-class GPU within Google Colab, although the modest model and dataset permit feasible CPU execution. Each epoch required roughly one second, facilitating rapid hyper‑parameter exploration. Performance monitoring was employed to assess accuracy and loss for both the training and validation partitions. Because accuracy saturated quickly, validation loss served as the primary early-termination criterion [27, 28]. The network converged swiftly, with validation accuracy increasing from 63% in the first epoch to 98% by the eighth epoch and reaching 100% by the fifteenth epoch. Consequently, optimisation ended well before the epoch limit, delivering a compact yet highly discriminative detector.

5.1 Impact of supervision and data quality

The results of our supervised CNN on UCSD Ped2 are exceptional, prompting a discussion on why the model performed so well and what insights can be drawn for the broader field of anomaly detection.

The primary factor in the model's performance is the availability of precise, frame-level labels for anomalies. By training on examples of actual anomalies, the CNN performs binary pattern recognition, learning the visual features that distinguish anomaly frames from normal frames. This is fundamentally easier than the unsupervised task of novelty detection, where the model must infer what is "unusual" without ever seeing an example of "unusual". In our experiments, we initially attempted an unsupervised approach (training an autoencoder on normal frames only). However, the results were markedly poorer – the model had high error on some normal frames (false alarms) and missed some anomalies. Those initial poor results highlighted the critical importance of correct labelling and balanced training data. Once we switched to the supervised paradigm and ensured the training set had a healthy mix of anomaly and normal frames, the model's performance improved dramatically. This underscores a general point: if one can afford to obtain labelled anomalies (perhaps via simulation or expert annotation), even in modest quantities, it can drastically reduce false positives and false negatives compared to unlabeled approaches. Our experiment serves as proof of concept for the potential of supervised anomaly detection.

However, it is essential to recognize that the UCSD Ped2 environment is somewhat controlled. The anomalies are limited to a few types (mostly people riding things), and the background and normal activities are fairly consistent. The model might be implicitly learning something like "wheels or very fast motion = anomaly". In a more complex scene with a greater variety of normal activities, the concept of anomaly might be more nuanced, and collecting a comprehensive, labelled set of anomalies would be challenging. This is why unsupervised methods remain dominant for many applications – because enumerating all anomalies is impractical. However, our study suggests that for specific applications where anomalies can be well-defined and examples can be procured, a supervised learning approach can yield superior and more reliable performance.

5.2 Architecture simplicity vs. complex models

Another notable aspect is that our CNN architecture was relatively simple (3conv layers, one dense layer). In the unsupervised domain, researchers have tried very complex architecture. These are necessary when the task is more challenging due to a lack of labels or the need to capture temporal dynamics. Our results indicate that structured simplicity can outperform complexity when guided by strong supervision. The straightforward nature of our model also aids in transparency – we can more easily interpret what the convolutional filters might be looking for. The one misclassification provided insight that the model was probably focusing on the presence of a bicycle wheel, even at a threshold of visibility. This type of specific interpretation is more challenging with generative models, where the anomaly score represents an abstract reconstruction error. In practice, a security operator might appreciate a system that says "an object that looks like a bicycle was detected” rather than one that says, “frame reconstruction error is high”.

5.3 Role of data balancing

We took care to balance the training data. If we had not – suppose we trained on all 2,550 normal frames and only 1,640 anomaly frames without balancing – the model might have been biased to predict "normal" more often due to class imbalance. In preliminary trials, we indeed noticed that an imbalanced training set led to slower convergence and a few more mistakes, as the model was essentially optimizing accuracy by leaning toward the majority class. Once we were stratified and even did some minor oversampling of anomalies during training, the model had no difficulty. This reflects a common deep learning practice: when anomalies are rare, one must be careful that the training objective does not get dominated by the negative class. Techniques like class weighting or oversampling can help. In our case, a simple split with stratification was sufficient given the moderate imbalance.

5.4 Generalisation considerations

While our validation performance is stellar, how would the model generalize to truly new data? This is a fair question – arguably, we demonstrated memorization of the anomaly concept in a fixed environment. If we were to test this model on a different surveillance camera or a different scene, it might not directly achieve 100%. However, because the model is learning fairly high-level features (such as the presence of a bike or skateboard), it may generalize to some extent. For instance, if applied to UCSD Ped1, it might catch bikes as anomalies but could fail for other types of anomalies. This points to a limitation of supervised models: they generalize to situations similar to what they were trained on. Unsupervised models, by focusing solely on the normal patterns of a specific scene, can adapt to each new scene without requiring anomalies from that scene. Therefore, a practical system might consider a hybrid approach: use unsupervised learning to adapt to a new camera's normals but incorporate a classifier like ours for known anomaly types (if such anomalies exist). Our results, in essence, define an upper bound on performance in the Ped2 scenario, serving as a baseline against which to strive with any method. Achieving near 100% detection with no false alarms is the ideal for any surveillance analytics.

5.5 Error analysis

The single error (false positive) we encountered provided a useful insight. The fact that it was an anomaly entering the scene slightly before the labelled boundary suggests that our labelling (ground truth) could perhaps be refined. If one were to use this in practice, one might adjust the anomaly labels to start earlier to capture such cases. It also highlights that the model has a temporal sensitivity, even though it is frame-based – it effectively learned from examples that whenever you see part of a bicycle, even if it is just emerging, that frame is likely anomalous. This suggests that adding a small quantity of temporal context (e.g., a short Long Short Term Memory (LSTM) or temporal window) might enable the model to be aware of continuity and possibly avoid minor jitters, such as that false positive. For example, a model that looks at a sequence of frames might realise "the bicycle is only fully there in the next frame, so maybe not signal yet." In our frame-level approach, it has no such context by design, and so it flagged at the first sign.

In conclusion to this discussion, our study demonstrates that the exceptional performance of our model is attributable to the combination of explicit supervision, a carefully curated training process, and a well-chosen simple architecture. It reaffirms the value of ground truth annotations – they dramatically simplify the problem. It also serves as a reminder that for specific deployments, training a supervised detector might be the most reliable solution. The trade-off is that one must gather anomaly examples for each application.

5.6 Limitations and possible improvements

Despite the good performance of the new CNN, in particular, a number of limitations were identified and indicate clear directions for further advancement. Firstly, the current model is confined to frame-level operations without incorporating any temporal information. Consequently, it cannot benefit from movement continuity or sequential patterns, both of which are necessary for gradual or motion-dependent anomalies. Future extensions might see our architecture become based on sequences, as for example 3D CNN or combined CNN-LSTM models specifically designed to capture the temporal dynamics of data.

Secondly, the model is only validated on the UCSD Ped2 dataset's, which is a relatively simple and well-controlled environment. This raises an issue regarding generations: what looks good when that particular camera is pointed at some specific object in luminous conditions. A wider validation onto more data sets, along with stronger data augmentation strategies can make the system less finicky.

Third, the approach relies heavily on frame-level annotated anomalies. While this allows it to achieve good results under supervision conditions, it is not something that easily scales up to practical cases in which dense anomaly annotation has to be done. Moving towards weakly supervised or semi-supervised frameworks—where only a small subset of anomalies are labeled—could break this dependency but still extract meaningful and rewarding features from view data.

Building on the findings of this research, several avenues emerge for future exploration:

7.1 Wider dataset evaluation

We plan to evaluate our supervised approach on other anomaly detection benchmarks to test its generalization. Each dataset has different scenes and anomaly types; applying our method would confirm if similar near-perfect performance can be achieved elsewhere or identify new challenges specific to those environments.

7.2 Temporal modelling extensions

While our current model analyses frames independently, many anomalies are inherently spatio-temporal. Future work includes developing temporal models such as CNN-LSTM or 3D ConvNet hybrids that consider sequences of frames.

7.3 Reducing label dependence

An interesting direction is to combine the strengths of supervised and unsupervised approaches. One idea is a semi-supervised or weakly-supervised framework where a small number of labelled anomalies guide an unsupervised model. Techniques like transfer learning or positive-unlabeled learning could leverage our results to inform models that must operate with fewer labels.

7.4 Interpretability and user feedback

To enhance trust in automated surveillance, future work could focus on explainability techniques for our CNN. For instance, using Grad-CAM or similar methods to highlight which part of the frame triggered the anomaly classification. This approach could be combined with a user-in-the-loop method, where security personnel can confirm or correct the model's detections, thereby progressively improving the system.

By pursuing these directions, we are courageous to bridge the gap between the idealized performance seen in this supervised setting and the constraints of real-world anomaly detection tasks. Ultimately, the goal is a highly accurate, interpretable, and deployable anomaly detection system that can enhance security and safety in various surveillance contexts.