A Comparative Study of Incremental and Batch Machine Learning Methodologies for Network Intrusion Detection

Cybersecurity and network security concerns have grown significantly with the rapid expansion of the Internet and communication technologies and the increasing variety of network applications. Network security has become a critical issue as attack methods have also evolved. Key challenges include the sheer volume of data to be analyzed and the rapid emergence of novel attack techniques.

Intrusion detection systems (IDS) play a pivotal role in identifying and mitigating cyber threats by monitoring network traffic to detect anomalies and cyberattacks [1]. However, intrusion detection faces new challenges, such as scalability, adaptability, and efficiency in the age of big data and rapidly evolving attack strategies. Previous studies have proposed several approaches to improve IDS performance using machine learning techniques. For instance, some works have focused on enhancing system performance through multi-stage frameworks, feature selection, and hyperparameter optimization [2]. Others, like the previous study [3], integrated chi-square feature selection with support vector machines (SVM) to improve accuracy and reduce false positives. Feature reduction techniques, such as CFS, IG, and GR, have also been utilized to enhance classifier efficiency [4]. Similarly, anomaly detection models for cloud computing environments have employed SVM with particle swarm optimization (PSO) to refine feature selection and improve classification results [5]. Incremental learning has emerged as a promising methodology in IDS, addressing challenges such as memory and computational requirements when handling large datasets. For example, researchers [6] demonstrated how incremental techniques could reduce training time and classifier size while maintaining performance comparable to batch learning. Other work [7], developed multi-classifiers based on incremental learning to improve training efficiency on massive datasets. Furthermore, distance-weighted outlier detection and domain-specific anomaly detection methods have been explored to detect hidden anomalies in cellular networks [8, 9]. Deep learning approaches have also been employed, such as the system proposed in previous study [10] using deep neural networks (DNN) and optimization algorithms to detect network intrusions in cloud environments. Two-layer anomaly detection models combining Naive Bayes and customized KNN classifiers have achieved reasonable detection rates with fewer false alarms, particularly for rare attack types like U2R [11].

Despite these advancements, there is a limited comparative analysis of incremental learning and batch learning methodologies in the context of IDS. While incremental learning offers adaptability to new threats, it may incur higher computational costs, whereas batch learning is computationally efficient but slower to adapt to dynamic environments [12]. This study addresses this gap by investigating the comparative effectiveness of incremental and batch-learning methodologies in network intrusion detection. The main contributions of this work can be summarized as follows:

·Comparing batch and incremental learning methodologies in IDS systems based on machine learning.

·Evaluating the performance of several machine learning algorithms using two recent datasets: CI-CIDS 2017 and UNSW-NB15.

The remainder of this paper is structured as follows: Section 2 introduces key concepts of intrusion detection and machine learning alongside a detailed review of relevant literature. Section 3 describes the methodology and model design for the comparative framework. Section 4 presents the experimental results, while Section 5 discusses the findings, highlighting the strengths and limitations of each methodology. Finally, Section 6 concludes the paper.

This section outlines the methodology used in this study, including data collection, preprocessing, model design, and evaluation. The proposed framework evaluates supervised machine learning algorithms for classifying activities as suspicious or non-suspicious. Four machine learning algorithms, Random Forest, Support Vector Machine (SVM), Naïve Bayes, and K-Nearest Neighbor (KNN), are applied independently to compare their performance. Incremental and batch learning approaches are evaluated separately to understand their strengths and weaknesses. Figure 4 provides an overview of the framework.

d3c6a412-3b73-49b0-b301-ab44a5c37e35.png

Figure 4. The general framework of the proposed methodology

3.1 Dataset

This study uses two widely recognized datasets:

·UNSW-NB15 Dataset: Collected by the Cyber Range Lab in 2015, this dataset integrates contemporary network traffic and various attack behaviors, including Backdoors, DoS, Exploits, Reconnaissance, and Worms. It consists of 2.5 million records with 49 features, and it is balanced, containing equal proportions of normal and attack logs. The dataset's richness and balance make it suitable for training and evaluating intrusion detection models [23].

·CI-CIDS 2017 Dataset: Developed by the Canadian Institute for Cybersecurity (CIC), this large-scale dataset comprises over 80 million records, each representing a network flow. It features both normal and malicious traffic and includes annotations for 15 types of attacks. With 81 features, the dataset provides extensive coverage of network behaviors, offering a robust basis for testing intrusion detection systems [24].

The datasets were selected for their balanced representation of attack and normal traffic, their scale, and the diversity of attack types. Thus, they are ideal for evaluating the generalizability and effectiveness of machine learning models.

3.2 Data preparation

To ensure that the data is suitable for analysis, preprocessing was performed in several stages:

1. Data Cleaning: Noise, incomplete, and irrelevant data were removed or modified. Errors were documented, and the effectiveness of the cleaning process was assessed against predefined requirements to ensure high-quality input data.

2. Feature Selection: Selecting relevant features is crucial for reducing computation time and improving model accuracy. The Binary Particle Swarm Optimization (BPSO) algorithm was used to rank features, with the SVM model as the fitness function. Features were retained if their value was set to 1 and removed if set to 0. This step minimized the impact of redundant features, improving detection time and classification accuracy.

3. Data Annotation: Data was labeled as "suspicious" or "non-suspicious" to facilitate supervised learning. This labeling ensures that models learn to differentiate between normal and malicious activities effectively.

4. Data Splitting: Datasets were divided into training and testing sets, with at least 20% of the data reserved for testing. This ensures that models are evaluated on unseen data, providing a reliable measure of their generalizability.

5. Data Transformation: It converts data into a more acceptable format for analysis. Cleaning, filtering, and standardizing the data may be required. The process of transforming data ensures that the data is clean and consistent. It refers to defining a restricted range of attribute values that might aid in improving detection results and avoiding numerical issues during calculations. This step is completed to guarantee that the detecting system can effectively process the data. Many traits have values that vary widely among species and ranges. As a result, to properly process and analyze the data, these numbers must be restricted to a particular range. Normalization methods based on min-max. Normalization of the average range [0, 1] (from least to most): This function converts attribute values to a value between 0 and 1. This is accomplished by subtracting the lowest value from each and dividing the result by the range (highest value minus lowest value) as in Eq. (1).

$X^{\prime}=\frac{x-\min A}{\max A-\min A}$     (1)

where, x and primes boldrimes are the normalized attribute value and the value to be normalized, respectively, before normalization, the minimum and maximum allowed values for attribute A are MinA and MaxA [10].

3.3 Model design

The machine learning models were designed with two primary phases:

·Training Phase: Models were trained on labeled data to learn the relationship between input features and their corresponding outputs. Parameters were iteratively adjusted to minimize prediction errors.

·Testing Phase: Models were evaluated on unseen data to measure their generalization ability to new situations. This step highlights the models' robustness, precision, and potential limitations, such as overfitting or underfitting.

3.4 Evaluation

The models were assessed using standard evaluation metrics derived from the confusion matrix:

(1) Confusion Matrix Metrics is to evaluate the models' ability to classify activities correctly, metrics such as the true positive rate (TPR), false positive rate (FPR), true negative rate (TNR), and false negative rate (FNR) were calculated.

a. TP represents normal conduct that is accurately expected and given in Eq. (2).

True Positive Rate $=\frac{T P}{T P+F N}$     (2)

b. FP represents normal behavior that is incorrectly supposed to be abnormal and given in Eq. (3).

False Positive Rate $=\frac{F P}{F P+T N}$     (3)

c. TN represents normal performance that is identified as correct and given in Eq. (4).

True Negative Rate $=\frac{T N}{T N+F P}$     (4)

d. FN represents aberrant performance that is misdiagnosed as normal and given in Eq. (5).

False Negative Rate $=\frac{F N}{F N+T P}$     (5)

(2) Accuracy is a trained model's proportion of true predictions. It is derived by dividing the number of forecasts by the number of correct guesses and given in Eq. (6).

Accuracy $=\frac{T P+T N}{T P+T N+F P+F N}$     (6)

(3) Precision is another metric showing the percentage of positive anticipated values. Precision aids in visualizing the machine learning model's dependability and classifying the model as positive and given in Eq. (7).

Precision $=\frac{T P}{T P+F P}$     (7)

(4) Recall measures the proportion of positive cases correctly identified by the model and given in Eq. (8).

Recall $=\frac{T P}{T P+F N}$     (8)

In the UNSW-NB15 dataset, KNN showed better incremental and batch learning accuracy, with a precision rate 0.99. Batch learning achieved the highest percentage of 55,519 TP cases, while incremental learning classified a larger number of 118,195 TN cases without abnormal or malicious activity. Regarding false positive (FP) cases, batch learning classified more cases than incremental learning.

Regarding SVM, batch learning was better in accuracy at 0.99, while incremental learning showed 0.98. Both batch and incremental learning performed well in classifying true positive (TP) cases, with batch learning achieving 54,844 TP cases and incremental learning achieving 56,000 TN cases. Regarding RF, batch learning was 1.00 better in recall, while incremental learning showed a 0.99 performance. Batch learning identified many actual network attacks (TP) and classified many normal activities (TN) as attacks, resulting in unnecessary alarms. Incremental learning significantly reduced the number of false alarms (FP) by classifying only 30 normal activities as attacks, while in batch learning, 91 normal activities were classified as attacks.

Regarding the CI-CIDS 2017 dataset, when learning incrementally, KNN performed better than batch learning (recall 0.989187 and F1-measure 0.989210), with recall (0.990397) and F1-measure (0.992272). KNN classified some negative cases as false positives by producing more false positives (3,701) with incremental learning compared to batch learning (828). Compared with batch learning, which produced 196 false negatives, KNN incremental learning produced 4370 false negatives, indicating that some positive cases were incorrectly classified. For SVM, incremental learning outperformed batch learning across parameters (F1 measure, precision, and recall), with incremental learning being 0.99 accurate compared to batch learning, which was 0.80 less accurate. On the other hand, incremental learning achieved a high recall rate of 0.99, but batch learning achieved a weak rate of 72%. Finally, the performance of batch learning in F-Score was 0.63, which is considered a very weak performance compared to the achievement of incremental learning with a higher performance rate of 0.99. RF achieves remarkable accuracy (0.9995 and 0.9998, respectively) and high F1 metrics (0.999470 and 0.999874) while maintaining good incremental and batch learning performance.

Batch learning was less effective than RF incremental learning regarding recall (1.0 vs. 0.99) or F1 scale (0.99987 vs. 0.999470). Both incremental RF and batch learning show false positive counts of (1604 and 1596, respectively), indicating their exceptional accuracy in identifying positive situations. False negatives (0) are also produced by batch learning or incremental learning of the RF, indicating that all positive cases are accurately identified. If the dataset is static and does not show significant changes over time, incremental learning may not be the most efficient or cost-effective approach. This is especially true for the UNSW-NB15 -NB15 dataset, where both incremental and batch learning showed similar performance results. The main benefit of incremental learning is its ability to adapt to evolving data patterns. This advantage is not fully utilized if the data set remains relatively constant. Incremental learning algorithms also require more computational resources than batch learning, especially when dealing with large data sets.

On the other hand, the CICID17 dataset showed a difference in performance between incremental learning and batch learning. The performance of incremental learning was better and more effective than batch learning, which means there is a great benefit and a big difference between using them. The CICID dataset shows the advantages of incremental learning in dynamic network intrusion detection scenarios. The ability of incremental learning to adapt to new data and reduce false positives makes it a promising approach for real-world cybersecurity applications.