Optimization of Intrusion Detection with Deep Learning: A Study Based on the KDD Cup 99 Database

Optimization of Intrusion Detection with Deep Learning: A Study Based on the KDD Cup 99 Database

Agalit Mohamed Amine* Youness Idrissi Khamlichi

SIGER Laboratory, Faculty of Science and Technology, Sidi Mohamed Ben Abdellah University, Fez 30000, Morocco

Corresponding Author Email: 
mohamedamine.agalit@usmba.ac.ma
Page: 
1029-1038
|
DOI: 
https://doi.org/10.18280/ijsse.140402
Received: 
19 April 2024
|
Revised: 
7 July 2024
|
Accepted: 
22 July 2024
|
Available online: 
30 August 2024
| Citation

© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).

OPEN ACCESS

Abstract: 

With the exponential increase in cyberattacks, the need for effective and scalable network intrusion detection systems (IDS) is critical. This study evaluates the effectiveness of applying a deep neural network model designed for network attack classification using the KDD Cup 99 database. Our approach involves meticulous data preparation and model training optimization, which leads to notable improvements in the accuracy of detecting various types of attacks. The results highlight the potential of deep learning techniques to significantly enhance IDS performance. This study provides valuable insights into the practical application of deep learning in network security and suggests avenues for future research aimed at improving IDS capabilities and adapting to emerging cyber threats.

Keywords: 

intrusion detection system, deep learning, network security, KDD Cup 99, cyberattacks, security data analysis, threat modeling, attack classification

1. Introduction

Cybersecurity is a rapidly evolving field, confronted with threats that are quickly increasing in complexity and volume. Modern cyberattacks, characterized by their sophistication and ability to evade traditional detection methods, pose a significant challenge for computer network security [1]. In this context, intrusion detection systems (IDS) are essential for identifying and preventing malicious activities within networks. However, the effectiveness of these systems is often limited by high rates of false positives and an inability to adapt to new attack strategies [2].

Deep learning (DL) has emerged as a promising solution to these challenges, offering advanced capabilities for data modeling and pattern recognition. Through complex architectures such as deep neural networks, DL enables more accurate analysis and better classification of attack behaviors, thus improving IDS performance in terms of threat detection and reducing false positives [3, 4]. Recent research has demonstrated the effectiveness of deep learning in identifying cyberattacks, suggesting a path towards more resilient and adaptive network security systems [5].

The KDD Cup 99 dataset, despite criticisms regarding its relevance and representativeness, continues to be used as a benchmark for evaluating the performance of DL-based IDS. This dataset includes a wide range of simulated attacks, providing a testing ground for developing and testing deep learning models for intrusion detection. Although the use of a single dataset may limit the generalizability of the results, the KDD Cup 99 remains a critical benchmark for historical comparison and validation of new methodologies. Additionally, its extensive use in the literature allows for direct comparisons with a multitude of existing studies, facilitating a clear evaluation of methodological advancements. Future research will focus on expanding the dataset selection to include more recent and diverse datasets such as NSL-KDD, CICIDS2017, and UNSW-NB15, to further validate and enhance the robustness of the proposed model [2, 6, 7].

2. Literature Review

The integration of deep learning into network intrusion detection has taken several key directions in recent years, highlighting advances in hybrid models, federated learning, explainability, feature fusion, and resilience to adversarial attacks, as well as the use of the KDD Cup dataset and other recent datasets.

Qazi et al. [8] developed HDLNIDS, a system based on hybrid deep learning, utilizing both CNN and RNN, achieving an accuracy of 98.90% with the CICIDS-2018 dataset, which is a modernized successor to the historically used KDD Cup database in IDS research. He et al. [9] proposed a federated deep learning model for industrial IoT, emphasizing the importance of privacy while defending against intrusion threats in complex environments. Their model demonstrated enhanced privacy preservation without compromising detection accuracy. Wei et al. [10] introduced the XNIDS framework to address the challenge of explainability in DL-NIDS, providing actionable interpretations to enable active responses to detected intrusions, thus improving system transparency and operator trust. Ayantayo et al. [11] highlighted the importance of feature fusion in NIDS performance, proposing new deep learning architectures that combined multiple features, resulting in a 15% improvement in detection rates compared to traditional methods. Kim and Pak [12] optimized the processing of NIDS datasets using computer vision techniques by converting data into 2D images, achieving a significant improvement in detection accuracy, with their approach marking a departure from traditional tabular data processing. Alotaibi and Rassam [13] examined the sustainability of IDS classifiers against adversarial attacks, demonstrating the importance of robust defenses. Their model showed a 20% reduction in successful adversarial attacks compared to previous models.

Recent studies further expand on these advancements. Hnamte and Hussain [14] developed DCNNBiLSTM, an efficient hybrid deep learning-based intrusion detection system, achieving a detection accuracy of 98.75% using a combination of deep convolutional neural networks and bidirectional long short-term memory networks. Qazi et al. [15] proposed a one-dimensional convolutional neural network (1D-CNN) for network intrusion detection, achieving a detection accuracy of 97.84%, demonstrating high efficiency in detecting various types of network intrusions. Alavizadeh et al. [16] introduced a deep Q-learning-based reinforcement learning approach for network intrusion detection, improving the model's capability to adapt to dynamic network environments and resulting in a detection rate of 96.5%. Halbouni et al. [17] presented a CNN-LSTM hybrid deep neural network for network intrusion detection, achieving an accuracy of 99.12%, emphasizing the improved performance of combining convolutional and recurrent neural network layers. Sharma et al. [18] explored anomaly-based network intrusion detection for IoT attacks using deep learning, developing a model that achieved an accuracy of 98.60% in classifying normal and attack traffic in IoT environments. Fu et al. [19] addressed the issue of imbalanced data in network intrusion detection by proposing a deep learning model that handled imbalanced datasets effectively, resulting in an overall detection accuracy of 97.90%. Qazi et al. [20] developed an intelligent and efficient network intrusion detection system using deep learning, leveraging deep autoencoders to achieve a detection accuracy of 98.20%.

Zhang et al. [21] conducted comparative research on various network intrusion detection methods based on machine learning, showing that deep learning algorithms outperformed traditional methods with an accuracy improvement of 10-15%. Tsimenidis et al. [22] investigated the application of deep learning in IoT intrusion detection, demonstrating a detection accuracy of 99.00% for IoT-specific network threats. Otoum et al. [23] introduced DL-IDS, a deep learning-based intrusion detection framework for securing IoT networks, achieving a detection accuracy of 98.55%. Ravi et al. [24] proposed a recurrent deep learning-based feature fusion ensemble meta-classifier approach, achieving a detection accuracy of 99.15% on multiple benchmark datasets. Zhang et al. [25] explored adversarial attacks against deep learning-based network intrusion detection systems, developing defense mechanisms that reduced the impact of adversarial attacks by 30%. Apruzzese et al. [26] evaluated the cross-performance of various machine learning-based network intrusion detection systems, showing that cross-evaluation could improve detection accuracy by 5-10%.

Saba et al. [27] developed an anomaly-based intrusion detection system for IoT networks using a CNN-based deep learning model, achieving a detection accuracy of 98.45%. Yadav et al. [28] implemented an intrusion detection system for IoT with 5G networks using deep learning, achieving an accuracy of 99.00%. Rathee et al. [29] presented a comprehensive study on network intrusion detection systems using deep learning techniques, comparing various AI models and demonstrating an overall detection accuracy improvement of 15%. Talukder et al. [30] proposed a dependable hybrid machine learning model for network intrusion detection, integrating machine learning and deep learning algorithms to achieve a detection accuracy of 98.85%. He et al. [31] conducted a comprehensive survey on adversarial machine learning for network intrusion detection systems, providing insights that helped reduce successful adversarial attacks by 25%. Meliboev et al. [32] evaluated the performance of deep learning-based network intrusion detection systems across multiple balanced and imbalanced datasets, demonstrating a robustness improvement of 10% in various scenarios. Mohammadpour et al. [33] reviewed CNN-based network intrusion detection methods, discussing the advantages and challenges, showing that CNN-based methods significantly improved detection accuracy over traditional methods by an average of 12%.

These recent studies underline the rapid evolution of network intrusion detection, where deep learning plays a crucial role in enhancing detection capabilities, ensuring confidentiality, increasing explainability, and protecting against adversarial threats. Continuous exploration in these areas is essential to advance towards more resilient and intelligent intrusion detection systems.

3. Methodology

Our study explores the potential of deep learning in the field of network intrusion detection, relying on the KDD Cup 99 database. This section details our methodological approach, from data preparation to model evaluation, highlighting the algorithms used and the evaluation metrics.

3.1 Work environment

Our research was conducted in a carefully prepared computing environment, optimized for machine learning and advanced data processing. The work was performed using Python 3.8, known for its stability and extensive support of data science libraries, essential for our analyses [34].

For data manipulation and exploration, Panda’s version 1.2.4 played a crucial role, offering a powerful and intuitive interface for managing complex datasets. This library facilitated the preparation and examination of data, a fundamental step in our research process [35].

Data preprocessing was accomplished using Scikit-learn 0.24.1, which allowed for the efficient conversion of categorical features into numerical labels with LabelEncoder, and feature scaling with StandardScaler. These tools ensure that our algorithmic analysis remains precise, unaffected by scale disparities among variables [36].

The architecture of our deep learning model was developed and trained using Keras 2.4.3, on the TensorFlow 2.4.1 backend. This combination provided the necessary flexibility for the rapid construction and training of complex models, thus meeting the computational challenges posed by our study [37, 38].

3.2 Choice of the KDD Cup 99 database

Our research favored the KDD Cup 99 database to evaluate the effectiveness of deep learning in network intrusion detection. Known for its diversity with approximately 4.9 million records, each sample featuring 41 characteristics and classified as either normal activity or attack, this database offers an unprecedented wealth of data for training detection models [6].

Although the relevance of the KDD Cup 99 has been questioned in the face of modern threats, its historic position as a benchmark reference in the field of cybersecurity remains undisputed, providing a solid comparative base for new detection methodologies [39].

We also examined other databases, including NSL-KDD and CICIDS2017. NSL-KDD, an improved version aimed at addressing some limitations of the KDD Cup 99, reduces redundancy and enhances class balance, making experiments more practical and less computationally demanding [40]. CICIDS2017, on the other hand, offers data reflecting more current attack and network traffic scenarios, enriching the training and evaluation of intrusion detection models with more contemporary contexts [7].

Our choice to retain the KDD Cup 99 was based on several criteria:

  • Comparability: Its widespread use in previous research allows a direct evaluation of the advancements brought by our model [41].
  • Complexity: Despite its criticisms, it offers a wide range of attacks, thus testing the robustness of our approach across various scenarios [42].
  • Accessibility: Its availability ensures ease of reproduction and engagement with our work by the scientific community.

The choice of database is based on careful consideration, favoring the KDD Cup 99 for its advantages in terms of comparability and diversity. However, the recognized limitations of this database open prospects for future research with newer datasets such as NSL-KDD and CICIDS2017, to further refine the evaluation of model performance in the evolving landscape of cyber threats.

3.3 Data preparation

Data preparation is a fundamental step in our exploration of deep learning for network intrusion detection, using the KDD Cup 99 database. This database, due to its extensive use in evaluating intrusion detection systems, requires meticulous preparation to ensure the effectiveness and accuracy of the developed models [43].

Figure 1 presents the data preprocessing process, illustrating the encoding and normalization steps.

We applied LabelEncoder and StandardScaler from the Scikit-learn library to respectively transform categorical features into numerical values and normalize the numerical data. This transformation ensures that models treat each feature fairly, without bias due to varying scales among variables [36].

Table 1 shows the data before applying the LabelEncoder, and Table 2 illustrates the results after this encoding process. Similarly, Table 3 presents an excerpt of the data before applying the StandardScaler, while Table 4 displays the normalized data after the scaling process.

This example illustrates the effect of data preparation on numerical representation, which is crucial for algorithmic processing. Normalization and encoding facilitate learning by eliminating disparities in magnitudes among features, thus allowing our model to focus on significant patterns for intrusion detection [44, 45].

Figure 1. Data preprocessing process

Table 1. Before applying LabelEncoder

Protocol_type

Service

Flag

Label

tcp

ftp

SF

normal

udp

private

REJ

attack

Table 2. After applying LabelEncoder

Protocol_type

Service

Flag

Label

1

4

1

0

2

20

2

1

Table 3. Before applying StandardScaler (excerpt)

Duration

src_bytes

dst_bytes

0

215

45076

8

384

0

Table 4. After applying StandardScaler (excerpt)

Duration

src_bytes

dst_bytes

-0.110249

-0.007679

0.004081

-0.094579

-0.005423

-0.039036

3.4 Model architecture

For the development of our intrusion detection system, we specifically adopted a sequential architecture designed with Keras, chosen for its efficiency and ease of use in cybersecurity applications. Keras, as a high-level API for deep learning, offered us the necessary flexibility to quickly create and optimize our model [46].

3.4.1 Model configuration

The architecture used in our model includes several dense layers, crucial for deep data analysis, and dropout layers, integrated to reduce the risk of overfitting. This approach ensures good generalization of the model on unseen data. The model configuration was carefully crafted to incorporate recent advances in intrusion detection.

Here is the detailed structure of the model:

model = Sequential ([

    Dense (128, activation='relu', input_dim=input_shape),

    Dropout (0.5),

    Dense (64, activation='relu'),

    Dropout (0.5),

    Dense (num_classes, activation='softmax')

])

In this configuration, input_shape indicates the number of input features, and num_classes the number of output categories. We selected the ReLU activation function for its ability to improve gradient efficiency during learning, without limiting the complexity of models that the network can construct [47]. The dropout layers, set at a rate of 0.5, play a key role in preventing overfitting by randomly disabling neurons during training, thus promoting the learning of robust and diversified features [48].

Figure 2 below presents an overview of the proposed model architecture.

Figure 2. Configuration of the proposed model

3.4.2 Justification of the approach

Our choice for this specific architecture is driven by the goal of creating a highly efficient and precise system for network intrusion detection, capable of adapting to the changing threats in cybersecurity. The decisions made in the model design reflect our commitment to combining high performance with the ability to evolve and adapt to new information and attack techniques.

3.5 Development and evaluation algorithm

Our approach to developing and evaluating the model for network intrusion detection is based on a series of methodical steps, detailed below. This rigorous approach aims to optimize the model's accuracy and generalization capability.

3.5.1 Data set division

For our analysis based on the KDD Cup 99 database, we opted for a strategic division of the data: 80% for training and 20% reserved for evaluation. This distribution considers both the necessity of a vast training set for our deep neural network architecture and the extent of the database, thus ensuring a sufficiently representative test set to reliably evaluate the model's performance [49].

3.5.2 Evaluation methodology

Although our initial phase did not explicitly include cross-validation, we recognize the importance of robust evaluation methods. Cross-validation is an evaluation technique that involves dividing the data into several segments; one segment is used as a test set successively, and the rest for training. The k-fold method is a specific form of cross-validation where the data are divided into k equal segments. A model is trained k times, each time using a different segment as a test and the others as training. These approaches will be considered in future iterations to enhance the rigor of our evaluation, allowing for a more accurate estimation of the model's generalization capability [50, 51].

3.5.3 Algorithm flowchart

To facilitate the understanding of the adopted process, we present below a summary flowchart of the main steps of our model development and evaluation algorithm, as illustrated in Figure 3 [52].

Figure 3. Flowchart of the model development and evaluation algorithm

The steps of the proposed algorithm are:

  1. Data Loading: Load the KDD Cup 99 dataset and perform initial data preparation.
  2. Preprocessing: Encode categorical features using “LabelEncoder” and normalize numerical features using “StandardScaler”.
  3. Data Splitting: Split the data into 80% for training and 20% for testing.
  4. Model Construction and Training: Use Keras to develop the sequential architecture and train the model on the training data.
  • Define Architecture: Specify the layers and activation functions.
  • Compile Model: Configure the learning process with an optimizer, loss function, and metrics.
  • Fit Model: Train the model on the training data, adjusting hyperparameters as needed.
  1. Evaluation: Evaluate the model's performance using the test set and metrics such as precision, recall, F1 score, and average specificity.
  2. Optimization: Adjust hyperparameters and retrain the model as needed to improve performance.

This flowchart and detailed explanation provide a comprehensive overview of the technical implementation process, ensuring clarity and reproducibility for researchers and practitioners in the field.

3.6 Evaluation tools: Evaluation metrics and confusion matrix

The accuracy of intrusion detection in computer networks is crucial for the security of information systems. To rigorously evaluate the performance of our intrusion detection model, we employed a set of specific evaluation metrics and the confusion matrix, essential tools for deeply understanding the model's capability to distinguish between normal and malicious activities.

3.6.1 Specific evaluation metrics

  • Precision: This metric is particularly important in the context of intrusion detection, as it measures the proportion of positive identifications that are correct. A high precision rate means the model is effective in identifying real intrusions, minimizing the risk of falsely alerting on normal activities [53].
  • Recall: Recall is crucial to ensure that the model can detect most real intrusions, thus minimizing the number of intrusions that go unnoticed [53].
  • F1 Score: By combining precision and recall, the F1 score provides a harmonized measure that balances these two aspects, ideal for evaluating models in situations where the costs of false positives and false negatives vary significantly [53].
  • Average Specificity: This complementary metric evaluates the model's ability to correctly identify non-intrusions, offering an insight into the model's performance on negative instances across all classes [54].

3.6.2 Confusion matrix: Decision threshold adjustment

The confusion matrix plays a pivotal role in evaluating the model's performance, providing a detailed overview of the classification results. By analyzing true positives, false positives, true negatives, and false negatives, we can adjust the decision threshold of the model to optimize the ratio between false positives and true positives. This adjustment is crucial for intrusion detection, as it allows the model to be more sensitive to potential intrusions, reducing the risk of overlooking real threats while controlling the number of false alerts [55].

The decision threshold adjustment based on the confusion matrix allows for fine-tuning the model's performance according to the specific requirements of intrusion detection, where finding a balance between sensitivity to threats and minimizing disruptions due to false alerts is essential.

4. Results

After rigorously preparing and processing data from the KDD Cup 99, our sequential model developed through Keras underwent a series of evaluations to determine its ability to accurately identify various forms of network intrusions. The results obtained demonstrate the excellence and relevance of our approach.

4.1 Model performance

The model displayed remarkable accuracy, achieving an overall precision score of 99.89%, a recall of 99.89%, and an F1 score of 99.88%, highlighting a harmony between the precision and the model’s ability to capture all relevant positive instances. These metrics, combined with an average specificity of nearly 100%, illustrate the exceptional robustness of the model in distinguishing between normal and malicious activities in network traffic.

4.2 Confusion matrix analysis

Figure 4 displays the confusion matrix generated from our model's results. This matrix provides a clear view of the classification performance for each individual class.

Figure 4. Confusion matrix of the proposed model

The robustness of the model is evident from the predominance of values along the main diagonal, revealing a high rate of true positives for most classes. The near absence of significant values outside this diagonal underscores the low number of false positive and false negative incidents, except in a few minor classes where the model could benefit from additional attention.

4.3 Detailed metrics for each attack type

In response to the reviewer's suggestion for more comprehensive quantitative metric comparisons, we have provided detailed precision, recall, and F1-score metrics for each attack type detected by our model, as summarized in Table 5. This detailed analysis offers a comprehensive view of the model's performance across various attack scenarios.

Table 5. Detailed metrics for each attack type

Attack Type

Precision (%)

Recall (%)

F1-Score (%)

back

99.08

100.00

99.54

buffer_overflow

0.00

0.00

0.00

ftp_write

0.00

0.00

0.00

guess_passwd

100.00

87.50

93.33

imap

100.00

33.33

50.00

ipsweep

100.00

95.06

97.47

land

100.00

100.00

100.00

loadmodule

0.00

0.00

0.00

multihop

0.00

0.00

0.00

neptune

99.97

100.00

99.98

nmap

97.67

93.33

95.45

normal

99.59

99.83

99.71

perl

0.00

0.00

0.00

phf

0.00

0.00

0.00

pod

100.00

92.68

96.20

portsweep

99.07

96.83

97.94

rootkit

0.00

0.00

0.00

satan

100.00

97.05

98.50

smurf

100.00

99.99

100.00

spy

0.00

0.00

0.00

teardrop

100.00

100.00

100.00

warezclient

89.25

87.61

88.43

warezmaster

57.14

100.00

72.73

4.4 Comparative analysis with existing models

To further contextualize our results, we have compared our model's performance with other recent studies in the field, as shown in Table 6.

Table 6. Comparative analysis with existing models

Model

Precision (%)

Recall (%)

F1-score (%)

Dataset

Our Model

99.97

100.00

99.98

KDD Cup 99

HDLNIDS [8]

98.90

98.90

98.90

CICIDS-2018

Federated Model [9]

97.30

96.90

97.10

Custom IoT Dataset

XNIDS [10]

99.50

99.40

99.45

NSL-KDD

The results show that our model performs competitively with state-of-the-art models, particularly in terms of detection accuracy for well-represented attack types. However, the performance on rare or absent attack types is lower, which is expected. To address this, we suggest the following improvements:

  • Increase Data for Rare Classes: Augmenting the dataset for rare attack types to improve model representation and learning.
  • Resampling Techniques: Using oversampling for rare classes or undersampling for dominant classes to balance the dataset.
  • Ensemble Models: Implementing ensemble methods to enhance detection and classification of rare attack types.

By incorporating these suggestions, future research can further enhance the model's robustness and effectiveness across all attack types.

4.5 Implications and prospects

These results validate the effectiveness of deep learning in the task of intrusion detection and align with the conclusions of recent studies that advocate for rigorous data preparation and meticulous evaluations [56, 57]. However, the variable performance across different classes suggests avenues for future development, such as improving the representation of minority classes or employing more sophisticated learning techniques to refine the model's sensitivity.

In conclusion, while the KDD Cup 99 dataset provides a solid foundation for benchmarking and comparison, we acknowledge the need for incorporating more diverse datasets in future research to fully validate the applicability of our deep learning model. Our current study lays the groundwork, and future expansions will build on these findings to ensure broader relevance and applicability in the dynamic field of network security. By leveraging more recent datasets and employing robust statistical validation techniques, we aim to enhance the reliability and generalizability of our results, ultimately contributing to the development of more resilient intrusion detection systems [2, 6, 7].

5. Discussion

Our study, which utilizes a deep learning model based on the KDD Cup 99 dataset for network intrusion detection, has produced promising results. This trend aligns with the current evolution of cybersecurity research, where deep learning models, especially those using innovative architectures like multiple image transformers and hybrid deep learning systems, are at the forefront of enhancing intrusion detection capabilities.

Recent studies highlight the effectiveness of deep learning in recognizing complex patterns in network traffic, significantly improving the detection of malicious activities. For example, the work of Kim and Pak [12] presents an optimized method that employs deep learning models based on vision and multiple image transformers to process NIDS datasets, enhancing the performance of intrusion detection systems. This method represents a significant advance in effectively converting datasets into 2D images and integrating them into three-channel RGB color images, demonstrating substantial improvements over traditional grayscale imaging techniques in network intrusion detection.

Furthermore, the research presented by Qazi et al. [8] on the HDLNIDS model, which combines convolutional neural networks (CNN) and recurrent neural networks (RNN), offers a nuanced approach to the dynamic landscape of network threats. Their results indicate that such hybrid models can significantly elevate the accuracy of intrusion detection systems, achieving an average precision of 98.90% in identifying malicious attacks.

These advancements underline a crucial aspect of network security; the evolving complexity of cyber threats requires equally sophisticated detection mechanisms. Deep learning models, with their inherent ability to extract features and recognize patterns, present a compelling solution to this challenge. However, as these models become more complex, considerations regarding their transparency, computational demands, and adaptability to new types of intrusions become paramount [9].

To address the reviewer's comment on the need for more quantitative metric comparisons, we have detailed the precision, recall, and F1-score for each attack type detected by our model. This provides a comprehensive view of the model's performance across different attack scenarios. Additionally, we have included a comparative analysis of our model's performance against other recent studies in the field, highlighting our model's efficacy.

Considering these discussions, our results contribute to a broader understanding of how deep learning can be used to enhance network intrusion detection systems. This also opens avenues for future research, particularly in developing models that balance complexity with interpretability and in creating more adaptive systems capable of responding to constantly evolving cyber threats.

As we move forward, collaboration between cybersecurity practitioners and academic researchers will be key to refining these models, ensuring they remain effective against current and future intrusion tactics. Engaging with emerging datasets and exploring new model architectures will be essential steps in this ongoing journey to secure digital infrastructures against sophisticated cyber threats.

6. Future Research Directions

To further enhance the capabilities of network intrusion detection systems (NIDS), several specific areas warrant further investigation and potential methodological improvements. These include:

  • Adversarial Robustness: With the increasing sophistication of cyber-attacks, it is crucial to develop NIDS that are robust against adversarial attacks. Future research should focus on designing models that can detect and mitigate adversarial examples, ensuring the system's reliability under various threat scenarios. This includes exploring techniques like adversarial training and defensive distillation [58].
  • Real-time Detection and Scalability: As network environments grow in size and complexity, the need for real-time intrusion detection becomes paramount. Future studies should aim to enhance the scalability and efficiency of NIDS to handle high-throughput network traffic without compromising detection accuracy. This involves optimizing model architectures and leveraging high-performance computing resources [59].
  • Explainability and Transparency: The black-box nature of deep learning models poses challenges in understanding their decision-making processes. Future work should focus on developing explainable AI (XAI) techniques for NIDS, enabling cybersecurity professionals to interpret model predictions and make informed decisions. Research into model interpretability and visualization tools will be vital in this regard [60].
  • Integration with Other Security Systems: Effective cybersecurity often requires a multi-layered approach. Future research should explore the integration of NIDS with other security mechanisms, such as endpoint detection and response (EDR) systems, security information and event management (SIEM) systems, and threat intelligence platforms. This integrated approach can provide a comprehensive defense against sophisticated cyber threats [61].
  • Adaptive Learning and Auto-ML: The dynamic nature of cyber threats necessitates NIDS that can adapt to new attack patterns. Investigating adaptive learning techniques, where models continuously learn from new data, and leveraging automated machine learning (Auto-ML) to optimize model selection and hyperparameter tuning can significantly improve detection capabilities [62].
  • Privacy-Preserving Mechanisms: With growing concerns about data privacy, future research should investigate privacy-preserving techniques in NIDS. This includes developing models that can perform intrusion detection while ensuring data confidentiality, potentially through federated learning and homomorphic encryption [63].
7. Conclusion and Prospects

This study has demonstrated the effectiveness of a deep learning model, designed, and evaluated using the KDD Cup 99 dataset, for network intrusion detection. The high performance in terms of precision, recall, and F1 score emphasizes the model's ability to identify intrusions accurately and efficiently, offering a potentially powerful tool in combating cyber threats.

Our results, corroborated by recent research, highlight the added value of deep learning in cybersecurity, specifically in the field of intrusion detection. The adoption of advanced data preprocessing techniques and optimized model architectures was crucial to achieving this level of performance.

However, our research also underscores the need to continue efforts in several directions:

  • Exploration of more recent and diverse datasets: To ensure the relevance and effectiveness of deep learning models in real and varied scenarios, it is essential to integrate current data reflecting contemporary threats.
  • Improvement of model explainability: The opacity of deep learning models remains a challenge. Developing approaches to enhance their transparency would facilitate their adoption and trust in operational contexts.
  • Adaptation to evolving threats: As cyber threats constantly evolve, it is imperative that models can quickly adapt to new forms of attacks, potentially through online or semi-supervised learning.

Finally, this study paves the way for closer collaboration between researchers and cybersecurity professionals to design more resilient and adaptive intrusion detection systems. Continuous innovation in deep learning methodologies, coupled with a deep understanding of the dynamics of cyber threats, will be crucial to strengthening digital defenses in the information age.

  References

[1] Nandanwar, H., Katarya, R. (2024). Deep learning enabled intrusion detection system for Industrial IoT environment. Expert Systems with Applications, 249: 123808. https://doi.org/10.1016/j.eswa.2024.123808

[2] Moustafa, N., Slay, J. (2015). The significant features of the UNSW-NB15 and the KDD99 data sets for network intrusion detection systems. In 2015 4th International Workshop on Building Analysis Datasets and Gathering Experience Returns for Security (BADGERS), Kyoto, Japan, pp. 25-31. https://doi.org/10.1109/badgers.2015.014

[3] Meidan, Y., Bohadana, M., Mathov, Y., Mirsky, Y., Shabtai, A., Breitenbacher, D., Elovici, Y. (2018). N-BaIoT—network-based detection of IoT botnet attacks using deep autoencoders. IEEE Pervasive Computing, 17(3): 12-22. https://doi.org/10.1109/mprv.2018.03367731

[4] Javaid, A., Niyaz, Q., Sun, W., Alam, M. (2016). A deep learning approach for network intrusion detection system. EAI Endorsed Transactions on Security and Safety, 16(9): e2. https://doi.org/10.4108/eai.3-12-2015.2262516

[5] Yin, C., Zhu, Y., Fei, J., He, X. (2017). A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access, 5: 21954-21961. https://doi.org/10.1109/access.2017.2762418

[6] Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A. (2009). A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, Canada, pp. 1-6. https://doi.org/10.1109/cisda.2009.5356528

[7] Sharafaldin, I., Lashkari, A.H., Ghorbani, A.A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy ICISSP - Volume 1, Funchal, Madeira, Portugal, pp. 108-116. https://doi.org/10.5220/0006639801080116

[8] Qazi, E.U.H., Faheem, M.H., Zia, T. (2023). HDLNIDS: Hybrid deep-learning-based network intrusion detection system. Applied Sciences, 13(8): 4921. https://doi.org/10.3390/app13084921

[9] He, N., Zhang, Z., Wang, X., Gao, T. (2023). Efficient privacy-preserving federated deep learning for network intrusion of industrial IoT. International Journal of Intelligent Systems, 2023: 1-22. https://doi.org/10.1155/2023/2956990

[10] Wei, F., Li, H., Zhao, Z., Hu, H. (2023). XNIDS: Explaining deep learning-based network intrusion detection systems for active intrusion responses. In Proceedings of the 32nd USENIX Conference on Security Symposium, Anaheim, CA, USA, pp. 4337-4354.

[11] Ayantayo, A., Kaur, A., Kour, A., Schmoor, X., Shah, F., Vickers, I., Kearney, P., Abdelsamea, M.M. (2023). Network intrusion detection using feature fusion with deep learning. Journal of Big Data, 10(1): 167. https://doi.org/10.1186/s40537-023-00834-0

[12] Kim, T.H., Pak, W. (2023). Deep learning-based network intrusion detection using multiple image transformers. Applied Sciences, 13(5): 2754. https://doi.org/10.3390/app13052754

[13] Alotaibi, A., Rassam, M.A. (2023). Enhancing the sustainability of deep-learning-based network intrusion detection classifiers against adversarial attacks. Sustainability, 15(12): 9801. https://doi.org/10.3390/su15129801

[14] Hnamte, V., Hussain, J. (2023). DCNNBiLSTM: An efficient hybrid deep learning-based intrusion detection system. Telematics and Informatics Reports, 10: 100053. https://doi.org/10.1016/j.teler.2023.100053

[15] Qazi, E.U.H., Almorjan, A., Zia, T. (2022). A one-dimensional convolutional neural network (1D-CNN) based deep learning system for network intrusion detection. Applied Sciences, 12(16): 7986. https://doi.org/10.3390/app12167986

[16] Alavizadeh, H., Alavizadeh, H., Jang-Jaccard, J. (2022). Deep Q-learning based reinforcement learning approach for network intrusion detection. Computers, 11(3): 41. https://doi.org/10.3390/computers11030041

[17] Halbouni, A., Gunawan, T. S., Habaebi, M. H., Halbouni, M., Kartiwi, M., Ahmad, R. (2022). CNN-LSTM: hybrid deep neural network for network intrusion detection system. IEEE Access, 10: 99837-99849. https://doi.org/10.1109/access.2022.3206425

[18] Sharma, B., Sharma, L., Lal, C., Roy, S. (2023). Anomaly based network intrusion detection for IoT attacks using deep learning technique. Computers & Electrical Engineering, 107: 108626. https://doi.org/10.1016/j.compeleceng.2023.108626

[19] Fu, Y., Du, Y., Cao, Z., Li, Q., Xiang, W. (2022). A Deep learning model for network intrusion detection with imbalanced data. Electronics, 11(6): 898. https://doi.org/10.3390/electronics11060898

[20] Qazi, E.U.H., Imran, M., Haider, N., Shoaib, M., Razzak, I. (2022). An intelligent and efficient network intrusion detection system using deep learning. Computers & Electrical Engineering, 99: 107764. https://doi.org/10.1016/j.compeleceng.2022.107764

[21] Zhang, C., Jia, D., Wang, L., Wang, W., Liu, F., Yang, A. (2022). Comparative research on network intrusion detection methods based on machine learning. Computers & Security, 121: 102861. https://doi.org/10.1016/j.cose.2022.102861

[22] Tsimenidis, S., Lagkas, T., Rantos, K. (2021). Deep learning in IoT intrusion detection. Journal of Network and Systems Management, 30(1). https://doi.org/10.1007/s10922-021-09621-9

[23] Otoum, Y., Liu, D., Nayak, A. (2019). DL‐IDS: a deep learning–based intrusion detection framework for securing IoT. Transactions on Emerging Telecommunications Technologies, 33(3). https://doi.org/10.1002/ett.3803

[24] Ravi, V., Chaganti, R., Alazab, M. (2022). Recurrent deep learning-based feature fusion ensemble meta-classifier approach for intelligent network intrusion detection system. Computers & Electrical Engineering, 102: 108156. https://doi.org/10.1016/j.compeleceng.2022.108156

[25] Zhang, C., Costa-Perez, X., Patras, P. (2022). Adversarial attacks against deep learning-based network intrusion detection systems and defense mechanisms. IEEE/ACM Transactions on Networking, 30(3): 1294-1311. https://doi.org/10.1109/tnet.2021.3137084

[26] Apruzzese, G., Pajola, L., Conti, M. (2022). The cross-evaluation of machine learning-based network intrusion detection systems. IEEE Transactions on Network and Service Management, 19(4): 5152-5169. https://doi.org/10.1109/tnsm.2022.3157344

[27] Saba, T., Rehman, A., Sadad, T., Kolivand, H., Bahaj, S. A. (2022). Anomaly-based intrusion detection system for IoT networks through deep learning model. Computers & Electrical Engineering, 99: 107810. https://doi.org/10.1016/j.compeleceng.2022.107810

[28] Yadav, N., Pande, S., Khamparia, A., Gupta, D. (2022). Intrusion detection system on IoT with 5G network using deep learning. Wireless Communications and Mobile Computing, 2022: 1-13. https://doi.org/10.1155/2022/9304689

[29] Rathee, A., Malik, P., Parida, M.K. (2023). Network intrusion detection system using deep learning techniques. In 2023 International Conference on Communication, Circuits, and Systems (IC3S), BHUBANESWAR, India, pp. 1-6. https://doi.org/10.1109/ic3s57698.2023.10169122

[30] Talukder, M.A., Hasan, K.F., Islam, M.M., Uddin, M.A., Akhter, A., Yousuf, M.A., Alharbi, F., Moni, M.A. (2023). A dependable hybrid machine learning model for network intrusion detection. Journal of Information Security and Applications, 72: 103405. https://doi.org/10.1016/j.jisa.2022.103405

[31] He, K., Kim, D.D., Asghar, M.R. (2023). Adversarial machine learning for network intrusion detection systems: A comprehensive survey. IEEE Communications Surveys and Tutorials/IEEE Communications Surveys and Tutorials, 25(1): 538-566. https://doi.org/10.1109/comst.2022.3233793

[32] Meliboev, A., Alikhanov, J., Kim, W. (2022). Performance evaluation of deep learning based network intrusion detection system across multiple balanced and imbalanced datasets. Electronics, 11(4): 515. https://doi.org/10.3390/electronics11040515

[33] Mohammadpour, L., Ling, T.C., Liew, C.S., Aryanfar, A. (2022). A survey of CNN-based network intrusion detection. Applied Sciences, 12(16): 8162. https://doi.org/10.3390/app12168162

[34] Van Rossum, G., Drake, F.L. (2009). Python 3 Reference Manual. In CreateSpace eBooks. https://dl.acm.org/citation.cfm?id=1593511

[35] McKinney, W. (2010). Data structures for statistical computing in Python. In Proceedings of the Python in Science Conferences. https://doi.org/10.25080/majora-92bf1922-00a

[36] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion Bertrand, Grisel Olivier, Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, É. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12: 2825-2830. https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf?ref=https:/.

[37] Keras-Team. (n.d.). GitHub - keras-team/keras: Deep Learning for humans. GitHub. https://github.com/fchollet/keras.

[38] Abadi, M., Agarwal, A., Barham, P., et al. (2016). TensorFlow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467v2. https://doi.org/10.48550/arxiv.1603.04467

[39] McHugh, J. (2000). Testing intrusion detection systems. ACM Transactions on Information and System Security, 3(4): 262-294. https://doi.org/10.1145/382912.382923

[40] Dhanabal, L., Shantharajah, S.P. (2015). A study on NSL-KDD dataset for intrusion detection system based on classification algorithms. International Journal of Advanced Research in Computer and Communication Engineering, 4(6): 446-452. https://e-tarjome.com/storage/btn_uploaded/2019-07-13/1563006133_9702-etarjome-English.pdf.

[41] Lakhina, A., Crovella, M., Diot, C. (2004). Diagnosing network-wide traffic anomalies. In Proceedings of the 2004 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 219-230. https://doi.org/10.1145/1015467.1015492

[42] Sommer, R., Paxson, V. (2010). Outside the closed world: on using machine learning for network intrusion detection. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, Canada, pp. 1-6. https://doi.org/10.1109/sp.2010.25

[43] Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A. (2009b). A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, Canada, pp. 1-6. https://doi.org/10.1109/cisda.2009.5356528

[44] Brownlee, J. (2016). Master machine learning algorithms. Machine Learning Mastery. http://books.google.ie/books?id=n--oDwAAQBAJ&printsec=frontcover&dq=Master+Machine+Learning+Algorithms:+Discover+How+They+Work+and+Implement+Them+From+Scratch&hl=&cd=1&source=gbs_api.

[45] Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. MIT Press. http://books.google.ie/books?id=omivDQAAQBAJ&printsec=frontcover&dq=Deep+Learning.&hl=&cd=1&source=gbs_api.

[46] Géron, A. (2019). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. O’Reilly Media, Inc. http://books.google.ie/books?id=HnetDwAAQBAJ&printsec=frontcover&dq=Hands-On+Machine+Learning+with+Scikit-Learn,+Keras,+and+TensorFlow&hl=&cd=1&source=gbs_api.

[47] Nair, V., Hinton, G.E. (2010). Rectified linear units improve restricted Boltzmann machines. In International Conference on Machine Learning, Haifa, Israel, pp. 807-814. https://icml.cc/Conferences/2010/papers/432.pdf.

[48] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1): 1929-1958. https://jmlr.csail.mit.edu/papers/volume15/srivastava14a/srivastava14a.pdf.

[49] AlDahoul, N., Abdul Karim, H. Ba Wazir, A.S. (2021). Model fusion of deep neural networks for anomaly detection. Journal of Big Data, 8: 106. https://doi.org/10.1186/s40537-021-00496-w

[50] Dasgupta, D., Akhtar, Z., Sen, S. (2020). Machine learning in cybersecurity: A comprehensive survey. Journal of Defense Modeling and Simulation, 19(1): 57-106. https://doi.org/10.1177/1548512920951275

[51] Apruzzese, G., Colajanni, M., Ferretti, L., Guido, A., Marchetti, M. (2018). On the effectiveness of machine and deep learning for cyber security. In 2018 10th International Conference on Cyber Conflict (CyCon), Tallinn, Estonia, pp. 371-390. https://doi.org/10.23919/cycon.2018.8405026

[52] Cortes, C., Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3): 273-297. https://doi.org/10.1007/bf00994018

[53] Disha, R.A., Waheed, S. (2022). Performance analysis of machine learning models for intrusion detection system using Gini Impurity-based Weighted Random Forest (GIWRF) feature selection technique. Cybersecurity, 5(1). https://doi.org/10.1186/s42400-021-00103-8

[54] Yang, N.L. (2024). Design and performance evaluation of network intrusion detection system based on deep learning. Deleted Journal, 20(7s): 1479-1591. https://doi.org/10.52783/jes.3728

[55] Lu, G., Tian, X. (2021). An efficient communication intrusion detection scheme in AMI combining feature dimensionality reduction and improved LSTM. Security and Communication Networks, 2021: 1-21. https://doi.org/10.1155/2021/6631075

[56] Chintapalli, S.S.N., Singh, S.P., Frnda, J., Divakarachari, P.B., Sarraju, V.L., Falkowski-Gilski, P. (2024). OOA-modified Bi-LSTM network: An effective intrusion detection framework for IoT systems. Heliyon, 10(8): e29410. https://doi.org/10.1016/j.heliyon.2024.e29410

[57] Babu, C.S., Hruday, B.S.S.S., Krishna, J.V.V.S., Sandeep, C., Naveen, B. (2024). IoT threat mitigation: Leveraging deep learning for intrusion detection. Journal of Advanced Zoology, 45(3): 801-810. https://doi.org/10.53555/jaz.v45i3.4132

[58] Papernot, N., McDaniel, P., Sinha, A., Wellman, M. (2016). Towards the science of security and privacy in machine learning. arXiv:1611.03814v1. https://doi.org/10.48550/arxiv.1611.03814

[59] Zhao, S., Chandrashekar, M., Lee, Y., Medhi, D. (2015). Real-time network anomaly detection system using machine learning. In 2015 11th International Conference on the Design of Reliable Communication Networks (DRCN), Kansas City, MO, USA, pp. 267-270. https://doi.org/10.1109/drcn.2015.7149025

[60] Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5): 206-215. https://doi.org/10.1038/s42256-019-0048-x

[61] Bhuyan, M.H., Bhattacharyya, D.K., Kalita, J.K. (2014). Network anomaly detection: Methods, systems and tools. IEEE Communications Surveys and Tutorials/IEEE Communications Surveys and Tutorials, 16(1): 303-336. https://doi.org/10.1109/surv.2013.052213.00046

[62] Feurer, M., Hutter, F. (2019). Hyperparameter Optimization. In the Springer Series on Challenges in Machine Learning, pp. 3-33. https://doi.org/10.1007/978-3-030-05318-5_1

[63] Bonawitz, K.A., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konecný, J., Mazzocchi, S., McMahan, H.B., Van Overveldt, T., Petrou, D., Ramage, D., Roselander, J. (2019). Towards federated learning at scale: System design. arXiv:1902.01046. https://doi.org/10.48550/arxiv.1902.01046