Hybrid Architecture for Distributed Intrusion Detection System

Intrusion detection can be stated as a process for observing, analyzing and taking decisions for abnormal activities which can be considered as violations of policies of network environment [1]. It mostly supports firewalls to work faster and in more accurate way. Sometimes IDS can detect the abnormal activities which cannot be detected by the firewalls. Denning [2] had provided the framework of IDS to detect the various kinds of attacks on networks, which explains how audit reports can be used to find out abnormal patterns to detect security violations. Different types of attacks or intrusions are mostly used by intruders to interrupt the network traffic for degrading network performance. Sudden change in the network traffic can affect the security of network. According to Yan, et al. [3], by observing pattern of actions intrusions can be find, which tries to compromise confidentiality, integrity or availability (CIA) of information resources; so it’s required to take action against such attempts to reduce security risks in network. Data mining and machine learning techniques can be used to build smart IDS which can minimize risk of intrusion.

One should aim to create a Novel IDS which incorporates the methodologies of Data Mining, Machine Learning and Artificial Intelligence to identify the attacks in the network correctly with very less number of misclassifications which otherwise would go unidentified in traditional Intrusion Detection Systems. The following section highlights the technologies that can be used to achieve better accuracy for detection and reduce false alarm rate.

1.1 Misuse-based IDS

Misuse based detection method is used for detection of known attacks. Definition of attacks is observed as signature in this type of detection. Signatures are represented in format such as change in packet payload, shutting down of process automatically or actions followed by other actions etc. Signature database is created to store all the signatures together to analyze normal and abnormal behavior of traffic or packets. Misuse type of IDS concentrates on matching the signature from database with capture packet rather than analyzing the source of attack generation. Analysis is done to find whether the packet is normal or malicious. Big challenge for this type of IDS is to collect signatures of all types of known attacks. Most of the IDS following signature database shows slow performance in attack detection as the size of database is large which stores signatures of multiple attacks. Due to this most of the attacks which works in distributed fashion can harm network. Signature based IDS shows good performance of all known attacks whose signature is available in database, but shows poor performance for unknown attacks whose signature is not available in database [4].

1.2 Anomaly-based IDS

To eliminate the limitation of signature based IDS which is unable to detect the unknown attacks, anomaly based IDS can be used. Intrusion detection in this IDS is done by observing the abnormal behavior of the network traffic. This method train classifier according to normal behavior of the network such as packets arriving per unit time, packet payload values or behavior of network application and traffic. Packets are classified as normal if it matches to the normal behavior of the network which is checked with the rules of identification, otherwise it is considered as the abnormal behavior and malicious packets which can be analyzed as attacks. Once the attack packet is detected alarm is generated to give indication to administrator about this malicious activity. Challenge in this type of IDS is to decide the normal behavior of network and to set threshold when to generate alarm. In this type of IDS sometimes wrong alarms can be generated by little deviation of network traffic from normal. Wrong alarm leads towards more false alarm generation; which can be responsible for poor performance of system and less detection rate [4].

Several Machine Learning algorithms, neural network [5-6], Support Vector Machine [5-6], Clustering Algorithms [7] with Artificial Intelligence, Random forest [8], Genetic algorithms [9] and ensemble of most classifiers are used by many researchers to improve accuracy of detection and reduce false alarm rate. In depth knowledge of KDD dataset is presented by Hasan, et al. [5] where four types of attacks were used as Basic, Content, Traffic and Host. Detection Rate (DR) and false alarm rate is used as evaluation metrics to check the performance of classifier. Authors create 15 subsets of instances using all attributes into above mentioned 4 classes. Most importance is given for reducing false alarm rate and increasing detection rate by class dominance. Farnaaz, et al. [8] explained various algorithms where a j48 algorithm is compared with random forest classifier. Dataset used is NSL-KDD to train the classifier. After pre-processing dataset is clustered into attack classes. Symmetrical uncertainty measure is used for reducing feature in pre-processing stage. 100 trees are created using random forest classifier to compare results using detection rate and false alarm rate.

Roberto, et al. [10] used ensemble of multiple SVM to create an IDS. 2v-grams technique is used to do analysis of packet payload. Value of v is obtained by a sequence of 2-grams which are extracted from payload at particular interval. Probability of attack in normal traffic is analyzed by varying value of v. Feature reduction clustering algorithm is used for representation of payload in feature space. SVM is modeled with the help of representation of feature space in training phase. Packet detection probability is found by SVM in testing phase. Authors adopt non trainable combiner to combine all probabilities, analyzed upon maximum, minimum, average and product. Model output the final probability of packet being normal. Finally, if value of probability is above threshold it outputs as normal packet.

An ensemble approach by combining SVM, DT, ANN and k-NN is used by Borgi [11] to create IDS. Dataset used to train and test all classifiers is DARPA. All the classifiers are compared to check the detection accuracy and false positive rate. Output is found by applying combination rule, majority voting rule and average rule to each classifier. Belief function is used to find the probability of class assigned to packet. Current IDS does signature based detection [12] which requires large amount of data. To calculate accuracy IDS performance plays important role which depends on data and features used for detection. Performance of IDS can be degraded if inaccurate data and features are given to the classifier as input. To avoid this most of the feature reduction and dimensionality reduction techniques such as data filtering, data clustering or feature selection are used. In this work feature selection technique is developed to reduce number of features use for prediction by classifier.

It is proved that ensemble of classifiers gives better result as compared to single classifier, whereas a novel approach which uses two classifiers SVM and KNN in ensemble approach presented in [13]. Dataset used to train and test classifiers is KDD’99. Combination rule is used to combine predictions of base classifiers. Author used techniques as combination function such as particle swarm optimization (PSO), variant of PSO and weighted majority algorithm. Experimental results show the improved classification accuracy given by ensemble classifiers as compared to base classifier alone.

Jha, et al. [14] explained the system created using an SVM classifier. Combination of Information Gain and k means is used for feature selection, where information gain is used to calculate importance of each feature. Features are selected by Information gain according to gain and then selected using k means algorithm. Authors calculated the accuracy by reducing the features to 23 and 30; which shows the accuracy changes by 0.05 %.

Jungsuk, et al. [15] test the real traffic obtained from Kyoto University. Unsupervised approach is further extended in order to improve the results without any intervention by the users. In this approach users are not setting any parameters in filtering and clustering phases. It is assumed by system. Number of clusters to form and ration of attacks with network traffic is automatically modeled by system in which both the approaches are compared. Automatic approach is found to be superior as compared to user intervention approach.

The analysis done by Choi, et. al. [16] shows that 41 features of KDD dataset are not relevant by using, information gain, Gain ratio and Correlation based features selection. To reduce feature attribute ratio with decision tree is used to find the importance of the feature which is compared with Information Gain and Gain Ratio. Authors proved that accuracy obtained by system is better after reducing 41 features to 22 features. Process of weighted majority algorithm is explained by Littlestone and Warmuth [17], which assigned right to the prediction given by each classifier used for finding final prediction class.

An unsupervised framework based on random forest is presented by Jiong, et al. [18]. The network traffic was captured to create dataset to train random forest algorithm in off-line mode in preprocessing phase.  Random forest requires more computational cost if it is used on-line to test the network traffic. Random forest is an ensemble of several trees generated by different bootstrap samples from the dataset. Different bootstrap samples are used to classify packet as normal or attack. Random forest ensemble votes from each class to get the final prediction about the packet.KDD’99 dataset is used to find accuracy and detection rate using random forest. Author proved that the unsupervised framework using random forest gives higher detection rate and low false positive rate.

Hasan et al. [19] implements IDS using SVM and Random Forest classifier. It is observed that SVM and Random Forest provides same accuracy; however Random Forest provides better computational time as compared to SVM. Accuracy of 92.99 % is obtained while testing SVM which use radial basis kernel method. Accuracy for Random Forest for testing is 91.41 %. Processing time required for attack detection is less for Random Forest as compared to SVM. Sindhu, et al. [20] proposed a novel architecture based ensemble of neural networks. KDD’99 dataset is used to train and test classifier. Detection accuracy of model is found using optimal subset method and Genetic algorithm. Model functionality is validated using detection accuracy fitness function. Complexity increases as neural networks classifiers are used for training models. Ada-boost algorithm is used to ensemble the classifiers. C4.5 is used is used to reduce the complexity of neural network. Experimental results show that classifiers decision tree and neural network if used alone is less as compared to ensemble approach if analyzed on parameters such as True positives, False positives, precision, recall and F-measure.

Unsupervised approach is used by many researchers to test various data mining algorithms for IDS system. Jungsuk, et al. [21] elaborates another unsupervised framework based on various data mining techniques. This framework consists mainly three phases i.e., filtering, clustering and modeling. This framework depends on the parameters set by the user in training and testing phase. In training phase the attack packets are filtered and eliminated from the training dataset. Elimination process is accomplished by algorithm working on notion of density, which uses the ration of attacks with respect to network traffic. This parameter is specified by user in training phase.  After filtering data clustering is performed on training dataset. Number of clusters to be obtained is decided by user which forms clusters for network traffic. In final phase a one-class SVM is trained for each cluster formed on the dataset. In testing phase, packets are evaluated by ensemble of SVMs obtained in previous phases. Packets are predicted as normal for any of the one-class SVM represents it otherwise consider as malicious packet.

Clustering algorithm BIRCH is used as classifier by Horng et al. [22]. KDD’99 dataset is divided into five parts to process with BIRCH depending on their principal classes. Principal classes are Probe, Dos, U2R, R2L and normal traffic. BRICH generate one clustering tree for each class. Feature tree which is a compact representation of dataset; represents whole dataset as a tree where each leaf node represents cluster. SVM is trained for each cluster representing one attack class. All SVMs are ensemble to get results in terms of accuracy. It is proved that ensemble SVM gives better accuracy as compared to decision tree and k-means. According to work performed by many researchers as discussed above, it is witnessed that, in today’s scenario it is difficult to handle most of the new attacks happening on network. Anyone can handle the attacks if it is known, but if unknown or anomaly type attack happened on system, single classifier fails to handle that attack. Accuracy can be increased by ensembling number to classifier together. It also supports in reducing false positive rate. Accurate attack detection is considered as true positive else false positive if misclassified. Hence the main objective behind the proposed work is use hybrid model to improve the detection accuracy of anonymous attacks by increasing rate of true positive and reducing rate of false positive.

On completing the modeling of the classifiers, training and testing them, the next step is to deploy them in a distributed network. For this purpose, a distributed architecture needs to be developed depicting how the system will function in the network in a distributed manner. Figure 2 shows the architecture of the system.

As seen in Figure 2, data packets in this system are captured through internal for live data. Most the network has switch/Hub at the entrance which allows the traffic or traffic flow coming from internet to enter into the network. Switch/Hub are the devices which are responsible to route the packets to the intended host. In the proposed system first all the packets where passed through firewall to avoid the malicious activities. The attack packets which are not recognized by firewall is detected using various classifiers installed in distributed manner. Packets entered in the network through the centralize server on which packet feature extraction is done with the help of KDD-Extractor. After extraction of features only some features where selected and passed to the classifiers which are working as individual node for classification. Each of the node use different classifier to predict the type of attack. The data is send to each node where classifier such as SVM, Random Forest, Naïve Bayes, KNN, DT, Logistic Regression and Stochastic Gradient Descent is used to analyze each packet of traffic. Each classifier generates the output and send to ensemble module where majority voting algorithm is used to find the final prediction of packet either attack or normal. For example, if the nodes running Naïve Bayes and random forest predict that the set of packets are malicious and the SVM node predicts it as normal, then a majority voting amongst them would classify the packet as malicious. From this discussion, it can be deduced that if we used only one classifier for prediction, say SVM, the packet would be misclassified which would damage the security of the network. From experimental results it is observed that prediction accuracy increases significantly if we ensemble the output of all classifiers together rather than using single classifier for prediction. The node on which attack is done is responsible for creating signature of the unknown attack and send back signature to center node. Center node save signature in its database and share it to remaining nodes so that it can be used for future attack detection.

figure_2.jpg

Figure 2. Proposed system architecture

5.1 Performance evaluation

As shown in Table 3 TP and TN articulate us when the classifier is classifying tuple correctly, while FP and FN tell us when the classifier is misclassifying tuples. To check the performance of proposed system real time traffic consisting of more than 1.5 lakhs packets were captured to develop values in confusion matrix as shown in Table 4. Each packet information is saved in .pcap file for testing using individual and hybrid classifier. This file is tested by classifiers to get the values mentioned in Table 4 as per confusion matrix shown in Table 3.

Observations shown in Table 4 are tested on all classifier in the real time environment. The input packet is taken from the network to test the distributed denial of services attack. Packets used for testing proposed model are more than 1.5 lakhs packets. Preprocessing and normalization is done for each classifier as per the requirement of classifier. Each classifier is train separately with NSL KDD dataset. Each classifier is tested separately to find detection accuracy and false alarm rate. From these observations it is seen that the random forest predicts more amount of packets correctly as compared to support vector machine and other classifiers. Firstly the true positive and false positive packets generated by each classifier are estimated and finally tried to estimate with the proposed hybrid system. From the observations presented in Table 4, it is concluded that proposed hybrid approach gives better rate of correct prediction of attack as compared to any of the single classifier. Less number of packets are misclassified in hybrid model as compared to single classifier which help us to increase detection rate of packets in real time environment.

5.2 Receiver operating characteristics curves

ROC is a tool for comparing two or more classifiers and represents it in the form of curves. ROC curve is a useful visual tool for comparing two or more classification models. ROC curves can represent the trade-off between true positive rate (TPR) and false positive rate (FPR) on y and x axis respectively for a given model. TPR is an amount of positive (C1) tuples which are correctly classified and labeled by model; FPR is an amount of negative ( ) tuples that are misclassified and mislabeled as positive instead of negative as describe in Table 3.

Table 4. Tested values of confusion matrix for all classifiers

SVM (Figure 3(a)) classifier shows the variation rate of true positive with rate of false positive. As curve is not close to the left hand border, results in less area under curve leads to less accurate outcome. From Figure 3 (b) it is observed that curve for NB is far away from left and right hand border which leads to less accuracy. In RF curve is closer to both the borders but as compared to DT it gives less accuracy as shown if Figure 3(c). Logistic Regression is having less accuracy as shown in ROC curve of Figure 3(d). Figure 3(e) shows the area covered under ROC curve for KNN which is closer to left hand border leads towards more accuracy. Figure 3(f) shows ROC for DT which is much closer to the left hand border and top border tends to more accurate results. From Figure 3(g) we can observe that accuracy reduce drastically for SGD. From Figure 8(h) it is seen that ROC curve for hybrid model covers more area under curve as it is closer to left hand border and on the top border. These curves are used to check the accuracy of each classifier with the help of confusion matrix.

Table 4 present the confusion matrix. Eqns. (5-7) are used to estimate accuracy, true positive rate and false positive rate.

Accuracy=(TP+TN)/TotalObservation (5)

TPR=TP/C₁ (6)

$FPR=FP/\overline{{{C}_{1}}}$ (7)

From Figure 3 (a) and Table 5 it is observed that SVM results in misclassification when it comes to unknown attack so it gives only 0.89 accuracy. Biased prediction can be given by SVM as it is a binary classifier. To avoid it proper preprocessing need to carry out for SVM. Naive Bayes classifier shows less accuracy of 0.82 with more false alarm rate of 0.34 with tends to more misclassification of packets. For RF the accuracy is better it is up to 0.97 if the number of trees is more. Detection accuracy increases as more umber of trees used. Time require to train will be more as we increase number of trees which is not feasible with live data. Logistic Regression shows less accuracy as compared to KNN algorithm as both are used to find change in behavior pattern of traffic. For KNN accuracy is up to 0.96, but again it fails to classify unknown attacks by checking behavior. Reduced false alarm rate id observed with help of KNN. For DT classifier its gives accuracy up to 0.97, but most of the time output can be biased as DT cannot handle overfitting of data in the large size of input. As the input is taken from internet lakhs or packets travel through the network in minimum time so to handle overfitting decision tree need to be used with some other classifier to improve classification accuracy. SGD also fails to improve accuracy as compare to remaining classifiers. After observing the results we can conclude that none of the single classifier is able to provide high detection rate with less false alarm rate. A new proposed system is develop called as hybrid system which will remove all the lacunas from mentioned classifiers. In proposed hybrid system the output of all classifiers is ensemble and tested on real time data to check the accuracy. This system obtained accuracy up to 0.98 (98%) with reduced false alarm rate of 0.09. This system is able to analyze known as well as unknown attacks with high accuracy.

figure_3a.jpg

a. ROC curve for SVM 

figure_3b.jpg

b. ROC curve for NB

figure_3c.jpg

c. ROC curve for RF 

figure_3d.jpg

d. ROC curve for LR

figure_3e.jpg

e. ROC curve for KNN 

figure_3f.jpg

f. ROC curve for DT

figure_3f2.jpg

f. ROC curve for SGD

figure_3g.jpg

g. ROC curve for Hybrid

Figure 3. (a-g) Receiver operating characteristics curves

Table 5. Comparison of classifier output with proposed hybrid model

Table 6 shows the comparison of other classifiers with proposed system. Aburomman et al. [25] explains various methods to obtain more accuracy with less false alarm rate that is false positive rate by ensembling more classifiers rather than using single classifier. Data mining supervised, unsupervised and semi supervised algorithms can be used to increase accuracy. In [26] authors use k means algorithm with two level hybrid approach to increase the detection accuracy. Multi-level SVM-EML model is used in hybrid manner in [27] to classify known attacks. Authors also try to observe behavior of network for anomaly detection using adaptive SVM model. Accuracy achieved is 95.86 % with using all 41 features of KDD99 dataset. Multi-class SVM is used by authors in [28] to increase accuracy by ensembling multiple SVMs together. Authors explained a hybrid approach [29] by combination of strangeness isolation and l nearest neighbor algorithm which is used for detection of all four types of attacks from KDD dataset. They obtained 95.1 % accuracy with 3.0 false alarm rate. In [31] authors used fuzzy rule-based system for proposing novel framework based on agent based strategy. It also used multi objective technique for improving detection accuracy. In [32] authors make use of three tier architecture for cleaning and preprocessing of data followed by SVM classifier for testing data. Accuracy obtained is 94.71 with 3.8 false alarm rate. Use of various techniques in collaborative and distributed intrusion detection system results in improving accuracy of detection [33]. Authors [34] used KNN algorithm for detection of misuse and anomaly based attack detection and obtained 93.29 % accuracy and 0.78 false alarm rate. From Table 7 and Figure 9 it is proved that single classifiers provide less accuracy of detecting signature and anomaly attack. Less true positive rate and falser positive rate is result of less accuracy. The ensembling assistances in improving the accuracy, true positive rate and reduces false positive rate.

Table 6. Comparison of proposed system with other classifiers