A Federated Metaheuristic Optimized Deep Learning Framework for Privacy-Preserving Intrusion Detection and Internet of Things Security Analytics

The rapid growth of the Internet of Things (IoT) has created a huge number of connected devices which can produce enormous amounts of diverse data within domains such as healthcare, smart cities and industrial automation. The speed at which IoT is growing continues to accelerate and needs new techniques to support intelligent decision-making. Therefore, it is urgent that we apply more sophisticated analytical methods and use Deep Learning (DL) to discover valuable insights in these massive volumes of heterogeneous data across multiple domains. Because traditional centralized learning methods require transferring all raw data from the end-user IoT devices to be stored on cloud servers, there are many risks related to transmitting data; risks include concerns related to privacy and security, as well as large amounts of communication overhead. Federated Learning (FL) is an effective solution to support decentralized training in distributed systems using a federated approach for training models. Instead of sending and transferring raw data to a central location, each of the IoT devices holds its own datasets locally, and then collectively trains a global shared model, keeping data on their devices at all times, thereby preserving privacy and mitigating the risks associated with transmitting data over the network [1].

Many researchers have examined the use of FL in various IoT environments and have reported that FL can effectively improve privacy and security when using distributed systems [2]. While FL helps preserve user privacy and security, it is also vulnerable to privacy loss through updates sent to the shared global model and suffers from issues related to data heterogeneity, communication inefficiencies, and slow model convergence [3]. FL frameworks have widely adopted diffusion privacy in order to help reduce risks associated with privacy. An IoT system's local differential privacy-based FL model [4] exhibits a high degree of data protection. In addition, an IoT-based FL hybrid-differential privacy method [5] has also enhanced security via IoT systems. These approaches have been continuously researched, and more recently, researchers have successfully combined differential privacy and FL in order to create a framework for cross-platform secure knowledge sharing [5, 6]. Although these approaches mitigate privacy risk, they tend to result in a compromise between privacy and model efficacy in IoT environments that are resource-starved [7].

Some of the challenges identified by several reviews and surveys regarding FL for IoT systems include non-optimized communications, scalability issues, and limited adaptation to waterfall models [8]. Research efforts have recently begun to integrate meta-learning and more advanced privacy-preserving technologies in order to fill these deficiencies via FL [9]. Nonetheless, all of these existing approaches were designed to either enhance privacy or enhance model efficacy but did not simultaneously address multiple conflicting objectives such as model Accuracy, Diversity, Adaptability (ADA), communication costs, and privacy preservation. One important area that current FL frameworks lack is the ability to adaptively optimize their parameters [10]. Most current models use a static hyperparameter setup, which does not work well in dynamic IoT environments where data is non-independent and non-identically distributed.

There is now a much larger attack surface available for bad actors to take advantage of due to the increasing volume of IoT and Industrial Internet of Things (IIoT). Thereby exposing systems to threats such as Denial of Service, Botnet (BoT) attacks and Reconnaissance attacks. Traditional centralized IDS are not able to function well in an IIoT environment due to issues such as lack of privacy, increased communication costs, and the inability of the system to expand its capability. Consequently, there is now a growing demand for FL-IDS as an alternative to traditional IDS. FL-IDS allows for distributed and privacy-preserving security analytics at the edge of the network rather than centralized.

Optimization methods have been developed for use in traditional machine learning, but your research into how those techniques fit into a federated framework, especially with regard to IoT data analytics, is still in its infancy. To address these limitations, we offer a new federated metaheuristic-optimized DL framework for privacy-preserving analytics in IoT environments. This new framework employs a unified approach that integrates FL and an adaptive optimization mechanism to enable dynamic tuning of model parameters, client participation and privacy budgets. The main benefit of our proposed approach is that it is capable of meeting multiple objectives, including maintaining the ADA of the created models, preserving user privacy, and minimizing communications costs at the same time, as well as providing a solution scalable enough for an IoT setting.

IDS currently used in IoT are dependent on centralised data, which leads to insecurity due to the risk of private information being disclosed, the possibility of delayed detection, as well as excessive communication overhead in real-time industrial applications. This study presents a method of providing a secure and effective means of intrusion detection via a framework for security analytics utilizing FL.

This research makes the following major contributions:

• Designed a federated DL framework to enable privacy-preserving analytics in the IoT using a metaheuristic optimization for adapting hyperparameter tuning and selecting clients.
• A multi-objective optimization approach for balancing the trade-offs between ADA, privacy, and communication costs. And the use of differential privacy techniques to protect against data exposure.
• A comprehensive experimental validation demonstrating the advantages of our methodology over many of the existing methods. The remainder of this paper is organized as follows: Section 2 provides a review of the related work with research gaps. Section 3 presents the proposed methodology and mathematical model. Section 4 discusses the experimental setup, dataset details, results and discussions. Finally, Section 5 concludes the paper.

The Federated Metaheuristic Optimized Deep Learning (FMODL) framework will allow users to do much more than just monitor their networks. With the FMODL framework, users will have the ability to analyze their network traffic for malicious activity such as Denial-of-Service (DoS) attacks, DDoS attacks, and reconnaissance attacks, while still protecting user privacy and personal information. This paper presents an efficient FMODL framework to overcome several IoT analytics challenges, including the issues of data privacy, communication overhead, and model adaptability. FMODL combines several technologies: i.e., FL for decentralized training of a model, metaheuristic optimization for adjusting the trained model’s parameters adaptively, and differential privacy to provide a secure communication mechanism. By using a multi-objective optimization strategy, the proposed FMODL model provides a means for balancing and optimizing ADA, privacy, and communication efficiency in the rapidly changing environment of IoT devices. Figure 1 shows the detailed design architecture of the proposed FMODL framework.

tu_pian_1.png

Figure 1. Proposed Federated Metaheuristic Optimized Deep Learning (FMODL) framework design diagram

This depicts an overview of an FL approach within an IoT setting that allows multiple devices to train local models based on their own data without direct sharing of it. To enhance model performance, each device employs metaheuristic strategies including the use of PSO and GWO, followed by implementing differential privacy to ensure privacy of the data.

Before model updates are sent from the devices to a central server for aggregation, the models' size is reduced via model compression to minimize the communication overhead associated with sending the model updates over a secured channel of communication. Once sent to the server, each respective federated aggregated model is then sent back to the devices to allow for continued training and enhancement of the local models based on the performance improvements achieved through FL methods. The data that the i-th client has is represented as ${{D}_{i}}$ by the formula presented in Eq. (1).

${{D}_{i}}=\{\left( {{x}_{j}},{{y}_{j}} \right)\}_{j=1}^{{{n}_{i}}}$           (1)

where, ${{x}_{j}},~{{y}_{j}}$ are the input feature vector and label of data point j, respectively, and ${{n}_{i}}$ is the number of samples that client i has. The data pre-processing steps done before training using the proposed model include data cleaning. Additionally, before using the raw input data obtained from IoT devices as the raw form of input with respect to the proposed model algorithm, the process of normalizing by Eq. (2) is also completed. The $x'$ will have a mean of 0 and a unit variance which improves convergence during the training phase and stabilizes the entire learning process across the multiple clients by determining how much each feature contributes to the overall model.

  ${x}'=\frac{x-\mu }{\sigma }$       (2)

where, the output $x'~$is returned when $x$ (the raw input feature vector from the IoT devices) is transformed by normalizing it using the mean $\mu $ and standard deviation $\sigma $ of the dataset.

Eq. (3) defines how the input interacts as it passes through each layer of the DL architecture. The term ${{h}^{l}}$ shows the output of the l-th layer, while ${{h}^{l-1}}$ indicates the output from the previous layer.

$h^{(l)}=\sigma\left(W^{(l)} h^{(l-1)}+b^{(l)}\right)$       (3)

where, ${{W}^{l}}$ refers to the weights between layers (l-1) and l, and ${{b}^{l}}$ is the bias vector for layer l. The function $\sigma \left( \cdot  \right)$ is a nonlinear activation function such as ReLU or sigmoid, which adds non-linearity to the system.

An entirely interconnected DNN is utilized for classification by each IoT client utilizing a local DL model. The network contains an input layer that matches the feature dimension of the dataset, three hidden layers containing 128, 64 and 32 neurons respectively, with ReLU activations applied to all neurons within the hidden layers. The application of the ReLU activation function ensures that there are non-linear features to be learned by the model and allows for a faster rate of convergence. A dropout regularization factor of 0.3 is applied to all hidden layers to mitigate overfitting within a heterogeneous and non-IID IoT environment. A softmax activation function is used within the output layer to perform multi-class classification. The model is trained using the Adam optimizer with categorical cross-entropy as the loss function, with a batch size of 64 (varying from 32–128 depending on resource availability) and 3–5 local epochs per round of federated communication. The learning rate is dynamically adapted to be within the range of [0.0001, 0.01] utilizing the proposed Particle Swarm Optimization-Grey Wolf Optimization (PSO-GWO) optimization strategy, and weight initialization using Xavier initialization improves convergence stability. All clients utilize the same architecture to ensure that all federated nodes and datasets are fair and reproducible. Therefore, the use of this function assists with the development of the model from the IoT data, allowing for identification of complex patterns formed by data during the local training phases. The loss function is given by Eq. (4).

${{\mathcal{L}}_{i}}=\frac{1}{{{n}_{i}}}\underset{j=1}{\overset{{{n}_{i}}}{\mathop \sum }}\,\ell (f({{x}_{j}};w),{{y}_{j}})$       (4)

where, ${{n}_{i}}$ represents the total number of training samples available for client i, which allows the prevalence of loss to be equalized across all local data. Each $\left( {{x}_{j}},{{y}_{j}} \right)$ pair corresponds to the feature vector and label. The function $f\left( {{x}_{j}};w \right)$ gives the output of the DL architecture by applying the parameters w. And, the function $l\left( \cdot  \right)$ is the loss function to use the DL model is built to learn by minimizing prediction error using IoT data that are readily available to the training clients.

The model update rule defines the new global model parameters after each communication round. The model updating process is given in Eq. (5).

$w_{i}^{t+1}={{w}^{t}}-\eta \nabla {{\mathcal{L}}_{i}}\left( {{w}^{t}} \right)$        (5)

where, the new local summary of the model is defined by the local model parameters ${{w}^{t}}$ at client i after each communication round. The learning rate is defined as $\eta $, which defines the magnitude of the update step. The gradient $\nabla {{L}_{i}}\left( {{W}^{t}} \right)$ points in the direction that the loss function is increasing the most. Therefore, subtracting the gradient will move in the direction of the minimum of the loss function. Each client performs this update locally so that the FL framework learns in a decentralized mode.

The proposed framework incorporates a hybrid metaheuristic method for optimization, which combines PSO and the GWO to dynamically tune the hyperparameters of an ML model such as learning rate, batch size, and network configuration. PSO allows exploration of the global search space for solutions, and GWO will allow exploitation of the local search space, thus leading to faster convergence and better ADA. The hybrid optimization strategy has been found especially useful for addressing non-IID and dynamic data distributions from IoT devices.

3.1 Hybrid Particle Swarm Optimization-Grey Wolf Optimization mechanism

The Particle Swarm Optimization equation defines how candidate solutions are updated by Eqs. (6) and (7).

$v_{i}^{t+1}=\omega v_{i}^{t}+{{c}_{1}}{{r}_{1}}\left( pbes{{t}_{i}}-x_{i}^{t} \right)+{{c}_{2}}{{r}_{2}}\left( gbest-x_{i}^{t} \right)$         (6)

$x_{i}^{t+1}=x_{i}^{t}+v_{i}^{t+1}$          (7)

where, the particle velocity is given by the equation $v_{i}^{t}$ and its current position is defined by the equation $x_{i}^{t}$ as potential candidates for solutions. $pbes{{t}_{i}}$ denote the local best position and $gbest~$global best position, we use the letters $p$ and $r~$for the control parameters $\omega $, ${{c}_{1}}$ and ${{c}_{2}}$ to determine how much exploration and exploitation will occur, and we use the random variables ${{u}_{1}}$ and ${{u}_{2}}$ in the range of 0 to 1. The position update generated by the GWO is achieved by means of Eqs. (9) and (10).

$D=\mid C\cdot {{X}_{p}}-X\mid $        (8)

$X\left( t+1 \right)={{X}_{p}}-A\cdot D$       (9)

${{X}_{new}}=\frac{{{X}_{\alpha }}+{{X}_{\beta }}+{{X}_{\delta }}}{3}$        (10)

where, ${{X}_{\alpha }}$ denotes the first best candidate solution, ${{X}_{\beta }}~$the second and ${{X}_{\delta }}$ the third best candidate solutions. These will be the best candidates to lead the leadership structure in GWO. With respect to the control parameters, A and C will allow for movement to be made towards the optimal solution and provide sufficient exploration and exploitation of the space of solutions.

The hybrid method combines the strengths of each of these optimization methods using Eq. (11).

${{X}_{hybrid}}=\lambda {{X}_{PSO}}+\left( 1-\lambda  \right){{X}_{GWO}}$       (11)

where, λ represents the weighting factor for determining the contribution of the PSO and GWO methods to the overall performance. The hybridization of these two methods allows for improvement in the speed of convergence of the method and the quality of the solution produced.

3.2 Privacy preservation mechanism

In order to protect the confidentiality of the data, the framework employs a differential privacy mechanism by introducing controlled noise to the local model update prior to transmission. This prevents any malicious party from being able to reconstruct sensitive data from the model parameter or gradient. Through the use of a tunable parameter, it is possible to adjust the level of privacy protection while still maintaining the ADA of the model. The addition of noise is represented in Eq. (12).

${{\tilde{g}}_{i}}={{g}_{i}}+\mathcal{N}\left( 0,{{\sigma }^{2}} \right)$         (12)

where, $N\left( 0,{{\sigma }^{2}} \right)$ is defined as a Gaussian distribution, $\mu $ = 0 and $\sigma ~=~{{\sigma }^{2}}$. Therefore, when generating the noisy gradient$~{{\tilde{g}}_{i}}$. It is impossible for adversaries to use any piece of local data to infer sensitive data. The value of the parameter σ will determine the amount of noise added. The privacy budget is defined by Eq. (13).

$\epsilon =\frac{\text{ }\!\!\Delta\!\!\text{ }f}{\sigma }$         (13)

where, $\epsilon $ measures how much privacy has been guaranteed to the user, $\Delta f$ represents how sensitive the output function is to changes in data, $\sigma $ is the standard deviation of the added $N\left( 0,{{\sigma }^{2}} \right)$, and it will correlate with how much an input datapoint impacts performance. Communication cost is defined as Eq. (14).

${{\hat{w}}_{i}}=Q\left( {{w}_{i}} \right)$         (14)

where, ${{w}_{i}}$ represents the local model parameters at client $i$, and $Q\left( \cdot  \right)$denotes a compression function such as quantization or sparsification. The communication cost is formulated as Eq. (15). This calculation provides a total amount of data sent between clients for every communication round. Reducing this communication cost is critical for effective FL solutions in large-scale IoT systems.

${{C}_{comm}}=\underset{i=1}{\overset{K}{\mathop \sum }}\,\mid {{w}_{i}}\mid $        (15)

where, ${{C}_{comm}}$ denotes the total communication overhead, $K~$denotes the total number of clients participating in the system, and $\left| {{w}_{i}} \right|$ represents the size of model parameters being sent by each client.

3.3 Federated aggregation strategy

The global server uses a weighted average aggregation procedure when compiling the weighted private updates it receives from the participatory clients. This process provides clients with larger datasets more benefit to the global model than clients with smaller datasets. Therefore, this weighted average aggregation process is performed repeatedly until the global model converges, as outlined by Eq. (16).

${{w}^{t+1}}=\underset{i=1}{\overset{K}{\mathop \sum }}\,\frac{{{n}_{i}}}{\mathop{\sum }_{j}{{n}_{j}}}\cdot {{\hat{w}}_{i}}$         (16)

where, the updated global model at round $t+1$ is denoted by ${{w}^{t+1}}$, and the parameters of the compressed and privatized local model are denoted by$~{{\hat{w}}_{i}}$. The number of samples at client i is represented by the term ${{n}_{i}}$ and, the total number of samples across all of the participating clients is indicated by the term$~\mathop{\sum }_{j}{{n}_{j}}$.

Select the most appropriate clients to participate in the training process; as such, it provides efficient and reliable training, while minimizing the consumption of resources. The client score for client i is given by Eq. (17).

${{S}_{i}}={{\alpha }_{1}}Ac{{c}_{i}}+{{\alpha }_{2}}{{E}_{i}}-{{\alpha }_{3}}{{L}_{i}}$        (17)

where, ${{S}_{i}}$ represents the score for client i based on the suitability for training, while the ADA of the local model for that client is given by the term $Ac{{c}_{i}}$. ${{E}_{i}}$ represents the energy available for the device of client i, ${{L}_{i}}$ indicates the communication latency. The coefficients ${{\alpha }_{1}},~{{\alpha }_{2}},~{{\alpha }_{3}}$ determine the weighting of these three factors for each client. The clients with the highest scores will be selected to participate in the training process, thereby maximizing the efficiency of the network resources and the stability of the training process.

3.4 Multi-objective optimization

The proposed framework for the learning process as a multiple-object optimization problem, rather than solely focusing upon achieving increased prediction performance. This model offers optimization of ADA, privacy and communication costs simultaneously and therefore provides a practical balanced solution for IoT systems in the real world as a whole. The overall optimization objective will be represented as defined in Eq. (18).

$\mathcal{F}=\alpha {{\mathcal{L}}_{acc}}+\beta {{\mathcal{L}}_{priv}}+\gamma {{\mathcal{L}}_{comm}}$        (18)

where, $F$ represents the total optimization objectives, ${{\mathcal{L}}_{acc}}$ denotes the ADA loss, ${{\mathcal{L}}_{priv}}$ denotes the privacy loss, and ${{\mathcal{L}}_{comm}}$ denotes the communication cost. The weights $\alpha ,\beta ,\gamma ~$represent the balance of competing objectives. This formulation ensures that model optimization can take place on all three objectives at once.

The iterative training process continues until the global model has converged. The convergence will be determined by changes in the model weights between successive rounds. Thus, ensuring that the model's optimization process does not create any unnecessary computations. The condition for convergence will be expressed in Eq. (19).

$\mid {{w}^{t+1}}-{{w}^{t}}\mid <\delta $           (19)

where, ${{w}^{t}}$ and $\left( {{w}^{t}}+1 \right)$ refer to the global parameters of the model during each of the two communication rounds, while the threshold value$~\delta $ is a very small number. Once two consecutive model parameters have been updated in such a way that the distance between them is less than δ, then the model has converged. This criterion guarantees that training is stable over time and avoids having to perform unneeded computations later.

The overall process starts from the IoT devices where data is collected to create a local model that trains and optimizes itself before sending an optimized update to the central server. The optimized update has been compressed and completes the privacy process before it is sent to the central server to be aggregated with other updates. After the central server has created an updated global model based on aggregation, the updated global model is sent back to clients for them to continue the process iteratively until the model has converged. The iterative way to build the model continues to allow learning while conserving privacy and minimizing communication cost.

This paper describes a novel FMODL framework for conducting IoT data analysis that ensures both user privacy and security. The FMODL Framework incorporates FL, a hybrid metaheuristic optimization methodology, and differential privacy to overcome common obstacles encountered in privacy-sensitive environments. In terms of performance metrics, the general outputs of the FMODL Framework are evaluated against benchmarked datasets provided an ADA rating of 96.4% and 94.8% with high FARC, DE, and ICP ratings. Similarly, the MMODL framework demonstrated good levels of communication efficiency compared to traditional benchmarked methods producing high levels of performance compared to those traditional benchmarked methods.

Future research will be concentrated on expanding the FMODL Framework to support large-scale, dynamic IoT systems, accommodating adaptable methods for maintaining data privacy, and evaluating lightweight approaches to implementing analytics on resource- limited edge devices. Further exploration of deploying the FMODL Framework in real time and validating performance results across varying domains will provide an opportunity to improve the overall scalability and generalization of performance outputs when deploying practical IoT solutions.