Split Federated Learning for Efficient and Privacy-Preserving Predictive Maintenance in Resource-Constrained Industrial IoT Systems

One of the major uses related to AI in smart manufacturing, as well as IIoT, is PdM, which is a fundamental component of Industry 4.0. For minimizing unscheduled downtime and maintenance expenses, PdM focuses on predicting machine failures using multi-sensor data, including temperature, vibration, and current signals.

2.1 Split Learning

About distributed ML, SL is considered a cooperative technique enabling many clients to train a common model without sharing raw data [13]. The technique divides the neural network (NN) at a certain layer called the cut layer. Just the intermediate output, known as smashed data, is sent to the server by each one of the clients after processing the first few layers of the model locally [14]. After that, the server computes the backpropagates, loss, and continues the computation on the remaining layers. In addition, the client then updates its local model segment using the gradients of the cut layer that the server has returned to it [15]. The two sides contribute to the model's overall training using such a back-and-forth process, which protects as well as preserves sensitive data [16]. A total of three client devices (client A, client B, and client C) are used in Figure 1 to show how SL operates. Raw data from clients is never seen by the server. Just the smashed data is sent to the server by each one of the clients, which trains its part regarding the model up to a specific layer known as the cut layer. For clients to update their local models, the server completes the forward pass, computes the loss, and returns gradients. Until the model is completely trained, the exchange is carried out over all client data in several training rounds.

1.png

Figure 1. Split Learning (SL) representation with three client devices

Two formal functions could be distinguished in the SL model:

$f(x)=f_s\left(f_c(x)\right)$          (1)

where:

• $f_c(\cdot)$ denotes the layers executed on the client side,
• $f_s(\cdot)$ denotes the layers executed on the server side,
• $x$ denotes input data.

Throughout the forward pass, the client computes:

$a=f_c(x)$          (2)

Here, $a$ (the smashed data) is sent to the server, which then produces the prediction:

$\hat{y}=f_s(a)$          (3)

The gradients are backpropagated, and the loss is calculated through the server. The gradient of the cut layer, $\nabla_a L$, is sent back to the client, which updates its local parameters:

$w_c=w_c-\eta \nabla_{w_c} L(y, \hat{y})$          (4)

Meanwhile, the server updates its parameters as:

$w_s=w_s-\eta \nabla_{w_c} L(y, \hat{y})$          (5)

2.2 Federated Learning

A distributed ML technique called FL allows several clients to work together to create a common model with no sharing of the raw data. By keeping the data local as well as just sharing model updates across the network, it lowers the communication overhead and also provides a number of important benefits, including data privacy, efficiency, security, and scalability. Federated Averaging (FedAvg), which is considered a model aggregation process used in ML [17], is performed on the server, in which a global model is produced through averaging local model updates. FL trains a shared global model with three clients, client A, client B, and client C, as illustrated in Figure 2 [18]. The model parameters are aggregated by the central server, and each one of the devices updates its own model solely with its own dataset. Raw data never leaves the clients; just the model updates (gradients or weights) are transferred, permitting collaborative training, and maintaining the privacy of data. Yet, when devices have varying capabilities, as is typical in IoT systems, training the entire model on the devices with restricted resources could be computationally demanding as well as slow [19, 20]. Notwithstanding such difficulties, FL offers a workable method for developing PdM models at several locations without disclosing private information [21]. FL's formal objective is to minimize the global objective function, which is specified as follows:

$f(w)=\frac{1}{n} \sum_{j=1}^n f_j(w)$          (6)

where, each $f_j(w)$ denotes the empirical risk computed on the client j's dataset. For a given j, this local object could be indicated as:

$f_j(w)=\frac{1}{\left|D_j\right|} \sum_{(x, y) \in D_j} \ell(x, y, w)$          (7)

Here $D_j$ represents the local dataset of client j, $\ell(x, y, w)$ is the chosen loss function. The most widely adopted aggregation algorithm in FL is Federated Averaging (FedAvg), where the global model is updated as follows:

$w^{(t+1)}=\sum_{k=1}^K \frac{n_k}{n} w_k^{(t)}$          (8)

with $w_k^{(t)}$ denoting the weights regarding the local model trained through the client $k$, $n_k$ is the number of local samples at client k, $\mathrm{n}=\sum_{k=1}^K n_k$ is the total number of samples over all clients.

2.png

Figure 2. Federated Learning (FL) representation with three devices

2.3 Split Federated Learning

Lately, a significant paradigm combining the advantages of SL and FL has surfaced: SFL. SFL lessens the computational load on edge devices and maintains the privacy of data through dividing processing between servers and clients [22]. This method is especially crucial for IoT-enabled settings, in which the privacy of data is a top priority and devices have limited resources. Every client sends their smashed data to the main server, and concurrently does forward propagation on their client-side model, including their noise layer. After that, the main server uses its server-side model for processing forward as well as back-propagation independently, in (relatively) parallel, using smashed data from each one of the clients. After that, it forwards the smashed data's gradients to the appropriate clients for back-propagation. The server then uses FedAvg, or weighted average regarding gradients calculated throughout back-propagation on each one of the client's smashed data, for updating its model. Each one of the clients computes its gradients at the client's end after back-propagating on its client-side local model, following obtaining gradients of its smashed data. The gradients are sent to the fed server. Furthermore, all participating clients receive FedAvg regarding client-side local updates, which is performed by the Fed server [23]. Table 1 compares FL, SL, and SFL, whereas Figure 3 depicts the architecture of SFL. FL's global purpose could be stated formally as follows:

$f(w)=\frac{1}{n} \sum_{j=1}^n f_j(w)$          (9)

In SFL, each client’s local model is split into two parts.

$\mathrm{f}(x)=f_s\left(f_c(x)\right)$          (10)

During training, the client computes.

Finally, instead of keeping each client’s model separate, SFL combines them using federated averaging to update a shared global client-side model.

$w^{(t+1)}=\sum_{k=1}^K \frac{n_k}{n} w_k^{(t)}$          (11)

where, $n_j$ is the local dataset size of client j and $n_j=n \sum_{j=2}^k$.

3.png

Figure 3. Overview of the Split Federated Learning (SFL) system model

Table 1. An abstract comparison of Federated Learning (FL), Split Learning (SL), and Split Federated Learning (SFL)

For collecting sensor data, the presented study makes use of a variety of sensors, such as contactless temperature sensors, 3-axis accelerometers, current sensors, and ambient temperature and humidity sensors. The vibration sensor successfully recorded a defect status per one-second sample time by producing three unique features corresponding to the Y, X, and Z axes. This resulted in multiple columns inside the same dataset. Offline as well as online data collecting methods were used for recording certain fault categories, including vibration, high load, high current, stop rotation, and normal behavior. For providing the required dataset for every form of fault, such criteria have been carefully created in the standard system. An imbalanced mass has been used for replicating vibrations in all three axes of the motor. Capacitors of various sizes have been used to simulate changes in motor current, and a mechanical brake system was used for simulating heavy loads as well as stop situations. The distribution and quantity of datasets for each one of the fault categories are shown in Table 3. The four categories of faults were categorized in the presented study using a deep neural network model, with the fifth class representing normal behavior. Before applying the DNN-based SFL algorithm, a dataset must be preprocessed, such as normalizing and labeling the different classes as follows:

Table 3. Class sizes and counts

i. Normalization: Dataset normalization is a curried pre-processing step in ML. It involves transferring the feature in the dataset to common scales, typically between send 1 or -1.

ii. Preprocessing for dataset labeling includes converting categorical features into a numerical representation suitable for ML models. This transformation utilizes a one-hot encoder to generate a binary column (values of either 0 or 1) for every distinct category within the original data.

iii. Dataset splitting: The Dataset has been split into 70% training and 30% testing, where the training set is used to train the model, which is normally the largest portion of data. Testing is used to assess the fine-tuned performance of the trained model on unseen data. This provides an estimate of how well the model generalizes. Figure 4 shows the collected dataset distribution for five types of classes, namely normal, high-current, high-load, stop, and vibration. As shown in this figure, the class is not uniformly distributed.

Figure 4. Dataset distribution

iv. Class balance: To resolve class imbalance in datasets, in this work, the Synthetic Minority Over-sampling Technique (SMOTE) is used. This gives a DNN model a more balanced perspective on the issue and increases the likelihood that it will learn to categorize all classes efficiently by boosting the representation of minority classes in training data. This contributes to the confirmation that the SMOTE operation was effective in achieving more equitable class allocations. Figure 5 shows the five classes after balance.

Figure 5. The distribution of the five datasets

The presented section explains the experimental findings obtained from implementing three distinct learning frameworks: FL, SL, and the proposed SFL. The analysis compares their performance in terms of communication efficiency, prediction accuracy, and data privacy, with an emphasis on their applicability in industrial PdM environments.

In SL, model training is split between the central server and clients, allowing each one of the client devices to transmit only intermediate feature representations rather than raw data. This approach ensures a strong level of data privacy but introduces significant communication overhead, as frequent exchanges of activations and gradients occur during each training iteration. Consequently, SL demonstrates reliable accuracy but suffers from high latency; thus, it is less effective with regard to real-time or large-scale industrial applications. Figure 7 illustrates the training process related to the global model, which aggregates the contributions from three client models. In this setup, the cut layer was chosen as the last layer (output layer), where the intermediate activations from all clients are combined on the cloud server to finalize the forward pass.

a) Accuracy                                                                                 b) Loss

Figure 7. Two types of performance metrics with a split (with Split Learning (SL)). a) Accuracy b) Loss

Federated learning (FL), on the other hand, enables each one of the clients to train a complete local model as well as share only the updated parameters with the central server. Therefore, this reduces communication frequency and enhances scalability compared to SL. However, FL lacks feature-level collaboration, as the local training occurs in isolation, which can limit model generalization, especially when client data are highly heterogeneous. Figure 8 explains employing FL for training a local model on distributed data.

a) Accuracy                                                                                 b) Loss

c) Accuracy                                                                                  d) Loss

e) Accuracy                                                                 f) Loss

Figure 8. DNN model performance for Stader system (with Federated Learning (FL))

The proposed SFL framework integrates the advantages of both SL and FL. In this hybrid configuration, the DL model is partitioned so that clients handle initial feature extraction, while the main server performs deeper representation learning. The federated server then aggregates model parameters across all participating nodes, maintaining global consistency. This design significantly reduces communication overhead, improves convergence speed, and maintains strict data privacy, making SFL the most efficient and reliable approach for PdM applications in industrial environments. Table 4 shows the performance comparison between FL, SL, and the suggested SFL framework, where FL achieved a high accuracy, followed by SL, while SFL achieved slightly lower. However, SFL offers a balanced trade-off among computational load, communication efficiency, and privacy, making it more suitable for resource-constrained IIoT environments. The proposed SFL framework. This indicates that the model generalizes well to unseen data, making it adequate for PdM applications, in which reliable performance is of utmost importance, as shown in Figure 9.

Table 4. Testing performance comparison between Split Learning (SL), Federated Learning (FL), and the proposed Split Federated Learning (SFL) framework

a) Accuracy

b) Precision

c) Recall

d) Loss

Figure 9. Shows testing results of the Split Federated Learning (SFL) framework: a) Accuracy, b) Precision, c) Recall, d) Loss