A Privacy-Preserving Federated Swin Transformer V2 Framework with Explainable Artificial Intelligence for Skin Lesion Classification

Transformer pruning methods can be divided into four groups. The former type removes unnecessary items and filters based on structural similarities in the model architecture. Such techniques are not based on semantic knowledge but rather on architectural redundancy. The second category identifies the significance of image patches and several attention heads as measured by L2 norm, entropy, and saliency. In other studies, the L1 norm can be used to trim Multilayer Perceptron (MLP) layers; however, these methods fail to account for semantic relevance [17]. The third group exploits explainability methods, including those proposed to preserve features important for prediction. In contrast to the proposed method, which relies on simple statistical metrics such as distortion, the methods are based on more complex interpretability processes [18]. The fourth one considers relevance not on a per-layer basis but at the encoder block level. Though this methodology also accounts for block-level interactions, it differs in that it uses a block-wise training approach. This approach differs markedly from prior research by further compressing an already efficiency-optimized SViT architecture, whereas most prior studies leverage a highly over-parameterized ViT [19].

Hybrid CNN models with a Support Vector Machine (SVM) classifier to enhance diagnostic accuracy when classifying skin lesions. Introduced a computerized diagnosis framework of skin cancer based on dermoscopic image, with the implementation of adaptive snake and region growing methods of segmentation, which is then followed by ANN and SVM-based classification. Introduced the approach of combining AI with DL in skin cancer detection, where the Contourlet Transform is used to extract features and the state-of-the-art neural networks are used to train them [20]. Explored CNN and SVM classifiers, which involve pre-processing, segmentation, and feature extraction steps to forecast the progression of skin cancer. Proposed a detection method that can improve feature extraction and classification through a combination of Sand Cat Swarm Optimization and a ResNet50-based model [21]. The main idea of EfficientNet modeling is that a compound scaling methodically increases a baseline CNN to a desired model size while maximizing the accuracy gain. This strategy uniformly scales network width, depth, and input resolution, allowing EfficientNet to achieve high performance with far fewer parameters and lower Floating-Point Operations per Second (FLOPS). EfficientNet is not only an optimal classifier but also highly computationally efficient. Convolutional-Deconvolutional Neural Networks (CDNNs) were introduced for segmenting skin lesions in dermoscopic images, producing binary masks through pixel-wise classification of skin and lesion regions [22]. The CDNN architecture comprises 29 layers, each with hyperparameters optimized via a grid search. The up-sampling and deconvolution layers maintain and recover image resolution. A collection of CDNNs is used as the main segmentation architecture, which proves more efficient in terms of computational cost and lesion segmentation quality [23]. To produce coarse segmentation maps, two Fully Convolutional Residual Networks (FCRNs) are trained on augmented images, including the original, flipped, and rotated versions of the sample. These crude maps are then refined using distance maps generated by a Locally Invariant Convolutional Unit (LICU), which are upsampled and warped to achieve fine segmentation results. The average of the probabilities from the refined maps is used to obtain the final lesion classification results [24].

Segmentation, filtering, and lesion enhancement methods of extracting Regions of Interest (ROIs) in lesion images via pre-processing. DL based as well as handcrafted features were obtained. Deep features were learned using CNNs, and handcrafted features were learned using ABCD rules for shape, color, and texture [25]. A linear classification system that was trained on features extracted by CNN was used to classify skin lesions. A strongly trained multi-scale network of skin cancer prediction and segmentation based on dermoscopic images was proposed. Supervised DL models have also outperformed unsupervised methods in analyzing skin lesion images [26].

Used a DL framework using Restricted Boltzmann Machines (RBMs) to learn unsupervised features in the images of brain lesions. A Random Forest (RF) classifier with high Dice coefficient values on brain MRI data was then used to segment brain lesions. Similarly, they used a DL model based on Deep Belief Networks (DBNs) to predict autism spectrum disorders. For classifying histopathological breast cancer images, an RBM-based deep neural network (DNN) architecture was proposed, achieving competitive results on breast cancer image datasets [27]. Transfer Learning (TL) Applied to a CNN trained on ImageNet was used on skin lesion data, with NASNet-Large, ResNet-101, and GoogleNet fine-tuned. Proposed an intelligent diagnostic framework of multi-class skin lesion classification based on a hybrid DCNN and SVM model with Error-Correcting Output Codes (ECOC) [28]. An AlexNet architecture was used to extract the features, and the network was trained. Proposed a two-step defense system against poisoning attacks in FL environments. The approach will minimize false-positive rates in detecting poisoned models by using two decision thresholds to prevent the rejection of borderline models and then testing model safety over trends in past performance [29].

The majority of currently available studies on skin lesion classification are based on centralized DL models, which require access to large volumes of dermoscopic data. This not only causes severe issues with patient privacy but also constrains real-world clinical implementation due to confidentiality and security policies. Despite the development of privacy-preserving alternatives to FL, numerous existing FL methods rely on early versions of ViT, or on existing CNNs, which cannot handle intricate model-lesion patterns and tend to suffer under non-identically distributed clinical data [30]. Existing federated skin lesion classification studies primarily aim to improve predictive accuracy but fail to address model transparency, leading to black-box decision-making that undermines clinical trust. The combination of an advanced transformer architecture with XAI in the FSViTV2 model, in particular, has been little studied for its ability to successfully capture local and global lesion features and provide clinically useful visual rationale. There is a serious gap in research on creating a single, privacy-sensitive federated framework that integrates XAI with the SViTV2 architecture to achieve accurate, robust, and clinically reliable skin lesion classification across a variety of healthcare settings.

Dermoscopic images were obtained at various distributed medical facilities, and each client had a local copy and processing of his/her skin lesion images to maintain patient confidentiality. Pre-processing of images was performed through resizing, color normalization, and data augmentation to minimize illumination variation and class imbalance. The SViTV2 model was initialized on the central server, and training was performed using the FL framework without exchanging raw data, as shown in Figure 3. Encrypted model updates were shared and averaged only through federated averaging. The hierarchical self-attention mechanism captured both global and fine-grained lesion features. Training and evaluation with non-identically distributed data were conducted across several rounds of federated performance, using standard performance metrics.

Figure 3. Proposed architecture

Figure 3 illustrated framework presents a federated learning (FL)-based skin lesion classification system built on SViTV2, where multiple distributed clients (private hospitals/clinics) collaboratively train a global model without sharing raw patient data. Each client $k$ locally optimizes its SViTV2 model using its private dataset $D_k$ by minimizing a local empirical risk function

$\mathcal{L}_k(\theta)=\frac{1}{\left|D_k\right|} \sum_{\left(x_i, y_i\right) \in D_k} \ell\left(f\left(x_i ; \theta\right), y_i\right)$   (4)

where, $f(;, \theta)$ denotes the Swin Transformer V2 with patch partitioning, hierarchical self-attention, and patch merging, and ℓ(⋅) is the cross-entropy loss.

After local training, only the encrypted model parameters $\theta_k^{(t)}$ are transmitted to the central server. The server performs secure aggregation (e.g., FedAvg) to update the global model: $\theta^{(t+1)}=\sum_{k=1}^K \frac{\left|D_k\right|}{\sum_{j=1}^K\left|D_j\right|} \theta_k^{(t)}$ ensuring privacy preservation through encryption and differential privacy mechanisms. The updated global Swin Transformer V2 is redistributed to all clients for the next training round. For clinical transparency, an XAI module (GradCAM++, Score-CAM, and SHAP) generates class-discriminative heatmaps, where Grad-CAM++ computes importance weights $\alpha_k^c=\sum_{i j} \frac{\partial^2 y^c}{\left(\partial A_{i j}^k\right)^2}$ highlighting lesion regions that contribute most to predictions such as Melanoma or Basal Cell Carcinoma. Overall, this architecture combines privacy-preserving federated optimization, hierarchical vision transformers, and explainable decision support, enabling accurate, trustworthy, and clinically interpretable skin lesion diagnosis.

4.1 Dataset description

To ensure medical relevance, heterogeneity, and realism in FL, both confidential clinical data and accessible dermoscopic databases, such as ISIC 2018, HAM10000, and PH2, as presented in Table 1, were used. These datasets are lesion-category datasets, including melanoma, nevus, carcinoma, and benign conditions, with varying class distributions. Federated clients were non-identically distributed to simulate a multi-institutional environment, and data were distributed across them in a heterogeneous fashion that would reflect the real world. All images were downsized to 224 × 224 pixels to address class imbalance. During training, encrypted model updates were shared between clients and the server, ensuring privacy. A safe and clear model analysis, and raw dermoscopic images were stored at each client.

Table 1. Dataset description

Federated clients were not allocated identical dermoscopic images by ISIC 2018, HAM10000, PH2, and private datasets to incorporate a realistic multi-institutional healthcare setting. Each client had locally held 224 × 224 lesion images from different classes, with only encrypted model updates to ensure privacy-preserving, cooperative skin lesion classification, as shown in Table 2. The dataset comprises 318 dermoscopic images representing various lesion classes, and each image is 640 x 480 pixels. The values of class are 21 Angioma, 46 Nevus, 41 Lentigo NOS, 68 Solar Lentigo, 51 Melanoma, 54 Seborrheic Keratosis, and 37 Basal Cell Carcinoma (BCC) images. Figure 4 presents representative samples from the dataset.

Figure 4. Sample skin lesion images

Table 2. Sample data

4.2 Pre-processing

The image pre-processing pipeline, as shown in Figure 5, consists of sequential operations designed to enhance dermoscopic image quality and improve model robustness. Let the raw dermoscopic image be denoted as $X \in R^{H \times W \times 3}$ First, resizing is applied to standardize the input dimensions using an interpolation function $R($.$):$

$X_r=R(X), \quad X_r \in R^{224 \times 224 \times 3}$   (5)

Figure 5. Steps of image pre-processing

Next, contrast adjustment and noise reduction are performed to enhance lesion visibility. Contrast enhancement can be expressed as:

$X_c=\alpha X_r+\beta$   (6)

where, $\alpha$ controls contrast and $\beta$ adjusts brightness.

Noise reduction is applied using a smoothing filter $F(.)$. , such as a Gaussian filter:

$X_e=F\left(X_c\right)$   (7)

Subsequently, data augmentation is employed to improve generalization by applying geometric transformations, including rotation, flipping, and zooming, represented as:

$\tilde{X}=A\left(X_e\right)$   (8)

where, A(.) represents a stochastic augmentation operator. Lastly, the preprocessed image $\tilde{X}$ is offered as the model input with uniform, improved, and varied representations to ensure a diverse representation to facilitate effective learning in the FSViTV2 model.

Various augmentation methods were used, including scaling, contrast and brightness, random rotations, and random vertical and horizontal flips, as presented in Figure 6. These augmentations allowed learning of the model using invariant representations, hence encouraging the strong recognition of monkey pox lesions and other skin diseases under different orientations, light conditions, and viewpoints.

Figure 6. Data augmentation process illustrating (a) the original image, (b) vertical flip, (c) horizontal flip, (d) rotation by 90°, and (e) rotation by −90°

4.3 Privacy-aware federated collaborative learning

The FSViTV2 architecture, which uses privacy-preserving techniques, enables skin lesion classification in a collaborative setting without sharing raw medical images. Every client in the local training of the SViTV2 model trains its own model with its own data and creates local model updates. These updates are encrypted and securely sent to a federated aggregator, which performs privacy-conscious aggregation using secure-computation methods. The shared SViTV2 model is then updated by the global server, while maintaining data confidentiality to facilitate shared learning, global generalization, and precise, explainable predictions.

Figure 7 depicts a multi-stage learning pipeline that combines FL, reinforcement learning, and hyperparameter optimization for skin lesions. Under Stage 1, labeled and unlabelled dermoscopic images are obtained with the labeled set $D_L$ containing classes of known lesions, and the unlabelled pool $D_U=\left\{i_y\right\}$. In Stage 2, a CNN-based feature extractor and fully connected layers are trained to learn discriminative representations and learn the local model parameters $\theta_k$ at each client with FL. The world model is revised through weighted averaging:

$\theta^{(t+1)}=\sum_{k=1}^K \frac{n_k}{N} \theta_k^{(t)}$   (9)

Figure 7. Privacy-aware federated collaborative learning

where, $n_k$ is the number of samples at client k, and N is the total sample count.

In Stage 3, a Q-network trained with focal loss and Proximal Policy Optimization (PPO) determines whether a sample should be annotated or rejected. The decision policy $\pi\left(a_x \mid s_x\right)$ maximizes the expected reward:

$R=E\left[\sum \gamma^t r_t\left(s_t, a_t\right)\right]$   (10)

where, $s_t$ represents the state of the model, $a_t$ represents the action (annotation or nonannotation), and $\gamma$ represents the discount factor. Lastly, Stage 4 uses the Artificial Bee Colony (ABC) optimization to tune hyperparameters by minimizing the classification loss L , producing a strong, optimized model. This unified pipeline enhances annotation efficiency, classification accuracy, and overall generalization without compromising data privacy.

Figure 8 illustrates the FL framework, which enables collaborative skin cancer image classification across multiple data owners while preserving data privacy. Each owner $k \in \{A, B, C\}$ retains its local skin lesion dataset $D_k$ and trains a local model by minimizing a classification loss, typically cross-entropy,

$L_k\left(\theta_k\right)=-\frac{1}{\left|D_k\right|} \sum_{\left(i_x, j_x\right) \in D_k} j_x \log f\left(i_x ; \theta_k\right)$   (11)

where, $\theta_k$ are the local model parameters. After local training, only the updated parameters $\theta_k^{(t)}$ are transmitted to the aggregation server, while raw images remain private. The aggregation server computes a global model using weighted federated averaging,

$\theta^{(t+1)}=\sum_{k=1}^K \frac{\left|D_k\right|}{\sum_{y=1}^K\left|D_y\right|} \theta_k^{(t)}$   (12)

Given that, clients with more data should play a proportionately larger role. The new international model is then reissued to all owners for the subsequent training cycle. This cycle helps the system be informed by strong, generative representations of skin lesions across heterogeneous data and ensures high data confidentiality and adherence to privacy limitations.

Figure 8. FL for skin image classification

4.4 Hierarchical Swin Transformer V2 for distributed skin lesion representation

It offers an efficient mechanism for capturing multi-scale lesion characteristics whilst retaining computational efficiency on large-scale dermatological data, as illustrated in Figure 9. Its hierarchical structure divides the dermoscopic images into non-overlapping windows and gradually refines them across phases, allowing the model to acquire both fine-grained local details and global context. At every step, shifted window self-attention is used, and it does not introduce quadratic computational complexity; it is highly applicable to high-resolution medical images. The successive-layered distribution of feature extraction enables the model to produce high-quality hierarchical embeddings that can represent lesion shapes, color differences, and textures in a meaningful way. In distributed and federated medical environments, this architecture is useful for lesion classification, segmentation, similarity retrieval, and more, providing a scalable, dependable, and comprehensible system for automated analysis of skin lesions. The proposed framework is based on the FL paradigm, according to which N local clients jointly train an SViTV2 model without sharing raw data. The client x has personal data.

$D_x=\left\{\left(i_y^x, j_y^x\right)\right\}_{y=1}^{n_x}$   (13)

Local training minimizes the empirical risk

$L_x(\theta)=\frac{1}{n_x} \sum_{y=1}^{n_x} l\left(f\left(i_y^x ; \theta\right), j_y^x\right)$   (14)

where, θ identifies model parameters and $f$(.) identifies the SViTV2. This decentralized optimization enables learning from heterogeneous data distributions while maintaining data locality. At the local client level, images are processed and forwarded through hierarchical window-based self-attention in SViTV2 in the form of

Attention $(Q, K, V)=\operatorname{Softmax}\left(\frac{q K^T}{\sqrt{d}}\right) V$   (15)

Figure 9. Hierarchical Swin Transformer V2 for distributed skin lesion representation

Explainability is achieved using LIME or SHAP, where feature attribution for input i is computed as

$\emptyset_k=\sum_{S \subseteq F \backslash\{k\}} \frac{|S|!(|F|-|S|-1)!}{|F|!}[f(S \cup\{k\})-f(S)]$   (16)

To preserve privacy, differential privacy adds noise to gradients:

$\tilde{g}_x=g_x+N\left(0, \sigma^2\right)$   (17)

securing resistance against data reconstruction attacks. The secure server combines encrypted client updates through privacy-preserving methods. Each client sends encrypted parameters $E\left(\tilde{\theta}_x\right)$, and secure aggregation computes

$\theta^{(t+1)}=\sum_{x=1}^N \frac{n_x}{\sum_y n_y} \tilde{\theta}_x$   (18)

Without the decryption of individual updates. Homomorphic encryption and Secure Multi-Party Computation (SMPC) ensure that intermediate values are not revealed and therefore remain confidential during aggregation. The global SViTV2 model is updated on the global server using aggregated parameters. Integrated Gradients are used to give global explainability, which is defined as

$I G_k(i)=\left(i_k-i_k^{\prime}\right) \int_0^1 \frac{\partial f\left(i^{\prime}+\alpha\left(i-i^{\prime}\right)\right)}{\partial i_k} d \alpha$   (19)

marking out clinically-relevant areas. The output provides interpretable and accurate predictions of melanoma, nevus, and carcinoma, with a balance between diagnostic accuracy and great privacy and security.

4.5 Explainable Artificial Intelligence (XAI) integration

FSViTV2 architecture incorporates XAI to make skin lesion classification more transparent and understandable. Despite the high accuracy of deep learning models like SViTV2, the lack of transparency in their decision-making makes them hard to implement clinically. To prevent this, the proposed framework will use XAI methods, such as attention-based Grad-Cam visualizations, to highlight the lesion regions that affect classification results. These visual explanations can help dermatologists prove model predictions, spot potential bias, and instill confidence in automated diagnostic systems. Figure 10 demonstrates that the XAI module assigns attention weights to lesion regions after federated model inference, revealing that features such as irregular borders, color differences, and texture patterns are critical. The integration is effective for decision-making, error analysis, model refinement, and accountability, especially in heterogeneous data and multi-institutional settings. XAI guarantees that the FSViTV2 framework is interpretable, clinically relevant, truthful, and does not invade privacy.

0928d1476e0e4498d943c6758435f6e3.png

Figure 10. Workflow of Explainable Artificial Intelligence (XAI)

In localization maps specific to the classes, it uses the gradients that traverse the attention layers. For a target class e, the gradient of the class score $j^c$ with respect to the feature map $A^k$ of the last transformer block or attention head is computed:

$\frac{\partial j^c}{\partial A^k}$

The importance weight $\alpha_k^c$ for each feature map channel k is obtained by global average pooling these gradients over the spatial dimensions:

$\alpha_k^c=\frac{1}{Z} \sum_x \sum_y \frac{\partial j^c}{\partial A_{x y}^k}$   (20)

where, $Z$ is the total number of spatial positions $(x, y)$ in the feature map. The Grad-CAM heatmap $L_{\text {Grad }-C A M}^c$ is then computed as the weighted combination of the feature maps,

$L_{G r a d-C A M}^c=\operatorname{ReLU}\left(\sum_k \alpha_k^c A^k\right)$   (21)

This heat map is upsampled to the resolution of the original image and superimposed, showing the important areas- irregular borders, pigmentation patterns, etc., that most contributed to the classification decision. With the FSViTV2 framework, the model offers interpretable, clinically relevant information for every distributed client without disclosing sensitive patient information.

4.6 Algorithm: Privacy-preserving FSViTV2 with Explainable Artificial Intelligence for skin lesion classification

Inputs: Distributed dermoscopic image datasets $D_1, D_2, \ldots, D_N$ at N clients (hospitals/clinics); Swin Transformer V2 architecture $f_\theta$ with parameters $\theta$; Number of federated rounds T; Learning rate $\eta$

Outputs: Federated model $f_\theta^*$ for skin lesion classification; Explainable heatmaps $L_{\text {Grad-CAM}}^C$ for each prediction

Step 1: Initialization

1. Initialize global model parameters $\theta_0$ on the server.

2. Set round counter $t=0$.

Step 2: Local Training at Client $\boldsymbol{x}$

For each client $x \in\{1,2, \ldots, N\}$

1. Receive global model $\theta_t$ from the server.

2. Train locally using dataset $D_x$ for E epochs:

$\theta_x^{t+1}=\theta_{+}-\eta \nabla_\theta L\left(f_\theta\left(D_x\right), i_x\right)$   (22)

where, $L$ is the cross-entropy loss and $j_x$ are the labels.

3. Apply privacy-preserving mechanisms (e.g., differential privacy noise $N\left(0, \sigma^2\right)$ ) to the gradients or weights:

$\tilde{\theta}_x^{t+1}=\theta_x^{t+1}+N\left(0, \sigma^2\right)$   (23)

4. Send $\tilde{\theta}_x^{t+1}$ to the central server.

Step 3: Global Aggregation at Server

1. Aggregate the client updates using Federated Averaging (FedAvg):

$\theta_{t+1}=\sum_{x=1}^N \frac{\left|D_x\right|}{\sum_y\left|D_y\right|} \tilde{\theta}_x^{t+1}$   (24)

2. Update global model $\theta_{t+1}$ and send back to all clients.

3. Increment $\mathrm{t}=\mathrm{t}+1$ and repeat Steps 2-3 until convergence $(\mathrm{t}=\mathrm{T})$.

Step 4: Inference and Explainable Al Generation

For a test image $i$:

1. Predict class $c^*=\arg \max f_\theta^*(i)$ with global model.

2. Compute Grad-CAM heatmap:

Compute gradient of target class score w.r.t. feature map

$A^k: \frac{\partial j^c}{\partial A^k}$

Compute channel importance weights:

 $\alpha_k^{c^*}=\frac{1}{Z} \sum_x \sum_y \frac{\partial j^c}{\partial A_{x y}^k}$   (25)

Generate heatmap:

$L_{G r a d-C A M}^{c^*}=\operatorname{ReLU}\left(\sum_k \alpha_k^{c^*} A^k\right)$   (26)

3. Overlay $L_{\text {Grad-CAM }}^{c^*}$ on input image $x$ for clinically interpretable visualization.

Step 5: Output: Federated model $f_\theta^*$ that is privacy-preserving.

Grad-CAM/XAI heatmaps for each skin lesion classification.

The hierarchical multi-scale attention of SViTV2 allows better localization of lesion features for Grad-CAM.

Differential privacy ensures patient data never leaves the local client, maintaining HIPAA compliance.

FedAvg balances contributions from each client while preserving the efficiency of distributed learning.