Enhancing Colon Cancer Diagnosis Through Explainable Vision Transformer Architecture with Selective Kernel Learning and Multi-Objective Differential Evolution

Colon cancer is observed as a prevalent cancer around the globe causing a major threat to the health globally [1]. Traditional method of such cancer diagnosis involves an invasive approach considering the histopathological analysis of biopsies. This approach is expensive and leads to inconsistencies. Methods such as colonoscopic screening used for predicting colorectal cancer leads to problems such as bleeding, colonic perforation and inability to detect all polyps [2], Early and proper diagnosis for such disease is important to improve patient outcomes and increase the survival rate. Survey reveals that the early stage diagnosis of colon cancer increases the survival rate of the patient by 90% [3]. However, for patients with cancer spread across different organs, the survival rate drops accordingly.

Deep learning models offer a better diagnosis for colon cancer using histopathological images by differentiating the normal and cancerous tissues. These models focus only on high accuracy for diagnosis, interpretability is also important to help the healthcare professionals to understand the predictions of the model to improve decision making. Recently deep learning architectures such as convolutional neural networks (CNNs) have been utilized in medical image analysis, particularly for diagnosis of critical diseases like colon cancer [4]. Nowadays, vision transformer (ViT) based deep learning architecture termed as a better alternative than CNN is used much in the Computer Vision domain [5]. The applications of deep learning methods in healthcare due to their “black box” nature lead to limitations such as lack of interpretability, limiting clinical adoption, and hindering trust in diagnosis [6]. This paper proposed a novel approach to enhance the diagnosis of colon cancer utilizing EViT architecture through features such as SKL and MODE. EViT is a modified version of the standard Vision Transformer architecture to capture long range dependencies in image data sing self-attention mechanism. SKL method utilizes learnable weights termed as gates in each kernel of the convolution layer in EViT architecture. This helps the model to suppress the channels that are less informative and improve the interpretability and efficiency. MODE is an evolutionary optimization method utilized to optimize the EViT architecture for objectives such as classification accuracy and interpretability. Also, developing deep learning models with interpretability leads to several benefits such as a better understanding of patterns and image features for improving classification accuracy. This process helps healthcare professionals generate valuable insights, facilitating improved diagnostics and better communication between patients and healthcare professionals.

Balancing factors such as accuracy and interpretability become a major challenge in medical diagnosis, especially when using deep learning models. Performing optimization for accuracy alone can lead to black box models with a lack of transparency in predictions. Alternately, focusing on the optimization of interpretability will become a challenge for the model to handle complex patterns leading to compromise in the predictions. The proposed approach can overcome these problems by utilizing EViT architecture and MODE by formulating interpretability as an objective function. By addressing these issues, the proposed EViT based model can be used as an efficient tool for accurate diagnosis of colon cancer.

The major work of the proposed system includes the design of EViT architecture with SKL approach to improve the interpretability of colon cancer diagnosis. The optimization of EViT architecture is performed using MODE for classification accuracy and interpretability to address the multi-objective nature while performing a diagnosis of colon cancer. The proposed approach has been evaluated using colon cancer dataset with histopathological images. The results were promising with high accuracy compared to other state-of-art methods. By combining the ViT architecture with the technique to enhance interpretability and Multi objective optimization, the proposed work contributes for accurate diagnosis of colon cancer.

3.1 Image preprocessing

Image preprocessing techniques are used to remove noise from images for better accuracy while performing classification tasks. Traditional filtering methods such as blurring and sharpening are not sufficient to perform classification for histopathological images. Considering this fact advanced filtering method termed Non-local Means filtering with patch similarity measure has been used to remove noise in histopathology image samples of colon cancer.

Input: Noisy Image (N), Patch Size(s)

Output: Denoised Image (M)

Algorithm 1:

1. Patch Extraction:

• Image N divided into overlapping patches with size (s×s)
• Each patch represented as S_j,k, j,k specifies pixel location

2. For each patch S_j,k

• Search the image for similar patch (P) considering average intensity within a patch

3. Weight Calculation:

• Assign a weight w to the similar patch (P) based on average intensity difference between S_j,k and P. Higher similarity means higher weight

4. Weighted Averaging:

• Calculate denoised pixel value $v_{j, k}$ by averaging the intensity of similar patch P, with weight w using formula specified in Eq. (1)

$v_{i, k}=(\mathrm{w} * \mathrm{P})+\left[(1-\mathrm{w}) * I_{i, k}\right]$          (1)

where, I_j,k represents the intensity of noisy pixel at (j,k)

5. Image Reconstruction:

• Repeat steps 2-4 for all pixels in the image

Combine denoised pixels $v_{j, k}$ to form denoised image (M)

The algorithm for image preprocessing using Non-local Means filtering is specified in Algorithm 1. This method performs a search over the entire image to identify similarity patches using parameters such as intensity, local features and texture and utilizes a weighted averaging approach to reconstruct the denoised image. This approach removes noise by preserving vital information such as tissue boundaries and improves the efficiency of colon cancer classification.

Histogram equalization is also performed to enhance the contrast to increase the visibility of the tissue features for the deep learning model during colon cancer classification. This process is performed by stretching the distribution of pixel intensities across the entire range of grayscale images. This process also addresses stain variations by normalizing the intensity distributions.

3.2 Colon cancer classification using EViT architecture

The proposed Enhanced Vision Transformer (EViT) architecture specified in Figure 1 addresses the challenge of traditional deep learning model being black box in nature lacking transparency. This helps in healthcare applications, particularly in cancer diagnosis to enhance interpretability. EViT architecture is the modified version of vision transformer utilized for the diagnosis of colon cancer through its capacity for hierarchical feature representation. This architecture uses self-attention mechanism to model long range dependencies across histopathological images. This enhancement reduces the computational complexity by utilizing sparse attention mechanism allowing effective processing of high dimensional data. This feature is introduced in token interactions in order to improve speed and memory efficiency during attention computation. Also, the hierarchical feature extraction process across various scales enables efficient localization of cancerours regions.

EViT architecture uses attention maps to address black box problem by providing explanations for their classification. These maps highlight the image regions that the model focuses allowing the healthcare professionals to understand the models decision in identifying whether cancerous or non-cancerous. Also, this architecture provides early detection insights for researchers by analyzing the image features that the EViT architecture prioritizes. This process helps to identify the visual patterns associated with colon cancer leading to better diagnosis.

The shortcomings of the model can also identify which helps to identify the areas of improvement or to address the biases in training data. Also, histopathological images are complex in nature to signify cancerous and non-cancerous samples which can be overcome by EViT architecture focusing on specific regions required in such scenarios. The tasks such as processing histopathological images and performing feature extraction are carried out by the Vision Transformer architecture. Metaformer block is used in the architecture to enhance the performance of transformer encoder. The method such as SKL has been utilized to enhance the feature extraction process concentrating on the informative region of the image.

The initial component specified in the EViT architecture is used to preprocess the input images using denoising method mentioned in Figure 1. Then normalization is applied to scale the images to a standard range to ensure the quality of image input and to improve convergence during training process. The denoised image of size h×w×c (height×width×channel), where channel is 3 for RGB, is considered for patch extraction and embeddings, discussed in detail in the sub-section (3.3).

1.png

Figure 1. EViT architecture for colon cancer diagnosis

During this process patch size is defined for example, P×P, then a grid of non-overlapping images will be performed. Each patch is flattened using a vector of size P²×c. During the process of embedding flattened patches into a format through linear projection that the transformer model can process. The CLS token is then added to the patch embeddings and then positional encoding is added to provide positional information. SKL is applied to the sequence of embeddings to select relevant kernels. This process selects the most relevant features dynamically based on the input data. This method is

The components of transformer encoder with metaformer block are specified in Figure 2. The patch embedding component divides the histopathological images into patches and converts them into lower dimensional vectors suitable for processing by transformer encoder. This process is performed through an embedding function preferably linear projection that divides image into patches. SKL uses pre-trained kernels to emphasize the image features related to colon cancer, particularly textures and shapes.

2.png

Figure 2. Transformer encoder architecture with metaformer block

Multi head self attention mechanism is used to capture the dependencies between tokens in the enhanced feature map. Also, it analyzes the image patches enhanced by the meta former block. During this process the patch embeddings transformed into query (Q), key (K) and value (V) matrices and then attention score is calculated as specified in Eqs. (2) and (3).

$Q=X w_q, K=X w_k, V=X w_v$          (2)

$\operatorname{Attention}(Q, K, V)=\operatorname{softmax}\left(\frac{Q k^T}{\sqrt{d_k}}\right) V$          (3)

where, w_q, w_k, w_v forms learnable weight matrices for query, key and value, d_k represents the dimensionality of key vectors. Concatenation of attention heads is performed and linear transformation is applied as in Eq. (4).

$M H A(Q, K . V)=\operatorname{concat}\left(h_1, h_2 \ldots h_h\right) w_y$          (4)

where, h_i=Attention(Q_i,K_i,V_i) and w_y forms learnable output weight matrix. Channel gating mechanism is used to activate channels in the feature map. This mechanism is applied as in Eq. (5) to enhance the feature maps.

$z_{g t}=\sigma\left(w_{g t} \mathrm{Z}+b_{g t}\right)$          (5)

where, σ forms sigmoid function, w_gt and b_gt are learnable parameters and z_gt forms gating weights.

Feed forward network is used to apply non-linear transformations to the features as in Eq. (6). Residual connections and layer normalization are applied to stabilize training and improve gradient flow as in Eq. (7).

$F F N(x)={ReLU}\left(X w_1+b_1\right) w_2+b_2$          (6)

where, w₁ and w₂ forms weight vectors and b₁ and b₂ form bias vectors of the feed forward network.

$Y=\left(\frac{X-{mean}(X)}{{stdv}(X)+\epsilon}\right) * \gamma+\beta$          (7)

where, mean(X) forms mean value of the input, std(X) forms the standard deviation of the input X, ϵ forms a small constant to avoid divide by zero error, γ, β forms learnable scaling and shifting parameters. The special token termed as class (CLS) token representing the entire sequence is used for classification task. Explainable classification head maps the class (CLS) token to output classes and then provides interpretability through linear transformations applied to the CLS token. Softmax function is used to produce probability distribution over classes using the technique Grad-CAM (Gradient weighted Class Activation Mapping). This method is a visualization technique used by the proposed EViT based model to make classification decisions. During training, the EViT architecture learns to extract features from images to differentiate from cancerous and healthy tissues.

Grad-CAM is then applied to a specific image to understand the models decision by focusing on the final layer of the model containing softmax function. During forward pass the image is divided into patches and positional encodings are added. Then SKL is applied to select appropriate features. The transformed embeddings passed through the transformer encoder with metaformer block using multi-head self attention, channel gating and feed forward network. The feature maps from SKL are selected to compute Grad-CAM through predicted class score with respect to these feature maps using Eq. (8).

$\frac{d y^c}{d m^f}$          (8)

where, y^c forms the class score for colon cancer and m^f specifies the feature map f for the selected convolution layer.

Then average the gradients to get importance of weights for each feature map f as specified in Eq. (9).

$D_f^c=\frac{1}{z_i} \sum_i \frac{d y^c}{d m_{i, j}^f}$          (9)

where, $m_f^c$ specifies importance weight for feature map f and class c. z represents number of pixels in the feature map and $m_{i, j}^f$ represents element at position (i,j) in feature map m^f. Compute the weighted sum of the feature weight using importance weight as in Eq. (10), where G^c provides heatmap of class c, ReLU represents rectified linear unit activation.

$G^c={ReLU}\left(\sum_k D_f^c m^f\right)$          (10)

Upsample the heatmap to the original image size as in Eq. (11). This output reveals the important region in the image highlighting the abnormal tissues which helps in generating meaningful visual representation helping in colon cancer diagnosis.

$G U^C={upsample}\left(G^c\right)$           (11)

3.3 Optimal kernel selection using SKL

SKL introduces the adaptive kernel selection mechanism into the model. This feature ensures that the model captures features across different spatial resolutions which are important for detecting patterns in colon cancer images. This method utilizes parallel convolutional pathways with varying sizes of kernel and integrates their outputs through attention mechanism in order to prioritize the most relevant scale. This mechanism focuses on features of interest considering the biological vision systems. Also, SKL enhances the ability of the model to detect fine-grained details and global patterns through feature aggregation process. SKL is used to dynamically combine different convolutional kernels to enhance feature representation. A kernel refers to a convolutional filter applied to a feature map to extract relevant features improving the learning process. Kernels are trained to detect features such as edges and textures. Multiple kernels are used to capture diverse features. The algorithm for SKL is specified in Algorithm 2.

Input: Feature Map (X) (h×w×c), Convolutional kernels {k₁,k₂…,k_n)

Output: Feature Map (M)

Algorithm 2:

1. Apply Convolutional Kernels

• Apply convolutional kernels k_i to feature map X to produce set of features X_i as mentioned in Eq. (12).

$X_i={Conv}\left(X, k_i\right) \forall i \in\{1,2, \ldots, \mathrm{n}\}$          (12)

2. Global Information Aggregation

• Apply Global Average Pooling to each feature map $X_i$ to produce global descriptor $d_i$ as in Eq. (13).

$\begin{gathered}d_i=G A P\left(X_i\right)=\frac{1}{h x w} \sum_{h e=1}^h \sum_{w e=1}^w X_i(h, w, c) \\ \forall c \in\{1,2, \ldots, \mathrm{n}\}\end{gathered}$          (13)

3. Concatenate Global Descriptors

• Concatenate the global descriptors from all feature map to form a single vector as in Eq. (14).

$d=\operatorname{concat}\left(d_1, d_2, \ldots d_k\right)$            (14)

4. Compute Importance Weights

• Pass the concatenated weight d through fully connected layers to derive importance weights w as in Eq. (15).

$\mathrm{w}=F C_2\left(\operatorname{ReLU}\left(F C_1(d)\right)\right)$           (15)

5. Softmax Normalization

• Normalize the importance weights using softmax function to ensure they sum to 1 as in Eq. (16).

$w_{\text {soft }, i}=\frac{\exp \left(w_i\right)}{\sum_{j=1}^k \exp \left(w_j\right)}$          (16)

6. Re-weight Feature Maps

• Multiply feature map by its normalized importance weights as in Eq. (17).

$M_i=w_{s o f t, i} * X_i$           (17)

7. Combine Re-weighted Feature Maps

• Add the re-weighted feature maps to produce final output feature map as in Eq. (18). The dimension of output feature map will be h×w×c.

$M=\sum_{i=1}^k M_i$          (18)

3.4 Hyperparameter optimization using MODE

MODE method is utilized for the optimization of hyperparameters balancing multiple objectives like computational efficiency, accuracy and generalization. MODE method explores pareto-optimal solution space to ensure that the model achieves balance between competing performance metrics. This method uses differential evolution method by incorporating multi objective differential evolution in order to maintain diversity in solution space ensuring better optimization. MODE is used to optimize the hyperparameter and feature selection to balance multiple objectives like accuracy and interpretability. This method uses the approach of evolutionary algorithm to improve the candidate solutions. Also, this method maximizes the accuracy and minimizes the complexity of the model by balancing the multiple objectives and identifying the best tradeoffs. The hyperparameters of model building such as learning rate, patch size, number of layers can be optimized using this method to enhance the efficiency of EViT model. The histopathological images of colon cancer dataset used to train the EViT architecture with different hyperparameter configurations.

The hyperparameters to be optimized includes: patch size, transformer encoder layers, self-attention heads, learning rate and drop-out rate. This method is well suited for optimizing problems with multiple objective functions. This process involves generating and iterating candidate solutions by updating the population based on pareto dominance. The algorithm for MODE to optimize hyperparameters is specified in Algorithm 3.

Input:

Objective Functions: O₁(x), O₂(x),…O_m(x)

Population Size: N

Scaling Factor: S

Crossover Probability: CP

Stopping Criteria: Maximum number of generations GE_max

Output: Pareto front,optimal parameter set for pareto front solutions

Algorithm 3:

Initialization

• Generate an initial population of size N with random parameter vectors as specified in Eq. (19).

$P=\left(y_1, y_2, \ldots, y_N\right)$          (19)

Evaluation

• Evaluate each individual in the population for all objective functions as in Eq. (20).

$f\left(y_i\right)=f_1\left(y_i\right), f_2\left(y_i\right), \ldots f_m\left(y_i\right)$          (20)

While stopping criteria not met

1.  Mutation

• For each individual $y_i$ generate mutant vector m_i as in Eq. (21).

$m_i=y_{r 1}+F \cdot\left(y_{r 2}-y_{r 3}\right)$           (21)

where r_1,r₂, r₃ form distinct random indices.

2. Crossover

• Create a trial vector v_i by combining m_i and y_i as in Eq. (22).

$v_i=\left\{\begin{array}{l}m_{i j}, { if } \,\, { rand }_j \leq C p \\ y_{i i}, \cdots \cdots \cdot { Otherwise }\end{array}\right.$          (22)

3. Selection

• Evaluate the trial vector $m_i$ as in Eq. (23).

$f\left(m_i\right)=f_1\left(m_i\right), f_2\left(m_i\right), \ldots f_m\left(m_i\right)$          (23)

• Update population based on pareto dominance as in Eq. (24).

$\left(v_i\right)^{c+1}=\left\{\begin{array}{l}m_i, { if } \,\, m_i \cdot { dominates } \cdot y_i \\ y_i, \cdots \cdots \cdots \cdots \cdot \cdots \cdot { Otherwise }\end{array}\right.$         (24)

4. Update Pareto Front

• Maintain a set of non dominated solutions representing current pareto front.

5. Increment generation counter as in Eq. (25).

$c=c+1$          (25)

Return

• The pareto front, set of non-dominated solutions

Optimal parameter set y for the solutions in the pareto front.

The proposed system has been evaluated using metrics like accuracy, precision, recall and F1-score. The accuracy measure is used to find the ratio of correctly classified samples indicating the overall correctness of the model as specified as in Eq. (26).

$Accuracy$ $=\frac{T R P+T R N}{T R P+T R N+F A P+F A N}$         (26)

where, TRP, TRN represents true positives and true negatives, FAP and FAN represents false positives and false negatives. The metric precision representing the ratio of true positive prediction over all positive predictions measures how many of the predicted positive cases are correct as in Eq. (27).

$Precision$ $=\frac{T R P}{T R P+F A P}$          (27)

The evaluation metric recall also termed as sensitivity provides the ratio of true positive predictions to all actual positive cases reflecting the model's ability to identify positive cases as in Eq. (28).

$Recall$ $=\frac{T R P}{T R P+F A N}$           (28)

The metric F1-Score represents mean of precision and recall used to evaluate the performance when the dataset is imbalanced as in Eq. (29).

$F 1-$ $Score$ $=2 * \frac{ { Precision } * { Recall }}{{ Precision }+ { Recall }}$          (29)

In this paper, the advanced model for prediction of colon cancer has been proposed by combining EViT architecture with SKL and MODE. The proposed methods show accuracy of accuracy of 93.2% outperforming the state-of-art methods. The utilization of SKL helps the model select optimal kernels for feature extraction, enhancing the capability of the model to identify important features. The MODE algorithm optimizes the hyperparameters specified in Table 2 ensuring the performance of the model by balancing between accuracy and interpretability. Also, the use of GRAD-CAM and Attention map generated by the proposed model specified in Figure 9 provides valuable insights making the model transparent and interpretable for clinical applications. Further, the utilization of advanced image pre-processing techniques like non-local mean filtering with patch similarity measure enhanced the image quality and feature extraction contributing to the better performance of the model. The model’s explainability feature helps medical practitioners to identify the critical regions that contribute much to the occurrence of colon cancer. This process facilitates better decision-making in clinical environment. Also, the proposed work has an impact on patient outcomes through early and accurate diagnosis of colon cancer. The significance of the proposed work extends beyond the technical advancements on clinical practices. The proposed model provides a reliable tool to healthcare providers for making informed diagnoses to improve patient outcomes. The innovation of the proposed work lies in the explainability features through GRAD-CAM and attention maps. These methods offer transparency by highlighting the regions of interest which helps medical practitioners to gain deeper insights into the diagnosis of colon cancer. The adoption of preprocessing methods such as non-local mean filtering with patch similarity enhances feature extraction and image clarity improving the performance of the proposed model.

Future work will focus on including more diverse samples and further larger datasets across different population considering histopathological image variations. There is also a potential to explore other types of cancers with the same architecture considering different image modalities and datasets. Implementing continuous learning mechanism will help the model to update and improve with new data, maintaining better performance. There will be a challenge deploying the model in real-time clinical settings considering the optimization of speed and efficiency without compromising the accuracy of the model. Integrating multimodal data such as histopathological images with genetic information has the possibility of enhancing the accuracy of diagnosis providing deeper insights about the disease.