Multi-Scale Transformer and Explainable AI Framework for Automated Detection of Circulating Tumor Cells

Cancer is the major cause of death worldwide, with millions of new cases diagnosed each year. Early detection of cancer diseases supports timely treatment and increases patient survival rates [1, 2]. One of the key biomarkers for early cancer detection is CTCs—tumor cells that detach from primary or metastatic tumors and enter the bloodstream. The early diagnosis and analysis of CTC is used for prevention and nursing response. But finding its presence is a difficult task due to its extreme rarity (as low as 1 CTC per billion blood cells) [3].

Different techniques have been developed for CTC detection. It can be categorised into biological, physical, and image-based approaches. Biological methods based on immunomagnetic separation and molecular markers to isolate and identify CTCs. Physical methods use size differences and density between CTCs with filtration techniques. Image-based methods use microscopic imaging and computational techniques for automated CTC identification [4].

Recently, the arrival of Artificial Intelligence (AI) techniques has received greater attention due to their accuracy. AI models can be classified into two categories: Machine Learning and Deep Learning [5]. In ML models, the feature learning and classification processes are processed independently. Conversely, in Deep Leraning (DL) models, the learning and categorisation are carried out in the architecture itself. The well-known ML models are catboost, Decision Trees and Random Forests (RF). The well-known DL models are Convolutional Neural Networks (CNNs) and Transformer-based architectures. However, these models have several limitations: limited spatial awareness, high computational complexity, lack of feature selection mechanisms and limited model interpretability. Conventional CNN models failed to capture long-range dependencies and to detect morphological differences between CTCs and normal blood cells. Deep CNN-based architectures require large datasets and extensive computational resources for processing them. Many DL models extract a high volume of redundant features which can lead to overfitting and decreased efficiency.

Based on the drawbacks of existing DL models, there is a need for an advanced, efficient and interpretable DL model. In this work, a novel threefold model is proposed for CTC classification. For improved feature extraction, the MS-Swin-T is proposed. Then, optimized with AO. The classification is done with Explainable Boosting Machines. It can overcome the limitations of existing DL models.

In this work, a novel threefold DL architecture is proposed for CTC categorisation. The conventional Swin Transformer is modified with multi-scale operation in patch embedding and window attention. The extracted feature is optimized using AO. The optimized features are classified using the Explainable Boosting Machines model.

3.1 Swin Transformer

The Swin Transformer is a hierarchical vision transformer designed for efficient high-resolution image processing. It applies self-attention globally for improved feature extraction. The working of the Swin Transformer involves three main processes: Window-based self-attention, Shifted windows and Hierarchical structure. Window-based self-attention is computed within fixed-size local windows. The shifted windows strategy is used for cross-window feature interaction without high computational cost.

3.2 Key mechanisms in Swin Transformer

3.2.1 Patch splitting and embedding

The input image XÎℝ^H×W×3 is divided into non-overlapping patches of multiple sizes P1×P1, P2×P2, …PN×PN which forms a sequence of feature vectors. These patches are linearly projected to obtain an initial feature representation.

${{F}_{patch~,i}}={{W}_{p,i}}X+{{b}_{p,i}}$                     (1)

where, ${{W}_{p,i}}$ is the projection matrix for the i-th patch size, ${{b}_{p,i}}$ is the corresponding bias term, ${{F}_{patch~,i}}$ represents the extracted patch features for scale i. The structure of the proposed Swin architecture is given in Figure 1.

Figure 1. Swin Transformer architecture

The final multi-scale feature representation is obtained by concatenation and transformation:

${{F}_{patch}}={{W}_{c}}\left[ {{F}_{patch,1}},{{F}_{patch,2}}\ldots ..{{F}_{pathc,N}} \right]+{{b}_{c}}$                  (2)

where, ${{W}_{c}}$ and ${{b}_{c}}$ are trainable parameters for feature fusion.

3.3 Multi-scale window-based self-attention

In standard Swin Transformer, self-attention is applied within fixed-size windows. However, in MS-W- MSA Swin Transformer applies self-attention within non-overlapping windows of size M×M. In the standard Swin Transformer, self-attention is applied within fixed-size windows. However, in MS-W-MSA, multiple window sizes W1×W1, W2×W2, ...WM×WM are used, leading to multi-scale spatial attention:

$\begin{align}  & MS-W-MSA\left( Q,K,V \right)= \\ & \sum\limits_{j=1}^{M}{{{W}_{j}}}Softmax\left( \frac{{{Q}_{j}}{{K}^{T}}_{j}}{\sqrt{d}}+{{B}_{j}} \right){{V}_{j}} \\ \end{align}$                  (3)

where, j indexes the window scale, Q, K, V are the query, key, and value parameters, ${{B}_{j}}$ is the correlated positional bias specific to each window, ${{d}_{k}}$ is the scaling factor, ${{W}_{j}}$ are learnable weights to adaptively merge information across scales. The final multi-scale attention output is obtained via weighted summation.

3.4 Shifted windows mechanism

To enable feature interactions across windows, windows are shifted by M/2 pixels before the next transformer block. It is used for neighboring patches interaction.

${{F}_{SW-MSA}}=MS-W-MAS(Shift\left( {{F}_{prev}} \right))$                       (4)

where, Shift(.) is a predefined shifting function.

3.5 Cross-scale feature interaction and fusion

The extracted multi-scale features from different patch and window sizes are aggregated using a weighted summation mechanism:

${{F}_{merged}}=\sum\limits_{i=1}^{N}{\sum\limits_{j=1}^{M}{{{W}_{ij}}{{F}_{MSA,ij}}}}$                (5)

where, ${{W}_{ij}}~$are learnable parameters ensuring adaptive cross-scale feature fusion.

The final feature representation is passed through a Feed-Forward Network (FFN) with GELU activation:

${{F}_{FFN}}=GELU\left( {{W}_{1}}{{F}_{merged}}+{{b}_{1}} \right){{W}_{2}}+{{b}_{2}})$                      (6)

where, ${{W}_{1}}$, ${{W}_{2}}$ are trainable matrices, b1, b2 are bias terms.

In our proposed multi-scale window attention mechanism (MS-W-MSA), multiple window sizes ${{W}_{1}}\times {{W}_{1}},{{W}_{2}}\times {{W}_{2}},...,{{W}_{M}}\times {{W}_{M}}$ are used to capture different spatial scales within the self-attention layers. The selection of these window sizes is guided by the task's need to capture both fine-grained details and broader contextual information. This choice is not arbitrary but designed to enhance performance in CTC detection. The learning strategy for the adaptive weights ${{W}_{j}}$ is handled through a Softmax normalization mechanism which ensures balanced attention across all window scales. This mechanism prevents any single scale from being overly emphasized or ignored. It is used for a robust fusion of multi-scale information. Moreover, the use of a SW-MSA ensures that neighboring patches can interact and increase the model’s ability to detect spatial relationships across the entire input. Thus, the multi-scale window sizes and their corresponding learnable weights are designed to optimize feature extraction from both local and global perspectives.

3.6 Feature selection using Aquila Optimizer (AO)

Once features are extracted from MS-Swin-T, AO is applied to select the most informative features. Feature optimization is crucial in DL models to enhance performance with reduced computational complexity. It is also used to eliminate redundant or irrelevant information. By optimizing the selected features, models become more interpretable and computationally efficient.

3.6.1 AO model

The Aquila is the well known bird of prey in the Northern region. While hunting, the male Aquila is the bird that can hunt its prey solo. It can utilize its sharp talons and speed to hunt its prey like squirrels, hares, rabbits and also the many birds and animals. Most likely Aquila diet is the ground squirrels. The Aquila has used four steps for hunting with several distinct differences. The ability of Aquila is cleverer and quicker to move a back-and-forth motion in its hunting strategy and depending on the situation. There are several steps taken for hunting methods of Aquila which are given in the following:

The first step is a heavy soar with a vertical stoop which is used to choose its prey. The second step is a contour flight with a short glide attack done by Aquila. The Aquila catches the prey that cannot run or fly from it. The third step is a slow descent attack by a low flight. The Aquila is bending low towards the ground, and next attacks gradually on the prey. It chooses its victim and lands on the neck and back of the prey that can be tried to penetrated. The different step is walking and grabbing the prey by walking on the surface and grabbing to pull its target. This strategy is used to grab a young or large prey like a deer or sheep in its region.

By observing the hunting behaviour of Aquila, it is clear that it is a more skilful and intelligent hunter and probably after humans. The AO is developed based on this strategy to apply to the TransUnet segmentation model. The AO can be derived with several steps which are given in the following.

(i) Solutions Initialization

The initialization is the basic step which is a population-based model that randomly places a population of candidate solutions (X’). The position is randomly initiated as follows:

$A_{ij}^{'}=r\times \left( UB_{j}^{'}-LB_{j}^{'} \right)+LB{{'}_{j}},$ i=1,2…Nj=1, 2…D                         (7)

where, A’ indicates a set of present candidate solutions, $\text{A}{{\text{ }\!\!'\!\!\text{ }}_{\text{i}}}$ indicates positions of the i^th solution, r indicates a random number, $\text{UB}_{\text{j}}^{\text{ }\!\!'\!\!\text{ }}$ indicates a j^th upper bound and $\text{LB}_{\text{j}}^{\text{ }\!\!'\!\!\text{ }}$ represents the j^th lower bound.

(ii) AO based Mathematical derivation

The AO method is derived from its four-step hunting behaviour that can be transferred from the exploration stage to the exploitation stage. Using these stages, the hunting behaviours are in the condition if $\text{t }\!\!'\!\!\text{ }\le \left( 23 \right)\text{*T }\!\!'\!\!\text{ }$.

Step 1: High soar with the vertical stoop (A1)

The AO is broadly explored from a high soar to recognise the search space area. Aquila’s behaviour of high soar with the vertical stoop is expressed as below:

$A_{1}^{\prime }\left( {{t}^{\prime }}+1 \right)=A_{best}^{\prime }\left( {{t}^{\prime }} \right)\times \left( 1-\frac{1}{T} \right)+\left( A_{M}^{\prime }\left( {{t}^{\prime }} \right)-A_{best}^{\prime }\left( {{t}^{\prime }} \right)*r \right)$                         (8)

Eq. (8) expressed the first step of the search model $A1$. The $A_{best}^{'}\left( {{t}'} \right)$ denotes the best fitness until t’^th$~$iteration to predict an approximate location of prey. The $1-\frac{1}{T}$ can be controlled an exploration through several iterations. $A_{M}^{'}\left( {{t}'} \right)$ indicates the locations of the present solution of mean value that is connected at t’^th$~$iteration is expressed in the below Eq. (9). In the above Eq. (8), the random value among a 0 and 1. t’ and T’ denotes a present iteration and the higher quantity of iteration.

$A_{M}^{\prime }\left( {{t}^{\prime }} \right)=\frac{1}{N}\sum\limits_{i=1}^{N}{A_{i}^{\prime }\left( {{t}^{\prime }} \right),\ {{j}^{\prime }}=1,2,\ldots ,D}$                         (9)

Step 2: Narrowed exploration (X’₂)

When the prey is identified from high, it can circle the aimed prey, observe the land, and then attack which is named contour flight with random movement. The Aquila selected a target area of prey narrowly in preparation for the attack. This narrow strategy of hunting is expressed in Eq. (10):

$A_{2}^{\prime }\left( {{t}^{\prime }}+1 \right)=A_{best}^{\prime }\left( {{t}^{\prime }} \right)\times levy\left( D \right)+A_{R}^{\prime }\left( {{t}^{\prime }} \right)+\left( {{y}^{\prime }}-{{x}^{\prime }} \right)*r$                       (10)

where, $A_{2}^{'}\left( {t}'+1 \right)$ indicates a next iteration solution of t’ by a second step search strategy ($A_{2}^{'}$), $A_{R}^{'}\left( {{t}'} \right)$ indicates a random result occupied in the range between zero ti N at the i^th iteration. Levy (D) denotes a Levy flight parameter that is expressed using Eq. (11):

$levy\left( D \right)=s\times \frac{u\prime \times \sigma \prime }{\left| v \right|\prime }$                     (11)

where, s denotes fixed values as 0.01, u and υ represent the random numbers among 0 and 1. $\sigma '$ expressed in the following Eq. (12):

${\sigma }'=\left( \frac{r'\left( 1+{\beta }' \right)\times \text{sin}\left( \frac{\pi \beta }{2} \right)}{\frac{{r}'\left( 1+{\beta }' \right)}{2}~\times \beta \times {{2}^{\left( \beta -\frac{1}{2} \right)}}} \right)$                  (12)

where, β’ indicates a tuning parameter. The Eqs. (13)-(17) below, y’ and x’ denoted the spiral structure in the search is expressed as:

 ${y}'=r'\times \text{cos}\left( \theta  \right)$                            (13)

${x}'=r'\times \text{sin}\left( \theta  \right)$                          (14)

where,

${{r}^{\prime }}=r_{1}^{\prime }+U\prime \times D_{1}^{\prime }$                     (15)

$\theta =-\omega \times D_{1}^{\prime }+{{\theta }_{1}}$                           (16)

${{\theta }_{1}}=\frac{3\times \pi }{2}$                          (17)

where, $r_{1}^{'}$ is valued between one to twenty for the constant search cycles, $D_{1}^{'}$ indicates an integer number from 1 to the search space length (D’),$~U'$ indicates a low value fixed to 0.00565 and ω denotes a minimum value fixed to 0.005.

Step 3: Expanded exploitation (X3)

If the prey region is marked accurately, then Aquila is ready for attack. This attack is named as slow descent attack by low flight which is the third step strategy of Aquila. Where, the optimizer executes the chosen place of the target to reach closer place of the target. This step is expressed in Eq. (18):

$\begin{align}  & A_{3}^{\prime }\left( {{t}^{\prime }}+1 \right)=    A_{best}^{\prime }\left( {{t}^{\prime }} \right)-A_{M}^{\prime }\left( {{t}^{\prime }} \right)\times {{\propto }^{\prime }}-r+\left( \left( U{{B}^{\prime }}-L{{B}^{\prime }} \right)\times r+L{{B}^{\prime }} \right)\times {{\delta }^{\prime }} \\ \end{align}$                      (18)

where, $A_{3}^{'}\left( {t}'+1 \right)$ denotes the next iteration solution of t’ derivate by the third search approach (A3). $A_{M}^{'}\left( {{t}'} \right)$ indicates a current solution mean value at t’ th iteration. $\propto '$ and δ’ $~$are the exploitation parameters to a minimum value between zero to one.

Step 4: Narrowed exploitation (A4)

If the Aquila is so close to the prey, the land and attack by walking which is the fourth step of the Aquila strategy named as walk and grab prey. This step is expressed in Eq. (19):

$\begin{align}  & A_{4}^{\prime }\left( {{t}^{\prime }}+1 \right)=   QF\prime \times A_{best}^{\prime }\left( {{t}^{\prime }} \right)\times \left( K_{1}^{\prime }\times {{A}^{\prime }}\left( t \right)\times r \right)-K_{2}^{\prime }\times levy\left( D \right)+r\times K_{1}^{\prime } \\ \end{align}$                    (19)

where, $A_{4}^{'}\left( {t}'+1 \right)$ indicates a next iteration solution of t’ by A4. QF’ represents the quality function of search strategies equilibrium given in Eq. (20) and then. $K_{1}^{'}$ indicates a different motion to track the prey in elopes that is expressed in Eq. (21). $K_{2}^{'}$ indicates a decreasing value from 2 to 0 of AO flight slope from the initial position to the final position that is given in Eq. (22):

$QF\left( {{t}'} \right)={{t}^{2\times r-1/{{\left( 1-T \right)}^{2}}}}$                  (20)

$G_{1}^{\prime }=2\times r-1$                         (21)

$G_{2}^{\prime }=2\times (1-\frac{t}{T})$                      (22)

The AO for feature selection begins with the initialization phase where a set of candidate feature subsets is generated randomly. Each subset represents a potential solution containing a selected group of features. The fitness function $f\left( \text{A} \right)$for a feature subset $\text{X }\!\!~\!\!\text{ }$can be expressed as:

 $f\left( \text{X} \right)=\frac{1}{1+\text{Classification} \text{ Error}\left( A \right)}$                                (23)

where,

$\text{Classification}\ \text{Error}\left( A \right)=\frac{FP+FN}{TP+TN+FP+FN}$                       (24)

Here, TP, TN, FP, and FN represent the number of true positives, true negatives, false positives, and false negatives, respectively. The aim of the optimizer is to reduce the classification error by selecting the optimal subset of features.

During the exploration phase, AO applies two strategies: high soar and narrowed exploration which help in identifying promising feature subsets by encouraging diverse searches across the feature space. This phase prevents premature convergence and ensures a broad search for optimal solutions. Following this, the exploitation phase is introduced to refine the selected features further. AO uses expanded exploitation and narrowed exploitation techniques to fine-tune the feature subsets by focusing on the most informative features while reducing redundancy.

Throughout the process, AO continuously updates the best solution keeping track of the optimal feature subset with the highest classification performance. The optimization process terminates when the convergence criteria are met or after a predefined number of iterations. At the end of the process, the most informative and compact feature subset is selected.

3.6.2 EBM-based classification

EBM is a variant of cyclic gradient boosting Generalized Additive Model designed to overcome the limitations of existing ML models like RF and catboost models. It resembles a tree-based structure to achieve higher accuracy. It can be expressed as follows:

$g\left( E\left[ y \right] \right)={{\beta }_{0}}+\sum{{{f}_{j}}({{x}_{j}})}$                               (25)

where, g denotes the link function. It can be used to configure the model as a regression or a classification. Compared to other models, it learns features using bagging and gradient boosting. For effective boosting, the round-robin fashion is applied in feature learning. This round robin fashion is used to overcome the effects of co-linearity and identify important features based on the best feature function fj. In addition, it can detect pairwise interaction automatically as follows:

$g\left( E\left[ y \right] \right)={{\beta }_{0}}+\sum{{{f}_{j}}({{x}_{j}})}+\sum{{{f}_{i,j}}({{x}_{i}},{{x}_{j}})}$                          (26)

This interaction increases EBM accuracy further. EBM is an additive model where each feature contributes to classifications in a modular way.