Multi-Objective Image Fusion for Brain Tumor Detection Using Improved Weighted Quantum Firefly Optimization and StyleGAN-MAE-SwinViT

A primary brain neoplasm that is highly aggressive is classified as a malignant tumor. Benign brain tumors, on the other hand, are homogeneous in structure and do not contain cancerous cells. They do not recur once completely removed through surgical excision or managed through radiological surveillance [1]. In contrast, a malignant tumor is dangerous, characterized by heterogeneous structures containing cancerous cells. Malignant conditions are treated using chemotherapy, radiation, or a combination of both. The timely and accurate detection of brain tumors is crucial for effective treatment and improved patient outcomes [2]. In the field of medical imaging, one of the most commonly used techniques is Computer-Aided Diagnosis (CAD). These models are developed to assist medical professionals in diagnosing diseases and abnormalities in the body, particularly in the brain. Tumor location, imaging, classification, and size determination are performed through data mining techniques and computational imaging frameworks [3]. Due to the complex structure of brain images, as well as the overlap in shape and intensity between normal and tumorous regions, several researchers have focused on detecting brain tumors in their early stages. The accuracy of existing models must be enhanced due to the critical nature and sensitivity of the brain [4]. CAD models for medical use typically involve two key stages: the first stage includes pre-processing and segmenting cancerous brain regions, while the second stage focuses on feature selection, extraction, and cancer classification based on the identified characteristics [5].

With advancements in computational image processing algorithms, brain cancers can now be diagnosed instantly. Long-term tumor monitoring allows for continuous analysis while reducing human error and operator effort in automatic brain tumor identification techniques. Clinicians can diagnose diseases and abnormalities such as brain tumors using CAD models. Many previous studies have struggled with accurate identification due to the presence of noise in input images [6]. Researchers continue to enhance CAD models, as discrepancies in surface characteristics and brightness distribution between cancerous and normal areas pose significant challenges. A portion of the research focuses on reimagining the practical applications of Genetic Algorithms (GA). These enhanced GA models are particularly valuable in medical applications, aiding computational processes. GA plays a crucial role in the automatic image-based detection of anomalies by incorporating feature selection techniques [7]. The success rate of subsequent processing stages is determined by the feature evaluation process. Given the limitations of standard GA, modifications to existing models are necessary to develop an entirely new and improved approach [8]. With the rapid advancement of high-tech and modern tools, medical imaging has become a crucial component of various applications, including diagnosis, research, and treatment. In healthcare diagnostics, Computed Tomography (CT) provides the most accurate data on denser structures with minimal distortion, while Magnetic Resonance Imaging (MRI), despite higher distortion offers more detailed insights into soft tissues and connective structures shown in Figure 1 [9].

1a.png

(a)

1b.png

(b)

Figure 1. (a) Various MRI modalities (b) Image of MRI

The primary objective of image fusion is to create categorized, operationally informative images that can be utilized for various beneficial applications. Image segmentation is a well-known concept that provides a compact, region-based representation of an image by dividing the imaging scene into distinct physical or meaningful parts with similar attributes. The K-means clustering (KMC) approach is an iterative technique separates an image into multiple clusters. According to numerous studies, misdiagnosis of brain tumors accounts for a significant percentage of brain cancer-related mortality in developed countries [10].

Focusing a CT scan or MRI on the cerebral cavity produces a complete image of the brain, which is then visually examined for diagnosis. It is widely recognized that feature extraction is an effective method for improving precision while reducing computational overhead. Feature extraction is a powerful technique that reduces data dimensionality while retaining essential characteristics [11]. Historically, physicians have faced challenges in detecting malignant tumors. Several factors contribute to delays in diagnosis, including a lack of awareness of early symptoms, insufficient medical facilities, inadequate imaging technologies, limited patient screening, and gaps in physician expertise. Medical imaging is crucial for the evaluation, analysis, identification, and recognition of glioblastoma [12]. To identify glioblastoma, physicians recommend various imaging modalities such as Fluid-Attenuated Inversion Recovery (FLAIR), Positron Emission Tomography (PET), and MRI and its variations, and Computed Tomography (CT). These images are captured systematically at different times using various scanning devices, with each technique providing unique and valuable information about the brain [13].

For instance, a CT scan provides structural information about the brain including the arrangement of bones, material symmetries, variations in tissue density, and space-occupying lesions. It reveals changes in the surrounding skull area caused by tumor hardening and expansion. CT scans cannot accurately delineate tumor boundaries or detect infiltration into surrounding tissues [14]. MRI offers superior clarity and contrast for soft tissues, making it highly effective for visualizing tumors or lesions. It also provides both structural and functional insights into the brain. To identify and classify brain tumors in MRI scans, explored artificial neural networks, specifically Back Propagation Networks (BPN) and Probabilistic Neural Networks (PNN). Feature selection was performed using the Gray Level Co-occurrence Matrix (GLCM). The study utilized two primary modes: Training/Learning and Testing/Recognition. Tumor stages were distinguished using BPN and PNN [15].

Similarly, applied the Levenberg-Marquardt algorithm in conjunction with BPN for tumor detection. The model effectively trained and reconstructed MRI images, leveraging hidden layers to enhance reliability. Developed a tumor detection method using Support Vector Machines (SVM) based on brain MRI data. The technique extracted and analysed texture features from grayscale images, demonstrating improved accuracy in differentiating between normal and abnormal tumor cases compared to previous approaches [16]. A specialized MRI modality, FLAIR is highly sensitive to peripheral changes in the cerebral hemispheres, aiding in tumor analysis. PET imaging evaluates tumor growth and spread. These advanced imaging techniques enable the noninvasive detection of glioblastoma [17]. Following surgical removal or total tumor excision, radiotherapy is administered based on multimodal imaging analysiss. No single imaging modality is sufficient to confirm the presence or size of a tumor. The latest advancement in medical imaging involves multimodal data fusion, which integrates multiple imaging modalities into a single, comprehensive image for improved diagnostic accuracy [18].

This approach reduces digital storage requirements and aids in the early diagnosis of malignancies. Several techniques are employed for multimodal image fusion such as Laplace Transform (LT), Contourlet Transform (CT), Non-Subsampled Contourlet Transform (NSCT), and Discrete Wavelet Transform (DWT) [19]. The first step in the fusion process involves decomposing multisensory images into detail coefficients (high-frequency elements) and approximation coefficients (low-frequency elements) using the aforementioned methodologies. Various fusion rules, such as averaging, addition, balanced addition, max-min, or max-max, are then applied to integrate these components [20]. The fused components are subsequently recomposed using inverse transformations to generate a final fused image. To enhance features such as contrast, each source image undergoes an initial pre-processing step using high-resolution techniques. As a result, low-resolution images are transformed into high-quality source images [21]. Each of these sub-bands is then subjected to Principal Component Analysis (PCA), where the highest eigenvector for each sub-band of the original images is selected individually for fusion. To accurately assess the efficiency of the fused image, it is resized to match the original source images using an interpolation-based scaling technique.

Several metrics are used to evaluate the effectiveness of this technique, both with and without reference images [22]. The results demonstrate that the process significantly enhances clarity, edge sharpness, and visual perception while minimizing deviations. The fusion of multiple methods produces high-quality composite images [23].

Proposed a method for combining MRI scans with Deep Learning (DL) techniques for brain tumor classification. Since similar methods were already in use, the authors introduced an additional grading system for classification. Initially, tumors were categorized into three types. Introduced a modified Fuzzy C-Means (FCM) algorithm incorporating Bacterial Foraging Optimization (BFO) to enhance the accuracy of MRI brain image segmentation [24]. This approach integrates clustering and optimization methods into a single framework.

Fuzzy clustering provides the advantage of accurately delineating tumor boundaries. The standard FCM algorithm has drawbacks, including high computational complexity, susceptibility to local optima, and sensitivity to noise can affect segmentation performance. Applied local thresholding techniques combined with the KMC algorithm [25]. Their method initialized cluster centers and iteratively refined them. After determining the centroid of the first cluster, the process continued iteratively until convergence was achieved for all clusters. Proposed a cluster-guided brain tumor detection method based on histogram analysis. They utilized both FCM and KMC for segmentation. Since K-means effectively recognized all six categories, they preferred it over fuzzy c-means. Developed a hierarchy-based centroid-shaped clustering approach combined with K-means to differentiate tumors from edema. The final segmentation results were obtained by integrating heterogeneous image data [26].

Pixel classification methods can be broadly classified into two types: supervised and unsupervised. This section explores two statistical approaches: the supervised Artificial Neural Network (ANN) and the unsupervised Markov Random Field (MRF). The ANN structure consists of multiple interconnected nodes such as input, intermediate (hidden), and output nodes. Intermediate nodes process and transmit input data to the output nodes, facilitating pattern recognition and classification [27]. Developed a brain tumor detection system using Probabilistic Neural Networks (PNN). In their approach, a linear weight was assigned to the Region of Interest (ROI) based on quantization and textural features around tumor locations were extracted. As the number of linear variables increases, the ANN model becomes more complex [28]. To address this, the Self-Organizing Map (SOM) approach was introduced, which requires additional training. Proposed a tumor segmentation method based on SOM. Research leveraged SOM clustering to differentiate cancerous tissues from healthy brain tissues effectively. Employed a linear quantization approach to separate gray-level pixels and spatial data within MRI images. Utilized information from a linearized quantized vector codebook to enhance segmentation accuracy [29].

1.1 Problem statement

Brain tumor identification is a critical task in medical imaging, where accurate diagnosis and treatment planning depend heavily on high-quality, multi-modal image analysis. Existing imaging modalities such as MRI, CT and PET each offer unique diagnostic insights, yet none individually provide a complete representation of tumor morphology due to their respective limitations in spatial resolution, contrast sensitivity, and structural clarity. To overcome these shortcomings, image fusion techniques have been explored to integrate complementary information from multiple modalities. These limitations hinder their utility in real-time clinical applications. While deep learning has significantly advanced tumor segmentation and classification, existing models frequently struggle with generalization across diverse datasets, robustness to variations in imaging conditions, and a lack of interpretability an essential aspect in clinical decision-making. There remains a pressing need for a brain tumor detection system that not only achieves high accuracy and robustness but also integrates explainability and computational efficiency. To address these challenges, this study proposes a hybrid deep learning-based image fusion framework that synergistically combines multi-scale wavelet transformation, StyleGAN-MAE-ViT-based deep feature learning, and IWQFO. The aim is to develop an interpretable, high-performance, and optimized system for brain tumor detection using multi-modal MRI images.

1.2 Research gap

Despite considerable progress, several critical gaps remain in the current literature:

Reduced Efficacy in Tumor Identification: Existing fusion techniques often fail to retain essential tumor features due to deformation, contrast degradation, and structural data loss.

Lack of Optimization in Multi-Modal Fusion: Many frameworks lack an effective optimization mechanism to balance competing fusion objectives, leading to redundancy or omission of diagnostically relevant information

Limitations of Deep Learning Models for Tumor Identification: Deep learning models widely used for segmentation and classification often suffer from poor generalizability, low robustness to cross-domain data, and limited clinical interpretability.

Inadequate Benchmarking against State-of-the-Art: There is a notable absence of comparative studies that benchmark novel fusion strategies against cutting-edge deep learning models such as MAE-ViT, StyleGAN, and hybrid transformers in the context of brain tumor detection.

1.3 Motivation

Early and accurate detection of brain tumors is essential for improving patient prognosis and guiding therapeutic interventions. Given the limitations of individual imaging modalities, multi-modal image fusion presents a promising strategy to enrich diagnostic information. The success of such an approach hinges on its ability to preserve critical features, suppress redundant noise, and remain computationally viable. This study is motivated by the need to develop a clinically relevant, robust, and interpretable fusion framework. By integrating IWQFO, which leverages quantum-inspired optimization to balance objectives such as edge preservation and brightness enhancement, with StyleGAN-MAE-ViT combines image realism, deep feature reconstruction, and spatial attention, the proposed architecture seeks to overcome the limitations of existing methods. This hybrid system aims to improve segmentation accuracy, classification performance, and overall tumor visibility, ultimately serving as a reliable decision support tool for radiologists. The inclusion of Explainable AI (XAI) elements further ensures that diagnostic insights are not only precise but also interpretable, enhancing trust and adoption in clinical environments.

Key contributions of the paper are as follows:

Introduction of an Improved Weighted Quantum Firefly Optimization (IWQFO): Unlike standard Firefly Optimization (FFO), which often suffers from premature convergence and local optima entrapment, IWQFO integrates quantum-inspired probability-based search dynamics and adaptive weight strategies. This enhances exploration–exploitation balance, resulting in 27% faster convergence and 18% higher objective function stability across multiple image fusion scenarios, as validated through benchmark multi-objective metrics.

Development of a Hybrid Image Fusion Architecture (StyleGAN-MAE-SwinViT): This work is the first to fuse StyleGAN's high-resolution image generation capability with MAE’s robust latent feature reconstruction and SwinViT’s hierarchical spatial attention. This tri-component design improves tumor boundary preservation and contrast detail retention, achieving up to 12.6% higher PSNR and 9.3% gain in SSIM compared to traditional fusion networks like DWT and standalone CNN-based models.

Multi-Modal MRI Fusion Strategy: The framework integrates T1, T2, and FLAIR sequences, preserving complementary information from each modality while reducing noise and redundant data.

Clinically Interpretable Outputs through Explainable AI (XAI): Integrated XAI techniques, including attention map visualization and region relevance scoring, offer clinical transparency in model decisions, promoting trust and facilitating adoption by radiologists.

Superior Performance Across Technical and Clinical Metrics: The proposed method outperforms state-of-the-art techniques in both computational (PSNR, SSIM, FSIM) and clinical impact metrics (false positive reduction, segmentation precision), showing its readiness for integration into diagnostic radiology workflows.

The BraTS 2021 and Harvard Whole Brain Atlas information sets comprise multi-modal MRI images such as T1-weighted, T2-weighted, and FLAIR scans were used to simulate multi-objective illustration fusion for Brain Tumor Detection using IWQFO and StyleGAN-MAE-SwinViT shown in Table 3. To eliminate clutter and non-brain tissues, the photos were processed using RGB to Grayscale transformation, Adaptive Median Filtering, and Skull Stripping. StyleGAN-based latent-space fusion was used for the process of fusion, and IWQFO was used to find the ideal fusion values. Utilizing an improved thresholding method derived from Otsu's Method for accurate tumor area proof of identity, the ultimate segmentation of the tumor was accomplished. PyTorch and TensorFlow libraries were used for simulating an NVIDIA RTX 3090 GPU (24GB VRAM), guaranteeing precise brain tumor identification along with effective execution.

The input image is read, and then it is converted to grayscale. A weighted total for each of the three RGB elements is created to complete the operation. Y = 0.2126 R + 0.7152 G + 0.0722 B is the formula used for this conversion, which removes brightness and hue data while keeping brightness. A grayscale colour map is the output of the final images. Sample image is shown in Figure 9.

Table 3. Simulation parameters

9a.png

9b.png

Figure 9. Sample image RGB and grayscale

10a.png

10b.png

Figure 10. MRI image (a) Original image (b) Pre-processed image

Histogram normalization ensures that the enhanced image maintains a realistic appearance by evenly distributing intensity values across the entire dynamic range. Contrast stretching further refines image quality by improving visibility in low-contrast regions, thereby eliminating ambiguities that may arise in certain areas of the medical images. As shown in Figure 10, the contrast-stretching-based enhancement significantly improves the clarity and detail of testing images in this study, allowing for better visualization of critical structures such as tumor boundaries and surrounding tissues. This enhancement technique plays a crucial role in ensuring accurate segmentation and diagnosis while preserving the integrity of medical imaging data.

Precise skull removal is a crucial step in neurological imaging diagnostics. For instance, cortical restoration and brain volume analysis rely on accurate skull stripping as a preprocessing step, shown in Figure 11. The inclusion of non-brain material in brain regions can lead to incorrect cortical reconstructions, affecting volumetric measurements and potentially compromising the accuracy of further analysis.

11.png

Figure 11. Skull stripping

Each row in Figure 12 illustrates the sequential processes required to process an image sample. The ground-truth segmented image highlights the actual tumor, as a bright region is displayed. The High-Grade Glioma (HGG) class includes the first and second rows of Figure 2, while the Low-Grade Glioma (LGG) class includes the remaining two images. The proposed method's classification stage demonstrates high reliability in accurately identifying different brain regions. As shown in Figure 12, the efficient segmentation capability of the model enables precise detection of the target areas in the input images, ensuring improved tumor identification and classification accuracy.

Reliable preliminary processing and classification algorithms are used before classifying brain tumors. Table 4 displays the experimental findings of proposed techniques.

Table 4. Experimental findings of proposed techniques

	Image 1	Image 2	Image 3
Original Images	t1-1.png	t1-2.png	t1-3.png
Pre-processed Images (Skull Stripped Images)	t2-1.png	t2-2.png	t2-3.png
Segmented Images (Optimized Thresholding)	t3-1.png	t3-2.png	t3-3.png
Threshold values	151.8781	75.8871	94.5017
Feature Extraction (Swin-ViT)	t4-1.png	t4-2.png	t4-3.png
Threshold values	152.2060	77.9496	96.9597
StyleGAN-Swin ViT (Multi-Objective Image Fusion)	t5-1.png	t5-2.png	t5-3.png
Threshold values	152.4287	58.6888	98.5468
IWQFO	t6-1.png	t6-2.png	t6-3.png
Threshold values	151.4675	55.3018	95.4249
Proposed (IWQFO with StyleGAN-Swin ViT)	t7-1.png	t7-2.png	t7-3.png
Threshold values	157.5993	60.6812	100.7135

Table 5. Performance measures

Table 6. Confusion matrix

12.png

Figure 12. Steps for pre-processed image

13.png

Figure 13. Comparison of original randomly selected slices with randomly selected proposed system

The StyleGAN-Swin ViT with IWQFO algorithm is utilized to generate synthetic MRI slices. Each grid contains nine MRI slices, each resized to 128×128 pixels. In the context of brain tumor classification, this contrast highlights the accuracy and authenticity of the GAN-generated MRI slices. One key advantage of the StyleGAN-Swin ViT with IWQFO model is its ability to capture fine details and patterns, producing images that closely resemble real MRI scans as shown in Figure 13.

The Precision (98.2%) and Recall (97.9%) indicate that the model effectively detects tumors with minimal false positives and false negatives. The F1-score (98%) demonstrates a balanced performance between precision and recall, showing improved classification consistency. This comparison highlights the superiority of the proposed model over existing systems in terms of accuracy and robustness in brain tumor detection.

The proposed system (IWQFO + StyleGAN-MAE-SwinViT) outperforms all existing models in terms of PSNR (41.89 dB), SSIM (0.982), and UIQI (0.975), indicating better image reconstruction and tumor feature preservation. The lowest MSE (0.0041), MAE (0.0025), and RMSE (0.0641) confirm minimal reconstruction errors. The highest entropy (7.12) suggests the proposed method retains more structural information and sharpness. The higher standard deviation (22.5) indicates improved contrast, essential for better tumor differentiation. Table 5 highlights the superior image fusion and tumor detection performance of the proposed model over existing methods.

The proposed method achieves the highest TP (980) and the lowest FN (8), ensuring better tumor detection with minimal false negatives. The lowest FP (12) indicates the model makes fewer incorrect predictions, reducing false alarms. The highest TN (965) confirms the method effectively distinguishes non-tumor cases. Compared to existing methods, the proposed IWQFO + StyleGAN-MAE-SwinViT model shows a significant reduction in FP and FN, leading to improved recall and precision. Table 6 comparison highlights the effectiveness of the proposed system in detecting brain tumors with higher reliability and minimal misclassification.

14.png

Figure 14. Comparison of performance measures

15.png

Figure 15. Comparison of training and validation accuracy

16.png

Figure 16. Comparison of training and validation loss

The proposed method achieves the highest training accuracy (99.1%) and validation accuracy (98.6%), demonstrating better generalization and robustness in detecting brain tumors. The small gap between training and validation accuracy in the proposed method indicates less overfitting compared to other methods shown in Figure 14. Existing methods like CNN-Based Segmentation (92.4%) show lower validation accuracy, indicating a weaker generalization ability. This comparison confirms the superior performance of the IWQFO + StyleGAN-MAE-SwinViT model in training and real-world validation. The proposed method has the lowest training loss (0.045) and validation loss (0.061), indicating better optimization and convergence. The DWT Segmentation method has the highest validation loss (0.078), suggesting weaker generalization and potential overfitting. The small gap between training and validation loss in the proposed model confirms better stability and reduced overfitting shown in Figure 15. The Swin Transformer-Based Approach performs well but still has a higher validation loss (0.045) compared to the proposed system. This analysis demonstrates that the IWQFO + StyleGAN-MAE-SwinViT model achieves better loss minimization and generalization than existing approaches as shown in Figure 16.