YOLO Object Detector and Inception-V3 Convolutional Neural Network for Improved Brain Tumor Segmentation

The most common brain tumor is glioma, which also has the highest rates of mortality and highest rates of morbidity. For the diagnosis and curing of glioma, the accuracy of segmentation is very critical. The treatment and diagnosis of brain tumor can be caused by using the necessary tool known as magnetic resonance imaging (MRI) [1]. Various MRI sequences can present various tissue structures of brain tumor, and multimodal MRI of brain tumors is usually used to segment brain tumors. As there is a complex structure of brain tumor, individual differences and uncertainty in the boundaries of the tumor, the segmentation of the brain is not an easy task to do [2]. A lot of time is required by doctors in order to complete the manual segmentation, and the accuracy of the segmentation is poor. During the last years, deep learning-based automatic segmentation methods have demonstrated promising results in the segmentation of medical imaging [3]. The classification, object detection, and segmentation are computer vision tasks; these tasks by the deep learning process, if based on CNN, will perform much better. A CNN can learn complex data features without depending on manual feature extractions during the training process, improving brain tumor segmentation accuracy [4, 5].

A fully convolutional neural network (FCN) was proposed by Long et al. [6], which used the upsampling for restoring the map of the output to the same size as the image input and achieved semantical end-to-end image segmentation by converting the full connection layer into convolutionary layer. A U-net technique is proposed by Ronneberger et al. [7] to segment biological cells, consisting of two paths, i.e. contraction and expansion.

A contraction path includes max pooling used for down-sampling and convolution block used for the extraction of features. An expanding path includes the up-sampling instead of the down-sampling and a convolution block. If the connection is skipped between these two paths (i.e. contracting path & expanding path), then the features of exact resolution will be fused. U-net consists of simple structures and can achieve better segmentation results in medical images with a small sample size. Contrary to this, image segmentation of brain tumors based on U-net applications should be improved [8]. The U-contracting net's path, on the other hand, employs the shrinking of feature map by pooling layer and thus, cause the expansion of the receptive field.

Continuous pooling can result in providing no details of the image and have an impact on segmentation results. On the other hand, the essential problems are the differences in the size, shape, and location of brain tumors, as well as in obtaining more detailed segmentation target features and multiscale features [9]. The residual network [10] was proposed for solving the problems of gradient disappearance and network degradation as network depth increases. The addition of identity mapping b/w the input & the output of numerous convolutional layers makes the network converge easier and prevents network degradation [11].

A DeepLab model is proposed by Chen et al. [12], which is used to obtain multiscale features. The final pooling layer was removed in this model, and atrous convolutions expanded the receptive field. The input is sampled in parallel position by atrous spatial pyramid pooling (ASPP) with atrous convolutions of many different dilation rates; then, the results are spliced together. The extraction of multiscale features is better with ASPP. An inception network was proposed by Szegedy et al. [13]. The network's width can be increased by using the inception module. In the ILSVRC 2014 competition, GoogLeNet, which included the inception module, had outstanding classification results and detection results. To reduce image data loss, this paper proposes extending the receptive field by utilizing atrocious convolution and decreasing pooling layers. An atrous convolution is a process that involves the insertion of holes in a standard convolution kernel for expanding the feature extraction’s receptive field with no additional parameters [14].

We discovered some issues in the automatic segmentation of brain tumors by looking through the literature: (a) Highly Imbalanced Tumor over Background, (b) Inconsistency in Appearance, and (c) Similarity between tumor and healthy tissue. The difficulty in segmenting brain tumors is primarily due to the wide range of sizes, shapes, regularity, location, and appearance of brain tumors. Segmenting tumors from background is a highly imbalanced dense prediction task because the tumor area only accounts for a small portion of the whole slice image in some benchmark datasets, usually less than 1%. When using a machine learning model to perform automatic segmentation, it's important to keep such issues in mind. Even if they belong to the same connected component, different parts of a tumor can have different appearances. The appearance of different slices of brain tumors is usually quite different. The difficulty of methods based on appearance models is exacerbated by the fact that some tumors have inconsistent appearances. Also, when using machine learning techniques, the features extracted should account for this type of variation in appearance. Surrounding healthy tissues can sometimes resemble tumors. The tumor and the background have a similar appearance in this case. Even if strong contrast boundaries exist between the tumor and healthy tissue, the intensity contrast information can still be used to segment the tumor. Otherwise, it is difficult for an untrained human to locate a tumor. A tumor can look a lot like other parts of the brain in many cases. This type of tissue can make the segmentation process even more difficult. It will be difficult to distinguish tumor from background and correctly predict labels using a machine learning model.

This paper represents the combination of inception and atrous convolution to form the A-Inception module, and new U-net-based network architecture is proposed. To increase the network’s depth & width and then obtain different sizes of the receptive field, the encoder of the network uses a module of A-inception. At the same time, the network is enhanced with atrous spatial pyramid pooling for extracting the image's multiscale features.

The proposed brain tumor segmentation model is trained on three benchmark datasets. The performance achieved by the best model is (a) mean Dice score: Enh = 0.923, Whole 0.931, Core = 0.886, (b) mean sensitivity score: Enh = 0.937, Whole 0.943, Core = 0.982, and (c) HAUSDORFF99: Enh = 1.76, Whole 1.57, Core = 2.48.

Section 2 consists of a detailed literature review and its summary; section 3 is the research methodology section in which the framework of the proposed model is discussed along with the mechanism of extracting learnable features and Yolo version 3 architecture for tumor segmentation, the section 4 contains information about the experimentation performed, performance evaluation and comparison of segmentation results with state-of-the-art. The article finishes with Section 5, which serves as a summary of the article and future directions of the research.

As shown in Figure 1, the proposed technique comprises four basic steps: (1) enhancement, (2) classification, (3) localization, and (4) segmentation. Homomorphic wavelet filers are used for enhancing the input images and inceptionv3 derived deep features are used for input image classification. The identified images are segmented using Kapur entropy and localised using the proposed YOLOv2-inceptionv3.

3.1 Noise elimination using homomorphic wavelet filter

The images obtained from an MRI procedure with adverse conditions may be affected by noise, lowering the disease detection rate. For noise reduction, there are several filters available. These filters are dependent on the sort of noise present in the photos. The images are represented in frequency domain using the wavelet transform. Image is divided into three bands via decomposition: high to high (i.e., HH), low to high (i.e., LH) & high to low (i.e., HL). This study looks into the homomorphic wavelet filter decomposition for removing speckle noise, which is described mathematically as follows:

$\log _{f(x, y)}=\log _{g(x, y)}+\log \eta_{m}(x, y)$       (1)

Figure 1 depicts the use of homomorphic filter for noise removing process and wavelet decomposition, with the image split into four bands: i.e., High-Low, Low-High, High-High, and Low to High-High to High. When compared to other bands like HL, LH & LH-HH, the HH bands improve the quality of image. As a result, the High-High band is used to accomplish accurate segmentation for further processing.

3.2 Extracted deep features using pre‑trained inceptionv3 architecture

The applications like deep learning, speech recognition and computer vision are commonly used in artificial intelligence applications. However, as the field of deep learning becomes more popular, categorization into corresponding categories has become a big issue. Because correct models & architecture are developed in the time-saving manners, this challenge might be handled through transfer learning. Instead, then learning new features from start, this method involves leveraging previously acquired patterns to address various challenges. For problem solving, transfer learning employs pre-trained models; these models have been learned over a large data.

As a result, for feature learning, this study employs a pre-trained inceptionv3 transfer-learning model that includes 01 image, 094 batch-normalization (bn), 094 Convolutional, 14 max-pooling, 094 ReLU, softmax with the function of cross-entropy, 015 depth concatenation and fully connected layers. As shown in Figure 2, features are retrieved from completely connected layers termed prediction and then transferred to NSGA for improved feature selection.

In the first phase of our framework that is depicted in Figure 1, an ensemble supervised learning classification model is developed; using the bagging strategy, multiple classifiers are trained, and in the validation phase, a voting scheme is applied to the output achieved through several classifiers and predict whether the image is from the normal class or abnormal class containing single or multiple tumors. In the second phase, a Yolo-V3 based on inception-V3 CNN features is trained on tumor containing MRI image along with its annotated ground truth that has been provided by the radiologist, the ensemble classification method helps to only process the images that contain tumor without wasting time for detecting of tumor in normal images. In most Yolo architecture, Darknet CNN, which is 153 layers model, is used for features learning; in this framework, the Darknet model has been replaced with inception-V3 315 layers model to extract more robust features from images and improve the detection accuracy of the YOLO detector. In Table 1, the details of our proposed segmentation models are discussed as: in step 1, the homomorphic filter is applied to enhance the quality of medical images. Step 2 is the features extraction phase, where inception-V3 CNN architecture is used for feature learning from normal and tumors containing MRI images. In step 3, an ensemble classification model is used to detect tumor and non-tumor images. Finally, in step 4, the Inception-V3 features based YOLO object detector is used to localize and segment the brain tumor regions.

1.png

Figure 1. Our proposed brain tumor segmentation and classification models architectures

2.png

Figure 2. Learning based features extraction using inceptionv3 CNN model

3.png

Figure 3. Localizing the tumor in MRI images using proposed model

Table 1. Proposed tumor segmentation framework details

3.3 Features selection

Using the inceptionv3 network, a deep feature vector (1 x 1000) is obtained. The engineering features are used to choose the best feature vector using NSGA II. As seen in Table 2, the NSGA parameters are as follows.

Table 2. The parameters about NSGA II

3.4 Localization using YOLO‑inceptionv3 model

To locate the tumor location, a YOLO-inceptionv3 model is proposed which have 174 layers, with inceptionv3 having 165 layers with 01 input, 50 Conv, 50 bn, 50 activations ReLU, 06 mixed (depth concatenation), 03 max-pooling, 05 average pooling, and 09 layers of tinyYOLOv2 [40] model. Table 3 consists of several hyperparameters used to train our proposed tumor localization and segmentation framework. The model is also trained on some other hyperparameters as well, but the hyperparameters in the table listed are the most optimal through which the tumor is segmented with higher accuracy. As seen in Figure 3, the proposed model more precisely localises the tumor location. The MSE loss between predicted bounds and boxes of ground truth was optimised using the YOLOv2 model. Three main sorts of losses are used to train the model: localization, confidence, and classification. Localization loss computes error utilising location, estimating box size & ground truth among the expected and ground truth boxes. The objectiveness having error with detected objects in the jth limited box of grid I cell is computed using the confidence loss. The classification loss was used to compute probability for each grid cell class.

Table 3. Adjusts the hyper-parameters ofinceptionv3 CNN based YOLOv2 model

Table 4. Details of the BRATS MRI images dataset

3.5 Lesion segmentation

Variability in medical data is a major difficulty in medical imaging. Different modalities, such as X-ray, MRI, CT, and PET, show variances in human anatomy. The illness severity levels are analysed using the segmentation region. McCulloch's Kapur entropy approach [41] is used to segment tumors in the proposed method. In this method, the foreground and background regions' likelihood of intensity values distributions are measured, and entropy is calculated independently for each region. To enhance the total of their entropies, the best threshold value is used. In Figure 4, four different images are visualized; Figure 4-a is the input of volumetric MRI series to the segmentation deep learning model, the Figure 4-b is the output of Kapur based entropy method for tumor localization. Figure 4-c is the segmented brain tumor (lesion) in multiple MRI slices, and Figure 4-d is the overlay images of the segmented tumor and original MRI images where the segmented and localized tumor can be seen in the red color.

3.6 Data and implementation details

The dataset of BRATS 2018 that comprises of brain tumor Multiview and multiseries MRI images along with ground-truth (segmented mask) by the domain experts, the brain tumor subjects have various histological subregions of heterogeneity, with a wide difference of aggressiveness & medical history. The MRI images of BRAT dataset is used for model training and model validation for performance analysis. These datasets Multi-Modal MR images details can be seen in the Figure 5, which are different from Computed Tomography (CT) images containing the dimensions of 240 240 150 and also were obtained clinically, utilising diverse strength of magnetic field, scanners, & varied protocols from several institutes. The MRI FLAIR series highlights the tumor region, whereas the cerebral spinal fluid, fat layer are not visible in this contrast level. The T2 weight series is consider to be the most appropriate and feasible for tumor analysis, the voxel contrast intensity of the tumor, cerebral spinal fluid and fat in the head region are almost the same, which makes it more challenging to detect and segment using an automatic diagnosis system. Other than FLAIR and T2 W MRI images, the T1 W and T1 Contrast series is also studied for tumor confirmation and treatment plans.

4.png

Figure 4. (a) Load Input Image (b) Kapur entropy-based image (c) segmentation of lesion (d) overlaying segmented tumor region over the original images

5.png

Figure 5. Overview of the benchmark BRATS dataset used for models validation

This dataset contains 75 LGG cases & 210 HGG cases that were divided randomly into the training data (i.e. 80%), validation data (ten percent), and test data (ten percent). In addition, neuroradiologists labelled images with tumor labels (necrosis is represented by 1, edema is represented by 2, non-enhancing tumor is represented by 3, and enhancing tumor is represented by 4). Also, a tissue with a value of zero is considered normal). The third label isn't used.

The proposed structure's experimental results were obtained using MATLAB 2020b on an Intel Core I7 9^th generation CPU, with a 48 GB of RAM and a total of 15 L3 cache. To speed up the training processing by incorporating parallel computing CUDA framework was used with RTX 2070 super 8 GB NVIDIA GPU, the MATLAB was running on windows 10 64-bit operating system. For the training stage, we used Adaptive Moment Estimation (Adam) with 2 batch sizes, 10^-5 weight decay and an initial rate of learning of 10^-4. We spent a total of 13 hours training and 7 seconds testing each volume.

3.7 Evaluation measure

Metrics for the enhancing core, tumor core (TC, which includes necrotic core & non-enhancing core), and total tumor are used to evaluate the approach's success (WT, which includes all the classes/types of tumor structures). In case of image segmentation, the Dice Similarity Co-efficient (DSC) is used as an evaluation metric that help to compute the number of overlapped pixel between the actual pixel (ground-truth) and the pixel predicted by the model. Three criteria were used to generate the experimental results: HAUSDORFF99, the Dice similarity, and models achieved Sensitivity. Hausdorff score is a measure to calculate the distance b/w a predicted region and ground-truth region. The overlap between the ground truths and the predictions is computed using a dice score as the evaluation metric. The measure of correctly determined non-tumor pixels is called specificity (real negative rate). Sensitivity refers to the number of tumor pixels that were estimated accurately. These 3 criteria are as follows:

$\operatorname{DICE}\left(R_{p}, R_{a}\right)=2 * \frac{R_{p} \cap R_{a}}{R_{p}+R_{a}}$       (2)

Sensitivity $=\frac{\left(R_{p} \cap R_{a}\right)}{R_{a}}$      (3)

where, R_p demonstrated the predicted tumor regions, R_a demonstrated the actual labels and Rn demonstrated the actual non-tumor labels.

A new brain tumor segmentation architecture is proposed in this manuscript, that takes advantage of the 4 MRI modalities' characterisation. It indicates that each and every modality contain its own set of properties that helps the network discern between the classes more effectively. We have shown that a CNN model may achieve performance comparable to the human observer by focusing just on brain image portion near the tumor tissue. Furthermore, an effective but simple cascade CNN models have been proposed for extracting features using state-of-the-art inceptionv3 315 layers models. After applying a robust preprocessing technique to enhance the quality of the MRI images, the MRI images along with its tumor ground truth are feeded in the inception+YOLO model for training purpose. As a result of YOLO model, the tumor detection is detected and localized with higher accuracy and improved efficiency, the computing time and capacity to make fast segmentation of tumor region in a clinical MRI image is reduced. When comparing the state-of-the-art alternatives, extensive experiments have demonstrated the usefulness of our proposed Attention mechanisms in our algorithm, in addition to the extraordinary capacity of the overall model.

Despite the proposed approach's superior outcomes when compared with the other recently published models, there are limits in our algorithms when dealing with tumor volumes more than 1/3^rd of the total brain volume. This is due to an increase in the tumor's predicted region size, which causes the feature extraction performance to suffer.