A Novel Saliency Region Enhanced Technique for Biomedical Image Indexing Using Deep Learning

The case-based analysis is a common practice for medicine experts. Finding the information of previous patients helps the doctor with additional support for better understanding of critical cases and proper diagnosis [1]. Content based medical image retrieval (CBMIR) system used to retrieve the images of similar cases from the large medical image database [2, 3]. The content refers to the visual image features derived from the textures, colors, shapes [4-7] etc. The feature matching exhibits visual similarities, which is based on similarity extracted features rather than the exact matching.  The retrieval results are then ranked accordingly to their similarity index [8-10]. Medical image analysis is cumbersome since radiological images are high-resolution images. Moreover finding the image features which are really meaningful for a radiologist is also challenging. Hence building a CBMIR system is a more tedious task than the CBIR system. Manual annotation of medical images with precise descriptions or labels is no longer possible due to its exponential growth, multimodal nature [11, 12], and lack of properly defined standards.

Traditionally handcrafted features are also not very useful due to its high resolution and complex possessions. The major challenges in medical image analysis are first most of the radiological imaging are gray-scale and multi-modal in nature, i.e. the X-rays images are used for detection of (a) bone fracture analysis (b) the chest X-rays are used for the diagnosis of COVID-19 and pneumonia [13-18] analysis (c) in case of Mammogram the X-rays image is used for breast cancer detection [19]. Likewise, the computed tomography (CT) scan image shows a detailed view of the bones, tissues and other organs inside the body [20]. The X-ray and CT scan images are also used for the Brain tumor detection [21-23]. In the case of a high resolution computed tomography (HRCT) scanning the slices are much thinner than that of a standard CT scan, so it gives a more comprehensive view of the chest. It maps the three-dimensional (3D) to a cross-sectional view in two-dimensional (2D) called “slice” [24]. Images produced by Magnetic resonance imaging (MRI) scan [25] are useful to understand the different abnormal conditions of the chest, abdomen and brain. Hence retrieving similar images from multiple modalities is more challenging. Second, the limited number of pathological information available for many critical cases.  With the availability of high-speed computing facilities using GPUs and TPUs and the evolution of Deep learning techniques, medical image analysis has become a more focused research domain. Computer-aided diagnosis (CAD) [26] models are developed to automate the diagnosis process. Deep learning is used for automatic medical image segmentation, i.e. to find the region-of-interest [27, 28] as well as for image classification using the Convolutional neural network (CNN) feature [29-34]. An efficient image retrieval system can be helpful for the automation of medical data management in hospitals. It will assist the physician for faster decision making by reviewing the previously available clinical cases [35-38]. The service can be extended to the remote health centers to support the newly appointed doctors as a cloud based service.

In this work our objective is to develop a more effective CBMIR system using CNN feature of the enhanced medical images. The CNN feature extraction uses the Max-pooling operation after each convolution operation, so the CNN feature of the enhanced image is expected to be a better image descriptor than the CNN feature of the original image. First the medical images are segmented using deep learning (U-Net) [39, 40] to find the region-of-attention i.e. the target region of a biomedical image according to their image class. Here the RoA segmented image is referred as mask image. Then the segmented images are enhanced. These enhanced segmented images are fused with its original image to produce a complete enhanced image. The proposed CBMIR system for the biomedical image analysis and retrieval is developed and tested using two publicly available biomedical image datasets i.e. Brain tumor dataset, it consists of four classes and Covid-19 dataset which consists of three classes. The classification and retrieval performances of both the datasets are presented in the result section.

The major achievements of the proposed method are:

·The deep learning technique is applied to extract the RoA as per the class of the radiological image.

·The RoA image is enhancement is done using Contrast-limited adaptive histogram equalization (CLAHE).

·The enhanced RoA image is fused with the input image to generate a complete enhanced image.

·CNN features of the fused enhanced image are used as the image descriptor.

·A CBIR model is developed and tested with two publicly available biomedical image dataset i.e. Brain tumor and COVID19 dataset.

·The performance retrieval performance of the original image and enhanced image datasets are compared.

The rest part of this paper is arranged as per the following order: Section 2 shows a review of medical image analysis and the CBIR system. Section 3 presents the proposed RoA segmentation technique using U-Net, the image enhancement and the CNN feature extraction. Section 4 shows the detailed classification and retrieval results. Section 5 presents the conclusion and future scope of the work.

In this work, a novel enhanced saliency region based CBIR model is proposed for the medical image retrieval. Here the CNN features are used as the image descriptor, which is used for finding the similarity indexes between the query image and the images in the database. Most of the medical images are high-resolution images and the doctor’s attention goes to only the region of ailment i.e. the RoA region. Many cases these RoA regions are less prominent in the original image. The factual features available in the less prominent RoA region of an image are suppressed by the MAX-pool operation during CNN feature extraction as the MAX-pool is applied recursively with each convolution operation [39]. The objective of this work is to enhance the RoA before feature extraction to derive a better image descriptor as the image class. Figure 1 shows the U-Net architecture used for segmentation of the RoA i.e. to generate the mask image as per their class information using supervised learning technique [40].

tu_pian_1.png

Figure 1. U-Net architecture for masked-image generation

Figure 2 shows the model for the proposed CBIR system, here medical images are segmented using deep-learning.  Where the segmented masks are enhanced using CLAHE and fused with the original image to produce final enhanced image, and are used for the CNN feature extraction [37]. The Table 2 shows the saliency region enhanced image with the original image and the mask-image of the Brain tumor and COVID19 dataset [36, 38]. The City-block distance measure is used for finding the similarity index between the query image and the images in the database. The performance of the proposed model are presented using precision, recall, f1-score, ROC analysis and retrieval rate for both the above mentioned datasets.

Table 2. Saliency region enhanced images

2.png

222.png

Figure 2. Proposed SRE-CBIR model

3.1 CNN feature extraction

The proposed model illustrated in Figure 1 shows the CNN features of the enhanced medical images are generated. Here the gray-scale image is resized to 512 x 512, which is given to the CNN model for training. The proposed CNN model contains of seven layers, here each layer consists a convolution, which is followed by the ReLU and then a MAX pool operation. The activation function  ReLU introduces non-linearity property to the convolution output. The image size is reduced by the MAX pooling operation with each convolution layer. The image is reduced to a 1D feature vector of 1 x 1024 by the flattened layer. Four fully connected layers are used for additional dimensional reduction. Soft-max operation converts the feature map to class label using the energy function [6].

$(I * f)_{x, y}=\sum_{s=1}^H \sum_{t=1}^W f_{s, t} \cdot I_{x+s-1, y+t-1}+b$    (1)

$\operatorname{ReLU}(x)=\operatorname{Max}(0, x)$   (2)

$E(w, b)=\frac{1}{n} \sum_{i=1}^n L\left(y_i, f\left(x_i\right)\right)+\alpha R(w)$      (3)

where, L: The loss value of the model; R: The regularization factor used to deal model complexity; α: strength control value of the regularization value.

The precision value represents the number of images rightly recognized by the system with respect to the number of image recognized by the system. Whereas the recall value indicates the number of images correctly recognized to a given class with respect to number of images available in the database for the same class. So the value of average recall rate is a more significant measuring parameter in case of retrieval analysis. The harmonic mean of all the classes is presented using f1-score.

$S_i(x)=\frac{1}{1+e^{-x}}$        (4)

$\operatorname{LogLoss}=-\frac{1}{N} \sum_{i=1}^N \sum_{j=1}^M y_{i j} \log P_{i j}$    (5)

The convolution operation at a point a (x, y) is represented in equation (1). Here the I represent a gray-image and the f is the operational filter, here H is the height and W is the width of the image. The ReLU operation is defined in equation (2). The Eq. (3) represents regularized training error of an instance. The S_i is the sigmoid function that maps the output value within (0, 1) is defined in Eq. (4). The penalty value for each iteration is  represented using Eq. (5) i.e. cross-entropy loss using the energy function [6].

3.2 Performance measures

The efficacy  of the proposed SRE-CBIR model is measured using quantifier parameters such as precision, f1-score, and recall these are defined using Eq. (6), Eq. (7), and Eq. (8) respectively [26].

Precision $=\frac{\text { True }+ \text { ve }}{\text { True }+ \text { ve }+ \text { False }+ \text { ve }}$     (6)

Recall $=\frac{\text { True }+ \text { ve }}{\text { True }- \text { ve }+ \text { False }- \text { ve }}$    (7)

$f 1-$ Score $=2 \times \frac{\text { Precision } \times \text { Recall }}{\text { Precision }+ \text { Recall }}$  (8)

here, true +ve value represents the total number images correctly recognized by the system to their belonging class. The false -ve value indicates number of images wrongly identified by the system for a particular class, and the false -ve value shows number of images rejected by the system falsely.

Receiver operating characteristics (ROC) is the curve shows the stability of the model, it is plotted using the values of true+ rate vs false- rate [26]. The City-block distance used to measure the similarity index value between the query image and the database image is shown in the Eq. (9).

City-block distance measure:

$D_{C T}=\sum_{i=0}^{L-1}\left|F_i^q-F_i^t\right|$        (9)

where, $F_i^q$ : feature vector of the query image; $F_i^t$ : feature vector of the database image.