Deep Learning Based Object Detection in Medical Image with YOLOv4-CSP with U-Net Algorithms

The provided diagram in Figure 1 illustrates the architectural framework of the proposed methodology. This system is designed to handle input in the form of images, from SIH DeepLesion Subset. A deep learning model with a hybrid approach that utilizes the U-Net and YOLOv4-CSP architectures for identifying objects in medical pictures. YOLOv4-CSP is the core of an effective and rapid object identification technology that uses the Cross Stage Partial (CSP) connections to enhance the scalability and representation of features. At the same time, the U-Net component is used for fine-grained semantic segmentation, thereby enabling precise localization of objects inside the identified areas. This uses a multi-task method of learning during training, where the model is learned using an assortment of regression and classification losses on both detecting and segmentation assignments simultaneously. In addition, the proposed method incorporates a local self-attention system into the model design to improve overall detection accuracy by enhancing the fusion of features and preserving contextual connections at various levels.

The sequential process involves the following key steps:

(1) Input processing

The journey commences with the intake of digital images of medical findings. These images might be any medical images with various modalities.

Figure 1. Architecture diagram of the proposed method

To optimize the input image for subsequent experimentation, this work engages in pre-processing techniques. This includes steps such as cropping, binarization using the Otsu method, and employing the CLAHE algorithm for image enhancement. These processes collectively serve to enhance image quality, eliminate salt and pepper noise, and smoothen the image, ensuring optimal conditions for object detection.

(3) Segmentation

The pre-processed image, now refined and prepared, undergoes segmentation. This is accomplished through the application of a U-NET. The primary objective here is to identify the lesions from a medical image.

(4) Object Detection using YOLOv4-CSP

The segmented medical image is fed into a YOLOv4-CSP algorithm. This sophisticated neural network is adept at object detection, leveraging its ability to learn hierarchical representations. This enables accurate object identification from a medical image.

4.1 Image pre-processing

Cropping:

Creating a cropped image entails isolating a specific rectangular Region of Interest (ROI) from an original image. Here's an alternative explanation for the process:

Procedure for Selecting and Cropping a Region of Interest (ROI) in an Image:

Step 1: Import the Original Image Begin by loading the original image into your preferred image processing software. This serves as the starting point for the cropping process.

Step 2: Define the Region of Interest (ROI) specify the ROI within the image by providing coordinates that outline a rectangular area. These coordinates determine the starting point (top-left corner) and dimensions (width and height) of the desired ROI.

Binarization using the Otsu Method:

Otsu's thresholding method is a widely used technique for automatically converting grayscale images into binary images. It operates by determining an optimal threshold that maximizes variance linking two classes of pixel intensities, effectively distinguishing foreground and background regions. Let's delve into the mathematical foundation of Otsu's thresholding algorithm. Consider a grayscale image represented by pixel intensities in the range [0, L-1], where L is the total number of possible intensity levels. The goal of Otsu's algorithm is to discover a threshold value, denoted as T, which partitions the image into two classes:

Background (B): Pixels with intensities less than T.

Foreground (F): Pixels with intensities greater than or equal to T.

The algorithm maximizes the separation between these two classes by leveraging the concept of intra-class variance and inter-class variance.

Step 1: Calculate the histogram of pixel intensities in the image, counting the frequency of each intensity value.

Step 2: Normalize the histogram so that the sum of all frequencies is 1.

Step 3: Compute the Cumulative Distribution Function (CDF) and the cumulative mean intensity up to each intensity level.

Step 4: Calculate the global mean intensity of an entire image

Step 5: For each intensity level k from 0 to L-1:

•Calculate the probabilities of background and foreground classes up to intensity level k.

•Calculate the mean intensities of the background and foreground classes up to intensity level.

•Calculate the between-class variance ($\operatorname{sigma}_b^2$) using the formula:

$\operatorname{sigma}_b^2=w b* w f *\left(\text {mean}_b-\text {mean}_f\right)^2$             (1)

where, w_b and w_f are the probabilities of the background and foreground classes, and mean_b and mean_f are the mean intensities of the background and foreground classes.

Step 6: Find the optimal threshold, The threshold that maximizes $\operatorname{sigma}_b^2$ corresponds to the optimal threshold T.

Image Enhancement using CLAHE:

Contrast-Limited Adaptive Histogram Equalization (CLAHE) is a widely used algorithm for enhancing the contrast in medical images. It's particularly effective in scenarios where traditional histogram equalization may lead to overamplification of noise. Here's a step-by-step explanation of how the CLAHE algorithm is applied to enhance medical images:

I(x,y): Original input image.

M×N: Size of the image.

L: Intensity levels (e.g., 256 for an 8-bit image).

S×S: Size of the contextual region (tile).

Step 1: Divide the Image into Non-Overlapping Tiles:

Divide the image I(x,y) into non-overlapping tiles of size S×S.

Step 2: Calculate the Histogram for Each Tile:

For each tile, calculate the histogram H_k(m,n) for intensity levels 00 to L−1.

$H_K(m, n)=\sum_{x^1=m}^{m+S-1} \times \sum_{y^1=n}^{n+S-1} I\left(x^1, y^1\right)$            (2)

Step 3: Apply Histogram Equalization to Each Tile:

For each tile, apply histogram equalization to obtain the cumulative distribution function (CDF)

$C_k(m, n)=\operatorname{cumsum}\left(H_K(m, n)\right) / S^2$               (3)

Step 4: Apply Contrast Limiting:

For each tile, limit the contrast by clipping the values of the CDFC_k(m,n) above a specified limit (clipLimit).

$\operatorname{Ck}(m, n)=\left\{\begin{array}{lr}\text {cliplimit if} \operatorname{Ck}(m, n)>\text {cliplimit} \\ \operatorname{Ck}(m, n)\quad \quad\quad\text {otherwise}\end{array}\right.$             (4)

Step 5: Interpolate Overlapping Tiles:

Interpolate the contrast-limited histograms of overlapping tiles.

Step 6: Map Original Intensities to Enhanced Intensities:

For each pixel (x,y) in the image, find the corresponding histogram equalized intensity from the interpolated histograms.

$I_{\text {enhanced }}(x, y)=C_k(m, n) \cdot(I(x, y)+1)$             (5)

It enhances the contrast in local regions of an image while avoiding the overamplification of noise by limiting the contrast in each tile.

Figure 2 shows the original input image taken for pre-processing and Figure 3 shows the output after applying pre-processing steps cropping and Figure 4 shows the image after applying Binarization, Figure 5 shows the output image after applying median and Gaussian filtering process.

3.2.png

Figure 2. Original input image

3.3.png

Figure 3. Pre-processed image (cropping)

3.4.png

Figure 4. Pre-processed image (binarization)

3.5.png

Figure 5. Pre-processing (CLAHE)

4.2 Image segmentation using U-Net algorithm

The U-Net algorithm, a convolutional neural network (CNN) architecture, is widely employed for image segmentation tasks, particularly in the domain of medical image analysis. Originally crafted for biomedical image segmentation, its primary objective is to discern and categorize distinct structures within an image, such as organs or tumors. Below is a detailed mathematical walkthrough of the U-Net algorithm:

The U-Net architecture is composed of two fundamental components: the contracting path (encoder) and the expansive path (decoder).

1. Contracting Path (Encoder):

Convolutional Block:

Let X be the input image.

Convolution: Z₁=Conv2D(X, W₁)+b₁, where W₁ is the filter, and b₁ is the bias.

Activation: A₁=ReLU(Z₁).

Down-sampling (Max-pooling):

P₁=MaxPooling2D(A₁).

Repeat Convolutional Block and Down-sampling:

Z₂=Conv2D(P₁, W₂)+b₂, A₂=ReLU(Z₂).

P₂=MaxPooling2D(A₂).

Repeat this process to create a contracting path with multiple convolutional blocks and down-sampling layers.

3.6.png

Figure 6. (a) Original image (b) to (j) Segmentation using U-Net algorithm

2. Bottleneck:

Central Convolutional Block:

Z_n=Conv2D(P_n, W_n)+b_n, A_n=ReLU(Z_n).

3. Expansive Path (Decoder):

Up-sampling (Transposed Convolution):

U₁=Conv2DTranspose(A_n, W_up)+b_up, where W_up is the transposed convolution filter, and b_up is the bias.

Concatenation and Convolutional Block:

Concatenate feature maps from the corresponding contracting path:

C₁=Concatenate(U₁,A₂).

Z_n+1=Conv2D(C₁,W_n+1)+b_n+1, A_n+1=ReLU(Z_n+1).

Repeat Up-sampling, Concatenation, and Convolutional Block:

Repeat this process to create an expansive path with multiple up-sampling, concatenation, and convolutional block layers.

4. Output Layer:

Convolutional Layer with Sigmoid Activation:

Z_output=Conv2D(A_{expansive_path},W_output)+b_output.

Apply a sigmoid activation function to obtain the final output:

Y_output=Sigmoid(Z_output).

Figure 6(a) shows the original image and Figures 6(b)-6(j) shows the output of the u-net segmentation algorithm which segments all objects from an input image that will be given as input to the next stage called object detection. Apart from that Figure 7 shows the output of contour-based segmentation which will help the experts to identify all the objects in a single image.

3.7.png

Figure 7. (a) Original image (b) Contour-based segmentation using U-Net algorithm (c) Binarized image of contour-based (d) Inverted image of contour-based for object identification analysis

Figure 7(a) shows the original image Figure 7(b) shows the contour-based segmentation using the U-Net algorithm, and Figures 7(c) and (d) show the binarized and inverted binarized image of the contour-based segmented image which will be used to identify the objects in the medical images accurately.

4.3 Object detection using YOLOv4-CSP algorithm

Renowned for its real-time object detection prowess, YOLO (You Only Look Once) excels by partitioning the input image into a grid and making predictions for bounding boxes and class probabilities within each grid cell. This unique approach allows for swift and accurate identification of objects in a single pass, distinguishing YOLO as a leading algorithm in real-time computer vision applications.

CSPNet (Cross Stage Partial Network) is a network module that aims to improve the information flow across different stages of a neural network. YOLOv4-CSP incorporates CSPNet into the YOLOv4 architecture to enhance its performance.

Step 1: Input Image:

Let I be the input image with dimensions H×W×C, where H is the height, W is the width, and C is the number of channels.

Step 2: Backbone Network (CSPDarknet53):

-The backbone network processes the input image and produces feature maps at different scales.

-Let F_backbone represent the feature maps generated by the backbone.

Step 3: Feature Pyramid Network (FPN):

-FPN processes the feature maps from the backbone to create a pyramid of features at different scales.

-Let F_FPN represent the feature maps from the FPN.

Step 4: Head Network:

-The head network processes the feature maps from FPN to make predictions.

-For each grid cell in the feature map, the head predicts bounding box coordinates (x,y,w,h), confidence scores, and class probabilities.

-Let B represent the number of bounding boxes predicted per grid cell.

-Predictions for each grid cell:

P=(x,y,w,h,confidence,class1,class2,...,classN)

where, N is the number of classes.

Step 5: Anchor Boxes:

-YOLOv4-CSP uses anchor boxes to improve bounding box prediction accuracy.

-Let A represent the number of anchor boxes. Each anchor box has a width and height.

Step 6: Loss Function:

The loss function is defined as a combination of localization loss, confidence loss, and class loss.

Localization Loss:

$\begin{gathered}L_{\text {conf }}=y_{\text {coord }} \sum_{i=0}^B 1_{o b j_i} {\left[\left(a_i-\hat{a}_i\right)^2+\left(b_i-\hat{b}_i\right)^2+\left(\sqrt{w}_i-\widehat{\sqrt{w}}_i\right)^2\right.} \left.+\left(\sqrt{h}_i-\widehat{\sqrt{h}}_i\right)^2\right]\end{gathered}$              (6)

Confidence Loss:

$\begin{gathered}L_{\text {conf }}=y_{\text {coord }} \sum_{i=0}^B 1_{o b j_i}\left[\left(c_i-\hat{c}_i\right)^2+\left(\mu_{\text {noob } j}-\hat{\mu}_{\text {inoob } j i}\right)^2\right. \left.+\left(c_i-\hat{c}_i\right)^2\right]\end{gathered}$             (7)

Class Loss:

$\begin{aligned} & \text { Lclass }=\lambda \text { class } \sum_{i=0}^B 1_{o b j_i} \sum_{c=1}^N 1_{o b j_i}\left(p_i(c)-\hat{p}_i(c)\right)^2\end{aligned}$            (8)

Total Loss:

$L=L_{\text {lacin }}+L_{\text {counf }}+L_{\text {class }}$            (9)

where, $o b j_i$ is an indicator function that is 1 if object i is present in the grid cell and 0 otherwise.

Step 7: Non-Maximum Suppression (NMS):

Following the generation of predictions, a crucial refinement step is employed through non-maximum suppression, strategically implemented to discard redundant bounding boxes. This post-processing technique ensures the final selection of the most accurate and relevant bounding boxes, enhancing the precision and efficiency of the object detection results.

Step 8: Post-processing:

-Bounding box coordinates are converted from the grid cell space to the image space.

-Predictions below a certain confidence threshold are filtered out.

Figure 8 shows the output after applying the YOLOv4-CSP object detection algorithm. The output image clearly shows that the proposed algorithm after the U-Net segmentation process, identifies all the objects in medical images. This will help the medical experts and also patients to identify the parts. If measurements are taken then if any lesions are there that also will be identified very easily by the medical experts.