Multiclass Classification of Cervical Pap Smear Images Using Deep Learning-Based Model

The fourth most common cancer among women is cervical cancer, which develops in the woman's cervix [1]. The World Health Organization (WHO) reports that 311 000 women died from cervical cancer in 2018 alone, with an estimated 570 000 women worldwide receiving a diagnosis [2]. Due to a lack of screening and treatment facilities, developed and underdeveloped countries account for more than 80% of instances of cervical cancer and 85% of fatalities from the disease [3]. The main risk factors for human papillomavirus (HPV) infection include smoking, early pregnancy, poor menstrual hygiene, and the use of oral preventatives [4]. The primary cause of cervical cancer is a long-term HPV infection, according to research [5]. However, if diagnosed early and given the proper care, the most treatable malignancy is cervical cancer [6].

Over 30 years old, cervical cancer diagnoses are made possible by women's routine screening, which is essential to preventing the disease [7]. Cervical malignancies are identified using cervical cytopathology (pap smear test); this screening technique is the most popular due to its affordability [8]. The malignancy is examined by extracting the cells from the cervix squamocolumnar terminal using a light microscope [9]. A single slide typically requires 5 to 10 minutes to analyze due to the cells' varying orientations and overlaps [10]. Each slide comprises over three million cells with varied directions and overlaps, making the human screening process laborious, repetitive, expensive, and prone to errors. As a result, a developed system that can accurately and effectively evaluate the pap cell is being developed [11].

The computer-aided diagnostic (CAD) system developed to assist medical professionals in monitoring cervical cancer has attracted much interest [12]. Cell segmentation (cytoplasm and nuclei), feature extraction, and classification are the three phases of the typical CAD system [13]. To improve the quality of the images, the system uses filtering-based pre-processing operations [14]. Then, cell nuclei are extracted. After that, the segmented nucleus is corrected through post-processing [15]. The segmented nucleus is then used to extract handcrafted features like texture, color metric, and morphological characteristics [16]. The most discriminating features are then determined using the feature selection technique, and lastly, the cell classification is performed using a network model [17].

Processing the data using the technique mentioned above involves several steps, and the extracted handcrafted features cannot guarantee superior classification performance. This also demonstrates the limitation of automated learning [18]. Deep learning-based feature extraction techniques outperform other machine learning (ML) algorithms to obtain an improved CAD system. The existing model's outcomes on complex tumor detection problems are being attained by DL-based algorithms [19]. One drawback of deep learning is that, in comparison to ML techniques, it requires more data to provide effective results, which is challenging to acquire in the medical field. Additionally, deep learning could perform better for multiple classifications, which is quite common in the medical industry and has a disparity in sample data distribution. Therefore, additional research and development are needed to analyze pap cells [20].

To effectively classify the cervical cytopathology cell, we have developed the Attention-based nested U-Net (Anu-Net) model based on deep learning in this research. To test the proposed model, the SIPAKMED dataset is used. The proposed methods outperformed the existing models.

The main contributions of the research are

• First, we performed image pre-processing to enhance the performance of classification and segmentation. Image enhancement and noise removal processes are performed in the image pre-processing stage. The gaussian filter removes noise in the image, and CLAHE (contrast limited adaptive histogram equalization) is used to enhance the image contrast.
• Following Nuclei and cytoplasm of the image are segmented. A shape-based iterative technique is used to segment nuclei, and the marker-controlled watershed technique is used to segment overlapping cytoplasm.
• After that, the significant relative features of the disease are then extracted. The optimum features are selected using simulated annealing integrated with a wrapper filter.
• Finally, the multiple cervical cancer classifications are classified using deep learning-based Anu-Net. The more effective performance is attained by the proposed model using the SIPAKMED dataset. It demonstrates the possibility of improving cervical cancer detection methods.

The rest of this paper has been divided into the following sections: The most current studies employing deep learning approaches for detecting and forecasting cervical cancer are discussed in Section 2. Section 3 provides the proposed methodology, which describes the overall workflow of the research. Afterward, Section 4 discusses the result and analysis. Finally, Section 5 provides the conclusion of the study.

The proposed deep learning-based cervical cancer classification model is shown in Figure 1. The proposed model's main steps are acquiring the image, image pre-processing, segmenting cell components, and image classification. The SIPaKMeD dataset is utilized during the image acquisition stage for multi-cells. The image quality is improved by enhancing and denoising the input pap smear images as an image pre-processing stage. Cell segmentation is the next stage. This step divided the input images into exciting areas, such as the nucleus and cytoplasm. The subsequent phase was feature extraction after segmentation. Distinct significant regions or features are extracted during feature extraction. The optimum features are selected by employing simulated annealing integrated with a wrapper filter. Classification of cervical cancer is the last step. A deep learning-based Anu-Net is used for classifying normal and abnormal cells at this stage. Each subsection has a description of each phase.

3.1 Image acquisition

SIPaKMeD recently released an accessible dataset, which is used in this research. The 966 full slide pap smear images are included in the database and manually cropped into 4049 single-cell pictures. The cells were divided into five classes according to their level of abnormalities benignity - parabasal cells (PC), superficial-intermediate cells (SIC), dyskeratotic cells (DC), koilocytotic cells (KC), and metaplastic cells (MC). The initial two are healthy, the next two are abnormal, and the final is benign. There appears to be a uniform image distribution among the single-cell image classes (831, 787, 825, 793, and 813, respectively). Table 1 includes a description of the dataset's details.

3.2 Image enhancement

Noises or other artifacts may be present in the pap smear images. Images from pap smears may have worse quality due to noise or contrast. Deleting noises and enhancing the image's quality in cell contrast is necessary. The noise is removed by using a Gaussian filter. The cell contrast is improved by using CLAHE. The input cell images, grayscale images, and pre-processed images for all classes are shown in Figure 2.

Table 1. Sipakmed (multi-cells) dataset five-class cell descriptions

1.png

Figure 1. Schematic structure of the proposed methodology

2.png

Figure 2. Pre-processing results for classification and segmentation

3.3 Segmentation of cytoplasm and nuclei

From each segmented cell, this stage involved the segmentation of the nuclei and cytoplasm that had emerged from the segmentation of overlapping cells. The segmentation stage of overlapping cells produced three different types of segmented cells. The cells were isolated, overlapping, and touching. Three separate sub-processes were established for the segmentation of the nucleus and cytoplasm. Segmenting the cell components from isolated cells is the first process. Segmenting the cell components from touching cells is the second process. Segmenting the cell components from overlapping cells is the last process. The results of the watershed transform technique for segmented cells could be used to extract the cytoplasm boundary of overlapping, touching, and isolated cells. It was suggested during the segmentation process for the overlapping cells. The image obtained during the segmentation did not contain enough touching patches and overlapping cytoplasm to depict the cytoplasm boundaries accurately. Therefore, we applied an edge-smoothing technique to smooth the cytoplasm boundaries.

Edges smoothing method's processing steps

• Grayscale image reading and binary image conversion
• Extracting only the most enormous blob
• Crop the top and left of the frame
• Fill holes
• Blurring the image
• Threshold again
• Display the smoothed binary image using the smoothing method

The shape-based iteration technique described below was used to segment the nuclei of overlapping, touching, and isolated cells.

Shape-based nuclei detection algorithm's processing steps

• Input color image reading and convert a grayscale image
• Using a Gaussian filter to remove noise
• Predefining the values of solidity, minimum and maximum intensity values, major and minor axis lengths, and minimum area
• Image binarizing using the highest and lowest thresholds.
• Removing the limited intensity and shape value regions
• Segmenting nuclei

Segmenting the nuclei was simpler than segmenting the cytoplasm. The nuclei in the image of multiple cells have effective qualities, such as the shapes are usually circular or oval and well-structured and low intensity. In the other regions, like cytoplasm or background, the nuclei are considerably different. The most significant characteristics of nuclei used for differentiating the background and cytoplasm are solidity, major and minor axis lengths, average intensity, and area. The nuclei are segmented and detected by developing the method based on these characteristics.

The original Pap smear images are processed with a Gaussian filter to eliminate the noise. The nuclei segmentation process uses shape-based features (solidity, minor axis length, significant axis length, intensity values, and area). The experiments showed that the smallest nucleus size was 362 µm². The solidity values were less than or equal to 0.98, and between 60 and 150 was the average intensity value. The length of the minor axis varied from 17 to 87 meters, whereas the size of the main axis for nuclei ranged from 24m to 117m.

Figures 3 and 4 demonstrate the results of the nuclei segmentation process. The effectiveness of nuclei detection using the proposed model is provided in Table 6. Tables 3-5 reveal the performance of segmentation, cytoplasm, and nuclei of isolated, overlapping, and touching cells. The segmentation for overlapping cells is tested on 4049 cytoplasm images. Isolating cells comprised 6.47% of the cytoplasm. In contrast, touching cells consisting 35.27% of the cytoplasm, and overlapping cells comprised 58.26% of the cytoplasm. Table 2 displays the overall segmented cytoplasm range and the percentage of segmented cytoplasm.

3.png

Figure 3. Detecting nuclei in Pap smear images of multiple cells

4.png

Figure 4. Detecting nuclei by cropping regions from single cells

Table 2. The percentage of cytoplasm that is well-segmented with the total number of cytoplasm

Table 3. Segmentation of cytoplasm and nuclei for isolated cells

biao_3.png

Table 4. Segmentation of cytoplasm and nuclei for touching cells

biao_4.png

Table 5. Segmentation of cytoplasm and nuclei for overlapping cells

biao_5.png

3.4 Features extraction

Feature extraction is the next stage after the cell segmentation stage. The cytoplasm and nucleus' essential features, such as color, shape, and texture, are now retrieved. The gray-level co-occurrence matrix (GLCM) algorithm extracts the disease's significant texture features. In a cervical pap smear image, the distribution of color and shape of abnormal and normal cells is very different. The morphology features are also extracted. Twenty-nine features are extracted from the nucleus due to the biological importance of the nucleus in the classification of cancer. The features include six texture features (entropy, energy, smoothness, variance, mean, and standard deviation) and three additional geometric features (eccentricity, compactness, and solidity).

3.4.1 GLCM features

The energy features of the cervical image are extracted using the GLCM feature extraction technique. One can be constructed for any single-channel image. A square matrix constitutes the GLCM. An equal range of columns and rows is present in the grayscale of the original image. In the fused cervical image, the GLCM matrix is generated by counting different orientations' grayscale intensity pixels. This work uses a 45° angle to build the GLCM matrix. From the healthy cervical image, the cancerous image is distinguished using GLCM features like correlation, entropy, energy, and contrast.

Correlation:

R calculates the relationship between adjacent pixels’ brightness. Row average represents ‘i’, and column average represents ‘j’ for GLCM.

$\operatorname{Correlation}(R)=\sum(i-\mu i)(j-\mu j) \frac{p(i, j)}{\left[\sigma_i, \sigma_j\right]}$     (1)

Contrast: The grayscale difference between two adjacent pixels is calculated by this texture feature.

Contrast $=\sum\left(|i-j|^2 \times p(i, j)\right)$      (2)

Entropy: This metric measures the proximity of the GLCM elements to the diagonal. The value ranges from 0 to 1, which is defined as,

Entropy $=-\sum p(i, j)\left[\log _2 p(i, j)\right]$     (3)

Energy: For all the GLCM components, the squares are added to determine the second angular moment.

$\operatorname{Energy}(E)=\sum p(i, j)^2$         (4)

For an image, the range of the homogeneity unit of measurement, called energy, is zero to unity.

3.4.2 Morphological features

The morphological properties are determined by cell size and shape. The morphological properties of cells are ascertained for this investigation using eight connected chain codes. The eight corresponding pixels are those that surround the eight-linked chain code. The nucleus-to-cytoplasm ratio is defined as,

$\frac{N}{C}=\frac{N u c l_{\text {area }}}{Nucl_{\text {area }}+C y t o_{\text {area }}}$        (5)

3.5 Features selection

Selecting the extracted feature subsets that produce better classification results is a feature selection technique. Some of the collected features contain noise, and the network model will not select these features for classification. Consequently, it is necessary to identify the optimum set of features by exploring all possible combinations. The algorithm's computational complexity is increased when using multiple features; the number of possible combinations explodes. The filter, wrapper, and embedding approaches are general classifications for feature selection algorithms.

Simulated annealing with a wrapper approach is combined in the proposed method. A double-strategy random forest technique is used in this research to assess the performance of the feature selection, although this method has already been proposed. The probabilistic method of "simulated annealing" can be used to approximate a function's global optimum. The technique works well to ensure that the best combination of features is chosen. The below algorithm shows the steps of the simulated annealing approach.

Begin simulated annealing

            S = Select starting point (typically around 0.2)

            I = number of iterations

            BF = all features (Backward selection)

            Actual error = Error (BF)

            For I=1 to Maximum iterations

                   I=I-1

                   Randomly select to add or remove a feature to BF

                   New error=Error (BF)

                   If(e((Actual error-New error)/S)>rand [0 . . . .1]

                            Accept and keep the new features in BF

                            Actual error=New error

                   Else

                            Keep the old features

                   End if

            End for

End

A fitness value serves as the direction for choosing the desired set. The fittest (i.e., the one that performs the best) feature subset is selected after all the simulated annealing subsets have been compared. A wrapper is utilized to acquire the fitness value search, and the classification algorithm's error was calculated using k-fold cross-validation. The retrieved traits are combined into various combinations, then created, assessed, and contrasted with other varieties. The evaluation of a set of features is then done using a prediction model, and a score is given according to the model's accuracy.

3.6 Classification using anu-net

A model's classification phase finally ends with the desired result of predicting the label. We classified the input image using the multiclass classification algorithm according to the selected features in the proposed approach. From pap smear images, cervical cancer is classified using the Anu-Net model in this research.

We develop an ANU-Net integrated network based on the Attention mechanism and Nested U-Net architecture to classify medical images. Figure 5 provides a framework for ANU-Net. The primary network framework ANU-Net uses is nested U-Net, as can be seen, and decoders and encoders are equally organized on both sides of the network. Dense skip connections are used to transfer the reliable data obtained by the encoder to the decoder of the relevant layers, allowing for the extraction of hierarchical features that are more effective.

5.png

Figure 5. Structure of ANU-Net

The two equal-scale feature maps are received by each decoder's convolution block as inputs when there are dense skip connections: (1) Using previous Attention Gate outputs coupled with a same depth skip connection, the intermediate feature maps are produced; (2) The result of the more profound block deconvolution operation is the final feature map. The decoder then reconstructs features from the bottom up after receiving and concatenating all feature maps. Earlier nested convolution block's feature maps in this layer accumulate and reach the present block because dense skip connections can fully utilize these feature maps.

The feature map is defined as follows, let the convolution block output is represented by Q_u,v, where the network's feature depth is represented by u, and the sequence of the convolution blocks along the skip connections in the uth layer is depicted by v; consequently, the retrieved feature map equation is defined as,

${{Q}_{u,v}}=\left\{ \begin{matrix}   \varphi \left[ {{Q}_{u-1,\,v}} \right]\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,,v=0  \\   \varphi \left[ \Sigma _{k=0}^{v-1}\,Ag\left( {{Q}_{u,k}} \right),\,{{U}_{P}}\left( {{Q}_{u+1,v-1\,\,}} \right) \right]\,\,\,\,\,\,\,,v>0  \\\end{matrix} \right\}$       (6)

where, the convolution block followed by concatenation merger is denoted by φ[]. The attention gate selection and mean upsampling are represented by A_g() and U_P(). Concatenating the Attention Gates' output results from a node Q_u,k=0 to Q_u,k=v-1 in the uth layer is indicated by $\sum_{k=0}^{v-1} A g\left(Q_{u, k}\right)$.

In the first layer, the process of feature maps traveling through the skip connection is detailed in Eq. (6).

As an illustration, a block Q_0,4 possesses five inputs. The first four are the previous v block outputs in this layer, and the remaining input is the second layer's upsampling feature from a block Q_1,3. ANU-Net's most significant innovations are: Dense skip connections were used in the network transfer to integrate hierarchical representations utilizing the quality that the encoder had retrieved. In the decoder path, the features retrieved at various levels are combined by implementing the Attention Gate between layered convolution blocks with a focused selection.

For cervical cancer classification, a new deep learning-based Anu-Net model is proposed in this research using Pap smear images. The nucleus and cytoplasm of each independent cell component are first segmented by the proposed model. Then a deep learning-based methodology was used to determine whether or not the cells were cancer. There are many methods have been developed in this direction in the past. However, it has not been discovered to be very accessible for accuracy.

The proposed model with selected useful features achieved a classification accuracy of 99.95% for a binary class problem, 99.98% for three class problems, and 99.74% for multiple classification problems in this paper. The overall classification performance is increased by the proposed model in classifying the abnormal cells from the normal ones. The proposed system outperforms other well-known deep learning network models in classification accuracy. The cervical cancer screening system detects malignancies early by using the proposed framework. We intend to incorporate an assessment of the risk factors for cervical cancer into the proposed model in further research.