Classification of Thyroid Nodules Using Follow the Regularized Leader Optimization Based Deep Neural Networks

One of the most common endocrine tumours in the human body today is thyroid cancer. Up to 68% of symptomatic individuals in the general population have thyroid nodules. 7-15% of thyroid nodules are affected by thyroid cancer, which is the disease with the highest rate of growth across all populations [1]. The thyroid gland, an endocrine gland, secretes thyroid hormones. These hormones largely control protein synthesis and metabolic rate. They also have a number of other effects, particularly those that are connected to the growth of the human body [2]. Their prevalence is very important to reduce the risks its depends on identification method used. A CAD technology that is based on image properties is the ultrasound approach to image categorization. utilising Deep Learning (DL) and cloud technologies to create an intelligent diagnosis model based on image classification [3]. The classification challenge for thyroid nodule CAD can be stated as a machine learning problem where the model automatically identifies between benign and malignant nodules on ultrasound images [4]. By decreasing effort and diagnostic variances across doctors, ultrasound can offer a trustworthy, consistent, highly effective, and repeatable ultrasound diagnosis pathway for thyroid nodules [5]. Recently, a number of researchers have conducted extensive research in the field of CAD to reduce the impact of subjective factors on physician diagnosis and improve diagnostic precision. The attributes are used to create a validation model, which is then examined to determine the applicability of each attribute selected for categorization [6, 7].

Techniques for thyroid nodule testing have significantly improved during the past 20 years. Ultrasound (US) imaging is frequently used as the primary point of diagnosis and for the evaluation of thyroid nodules because it successfully pictures and visualises soft tissue features [8]. It is also the most widely used and economical imaging technique, free of ionising radiation. The structure and characteristics of nodules can be learned through a thyroid ultrasound scan [9, 10]. As a result, thyroid issues such echogenicity, texture, a lack of microcalcification, a halo, and intra-nodular blood flow can be more easily found. Numerous machine learning-based techniques have been proposed [11] for autonomously identifying and assessing thyroid nodules in ultrasound images. These methods are separated into two groups: radionics and DL-based techniques. A radionics-based technique was introduced to extract quantitatively high throughput image features from medical images, such as the Gray-Level Co-occurrence Matrix (GLCM), Histograms of Oriented Gradients, and Grey level run-length matrix [12, 13]. Recently, computer-aided diagnosis systems have performed as well as or better than radiologists at making diagnoses from medical images as a result of the advancement of DL, notably Convolutional Neural Network (CNN) [14]. The complex nodule detection model utilising DL requires a hyper parameter adjusting technique to improve system performance. It is occasionally required to tune the hyperparameters to make the smart search algorithm faster than random search. The suggested model used an optimisation strategy for hyper parameter adjustment to address this issue [15]. The current techniques are unable to classify thyroid nodules accurately. Additionally, the approaches used to choose useful criteria for thyroid nodule classification are ineffective. The suggested Regularised Learning based Deep Neural Network (FTRL-DNN) approach aims to address the shortcomings of existing systems. The contribution of the research are as follows:

• The Regularised Learning based Deep Neural Network (FTRL-DNN) algorithm is suggested for a successful categorization of thyroid nodules.
• Because ResNet-18 has so many hidden layers, AlexNet and ResNet-18 are used to successfully extract the features from the photos.
• The best features are chosen from the feature set using the RREMI algorithm-based feature selection.

The remaining of the paper is mentioned as follows: Section 2 provides the existing work information about thyroid nodule classification. The clear details about the FTRL-DNN based thyroid nodule classification is given in section 3. Section 4 provides the outcome of the FTRL-DNN whereas the conclusion is presented in section 5.

The block diagram of the proposed research was shown in Figure 1. Database, Feature Extraction, Feature Selection, Hyper Parameter Optimization and Classification using LSTM classifier are described below.

1.png

Figure 1. Block diagram of the proposed work

3.1 Dataset collection

The Digital Database of Thyroid Images (DDTI) dataset, provided by Columbia National University, is used in this study to classify thyroid nodules [20]. This collection includes 480 US pictures from 400 thyroid illness patients. After ignoring the 480 faulty images, 427 US images are all that are left. Based on the placement of thyroid nodules in each class of the different (TI-RADS) standard, the thyroid pictures in this dataset are divided into six classes: 2, 3, 4A, 4B, 4C, and 5. According to the TI-RADS recognition concept, the classes "2, 3" are designated as benign nodules and "4A, 4B, 4C, 5" are designated as malignant nodules. Then, the sample images of benign and malignant nodules are shown in the Figure 2 (a) and Figure 2 (b) respectively.

2.png

Figure 2. Sample images: (a) Benign nodules; (b) Malignant nodules

3.2 Feature extraction

After gathering the data, AlexNet and ResNet-18 were used to effectively extract the features. Eight successive layers make up the AlexNet model, the first five of which are convolutional layers and the final three of which are fully connected layers. The greater number of hidden layers included in the network allows the AlexNet to extract features effectively. The bias layer of the ResNet-18 CNN is a convolutional layer, and it passes the input picture through a convolutional filter to produce a characteristic map. The kernel designates the 55 shaped matrix that had to be transformed into the input pattern matrix. The convolutional layer's output is shown in Eq. (1).

$x_j^l=f\left(\sum_{a=1}^N W_j^{l-1} * Y_a^{l-1}+b_j^l\right)$                     (1)

where, the feature map $j$ at $l$ th layer is represented as $x_j^l$; the activation function is represented as $f()$; total amount of features on the layer $l-1$ is denoted as $N$; the kernel on the layer $l-1$ is $W_j^{l-1}$; the feature map at $l-1$ layer is denoted as $Y_a^{l-1}$; the convolution mode is denoted as (*) and the bias value of the convolutional layer is denoted as $b_j^l$. The second layer after the convolution layer is pooling layer which used to perform the down-sample to input feature data. Moreover, the data's space size and parameter quantity for model's each layer are decreased by accomplishing the pooling operation. The maximum value of the neural unit is taken by max-pooling layer at certain field, therefore the new feature from the pooling kernel is expressed in Eq. (2).

$x n_j^l=\beta_i^l \operatorname{down}\left(x_j^{l-1}\right)+b_j^l$                      (2)

where, the $j$ th multiplication of $l$ th layer is denoted as $\beta_j^l$ and the pooling function is represented as down(.).

The crucial CNN layer is a fully interconnected layer that functions as a multilayered perception. Since it was designed to address the gradient vanishing problem, this connected layer uses the Rectified Linear Unit (ReLU) described in Eq. (3) as an activation function.

$\operatorname{ReLU}(x)= \begin{cases}0, & x<0 \\ x, & x \geq 0\end{cases}$                        (3)

3.3 Feature selection

In order to choose the best features from the feature set $(\mathrm{xn})$, the input feature extraction is sent to the feature selection process using the RREMI algorithm. When utilising the traditional Relief approach, the feature is solely selected based on its weight, which is calculated using the closest target. Here, just the 0 s and 1 s (i.e., weight values) are taken into account when choosing the features, which causes even the features with the greatest solutions to be overlooked. In order to lessen the redundancy of the features, the Renyi entropybased mutual information is combined with the weight values of traditional Relief. Consider the following: the subset has N_s features, the feature subspace is indicated by the letter s, and the dataset contains $M$ samples with labels. Hence, the sample data $m(1 \leq m \leq M)$ with labels in the dataset is expressed as vectors $x n^{(m)}=\left(x n_1^{(m)}, \ldots, x n_N^{(m)}\right)$ in the space which is generated by $N$ amount of features from complete feature set. Moreover, the $x n_s^{(m)}$ denotes the certain sample data $m$ with the labels in the dataset at the $N_s$ dimensional feature subspace.

Eq. (4) shows the computed distance between the two points in the subspace $S$ chosen by the feature subsets. Here, the normalization method is used to normalize the results.

$\begin{aligned} & d\left(x n_s^{\left(m_1\right)}, x n_s^{\left(m_2\right)}\right) =d_M\left(x n_s^{\left(m_1\right)}, x n_s^{\left(m_2\right)}\right) \exp \left(-\frac{d_M\left(x n_s^{\left(m_1\right)}, x n_s^{\left(m_2\right)}\right)}{d i s t_{\max }}\right)\end{aligned}$                             (4)

where, $1 \leq m 1 ; m 2 \leq M$; the Manhattan distance between two different vectors is denoted as $d_M$ and the $dist_{\max }$ is expressed in the following Eq. (5).

$d i s t_{\max }=\max d\left(x n_s^{(m)}, x n_s^{(k)}\right)  \\m \neq k, m, k \in\{1, \ldots, M\}$                       (5)

3.4 Hyper parameter optimization using FTRL-DNN

After feature selection, we used FTRL-DNN to classify data utilising hyper parameter optimisation to fine-tune the hyper parameters. The assigned records and variable data were utilised to forecast the solar array output power using the FTLR-DNN algorithm. The learning rate value is one of the additional factors that must be given for the optimisation process. By testing a set of parameters first and then adding new ones to check if the optimisation approach performs better, the FTRL-DNN method of obtaining the hyperparameters is manual. Traditional manual tuning takes a long time, and it needs a domain expert to speed things up. To apply the FTRL-DNN algorithm in the predication process, first some notations was established. $\left(g_t, y_t\right)$ is the instances in the recorded sample data. It represents input vector of the prediction model and there are $d$ entries including irradiance profile and the switch state with ith entry denoted by $g_t$. Besides, $y_t$ describes the photovoltaic array of output power in the instance which is expected by the prediction model. The prediction of $y_t$ is described by $h_t(w)=\sum w_t x_t$. Moreover, the function is defined in Eq. (6),

$J_t(w)=\frac{1}{2}\left(b_t(w)-y_t\right)^2$                         (6)

To decide the weights w, the FTRL-DNN was used. Besides, to fasten the calculation, L1 and L2 regularization are added. The FTRL-DNN based regression prediction model is then operated with two parts, that is prediction process and model updating was showed in Eq. (7).

$w_{t, i}=\left\{\begin{array}{c}0|z| \leq \lambda_t, \\ -\left(\frac{\beta+\sqrt{n_i}}{\alpha}+\lambda_2\right)\left(z_i-\operatorname{sgn}\left(z_i\right) \lambda_1\right) \text { otherwise },\end{array}\right.$            (7)

where,

$\alpha \rightarrow$ learning rate

$\beta \rightarrow$ Smoothing parameter for adaptive learning rate $\lambda_1, \lambda_2 \rightarrow L_1, L_2$, regularization

$Z_1, n_i, \rightarrow$ Initialized to be 0

3.5 Long Short Term Memory (LSTM)

The LSTM classifier is provided the feature selection to show performance when huge datasets are managed. Large datasets demand more memory to use, which raises the computational difficulty. To address the aforementioned downgrades, the developed model uses the Firefly Algorithm with LSTM-produced hyper parameters. Key factors like the number of epochs and hidden layers must be optimised in order to surpass the LSTM model. The LSTM unit computing process is described in the following points.

Here, the patient memory cell $\widetilde{c_t}$ is computed, $b_c$ is represented as bias, and $W_c$ is denoted as weight matrix, as indicated in Eq. (8).

$\widetilde{c_t}=\tanh \left(W_c \cdot\left[h_{t-1}, x_t\right]+b_c\right)$                    (8)

As illustrated in Eq. (9), the input gate $i_t$ is computed, present input data update of memory cell state value is controlled by the input gate, $b_i$ is stated as bias, $W_i$ is stated as weight matrix, and $\sigma$ is defined as sigmoid function.

$i_t=\sigma\left(W_i \cdot\left[h_{t-1}, x_t\right]+b_i\right)$                     (9)

The value of forget gate $f_t$ is computed, forget gate handles a memory cell state value depends historical data update, $b_f$ is stated as bias, $W_f$ is stated as weight matrix, as shown in Eq. (10).

$f_t=\sigma\left(W_i \cdot\left[h_{t-1}, x_t\right]+b_f\right)$                          (10)

The LSTM model is easy to update, reset, read, and store long-term data since it relies on memory cells and control gates. The internal parameter sharing mechanism of the LSTM operates the output dimensions in accordance with the setups for the weight matrix dimensions. The class probabilities are determined from the data, and the deep co-learning approach identifies labels based on phrases that were not previously detected.