A Meta-Learning Approach for Diabetic Retinopathy Severity Grading

According to projections made by the World Health Organization, there will be 300 million individuals living with diabetes by the year 2025. One of the most severe effects that diabetes may have on a person's eyes is a condition known as diabetic retinopathy [1]. The leading cause of visual impairment and blindness in diabetics is a condition called diabetic retinopathy (DR) [2]. According to "Prevent Blindness," the number of persons with age-related eye illnesses and visual impairment will triple in the next three decades. Diabetes mellitus destroys the retina's tiny blood vessels, which may lead to blindness if left untreated [3]. Diabetic retinopathy affects 80% of individuals with diabetes for at least ten years, making it the leading cause of new blindness in ages 25 to 74.

The two forms of diabetic retinopathy are non-proliferative diabetic retinopathy (NPDR) and proliferative diabetic retinopathy (PDR). NPDR is the first stage of diabetic retinopathy and the most common. Because blood vessels in the retina have burst, this disease causes minimal blood and other fluid to leak into the eye [4]. PDR develops when the blood vessels in the retina tighten and restrict proper blood flow. This results in decreased retinal blood volume. Many technologies were developed to detect DR automatically from the RGB fundus images. However, those approaches have limitations in estimating the severity level due to the large-scale unavailability of datasets. Over the past several years, many diagnosis tools have been developed [5] to detect DR to classify fundus images automatically. The extraction of handcrafted features is a time-consuming process. Therefore, a new technology that can learn features from the images evolved and attracted the researcher's attention. Only when DR is diagnosed early on can it be adequately treated. As a result, early detection through routine retinal screening is crucial for people with diabetes.

As a consequence of this, early diagnosis and screening of people suffering from retinopathy will be able to contribute to the prevention of vision loss. Analysis of medical images is a burgeoning field of research that is gaining favor among researchers and practitioners in the medical field [6]. Using different methods of neural networking, a large number of studies are now being carried out to identify and grade DR. A CNN method for diagnosing and categorizing DR from fundus pictures based on severity was shown to have an average accuracy of 75% after being tested on 5,000 validation images [7]. This model, however, has a high time complexity, which may have an impact on the classifier's performance. To address these challenges, this work developed a weighted stacking-based ensemble classifier (WSEC) with a meta-learning (ML) technique and multipath CNN-based feature extraction to improve diabetic retinopathy detection performance. The purpose of this research is to devise a strategy that is automated, suitable, and as cutting-edge as possible and that makes use of image processing to detect DR at an early stage and reduce retinal damage.

The remaining portion of this work is organized in the way that follows below. Section 2 is a review of similar work that has recently been published in the literature. The proposed methodology and classification strategy for diabetic retinopathy diagnosis are discussed in detail in Section 3. In Section 4, you will find a presentation of the effectiveness analysis of the suggested technique. The concluding remarks were added in Section 5.

This research suggests combining weighted stacking-based ensemble classifiers (WSEC) with meta-learning (ML) to improve diabetic retinopathy detection performance. The job may be broken down into the following four sections: preprocessing, feature extraction, meta-learner-based prediction using weighted stacking-based ensemble classifiers (WSEC), and classification module.

·Initially, the preprocessing process is done.

·The next step is feature extraction, which is carried out with the assistance of a Multipath Convolutional Neural Network (M-CNN), which is utilized for the extraction of global and local features from pictures.

·Third, the weighted stacking-based ensemble classifier (WSEC) based prediction is processed. Here, the ensemble classifiers are Support Vector Machine (SVM), Fuzzy Neural Network (FNN), and AdaBoost classifier.

·Finally, the Enhanced Recurrent Neural Network (ML-ERNN) based meta-model is developed and implemented with the purpose of improving the overall level of efficiency of the categorization during the detection of diabetic retinopathy. Figure 1 illustrates the process of the proposed methodology.

1.png

Figure 1. The overall process of the proposed methodology

3.1 Dataset description

APTOS and Kaggle (for DR detection) are the datasets used in this study. APTOS has 3,662 images, while the Kaggle dataset has 35,126. Table 2 contains a description of the dataset, and Figure 2 gives a description of class-wise categorization. The features of Diabetic Retinopathy (DR) are retrieved independently from the images in these datasets and utilized to train the Meta-Learning (ML) classifiers. The M-CNN will be used for feature extraction and a machine.

Table 2. Dataset description

Learning classifier will be used for DR multiclass classification/severity grading. M-CNN was trained using 31029 DR category fundus images from the Kaggle and APTOS datasets (not utilized for performance evaluation of the system).

2.png

Figure 2. Datasets analysis on class-wise categorization

3.2 Preprocessing

Before feature extraction on a retina fundus image, preprocessing increases the image's desirable features. The common strategies for preparing retinal pictures of the eye introduced in this work are detailed in depth. The image is enlarged in this step to help with the subsequent Procedure for detecting different stages of diabetic retinopathy.

Image Resizing:

Resizing an image alters its proportions, affecting file size and image quality. The most common reason for scaling an image is to reduce the size and complexity of large files. Two datasets, APTOS and Kaggle, are used in this research. The input image size in APTOS is (2848×4288×3) (Figure 3(a)), and the image is downsized to (224×224×3) (Figure 3(b)) after preprocessing, as shown in Figure 3.

3.png

Figure 3. The preprocessing process of images

3.3 Data augmentation

The supplied data in many classification situations is inadequate to train classifiers that have high accuracy and robustness. The use of data augmentation is a common strategy for coping with the limited amount of data available in comparison to the number of variables that may be adjusted in a classifier. Figure 4 represents the transformation and rotation of the image in augmentation for both datasets.

4.png

Figure 4. Transformation and rotation of the image in augmentation

3.4 Feature extraction

The disease is graded using popular meta-machine learning classifiers and a unique M-CNN architecture for the extraction of characteristics from images of the retinal funds in order to grade DR. The transfer learning via feature extraction method is used. Figure 5 depicts M-CNN architecture. CNN's input layer picture is 196 × 196 pixels. Two feed-forward paths are presented.

5.png

Figure 5. M-CNN architecture

First, there's mainstream CNN. Second, multipath properties are extracted. After the initial CL, both feature maps are concatenated before linking. ReLU is a superior activation function for gradient change to sigmoid and tan. The first convolutional layer is composed of an eight-kernel 5×5 convolutional operation. The way is then divided.

The feature maps from the previous layer are moved to the second route, which is responsible for carrying out 9×9 convolutional operations using 32 kernels and a max-pooling layer. Following the primary route, one arrives at a max-pooling layer, where one then loops back around to a 5×5 convolution layer.

The number of kernels that make up the network is determined by a process of trial and error.

The M-CNN architecture is designed with kernel settings that have a 0.97 percent error rate. In the max pooling process, the input is down-sampled utilizing the value that was highest among each cluster of neurons in the layer that came before it. Through the process of downsampling, the spatial resolution of the succeeding layers is reduced, but the essential local structures are maintained. After that, the feature maps go through an integration process in the weighted transform layer (1-1 without bias) before being sent to the Fully Connected Layer (FCL).

Both the first and second FCLs have hidden neuron populations of 128 and 64, respectively. The performance of the network's prediction is hindered when the softmax classifier is used. After the first fully connected layer, a dropout (the process of firing out units in a neural network) of 0.5 is applied before getting characteristics from the second fully connected layer. This is done to prevent overfitting during the machine learning classifier training process. As a consequence of this, integrating a CNN with an ML classifier has the power to significantly boost the efficiency of the classification system as a whole.

3.4.1 Procedure of feature extraction using M‑CNN

The features produced by a standard neural network may have losses in the global structures if the simple route is either extremely short or excessively lengthy. Shortcut pathways can be used to fix this. The multipath extracted features aid in the preservation of global structure losses, resulting in the M-CNN producing more meaningful global and local structures.

The second iteration is when the result characteristic matrices that may be used for categorization are gathered from FCL after concatenating the feature maps from the two routes. (1) The opportunity to deceive CNN in its evaluation of global structures while simultaneously moving features from the currently active layer to the shortcut route is one of the primary challenges encountered in multipath feature extraction. (2) The network will be prone to global noise interference if the image resolution is inadequate. Because the shortest route has a sufficient number of convolutional layers, these difficulties can be overcome. The recommended M-CNN has the highest level of performance in the shortest route for DR classification when it consists of a 9×9 convolutional layer and 32 kernels.

3.4.2 Feature extraction using pre‑trained networks

There are currently pre-trained networks that can be utilized to solve categorization challenges. Deep convolutional neural networks (CNNs) like these may double as feature extractors if necessary. In these kinds of situations, the characteristics are gleaned from the levels in between. In this research, the integrated networks ResNet-50 [17] and VGG-16 are used to extract DR characteristics from images of the retinal fundus.

3.4.3 Feature extraction using grey level co-occurrence matrix (GLCM)

The gray-level co-occurrence matrix, sometimes known as the GLCM, is a tool that analyses the spatial relationship of pixels in a texture. The idea is to calculate the co-occurrence matrix for tiny picture parts and then utilize that matrix to derive statistics such as contrast, correlation, uniformity, homogeneity, probability, inverse, and entropy. This research implements the Angular Second Moment (energy), Inertia Moment, Correlation, and Entropy Moment. Inverse Difference Moment is also important.

a) Angular second moment

Uniformity or Energy are other names for Angular Second Moment. GLCM squares sum A2M assesses image homogeneity. A high Angular Second Moment indicates visual homogeneity or comparable pixels.

$A S M=\sum_{i=0}^{N_{g-1}} \sum_{i=0}^{N_{g-1}} P_{i j}^2$                    (1)

b) Inverse difference moment

The local homogeneity is denoted by the acronym IDM, which stands for "inverse difference moment." When the local gray level is consistent throughout, it has a high value, and the inverse GLCM also has a high value.

$I D M=\frac{\sum_{i=0}^{N_{g-1}} \sum_{j=0}^{N_{g-1}} P_{i j}}{1+(i-j)^2}$           (2)

The value of the IDM weight is the opposite of the Contrast weight.

c) Entropy

Entropy is a measure of the amount of information contained in a picture that must be sacrificed in order to achieve a certain level of compression. Entropy is a measurement that may be applied to both the image information and the loss of information or message that occurs during the transmission of a signal.

$ Entropy =\sum_{i=0}^{N_{g-1}} \sum_{j=0}^{N_{g-1}}-P_{i j} * \log P_{i j}$               (3)

d) Correlation

A linear connection between the grey levels of nearby pixels may be measured via correlation. The optical technology known as digital image correlation uses tracking and image registration methods to provide precise 2D and 3D measurements of the changes that occur in pictures. Although it is most often employed to detect deformation, displacement, strain, and optical flow, this technique finds widespread use in a variety of scientific and technical fields. The measurement of the movement of an optical mouse is an example of a fairly popular application.

$ Correlation =\frac{\sum_{i=0}^{N_{g-1}} \sum_{j=0}^{N_{g-1}}(i, j) p(i, j)-\mu_x \mu_y}{\sigma_x \sigma_y}$                     (4)

3.5 Weighted stacking ensemble learning model

For further prediction, the weighted stacking ensemble classifier based on majority votes is used in this study. The retrieved M-CNN features are used to train and evaluate the classifiers. It is recommended to use an ensemble classifier rather than individual classifiers because of the restrictions imposed by these classifiers. As a result, the error made by one classifier differs from that of others. The proposed ensemble classification model is shown in Figure 6.

The most straightforward kind of majority voting, known as hard voting, forecasts the ultimate class level in accordance with the prediction of the base classifier.

$Y=\operatorname{Max}_{j=1}^C \sum_{i=1}^N d_{i, j}$           (5)

where, N is the number of base classifiers, while C denotes the number of classifications. C=2 and N=3 in this work.

6.png

Figure 6. Proposed weighted stacking ensemble model

3.5.1 Support vector machine (SVM)

The support vector machine (SVM) is a machine that directly predicts decision surfaces. It has excellent performance indicators. The distance between the hyper-planes is specified as the margin [18]. The training examples are used to define an SVM classifier. Data that can only be separated using a nonlinear decision surface is used in real-world classification. The usage of a kernel-based transformation is included in the optimization of the input data in this situation.

$K\left(X_i, X_i\right)=\Phi\left(X_i\right) \cdot \Phi\left(X_j\right)$             (6)

Kernels calculate dot products in high-dimensional spaces without mapping data. Decision functions include:

$f(x)=\sum_{i=1}^N \alpha_i y_i K\left(X, X_i\right)+b$               (7)

This work explores two commonly used kernel functions, and they are:

$K(x, y)=(x . y+1) polynomial \, with \, degreed $                       (8)

$(x, y)=\exp \left\{-\gamma|x-y|^2\right\} radial \, basis \, function $              (9)

A data-dependent kernel known as a radial basis function (RBF) kernel has evolved as a potent alternative tool. RBF kernels have a slower convergence time than polynomial kernels, but they provide superior results. Dot products are used in the classification process. The number of support vectors scales linearly with the classification job. For non-separable data, soft margin classifiers are utilized. Slack variables are used to ease the separation requirements.

$x_i \cdot w+b \geq+1-\xi_i, \quad$ $ \, for \, y_i=+1$             (10)

$x_i \cdot w+b \leq-1+\xi_i, \quad$ $ \, for \, y_i=-1$             (11)

$\xi_i \geq 0, \quad \forall i$             (12)

According to the results of the preceding investigation, the proposed hybrid feature selection methodology is a very efficient method for assessing the data.

3.5.2 Fuzzy neural network (FNN)

There are three layers in a fuzzy neural network: Fuzzy input, fuzzy rules, and fuzzy output [19]. A popular neural network uses Back Propagation (BP) least mean-square learning. The structure is shown in Figure 7. Edges link neuron processing units. Each neuron input is weighted to indicate its relevance. The net value is found by linearly combining the inputs from the neurons.

7.png

Figure 7. Example neural network with single-output neuron and a hidden layer

The Fuzzy Backpropagation (FBP) estimates net value using a fuzzy LR number and doesn't assume criterion independence. The FBP algorithm never oscillates and never falls into a local minimum, another benefit [20].

Fuzzy Back Propagation algorithm (FBP algorithm)

Step 1: The input hidden layer's initial weight sets, w, should be generated at random such that each w_ji=(w_mji, w_αji, w_βji) is a fuzzy number of the LR type. Produce the weight set W' for the concealed output layer as well.

where, $w_{k j}^{\prime}=\left(w_{m k j}^{\prime}, w_{\alpha k j}^{\prime}, w_{\beta k j}^{\prime}\right), w_{j i}=\left(w_{m j i}, w_{\alpha j i}, w_{\beta j i}\right)$, $w_{k j}^{\prime}=\left(w_{m k j}^{\prime}, w_{\alpha k j}^{\prime}, w_{\beta k j}^{\prime}\right)$.

Step 2: Let (I_p, D_p) p=1, 20, ..., N. The fuzzy back propagation method requires a certain input-output pattern set in order to be trained. Here I_p=(I_p0, I_p1, I_p1), where each I_pi is an LR-type fuzzy number.

Step 3: Assign values for α and η; Alpha=0.1 Neta=0.9.

Step 4: Get the next pattern set (I_p, D_p) Assign (O_pi=I_pi, i=1, 2, 3...)

Step 5: Compute the input to hidden neurons$O_{p j}^{\prime}=f\left(N E T_{p j}\right), j=1,2 \ldots, m ; O_{p 0}^{\prime}=1$.

where, $N E T_{p j}=C E\left(\sum W_{j i} O_{p i}\right)$.

Step 6: Compute the hidden to output neurons $O^{\prime \prime}{ }_{p k}=f\left(N E T{ }^{\prime}{ }_{p k}\right), k=1,2, \ldots n ;$

where, $N E T^{\prime}{ }_{p k}=C E\left(\sum W_{j i} O^{\prime} p_{j j}\right)$.

Step 7: Compute the change of weights ∆ w'(t) for the hidden output layer as follows:

Compute $\Delta E_p(t)=\left(\partial E_p / \partial w^{\prime}{ }_{m k j}, \partial E_p / \partial w \alpha k j, \partial E p / \partial w^{\prime}{ }_{\beta k j}\right)$.

Compute $\Delta w^{\prime}(t)=-\eta \Delta E_p(t)+\alpha \Delta w^{\prime}(t-1)$

The updated weight i of hidden to output neuron is W’(t)=W’(t-1)+∆W’(t).

Step 8: Compute the change of the weights ∆ w'(t) for the input hidden layer as follows:

Let

$\begin{aligned} & \delta_{p m k}=-\left(D_{p k}-O^{\prime \prime}{ }_{p k}\right) O^{\prime \prime}{ }_{p k}\left(1-O^{\prime \prime}{ }_{p k}\right) \cdot 1 \\ & \delta_{p m k}=-\left(D_{p k}-O^{\prime \prime}{ }_{p k}\right) O^{\prime \prime}{ }_{p k}\left(1-O^{\prime \prime}{ }_{p k}\right) \cdot\left(-\frac{1}{3}\right) \\ & \delta_{p m k}=-\left(D_{p k}-O^{\prime \prime}{ }_{p k}\right) O^{\prime \prime}{ }_{p k}\left(1-O^{\prime \prime}{ }_{p k}\right) \cdot\left(\frac{1}{3}\right) \\ & \end{aligned}$

Compute

$\Delta E_p(t)=\left(\partial E_p / \partial w^{\prime}{ }_{m j i}, \partial E_p / \partial w \alpha_{j i}, \partial E p / \partial w^{\prime}{ }_{\beta j i}\right)$

Compute

$\Delta w^{\prime}(t)=-\eta \Delta E_p(t)+\alpha \Delta w^{\prime}(t-1)$

Step 9: Update the weight for the input-hidden-output layer as:

$\begin{aligned} W(t) & =W(t-1)+\Delta W(t) \\ W^{\prime}(t) & =W^{\prime}(t-1)+\Delta W^{\prime}(t)\end{aligned}$

Step 10: p=p+1; if (p<=N) go to step 5;

Step 11: count_of_itrns=count_of_itrns+1;

         if count_of_itrns<itrns

         {

         Reset the pointer of the first pattern in the training set;

         P=1;

         Go to step 5;

         }

3.5.3 AdaBoost classifier

One kind of ensemble boosting classifier is known as a-boost, which stands for adaptive boosting. Integrating classifiers improves their accuracy. This strategy uses iterative ensemble algorithms like AdaBoost. AdaBoost merges lesser classifiers to generate a strong, accurate classifier. AdaBoost works by setting the classifier weights and training the data sample in each iteration to forecast unusual occurrences correctly.

Let Y represent the set of potential class labels and X represent the input data space. We analyze the situation of two classes, Y={-1, +1}, and assume that X=Rn. Creating a mapping function is the objective here. F: X→Y that, taking into consideration the feature vector x∈X, calculates the (correct) class label y. Additionally, we take into consideration the scenario in which a collection of labeled data for the purpose of training is available:

$\left(x_1, y_1\right), \ldots .\left(x_N, y_N\right) ; x_i \in X ; y_i \in Y$               (13)

All strategies that are based on AdaBoost may be thought of as a greedy optimization strategy for the purpose of reducing the exponential error function:

$\sum_{i=1}^N e^{-y_i \cdot F\left(x_i\right)}$               (14)

Classifier decision F(x) Boosting's strong approximation skills are theoretically demonstrated, but its amazing generalization capabilities are only empirically validated, leaving room for future advancements. Other boosting techniques lack a natural stopping criteria like ours. The results are compared using Gentle AdaBoost in Matlab. The boosting strategy surpasses Gentle AdaBoost in generalization error (measured on a control subset), but lowers training error more slowly.

3.6 Classification using meta-learning model

A machine learning technique called meta-learning explores the learning process and identifies its mechanisms in order to reuse them in the future. The goal is to build a flexible automatic learning machine that can solve different learning problems using meta-data, such as learning algorithm properties, learning problem traits, or previously discovered patterns derived from the relationship between learning problems and the efficacy of different learning algorithms. This helps learning algorithms.

3.6.1 Enhanced recurrent neural network (ERNN)

Recurrent Neural Networks (RNNs), which are used to handle sequential input, are one of the most significant subfields in deep learning. Considering that input is processed sequentially, RNNs are naturally deep in time. When performing tasks like language translation and natural language processing, RNN has outperformed other sequence learning algorithms [21].

This paper proposes an Enhanced Recurrent Neural Network (ERNN), which overcomes the drawbacks of RNN by leveraging the absolute value of the recurrent layer output. The suggested RNN gains nonlinearity from absolute value while maintaining information flow during forward and backpropagation. The proposed approach uses two recurrent layers. As the recurrent layer's final output at each time step, the mean modulus of these two layers is determined.

It employs two parallel channels, $h_t^{(1)}$ and $h_t^{(2)}$, and is given by:

 $h_t^{(1)}=x_t w_{x h}^{(1)}+h_{t-1} w_{h h}^{(1)}+b_h^{(1)}$                 (15)

$h_t^{(2)}=x_t w_{x h}^{(2)}+h_{t-1} w_{h h}^{(2)}+b_h^{(2)}$               (16)

As seen below, the hidden layer activation is calculated.

$h_t=\left(\bmod \left(h_t^{(1)}\right)+\bmod \left(h_t^{(2)}\right)\right) / 2$            (17)

The output is given by:

$y_t=w_{h y}+b_y$              (18)

Rather than initializing w_hh as identity as in ERNN, $w_{h h}^{(1)}$ and $w_{h h}^{(2)}$ are initialized in such a way that the gradient flow is preserved. The variables $b_h^{(1)}$ and $b_h^{(2)}$ are both set to zero. Backpropagation with an adaptive learning rate is used for training. Despite the fact that ML-ERNN has more parameters than ERNN, the calculations of the two channels in ML-ERNN are independent of one another and can be done in parallel. In ERNN, h_t is dependent on h_t-1 at time step t through the identity matrix w_hh.

The hidden layer activation h_t=0; if input<0, hence information about the prior hidden state h_t-1 is lost. In the suggested method, input data is never lost while it passes through both $h_t^{(1)}$ and $h_t^{(2)}$. Because when considering their absolute values, a big negative value is more dominant than a small positive value, the suggested ML-ERNN uses the channels' absolute values.

As a result of this research, a weight updating factor was devised to solve the problem. The Gaussian distribution factor-based improvement for improving prediction rate performance is described in the section below.

Gaussian Distribution Factor (GDF):

This paper proposes a modified technique to solve these disadvantages. A type of relationship known as a Gaussian distribution is used in this strategy to factor its neighboring data toward its own data. This Gaussian distribution factor is influenced by two factors: data intensities, or feature attraction λ(0<λ<1), and spatial position of neighbors, or distance attraction ξ(0<ξ<1), which is also influenced by neighborhood structure. Taking into account the Gaussian distribution factor, which is defined as follows:

$s d^2\left(x_i, v_i\right)=\left\|x_i-v_i\right\|^2\left(1-\lambda H_{i j}-\xi F_{i j}\right)$                (19)

where, H_ij indicates the primary focal point of interest, and F_ij exemplifies the allure of keeping your distance. The underlying criteria λ and ξ rebalance the proportions of the two attractions in the community. Last but not least, the ERNN-based meta-learning model yields the highest quality outcomes.

In this study, the authors aimed to develop an Enhanced Recurrent Neural Network (ML-ERNN) with the purpose of improving the overall level of efficiency of the categorization during the detection of diabetic retinopathy. In this work, the M-CNN extraction was used to extract essential properties from fundus photographs and classify lesions according to varying degrees of severity. SVM, FNN, and AdaBoost were the EML classifiers used in the trials. Finally, a meta-model-based classifier is developed to improve the accuracy of diabetic retinopathy identification. The attributes that were retrieved by employing pre-trained networks are also employed for the assessment process. This multipath network may be altered to forecast a variety of retinal illnesses, which will result in an improvement to the system for monitoring the health of the retina. The proposed approach yielded a high accuracy of 99.96% on the testing set of the Kaggle dataset and 99.93% on the testing set of the APTOS dataset. Following a series of experiments, it was shown that the M-CNN features perform most effectively when combined with the ML-ERNN classifier. Due to the fact that we have only dealt with voting by a strict majority, one way that this work might be expanded is via the use of voting by a more flexible majority. For ophthalmologists, the proposed model will be immensely useful in assisting doctors in improving patient outcomes by leveraging insights gained from a broad range of patient data and treatment experiences. This model can analyze historical data to identify patterns associated with patients who are at a higher risk of developing severe diabetic retinopathy.