Diabetic Retinopathy Recognition and Classification Using Transfer Learning Deep Neural Networks

Diabetic retinopathy (DR), which damages the retina as a result of diabetes, can lead to blindness if left untreated. Diabetic retinopathy can be diagnosed using optical techniques such as fundus photography and optical coherence tomography (OCT). While fundus photography captures detailed images of the retina, OCT creates cross-sectional images that show abnormalities in the layers of the retina. For automatic DR diagnosis, deep learning models, usually trained using color images, can detect and classify DR lesions. Transfer learning can train deep learning models on small datasets and improve classification accuracy. In this study, we investigated the effectiveness of adaptive models (DenseNet, EfficientNet, VggNet, ResNet) in diagnosing DR using retinal images. We also examine the influence of Ben Graham before this movement. Color imaging of the retina plays an important role in the evaluation and diagnosis of DR and provides information about the location, extent and severity of damage. Automated diagnosis using machine learning, especially deep learning, can increase accuracy and speed. In recent years, transfer learning has become an effective method for training deep learning models on small data sets. Transfer learning has proven effective in many image classification tasks. It is possible to increase the accuracy and speed of automatic DR diagnosis using machine learning. A deep learning method was developed to detect and classify DR using color images. In this process, deep neural networks are often trained using large datasets of recorded images, where color information is used to identify and distinguish lesions. The output of the network can be used to provide diagnoses and weighted scores to guide treatment decisions. Although other imaging techniques such as OCT and fluorescein angiography provide additional information about structure and function, color imaging is still an important part of DR screening and diagnosis. Early diagnosis and treatment of DR is important to prevent blindness. However, manual testing of DR is time-consuming and error-prone. Diabetic retinopathy can be identified and diagnosed using color DR. Color images of the retina are frequently used to screen and diagnose DR because they provide information about the location, extent, and severity of damage. In particular, color analysis can help determine the appearance of hemorrhages, exudates, and other lesions indicative of DR. These lesions appear as light or dark spots on the retina and can be distinguished from healthy tissue by their color and texture. In summary, our work explores the potential of modifying learning models and image preprocessing techniques to improve the efficiency and accuracy of automated DR testing. These options are designed to enhance the limitations of clinical guidelines and provide a basis for better clinical decision-making.

The primary objectives of the paper are - to assess the effectiveness of transfer learning models, namely DenseNet, EfficientNet, VggNet, and ResNet, in diagnosing diabetic retinopathy using retinal images, to harness their learned features and optimize their performance for diabetic retinopathy diagnosis, to explore the impact of applying Ben Graham's preprocessing technique to the retinal images before feeding them into the transfer learning models to improve the model's ability to capture relevant features and patterns.

5.1 Ben graham preprocessing

The technique involves several steps aimed at improving the quality of the image and making it more suitable for analysis. The first step is to clean up the image's noise. This is typically done using a filter or smoothening operation to remove any high-frequency noise that may be present in the image. Next, the image is normalized to a common range to ensure that different images can be compared and analyzed together. This involves scaling the pixel values of the image so that they fall within a specific range, such as between 0 and 1. After normalization, the image is often subjected to further processing, such as edge detection, segmentation, or feature extraction. These techniques are used to identify specific regions or features within the image that are relevant to the analysis being performed. Some mathematical equations involved in the Ben Graham image preprocessing technique, Normalization equation:

$x^{\prime}=\frac{x-x_{\min }}{x_{\max }-x_{\max }}$        (1)

where, x' is the image's normalised pixel value, x is the original pixel value, x_min is the image's lowest pixel value, and x_max is its highest. Equation for detecting edges:

$\Delta f=\left[\frac{\delta f}{\delta x}, \frac{\delta f}{\delta y}\right]$         (2)

where, $\Delta f$ is the gradient of the image.

$\delta f / \delta x$ is the partial derivative of the image with respect to x, and $\delta f / \delta y$ is the partial derivative of the image with respect to y. Segmentation formula:

$C(i, j)={argmin}\left(I(i, j)-M(k, l)^2\right)$          (3)

where, $C(i, j)$ is the label assigned to the pixel at location $(i, j)$, $I(i, j)$ is the pixel value at location $(i, j), M(k, l)$ is the mean pixel value of the region around pixel $(k, l)$, and argmin denotes the label that minimizes the expression. Following is the pre-processing pseudo code:

Each preprocessing step contributes to achieving better results by addressing specific challenges associated with diverse image characteristics. Circular cropping focuses the analysis on the central region of the retinal images, which is often the most diagnostically relevant area. By discarding irrelevant peripheral information, circular cropping reduces noise and concentrates on the crucial features, potentially improving the model's ability to identify diabetic retinopathy-related patterns. Resizing standardizes the image dimensions, ensuring consistency and compatibility across the dataset. It also simplifies computational requirements. Uniform image sizes facilitate model training, making it more robust and reducing the risk of bias towards specific resolutions. This step aids in streamlining subsequent processing steps. Applying a weighted blur helps reduce high-frequency noise and enhances the generalization capability of the model. The blur operation can smooth out minor irregularities, making the model less sensitive to small variations and potentially improving its performance on diverse retinal images. Thresholding and cropping help eliminate background noise and focus on relevant details within the images. Removing non-contributory background pixels reduces interference, ensuring that the model primarily processes the informative regions of the retinal images. This step is particularly important for handling variations in image backgrounds. Converting the image from BGR to RGB ensures consistency in color representation across different systems and libraries. Uniform color representation facilitates better compatibility with pre-trained models, avoiding potential discrepancies in feature extraction due to color variations. Converting the image to grayscale simplifies the image representation, often preserving essential features while reducing computational complexity. Grayscale images are computationally more efficient and may retain sufficient information for diabetic retinopathy detection. This transformation also aids in standardizing input formats for diverse models. Reassembling the image after processing ensures that relevant color channels are retained for subsequent analysis. The final image retains essential information while discarding non-informative pixels, providing a clean input for the model and potentially improving its accuracy in identifying diabetic retinopathy-related features. The pre-processed images of three different output classes are represented in Figure 1.

2-1.png

Figure 1. (a) (c) (e) are original images, (b) (d) (f) are preprocessed images using Ben Graham’s method

5.2 Data augmentation

Out of 35,108 images, there were 25,802 images for the class Normal, 2,438 and 5,288 images for the classes mild and moderate respectively and 872 images for severe and 708 for the proliferative class. This made the dataset unbalanced. In order to balance an unbalanced dataset, data augmentation is used. Data augmentation is a strategy for augmenting data using existing training models as a starting point. The advantage of data augmentation is that there is no need to collect additional data, which can be time-consuming and expensive; It can be used to expand the size of the dataset, help improve the accuracy of model learning, and help build machine learning models. It is more robust to changes in input data (such as changes in lighting, orientation, or scale) and increases the diversity of training data, thus preventing overfitting and improving the overall ability of machine learning models. Various transformations applied to the dataset are rotation of 15-30 degrees where the image is rotated by a certain angle, translation of where the image is shifted horizontally or vertically by a certain distance, scaling of 1.1$x$ -1.2$x$ where the image is scaled up or down by a certain factor helping to simulate objects of different sizes, horizontal and vertical flipping, cropping where a smaller rectangular portion of the image is extracted and color jittering where the color of the image is modified by changing the hue, saturation, and brightness helping to simulate different lighting conditions. Rotating images introduces variability in the orientation of features, helping the model generalize better to images with different perspectives. This is particularly relevant in medical imaging where the orientation of retinal structures may vary across patients. Horizontal and vertical shifts simulate different viewpoints, providing the model with variations in image composition. This helps the model become less sensitive to the specific positioning of retinal structures, enhancing its ability to detect diabetic retinopathy across diverse images. Scaling images up or down simulates objects of different sizes. This is important in capturing variations in the size of retinal structures or anomalies, making the model more robust to different scaling factors commonly encountered in medical imaging datasets. Horizontal and vertical flipping simulate objects appearing in different orientations. This augmentation technique aids in addressing the potential bias introduced by the specific orientation of retinal features in the original dataset. It ensures the model is exposed to a diverse range of orientations during training. Extracting smaller rectangular portions of images focuses the model's attention on specific regions of interest. This is particularly useful for diabetic retinopathy detection, where certain pathological signs may manifest in localized regions. Cropping helps the model learn to identify relevant features in different areas of the retinal images. Modifying the color of images through changes in hue, saturation, and brightness simulates different lighting conditions. This is crucial for training a model that can handle variations in illumination often encountered in real-world scenarios. Color jittering ensures the model is not overly sensitive to specific color tones or lighting conditions present in the original dataset.

5.3 Model description

5.3.1 VGG19

VGGNet19 architecture has 19 layers, including 3 fully connected layers, 5 maximum pooling layers and 16 convolutional layers. The max pooling layer has a 2-step 2×2 filter, while the convolution layer has a small 3×3 filter. Each convolutional layer has a number of different filters, ranging from 64 in the first layer to 512 in the last layer. The number of units in the last connected layer is the same as the number of clusters in the work division, and there are a total of 4096 units in each of the first two connected layers. After each layer and layer are connected to all, the layer adjustment (ReLU) function is used to realize the function. To prevent over-settlement, release treatment is also applied to all layers.

5.3.2 ResNET50

Each of the five layers that make up the 50 layers of the ResNet50 architecture has a remaining number of layers. Two convolutional layers with batch normalization and cross-connection that add input to blocks along with ReLU activations generate the residuals. Convolution function and maximum pooling are used in the first stage to reduce the sample size. More blocks and more filters are gradually added in later stages to capture increasingly complex messages. The last layer consists of combining with softmax and global averaging in order to make the distribution.

5.3.3 DenseNET121

The CNN creator, called DenseNet121, has 121 layers and around 8 million nodes. Dense Block has several levels and is the core of DenseNet121. Each layer in a density receives private maps from all its layers and passes the private maps to the layers behind it. To improve the performance and stability of the model, the transformation process of dense blocks consists of batch normalization and rectified linear unit (ReLU) activation. The global average layer of the network calculates the average of each map specification across all dimensions. The final classification is done by feeding positive-valued vectors into a fully connected cluster.

5.3.4 EfficientNET_b0

EfficientNET_b0 architecture consists of a backbone, several blocks and a header. The root consists of multiple convolutional layers followed by maximum pooling and batch normalization techniques. Each block has a collection of individual links and cross-links, and the blocks are ordered hierarchically. In each block, the resolution and number of channels of the particular input map are increased. The head consists of a softmax layer for distribution, a fully connected layer, and a global neutral layer. Composite scaling is a state-of-the-art method used by EfficientNET_b0 to scale models to multiple dimensions. Depending on the desired solution, the design can be scaled up or down so that resources can be used efficiently. Below is the final algorithm:

Using a dataset of 35,108 eye pictures divided into 5 classes, several deep learning models were examined for the diagnosis of diabetic retinopathy in this work. We used Ben Graham's preprocessing method to boost the models' performance since it made the picture simpler and made it easier to tell apart different characteristics. Additionally, we balanced the dataset using augmentation to mitigate the imbalanced class distribution. We applied 4 transfer learning models, including VGGNet19, ResNet50, DenseNet121, and EfficientNet_B0, to both the processed and unprocessed images. Results showed that the best validation accuracy for unprocessed images was achieved by the DenseNet model, which was 78.6%. However, overfitting was observed for all models on the unprocessed images. In contrast, for the processed images, we did not observe any overfitting, and the highest validation accuracy was achieved by the DenseNet model, which achieved an accuracy of 97.7%.

Overall, the results indicate that Ben Graham's preprocessing method can enhance the effectiveness of machine learning models for detecting diabetic retinopathy. Additionally, transfer learning models can achieve high accuracy on processed images, which can be beneficial for clinical diagnosis. However, overfitting is a potential issue for unprocessed images, which may require further investigation. Our work lays the foundation for further investigations into refining preprocessing techniques, exploring additional deep learning architectures, and addressing potential overfitting challenges. Additionally, as we contribute a balanced dataset and showcase the efficacy of augmentation, future studies may build upon these insights to improve model generalization and real-world applicability. In conclusion, our study contributes to the development of accurate and reliable machine learning models for diabetic retinopathy detection which may help in the early detection and care of this illness.