The Effect of Color and Augmentation on Corn Stalk Disease Classification Using ResNet-101

This research focuses on classifying diseases in corn stalks using digital images. One of the key elements of data mining is classification. Data mining is the process of using mathematical techniques, artificial intelligence, statistics, and machine learning to extract, identify, and analyze different types of data in order to find patterns within them [11, 12]. Knowledge discovery from databases is another name for data mining. Data mining can be classified into a number of categories based on its functions, including description, prediction, estimation, classification, clustering, and association [13]. In this research, the classification of digital images involves three main stages, starting with dataset collection, followed by the training process, and finally, the evaluation process to determine how well the trained model performs.

2.1 Color SpaceRGB (red, green, blue)

The RGB color space is a color system that uses three basic components: red, green, and blue, to produce a variety of other colors [14, 15]. Each color in this space is determined by the combination of intensities of the three components. By adjusting the intensity of each RGB component within a specific range, such as from 0 to 255 in an 8-bit representation, different colors can be created. For example, the color white is produced with full intensity from all three components (255, 255, 255), while the color black is produced with zero intensity (0, 0, 0) [16].

In digital image classification, the RGB color space is often used due to its ability to represent colors in a way that closely aligns with human perception. Each digital image typically consists of three primary colors: red, green, and blue, each containing information about the intensity of the color for each pixel. The classification process utilizes this information to identify and categorize objects or patterns present in the image. Thus, by using the RGB color space, classification algorithms can analyze color features from the images to assist in the identification process. For example, in the research, the classification model is trained to recognize specific color characteristics that distinguish between several categories. The RGB color space provides a strong foundation for feature extraction and image analysis due to its ability to capture and store detailed color information, which is crucial for achieving high classification accuracy. In Figure 1, this is an image of a corn stalk in RGB color space.

1.png

Figure 1. RGB corn stalk image

HSV (Hue, Saturation, Value)

The HSV color space is a color representation system that consists of three main components: Hue (H), Saturation (S), and Value (V) [17]. Hue refers to the base color perceived, such as red, blue, or green, and is measured in degrees from 0 to 360 on the color wheel [18]. Saturation describes the intensity or purity of a color, where high values indicate bright and pure colors, while low values indicate more muted or grayish colors [18]. Value, or brightness, measures how light or dark a color is, with a value of 0 indicating black and a full value representing the brightest form of that color [18, 19].

In the context of digital image classification, the HSV color space has the advantage of separating color information from brightness information. This allows the classification model to focus on the Hue component to recognize specific colors without being affected by variations in lighting or brightness. This is particularly useful when dealing with varying or inconsistent lighting, as the Saturation and Value components enable the system to manage differences in intensity and color saturation [20].

2.png

Figure 2. HSV corn stalk image

The use of the HSV color space in digital image classification allows algorithms to be more accurate in recognizing and classifying objects based on their colors, as the HSV color space facilitates the separation and identification of colors even when lighting conditions vary. Thus, HSV helps improve the performance of image classification systems by making them more robust against changes in lighting and color saturation. This is as depicted in Figure 2, which is a picture of a corn stalk in HSV color space.

2.2 CNN model: ResNet-101

ResNet-101 is one of the Residual Network (ResNet) architectures designed to address issues in training deep neural networks [21]. ResNet, introduced by Kaiming He and his team on the research in 2015 [22], that introduces the concept of residual learning to address the common vanishing gradient problem encountered in very deep networks. ResNet-101 is an enhanced version with 101 layers and includes various modern optimizations to improve performance in image recognition.

ResNet are designed to enable deep networks to be trained effectively without a decline in accuracy as the number of layers increases. This architecture leverages residual blocks, which feature shortcut connections that allow gradients to pass through several layers without losing important information.

Table 1 shows that there are 5 main layers in the ResNet-101 architecture [23]. The first layer is the Conv Layer (Initial Convolution Layer) which consists of Conv1 and pooling. In Conv1, the network begins with a standard convolution layer with a kernel size of 7×7, stride 2, and padding 3. This is the first layer that functions to extract basic features from the input image. The output is then sent to the max pooling layer with a kernel size of 3×3 and stride 2, which reduces the output size. After the first convolution, the image is processed through max pooling to reduce its dimensions and speed up computation. The second layer is the Residual Block. ResNet-101 has four main stages, each consisting of several residual blocks, with each stage having a different number of blocks. Stage 1 (Conv2_x) has 3 residual blocks. Each block consists of three layers: two 1×1 convolutions (to reduce and restore dimensions) and one 3×3 convolution in between. In addition, shortcut connections in each block connect the initial input to the output after the convolution operation, allowing information to be passed directly. Stage 2 (Conv3_x) Has 4 residual blocks with a similar architecture. The first convolution in each block is responsible for reducing the dimensionality of the features. While Stage 3 (Conv4_x) has 23 residual blocks, making it the deepest part of the network. At this depth, the network is able to learn complex features from the image, such as complex texture patterns, object shapes, or special characteristics of the image. and the last stage, Stage 4 (Conv5_x) has 3 residual blocks. This is the final part of the convolutional network before the features are sent to the fully connected layers. These residual blocks are responsible for learning the last high-level features of the image, which helps improve classification accuracy. Layer 3 is Shortcut Connections, where each residual block uses shortcut connections, allowing information to pass directly from input to output. This helps overcome the vanishing gradient problem and makes it easier for the network to learn, even as the number of layers increases. layer 4 is Global Average Pooling. This layer functions to flatten all extracted features into a one-dimensional vector. This process reduces the dimensionality of the data without losing important information, compressing the output before sending it to the fully connected layer. and the last layer is the Fully Connected Layer and SoftMax. This layer is responsible for mapping the learned features to probability values. while SoftMax is used to determine the final class of the input based on these probabilities. This allows the network to make classification predictions more efficiently.

Table 1. ResNet-101 architectures

2.3 Confusion matrix

One approach to evaluate how effectively a machine learning classification model performs is a confusion matrix, which shows how well or weakly the model accurately classifies data. Several performance evaluation metrics, including accuracy, precision, recall (sensitivity), and F1-score, can be computed using the confusion matrix data [24]. Table 2 shows the confusion matrix that can serve as a reference for its calculation.

Table 2. Confusion matrix

The following formulas can be used to determine the accuracy, precision, recall, F1-score values based on the confusion matrix table [25].

1. Accuracy

$Accuracy=\frac{T P+T N}{T P+T N+F P+F N}$               (1)

2. Precision

$Precision=\frac{T P}{T P+F P} \times 100 \%$             (2)

3. Recall

$Recall=\frac{T N}{T N+F N} \times 100 \%$            (3)

4. F1 Score

$F- measure =2 \times \frac{ { Recall } \times { Precision }}{ { Recall }+ { Precision }} \times 100 \%$             (4)

where,

TP: The number of instances where the model correctly predicted the positive class,

TN: The number of instances where the model correctly predicted the negative class,

FN: The number of instances where the model incorrectly predicted the negative class when the actual class was positive,

FP: The number of instances where the model incorrectly predicted the positive class when the actual class was negative.

In this results and discussion section, the performance of the ResNet-101 model in classifying corn stalks will be presented and analyzed. The classification model is formed using a dataset of 750 samples with five target classes. The parameters used in the system modelling with ResNet-101 are as follows:

Learning rate (Lr) = 0.001

Batch size = 8

Optimizer = Adam

Number of Epoch = 10

With these parameters, the model training process was conducted using both the augmented dataset and the non-augmented dataset. The results obtained are displayed in Table 4 and Table 5.

Table 4. Performance results of ResNet-101 with augmentation

Table 5. Performance results of ResNet-101 without augmentation

Based on the two tables above, the ResNet-101 model with RGB images achieved an accuracy of 74.67%, with precision at 75.87%, recall at 72.58%, and an F1-Score of 74.19%. Although these results are good, they indicate that without data augmentation, the model has limitations in recognizing the actual variations in image patterns. However, with the implementation of augmentation, the accuracy significantly increased to 89.47%. This improvement in accuracy demonstrates that data augmentation successfully added variation that helps the model learn more patterns, thereby enhancing its generalization and classification accuracy.

Meanwhile, the use of HSV image data yielded the best results without augmentation, achieving an accuracy of 90.67%, precision of 92.22%, recall of 90.92%, and an F1-Score of 92.57%. This indicates that the conversion to HSV helps in recognizing color features more distinctly, which are important indicators in identifying corn stalk diseases. However, when augmentation was applied to the HSV images, the accuracy dropped to 67.11%, which is lower than all other tests. This decrease may be due to the augmentation introducing unnatural color variations, making it difficult for the model to recognize relevant patterns. This highlights the need for augmentation to be tailored to the type of color format used to achieve accurate and effective results.