The Impact of Color Space with Augmentation on ResNet-50-Based Corn Stalk Disease Classification

This study focused on developing a corn stem disease classification model using the CNN ResNet-50 architecture. The main goal of this research was to create a model that can accurately identify and classify five types of corn stem conditions: healthy corn, and diseases caused by Erwinia carotovora, Pythium, Stenocarpella, and Gibberella.

Classification is one of the main techniques in data mining. Data mining is the process of extracting useful information or knowledge from large and complex datasets [11]. This technique involves analyzing and discovering hidden patterns in the data, which can be used to make decisions or develop predictive models. Data mining encompasses various techniques, such as classification, clustering, regression, and association, all designed to uncover valuable insights from data [12]. Classification itself is a technique in machine learning aimed at grouping data into predefined categories based on certain features [13]. In this context, classification models learn from labeled training data to recognize patterns or characteristics that distinguish one category from another, enabling them to predict the category of new, unseen data.

2.1 Color space

Color space is a system for representing color in numerical format that allows computers to process and display digital images [14]. Color space defines how colors are represented through combinations of different color channels, facilitating visual analysis and image processing. In the field of image processing, selecting the right color space is crucial because it can affect how well the visual features of an image can be identified and processed by algorithms [15]. Common color spaces include RGB, HSV, YCbCr, and LAB, each with specific advantages and characteristics. In this research, two types of color spaces are used, namely RGB and HSV, to explore the model's effectiveness in detecting corn stem diseases. The selection of these two-color spaces is based on their ability to capture different visual features, with RGB working with basic color intensities, while HSV emphasizes hue, saturation, and brightness which is more similar to human perception.

A. RGB

The RGB color space is one of the most common and widely used color models in digital image processing [16]. In this model, each color is represented as a combination of three primary colors: red (R), green (G), and blue (B) [17]. The value of each color channel typically ranges from 0 to 255 in 8-bit representation, enabling more than 16 million different color combinations. The combination of intensities from these three channels produces various color spectrums visible on monitor screens, televisions, digital cameras, and other visual devices.

The color formation process in RGB color space follows the additive color model principle, where the primary colors (red, green, and blue) are added at various intensities to produce the final color. For example, when all three channels are given maximum values (255, 255, 255), the result is white, while if all values are zero (0, 0, 0), the result is black [18]. Other value combinations will produce different spectrum colors such as purple, brown, yellow, and others.

The advantage of RGB color space lies in its ease of digital image processing, as sensors in cameras and other digital devices also capture images in this format. Additionally, many image processing algorithms, such as edge detection, color segmentation, and pattern recognition, use RGB as standard input due to its simplicity in managing color based on intensity. However, one weakness of RGB is its dependence on lighting conditions, which can affect color recognition performance in different environments [19]. This makes RGB less effective when variations in light intensity or shadows affect the image, leading to other color spaces like HSV being often chosen for such situations.

In this research, the use of RGB color space enables the model to learn from differences in primary color intensities in detecting disease symptoms in corn stems. Images are processed in RGB format, where color changes in disease-infected stems can be recognized by the model based on intensity variations in each color channel.

B. HSV

The HSV color space is designed to more closely match human color perception compared to the RGB color space. HSV separates color information (hue), intensity (value), and clarity (saturation), enabling easier analysis under various lighting conditions [20]. In this color space, each color is represented by three main components: Hue (H), which describes the tone or type of basic color such as red, green, blue, and others; Saturation (S), which indicates how pure or saturated the color is; and Value (V), which represents the brightness level of the color [21].

Calculations in HSV involve transforming RGB values into cylindrical coordinates. Hue is measured in degrees from 0° to 360°, where 0° represents red, 120° green, and 240° blue, with intermediate values producing mixed colors. Saturation ranges from 0% (no saturation, resulting in grayscale) to 100% (full color without white mixing). Value or brightness is also measured in percentages, with 0% being total black and 100% being the brightest color [22].

The main advantage of HSV color space is its ability to isolate color hue from lighting and saturation, allowing color differences to be recognized even when images are taken under varying lighting conditions [23]. This is particularly useful in image processing tasks such as color segmentation, object detection, and image analysis in real-world environments, where light changes can affect image appearance. For example, in plant recognition, HSV allows the model to more easily detect color changes caused by disease without being overly affected by shadows or uneven lighting.

Therefore, in this research, HSV color space is used as an alternative to RGB to test the model's ability to identify corn stem diseases. By separating hue from lighting, HSV enables more stable detection of color changes in infected areas, which might be more difficult to detect in RGB color space. Thus, the model can be more adaptive to lighting variations while remaining accurate in detecting early signs of disease in corn stems.

2.2 Data augmentation

Data augmentation is a method in machine learning and deep learning employed to increase the size and diversity of training datasets by modifying existing data. In the context of image processing, data augmentation generates new images are generated by applying different transformations to the original images, such as cropping, horizontal or vertical flips, scaling, rotation, contrast and brightness adjustments, and noise addition [24]. The main purpose of data augmentation is to introduce new variations into the training data so that the model can learn from more examples without needing to collect large additional datasets.

Augmentation techniques are particularly important when the original dataset size is limited, as deep learning models like CNN tend to require large amounts of data to effectively recognize complex patterns. If the training dataset is too small or lacks variation, the model can experience overfitting, where it learns too specifically from the training data and is unable to generalize to new data. Data augmentation helps address this issue by providing various examples of variations, making the model more accurate and having better generalization capabilities.

Several commonly used data augmentation methods include [25]:

a) Rotation and Flip: Changing image orientation by rotating or flipping to create new variations. This helps the model learn to recognize objects from various angles and orientations.

b) Scaling and Cropping: Modifying image size or cropping certain parts of the image to introduce variations in scale. This technique helps the model learn to recognize objects despite varying sizes.

c) Contrast and Brightness Adjustment: Altering image light intensity or contrast to simulate different lighting conditions. This is useful so that the model doesn't become too dependent on specific lighting conditions when recognizing objects.

In this research, data augmentation is used to expand the variation of the collected corn stem image dataset. The augmentation techniques applied include rotation, flip, and brightness adjustment to ensure the model can recognize patterns from various angles and lighting conditions. Thus, the developed ResNet-50 model will be more robust and accurate in classifying types of corn stem diseases under different conditions. These augmentation techniques not only help improve model accuracy but also reduce the risk of overfitting, resulting in better model performance when applied to new data or real data in the field.

2.3 ResNet-50

ResNet-50 architecture is a CNN model consisting of 50 layers with a residual mechanism. This approach was introduced to address the degradation problem that commonly occurs in very deep networks, where accuracy tends to decrease as the number of layers increases. ResNet50 uses the concept of skip connections in other words also called as shortcut connections that allow signals to bypass one or more layers, it is making easier for the network to learn more complex features without experiencing the vanishing gradient problem [8].

ResNet50 uses residual blocks that allow the network to learn the identity function from previous layers. These blocks consist of two or three layers with direct connections known as skip connections or shortcuts. The main concept of residual blocks is to simplify the network's learning of the differences (residuals) between the input and output, rather than learning the complete transformation function. In general, the ResNet50 architecture consists of the following parts [26]:

a. 1 initial convolutional layer (7 × 7 kernel)

b. 4 main residual blocks, each having a different number of layers

c. Global average pooling layer at the end

d. Fully connected layer as output

The ResNet-50 architecture is shown in the Table 1.

Table 1. ResNet-50 architecture

In this research, the ResNet-50 architecture was used to classify corn stem diseases from RGB and HSV images. This model was chosen due to its proven capabilities in various image classification tasks, including object recognition and plant disease detection. The basic ResNet-50 architecture consists of several residual blocks containing convolution layers, ReLU activation, and batch normalization. These residual blocks are equipped with shortcut paths that connect input directly to output, thus accelerating convergence and improving model performance during training.

2.4 Confusion matrix

In this research, a confusion matrix was used to assess the performance of the ResNet-50 model in classifying corn stem diseases. A confusion matrix is an evaluation tool used to measure the effectiveness of classification models by providing detailed insights about model predictions compared to actual values [27, 28].

This matrix will help identify how well the model recognizes each different category and examine if certain categories are frequently misclassified by the model. Thus, the results from the confusion matrix can be used for further analysis in optimizing the model or identifying specific weaknesses that need improvement, such as adding more training data for less accurate classes or using more appropriate augmentation techniques.

tu_pian_8.png

Figure 1. Confusion matrix

Figure 1 illustrates the use of a confusion matrix for evaluating model performance. The matrix includes four key elements: True Positive (TP), False Positive (FP), True Negative (TN), and False Negative (FN), arranged in a matrix format. In multi-class classification, the confusion matrix can be extended to assess the model’s accuracy in classifying each class separately. Below are explanations of these four components.

• The first metric is the true positive rate (TP), which is the percentage of times the model gets the sample labels right. For example, the model predicts 'disease A' and the original label is also 'disease A'.
• The number of cases where the model correctly identifies that a sample is not part of the target class is the definition of the true negative rate. For example, the model predicts 'not disease A' and the original label is also 'not disease A'.
• The false positive rate is the number of cases where the model incorrectly classifies samples as the target class when they should not be.
• The false negative rate is the number of cases where the model fails to identify samples as the target class when they should be.

We can compute a number of model performance evaluation metrics, that include recall, accuracy, precision, and F1-score, using data from the confusion matrix [29]. Here are the equations:

• Accuracy

$Accuracy=~\frac{TP+TN}{TP+TN+FP+FN}$             (1)

• Precision

$Precision=~\frac{TP}{TP+FP}\times ~100%$              (2)

• Recall

$Recall=~\frac{TN}{TN+FN}\times ~100%$             (3)

• F1 Score

$F-measure~=2~\times ~\frac{Recall~\times ~Precision}{Recall+Precision}\times ~100%$          (4)

In this study, a CNN model with the ResNet50 architecture was used to classify corn stalk diseases based on images processed in two main scenarios: without augmentation and with data augmentation. The parameters applied in training the model include an input image size of 224 × 224 pixels, a batch size of 8, and a total of 10 epochs. The optimizer used is Adam with a learning rate of 0.001, aimed at accelerating convergence during the training process. The available image data is split with 80% allocated for training and 20% for testing data. In order to make the model better at identifying various disease patterns, data is also added to the training dataset to increase image variability. The classification results using the proposed method are displayed in the Table 3.

Table 3. Result of classification using ResNet-50

Table 3 shows that the results indicate that data augmentation significantly improves model performance across both RGB and HSV color spaces. In the RGB color space, accuracy improved from 80.00% to 97.76% after augmentation, with corresponding increases in precision, recall, and F1-score. However, this improvement comes at the cost of increased running time, from 2410.38 seconds to 5752.3 seconds seen in Figure 5 and Figure 6.

tu_pian_12.png

Figure 5. Confusion matrix for augmentation RGB

tu_pian_13.png

Figure 6. Training and validation for augmentation RGB

On the other hand, in the HSV color space, accuracy also increased—from 86.67% to 90.13% with augmentation. The F1-score improved from 85.26% to 87.81%, showing more consistent gains across all metrics compared to the unaugmented version, although with increased computation time as well, from 2565.45 seconds to 5302.75 seconds seen in Figure 7 and Figure 8.

tu_pian_14.png

Figure 7. Confusion matrix for augmentation RGB

tu_pian_15.png

Figure 8. Training and validation for augmentation RGB

Overall, RGB with augmentation achieved the highest accuracy (97.76%), but HSV without augmentation already outperformed RGB without augmentation. This suggests that HSV may inherently provide better feature separation, while augmentation further enhances performance by providing more diverse training data.

The results of the testing indicate that data augmentation has a significant impact on enhancing the model's performance when utilizing RGB images. In the absence of augmentation, the model achieved an accuracy of 80.00%, with precision at 80.78%, recall at 80.30%, and an F1-score of 80.54%. Following the implementation of data augmentation, the accuracy drastically increased to 92.76%, accompanied by corresponding improvements in precision (91.91%), recall (92.43%), and F1-score (92.17%).

These findings suggest that data augmentation facilitates the model's ability to learn more effectively from the variations in features present within the dataset, thereby enhancing its capability to recognize distinct patterns in images of healthy and diseased corn stalks. However, it is noteworthy that the training duration increased to 5752.3 seconds due to the model processing a larger volume of data during the learning phase.

Additionally, the table shows a comparison of performance between the models trained using images in RGB format and HSV format. In the scenario without augmentation, the model using HSV images demonstrated better performance compared to RGB, with an accuracy of 86.67% compared to 80.00% for RGB. The precision value was 83.54%, recall was 87.05%, and F1-score was 85.26%, all of which were higher for HSV. This suggests that the model is more effective at recognizing disease patterns when using images in the HSV color format. This may be due to the different ways colors are represented in the HSV format, allowing the model to focus more on variations in hue, saturation, and value, which enables better separation between areas infected by pathogens and healthy areas.

However, in the application of augmentation, images in the RGB color space perform better in classifying the dataset compared to those in the HSV space. Therefore, both data augmentation techniques and the selection of appropriate color representations are crucial for improving the performance of deep learning models.