Smart Harvest: Web-Integrated Ripeness Detection for Apples with CNN Algorithm

Apples are one of the fruit crops in Indonesia with considerable production. Based on data from BPS, apple production in Indonesia in 2022 reached 523.596 tons and East Java became the province with the most apple production [1, 2]. This makes apples one of the fruits that are often consumed and favored by the people of Indonesia, especially those cultivated in East Java, such as Rome Beauty apples. Epidemiological studies have linked eating apples to a decreased risk of diabetes, asthma, cardiovascular disease, and cancer.

In the laboratory it is known that the anti-oxidant activity of apples is quite potent, inhibiting the growth of cancer cells, lowering lipid levels and increasing cholesterol. Chlorogenic acid, phloridzin, catechin, and quercetin are just a few of the potent antioxidants found in apples [3, 4]. Therefore, it would be very beneficial to conduct research that could inform people who are presently grading apples whether they have reached a suitable ripeness level for estimation based on the colour of their outer skin because colour is an important indicator in classifying fruit ripeness [5, 6]. To determine the apples' level of maturity, local Rome Beauty apples were used as test apples.

This apple comes from East Java with a bright reddish light green colour when it is ripe and light green when it is still raw. The colour of the skin on apples is one indicator of ripeness.

However, currently there are still many apple farmers, sellers, or buyers, especially in Indonesia, still using traditional methods to determine the level of maturity of apples. This certainly has disadvantages where it requires more energy and sometimes has a lack of accuracy because it depends on the judgment and perception of each human being [7, 8]. Therefore, a system is needed that can help determine the maturity level of apples.

In this research, we used a Convolutional Neural Network to classify Rome Beauty apple images into ripe or raw categories. CNN's feature extraction layers allow the model to capture a large number of features in an image, which improves performance. This research uses 2400 Rome Beauty apple images as a dataset with a division of 1600 for ripe Rome Beauty apple images and 1600 for raw Rome Beauty apple images. This study's goal is to classify ripe or raw Rome Beauty apples based on skin colour which has different levels of colour brightness.

The portions of this paper are as follows. Section I provides an introduction and background for this research. Section II presents related work, whereas Section III discusses the proposed method for determining apple maturity. Section IV presents the results and discussion, while Section V explains how to develop an apple ripeness detection system utilizing the CNN algorithm on a web application. Section VI finishes with a review of the study's findings and future directions.

ping_mu_jie_tu_2024-12-25_180521.png

Figure 1. CNN architecture model

image004.png

Figure 2. Flowchart classification model

The CNN architectural model features three hidden layers made up of a 2D Max Pooling layer and a 2D Convolutional layer, each of which contains an input layer with pre-labelled and uniformly shrunk apple images. The 2D Convolution layer, one of the feature extraction layers, extracts skin colour features from the input layer's image using a 3×3 filter size and matches all of the filters observed during testing. The Max Pooling Layer reduces the input image by selecting the maximum value from each array and retaining all pertinent information, including image features. The Rectified Linear Unit (ReLU) serves as an activation function in hidden layers, transforming the CNN input array into values between 0 and 1. The feature extraction layer, which generates two maturity categories to categorize the apple maturity level, is a preprocessing layer that preprocesses the input image before labelling and categorizing it in the fully connected layer.

A brief overview of the flowchart of the process for classifying the maturity level of apples is shown in Figure 2.

3.1 Image acquisition

The process of capturing images with an appropriate camera before they are examined is known as image acquisition [23]. The Rome Beauty apple image dataset is needed to categorize apple ripeness. It is possible to use as many as 2400 real Rome Beauty apples for both ripeness levels where 1200 images each for both raw and ripe apples using the phone's camera with 4080×3060 resolution for image acquisition of both types of apple images. Images are pre-labelled and categorized at this stage according to the appropriate level of maturity using the fixed background modeling method. To reduce image noise and improve CNN feature extraction, all background images obtained have no background. Since the background image remains unchanged during acquisition, fixed mobile image acquisition is used [24].

Figure 3 presents images of raw and ripe Rome Beauty apples that have a background and those that don’t have a background.

image005.png

Figure 3. Raw and ripe Rome Beauty apples, with a background and without a background

The image capture method was performed under meticulously regulated settings to guarantee the consistency and reliability of the categorization results. Uniform lighting was maintained using LED lights to minimize shadows and variations in brightness, ensuring that the colour features of the apples were accurately captured. The ambient lighting setup was calibrated to provide neutral, diffused illumination, simulating natural daylight conditions and eliminating any harsh reflections or glare that could distort the visual features of the apples. Images were captured from a fixed distance of 30 cm using a high-resolution smartphone camera, ensuring uniformity in the size and scale of the objects across the dataset. The shooting angle was kept perpendicular to the apple's surface to reduce perspective distortion and maintain consistent framing. This configuration standardized image quality, rendering the dataset robust and appropriate for precise feature extraction and classification by the CNN model.

The dataset, comprising 2400 images of Rome Beauty apples, was systematically partitioned into training, validation, and test sets to guarantee a balanced and impartial model training procedure. Specifically, 60% of the images (1440 images) were designated for training, 20% (480 images) for validation, and the remaining 20% (480 images) for testing. This division ratio was chosen to provide the model with a sufficient number of images during training, enabling it to learn diverse features effectively, while also reserving an adequate set for validation to fine-tune the model parameters and prevent overfitting. The final test set, completely separate from the training and validation data, ensures an unbiased assessment of the model's performance. This method preserves the integrity of the training process and offers a dependable assessment of the model's generalization capability, essential for appraising the robustness of the CNN in practical applications.

3.2 Image augmentation

Image augmentation is a method for altering data while preserving the original significance or content of the image. The purpose of image augmentation is to train the task model using the training dataset to ensure adequate generalization to the testing dataset [25]. Image augmentation is utilized to enhance CNN performance and mitigate overfitting, as it requires substantial data to yield reliable predictive outcomes. In this work, the geometric approaches Horizontal Flip, Vertical Flip, Rotation Range [26], and Zoom Range were applied for image augmentation. Following image augmentation, 600 test images of Rome Beauty apples and 600 training images of Rome Beauty apples were acquired. Several examples of the picture augmentation that has been done to the two varieties of apples are shown in Figures 4 and 5.

image006.png

Figure 4. Image augmentation on ripe Rome Beauty apples

image007.png

Figure 5. Image augmentation on raw Rome Beauty apples

Table 1. Image augmentation parameter settings

To enhance the model's generalization capacity and reduce overfitting, a range of image augmentation techniques were utilized with clearly stated parameters. These augmentations were chosen to simulate natural variations that could occur in real-world apple images while preserving the essential features needed for classification. The rotation augmentation applied random rotations within a range of -20° to +20°, allowing the model to recognize apples from different orientations. Horizontal and vertical flips were introduced with a probability of 50%, ensuring that the model could learn from mirrored images and become more robust to variations in viewpoint. The zoom augmentation was configured with a range of 0.8x to 1.2x, mimicking small changes in the distance of image capture and ensuring the model's robustness to variations in apple size. Furthermore, a brightness adjustment within a range of ±10% was applied to accommodate slight differences in lighting conditions, enhancing the model’s adaptability to varying environments, as illustrated in Table 1.

3.3 Training and testing

The efficacy of Machine Learning, particularly the utilized CNN model, is profoundly affected by the training and testing procedures [27]. The model summary for CNN architecture, which was completely self-engineered, was displayed in Table 2. Table 2 reveals the presence of one sequential layer, one rescaling layer, three convolutional layers, three max pooling layers, four batch normalization levels, one dropout layer, one global average pooling layer, and two dense layers. This model summary delineates the output format and the aggregate count of trainable parameters employed during the training and testing phases. To achieve an adequate probability categorization prediction result, the architectural models employed the SoftMax activation function. This approach enables the cross-entropy loss function to yield a suitable model loss value [28]. CNN's training dataset is divided equally, with 50% allocated to training and 50% to testing data. The testing data constituted a distinct dataset utilized to identify errors during training and mitigate overfitting [29] which is a primary issue encountered in the training process [30].

Table 2. CNN model summary

The CNN architecture used for apple ripeness classification has been carefully configured to optimize performance and ensure reproducibility. Each design choice, including the convolution kernel size, stride, and activation functions, has a specific rationale to enhance the model’s feature extraction and generalization capabilities. The convolution kernel size of 3×3 was chosen because it effectively captures local features such as skin colour variations and subtle texture changes while maintaining computational efficiency. The stride is set to 1 to ensure no significant loss of spatial information, enabling the model to analyze fine details critical for accurate classification. The ReLU activation function is utilized for its non-linearity, enabling the model to acquire intricate patterns, and its effectiveness in alleviating the vanishing gradient issue during backpropagation.

Table 3 illustrates the loss and accuracy outcomes for the training and validation of Rome Beauty apples. The epochs and batch sizes were meticulously evaluated during training and validation. Epochs refer to the total instances in which the learning classifier traverses the entire training dataset [31]. In the end, 32 batch sizes and 20 epochs were used to achieve the desired accuracy and loss levels.

Table 3. Summary of training and validation model of Rome Beauty apple

A series of experiments were performed to assess the influence of varying batch sizes on the training efficacy of the CNN model. Batch sizes of 16, 32, 64, and 128 were tested to determine the optimal setting that balanced training speed, memory efficiency, and model accuracy. We observed that a batch size of 16 led to more frequent updates to the model's parameters, resulting in unstable training with fluctuating accuracy and a higher likelihood of overfitting. Conversely, a batch size of 128 exhibited a more stable training curve but was prone to underfitting the data, as the reduced number of updates per epoch constrained the model's capacity to acquire intricate features. A batch size of 64 provided moderate results, but training was still less stable compared to smaller batch sizes. Ultimately, a batch size of 32 was found to offer the best balance: it provided stable and efficient training with a high level of accuracy and minimized the risk of overfitting or underfitting. The chosen batch size allowed for efficient use of GPU memory while ensuring that the gradient updates were neither too frequent nor too sparse, resulting in an overall improvement in the model's performance and convergence speed.