Comparative Analysis of Classical Machine Learning and Deep Learning Methods for Fruit Image Recognition and Classification

In this examination of the multifaceted role played by fruits in our lives, encompassing dietary significance, cultural value, and ecological importance [1], the intricate ways in which these vibrant earthly treasures contribute to the human experience are delved into [2]. A pivotal aspect involves the visual identification and classification of diverse fruit varieties, requiring the application of computer vision and machine learning (ML) techniques, commonly referred to as fruit recognition or classification [3]. Substantial time, cost, and labor losses are incurred through the hand and eye sorting of fruits based on their visual characteristics [3, 4]. The challenges posed by small visual differences between fruits are recognized [5, 6], and efforts are made in this study to address the limitations of current fruit classification methods, particularly in the agriculture sector, where real-time use is hindered by lengthy training and testing times or inaccuracies [7, 8]. In response to these identified gaps, the primary objective of this work is to have picture features extracted and preprocessed from the comprehensive Fruit-360 dataset, utilizing Principal Component Analysis (PCA), color, and texture features. Subsequently, a combination of classical machine learning methods (SVM, KNN, and DT) and the deep learning classification network AlexNet is employed to classify diverse fruit varieties [9, 10]. The analysis of classification results aims to identify the most effective ML and DL models for the Fruit-360 dataset, thereby addressing the pressing challenges in fruit recognition and classification. The organization of this paper can be outlined as follows: Section 2, titled "Related Work," is dedicated to reviewing prior research. In Section 3, we delve into the foundational theory and present our proposed system. Section 4 is focused on presenting the results and engaging in a detailed discussion. Finally, the paper culminates with a conclusion in Section 5.

3.1 Proposed system

Three primary components make up the proposed system: (Preprocessing, Feature extraction, and classification). The suggested system's flow diagram is depicted in Figure 1.

1.png

Figure 1. Flow diagram

3.1.1 Pre-processing

Utilizing segmentation techniques, it is crucial to isolate the fruit region from the background of the image in order to obtain the necessary features for classification. To speed up processing, the image is first downsized from 100×100 to 150×150 pixels in the pre-processing stage. By removing any unwanted contaminants, the image is subsequently converted to grayscale, gray threshold, and finally black and white. Utilize the region of interest from the segmented fruit in the final fruit mask. As shown in Figure 2.

2.png

Figure 2. Pre-processing steps

3.1.2 Feature extraction

Feature extraction is a technique for discovering more precise information about significant elements in a picture [21]. A classification approach that overfits the training data and performs badly when applied to new samples is frequently needed for analyses with a lot of variables. Feature extraction is a technique for combining the variables to get around these problems while still accurately characterizing the data [22]. Pattern recognition features frequently include context, PCA, shape, color, texture, or grayscale data. In image processing or machine vision, a pattern measurement from the beginning or particular consecutive measurement patterns are transformed into a new pattern feature [23]. The process of classifying an object into a category or a class using higher-level data or characteristics that have been retrieved from the object is known as pattern classification. It creates the categorization strategy while also automatically identifying things in the image [24]. Finding patterns, especially visual and aural patterns, is a major focus of the field of pattern recognition in computer science. It employs methods from various academic fields, such as ML, statistics, and others [24].

Principal component analysis (PCA). Principal Component Analysis, a dimensionality reduction technique, can be used to solve compression and recognition problems (PCA). Other names for PCA include hoteling or eigenspace projection [25]. The original data space or image is converted into a subspace collection of principal components (PCs) using PCA, where the diversity between the photos is primarily captured by the first orthogonal dimension. The final dimension of this subspace exhibits the smallest amount of variation between the photos according to the statistical characteristics of the targets [26]. When characterizing the original vector using the orthogonal or uncorrelated output components from this transformation, the mean square error may be the lowest. Results from common transform techniques like PCA are not directly related to any one feature of the original test. By extracting features, PCA can identify the sample data elements that show the greatest change. From all the features, this can be utilized to select a few fascinating people.

The projection matrix W_opt that maximizes the determinant of the total scatter matrix of the projected samples is often found using the PCA method [27] as follows.

where, w is:

$W_{o p t}=\arg \max _w \frac{\left|w^T s_T w\right|}{\left|w^T s_w w\right|}$    (1)

And the total scatter matrices are ST:

$\mathrm{ST}=\sum_{1=1}^c\left(x_i-\mu\right)\left(x_i-\mu\right)^T$           (2)

The Symbol µ denote the mean of vector feature for all sample in set of training and xi is the i-th of feature vector of sample and c are the amount of training sample numbers.

Color feature. Researchers have utilized the fruit's color as a defining characteristic for classification quite a bit. In this study, RGB color features that are observed in the fruit region are extracted to obtain color information.

RGB color model. The RGB model, commonly known as the three primary colors, is made up of Through the use of the three brightness channel combinations, a wide range of colors can be manipulated. three-color channels: red, green, and blue. Through the use of the three brightness channel combinations, a wide range of colors can be manipulated. Channels often have a limited range, such as 0 to 255. There are typically 24 color photos utilized [28]. Additionally, eight shades of gray are used to represent each color component. The total number of colors can be expressed using three color components as follows: 288+8+8=224=16,777,216. In essence, all the colors that the human eye can see are contained in the color-coded representation of their appearance. The most popular color model is this one.

The RGB color space's coordinate system is seen in Figure 3 as a unit cube. Black is in the coordinate (0,0,0), white is in the coordinate (0,0,1), green is in the coordinate (0,1,0), and blue is in the location (0,0,0). (1,1,1). The cube's diagonal is covered in several shades of gray. All the hues are present in the cube; for example, white can be described as (1,1,1) and gray as (0.5, 0.5, 0.5).

Simple color representations include the RGB color space. However, it falls short of the visual experience and lacks an intuitive expression style [29-31]. It is impossible to comprehend the equivalent combination of colors based on the three main color components. Additionally, the three-color components must vary if you want to alter the image's color. Consequently, the finished product cannot accurately depict the color that the human eye actually sees [30]. When doing a statistical texture analysis, the characteristics of the texture are calculated using the statistical distribution of the observed combinations of intensities at certain locations within the image in relation to one another. In this work, LBP types of texture characteristics are investigated. Depending on how many intensity points (pixels) there are in each pair, statistics are categorized as first-order, second-order, and higher-order statistics.

Local binary pattern (LBP). The majority of texture models have an excessively high level of computational complexity. The local binary pattern operator (LBP) is a straightforward texturing model that the authors selected as a result [32]. The operator labels the pixels in an image block by setting the center value as the threshold for each pixel's neighborhood, and then interprets the output as a binary number (LBP code):

$\operatorname{LBP}\left(x_c, y_c\right)=\sum_{p=0}^{p-1} s\left(g_p-g_c\right) 2^p$             (3)

where, g_c corresponds to the value of the center pixel (x_c, y_c) and g_p represents the grey values of the P neighborhood pixels. The function s(x) is defined as follows:

$s(x)= \begin{cases}1 & \text { if } x>0 \\ 0 & \text { if } x \leq 0\end{cases}$       (4)

Uncertainty has been given to the definition of the h_texture in order to test for similarity between an image block texture and the backdrop texture.

$\begin{aligned} & h_{\text {texture }}\left(x_c, y_c\right)= \begin{cases}G_t\left(x_c, y_c\right) / G_B\left(x_c, y_c\right) & \text { if } G_t\left(x_c, y_c\right)<G_B\left(x_c, y_c\right) \\ 1 & \text { if } G_t\left(x_c, y_c\right)=G_B\left(x_c, y_c\right) \\ G_B\left(x_c, y_c\right) / G_t\left(x_c, y_c\right) & \text { if } G_t\left(x_c, y_c\right)=G_B\left(x_c, y_c\right)\end{cases} \end{aligned}$      (5)

where, $G_B\left(x_c, y_c\right)$ is the texture LBP of pixel $\left(x_c, y_c\right)$ in background and Gt $G_t\left(x_c, y_c\right)$ is the texture LBP of pixel $\left(x_c, y_c\right)$ in time $\mathrm{t}$ video frame. $h_{\text {texture }}$ is close to one if $G_B\left(x_c, y_c\right)$ and $G_t\left(x_c, y_c\right)$ are extremely comparable [33] show Figure 4 for example.

3.png

Figure 3. RGB color cube

4.png

Figure 4. Bilinear interpolation is used to estimate the values of nearby neighbors

3.3 ML classification method

The choice of classifier is a crucial step since the same collection of features may yield different results when used with various classification approaches. In this study, we classified a variety of fruits using ML and DL algorithms. In this work, the effectiveness of ML (KNN, DT, and SVM) and DL (AlexNet) approaches for classifying fruits was compared. We selected these ML and DL methods despite the fact that there are many more that have been explored in the literature since they are popular and widely accepted as being successful. The choice of classifier is a crucial step since the same collection of features may yield different results when used with various classification approaches. In this study, we classified a variety of fruits using ML and DL algorithms. In this work, the effectiveness of ML (KNN, DT, and SVM) and DL (AlexNet) approaches for classifying fruits was compared. We selected these ML and DL methods despite the fact that there are many more that have been explored in the literature since they are popular and widely accepted as being successful.

3.3.1 SVM

The SVMs are also called support vector networks [34]. The trained model is a non-probabilistic binary linear classifier since it categorizes newly sampled data into one of the categories. The data in SVM can be seen as points in space that are mapped to another space and then separated by a hyperplane. The new test data is then translated into the same space and classified according to the hyperplane. By utilizing a kernel, SVM can also be utilized for non-linear classification. High-dimensional feature spaces like kernel maps make separation simple [35, 36].

3.3.2 KNN

The KNN algorithm is a widely recognized statistical method for pattern recognition [36]. It is both simple and efficient. The classifier functions independently of the training dataset, and there is no complexity involved during the training phase. Nevertheless, the size of the training set significantly influences the computational complexity of the KNN algorithm. This approach is particularly effective for separating samples with numerous intersections or overlaps within the class domain, outperforming other methods [37].

3.3.3 DT

Instances are categorized using DT, which order instances based on the values of their features. A decision tree's branches each reflect a potential value for the node, which in turn represents a feature in an instance that needs to be categorized. Instances are grouped and sorted starting at the root node based on the feature values they contain. In data mining and ML, DT learning employs a decision tree as a prediction model to connect observations about an object to judgments about the value it should have. Classification trees or regression trees are suitable names for these tree models [38]. In order to evaluate the performance of decision trees while they are pruned using a validation set, decision tree classifiers frequently use post-pruning approaches. Any node may be deleted and assigned the training instances' most prevalent class [38].

In essence, the selection of SVM, KNN, and DT is based on their distinctive advantages: SVM stands out in precise binary classification and has the capability for non-linear separation; KNN proves to be efficient and proficient, especially in scenarios where class domains overlap; and DT offers a transparent hierarchical framework for decision-making relying on image features. The intent behind utilizing this combination is to harness the individual strengths of each method, thereby improving the overall accuracy and performance in the realm of fruit image recognition and classification.

3.4 DL classification method

In contrast to traditional machine learning, deep learning necessitates less manual feature engineering, as seen in algorithms like SVM and KNN. For instance, CNNs, a subtype of deep learning models, excel in the process of feature extraction. CNNs possess the ability to map input data through multiple layers, allowing them to learn from each layer individually, thereby enabling the extraction of meaningful features from large datasets. Deep learning classification models often incorporate convolutional, pooling, and fully connected layers [39]. When dealing with images of plant leaves, the convolutional layer primarily extracts image features. The intermediate layer is responsible for extracting intricate texture information and a portion of the semantic details, while the deep layer captures high-level semantic features, and the shallow layer retrieves edge and texture information. A max-pooling layer is employed to preserve crucial information in the image following the convolutional layer [40]. The high-level semantic features extracted by the feature extractor are categorized using a classifier composed of fully connected layers located at the model's end. In this study, the input image dimensions were set at 270×270×3, divided into several depth slices with a significant number of neurons in each slice. Think of square filters like 16×16, 9×9, or 5×5 convolution kernels as weights for these neurons, each related to a specific local region in the image from which it extracted features.

If we assume the input image size is denoted as W, the convolution kernel size as F, and the stride of the convolution kernel as S (typically S=2), padding P is used to fill the input image boundary, with P usually set to 0. The size of the image following convolution can be calculated as (W-F+2P)/S+1. Each output map feature employs convolutions to combine various input maps, and typically, Eq. (6) can be used to represent the result. Semantic information. The input image size for this investigation was set to 270×270×3. In the direction of depth, it was divided into numerous slices.

$x_j^l=f\left(\sum_{i \in M_j} x_i^{l-1} * k_{i j}^l+b_j^l\right)$        (6)

where, i represents the I layer, k_ij represents the convolutional kernel, b_j represents the bias, and M_j is a collection of input maps. It seems to suggest that when implementing CNNs in a more detailed or advanced manner, many people use the sigmoid activation function, a tanh function, or an additive bias. For instance, the unit's value at the place x, y in the (j-th) feature map and the (i-th) layer, denoted as $v_{i j}^{x y}$ is given in Eq. (7).

$\begin{aligned} & v_{i j}^{x y}=\operatorname{sigmoid}\left(b_{i j}+\sum_{p=0}^{P_i-1} \sum_{q=0}^{Q_j-1} w_{i j}^{p q} v_{(i-1)}^{(x+p)(y+q)}\right)\end{aligned}$        (7)

where, ${sigmoid (.)}$ is the sigmoid function $b_{i j}$ is the bias for the feature map, $P_i$ and $Q_j$ are the height and width of the kernel, and $w_{i j}^{p q}$ is the kernel weight value at the position (p, q) connected to the $(\mathrm{i}, \mathrm{j})$ layer. The parameters of CNNs, such as the bias $b_{i j}$ and the kernel weight $w_{i j}^{p q}$, are usually trained using unsupervised approaches. Different DL classification models have been developed for image classification problems. We applied AlexNet DL classification models in this research because AlexNet is a well-known CNN model used for fruit recognition and classification. Its deep layers and convolutional filters excel at capturing complex fruit features, making it effective at distinguishing different fruit types. This success in image classification and its capacity to learn hierarchical features contribute significantly to improving the accuracy and efficiency of fruit recognition systems. AlexNet, developed by Alex Krizhevsky, achieved a groundbreaking victory in the 2012 ImageNet Large Scale Visual Recognition Challenge, signifying a significant leap forward in computer vision. With a structure encompassing eight layers, featuring five convolutional layers and three fully connected layers, AlexNet serves as the foundational framework for contemporary deep neural networks. The convolutional layers, essential for extracting features and identifying objects, employ diverse filters, including sizes like 11×11 and 5×5, among others. Following each convolutional layer, max-pooling layers efficiently reduce spatial dimensions. The three fully-connected layers, each hosting 4096 neurons, play a vital role in object categorization, while the final SoftMax layer predicts object classes in the images. This architectural design, coupled with specified filter sizes, has become a cornerstone in the field of deep learning [41, 42]. The entire course of this study is depicted in Figure 5. We manually retrieved characteristics from the preprocessed fruit dataset in order to employ ML techniques to categorize fruits. It was unnecessary to manually extract features in this case because the DL classifier could do so automatically. The DL and ML networks, respectively, received input from the preprocessed images and the extracted features for training. We acquired the trained models after the training procedure was finished. The trained model was then used to classify the test dataset [42].

Essentially, the strengths of deep learning, exemplified by the proficiency of AlexNet, originate from its inherent ability to autonomously acquire intricate features, comprehend hierarchical abstractions, and provide heightened flexibility and scalability. This, in turn, leads to superior performance in image recognition when compared to conventional ML approaches.

5.png

Figure 5. Flowchart of fruits classification

3.5 Data set

The dataset utilized in this paper is the Fruit 360 dataset [43], comprising 131 categories of fruits and vegetables and encompassing a total of 90,483 images. The training dataset consists of 67,692 images, while the testing dataset contains 22,688 images, with each image featuring a single fruit or vegetable. All images within the dataset have been standardized to 100×100 pixels to ensure uniformity. To enhance classification accuracy, the fruits and vegetables were isolated from their backgrounds due to varying lighting conditions. To create an effective model with high accuracy, it is essential to maintain a balanced distribution of data in both the training and testing datasets. A comparison of the two datasets reveals a similar image distribution ratio across all categories, which can be verified by calculating the ratios for specific classes in both datasets. As illustrated in Figure 6, images of various fruits from 360 datasets.

6.png

Figure 6. Fruits classes in 360 datasets

3.6 Confusion matrix

The confusion matrix is a vital tool in ML and DL for evaluating the performance of classification model performance. It provides a comprehensive analysis of how a model's predictions compare to the actual labels derived from the ground truth. Although it may be used in multi-class settings as well, binary classification issues (two classes) are where it shines [44].

The confusion matrix consists of four essential components:

(1). True Positives (TP): In these cases, the model successfully predicted the positive class.

(2). True Negatives (TN): In each of these situations, the model properly predicted the negative class.

(3). False Positives (FP): Here, the model predicted the positive class when the actual class was negative, which was incorrect.

(4). False Negatives (FN): In these cases, the model predicted the erroneous class, which was negative whereas the true class was positive.

The Confusion Matrix is frequently depicted as a figure, roughly resembling this:

Calculating the proportion of samples that were correctly classified to all samples, as demonstrated in the example below, can be used to evaluate the classification models' accuracy.

Accuracy $=\frac{T P+T N}{T P+T N+F N+F P}$        (8)

To determine the accuracy for the TP divided by the total number of items with positive labels (TP plus FP added together), as stated in formula 9, High precision indicates that the model and categorization are producing more useful results.

Precision $=\frac{T P}{T P+F P}$       (9)

Calculating the recall by dividing the total number of components that genuinely belong to the positive class TP will allow you to determine the model's sensitivity (10).

Sensitivity $($ recall $)=\frac{T P}{T P+F N}$       (10)

The harmonic mean of sensitivity and precision is used to determine the F1-Score.

$F 1=2 * \frac{(\text { Precision } * \text { sensitivity) }}{\text { (Precision }+ \text { sensitivity })}$        (11)

Data scientists and ML practitioners can assess how well their models are working and make educated decisions regarding model modification and deployment using the Confusion Matrix and related metrics [45].