Improving the Approval Process for Durum Wheat Grain Quality in Algeria Using Computer Vision and Machine Learning

Our research methodology consists of a sequence of modules, beginning with data acquisition, followed by dataset refinement, and culminating in the presentation of model development. Finally, we conclude with a thorough examination of the results and a comprehensive discussion to assess the performance of our model, comparing its findings with related research.

3.1 Data collection

We obtained samples of wheat seed and impurities from two institutions: the Seed Batch Control Centre (CCLS) and the National Crop Control Centre (CNCC). The CNCC samples included four different varieties of durum wheat, namely Vitron, Bousellam, GTA, and Oued El-Bared. Meanwhile, the CCLS samples included three other cereal species, including bread wheat, barley, triticale, and oats. The sample collection also included two types of impurities, mitadine and lean grains. This approach resulted in a total of nine different classes.

The process of capturing images of these grains followed a specific procedure. Grains from each class were photographed using a scanner (brother dcp-t310) and organized into folders corresponding to their respective classes. A black background was used during the scanning process to minimise the effect of shadows. The different types of grain were positioned on an A4 paper before scanning. The scanner's image resolution was configured at 600 dots per inch (dpi), with lighting conditions and contrast both set to 50%. Additionally, the scanning mode was selected as colour with 24 bits per pixel.

3.2 Dataset preparation

The performance of a machine learning (ML) model largely depends on the quality of the dataset used for its training. Several fundamental conditions, such as the adequate dataset size, class balance, and data quality, play a crucial role in determining this quality. Figure 1 illustrates the different steps we took to prepare a high-quality dataset, accompanied by a detailed explanation of each step that follows.

3.2.1 Image segmentation

Image segmentation plays a central role in the identification and classification of various objects within an image, with a particular focus on its utility in the classification of wheat grains. This task can be performed either by classical techniques such as thresholding, edge detection, region-based methods and watershed analysis [31, 32], or by approaches based on artificial intelligence (AI) [33]. The latter category includes unsupervised clustering methods such as k-means and mean shift, and deep learning techniques such as convolutional neural networks (CNN), region-based CNNs (R- CNN), the fastest, fastest R-CNNs. R-CNN, R-CNN mask and object detection.

According to the findings from the literature review, classical methods hold a prominent position in grain classification applications. Techniques such as Otsu's thresholding and the Canny edge detector are widely favoured due to their simplicity and minimal computation time. It's important to highlight that their effectiveness depends largely on meticulous consideration of the environmental conditions to attain optimal results. These achievements are commonly realized in industrial settings where an industrial camera is used within a controlled processing setup, to ensure a stable image background.

In such well-controlled environments, the use of classical image segmentation methods proves to be not only sufficient, but also optimal to achieve the desired objectives. However, it's important to recognize that challenges may arise in scenarios where grains are in close contact, particularly during the scanning of wheat grains that contain different species or impurities. In such complex situations, classical approaches are often limited and may not provide the desired results.

1.png

Figure 1. Dataset preparation steps

Therefore, the image segmentation process in our system involves four sequential steps, namely image pre-processing, Otsu's thresholding, Canny edge detection, and image post-processing. These deliberate steps are strategically designed to improve the quality of the segmentation results while effectively addressing potential challenges posed by situations involving multiple grain contacts.

Once the image segmentation has been successfully completed, the next step is to precisely crop the segmented image into discrete individual grain images. This precise partitioning/segmentation/separation allows each grain to be isolated and analysed individually. This meticulous process enables us to curate a dataset that is well suited to efficient classification algorithms and the extraction of key characteristics related to cultivated grains.

Step 1: Image Pre-processing

Prior to the actual segmentation process, we implemented three distinct methods. The "Grayscale" conversion technique was applied to transform the colour image into grayscale. Following this, we employed the Adaptive Median Filter [34, 35] to effectively eliminate salt and pepper noise from the image. Additionally, the Gaussian Filter [36] was utilized to efficiently attenuate high-frequency noise and impart smoothness to the grayscale image.

Step 2: Image Segmentation Combination with Two Methods

In this step, there is a combination of two image segmentation methods: Otsu’s thresholding [37] and Canny edge detection [37, 38]. To improve the accuracy and quality of the overall image segmentation, we use a bitwise OR operator to combine the results of both methods, see (Figure 1).

Step 3: Image post-processing

After combining the two segmented images, the resulting image undergoes a sequence of post-processing procedures. In our approach (as shown in Figure 2), we use two different morphological operations: erosion and dilation.

Erosion, a morphological operation, causes the foreground regions of an image's boundaries to contract or erode. Conversely, dilation, the opposite of erosion, expands the boundaries of an image's objects by scanning the image and adding pixels to the edges of the object.

2.png

Figure 2. Morphological operations on an image contain multiple touching grains

3.2.2 Contour detection and image cropping

The next step after segmentation is contour detection. Contours are continuous curves that follow the edges of objects in an image. Once the contours are detected, the image is cropped to isolate individual grains. The cropping process uses OpenCV's boundingRect function to obtain the coordinates of the bounding rectangle for each contour. Criteria such as the minimum contour area are used to filter out undesirable contours.

Then, in order to standardize the input for classification while preserving the distinct characteristics of different grains, the cropped grain images are centred within fixed-size black images. A custom function is implemented using Python and OpenCV to generate fixed-size black images, calculate optimal positions for placing the grains at the center, and accurately position the grains within the fixed-size images. (Figure 1).

3.2.3 Dataset equalization and partitioning

As a result of the image segmentation process and subsequent image cropping, we obtained individual images each with a single grain, from the scanned images. However, this process resulted in an unbalanced dataset where the number of images in each class varied significantly. For example, there were 4500 images of barley, 2051 images of oats, but only 750 images of Mitadine. It was noted that this case of unbalanced data leads to the problem of a poorly trained model and therefore the 'Accuracy' evaluation metric can be misleading because it does not reflect the training state of the model.

To avoid this problem, we need to ensure a balanced representation of classes within our dataset. To do this, we followed a systematic approach. Instead of using traditional methods such as data augmentation or resampling, we divided the dataset into three separate groups: one for species, one for impurities, and one for varieties. Each of these groups was meticulously organized to contain an equal number of samples for each class. This deliberate distribution allows us to maintain uniformity in the number of images across all classes, making it easier to train and evaluate fair and efficient models.

Subsequently, we collectively allocated 80% of each group to the training and validation subsets. The remaining 20% were reserved for testing and evaluation, where the performance of the models would be comprehensively evaluated. For clarity, let’s break down the breakdown of each group.

Group 01: Contains a total of 10,000 images of 5 different species. We assigned 8,000 images for training and validation (80%/20%), while the remaining 2,000 images are designated for testing as shown in the Table 2.

Table 2. Dataset partitioning of species

Group 02: Contains a total of 4,200 images of 4 different Durum wheat varieties. We assigned 3,360 images for training and validation (80%/20%), while the remaining 840 images are designated for testing as shown in the Table 3.

Table 3. Dataset partitioning of durum wheat varieties

Group 03: Contains a total of 2,860 images of 4 different impurities. We assigned 2,288 images for training and validation (80%/20%), while the remaining 572 images are designated for testing as shown in the Table 4. Note that we have included durum wheat in this category to enable the model to distinguish among the three impurity types and durum wheat grains.

Table 4. Dataset partitioning of durum wheat impurities

In summary, we meticulously balanced our dataset by creating three distinct groups for species, impurities, and varieties. Each group maintained an equal distribution of samples across all classes to ensure fairness in model training and evaluation. Subsequently, we divided each group into 80% training and validation data, reserving 20% for comprehensive testing and performance assessment. This strategic approach lays the foundation for robust model training and accurate quality assessment of wheat grains.

3.3 Model construction

The construction of our model involves several stages, each of which requires the application of various methods in order to achieve an optimal classification model with a high level of accuracy for durum wheat grains and their different varieties and impurities.

We first discuss the methods used to extract descriptors from the images in our dataset, followed by a review of the normalisation techniques applied to these descriptors.

We then explain the adaptation of the Support Vector Machine (SVM) model for multiclass classification. We present the specific methods used to find the optimal hyperparameters of the model. During the training phase, we also explore different combinations of the extracted descriptors to determine the most effective combination among them.

3.3.1 Image descriptors

In the context of using machine learning and computer vision, descriptors involve capturing and quantifying significant information from the image of interest, which is in our case is the image of raw grains and impurities. These image descriptors play a key role in the ongoing image classification tasks that we are undertaking in our research. We have classified the image descriptors into three distinct categories: morphological descriptors, color descriptors, and texture descriptors [39].

For the morphological descriptors, we extracted eight essential features, including area, perimeter, circularity, aspect ratio, as well as shape factor1(sf1), shape factor2 (sf2), shape factor3(sf3), and shape factor4 (SF4). These descriptors provide insight into the size, shape, and geo-characteristics of the grains.

In terms of colour features, we carried out comprehensive assessments across different colour spaces and channels, such as RGB, HSV, XYZ, LAB, and YCbCr. In our research, we computed statistical metrics such as mean, standard deviation, skewness, kurtosis, and entropy for each of these different colour channels. This approach results in a total of 90 colour-related features.

In terms of textures, we calculate 102 features using both the Gray-Level Run Length Matrix (GLRM) and Gray-Level Co-occurrence Matrix (GLCM) algorithms [40, 41]. For the GLRM, we consider multiple distance and angle configurations, including distances of 1, 2, 3, and 4 units, and angles of 0, π/4, π/2, and 3π/4 radians, effectively capturing texture information across various spatial relationships. Additionally, for the GLCM, we focus on distances of 1 unit and angles of 0, π/4, π/2, and 3π/4 radians, ensuring a comprehensive analysis of texture patterns in our wheat seed image dataset.

The computed image descriptors, comprising a total of 200 features including 102 texture features, 8 morphological features, and 90 colour features, enable comprehensive characterization of the raw grain images. This combined feature set contributes to robust and discriminative image classification.

3.3.2 Features normalisation

In our relentless pursuit of precision in classifying different wheat grains, we’ve ventured into the realm of feature normalization. This crucial step, often underestimated, plays a key role in ensuring that our machine learning model can make the most of the diverse range of image descriptors extracted from our wheat grains images.

First, we explored the Min/Max normalization technique also known as Min-Max Scaling. Its main advantage lies in its ability to place all features on a uniform scale, neatly confined between the values of 0 and 1. This is particularly beneficial when dealing with descriptors that encompass different units and magnitudes. The Max/Min Normalization technique ensures that no feature dominates the learning process due to its inherent scale. Despite these advantages, Min/Max normalization also has its limitations: while it's successful at rescaling features within a certain range, it can be sensitive to outliers. Outliers can have a disproportionate impact on the scaling process, potentially leading to a loss of information or distortions/bias in the data [42].

In our search for robustness in the face of outliers, we turned our attention to Robust Normalization. Robust Normalization takes refuge in the shelter of the median and interquartile range (IQR). In the presence of outliers, which can be a turbulent force in real-world datasets, Robust Normalization stands as a steadfast guardian. It shields our model from the disturbances caused by extreme values. While robust Normalization is effective in reducing the influence of outliers, it can lead to non-zero means and varying standard deviations across features, which may impact model convergence and interpretation.

In contrast, Standardization Normalization, also called Z-Score, ensures a mean of 0 and a standard deviation of 1 for all features, promoting consistent scaling and aiding model convergence. This uniformity in feature scaling allows for fair comparisons and mitigates the dominance of any single feature. Consequently, in our dataset where consistency, robustness, and interpretability are paramount, Z-Score Normalization emerges as the optimal choice, overcoming the limitations of Robust Normalization and improving the precision and stability of our classification model.

3.3.3 Support Vector Machine (SVM) model

Machine learning is a discipline within the realm of artificial intelligence that focuses on developing algorithms and statistical models where computer systems learn and improve their task performance by analysing and adapting to data patterns without being explicitly programmed for those tasks [43]. This technology has been applied in computer vision, which aims to give machines the ability to interpret and understand visual information extracted from images or videos, emulating the human capacity for visual perception and interception [44].

In the overarching context, we look at the application of machine learning and computer vision to the complex task of wheat seed species classification. This task involves the systematic categorization and differentiation of various wheat seed species based on their distinctive visual characteristics.

Among the many machine learning algorithms available, we selected the Support Vector Machine (SVM) [39] as the cornerstone of our classification machine learning model. SVM is known for its proficiency in handling high-dimensional data, which makes it particularly well-suited for the multi-feature descriptors derived from our seeds and impurities images. However, it's important to note that SVM is inherently designed for binary classification tasks, where it seeks to find an optimal hyperplane in feature space to separate two distinct classes.

To harness the full potential of SVM in our multi-class classification challenge, we've strategically employed the One-vs-All (OvA) approach [45]. This technique transforms our multi-class problem into a series of binary classification subproblems, where each class is treated as a separate binary classification task against all other classes.

In the following section, we delve into the process of training our SVM model, using the standardized normalization features and employing the one-vs-all approach to address multi-class classification.

3.3.4 Fine tuning SVM model and training

In almost every machine learning project, we train different models on the dataset and select the one with the best performance. We found that an important factor in the performance of these models is their hyperparameters. Once we set appropriate values for these hyperparameters, the performance of the model can improve significantly. Note that there is no way to know the best values for the hyperparameters in advance, so ideally, we should try all possible values to find the optimal ones. Doing this manually could take a lot of time and resources, so we use the grid search method to automate hyperparameter tuning. In this article, we have used the grid search technique, which is based on the k-fold cross-validation approach.

K-Fold Cross validation: is a fundamental technique widely used to evaluate the performance of machine learning and deep learning models by partitioning the dataset into ‘k’ subsets (folds) of equal size. The model is trained and evaluated ‘k’ times, with each fold serving as a validation set once, while the remaining folds are used for training. This process ensures that each data point is part of the validation set exactly once, allowing for a comprehensive evaluation of the model’s performance [46].

The choice of K should strike a balance between the representativeness of each fold and computational efficiency. In our research, we carefully considered this balance and decided to use K = 5 as the number of folds for our experiments. This choice ensured a meaningful evaluation of model performance while making efficient use of our computational resources.

Grid Search: is the process of performing hyperparameter tuning in order to determine the optimal values for a given model [47, 48]. We pass predefined values for hyperparameters to the grid Search function. We do this by defining a dictionary in which we mention a particular hyperparameter along with the values it can take. The Grid Search method tries all combinations of the values passed in the dictionary and evaluates the model for each combination using the Cross-Validation method. Hence after using this function, we get the accuracy/loss for each combination of hyperparameters and we can choose the one with the best performance. C and kernels are some of the hyperparameters of an SVM model that we are interested in this article.

kernel: It represents the type of kernel to use (RBF is the default). Choosing the right kernel is crucial, and each kernel has its own advantages. For example, in the case where the dataset is linearly separable, it is preferable to use the linear kernel. In the case where the dataset has several classes, the best choice is the RBF (Radial Basis Function) because it supports projection into high-dimensional spaces. Note that this choice is more questionable in terms of computing power.

3.png

Figure 3. Scheme of 5-Fold Cross validation with SVM for optimal hyperparameters