Early Terrain Identification for Mobile Robots Using Inertial Measurement Sensors and Machine Learning Techniques

Numerous techniques have been suggested in scholarly works for distinguishing the surface over which a vehicle or robot moves. Various sensor principles are employed to aid in identifying the physical features [11], which provide valuable terrain data for the vehicle's operation. The methods outlined in the literature can be classified into IMU-based identification, tactile sensor-based identification, image-based identification, and acoustic-based identification [12].

2.1 IMU-based identification

Surface identification using IMUs is widely used. IMUs are strategically placed and equipped with multiple sensors to measure acceleration and angular velocity. Typically, the IMU is affixed to the vehicle, such as on the chassis. Algorithms are then utilized to analyze the sensor data, determine the difference in linear and angular acceleration, and assign a corresponding surface label to the feature set. Nampoothiri et al. [13] present a comprehensive survey of advanced algorithms used for terrain identification, focusing on deep learning and fuzzy logic methods. Tick et al. [6] introduced a novel approach that utilized a classifier for surface identification, achieving an accuracy of approximately 90% over a 20-minute drive. The measurements were taken at various velocities, enhancing the algorithm's reliability. Before the experiment, the IMU data was filtered to address significant noise and gyro sensor drift, often using a Kalman filter to refine the data. In one of the studies,

Csík et al. [14] proposed a new approach for terrain classification using a multi-layered perceptron (MLP) neural net as a classifier. A prototype measurement system gathered data from inertial sensors across diverse outdoor terrains. Both the accelerometer and gyroscope data were independently and jointly tested with different processing window sizes. The results indicate that the proposed system can achieve a classification efficiency of over 99%. Shin et al. [15], in one of the studies, used IMU sensor data and a Gaussian mixture model to develop a system for terrain identification. In their study, Knuth and Groves [16] used IMU to record acceleration, angular rate, and magnetic field data. A total of 44 features are extracted from the magnitude of the data, including time domain and frequency domain features. Five classifiers were trained and tested on these features to classify terrain types. In another study, Thavitchasri et al. [17] used IMU sensor data to predict surface type to facilitate improved navigation for autonomous tractors. IMU sensor data was recorded for seven different surface types, and various ML algorithms like logistic regression, SVM, random forest, AdaBoost, and XGBoost were used to classify surface types.

2.2 Tactile sensor-based identification

In this approach, an IMU is also employed to identify the surface by positioning the IMU on the rod attached to the vehicle's rear end, making direct contact with the surface. This direct contact facilitates the capture of characteristic irregularities more effectively in the sensor data. Tactile sensing is essential, especially when optical sensors cannot directly observe surface properties, for instance, in fog and dust scenarios [18]. Mason et al. [19] presented a study on the use of acoustic tactile sensors to enhance the identification of terrain materials, types, and structures for mobile robots. In a study by Giguere and Dudek [20], ten different surfaces were examined with variable time windows achieving the highest accuracy of 94.6%, respectively. These experiments were conducted at a constant speed. So, there is a need for further validation at the robot's varying speeds. Moreover, it's essential to note that in practical applications, using a rigid rod to contact the ground is not feasible. This approach introduces complexities in the vehicle's kinematics and requires additional design considerations, which are not currently addressed in the kinematic model. In another research, Nagy et al. [21] used tactile sensor data and machine learning models for road quality classification. Accelerometer and gyroscope sensors were used to record the data. Further machine learning models like decision tree were used to classify road quality based on features obtained using power spectrum and principal component analysis.

2.3 Image-based identification

Numerous methods in literature used the most popular convolutional neural networks (CNN) for image-based surface identification [22]. The difficulty with such methods lies in generating a sufficient number of images for proper training of the neural networks. A benefit of such an approach is its capability to identify the surface even when the robot is not in motion. Wang et al. [23] present a deep learning and support vector machine-based combined approach to identify types of terrains for the robots. The algorithms were trained using a dataset comprising 30,000 images representing six distinct types of terrain. This method achieved an accuracy of up to 87%. Demirtaş et al. [24] introduced a method for classifying indoor and outdoor surfaces based on images. Initially, they created a database of 2081 images representing various surfaces like carpets, tiles, and wood. The surfaces above were effectively categorized utilizing a robust deep-learning model, resulting in an outstanding accuracy rate of 99.52%. Additionally, the authors expressed a high level of confidence in asserting that their model offers expedited loading and decreased processing durations compared to other models documented in the existing literature. In another study, the authors combined vibration and image texture data for a terrain identification task [25]. Time and frequency domain features of vibration signal and Gray-level co-occurrence matrix feature from the image are extracted, and a weighted k-nearest neighbor classifier is used further to classify different terrains.

2.4 Acoustic-based identification

The utilization of this technique for surface identification is uncommon. Acoustic-based surface identification is advantageous for mobile robots operating outdoors, where conventional visual or tactile sensing techniques may encounter limitations or inconsistencies. These methods involve capturing and analyzing acoustic signals generated during the interaction between a robot and various surfaces. In their work, Libby and Stentz [26] explore the application of sound data as an innovative sensing modality for categorizing different surfaces on which a robot traverses in an outdoor setting.

Acoustic recordings were acquired by mounting them on a mobile robot that navigated diverse outdoor terrains. The measurements were taken for different velocities and features like spectral flux and rate of energy change, which were considered in the classification model building. Subsequently, the data is assigned with labels and used to train the multi-class classifier. This trained classifier can then discern between distinct interactions solely based on acoustic data and is subsequently used to classify five different surfaces. Overall, they achieved a 92% true positive rate. Valada et al. [27] presented a standard method for categorizing audio sequences, utilizing spectrogram extraction from recorded audio data for classification with a CNN. Their dataset spans 2.15 hours and encompasses nine distinct surfaces, with recordings conducted at various vehicle speeds. They report an impressive overall classification accuracy of 98.5%. However, it is worth mentioning that classification accuracy notably declines when the vehicle speed falls below 1 m/s.

This section focuses on the experimentation carried out using the proposed system, the results obtained for different machine learning methods, their performance comparison, and the discussion about the real-time use of the proposed system.

The robot has IMU sensors to record movement readings and create a dataset. The following three parameters are measured using the IMU sensors [32].

Orientation: Four values for attitude quaternion channels represent the scalar and vector values.

Angular rate: Comprises three orthogonal IMU coordinate axes X, Y, and Z.

Acceleration: Three force values corresponding to three orthogonal IMU coordinate axes X, Y, and Z.

3.png

Figure 3. Density plot for linear acceleration in X direction

After data pre-processing and initial analysis, it was observed that a few features were not linearly separable. Figure 3 shows the density plot for linear acceleration in the X-direction, measured using the IMU sensors. This figure shows that the data is not linearly separable and is almost centered at zero. So, to overcome this problem, we have extracted the 12 statistical features from the dataset. These features are then used for the proposed system's implementation and experimental evaluation. Feature extraction is performed in the time domain and aims to acquire a concise and straightforward representation of the reading from the IMU sensors. Here, we have extracted 12 features, including mean, standard deviation, maximum and minimum values, median absolute deviation, mean absolute change, absolute minimum, absolute maximum, absolute average, quantile, skewness, and kurtosis. These derived features represent various statistical characteristics and insights into the nature of the motion being measured.

The various machine learning algorithms are then implemented using the derived features, and their performance is evaluated using different metrics such as accuracy, precision, recall, and F1-score, as shown in Eqs. (1)-(4), respectively.

${Accuracy}=\frac{T P+T N}{T P+F P+F N+T N}$           (1)

${Precision} (P)=\frac{T P}{T P+F P}$              (2)

${Recall}(R)=\frac{T P}{T P+F N}$              (3)

$F 1- {score} =\frac{2 \times P \times R}{P+R}$                (4)

where, TP is true positives, TN is true negatives, FP is false positives and FN is false negatives.

Thus, we have finally selected LGBM to implement the system. Extensive experimentation is carried out using the LGBM classifier. The classification report of the LGBM classifier is shown in Table 1.

Table 1. Classification report for LGBM classifier

Table 2. Comparative analysis of the performance of different classifiers for terrain identification

We have implemented and evaluated 11 machine-learning algorithms on the dataset for terrain classification. Table 2 presents comparative results for various machine learning models. The LGBM classifier outperforms all other machine learning algorithms. It is a fast, distributed, high-performance gradient-boosting framework constructed using decision tree classifiers. It uses a gradient-boosting framework to construct the ensemble of weak learners sequentially such that a new weak learner is constructed to reduce the classification errors made by the previous learner. Unlike the traditional gradient boosting techniques, it uses a histogram-based algorithm to select the best feature to split at the node of the decision tree. Thus, its ability to handle large datasets with higher dimensional feature sets, speed, and memory efficiency makes it the best-performing and most popular in the literature.

From experimental evaluation, it is clear that the proposed method achieved an overall classification accuracy of 91%. Table 1 also shows the precision, recall, and F1-score [33], along with the support for all the nine classes in the dataset. For most of the classes, these values are greater than 90%. This indicates a high-performance level and demonstrates the proposed approach's effectiveness in predicting the terrain on which the robot is moving. The achieved accuracy highlights the potential of our model in mobile robot terrain classification.

The robot's speed impacts the accuracy of terrain classification. This impact is due to how sensor data is collected and processed. At higher speeds, there is increased noise in the IMU sensor’s data. This increases the fuzziness in the distinct patterns associated with different terrains and reduces the resolution of the terrain's features. Lower speed captures the stable data and better resolution of terrain-specific features. Also, speed affects how the robot wheels interact with the terrain. On slippery surfaces (e.g., tiles), higher speed might cause skidding, which could distort classification patterns. In this study, the impact of the robot's speed on terrain classification accuracy was not explicitly considered. The primary objective was to evaluate the system's ability to distinguish between nine distinct terrains under consistent operational conditions. While speed influences sensor readings and classification performance, it was assumed the robot would operate within a moderate and constant speed range appropriate for a typical surface and environment.

4.png

Figure 4. Normalized confusion matrix of LGBM classifier

Figure 4 shows the normalized confusion matrix for the LGBM classifier. The rows in the confusion matrix represent the instances of the actual class, that is, the true floor type, and the columns represent the surface types predicted by the classifier. Consequently, the diagonal values represent correctly predicted classes, that is, the accuracy of the classifier in predicting the instances of a particular class. On the other hand, the off-diagonal values represent the degree of incorrectly classified instances.

The receiver operating characteristic (ROC) and the area under the ROC curve (ROC-AUC) are important metrics for evaluating the classification model [34]. The x- and y-axes represent false positive rate (FPR) and true positive rate (TPR) defined in Eq. (5) and Eq. (6), respectively. The false positives (FP) represent the number of negative instances wrongly classified as positive, true negatives (TN) represent the number of instances correctly classified as positive, and false negatives (FN) represent the number of positive instances wrongly classified as negative.

FPR=FP/(FP+TN)                (5)

TPR=TP/(TP+FN)                (6)

The ROC curve and ROC-AUC for the classifier are shown in Figure 5. For almost all classes, the ROC-AUC is above 0.94. This indicates that the classifier has excellent discriminatory power and performs very well in distinguishing the positive and negative classes in the classification problem. As ROC-AUC values are close to 1, this specifies that the classifier's predictions are highly consistent with the true class labels and the model is making accurate predictions and is reliable for predicting the terrain type on which the robot is moving.

5.png

Figure 5. ROC curve for LGBM classifier

The proposed system has two limitations. First, the terrain classification system may struggle to obtain clear, consistent data if the robot's sensors are blocked or misaligned due to physical objects. This could lead to misclassifications, as the terrain data might not represent the actual surface but rather a disturbed sensor reading caused by the obstruction. Second, in the proposed system the impact of the robot's speed on terrain classification accuracy was not explicitly considered.

Our analysis revealed that misclassified samples are complex and of poor quality, making them hard to classify. To address this, we plan to improve data quality and diversity by incorporating more complex samples. The model will be retrained and fine-tuned with these enhancements. This approach aims to make the model more robust for challenging cases.