A Novel Framework for LiDAR Data Classification Using PCA and Ensemble Learning on Mobile Robots

The emergence of autonomous indoor navigation for robot operations has enhanced its capabilities in navigating areas such as hospitals, factories, smart buildings, and complex environments [1]. The essence of the robot's autonomy is the ability to classify its environment, such as corridors, doorways, rooms, and halls, based on sensory data [2]. For such tasks, LiDAR sensor is utilized (light detection and ranging) to provide the robot with a spatial, 3D/2D illustration-rich dataset of the target environment [3]. However, the complex and large datasets produced by LiDAR pose significant challenges, particularly when scaling up the target application [4]. LiDAR sensors generate a significant amount of data each frame; for instance, 2D LiDAR scans, which cover a 360° space, can yield up to 360 data points per frame [2]. On the other hand, 3D LiDAR can generate even more data, reaching thousands of points per frame [4]. Therefore, in applications that use LiDAR in large environments with complex details, the size of the dataset can exponentially grow as the number of features and objects increases [3].

Mapping large indoor environments, such as hospitals, can result in an abundance of features in datasets [5]. This behavior can exacerbate the issue of dimensionality, increase the computational costs associated with data processing, and complicate real-time classification [6]. The integration of machine learning with LiDAR technology has demonstrated its effectiveness in surmounting the challenges presented by high-dimensional LiDAR datasets and presents promising opportunities for improving localization capabilities [7]. Machine learning methods can powerfully process the massive amounts of data generated by LiDAR sensors, allowing for precise and real-time classification of different environments [8]. This ability is essential for applications such as indoor navigation, where accurate detection of elements is critical [9].

By merging LiDAR's rich datasets with various machine learning methods, robots can attain superior performance in complicated tasks, promoting scalability, decreased computational costs, and improved adaptability through different environments [10, 11]. We propose a novel approach to address these challenges in LiDAR technology by leveraging principal component analysis PCA in conjunction with ensemble machine learning methods. PCA greatly reduces the number of dimensions in LiDAR data by taking out the most important features and getting rid of unnecessary data. This process makes the dataset smaller and more representative. This approach effectively addresses the issue of dimensionality, leading to a significant reduction in runtime. On the other hand, the ensemble methods maintain and enhance the classification accuracy.

The main contributions of this paper are summarised as follows:

1. Dimensionality Reduction for Scalability:

We present a PCA-based preprocessing pipeline that significantly reduces the dimensionality of LiDAR data while preserving essential structural features.

2. Enhanced Classification Accuracy:

The proposed model delivered a high classification accuracy of 99.20% across various indoor environments by integrating PCA with ensemble machine learning methods.

3. Real-Time Efficiency:

The proposed approach offers the reduction of computational cost and a real-time deployment solution on resource-constrained platforms.

This work's significance extends beyond individual applications to diverse tasks such as robotic surgery and autonomous driving that require an efficient and scalable real-time processing solution. The proposed solution can manage a vast LiDAR dataset with rapid and accurate classification, which can pave the way for real-time, efficient, and responsive robotic systems.

We organise the paper as follows: Section 2 discusses the related work on LiDAR-based classification accuracy and runtime overhead management. Section 3 illustrates the proposed methodology we implement. Section 4 details the experimental results with a focus on the accuracy improvements and computational reduction. Section 5 delves into the implications of the results for real-world applications. Section 6 brings the work to a close by exploring potential avenues for future research.

3.1 Methods and materials

The dataset used in this research is the LidarDataFrames dataset, which is publicly available on Kaggle [33]. The dataset is introduced as a wide collection of LiDAR data derived from a mobile robot equipped with an RPLIDAR-A1 Laser Range Scanner. The dataset contains signal scans from four different environments: rooms, doorways, corridors, and halls, with each environment providing 360 different LiDAR signals. 360 input features describe each instance in the dataset, indicating the diverse LiDAR signal measurements, while a target feature reveals the type of environment. The LidarDataFrames dataset offers a complete view of the complex nature of LiDAR data, making it a valuable resource for developing robust prediction solutions because it provides broad information on factors associated with LiDAR-based localization [27].

The dataset is well-structured with a near-balanced class distribution, confirming that each environmental type is represented by a similar number of instances and minimizing the risk of classification bias. According to the dataset's paper [27], it has already undergone essential data cleaning and preparation, involving the elimination of missing or noise values. Hence, no further data preprocessing phases were necessary before starting model development. This cleaned and balanced dataset serves as the foundation for feature extraction and model training proposed in this paper.

3.2 Experimental design and workflow

In this study, we propose a predictive model for detecting the type of environment based on LiDAR data. We deployed three different experimental setups to evaluate the performance of several ensemble classifiers and dimensionality reduction techniques. Each experiment employed different predictive model strategies to evaluate the impact of these methods on the classification accuracy. The overall experimental design and workflow of the proposed LiDAR classification system is illustrated in Figure 1, which summarizes the main processing stages used in the proposed method.

Figure 1. The detailed experimental design and workflow of the proposed methodology

3.2.1 Experiment 1: Baseline ensemble classifier deployment

In the first experiment, we applied eight different ensemble classifiers to the dataset in its original form (360 features) without using any dimensionality reduction techniques. These classifiers include:

• Random Forest: Is a powerful bagging-based ensemble algorithm that builds multiple decision trees and aggregates their predictions to enhance accuracy and decrease overfitting. People commonly use it due to its ability to handle high-dimensional data and noise.
• AdaBoost: A boosting algorithm that integrates weak classifiers in a sequential manner, iteratively adjusting the weights of misclassified cases.
• Gradient Boosting: Another boosting method that constructs decision trees sequentially, where each decision tree aims to resolve the errors of the previous decision tree. It reduces an indicated loss function through gradient descent, making it greatly efficient for classification tasks.
• Extreme Gradient Boosting (XGBoost): A highly efficient and scalable extension of the Gradient Boosting technique, implemented for optimized speed and performance. It presents advanced techniques such as regularization and tree pruning, which help avoid overfitting and enhance model accuracy.
• Categorical Boosting (CatBoost): A gradient boosting method specially implemented for categorical data, applying effective handling of categorical features without involving broad preprocessing steps.
• Extra Trees: An ensemble learning algorithm that builds multiple decision trees, randomly selecting both the data objects and splits for each tree. Extra Trees presents more randomness by selecting split thresholds randomly, which can enhance computational efficiency and avoid overfitting in some situations.
• Light Gradient-Boosting Machine (LightGBM): A gradient boosting algorithm implemented for speed and efficiency, especially on big datasets. It attains its superior performance by using methods such as histogram-based learning to decrease computational costs and improve scalability.
• The Balanced Bagging Classifier: Is an ensemble learning technique that utilizes bagging and balanced resampling to tackle class imbalance issues in datasets, ensuring equal representation of each class in the training set. This technique trains multiple base estimators on the balanced training set and combines their predictions, enhancing the performance of minority classes and decreasing bias to majority classes.

In this experiment, each of these techniques was trained and evaluated on the original dataset to compare their performance in classifying the environment type based on the LiDAR data.

Hyperparameter tuning was performed to optimize each model’s performance. The training hyperparameters for the implemented classifiers are presented in Table 1. We relied completely on the default hyperparameters specified by each classifier’s official implementation. Our objective was to assess the effect of PCA-based dimensionality reduction across a diverse set of ensemble algorithms under fair and consistent conditions, rather than to perform extensive hyperparameter optimization. Using default settings ensures a widely accepted, reproducible, and unbiased baseline across all classifiers.

Table 1. Default hyperparameters used for each classifier

3.2.2 Experiment 2: Deployment of ensemble classifier and PCA – full component retention

The second experiment presented dimensionality reduction through PCA. PCA was preferred because LiDAR signals present high dimensionality and strong correlations across neighboring beams, which lead to redundancy and increase computational cost. PCA transforms the original feature space into a set of orthogonal components sorted by variance, allowing the model to retain the most informative structure of the LiDAR data while removing noise-dominated directions.

PCA was applied to the dataset to reduce input variable redundancy, retaining the most significant variance in the dataset. PCA was implemented on the 360 variables, and all principal components were preserved during the transformation. After performing PCA, the obtained principal component matrix was used to train the same set of ensemble classifiers used in the first experiment. This step was added to see what effect orthogonal feature space transformation had on the performance of the classifier and to see if using PCA would make it work better or be more accurate.

3.2.3 Experiment 3: Deployment of ensemble classifier and PCA (reduced component retention)

The third experiment extended the second experiment by reducing the number of principal components used for model training. Several component sets were tested to find the best number of dimensions that would balance the need for fast computing with keeping all the original data's variance. We trained and tested the same set of ensemble classifiers on the reduced PCA-transformed datasets. This experiment aimed to explore how the decrease in the number of principal components affects both classification accuracy and computational efficiency. Here's how we achieved the integration of PCA and ensemble learning: PCA acted as the feature extraction phase, while the ensemble classifiers were implemented as the predictive models on the PCA-transformed feature space. This integration is intended to enhance classifier generalization by decreasing the risk of overfitting associated with high-dimensional datasets.

3.3 Data splitting and evaluation

For all experiments, the dataset was divided into a training set (70%) and a test set (30%). The training set consisted of 287 objects, while the test set contained 124 objects. Reproducibility was guaranteed by choosing random seeds during data splitting and model training.

The proposed model performance was tested and evaluated using several standard classification metrics, including:

• Accuracy: The percentage of correct predictions made out of all instances. The accuracy formula is illustrated below:

$Accuracy=~\frac{TP+TN}{TP+TN+FN+FP}$

where TP, TN, FN, and FP refer to the number of true positive cases, false positive cases, false negative cases, and false positive cases, respectively.

• Recall: The proportion of true positive instances identified by the model among all actual positive instances. The recall formula is presented below:

$Recall=~\frac{TP}{TP+FN}$

• Precision: The percentage of genuine positives detected by the model out of all anticipated positives. The precision formula is shown below:

$Precision=~\frac{TP}{TP+FP}$

• F1 Score: is the harmonic mean of precision and recall, which provides a balanced measure of model performance. Below, we present the formula for the F1 score.

$F1-Score=\frac{2.~Precision.~Recall}{Precision+Recall}$

We selected these metrics to provide a comprehensive evaluation of the model's performance, taking into account its ability to accurately classify positive instances from each environment while maintaining a balance between false positives and false negatives.

After all the predictive models were tested, the one that did the best (based on accuracy and performance across all metrics) was put through more tests to find out what it learnt and what its analytical features were. To establish the optimal cut-off values, the Youden combined value of sensitivity and specificity. We also used a confusion matrix to clearly illustrate prediction performance and highlight potential misclassification areas. The performance metrics used for model evaluation included sensitivity, specificity, and AUC-ROC. In addition, the accuracy progression was tracked over training iterations, providing valuable insights into model convergence and overall stability. The loss function was also monitored throughout the iterations to evaluate the learning dynamics of the model, assess convergence behaviours, and detect signs of overfitting.

3.4 Implementation and computational setup

We conducted the entire study using a Jupyter Notebook (Anaconda3), running Python (version 3.9) on a MacBook Big Sur (version 11.7.10), equipped with 16GB of RAM and an Intel® Core i9 CPU @ 2.3 GHz. The following Python packages were utilised in this study:

• imbalanced-learn (version 0.12.3): Used for implementing the balanced bidding classifier.
• xgboost (version 2.1.2): implemented for the XGBoost classifier.
• lightgbm lightgbm lightgbm (version 4.5.0): Used to implement the LightGBM classifier.
• catboost (version 1.2.7): Utilized for the CatBoost classifier.
• scikit-learn (version 1.4.2): Used for implementing other ensemble classifiers, PCA, evaluation metrics, and data splitting.

Reducing the classification computational overhead is a vital factor for supporting the real-time processing in the mobile robot's autonomous navigation using LiDAR sensors. Furthermore, it plays a crucial role in improving the application's scalability. The capability of the classifier to classify LiDAR datasets quickly and accurately makes it an appropriate solution that can operate in dynamic indoor settings, where rapid decision-making is essential. These findings of this study demonstrate the significant benefits of combining ensemble learning and PCA for accurately sorting 2D LiDAR data into rooms, doorways, hallways, and other indoor environments and objects.

Advances in machine learning is not always about utilizing the most complicated methods or extensive feature engineering. Sometimes, simplifying model development to improve generalizability and usability represents an important innovation. Simplified models are easier for researchers to replicate and validate and are more realistic to deploy in practical, real-world environments. Our study validates this principle by attaining notable classification performance with an ensemble learning approach and dimensionality reduction via PCA, emphasizing simplicity and efficiency. The choice of methods in this study was essential in ensuring robust prediction performance. Ensemble algorithms, in general, have consistently showcased their reliability in enhancing prediction accuracy by integrating the strengths of multiple learners and mitigating individual model limitations [27]. This study highlights the effectiveness of this approach in the context of mobile robot localization using LiDAR data.

This work presents a promising contribution for scalable mobile robot indoor navigation tasks, as it provides significant accuracy with low runtime overhead and real-time applicability solutions. Specifically, integrating PCA-related features with the CatBoost classifier provided 99.2% accuracy, in addition to F1-scores, precision, and recall of 99.2% each. The performance results show that the PCA features can reduce the number of dimensions in the 2D LiDAR dataset while keeping its main features.

Previous research has highlighted the significance of AUC as an indicative metric for evaluating a model’s ability to distinguish between positive and negative instances, ensuring reliable and effective classification performance [34]. The proposed solution achieved a perfect 1 score in AUC, ensuring the predictive model's robustness and reliability. In addition, this score illustrates how specific and sensitive the model is in indoor environment classification. Most of the environment features scored a perfect classification rate under the AUC scope; however, the corridor class AUC is 0.99, which is slightly lower than the perfect score of 1.0. This minor reduction can be a result of overlapping structural features with other classes, like doorways, which impose a margin of classification ambiguity. Despite this, the corridor classification achieved an impressive score of 0.99 AUC, demonstrating the model's practicality for such practical applications. Furthermore, the optimal threshold probability of 0.43 guided by the Youden index highlights the balanced of sensitivity and specificity in the study. This guarantees not only the accurate classification of indoor environments but also minimizes false alarms, which is critical in real-time robotic applications to avoid navigation errors and unnecessary corrective actions. The achieved AUC and Youden Index demonstrated the model’s significant predictive ability.

CatBoost steadily outperformed the other ensemble methods in this experiment, and this performance aligns with its design characteristics. Unlike conventional gradient boosting methods, CatBoost uses a well-ordered boosting scheme that reduces prediction shift and decreases overfitting when dealing with high-dimensional or partially sparse datasets. The LiDAR feature space in this study consists of correlated and redundant measurements, which CatBoost handles effectively through its symmetric tree structure and built-in regularization. Moreover, CatBoost is recognized for its robustness in handling noisy data and does not require extensive preprocessing or hyperparameter tuning to attain stable performance. These properties make it specifically appropriate for LiDAR-based classification tasks, where data distributions are complicated and feature importance varies across environments.

In addition, the use of PCA with the CatBoost classifier to reduce the 2D LiDAR dataset features has significantly reduced CatBoost's runtime by 92.4%, from 84.72 seconds to 6.48 seconds. This improvement is largely due to PCA reducing the number of features, which decreases the computational complexity and speeds up both training and inference. While the runtime reduction is impressive, it is vital to note that this work was attained using standard computational resources (an Intel® Core™ i9 processor with 16GB RAM), making the approach feasible for resource-constrained mobile robots. However, the trade-off of dimensionality reduction is the potential loss of fine-grained data, which could impact performance in more complex environments. In this study, careful selection of principal components helped maintain high accuracy while improving efficiency.

Moreover, it is worth noting that AdaBoost showed the lowest classification accuracy among the ensemble classifiers evaluated in this study, with an accuracy of 41.9%, when used without applying dimensionality reduction. After applying PCA with 28 principal components, the performance of AdaBoost improved to 91.1%. However, even with this enhancement, AdaBoost’s performance remained significantly lower compared to other models. This performance gap may be attributed to AdaBoost’s sensitivity to noisy data and its sequential learning nature, which can exacerbate the impact of misclassified instances. These findings highlight the challenges of using AdaBoost in complex, high-dimensional datasets like LiDAR signals, where more robust models are needed to ensure reliable and generalizable performance.

Table 6 presents a comparison of the most pertinent studies that align with our work. We compared our work to thirteen studies using six criteria: the type of Lidar used, the number of objects classified in each study, the accuracy rate, the runtime, the machine learning algorithms used, and the processing element that ran the proposed methods. This helped us figure out where our work stands in comparison to these other works. The comparative analysis illustrates that there are performance trade-offs between computational efficiency and the LiDAR-based ML classification methods. Our work has a high classification accuracy (99.2%) using the suggested methods, which shows that the PCA with the ensemble learning hybrid approach works well. Leveraging the strengths of multiple machine learning classifiers (SVM, Random Forest, CatBoost, Decision Tree, LightGBM, and Naïve Bayes), our model maximises classification reliability compared to other approaches, such as deep CNNs or transformer-based architectures.

Table 6. Comparison of key studies relevant to the evaluation case, highlighting LiDAR types, target objects, achieved accuracy, runtime performance, applied machine learning algorithms, and the processing elements used

While some methods achieve high-speed runtimes (e.g., 70 FPS, 40 FPS, 32 FPS) using deep learning and high-performance GPUs (e.g., RTX 3090, NVIDIA Titan X, A100), our method achieves a high processing speed of 63.43 FPS on an Intel Core i9 processor, making it a competitive real-time model with a superior accuracy rate. The results achieved in this work demonstrate that our model has significant potential for real-time applications, including mobile robotics and autonomous navigation. The data comparison shows that no optimal proposed method that can balance the computational efficiency and the accuracy rate across all applications.

Our work provides a new benchmark in LiDAR data classification with an accuracy of 99.2% and a competitive real-time processing rate of 63.43 FPS. By incorporating a hardware acceleration design and transitioning to an embedded AI accelerator such as FPGAs or NVIDIA Jetson. In addition, by applying model optimisation techniques such as quantisation, pruning, and knowledge distillation, further improvements in runtime, efficiency, and scalability can be achieved. We should know that the PCA method might lose some fine-grained features in environments that change a lot. For example, the corridors class's lower AUC compared to the other classes suggests that the system needs further refinement, including the integration of more contextual features that aid in distinguishing overlapping structure settings. In addition, using techniques like hybrid sensor fusion and deep learning-based feature extraction can help the model work better in different situations. Furthermore, the model currently utilises 2D LiDAR data, necessitating further enhancements to incorporate 3D LiDAR data analysis.

LiDAR technology has proven its vital role in different real-world application domains, such as logistics robots, industrial automation, and autonomous cars. In logistics robots, LiDAR is a vital piece that provides path planning and item tracking functionality, whereas in autonomous cars, LiDAR is an important component for navigation and avoiding obstacles. In addition, industrial automation and LiDAR play a vital role in real-time monitoring. These different tasks all rely on high LiDAR data classification accuracy; however, it struggles with runtime overhead with the 3D point cloud value of data for autonomous cars, while the computational efficiency might interrupt factory operations in industrial automation. Therefore, our contribution is vital and well-suited to this research-related context. PCA employment to reduce dimensionality in such applications that produce large-scale LiDAR data is vital to manage the runtime overhead.

In theory, if our proposed model is applied to LiDAR data in such large-scale applications that have a vast number of classes, our model would provide efficient computational cost with high classification accuracy. For instance, our model would score above a 95% accuracy rate classifying a LiDAR dataset that has a tenfold increase in size and scale the runtime overhead logarithmically rather than linearly. Although a validation study for such a promising theoretical prediction is necessary, our model's efficiency and scalability show promising figures for real-world implementation.