Wearable Sensor-Based Human Activity Recognition for Smart Healthcare and Behavior Monitoring over Efficient Feature Selection and MLP

Human Activity Recognition (HAR) is an emerging research focus, particularly the ability to understand, analyze, and classify human activities from wearable sensor data. This field finds its application in systems like health monitoring, fitness, and smart homes. Nonetheless, shortcomings are attached to using HAR because of the complexity of human activities, differences in the location of sensors, and noise in captured signals. These factors could mask important emergent structures needed for activity identification. Therefore, the formulation of effective methodologies that can well handle and analyze such sensor data is important.

Wearable sensor data presents new levels of challenge in decision-making and data analysis. Signal fluctuations caused by external noise or wearer mobility, as well as inter-individual disparity, require signal preprocessing as well as feature extraction. Many sensors used in a similar setting produce high-dimensional data, which may contain noise or irrelevant features, making classification even more challenging. To address these issues, the best solution can be considered the application of modern methods of data preprocessing and optimization of the set of decision-making factors.

In this study, to overcome the above challenges, we choose three wearable sensor datasets that are accessible to the public. The first process includes pre-processing the data to remove the noise from the signal but keeping most of its features by using the fourth-order median filter. Subsequently, a Hamming window was applied to further smooth the signal and bring it into the best possible state to extract features. The extracted features were further enhanced by Particle Swarm Optimization (PSO), a metaheuristic in nature that was installed to minimize the dimensions of the features while keeping the most significant ones. Last but not least, in the last step, the different human activities were categorized with the help of the MLP classifier since it is successful in training high-level features and identifying nonlinear relationships in the optimized feature space.

The generalized methodology suggested in this paper provides a framework for enhancing the accuracy and efficiency of HAR. Our goal with the proposed approach includes handling difficulties associated with wearable sensor data, such as the non-stationary nature of the signal, by combining feature optimization and machine learning with advanced signal processing. The remaining paper is structured as follows: Section 2 surveys related studies and examines prior methods of HAR. Section three provides more information about the proposed methodology. A discussion of results is given in section 4, where the efficiency of the proposed approach will be illustrated. Last but not least, Section 5 provides a general summary of the paper and future research lines.

In this work, we presented an integrated approach towards HAR through three databases of wearable sensors. The first step involved data preprocessing of the raw sensor data: we filtered the signals using a 4th order median filter to condemn noise and smoothen the data’s readings. Subsequently, Hamming window was used on the obtained data to minimize the spectral leakage in preparation of feature extraction. The most relevant factors were initially derived from the processed data to extract the most important patterns that define different activities. To improve feature space, extracted features underwent PSO, which improves the selection of features so as to make the performance higher as compared to the classification task. Then, the important features were presented to an MLP for activity classification where the features were optimized.

The overall strategy of preprocessing, followed by feature optimization and classification, was found to provide reasonable performance for the final classifier with three datasets used to recognize human activities from wearable sensor data. Figure 1 demonstrates the overall functionality of our proposed method.

1.png

Figure 1. All-encompassing depiction of the suggested framework

3.1 Data pre-processing

In the pre-processing step, we performed preprocessing on all sensor data in the three datasets. More details on the 4th-order median filter and its application to the pre-processing step along with the Hamming window, is made available in the subsequent subsection.

3.1.1 4^th order median filter windowing

In this paper, it is shown that pre-processing can significantly enhance the capabilities of sensor networks through steps such as applying a 4th order median filter to the data produced by the sensors. This filter is meant to help reduce noise and improve the quality of data to make the signal much better. Collection circumstances from which data were obtained in the case of datasets are often comprised of noise such as spikes and fluctuations due to the limitations of a particular sensor or different conditions of the environment. To remove such interfering signal, we applied 4th order median filter on the data obtained from all the three types of dataset. The same data, filtered, is presented in Figure 2 which shows how this pre-processing enhance the HCI signal quality along x axis of accelerometer sensor.

2.png

Figure 2. Original vs filtered signal over HCI dataset

3.1.2 Windowing

Subsequent to the median filtering at the 4th order, the data was subjected to a Hamming window basically to enhance the signal. The Hamming window is one of the windows used in tapering functions for the reduction of spectral leakage by preventing a change of the frequency of the signal by sudden variation of the amplitude of the signal. The Hamming window function W (n) is given by:

$W(n)=0.54-0.46. \cos \left(\frac{2 \pi n}{N-1}\right), 0 \leq n \leq N-1$          (1)

where, N is the size of the window which has been chosen and n is the position of the data point in the chosen window. This windowing function is especially helpful in time-serial data to diminish the data and enhance the precision of feature extraction plus categorization. We used the Hamming window to the filtered data, and this further enhanced the signal so that other forms of distortions are minimized. In Figure 3, windowed data along all dimensions of both sensors from the HCI dataset are demonstrated.

3.png

Figure 3. Outcomes of accelerometer and gyroscope data after applying a hamming window over HCI dataset

3.2 Features extraction

For the filtered data, after applying the Hamming window, several features were derived to get the different characteristics of the signal. Features used in this study include Shannon entropy, wavelet transform features, quaternion-based features, time-frequency features, recurrence plot features, empirical mode decomposition features, mean, variance, Root Mean Square feature, and standard deviation. The present features offer a broader representation of the sensor data that will help improve the recognition of the activity.

3.2.1 Shannon entropy

Shannon entropy, this measure is called the entropy of the signal is defined by the level of interactions between the signal and the measure of the unpredictability or randomness. It quantifies the amount of uncertainty in the signal and is defined as:

$H(X)=\sum_{i=1}^n p\left(x_i\right) \log \left(p\left(x_i\right)\right)$            (2)

where, p(x_i) is the probability of a member's signal x_i, it is the likelihood that this signal value occurs in a specific dataset. Shannon entropy functions as a differentiation tool between human activities within HAR through its ability to evaluate signal complexity. Walking and similar repetitive movements generate lower entropy levels than tooth brushing, along with other random activities, because walking produces predictable patterns. Figure 4 demonstrates the result of applying the Shannon entropy feature extraction that determined the signal randomness in each point.

4.png

Figure 4. Shannon entropy alongside accelerometer x-axis data

3.2.2 Wavelet transform feature

The features derived from wavelet transforms are obtained from the analysis of the signals in terms of frequency contents. The continuous wavelet transform (CWT) of the signal f(t) is defined as:

$W(a, b)=\int_{-\infty}^{\infty} f(t) \Psi *((t-b) / a) d t$              (3)

The wavelet function is represented by $\psi$, the scaling parameter is a , and the translation parameter is b. The combination of time and frequency information accessible through wavelets in HAR makes them an effective tool for detecting standard movement patterns. This differentiates movements between walking, running, and standing activities. Figure 5 presents the visuals of Wavelet Transform feature extraction in some data points of HCI data.

5.png

Figure 5. Wavelet transform feature extraction by gyroscope x-axis data

3.2.3 Quaternion-based feature

Quaternion-based features preserve the most of the rotational and directional characteristics of the signal in 3-D space. The quaternion representation q(t) is defined as:

$q(t)=a(t)+b(t) i+c(t) j+d(t) k$          (4)

where, a(t), b(t), c(t), and d(t) are components contributing to signal orientation while i,j, and k are the imaginary parts. The orientation and rotational changes in body movements can be effectively detected using quaternion features in HAR since these features allow distinction of activities that share similar movements yet present different postures or directions. This helps in recognizing complex three-dimensional motions, including activities that differ between sitting down versus lying down and walking versus turning. Figure 6 includes the quaternion-based features of the data where the signal in the time domain rotates and shifts in 3D space.

6.png

Figure 6. Quaternion-based feature extraction by accelerometer x-axis

3.2.4 Time-frequency feature

The time-frequency features can be described by performing the analyses in both in time domain and frequency domain. One common method for calculating time-frequency representations is the Short-Time Fourier Transform (STFT), which is defined as:

$\operatorname{STFT}\{f(t)\}(t, \omega)=\int_{-\infty}^{\infty} f(\tau) h(t-\tau) e^{-i \omega \tau} d \tau$            (5)

where, $h(t-\tau)$ is the window function, $\omega$ is the frequency, and $t$ is the time variable. The identification of transitions between activities depends on time-frequency features since these patterns monitor motion patterns through time to differentiate between actions that share similar frequency ranges yet have dissimilar timing rhythms (walk and stair climb as an example). Figure 7 shows the time frequency characteristics of the data for frequency aspects of the signal with time.

7.png

Figure 7. Time-Frequency feature extraction from gyroscope x-axis

3.2.5 Recurrence plot feature

Based on the idea of time, the features of the recurrence plot result from the recurrence of the signal patterns. The recurrence matrix R is defined as:

$\mathbf{R}_{\mathrm{ij}}=1$ if $\left|\mathrm{x}_{\mathrm{i}}-\mathrm{x}_{\mathrm{j}}\right|<\epsilon \quad \mathbf{R}_{\mathrm{ij}}=0$ if $\left|\mathrm{x}_{\mathrm{i}}-\mathrm{x}_{\mathrm{j}}\right| \geq \epsilon$           (6)

where, $\epsilon$ is a threshold distance, and $x_i x_j$ are the signal values of the signal at time $i$ and time j . HAR systems utilize recurrence plots to detect regular patterns in motion signals which identifies periodic actions such as walking or running as well as to track modifications in the temporal structure of irregular or non-periodic movements. The last row of features is presented in the form of a recurrence plot feature, which captures the periodicity of the signal in the time domain as shown in Figure 8.

8.png

Figure 8. Resulting recurrence plot feature values on x-axis of accelerometer

3.2.6 Empirical mode decomposition feature

It is another robust, adaptive, and efficient feature extraction technique for a signal to be analyzed. Through EMD, the signal is subjected to a decomposition process into intrinsic mode functions (IMFs). The signal f(t) can be represented as: 

$f(t)=\sum_{i=1}^n \operatorname{IMFi}(t)+r N(t)$           (7)

where, IMFi(t) are the instrinsic mode functions and rN(t) is the residual after decomposition. Non-stationary motion signals undergo analysis in HAR through EMD because it decomposes signals into simpler oscillatory components i.e. IMFs which represent activity-unrelated patterns that help differentiate activities with multiple simultaneous motions like jogging with arm movements. Figure 9 displays the result of the empirical mode decomposition feature, in terms of the resulting IMFs that define and capture local features of the signal. Figure 9 indicates the illustrations of Empirical Mode Feature extraction on data from accelerometer x-axis data coming from HCI dataset.

9.png

Figure 9. EMD feature extraction results of gyroscope on x-axis

3.2.7 Statistical features: mean, variance, root mean square, standard deviation

The mean feature represents the average value of the signal in each frame, calculated as:

$\mu=1 / N \sum_{i=1}^n x_i$           (8)

The variance feature measures the spread of the signal values, calculated as:

$\operatorname{Var}(x)=1 / N \sum_{i=1}^n\left(\mathrm{x}_{\mathrm{i}}-\mu\right) 2$           (9)

The RMS feature represents the magnitude of the signal, calculated as:

$R M S=\sqrt{1 / N \sum_{i=1}^n\left(\mathrm{x}_{\mathrm{i}}\right) 2}$            (10)

The standard deviation feature measures the dispersion of the signal, calculated as:

$\sigma=\sqrt{1 / N \sum_{i=1}^n\left(\mathrm{x}_{\mathrm{i}}-\mu\right) 2}$              (11)

In Eqs. (8)-(11), N is the number of samples and x_i is an estimate of the signal that will be taken at the i^th sample. The signal characteristics measured through statistical features generate an efficient summary that identifies differences between sedentary states and energetic states (sitting vs. running) in HAR platforms. Statistical features extracted are depicted in Figure 10.

10.png

Figure 10. Illustration of statistical features extracted from accelerometer

Class Imbalance and Feature Balancing. After extracting features, the approach for balancing classes was implemented to handle dataset imbalance. The prediction performance degrades and becomes biased because of underrepresented activity classes in class-imbalanced datasets. A resampling technique was used as an intervention, which involved a focus on oversampling minority class instances.

For instance, in the HMP dataset, the "BrshTth" activity instances were substantially not equal to the "Wlk" category. We remedied the imbalance between minority and majority class instances by duplicating instances from the class having fewer samples until they matched the count of instances in the one having more training data points. The training set contained equal portions from each class because of this procedure. The process began with activity class segregation followed by oversampling all minority class data points until their distribution became equal.

The approach was implemented identically on each dataset collection (HMP, HCI, and WISDM) to achieve proper representation of all activities without preference for majority activities.

3.3 Features optimization

The research used Particle Swarm Optimizer (PSO) to find valuable features and exclude unnecessary ones to enhance system effectiveness. PSO utilizes bird and fish group behaviors to enhance solutions by testing particle fitness multiple times. A particle searches for better solutions in feature space when it evaluates both its own history and data shared by other particles. The position ($\mathrm{x}_{\mathrm{i}}$) and velocity ($\mathrm{v}_{\mathrm{i}}$) of each particle are updated iteratively using the following equations:

$\begin{gathered}v_i(t+1)=w \cdot v_i(t)+c 1 \cdot r 1\left(p i^{\text {best }}-\mathrm{x}_{\mathrm{i}}(t)\right)+ c 2 \cdot r 2\left(g^{\text {best }}-\mathrm{x}_{\mathrm{i}}(t)\right), \mathrm{x}_{\mathrm{i}}(t+1)=\mathrm{x}_{\mathrm{i}}(t)+v_i(t+1)\end{gathered}$            (12)

Here w stands for the PSO inertia weight while c1and c2 establish the PSO coefficients and r1and r2 are random variables within 0 and 1. pi^best is the best position of particle and g^best is the global best position. Through the PSO algorithm MLP predictions achieved better results by using optimal wavelet transforms and Shannon entropy features. The studies found in papers [9-12] show PSO improves the precision of classification tasks.

11.png

Figure 11. Pointing out selected features with respect to total features and fitness values

12a.png

(a)

12b.png

(b)

Figure 12. PSO Convergence-Accuracy vs. Iterations (a) for maxitr=5 and (b) for maxitr=12

Through this algorithm, we seek to enhance model accuracy by reducing negative accuracy after selection of features.

Figure 11 shows how selected features (x-axis) correlate with total extracted features (y-axis) and each match's fitness (accuracy) value position (z-axis) where selected features appear in red.

During a maximum of 5 PSO iterations, the process reached an accuracy level of 0.8770. An experiment was conducted using 12 iterations to check for premature convergence or local minimums. The accuracy turned out to be 0.8763. Research about convergence required the creation of convergence charts for both configurations. An examination of these graphs confirms how the optimization system reached stability shortly after its initial few iterations and did not attain any additional improvement after the fifth step. The identical convergence patterns showed that the model was not trapped in a local minima problem.

The selected iteration count reached optimal performance within five iterations because unnecessary computational overhead was avoided, thus making it appropriate for resource-constrained wearable systems.A comparison of convergence curves appears in Figure 12, having swarm_size=30 with sub-figure a) representing max_iter = 5 while sub-figure b) represents max_iter = 12.

3.4 Classification through MLP

Our MLP network model identified activities in these three datasets, i.e., HCI, WISDM, and HMP, and it delivered good predictions when the inputs and outputs were linked properly. Our MLP network model contains one input layer plus one or more hidden layers before the output layer, which produces predicted results. This system finds links between what enters the system and what it produces using activation methods. Our learning process adjusts weight and bias values until achieving the required loss performance target [13]. We explored various layers and activation functions to determine optimal values for learning rate and regularization strength [14, 15]. In an MLP, the output of a lone neuron in the hidden and output layer is mathematically implied as:

$z k=\sum_{i=1}^n w k i . x i+b k$            (13)

$a k=f(z k)$             (14)

where, zk is weighted sum of inputs, wki is the weight connecting the i^th input to k^th neuron, xi is the i^th input, bk is the bias term and f(zk) is the activation function.

In context of HCI data our experiment's hyperparameter grid uses GridSearchCV to analyze the impact of various MLP settings including hidden units, activation functions, regularization strength, and learning rate values. GridSearchCV was applied to optimize MLP architecture tuning by evaluating multiple hidden layer combinations such as (100,), (100, 50), (100, 100), and (200, 100) while adjusting activation functions (relu, tanh) and regularization (alpha) and learning rate strategies. The best MLP model performs predictions and testing according to results from cross-validation. Our model testing uses accuracy results alongside confusion matrix and classification reports to show how well it separates different classes. Figure 13 shows the MLP detailed structure over HCI data.

13.png

Figure 13. MLP over HCI dataset

14.png

Figure 14. Training loss curve showing convergence

The training loss curve in Figure 14 displays a consistent and descending pattern, which proves model convergence during 200 iterations to guarantee proper training operation.

After completing a 5-fold cross-validation process, the final configuration is shown in Table 2.

Table 2. Final Hyperparameter settings for MLP Classifier Identified via GridSearchCV

3.4.1 Computation Time and Resource Utilization of PSO

A study on the PSO-based feature selection approach involved runtime measurements as well as hardware resource analysis during the offline training phase. The summary of execution details and key parameters exists in Table 3.

Table 3. Computation time and resource utilization during the PSO-based feature selection

Some obstacles exist in deploying the proposed system for HAR when using wearable sensor data. One major limitation is the generalizability of the dataset, as the employed datasets (HCI, HMP, and WISDM) are collected in controlled environments. Real-world dynamic application scenarios are challenging for the model since they require different sensor placements and changing environmental factors and subject behaviors to determine classification accuracy.

The HMP, WISDM, and HCI datasets provide their data within controlled environmental conditions that limit their general application to real-world operations with flexible sensor positions and natural ecological interferences. The incorporation of median filtering and Hamming windowing in our framework serves to improve noise-resistant capabilities and reduce spectral leakage. So our model can perform well in data processing in real-world scenarios. The model will be subject to validity tests in our future work by evaluating its deployment readiness through experiments using data from uncontrolled environments.

5.1 Limitations of sensor fusion and single-sensor scenarios

There is effective sensor fusion through accelerometer-gyroscope combinations based on HCI datasets, although these multiple-sensor systems do not necessarily exist practically because hardware limitations or budgetary restrictions make them impractical. The system needs to maintain reliable functioning when provided with single-sensor data. The performance evaluation involved analyzing results from both the WISDM and HMP datasets because their data collection included only accelerometer measurements. The model demonstrated robust performance during operation with only accelerometer sensor data, which confirmed its capability to run using a single sensor. The sole utilization of sensor data led to decreased performance for detecting activities that require both high-dynamic actions and orientation sensitivity. Our experiment results showed that our proposed method still performs successfully even when only a limited number of sensors are available.

5.2 Deployment considerations on resource constrained devices

The proposed MLP model achieves computational and memory efficiency effectively since its running time operates at O (n × m × h) while using only O (m × h) storage space. The model functions effectively on wearable devices through its performance-related features, which meet processing and memory limits.

Real-time implementation of the model on embedded systems faces compliance issues due to hardware limitations that affect RAM capacity along with CPU performance and battery consumption. These problems require solutions that bring the following benefits to the model:

(1) The model has a minimalist design that consists of a few layered connections.

(2) The feature optimization process happens offline through PSO to lower runtime requirements.

(3) The implementation utilizes processed and chosen features, which both decrease input dimensions while minimizing computational loads.

5.3 Error analysis and misclassification patterns

The confusion matrices showed some wrong classifications, especially when the model confused Sit with Stand during HCI dataset activity monitoring. The sensor signal patterns from these activities maintain close similarities, which results in overlapping accelerometer and gyroscope sensor measurements, mainly affecting the vertical axis. The overall model performance remains strong, but slight variations in movement between activities cause challenges in correct classification.

Future development of the feature extraction method should include orientation-based features because they help identify subtle patterns in addition to temporal characteristics. The model requires additional sensor fusion methods together with improved models to better detect activities when movement is minimal.

This research is supported and funded by Princess Nourah bint Abdulrahman University Researchers Supporting Project number (PNURSP2025R410), Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia.