Artificial Butterfly Optimizer Based Two-Layer Convolutional Neural Network with Polarized Attention Mechanism for Human Activity Recognition

Human activity recognition (HAR) in the home, achieved by establishing a network of sensors to detect human behavior, can enhance people's ability to maintain independence and quality of life [1]. HAR is a dynamic and complex research area [2-4], driven by the increasing demand for medical diagnosis of chronic diseases, active home automation [5], and convenient services. Methods based on affordable and user-friendly sensors for monitoring and recording human daily actions can eliminate the constraints of human subjectivity. Doctors can gain insights into managing patients with chronic illnesses by reviewing data collected from patients' daily activity logs. Thanks to sensor devices, patients can undergo continuous monitoring and treatment even in the absence of medical professionals. However, other internal variables significantly impact human health and quality of life.

HAR serves as the primary method for integrating smart sensor monitoring equipment into the natural surroundings of a smart environment. Research challenges with practical implications [6] arise from the complexity of sensor data across various configurations and locations. Issues include determining how to use the features of sensor data to extract discriminating characteristics and how to enhance reliability based on sensor data.

Human behavior is unpredictable, influenced by individual differences in taste and lifestyle habits. When a sensor used for feature acquisition cannot discern a subject's primary characteristics, extracting the subject's behavior feature space becomes challenging. Implementing human body tracking technology in a visual sensor ensures constant visibility of the subject. However, visual monitoring involves large data volumes, making it resource-intensive to process, with privacy leaks being a concern [7]. Wearable sensors, while providing reliable data collection, have drawbacks such as being cumbersome and difficult to use for extended periods, limiting their suitability for identifying complex activities [8].

Integrated multitype sensors placed in the subject's environment track the current status of the smart space and gather data about the inhabitants without disruption [9]. However, environmental sensors may introduce noise data into the stream, impacting feature definition [10]. Analyzing features from different datasets and identifying significant aspects of behaviors can aid in optimizing the selection and deployment of environmental sensors [11], leading to improved recognition accuracy and more precise conclusions when combined with high-quality data feature sets.

Significant intraclass and modest interclass variances affect the classifier's performance in activity detection. Time-stamped streams of sensor activations are often presented as gathered data [12], segmented into activity instances represented by sequences of sensor events [13]. Two types of techniques for extracting HAR features from sensor sequence instances are conventional feature extraction and expression techniques [14]. Conventional methods require manual extraction of time domain features and other feature vectors [15], providing only superficial characteristics. Deep learning technology, without human interaction, can automatically extract richer and more nuanced characteristics from raw data, efficiently resolving intraclass differences and interclass similarities [16].

In the proposed scheme, spectral and spatial information of HAR are separately extracted using a two-branch structure. To adapt the network to small sample contexts, complexity is reduced by minimizing the window widths of the 3D layers. Additionally, the network's convolutional layers are connected using a one-time connection, allowing the shallow layers to be retained in the upper layers and retrieved with deep semantic features. This one-time connection benefits the network's ability to extract feature maps across layers and effectively extract training-sample features. The use of Batch Normalization (BN) layers and Parametric Rectified Linear Unit (PReLU) layers mitigates the gradient problem and network degradation. After the two-branch, an enhanced full attention (FLA) mechanism is adopted to combine spectral and spatial characteristics acquired from the two-branch network. The Artificial Butterfly Optimization Algorithm (ABOA) handles the fine-tuning process.

The rest of the paper is organized into five sections: 2. Background Work; 3. Proposed Method; 4. Investigation and Discussion of Results; and 5. Conclusion.

1.png

Figure 1. Instance of the UCF50 dataset [23]

There were 50 different categories of data in the UCF50 dataset. The dataset includes events like a baseball pitch, a basketball shot, a billiards break, a clean and jerk, a dive, a drum solo, a fencing stroke, a golf swing, a guitar solo, a horse race, a horseback ride, a high jump, a horseback ride, a hula hoop, a rope, a jumping jack, a kayak Indoor rock climbing, rowing, salsa spins, skateboarding, skiing, skijet, juggling soccer balls, swing dancing, playing the tabla, tai chi, swing tennis, playing the violin, spiking volleyball, walking a dog, and yo-yoing are all in the list of 24 activities.

Both datasets were used to train the network, and the consequences were associated. The dataset was split into 10 layers. The smaller dataset is known as UCF50mini. The whole and reduced datasets were split into three groups: the training set.

3.1.2 LoDVP abnormal activities dataset

There are 1069 clips in all that make up the LoDVP Abnormal Activity collection. Videos depicting fictitious situations staged by amateur actors. The database contains only credible videos. The parking lot, woods have all been cited as sites of incidents. Multiple camera views of the scenes were taken. There are around 100 movies in each of the 11 categories that make up the dataset. The incident determines how long the video is, which might range from 1 second to 30 seconds. Characteristics shared among videos of the same type include a similar scenario, camera angle, and recurrent characters [24].

3.jpg

Figure 2. LoDVP Abnormal Events dataset: (a) Begging; (b) Drunkenness; (c) Fight; (d) Harassment; (e) Hijacking; (f) Knife danger; (g) Regular movies; (h) Pollution, (i) Pollution broken things; (j) A theft; (k) Terrorist acts [25]

Figure 2 displays the many categories that make up the LoDVP Abnormal Activity dataset. This dataset, like UCF50mini, was split into a training test and an authentication set for our work. The breakdown was as follows: 70%; 0%; 10%.

3.1.3 MOD20 dataset

Six films were shot with a quadrotor UAV, while the remaining 2318 were pulled from YouTube to make up the MOD20 dataset. Media files are all full screen. The frame rate of the videos was below 29.97 fps. Both stationary and mobile cameras were used to capture the videos included in the dataset. Twenty different subject areas are included in the movies (see Figure 3) [26].

3.2 Classification

In this work, a two-branch network is proposed to extract the rich spectral and spatial information present in HAR. Figure 4a depicts the process flow of the projected network. The two-branch network make up the suggested network.

The spectrum information is extracted along the spectral dimension, while the spatial information is extracted along the spatial dimension, in a two-branch spectral-spatial network.

There are eight convolutional layers in the spectral feature extraction sub-branch, including the BN layer and the PReLU activation layer. We begin by increasing the number of feature maps and decreasing the spectral dimension using a layer with a window size of 711. After that, a total of five convolutional layers are employed to further extract the spectral information, each with a 711-window size. These convolutional layers save their prior feature maps thanks to a one-time link between them. The feature maps are then compressed using a size of 111. Additionally, the spectral dimension is layer with a C11 window size and a reshape operation. Similarly, eight convolutional layers are used in the spatial feature extraction sub-branch, utilizing BN and PReLU. To accomplish both of these goals, a convolutional layer with a 711-window size is first developed. After that, we reduce the spectral dimension by using a convolutional layer with a window size of C11. The spatial info is then extracted using layers with a 133-window size. These convolutional layers are connected by a one-time link to preserve the data. After that, the dimensionality of the spectrum is reduced and the number of feature mappings is compressed using a layer with a window size of 111. The final feature maps are a combination of the spectral branch's and the spatial branch's outputs.

2.jpg

Figure 3. The instance of the MOD20 dataset [26]

4a.jpg

(a) The way the PTCN works

4b.jpg

(b) The way the PFLA works

Figure 4. How the suggested network is put together

3.2.1 Hyper-parameter tuning using artificial butterfly algorithm

The territoriality of butterflies provides a useful model system in which to examine the effects of life history optimization on the lifetime cost of aggressive behavior [27]. A novel optimization technique can woodlands. We introduced the Artificial Butterfly Optimization (ABO) technique, which is inspired by the mating behavior of woodlands. Table 4 presents the ABO algorithm's pseudo code. The ABO procedure follows several criteria that idealize the butterflies' methods of mating:

a) In order to increase their chances of seeing butterflies strive to move into a better position known as a sunspot.

b) Every butterfly on a sunspot is constantly looking to move to a more favourable location on the sun.

c) Every canopy butterfly, in a never-ending competition for the sunspot, flies straight at any sunspot butterfly.

Table 4. Pseudo code of ABO algorithm

In Table 4, you'll see how the original population of butterflies was divided into a healthier and a less healthy subset. A separate flight plan is prepared for each party. There are now some similarities to niching approaches. It is standard practice to modify the behavior of a classical procedure using a niching technique [28, 29] to maintain distinct groups within the chosen populace component in order to effectively identify several optimum solutions. You may choose among the sunspot, canopy, or free flight modes. For these configurations, a wide range of flight methods is at your disposal. In other words, ABO may yield a new algorithm whenever one of the three flight manners is given a unique flying strategy.

Here are three distinct butterfly flying scenarios. In this research, we use a D- vector to characterize the site of a digital butterfly.

According to Eq. (2), each butterfly at random. This tactic is implemented in the ABO mode.

$X_{i_i}^{t+1}=X_{i, j}^t+\left(X_{i, j}^t-X_{k, j}^t\right) \cdot$ rand ()                (2)

In this play, I butterfly. In this expression, j is a dimension index selected at accidental among [1,D], t is the iteration count, rand() generates an among [-1,1], and k is a butterfly generated at random. Here, k is different from i.

Each direction picked at random by a neighbor according to Eq. (3). This strategy is utilized by both the flight modes of the ABO procedure.

$X_i^{t+1}=X_i^t+\frac{X_k^t-X_i^t}{\left\|X_k^t-X_i^t\right\|} \cdot(U b-L b)$. step $\cdot$ rand ()             (3)

where, i is the replicated, t is the number of repetitions, $X_i^{t+1}$ is the butterfly's new position, step is an accidental value among 0 and 1, and k is a haphazardly selected butterfly. In this case, k is dissimilar to i. The butterfly has a flight range among L_b and U_b, where L_b value. The values of L_b and U_b are significant in this context.

According to Eq. (4), each butterfly randomly chosen neighbor. In the exploration phase, the same method has been used to look for a new-fangled place. This process is realized in the ABO flight mode.

$X_i^{t+1}=X_k^t-2 \cdot a \cdot \operatorname{rand}()-a \cdot D$          (4)

where, $X_i^{t+1}$ is the novel site of the i^th digital produces an accidental value among 0 and 1, and a decrease linearly from 2 to 0 with each repetition. k is a butterfly generated using Eq. (5).

$D=\left|2 \cdot \operatorname{rand}() \cdot X_k^t-X_i^t\right|$           (5)

If i produces a random number among zero and one. The value of the step parameter in Eq. (2) is variable. As shown in Eq. (6), we employ a diminishing method. In a linear fashion, with cumulative iterations, step declines from 1 to step_e. A better step value in the beginning increases the global searching ability and variety. In the final phase, better convergence is achieved with smaller step values since massive leaping is avoided.

step $=1-\left(1-\right.$ step $\left._e\right) \cdot \frac{E}{\max E}$              (6)

where, E is present assessments total and maxE is the max assessments count.