EEG Based Emotion Detection by Using Modified Tunicate Swarm Optimization Algorithm

Emotions play a crucial role in human life, impacting cognitive function, mental well-being, and decision-making [1]. They are deeply personal and influenced by numerous factors, including external stimuli, memories, experiences, personality, and mood [2]. Positive emotions can improve human behaviour, while negative emotions can undermine and diminish it. Particularly, intense negative emotions like depression can have deep detrimental effects on both physical and mental health [3]. Accurate identification and understanding of emotions hold immense potential and practical significance across various domains [4]. Emotion recognition has useful applications in psychology, healthcare, and human-computer interaction. Emotional states are analyzed in psychology to help understand and treat mental health disorders. It allows healthcare providers to monitor patient health and emotional responses. In human-computer interaction, it improves user experience by modifying interfaces to emotional inputs. Overall, emotion identification contributes to individualized therapies, enhanced mental health care, and more instinctive human-computer interactions, encouraging developments in various fields. While behavioural signals like facial expressions and speech are commonly utilized for affective computing, physiological signals offer incomparable advantages due to their spontaneity [5]. EEG signals characterize underlying brain activity by detecting the electrical patterns created by neurons. Emotional emotions alter brain activity, resulting in different EEG patterns while increasing beta and gamma frequencies are associated with higher emotional arousal, whereas enhanced alpha and theta rhythms are associated with decreased arousal. Analyzing these frequency shifts helps identify emotional states, making EEG a helpful tool for understanding and interpreting emotional experiences by recording the dynamic interplay of brain activity associated with distinct feelings. Its high temporal resolution allows for the study of cross-frequency and neuronal oscillations coupling, which involves connections among diverse frequency bands [6]. Due to these capabilities, many affective computing researches approaches have shifted their focus towards analysing physiological signals [7]. Emotion recognition faces the challenge of accurately determining the true emotional context of stimuli during experiments through annotation or interpretation. Defining emotions precisely poses difficulties, making it complex to establish a clear ground truth for emotions. However, subjective ratings or self-reporting by test subjects during emotional trials have emerged as the most effective method to ascertain and interpret emotions in experiments [8, 9]. While EEG-based recognition systems have made advances in recognizing emotions, limited efforts have been dedicated to artificially eliciting emotional states for improved accuracy in emotion recognition. By adopting a novel approach based on central nervous system signals, this method addresses the limitations of previous emotion recognition techniques, as these signals remain minimally influenced by the previously identified factors. By employing wavelet-based and statistical-based analyses of EEG signals, the technique extracts feature and, when integrated with classification methods, has shown moderate success in identifying up to three, four, and five distinct emotions. Nevertheless, despite EEG's potential for emotion recognition, additional research was required to improve recognition rates of emotion mechanisms [10]. The contribution of this research is listed as follows:

· In this research, Modified Tunicate Swarm Optimization Algorithm (MTSOA) is suggested for feature selection to improve emotion recognition accuracy. MTSOA optimizes the feature subset by adaptively selecting the relevant features, which increases the model's discriminatory strength. This increases the accuracy of emotion identification by evaluating the most informative components of the input data.

· The DEAP database is the source of the collected EEG signals, then EMD and VMD filters was used to decompose the signal. These filters effectively remove noise and artifacts, ensuring that the EEG signals for further processing and classification.

· A Bi-LSTM network was developed to classify human emotions as valence and arousal to enhance computational effectiveness and save time, for optimal hyper-parameters.

The remainder of the article is organized as follows: Section 2 presented the related research articles, while Section 3 showed an example of how the suggested methodology of the study could be applied. The findings and conclusion of this thorough study are presented in Sections 4 and 5.

Increasing the accuracy of EEG signals based on emotion recognition is the aim of the proposed ITSOA.The process involved in various stages of classifying the emotions that are data collection, pre-processing, feature extraction, feature selection, Hyper parameter optimization and classification. The initial step involves acquiring data from a publicly available dataset, followed by a pre-processing phase to eliminate irrelevant or inappropriate features. Subsequently, a feature selection process is applied to choose relevant and non-redundant features. Furthermore, optimal hyper-parameters are selected by utilizing IRSOA. Finally, an efficient classification is conducted using a Bi-LSTM to achieve accurate predictions. The Flow diagram of the suggested method is illustrated in Figure 1.

1.png

Figure 1. Flow diagram of the suggested method

3.1 Data collection

In this research, the raw data is obtained from publicly available dataset namely DEAP dataset [18]. The description of the mentioned dataset is mentioned as follows:

The DEAP database comprises 40-channel EEG data collected from 32 individuals. It includes four emotional variables: arousal, valence, dominance, and liking, for this study, 40 EEG trials featuring 40 films were utilized, with scores ranging from 0 to 9. Each trial had a duration of 60 seconds, and specifically, the second-half EEG trials were selected for analysis. The DEAP database contains EEG data from 32 channels during emotional stimulations for 32 varience, with each session lasting 42 seconds. Emotional dimensions with rating scores in the range of 0-5 are considered low, while those in the range of 5-9 are considered high. It is a four-class classification problem because the emotions in the DEAP database are divided into four quadrants: low arousal high valence (LAHV) to relaxation, High Arousal Low Valence (HALV) to anger, High Arousal High Valence (HAHV) to happiness, and Low Arousal Low Valence (LALV) to sadness.

Figure 2 shows an example of the obtained EEG signals.

2.png

Figure 2. Sample acquired EEG signals

3.2 Data Pre-processing

After the stage of data collection, the pre-processing was utilized to transform the data into a processed format without complexities. Prior to applying decomposition filters, EEG signals undergo pre-processing that includes noise filtering, re-referencing, and artefact removal. Empirical mode decomposition and variational mode decomposition are two preprocessing processes. These techniques improve the quality of EEG data, resulting in more accurate findings when utilizing decomposition filters for feature extraction in emotion detection algorithms. It involves EMD [19] and VMD [20] filters, the process of the proposed method is briefly outlined as follows.

•Empirical mode decomposition

After obtaining EEG signals from the DEAP database, researchers employ the EMD technique to effectively decompose the non-linear and non-static signals, thereby eliminating undesired noise and artifacts. The initial phase involves categorizing the collected EEG signals into multiple IMFs using the EMD method. Each IMF then functions as a sub-band gesture, facilitating the subsequent decomposition of the sub-strip signal. Eqs. (1) and (2) the mean value of both the lower and upper envelopes.

$x(t)=\sum_{m=1}^M D_m(t)+r_M(t)$       (1)

where, $D_m(t)$ corresponds to the IMFs obtained through the EMD of the time-series EEG signal $x(t)$. Where, $x(t)$ can be expressed as the sum of all the decomposed IMFs along with the residue.

$m(t)=\frac{e_m(t)+e_l(t)}{2}$       (2)

where, $e_m(t)$ represents the lower envelope and $e_l(t)$ represents the upper envelope, then $m(t)$ represents lower envelope. Two crucial criteria must be satisfied during the decomposition process. Two main conditions are followed during each iteration of decomposition. The EMD model for sample-decomposed signal is depicted in Figure 3.

•Variational mode decomposition

VMD is a method of time-frequency decomposition.Its basic idea is to break down the original signal into individual modulations (IMFs) with different centre frequencies and bandwidths by iteratively solving the variational equation.

VMD is particularly well-suited for processing nonlinear and non-stationary signals, offering clear advantages over EMD. It effectively addresses the mode aliasing issue encountered in EMD. The mathematical representation of the variational problem with constraints is depicted in Eq. (3):

$\begin{gathered}\left\{v_k\right\}^{\min, } \\ \left\{\omega_k\right\}\left\{\sum_k\left\|\partial_t\left[\left(\delta(t)+\frac{j}{\pi t}\right) * v_k(t)\right] e^{-j w_k t}\right\|_2^2\right\} \\ \text { s.t. } \sum_k v_k=f\end{gathered}$        (3)

$v_k=\left\{v_1, \ldots . v_k\right\} \quad$ represents IMFs after VMD decomposition, $K$ represents the number of IMFs, $\left\{\omega_k\right\}=$ $\left\{\omega_1, \ldots . \omega_k\right\}$ represents the centre frequency of each component. $f$ is original input signal, $j$ is an imaginary unit, $\partial_t$ is the partial derivative of $t, *$ represents the convolution, $\delta(\mathrm{t})$ is the impact signal, $\lambda(\mathrm{t})$ are introduced into Eq. (3) to convert the constrained issue. $\left(\sum_{k:} \sum_{k=1}^k\right)$ represents the sum of all modes $\alpha$ is the balance stricture of data fidelity constraint, then the function expression in Eq. (4).

$\begin{gathered}\left(\left\{v_k\right\},\left\{\omega_k\right\}, \lambda\right) \\ =\alpha \sum_k\left\|\partial_t\left[\left(\delta(t)+\frac{j}{\pi t}\right) * v_k(t)\right] e^{-j w_k t}\right\|_2^2 \\ +\left\|f(t)-\sum_k v_k(t)\right\|_2^2+\left\langle\lambda(t) \sum_k v_k(t)\right\rangle\end{gathered}$      (4)

To achieve the optimal solution of Eq. (4), is employed, enabling continuous informs $v_k^{n+1}, \omega_k^{n+1}, \lambda_k^{n+1}$ Through iterative processes, the irregular direction multiplier process is used to attain the "saddle point" of Eq. (5) $v_k^{n+1}$ is:

$\begin{aligned} v_k^{n+1}= & \arg \min \left\{\alpha \| \partial_t\left[\left(\delta(t)+\frac{j}{\pi t}\right)\right.\right. \\ & \left.* v_k(t)\right] e^{-j w_k t} \|_2^2 \\ & \left.+\left\|f(t)-\sum_i v_i(t)+\frac{\lambda(t)}{2}\right\|_2^2\right\}\end{aligned}$          (5)

where, $\omega_k=\omega_k^{n+1}, \sum_i v_i(t)=\sum_{i \neq k} v_i(t)^{n+1}$.

Eq. (6) is subjected to Fourier transform to derive the frequency domain expressions of the IMF and the centre frequency, respectively.

$\hat{v}_k^{n+1}(\omega)=\frac{\hat{f}(\omega)-\sum_{i \neq k} \hat{v}_i(\omega)+\frac{\hat{\lambda}(\omega)}{2}}{1+2 \alpha\left(\omega-\omega_k\right)^2}$   (6)

$\omega_k^{n+1}=\frac{\int_0^{\infty} \omega\left|\hat{v}_k(\omega)\right|^2 d \omega}{\int_0^{\infty}\left|\hat{v}_k(\omega)\right|^2 d \omega}$          (7)

In each iteration $v_k^{n+1}$ is obtained through Wiener filtering of $\hat{f}(\omega)$ minus the sum of all other $\hat{f}(\omega)-\sum_{i \neq k} \hat{v}_i(\omega)$, where $i$ is not equal to k . The term $\omega_k^{n+1}$ represents the centre frequency, and $\hat{v}_k(\omega)$ denotes the inverse Fourier transform. The real part of the result is represented as $v_k(t)$. EEG signals exhibit evident nonlinear and non-stationary characteristics, and the application of VMD proves beneficial in reducing their non-stationarity. The hybrid model for sample-decomposed signal was illustrated in Figure 4.

3.3 Feature extraction

The first pre-processing step yields the pre-processed output, which is subsequently used as input for the feature extraction stage that follows. The goal of extracted features is to help with analysis and learning algorithms in a variety of applications, including machine learning and pattern recognition, by identifying and selecting the most relevant and useful properties from raw data. The hybrid feature extraction method encompasses a diverse set of statistical techniques for extracting feature values from the processed signals. These techniques include, sample entropy, zero-cross rate, Hjorth activity, mean-curve-length, standard deviation, variance, Shannon entropy, normalized first difference, Hjorth complexity, auto-regressive model, mean Teager energy, Hjorth mobility and various band power measurements such as delta, beta, alpha and gamma. Additionally, the feature extraction involves the ratio of band power between alpha and beta frequencies, as well as minimum and maximum computations [21]. This hybrid feature extraction produces a significant feature length of 2676 in the context of two-dimensional emotion prediction. It is further optimized to 1204 useful features for both valence and arousal classes.

The operation of this hybrid approach offers several advantages, including improved accuracy, reduced risk of overfitting, enhanced data visualization, and accelerated training process when integrated with Bi-LSTM networks.

3.4 Feature selection

The step of suggested feature selection receives the pre-processed output, which is obtained from the feature extraction stage and fed as input. Because of its adaptive feature space exploration, TSOA performs better when it comes to feature selection for emotion categorization. In order to identify the most crucial attributes, TSOA carefully looks over feature subsets, striking a balance between exploration and exploitation. Its adaptability to alter the search approach guarantees the inclusion of discriminative features, enhancing the model's capacity to identify complex emotional patterns. By assessing the extraction of critical information from the input data, the dynamic nature of TSOA improves feature selection over prior methods, leading to more reliable and efficient models. It also demonstrated higher accuracy in emotion categorization. Below is a brief description of the feature selecting approaches procedure.

3.4.1 Tunicate swarm optimization algorithm

Because it finds better optimal solutions than competing techniques, the new meta-heuristic optimization algorithm TSA is used in this work. TSA shows to be especially useful in solving practical engineering design issues. When living in aquatic settings, tunicates use two methods to find food: they use swarm intelligence, and they determine the best state to locate food in.

(1) Search agents for avoiding conflicts

Eqs. (8)-(10) establishes the role of a search agent, which incorporates the $\vec{A}, \vec{G}$, and $\vec{M}$ vectors in order to prevent disagreements between tunicates.

$\vec{A}=\frac{\vec{G}}{\vec{M}}$      (8)

$\vec{G}=c_2+c_3-\vec{F}$      (9)

$\vec{F}=2 . c_1$    (10)

Advection of water flow in the deep ocean is denoted by $\vec{F}$, whereas $\vec{G}$ stands for gravity and $\vec{M}$ for social interactions. $C_1$, $C_2$, and $C_3$ are random variables that fall inside the interval [0 , 1]. The estimation of $\vec{M}$ can be obtained using Eq. (11).

$\vec{M}=\left[P_{\min }+c_1 \cdot\left(P_{\max }-P_{\min }\right)\right]$     (11)

where, P_min and P_max are the minimum and maximum speeds correspondingly for social interface. It is essential to note that these values for P_min and P_maxhave been set at 1 and 4 respectively.

(2) Move to the best neighbor direction

The motion of the tunicates according to Eq. (12) is in the direction of their best neighbour.

$\vec{P} D=\left|\vec{F} S-r_{\text {and }} \cdot \vec{P}_p(x)\right|$      (12)

Here, rand is a random volume within the interval $[0,1]$, distance refers to the food source, tunicate is represented as $\vec{P} D$, x is the optimum food source which is determined by $\vec{F} S$, and the location of the tunicate is expressed as $\vec{P}_p(x)$ transmitting towards best search agent. Eq. (13) gives the optimal position of the tunicate with respect to the food location.

$\vec{P}_p(x)=\left[\begin{array}{l}\vec{F} S+\vec{A} \cdot \vec{P} D ; \text { if }: r_{\text {and }} \geq 0.5 \\ \vec{F} S-\vec{A} \cdot \vec{P} D ; \text { if }: r_{\text {and }}<0.5\end{array}\right]$      (13)

Finding the optimal solutions between the first two optima is the first step towards simulating tunicate features. The locations of additional search agents are then effective, accounting for the best tunicate's location. Eq. (14) can be utilized to estimate the tunicates' swarm characteristics.

$\vec{P}_p(\vec{x}+1)=\frac{\vec{P}_p(x)+\vec{P}_p(\vec{x}+1)}{2+c_1}$      (14)

3.4.2 Modified tunicate swarm optimization algorithm

An MTSOA is incorporated in the original $\vec{A}, \vec{G}$, and $\vec{M}$  vectors to improve the global and local search capabilities of the standard TSA. By using these vectors, search agents can traverse the search space at random and facilitate conflict-free exploration. By modifying these vectors, it is possible to achieve better phases for exploration and exploitation. The reason MTSOA is used for feature selection in emotion detection is because it can navigate complex search spaces with ease.

Through feature subset optimization, MTSOA chooses the most pertinent EEG signal elements to enhance model performance. Its efficiency and adaptability contribute to increased accuracy in emotion identification systems. However, because some differences become caught in local optima, finding the global optimum may be challenging, particularly in higher-dimensional and complex situations. This study enhances TSA's exploring capability in the ways that follow:

(1) Improved exploration ability

The study's initial values for the numerator and denominator are set to be low and high, respectively. This modification allows search agents to cover a wider area of the search field by allowing them to travel at a slower pace.

Because the targets of this enhanced approach are local searches, striking a balance between exploration and exploitation is important. As the number of iterations increases, the values gradually decline to lower levels. By implementing these adjustments, the exploitation phase is enhanced and leads to more effective local explorations.

The improved form of vector $\vec{A}$  can be represented by Eq. (15).

$\vec{A}=\frac{2-4\left(t / t_{\max }\right)}{4-3\left(t / t_{\max }\right)}$      (15)

where, the values for the current iteration and the maximum number of iterations are represented by t and t_max.

Therefore, the result of the recommended feature selection is subjected to hyperparameter optimisation in order to enhance the accuracy of emotion recognition, as briefly described in the stage below.

3.5 Hyper parameter optimization

In order to increase the accuracy of emotion identification, the hyper parameter optimisation was improved utilising the RSOA after the suggested feature selection step. The RSOA is used in emotion recognition to choose hyperparameters due to its ability to efficiently search parameter spaces. Better accuracy and robustness are attained by the model's optimal configuration for emotion detection tasks, which is ensured by its adaptive nature and ability to find global optima.

Hyperparameters regulate how machine learning models behave, affecting both their effectiveness and generalization. The parameter includes learning rate of 0.001 to 1, hidden units of 10 to 500, regularization strength of 0.01 to 0.1, batch size of 32 to 512, and activation functions of ReLU, sigmoid, respectively. A powerful metaheuristic optimisation method called RSOA was developed by observing how attacking and following rats behave. This method conducts hyperparameter optimisation by modelling the aggressive and follow-the-rat behaviour, which ultimately leads to improved performance [22]. Eq. (16) scientifically indicates that the search space is initialised by the RSOA.

$x_i=x_{i \text { min }}+\operatorname{rand} \times\left(x_{i \text { max }}-x_{i \text { min }}\right), i=1,2, \ldots N$    (16)

In Eq. (17), $x_{i \text { max }}$ and $x_{i \text { min }}$ represent the upper and lower bounds respectively, of the i^th variable, while N stands for the total number of agents. Through this aggressive process, therefore, the rat is scientifically shown in Eq. (17) to create updated positions of the rat.

$\vec{P}_i(x+1)=\left|\vec{P}_r(x)-\vec{P}\right|$     (17)

In this context, $\vec{P}_i(x+1)$ signifies the efficient locations of the $i^{\text {th }}$ rats, and $\vec{P}_r(x)$ denotes the best optimum resolution. The computation of the time $\vec{P}$ is carried out using Eq. (18).

$\vec{P}=A \times \vec{P}_i(x)+C \times\left(\vec{P}_r(x)-\vec{P}_i(x)\right)$    (18)

In this instance, $\vec{P}_i(x)$ denotes the position of the i^th rat, whereas the matrices A and C are found by consuming Eqs. (19) and (20), respectively.

$A=R-x \times\left(\frac{R}{\text { Iter}_{\max }}\right), x=1,2,3, \ldots$. Iter $_{\max }$      (19)

$C=2 \times$ rand      (20)

In the RSOA, Iter$_{\text {max }}$ described the maximum volume of iterations, $x$ denotes the current iteration, $R$ represents a random amount within the range [1,5], and $C$ indicates a random amounts within the range [0,2]. N-dimensional vector $X$ is scientifically represented in Eq. (21).

$X=\left(x_1, x_2, \ldots . x_N\right)$     (21)

where, $x_i \in\left[x i_{\text {min }}, x_{\text {imax }}\right]$, and the opposed point of $x_i$ is characterized as $\bar{x}_i$ and it is revealed in Eq. (22).

$\bar{x}_i=\left(x_{\text {imax }}, x_{\text {imin }}\right)-x_i, i=1,2, \ldots, N$   (22)

Conversely, to enhance the search exploration in the ITSOA at each iteration, the best resolution is substituted with a new resolution based on Eq. (23).

$x_{\text {worst }}=\left\{\begin{array}{cc}\operatorname{rand}_1 \times \vec{P}_r(x), & \text { if rand}{ }_3 \leq 0.5 \\ \left(x_{\text {imax }}-x_{\text {imin }}\right)-x_i, & \text { if rand}{ }_3>0.5\end{array}\right.$       (23)

In this context, rand $_1$, rand $_2$, and rand $_3$ represent random records within the range of 0 to 1 , and $x_{\text {worst }}$ indicates the resolution with a high value by applying these concepts, the hyper parameter optimization outcomes are improved as the vector locations of the least hierarchical rats are exchanged with the finest results attained distant, denoted as $\vec{P}_r(x)$, in each group.

3.6 Classification using Bi-LSTM

Using the DEAP dataset, the Bi-LSTM is run after the hyper parameter optimisation stage to increase the accuracy of emotion categorization. Because it can identify temporal correlations in emotional sequences and both past and future contextual information, the Bi-LSTM is crucial for classifying emotions. One of its advantages is that it can manage bidirectional dependencies, which makes it possible to understand emotional expression more accurately. Bi-LSTM is selected because of its ability to describe intricate temporal patterns. It performs better than alternative techniques by effectively capturing long-range relationships, enhancing the precision of emotion classification models for a more complex and contextually sensitive emotional content interpretation. An effective deep learning model for classifying emotions is called Bi-LSTM [23]. The bidirectional feature improves the model's comprehension of context by allowing it to take into account information from the past as well as the future. Bi-LSTM uses text inputs to represent the emotional context of the text in classifying the emotions. This is a popular choice for classifying tasks related to emotion in natural language processing, due to the ability to gather context and sequence information that furthers our understanding of a multitude of applications of human emotions and sentiment.

The structure of a Bi-LSTM is shown in Figure 5.

3.png

Figure 3. Sample-decomposed EEG signal of EMD model [24]

4.png

Figure 4. EMD+VMD technique for sample-decomposed EEG signal [24]

5.png

Figure 5. Structure of a Bi-LSTM architecture

A forward calculation with an RNN in positive direction, notationally represented by $\mathrm{A}_1 \rightarrow \mathrm{A}_2 \cdots \rightarrow \mathrm{A}_t$, proceeds. At time $t$, the input is the sequential data $x_t$, and the output is the A$_{(t-1)}$ of the time before. In other parts, a reverse calculation that runs in the opposite way has a direct RNN in positive direction, symbolically noted again as $\mathrm{A}_1 \rightarrow \mathrm{A}_2 \cdots \rightarrow \mathrm{A}_{\mathrm{t}}$. The input at time $t$ is again sequential data $x_t$, and the output is now the A$_{(t+1)}$.