PsySFA-Net: A Lightweight Psychological Face Analysis Framework Using Spatial-Frequency Attention for Emotion Detection from Facial Images

Emotion recognition and psychological face analysis are interdisciplinary fields that use facial expressions to understand human emotions and mental states [1]. This analysis is an essential field of research with far-reaching applications across multiple domains like healthcare, human-computer interaction, and mental health monitoring. The ability to accurately detect emotions from facial expressions is used for personalised care and stress detection. However, emotion recognition remains a complex challenge due to the variability of emotional expressions.

Existing emotion recognition systems are based on hand-crafted features [2]. The facial landmarks like texture patterns and geometric features are manually selected and processed to identify emotions [3]. But these methods have failed to reach higher accuracy due to the factors of the complexity of human emotions and individual differences in facial expressions. Also, the difficulties in handling dynamic changes affect the accuracy of detecting emotional states. The manual feature extraction techniques struggled to adapt to new and unseen data.

Recently, deep learning (DL) models [4-6] like convolutional neural networks (CNNs) have significantly advanced emotion recognition by learning hierarchical feature representations from raw data. These models are capable of automatically extracting discriminative features from large datasets. Despite their success, traditional DL models face several limitations. These models still struggle with various factors. It involves noise, differences in lighting, head pose and occlusions etc. In addition, DL models are less effective at capturing subtle or suppressed emotions. This model requires more sophisticated feature extraction techniques.

The DL model is capable of extracting texture and geometric features from facial images. But it failed to capture contextual psychological cues. The psychological cues refer to subtle and implicit facial patterns. It denotes the emotional and cognitive states beyond obvious expressions. In computer vision, these cues are represented through attention-weighted feature maps and fine-grained texture variations. These cues are used for the model to capture both explicit and suppressed emotions. It mainly focuses on important facial regions and reduces irrelevant information. In expression detection, the contextual psychological cues are very important for classification when emotions are suppressed or hidden. These challenges highlight the need for more sophisticated models capable of combining multiple types of information and maintaining robustness against noise and emotional ambiguity.

To solve the above issues, a DL model is proposed in this work based on multiple feature extractions and attention mechanisms. It involves Fractal, Spectral, and Psychological Attention modules to enhance the model’s ability to focus on relevant information and to suppress the unwanted emotional noise. Also, the model parameters are tuned using the Modified Bobcat Optimization Algorithm (MBOA) to achieve higher classification accuracy.

In this work, a DL model called PsySFA_Net is proposed to handle complex facial expression recognition tasks. It integrates fractal, spectral, and attention-based features into a dynamic fusion mechanism. To achieve higher accuracy, the model parameter is tuned using MBOA. The overall workflow is shown in Figure 1.

Figure 1. Overall workflow of PsySFA_Net

3.1 Fractal block

The Fractal Block is responsible for capturing the intricate and self-similar patterns of the input facial image. It calculates these texture patterns by performing the absolute difference between diagonally adjacent pixels. This operation highlights the local textural complexity, similar to how fractal analysis identifies patterns that repeat at different scales. The operation can be mathematically represented as:

${diff}(x, y)=|I(x, y)-I(x-1, y-1)|$          (1)

where, I(x,y) is the pixel value at coordinates (x,y).

Then, the module applies a standard convolution and ReLU activation on this difference map to extract higher-level features from the textural information. In this block, the diagonal pixel difference operation is used as a computationally efficient approximation of local self-similarity. It is used to capture directional intensity variations associated with micro-textural changes in facial expressions. The conventional descriptors like Local Binary Patterns (LBP) or Gabor filters are mainly based on predefined patterns and multi-scale filter banks. The proposed operation is learnable and seamlessly integrated within the convolutional system. Also, it has minimal computational overhead and preserves sensitivity to subtle diagonal transitions.

3.2 Spectral block

The Spectral Block analyzes the image in the frequency domain to identify underlying structural information that might be obscured in the spatial domain. It uses the Discrete Cosine Transform (DCT) to convert the image into its spectral coefficients. The 2D DCT of an N×N image I is defined as:

$\begin{gathered}C_{u v}=  \alpha_u \alpha_v \sum_{x=0}^{N-1} \sum_{y=0}^{N-1} I_{x y} \cos \left[\frac{\pi u(2 x+1)}{2 N}\right] \cos \left[\frac{\pi v(2 y+1)}{2 N}\right]\end{gathered}$          (2)

where, $\alpha_u=\frac{1}{\sqrt{N}}$ for $u=0$ and $\alpha_v=\sqrt{2 / N}$ for $u>0$

In the Spectral Block, the selection of mid-frequency DCT coefficients is motivated by their ability to capture structural and discriminative facial details. It eliminates redundancy and noise. The low-frequency components primarily represent global illumination and coarse intensity information. These components contribute limited discriminative power for emotion recognition. In contrast, high-frequency components tend to capture fine edges and noise. These components are sensitive to variations such as illumination changes and compression artifacts. The mid-frequency band provides a balanced representation. It preserves essential texture and structural variations relevant to facial expressions.

3.3 Psychological attention

The psychological attention module is a form of spatial attention strategy. This module uses a larger convolutional kernel (7 × 7) to capture a broader contextual understanding of the image. This is based on the idea that human visual perception focuses on salient regions to understand a scene. The larger kernel is used for the network to pay attention to wider regions to identify important features. The convolution operation for this module can be written as:

$A_{ {out }}(x, y)={ReLU}\left(\sum_{i=-3}^3 \sum_{j=-3}^3 W_{i j} . I(x+i, y+j)+b\right)$          (3)

where, W is the 7 × 7 convolutional kernel and b is the bias. This produces a feature map that highlights the regions deemed most important by the larger receptive field. The choice of a 7 × 7 convolution kernel in the Psychological Attention Module is motivated by the need to capture broader contextual dependencies across facial regions. Compared to smaller kernels, a 7 × 7 kernel provides a larger receptive field and allows the model to integrate information from multiple facial components simultaneously.

The Fractal, Spectral, and Psychological Attention modules interact through a parallel feature extraction and unified fusion mechanism. Each module captures complementary information like spatial textures, frequency-domain structures and context-aware salient regions. Then, these features are mapped into a common feature space and combined using multi-head attention. It learns their interdependencies and assigns adaptive weights.

3.4 Dynamic feature fusion block

The dynamic feature fusion block combines features from multiple sources using a Multi-head Attention Mechanism. This fusion process is used for the model to dynamically focus on the most important features across all modalities. Given feature maps f from the Fractal Block, s from the Spectral Block, and a from the Psychological Attention Block, the features are concatenated:

$\begin{gathered} { Combined\ features }={concat}(f.{ flatten(2),s.flatten(2),a.flatten(2)) }\end{gathered}$          (4)

The fused feature set is then passed through the Multi-head Attention mechanism:

$\begin{gathered}{ attention\ output }=   { MultiheadAttention(combined\ feature) }\end{gathered}$          (5)

The resulting output is reshaped and passed forward in the network for further processing.

The attention weights are computed using scaled dot-product attention:

Attention $(Q, K, V)=\operatorname{Softmax}\left(\frac{Q K^T}{\sqrt{d_k}}\right) V$          (6)

which captures relationships between fractal, spectral, and psychological features.

In addition, attention map visualization is performed to interpret feature contributions.

3.5 CNN block

The CNN block includes a standard convolutional layer, ReLU activation, and max pooling. This block refines the fused features before passing them into the Fully Connected (FC) layers for classification. The output from the Dynamic Feature Fusion Block is passed through several convolutional layers, each followed by ReLU and Max Pooling:

$out ={ReLU}({Cov} 2 D(x))$          (7)

After passing through these layers, the feature map is downsampled using max pooling and extracts relevant patterns for further processing.

3.6 Emotion suppression block

The emotion suppression block is used to suppress irrelevant or noisy features. In this block, an attention mechanism is applied to focus on the most salient regions. This block uses Global Average Pooling (GAP) followed by an FC layer to produce a suppression factor. The GAP operation computes the average value of each channel:

$w={GAP}(x)$          (8)

The weight w is then passed through an FC layer:

$w={sigmoid}(f c(w))$          (9)

The suppression factor is applied to the input features using the following attention mechanism:

$attention\ output =MultiheadAttention (w)$          (10)

The final output is obtained by suppressing less important features:

 $out=x \times(1-attention\ output )$          (11)

Compared to SE and CBAM attention mechanisms, the proposed block performs negative attention by suppressing irrelevant features instead of only enhancing important ones. The SE focuses on channel-wise scaling and CBAM applies sequential channel and spatial attention. The proposed method reduces redundant activations using:

$x^{\prime}=x \cdot(1-\alpha)$          (12)

where, $\alpha$ represents the learned suppression weights. This mechanism improves robustness by attenuating noisy or overlapping emotional features.

3.7 FC layer

The FC layer flattens the output from the previous convolutional block. Then, the flattened output is passed through a series of fully connected layers. These layers are responsible for performing the final classification into one of the target facial expression categories. The flattened features are passed through two Linear layers, with ReLU activation and Dropout for regularisation.

3.8 Modified Bobcat Optimization Algorithm

The Bobcat Optimization Algorithm (BOA) is a novel bio-inspired metaheuristic algorithm designed to solve complex optimization problems. BOA is inspired by the hunting strategy of bobcats for their survival. The optimization of BPA is modelled in two primary phases: exploration and exploitation. The optimization of conventional BOA is based on a single prey (to guide the position updates during the exploration phase. As a result, BOA may become prematurely focused on local optima for high-dimensional problems.

To solve these issues, in this work, a MBOA is proposed. Here, in the exploration phase, the multiple prey for each bobcat is used and a randomised approach is used to update positions.

Stages of MBOA

Initialization

Initialise the population of bobcats in the solution search space. The population represents candidate solutions, each with a position in the solution space. The initial locations of the bobcats are randomly generated within predefined bounds for each decision variable as follows:

$X\left[\begin{array}{ccc}x_1 & \cdots & x_m \\ \vdots & \ddots & \vdots \\ x_N & \cdots & x_m\end{array}\right]$          (13)

where, X is the population matrix, $x_i$ is the position of the i-th bobcat, and N is the number of bobcats, m is the number of decision variables.

Each bobcat's location is initialised as:

$x_{i, d}=l b d+r(u b d-l b d)$          (14)

where, lbd and ubd are the lower and upper bounds for the decision variable d, and r is a random number in the interval [0,1].

Exploration Phase (Tracking Prey)

The bobcats move towards the prey (better solutions) in search of optimal areas in the solution space. This phase explores the global search space to identify promising regions. Each bobcat's position is updated by moving towards the best candidate solution (prey) in the population. This mimics the bobcat’s behavior of tracking its prey in the wild. The candidate prey set for each bobcat is determined by:

$C P_i=\left\{X_k: F_k<F_i\right.$ and $\left.k \neq i\right\}$          (15)

where, $C P_i$ is the set of preys for the i-th bobcat, $X_k$ is a bobcat with an improved objective function rate than $X_i$, and $F_k$ is its objective rate. A new place is identified for each bobcat by simulating its move towards the selected prey:

$x_{P 1_{i, j}}=x_{i, j}+W_{{exploration }}(t) \cdot\left(\sum_{k=1}^n \gamma_k\left(S P_{i, j}-x_{i, j}\right)\right.$          (16)

where, $x_{P 1_{i, j}}$ is the new position of the i-th bobcat in the j-th dimension after the update during the exploration phase. $x_{i, j}$ is the current position of the i-th bobcat in the j-th dimension. $W_{{exploration}}(t)$ is the exploration weight at iteration t. $S P_{i, j}$ is the j-th dimension of the position of the prey (better solution) for bobcat i. $\gamma_k$ is the random weight factor between 0 and 1 for each prey, n is the number of prey considered and t is the current iteration of the algorithm.

Each bobcat i considers multiple prey and uses them to update its position in the search space. The weighted sum of the difference between each prey’s position and the bobcat’s current position is used to compute the new position. The random factor $\gamma_k$ is used for the bobcat to explore various areas of the search space. The exploration weight $W_{{exploration}}(t)$ gradually decreases as the algorithm progresses, which shifts the focus towards exploitation in later iterations. If the new position improves the objective function, it replaces the previous position:

$X_i=X_{P 1,}\  if \cdot F_{P 1 i} \leq F_i$          (17)

Exploitation Phase: Chasing the Prey

Once a promising region is found during exploration, the algorithm switches to local search to refine the solution and exploit the potential of the best region. The bobcats simulate a chase behavior and refine their position by performing smaller changes to exploit the local search space. This phase is used to fine-tune the solutions around the best-found area. Each bobcat's new position near the prey is calculated as:

$x_{P 2_{i, j}}=x_{i, j}+\left(1-2 \cdot r_i\right) \cdot \frac{1}{1+t} \cdot x_{i, j}$          (18)

where, t is the iteration counter. If the new position improves the objective function, it replaces the previous position:

$X_i=X_{P 2_i} \ if \ F_{P 2 i} \leq F$          (19)

Repeat the exploration and exploitation phases for several iterations until a termination condition is met. In each iteration, the bobcats update their positions based on the exploration and exploitation strategies. The best solution found so far is updated and stored.

3.9 Hyperparameter tuning using MBOA

In the proposed PsySFA_Net architecture, the hyperparameter contributes major role in classification performance. The important hyperparameters are learning rate, batch size, dropout rate, and number of filters in the convolutional layers. The performance of PsySFA_Net and convergence speed mainly depend on these parameters. 

In this work, the MBOA is used to adjust the hyperparameters. The learning rate (LR) controls how quickly the model adjusts its weights in response to errors. The batch size (BS) determines the number of training samples processed before updating the model's parameters. Likewise, the dropout (DR) is used to prevent overfitting by randomly "dropping out" neurons during training. The number of filters (NF) in the convolutional layers controls the capacity of the model to capture complex features from the input image. The fitness function f(LR,BS,DR,NF) is defined as the validation loss of the model trained using these hyperparameters. The objective is to minimise this function since lower validation loss implies better model performance.

$\begin{aligned} & f(L R, B S, D R, N F)  ={ ValidationLoss }(f(L R, B S, D R, N F))\end{aligned}$          (20)

where, the validation loss is the average loss computed over the validation set after training the model for a specified number of epochs using a set of hyperparameters.

Pseudocode for MBOA

# Step 1: Initialize Population (bobcats)

population_size = 10     # Number of bobcats (solutions)

num_iterations = 5       # Number of iterations for the optimization

lower_bounds = [np.log10(1e-5), 8, 0.1, 8]  # Lower bounds for LR, BS, DR, NF

upper_bounds = [np.log10(1e-2), 64, 0.5, 32] # Upper bounds for LR, BS, DR, NF

# Step 2: Generate Initial Bobcat Population

Bobcat = initialize_population(population_size, lower_bounds, upper_bounds)

Step 3: Evaluate Fitness of Each Bobcat (using the fitness function)

fitness = []

    lr, batch_size, dropout_rate, num_filters = Bobcat

    # Apply the hyperparameters to train the model and evaluate validation loss

    val_loss = evaluate_model(lr, batch_size, dropout_rate, num_filters)

    fitness.append(val_loss)  # Fitness is the validation loss

# Step 4: Find Best Bobcat (Global Best Solution)

best_ bobcat = find_best_ bobcat (fitness, bobcats) # Find the bobcat with the lowest validation loss

best_fitness = min(fitness) # Best fitness (minimum validation loss)

# Step 5: Iterative Optimization (Echolocation Update)

for iteration in range(num_iterations):

        # Step 5a: Exploration and Exploitation

        explore_or_exploit(bobcat, best_ bobcat)  # Update bobcat’s position based on echolocation

               # Step 5b: Update Bobcat Position (Hyperparameters)

        update_ bobcat _position(bobcat)

               # Step 5c: Evaluate fitness again after update

        lr, batch_size, dropout_rate, num_filters = bobcat

        val_loss = evaluate_model(lr, batch_size, dropout_rate, num_filters)

                # If new fitness (validation loss) is better, update the bobcat’s position

        if val_loss < best_fitness:

            best_ Bobcat = Bobcat

            best_fitness = val_loss

# Step 6: Output Best Hyperparameters and Final Fitness

print("Optimized Hyperparameters:", best_ bobcat)

print("Minimum Validation Loss:", best_fitness)

The MBOA works by initialising a population of bobcat, each representing a potential solution with a random set of hyperparameters within defined bounds. The fitness of each bobcat is evaluated by training the model with its specific hyperparameters and calculating the validation loss; the bobcat with the lowest validation loss is considered the best solution. In each iteration, the bobcat update their positions using a mix of exploration and exploitation. This process is used to improve the overall hyperparameter configuration. After several iterations, the algorithm identifies the optimal set of hyperparameters that minimize validation loss. Finally, the best-performing hyperparameters are output by the optimizer.