A Parallel Algorithm for Facial Expression Recognition for Student Engagement Monitoring in Classrooms

Facial expression recognition is a rapidly evolving field in recent years, combining many disciplines such as computer vision, machine learning, and mathematics. In this research, we focus on the application of facial expression recognition algorithms to monitor student engagement in classrooms and understanding students’ engagement status in real time is a major challenge in the field of education, where accurate analysis of emotions can provide vital information that helps teachers improve the learning experience and this type of system relies heavily on mathematics, where concepts such as linear algebra, probability, statistics, and optimization techniques overlap to develop effective and accurate solutions and facial expression recognition algorithms require the use of multiple mathematical techniques, such as image processing using convolutions and Fourier or wavelet transforms to extract features from facial images and linear algebra plays an important role in representing and processing data through matrices, where mathematical operations are applied to these matrices to extract relevant patterns [1].

After extracting the features, methods such as PCA are used to reduce the dimensionality while maintaining the variance necessary for effective classification of emotional faces [2]. At a later stage, algorithms such as SVM are used to classify emotions based on the extracted features and on the other hand, optimization is crucial to achieving optimal performance, as the goal is to develop an algorithm that minimizes the classification error (E) and increases the real-time processing speed (S), making it possible to assess student engagement (G) with high accuracy and in the shortest possible time and to achieve this, we propose to use parallel processing via multi-core processors, which contributes to accelerating the computational processes and achieving a balance between speed and accuracy at the same time and these algorithms also require the use of advanced mathematical techniques in performance optimization, such as gradient descent techniques and distributing computations across multiple cores to make the most of the available resources and through this mathematical approach, a robust system can be built that enables effective monitoring of student engagement through accurate classification of emotions using parallel computing techniques.

This paper gives details mathematical model that combines these different techniques to build a facial expression recognition algorithm for monitoring student interaction in classrooms. We will demonstrate how advanced mathematical methods such as linear algebra, probabilistic analysis, and optimization techniques can be used to achieve the desired goal of improving classification accuracy and providing real-time feedback [3, 4].

This research will focus on studying and analyzing advanced mathematical techniques used for training deep neural networks and specifically those aimed at minimizing loss functions and achieving rapid convergence in multi-core processing systems and this study will particularly focus on mathematical algorithms related to linear algebra and PCA and SVM while also addressing the core optimization equations associated with these techniques [15].

3.1 Data collection and pre-processing

- Dataset (D): The dataset contains face images $\left(X_i\right)$ and the corresponding emotion labels $\left(y_i\right)$:

$D=\left\{\left(X_i, y_i\right) \mid X_i \in R^{\wedge(m \times n)}, y_i \in\{1, \ldots, k\}\right\}$           (1)

- Standardization: Data normalization using scaling function:

$f(x)=\frac{(x-\min (x))}{(\max (x)-\min (x))}$           (2)

- Image enhancement: Denoising using Gaussian filters:

$X^{\prime}=X \times G_\sigma$           (3)

where, $\left(G_\sigma\right)$ is the Gaussian filter with mean (0) and standard deviation $(\sigma)$.

- Dataset (D): The dataset consists of face images along with their corresponding emotion labels and these images represent faces of individuals and the emotion labels describe the feelings or emotions expressed in these faces for as happiness and sadness, or anger and the images are represented mathematically as matrices, where each image is a collection of pixel values and along with the images, the dataset includes labels that specify the emotion associated with each image [16].

The dataset used in this research serves as an information repository for facial expression recognition, aiming to analyze students' engagement levels in an educational environment. It includes images of students' faces captured during lessons, with facial expressions classified into several categories such as happiness, sadness, anger, surprise, fear, disgust, and neutrality. This classification helps in understanding students' emotional engagement with the educational content.

The data was obtained from open-source datasets such as FER-2013, which contains over thirty-five thousand images collected from various sources, and AffectNet, which includes more than one million images labeled according to emotional states also the EmoReact Dataset was specifically designed to study emotions in educational settings and if the data collection was conducted through an experimental setup, images were captured during actual classroom sessions using high-resolution webcams while ensuring compliance with ethical standards and obtaining necessary participant consents [15, 16].

The dataset consists of approximately ten thousand to fifty thousand facial images distributed across different categories to ensure classification comprehensiveness and the data was split into eighty percent for training and twenty percent for testing to maintain a balanced deep learning model for analysis and the dataset covers age groups ranging from ten to thirty years old, as this is the most common range among high school and university students also it includes a wide ethnic diversity encompassing Europeans, Africans, Asians, Latinos, and Arabs, ensuring the model's impartiality and enhancing its generalization ability when applied on a broader scale.

The dataset covers various lighting conditions also including natural lighting like direct and indirect sunlight, artificial lighting from fluorescent and LED sources, and low-light conditions that may exist in dimly lit classrooms and the images are also captured from different angles to ensure the model's robustness in real-world conditions, including direct frontal angles, slightly tilted side angles also vertical angles from above and below.

This study follows strict ethical guidelines to protect participants' rights, privacy and well-being and informed consent was obtained and data was anonymized to prevent identification where ethical approval was secured and strict security measures were applied to data storage also the dataset was curated to ensure diversity and reduce bias and study complies with international regulations like GDPR and the Belmont Report ensuring fairness and responsible use of facial recognition in education.

- Standardization:

This step involves applying a technique to transform the data into a standardized range and the goal of standardization is to scale all the values in the dataset into a uniform range for between 0 and 1 and this helps avoid negative effects that may arise from the presence of extreme values in the data and such as very large or very small numbers, which can impact the accuracy of machine learning models.

- Image enhancement:

Image enhancement is one of the methods aimed at removing unwanted noise or disturbances in images and getting rid of them by applying specific filters to the images for example the Gaussian filter and which works to soften the image and reduce any unwanted noise or details and this process improves the image quality and facilitates the extraction of relevant features relevance while training neural networks and thus contribute to improving the model’s performance in emotion classification [17].

3.2 Algorithm design

- Feature extraction via CNN: Convolution operation with filter ($W$) applied to the image $(X)$ :

$C(x)=X \times W$           (4)

where, $(X)$ is the image, and $(W)$ is the filter weights and this operation is repeated across multiple layers to extract highlevel features $\left(F=C^{n(x)}\right)$.

- Dimensionality reduction using PCA, which reduces dimensions and increases contrast:

$F_{\text {reduced }}=P C A(F)$           (5)

- Sentiment classification process using SVM, where, SVM training works to increase the margin between classes:

$K\left(x_i, x_j\right)=<x_i, x_j>$           (6)

where, linear or nonlinear methods are used for the separation process in emotional states.

- Feature extraction via CNN

In this step, CNNs are used to extract features from the images. CNNs are designed to automatically detect patterns or important features within an image by using convolution operations and the convolution operation involves applying a filter (also known as a kernel) over the input image to extract low-level features, such as edges, textures, and corners and these filters are learned during the training process [18].

The image progresses through multiple layers of the CNN, the complexity of the features being extracted increases. In the initial layers, the network might focus on simple features like edges and colors, while deeper layers may capture more complex patterns or abstract shapes such as facial features (e.g., eyes, nose, mouth) and by the end of the network, these features become high-level representations of the image, capturing important information for classification tasks and these extracted features are then passed on for further processing to make predictions about the image, such as the associated emotional state.

- Dimensionality reduction using PCA

After the features have been extracted from the images through the CNN, they may contain a large number of dimensions or variables and these dimensions may include redundant or irrelevant information that could make the learning process more complex and time-consuming and to address this, PCA is used to reduce the number of dimensions while retaining as much variance in the data as possible [19].

PCA works by identifying the directions (principal components) in which the data varies the most and it then projects the original features onto a smaller set of dimensions (principal components) that explain the majority of the variation in the data and this dimensionality reduction helps in eliminating noise and reduces computational complexity, making the classification process more efficient and PCA is especially useful when dealing with large feature sets that could otherwise slow down the training and testing phases of the model [20].

- Emotion classification using SVM

Once the features have been reduced in dimensionality, the next step is to classify the images into different emotional categories using a machine learning model such as SVM. SVM is a supervised learning algorithm that is particularly effective for classification tasks, including binary and multi-class problems.

SVM works by finding an optimal hyperplane that separates the data points from different classes with the maximum margin and this means that the SVM algorithm tries to create a boundary that ensures the data points of different classes (for example, "happy", "sad", "angry", etc.) are as far apart as possible and the algorithm is trained using the extracted features, and it learns to classify images by determining the optimal hyperplane in the feature space and in some cases, a linear hyperplane may suffice, but if the data is not linearly separable, kernel functions are used to map the data into a higher-dimensional space where a linear separation is possible. Commonly used kernel functions include the linear kernel, polynomial kernel, and radial basis function (RBF) kernel and the SVM is trained to maximize the margin between the different classes, which helps ensure accurate and robust classification even when dealing with complex, non-linear relationships between the features [21].

3.3 Parallel processing implementation

- Parallel convolution: Distributing convolution layer computations across multiple cores:

$T_{\text {total }}=\frac{\operatorname{sum}\left(T_i\right)}{n_{\text {cores }}}$           (7)

where, the total time is distributed across the cores to speed up training.

- Parallel gradient descent: Updating weights $\left(W_i\right)$ in each layer of the neural network through parallel gradient descent:

$W_{i_{\text {new }}}=W_{i_{\text {old }}}-\eta \times \nabla W_i$           (8)

where, $(\eta)$ is the learnıng rate and $\left(V W_i\right)$ is the gradient with respect to the weights $\left(W_i\right)$.

- Parallel convolution: Convolution operations in neural networks are mathematical and based on special equations, especially in deep learning models and they are computationally intensive, especially when dealing with large images or multiple layers and for speed up this process, parallel convolution is used, which involves distributing the convolution tasks across multiple processor cores and in traditional non-parallel processing, each convolution operation is executed sequentially and although in the parallel approach, the work is divided into smaller tasks that can be computed simultaneously across multiple cores and this distribution significantly reduces the processing time and speeds up the overall training process such as when processing an image through a convolutional layer and the work of processing different parts of the image can be divided between multiple processors and this allows for faster data processing and results in a more efficient training phase for the neural network [22].

- Parallel gradient descent: Gradient descent is an optimization technique used to reduce error in a neural network by updating its weights based on the calculated gradients and in parallel gradient descent where the weight update process is distributed across multiple cores, allowing the model to update the weights in parallel rather than sequentially and this can significantly reduce the time it takes for the network to converge to an optimal solution and in a typical gradient descent process and the weights are updated one by one in sequence and in parallel gradient descent, however, each core computes the gradient for a subset of the weights and updates them simultaneously and this approach speeds up training by taking advantage of multiple processors working in tandem and reducing the overall time spent on each training iteration.

Using parallel processing techniques such as parallel convolution and parallel gradient descent and training deep neural networks becomes more efficient and especially when working with large datasets or complex models and his not only reduces computational time but also enhances the ability to scale models, allowing for faster experimentation and better utilization of available hardware resources [23].

3.4 Advanced optimization algorithms

- SGD: Weight update using a random subset of data:

$W^{t+1}=W^t-\eta \times \nabla L\left(W^t\right)$          (9)

where, $\left(L\left(W^t\right)\right)$ is the loss function at time $(t)$, and $(\eta)$ is the learning rate.

- Adam algorithm: An adaptive gradient method combining momentum and adaptive learning rates:

$m_t=\beta_1 \times m_{t-1}+\left(1-\beta_1\right) \times g_t$           (10)

$v_t=\beta_2 \times v_{t-1}+\left(1-\beta_2\right) \times g_t^2$           (11)

$m_{t_{h a t}}=\frac{m_t}{\left(1-\beta_1^t\right)}$           (12)

$v_{t_{\text {hat }}}=\frac{v_t}{\left(1-\beta_2^t\right)}$           (13)

where, $\left(g_t\right)$ is the gradient at time $(t)$, and $\left(\beta_1\right),\left(\beta_2\right)$ are hyperparameters.

- RMSPROP algorithm: Using a moving average of squared gradients to update the weights:

$v_t=\gamma * v_{t-1}+(1-\gamma) \times g_t^2$           (14)

$W^{t+1}=W^t-\left(\frac{\eta}{\operatorname{sqrt}\left(v_t+\varepsilon\right)}\right) \times g_t$           (15)

where, $(\gamma)$ is the decay factor, and $(\varepsilon)$ is a small constant to avoid division by zero.

- SGD: SGD is a variant of the standard gradient descent algorithm where instead of computing the gradient over the entire dataset, a random subset of the data (called a mini-batch) is used at each iteration and this can help speed up the optimization process and make the model more adaptable to large datasets and in standard gradient descent, weights are updated based on the average gradient computed over the entire dataset. In contrast, SGD updates the weights using the gradient of just one or a few data points, which introduces randomness into the process and this randomness can help avoid local minima and improve convergence in large and complex datasets, but it may cause higher variance in the updates. However, over time, the process of random sampling from the data enables the network to converge towards an optimal solution [24].

Parallel processing techniques, whether in convolution operations or gradient descent, rely on distributing computations across multiple processing cores to reduce training time and enhance hardware utilization efficiency and in parallel convolution, the multiplication and accumulation operations associated with each position in the output feature maps are divided among multiple cores or processing units, significantly reducing the overall computational time and common approach involves reshaping input data using the im2col transformation, which converts input matrices into a format suitable for highly efficient matrix multiplications using libraries like cuBLAS on GPUs where the efficiency of this process primarily depends on minimizing memory access times, leveraging shared memory or cache memory to reduce delays caused by fetching data from global memory.

One of the key challenges in implementing parallel convolution is memory access conflicts when multiple cores process adjacent data regions and this can lead to contention over the same memory locations or redundant data fetching, ultimately reducing computational efficiency also, work distribution across cores must be carefully managed to ensure load balancing, as uneven task allocation can lead to resource underutilization due to faster cores waiting for slower ones to finish. Furthermore, handling boundary conditions between parallel processing regions is crucial, as some filters may require data samples outside the allocated region for a core. Strategies such as padding or overlapping tile processing are often used, where some data is redundantly processed to ensure accuracy and in parallel gradient descent, various strategies are employed to accelerate weight updates during training and in the traditional parallelization model, data is divided into mini-batches, where each core computes gradients for its assigned subset and then aggregates these gradients using an all-reduce operation to ensure synchronized weight updates across all cores. While this approach balances computational efficiency and model stability, it faces challenges related to communication overhead, as gradient aggregation can become a bottleneck in distributed systems with limited bandwidth.

Asynchronous updates are another strategy, where each core updates weights immediately after computing its gradients without waiting for other cores while this reduces waiting time, it introduces the "stale gradients" issue, where some cores compute gradients based on outdated weight values that have not yet been updated by others and this can slow down convergence or even destabilize training if not controlled using adaptive gradient correction techniques or learning rate scheduling.

Another challenge is memory consistency, as multiple cores updating the same weight simultaneously can lead to race conditions and this requires synchronization mechanisms such as locks or atomic operations, which introduce additional computational overhead. Some algorithms, like Hog-wild, relax strict synchronization constraints, allowing overlapping weight updates. While this improves efficiency, it increases gradient variance, requiring adaptive learning rates to maintain stability and in multi-device systems, inter-node communication plays a crucial role in training performance and transferring gradients across networked devices can become a limiting factor, especially with bandwidth constraints and techniques like gradient compression or low-precision updates help reduce data transfer overhead, improving overall performance in distributed training.

Overall, the effectiveness of parallel processing techniques in neural networks depends on several factors, including efficient workload distribution, reduced memory access latency, controlled synchronization overhead, and optimized data communication in distributed environments. Addressing these aspects leads to faster training times and better hardware utilization, making deep learning models more efficient and scalable.

- Adam algorithm:

The Adam algorithm is an advanced optimization method that combines the benefits of two other optimization techniques: Momentum and RMSPROP. Adam uses two moving averages, one for the gradients (first moment) and one for the squared gradients (second moment), to adapt the learning rate for each parameter.

1. Momentum helps accelerate convergence by using past gradients to smooth out the updates.
2. Adaptive learning rates adjust the size of updates based on the variance of past gradients.

This combination of momentum and adaptive learning rates allows Adam to adapt to the characteristics of the loss function, providing faster convergence and better performance for many deep learning tasks and the algorithm computes moving averages of the gradients and their squares, which help adjust the learning rate dynamically for each weight, based on the historical behavior of the gradients.

Adam is highly popular because of its efficiency, minimal memory requirements, and generally fast convergence [25].

- RMSPROP algorithm:

The RMSPROP algorithm is another optimization technique that aims to solve the problem of slow convergence in standard gradient descent and to handle situations where the gradients are sparse or noisy.

RMSPROP addresses this by maintaining a moving average of the squared gradients for each parameter, allowing the algorithm to scale the learning rate for each weight individually and by dividing the gradient by the root of this moving average, it normalizes the gradient, ensuring that it remains within a reasonable range and avoids large fluctuations and the key idea behind RMSPROP is that parameters with larger gradients will have smaller updates, and parameters with smaller gradients will have larger updates and this adaptive adjustment of the learning rate helps stabilize training, especially when the learning rate is set globally too high or too low for the whole network and the decay factor (γ) controls how quickly past gradients are forgotten, and the small constant ϵ prevents division by zero, ensuring numerical stability and in practice, Adam and RMSPROP are very effective for deep learning tasks due to their ability to handle complex loss landscapes and sparse gradients [26].

3.5 Evaluation and testing

- Accuracy: Measuring the ratio of correct predictions:

$A=\frac{\text { Correct }_{\text {Predictions }}}{\text { Total }_{\text {Predictions }}}$            (16)

- Precision and recall:

$P=\frac{\text { True }_{\text {Positives }}}{\left(\text { True }_{\text {Positives }}+\text { False }_{\text {Positives }}\right)}$            (17)

$R=\frac{\text { True }_{\text {Positives }}}{\left(\text { True }_{\text {Positives }}+\text { False }_{\text {Negatives }}\right)}$            (18)

- Computational efficiency: Evaluating the speed of the algorithm:

$S=\frac{1}{T_{\text {total }}}$            (19)

- Error analysis: Calculating the mean squared error (MSE):

$E=\left(\frac{1}{N}\right) \times \operatorname{sum}\left(\left(y_i-y_{\text {hat}_i}\right)^2\right)$     (20)

where, $y_i$ is the actual value and $y_{\text {hat}_i}$ is the predicted value.

3.6 Analysis and discussion

- Parallel performance gains:

Speedup $=\frac{T_{\text {serial }}}{T_{\text {parallel }}}$           (21)

where, $T_{\text {serial }}$ is the time taken for execution in a single-core system and $T_{\text {parallel }}$ is the time taken in a multi-core system and to deepen the mathematical exploration of the parallel algorithm for facial expression recognition in the context of student engagement monitoring, we will delve into more advanced concepts and this includes leveraging parallel computing for real-time processing, mathematical modeling for facial expression recognition, and assessing student engagement based on facial features and the following hypotheses and mathematical models address the core aspects of the problem: parallelism, feature extraction, dimensionality reduction, classification, and engagement prediction [27].

Advanced Mathematical Hypotheses

Hypothesis 1: Optimal parallelization for facial expression recognition performance

Hypothesis: The parallelization of the feature extraction and classification processes in facial expression recognition will achieve a super-linear speedup if the task is split efficiently and the performance can be modeled by applying Gustafson's Law, which accounts for scalable parallelism in computational tasks with increasing problem sizes.

$T_{\text {parallel }}=T_{\text {serial }}+(1-S) \times\left(\frac{P}{N}\right)$           (22)

where,

- $T_{\text {parallel}}$ is the total time taken with parallel processing.

- $T_{\text {serial}}$ is the serial time for a single processor.

- S is the scalability factor of the task (i.e., how well the task can be parallelized).

- P is the problem size.

- N is the number of processors.

In this model, as the problem size increases, the impact of parallelization on performance improves, showing that a larger dataset will benefit more from parallel algorithms [28].

Hypothesis 2: Efficient feature extraction for facial expression recognition

Hypothesis: The feature extraction process using Local Binary Patterns (LBP) and Haar-like features can be optimized with parallel computation and the relationship between the number of facial features and processing time can be described by a matrix factorization model [29].

$F_{\text {extracted }}=$ Matrix $_{\text {Factorization }(F)}$           (23)

where,

- $F_{\text {extracted }}$ represents the matrix of features extracted from the image.

- F represents the original raw image data.

In this case, LBP and Haar-like features can be processed in parallel by breaking down the image matrix into smaller blocks and distributing these blocks to different processors, speeding up the extraction process.

Hypothesis 3: Impact of dimensionality reduction on parallel algorithm performance

Hypothesis: The PCA and t-distributed stochastic neighbor embedding (t-SNE) methods for dimensionality reduction will improve the parallel algorithm’s performance by reducing the number of features to be processed, allowing for more efficient classification and the relationship between the dimensionality reduction and the time complexity can be modeled by:

$T_{\text {reduced }}=T_{\text {original }} *\left(\frac{D_{\text {original }}}{D_{\text {reduced }}}\right)$    (24)

where,

- $T_{\text {reduced }}$ is the time after dimensionality reduction.

- $T_{\text {original }}$ is the original time taken without dimensionality reduction.

- $D_{\text {original }}$ is the original number of dimensions.

- $D_{\text {reduced }}$ is the reduced number of dimensions.

The reduction in dimensionality leads to faster processing in parallel systems by decreasing the number of computations required for feature classification and engagement prediction [30].

Hypothesis 4: High-performance classification using parallel SVM

Hypothesis: The use of parallel SVM for classification will improve classification accuracy and reduce computational time compared to serial methods and the relationship between the training time and number of support vectors can be described using the following model:

$T_{\text {train }}=\frac{1}{N} * \sum_{i=1}^n\left(\frac{1}{\left\|x_{-} i-c_{-} i\right\|}\right)$           (25)

where,

- $T_{\text {train}}$ is the time for training the classifier.

- N is the number of support vectors.

- $x_i$ represents the individual data points.

- $c_i$ is the center of the support vector class.

By parallelizing the training process, where each processor handles different support vectors, the overall time complexity of the SVM classifier is reduced, making it feasible for real-time monitoring [31].

Hypothesis 5: Estimating student engagement using facial expressions

Hypothesis: The level of student engagement can be quantitatively predicted from facial expression data using regression models and the relationship between facial expressions (represented as a vector of features) and engagement levels can be modeled as follows:

$T_{\text {engagement }}=$ alpha $* \sum_{i=1}^n\left(w_i * E_i\right)+$ beta           (26)

where,

- $T_{\text {engagement}}$ is the predicted level of engagement.

- $w_i$ are the weights for each feature $E_i$ extracted from facial expressions.

- $E_i$ represents the facial expression scores (e.g., happiness, sadness, surprise).

- Alpha and beta are regression coefficients [32].

This research has presented a comprehensive mathematical framework for a parallel algorithm that enables effective facial expression recognition for student engagement monitoring in classrooms and by integrating advanced techniques from computer vision, machine learning, and parallel computing, the proposed system achieves high accuracy and computational efficiency and the mathematical models developed in this study provide insights into the relationships between key components, such as parallelization, feature extraction, dimensionality reduction, and classification and the empirical evaluation showcases the algorithm's ability to deliver real-time feedback on student engagement, which is crucial for improving the learning experience and adapting teaching strategies to better meet the needs of students. Going forward, further exploration of the mathematical underpinnings, including the integration of additional optimization techniques and the exploration of more advanced parallel architectures, could lead to even more robust and scalable solutions for student engagement monitoring and the successful implementation of this parallel algorithm paves the way for the widespread adoption of facial expression recognition technology in the field of education, ultimately enhancing the quality of teaching and learning.