Facial Expressions Recognition Based on Delaunay Triangulation of Landmark and Machine Learning

During the past decades, there is a lot of research interest in the development of efficient Facial Expression Recognition (FER) systems intending to improve quality of life. The range of FER applications varies from robotics, e-health, enhanced learning and e-crimes that analyze facial expressions and estimate serious tendencies. On the other hand, FER can benefit medical applications by assisting patients with facial paralysis during rehabilitation treatment. Although a robotics teacher can provide a better learning experience to students and can help them build a well understanding of their feelings. For security and surveillance at airports, several FER systems still face several challenges such as reliability and robustness to ensure the safety of drivers due to huge variability in facial appearance such as changes of light, sunglasses, and masks, placement of head and partial occlusions caused to hairs.

Indeed, facial expressions play a very important role in social communication between humans to convey energies, emotions and information without using any words. However, understanding the internal states of a person without seeing their face or facial expressions can be complex and difficult task. There are various channels for conveying non-verbal information to observers such as body gestures, face and voice, but facial expressions remain one of the best mediums in terms of capturing emotional state and signal pain, alertness, intentions and personality traits. In addition, verbal and vocal parts only contribute 7% and 38% respectively, to the impact of the spoken message while facial expression contributes 55% [1]. Thus, the face reflects much more ability to communicate emotions and the recognition of facial expression offers a very effective way to measure the different emotional states of human beings.

The human face is capable of generating many different facial expressions. In the field of scientific research, only six of these expressions are identified according to work-in-depth of behavioral specialist Ekman [2] called Anger (AN), Disgust (DI), Fear (FE), Happiness (HA), Sadness (SA), and Surprise (SU) as “atomic expressions” [2-4]. They also proved that these six expressions are unique among different races, religions, cultures and groups [4]. Some researchers consider the neutral face as a standard set of seven emotions [5-7]. Classifying facial images using the discriminative features of facial expressions can be a complicated task for machines. Machines need to discover and correctly classify numerical facial expressions. However, we need well-designed and adopted systems, which can be achieved through a wide variety of special textural expressions and significant positions of landmarks.

Despite the rapid growth of FER applications, FER system suffers from the main real challenge to find the reliable and efficient feature extraction method that reduces and extracts relevant discriminative features of facial expressions, that giving very appreciable accuracy rate, a complete description of facial expression and fast classification time. Several extraction techniques of features emotions [8-16] can make a significant contribution to classifying facial images. We can distinguish between two types: geometric techniques as well as non-geometric techniques, and hard to pin down for multiple reasons. Several emotional expressions cannot be distinguished only by subtle discriminative features in facial expression images like jupons and china. These facial images are often expressed in very smoothest ways.

In this paper, we propose a new approach addressing some important issues in the recognition of facial expressions through interesting facial positions with less complexity in terms of computation time. The approach is mainly based on landmarks based on the Delaunay triangulation technique to extract only the discriminative features of facial emotions in the face image as a one-dimensional vector. This vector will be transmitted to machine learning and be classified among six facial expressions. Most machine learning techniques are applied to evaluate the performance of the proposed approach and find out which one is better suited to our FER system. This paper seeks to achieve the following contributions:

(1) To the best of our knowledge, few studies have investigated the problem of facial expression features extraction and FER classification and recognition in the context of smoothest textural regions using the enhanced Delaunay Triangulation of landmark performance and machine learning techniques;

(2) The proposed approach is based on the enhanced Delaunay Triangulation of landmark for reduced and relevant facial features extraction and six machine learning techniques for expression’s recognition;

(3) Experiments that are conducted on four standard facial databases showed that the proposed approach achieves a precision of 96.94% for FER recognition.

The rest of this paper is organized as follows. Section 2 presents related works for features extraction and FER recognition. Section 3 details the proposed approach. Section 4 presents the results obtained from our experiments. Finally, Section 4 concludes the paper and provides potential future work.

This process of extracting relevant and discriminative landmarks in facial images is based on the Delaunay triangulation by localizing the face, applying an active appearance model, adapting and normalizing the face’s model, triangularization landmarks and constructing the feature vector in the face images as illustrated in Figure 1.

1.png

Figure 1. Obtained results of landmarks on six well-known facial expressions

This process is necessary before machine training and test algorithms to achieve high accuracy and increase performance of the system. In the next subsections, the different phases composing the proposed approach are detailed.

3.1 Face detection phase

For this phase, the input is an image of and the output is an image enclosing the detected face. This phase is automated by applying the Viola and Jones algorithm which solves the complex problems of edge and changes in brightness [18]. Our choice of Viola and Jones algorithm is justified by its noise invariant and good scaling in detecting objects from images in several different applications. To detect faces from facial images, the Adaboost of the Viola-Jones algorithm consists of two successive steps: (1) Extraction of HAAR features; (2) classification using Adaboost.

3.2 Building the active appearance model (AAM)

The active appearance model (AAM) [13, 14, 16, 18] aims to create a face model and describe the main features of the face namely nose, mouth, eyes and eyebrows by representing faces images labeled with landmarks that define the general geometric shape of a face. Shape models are essential to generate a statistical model in order to customize and control the variations in the shape of a face. In face representation phase, the AAM model is approximated with any new example. The AAM model is constituted of three main phases: 1 Shape model-building phase; 2) Texture model-building phase; 3) Combined appearance model.

Phase 1. Building the Shape Model [19]. Apply shape model L of 2-dimensional including n=68 points. The points are placed in face regions that contain rich textural information. Each instance s of the shape model can be described as a vector consisting of 2n elements:

$s=\left(x_{1}, x_{2}, \ldots, x_{n}, y_{1}, y_{2}, \ldots, y_{n}\right)^{T}$    (1)

To detect these points from face images, the process of the shape model consists of four successive steps:

(1) First, the Generalized Orthogonal Procrustes Analysis (GPA) [20] is applied to shape model to eliminate the geometric effect caused by translation, rotation and scaling. In addition, the GPA algorithm generates additional shapes by mirroring the learning shapes horizontally as its goal is to achieve a reliable, efficient, and robust recognition system. This returns new shapes for training of 2N examples.

(2) The mean shape $\bar{s}$ can be computed on new learning data set as follows:

$\bar{s}=\frac{1}{2 N} \sum_{i=1}^{2 N} L_{i}^{\prime}$    (2)

where, $L_{i}^{\prime}$ denotes a new training data set.

(3) The Principal Component Analysis (PCA) is applied to this data set to calculate Eigen-vectors and Eigen-values. This allows us to build the AAM model and subsequently to generate new shapes according to the constructed model. The m components with the biggest Eigen-values will be selected as the shape components L₁ to L_m.

$s=\bar{s}+\sum_{i=1}^{m} p_{i} * b_{i}$    (3)

where, $\bar{s}$ denotes the average form; p_i denotes a set of orthogonal modes of grey variation; b_i denotes a set of grey-level parameters

Phase 2. Building the texture model [19]. The second main phase in AAM model is to build the texture model that transforms the input images of large dimensions into a linear subspace of cleaned faces to reduce the representation space of the information. Steps that are used in the texture model are listed below:

(1) Image-Texture piecewise transformation: The first step of building texture Model consists of converting each image I(x) into a basic form $\bar{s}$. This transformation is performed by applying apiece wise function I(W(x, p)) [16], which transforms an input image I(x) with the shape s and parameters p to the basic form $\bar{s}$ as shown in Eq. (3) The average of the texture images A₀(x) can be calculated as follows:

$A_{0}(x)=\frac{1}{N} \sum_{i=1}^{N} I(W(x, p))$    (4)

(2) Image-eigenvalues transformation: In this step, the texture images are converted into components using PCA (Principal Components Analysis). We keep only k components of appearance with the biggest eigenvalues A₁(x) to A_k(x). Finally, the AAM model A(x) can be represented as an average appearance A₀(x) and a linear combination of the one-appearance vectors A_i(x).

$A(x)=A_{0}(x)+\sum_{i=1}^{m} \lambda_{i} * A_{i}(x)$    (5)

Phase 3. Building the combination appearance model [19]. The AAM appearance model is the combination of the shape model and the texture model. For this, we combine the appearance parameters λ=(λ₁, λ₂, ..., λ_m) and the shape parameters p=(p₁, p₁, ..., p_m).Using Eq. (5). The image A(x) in the form of the base shape S₀ can be calculated. After that, the image A(x) can be transformed into the form S using the warp function W(x, p).

3.3 Model adaptation phase

After the AAM model phase, the adaptation of the AAM model is necessary before identifying the landmark points. This step allows adapting the trained model to an unknown input image I(x). We solve the optimization problem of Eq. (6) and find the optimal parameters p and λ.

$\arg \min _{p, \lambda} \sum_{x \in s_{0}}\left[A_{0}(x)+\sum_{i=1}^{m} \lambda_{i} A_{i}(x)-I(W(x, p))\right]^{2}$    (6)

3.4 Shape normalization phase

This phase aims to eliminate the effects of scale, rotation and translation between all shapes using the ASM model to identify the relevant facial information that will greatly increase and improve the recognition rate of the proposed system. The Generalized Procrustes Analysis (GPA) [20] is applied to normalize the different shapes that will greatly influence the quality and reliability of the FER system.

Figure 2 shows an example of shapes before and after the application of GPA.

2.png

Figure 2. Shapes before and after the application of GPA

3.5 The Delaunay triangulation phase

At this phase, faces are represented by geometric shapes. Each shape contains68 facial landmarks. It is detected using the positions of eyebrows, eyes, nose, lips and the contour of the face. These facial landmarks change over time according to continuous movements of facial muscles. These landmarks are then connected to obtain a graph. The density of the graphics varies in each facial expression due to facial movements. We use the famous Delaunay triangulation to connect a set of landmarks to construct a triangular mesh. Therefore, after the graph is connected, we obtain 109 triangles formed on the human face as shown in Figure 3.

3.png

Figure 3. A triangular mesh (a) Facial landmarks, (b) C among landmarks

The proposed system deals with the recognition of facial expressions. The difference between the facial expressions of an individual or several individuals naturally occurs on facial movements. At the first step, we use the AAM model to locate landmarks of facial components as named eyes, eyebrows, nose, and mouth. Then, we use Generalized Procrustes Analysis (GPA) [20] for face normalization to reduce as much as possible the difference in geometric shapes. The obtained results are represented by Delaunay's triangulation as a connected graph composed of triangles that simulate the geometric shape of facial expression. Finally, we apply machine learning to classify the facial expressions to improve the reliability and robustness of face recognition method.

3.6 Feature vector construction phase

This phase begins with a geometric representation of the features extracted from a face image to design a connected graph composed of several triangles as illustrated in Figure 4. Each triangle has three points called Centroid, Incenter, and Circumcenter. Two different approaches can be used to define the final representation of the data. The first is to use the distances and slopes between the different points of a triangle including Centroid, Incenter, and Circumcenter. The second only uses the coordinates of points (its abscissas and ordinates) as an information tool. We choose to apply the localization of points for final data representation that provides a higher recognition rate compared to the first approach based on several test results. After testing the two approaches used, we can decide to perform the second approach with three points of the triangle as well as Centroid point to find the best results. Now, a one-dimensional feature vector is used to represent each expression in a face image. All the feature vectors are finally constructed are stored in a database representing all the facial expressions of the face images of the initial database. Let F be the database containing feature vectors defined by:

$F=\left(q_{1} ; q_{2} ; \ldots ; q_{m}\right)$    (7)

where, q_i denotes the shape vector; m is the number of shapes.

4.png

Figure 4. Construction of features vector

In this section, we will detail the implementation of the proposed work and the obtained results using the proposed approach.

5.1 Data sets

To validate the proposed facial expression recognition approach, we have tested our proposed framework on four well-known facial expressions databases:

(1) Extended Cohn-Kanade (CK+) dataset is a complete dataset for action unit and emotion-specified expression which contains 593 sequences from 123 subjects [27];

(2) Japanese Female Facial Expression (JAFFE) dataset contains 213 expressions of 10 Japanese females with each object posed four examples for each of the seven basic expressions (six emotional expressions plus neutral face) [28];

(3) Karolinska Directed Emotional Faces (KDEF) dataset consists of 4900 images of human facial expressions. The set of images contains 70 individuals displaying seven different emotional expressions [29];

(4) Multimedia Understanding Group (MUG) dataset contains 1462 images and provides a set of landmark point annotations for 1222 images [30].

All experiments were carried out on Intel(R) Core (TM) i5-3217U CPU @ 1.80GHz with 4GB RAM Laptop, using Windows 8 (64 bit) stem. We have used Dlib prediction model library for extracting 68 facial landmark points using open CV and Python [31]. Note that AAM model achieved 100% detection accuracy in CK+, JAFFE, MUG database, and 99.2% in KDEF database.

5.2 Splitting the data

We divide the data set into two parts: the first part uses the training data set with a size of 80%, the test part has a size of 20%. We have carried out several performance experiments using the most used learning algorithms. Our work is based on the applications of the Delaunay triangulation of landmark and six classifiers to validate the accuracy and execution time of each one to find out which one gives high accuracy and short execution time.

5.3 Evaluation metrics

We evaluated and compared the performance in automatically recognizing facial expressions in terms of effectiveness and efficiency.

5.3.1 Effectiveness metrics

Accuracy (A) to measure how a classifier can expect the correct predictions.

Precision (P) to identify the number of relevant faces among the classified ones.

Recall (R) to identify the number of corrected classified faces images over all the faces images.

F1-score to evaluate a weighted Precision and Recall.

$A=\frac{T P+T N}{T P+T N+F P+F N}$    (8)

$P=\frac{T P}{T P+F P}$    (9)

$R=\frac{T P}{T P+F N}$    (10)

$F_{1}=2 \times \frac{P \times R}{P+R}$    (11)

where, TP, True Positive, is the number of facial expressions which are correctly recognized; FP, False Positive, is the number of facial expressions which are incorrectly recognized; TN, True Negative, is the number of correct facial expressions that are incorrectly recognized; FN, False Negative, is the number of correct facial expressions that are incorrectly recognized.

PSNR [29] is a logarithmic function of Mean Square Error (MSE) interpreted as a corrected version of the Signal-to-Noise Ratio. A perfect similarity between two images is expressed when PSNR→∞, which means that the two images are perfectly identical. The PSNR is calculated using Eq. (12):

$\operatorname{PSNR}(X, Y)=10 \log _{10}\left(\frac{255^{2}}{\frac{1}{\mathrm{M} \times \mathrm{N}} \sum_{i=1}^{M} \sum_{j=1}^{N}\left(X_{i j}-Y_{i j}\right)^{2}}\right)$    (12)

where, x_ij, y are the pixel (i, j) in the original image x and the watermarked imagey respectively. M×N is the size of the image.

Average difference (AD) [32] expresses the similarity between the original image and the distorted one. Good classification means having the lowest difference value. Usually, it can be considered good if AD is close to or equal to zero. This metric is computed as follows:

$A D(x, y)=\frac{1}{N M} \sum_{i=1}^{N} \sum_{j=1}^{M}\left|x_{i, j}-y_{i, j}\right|$    (13)

Maximum difference (MD) [32] provides the maximum difference between the original image and the distorted one. It is used to measure the maximum accuracy of the proposed descriptors and it is defined as follows:

$M D(x, y)=\max \left|x_{i, j}-y_{i, j}\right|$    (14)

Structural content (SC) [32] is expressed as follows:

$\operatorname{SC}(x, y)=\frac{\sum_{i=1}^{N} \sum_{j=1}^{M} \quad x_{i, j}{ }^{2}}{\sum_{i=1}^{N} \sum_{j=1}^{M} \quad y_{i, j}{ }^{2}}$    (15)

Normalized absolute error (NAE) [32] expresses the similarity between the original image and the distorted one. A lower NAE value shows that image is of good classification. It is defined as follows:

$\operatorname{NAE}(x, y)=\frac{\sum_{i=1}^{N} \sum_{j=1}^{M} \quad \left|x_{i, j}-y_{i, j}\right|}{\sum_{i=1}^{N} \sum_{j=1}^{M} \quad \left|x_{i, j}\right|}$    (16)

5.3.2 Efficiency metrics

We also use execution time for comparing the presented approach using six classifiers. In the next section, we explain different experiments and the results

5.4 Experiments on CK + database

The CK+ expression dataset is a combination of posed and spontaneous expressions. The resolution of all images for training, validation and testing of size 256×256. The images set ranging from18 to 50 years old age consisting of 69 female, 81Euro-American, 13 Afro-American, and 6 other groups have 326 images of peak facial expressions for seven emotion categories are anger (AN), contempt (CO), disgust (DI), fear (FE), happiness (HA), sadness (SA), and surprise (SU), varying between posed and non-posed facial expressions of 210 adults. We consider the best quality to consider these properties of a good dataset. Figure 6 demonstrates samples of CK + database.

6.png

Figure 6. Samples of CK + database

5.4.1 Evaluation and comparison regarding accuracy

Table 1 shows the comparison of the precision, recall, and F1-score of the proposed approach using the most standard classification approaches (KNN [21], NB [22], SVM [23], DT [24], QDA [25], RF [26]). The QDA [25] based on Delaunay triangulation of landmark attains a promising precision as compared to other classifiers with 100% on CK+ database. Besides, QDA with the Delaunay triangulation achieves 100% Recall and demonstrates a higher F1 score rate of 100%.

Table 1. Accuracy comparison for different classification approaches on CK+ database

Table 2. Confusion matrix of CK+ database of KNN and Delaunay triangulation

Table 3. Confusion matrix of CK+ database using NB and Delaunay triangulation

Table 4. Confusion matrix of CK+ database of KNN using DT and Delaunay triangulation

Table 5. Confusion matrix of CK+ database of KNN using QDA and Delaunay triangulation

Confusion matrix are measured between the predicted facial expressions and actual facial expressions. The results of our proposed approach on CK+ Dataset are shown in Tables 2, 3, 4, 5 and 6 and noticeably outweigh seven of well-known facial expressions. According to Table 2, 3, 4, 5, 6 and 7, SVM and QDA based on Delaunay triangulation show close results with respect to expression facial features and even QDA is a bit better. The reason comes from the relevant feature extraction in the Delaunay triangulation algorithms and the powerful classifier QDA. QDA considers the highest importance to the most interesting point of high texture.

Table 6. Confusion matrix of CK+ dataset of KNN using RF and Delaunay triangulation

Table 7. Confusion matrix of CK+ database of KNN using SVM and Delaunay triangulation

5.4.2 Evaluation and comparison regarding similarity measures

The similarity is evaluated through all mentioned measures needed for classifying an input image. Tables 8 shows a comparison of the similarity measures on CK+ Dataset for six classifiers. From that Table 8, it is noted that the value of similarity of QDA based on Delaunay triangulation is better than other classifiers (KNN [21], NB [22], SVM [23], DT [24], RF [26]). Such high similarity measure and low error indicated that the proposed Delaunay triangulation will achieve higher similarity values (100%) and is therefore suitable for facial expression recognition.

Table 8. Comparison of the similarity measures on the CK+ database for six classifiers

5.5 Experiments on KDEF database

KDEF database regroups facial expression images from 140 amateur actors (70 men and 70 women) all of them between 20-30 years old, at five different view angles. In our work, we have only considered frontal images. We have conducted the experiments in 490 facial expression images (70 Surprise, 70 Disgust, 70 Happy, 70 Sad, 70 Angry, 70 Fear, and 70 Neutral). All images are in the size of 128x128 and divided into 8x8 = 64 equally blocks. Figure 7 illustrates samples of KDEF dataset.

7.jpg

Figure 7. Samples of KDEF database

5.5.1 Evaluation and comparison regarding accuracy

According to the results of Table 9, QDA shows best accuracy of 96.94%, recall 97.26% and F1-score 97.10% than other classifiers (KNN [18], NB [19], SVM [20], DT [21], RF [23]). The reason goes to the introduction of shape normalization as well as localization of face’s components points.

Tables 10-15 indicate remarkable outcomes of the proposed method with the most standard machine learning in the recognition of facial expressions. As it is shown in Table 13-15 in comparison of QDA, SVM, and DT with other classifiers, values of the confusion matrix of these classifiers are high, while the other classifiers KNN, NB, and RF for FER, are low.

Table 9. Accuracy comparison for different classification approaches on KDEF database

Table 10. Confusion matrix of KDEF database of KNN and Delaunay triangulation

Table 11. Confusion matrix of KDEF database using NB and Delaunay triangulation

Table 12. Confusion matrix of KDEF database of KNN using DT and Delaunay triangulation

Table 13. Confusion matrix of KDEF database of KNN using QDA and Delaunay triangulation

Table 14. Confusion matrix of KDEF database of KNN using RF and Delaunay triangulation

Table 15. Confusion matrix of KDEF database of KNN using SVM and Delaunay triangulation

5.5.2 Evaluation and comparison regarding similarity measures

In Table 16, values concerning PSNR, NAE, and MSE of the QDA classifier are around 57.25, 0.0612 and0.1224 respectively. They are the highest relatively those of (KNN [21], NB [22], SVM [23], DT [24], and RF [26]).

Table 16. Comparison of the similarity measures on the KDEF database for several classifiers

5.6 Experiments on JAFFE database

In this experiment, JAFFE database is exploited. This dataset includes 213 peak facial expressions from ten subjects. Seven emotion categories are considered, happiness, sadness, surprise, anger, disgust, and natural. The gray level images are of size 256×256. We used the fdlibmex library, free code available for MATLAB for face detection. We normalized all the evaluated face images before the experimentation in size of 128×128 pixels. Figure 8 presents samples of the JAFEE database.

8.jpg

Figure 8. Simples of JAFFE database

5.6.1 Evaluation and comparison regarding accuracy

In Table 17, accuracy, recall, and F1-score values are computed for all six classifiers based on Delaunay triangulation during the classification phase. The obtained results of SVM are slightly better than other classifiers (KNN [21], NB [22], DT [24], RF [26]) as their higher values are respectively 71.43, 73.57 and 33.61. SVM succeed on JAFEE database.

Table 17. Comparison of the similarity measures on the KDEF database for several classifiers

Table 18, 19, 20, 21, 22, and 23 describes the classification performance based on the proposed Delaunay triangulation for facial feature extraction with the most standard classifiers. It shows that proposed Delaunay triangulation features with QA and SVM predict better results for all classes of facial expressions.

Table 18. Confusion matrix of JAFFE database using KNN and Delaunay triangulation

Table 19. Confusion matrix of JAFFE database using NB and Delaunay triangulation

Table 20. Confusion matrix of JAFFE database using DT and Delaunay triangulation

Table 21. Confusion matrix of JAFFE database using QDA and Delaunay triangulation

Table 22. Confusion matrix of JAFFE database using RF and Delaunay triangulation

Table 23. Confusion matrix of database of JAFFE Dataset using SVM and Delaunay triangulation

5.6.2 Evaluation and comparison regarding similarity measures

Table 24 reports some calculated similarity measures, those of the proposed PSNR are very high and tend to 43 db. Furthermore, Table 24 demonstrates higher PSNR values and lower NAE values of 46.7077 and 0.5714 correlation in QDA based on Delaunay triangulation.

Table 24. Comparison of the similarity measures on the JAFFE database for several classifiers

5.7 Experiments on mug database

The MUG [30] Facial Expression database contains 86 subjects performing facial expressions. The background was a blue screen. In this dataset participated 86 people, all of Caucasian origin between 20 and 35 years of age, 35 women and 51 men. But only 52 subject images are accessible to authorized users. A total we have 50 to 160 images sequence are available, For each expression. Each image was saved with a jpg format, 896×896 pixels and a size ranging from 240 to 340 KB. we have used 1260 images of the MUG database, 180 images for each facial expression. Figure 9 illustrates samples of MUG database.

9.png

Figure 9. Samples of MUG database

5.7.1 Evaluation and comparison regarding accuracy

In Table 25, accuracy, recall and F1-score concerning QSA based Delaunay triangulation are 100%. They are the highest relatively those of other classifiers.

Table 25. Accuracy comparison for different classification approaches on MUG database

Table 26. Confusion matrix of MUG database using and Delaunay triangulation

Tables 26, 27, 28, 29, 30, and 31 are respectively confusion matrix of the MUG database using KNN, NB, DT, QDA, RF and SVM.

Comparing confusing matrix results of the proposed Delaunay triangulation in Tables 26, 27, 28, 29, 30, and 31 illustrates QDA has higher matched values. Although RF demonstrates weaker values in this database.

Table 27. Confusion matrix of MUG database using NB and Delaunay triangulation

Table 28. Confusion matrix of CK+ database of KNN and Delaunay triangulation

Table 29. Confusion matrix of MUG database using QDA and Delaunay triangulation

Table 30. Confusion matrix of MUG database of KNN and Delaunay triangulation

Table 31. Confusion matrix of MUG database of KNN and Delaunay triangulation

5.7.2 Evaluation and comparison regarding similarity measures

Table 32 describes the similarity results based on the triangulation feature with six well-known classifiers. It shows that the proposed Delaunay triangulation feature with QDA predicts the better result for all the measures, which is MSE, PSNR, MNCC, AD, SC, and NAE.

Table 32. Comparison of the similarity measures on the MUG database for several classifiers

5.8 Discussion

The comparative analysis of six classification approaches used for facial expression recognition reveals the following important points:

(1) Among the six classification approaches, QDA based on the Delaunay triangulation gives the highest accuracy, recall and F1-score for facial expression recognition. However, DT approach gives lowest accuracy. (2) In addition, the QDA based on the Delaunay triangulation shows highest value of similarity between the predicted facial expressions and actual facial expressions with 100% in the CK+ and MUG dataset. (3) Among the six classification approaches, QDA based on the Delaunay triangulation presents the best algorithm for facial expressions recognition with highest PSNR values and lowest error rate.

However, QDA based on the Delaunay triangulation offers the best facial expressions recognition approach with the highest accuracy and recall rates and good similarity measures.