Bandaru Kanaka Durga*
| Vullanki Rajesh | Sirisha Jagannadham | Pattem Sampath Kumar | Ahmed Nabih Zaki Rashed | Kayam Saikumar
© 2023 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
The scientific community and media have increasingly recognized the significance of micro-expressions as indicators for detecting deception, as they reveal genuine emotions that individuals attempt to conceal. To capitalize on these subtle cues of deceit, researchers have developed applications capable of automatically detecting and recognizing micro-expressions, which are typically imperceptible to the human eye. Facial expressions serve as fundamental ground truth determinants in multimedia applications. Earlier models, such as GA, RFO, X-Boosting, and Gradient Boosting, demonstrate greater efficiency in terms of time and accuracy. However, not all applications are capable of detecting micro facial expressions. In this study, a deep learning-based Tiefes FCNN model is designed specifically for micro facial expression recognition. Implemented using Python software, the proposed model consists of two stages: first, pre-processing is performed using image segmentation, followed by the application of a deep learning model employing Tiefes FCNN technology in the second stage. The experimental results exhibit significant performance improvements, including an accuracy of 99.02%, precision of 98.82%, F1-score of 97.8%, PSNR of 56.31, and CC of 96.31.
micro expression, facial expression, Tiefes FCNN, deep learning
Micro-expressions (MEs) are subtle facial gestures that reveal concealed emotions and feelings, with potential applications in various fields such as clinical diagnostics, safety surveillance, and investigations [1-3]. The concept of MEs gained popularity through media exposure, including the television show ME32wsaz, and holds promise for detecting deception, as these expressions may disclose genuine sentiments [4]. The term "suppressed emotions" was first introduced by Miao et al. [4] when they identified micro-momentary expressions. Subsequently, Ekman discovered micro-expressions during a video interview of a suicidal patient in 1969 [5], and since then, numerous studies have been conducted in the field of MEs.
A key characteristic of MEs is their brief duration, usually less than 0.5 seconds [6]. MEs also tend to have lower intensities [7] and are typically undetectable by the untrained eye [8]. Automated systems for detecting and recognizing MEs in lie detection could have significant implications for national security, transportation safety, and clinical diagnosis. Although considerable effort has been dedicated to expression recognition in the past, ME detection has only recently garnered similar attention. The short duration and low intensity of MEs pose challenges for computer vision, with the lack of robust datasets further exacerbating the issue.
Several organizations have recently produced micro-expression databases, although these databases face numerous challenges. The current study introduces the CASME collection, which aims to address some of these issues. Micro-expressions, as brief, low-intensity facial gestures, differ from regular facial expressions in several aspects (Figure 1). Due to their ability to reveal genuine emotions and detect deception, MEs are considered valuable in various domains, including clinical diagnostics, national security, and interrogations [9]. Traditional lie detection methods, such as polygraphs, require subjects to remain connected to the device throughout the session, potentially prompting individuals to adopt countermeasures. In contrast, lie detectors based on MEs are less likely to induce such defensive behaviors.
Despite their potential usefulness, humans often struggle to detect and recognize MEs, possibly due to their short duration, low intensity, and fragmented action units. Although disagreements regarding the exact duration of MEs exist, a time constraint of 0.5 seconds is widely accepted [6]. Individuals tend to exercise control over their micro-expressions, making them even more challenging to detect. Furthermore, MEs often display only a fraction of the action units present in full facial expressions, with simultaneous movement units limited to the upper and lower face. To improve human performance in recognizing the seven categories of MEs established by González-Lozoya et al. [9], the Micro-Expression Training Tool (METT) was developed.
Advancements in computational power and the need to monitor numerous expressions have prompted researchers to explore automated ME assessment methods. One such approach includes the Tensor Independent Color Space (TICS), an advanced color space model presented at the ICPR conference, which was specifically designed to aid in ME detection. Subsequent research demonstrated that CIELab and CIELuv are also effective for detecting micro-expressions. The dynamic texture features of micro-expression clips in these color models are extracted using LBP-TOP, with an SVM classifier employed to identify MEs by concatenating the LBP-TOP codes' histograms into a feature space and providing them as inputs. TICS results differ from those of CIELab and CIELuv, primarily due to the distinct simulation of TICS, which maintains the three color components as separate entities. The mutual information between TICS, CIELab, and CIELuv color spaces is investigated to determine their superiority over RGB [10]. Additionally, the study introduces the topological orthogonal tensor orthogonal tensor orthogonal LBP (TO-LBP) and demonstrates that LBP-TOP is a special case of TO-LBP in three-dimensional space.
This paper builds upon earlier work [11], which presented several new findings on MEs, identified potential challenges, and offered suggestions for automating ME identification using current databases [12]. However, existing databases may suffer from issues related to the elicitation or categorization of micro-expressions. To address these concerns, the present study utilizes frame spotting, AU coding, and ME labeling to create a dataset of unstructured micro-expressions [13]. This paper provides an overview of MEs in a laboratory setting, including details on their collection, as well as considerations to assist researchers in employing the ME dataset and enhancing ME identification.
Figure 1. Facial expression
1.1 The past ME databases
This segment delved into the already-existing information. Table 1 offers a general summary for each database. The difficulty with USF-HD & Polikovsky's dataset is that they rely on pre-recorded expressions rather than real-life ones. On the other side, it is thought to be challenging to conceal ME. Micro-expressions are allowed to last longer in the USD HD than in the widely recognized half-second restriction. Micro-expressions in YorkDDT [14] have high ecological validity, but they are accompanied by additional head and face expressions that are not important to the conversation which was shown in Figure 2.
Table 1. ME existing database
|
Dataset |
Profile |
Issue(s) |
|
USF-HD |
There are one hundred MEs in it. A variety of facial expressions, both macro & micro, were requested of the participants. |
Instead of a natural reaction, posed MEs are used. |
|
Polykovsky’ sdataset |
There are 10 models in all, and they were all tasked with mimicking ME movement. |
Instead of a natural reaction, posed MEs are used. |
|
York DDT |
This document has 18 MEs extracted from Warren's research recordings. |
face expressions that seem to be very natural yet have nothing to do with the context. |
|
SMICdatabases |
The 100-fps camera captured 77 spontaneous MEs in the document. |
We may categories spontaneous MEs as either good or negative. |
Figure 2. Micro facial features extraction
Table 2. Face-labeling parameters & dataset frequencies
|
MEs |
Requirements |
Total tests performed |
|
Expression as Happiness |
Must present, either AU6 OR AU12. |
09 |
|
Expression as Sadness |
Must present AU1. |
06 |
|
Expression as Disgust |
Must present, at least one of the AU9, AU10. |
44 |
|
Expression as Surprise |
Must present, AU1+2, AU25 or AU2. |
20 |
|
Expression as Fear |
Must present, either AU1+2+4 or AU20. |
02 |
|
Expression as Repression |
Present alone or in combination of AU14, AU15 or AU17. |
38 |
|
Expression as Anxious |
PresentAU4 or other emotion related facial expressions. |
69 |
The collected database is from Polikovsky1 and USF-HD samples which are experimentally approved micro facial features. At this level of micro expression identification, intricate facial movements are not optimal due to the difficulty in recognizing them. In addition, since micro-expressions are difficult to elicit using the "telling falsehoods” Strategy, this dataset has extremely few of them. These micro-expressions were recorded in the laboratory and may be found in the SMIC database [14]. If we compare this to the posed micro-expression datasets, we can see a significant improvement. There were no AUs in this database and micro-expression classification relied only on individuals' self-reports. Problems may arise when videos may transmit many different kinds of emotions, making it difficult to provide a detailed account of what was seen (for example, eating worms can be gross but can also be humorous or startling).
The deep learning algorithms are intelligence in nature and extracting features in effective manner with layered architecture. The max pooling layer, flatten layer and dense layers have been normalize the features. The micro features are built from above three layers via mean statistical analysis.
In addition, certain facial movements, such as brow movement owing to changes in vision, may be emotion-irrelevant. There should be an end to these unnecessary facial expressions. A better micro-expression database, based on earlier challenges and shortcomings, is being built to aid the expansion of an effective automatic ME identifying system shown in Table 1 and Table 2.
In this section a brief survey on facial expressions and micro facial expressions have been discussed. The merits and de-merits were explained for future implementations. It was shown that micro-expressions durations and onset times were distributed. Micro-expression may be defined as an event with a start time less than 260.00ms or a total period less than 500.00ms, according to the distributions and estimate. These discoveries may pave the way for future research into micro-expressions. A real time interpolation framework and the first complete corpus of spontaneously micro-expressions are used in this research to demonstrate the accuracy with which humans can identify these very brief manifestations. We used a high-speed camera captures the new corpus during an experiment conducted the induction of emotional repression. When compared to human micro-expression detection performance, this scheme is initial to recognize unexpected FME. It shows great promise [15]. As a result of Parkinson's disease, we anticipated that deliberate facial expressions may be delayed (bradykinetic) and entail less movement, just as other purposeful motions are impacted. In order to test this idea, we employed high-tech computer imaging methods to measure face motion. With Parkinson's disease, facial expressions were more erratic (entropy) and took longer to reach their peak (i.e., bradykinesia). According to these results, basal ganglia are involved in influencing facial expressions.
Static facial expressions and micro-expressions are the two main types of face expressions. Face recognition system has a broad variety of potential customers, including the identification of pain, the identification of lies, and the babysitting of children. When used to identify micro-expressions, traditional convolutional neural network (CNN) approaches have two major drawbacks. A major drawback of deep architectures is that they tend to overfit smaller information [16].
Face expressions known as "micro-expressions" are brief, transitory displays of emotion that individuals often strive to keep hidden. These indicators may be used in a variety of disciplines, including the medical profession and national security, since they can be used to identify falsehoods and risky conduct. However, it is quite difficult to recognize anything with the naked sight. The dearth of spontaneous micro-expression datasets has hindered computer vision researchers from developing micro-expression detection and identification techniques. Micro-expressions evoked by faces that had been neutralised were the focus of this investigation [17].
To better understand the tiny motions of the face, researchers provide a new Eulerian paradigm in this research. It concurrently extends the length of the micro-expressions and intensifies the muscular motions. The suggested technique is able to analyse ME clips more quickly and discern tiny ME motions more clearly than other methods. It has been shown via testing on two public MER datasets that our model is faster and more accurate than any other currently available [18]. It has been shown that this technique could be used to read eight facial movements: happy, sorrow, joyful, angry, terror, wonder, disgust, and contempt, as well as recognising facial micro-expressions. Other disciplines of study such as psychological evaluation may benefit from its precise output. The findings show a high degree of accuracy, paving the way for the creation of programmes that can react in real time to changes in facial movements [19]. Sentimental categories vs subjective categories in labelling, facial areas in DBMS, standardization of measures, and the needs of real-time application are all explored. Finally, this suggests several interesting future areas for the advancement of micro-expressions studies as our last conclusion [20]. Emotions and objectives may be conveyed via the use of face expressions. Several personal behaviours analysing tasks, including as interviewing, self-directed driving, and therapeutic dealings, are relying on automated facial expression recognition (FER) in recent years. A technique for facial expression detection using features extracted using CNNs is presented in this study, taking use of a model that has already been trained on comparable tasks. While other methods focus on a single database, the FER technique uses data from several sources to improve generalisation, a critical problem in ML. Six basic emotions phrases are identified with just accuracy of above 92.00% when utilising the five most commonly-used databases [21] Detecting aggressive intent and hazardous demeanours may be aided by studying facial micro-expressions. In this research, researchers offer a unique method for recognising face micro-expressions in surveillance videos. It is first necessary to take the image of the face using most efficient resolution cameras capable of 200 images or frames per second. Second, a three-dimensional distribution of orientations descriptor is used to segment the face into discrete sections, and the movements of every region is then detected. We used a high-speed camera to manually tag a fresh dataset of face micro-expressions as a ground truth for evaluating our strategy. The findings of Thirteen distinct micro-expressions are presented in this study. Using a new low-stakes videos datasets, investigators assess the effectiveness of numerous extraction features models and ML algorithms (i.e., SVM vs. DNN) to differentiate from truth-tellers. Interviewers used a tactic to make it harder for liars to hide their falsehoods by increasing the cognitive burden they were under [22]. Malingering encompasses a wide variety of behaviours, ranging from distorted self-perception to outright fabrication of material facts. Self-awareness or purposeful control is one of the continuum's components. Data from a 40-year study on deception is used to investigate Darwin's writings on the importance of intentional and involuntary activation of face muscles [23].
The SETT, but not the METT, showed a strong correlation between emotional lie detection accuracy and reported usage of facial expressions and performance. As the findings show, it's critical to consider the sort of deception being perpetrated when evaluating deception detection abilities.
Non-verbal leaking and statement analytics clues were coupled to try to distinguish truth-tellers from liars, but the results remained similar with earlier studies. 90% of the individuals in the films were accurately classified as liars or truthtellers based on facial expressions and statement analysis annotations, according to the researchers. This suggests that behavioural clues in both verbal statements and nonverbal actions together give a far superior source for judging sincerity. Investigators may be able to use this to their advantage while conducting investigations and questioning suspects [24].
Lying detection is essential in numerous fields, including airport security and police inquiry. Face ME, which are fleeting, spontaneous emotions of the face of individuals while they are attempting to suppress or hide their expressional feelings, may be used to identify lying. It is difficult, time-consuming, and incorrect to quantify MEs by hand. Facial Micro-Expressions are used to create a Lie Detection System, which is described in this study [25].
Face expressions throughout civilizations are examined in this article. Three methodologies are used to analyse emotional facial reactions: sociological, physiological and computerized Presentations from world-renowned experts in behavioural, physiological, and mathematical disciplines are included [26].
While neutralization was taking place, high impact emotions were more likely to show through in the upper face because they were harder to keep under wraps. The most and least amount of emotional leakage was seen with emotions of fear and delight, correspondingly. Real and fraudulent emotions could not be distinguished beyond the level of chance by untrained observers [27].
Micro-expressions were evoked in the experiment and a database was developed with the help of psychologists, which is explained here. Also included are concerns that may aid researchers in their usage of micro-expression databases and in their ability to recognize micro-expressions. Falsehood may be detected by the use of micro-manifestations, which are quick, automatic expressions of real emotion. Their numerous prospective uses have drawn the attention of researchers from a wide range of disciplines. The employment of two perception color images may enhance the identification of facial expressions (CIELab and CIELuv). It is our hope that the tensor independent colour space (TICS), which was first proposed in our article for the International Conference on Pattern Classification, may aid in the identification of ME in future work.
When it comes to spotting lies and harmful demeanours, micro-expressions are a crucial behavioural indication. On the other side, MEs are difficult for people to pick up on. We provide a novel method for automatically identifying little facial expressions in this work. A frame-by-frame operation ensures that the system is completely automated. This tool uses Gabor filters to automatically identify a face and extract its characteristics.
In this research, researchers present a CapsuleNet for ME identification that is both simple and effective shown in Figure 3. With the leave-one object-out cross-validation, this examined the presence of these suggested approaches on the cross-database micro-expression benchmark. According to the findings of the studies, our technique outperformed both the baseline method (LBP-TOP) and other leading edge CNN algorithms. MEs are transient, spontaneous, and low-intensity movements of the face. Facial micro-expressions are particularly difficult to pick up on in the moment. For the identification of micro-expressions, we provide a basic yet effective MDMO feature in this study. On micro-expression video clips, we use a reliable optical flow approach to divide the face area into ROIs based in large part on action units [28].
Micro-expressions, which vary from typical facial emotions in subtlety and length, are quick, involuntary FME that arise on a person's face while they hide an emotion. Data from the CASME II currently uses action units and self-reports as the basis for emotion classifications, which causes problems for machine learning training.
Figure 3. Earlier facial expression model
Deception was not a primary concern for Darwin. The topic was just briefly touched a few times in his work. An intriguing point was raised on whether it is complicated to control one's expressional displays that are most hard to control. one more hypothesis was that the lack of the difficult-to-voluntarily-generate facial motions may uncover a faked expression. Another was that bodily motions were more readily concealed than facial expressions when people were in the midst of an emotional state. Other studies on deceit, which Darwin did not anticipate, is also included in the study [29]. micro facial expressions in picture sequences have made use of a broad range of face models [30]. Existing approaches, on the other hand, only deal with one kind of expression at a time, since the amplitude and/or texture changes in micro-expressions vary greatly from those in full expressions. Without appropriate processing, the visual stream collected from the face is noisy and cannot meet expression identification standards, particularly for MEs [31]. The above all methods are most useful for facial expressions estimation, in these many limitations are there like maskable, spectacle, and blurred images [32].
2.1 Micro facial processing
Micro expressions, on the other hand, are facial features that appear and disappear in a microsecond, which can be as quickly as 1/30 of a second. They move so quickly that you would miss them if you blink. Micro facial expressions are often indicators of hidden intentions. The Micro facial emotions are coded with 700 high-stakes real & invented expressions with emotions & exposed just 2% micro expressions in one of the few studies on the topic.
$P=[T 0 \rightarrow 1(C s+C l)+T c C c] V 2 d d F c k$ (1)
Pixel density estimation is performed through T0-and Cs
$T C=K 1 T 1+K 2 T 2+K 3 T 3+K 4 T 4$ (2)
$T 2+T 3+T 4+2 T 1 * * *<T 1+2 T 4 * * *$ (3)
Pre-processing is the start of overseeing facial image reading. This level is to enhance the character of the facial image in light of the fact that the image regardless of the pixels analysis [27].
Figure 4. Micro features pre-processing
This features extraction starts off evolved via isolating, the sifting manner utilizing the middle channel, which serves to take out samples [28]. The respective pre-processing assists with halting the images but at the identical time holds the picture structure, and the results can be prepared in a pre-processing manner [29]. A complementary filtering approach is a versatile approach for improving the images of the nonlinear properties [30]. This strategy is a combination of filtering techniques, in particular, places pixels and samples are verified. The combination of these two techniques first-rate enhancements which are progressively compelling against facial image-based pixels. The complementary bands approach is makes use of forms with weighting on pixels. The facial image to supply gradually exact counts contrasted with the past strategies. All discussion is explaining about various pre-processing techniques on facial images. The division manner is beneficial for keeping apart a photograph into some locales as according to the precise guidelines shown in Figure 4. This division degree begins with a side process; there is additionally a sort of area applied, to be unique, limit paired reversed. In the restrict parallel shifted, the object that has esteem now not precisely the brink, the cost will be one or white. Though those who contain a sum extra than the edge, the pixel esteem is well worth zero or dark [5]. The tasks of limit parallel rearranged may have seemed inside the accompanying circumstance:
Segment $=\{0$ if $T h>p(x, y) ;$ else max value
Clarification: P (x, y) is a darkish picture that will be handled, sift is the brink esteem, max Val is the greatest really worth, this restriction manner works in finding out instructions at consist of extraction.
In this work, adaptive Tiefes FCNN based micro facial expressions is detected using the Kaggle dataset. At the primary stage, images are trained with an adaptive median filter, after filtering samples are applied to Tiefes FCNN. The training can be processed through two benches like feature extraction and exercise. The complete testing process is enhanced through the proposed Tiefes FCNN.
Figure 5. The flow of work
The above Figure 5 is clearly explaining about micro facial expressions workflow in this at 1st stage all input samples are to be collected using the Kaggle dataset. The following samples are training with pre-processing as well as Tiefes FCNN deep learning. The testing purpose real-time image frames and videos are giving, finally, it is identified that proposed design is easily extracting the features with micro expressions.
Figure 6. Proposed work block diagram
The above Figure 6 explains about proposed work block diagram with Tiefes FCNN deep learning. In this feature extraction, training, testing and micro expressions are estimated with accurate deep learning techniques.
Algorithm: Tiefes FCNN deep learning
Step 1. Collect Kaggle micro facial samples
Step 2. Apply pre-processing
Segment $=\{0$ if Th $>p(x, y) ;$ else max value
$T 2+T 3+T 4+2 T 1 * * *<T 1+2 T 4 * * *$
Step 3. Apply Tiefes FCNN deep learning for training
$\epsilon_t=\sum_{i: f_i\left(X^i\right) \neq y^i} w_t^i / \sum_{i=1}^n w_t^i$
$w_{t+1}^i=w_t^i *\left(\epsilon_t /\left(1-\epsilon_t\right)\right)^{1-\left|f_t\left(x^1\right)-y^1\right|}$
Step 4. Micro features extraction
$\hat{y}=\operatorname{sgn}\left(\sum_{t=1}^T c_t f_t(x)\right)$
Step 5. Testing on real time samples
$c_t=\log \left(\left(1-\epsilon_t\right) / \epsilon_t\right)$
$f=\frac{1}{n} \sum_{i=1}^n \exp \left(-\sum_t y^i c_t f_t\left(x^i\right)\right)$
Step 6. Estimating performance measures
$g(y \mid x, M)=k\left[\prod_t c_t^{\frac{1}{2}}\right] \exp \left[-\sum_t c_t\left(y-f_t(x)\right)^2\right]$
Accuracy, sensitivity, recall, throughput
Step 7. Stop the process
Tiefes FCNN deep learning Index (TFCNNI) produced through the Reflectance images by the given equation below:
NDVI=(ρnir-ρred)/(ρnir+ρred)
where, ρ is reflectance.
Afterwards, categorization based on pixels Since the TFCNNI index is a direct measure of tree canopies, it might be used to obscure cityscapes. Subsequently, the masked pictures were subjected to supervised classification using the Maximum Likelihood Classifier (MXL) method, with the training samples having been created using ground truth data and reference from Google Earth. All the scenes were analyzed independently, and then the TFCNNI areas shared between them were pooled to identify shared micro-pixels for extraction in the Extraction of Common Facial Image Classes.
The picture is broken up into sections that are then ordered in accordance with the spectral, geometric, textural, and other features of the objects in the image, as determined by the object-based categorization method. Here, we make advantage of Raster's pixel processor, which does its processing using the SFP function on individual pixels. From the values of the input pixels and the samples used for training, it assigns a probability score (from 0 to 1) to each pixel (TFCNNI). To conclude, Raster Object Creator Then, using the Segmentation Lambda Schedule (FLS) method, the pixel layer is organized into meaningful objects. Operation on rasterized objects the likelihood filter was used to better the segmentation outcomes. The operator gets rid of any raster objects with a mean zonal probability lower than the threshold you set. Then Transformation of Bitmaps into Vectors Polygon trace was used to transform the raster objects created in the previous stage into vector objects. The edges of orchard polygons may be smoothed out by using a Smooth filter, which is a Vector object operation. An object vector processor, at this point, the vector layer is processed. The vector objects were cleaned up based on area and eccentricity. Eliminator of unwanted vectors, in order to eliminate unwanted faces and other vector objects, we used Arc GIS's on-screen visual interpretation and non-vegetation mask to clean up our vector layers. The micro expression map was completed by summing the attention focal points from all the double-dated and overlapping scenes. Accuracy was evaluated using ground truth points for the final output. The other visual interpretations like SVM (Support Vector Machine), PSO (particle Swarm optimization) and DAV (Differential Annealing Vector) models are facing ToC issues long with less visual interpretations.
In this section a brief note on micro facial images expression and its analysis is explained. This work to identify the visible defects in facial based on image system classification method. The present model exactly identifies the real time image visible defects, so proposed O-XB overcome the limitations of visible defects. At final estimating performance measures such as accuracy 98.96%, sensitivity 98.57%, recall 0.962, F1 score 0.971, correction factor 0.961 and machine count 78% has been achieved. The simulated results performed on python software and compared with existing methods shown in Figure 7.
Figure 7. FCNN based micro facial expressions
The above figure clearly explains about micro facial image expressions dataset in this all variety of samples are detected with proposed FCNN model.
Figure 8. Web cam base facial expression estimation
Figure 9. Comparison of micro expression with 2 databases
The above Figure 8 is explaining about web cam based micro facial expressions estimation model in this response graph is giving full analysis of functioning.
The above Figure 9 is explaining about micro facial expressions of 2 databases like ISTMIVI and ISTM 4V1. In this at all conditions proposed model is attains more improvement compared to earlier models. The epochs time is more improved compared to earlier models like SVM and x-boosting methodologies.
Figure 10. Comparison of results
Table 3. Comparisons of results
|
S NO |
PARAMETER |
X boosting |
SVM |
FCNN |
|
1 |
Calculation prospect |
[09.926e-01] |
- |
- |
|
2 |
Accuracy |
99.01 |
89.87 |
99.02 |
|
3 |
Precision |
97.63 |
93.6 |
98.82 |
|
4 |
Recall |
98.57 |
94.4 |
99.3 |
|
5 |
F1 score |
96.43 |
92.3 |
97.8 |
|
6 |
PSNR |
51.82 |
49.53 |
56.31 |
|
7 |
CC |
92.83 |
95.12 |
96.23 |
The above Figure 10 and Table 3 clearly expanding about various facial expressions analysis in this compared to earlier models proposed model is attains more improvement.
In many applications, face expressions are crucial ground truth decision variables. The previous models, such as GA, RFO, X-Boosting, and gradient boosting, are inefficient in terms of ToC and accuracy. Furthermore, available facial applications are failed to detect micro facial expressions, so that a deep Tiefes FCNN model has been implemented. In this work, to recognise micro facial expressions using segmentation as well as feature extraction. The implantation is carried out using Python 3.7.0 software tool and the results of experiments are determined with comparison. The facial expression model consists of pre-processing and segmentation stages. The Tiefes FCNN technology is used to extract micro features using arc visualisation. The Accuracy was 99.02 percent, precision was 98.82 percent, F1 score was 97.8 percent, PSNR was 56.31, and CC was 96.31 attained. This proposed model is competing with present technology and outperformance the outcomes. To design proposed application with cloud platform then it is very useful to future generations.
[1] Yan, W.J., Wu, Q., Liang, J., Chen, Y.H., Fu, X. (2013). How fast are the leaked facial expressions: The duration of micro-expressions. Journal of Nonverbal Behavior, 37: 217-230. https://doi.org/10.1007/s10919-013-0159-8
[2] Pfister, T., Li, X., Zhao, G., Pietikäinen, M. (2011). Recognising spontaneous facial micro-expressions. In 2011 International Conference on Computer Vision. IEEE, pp. 1449-1456. https://doi.org/10.1109/ICCV.2011.6126401
[3] Bowers, D., Miller, K., Bosch, W., Gokcay, D., Pedraza, O., Springer, U., Okun, M. (2006). Faces of emotion in parkinsons disease: Micro-expressivity and bradykinesia during voluntary facial expressions. Journal of the International Neuropsychological Society, 12(6): 765-773. https://doi.org/10.1017/S135561770606111X
[4] Miao, S., Xu, H.Y., Han, Z.Q., Zhu, Y.X. (2019). Recognizing facial expressions using a shallow convolutional neural network. IEEE Access, 7: 78000-78011. https://doi.org/10.1109/ACCESS.2019.2921220
[5] Yan, W.J., Wu, Q., Liu, Y.J., Wang, S.J., Fu, X. (2013). CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1-7. https://doi.org/10.1109/FG.2013.6553799
[6] Peng, W., Hong, X., Xu, Y., Zhao, G. (2019). A boost in revealing subtle facial expressions: A consolidated eulerian framework. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition, pp. 1-5. https://doi.org/10.1109/FG.2019.8756541
[7] Owayjan, M., Kashour, A., Al Haddad, N., Fadel, M., Al Souki, G. (2012). The design and development of a lie detection system using facial micro-expressions. In 2012 2nd International Conference on Advances in Computational Tools for Engineering Applications (ACTEA), pp. 33-38. IEEE. https://doi.org/10.1109/ICTEA.2012.6462897
[8] Merghani, W., Davison, A.K., Yap, M.H. (2018). A review on facial micro-expressions analysis: datasets, features and metrics. arXiv Preprint arXiv: 1805.02397. https://doi.org/10.48550/arXiv.1805.02397
[9] González-Lozoya, S.M., de la Calleja, J., Pellegrin, L., Escalante, H.J., Medina, M.A., Benitez-Ruiz, A. (2020). Recognition of facial expressions based on CNN features. Multimedia Tools and Applications, 79: 13987-14007. https://doi.org/10.1007/s11042-020-08681-4
[10] Polikovsky, S., Kameda, Y., Ohta, Y. (2009). Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor. In 3rd International Conference on Imaging for Crime Detection and Prevention (ICDP 2009). https://doi.org/10.1049/ic.2009.0244
[11] Monaro, M., Maldera, S., Scarpazza, C., Sartori, G., Navarin, N. (2022). Detecting deception through facial expressions in a dataset of videotaped interviews: A comparison between human judges and machine learning models. Computers in Human Behavior, 127: 107063. https://doi.org/10.1016/j.chb.2021.107063
[12] Ekman, P., O'Sullivan, M. (2006). From flawed self‐assessment to blatant whoppers: The utility of voluntary and involuntary behavior in detecting deception. Behavioral Sciences & the Law, 24(5): 673-686. https://doi.org/10.1002/bsl.729
[13] Warren, G., Schertler, E., Bull, P. (2009). Detecting deception from emotional and unemotional cues. Journal of Nonverbal Behavior, 33: 59-69. https://doi.org/10.1007/s10919-008-0057-7
[14] Matsumoto, D., Hwang, H.S., Skinner, L., Frank, M. (2011). Evaluating truthfulness and detecting deception. FBI L. Enforcement Bull., 80: 1.
[15] Haritha, I.V.S.L., Harshini, M., Patil, S., Philip, J. (2022). Real Time Object Detection using YOLO Algorithm. 2022 6th International Conference on Electronics, Communication and Aerospace Technology. Presented at the 2022 6th International Conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India. https://doi.org/10.1109/iceca55336.2022.10009184
[16] Frank, M.G., Svetieva, E. (2015). Microexpressions and deception. Understanding Facial Expressions in Communication: Cross-Cultural and Multidisciplinary Perspectives, 227-242. https://doi.org/10.1007/978-81-322-1934-7_11
[17] Porter, S., Ten Brinke, L., Wallace, B. (2012). Secrets and lies: Involuntary leakage in deceptive facial expressions as a function of emotional intensity. Journal of Nonverbal Behavior, 36: 23-37. https://doi.org/10.1007/s10919-011-0120-7
[18] Yan, W.J., Wang, S.J., Liu, Y.J., Wu, Q., Fu, X. (2014). For micro-expression recognition: Database and suggestions. Neurocomputing, 136: 82-87. https://doi.org/10.1016/j.neucom.2014.01.029
[19] Wang, S.J., Yan, W.J., Li, X., Zhao, G., Zhou, C.G., Fu, X., Yang, M., Tao, J. (2015). Micro-expression recognition using color spaces. IEEE Transactions on Image Processing, 24(12): 6034-6047. https://doi.org/10.1109/TIP.2015.2496314
[20] Wu, Q., Shen, X., Fu, X. (2011). The machine knows what you are hiding: an automatic micro-expression recognition system. In Affective Computing and Intelligent Interaction: Fourth International Conference, ACII 2011, Memphis, TN, USA, October 9-12, 2011, Proceedings. Springer Berlin Heidelberg, Part II, pp. 152-162. https://doi.org/10.1007/978-3-642-24571-8_16
[21] Van Quang, N., Chun, J., Tokuyama, T. (2019). CapsuleNet for micro-expression recognition. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pp. 1-7. https://doi.org/10.1109/FG.2019.8756544
[22] Liu, Y.J., Zhang, J.K., Yan, W.J., Wang, S.J., Zhao, G., Fu, X. (2015). A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Transactions on Affective Computing, 7(4): 299-310. https://doi.org/10.1109/TAFFC.2015.2485205
[23] Davison, A.K., Merghani, W., Yap, M.H. (2018). Objective classes for micro-facial expression recognition. Journal of Imaging, 4(10): 119. https://doi.org/10.3390/jimaging4100119
[24] Ekman, P. (2003). Darwin, deception, and facial expression. Annals of the New York Academy of Sciences, 1000(1): 205-221. https://doi.org/10.1196/annals.1280.010
[25] Srinivas Rao, K., Divakara Rao, D.V., Patel, I., Saikumar, K., Vijendra Babu, D. (2023). Automatic prediction and identification of smart women safety wearable device using Dc-RFO-IoT. Journal of Information Technology Management, 15(Special Issue): 34-51.
[26] Koppula, N., Sarada, K., Patel, I., Aamani, R., Saikumar, K. (2021). Identification and recognition of speaker voice using a neural network-based algorithm: Deep learning. In Handbook of Research on Innovations and Applications of AI, IoT, and Cognitive Technologies. IGI Global, pp. 278-289. https://doi.rog/10.4018/978-1-7998-6870-5.ch019
[27] Rao, K.S., Reddy, B.V., Sarada, K., Saikumar, K. (2021). A sequential data mining technique for identification of fault zone using facts-based transmission. In Handbook of Research on Innovations and Applications of AI, IoT, and Cognitive Technologies. IGI Global, pp. 408-419. https://doi.org/10.4018/978-1-7998-6870-5.ch028
[28] Saikumar, K., Rajesh, V., Babu, B.S. (2022). Heart disease detection based on feature fusion technique with augmented classification using deep learning technology. Traitement du Signal, 39(1). https://doi.org/10.18280/ts.390104
[29] Kailasam, S., Achanta, S.D.M., Rama Koteswara Rao, P., Vatambeti, R., Kayam, S. (2022). An IoT-based agriculture maintenance using pervasive computing with machine learning technique. International Journal of Intelligent Computing and Cybernetics, 15(2): 184-197. https://doi.org/10.1108/IJICC-06-2021-0101
[30] Saikumar, K., Rajesh, V. (2022). A machine intelligence technique for predicting cardiovascular disease (CVD) using radiology dataset. International Journal of System Assurance Engineering and Management, 1-17. https://doi.org/10.1007/s13198-022-01681-7
[31] Jyothsna, K., Kumar, V.M., Shambharkar, S., Reddy, D.A., Bhoyar, C.N., Somkunwar, R.K. (2022). Face recognition automated system for visually impaired peoples using machine learning. 2022 International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI). Presented at the 2022 International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI), Chennai, India. https://doi.org/10.1109/icdsaai55433.2022.10028976
[32] Sowmya, N., Babu, I.A. (2022). Machine learning system for recognition and classification of overlapped fingerprints. 2022 Smart Technologies, Communication and Robotics (STCR). Presented at the 2022 Smart Technologies, Communication and Robotics (STCR), Sathyamangalam, India. https://doi.org/10.1109/stcr55312.2022.10009199
