İdil Işıklı Esener*
| Onur Kılınç | Burak Urazel | Betül N. Yaman | Demet İ. Algın | Semih Ergin
© 2023 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Advancements in EEG biometric technologies have been hindered by two persistent challenges: the management of large data sizes and the unreliability of data resulting from various measurement environments. Addressing these challenges, this study introduces a novel methodology termed 'Cube-Code' for cognitive biometric authentication. As a preliminary step, Automatic Artifact Removal (AAR) leveraging wavelet Independent Component Analysis (wICA) is applied to EEG signals. This step transforms the signals into independent sub-components, effectively eliminating the effects of muscle movements and eye blinking. Subsequently, unique 3-Dimensional (3-D) Cube-Codes are generated, each representing an individual subject in the database. Each Cube-Code is constructed by stacking the alpha, beta, and theta sub-band partitions, obtained from each channel during each task, back-to-back. This forms a third-order tensor. The stacking of these three sub-bands within a Cube-Code not only prevents a dimension increase through concatenation but also permits the direct utilization of non-stationary data, bypassing the need for fiducial component detection. Higher-Order Singular Value Decomposition (HOSVD) is then applied to perform a subspace analysis on each Cube-Code, an approach supported by previous literature concerning its effectiveness on 3-D tensors. Upon completion of the decomposition process, a flattening operation is executed to extract lower-dimensional, task-independent feature matrices for each subject. These feature matrices are then employed in five distinct deep learning architectures. The Cube-Code methodology was tested on EEG signals, composed of different tasks, from the PhysioNet EEG Motor Movement/Imagery (EEGMMI) dataset. The results demonstrate an authentication accuracy rate of approximately 98%. In conclusion, the novel Cube-Code methodology provides highly accurate results for subject recognition, delivering a new level of reliability in EEG-based biometric authentication.
biometric authentication, EEG, higher-order singular value decomposition, 3-D feature extraction
Researches on authentication systems have gained momentum due to the increased demand for both their comfortable use by disabled people and the intensified security in internet-enabled commercial applications. An authentication procedure can be comprised of three main principles: (i) what the person knows (password/personal identification number), (ii) what the person has (key, smart card, etc.), and (iii) who the person is (fingerprint, face, etc).
The systems based on what the person knows and/or owns have the disadvantage in case of personal information forgetting. On the other hand, the “who the person is” systems, known as traditional biometric systems, is based on the physiological (face, fingerprint, palm, retina, etc.) and behavioral (signature, voice, etc.) characteristics of individuals. The existence of these universal, permanent, time-invariant, collectable, and distinctive characteristics for all individuals makes biometric systems even more important in emerging technology.
Fingerprint is the first biometry preferred in traditional biometric systems in the literature [1-8]. Besides, face [9-12], retina [13], iris [14-16], hand geometry [17-20], finger geometry [21], palms [22, 23], ear shape [24-26], hand vein [27], palm vein [28], and finger vein [29] as well as some behavioral characteristics such as signature [30, 31], voice [32], eye movements [33-35], and keystroke dynamics [36, 37] are also frequently used for biometric systems. Furthermore, biometric systems performed by collaborating more than one biometrics, are also mentioned in the literature [38-41].
Although these biometrics gain fabulous popularity by satisfying more security than the what the person knows and what the person has policies, they can easily be falsified due to malicious technology usage [42]. Besides, verification of the person’s aliveness is another problem that the authentication systems face [43]. Due to these reasons and the motivation from the “Pass-Thoughts” principle [44], research on cognitive biometric authentication systems has come to the fore [45]. These systems investigate the electroencephalogram (EEG), which is a noninvasive electrical recording method for monitoring brain activity [46], signals based on “what the person thinks” [47]. The EEG biometry is a measure of brain activity collected during cognitive or emotional stimulus; and provides an advantage over the aforementioned shortcomings due to its anti-spoofing capability, privacy compliance, shoulder-surfing resistance, and inherent verification of the person’s aliveness [42, 44, 45, 48, 49]. Besides the forgery circumvention, the sensitivity of EEG signals to environmental conditions also provides the advantage of access security to cognitive biometrics by preventing the access of the high-stress-detected user who is forced to enter the system [50].
An EEG-based biometric system, like all biometric systems, consists of an enrollment and a recognition phase while the recognition phase can be operated in two modes based upon the application: Verification or identification. In an enrollment phase, any individual is registered by constructing a relation between itself and its biometric characteristics. In the verification mode of the recognition phase, one-to-one comparisons are accomplished to validate the identity that an enrolled user claims to be, in order to prevent multiple user match with the same identity. On the other side, the identification mode tries to recognize an enrolled user via one-to-many comparisons.
In the literature, Thorpe et al. recorded and processed brain signals using a brain-computer interface, so-called “Pass-Thoughts” using the P300 method. They state that a thought which belongs to an image, an imagined movement, an emotion or a reminiscence can be used as a biometric characteristic, if the recording and processing of collected brain signals is precise and repeatable [44]. In the implementation of the “Pass-Thoughts” protocol proposed in [44, 45], the brain signals of 15 people are recorded under various tasks with the Neurosky MindSet headset, which includes an EEG sensor measurement system from the left-frontal lobe [45]. These tasks are stated as breathing, thinking about finger movements, thinking about a repetitive sports movement, singing a song or thinking about reading a passage, moving eye with music, counting colors and thinking about a code. In these given tasks, alpha and beta sub-bands were selected from the recorded EEG signals and these sub-bands were compressed to obtain spatially one-dimensional signals. The percentage difference values for self-similarity and cross-similarity metrics were expected to be equal to or greater than a certain threshold value, and finally 99% authentication accuracy is achieved. Ishikawa et al. followed the same approach, and they obtained a maximum accuracy of 98.2% for the EEG signals collected from the 12 electrodes of 16-electrode BioSemi medical EEG headset [51].
Marcel and del Millan [52] took EEG measurements from a 32-electrode BioSemi medical EEG headset under three tasks: thinking of repetitive movement of left hand, thinking of repetitive movement of right hand, and generating words with the same random letter. They extracted power spectral density features from noise-removed signals by Laplace filtering, and the identification was performed with the Gaussian modeling with the largest posterior adaptation. Sohankar et al. [53] performed a feature extraction step from EEG signals measured on a single channel from 15 people without preprocessing. As a result of Naive Bayes classification of alpha sub-bands, an authentication accuracy of 80% was provided. On the other hand, Riera et al. [54] performed two-channel EEG measurement in the resting state of 40 people and extracted two different feature groups. The first group of these features is the autoregression and Fourier transform coefficients while the other group was the measure of mutual information, consistency, and correlation. It was stated that an accuracy of 79.2% was obtained from the features classified by Fisher Linear Discrimination Analysis (FLDA).
Ruiz-Blondet et al. [48] discussed that the sensitivity of biometric verification can be improved as a result of combining the measurements taken from many channels and checking the cognitive status of individuals instead of using a single-channel EEG signal. They made measurements using 26 electrodes from 56 people while showing pictures of black-white sinus grids, low-frequency words, food, and famous faces. Wu et al. [50] recorded 16-channel EEG measurements from 40 person by showing one of their own faces and nine of familiar or unfamiliar face images. Besides, Zeng et al. [55] used person's own face and other different faces for EEG stimulation. In a recent study for EEG biometrics, Goudiaby et al. [56] performed multi-channel verification using the Emotiv Epoc system, and Zeynali and Seyedarabi [57] classified single channel signal recordings using ANN, Support Vector Machine (SVM) and Bayes classifier. Orrú et al. [58] showed that an effective EEG-based verification can be made by using a smaller number of sensors, and they confirmed that the gamma sub-band contains the best distinguishing data in person verification.
Despite its advantages, EEG biometrics also has its intrinsic problems: (1) large data size, and (2) sensitivity of EEG signals to environmental conditions [59, 60]. Projection of high-dimensional data into a lower-dimensional space is proposed in the literature to overcome the large data size problem [59]. Kumar et al. [61] proposed a new approach for modelling multi-channel EEG data as biometric signatures regardless of task/condition by using the fundamentals of subspace-based text independent speaker verification. They applied the high dimensional statistics EEG signals, then projected them into a lower dimensional subspace. The authentication accuracies of 86.4% and 35.9% are obtained by their proposed approach on datasets with 30 and 920 subjects, respectively.
Although the affectability of the EEG signals due to the changing conditions is able to provide access security, it may also disable to construct a robust model since the EEG signals measured via different sensors and/or under different tasks drastically differ [59]. He proposed an independent component-based approach for the feature extraction process on EEG signals measured from several sensors [59] while Rahman et al. [62] proposed a novel multimodal biometric system that is a hybrid of EEG and keystroke dynamics. The main idea is to overcome the deficiency of being sensitive to different psychological and physiological conditions of EEG biometrics with the higher accuracy of a key-stroke dynamic based system.
The motivation of this paper is to hit the two major problems of EEG biometrics with one arrow by the proposal of a Cube-Code methodology. The proposed methodology was verified on the EEG signals composed of different tasks on PhysioNet EEG Motor Movement/Imagery (EEGMMI) dataset [63], in this paper. In the proposed methodology, first of all, an effective Automatic Artifact Removal (AAR) algorithm was applied on EEG signals. An EEG signal comprises of five frequency components which are Delta ($\delta$) [0.5 - 4Hz], Teta ($\theta$) [4 – 8 Hz], Alfa ($\alpha$) [8 – 14 Hz], Beta ($\beta$) [14 – 30 Hz], and Gama ($\gamma$) [ > 30 Hz] [47]. Being the major ones, the theta, alfa, and beta sub-bands are thought to be critically important for authentication. Analyzing these three sub-bands separately may cause information loss due to the neglection of their mutual complementary relationships and require detecting fiducial components such as P300 or N400 on the signals. Besides, analyzing the sub-bands separately results in further increment on data size. On the contrary, a novel 3-Dimensional (3-D) Cube-Codes were created by jointly combining the theta, alfa, and beta sub-bands of EEG signals of a person to separately represent each person (subject) in database, in this paper. 3-dimensionalization of these critically important sub-bands instead of concatenating them, prevents the further increment of the data dimension. In addition, using the 3-D Cube-Code representation rather than the 1-dimensional signal measured on any task from any sensor enables the direct usage of the non-stationary data. Thus, the necessity of fiducial component detection is eliminated; therefore, both time consumption for the detection and performance loss due to possible false detection are prevented. The generated Cube-Codes are essentially 3-D tensors, and these tensors are constructed on each signal measured from each sensor during each task, as shown in Figure 4. Direct use of these tensors in the feature extraction process still causes dimensionality problems, and also the system becomes task-dependent this way. Motivated by the proven performance of Higher-Order Singular Value Decomposition (HOSVD) on 3-D tensors [64], the HOSVD on these Cube-Codes was attained. After this decomposition procedure, a matricizing (also referred to as flattening) operation was accomplished in order to extract the lower-dimensional, and task-independent feature matrices of a person. Finally, five different well-celebrated deep learning architectures are separately applied on these feature matrices, and approximately 98% biometric authentication accuracy rates were attained. The proposed novel approach, named as Cube-Code, provides fairly satisfactory results for subject recognition.
This paper is organized as follows: In the next section of this paper, HOSVD of 3-D tensors and the deep learning architectures utilized to perform the authentication process are explained in detail. In the third section, the EEG data acquisition including the database information, pre-processing, Cube-Code generation procedures, and the revealed results are elaborately given. Ultimately, all of the conclusions and future work are presented in the last section.
2.1 Database
The EEGMMI dataset [63], a popular benchmark dataset publicly available in PhysioNet [65], is used for the verification of the proposed method. This database consists of one- and two-minute EEG recordings of 109 subjects collected during 14 experimental runs (tasks) from 64 electrodes distributed over the scalp according to the international 10-10 configuration and sampled at 160 Hz. The first two experimental runs are one-minute baseline runs with eyes open and eyes closed, respectively. The rest 12 experiments are performed during two-minute runs under different motor movement/imagery tasks of: (1) opening and closing of left or right fist, (2) imagination of opening and closing of left or right fist, (3) opening and closing of both fists or both feet, (4) imagination of opening and closing of both fists or both feet.
2.2 Higher-order singular value decomposition (HOSVD)
The traditional Singular Value Decomposition (SVD) of a matrix is a very beneficial stage in the classification problems [64]. Since any real-valued matrix F is given as
$\boldsymbol{F} \in \mathbb{R}^{M \times N}$ and $M \geq N$ (1)
the matrix F can be decomposed into three different matrices by SVD, such that:
$\boldsymbol{F}=\boldsymbol{U} \Sigma \boldsymbol{V}^T$ (2)
where, the matrix U spans the row space of F, the matrix V spans the column space of F and Ʃ is a diagonal matrix of singular values. Alternatively, F is expressed in terms of the n-mode products:
$\boldsymbol{F}=\Sigma \times_1(\boldsymbol{U}) \times_2(\boldsymbol{V})$ (3)
where, ×1 is 1-mode product and ×2 is 2-mode product.
HOSVD can be considered as one type of generalization of the matrix singular value decomposition (SVD) which is pretty successful and preferred decomposition method especially in machine learning, signal processing and computer vision [66]. In other words, it is multilinear algebraic version of SVD, and it is used for “n” dimensional data. The “n” dimensional data is also termed as “nth – order” tensor that has “n” indices. Since the three prominent and important EEG sub-bands (theta (θ), alpha (α), and beta (β)) constitute a 3-D shape thoroughly explained in the experimental study section, a multilinear counterpart of the classical SVD methodology, HOSVD, is favored for the feature extraction process from 3-D sub-band structures. Therefore, a third order tensor is formed for each class, and “n” equals to 3 in this paper. The singular values have a very close relationship with eigenvalues. Since eigenvalues explicitly indicate the inherent scattering of data, the singular values, which are nonnegative, also give the characteristic and dimension-based properties of a tensor.
A third order tensor $\mathfrak{T}$ can be written as:
$\mathfrak{T}=(\boldsymbol{\delta}) \times_1(\boldsymbol{U}) \times_2(\boldsymbol{V}) \times_3(\boldsymbol{W})$ (4)
where, $\delta$ is called as core tensor and $U, V$, and $W$ are the orthogonal matrices. Also, $\times_1$ is 1 -mode product, $\times_2$ is 2 -mode product and $\times_3$ is 3 -mode product. The restructuring of the elements for a tensor is referred to as flattening of a tensor. The "mode- $n$ flattening" of a tensor $\mathfrak{T}$ is an operation in which the tensor $\mathfrak{T}$ is converted into matrices based upon its dimension " $n$ ", denoted as $\mathfrak{T}_{(n)}$. The HOSVD of a tensor $\mathfrak{T}$ can be performed by separately computing the orthogonal matrices $\boldsymbol{U}, \boldsymbol{V}$, and $\boldsymbol{W}$ which can be calculated by applying three distinct SVDs onto each mode- $n$ flattening of tensor $\mathfrak{T}$, separately:
$\begin{aligned} & \mathfrak{T}_{(1)}=\boldsymbol{U} \Sigma_{\mathbf{1}}\left(\boldsymbol{Y}_1\right)^{\mathrm{T}} \\ & \mathfrak{T}_{(2)}=\boldsymbol{U} \Sigma_{\mathbf{2}}\left(\boldsymbol{Y}_2\right)^{\mathrm{T}} \\ & \mathfrak{T}_{(3)}=\boldsymbol{U} \Sigma_{\mathbf{3}}\left(\boldsymbol{Y}_3\right)^{\mathrm{T}}\end{aligned}$ (5)
where, $\boldsymbol{Y}_i$ 's are the right singular matrices of $\mathfrak{T}_{(i)}$ 's. Then, the core tensor $\delta$ is evaluated as:
$\boldsymbol{\delta}=(\mathfrak{T}) \times_1\left(\boldsymbol{U}^{\mathrm{T}}\right) \times_2\left(\boldsymbol{V}^{\mathrm{T}}\right) \times_3\left(\boldsymbol{W}^{\mathrm{T}}\right)$ (6)
Finally, the basis matrices (Xi) of tensor $\mathfrak{T}$ are found as:
$\boldsymbol{X}_{\boldsymbol{i}}=[\boldsymbol{\delta}(:,:, \boldsymbol{i})] \times_1(\boldsymbol{U}) \times_2(\boldsymbol{V}) i=1,2, \ldots, P$ (7)
where, P is the value for third dimension of tensor $\mathfrak{T}$ in each class. Xi's are orthogonal to each other. The singular values corresponding to these basis matrices are non-negative and they are calculated as:
$\boldsymbol{\sigma}_{\boldsymbol{i}}=\|\boldsymbol{\delta}(:,:, \boldsymbol{i})\|_F \quad \sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_P \geq 0$ (8)
2.3 Deep learning network architectures
There are various deep learning network architectures that were designed since the last decade. In this paper, five prominent architectures were implemented and brief information about these architectures is given in the following subsections.
2.3.1 MobileNetV2 architecture
Recently, a team of researchers from Google released an architecture MobileNetV2, which is optimized especially for mobile devices [67]. MobileNets are small, low-latent, low-power architectures parameterized in order to satisfy the source restrictions of many devices. MobileNetV2 is built on a reversed residual architecture where the connections are made between the bottleneck layers. The middle extension layer uses lightweight convolutions to filter out the features resulted from the non-linearity issue. The architecture of MobileNetV2 comprises of an initial fully convolution layer with 32 filters followed by 19 bottleneck layers. MobileNetV2 architecture improves the state-of-the-art performance of mobile architectures with a spectrum of different model sizes. The depth-wise (cross-channel) architecture of MobileNetV2 with separable convolutions focuses on the reduction of the model size and complexity without compromising the accuracy when computational power is very limited and input data is very complex. This focus attracts the use of this network for high-dimensional data such as multi-channel EEG signals.
2.3.2 InceptionV3 architecture
Inception Networks are designed with parallel layers instead of deep layers in order to overcome the memory and computational cost restrictions of VGGNets [68]. The InceptionV3 architecture is an optimized version of its predecessors. Factorization of large convolutions in InceptionV1 architecture into smaller convolutions and applying spatial factorization into asymmetric convolutions are major optimizations building the InceptionV2 architecture. The InceptionV3 architecture differs from the InceptionV2 architecture in using auxiliary classifiers, which act as regularizers satisfying higher accuracies. Another major modification done on the InceptionV3 architecture is reducing the grid size by expanding the activation dimension of the network filters. Utilization of multiple kernel sizes in the same layer enables both local and global feature extraction which is very beneficial for complex EEG data.
2.3.3 Xception architecture
Xception architecture [69] is the upgraded version of Inception network architectures. It uses a modified form of classic depth-wise separable convolution. The first difference between the architectures of Xception and Inception networks is that the original depth-wise separable convolutions initially accomplish channel-wise spatial convolution and then implement 1×1 convolution whereas the modified depth-wise separable convolution initially achieves 1×1 convolution and then channel-wise spatial convolution. Therefore, the order of operations is changed. The second difference is that there is non-linearity after the first convolution operation. In the Xception architecture, there is no intermediate non-linearity. The key advantage of Xception is its consideration of cross-channel and spatial correlations within each channel separately, allowing it to model more complex features in the data. This is particularly beneficial for EEG data, which has complex spatial and temporal correlations in different channels with distinct information.
2.3.4 ResNet50 architecture
Residual Network (ResNet) architectures are introduced by Microsoft intending to solve the vanishing gradients and degradation problem exposed by adding shortcut connections in the residual blocks of the network [70]. These connections create shortcut paths for the gradient to flow through by skipping one or more layers that hurt the performance of the architecture. Therefore, the ResNet architecture is satisfactorily trained by deeper networks with a minimized error percentage. This is beneficial for EEG-based authentication since an EEG signal often contains complex and subtle patterns that generally deeper networks may capture better. The ResNet50 architecture is a variant of ResNets which has 48 Convolution, one Maximum Pool and one Average Pool layers.
2.3.5 EfficientNet–B0 architecture
EfficientNet architecture scales up the depth, width, and resolution of the networks uniformly using a fixed set of coefficients of d, ω, and r, respectively, which is defined by a compound coefficient $\varnothing$ [71].
$\begin{array}{ccc}d=\alpha^{\varnothing} & \omega=\beta^{\varnothing} & r=\gamma^{\varnothing} \\ & \text { s.t. } & \alpha \cdot \beta^2 \cdot \gamma^2 \approx 2 \\ & & \alpha \geq 1, \beta \geq 1, \gamma \geq 1\end{array}$ (9)
The constants $\alpha, \beta$, and $\gamma$ in Eq. (9) indicate how to assign the computational resources to the network depth, width, and resolution, respectively, while the number of these resources are controlled by the compound coefficient $\varnothing$. Different models of EfficientNets are obtained by using different $\varnothing$ values. The EfficientNet-B0 Architecture uses the values of $\alpha=1.2, \beta=1.1$, and $\gamma=1.15$ with $\varnothing=1$. The efficiency of uniform scaling of all dimensions, for multi-channel EEG signal, and less computational cost acquiring similar accuracy to other complex models make it worth comparing the functionality of EfficientNet-B0 in EEG-based authentication systems.
Table 1. Dataset description
|
Task Code |
Task Description |
Dimension of Data |
Duration |
|
T01 |
Baseline run while eyes are open |
9760×64 for all subjects |
61 seconds for all subjects |
|
T02 |
Baseline run while eyes are closed |
9760×64 for all subjects |
61 seconds for all subjects |
|
T03 |
Open and close left or right fist |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T04 |
Imagine opening and closing left or right fist |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T05 |
Open and close both fists or both feet |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T06 |
Imagine opening and closing both fists or both feet |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T07 |
Open and close left or right fist |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T08 |
Imagine opening and closing left or right fist |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T09 |
Open and close both fists or both feet |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T10 |
Imagine opening and closing both fists or both feet |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T11 |
Open and close left or right fist |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T12 |
Imagine opening and closing left or right fist |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T13 |
Open and close both fists or both feet |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
|
T14 |
Imagine opening and closing both fists or both feet |
20000×64 for S001, S003, and S007 19680×64 for other subjects |
125 seconds for S001, S003, and S007 123 seconds for other subjects |
All the extracted feature matrices are reshaped into corresponding matrix sizes for each deep learning network architecture so that they are properly treated in the layers of architectures. The number of epochs is preferred as ten by carefully tracking the convergence of the loss function for each architecture.
The proposed biometric identification system in this paper consists of two major phases: enrollment and recognition. The initial step of this system is the data acquisition. The EEGMMI dataset [63] is used for the verification of the proposed system. The enrollment phase of the system is responsible for the registration of individuals performing feature extraction after several pre-processing steps applied to the EEG signals. On the recognition side, identification is realized using these features in the classification. The performance of the proposed system is evaluated based on the recognition phase findings.
3.1 Data acquisition
A subset belonging to the first 20 subjects of the EEGMMI dataset is used in this paper. This subset includes EEG recordings collected from each of 64 channels during 14 experimental runs. Table 1 gives a brief description of the tasks in experiments and indicates the duration and dimension of data per task for each of the 20 subjects [63]. The data per channel is constructed by column-wise concatenation of EEG signal samples for each channel. Subjects 1-20 and experiments 1-14 are coded as S001-to-S020 and T01-to-T14, respectively in the rest of the paper. As a sample of the dataset, the 10-second-EEG recordings of S001, collected from FP1 sensor during each experiment and from the FP1, TP7, and Po8 during T01, are plotted in Figures 1 and 2, respectively.
3.2 Pre-processing
The following operations are applied on the recorded EEG signals in the pre-processing stage:
(1) Signal Standardization: The per-subject EEG recordings collected during 14 experimental runs from different channels are fused for the proposed authentication system. The analysis of Figures 1 and 2 reveals that these recordings exhibit heterogeneity generated either from they are measured from the same channel during different experiments or from different channels during the same experiment. Hence, these recordings should be transformed into a common domain before fusion. Therefore, the z-score normalization [72] is applied on the collected signals to standardize the data in this study. Mathematical representation of z-score normalization of any data X is given as Xstandard in Eq. (10) where μX and σX refer to the mean and standard deviation of X, respectively. The rest of the pre-processing steps are thereafter realized on the computed data as Xnew.
$\boldsymbol{X}_{\text {standard }}=\frac{\boldsymbol{X}-\boldsymbol{\mu}_{\boldsymbol{X}}}{\sigma_{\boldsymbol{X}}}$ (10)
$\boldsymbol{X}_{\text {new }}=\frac{\boldsymbol{X}_{\text {standard }}}{\left\|\boldsymbol{X}_{\text {standard }}\right\|}$ (11)
(2) Sub-band Decomposition: The simultaneous usage of three EEG sub-bands instead of utilizing raw EEG recordings from skull measurement locations not only gives us a very thorough analysis of an EEG recording but also combines the information lying under different EEG frequency ranges obtained in distinct brain conditions of a person. Thus, the need for the exhaustive determination and calculation of EEG fiducial components in an EEG processing stage can be removed. Thus, the standardized data is decomposed into theta (θ), alpha (α), and beta (β) sub-bands using Chebyshev Type II band-pass filter. The order of the filter is specified as n=4, and it is employed by the MATLAB function ‘cheb2ord’ with the input parameters given in Table 2.
(3) Data Partitioning: The EEG recordings utilized in this paper have a huge number of data points, as given in Table 1, for a feasible artifact removal and feature extraction scheme. So, a data partitioning operation is applied on each sub-band via a sliding window technique with a fixed-window size of 10 seconds. This operation results in 1600×1–sized partitions for each sub-band decomposed from each channel for each subject. The number of partitions for experiments T01 and T02 is six for each subject whereas it is twelve for the rest of the experiments.
(4) Artifact Removal: The related studies in the literature point out that the most challenging issue for EEG interpretation is the noise caused due to non-physiological effects such as electrode impedance changes, and physiological activities such as eye blinking, muscle movements, or cardiac effects [73-80]. Consequently, an automatic artifact removal process is implemented on each partition segmented in the previous stage via Wavelet Decomposed Independent Component Analysis (wICA) to get rid of the ocular and muscular artifacts using the EEGLAB platform [81, 82]. The independent components of the partitions are computed by the FastICA algorithm [82, 83], and the artifact-removed partitions are concatenated. Finally, the artifact-removed theta, alpha, and beta sub-bands are segmented into 2-second, 320×1–sized parts with an overlapping rate of 25 percent. Ultimately, the pre-processing stage is concluded with 39 partitions for T01 and T02 tasks, whereas 79 partitions for the tasks T03-T14 for each sub-band signal obtained from each channel. A simple illustrative flowchart of the pre-processing stage on a sample EEG recording is given in Figure 3.
Figure 1. Sample EEG recordings of S001 measured from the channel Fp1 during 14 tasks
Figure 2. Sample EEG recordings of S001 measured from three different channels during T01
Table 2. Parameters for sub-band decomposition
|
$n=\operatorname{cheb2ord}(W p, W s, R p, R s)$ (MathWorks) |
|||||
|
Parameter |
Definition |
|
$\theta$ -band |
$\alpha$ -band |
$\beta$ -band |
|
$W p$ |
passband edge frequency |
$\longrightarrow$ |
$\frac{\left[\begin{array}{ll}4 & 8\end{array}\right]}{F_s / 2}$ |
$\frac{\left[\begin{array}{ll}8 & 14\end{array}\right]}{F s / 2}$ |
$\frac{\left[\begin{array}{cc}14 & 30\end{array}\right]}{F s / 2}$ |
|
$W s$ |
stopband edge frequency |
$\longrightarrow$ |
$\frac{\left[\begin{array}{ll}3.5 & 8.5\end{array}\right]}{F s / 2}$ |
$\frac{\left[\begin{array}{ll}7.5 & 14.5\end{array}\right]}{F s / 2}$ |
$\frac{\left[\begin{array}{cc}13.5 & 30.5\end{array}\right]}{F_S / 2}$ |
|
$R p$ |
passband ripple |
$\longrightarrow$ |
30 dB |
||
|
$R s$ |
stopband attenuation |
$\longrightarrow$ |
10 dB |
||
| $F s$ | sampling frequency |
$\longrightarrow$ |
160 |
||
Figure 3. Flowchart of the pre-processing stage on a sample EEG recording
Figure 4. The proposed feature matrix construction scheme
3.3 HOSVD of EEG data
The 320×1–sized pre-processed sub-band partitions measured from each channel during T03-T14 tasks are then individually stacked back-to-back resulting in a third-order tensor of size 320×3×12, for each subject. This tensor is called as the Cube-Code of a particular subject in this paper. A feature extraction process is implemented by applying HOSVD onto the Cube-Codes; and 12 basis matrices with a size of 320×3 are computed. These basis matrices of each Cube-Code are then concatenated side-by-side to form the 320×36–sized feature matrices. Thus, 5056 task-independent feature matrices are constructed for each subject. This process is elaborately explained in Figure 4.
3.4 Classification and performance evaluation
The identification problem in this paper is defined as a 20-class classification problem and is solved by using the MobileNetV2, ResNet50, Xception, InceptionV3, and EfficientNet-B0 architectures. The 5-fold cross-validation technique is used for the classification which means that these networks are trained by 80% of the randomly selected feature matrices in each class and tested by the rest 20%. The proposed authentication method is trained using the Deep Learning Toolbox of MATLAB 2020a in an Intel(R) Core(TM) i7-10750H CPU 2.60 GHz with 16 GB of RAM, and Nvidia GeForce RTX 2070 GPU with 8 GB VRAM. The stochastic gradient descent method with the learning rate of 0.0001 is used to train our networks and the cross-entropy is used for the loss function. The batch size is chosen as ten and the number of epochs is selected as ten by rigorously taking the convergence of loss function into consideration for each architecture. The performance of the proposed authentication method is evaluated in terms of accuracy (ACC), computed as given in Eq. (12). The terms TP, TN, FP, and FN are the numbers of true positives, true negatives, false positives, and false negatives, respectively.
$A C C=\frac{T P}{T P+T N+F P+F N}$ (12)
Table 3. The average accuracy values achieved using five different network architectures
|
Name of Deep Network Architectures |
Five-Fold Cross Validation Accuracy (%) |
|
MobileNetV2 |
90.09 |
|
ResNet50 |
90.99 |
|
Xception |
88.76 |
|
InceptionV3 |
98.08 |
|
EfficientNet-B0 |
94.96 |
The average of the obtained ACC values computed for each fold is evaluated as the authentication system performance. The average ACC values succeeded using the above-mentioned network architectures are given in Table 3.
The comparison of the results achieved by the proposed study with the prominent and current studies in the literature is given in Table 4.
Table 4. Related works using the EEG Motor/Movement Imaginary Dataset for EEG-based biometric authentication
|
Reference |
Duration of EEG Segments |
Number of Tasks Used |
Number of Sensors Used |
Frequency Bands Used |
Architecture/Classifier |
Accuracy (%) |
|
|
[84] |
5 sec |
2 (T01-T02) |
8 |
0-10 Hz |
CNN |
74.20 |
|
|
10-20 Hz |
73.50 |
||||||
|
20-30 Hz |
72.80 |
||||||
|
30-40 Hz |
74.90 |
||||||
|
40-50 Hz |
72.90 |
||||||
|
50-60 Hz |
68.20 |
||||||
|
[85] |
1 sec |
- |
32 |
- |
CNN + Dense |
100 |
|
|
64 |
100 |
||||||
|
[86] |
10 sec |
12 (T03-T14) |
64 |
<40 Hz |
Magnitude Squared Coherence |
100 |
|
|
[87] |
12 sec |
1 (T01) |
64 |
$\delta$ -band (0.5-4 Hz) |
Eigenvector |
11.30 |
|
|
$\theta$ -band (4-8 Hz) |
7.70 |
||||||
|
$\alpha$ -band (8-13 Hz) |
15.70 |
||||||
|
Low $\beta$ -band (13-20 Hz) |
67.00 |
||||||
|
High $\beta$ -band (20-30 Hz) |
81.90 |
||||||
|
$\gamma$ -band (30-50 Hz) |
96.90 |
||||||
|
1 (T02) |
$\delta$ -band (0.5-4 Hz) |
5.90 |
|||||
|
$\theta$ -band (4-8 Hz) |
7.50 |
||||||
|
$\alpha$ -band (8-13 Hz) |
19.90 |
||||||
|
Low $\beta$ -band (13-20 Hz) |
55.80 |
||||||
|
High $\beta$ -band (20-30 Hz) |
60.00 |
||||||
|
$\gamma$ -band (30-50 Hz) |
92.60 |
||||||
|
[88] |
2 sec |
1 (T01) |
64 |
$\delta$ -band $\theta$ -band $\beta$ -band $\gamma$ -band |
SVM |
with PCA |
97.64 |
|
without PCA |
96.88 |
||||||
|
RF |
with PCA |
98.16 |
|||||
|
without PCA |
95.78 |
||||||
|
1 (T02) |
SVM |
with PCA |
96.02 |
||||
|
without PCA |
96.02 |
||||||
|
RF |
with PCA |
97.30 |
|||||
|
|
without PCA |
93.21 |
|||||
|
[89] |
1 sec |
14 (T01-T14) 1 (T02) |
4 |
- |
CNN |
94.34 |
|
|
LSTM |
90.36 |
||||||
|
1-D Convolution LSTM |
94.28 |
||||||
|
16 |
CNN |
98.07 |
|||||
|
LSTM |
95.94 |
||||||
|
1-D Convolution LSTM |
99.58 |
||||||
|
32 |
CNN |
98.50 |
|||||
|
LSTM |
96.71 |
||||||
|
1-D Convolution LSTM |
99.50 |
||||||
|
64 |
CNN |
98.87 |
|||||
|
LSTM |
96.39 |
||||||
|
1-D Convolution LSTM |
99.58 |
||||||
|
|
|
|
|
|
|
|
|
|
[90] |
1 sec |
1 (T01) |
64 |
- |
PLV+Graph CNN |
99.97 |
|
|
1 (T02) |
|
|
|
99.88 |
|||
|
6 (T03, T05, T07, T09, T11, T13) |
99.99 |
||||||
|
6 (T04, T06, T08, T10, T12, T14) |
100.00 |
||||||
|
[91] |
- |
12 (T03-T14) |
35 |
- |
FPA + β-hill |
96.05 |
|
|
[92] |
- |
T01-T02 |
48 |
δ-band |
LSTM-MLP |
99.70 |
|
|
Proposed Method |
2 sec |
12 (T03-T14) |
64 |
θ-band (4-8 Hz) |
CNN |
98.08 |
|
|
α-band (8-14 Hz) |
|||||||
|
β-band (14-30 Hz) |
|||||||
Biometric person authentication is a process that is implemented with unique feature vectors extracted from the inimitable characteristics of a person. Although universality, permanency, time-invariance, collectability, and distinctiveness of biometric systems overcome most of the weaknesses of traditional authentication systems, ongoing ease of imitation and incapability of aliveness verification motivate researchers to focus on cognitive biometric systems. These systems utilizing EEG signals also provide an advantage of analyzability of whether a person consents or is compelled to authenticate, using the change that occurs in EEG signals due to varying stress levels.
On the other side, the inconsistency of these signals due to varying physiological and psychological conditions points out the reliability problem of EEG signals under different motor/mental tasks. Besides, the large data size of EEG signals is another problem [93] for cognitive systems. Therefore, a Cube-Code methodology is proposed to overcome these problems in this paper.
The inherent characteristics of an EEG signal are composed of three major sub-bands named theta, alfa, and beta, and these sub-bands essentially lie in some particular frequency intervals. Since they seem to be distinct signals, their contributions are separately examined in general. However, the theta, alfa, and beta sub-bands are accepted as mutually complementary with each other; and therefore, they are merged together in this paper so that novel 3-D Cube-Codes are generated for separately representing each person. The main idea behind this novel methodology is grounded that these three sub-bands equally contribute the whole intrinsic characteristics of an EEG signal for a person; and thus, the different EEG measurement locations (channels) that basically control several brain activities, such as speech, problem solving, motor control, vision, etc., are not important. Therefore, it can be concluded that the primary claim of this article is that EEG signals collected from different EEG measurement locations can be jointly combined to form a person's Cube-Code. From this point of view, 3-D Cube-Code representation of EEG signals provides the utilization of the sub-bands directly without fiducial component detection. This representation also prevents the further increment in large data size of these sub-bands when they are concatenated; so that, the generated Cube-Codes are distinguishable and effectively express a person. However, since these Cube-Codes for a person are generated for each signal measured from each sensor during each task, these codes are task-dependent yet, and large data size still poses a problem for feature extraction. Utilization of the HOSVD handles these problems by performing subspace analysis of the Cube-Codes and results in obtaining lower-sized and task-independent feature matrices of a person. This is another interesting and remarkable aspect of this paper since Cube-Codes may neither directly used nor easily treated on a simple electronic device.
The proposed Cube-Code methodology was implemented on EEG signals composed of different tasks on PhysioNet EEG Motor Movement/Imagery (EEGMMI) dataset, and the attained authentication results support the above-mentioned claims since approximately 98% authentication accuracy is obtained. This noticeably high recognition result also motivates a task-independent authentication approach although many papers in the literature utilize and propose the concept of the importance on task-dependency when they are including numerous EEG applications. As a conclusion, the proposed methodology has not only advanced identification accuracy but also ensures the way for more efficient and reliable biometric person authentication methods.
The effect of the direct usage for Cube-Codes in deep tensor neural networks is left beyond the scope of this paper and proposed as a possible extension for future work. Another potential area of research could involve the use of other frequency bands, like lower-beta, gamma, or delta sub-bands. Additionally, hybrid deep learning architectures could be explored and compared to further enhance model performance for EEG-based biometric identification. Finally, a comprehensive comparison between task-independent and task-dependent approaches for EEG-based identification could offer further insights. This could allow for faster authentication with fewer tasks when utmost security is not required, which could be more beneficial for real-time applications.
This research is supported by Eskisehir Osmangazi University, Fund of Scientific Research Projects (Grant No.: 202115017).
[1] Jain, A.K., Hong, L., Pankanti, S., Bolle, R. (1997). An identity-authentication system using fingerprints. Proceedings of the IEEE, 85(9): 1365-1388. https://doi.org/10.1109/5.628674
[2] O’Gorman, L. (2000). Practical systems for personal fingerprint authentication. IEEE Computer, 33(2): 58-60.
[3] Ratha, N.K., Connel, J.H., Bolle, R.M. (2001). Enhancing security and privacy in biometrics-based authentication systems. IBM Systems Journal, 40(3): 614-634. https://doi.org/10.1147/sj.403.0614
[4] Ross, A., Dass, S., Jain, A. (2005). A deformable model for fingerprint matching. Pattern Recognition, 38(1): 275-289. https://doi.org/10.1016/j.patcog.2003.12.021
[5] Jain, A.K., Ross, A., Pankanti, S. (2006). Biometric: A tool for information security. IEEE Transactions on Information Forensics and Security, 1(2): 125-143. https://doi.org/10.1109/TIFS.2006.873653
[6] Ratha, N.K., Chikkerur, S., Connell, J.H., Bolle, R.M. (2007). Generating cancellable fingerprint templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4): 561-572. https://doi.org/10.1109/TPAMI.2007.1004
[7] Lin, C., Kumar, A. (2018). Matching contactless and contact-based conventional fingerprint images for biometrics identification. IEEE Transactions on Image Processing, 27(4): 2008-2021. https://doi.org/10.1109/TIP.2017.2788866
[8] Cappelli, R., Ferrara, M., Maltoni, D. (2010). Minutia cylinder-code: A new representation and matching technique for fingerprint recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(12): 2128-2141. https://doi.org/10.1109/TPAMI.2010.52
[9] Cardinaux, F., Sanderson, C., Bengio, S. (2006). User authentication via adapted statistical models of face images. IEEE Transactions on Signal Processing, 54(1): 361-373. https://doi.org/10.1109/TSP.2005.861075
[10] Battaglia, F., Iannizzotto, G., Lo Bello, L. (2014). A biometric authentication system based on face recognition and RFID tags. Mondo Digitale, 13(49): 340-346.
[11] Hossain, M.S., Muhammad, G., Rahman, S.M.M., Abdul, W., Alelaiwi, A., Alamri, A. (2016). Toward end-to-end biometrics-based security for IoT infrastructure. IEEE Wireless Communications, 23(5): 44-51. https://doi.org/10.1109/MWC.2016.7721741
[12] Battaglia, F., Iannizzotto, G., Lo Bello, L. (2017). A person authentication system based on RFID tags and a cascade of face recognition algorithms. IEEE Transactions on Circuits and Systems for Video Technology, 27(8): 1676-1690. https://doi.org/10.1109/TCSVT.2016.2527299
[13] Marino, C., Penedo, M.G., Penas, M., Carreira, M.J., Gonzalez, F. (2006). Personal authentication using digital retinal images. Pattern Analysis and Applications, 9: 21-33. https://doi.org/10.1007/s10044-005-0022-6
[14] Pillai J.K., Patel V.M., Chellappa, R., Ratha, N.K. (2011). Secure and robust iris recognition using random projections and sparse representations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(9): 1877-1893. https://doi.org/10.1109/TPAMI.2011.34
[15] Hollingsworth, K.P., Bowyer, K.W., Flynn, P.J. (2011). Improved iris recognition through fusion of Hamming distance and fragile bit distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12): 2465-2476. https://doi.org/10.1109/TPAMI.2011.89
[16] Sui, Y., Zou, X., Du, E., Li, F. (2014). Design and analysis of a highly user-friendly secure, privacy-preserving and revocable authentication method. IEEE Transactions on Computers, 63(4): 902-916. https://doi.org/10.1109/TC.2013.25
[17] Sanchez-Reillo, R., Sanchez-Avila, C., Gonzales-Marcos, A. (2000). Biometric identification through hand geometry measurements. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10): 1168-1171. https://doi.org/10.1109/34.879796
[18] Kumar, A., Zhang, D. (2006). Personal recognition using hand-shape and texture. IEEE Transactions on Image Processing, 15(8): 2454-2461. https://doi.org/10.1109/TIP.2006.875214
[19] Kanhangad, V., Kumar, A., Zhang, D. (2011). Contactless and pose invariant biometric identification using hand surface. IEEE Transactions on Image Processing, 20(5): 1415-1424. https://doi.org/10.1109/TIP.2010.2090888
[20] Ramalho, M.B., Correia, P.L., Soares, L.D. (2012). Hand-based multimodal identification system with secure biometric template storage. IET Computer Vision, 6(3): 165-173. https://doi.org/10.1049/iet-cvi.2011.0095
[21] Malassiotis, S., Aifanti, N., Strintzis, M.G. (2006). Personal authentication using 3-D finger geometry IEEE Transactions on Information Forensics and Security, 1(1): 12-21. https://doi.org/10.1109/TIFS.2005.863508
[22] Huang, D.S., Jia, W., Zhang, D. (2008). Palmprint verification based on principal lines. Pattern Recognition, 41(4): 1316-1328. https://doi.org/10.1016/j.patcog.2007.08.016
[23] Zhang, D., Kanhangad, V., Nan, L., Kumar, A. (2010). Robust palmprint verification using 2D and 3D features. Pattern Recognition, 43(1): 358-368. https://doi.org/10.1016/j.patcog.2009.04.026
[24] Yan, P., Bowyer, K.W. (2007). Biometric recognition using 3D ear shape. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(8): 1297-1308. https://doi.org/10.1109/TPAMI.2007.1067
[25] Kumar, A., Wu, C. (2012). Automated human identification using ear imaging. Pattern Recognition, 45(3): 956-968. https://doi.org/10.1016/j.patcog.2011.06.005
[26] Prakash, S., Gupta, P. (2014). Human recognition using 3D ear images. Neurocomputing, 140: 317-325. https://doi.org/10.1016/j.neucom.2014.03.007
[27] Wang, Y., Xie, W., Yu, X., Shark, L.K. (2015). An automatic physical access control system based on hand vein biometric identification. IEEE Transactions on Consumer Electronics, 61(3): 320-327. https://doi.org/10.1109/TCE.2015.7298091
[28] Zhou, Y. Kumar, A. (2011). Human identification using palm-vein images. IEEE Transactions on Information Forensics and Security, 6(4): 1259-1274. https://doi.org/10.1109/TIFS.2011.2158423
[29] Liu, Z., Song, S. (2012). An embedded real-time finger-vein recognition system for mobile devices. IEEE Transactions on Consumer Electronics, 58(2): 522-527. https://doi.org/10.1109/TCE.2012.6227456
[30] Bhattacharya, D., Bandopadhaya, S.K., Das, P, Ganguly, D., Mukherjee, S. (2008). Statistical approach for offline handwritten signature verification. Journal of Computer Science, 4(3): 181-185. https://doi.org/10.3844/jcssp.2008.181.185
[31] Diaz, M., Ferrer, M.A., Eskander, G.S., Sabourin, R. (2017). Generation of duplicated off-line signature images for verification systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(5): 951-964. https://doi.org/10.1109/TPAMI.2016.2560810
[32] Çamlıkaya, E., Yanıkoğlu, B., Erdoğan, H. (2007). Saklı Markov modeli kullanarak ses ile metin bağımlı kimlik doğrulama. 2007 IEEE 15th Signal Processing and Communications Applications, Eskişehir, Turkey, pp. 1-4. https://doi.org/10.1109/SIU.2007.4298722
[33] Zhang, Y., Rasku, J., Juhola, M. (2012). Biometric verification of subjects using saccade eye movements. International Journal of Biometrics, 4(4): 317-337. https://doi.org/10.1504/IJBM.2012.049736
[34] Abo-Zahhad, M., Ahmed, S.M., Abbas, S.N. (2015). A novel biometric approach for human identification and verification using eye blinking signal. IEEE Signal Processing Letters, 22(7): 876-880. https://doi.org/10.1109/LSP.2014.2374338
[35] Lyamin, A.V., Cherepovskaya, E.N. (2017). An approach to biometric identification by using low-frequency eye tracker. IEEE Transactions on Information Forensics and Security, 12(4): 881-891. https://doi.org/10.1109/TIFS.2016.2639342
[36] Monrose, F., Reiter, M.K., Wetzel, S. (2002). Password hardening based on keystroke dynamics. International Journal of Information Security, 1(2): 69-83. https://doi.org/10.1007/s102070100006
[37] Bergadano, F., Gunetti, D., Picardi, C. (2002). User authentication through keystroke dynamics. ACM Transactions on Information and System Security, 5(4): 367-397. https://doi.org/10.1145/581271.581272
[38] Ribaric, S., Fratrick, I. (2005). A biometric identification system based on fingerpalm and eigenfinger features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(11): 1698-1709. https://doi.org/10.1109/TPAMI.2005.209
[39] Conti, V., Militello, C., Sorbello, F., Vitabile, S. (2010). A frequency-based approach for features fusion in fingerprint and iris multimodal biometric identification systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(4): 384-395. https://doi.org/10.1109/TSMCC.2010.2045374
[40] Koike-Akino, T., Mahajan, R., Marks, T.K., Wang, Y., Watanabe, S., Tuzel, O., Orlik, P. (2016). High-accuracy user identification using EEG biometrics. 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, pp. 854-858. https://doi.org/10.1109/EMBC.2016.7590835
[41] Hezil, N., Boukrouche, A. (2017). Multimodal biometric recognition using human ear and palmprint. IET Biometrics, 6(5): 351-359. https://doi.org/10.1049/iet-bmt.2016.0072
[42] Kavitha P.T., Vinod, A.P. (2017). Toward EEG-based biometric systems: The great potential of brain-wave-based biometrics. IEEE Systems, Man, and Cybernetics Magazine, 3(4): 6-15. https://doi.org/10.1109/MSMC.2017.2703651
[43] Hu, J-f. (2009). New biometric approach based on motor imagery EEG signals. 2009 International Conference on Future BioMedical Information Engineering, Sanya, China, pp. 94-97. https://doi.org/10.1109/FBIE.2009.5405787
[44] Thorpe, J., van Oorschot, P.C., Somayaji, A. (2005). Pass-thoughts: Authenticating with our minds. 2005 Workshop on New Security Paradigms, Lake Arrowhead California, USA, pp. 45-56. https://doi.org/10.1145/1146269.1146282
[45] Chuang, J., Nguyen, H., Wang, C., Johnson, B. (2013). I think, therefore I am: Usability and security of authentication using brainwaves. In: Financial Cryptography and Data Security, Lecture Notes in Computer Science; Adams, A.A., Brenner, M., Smith, M. (eds). Springer, Berlin, Heidelberg, Germany, 7862: 1-16. https://doi.org/10.1007/978-3-642-41320-9_1
[46] Veeramalla, S.K., Talari, V.K.H.R. (2019). Estimation of neural sources from EEG measurements using sequential monte carlo method. Ingénierie des Systèmes d’Information, 24(4): 411-417. https://doi.org/10.18280/isi.240408
[47] Reshmi, K.C., Ihsana Muhammed, P., Priya, V.V., Akhila, V.A. (2016). A novel approach to brain biometric user recognition. Procedia Technology, 25: 240-247. https://doi.org/10.1016/j.protcy.2016.08.103
[48] Ruiz-Blondet, M.V., Jin, Z., Laszlo, S. (2016). CEREBRE: A novel method for very high accuracy event-related potential biometric identification. IEEE Transactions on Information Forensics and Security, 11(7): 1618-1629. https://doi.org/10.1109/TIFS.2016.2543524
[49] Su, F., Zhou, H., Feng, Z., Ma, J. (2012). A biometric-based covert warning system using EEG. 2012 5th IAPR International Conference on Biometrics, New Delhi, India, pp. 342-347. https://doi.org/10.1109/ICB.2012.6199830
[50] Wu, Q., Zeng, Y., Zhang, C., Tong, L., Yan, B. (2018). An EEG-based person authentication system with open-set capability combining eye blinking signals. Sensors, 18(2): 335-352. https://doi.org/10.3390/s18020335
[51] Ishikawa, Y., Yoshida, C., Takata, M., Kamo, H., Joe, K. (2015). A personal classification method using spatial information of multi-channel EEG. 21st International Conference on Parallel and Distributed Processing Techniques and Applications, Monte Carlo Resort, Las Vegas, USA, pp. 229-235.
[52] Marcel, S., del Millan, J. (2007). Person authentication using brainwaves (EEG) and maximum a posteriori model adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4): 743-748. https://doi.org/10.1109/TPAMI.2007.1012
[53] Sohankar, J., Sadeghi, K., Banerjee, A., Gupta, S.K.S. (2015). E-BIAS: A pervasive EEG-based identification and authentication system. 11th ACM Symposium on QoS and Security for Wireless and Mobile Networks, Cancun, Mexico, pp. 165–172. https://doi.org/10.1145/2815317.2815341
[54] Riera, A., Soria-Frisch, A., Caparrini, M., Cester, I., Ruffini, G. (2009). Multimodal physiological biometrics authentication. In: Biometrics: Theory, Methods, and Applications, Boulgouris, N.V., Plataniotis, K.N., Micheli-Tzanakou, E. (eds). Wiley-IEEE Press, pp. 461-482. https://doi.org/10.1002/9780470522356.ch18
[55] Zeng, Y., Wu, Q., Yang, K., Tong, L., Yan, B., Shu, J., Yao, D. (2018). EEG-based identity authentication framework using face rapid serial visual presentation with optimized channels. Sensors, 19(1): 6. https://doi.org/10.3390/s19010006
[56] Goudiaby, B., Othmani, A., Nait-ali, A. (2020). EEG biometrics for person verification. In: Hidden Biometrics, Series in BioEngineering, Nait-ali, A. (eds). Springer, Singapore, pp. 45-69. https://doi.org/10.1007/978-981-13-0956-4_3
[57] Zeynali, M., Seyedarabi, H. (2019). EEG-based single-channel authentication systems with optimum electrode placement for different mental activities. Biomedical Journal, 42(4): 261-267. https://doi.org/10.1016/j.bj.2019.03.005
[58] Orrú, G., Garau, M., Fraschini, M., Acedo, J., Didaci, L., Ibáñez, D., Soria-Frish, A., Marcialis, G.L. (2019). Personal identity verification by EEG-based network representation on a portable device. In: Computer Analysis of Images and Patterns, Lecture Notes in Computer Science; Vento, M., Percannella, G. (eds). Springer, Cham, pp. 164-171. https://doi.org/10.1007/978-3-030-29891-3_15
[59] He, C. (2009). Person authentication using EEG brainwave signals. Master of Applied Science dissertation, Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC Canada.
[60] Pan, F.Q., Yang, Y.Z., Zhang, L.X., Yang, X.X., Yang, J.S., Liu, M.J. (2021). Analysis of EEG characteristics of drivers at the entrance and exit of an undersea tunnel and research on driving safety. International Journal of Safety and Security Engineering, 11(2): 155-165. https://doi.org/10.18280/ijsse.110204
[61] Kumar, M.G., Narayanan, S., Sur, M., Murthy, H.A. (2021). Evidence of task-independent person-specific signatures in EEG using subspace techniques. IEEE Transactions on Information Forensics and Security, 16: 2856-2871. https://doi.org/10.1109/TIFS.2021.3067998
[62] Rahman, A., Chowdhury, M.E.H., Khandakar, A., Kiranyaz, S., Zaman, Kh.S., Reaz, M.B.I., Islam, M.T., Ezeddin, M., Kadir, M.A. (2021). Multimodal EEG and keystroke dynamics based biometric system using machine learning algorithms. IEEE Access, 9: 94625-94643. https://doi.org/10.1109/ACCESS.2021.3092840
[63] Schalk, G., McFarland, D.J., Hinterberger, T., Birbaumer, N., Wolpaw, J.R. (2004). BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering, 51(6): 1034-1043. https://doi.org/10.1109/TBME.2004.827072
[64] Savas, B., Eldén, L. (2007). Handwritten digit classification using higher order singular value decomposition. Pattern Recognition, 40(3): 993-1003. https://doi.org/10.1016/j.patcog.2006.08.004
[65] Goldberger, A.L., Amaral, L.A., Glass, L., Hausdorff, J.M., Ivanov, P.C., Mark, R.G., Mietus, J.E., Moody, G.B., Peng, C.K., Stanley, H.E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23): e215-e220. https://doi.org/10.1161/01.cir.101.23.e215
[66] Wang, H., Ahuja, N. (2003). Facial expression decomposition. 9th IEEE International Conference on Computer Vision Nice, France, pp. 958-965. https://doi.org/10.1109/ICCV.2003.1238452
[67] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L-C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 4510-4520. https://doi.org/10.1109/CVPR.2018.00474
[68] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z. (2016). Rethinking the inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 2818-2826. https://doi.org/10.1109/CVPR.2016.308
[69] Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 1800-1807. https://doi.org/10.1109/CVPR.2017.195
[70] He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 770-778. https://doi.org/10.1109/CVPR.2016.90
[71] Tan, M., Le, Q.V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. ArXiv, abs/1905.11946. https://doi.org/10.48550/arXiv.1905.11946
[72] Palanichamy, I., Ahamed, F.B.B. (2022). Prediction of seizure in the EEG signal with time aware recurrent neural network. Revue d'Intelligence Artificielle, 36(5): 717-724. https://doi.org/10.18280/ria.360508
[73] Lopes, F., Leal, A., Medeiros, J., Pinto, M.F., Dourado, A., Dümpelmann, M., Teixeira, C. (2021). Automatic electroencephalogram artifact removal using deep convolutional neural networks. IEEE Access, 9: 149955-149970. ttps://doi.org/10.1109/ACCESS.2021.3125728
[74] Jiang, X., Bian, G.-B., Tian, Z. (2019). Removal of artifacts from EEG signals: A review. Sensors, 19(5): 987. https://doi.org/10.3390/s19050987
[75] Abdi-Sargezeh, B., Foodeh, R., Shalchyan, V., Daliri, M.R. (2021). EEG artifact rejection by extracting spatial and spatio-spectral common components. Journal of Neuroscience Methods, 358: 109182. https://doi.org/10.1016/j.jneumeth.2021.109182
[76] Tamburro G., Croce, P., Zappasodi, F., Comani S. (2021). Is brain dynamics preserved in the EEG after automated artifact removal? A validation of the fingerprint method and the automatic removal of cardiac interference approach based on microstate analysis. Frontiers in Neuroscience, 14: 577160. https://doi.org/10.3389/fnins.2020.577160
[77] Mumtaz, W., Rasheed, S., Irfan, A. (2021). Review of challenges associated with the EEG artifact removal methods. Biomedical Signal Processing and Control, 68: 102741. https://doi.org/10.1016/j.bspc.2021.102741
[78] Yang, B., Duan, K., Fan, C., Hu, C., Wang, J. (2018). Automatic ocular artifacts removal in EEG using deep learning. Biomedical Signal Processing and Control, 43: 148-158. https://doi.org/10.1016/j.bspc.2018.02.021
[79] Mannan, M., Kim, S., Jeong, M., Kamran, M. (2016). Hybrid EEG—Eye tracker: Automatic identification and removal of eye movement and blink artifacts from electroencephalographic signal. Sensors, 16(2): 241. https://doi.org/10.3390/s16020241
[80] Gomez-Herrero, G., Clercq, W., Anwar, H., Kara, O., Egiazarian, K., Huffel, S., Paesschen, W. (2006). Automatic removal of ocular artifacts in the EEG without an EOG reference channel. 7th Nordic Signal Processing Symposium, Reykjavik, Iceland, pp. 130-133. https://doi.org/10.1109/NORSIG.2006.275210
[81] Delorme, A., Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1): 9-21. https://doi.org/10.1016/j.jneumeth.2003.10.009
[82] Castellanos, N.P., Makarov, V.A. (2006). Recovering EEG brain signals: Artifact suppression with wavelet enhanced independent component analysis. Journal of Neuroscience Methods, 158(2): 300-312. https://doi.org/10.1016/j.jneumeth.2006.05.033
[83] Hyvarinen, E., Oja, E. (1997). A fast fixed-point algorithm for independent component analysis. Neural Computation, 9(7): 1483-1492. https://doi.org/10.1162/neco.1997.9.7.1483
[84] Svetlakov, M., Kovalev, I., Konev, A., Kostyuchenko, E., Mitsel, A. (2022). Representation learning for EEG-based biometrics using Hilbert–Huang transform. computers. Computers, 11(3): 47. https://doi.org/10.3390/computers11030047
[85] Bidgoly, A.J., Bidgoly, H.J., Arezoumand, Z. (2022). Towards a universal and privacy preserving EEG-based authentication system. Scientific Reports, 12(1): 2531. https://doi.org/10.1038/s41598-022-06527-7
[86] Singh, B., Mishra, S., Tiwary, U.S. (2015). EEG based biometric identification with reduced number of channels. 2015 17th International Conference on Advanced Communication Technology (ICACT), PyeongChang, Korea (South), pp. 687-691. https://doi.org/10.1109/ICACT.2015.7224883
[87] Fraschini, M., Hillebrand, A., Demuru, M., Didaci, L., Marcialis, G.L. (2015). An EEG-based biometric system using eigenvector centrality in resting state brain networks. IEEE Signal Processing Letters, 22(6): 666–70. https://doi.org/10.1109/LSP.2014.2367091
[88] Kaur, B., Singh, D. (2017). Neuro signals: A future biometric approach towards user identification. 2017 7th International Conference on Cloud Computing, Data Science & Engineering-Confluence, Noida, India, pp. 112-117. https://doi.org/10.1109/CONFLUENCE.2017.7943133
[89] Sun, Y., Lo, F.P.W., Lo, B. (2019). EEG-based user identification system using 1D-convolutional long short-term memory neural networks. Expert Systems with Applications, 125: 259–267. https://doi.org/10.1016/j.eswa.2019.01.080
[90] Wang, M., Hu, J., Abbass, H. (2019). Stable EEG biometrics using convolutional neural networks and functional connectivity. Australian Journal of Intelligent Information Processing Systems, 15(3): 19-26.
[91] Alyasseri, Z.A.A., Khader, A.T., Al-Betar, M.A., Alomari, O.A. (2020). Person identification using EEG channel selection with hybrid flower pollination algorithm. Pattern Recognition, 105: 107393. https://doi.org/10.1016/j.patcog.2020.107393
[92] Balcı, F. (2023). DM-EEGID: EEG-based biometric authentication system using hybrid attention-based LSTM and MLP algorithm. Traitement du Signal, 40(1): 65-79. https://doi.org/10.18280/ts.400106
[93] Ahmed, M.I.B., Zaghdoud, R.A., Al-Abdulqader, M., Kurdi, M., Altamimi, R., Alshammari, A., Noaman, A., Ahmed, M.S., Alshamrani, R., Alkharraa, M., Rahman, A., Krishnasamy, G. (2023). Ensemble machine learning based identification of adult epilepsy. Mathematical Modelling of Engineering Problems, 10(1): 84-92. https://doi.org/10.18280/mmep.100110
