Investigating the Effect of COVID-19 Infection on P300 Based BCI Application Performance

In this study, experiments were carried out utilizing the visual P300 stimulus model, which was proposed by Magliero et al. [17]. Also an Artificial Neural Networks (ANN) model was used to detect the P300 signal. In the following subsections, first the experimental design is introduced. Then the experimental procedure is defined. And finally, P300 signal processing method is given in the last subsection.

2.1 Experimental design

EEG dataset was recorded from six participants, aged between 26 and 46. They did not used any neurological drug and had a neurological disorder until had not COVID-19 infection. Data collection process was approved by the Health Sciences Institute Ethics Board of Ataturk University, and all participants signed the Consent Form, verified the board, before the start of EEG recording session. All EEG data was obtained in two sessions from these subjects, before and after the diagnosis of COVID-19. All participants reported that they have no visual impairment. The exact dates of the before the COVID-19 infection EEG recordings along with the infection diagnosis dates are given in Table 1. EEG recordings were obtained in about 15-30 days following the COVID-19 infection.

In the current study, the row-column based visual P300 spelling paradigm, first proposed by Donchin et al., is used. A 6x6 character matrix, shown in Figure 1a, is presented to the participants in order to collect target stimuli data. In each trial, the subject was asked to focus a character which was pre-determined. This character was verbally announced by the researcher while it was displayed on the monitor as shown in Figure 1b. A total of 85 letters were shown as target characters during each experiment. For that, a row or a column, out of 12 in the character matrix, is flashed randomly for each target character. In each set of flashings generated for a specific target character, all rows and columns are flashed once. Therefore, 12 flashings occur while the target character appears in 2 contrary to other 10 flashings as the character is included in a row and in a column.

1.png

Figure 1. Visual paradigm: (a) Character matrix (b) A sample target character to focus on

EEG recordings were collected with Brain Product ActiChamp device in all conducted experiments. Electrodes were placed according to the international 10/20 system and unipolar EEG recordings were performed. 'F2' channel is used as the reference electrode. Sampling frequency was determined as 250 Hz for all recording. The experimental system consists of two computers and the EEG device. EEG device combines data acquired from the electrodes with markers generated by the visual stimulation presentation computer and forward all information to the recording software running on the second computer. As a result, EEG data and visual stimulus type (markers) are recorded simultaneously. The whole data acquisition block-diagram is presented in Figure 2.

2.png

Figure 2. Data acquisition block diagram

Table 1. Experiment and diagnosis dates

2.2 Experimental procedure

In the proposed study, the row-column based visual P300 spelling paradigm is used. The visual stimuli were shown on a 1920 x 1080 resolution LED display. Experiments were carried out while the participants were sitting on a comfortable armchair with their head about 100 cm away from the screen. The visual stimuli were shown on the screen for 100 ms with interstimulus interval (ISI) of 75 ms as demonstrated in Figure 3.

Each run for which a character is flashed twice was completed with randomly flashing 6 rows and 6 columns together with the ISI periods. During the experiments, each run for a specific character is repeated consequently for 15 times, called trial, and the experiment is performed for 85 letters. A three-minute break was given after 23 character display trials with a last session performed for 16 characters. Also, the following target character was shown on the screen during 5 seconds after each trial. This way, an additional rest time is provided for the participant while the target character location is introduced. That is, each run is completed in 2.1 seconds while a trial takes about 36.5 seconds yielding an experiment time of 61 minutes including the breaks. The timing details are demonstrated in Figure 3.

3.png

Figure 3. Experiment paradigm

2.3 P300 signal processing

EEG signals were recorded on 10 channels for O1, Oz, O2, P3, P4, Pz, P7, P8, ground and reference electrodes, located according to 10-20 international location procedure. Electrodes used in the analysis are shown in Figure 4. The general flow chart of the P300 signal processing procedure is shown in Figure 5.

4.png

Figure 4. (a) Electrodes, (b) electrode groups, (c) equipment, (d) a participant

5.png

Figure 5. General flowchart

Independent component analysis (ICA), time/frequency analysis, artifact elimination and determination of event-related potentials were performed on each recorded EEG data.

At the very beginning a 0.1-10 Hz bandpass filtering is applied since P300 signals have low frequency components [18]. After that, ICA was performed to define independent EEG channels. With this approach, noise in the EEG channels can be determined and eliminated [19]. Artifacts, formed in the frontal region on EEG channels depending on eye movements or noise and occurred in the EEG channels due to various factors such as movement of the subject during the experiment, is suppressed by ICA analysis [20, 21]. This way, higher classification accuracy (CA) was achieved by removing eye and other artifacts utilizing ICA.

In the second step, EEG data was divided into two classes as data with and without target stimuli depending on the specific flashing during the data collection. Hence, the response of the brain to 2 flashings, which include the stimulus, out of 12 flashings is different from all others since these two results in P300 waves. However, the P300 wave is not observable in EEG signals of a single trial. Instead, it is obtained by averaging more than one EEG signals with target stimuli. This is shown in Figure 6. Here, one-second EEG signals for 12 flashings are shown for 15 repetitions as well as their averages obtained for target character “S”. As can be seen clearly, the P300 wave is observable only in two average EEG flashings, one for row with target character (number 4) and the other for column (number 10). During the data recording, 85 characters were shown to each participant as target. This results in, EEG data collection of 170 (2×85) with P300 wave and 850 (10×85) with non-P300 in each experiment per subject.

6.png

Figure 6. P300 waves obtained by averaging the EEG signals

This obtained data samples was randomly divided into 70% training, 15% validation, and 15% test partitions for each subject. It should be noted that the aforementioned P300 signal processing procedure is repeated twenty times in order to show the robustness of the method and avoid the problems of the random selections of the training, validation and test sets. The average CA was calculated for the Parietal and Occipital region EEG electrodes, used in this study, as they are related to visual activities [22, 23]. Since the inequality between the sample size of the two classes has a negative effect on the classification task success, synthetic data was generated to eliminate this imbalance between the classes. For this purpose, synthetic P300 data was created by adding Gaussian distributed noise, with 0 mean and 0.01 standard deviation, to the existing P300 samples as suggested by [22, 24, 25]. Thus, the imbalance between the classes was removed by increasing the samples of P300 data from 170 to 850, that of the non-P300 data samples. Then, baseline noise was removed from P300 and non-P300 EEG data sets by evaluating the 200ms EEG data before the arrival of the target stimulus, and using this information for the baseline removal of 1000ms EEG data after the arrival of the target stimulus as shown in Figure 7.

7.png

Figure 7. P300 data example before and after the baseline removal

8.png

Figure 8. Two-layer artificial neural network model

The resulting data set, cleared of baseline noise and equalized in both classes sample sizes are then used in a two-class classification problem, namely P300 and non-P300. Finally, a single-output two-layer ANN, including one hidden layer with 50 neurons [26, 27], was used for classifying the obtained EEG signals. The training of ANN is performed by updating the model weights in an iterative manner using the supplied training data, and it is verified using the validation data. At the end, the model is tested using test data. To evaluate the performance, an error term, the difference between the estimated value through the testing and the corresponding data label, is calculated. A sample neural network model with single output and two-layers is shown in Figure 8, and given by Eq. (1).

$\hat{y}=\tilde{g}\left(\sum_{j=1}^{M} w_{1 j}^{(2)} * g\left(\sum_{i=1}^{d} w_{j i}^{(1)} * x_{i}+w_{j 0}^{(1)}\right)+w_{10}^{(2)}\right.$       (1)

where, $\tilde{g}, g, w^{(1)}$ and $w^{(2)}$ represent the linear function, the tan-sigmoid function, hidden layer weights and output layer weights, respectively. The error function J(w), on the other hand, is represented by Eq. (2).

$J(w)=-\frac{1}{N} \sum_{n=1}^{N}\left[y_{n} \log \hat{y}_{n}+\left(1-y_{n}\right) \log \left(1-\hat{y}_{n}\right)\right]$       (2)

where, N is the number of samples in the test set, $y_{n}$  is the real value (label) and $\hat{y}_{n}$ is its estimated label produced by the model. This error term is subjected to a minimization process during the training for a satisfactory result.

In this study, it is investigated the effect of COVID-19 infection on the performance of P300 based BCI application as well as change in the P300 wave latency using the data of six subjects. For that, corresponding results, obtained before and 15-30 days after the COVID-19 infection, are compared with each other. It is worthwhile to mention that because the participants did not used any neurological drug and had not a neurological disorder until had COVID-19 infection, we believe that the brain activity were not distorted by any daily life changes.

It is observed that there is a visible difference in the BCI performance, mostly in the occipital region electrodes. Therefore, it is thought that this may cause vision problems in people. However, it is observed from the t-test results that there is a significant difference only on the performance of the Oz channel for alpha=0.05 while there is no difference for any EEG channel for alpha=0.01. On the other hand, latency comparison for before and after the COVID-19 infection is subject dependent as it increased for some subjects while decreasing for others regardless of their sex or age. The corresponding t-test also showed that there is no difference for latency. As a result, we concluded that the COVID-19 infection, in general, has no negative effect on the EEG data from a statistically point of view.