EEG Classification to Food Stimuli in Diverse Weight Groups with Regression Analysis of Eating Behavior Questionnaires

2.1 Participant

In this research, EEG data were acquired from 20 male volunteers between the ages of 21 and 41 with a BMI ranging from 21.45 to 39.43kg/m² from Yozgat Bozok University, Türkiye. The characteristics of the participants are displayed in Table 1. Participants were excluded from the study if they disclosed having a mental, neurological, or somatic condition or taking any medication within the previous month that could have influenced their eating patterns, body weight, or EEG activity. Each participant conducted three or four trials, with a ten-minute break between each session, for a total of seventy trials.

Table 1. Details of participants

Age	Weight	Height	BMI
35	79	175	25.80
28	66	170	22.84
21	62	170	21.45
22	110	180	33.95
31	72	174	23.78
22	105	193	28.19
35	85	170	29.41
41	73	181	22.28
27	130	186	37.58
23	76	182	22.94
35	118	173	39.43
22	70	170	24.22
26	75	169	26.26
38	105	182	31.70
37	80	176	25.83
33	87	188	24.62
35	90	170	31.14
25	77	183	22.99
35	82	174	27.08
26	86	175	28.08

2.2 Data acquisition and filtering

In this study, 19 channels were used with experimental parameters to record EEG signals over the scalp. Using the international 10-20 electrode placement technique, the electrodes were positioned non-invasively. A1 and A2's left-right earlobes were designated as the reference points, while the left eyebrow was designated as the ground. The EEG electrodes for monopolar placement are listed in Figure 1. Each electrode positioning point is designated by a letter associated with the specific brain lobe or region it is recording from: frontal (F), temporal (T), parietal (P), occipital (O), and central (C). The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of Yozgat Bozok University (protocol code: 2017-KAEK-189_2022.08.25_08 and date of approval: 25/08/2022) for studies involving humans.

In this investigation, images from the FoodPics dataset were used as stimuli. In total, 34 photographs (17 food and 17 non-food) were chosen. All images in this dataset are colour photographs with a resolution of 600x450 pixels (96 dpi, sRGB colour format). In terms of viewing distance (up to 80 cm), angle, and fundamental figure-ground composition, images were selected/edited to be relatively uniform. Photos were standardised based on the colour of the background (white) [29]. During the experiment, each participant sat in a chair 50 cm from the computer monitor and was instructed to focus on the images. Each image was shown for two seconds, and an example trial is depicted in Figure 2.

1.png

Figure 1. Channel names of EEG electrodes and placement

2.png

Figure 2. A depiction of the experimental setup for the examination of visual stimuli

3.png

During the research, EEG signals generated in response to visual stimuli were recorded using a Micromed SAM32RFO signal acquisition device and a Medcap electrode cap. The impedance of each channel was maintained below 10 kOhm, and each trial recording lasted 134 seconds. The sampling rate was set to 4096Hz, and the data was filtered at a frequency range of 0.03-70Hz. After the raw EEG data collection procedure was completed, the filtering step was applied. First of all, a 50Hz notch filter was applied to reduce power line noise. Then, a discrete wavelet transform was applied. In this step, a 10^th-order Butterworth bandpass filter was used to filter the four sub-band frequency components: alpha, beta, theta, and delta. This step helps to remove artifacts, which can affect the proper classification results badly. Due to the superiority of the delta wave result, a classification frequency of 0.03-4Hz was chosen. After the filtering step, data normalization was applied. The data matrix was centered by making the mean value zero. Figure 3 illustrates the fundamental steps of this work. Multi-Task Process (MTP) approach can also be seen in the figure, which is explained in detail in the study [30]. In this method, feature extraction is followed by feature data augmentation. Then, the tabular data is converted into images and given to classification algorithms. In this work, Principle Component Analysis (PCA) is added as an extra step because the size of the data matrix is large, and using PCA, features that do not have much impact can be removed. Additionally, eating behavior questionnaires and EEG results were analyzed in order to find any relation between participant answers to questionnaires and EEG recordings. In this step, three different methods, which are Random Forest, XGBoost, and LightGBM, were used. Finally, using the combination of these methods, a voting regressor was applied.

2.3 Feature extraction

The feature extraction stage involves the transformation of unprocessed signal data into a feature vector in order to extract crucial information. This stage is capable of emphasizing the distinguishing characteristics of the signal. Using the Burg method, power spectral density (PSD) was utilized in this investigation for future extraction. PSD is a valuable signal processing technique for stationary signals, and it is particularly effective for narrowband signals. It is a standard method of signal processing that distributes signal power across frequencies and displays signal energy as a function of frequency [31].

Burg's method generates an estimate of reflection coefficients by minimizing the mean of prediction errors in both forward and backward directions, while also adhering to the Levinson-Durbin recursion. This method resolves closely spaced, low-level sinusoids and estimates transient data records [32]. Following is an explanation of the Burg method:

${Pburg}(f)=\frac{\hat{\mathrm{E}}_P}{\left|1+=\sum_{k=0}^p \hat{\mathrm{a}}_p(k) e^{-2 \pi f}\right|^{\wedge} 2}$                    (1)

In this investigation, raw data was collected in segments of size (2*67*4096)*19 for each trial, where 19 represents the number of channels, 4096 the sampling frequency, 67 the number of images, and 2 the duration of each image in seconds. After collecting raw data, responses to 2-second food stimuli for each participant were subtracted. 17 food images multiplied by 2 seconds equals 34 data length, yielding 34*4096 data matrices per trial. Finally, the method of feature extraction was implemented. Using the Burg method, a 34*129 data matrix was acquired, and for 70 trials, a 2380*129 data matrix was created.

2.4 Tabular data augmentation

In the field of data science, datasets can be divided into structured and unstructured data. Tabular data, the most prevalent form of data in real-world applications, consists of a set of rows and columns containing features. Many difficulties arise when supplying them to deep neural networks, such as lack of locality, absent values, mixed feature types, and, unlike text or images, lack of prior knowledge of the data set's structure [33]. It can be said that there are several reasons for data augmentation [34], such as: limited training data [35], lack of relevant data [36], model overfitting [37] and imbalanced data [38].

There are a total of 2380 data columns in the dataset compiled for this study, of which 1462 represent individuals who are overweight/obese and 918 represent individuals who are normal. In order to eliminate imbalance and overfitting problems, data augmentation is applied. After synthetically increasing the data, the total number reached 2924 (1462 overweight/obese and 1462 normal).

2.5 Dimension reduction

PCA is a method for reducing the number of features. It calculates the variance within the data by creating a new set of perpendicular characteristics called principal components (PCs). PCA endeavors to eradicate duplicate dimensions from a given data space. This provides an easy method for ML algorithms to generate reduced data sets [39, 40]. In this investigation, feature-extracted data for each trial was reduced to 2924*2 dimensions using the Burg method. Figure 4 illustrates the principal components extracted for the data set used in the study.

4.png

Figure 4. Principal component distribution of used data

2.6 Feature embedding into the images

In this stage, the SuperTML method was used in order to convert the tabular data into images. This method is motivated by the resemblance between the challenges encountered in TML and those in text classification tasks [41]. Using this method, all instances and their features were embedded in the images, resulting in 2924 images. The dimension of the images was chosen as 224*224 because of the deep model input size. Figure 5 shows the tabular data conversion to images and filing procedures.

5.jpg

Figure 5. Feature conversion from tabular data to images

After all the pictures were saved in class folders, the data was split into train and test parts with a ratio of 80:20. Then, 30% of the train data is set aside for validation with the Keras image data generator pipeline. As a result, 2339 of 2924 data points were reserved for training, and 702 of them were selected for validation. For the test set, 585 pieces of data were reserved.

2.7 Classification using CNN models

CNNs have attained remarkable efficacy in image classification and detection. A CNN identifies an object by searching for basic characteristics, such as edges, lines, and curves, and then constructs additional features with an overall perspective. DL draws its inspiration from traditional neural networks but significantly outperforms its predecessors. Furthermore, DL utilizes transformations and graph technologies concurrently to construct multi-layer learning models [42].

Performance evaluation metrics are crucial evaluation criteria for the ML system used in biomedical signal processing. This study included four assessment metrics: accuracy, precision, recall, and F1-Score. The following are mathematical representations of assessment metrics:

$Accuracy =\frac{T P+T N}{T P+T N+F P+F N}$                   (2)

$Precision =\frac{T P}{T P+F P}$                   (3)

$Recall=\frac{T P}{T P+F N}$                 (4)

$F 1 \,\, Score =2 * \frac{{ Precision } *  { Recall }}{{ Precision }+  { Recall }}$                 (5)

Here, TP represents true positive decision, TN means true negative, FP stands for false positive and FN is false negative decision.

In this study, four different CNN models (DenseNet121, Inception-v3, ResNet50-v2 and ResNet152-v2) were used. These models are well known and widely used in the field of biomedical signal processing. One of our aims in using these models is to compare our results with established benchmarks in the literature, which provide a reference point for evaluating the effectiveness of our approach. It is important for the study that each of the models used has a different level of complexity and capacity. Here, ResNet-50 is a lighter model than ResNet-152 or DenseNet-121. Using these models, it is aimed at exploring how model complexity affects performance on a given task. Different architectures can capture different types of features from input images. For example, Inception-v3 is known for its ability to capture fine-grained details due to its use of multiple filter sizes. By using various models, different types of features were utilized, and higher classification results were sought. These models, which have been tested for robustness and generalization using different datasets and architectures, help to see the robustness of our findings on different architectures. These models have been used with high accuracy in various studies in the literature. They are also used both in the processing of the EEG signal and in the processing of the EEG signal converted into an image [43, 44]. The pre-trained CNN models were supplied with ImageNet weights for transfer learning, and the model's top property was set to false. Gaussian noise was added at a rate of 0.7, and dropout was chosen as 0.3 in order to avoid the model becoming overfitted. Additionally, 4096 neurons were added for fully connected layers. CNN models were trained with Keras over the course of 30 epochs. All convolutional layers were defrozen, time-based decay was implemented, and the initial learning rate was set to 0.004. The rate of decay is proportional to the rate of learning, the epoch value, and momentum. Momentum applied 0.4 to the Stochastic Gradient Descent (SGD) optimizer, which tends to produce superior outcomes.

2.8 Application of eating behavior questionaries

In the paper, two different eating behavior questionnaires were applied to the participants. These are Turkish versions of the Dutch Eating Behavior Questionnaires (DEBQ) and Three-Factor Eating Questionnaires (TFEQ). Validity and reliability of the Turkish versions of these questionnaires were proven in the researches [45, 46]. While the questionnaires were scored, the Likert scale was used and the results are given in Table 2.

Table 2. Questionnaire results

After questionnaires were scored, regression analyses were carried out in order to seek out the correlation between the EEG recording of participants and the questionnaire results. In this step, the features extracted from the EEG using the burg method were augmented utilizing the SMOGN algorithm. This technique can be used in order to eliminate imbalance problems in regression models and is explained in detail in the research [47]. A widely used technique for model selection is the method of cross-validation. Essentially, this involves partitioning the data into subsets. One portion is used for fitting each candidate model or method, and the remaining portion is used for evaluating their performance. The model exhibiting the most favorable overall performance is ultimately chosen [48]. In this study, 10-fold cross validation is applied. Then, Random Forest, XGBoost, and LightGBM techniques were applied for regression analysis using the best-scoring EEG channel, which is T4. Finally, the voting regressor, which is combination of these three methods, was applied. Random Forest, XGBoost, and LightGBM, all tree-based ensemble methods, are based on different algorithms and hyperparameters. Using a combination of these models can capture the various patterns and relationships present in the data. The aim of using an ensemble approach is to reduce the weaknesses of individual models and improve their overall performance. These ML-based algorithms can manage complex relationships between predictor variables and target variables, making them suitable for capturing fine details in survey data. Ensemble methods such as Random Forest and boosting algorithms (XGBoost and LightGBM) are less likely to suffer from overfitting than some other ML methods. This is important in regression tasks where the goal is to generalize well to unseen data, ensuring that the predictions of the model created are reliable. Random Forest, XGBoost, and LightGBM have built-in mechanisms to evaluate feature importance. This feature can be valuable for interpreting the results of your regression analysis. Knowing which characteristics have the most significant impact on eating behavior may provide insight for further research. It is possible to improve the overall performance and robustness of regression analysis by combining the predictions of several models using a voting regressor. The ensemble reduces the risk of making decisions based on the weaknesses of a single model, while exploiting the strengths of each model. In addition, these models are widely used in regression analysis and give very promising results [49, 50].

Using CNNs, this paper attempts to examine the variations in how people in various weight groups react to food imagery. During the data preprocessing, tabular data augmentation techniques are applied in order to eliminate overfitting and imbalance, and the data is also converted to images to show the applicability of the SuperTML method on EEG signals. At this stage, it is seen that this method gives effective results and can be used for EEG data. The regression analysis is also undergone in this study in order to show the correlation between questionnaires and EEG records, three different algorithms and the ensemble model.

In the study, each session lasts 134 seconds, and the sampling frequency is determined as 4096Hz. This actually shows that the data received from a person is at a quite acceptable level. In addition, the received data is subjected to a second process, the data augmentation technique. For this reason, it is thought that the sample size of the data obtained is sufficient. In addition, it has been observed that the number of participants used in numerous studies on EEG in the literature is similar to or less than this study.

Since it is believed that different responses may occur in various brain regions, each of the 19 channels has its own data analysis. Examining the channel findings reveals that channels F4 and T4 perform better in terms of classification than other channels that are comparable to the study [52], channel Cz, in comparison, receives the lowest accuracy ratings. In addition, 5 different brain regions are examined to seek out regional differences. It is seen that temporal lobe results give more successful average accuracy rates of around 0.90 compared to other regions. Similar results were observed in the study [53] whereas the central region seems to be the least successful part. In terms of CNN algorithms, the best results are obtained by using DenseNet-121 with a rate of 0.97. Looking at the differences in the right and left hemispheres of the brain, it is seen that the classification result is higher on the right side. This can be shown as a sign of increased activation in this region, as a matter of fact, similar results were obtained in another study. In addition, in the same study, it was emphasized that the differences in the frontal region were more pronounced, which is parallel to our study [54].

In terms of regression analysis, it can be said that the used models show excellent performance. It is evident that EEG data and questionnaires have a strong correlation. When the subsection of questionnaires is considered, the restrain eating section of DEBQ has the best correlation among the others, while the mean result of TFEQ is the least successful section.

Additionally, classification results show that brain signals fluctuate when a person's weight status varies. This indicates that the human brain is significantly impacted by body weight, as given in the researches [5, 55]. Being overweight can have negative psychological impacts in addition to physical ones because it alters brain messages. On the other side, people might be more prone to putting on too much weight as a result of these variations in brain impulses.

Considering the limitations of the study, the first thing that can be said that there is no strict control over whether participants received food on the day of the trial. In addition, there is no information about any emotional state changes experienced by the participants that day, which could change the data structure. Another issue is that data is collected only from male individuals, which may be one of the limitations of the study because women's and men's reactions to stimuli may be different. Additionally, the lack of a regression study between eating behavior questionnaires and EEG prevented us from performing a comparative analysis. However, presenting this first also shows the novelty of our work.