Predictive Modelling of Neurological Disorders Using Machine Learning: Insights from EEG, MRI, and Cognitive Data

Neurological diseases are disorders of the brain, spinal cord and the nerves that connect them. The severity of these diseases is such that they disrupt normal functioning. These Include a variety of disorders, ranging from neurodegenerative disorders such as Alzheimer’s disease to Parkinson’s disease, multiple sclerosis, epilepsy, and stroke [1-3]. Different causes underlie each of these diseases, making them especially complex. They can be caused by your genes and whether or not your environment and lifestyle help your genes express. Neurological diseases can have a serious impact on quality of life, causing disability and the need for long-term medical care [4, 5]. Neurological diseases are getting more common likely because of aging and better diagnosis at an advanced level. WHO organization states that the neurological disorders are causes of death and disability in similar measures [6, 7]. Millions of people around the world are affected by Alzheimer’s disease and other dementias, and this will rise in the coming decades [8, 9]. Due to their progressive nature, it is essential to diagnose and manage interventions as soon as possible.

1.1 Historical background of neurological disease diagnosis

Long ago in ancient civilizations, certain conditions, namely epilepsy, were blamed on supernatural causes, signifying the start of neurology history [10, 11]. The Greek doctor Hippocrates was one of the first people to say that disorders of the nervous system are caused by natural (a physiological dysfunction) not divine punishment. During the Renaissance, more advanced anatomical studies were used to better understand the nervous system [12]. In the 19th and 20th centuries, neuroanatomy and neuropathology were revolutionised by pioneering scientists like Jean-Martin Charcot who made substantial contributions to neurological diseases understanding [13, 14].

CTs and MRIs came into the picture during the 20th century and these imaging techniques helped diagnosis of diseases of the nervous system [15, 16]. Non-evasive imaging technologies were developed to visualize brain structures and detect abnormalities early on. The ability of Electroencephalography (EEG) to diagnose epilepsy as well as the introduction of positron emission tomography (PET) and functional MRI (fMRI) to assess brain function and metabolism [17-19]. Even with these bettering methods, the traditional diagnostic methods focus on clinical examination, patient history, and symptomatology, resulting in delay or misdiagnosis.

The development of artificial intelligence (AI) and machine learning (ML) technology provides promising solutions to enhance diagnostic precision while reducing healthcare system workload. Research has proven that ML algorithms can identify neuroimaging and clinical data patterns which human analysts would typically overlook. The implementation of AI models faces ongoing obstacles because of biased datasets and unclear decision-making processes and insufficient clinical validation. 

This research evaluates multiple ML approaches through a structured dataset to identify neurological disorders across different data formats. Our evaluation of model performance along with predictive feature identification will help us understand the most effective machine learning applications for this essential medical field.

The dataset that is used in the current study enables the early prediction of neurological disorders using several diagnostic and clinical features. Neurological disorders, like Alzheimer’s, Parkinson’s, and other cognitive deficits, exhibit early signs. When these are detected in a timely manner, prognosis improves. This dataset involves eight independent variables, including sex, age, and genetics. Reliable statistics were made possible by the size of 3000 records. The goal is to find patterns which may help in early detection through ML models.

3.1 Data collection and features

The dataset comprises 3,000 patient records with 18 features, including age, gender, genetic markers, EEG/MRI results, cognitive test scores, and diagnoses. Data were collected from anonymized hospital records with appropriate ethical considerations. Table 1 is a summary of the key features as below:

Table 1. Description of the key features included in the dataset for predicting neurological disorders

3.2 Feature encoding

Categorical variables were numerically encoded. Missing numerical values were imputed with the median; categorical with the mode. Features were scaled using standardization. Class distribution was balanced (approx. 51% positive, 49% negative), so no resampling was applied.

We encoded the categorical variables (Gender, Family History, Speech Impairment) to numerical values so that they are compatible with machine learning algorithms. Changing categorical information into numbers is important because machine learning models need numbers. For example, we mapped Gender (Male/Female) to (0/1) to ensure no ordinal encoding bias was introduced. In the same way, Family History denoting the hereditary nature of neurological disorders was encoded as 1 (Yes) and 0 (No). To keep the order intact, classification, which was classified into Low, Medium and High, was mapped to an ordinal numeric scale like 0, 1 and 2 respectively. Furthermore, EEG and MRI abnormalities which are required for diagnosis were conversion of 0 for normal and 1 for abnormal. As a result, the interpretation and model training became easier to identify the patients with and without abnormal. By encoding categorical variables into a numerical format, the relationship between features can be captured. In addition, it was an important step for dimensionality reduction methods, like PCA and feature selection methods like Mutual Information and Recursive Feature Elimination (RFE) to make sure only the most important variables were retained. Also, standardization encoded data plays an important role in zooming into data distribution through heatmaps (Figure 1) and pair plots for checking relationship by number. The encoding process also allowed advanced preprocessing techniques to be used, such as scaling and normalization. These techniques are critical for optimizing the performance of models, especially those that are sensitive to the magnitude of features, such as SVMs and neural networks. Through the addition of feature encoding, a structured dataset could be created that doesn’t introduce bias to the modeling process.

1.png

Figure 1. Heatmap displaying Pearson correlation values among numerical variables, highlighting relationships between features

3.2.1 Data preprocessing

Prior to training machine learning models, preprocessing steps were applied to clean the data. The Pandas' function isnull() was used to check for missing values in the data set. We looked at all numerical features of the dataset to see if there were missing values. Missing values present in any column of the training dataset were replaced by the median of the column. Similarly, for categorical features, missing values were replaced by the value that occurred most frequently. Standardization was utilized for the continuous variables like age, cognitive test score so that all the continuous variables have all similar scales.

Features such as 'Brain Volume' were initially considered but excluded from final model training due to minimal contribution in RFE and PCA-based analyses. This helped reduce dimensionality and improve model generalization.

3.2.2 Exploratory Data Analysis (EDA)

Correlation analysis and visualization were performed to better understand the data set.

a) Correlation analysis

The connection between features was examined by computing the Pearson correlation matrix. The following strong correlations were observed There is a negative correlation between the cognitive test score and speech impairment, which indicates that patients with lower cognitive abilities have higher speech impairments. MRI abnormalities & EEG abnormalities showed a positive correlation, suggesting that if one diagnostic test is abnormal, the other is likely abnormal as well.

b) Distribution of key features

Most of the patients aged between 40 to 70 years with few deviated from age range as mentioned in Figure 2(a). We noticed a few younger and older patients outside of this range. The results of cognitive tests were left skewed meaning that a greater number of patients have lower scores. This means that cognitive decline was common in participants of the study. The analysis showed that there are fewer patients are getting higher cognitive test marks (Figure 2(b)) thus confirming the cognitive losses in the sample.

2a.png

(a)

2b.png

(b)

Figure 2. Histograms showing the distribution of age and cognitive test scores in the dataset

c) Feature importance analysis

We performed Mutual Information (MI) (Figure 3) analysis and RFE (Figure 4) for building a good machine learning model using our data. These techniques help pick features to predict the neurological disorder early. Moreover, we applied PCA for dimensionality reduction to enhance feature selection further. MI measures how dependent a particular feature is on the target variable. It is used to assess the increased probable accuracy of a classification Eq. (1).

$M I(X, Y)=\sum x \in X \sum y \in Y P(x, y) \log \frac{P(x, y)}{P(x) P(y)}$             (1)

where, $P(x, y)$ is the joint probability of feature $X$ and target $Y, P(x), P(y)$ are the marginal probabilities of $X$ and $Y$.

3.png

Figure 3. MI analysis results, illustrating the dependency of features on the target variable

4.png

Figure 4. RFE results, identifying the most important features for model prediction

RFE is an iterative feature elimination technique that identifies the best k features by training a model and removing the least significant feature in each iteration. The importance score for each feature is calculated as Eq. (2):

$W_i=\frac{1}{n} \sum_{j=1}^n B^2 i j$              (2)

where, $W_i$ is the importance of feature $X_i, n$ is the number of iterations, $B i j$ is the coefficient of $X_i$ in iteration $j$.

The study found that cognitive test score, EEG abnormalities and MRI Abnormalities are the most important features. These features were important in differentiating different types of patients. Reducing the dimensionality through PCA revealed that the first five principal components accounted for more than 85 percent variance. This means that some features didn’t add much to the overall variability and could be discarded to simplify the analysis. Taking out features less important features will not affect the prediction of the model (Table 2).

d) Time-series trend analysis

The ratio of age to diagnosis revealed younger patients presented more as compared to patients who were older, who primarily presented degenerative diseases. People are getting genetic disorders at early age and degenerative disease at older age. Cognitive test scores were other neurophysiological features which declined significantly with age. These disorders essentially become clearer and more obvious with age. A line plot of important feature values by age demonstrates these trends well.

Table 2. Summary of observations from correlation analysis, feature importance, PCA, and time-series trends

Table 3. Detailed description of the features used in the study, including their types and relevance

There’s a strong probability of a linear relationship between the MRI and the EEG abnormalities (Table 3). An analysis of feature importance showed that cognitive test scores and neuroimaging abnormalities were key predictors. PCA analysis suggested to reduce dimensions of the model to essential components only. Also, time series analysis showed continuous decline in neurological functions with age.

Exploration of data through feature selection, and a thorough preliminary data analysis was done on the datasets. Important factors for the prediction of neurological disorder were identified as cognitive test score, EEG abnormalities, MRI abnormalities, and their trends with age. The results show we should analyze EEGs and so on to detect future neurologic issues. The analysis will help create models using machine learning.

We applied Various classification models like Random Forest, XGBoost, LightGBM and CatBoost to check their performance on our data. These models were chosen for their effectiveness with structured data and their frequent application in predictive modeling. We optimized the dataset and tuned the hyperparameters for better performance. The Tables 4-11 show the accuracy, confusion matrices, and other evaluation metrics of the results.

5.1 Random Forest

Random Forest achieved an accuracy of 82.18%, demonstrating a balanced trade-off between precision and recall. The model performed well in both classes, as reflected in its confusion matrix (Tables 4-5).

Table 4. Confusion matrix for the Random Forest model's performance

Table 5. Evaluation metrics (Precision, Recall, F1-Score, Support, etc.) for the Random Forest model

5.2 XGBoost

The accuracy of XGBoost is 79.70%. It performed slightly lesser than RF. The confusion matrix indicates there are more wrong predictions than the Random Forest model (Tables 6-7).

Table 6. Confusion matrix for the XGBoost model's performance

Table 7. Evaluation metrics for the XGBoost model

5.3 LightGBM

LightGBM showed moderate performance with an accuracy of 81.19%, improving upon XGBoost but slightly underperforming compared to Random Forest (Tables 8-9).

Table 8. Confusion matrix for the LightGBM model's performance.

Table 9. Evaluation metrics for the LightGBM model

5.4 CatBoost

The CatBoost model achieved the highest accuracy of 83.17%, outperforming other models, showing superior classification performance across both classes (Table 12).

Table 10. Confusion matrix for the CatBoost model's performance

Table 11. Evaluation metrics for the CatBoost model

Table 12. Comparison of specificity, false positive rate, and negative predictive value across all models (Random Forest, XGBoost, LightGBM, CatBoost)

8a.png

8b.png

Figure 8. Bar graphs comparing the performance metrics (Accuracy, Precision, Recall, F1-Score) of all models (Random Forest, XGBoost, LightGBM, CatBoost)

CatBoost is the best method overall with the accuracy of 83.17% and having the highest precision, recall, F1-score, specificity and NPV and lowest FPR. XGBoost achieves lowest accuracy at (79.70%) and does slightly worse on other metrics. All models give a balanced performance for both classes (0 and 1). The precision and recall values are close too, thus indicating no significant class imbalance issues in these results (Figure 8).

5.5 Evaluation metrics and evaluation

We evaluated model performance using multiple classification metrics, including accuracy, precision, recall, F1-score, specificity, negative predictive value (NPV), and the Area Under the Receiver Operating Characteristic Curve (AUC). These metrics provide a balanced assessment of model performance beyond overall accuracy. Comparison of ROC curves for CatBoost, Random Forest, LightGBM, and XGBoost. CatBoost shows the highest AUC (0.89), confirming its leading performance. Curves are based on representative distributions modelled from the evaluation metrics (Figure 9).

9.png

Figure 9. ROC curves for all machine learning models

To visualize discrimination ability, ROC curves were generated for each model. Confidence intervals (95%) for accuracy and AUC were estimated using 1,000-sample bootstrapping to validate the stability of performance metrics. Figure 9 illustrates the ROC curves for all four machine learning models, showing that CatBoost achieved the highest AUC of 0.89.

Compared to earlier studies, our model suite achieved higher accuracy with stronger validation. For instance, CatBoost outperformed models in similar research that lacked categorical optimization. Unlike prior work, we emphasized interpretability and validated results through ROC/AUC, adding reliability to our findings

Our findings suggest that ML models can offer valuable decision-support tools in neurological diagnostics, provided they are supplemented with clinical validation and interpretability. The use of machine learning in neurophysiological and clinical assessment can enhance diagnostic accuracy and scalability. This research highlights the need for multiple tests, including assessments of speech, cognition, MRI, and EEG, to make an accurate diagnosis. The models based on ML have a great scope, however, challenges regarding data quality, interpretability and real-world applicability are all serious issues. By fixing these problems, AI-based diagnostic systems can help doctors do their jobs better by making things more accurate, timely, and useful for patients. It is necessary to research the use of a vast range of data sets and enhanced features selection to improve the efficiency and the accuracy of machine learning models for diagnosis of neurological disorder. By incorporating diverse demographic and clinical profiles, we can enhance the model's generalizability.