Improving Cardiovascular Disease Prognosis Using Outlier Detection and Hyperparameter Optimization of Machine Learning Models

The healthcare sector generates an enormous volume of data pertaining to patients, diseases, and medical evaluations. However, the efficient analysis of this data remains largely untapped, thereby limiting its potential impact on patient health. At the forefront of global mortality causes is heart disease, accounting for an alarming number of fatalities each year. As per the World Health Organisation [1], cardiovascular diseases (CVDs) represent the leading cause of death globally, claiming an estimated 17.9 million lives annually. CVDs encompass a range of conditions such as coronary artery disease, rheumatic heart disease, vascular disease, along with various disorders affecting the heart and blood vessels. Notably, strokes and heart attacks constitute four out of every five CVD deaths [2].

A multitude of risk factors contribute to heart disease, including sex, smoking habits, age, family medical history, poor diet, high cholesterol, physical inactivity, high blood pressure, obesity, and alcohol consumption. Inherited risk factors encompass conditions such as diabetes and high blood pressure [3]. Secondary factors, such as physical inactivity, obesity, and unhealthy diets, further amplify this risk. Typical signs and symptoms include generalized weakness, fatigue, palpitations, excessive sweating, back pain, chest pain, discomfort in the shoulder and arm, and shortness of breath. Chest pain, medically referred to as angina [4], remains the most common symptom of insufficient blood flow to the heart. Diagnostic tests such as X-rays, MRI scans, and angiography are employed to confirm the diagnosis.

However, instances arise where an inadequacy of medical equipment leads to a resource deficit during emergencies. The urgency of diagnosing and treating cardiovascular disease cannot be overstated, as every second is of the essence. Cardiac centers and outpatient clinics generate voluminous data concerning heart disease detection, highlighting a significant potential need for enhanced big data analytics in the context of cardiovascular care and patient outcomes [4, 5].

Yet, the task of deriving precise, accurate, and valid conclusions is often impeded by disturbances, inconsistencies, and variability in the data. With the advent of significant technological advancements in big data, knowledge storage, acquisition, and recovery, artificial intelligence (AI) has become crucial in the field of cardiology [6]. Multiple machine learning (ML) models have been employed by researchers to make decisions, following the pre-processing of data using various data mining techniques [7]. A broad spectrum of ML algorithms and variants are utilized in monitoring hereditary cardiac disorders and control, aiming to forecast the early stages of heart failure [8]. A host of heart attack prediction algorithms, including KNN, DT, SVC, LR, and RF machine algorithms, have been explored [9, 10].

Unsupervised ML: data-driven, unlabelled data (clustering); Supervised ML: task-driven, labelled data (classification/regression); among the three kinds of machine learning approaches is reinforcement learning, which involves learning from failures while engaging in games [11]. In this study, supervised machine learning (ML) classifiers including Logistic Regression (LR), k-Nearest Neighbours (kNN), Support Vector Machine (SVM), and others are used to demonstrate how various models may predict the presence of heart disease and to compare the accuracy of these classifiers. Neural Network (NN), Decision Tree (DT), Random Forest (RF), FCC Means – SVM, FCC Means – NN, KNN Means – SVM and KNN Means – NN. The remainder of the paper is organized as follows: The literature review is found in Section 2. Section 3 discusses the recommended technique. Section 4 discusses the experiment's findings. To summarise, Section 5 contains the conclusions.

Problem Statement: Analysis of various traditional ML algorithms and their ensemble approaches were adopted for detecting cardiac diseases. Moreover, enhancing classification rate from optimization algorithms with hyperparameter-based tuning algorithm and grid search CV method. This paper uses the shorter population of critical medical data for a relatively smaller set of patients as the large dataset is recommended for such analytical studies. In order to address such practical constraints, it envisages exploring to compare the performance of each of the traditional models in regard to its counterpart in an attempt to choose an optimized model. The effect of hyper-tuning of the metric parameters on accuracy improvement shall be the point of interest. Also, possible outliers must be detected and processed to evaluate the prediction accuracy of these models [19]. Analysis of various traditional ML algorithms and their ensemble approaches were adopted for detecting cardiac diseases. Moreover, enhancing the classification rate of optimization algorithms with a hyper-parameter-based tuning algorithm and grid search CV method.

This research study discusses the use of traditional machine learning algorithms and their ensemble approaches for detecting cardiac diseases as an attempt to choose the best possible existing algorithms fit for such critical applications in medical science. The study uses a relatively smaller population dataset from the UCI Machine Learning Repository, which represents a challenge to AI/ML community and considered only fourteen important attributes for the proposed model in consultation with experts. Data cleaning was not availed as it is pre-processed dataset. Feature scaling was done using three scaling datasets, and the best accuracy was obtained from the standard scalar. Data splitting was done to train and test SVM, RF, and NN models [20]. The performance was analyzed for different split ratios, and the 70:30 split ratio was used for further experimentation to validate the data-splitting practice for critical medical applications. Data visualization was done using a histogram, which helped to identify the attributes that have a higher correlation with heart disease through analysis.

The novelty in this paper appears to be the proposed method for detecting cardiac diseases using various traditional machine learning algorithms and their ensemble approaches [21], as well as enhancing the classification rate through optimization algorithms with hyper-parameter-based tuning algorithm and grid search CV method. The authors also conducted experiments on a pre-processed dataset and evaluated the performance of different scaling techniques and data-splitting ratios. The data visualization and analysis of different attributes of the dataset for predicting and validating the proposed model is also a novel aspect of this text.

The objective of this paper is to explore feature selection techniques and it also emphasizes analyzing the behavior of the used model in the presence of an outlier in the smaller dataset which poses a challenge to the ML community. Cardiac disorder being a very critical application to human life, an outlier in the dataset must be included in practice time, and evaluating model performance in such a scenario proves to be useful to physicians.

This work performs a comparative study with the feature selection techniques. One can conclude and see which type of feature selection technique works relatively better if the model is random forest-based or ANN based or SVM based. Although the paper does not touch upon any information about the architecture of the models, this model is accurately used for comparing the feature selection techniques for better performance in the context of smaller datasets as an attempt to come up with a recommendation to the physician for selecting specific physiological attributes. The approach can be run in a pipeline wherein the physician can recommend a few corrective measures in selecting the attribute to see performance analytics. The proposed method is depicted in Figure 1.

1.png

Figure 1. Proposed method block diagram

3.1 Details of dataset

The proposed method has been evaluated and validated on UCI Machine Learning Repository datasets. It consists of 76 attributes. For experimentation purposes, we considered only 14 attributes for the significance of the proposed model. Some attributes give the same knowledge while other attributes are not relevant to the targeted attribute. Thus, redundant attribute data have been discarded from the data. Only relevant and discriminating attributes are taken into consideration. Table 1 provides a detailed description of data attributes used to predict and validate the proposed model.

3.2 Splitting and pre-processing of data

1. Cleaning of data: During the cleaning process, missing or null values are imputed or dropped. However, the dataset used for experimentation is pre-processed; hence, there is no specific requirement for data cleaning.
2. Scaling of features: ML models require numbers, if data is more discriminant, higher values have superiority in the data. Hence, a significant number plays an important role in training the ML model. Thus, feature scaling requires while dealing with different scale attributes. Otherwise, it results in a dilution of another significant attribute [22]. The dataset used is most distinct, hence prudent to consider feature scaling. These are classified as below:

• Standard Scalar (SLRstd): These are generated by subtracting the mean. Afterward, scaling has been done with unit variance. It provides the standardized data with 0 mean and unit standard deviation. The resultant data is normally distributed which can be used for classification.
• Min Max Scalar (SLRmm): It rescales the feature in provided range by preserving the original distribution shape. The individual feature is scaled and translated using a formula.

$\frac{X_{\text {scaled }}=X-X_{\min }}{X_{\max }-X_{\min }}$                    (1)

where X indicates feature, Xmin, Xmax, and Xscaled refer to the minimum, maximum, and scaled feature values respectively. Table 1 lists the details of the attributes pertaining to the heart dataset.

Table 1. Heart dataset attribute information

Data Attributes	Description
Sex	Gender, 0 = Female, 1=Male
Age	Age (years)
Chest Pain (Cp)	Chest pain Types (3: Asymptomatic, 2: Non-angina, 1: Atypical angina, 0: typical angina.)
Chol	Serum cholesterol in mg/d
Tresbps	Blood Pressure calculated at rest position (in mm Hg on admission to the hospital)
Ca	Major vessels (0-3) coloured by fluoroscopy
FBS	Fasting blood sugar >120 mg/dl), (0 = false, 1 = true) 1. Resting electrocardiographic results 2: probable or definite left
Restecg	ventricular hypertrophy by Estes criteria, 1: ST-T wave abnormality, 0: normal)
Exang Thalach	Exercise-induced angina (0 = no, 1 = yes) Maximum heart rate achieved
Oldpeak	ST depression induced by exercise relative to rest
Thal	7 = reversible defect; 6 = fixed defect; 3 = normal
Slope	Peak exercise Slope (2: down sloping, 1: flat, 0: up sloping)
Num	1: Unhealthy, 0: Healthy

• Robust Scalar (SLRr): It has the ability to reduce outliers’ effect, compared to SLRmm. It doesn’t provide the data scaling in predetermined intervals, unlike SLRmm. The ML models are trained using all three scaling datasets (as discussed above) to find the best fit for our dataset. Prediction accuracy was recorded during each case. The classification rate of individual methods has been mentioned in Table 2.

Table 2 indicates the best accuracy obtained from the standard scalar. Hence, the standard scalar is applied for further feature scaling in the experimentation. The Standard Scalar scales the data at zero mean and unit SD by assuming the data is normally distributed within each feature.

3. Data Split for Training and Testing: Training and testing dataset splitting is needed for data processing of the ML models. For e.g., generally, a 50:50 data split is not preferred for training and testing of data [23].

Table 2. Comparison of the three scalers' performance in terms of accuracy (%)

Scalar	SLRstd	SLRmm	SLRr
ACC (%)	86.78	84.90	85.84

For any model, maximum training data makes the system robust. The 50:50 data for training and testing is not a good choice for less data. The in present study, the four different ratios were selected to train, test SVM, RF, and NN, and noted the performance in every case. Table 3, indicates the analysis of training ACC (%) for different split ratios. It was noted that the 70:30 split provide des highest ACC. Hence, the 70:30 split ratio is used for further experimentation [24].

Table 3. Performance of SVM, NN, RF for different split ratios in terms of accuracy (%)

Algorithm	60:40	70:30	75:25	80:20
RF	86.74	92.45	90.31	85.12
NN	84.53	84.91	84.14	82.64
SVM	86.70	86.79	85.02	84.30

3.3 Visualization of data

Data Visualization provides more insights into the evaluation process hence readers engage and interact more closely with new findings. It shows an easier way to communicate the research in the community. Figure 2 shows a graphical representation f the present dataset for good understanding. In the histogram, the plot y-axis represents the count of the respective attributes whereas different attributes of the dataset have shown on the x-axis.

The distribution of heart disease patients with various attributes is presented in Table 1 viz. sex, age, Cp, etc. It’s clearly noted from the plots which attributes are belonging categorical numerical variables. Interpretation from histogram plots is discussed below [24].

• Age: From Figure 2, we can be noted that people around 60 years are more prone to cardiac attacks.
• Sex: Cardiac disorders in women (value 0) are lower compared to men (value 1). Hence, there is more possibility of male cardiac diseases compared to females.
• Cp: There are few people suffering from 0 type chest pain, the majority of these are suffering from type 3.
• Trestbps: People in a resting state BP of 140 are widely prone to cardiac disorders.
•  Chol: There are many people encountered with more cardiac disease who have a Cholesterol level of 250 and few outliers with a 500-cholesterol score.
• FBS: People who have an FBS of more than 120 are denoted with 1 while others have FBS of less than 120. Hence heart disease patients have FBS of more than 120mg/dl.
• Restecg: Many people are suffering from ST - T wave abnormality (value = 1) and very few people indicating definite or probable left ventricular hypertrophy.
• Thalach: Most patients have heart rate more than 150.
• Exang: Majority of people have 0 exang value. Thus, people having heart disease will not have any exercise-generated angina.
• Oldpeak: Those who have Oldpeak value in range 0 to 1 have maximum chances of cardiac disease.
• Slope: People having a 0 value suffer from the least heart diseases whereas slope value is 1 and 2 for the majority of people.
• Ca: People with minor vessels colored by fluoroscopy are more in terms of heart diseases.
• Target: The present dataset has many people with Unhealthy (value = 1).

2.png

3.4 ML models for prediction of cardiac disorders

3.4.1 Support vector machine

It is a linear classifier. The two-class separating hyper-plane is selected to optimize desired classification error for unknown test data. SVM is a binary classification model. SVM classifies the test data point to the class nearest point in the training data. Non-linear and non-separable data is transferred to higher dimension feature space and separated by a hyperplane. The process of mapping to higher-dimension feature space is performed by using different kernels. SVM provides [6] give the best two-class separating hyperplane that maximizes the separating distance.

$y=w x+b$                 (2)

3.4.2 Neural Networks (NN)

It simulates the system which can take decisions like human brain. ANN has ability to learn and realize various study patterns to obtain information. This creates a link between input neuron to output neuron. Neurons are functions of weighing factors. The output can be measured by taking product of specific neuron weight with input. Afterword, it compared with threshold value. The result is greater than given threshold value. It is considered as output. Practically, it is impossible to evaluate how many layers or nodes to be utilized in ANN to address a specific application. It has to evaluate by systematic experimentation to find best for the given data. Thus, after using various values of hidden layers, best suitable results were noted with 100 hidden layers [25].

3.4.3 Random Forest (RF)

RF works on the principle of decision trees at training. Predictions of different trees are pooled to find final prediction. The feature importance of decision tree can be evaluated as,

$F i_i=\frac{\sum_{j=n o d e s}^n n i_i}{\sum_{k=1}^n n i_k}$                        (3)

where Fi_i is feature importance, node j has importance ni_j. The mean of all the trees provides the final feature importance value. It can be calculated by sum of the feature’s important value of each tree, divided by the value of total trees [6].

$R F f i_i=\frac{\sum_{j \in t r e e s}^n n o r m f i_{i j}}{T}$                        (4)

where, RFfi_i denotes feature importance when evaluated from all trees in the RF model, norm fi_ij. Indicates the normalized feature importance for i in the tree.

3.4.4 K-Nearest Neighbors

It is a simple ML algorithm. It works on the principle of majority votes of neighbors to classify the unknown object. When the unknown sample is provided, the cluster area for k training samples, which are close to unknown samples, is searched by KNN algorithm. Euclidean distance between two given points Y = (y₁, y₂. . . y_m) and X = (x₁, x₂ ... x_m) can be given by,

$d(x, y)=\sqrt{\sum_{i=1}^m\left(x_i-y_i\right)^2}$                      (5)

3.4.5 K-Means-based SVM

The prediction analysis of this model has two stages. Initially, the k-mean clustering has employed which clusters hybrid datasets. Finally, the SVM model has been employed that will classify the data.

3.4.6 K-Means based Neural Network

The performance of classification analysis model has two phases. Firstly, k-mean clustering has evaluated on data that results in cluster dissimilar and similar type of data. Afterwards, the NN model has employed to separate data types. Here, the first step will be same as K means based SVM.

3.4.7 Fuzzy C-Means based Neural Network

It is a soft clustering method. It allows partial belongings of pixels in various clusters. So as to addition of all degrees is 1 for any data types. This method is applicable to segmentation applications compared to hard clustering algorithms. This model evaluates ‘c’ clusters by optimizing the objective function defined as,

$J_{F C M}=\sum_{k=1}^n \sum_{i=1}^c\left(u_a\right)^q d^2\left(x_k, v_i\right)$                   （6）

where, data points are denoted by x_k = x₁, x₂ ... x_n, the data items are indicated by n, c provides clusters count, the membership degree of x_k in the i^th cluster is denoted as u_ik, weighting exponent of membership is shown by q, v_i is cluster center, d²(x_k, v_i) The distance in cluster center v and data point x_k is denoted by d. In the proposed hybrid model, the output of Fuzzy C-means has been provided as input to the NN for best classification. Maximum accuracy is obtained from the fuzzy c-partitioned matrix. It is noted that the accuracy is improved by implementing this Hybrid method compared to only Neural Network.

3.4.8 Fuzzy C-Means based SVM

This algorithm is implemented as per the following steps:

To begin, the Fuzzy C model is applied to the provided dataset. Following that, a Fuzzy c-partitioned matrix is evaluated.

As input to the SVM method for classification, a combination of Fuzzy partition matrix and feature set is already available.

It has been discovered that when the Hybrid approach is used, classification accuracy improves when compared to using solely SVM.

The current study's goal is to diagnose heart disease using various ML algorithms and compare them to hybrid models. The hybrid models like the Fuzzy C-means clustering-based neural network and K Means Neural network performed better for training classification accuracy. K Means Neural Network obtained a prediction accuracy of 91.98%, which is enhanced in Fuzzy C-Means to Neural Network with 95.05% without hyperparameter tuning. The classification accuracy increased to 98.78% after a systematic search of hyperparameters with GridsearchCV. for the FCM-based neural network. Performances of the combination of clustering techniques with Neural networks and SVM and statistical classifiers have been compared. It was reported that the hybrid models performed better compared to the classical model. Afterward, the training testing accuracy by removing outliers and thereafter interpolating the rows have been implemented to evaluate the impact of outlier removal. It is discovered that the proposed strategy outperforms the dataset with anomalies. Hence, outlier removal can be recommended for the accurate detection of cardiac disorders. As per the suggestions of cardio, thoracic surgeon, and cardiologist, still there is more scope to work on physiological data like thyroid values, cardiac risk elements, parameters belonging to the clinical background of the family etc. with real-time application of more than 5000 patients. This large dataset can be optimized to have zero classification rate of prediction of cardiac disorders. Hyperparameter tuning is indeed an important aspect of machine learning model development, especially when working with large and diverse datasets. The performance of a model depends not only on the chosen algorithm but also on the values assigned to the hyperparameters, which are the settings that control the learning process. Hyperparameter tuning involves systematically searching for the optimal combination of hyperparameter values that results in the best model performance. This can be done using various techniques such as grid search, random search, Bayesian optimization, or genetic algorithms.