A Hybrid CNN–LSTM Framework for Diabetes Risk Prediction in COVID-19 Patients

The COVID-19 pandemic is a consequence of the novel coronavirus, SARS-CoV-2, which has a detrimental effect on communities and health systems worldwide. The virus first appeared at the end of 2019 and swiftly expanded over the world, killing a significant number of people and inflicting severe illness [1]. The patients who are exposed to risk of infection are those who are chronically sick like diabetes or a metabolic condition where they have high levels of blood glucose. Diabetes comes with impaired immunity, which exposes vulnerable persons to infectious diseases [2]. Since diabetes is a disorder that impacts about 422 million individuals worldwide, determining the interplay between COVID-19 and this prevalent disorder is extremely important.

Additional comorbidities, including obesity and cardiovascular diseases, as well as inadequate immune functions and chronic inflammation, can also increase this level of risk [3]. As such, predictive models are required to determine high-risk individuals and determine the risk of diabetes among COVID-19 patients to help verify that adverse outcomes are avoided.

Several approaches have been proposed to improve diabetic patient care during the COVID-19 epidemic. Diabetes risk classification and prediction using ML and DL techniques have shown a great deal of promise for improving patient outcomes. The most typical forms of risk assessment are based on the ML algorithms, i.e., on Logistic Regression (LR), Decision Trees (DT), and Support Vector Machines (SVM) that process clinical data to reveal the key predictors [4, 5]. By contrast, DL models, specifically CNNs and RNNs are more appropriate in working with complex data, such as medical images and patient records in sequence [6, 7]. These computational techniques show high potential to assist in managing diabetes risks among COVID-19 patients and allow them to be provided with a personalized treatment approach.

This study evaluates the risk of diabetes in COVID-19 patients using a predictive modeling technique. A wide range of ML algorithms are used to compute on a large data set of demographics, clinical characteristics, diagnostic outcomes, and comorbidities. The primary objective is to develop a useful model that may identify individuals who are more likely to experience unfavorable consequences. By identifying the factors linked to poor COVID-19 results in diabetics, this article should help a healthcare professional make a therapeutic choice and manage a patient.

Additionally, the Fine-Tuned Hybrid CNN-LSTM model, a novel hybrid deep learning model that combines the benefits of two deep learning models, will be presented in this work as a diabetes risk prediction. The recommended framework will be made up of four key steps namely, preprocessing data and features, model fusion, and classification. This paper's primary conclusions can be summed up as follows:

• To utilize the data balancing procedures and feature selection with the help of the Random Forest (RF) to enhance the quality of the dataset so that the model was effective at detecting the relevant features and could avoid biases in predictions.
• • The traditional ML and DL models are evaluated on the common classification tasks in terms of such metrics as accuracy, precision, recall, and F1-score on publicly available datasets.
• A hybrid CNN-LSTM model that combines CNN and LSTM was recently proposed with the goal of increasing the accuracy of diabetes risk prediction in COVID-19 patients. By incorporating CNNs to learn geographical information and LSTMs to learn sequential patterns, this system enhances the quality of submission prediction.
• The second step is to adjust the hybrid CNN-LSTM model's hyper-parameters so that it can identify the different factors that contribute to diabetes risk in COVID-19 patients and get the highest level of prediction power. This tuning process is more equipped against noise and maximizes generalization capabilities of the model when extended even to clinical practices.

The paper's outline is as follows:  A brief synopsis of related works from the past is given in Section 2. Section 3 outlined the planned deployment of the proposed fine-tuned hybrid CNN-LSTM model, and Section 4 included a detailed description of the suggested method of forecasting patients' risk of developing diabetes. The experimental design and findings are described in depth in Section 4, the work is summarized and possible future directions are suggested in Section 5, and the conclusion is covered in Section 6.

The methodology of diabetes risk prediction in COVID-19 patients begins with loading a publicly available dataset that includes the clinical features of a patient like age, sex, glucose level, status of diabetes, and so on. Imputation of missing values, normalization of numerical characteristics, and encoding of categorical variables are the earliest steps in preprocessing. As a first step in data preparation, we fill in missing values, standardize numerical characteristics, and use label encoding to encode categorical variables.

The outliers are either transformed or removed in order to minimize bias. In order to handle class imbalance, SMOTE is utilized to balance diabetic and non-diabetic classes. Feature selection is done using RF-based methods, which have the most influential features to reduce dimensionality in such a way that model efficiency is improved. The suggested hybrid DL model takes the best features of both LSTM and CNN networks and uses them to generate a spatial and sequential pattern in the input data. This hybrid architecture allows the model to potentially uncover complex relationships and time-dependent interactions among patient characteristics.

Methods like Grid Search are used for hyper-parameter tuning, which optimizes model architecture, batch sizes, and learning rates. Finally, a number of performance indicators show that the proposed model is helpful in predicting the risk of diabetes in COVID-19 patients. The proposed system architecture is shown in Figure 1, and the following section delves into the details of each stage.

Figure 1. Proposed system architecture

3.1 Dataset description

The Mexican government made available on Kaggle the dataset that was used in this investigation [33]. It has anonymized-clinical and demographic data of COVID-19 tested persons. The data consists of 1,048,576 records about patients characterized by 21 variables, including symptoms, comorbidity, treatment, and outcomes. The binary variables are coded (1 Yes and 2 No) and 97 and 99 represent missing or unavailable information, Table 1 shows the dataset feature and its description.

Table 1. Dataset feature and description

Figure 2. Dataset sample

Figure 3. Label counts (without balancing)

The first thing that needs to be done is to load the dataset that contains information about diabetes, which has normally been saved in CSV format. Among the many characteristics that may be included in this dataset are the patient's age, glucose levels, body mass index, blood pressure, and outcomes There is a possibility that the dataset includes information about COVID-19 infection, which is essential for the investigation being conducted. Reading the CSV file into a data structure, like a pandas Dataframe in Python, simplifies data manipulation and analysis during the dataset loading process. Figure 2 illustrates the dataset sample while Figure 3 illustrates the diabetes data count with 1 indicted diabetic patients, 2 is non diabetic and 98 related to others or having not identified.

3.2 Dataset pre-processing

Important pre-processing operations are performed to prepare the raw data for analysis. At this point, you should deal with missing values in one of two ways: either fill them in using measurement statistics like the mean or median, or remove rows and columns with too much missing data. It is necessary to identify any outliers that have the potential to bias the study and then either convert them or delete them. Data normalization or standardization is followed to make sure that the model does not get disproportionately influenced by characteristics having different scales. This may require encoding categorical data into numerical values, and feature scaling is done so that all characteristics fall into a scale comparable across one another. Dataset pre-processing comprises the following steps:

• Drop Rows with Label “98”.
• Handle missing values, and noise in the dataset.
• Normalize or scale numerical features.
• Min-Max Scalar.
• Encode categorical variables and labels.

3.2.1 Drop rows with label “98”

This is generally done when the label "98" in data preprocessing indicates an invalid or irrelevant category. In such cases, the removal of such rows helps the model not learn a pattern from erroneous or extraneous data points, which otherwise may impair the model's performance and predictions.

3.2.2 Handle missing values, and noise

High-quality data is very important for building reliable models; having missing values or outliers, or noisy data points will lead to biased results. When a value is missing, an imputation method that takes the feature's mean, median, or mode into account fills in the blanks. To solve the issue of missing values, mean values are utilized in this case. Likewise, noise in data-smoothening unexpected or meaningless variations-is achieved by filtering techniques. Removing these discrepancies makes the dataset more accurate reflection of real-world situations and reduces the chances of overfitting.

3.2.3 Normalize or scale numerical features

In cases when variables are on different scales or range, then standardization or scaling of numerical features must be done. This is because features with higher values may dominate others at the time of model training. One common technique for normalization is called Min-Max Scaling, in which features are transformed to fall within a scale in between 0 and 1. To accomplish this, we use the formula:

$X_{\text {scaled }}=\frac{X-X_{\min }}{X_{\max }-X_{\min }}$

where, X_min and X_max are the feature's minimum and maximum values, respectively, and X is the initial value. Learning algorithms are made more stable and converge faster with Min-Max scaling, which makes sure that all numerical features are on the same scale.

3.2.4 Encode categorical variables and labels

Since ML models can only take numerical input, it is necessary to convert categorical variables to numbers.  Encoding describes this procedure. For ordinal variables, which have an inherit order, we proceed with Ordinal Encoding - simply replacing each level of a categorical variable by a unique integer. For nominal (non-ordered) categories we should not rely on different weights and in such situations, it is appropriate to use one-hot encoding, where binary columns are created for each category to let the model know that it shouldn't assume any order of categories. Labels in the target variable are also encoded using Label Encoding if they represent discrete categories.

The correct interpretation of the categorical data by ML algorithms is dependent on this conversion, which in turn allows the model to successfully detect patterns and relationships. The Pre-processed dataset samples after encoding categorical variable is shown in Figure 4.

Figure 4. Pre-process dataset samples

3.3 Applying feature selection techniques

Identification and selection of the most pertinent features from the dataset that make a significant contribution to the prediction of the diabetic risk factor are considered to be part of the feature selection process. This is accomplished by the utilization RF feature importance model. The dimensionality of the dataset reduced through the use of feature selection, which also helps eliminate redundant or irrelevant data. As a result, the model's performance can be enhanced and calculation times can be reduced. Figure 5 shows the features ranking (graph) from high to low. We have selected features whose threshold is greater than 0.02 as shown in Table 2.

Figure 5. Feature importance score

Table 2. Selected features list

3.4 Applying data sampling technique

SMOTE is one of the data sampling approaches that is utilized here as the dataset is imbalanced (for example, there are a significantly greater number of patients who do not have diabetes than diabetic patients) (as shown in Figure 3). By taking this step, we ensure that the model does not develop a bias towards the class that constitutes the majority. It is possible to achieve a balanced dataset with the help of proper sampling, which ultimately results in more reliable model predictions, we used SMOTE for balancing the samples, the balance sample label count is shown in Figure 6.

Figure 6. Label counts (with smote balancing)

3.4.1 Train – test split of dataset

Data cleansing and feature selection are followed by the creation of a training set and a testing set from the dataset. Two common ratios for the sets are 80:20 and 70:30. We opted on a 70:30 split for our studies. By dividing the data in this way, we may access the model on unobserved data and have a good idea of how it will perform in the actual world. To eliminate any possibility of bias and to guarantee that both sets are representative of the entire dataset, the split must to be carried out in a random fashion.

3.4.2 Machine learning models for classification 

ML models [33-35] play a critical role in predicting diabetes risk among COVID-19 patients by leveraging clinical and demographic data. These models examine patient data for trends that suggest an elevated risk of diabetes, especially in the presence of COVID-19. Based on the labeled data used to determine the risk class—which includes blood glucose readings, age, BMI, comorbidities, inflammatory markers—the algorithms can now forecast and categorize individual patients into diabetic and non-diabetic groups. Various categorization methods allow these models to distinguish between high-risk and low-risk patients.  The goal is to generate early warnings in order to implement targeted interventions and personalized therapies. Here the diabetic risk factor is predicted using a number of different ML models, including the following:

Decision Tree:

Decision trees organize datasets hierarchically by routinely dividing them into subsets according to feature values that maximize class separation.  Each division is identified using criteria such as Gini Impurity or Information Gain. A node's Gini Impurity is determined as follows:

$\operatorname{Gin} i=1-\sum_{i=1}^C P_i^2$

In this case, the graph shows the correlation between the two variables, where pi is the likelihood that class I will be present at the node and C is the total count of classes at the node. The tree is constructed until a stopping requirement, like the minimum number of samples per node or the maximum depth, is satisfied. Deep trees are useful, but they can overfit, capturing noise and decreasing generalization.

Random Forest:

In order to generate numerous decision trees, RF is a combination method that makes use of bootstrapped samples and randomly selected subsets of features for each split. For classification tasks, the model uses majority voting, and for regression tasks, it uses averaging to aggregate the results from individual trees. The final output Y for classification is determined as:

$Y=\operatorname{Mode}\left(T_1, T_2, \ldots T_n\right)$

where, $T_n$ is the prediction from the n th decision tree in the ensemble. This reduces overfitting by averaging out errors across trees, making the model more robust and accurate.

Logistic Regression:

Logistic regression (LR) is a statistical method that models the probability of a binary outcome using the logistic (sigmoid) function. The logistic function is used to do this. Given the input features X and coefficients γ, the following expression can be used to express the probability P of the positive class:

$P(Y=1 \mid X)=\frac{1}{1+e^{-\left(\beta_0+\beta_1 X_1+\cdots+\beta_n X_n\right)}}$

where, the feature coefficients are denoted by β_i and the intercept by β₀. A threshold (often 0.5) is applied to determine the anticipated class. For linearly separable data, the model is effective and interpretable since it employs Maximum Likelihood Estimation (MLE) to determine the optimal parameters.

AdaBoost (Adaptive Boosting):

The iterative ensemble AdaBoost method can be used to produce a powerful classifier. In this method, many weak learners are combined including shallow decision trees to achieve the classifier. The model assigns weights to samples which have been misclassified during each iteration and this means the successive learner will focus on the difficult to classify cases. The prediction given by the final model is the weighted sum of the output of every weak learner:

$F(x)=\operatorname{sign}\left(\sum_{m=1}^M a_m h_m(x)\right)$

This is where M is the total number of iterations, $a_m$ is the weight provided to the weak learner, and h_m (x) is the m^th weak classifier. The effectiveness of increasingly precise classifiers in reaching the ultimate decision boundary is shown by computing the weights α_m according to the accuracy of each learner.

3.5 Deep learning models for classification

A subfield of machine learning known as "deep learning," these algorithms automatically learn intricate data patterns by means of artificial neural networks. These models are built to be able to classify incoming data into preset categories based on features learned in order to achieve classification tasks. Since it can model non-linear and detail-rich relationships and high-dimensional data, deep learning would best fit the management of complicated datasets. Deep learning excels in this area. These types of models are used in a subfield of machine learning known as "DL," and they automatically discover complicated patterns in data by means of artificial neural networks. 

3.5.1 Convolutional neural network

CNNs [36, 37] differ from conventional neural networks in that they employ convolutional layers as filters on the input data to identify hierarchical traits and local patterns. CNNs prove to be very useful in image and sequence classification processes since they can follow spatial data and features of images and find intricate patterns. These are filters that slide over the input, and they perform a mathematical operation popularly referred to as convolution, which assists in identifying features of a picture, including edges and tendencies of sequential data. The adaptation for use on structured or sequential data is done by applying a convolution operation along a single dimension. This makes it fit for such tasks as time-series analysis or even classification of genomic sequences. Utilizing a network of interconnected processes, CNN automatically extracts characteristics from unprocessed input data for application in result regression or object classification. Included in CNNs are pooling layers, completely linked or thick layers, and convolutional layers. To define the output feature y for 1D Convolution:

$y(t)= \sum_{i=0}^{k-1} x(t+i) \cdot w(i)+b$

where:

x(t): Input sequence at position t

w(i): Kernel weight at position i

k: Size of the kernel

b: Bias term

y(t): Output at position t

This method is repeated over the whole input sequence, yielding a new feature map that highlights helpful patterns in the data. To lower the dimensionality, pooling is typically used after convolutional layers, and fully connected layers are used for final classification. Figure 7 shows the CNN architecture used to classify patients and forecast their risk of developing diabetes.

Figure 7. CNN architecture

3.5.2 Long short-term memory

RNNs come in the form of LSTMs [38], which are specifically made to efficiently detect temporal dependencies in a sequence. These were created to solve the issue of disappearing gradients that conventional RNNs have. Input, forget, and output gates are essential components of the LSTM cell architecture, which allows the cell to efficiently control the flow of information. LSTMs are optimal for language modeling, voice recognition, and time series prediction due to their architecture and, among other things, their long-term memory capacity. LSTMs have the ability to learn and reflect intricate dependencies between sequential pattern pieces, unlike ordinary RNNs that struggle to replicate such dependencies due to their limited memory. Figure 8 shows the LSTM structure.

Figure 8. LSTM architecture

3.5.3 Proposed hybrid CNN+ LSTM model

Incorporating the greatest aspects of both convolutional and recurrent neural networks, the hybrid CNN-LSTM model provides a powerful tool for assessing sequences across applications. These two powerful deep learning approaches allow our model to be particularly well-suited for dealing with time-series signals, text sequences and physiological data. The CNN part is a strong feature extractor that implicitly learns and stores spatial patterns of raw input sequences. The LSTM layers, which are good at capturing long-term patterns in sequential information and sequence dependence, are then fed with these extracted characteristics. The result is a flexible model that can perform both complex feature extraction and temporal sequence modeling. Here, A 1D-CNN is used to capture local patterns and feature interactions across clinical attributes. The extracted features are then passed to an LSTM layer, which models dependencies within the learned feature representations, the details of model is provided in Model Architecture Overview section.

3.6 Model architecture overview

3.6.1 1D convolutional neural network layer for feature extraction

The final set of features selected through feature importance analysis is used as the input to the 1D-CNN layer. Typically, one or more 1D convolutional layers are incorporated in the first layers of this model to preprocess data from the input sequences. These are essentially filters that slide over the input sequence and create a convolution responsible for recognizing local patterns and identifying high-level features. Typically, each convolutional layer is succeeded by an activation function (e.g., ReLU) to introduce non-linearity, followed by a pooling layer or subsampling technique (MaxPooling) to reduce spatial resolution. This architecture must be considered what CNN should play as the primary role, in that it just compresses the raw data into a feature set with more structured format by extracting some important patterns such as spikes, trends and anomalies.

3.6.2 Flattening layer

A flattening layer was employed subsequent to the 1D convolutional layers to diminish the dimensionality of the feature maps from 2D to 1D. By serving as an important intermediary between the architecture's convolutional and recurrent parts, this transformation prepares spatially-derived features in a pacy way for upcoming LSTM layers to consider temporally. The 'flatten' operation keeps anything we have learned thus far, but restructures the previously learnt features into a sequence that RNN layers can analyses - this in effect enables the model to process spatial feature extraction and temporal pattern recognition together.

3.6.3 LSTM layer for temporal dependencies

The flattened feature vector is subsequently input into the LSTM layer(s) to acquire temporal correlations among the sequence data. LSTMs regulate information dissemination through memory cells and gates (input, forget, and output gates). The network is able to remember meaningful patterns over time while ignoring less significant ones because of this. For instance, in diabetes risk prediction, the LSTM could capture real-world similitudes and deviations of health metrics in a span of several days or weeks, discovering patterns that are linked to worsening (or improving) conditions. Input, forget, and output gates, along with memory cells, govern the dissemination of data in LSTMs. This enables the network to acquire and preserve significant patterns while eliminating less critical ones.

3.6.4 Fully connected (dense) layer

The final output of the LSTM layer encodes the learned temporal patterns, and for classification, one or more dense layers are employed to interpret this output. The LSTM model decodes high order sequence information that these deep layers integrate and translate into class predictions. Each dense layer learns complex, non-linear data patterns by weighting its inputs and activates these weights (most often using ReLU for the hidden layers). The hybrid architecture's classification pipeline is completed by the last dense layer, which generates probabilistic outputs for target classes using an activation function such as softmax or sigmoid.

3.6.5 Output layer

The sigmoid function is employed for binary classification, whereas the softmax activation function is employed for multi-class classification. These two functions are utilized by the output layer. A network's outputs can be normalized into probability distributions across many classes using the softmax function, and for binary outcomes, independent probabilities can be generated using the sigmoid function. This probabilistic output enables precise class predictions, such as stratifying COVID-19 patients into high, medium, or low diabetes risk categories. The complete architecture of our hybrid CNN-LSTM model, illustrating these components and their interactions, is presented in Figure 9.

Figure 9. Proposed hybrid CNN-LSTM architecture

3.6.6 Hyper parameter tuning and model optimization for diabetic risk prediction

After Applying Proposed CNN-LSTM model, to enhance the performance of model hyper-parameters tuning and (L2 regularization) optimization of parameters is performed. The process of hyper parameter tuning involves optimizing the parameters that regulate the training process of a model.

By preventing the use of excessively high weight values, L2 regularization helps to decrease model overfitting. The loss function is then amended to include a penalty term that grows in square root of the model weights' size. Objective Function with L2 Regularization:

$J(\theta)=L(\theta)+\lambda \sum_{j=1}^n \theta_j^2$

where,

$L(\theta)$ = original loss (e.g., cross-entropy, MSE)

$\theta_j$ = model parameters

$\lambda$  = regularization strength (hyperparameter)

$\sum \theta_j^2$ = L2 norm (squared)

Table 3. Hyper parameter description

The hyper-parameters utilized by the suggested hybrid CNN-LSTM model are displayed in Table 3. Now that the models have been trained and optimized, they can be used to predict the diabetic risk factor using fresh patient data. With the help of this prediction, it is possible to determine who is more likely to get diabetes, which paves the way for early intervention and individualized treatment strategies. Healthcare practitioners should be able to comprehend and take action based on the outcomes of the prediction, which will provide them with insights into the elements that are driving the risk.

3.7 Performance analysis

A variety of indicators, including Accuracy, Precision, Recall, F1-Score, Accuracy and Loss Graph, and Confusion Matrix, are used to evaluate the models' performance in the final stage.