Cloud-Based Parkinson’s Disease Diagnosis Using Machine Learning

Elderly people, with the age of 50 or more, may be affected seriously by neurodegenerative disorders, resulted from Parkinson’s Disease (PD) [1]. This disease affects the nervous system and leads to finally loss of control of the muscles. Several studies have revealed that the genetic factor is considered one of the main factors causing PD [2]. Tribulation dementia is one of the symptoms of PD compared to healthy people. Though, the patient could benefit from the scheduled treatment. Several symptoms occur with PD including trembling in arms, hands, head, jaw, legs, limbs' stiffness, slowness in movement, and impaired balance. Additionally, approximately 90% of PD patients suffer from speech difficulties represented by dysphonia that impaired speech production and dysarthria which means difficulties in speech articulation. Consequently, these known facts can be utilized for PD diagnosis [3]. Although there are several research studies have been conducted to diagnose PD, still there is a need for a more accurate detection system that achieves the characteristics of usability and the availability of the test in the form of a mobile application.

In this work, a cloud-based system is proposed to diagnose PD in the earlier stage. This can be achieved by using different artificial intelligence algorithms of machine learning, for instance, feature selection and classification methods over Parkinson’s dataset. Feature selection methods will be used to minimize the feature space for the input data while considering the classification accuracy to be maximum. Then trying to validate different methods for generating train and test data for applied classifiers such as LSTM and RF as well as calculating the accuracy and specificity of each classifier to choose the proper method to diagnose Parkinson’s disease with maximal success rate. Cloud computing technology is a model that allows access to an unlimited number of computing resources under favorable conditions and optionally anytime, anywhere.

The benefits of cloud computing can be summarized as follows [4]:

• Cost saving: cloud technology; eliminates investment expenses such as server racks and IT professionals working to manage infrastructure, which is a significant advantage in terms of costs.
• Speed: with the provision of many services of cloud computing, which can be provided easily without big efforts, this reduces the pressure of businesses regards capacity planning.
• Global-scale: services can be provided by cloud technology anytime, anywhere in a scalable manner.
• Productivity: in fact, data centers require a large number of “stacking and racking” operations. These are processes that include hardware tuning, patching software, which can be considered as time-consuming for the Information Technology (IT) management tasks. Cloud technology eliminates the requirement for a large number of tasks. This enables IT, teams, to utilize time for other business goals.
• Performance: clients should deal with cloud computing services due to many reasons. One of these reasons is that the data centers include the latest versions of hardware, which are updated regularly.
• Reliability: cloud technology facilitates the process of data backup and saving data in redundant locations.

The contribution of this paper is to develop a cloud-based PD diagnosis intelligent system with a mobile application using Machine Learning (ML) LSTM algorithms.

The rest of the paper is arranged as follows: in section 2 the prior research studies that have been published within the last decade are reviewed. Section 3 presents the proposed system of Parkinson's disease diagnosis. Section 4 includes the design of the cloud-based PD diagnosis system for PD diagnosis. The results of the performance evaluation are presented in section 5. Finally, section 6 illustrates the conclusion of the paper.

The main components of the paper are described in this section, including features selection algorithm and machine learning approaches for classification along with the details of cloud computing requirements.

3.1 Optimized features selection based on mRMR

Feature selection represents a search problem to find the optimal (m) subset features among (M) features. In other words, redundant and irrelevant features are excluded in the process of feature selection. This reduces both the complexity and computing time of the system, which improves the recognition accuracy [9, 10].

In this work, the Minimum-Redundancy-Maximum-Relevance selection (mRMR) feature selection method is used [11]. mRMR is employed to select the relevant features, where the aim is to find m subset of features and separate the target labels based on their common information. The details of mRMR feature selection method are described as follows.

For discrete random variables (x and y) the mutual information estimation between x and y is squared off using their individual probabilities P (x), P (y), and their joint probability P (x, y) as shown in Eq. (1) [12].

$\mathrm{I}(\mathrm{X}, \mathrm{Y})=\sum_{x \in X} \sum_{y \in Y} p(x, y) \log \left(\frac{p(x, y)}{p(x) p(y)}\right)$        (1)

where, x denotes a characteristic from the set of carefully chosen features X, and y represents the class label of the prototypes Y set.

In the case of having two features, if any of these features, which are highly dependent on each other, is removed, no big change in the discriminative power for this feature. The aim of mRMR is to find the minimum set of features (S) that contains the (n) attributes {x_i}, which have the largest dependency on the target class y and minimum redundancy between them. The algorithm of mRMR feature selection is demonstrated in Algorithm 1, which is implemented using Python.

Eq. (2) illustrates the maximization of the dependency between the feature variable xi with the class label y, while using Eq. (3) shows the minimization of the dependency for $\left(x_{i} \text { and } x_{j}\right)$ features. The constraint represents a filter for the mutually exclusive features.

$\operatorname{MaxRv}\left(\boldsymbol{x}_{i}, y\right)=\frac{1}{\|m\|} \sum_{x_{i} \in S} I\left(\boldsymbol{x}_{i}, y\right)$        (2)

$\operatorname{Min} R d\left(\boldsymbol{x}_{i}, \boldsymbol{x}_{j}\right)=\frac{1}{\|m\|^{2}} \sum_{\boldsymbol{x}_{i}, \boldsymbol{x}_{j} \in S} I\left(\boldsymbol{x}_{i}, \boldsymbol{x}_{j}\right)$        (3)

Finally, the mRMR algorithm chooses the subset Xm, where (m < n) from the set of main features Xn, which maximized Eq. (4), and uses a process of an incremental search for finding the possible optimal feature set S as shown in Eq. (5).

$\max \varphi(R v, R d), \varphi=R v-R$        (4)

$\max _{\boldsymbol{x}_{j} \in X-S_{m-1}}\left[I\left(\boldsymbol{x}_{i}, y\right)-\frac{1}{m-1} \sum_{\boldsymbol{x}_{i} \in S_{m-1}} I\left(\boldsymbol{x}_{i}, \boldsymbol{x}_{j}\right)\right]$        (5)

In this work, the mRMR method is applied to select the minimum data features as possible while keeping the high accuracy after train the classifier with selected features. This can be achieved by removing the features that are irrelevant to the target classes which can lead to confuse the classification model decision and impact the accuracy. Farther more removing the redundant features leads to reduce the model complexity.

Algorithm 1: mRMR feature selection [10]

Input: Dataset $D(X, Y)$.  

Output: Reduced Dataset S (Xm, Ym).

1. $\mathrm{S} \leftarrow \emptyset$  // Initialization
2. Add Xi= argmax $j \in \varphi$ D(Xj,Y) to S
3. For t=1: k-1 do
4. Add feature satisfies Eq. 5 to S
5. End for
6. Return S
7. End

3.2 Machine learning approaches for classification

The classifier represents a machine learning tool that receipts data and places them in one of the k classes. The intended goal is to learn the classifier how to classify the test sample at high accuracy with a minimum number of samples and features [13]. In this work, two machine learning approaches have been used, which are explained as follows:

3.2.1 Long Short-Term Memory Approach

The Long Short-Term Memory (LSTM) is a type of artificial recurrent neural network approach, which can be utilized in deep learning [14]. It can keep the data for a long period. LSTM can be applied in classifying the predicting and processing of data. Figure 1 illustrates the general structure of LSTM. The typical structure of LSTM contains four layers with one input layer, two hidden layers, and one output layer. LSTM also involves different memory blocks called cells. The data is retained in the cells and the manipulations of memory are achieved in what so-called gates. The structure of LSTM consists of three kinds of gates which are the input gate, forget gate, and output gate. The forget gate is used to elects the reserved or removed information in the cell. The number of states at the previous moment has been retained to the current moment is obtained by the Foregate gate [15]. The input gate is in control of the cell state via adding the useful information, which is finally extracted from the current cell state to be presented as an output or (the input to the next cell) is achieved by the output gate [14]. The general process of LSTM is described as follows.

1) Foregate gate: The first step in LSTM is to use the Foregate gate to decide whether to keep or remove the data from the cell. The output of the Foregate gate $f_{t}$ is shown in Eq. (6).

$f_{t}=\sigma\left(W_{f}\left[h_{t-1}, X_{t}\right]+b_{f}\right)$        (6)

where, $\sigma$  is the sigmoid function, $W_{f} \text { and } h_{t-1}$ represent the weight matrices of Foregate gate,   $b_{f}$ is the bias and $X_{t}$ is the input features.

2) Input Gate: The second step is used to save the new data in the cell state by using the output of the input gate which is calculated in Eq. (7), and then updating the cell memory by the candidate values as illustrated in Eq. (8).

$i_{t}=\sigma\left(W_{i}\left[h_{t-1}, X_{t}\right]+b_{i}\right)$        (7)

$C=\tanh \left(W_{c}\left\lceil h_{t-1}, X_{t}\right]+b_{c}\right)$        (8)

where, $W_{i} \text { and } W_{c}$ are wights of the input layer, $b_{i} \text { and } b_{c}$ are the bias terms for the input gate and the tanh represents the activation function.

3) Cell State: At the current moment, the cell state is calculated by the product of the forget gate and the previous moment state to be added with the product of the input gate parts as shown in Eq. (9).

$C_{t}=f_{t} * C_{t-1}+i_{t} * C$        (9)

where, $i_{t}$  is the output of input gate, C is the cell state, $C_{t-1}$  is the previous cell state and $C_{t}$  represents the current cell state.

4) Output Gate: The final step is completed by determining the decision in the output gate shown in Eq. (12) via passing the initial output that is calculated earlier from Eq. (10) and through the sigmoid activation function illustrated in Eq. (11).

$O_{t}=\sigma\left(W_{o}\left[h_{t-1}, X_{t}\right]+b_{o}\right)$        (10)

$h_{t}=O_{t} * \tanh \left(C_{t}\right)$        (11)

$\text { Output class }=\sigma\left(h_{t} * W_{\text {outparameter }}\right)$        (12)

where, $W_{o}, W_{\text {outparameter }}$ are the weights of output gate; $b_{o}$ is the bias; $\text { Output }_{\text {class }}$ is the classification output, and $\sigma$ is the sigmoid activation function.

1.png

Figure 1. The structure of the LSTM network [13]

3.2.2 Random Forests Approach

Random Forest (RF) is a type of data mining algorithms that is mainly used for supervised learning by randomly generates a forest. Then the “forest” that has been established represents a group of decision trees that are mostly trained using the “bagging” method, where the bagging method is simply representing a collection of learning models for increasing the accuracy of all results in the classification phase [16].

RF consists of multiple binary decision trees as shown in Figure 2. It is used for many purposes, including regression, classification, and different tasks via constructing numbers of decision trees at the time of training. The main characteristics of RF are:

• RF can handle naturally (multiclass) classification and regression.
• It is fast in training and prediction.
• RF can depend on one or two parameters in tuning.
• It has a built-in estimate of generalization error.
• It can be applied to high-dimensional problems.
• Parallel implantation of RF is applicable.
• It can provide measurement for the variable importance.
• Supporting differential class weighting.
• RF is unsupervised learning and can provide a visualization with outlier detection.

The operation process of the RF is explained briefly as follows:

Assume having the real value input of predictor variables X = $\left(X_{1} \ldots X_{p}\right)^{T}$ with p values. The first step is to calculate the loss function shown by Eq. (13).

$E_{X Y}=(L(Y, f(X)))$        (13)

where, $L(Y, f(X))$ is the loss function for calculating the prediction function $f(X)$ and Y is the actual value. The typical loss function can be represented by root mean square error. However, the entropy information gain of the Random Forest classifier is used as a loss function in normal circumstances [17]. The loss function L(Y, f (X)) is used for measuring the closeness between the prediction and the actual value as shown in Eq. (14).

$L(Y, f(X))=I(Y \neq f(X))=\left\{\begin{array}{l}

0 \text { if } Y=f(X) \\

1 \text { otherwise }

\end{array}\right.$        (14)

For the classification problem, the possible values of Y is represented by $\mathbb{R}$ , minimizing EXY (L(Y, f (X))) for zero-one loss gives Bayes rule shown in Eq. (15).

$f(X)=\operatorname{argmax}_{y \in \mathbb{R}} P(Y=y \mid X=x)$        (15)

Ensemble predictor f which is a collection of base learners represented by hJ(x) calculates the predicted final class by voting as shown in Eq. (16).

$f(X)=\operatorname{argmax}_{y \in \mathbb{R}} \sum_{j=1}^{J} I\left(y=h_{j}(x)\right)$      (16)

where, Y is the random variable that represents the target, $f(X)$ is the prediction function for predicting Y, $E_{X Y}$  is expected value of the loss, $\mathbb{R}$ is the set of possible values of Y and $h_{j}(x)$ the jth base learner;

2.png

Figure 2. Random forests [14]

3.2.3 Google Application Engine

Google App Engine (GAE) is a part of the services provided by the Google cloud platform. In 2008, Google was declared the App Engine platform to develop and host different web applications in Google data centers, which represents the first cloud computing service from the company. The service was added to the Google platform with multiple cloud services, making it available publicly in November 2011. Google App Engine is essentially an entire file system (not just a locked folder), where the users have more power than a Standard engine, for example, it has read/write permission. GAE is considered a platform as a service with serverless computing environments. The users in the GAE standard can have a fixed array of libraries already set up and the user cannot distribute 3rd party libraries. In a flexible environment, the user can load the library your application requires, including custom build environments (such as Python 3) [18].

3.2.4 Representative State Transfer

Representative State Transfer (REST) is mainly used as a web service design architecture. At present, the most common types of service communication can be divided into two as REST and Simple Object Access Protocol (SOAP). REST is a more preferred design architecture than SOAP since it provides an advantage over internet usage thanks to its (due to its) less bandwidth consumption. SOAP is a client/server logic protocol that uses the Remote Procedure Call (RPC) model for service calls. In simplified form, when the user creates a web service with SOAP, the service becomes an XML document and then it can call the methods using the definitions in this XML document. The exchange between server and client is always over XML when using SOAP, which means high bandwidth usage. The key difference between REST and SOAP is that the SOAP operates on a standard-based which is XML exchange, whereas the REST does not have such a standard. Sends and retrieves messages in REST, uses XML or JavaScript Object Notation (JSON) format. Messages are forwarded to the server as HTTP requests. Companies that offer cloud-based Application Programming Interface (API) such as Google, Amazon, Twitter, and Microsoft prefer to use the REST architecture due to its simplicity and lightweight structure. When a web service uses REST architecture, it is called RESTful API [19].

This section is divided into two main parts, in the first part, the focus is on the training of the classification models using the data set illustrated earlier in Section 4.1 and a comparison between the results of machine learning (RF) and deep learning (LSTM) is also achieved. In the second part, the best method (LSTM) is compared with the other methods in the related works. The training parameters used in this work for each method are described in Table 2.

Table 2. Models training parameters

Table 3. Confusion matrix

In the first part, in order to choose the best classification method for the proposed PD diagnosis system, we carried out a comparison between two classification methods, which are a regular machine learning method called RF and a deep learning method called LSTM. To evaluate and compare the performance of the trained classification methods, two different evaluation metrics are considered including F-score and ROCs [22] to measure the accuracy of the classification models, which can be defined according to the confusion matrix illustrated in Table 3. The classification method with the best performance then will be used in the proposed Parkinson's disease diagnostic system.

• The accuracy of the classification is: (TP + TN) / (TP + TN + FP + FN)
• F-score: is the proportion of actual negative, which are predicted negative: TN / (TN + FP)
• ROC Receiver Operating Characteristics curve is a 2-D curve parameterized with one parameter of the classification algorithm, such as some threshold in the (true positive rate / false positive rate) space, where the Area Under Curve (AUC) is between 0 and 1.

The procedure of the conducted experiment can be summarized as follows: Selecting N best features from our generated PD dataset using the mRMR feature selection method where N represents several desired features. For each experiment, the percentage number of desired features for the mRMR method is used as (5%, 10%, 25%, 50%, 75%, 100%). The dataset with selected best N features is used to train each one of the proposed classification methods. After the training process is completed, the results of classification models have tested as well as the evaluation metrics F-score and ROCs are calculated. The evaluation strategy is presented by dividing the dataset into 70% for training samples and 30% for testing samples, which is utilized to evaluate the accuracy of the trained model. Table 4 shows the calculated F-score for each one of the classification methods based on the percentage of desired features selected by the mRMR feature selection method.

Table 4. The F-scores for each classification method based on the percentage of selected features using the mRMR method

For better understanding, the obtained result in Table 3 is visualized as shown in Figure 4 below.

4.png

Figure 4. F-scores for each classifier versus the percentage of the selected features

The results in Figure 4 demonstrate that the best performance of the two classification methods is obtained using only 25% of the features selected by the feature selection method. This means that mRMR reduces the number of total features by 75%, which represents selecting only 27 features out of a total of 110 features.

Therefore, the second experiment will involve comparing the two-classification methods based on ROC using the best 25% of the features obtained by the mRMR method as shown in Figure 4. We trained each classifier using the best 25% features selected by the mRMR method and calculate the ROC for each one, as shown in Figure 5.

5.png

Figure 5. ROC receiver operating characteristic curve results for LTSM and RF classifiers model with the best 27 features selected mRMR

Based on the results of ROC curves shown in Figure 5, it is clearly seen that the LTSM classifier has superior performance compared with the RF classifier since the AUC of LSTM is 0.91, which is greater than the AUC of the RF (0.83). Therefore, the LSTM classifier will be used for building the classification model using the PD dataset with 27 features selected by the mRMR method in designing the cloud-based Parkinson’s disease diagnosing system.

Table 5. A comparison between the results of proposed deep learning PD diagnosis model and the other related works

The LSTM model is built and trained using Python and saved on Google cloud Application Engine to be used in the RESTful API and the mobile application, which represents the implementation of the second stage of this work. Figure 6 shows the results of testing the mobile application for diagnosing PD.

In the second part, the results of the proposed deep learning model (LSTM) are compared with related deep learning methods from the literature that have been applied to PD detection based on voice recordings data. Table 5 shows a brief comparison with the prior research studies in teams of the achieved accuracy. Based on the comparison of Table 5, it can be noticed that the proposed PD diagnosis system outweighs a number of existing studies in terms of detection accuracy.

6.png

Figure 6. Mobile application testing screens