Automatic Detection of COVID-19 Disease by Using Transfer Learning of Light Weight Deep Learning Model

Automatic Detection of COVID-19 Disease by Using Transfer Learning of Light Weight Deep Learning Model

Fatih Özyurt

Department of Software Engineering, Firat University, Elazig 23119, Turkey

Corresponding Author Email: 
fatihozyurt@firat.edu.tr
Page: 
147-153
|
DOI: 
https://doi.org/10.18280/ts.380115
Received: 
2 July 2020
|
Revised: 
2 October 2020
|
Accepted: 
16 October 2020
|
Available online: 
28 February 2021
| Citation

© 2021 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).

OPEN ACCESS

Abstract: 

COVID-19 began to appear in China in December 2019 with cases of pneumonia. Controlling the effects of this virus type is closely related to all scientists, especially medical doctors. In this study, ShuffleNet CNN architecture, which is suitable for mobile devices, is one of the deep learning models that perform today's most successful image recognition and classification processes. Obtaining a large data set in the training process is an important problem for the diagnosis of COVID-19 disease. However, rapid diagnosis of COVID-19 X-ray or CT images plays a crucial role in the immediate decision of medical doctors. However, the lack of training data causes poor performance due to the problem of over-fitting. Also, the back propagation algorithm used in CNN training is often very slow and requires the tuning of different hyper parameters. To overcome these disadvantages, this study proposed a new approach based entirely on the machine learning algorithms, extracting CNN features from the light weight ShuffleNet architecture. In this study, a total of 300 images were used, the images diagnosed as COVID-19 and normal chest images. During the classification phase, 10-fold cross validation was applied to the database and 99.98% accuracy rate was obtained.

Keywords: 

COVID-19, deep learning, Shufflenet, transfer learning, feature reduction

1. Introduction

The new Coronavirus (COVID-19) is a virus first identified on January 13, 2020. It was found as a result of research conducted in a group of patients who developed respiratory tract symptoms (fever, cough, shortness of breath) in late December in Wuhan Province, China [1]. The outbreak was initially detected in the animal market in this region. Then it spread from person to person and spread to other cities in Hubei province, especially Wuhan, and other provinces of China and other world countries. Coronaviruses are a large family of viruses that can cause disease in animals or humans. In humans, several coronaviruses are known to cause respiratory infections, from the common cold to more severe diseases such as the Middle East Respiratory Syndrome (MERS) and Severe Acute Respiratory Syndrome (SARS) [2-4].

Coronaviruses are, in fact, a large group of viruses of different types, although it comes with the recent outbreak. It carries RNA as genetic material, and is the group of viruses with the largest genome among viruses carrying RNA. Coronavirus is an enveloped virus and has spike-shaped protrusions around its envelope. This structure gives it an image of a royal crown when viewed under an electron microscope. For this reason, the virus was named Corona, which means the royal crown in Latin [4-8].

It is determined that hundreds of thousands of people worldwide have COVID-19 disease. In addition, tens of thousands died of this disease [9]. It is believed that mortal cases will increase as the rate of spread of the COVID-19 virus increases exponentially. Therefore, rapid diagnosis of COVID-19 cases is vital. The list of countries most affected by COVID-19 disease is provided in reference [9].

Nowadays, it is very important to use systems that can diagnose from biomedical images. Deep learning has become an area used by academic and industry environment. Industrial applications of various mobile sensing methods ranging from robotics, health, biometric, autonomous driving and defense are made with deep learning architectures. Advances in deep learning are often tied to the availability of powerful computing resources and a large amount of trained data. A high performance deep learning application requires a machine learning model, trained with large amounts of data. This is usually achieved using high-performance Central processing units (CPU), Graphics processing units (GPU), Tensor Processing Units (TPU) or combinations of these [10-15]. Various highly trained Convolutional Neural Network (CNN) architectures suitable for mobile devices are recommended because the high computational costs are very low on mobile devices. When CNN architectures are examined, the highest accuracy architectures usually have tens of layers, so billions of bits require computation [16-18]. This situation requires a very high calculation cost. The use of lightweight CNN architectures is crucial as mobile computing capacity is very low.

ShuffleNet is a CNN architecture with a very high computing efficiency, specially designed for mobile devices with very limited computing power (for example, 10-150 MFLOP). This new architecture was proposed by Zhang [19] and his team to greatly reduce the cost of computing and at the same time maintain accuracy. It is proposed when state-of-the-art basic architectures such as ResNeXt [18] and Xception [20] are found to be less efficient in very small networks due to the highly dense 1 × 1 convolutions. Pointwise group convolutions were used to reduce the calculation complexity of 1 × 1 convolutions.

In this article, lightweight CNN based transfer learning method is proposed for rapid diagnosis of COVID-19 diseases from chest images. The main contributions of the proposed method can be listed as follows.

  1. Lightweight ShuffleNet architecture is used with Transfer Learning Method.
  2. The proposed method is designed for use in lightweight devices with very limited computing power to increase portability.
  3. Due to the small amount of data, overfitting problem is avoided by using classical machine learning classifiers.
  4. 99.98% performance was achieved in COVID-19 detection in chest images.

The rest of this article is organized as follows: Section 2 introduces ShuffleNet architecture. Section 3 describes the proposed method in detail. Section 4 presents the experimental results of the proposed method. Section 5 includes discussion part and section 6 concludes the study.

1.1 ShuffleNet architecture

Classical convolutional neural networks [11, 12, 16, 17, 21] usually consist of repeated building blocks of the same structure. Among them, ResNeXt [19] and Xception [20] networks offer efficient depth-split convolutions or group convolutions to building blocks to provide a balance between representational ability and computational cost. Group convolution significantly reduces the cost of computing, ensuring that each convolution only works in the respective input channel group. However, if multiple group evolutions are stacked together, there is a side effect: outputs from a particular channel are obtained only from a small part of the input channels. Figure 1 (a) shows the state of the two stacked group convolution layers. It is clear that the outputs from a certain group are related only to the inputs within the group. This feature blocks the flow of information between channel groups and weakens the display. Figure 1 shows the channel of ShuffleNet architecture.

The fact that group convolution obtained input data from different groups is as shown in Figure 1 (b). The input and output channels will be completely interrelated. Specifically, for the feature map created from the previous group layer, the channels in each group are first divided into several subgroups, then each group in the next layer is fed with different subgroups. This can be applied efficiently by channel shuffling. detailed information can be found in the reference [19].

Figure 1. (a) Two stacked convolution layers with the same number of groups. (No cross talk); (b) When GConv2 receives data from different groups after GConv1, the input and output channels are completely related; (c) An application equivalent to using channel mixing in b [19]

2. Proposed Method

Medical images are usually available in X-ray or Computer Tomography (CT) format. The processing of these images is usually divided into three stages: (i) data collection, data segmentation and denoising, (ii) feature extraction / size reduction, and (iii) classification. In the first stage, a data set with Chest X-ray or CT images was obtained for the detection of COVID-19 case [22]. Normal chest data were obtained from a different source [23]. In the second stage, the lightweight CNN architecture, ShuffleNet architecture, was used to take advantage of the automatic feature extractor of deep learning architectures. 1,000 features were extracted from “node_202” layer, which is the last layer of this architecture.

Then feature reduction has been applied in the extracted feature set to eliminate unnecessary information and obtain distinctive features. In the last stage, the performance of the reduced features was observed using several classification methods. The high number of features does not always affect the classification performance positively. In order to establish the COVID-19 diagnosis from the chest images, it is necessary to determine the best feature extraction, size reduction and classification method.

Therefore, the main purpose of this study is to use lightweight CNN feature extraction method with appropriate feature reduction and classification technique for high accuracy COVID-19 diagnosis. The developed detection of COVID-19 disease by using transfer learning of light weight deep learning model framework includes feature extraction using layers of ShuffleNet architecture, feature selection with ReliefF and neighborhood component analysis (NCA) and classification. The 100 most distinctive features are selected using ReliefF and NCA based feature selection methods. Machine learning based classifiers are used to demonstrate the performance of the developed method. Graphical presentation of the proposed method is given in Figure 2.

Figure 2. Graphical presentation of the proposed method

The pseudo code of the proposed method is given in Algorithm 1.

Algorithm 1. Pseudo code of the proposed method

Input: Im is image with size of w x h.

Output: Obtained Results (OR)

1: Load Chest Dataset (COVID-19, Normal)

2: Resize Image(Im) 224 x 224

3: features = ShuffleNet (Im); Features obtainef from node_202 layer of the ShuffleNet

4: Define Target value

5: Concatenate features with their Target value

   Final Data = features | Target

6: Select the most 100 distinctive features with ReliefF and NCA

7: Use machine learning algorithms with 10-fold cross validation

8: Calculate Results using OR

9: Interpret these results.

2.1 Feature extraction via ShuffleNet architecture

Unfortunately, CNN architectures running on high performance computers cannot be used in devices that do not have the capacity to perform intensive operations such as drones, mobile phones and tablets. Today, high-performance computers are used for analysis after collecting data through different sensors or cameras to use it in devices with low processing capacity. However, the bottlenecks during the sending of the data communication mean additional costs.

High computational resource requirement for image processing may cause deep learning algorithms not to be used in platforms with low processing capacity. To solve these problems, the development of CNNs suitable for built-in real-time object detection is important to reduce model parameters and speed up their calculations. For this purpose, light versions of CNN architectures are used in limited hardware. In this study, Lightweight ShuffleNet CNN architecture was used to use in devices with low computing power such as mobile phones, drones and tablets. 1000 features were obtained from node_202 layer of ShuffleNet architecture. The demonstration of the proposed feature extraction and selection method is given in Figure 3.

Figure 3. Demonstration of the proposed feature extraction and selection method

2.2 Feature selection

During the feature extraction phase, 1,000 features were extracted in the "node_202" layer of Lightweight ShuffleNet. Two-layer feature selection method has been applied for the 100 most distinctive features. ReliefF and NCA algorithms were used to select these features, respectively. The weights produced by ReliefF show how much the related feature represents the relevant class [24]. Relief is a kind of feature selection method to remove unnecessary features from the data set. ReliefF aims to reveal the correlations and consistencies in the characteristics of the data set. In addition, it determines the important features in the data set by considering the closes to the samples in the same class and the distance to different classes. NCA produces only positive weights. First, ReliefF is applied to 1,000 features and produces 1,000 weights. Negative weights produced by ReliefF algorithm are eliminated and given to NCA algorithm. NCA is a distance-based feature weighting method. Positive weights are created for each feature. Each positive weight expresses how distinctive the attribute is. In other words, the feature with the highest weight can be called the most distinctive feature. Therefore, the 100 most distinctive features are selected and given to the classifiers. The steps of the feature selection algorithms used are given below.

Step-1: Using ReliefF algorithm to generate weight of features by using Eq. (1):

$w^{f R}=$ ReliefF $($ features, target $)$      (1)

where, $w^{f R}$ is weights of the ReliefF with size of 1,000.

Step-2: Eliminate negative weighted features by using Algorithm 2.

Algorithm 2. Negative weighted features elimination using ReliefF

Input: Feature (X) with size of 1000, weights of the features $\left(w^{f R}\right)$ using ReliefF.

Output: Positive weighted features (pf) with size of k.

1: pos = 1;

2: for i=1 to 1000 do

3:      if $w^{f R}(i)>0$ then

4:           $p f(p o s)=X(i)$;

5:           $p o s=p o s+1$;

6:      end if

7: end for i

Step-3: Using NCA algorithm to generate weight of features by using Eq. (2)

$w^{f N C A}=N C A(p f,$ target $)$     (2)

where, $w^{f N C A}$ is weights of the NCA.

Step-4: Select the most 100 discriminative features of the pf by using NCA weights.

This selection process is given in Algorithm 3.

Algorithm 3. NCA based feature selection method

Input: ReliefF features (pf).

Output: Final feature (feat) with length of 100.

1: [weights, indices] $=\operatorname{sort}(p f,$ desc $)$; // Sort pf by descending.

2: for i=1 to 100 do

3:      $f \operatorname{eat}(i)=p f($ indices $(i))$;

4: end for i

The selected final features are used as an input to the used machine learning classifiers.

2.3 Transfer learning with pre-trained ShuffleNet

ShuffleNet [19] is an efficient light weight CNN architecture for portable devices with low computing capacity. It is known to provide better performance than MobileNet [25]. ShuffleNet architecture uses two new processes to reduce computing cost while maintaining accuracy. These processes are pointwise group convolution and channel shuffle. Channel shuffle allows split the channels in each group into several subgroups, then feed each group in the next layer with different subgroups. ShuffleNet can be 13 times faster on an ARM-based mobile device than an AlexNet architecture, while maintaining comparable accuracy [19]. The example of Transfer learning with ShuffleNet architecture is given in Figure 4.

Figure 4. Transfer learning with ShuffleNet architecture

Normally a separate learning is performed for each task in machine learning. However, it is possible and advantageous to use some information learned from some tasks in other tasks. In this case, the information obtained from the source tasks is used for the solution of the target task. The features, weights etc. obtained from previously trained models with transfer learning are used for a new task. In this study, pretrained ShuffleNet architecture was used up to the node_202 layer. Then, feature reduction was applied to the features obtained from the node_202 layer and given to different classifiers.

3. Result and Discussion

3.1 Dataset

The datasets used in the study were collected from 2 different sources on the internet. COVID-19 images were individually collected from a total of 144 chest images with different diseases [22]. However, only 118 of these images were diagnosed with COVID-19. To increase the performance in the proposed study, coronal images (also known as the frontal plane) were selected in CT images and Poster anterior (PA) images were selected in X-ray images. In general, a total of 87 COVID-19 images, 3 from the CT coronal plane and 84 from the x-ray PA plane, were obtained. On the other hand, normal chest images were obtained from different open sources on the internet [23]. Normally, there are 1,583 normal chest images in this database. Due to the low COVID-19 data set, only 150 normal images were used. To balance the dataset equally 87 COVID-19 chest images have been increased to 150 images by applying flipping and rotating augmentation techniques. Sample display of the images used in this study is given in Figure 5.

Figure 5. Sample images from the database

3.2 Experimental result

In the proposed method, a comprehensive classification performance study was carried out to diagnose COVID-19. In the proposed method obtained 100 features were evaluated in Naïve Byes (NB) [26], Linear Discriminant Analyzer (LDA) [27], Support Vector Machine (SVM) [28] and K-Nearest Neighbor (KNN) [29] classifiers. Each study is classified by performing 10-fold cross validation.

Metrics such as accuracy, precision, geometric mean and F score were used to evaluate the proposed method. True positives (tp), true negatives (tn), false positives (fp) and false negatives (fn) are used to calculate these performance evaluation metrics [30-32]. Mathematical explanations of accuracy, precision, geometric mean and F-score are given as Eqns. (3)-(6).

Equations are as follows.

accuracy $=\frac{t p+t n}{t p+t n+f p+f n}$     (3)

geometric mean $=\sqrt{\frac{t p * t n}{(t p+f n) *(t n+f p)}}$      (4)

precision $=\frac{t p}{t p+f p}$     (5)

$F 1$ score $=\frac{2 * t p}{2 * t p+f p+f n}$      (6)

As shown in Table 1, Accuracy, Geometric Mean, Precision and F1-Score values of four different classifiers were calculated. Each classifier was run 100 times to demonstrate the consistency of the proposed method. The mean values of metrics such as Accuracy, Geometric Mean, Precision and F1-Score along with the standard deviation values are also shown in the table. The lowest accuracy rate was obtained in the LDA classifier with 98.67%. It is clear that other classifier values are greater than 99%. In addition, the highest value was obtained with the SVM classifier at the rate of 99.98%.

Figures 6 and 7 are boxplot graphics of the extracted features. In statistical science, the box chart is a descriptive statistic and statistical graph instrument, as a descriptive data analysis tool under the name box-and-whiskers graph to visualize quantitative data [33]. The median (median), Xmin (smallest observation value), Q1 (first quartile), and Q3 (third quartile) and Xmax (largest observation value) values of the features are shown on the chart. It offers a visual design to see in which range the data is distributed. It is clearly seen that the chest images diagnosed as COVID-19 in Figure 6 remain between -5 and +5, while the chest images in Figure 7 are between -4 and +4. These figures show that the extracted features of both classes represent the related classes very well.

Table 1. Experimental results of proposed method

Classifier

Accuracy

Geometric Mean

Precision

F1 Score

NB

99.39±0.0018

99.39±0.0018

99.40±0.0018

99.39±0.0018

LDA

98.67±0.0049

98.67±0.0049

98.67±0.0049

98.67±0.0049

SVM

99.98±0.0001

99.98±0.0001

99.98±0.0001

99.98±0.0001

KNN

99.64±0.0001

99.64±0.0001

99.64±0.0001

99.64±0.0001

Figure 6. Boxplot analysis of COVID-19 disease chest images features

Figure 7. Boxplot analysis of normal chest images features

4. Discussion

In this study, a lightweight deep learning based method that can detect COVID-19 disease is proposed. Collecting chest images diagnosed with COVID-19 in terms of correct plane for image processing is one of the biggest problems. Therefore, datasets were collected from 2 different sources from the internet [22, 23]. Images that do not have the correct position were eliminated. For this reason, coronal images in CT format and PA images in x-ray format were collected. 4 traditional classifiers have been used to comprehensively evaluate the proposed method. According to the results, the best classifier is SVM because the highest classification metrics (accuracy, precision, geometric mean, F-score) among the classifiers used were achieved using SVM. These results are given in Table 1. To prove the success of the extracted features, the scatter plot of the first 2 features is given in the Figure 8.

Figure 8. Scatter plot of the first 2 features

The results listed in Table 1 are the mean results of these methods. As seen in Table 1, the proposed method has achieved classification accuracy between 98.67% and 99.98%. Considering that these performance rates are obtained by running 100 times, the consistency of the proposed method can be clearly seen.

5. Conclusion

It is seen that COVID-19 disease is spreading in the world. The rapid diagnosis of this disease has a historical significance for humanity. The aim of this study is to propose a fast, high accuracy diagnostic method for the detection of COVID-19 disease. Therefore, a method with portability, which does not require powerful graphics cards and processors for deep learning, was proposed by obtaining the strong features of CNN architecture.

In this study, an automatic detection of COVID-19 disease by using lightweight deep learning method was developed. The ShuffleNet CNN architecture used in the proposed method is an architecture with low computational complexity but high recognition rate. Features from the node_202 layer of ShuffleNet architecture were obtained using the transfer learning method. Then, 2-layer feature selection method was applied to the obtained features and traditional classifiers were used. Features were reduced to 100 using ReliefF and NCA. It is understood from the scatter plot and boxplot analyzes that these features have strong representational capability. In addition, the classification accuracy, precision, geometric mean, and F1 score rates obtained from the classifiers show that it is possible to recognize COVID-19 disease from the chest images of the method. Although the data set is small, achieving very high performance rates with the 10-fold cross validation method clearly demonstrates the success of the proposed method.

In future studies, a more comprehensive chest disease recognition framework can be proposed by collecting chest images of different classes. In addition, a mobile based application can be developed using the proposed method in this study. Doctors can get an idea of whether the chest image will have COVID-19 disease by using this application.

  References

[1] Tuncer, T., Dogan, S., Ozyurt, F. (2020). An automated residual exemplar local binary pattern and iterative relieff based COVID-19 detection method using chest X-ray image. Chemometrics Intelligent Laboratory Systems, 203: 104054. https://doi.org/10.1016/j.chemolab.2020.104054

[2] Sohrabi, C., Alsafi, Z., O'Neill, N., Khan, M., Kerwan, A., Al-Jabir, A., Iosifidis, C., Agha, R. (2020). World Health Organization declares global emergency: A review of the 2019 novel coronavirus (COVID-19). International Journal of Surgery, 76: 71-76. https://doi.org/10.1016/j.ijsu.2020.02.034

[3] Bai, L., Yang, D., Wang, X., Tong, L., Zhu, X., Zhong, N., Bai, C., Powell, C.A., Chen, R., Zhou, J., Song, Y. (2020). Chinese experts’ consensus on the Internet of Things-aided diagnosis and treatment of coronavirus disease 2019 (COVID-19). Clinical eHealth, 3: 7-15. https://doi.org/10.1016/j.ceh.2020.03.001

[4] Shen, M.Z., Zhou, Y., Ye, J.W., AL-maskri, A.A.A., Kang, Y., Zeng, S., Cai, S. (2020). Recent advances and perspectives of nucleic acid detection for coronavirus. Journal of Pharmaceutical Analysia, 10(2): 97-101. https://doi.org/10.1016/j.jpha.2020.02.010

[5] Rodriguez-morales, A.J., Cardona-Ospina, J.A., Gutiérrez-Ocampo, E. (2020). Clinical, laboratory and imaging features of COVID-19 : A systematic review and meta-analysis. Travel Medicine and Infectious Disease, 34: 101623. https://doi.org/10.1016/j.tmaid.2020.101623

[6] Wilder-Smith, A., Chiew, C.J., Lee, V.J. (2020). Can we contain the COVID-19 outbreak with the same measures as for SARS? Lancet. Infect. Dis., 20(5): E102-E107. https://doi.org/10.1016/S1473-3099(20)30129-8

[7] Li, X.W., Geng, M.M., Peng, Y.Z., Meng, L.S., Lu, S.M. (2020). Molecular immune pathogenesis and diagnosis of COVID-19. J. Pharm. Anal., 10(2): 102-108. https://doi.org/10.1016/j.jpha.2020.03.001

[8] Chang, L., Yan, Y., Wang, L. (2020). Coronavirus Disease 2019: Coronaviruses and blood safety. Transfusion Medicine Reviews, 34(2) 75-80. https://doi.org/10.1016/j.tmrv.2020.02.003

[9] Worldometer (2020). Coronavirus Cases. [Online]. Available: https://www.worldometers.info/coronavirus/, accessed on 23-Mar-2020.

[10] Krizhevsky, A., Sutskever, I., Hinton, G.E. (2017). ImageNet classification with deep convolutional neural networks. Commun. ACM, 60(6): 84-90. https://doi.org/10.1145/3065386

[11] He, K.M., Zhang, X.Y., Ren, S.Q., Sun, J. (2016). Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 770-778. https://doi.org/10.1109/CVPR.2016.90

[12] Szegedy, C., Liu, W., Jia, Y.Q., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A. (2015). Going deeper with convolutions. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Boston, MA, pp. 1-9. https://doi.org/10.1109/CVPR.2015.7298594

[13] Girshick, R., Donahue, J., Darrell, T., Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Columbus, OH, pp. 580-587. https://doi.org/10.1109/CVPR.2014.81

[14] Ren, S.Q., He, K.M., Girshick, R., Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell., 39(6): 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031

[15] Shelhamer, E., Long, J., Darrell, T. (2017). Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 39(4): 640-651. https://doi.org/10.1109/TPAMI.2016.2572683

[16] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z. (2016). Rethinking the inception architecture for computer vision. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Las Vegas, NV, pp. 2818-2826. https://doi.org/10.1109/CVPR.2016.308

[17] Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A. (2017). Inception-v4, inception-ResNet and the impact of residual connections on learning. 31st AAAI Conf. Artif. Intell. AAAI 2017, pp. 4278-4284.

[18] Xie, S.N., Girshick, R., Dollár, P., Tu, Z.W., He, K.M. (2017). Aggregated residual transformations for deep neural networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 5987-5995. https://doi.org/10.1109/CVPR.2017.634

[19] Zhang, X.Y., Zhou, X.Y., Lin, M.X., Sun, J. (2018). ShuffleNet: An extremely efficient convolutional neural network for mobile devices. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Salt Lake City, UT, pp. 6848-6856. https://doi.org/10.1109/CVPR.2018.00716

[20] Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 1800-1807. https://doi.org/10.1109/CVPR.2017.195

[21] Simonyan, K., Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., pp. 1-14.

[22] ieee8023/covid-chestxray-dataset: We are building an open database of COVID-19 cases with chest X-ray or CT images. [Online]. Available: https://github.com/ieee8023/covid-chestxray-dataset?fbclid=IwAR0FW3uGU0YQuPyDJ-HX2WIJnNuxCuoxU9j9-CbfYqdPtEjKo4hEfi2rkmg, accessed on 19-Mar-2020.

[23] ‘Chest X-Ray Images (Pneumonia) | Kaggle’. [Online]. Available: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia, accessed on 19-Mar-2020.

[24] Tuncer, T., Dogan, S., Subasi, A. (2020). Surface EMG signal classification using ternary pattern and discrete wavelet transform based feature extraction for hand movement recognition. Biomed. Signal Process. Control, 58: 101872. https://doi.org/10.1016/j.bspc.2020.101872

[25] Abdelmalek, B., Ahmed, K., Amine, T.M. (2019). A survey on lightweight CNN-based object detection algorithms for platforms with limited computational resources. International Journal of Informatics and Applied Mathematics, 2(2): 28-44.

[26] Rish, I. (2001). An empirical study of the naïve Bayes classifier. IBM Research Division, pp. 41-46.

[27] Tharwat, A. (2016). Linear vs. quadratic discriminant analysis classifier: A tutorial. Int. J. Appl. Pattern Recognit., 3(2): 145. https://doi.org/10.1504/IJAPR.2016.079050

[28] Bennett, K.P., Bredensteiner, E.J. (2000). Duality and geometry in SVM classifiers. ICML, pp. 57-64.

[29] Bilgin, T.T., Erdogan, S.Z. (2012). A data mining approach for fall detection by using k-nearest neighbour algorithm on wireless sensor network data. IET Commun., 6(18): 3281-3287. https://doi.org/10.1049/iet-com.2011.0228

[30] Özyurt, F., Tuncer, T., Avci, E., Koç, M., Serhatlio, I. (2018). A novel liver image classification method using perceptual hash-based convolutional neural network. Arab J Sci Eng, 44: 3173-3182. https://doi.org/10.1007/s13369-018-3454-1

[31] Tuncer, T., Aydemir, E., Dogan, S. (2020). Automated ambient recognition method based on dynamic center mirror local binary pattern: DCMLBP. Appl. Acoust., 161: 107165 https://doi.org/10.1016/j.apacoust.2019.107165.

[32] Özyurt, F. (2020). Efficient deep feature selection for remote sensing image recognition with fused deep learning architectures. The Journal of Supercomputing, 76: 8413-8431. https://doi.org/10.1007/s11227-019-03106-y

[33] Potter, K. (2006). Methods for presenting statistical information: Visualization of Large and Unstructured Data, GI-Edition Lecture Notes in Informatics (LNI), S-4: 97-106.