VirLesDetNet: Pre-Trained Hybrid Deep Learning Approaches for Virus-Based Skin Lesion Detection

1.1 Background

The monkeypox virus, part of the Orthopoxviridae family, is a highly infectious DNA virus with a double-stranded structure [1-3]. It exhibits clinical symptoms similar to those of chickenpox and measles, and is categorized under the Poxviridae family and Orthopoxvirus genus [1, 4]. Identified initially in a monkey at a research facility in 1959, it was subsequently named monkeypox [5, 6]. The virus’s natural hosts include animals such as squirrels, Gambian opossums, and primates [1, 3], while transmission frequently occurs through rats and monkeys, with human-to-human spread also reported [7, 8].

The first recorded human infection of monkeypox occurred in 1970 in the Democratic Republic of Congo, coinciding with the global smallpox eradication campaign [9]. The virus spreads through direct physical contact, respiratory droplets, bites from infected animals, or exposure to contaminated surfaces [10, 11]. Monkeypox has an incubation period of 5 to 21 days, with illness severity influenced by the individual's immune response and viral load [1]. Early symptoms, generally appearing within the first 1-5 days, include fever, swollen lymph nodes, muscle pain, fatigue, and headaches. A rash usually follows, appearing on areas such as the face, hands, and feet, and progresses through five stages: macules, papules, vesicles, pustules, and finally, crusted lesions, over a period of 2-4 weeks [12]. Rising transmission rates of monkeypox have led to increased concern. While only about 50 cases were reported in Western and Central Africa in 1990 [13], by January 2023, cases had surged to 65,353 across 100 countries, spanning the Americas, Europe, Africa, the Eastern Mediterranean, and the Western Pacific regions [4, 8, 14, 15]. The World Health Organization (WHO) has classified monkeypox as a moderate health risk, underscoring the need for swift global response [1]. Despite considerable efforts, the Centers for Disease Control and Prevention (CDC) states that no fully effective treatment currently exists for monkeypox [12]. However, two drugs, Brincidofovir and Tecovirimat, which the CDC initially approved for chickenpox, are also available for treating monkeypox [16]. Though a vaccine is considered the ultimate solution for prevention, a reliable vaccine specifically for monkeypox in humans has not yet been developed, although research continues. Some countries have reportedly utilized varicella virus vaccines for monkeypox treatment [17].

Diagnosis typically starts with a visual examination of skin lesions, followed by a review of exposure history. The virus can be confirmed most accurately through electron microscopy of skin lesions. Polymerase chain reaction (PCR) testing from skin rashes, samples, fluids, crusts, or biopsies is another method used to detect monkeypox [1, 12, 13, 18], although PCR testing is not universally accessible. In certain cases, antigen and antibody detection methods can mistakenly identify monkeypox as the highly contagious varicella virus [18]. Mortality rates for monkeypox are estimated between 3% and 6% [19], making early detection and isolation crucial to controlling the virus’s spread. While PCR is the preferred confirmation method, image analysis can be an effective preliminary diagnostic tool in regions with limited resources, helping to curb outbreaks. Monkeypox poses diagnostic challenges for clinicians due to its rarity and its skin manifestations, which can resemble those of chickenpox, measles, and smallpox [1, 4]. This resemblance has led researchers to focus on developing image-based classification models for skin lesions related to various viral diseases, including monkeypox, chickenpox, and measles.

This study presents a hybrid model aimed at classifying skin lesions from monkeypox, chickenpox, measles, and healthy skin. The structure of the paper is outlined as follows: Section 2 details the dataset used, data augmentation techniques, and the proposed detection framework; Section 3 reports the model’s results and compares them with findings from other research, based on performance metrics; Section 4 offers a summary of the key findings and suggests areas for further research.

1.2 Contribution and novelty

Research on detecting virus-related diseases through deep learning (DL) has largely relied on transfer learning, using well-established pre-trained DL models. However, previous studies highlight three main limitations that this research seeks to overcome:

1. Most models are designed for binary classification, typically distinguishing monkeypox from other conditions. However, differentiating monkeypox lesions from chickenpox and measles lesions is essential, given their similar visual characteristics. This study develops a DL model for a four-class classification problem (normal, monkeypox, chickenpox, and measles), addressing these distinctions more comprehensively.
2. Previous studies often employ outdated DL models for predictions through transfer learning. In contrast, this study introduces an ensemble deep neural network framework to improve monkeypox lesion detection accuracy. Seven pre-trained DL models were evaluated, and the top three were combined using majority voting, creating a more robust final prediction output.
3. Limited datasets in prior studies have restricted model accuracy and reliability. This research utilizes the largest publicly available monkeypox dataset, with additional data generated through augmentation techniques to ensure broader and more reliable model training.

This study’s contributions and innovations include the following:

• By including three conditions with visually similar lesions, this study addresses a four-class problem, advancing holistic problem-solving in disease classification.
• Results for all seven pre-trained DL models in monkeypox detection were generated and compared using a refined architecture.
• The three highest-performing pre-trained models were integrated through majority voting to strengthen detection reliability and success rates.
• Results of the proposed ensemble framework are compared with individual pre-trained network architectures, highlighting the advantages of the ensemble method.

1.3 Related works

Research efforts towards early detection of the monkeypox virus have been somewhat limited, leaving room for improvement in the results. The studies in this area revolve around detecting the Monkeypox virus, detailing the datasets used, classification methods employed, and the achieved outcomes.

Nayak et al. [20] leveraged deep learning methods to diagnose monkeypox by analyzing visual features in skin lesion images. To evaluate the effectiveness of their approach, they used a publicly accessible dataset and tested it on five well-established pre-trained neural networks: GoogLeNet, Places365-GoogLeNet, SqueezeNet, AlexNet, and ResNet-18. Each of these models brings unique strengths to feature extraction and image recognition, offering a diverse set of architectural frameworks for comparison. By applying these varied models, Nayak et al. aimed to determine which architecture would best distinguish monkeypox from other conditions based on skin lesion imagery, contributing to a more accurate and efficient diagnostic approach. The optimal parameters were selected using hyperparameter. With an accuracy of 99.49%, ResNet18 achieved the highest rating among the aforementioned models. A distinction was drawn between monkeypox and other diseases, citing Alakus [21]. Images of various illnesses as well as chickenpox were used. In the second phase, data classification was performed using a Siamese deep learning model, achieving an accuracy score of 91.09%. Örenç et al. [22] created EfficientNetB3, ResNet50, and InceptionV3 using picture datasets that they acquired from Kaggle. The best model is resNet50, based on the three models' results. With an accuracy of 94.00%, resNet50 has the greatest accuracy value. Akın et al. [23] explored various deep learning models for classifying monkeypox skin lesions, including ResNet-18, ResNet-50, VGG-16, DenseNet-161, EfficientNet B7, EfficientNet V2, GoogLeNet, MobileNet V2, MobileNet V3, ResNeXt-50, ShuffleNet V2, and ConvNeXt. Among these models, MobileNet V2 demonstrated the highest performance, achieving an accuracy of 98.25%, sensitivity of 96.55%, specificity of 100.00%, and an F1-score of 98.25%, making it the top performer in their analysis. Additionally, Özaltın and Yeniay [24] leveraged MobileNet V2 as a feature extractor, integrating it with a Support Vector Machine (SVM) for classification purposes. By applying the Chi-Square method to select relevant features, they successfully classified 500 features with SVM, reaching an accuracy rate of 99.69%. This approach highlighted the effectiveness of combining deep learning-based feature extraction with traditional machine learning classifiers for improved accuracy in identifying monkeypox lesions.

In a related study, Sathwik et al. [25] explored models such as SVM, CNN, VGG16, VGG19, ResNet50, ResNet101, EfficientNet B0, EfficientNet B1, and EfficientNet B2 for detecting monkeypox lesions using images from the Kaggle open-source monkeypox dataset. Among these models, ResNet and VGG19 reached the highest accuracy, each achieving 92% in identifying the lesions.

A Q-learning approach for multilayer neural network parameter tuning and reinforcement learning for the classification of monkeypox images was created by Velu et al. [26]. For monkeypox, they received 96% f1 scores, 95% recall, and 95% accuracy. Almufareh et al. [27] utilized deep learning models (Inception V3, ResNet50, MobileNet V2, and EfficientNet B4) to classify skin lesions as either MPXV positive or negative. Their methodology was evaluated using both the Monkeypox Skin Image Dataset (MSID) and the Kaggle Monkeypox Skin Lesion Dataset (MSLD), assessing model performance based on balanced accuracy, sensitivity, and specificity. In related research, Lakshmi and Das [28] employed models such as VGG16, VGG19, ResNet50, ResNet101, DenseNet201, and AlexNet for the detection and classification of monkeypox symptoms, with ResNet101 achieving the highest accuracy at 98.59%. Uysal [29] developed a hybrid AI system for identifying monkeypox from skin images, utilizing a four-class classification scheme (chickenpox, measles, monkeypox, and normal). Models including CSPDarkNet, InceptionV4, MnasNet, MobileNetV3, RepVGG, SE-ResNet, and Xception were tested. To improve classification, the top two models were combined with a long short-term memory (LSTM) network, resulting in a unique hybrid model that achieved 87% test accuracy. Ahsan et al. [13, 30] introduced the "Monkeypox2022" dataset, the first open-access dataset for monkeypox classification, containing 1905 images in four categories: monkeypox, chickenpox, measles, and normal. This dataset, gathered from online sources, was shared on GitHub. By applying their modified VGG16 deep neural network model, Ahsan et al. achieved a 78% accuracy in distinguishing monkeypox lesions in a binary classification problem (monkeypox vs. other). Ali et al. [19] also compiled skin lesion images into four distinct categories—monkeypox, chickenpox, measles, and normal—forming the "Monkeypox Skin Lesion Dataset (MSLD)," which has become a valuable resource for advancing research in this field.

They employed VGG-16, ResNet50, and InceptionV3 deep neural network architectures for lesion class detection, with ResNet50 achieving 92.96% accuracy, VGG-16 reaching 81.48% accuracy, and their ensemble model achieving 79.26% accuracy. They also developed a web application for online image detection, emphasizing the need for a larger dataset to account for demographic differences. Sitaula and Shahi [31] used thirteen pre-trained deep neural network architectures on the Monkeypox2022 dataset (three lesion variants and one normal class). The highest performing Xception and DenseNet169 DL models were combined using the majority vote approach to create an ensemble model with 87.13% accuracy. Islam et al. [32, 33] addressed data limitations by web scraping, curating the Monkeypox Skin Image Dataset 2022. Their analysis of skin images in six categories resulted in ShuffleNet-V2 achieving the highest success with 79% accuracy. They further improved predictions with an ensemble model, reaching 83% accuracy.

Alakus and Baykara [34] examined the similarity between monkey flower visuals and some types of sigils that could cause confusion during diagnosis. Ali et al. [19] designed a deep neural network to detect monkeypox by analyzing DNA sequences of HPV and MPV, reaching an average accuracy of 96.08% and an F1 score of 99.83% in classifying DNA maps. Eid et al. [35] introduced a parameter-optimized LSTM deep network for monkeypox detection, achieving a predictive success rate of R² = 0.73 through hyperparameter tuning using the BER optimization algorithm. Abdelhamid et al. [36] enhanced monkeypox image classification by employing AlexNet, VGG-19, GoogleNet, and ResNet-50 models with transfer learning for feature extraction, reaching a 93.80% classification accuracy with GoogleNet and a hybrid feature selection method on the Monkeypox Skin Images Dataset (MSID).

Sahin et al. [37] created an Android app for detecting monkeypox, leveraging pre-trained models such as ResNet18, GoogleNet, EfficientNetb0, NasnetMobile, ShuffleNet, and MobileNetv2, which were specifically optimized for monkeypox detection using MATLAB. MobileNetv2 demonstrated the highest accuracy, reaching 91.11%, and was subsequently converted to a TensorFlow Lite model for integration into the Android platform. On three different mobile devices, average processing times for image extraction were recorded as 197 ms, 91 ms, and 138 ms. Miran et al. [38] employed an LSTM model to predict chickenpox cases using data from the CDC and web sources, achieving a correlation coefficient of 0.97114 and a sum of squared errors of 341.01547, outperforming linear regression methods. Haque [39] applied five deep neural network models—VGG19, Xception, DenseNet121, EfficientNetB3, and MobileNetV2—enhanced with a convolutional block attention module (CBAM) for monkeypox detection. The Xception-CBAM-Dense architecture yielded the best results, with an accuracy of 83.89%.

Kumar [40] worked on classifying monkeypox lesions using the Monkeypox-Skin-Lesion-Dataset. He extracted features with AlexNet, GoogleNet, and VGG16Net and utilized classifiers such as SVM, KNN, Naïve Bayes, Decision Tree, and Random Forest. The highest classification accuracy, 91.11%, was obtained by combining the Naïve Bayes classifier with features from VGG16Net. Dwivedi et al. [41] compared three deep neural network models for monkeypox lesion detection. Their study on the Monkeypox2022 data set resulted in detection accuracies of 84%, 87%, and 77% for the ResNet50, EfficientNetB3, and EfficientNetB7 models, respectively. Ozsahin et al. [8] curated a data set with two classes (monkeypox and chickenpox) using images from two open-source datasets. They developed a CNN model with four convolutional layers to identify and categorize monkeypox and chickenpox lesions. Comparing their model with five well-known DL models (VGG16, VGG19, ResNet50, AlexNet, InceptionV3), their suggested technique significantly outperformed the others, achieving a test accuracy of 95%. A novel diagnostic approach was proposed by Almutairi [4] to divide patient cases into two categories as pox (monkeypox, chickenpox, and measles) and normal). They used five DL models that have already been trained (Xception, MobileNet, MobileNetV2, VGG19, VGG16, and MobileNet) for diagnostic framework. The hyperparameters of these models have been determined with a metaheuristic algorithm to provide high performance. In addition, seven machine learning models were utilized to classify the extracted features in this study. The effectiveness of the proposed framework was evaluated on two publicly accessible datasets, MSID and MPID, with results presented accordingly. The VGG16 model combined with maximum voting (k=7) yielded the highest performance within the diagnostic framework, achieving accuracy, sensitivity, specificity, precision, and F1 score values of 97.44%, 94.81%, 97.67%, 94.72%, and 94.67%, respectively. Gairola [42] employed the Monkeypox Skin Lesion Dataset (MSLD) for a two-class classification task, using feature extraction with three deep learning models and a fusion approach involving six different machine learning models (DT, SVM, KNN, LR, NB, RF). Their approach achieved the highest performance using the AlexNet+VGGNet+NB combination, with an accuracy of 95.55%.

Irmak et al. [43] trained and tested their classification approach for monkeypox lesions using the Monkeypox Skin Image Dataset, comparing the performances of three pre-trained models—MobileNetV2, VGG16, and VGG19. Their findings indicated that MobileNetV2 had the highest predictive accuracy, reaching 91.38%, with precision at 90.5%, recall at 86.75%, and an F1 score of 88.25%. Agrawal et al. [44] used ResNet50, EfficientNet-B3, VGG16, and InceptionNetV3 on an image dataset from Kaggle, with EfficientNet-B3 demonstrating the highest accuracy at 93%. Haque et al. [45] enhanced monkeypox detection through deep transfer learning combined with a convolutional block attention module (CBAM). They applied channel and spatial attention mechanisms to five models (VGG19, Xception, DenseNet121, EfficientNetB3, and MobileNetV2), achieving an accuracy of 83.89% with an Xception-CBAM-Dense-based architecture for distinguishing monkeypox from other conditions.

Within the scope of this study, all image preprocessing, data replication, model creation, model training and calculation of performance metrics were developed using Keras package and Python programming language. The generated ensemble diagnostic method model was created utilizing architectures on the Kaggle platform and was accelerated using the platform's GPU capability. Table 4 shows the distribution of train, validation and test datasets for both cases. In this distribution, the ratio of 80:20:10 was adopted and the selection was made randomly. Data augmentation was applied solely to the training and validation sets, keeping the test set unchanged. This approach enhances test reliability by evaluating model performance on original data.

Table 4. Assignment of dataset split to train, validation and test

Figure 3 and Table 5 present a performance comparison chart of seven pre-trained DL models and the proposed ensemble diagnostic framework approach. In addition, the matrices of the confusion matrices showing how many of the models predicted to belong to the true class and how many to the false class from the 77 test image data are presented in Figure 3.

Accuracy alone does not adequately capture the models' performance. Thus, the proposed models were further assessed using precision, recall, and F1 score, offering a more comprehensive evaluation. Table 5 summarizes obtained results. When the data are reviewed, it is clear that the ResNet152 design had the best success rate (94%), while the DenseNet121 architecture had the lowest success rate (75%). The Proposed Ensemble model outperforms the best pre-trained model by 2% in classification success. The confusion matrices of five models are shown in Figure 3.

Table 5. Performance results of models for test dataset

3a.jpg

3b.jpg

3c.jpg

3d.jpg

Figure 3. Confusion matrix of used all model for test dataset

Table 6. Comparison of the results obtained with the literature

Although there are few research studies on Monkeypox lesion categorization in the literature, we believed that comparing the findings of the approach we developed with existing literature would be informative. Table 6 provides a summary of studies that have used the Monkeypox Skin Lesion Dataset (MSLD) along with the results of the current investigation. Upon analyzing previous studies, we could not find any other study that analyzed the same dataset using four categories, except for Uysal [29] and Irmak et al. [43]. Irmak et al. [43] employed MobileNet architectures and achieved a 91% accuracy rate, while Uysal [29] attained an 87% accuracy rate with LSTM. In the current study, we present the performance of a network architecture developed for four categories, anticipating a lower classification success compared to the two-category problem. However, it is evident that the proposed method demonstrates classification performance even higher than the majority of two-category monkeypox classification problems found in the literature.

We developed an ensemble model by synthesizing the estimation results of the MobiNetV3, ResNet152, and EfficientNetB3 architectures with the hybrid diagnostic framework approach we proposed. We successfully increased the estimation success rate by 3% for multiclass with this approach. However, as seen in the Current Study, the combination of multiple models and the classification of multiple diseases can enhance classification performance.