Leveraging Stacked Ensemble Meta-Learning and Deep Neural Networks for Improved Skin Cancer Diagnosis

In medical image analysis, deep convolutional neural networks have proven highly effective and widely adopted, particularly in classification and object detection tasks [10, 11]. The potential of machine learning and deep learning in early skin cancer detection has been widely studied, with a focus on improving diagnostic outcomes and assisting dermatologists in making informed clinical decisions. Automated image analysis techniques enable these approaches to classify skin lesions as malignant or benign, providing an efficient and scalable alternative to standard diagnostic methods. This section provides an overview of the latest advancements in deep learning, ensemble learning, and meta-learning strategies for skin cancer detection.

Many studies have leveraged pre-trained models like XceptionNet, EfficientNetV2, and ResNet for skin lesion classification. For example, Thapar and Tiwari [12] employed XceptionNet for skin lesion classification, achieving an accuracy of 88.72% and highlighting its effectiveness in medical image analysis. The authors employed multiple advanced deep learning architectures for the diagnosis of skin cancer, specifically leveraging architectures like XceptionNet, EfficientNetV2S, InceptionResNetV2, and EfficientNetV2M. While their work emphasizes the integration of explainable AI (XAI) to clarify model decision-making, it does not specifically address stacked ensemble meta-learning. With XceptionNet reaching an accuracy of 88.72%, the study illustrates how such technologies can elevate diagnostic accuracy, facilitate clinical workflows, and positively impact patient outcomes. Ezeddin et al. [13] presented a method which highlights the optimization of deep ensemble learning for melanoma skin cancer classification, utilizing state-of-the-art convolutional neural networks (CNNs) like ResNet101 and ResNext101. By employing a weighted averaging ensemble approach, the study achieved a classification accuracy of 96.12%, demonstrating significant advancements in detection methods. This integrated architecture enhances diagnostic precision in medical imaging, indicating its effectiveness for early clinical detection and management of melanoma, with the potential to significantly improve patient care.

A max voting ensemble method for skin cancer classification was proposed by Hossain et al. [14], utilizing the combined strengths of MobileNetV2, VGG16, and ResNet50. With an accuracy of 93.18% and an AUC of 0.9320 on the ISIC 2018 dataset, the ensemble model significantly boosted diagnostic accuracy. This approach leverages multiple models to aid clinicians in making precise and prompt diagnostic decisions, ultimately contributing to improved patient care. It may not always capture the nuances of complex cases where a single model might provide a more accurate prediction than the majority, potentially leading to suboptimal outcomes in certain scenarios.

Maurya et al. [15] introduced a hybrid model combining topological data analysis (TDA) and deep learning (DL) for basal cell carcinoma (BCC) diagnosis, achieving 97.4% accuracy and an AUC of 0.995. Persistent homology is utilized to capture complex topological patterns from telangiectasia and skin lesions, contributing to improved performance of deep learning models. While it focuses on BCC, the integration of TDA with DL represents a significant technological advancement in skin cancer detection. This study utilizes a dataset of 395 skin lesion images, with a focus on telangiectasia features that are processed through automated segmentation and analyzed using both topological data analysis (TDA) and deep learning techniques. A larger dataset could enhance the robustness and generalizability of the model. A limited dataset may not capture the full variability of BCC presentations in different populations. In their study, Daghrir et al. [16] introduced a hybrid strategy that merges deep learning and classical machine learning techniques to improve the accuracy of skin cancer diagnosis. The approach integrates a CNN for extracting deep features automatically from dermoscopic images. Simultaneously, KNN and SVM classifiers were trained on manually crafted features describing the lesions’ texture, shape, and color. To improve classification accuracy, the outputs of these models are aggregated using a majority voting approach. The experiment was carried out on a publicly accessible ISIC dataset, where the authors selected 640 representative images, both benign and malignant, from a total of 23,000 melanoma samples. Experimental results indicate that CNN achieved an accuracy of 85.5%, while SVM and KNN reached 71.8% and 57.3%, respectively. The ensemble model, leveraging majority voting, further improved the accuracy to 88.4%. However, the study highlights key challenges, including a limited amount of labelled training data, which the authors suggest addressing through semi-supervised learning in future work. To enhance melanoma classification, Filali et al. [17] later proposed a method that integrates handcrafted features with deep representations extracted from pre-trained CNNs. The proposed approach leverages feature-level fusion to combine the discriminative power of both feature types, aiming to improve classification accuracy. To ensure robustness, the model was evaluated on datasets of different sizes, namely the PH2 and the ISIC challenge datasets. The fusion approach yielded significant improvements in classification accuracy, particularly on the PH2 dataset. The integration of handcrafted and deep features in this study demonstrates a robust and effective approach for melanoma detection. Amin et al. [18] further enhanced skin cancer detection by integrating deep feature fusion techniques for both localization and classification. The study employs multiple deep learning models, including AlexNet and VGG-16, to extract important features from segmented skin lesion images. Principal Component Analysis (PCA) is applied to the extracted features to retain the most relevant features, thereby improving classification performance. Deep feature integration, when trained and validated on a curated dataset, proves effective in boosting both the detection accuracy and localization precision of skin cancer.

Devadhas et al. [19] presented a deep learning framework based on stacking ensemble techniques, attaining a 97% accuracy rate in detecting skin cancer. It utilizes a combination of CNN and deep neural networks (DNN) as input for a Long Short-Term Memory (LSTM) meta-classifier. The approach includes preprocessing of skin images, segmentation using Fuzzy-C-Means clustering, and feature extraction via Local Binary Pattern (LBP). The absence of a thorough evaluation limits understanding of the model’s performance, which is critical in medical applications where diagnostic errors can have significant consequences. In another study, Chiu et al. [20] developed an AI-driven skin cancer diagnosis model using a two-stage voting ensemble approach, significantly reducing false negatives and improving diagnostic accuracy in both ISIC and CSMUH datasets, with potential to aid resource-limited medical settings. Mary et al. [21] presented an advanced ensemble model combining ResNet, EfficientNet, and MobileNet to enhance skin cancer diagnosis. It addresses limitations of previous models by leveraging the strengths of these CNNs, integrating advanced preprocessing, data augmentation, and transfer learning. With an accuracy of 96.33% on the HAM10000 dataset, the model demonstrates strong potential for precise and early skin cancer detection.

The study by Kaur et al. [22] highlights the implementation of advanced deep learning models tailored for melanoma detection, with particular emphasis on automating the early diagnostic process. The method involves a hybrid pipeline, starting with morphological and context aggregation techniques for preprocessing, and concluding with a segmentation network to extract lesion areas. The classification model achieves a 93.40% accuracy rate using cleaned and segmented images. The BEDLM-CMS model proposed by Shah et al. [23] employs an ensemble of LSTM, BLSTM, and GRU networks to detect mutations responsible for cutaneous melanoma. It addresses challenges like data scarcity and overfitting, achieving an accuracy rate of 97% in independent tests and 94% in tenfold cross-validation. Liu et al. [24] developed a hybrid deep network combining Resnet-50 and CrossViT, which enhances feature representation and diagnostic accuracy through improved classification performance and stable feature fusion. This approach aims to refine early diagnostic accuracy in skin cancer, ultimately contributing to better patient survival rates. Qureshi and Roos [25] addressed the challenge of class imbalance in skin cancer detection by proposing an ensemble CNN model that integrates diverse CNN architectures along with metadata. Using 33,126 dermoscopic images, it combines pre-trained and custom-trained models with an SVM meta-learner, achieving enhanced accuracy. In their work, Rahman et al. [26] utilized anisotropic diffusion filtering to enhance image quality and applied a fast-bounding box (FBB) method for efficient segmentation of skin lesions. It integrates hybrid feature extractor (HOG, LBP, SURF) and VGG19-based CNN for feature fusion. The approach achieves 91.65% sensitivity and 95.70% specificity using ISIC 2017 and the academic torrents datasets. In a subsequent effort, Hemantkumar et al. [27] focuses on a deep learning feature fusion approach combined with an extreme learning machine (ELM) to classify benign and malignant skin lesions. The study highlights the use of a curated dataset from HAM10000 and Dermis.net, achieving a high accuracy of 98.0% for the validation set. Additionally, several studies have tackled the challenge of multi-class skin cancer detection using optimized hybrid deep learning models [28-30]. Table 1 provides a summarized comparison of the reviewed literature, highlighting the models used, key findings, and datasets employed in various skin cancer detection studies.

This section reports and analyzes the results of the proposed stacked ensemble meta-learning model for multi-class skin cancer classification. The performance of the ensemble model, incorporating four base CNN models (Custom CNN, InceptionResNetV2, ResNet101V2, and DenseNet201) and five meta-classifiers (Logistic Regression, Random Forest, Decision Tree, Gradient Boosting, and XGBoost), is assessed by means of standard metrics, including accuracy, precision, recall, and F1-score. Furthermore, training and validation curves for the base CNN models are analysed to assess their learning behaviour and a comparative analysis between the best-performing ensemble and the individual base models are presented. Finally, confusion matrix and detailed classification results are described for the best performing ensemble model, and a comparison with other similar works is provided.

4.1 Performance of stacked ensemble with different meta-learners

To evaluate the effectiveness of different meta-classifiers in the stacked ensemble framework, we assessed the final classification performance of each ensemble configuration. Table 4 summarizes the results in terms of accuracy, precision, recall, and F1-score. The evaluation metrics considered offer a comprehensive assessment of each meta-learner’s capacity to aggregate predictions from the base learners and generalize to unseen data. The results demonstrate the efficacy of the ensemble approach and highlight the varying performance of Logistic Regression, Random Forest, Decision Tree, Gradient Boosting, and XGBoost as meta-learners.

Table 4. Performance comparison of stacked ensemble models with different meta-learners

The results reveal a significant disparity in performance across the different meta-classifiers. Notably, Random Forest and XGBoost achieved excellent results, with accuracy and F1-scores exceeding 98%. This indicates their ability to effectively leverage the combined predictions of the base CNN models. With an accuracy of 98.63% and precision, recall, and F1-score all at 98.64%, the Random Forest meta-classifier demonstrated the best overall performance. Random Forest excels in ensemble frameworks by building multiple decision trees on bootstrapped data samples, allowing it to handle high-dimensional and complex input effectively. As a meta-learner, it can effectively learn non-linear interactions among base model predictions, capture feature redundancies, and reduce overfitting via averaging. Such properties enhance Random Forest’s robustness, especially in scenarios where base learners generate varied predictions, as frequently observed in complex tasks like skin cancer detection

XGBoost, another tree-based ensemble method, also demonstrated excellent performance, with an accuracy of 98.53%, precision of 98.63%, and F1-score of 98.42%. Although marginally lower than Random Forest, XGBoost remains highly competitive due to its gradient boosting framework, which optimizes the meta-model in a stage-wise manner. This allows it to focus on hard-to-classify samples and systematically reduce bias, which is critical in medical diagnosis where misclassification can have serious consequences. Its slight underperformance in recall (98.24%) compared to Random Forest may be attributed to overfitting tendencies in boosting methods when dealing with highly similar predictions from base models.

The Gradient Boosting and Decision Tree meta-classifiers also performed well, with accuracies of 96.85% and 96.64%, respectively. Their high precision and recall suggest that they are capable of capturing meaningful interactions among base learner outputs. However, their slightly lower scores compared to Random Forest and XGBoost indicate limitations in their ability to generalize from ensemble predictions when used in isolation. Similar to XGBoost, Gradient Boosting leverages an ensemble of weak learners to improve predictive accuracy. In this case, overfitting can occur without careful regularization, while a single Decision Tree, being non-ensemble in nature, lacks the error correction and variance reduction benefits that ensemble-based meta-learners inherently provide. Moreover, Decision Trees, being simpler models, may not have fully captured the complex interactions between the base CNN model outputs.

Logistic Regression, in contrast, delivered the weakest performance, with an accuracy of 73.17%, a high precision of 87.73%, but much lower recall (66.72%) and F1-score (71.60%). This sharp performance drop highlights the limitations of linear models in high-complexity, non-linear classification problems. As a meta-classifier, Logistic Regression assumes linear separability in the prediction space of the base models. However, the outputs of complex base learners, especially in deep learning and ensemble settings, are often non-linearly distributed. Consequently, Logistic Regression struggles to draw effective decision boundaries, especially for harder-to-classify minority or borderline cases, which is reflected in its poor recall.

The significant difference in performance between linear (Logistic Regression) and non-linear (Random Forest, XGBoost) meta-classifiers underscores the importance of selecting meta-learners capable of capturing complex, non-linear relationships in ensemble learning settings. Simpler models like Logistic Regression lack the expressive capacity to effectively integrate diverse base learner outputs, particularly in high-dimensional, intricate domains such as medical image classification. In contrast, the superior performance of Random Forest and XGBoost demonstrates that ensemble methods, especially those built on decision tree-based algorithms, are highly effective at aggregating predictions from deep CNN models. Their ability to model complex feature interactions and their robustness against overfitting contribute significantly to improved diagnostic accuracy. These findings support the core hypothesis of this study: meta-learning architectures benefit most from strong, non-linear meta-learners that can capture subtle dependencies among base model predictions, particularly in critical, imbalanced, and complex classification tasks like skin cancer diagnosis.

4.2 Comparison with base CNN models

The effectiveness of the ensemble strategy is further evaluated by comparing the Random Forest-based ensemble model with each individual base CNN. Figure 5 presents the accuracy and loss trends during training and validation phases for each base CNN model, illustrating their convergence patterns and generalization capabilities. As observed, the Custom CNN model demonstrates rapid convergence, with both training and validation accuracy reaching a plateau after 20 epochs. The validation loss shows a slight increase after 16 epochs, suggesting potential overfitting, which was mitigated using dropout layers. In contrast, InceptionResNetV2, ResNet101V2, and DenseNet201 models with fine-tuning, exhibit faster convergence and higher validation accuracy, indicating their strong feature extraction capabilities.

Table 5. Comparison of classification performance: Best ensemble vs. base CNN models

Table 5 provides a comparative evaluation of the best-performing stacked ensemble model (utilizing Random Forest as the meta-classifier) against individual base CNN models. With an accuracy of 98.63% and precision, recall, and F1-score all at 98.64%, the stacked ensemble model outperformed all individual base CNNs. This reinforces the ensemble model’s strength in leveraging the diverse predictive behaviors of base learners to produce a more stable and generalized decision boundary. The model’s ability to capture complementary patterns from multiple CNNs and synthesize them through a robust meta-learner contributed to its excellent diagnostic performance.

a1.png

(a) Custom CNN model

b.png

(b) InceptionResNetV2

c.png

(c) ResNet101V2

d.png

(d) DenseNet201

Figure 5. Training and validation accuracy and loss plots for: (a) a custom-designed CNN; (b) InceptionResNetV2; (c) ResNet101V2; and (d) DenseNet201

Among the individual CNNs, InceptionResNetV2 emerged as the top-performing base model, with a validation accuracy of 94.06%, and estimated F1-score of 93.74%. Its deep hybrid architecture, combining Inception modules with residual connections, likely enhances its capacity to learn both fine-grained local textures and broader contextual features, critical for accurate skin lesion classification. DenseNet201 and ResNet101V2 also performed competitively, achieving corresponding accuracies of 92.67% and 90.78%, respectively. DenseNet201's high recall (93.42%) and F1-score (92.59%) reflect its ability to promote efficient feature propagation and reuse, while ResNet101V2’s slightly lower performance may be attributed to overfitting or diminishing returns from deeper layers in the absence of ensemble integration. Finally, custom CNN, although specifically designed for this task, showed a lower corresponding accuracy of 89.67%, highlighting the performance gap between hand-crafted architectures and pre-trained deep models that benefit from large-scale prior knowledge.

Overall, the results clearly demonstrate the effectiveness of the proposed stacked ensemble approach in enhancing the accuracy and reliability of skin cancer diagnosis. By integrating diverse base CNN models and employing a powerful meta-learner, the ensemble achieved a significant performance improvement over individual models. This approach not only boosts overall classification accuracy but also achieves a more balanced trade-off between sensitivity (recall) and specificity (precision), a critical factor in medical diagnosis, where both false positives (FP) and false negatives (FN) can lead to serious clinical consequences. As shown in the confusion matrix values for Class 3 (e.g., melanoma) in Table 6, the performance of the stacked ensemble model across different meta-classifiers demonstrates consistently low counts of FP and FN, highlighting its effectiveness over individual CNN sub-models. Notably, the Random Forest and XGBoost meta-classifiers produce the lowest FP (4) and FN (3 and 6, respectively), indicating their ability to accurately identify true cases while minimizing misclassifications. A lower FP count reflects fewer instances where non-melanoma cases are incorrectly classified as melanoma, thereby enhancing the model’s precision. More critically, keeping the FN count low is essential in medical diagnostics, as misclassifying a melanoma patient as healthy (a FN) can lead to delayed or missed treatment, which could have serious consequences. These results affirm that the stacked ensemble approach, particularly with Random Forest and XGBoost as meta-learners, demonstrates superior diagnostic reliability in skin cancer detection, outperforming individual CNN models.

Table 6. Confusion matrix values for Class 3 using different meta-classifiers in the stacked ensemble model

4.3 Comparison with existing work

Achieving 98.63% accuracy and precision and recall scores of 98.64%, the proposed stacked ensemble model proved highly effective for skin cancer classification. The findings confirm the model’s high reliability in differentiating benign from malignant lesions, with minimal errors in classification. The integration of multiple fine-tuned CNNs at the base level and diverse meta-classifiers at the top level contributed to the model’s robust predictive performance. When compared to existing studies in the literature, our model consistently outperforms other approaches. For instance, Thapar and Tiwari [12] employed a combination of XceptionNet, EfficientNetV2, and InceptionResNetV2 on a curated dataset, with the best-performing model (XceptionNet) achieving only 88.72% accuracy, nearly 10% lower than our ensemble. Similarly, Ezeddin et al. [13] used an ensemble of ResNet101 and ResNext101, obtaining 96.12% accuracy, which is 2.5% lower than our results. Hossain et al. [14] used a max-voting ensemble of MobileNetV2, VGG16, and ResNet50 on the ISIC 2018 dataset, reporting 93.18% accuracy, along with balanced precision (93.10%) and recall (93.17%). Although their ensemble approach was effective, it still falls short in comparison to our stacked ensemble, which leverages more advanced meta-learning strategies. Filali et al. [17] explored handcrafted and deep features, reporting strong performance on the PH2 dataset (accuracy of 98%) but only 87.8% accuracy on the more challenging ISIC dataset, where our model clearly excels. Further, Mary et al. [21] proposed an ensemble of ResNet, EfficientNet, and MobileNet, achieving 96.33% accuracy on the HAM10000 dataset, which again is surpassed by our model. Rahman et al. [26] combined handcrafted features with VGG19, achieving 91.65% sensitivity and 95.70% specificity, indicating solid clinical relevance, yet not outperforming the comprehensive metrics reached by our ensemble. Likewise, Hemantkumar et al. [27] used deep feature fusion with an Extreme Learning Machine (ELM), achieving 98.0% accuracy, still trailing behind the 98.63% accuracy of our proposed method. Table 7 clearly highlights that our stacked ensemble model consistently achieves the highest accuracy, precision, and recall rates across all reviewed studies. This establishes its potential as a state-of-the-art approach in skin cancer detection.

Table 7. Performance comparison with similar existing studies for skin cancer detection