An Explainable Stacking Ensemble Framework for Student Academic Performance Prediction in Learning Management Systems

The growing digitization of education has altered traditional learning environments by including Learning Management Systems (LMS). LMS platforms make learning content distribution easier and allow for the systematic collection of behavioral data, such as login frequency, time spent on learning modules, and activity completion rates [1, 2]. This information opens up many doors for Educational Data Mining (EDM), a field that uses data mining methods to learn more about students and how they learn [3].

Recent advances in machine learning have shown promising results in predicting student academic outcomes, enabling timely and personalized interventions [4]. Different challenges, though, make predictive models useless in the classroom. First, class imbalance where high-performance students outshine failing or at-risk children is common in educational databases. This bias causes models to favor majority classes and overlook minority trends, reducing early-warning effectiveness [5]. Secondly, while complex models like ensemble methods (e.g., stacking, boosting, bagging) deliver higher accuracy, they often act as "black boxes"—provides little to no insight into how predictions are produced [6]. In the context of education, where trust, transparency, and accountability are critical, insufficient interpretability may impede acceptance by stakeholders, including educators, parents, and legislators [7]. Thus, a growing demand exists for interpretable and Explainable Artificial Intelligence (XAI) approaches that preserve model performance while making outcomes understandable [8].

This study proposes a stacking ensemble with Logistic Regression (LR) as a transparent meta-learner, enabling interpretable aggregation of heterogeneous classifiers while preserving interpretability. Shapley Additive Explanations (SHAP) allocates a contribution value to each feature for a given prediction, enabling both local (instance-level) and global (dataset-level) explanations [9]. This research also addresses class imbalance using Synthetic Minority Over-sampling Technique – Edited Nearest Neighbors (SMOTE-ENN), a hybrid resampling method that combines over-sampling of minority classes with noise reduction through edited nearest neighbors [10]. The framework is optimized using Grid Search and evaluated with stratified 5-fold cross-validation (CV), ensuring competitive and generalizable performance.

Kaggle xAPI-Edu-Data in Kalboard 360 (https://www.kaggle.com/aljarah/xAPI-Edu-Data, accessed June 8, 2023) offers the data needed for this research. Its parts related to student involvement and academic performance are behavior-driven. Its pragmatic base makes it ideal for developing interpretable models pertinent to learning environments in the real world. This research aims to: 1) create a highly competitive predictive model using stacking ensemble learning, 2) effectively address class imbalance through hybrid resampling SMOTE-ENN, 3) improve interpretability through SHAP for local and global explanations, and 4) identify and analyze critical behavioral features that impact academic performance.

This work contributes to the literature of EDM and learning analytics in four major ways:

1. We present a leakage-aware stacking ensemble architecture for LMS-based academic performance prediction. Unlike typical workflows, which may lead to optimistic bias during resampling, the proposed system uses SMOTE-ENN only in training folds during stratified cross-validation.
2. The method combines heterogeneous ensemble learning with an interpretable LR meta-learner, which enables the interpretable aggregation of nonlinear behavioural patterns and maintains model transparency.
3. The prediction pipeline of the study directly integrates SHAP-based explainability to facilitate global and local interpretations of student behavioural characteristics that are associated with academic achievement.
4. The proposed framework supports early warning analytics and educator-oriented intervention processes as a modular LMS decision support component.

Beyond predictive accuracy, this framework is explicitly conceived as a decision-support component for LMS-based educational information systems. By integrating a transparent meta-learner with SHAP-based explanations and leakage-safe resampling, the framework supports deployment scenarios that require institutional trust, accountability, and data governance.

Predicting student academic performance has become a key research direction in EDM and Learning Analytics, particularly as the adoption of LMS increases across formal and informal learning environments. LMS creates detailed behavioral traces, like logins, resource visits, assignment submissions, and participation in discussion forums, that can be used to model student involvement and learning progress based on data [11, 12].

Dogan et al. [13] earlier surveys showed that classical machine learning methods performed well for predicting academic success. A recent study found that behavior and relationships are more important than demographic traits for making accurate predictions [14-16]. These results show how important it is to include behavior-driven indicators in LMS-based predictive models and to select and analyze them properly.

Traditional machine learning models, including decision trees (DT), LR, and support vector machines (SVMs), have been widely used for academic performance classification. These models are easy to understand and use, but they do not always work well when dealing with LMS data that is different from one another. Recent work has focused on ensemble-based learning as a solution to this problem, using bagging, boosting, and stacking to combine multiple models. Ensembles frequently achieve better results than individual models in educational datasets, according to earlier studies. This is particularly the case when there are interaction effects or nonlinear correlations in the student's behaviors [17, 18]. Nonetheless, ensemble models often behave as "black boxes," limiting their suitability for educational settings where transparency, trust, and accountability are essential for adoption by instructors and policymakers [19].

To resolve interpretability challenges, XAI has gained substantial attention in the EDM community. At the same time, XAI addresses the black-box nature of prediction models [19] and enhances their performance by leveraging multidimensional behavioral, motivational, and demographic data to uncover patterns that traditional methods cannot. Artificial Intelligence (AI)-driven models can combine several types of data, such as simulation logs, engagement scores, and test outcomes, to provide you with individualized feedback and predictions [20]. SHAP and LIME are two techniques that have been shown to help determine the importance of features and model logic, thereby building confidence and ensuring that AI is used ethically in schools [21]. This transparency is especially critical in high-stakes settings where decisions based on algorithmic predictions must be justifiable. SHAP provides one of the most powerful model-agnostic frameworks for decomposing predictions into additive feature contributions. SHAP has been successfully used to identify important learning behaviors and provide interpretable explanations at both global and local levels [22]. For instance, SHAP was used by Swamy et al. [23] to understand dropout rates and by Tong and Li [24] to analyze logs of massive open online course (MOOC) enrollment.

The foregoing review reveals a consistent pattern across three interconnected research directions — ensemble learning, class imbalance handling, and XAI — each of which has advanced substantially in isolation but has rarely been integrated into a coherent, system-oriented architecture for LMS environments. The following critical gaps emerge from this analysis.

Notwithstanding these gains, some significant deficiencies persist. First, most previous ensemble-based EDM studies either ignore class imbalance or treat it superficially, even though imbalance is endemic in educational data (with more high-performing than at-risk students). Only a few works explore hybrid resampling, such as SMOTE-ENN, and even fewer apply them rigorously within cross-validation folds [5, 25]. Second, even though more and more people are using SHAP, it is most often used as a post-hoc analysis rather than as part of a single prediction pipeline. This can lead to explanations that do not fit with the techniques used to optimize models. Third, earlier research has rarely examined the stability and reliability of SHAP explanations, which is a key need for real-world LMS deployment, as model transparency must be consistent across different data splits and student cohorts. Fourth, limited research offers a systemic architecture perspective, incorporating ensemble modeling and explainability as functional components within an academic decision-support system.

No previous research has explicitly built an architecture for LMS decision-support processes that combines stacking ensemble learning, full SHAP-based interpretability, and hybrid class-imbalance handling via SMOTE-ENN synthesis (see Table 1). Furthermore, the current literature seldom situates models within an information systems context, which limits their usefulness for actual institutional implementation. Using a modular, deployment-aware architecture that views explanation, imbalance handling, and prediction as co-designed functional components inside an LMS-based academic information system, this study fills these overlapping gaps.

Table 1 summarizes the methodological distinctions between the proposed framework and representative prior studies. The study shows four major gaps: (1) most ensemble-based EDM studies do not use hybrid resampling or do so outside of cross-validation, which could cause data leakage; (2) SHAP is mostly added after the fact rather than being an integral part of the pipeline; (3) the choice of meta-learner in stacking architectures is not always backed up by evidence; and (4) no previous work on the Kalboard 360 dataset has looked at all three problems (imbalance, stacking, and interpretability) from the point of view of deployment-aware information systems. The proposed framework is created to address these four gaps cohesively.

Table 1. Gap analysis of previous studies and the proposed framework

The following framework organizes the research discussion: Section 2 describes the method used. We will present and analyze our experimental findings and their implications for XAI methods, and describe how XAI improves the EDM system, as mentioned in 3. This study concludes in Section 4.

Eight machine learning algorithms have been applied to the LMS Kalboard 360 dataset. The investigation the setup and protocol of the experimental, also evaluation. Some prepossessing was carried out on the dataset before applying the algorithm. Following the application of the eight classic ML algorithms, bagging, and voting to the preprocessed dataset, an explainable stacking ensemble method was developed. The examine focuses on performance, specifically how the ensemble method enhances performance when compared to earlier research to predict student academic success. Futhermore, explainability refers to whether SHAP values can explain the ensemble method's internal logic. The predictor model calculates the performance metrics, whereas the interpreter summarizes the consequences of negative and positive attributes. In this study, the model was initialized using an ensemble learning meta-learner and Shapley values to build a stacked ensemble learning model and an XAI model.

3.1 Experimental setup

The proposed framework was implemented using Python with scikit-learn [29]. SHAP libraries [52] was used to interpret the model that provides local and global explanations. Imbalanced-learn with SMOTE-ENN [53] was used to combine under-sampling and over-sampling to handle imbalanced data. All experiments were conducted on a machine with 16GB RAM and an AMD Ryzen 7000 series processor.

To ensure unbiased evaluation, SMOTE-ENN was applied exclusively within the training portion of each stratified cross-validation fold. No resampling was performed on the validation or test partitions. This leakage-aware protocol prevents synthetic information from influencing evaluation data and guarantees consistent performance estimation.

3.2 Model performance evaluation

A hybrid resampling (SMOTE-ENN) method was used in the experiment to address issues arising from an unbalanced dataset. We then looked at a five-fold cross-validation method for training the model, since validation is an important step in making useful predictive models. In order to construct models for the purpose of forecasting students' performance, two classes of machine learning algorithms were employed: classical methods (KNN, LR, DT, RF, SVM, XGB, ETC, LGBM) and ensemble methods (bagging, voting, and stacking).

This study compared eight different machine learning models with a stacked ensemble learning approach for predicting student academic performance to demonstrate the empirically effectiveity of the ensemble learning method. When testing the selected stacking ensemble model, five fundamental learning models: KNN, RF, ETC, XGB, and LGBM were used to build a stacked classifier. LR was the meta-model used.

The comparative experiment was conducted on candidate meta-learners, LR and SVM, to justify the selection of LR as the meta-learner empirically. All candidates are tested using the same stacking architecture and 5-fold cross-validation conditions. Table 3 presents the findings of the comparison to identify the best meta-learner. In terms of mean accuracy (75.83% vs. 70.83%) and macro F1-score (76.55% vs. 71.68%), Stacking + LR performed better than Stacking + SVM. The 5-point accuracy discrepancy remains unaddressed, despite Stacking + SVM having a smaller standard deviation (1.47% vs. 5.41%). Because of its high computational efficiency, compliance with accurate SHAP-based interpretability, and excellent predictive accuracy, LR was chosen as the meta-learner.

Table 3. Comparison of meta-learner on stacking classifier (5-fold CV)

Using a stratified approach, the data were divided into training and testing subsets at an 80:20 ratio. Moreover, to avoid models suffering from the overfitting problem, the k-fold cross-validation technique was used with the value of k set to 5 using GridSearchCV as hyperparameter tuning. Table 4 presents the optimal hyperparameter configurations obtained through grid search optimization with 5-fold cross-validation for each classifier. Evaluation metrics were employed to measure and compare the performance of stacking ensembles against other ML models such as bagging and voting approaches.

Table 4. Hyperparameter search space for grid search cross-validation (CV)

The results of all the models' predictions, as measured by Accuracy, Precision, Recall, and F1-score. The model successfully reduces the likelihood of inaccurately identifying students as top performers by efficiently minimizing false positive predictions, as seen by the high precision value. At the same time, the model was able to accurately identify most high-performing students despite the initial class imbalance in the dataset, as shown by its strong recall performance. The model appears to strike a good compromise between sensitivity and prediction dependability, as evidenced by the relatively tight association between F1-score and precision.

These results are the product of the hybrid features of the SMOTE-ENN resampling approach, which incorporates synthetic minority oversampling and noise reduction through Edited Nearest Neighbors (ENN). To reduce majority-class bias and increase the classifier's discriminative strength, this strategy helps to establish a more even distribution of training data. Figure 5 shows the best score evaluation measurement for predicting students' academic performance that is produced by combining stacking ensembles with hybrid resampling.

Figure 5. The best score evaluation measurement

Table 5 shows the model performance in terms of Accuracy, Precision, Recall, and F1-score using SMOTE-ENN resampling. The Stacking Classifier performed best overall (Accuracy: 96.57%, Precision: 96.34%, Recall: 93.26%, and F1-score: 94.40%), followed by ETC and LGBM. LR performed the worst among all the metrics.

Table 5. Performance comparison of the proposed classifier models

Figure 4 shows a comparison of metrics (accuracy, precision, and F1-score) of eleven classification models (KNN, DT, LR, SVM, RF, ETC, XGB, LGBM, Bagging, Voting, and Stacking) in four resampling situations (none, SMOTE, ENN, and SMOTE-ENN). The results show that, in all metrics, SMOTE-ENN consistently achieves the highest performance. The worst scores are achieved without resampling at the same time, which indicates that the resampling technique has a considerable influence on the model performance for imbalanced class distributions.

Moreover, the low standard deviation values observed across all evaluation metrics confirm the resilience and reproducibility of the proposed framework during stratified 5-fold cross-validation. This indicates that the stacking-based SMOTE-ENN model can maintain a competitive predictive performance despite variations in data partitioning.

The results (see Table 5) show that the ensemble-based models are more effective for the unbalanced classification tasks after SMOTE-ENN resampling. This improvement indicates that the suggested framework captures more discriminative patterns in the data when compared to earlier results on the same dataset, such as Abu Amrieh et al. [26]. This improvement is significant. Three pieces of evidence bolster the credibility of this discovery. To begin, the very consistent fold-wise performance is seen by the small error bars in Figure 6(a-c). Secondly, the model demonstrates that promising generalization is maintained across all evaluation criteria, as seen by the uniformly low standard deviations across accuracy, precision, and F1-score. Section 3.3's SHAP analysis demonstrates, thirdly, that the model is dependent on significant behavioral variables, most notably StudentAbsenceDays.

(a) Accuracy per model and resampling

(b) Precision per model and resampling

(c) F1-score per model and resampling

Figure 6. Comparison of performance metrics of selected models

The results of the Wilcoxon signed-rank test indicated no statistically significant differences at α = 0.05 (p > 0.05), even though the proposed stacking structure regularly scored the highest mean F1-score under the SMOTE-ENN configuration. It is clear from the low fold-wise variability and consistent performance gains that the proposed framework offers dependable predictive capability for LMS-based student performance prediction. Detailed statistical test results are presented in Table 6.

Table 6. Wilcoxon signed-rank test: stacking vs baseline models (metrics: F1, resampling: SMOTE-ENN, a = 0.05)

3.3 Result of Explainable Artificial Intelligence: Shapley Additive Explanations

The results of the interpretability analysis utilizing the SHAP methodology are presented below. Each attribute is represented by a SHAP summary graphic, which shows how it affects the student's academic achievement.

The explanation for Figure 7 shows that StudentAbsenceDays is the most dominant feature, surpassing all others. This indicates that student attendance has the greatest influence on predicting academic performance. The second most influential feature is VisitedResources, which reflects the intensity of students' access to learning materials within the LMS. Subsequent features, such as ParentAnsweringSurvey, Relationship, RaiseHands, ParentSchoolSatisfaction, and AnnouncementView, make relatively moderate yet significant contributions. Semester and gender contribute the least among the features displayed individually. These findings confirm that the optimal stack ensemble is primarily driven by behavioural engagement features, reinforcing the importance of LMS interactions and attendance patterns in predicting students' academic performances. The model is easy to understand using the SHAP method, which helps identify important factors for early teaching support.

7.png

Figure 7. A global explanation of the stacking model SHAP explainer

Figure 8 shows the SHAP summary plot, which reveals both the magnitude and directional influence of each feature on the model's predictions. StudentAbsenceDays exhibits the widest SHAP value distribution, with clear polarization by colour: low absence values (blue) consistently push predictions in a positive direction. In contrast, high absence values (red) exert strong negative impacts, confirming that absenteeism is the most decisive and consistent behavioural determinant in the model. VisitedResources demonstrates the opposite pattern, where higher feature values yield positive SHAP contributions, affirming that active engagement with learning materials meaningfully elevates predicted academic performance. ParentAnsweringSurvey and Relation show similar trends, suggesting that when parents are involved, and there is a good relationship between home and school, it positively influences predictions. RaisedHands and AnnouncementsView similarly exhibit positive directional associations, reflecting the predictive value of interactive participation and responsiveness to instructional communications within the LMS. In contrast, demographic and institutional features, including gender, Semester, StageID, GradeID, SectionID, Nationality, and PlaceofBirth, display SHAP distributions tightly concentrated near zero, confirming their negligible predictive contribution. The summary plot substantiates that the model's predictive behaviour is predominantly governed by dynamic behavioural engagement indicators rather than static student attributes, demonstrating both statistical validity and pedagogical interpretability.

8.png

Figure 8. Shapley Additive Explanations (SHAP) summary plot based on mean SHAP values

Figure 9 presents a SHAP force plot providing a local explanation for an individual prediction. The visualisation illustrates how specific features shift the prediction away from the baseline, either increasing or decreasing the final output. The prediction ultimately leans toward the high-performance class, indicating that the model classifies the student as likely to achieve excellent academic results. Low absence emerges as the primary positive contributor, substantially strengthening the prediction. Active engagement with learning resources and classroom participation reinforces this effect, further pushing the output toward high performance.

9.png

Figure 9. Force plot depicting feature contribution toward a single prediction

In contrast, certain parental-related variables exert minor negative contributions, slightly moderating the prediction. Overall, the combined influence of behavioural engagement factors outweighs opposing signals, resulting in a confident, high-performance classification. This local explanation aligns with the global SHAP analysis, reaffirming that attendance and LMS engagement are the central drivers of academic performance predictions.

To enhance the interpretability of the Stacking Classifier, SHAP dependence plots were generated for the most influential feature identified in the model. Figure 10 presents two dependence plots for StudentAbsenceDays—encoded ordinally as 0 (above 7 days, high absenteeism) and 1 (under 7 days, low absenteeism)—with interaction effects colored by VisITedResources (Figure 10(a)) and raisedhands (Figure 10(b)), both evaluated with respect to the High Performance (H) class prediction.

(a) VisitedResources

(b) RaisedHands

Figure 10. Shapley Additive Explanations (SHAP) dependence plots for studentabsencedays with interaction effects on high performance (H) class prediction

The plots show a clear bifurcation in SHAP values. Students with a low absence rate (encoded as 1) tend to have very high SHAP values (around +0.5 to +0.6), indicating that the model predicts they will perform well. On the other hand, students who are frequently absent from class (encoded as 0) have low SHAP values (about -0.4 to -0.6), which reduces the likelihood of being labeled as high performers—even if other behavioral characteristics still have favorable values. That StudentAbsenceDays is the principal gating characteristic in the model's decision-making process is supported by this pattern, which aligns with educational research findings that consistent attendance is necessary for good learning outcomes.

Beyond the main effect, the interaction coloring in Figure 10(a) reveals that among students with low absenteeism, those who more frequently access digital learning resources (VisitedResources, higher values shown in warmer colors) tend to obtain marginally higher SHAP contributions. This suggests a synergistic relationship between physical attendance and digital engagement, in which both behaviors jointly increase the likelihood of high academic performance. Similarly, Figure 10(b) demonstrates that students who combine consistent attendance with active classroom participation (RaisedHands, higher values in warmer colors) attain the highest SHAP values, reflecting the principle of active learning, in which participatory behavior reinforces comprehension and correlates positively with academic achievement.

The pedagogical implications of these findings are direct. Before cumulative learning impairments become irreversible, academic counselors should intervene early when an individual's absenteeism exceeds 7 days. Second, attending class is not enough to guarantee good grades; teachers also need to actively encourage students to participate and make effective use of digital tools. Thirdly, the Stacking Classifier's SHAP values are interpretable, which makes the model a good early warning system. It can identify students who are struggling before the end-of-semester exams and then help them with quick, evidence-based interventions.

3.4 Comparison with previous studies

These results offer important new information about the advantages of using stacking ensemble techniques for predicting student academic performance, a basic strategy. Furthermore, in Table 7, we compared the proposed student performance prediction model with previous models (i.e., Abu Amrieh et al. [26], Alsulami et al. [15], Adnan et al. [27], Jang et al. [28], and Tong and Li [24]) in addition to each of the machine learning models that make up the stacking ensemble (KNN, RF, XGB, ETC, and LGBM). An outline of the baseline is provided below:

Abu Amrieh et al. [26]: Using ANN, NB, DT, and ensemble methods (bagging, boosting, and RF) to improve classification performance. The conducted feature selection and 10-fold CV, but did not resample the data. The recorded accuracy was 82%, precision was 85%, recall was 82%, and F1-score was 82%. The ensemble results increased accuracy by up to 25.8%.

Alsulami et al. [15]: Used the Kalboard 360 LMS dataset, the Boosting+DT algorithms achieved an accuracy of 77.9% with hyperparameter optimization, and 10-fold CV was used.

Table 7. Comparing the proposed model with previous model

Adnan et al. [27]: Using the OULAD dataset, the RF algorithms achieved 92% accuracy and precision. A 10-fold CV was used, with 80% and 20% data splits, and then 10-fold cross-validation was added. Jang et al. [28]: Using the Korea University LMS dataset, the selected STC algorithm successfully achieved 94% accuracy and precision, and a 10-fold CV was used. Tong and Li [24]: The conducted research using the LMS dataset in China. Applied six algorithms and a stacking ensemble with a meta-learner LR on online student behaviour logs and demographic data. The split is 80:20, with 5-fold CV and OneSideSelection resampling. The best results are from the stacking model with meta-learner LR, which achieved 98% accuracy and precision. The proposed model, which used ensemble stacking with an LR meta-learner, got 96.57% accuracy, 96.34% precision, 93.26% recall, and 94.40% F1. A 5-fold CV grid search was used with an 80:20 data split. Also, SMOTE-ENN resampling was utilised to fix the imbalance.

Our stacked ensemble strategy addresses these issues by combining the best features of multiple models. Mixing the outputs of several base learners with LR meta-learners improves performance by combining the simplicity and complexity of linear and nonlinear models. By immediately correcting data imbalances, our strategy improves projection performance metrics. Our primary goal is to utilize interpretability modeling to gain a deeper understanding of education. SHAP integration has made our method more competitive and consistent than independent models for academic performance.

3.5 Discussion

These findings suggest that behavioral engagement within LMS environments may partially compensate for certain academic risk factors, thereby providing meaningful pedagogical insights into student learning behavior. The integration of SHAP-based interpretability further enables transparent identification of influential behavioral attributes, facilitating transparent interpretation of how specific LMS engagement behaviors contribute to student performance prediction. Consequently, the proposed framework may support targeted intervention strategies based on explainable behavioral indicators rather than demographic profiling alone.

Several limitations of this study should nevertheless be acknowledged. First, the experimental evaluation was conducted using the Kalboard 360 dataset (n = 480). Although this benchmark dataset has been widely utilized in EDM research, reliance on a single institutional dataset may not adequately represent the behavioral diversity and contextual variability of broader LMS ecosystems. Furthermore, institution-specific learning practices and learner interaction patterns may limit the transferability of the proposed framework across different educational settings. The relatively limited sample size may also constrain model generalizability in larger-scale educational environments. Nevertheless, the use of the Kalboard 360 benchmark enables meaningful comparison with prior studies and provides a controlled setting for evaluating the methodological contribution of the proposed framework.

Second, the proposed approach relies primarily on static tabular features and does not explicitly model temporal learning trajectories, despite the increasing importance of sequential interaction data in modern LMS platforms. Consequently, longitudinal engagement dynamics and evolving learning behaviors remain unexplored in the present study.

Third, although the framework demonstrated promising predictive performance under experimental conditions, its practical deployment feasibility has not yet been evaluated through real-time inference scenarios, long-term institutional implementation, or user acceptance assessment involving educators and academic administrators.

Future research should therefore investigate cross-institutional validation and sequence-aware learning analytics models capable of capturing temporal engagement dynamics from continuous LMS interaction data. Additional studies may also explore deployment-oriented evaluation frameworks to assess scalability, transparency, and practical integration within real educational environments.