Integrating Indigenous Knowledge with Meteorological Data for Explainable Climate Prediction: An LSTM-SHAP Framework for Archipelagic Regions

Various studies show that the application of XAI in climate and environmental science is increasingly important to address the “black box” nature of complex AI models and to increase public and policymakers' confidence in prediction results. A comprehensive review by Huang et al. [12] highlights that XAI in Earth System Science not only improves the interpretability and transparency of AI models but also serves as a scientific tool to uncover dominant patterns and physical relationships underlying climate processes, effectively bridging the gap between process-based and data-driven approaches [12]. Meanwhile, in their systematic literature review, Schiller et al. [13] analyzed 575 publications on the use of XAI in environmental and Earth system sciences. They found that although methods such as SHAP and LIME are widely applied, few studies systematically evaluate the faithfulness and robustness of explanations. This indicates the need for standardized evaluation frameworks to ensure the reliability and trustworthiness of XAI applications in complex environmental systems [13].

In the context of explainable climate prediction, some studies have begun shifting their focus from post-hoc explanations to inherently interpretable models. González-Abad et al. [14] demonstrate how XAI can be used to improve deep learning-based statistical downscaling processes by analyzing the internal behavior of models to improve spatial and temporal generalization [14]. Materia et al. [15] emphasize that a model's ability to explain the basis for its decisions is an essential requirement for the application of AI in predicting extreme climate events such as heat waves and floods, as model transparency is crucial for policymakers' confidence [15]. Furthermore, Mamalakis et al. [16] identify limitations of several XAI methods on convolutional neural networks (CNN) in geoscience, such as gradient shattering, which can reduce the quality of interpretation [16].

One increasingly emphasized innovative direction is the integration of scientific knowledge and local or indigenous knowledge to enrich XAI-based climate prediction systems. According to Sohad and Mafrolla [17], local knowledge plays a crucial role in understanding the dynamics of environmental change at the micro level because it is rooted in the direct experiences of communities [17]. However, the literature still rarely combines the XAI framework with these traditional knowledge systems. A study by Okedele et al. [18] found that the integration of IKS into global climate change adaptation policies is essential for creating inclusive, equitable, and sustainable strategies [18]. IKS reflects ecological insights and sustainability practices developed by indigenous communities over centuries, offering unique perspectives on biodiversity conservation, natural resource management, and climate resilience. In a similar context, Bommer et al. [19] state that a systematic evaluation framework for assessing the robustness, faithfulness, and localization aspects of various XAI methods can serve as an interpretive bridge between data-driven models and scientific understanding of climate processes [19].

4.1 Experimental results

This experiment tested the LSTM model's ability to predict climate conditions by integrating environmental variables and IKS indicators. The dataset used in this experiment was a field-informed multivariate time-series dataset representing ecological conditions across 12 locations in the North Maluku islands over 450 days of observation. The dataset integrates empirically derived Indigenous Knowledge indicators obtained from ethnographic fieldwork with observed meteorological variables, with limited scenario-based augmentation applied solely to support experimental evaluation while preserving field-observed relationships. Each daily data point includes environmental variables such as rainfall, temperature, humidity, and wind direction, as well as IKS indicators such as leaf curling, cloud over mountains, muddy water, insect activity, and bioluminescent plankton. The data were normalized using StandardScaler and split into training (64%), validation (16%), and test (20%) sets. To capture temporal dynamics, the data were arranged in 14-day sequences (SEQ_LEN = 14) before being fed into the model. The model architecture consisted of a Bidirectional LSTM with 128 units followed by a 96-unit LSTM, dropout of 0.30 and 0.25 to prevent overfitting, and a Dense layer with ReLU and sigmoid activations on the output layer. The model was compiled with the Adam optimizer (learning rate 4×10⁻⁴) and binary cross-entropy loss, then trained for up to 90 epochs with early stopping and ReduceLROnPlateau. After training, the model was evaluated on test data using accuracy, precision, recall, and F1-score metrics, and the results were visualized using training curves and confusion matrices. To interpret the results, the SHAP (SHapley Additive Explanations) method was applied to identify the relative contribution of each feature to the rainfall prediction. Figure 2 shows the training and validation curves.

(a) Training vs Validation accuracy

(b) Training vs Validation loss

Figure 2. Training and validation curves of the Long Short-Term Memory (LSTM) model

Based on the training results shown in Figure 2, the LSTM model demonstrates a stable and focused learning process. In Figure 2(a), the training and validation accuracies gradually increase to around 0.58, indicating that the model can recognize data patterns well, even though the validation accuracy still shows fluctuations. Meanwhile, Figure 2(b) shows a consistent decrease in loss from around 0.76 to 0.68 without a significant increase in validation loss, indicating that overfitting did not occur. Overall, both curves suggest that the model has converged and maintains a balance between learning and adaptation to new data.

After the LSTM model converged during training, the next step was to evaluate its performance on previously unused test data. The evaluation assessed the model's ability to predict rainy and non-rainy conditions using a combination of environmental variables and IKS indicators. Figure 3 presents the model evaluation results as a confusion matrix.

Based on the training and evaluation results shown in Figure 3, the LSTM shows a classification bias towards the positive class (“Rainy”). Most of the actual rainfall data was correctly predicted by the model (high actual positive rate), reflected in the nearly perfect recall of 99.79. However, there are still many cases where the model predicts rain even when the exact condition is not rainy (false positives), resulting in a precision of 51.37%. As a result, the model's overall accuracy is only 51.32%, with an F1-score of 0.6783, indicating a moderate balance between rain detection and prediction accuracy.

Figure 3. Confusion matrix of model evaluation result

Furthermore, to interpret the LSTM model's decisions, SHAP (SHapley Additive exPlanations) analysis was performed on the test sample. Since the model input is a time series, each feature is evaluated based on its contribution at each time step, and these values are averaged over 2 weeks to obtain the overall importance of each feature. The results of this analysis are shown in Figure 4, which displays the distribution of each variable's contribution to the model's decision when predicting the probability of rain. The color of the dots represents the original value of the feature (red for high values, blue for low values). In contrast, the position of the dots along the horizontal axis indicates the direction and magnitude of their influence on the prediction results: dots that shift to the right increase the probability of rain (Rainy), while those that change to the left decrease the probability (Non-Rainy).

Based on the SHAP analysis results in Figure 4, environmental features such as rainfall, humidity, and temperature have the highest contribution values to the model prediction, indicating that these three variables are the main determinants influencing the probability of rainfall. This is consistent with the characteristics of a humid tropical climate in archipelagic regions such as North Maluku, where increased humidity and decreased temperature are often early indicators of convective cloud formation. In addition, local knowledge indicators such as clouds over mountains and leaf curling also make a significant positive contribution to rainfall prediction. These two indicators represent ecological signals that indigenous peoples have long used to indicate weather change. In contrast, the bioluminescent plankton feature shows a negative association with the probability of rain, consistent with the calm seas and clear skies that often accompany this phenomenon. Overall, these XAI results indicate that Indigenous Knowledge indicators contribute to the model’s predictions alongside meteorological variables. However, feature contribution alone does not constitute definitive evidence of meaningful knowledge integration, which requires alignment between the learned contribution patterns and the underlying ecological interpretations within IKS.

Figure 4. SHapley Additive exPlanations (SHAP) analysis results

To assess whether the observed SHAP contributions reflect meaningful integration rather than statistical coincidence, the contribution patterns of selected Indigenous Knowledge indicators were examined in relation to their ecological interpretations. For instance, the “clouds over mountains” indicator exhibits positive SHAP values predominantly under conditions of increased humidity and rainfall probability, which is consistent with its traditional interpretation as an early signal of the rainy season. Similarly, the positive contribution of the leaf curling indicator aligns with pre-rainfall moisture accumulation observed in local ecological practices.

Nevertheless, this analysis remains indicative rather than conclusive. While the observed alignment suggests semantic consistency between model behavior and Indigenous Knowledge interpretations, the study does not perform formal causal validation across multiple seasonal cycles. As such, the results should be interpreted as evidence of preliminary alignment rather than definitive proof of full knowledge integration. Further work is required to evaluate the robustness of these patterns across extended temporal and spatial contexts.

4.2 Root-cause analysis of model performance

Despite the successful convergence of the LSTM model and its ability to capture temporal patterns, the overall classification accuracy remains moderate. To address this limitation, a root-cause analysis was conducted to examine the underlying factors contributing to the observed performance, particularly the imbalance between recall and precision.

First, data distribution and classification threshold effects play a central role. The dataset exhibits a dominance of rainfall-related events, which biases the learning process toward the positive (“Rainy”) class. This imbalance, combined with a relatively low decision threshold, results in very high recall (99.79%) but increased false positives, thereby reducing precision and overall accuracy. While this behavior is suboptimal for accuracy-oriented evaluation, it is consistent with the model’s design objective as a risk-averse early warning system, where minimizing missed rainfall events is prioritized over avoiding false alarms.

Second, the encoding strategy of IKS indicators introduces additional limitations. In the current implementation, several indigenous indicators—such as leaf curling, cloud formation over mountain peaks, and seawater turbidity—are represented using binary or coarse categorical encodings. Although this approach enables straightforward integration into the machine learning pipeline, it inevitably simplifies complex ecological phenomena. Important nuances, such as the intensity, duration, or gradual progression of these indicators, may be lost during discretization. This reduction in representational richness can weaken the discriminative power of IKS features and partially explain the model’s limited accuracy.

Third, the suitability of the LSTM architecture for hybrid feature sets must be considered. LSTM networks are well suited for modeling continuous numerical time-series data, such as rainfall, temperature, and humidity. However, the hybrid nature of the input—combining sequential meteorological variables with categorical or semi-static indigenous indicators—may not be optimally handled by a purely recurrent architecture. The lack of dedicated embedding layers or attention mechanisms for categorical features may constrain the model’s ability to fully exploit the informational content of IKS indicators.

Finally, dataset construction and augmentation constraints may also contribute to performance limitations. Although the dataset is field-informed and grounded in ethnographic observations, the temporal and spatial coverage remains limited. Scenario-based augmentation was applied only to support experimental evaluation, but such augmentation may still introduce simplified patterns that do not fully reflect the causal complexity of real-world climate–ecology interactions. This limitation can lead to spurious correlations and reduce generalization performance on unseen data.

Overall, this root-cause analysis indicates that the moderate accuracy observed in the model is not attributable to a single factor, but rather to the combined effects of data imbalance, simplified feature encoding, architectural constraints, and dataset scale. These findings underscore the importance of aligning evaluation metrics with application goals and motivate future improvements, including richer encoding schemes for indigenous indicators, hybrid or attention-based model architectures, and the expansion of field-validated joint datasets.

4.3 Discussion

The experimental results demonstrate that the LSTM-based sequence learning approach is capable of capturing temporal relationships between meteorological variables and IKS indicators. The high recall value indicates that the model is highly sensitive to rainfall-related signals, reflecting its tendency to prioritize the detection of rainy conditions based on both scientific climate variables (e.g., rainfall and humidity) and selected indigenous ecological indicators such as clouds over mountains and leaf curling. This behavior is consistent with a risk-averse early warning perspective, where minimizing missed rainfall events is prioritized.

However, the relatively low precision and overall accuracy reveal important limitations that must be interpreted carefully. The observed bias toward the positive class results in frequent false alarms, which can be attributed to data imbalance, simplified encoding of IKS indicators, and architectural constraints of the LSTM when handling hybrid numerical–categorical feature sets. These limitations indicate that the proposed model should not be interpreted as an optimized or operational climate prediction tool.

Importantly, the primary contribution of this study does not lie in achieving high predictive accuracy, but in proposing and empirically examining a methodological framework for integrating IKS with scientific climate data using XAI. Within this framework, XAI is employed not to improve raw model performance, but to provide interpretability and transparency regarding how indigenous indicators and meteorological variables jointly influence model decisions. This distinction is crucial, as SHAP-based explanations reveal patterns of feature contribution that can be compared against established indigenous ecological interpretations, thereby enabling an initial assessment of semantic alignment rather than definitive causal validation.

From this perspective, the results should be understood as a proof-of-concept and exploratory validation of knowledge integration, demonstrating how indigenous ecological indicators can be operationalized, encoded, and interpreted within a data-driven modeling pipeline. While the current implementation exhibits performance limitations, it provides a transparent foundation upon which more advanced architectures, richer encoding schemes, and larger field-validated datasets can be developed.

Future research should therefore focus on improving representational fidelity of IKS indicators (e.g., ordinal or fuzzy encoding), exploring hybrid or attention-based models to better handle heterogeneous feature types, and conducting longitudinal validation across multiple seasonal cycles. These steps are necessary before the framework can be extended toward robust, application-oriented climate services.