Comparing the Performance of Machine Learning Models for a Day Ahead Rainfall Prediction

Weather forecasting, the science of predicting atmospheric conditions in a specific area, plays a crucial role in shaping people's daily lives. It is not only essential for human activities but also impacts non-living systems, influencing decisions ranging from crisis management to addressing agricultural challenges [1].

In recent decades, meteorological science in India has made significant advancements. However, due to the inherent complexity and chaotic nature of weather variables, accurate forecasting remains a challenge. The unpredictability of atmospheric movements makes it difficult to provide reliable forecasts, even for the following day [2].

Although extreme weather events are rare and typically short-lived, they can be highly destructive, leaving long-lasting impacts on the environment. While preventing such occurrences is impossible, effective mitigation strategies can help minimize their effects. Therefore, prioritizing preparedness, prevention, and rescue operations is essential.

The origins of contemporary numerical weather prediction (NWP) may be traced to the 1920s [3]. Today, it is a crucial instrument in economic planning for several industries, including transportation, logistics, agriculture, and energy production. These forecasts have been crucial in saving many lives by giving early warnings of hazardous weather occurrences, with their accuracy having considerably improved over the years. The improvement in weather prediction can be due to better processing power, a better comprehension of micro-scale processes, and higher-quality atmospheric observations, which result in better model initialization through data assimilation. However, NWP also has some challenges and limitations [4]. Nonetheless, this challenge also allows machine learning (ML) models to learn patterns from data. Weather prediction can also be considered a Big Data Analytics issue. Continuous training for ML model often requires enormous data.

The ML techniques used by researchers for weather forecasting differ in many ways. They vary in the way weather data is used, the way in which the weather or climate data is processed, input variables (univariate/multivariate) used for predicting a variable, the scale of prediction, and the region for which the prediction is being made. Some researchers have used classification approaches such as the temperature will be high or normal or low; it is raining or not raining [5]. Some researchers have used a regression approach to predict weather variables. Hence, the results they obtained cannot be compared fairly.

This paper focuses on one weather variable, i.e., rainfall, and fully utilizes ML's potential for forecasting day-ahead rainfall conditions. ML has revolutionized several sectors and is the perfect route for enhancing prediction efficiency and accuracy in weather forecasting. ML is a technique for automating the learning of a task and for enhancing performance by iteratively improving the learning process. The objective of this study is to explore the basic ML techniques and figure out the correlation between all these ML techniques in forecasting rainfall patterns. The algorithms used here are MLR, DT, RF, and K-NN. The performance of all the models has been analyzed on the basis of MAE, RMSE, and correlation coefficient (R-value). One more parameter for comparison is considered here is the capability of the model to predict the maximum rainfall. As the rainfall pattern varies from place to place, two different regions on the same longitude but on different longitude are considered here to gauge how well ML techniques can predict a day ahead rainfall.

The paper is organized into five sections. After the introduction, the second section reviews the literature; the third section details the methodology. The methodology section covers the dataset description, and a brief description of the ML techniques used in the work. Results are discussed in the fourth section, and the fifth section concludes the paper.

3.1 Collection of data

The rainfall data used here is made freely available by IMD, Pune [22]. This is gridded daily rainfall data with a high spatial resolution of 1.0° latitude by 1.0° longitude. The dataset over India is available from the year 1901 to 2023. The data is available as a NetCDF file for every year. The data was downloaded for the years 2015 to 2023. The rainfall data provided by IMD, Pune, has three dimensions: Longitude, Latitude, and Time. The data is arranged in 35 × 33 grid points, i.e., longitudes and latitudes. The first record in the data is at 6.5 N and 66.5 E, and the last record in the data is at 38.5 N and 100.5E. The unit for the rainfall data is in millimeters (mm), and the time is 1st January to 31st December for each year. The data was downloaded for 9 years, starting from 2015 to 2023.

3.2 Data visualization and preprocessing

Panoply software, an open-source software for climate data visualization provided by NASA, is used to visualize rainfall data. Panoply helps extract data from NetCDF files in CSV format. The software facilitates visualization on a world map in various types of maps. The data was visualized by creating a georeferenced latitude-longitude plot with color contour. It was adjusted to the Equirectangular Regional projection to closely observe the Indian region. The sample view of rainfall on 27^th July 2023 is shown in Figure 1.

1.png

Figure 1. Visualization of rainfall on 27th July 2023 using Panoply

For rainfall prediction, data acquisition and preprocessing is highly important. Rainfall data can be very difficult to preprocess for ML models as, for a major part of the year, there is none to very minimal rainfall in India. Major rainfall occurs during the monsoon season, which usually starts in June and lasts until September. Therefore, the NaN values needed to be addressed properly so that they didn't negatively affect the models and hinder the prediction process. Another important aspect with the data was multidimensionality of the data.

After extracting the data from NetCDF files to CSV files using Panoply, Python libraries combined the CSV data files of all the years into a single data frame. Data from the years 2015 to 2023 were used. The data from all files was combined to form a single file with daily rainfall data over the above-mentioned latitude and longitude. This single data frame was then further preprocessed. For the latitudes and longitudes considered for this study, the NaN values were first identified, and number of such values was counted. The percentage of NaN values was found to be as negligible as 0.56% and hence they were changed to zero in the data frame. NaN values are not assumed or interpolated as it would have affected the training [19]. The sample data frame after preprocessing is shown in Table 1.

Table 1. Sample data frame after preprocessing

The data was further processed to remove the records of latitudes and longitudes other than the desired latitude and longitude. Two regions were considered for the study. The first was rainfall over 18.5 N and 73.5 E, and the second region was 15.5 N and 73.5 E. These regions are from Maharashtra and Gujarat states of India, respectively. Figure 2 shows the regions on the map.

2.png

Figure 2. Regions for rainfall prediction

After the preprocessing of data, various ML models were employed to predict rainfall of the next day. The methodology adopted for this study is as presented in Figure 3.

3.png

Figure 3. Methodology

The prediction models are described briefly in the following subsections. The performance of all the models deployed in the study was compared using various parameters such as mean absolute error, root mean squared error, correlation coefficient, and the maximum rainfall prediction capability of the model.

3.3 MLR model

Linear Regression is the most basic algorithm that provides a straightforward approach to modeling relationships between variables. The simplicity of linear regression provides us with the baseline model for predicting rainfall based on linear relationships.

Regression is a method used to examine the relationship between an independent variable or variables (X) and a dependent variable (y). The dependent variable is also called the response variable. Here, time series forecasting is done where an earlier day's rainfall is considered an independent variable to predict the next day's rainfall. The number of independent variables, if increased, is called MLP. Mathematically, it can be defined as shown in Eq. (1).

$Y=\beta_0+\beta_1 X_1+\beta_2 X_2+\beta_3 X_3+\varepsilon$               (1)

where,

Y is the dependent variable (rainfall).

β₀, β₁, β₂ are the coefficients to be estimated.

X₁, X₂, X₃ are the values of the independent variables (previous 3 days’ rainfall).

ε represents the error term.

The outcome, Y, represents the expected quantity of rainfall based on the linear relationship between the time variables provided. The goal is to reduce the residual sum of squares (RSS), which represents the sum of the squared variations between the rainfall's actual values and those predicted by the MLR model.

Mathematically, it is as given in Eq. (2) [23].

$\operatorname{RSS}\left(\beta_0, \beta_1\right)=\sum_{i=1}^n\left[y_i-\left(\beta_0+\beta_1 x_i\right)\right]^2$             (2)

The MLR model looks for a linear equation that best fits the data to make predictions. Thus, the model's goal is to forecast the amount of rainfall likely to happen at a specific site the next day.

3.4 DT

The DT is a non-parametric supervised learning technique used in regression and classification applications. Its internal nodes, leaf nodes, branches, and root nodes make up its hierarchical tree structure. This arrangement enables a sequential decision-making process.

An important component of the DT algorithm is entropy. It is a measure used at each node in the tree (t_i) to decide how the data should be divided. It is used to quantify disorders or impurities in a dataset. Entropy quantifies the degree of uncertainty or unpredictability surrounding the target variable. The decision tree uses entropy analysis to produce splits that maximize information gain and, as a result, produce classification or regression results that are more exact and accurate.

In a regression problem, node t_i gives a prediction of the value of the dependent variable equal to the average value of this variable y for the elements of the training set assigned to t_i [24].

Mathematically, it is defined as given in Eq. (3).

$\bar{y}_l=\frac{1}{N_i} \sum_{n=1}^{N_{\text {train }}} \mu_i\left(x_n\right) y_n$           (3)

where,

$\mu_i(x)$ is a membership function by which node t_i is characterized.

$N_i$ is the number of training samples

The tree is built using a divide-and-conquer approach, where the attribute space is divided into non-overlapping regions through a series of Boolean tests arranged in a hierarchy. Each region simplifies the decision problem, and each test in this hierarchy represents an internal node in the decision tree [24].

DT Regression aims to construct a tree structure for accurate rainfall forecasts using the previous day's rainfall. A parameter called max depth determines how complex the tree is. While deeper trees may prevent overfitting, they do catch finer rainfall distinctions. A suitable max depth is selected by calculating the Mean Squared Error (MSE) at various depths. To provide precise forecasts for rainfall based on geographic coordinates, this balance enables the model to identify important trends without overfitting. The depth of DT used in the model was 3 where minimum MSE was obtained. DT also helps us understand and visualize the decision-making process more easily, which helps us gain valuable insights into the contributing factors to rainfall conditions.

3.5 K-NN

K-NN is another supervised ML technique that is nonparametric [25]. K-NNs are also helpful in dealing with the complexities of rainfall patterns as this non-parametric algorithm doesn’t make underlying assumptions about the data distribution. K-NN uses Euclidean, Manhattan, and Minkowski distance measurements to classify data. Studies reveal that when K-NN is used on datasets with a smaller feature set, it performs better. The variables x_ij and x_io are used to calculate the Euclidean distance. x_ij stands for the i^th data point in the j^th predictor, and x_io for the i^th predictand (Eq. (4)).

Mathematically,

$d_j=\sum_{i=1}^n\left(x_{i j}-x_{i o}\right)^2$              (4)

$Z_r=\sum_{k=1}^K f_k\left(d_j \times Z_k\right)$             (5)

The idea of Euclidean distance is presented in these formulas, which measure the difference in similarity between data points. In addition, Z_r and Z_k represent the expected and neighboring data, respectively, while f_k(d_j) is the kernel function, a key element of the K-NN method. These mathematical formulas are incorporated to highlight the complexity of K-NN (Eq. (5)), a flexible method that is frequently used for data classification. For regression using K-NN, prediction is the average of the K-NN outcome [26].

Every data point's location is determined by its previous day's rainfall amount, and the recorded precipitation is indicated by the related rainfall. By iterating through various neighbor counts (k), ranging from 1 to 20, the performance of the K-NN regression model is assessed. The model is trained and tested, and the MSE is calculated for each k. The MSE results are graphically displayed to help choose the best k value. The K-value used for the model was 9. This helps to shed light on the trade-off between bias and variance for accurate rainfall predictions.

3.6 RF

qRF is a popular ensemble learning method that works well for regression and classification problems. To reduce overfitting and improve prediction accuracy, it combines several decision trees. Two important variables contribute to RF's "random" nature, which makes it unique. First, it uses random feature selection for every tree in the ensemble. Let's say there are M features in total. For every tree, a random subset of m features is chosen; m is usually significantly fewer than M. Only these m features are considered when building each tree in terms of node splits.

Secondly, RF introduces randomness by using random subsampling of the training data. Each tree in the ensemble is trained on a different subset of data points. Thirdly, the final prediction in an RF ensemble is calculated as the average of the predictions from all individual trees as given in Eq. (6).

Final Prediction $=\frac{1}{N} \sum_{i=1}^N \operatorname{Tree}_i$           (6)

where,

Final Prediction: Represents the overall prediction made by the RF.

N: The total number of decision trees in the RF.

Tree_i: The prediction made by the i^th decision tree in the ensemble.

RF generates a more accurate and dependable overall prediction by averaging the guesses. Multiple trees’ predictions work together to reduce individual errors and variations, creating a regression model that is both reliable and strong. An RF Regressor model is applied to forecast rainfall based on temporal data. The dataset is divided into training and testing sets for model evaluation. The initialization, training, and use of a RF Regressor to forecast rainfall values on the test set. Finally, the model's performance is assessed using various evaluation parameters.

3.7 Performance evaluation

In the last stage, the performance of all the models was evaluated on MAE, RMSE, R-value and the maximum rainfall prediction capability of the model for the test set. The formula for calculating MAE is given in Eq. (7) [27].

$M A E=\frac{1}{n} \sum_{i=1}^n\left|e_i\right|$             (7)

where, n is the number of samples, e is the error calculated as the difference between observed and predicted values. MAE is an appropriate measure of performance evaluation over RMSE as RMSE increases with increased variance in error [27] The formula to calculate RMSE is given in Eq. (8).

$R M S E=\left[\frac{1}{n} \sum_{i=1}^n\left|e_i\right|^2\right]^{1 / 2}$             (8)

Microsoft Excel was utilized to compute R-value. The formula for the correlation coefficient is given in Eq. (9).

$R-$ Value $=\frac{\sum(x-\bar{x})(y-\bar{y})}{\sqrt{\sum\left(x-\bar{x}^2\right)} \sqrt{\sum\left(y-\bar{y}^2\right)}}$           (9)

where, x and y are two arrays of samples and x̅ and y̅ are the mean of all values of x and y, respectively. In each example, the maximum rainfall capacity of the network was estimated independently for the testing datasets. This indicates the observed maximum rainfall during the testing period and the maximum predicted rainfall during the same period. This parameter assisted in evaluating the network's ability to predict excessive rainfall.