Comparative Analysis of LSTM, ARIMA, and Hybrid Models for Forecasting Future GDP

Forecasting time series accurately is a critical challenge with significant implications, especially in predicting economic growth. Gross Domestic Product (GDP) is one of the most important economic indicators for comparing and evaluating the economic performance of countries and regions, and economists are constantly in search of better ways to predict GDP [1, 2].

Gross Domestic Product (GDP), a comprehensive indicator, is determined through various macroeconomic measures. These indicators several multiple areas, including rural development, economy and growth, education, energy and mining, environment, financial sector, public and private sectors, and technology. The collective dynamics of these indicators significantly influence the overall performance and fluctuations in GDP [3].

Accurately forecasting GDP is a major challenge due to the complexity of these economic factors. Traditional forecasting techniques such as ARIMA, SARIMA, and ARMA have seen extensive use in predicting GDP time series [4, 5]. However, these methods face limitations and might not be well-suited for non-linear data or time series characterized by complex trends and seasonality.

In recent years, the field of predictive modeling has been transformed by machine learning, particularly through the advancements in deep learning algorithms. These methods have enabled a novel way of understanding the interactions between variables by employing a sophisticated, multi-layered structural approach [6]. Among these, machine learning techniques like Support Vector Machines (SVM) and Random Forests (RF), as well as deep learning algorithms including Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks, have drawn significant interest. These techniques are widely applied across different fields, especially in finance [7-9]. Deep learning methods excel at recognizing the structure and patterns in data [10], including non-linearity and complexity, particularly within the context of time series forecasting. In particular, the LSTM architecture, developed in 1997 by Hochreiter and Schmidhuber [11], offers a more fitting solution. This architecture, an extension of the recurrent neural network (RNN), is capable of handling the temporal nature of sequential data.  LSTMs excel at capturing long-term dependencies and non-linear relationships in time series data [12-16], making them powerful tools for forecasting GDP time series.

This article investigates whether hybrid models can outperform traditional forecasting methods for GDP time series data. It will model GDP data and examine three methods. The first approach is a connectionist method using a Long Short-Term Memory (LSTM) network, which is capable of self-learning and capturing long-term dependencies. The second method is a statistical approach, employing an Autoregressive Integrated Moving Average (ARIMA) process, based on the Box and Jenkins methodology, recognized for its effectiveness in modeling time series dependencies. Finally, we will examine a hybrid ARIMA-LSTM model that combines the strengths of both approaches.

The novelty of our work lies in the proposal of a hybrid approach to GDP forecasting, an approach not employed in the existing literature. This approach combines the seasonal decomposition method for time series analysis, thus combining the strengths of statistical techniques and neural networks. Our study focuses on prediction from the quarterly GDP dataset, with the aim of evaluating the performance of three distinct approaches: LSTM, ARIMA and the hybrid model.

The organization of this article is as follows: Section two contains a review of the literature on GDP time series forecasting methods, touching on traditional, machine learning, and hybrid approaches. Section three dives into the specifics of LSTM and ARIMA models and their implementation in time series forecasting. Section four provides details on the dataset used and the methodology for evaluating the accuracy of the LSTM, ARIMA, and hybrid models. The results of the three models are presented in section five. In section six, there's a discussion on the findings, including a comparison of the models' performances against some previous works. The conclusion highlights the primary insights of the study.

In the field of economics, ARIMA and LSTM models offer powerful frameworks for the analysis and forecasting of time series data in economics. By capturing the intricate dynamics of economic variables, these models facilitate the prediction of trends and patterns, contributing significantly to economic analysis and decision-making processes. A brief background on ARIMA and LSTM will be presented in this section.

3.1 ARIMA

The ARIMA (Autoregressive Integrated Moving Average) model is primarily used for forecasting a single time series variable by encapsulating it within a framework that integrates autoregressive (AR) and moving average (MA) processes [28]. The foundational equation for the AR component is illustrated below:

$x(t)=i=1 \sum p \alpha i x(t-i)$       (1)

where, t is the time step, x(t) the predicted value, p the number of AR terms, and α the AR coefficients. The moving average (MA) part is:

$x(t)=i=1 \sum q \beta i \epsilon(t-i)$      (2)

with q as the MA terms count, β the MA coefficients, and ϵ the error term. Integrating both, the ARIMA model can be expressed as:

$x(t)=i=1 \sum p \alpha i x(t-i)+i=1 \sum q \beta i \epsilon(t-i)$     (3)

3.2 LSTM

LSTMs, represent an advanced variant of Recurrent Neural Networks (RNNs), engineered specifically to address the issue of vanishing gradients, a significant limitation identified in conventional RNNs by Hochreiter and Schmidhuber [29] in 1997. By incorporating a system of gates, LSTMs can smartly decide which information to keep and which to let go, allowing them to remember important data over long periods without losing track due to the vanishing gradient problem. This gate system includes three key components: an input gate to add information to the memory cell, a forget gate to remove unneeded data, and an output gate to control the flow to the next network layer [22]. This setup makes LSTMs particularly adept at forecasting and analyzing sequences with long-term dependencies, outperforming traditional RNNs that struggle with these tasks. The LSTM cell, as depicted in Figure 1.

1.jpg

Figure 1. The LSTM cell, as conceptualized by Hochreiter and Schmidhuber in 1997 [30]

In an LSTM model, input data and previous output are processed at each time step. The model's hidden output, ht, is calculated using specific equations for the forget gate (Ft), input gate (It), and output gate (Ot). These components control the flow of information, allowing LSTMs to retain relevant data over time and address issues like vanishing gradients.

$F t=\sigma\left(w_f x_t+w_{h f} h_{t-1}+b_f\right)$      (4)

$I t=\sigma\left(w_i x_t+w_{h i} h_{t-1}+b_i\right)$    (5)

$O t=\sigma\left(w_o x_t+w_{h o} h_{t-1}+b_O\right)$    (6)

$c t=F_f c_{t-1}+I_t \tanh \left(w_c x_t+w_{h c} h_{t-1}+b_c\right)$    (7)

$h t=O_t \tanh \left(c_t\right)$    (8)

In the equations, σ denotes the sigmoid activation function, w terms stand for the weight matrices, and b signifies the biases in the model. Here, ht−1 is the output from the previous step, x_t is the present input, ct−1 represents the previous memory cell's state, and c_t indicates the current memory cell's state. These elements are crucial for calculating the LSTM model's operations, facilitating the management and update of information over time.

Long Short-Term Memory (LSTM) neural networks are extensively applied across a variety of fields, chiefly for their adeptness in managing sequential or temporal data. These networks are particularly valued for their capability to retain information over long periods, thus adeptly surmounting issues such as gradient vanishing or saturation, which are prevalent in traditional recurrent neural networks (RNNs) [31]. In our research, aiming to boost the LSTM model's efficacy in predicting GDP, we adopted the He initialization method. This approach is especially apt for LSTM models that employ the Rectified Linear Unit (ReLU) activation function, especially relevant in the analysis of time series data. The application of He initialization [32] has been instrumental in averting neuron saturation, culminating in marked enhancements in the model's forecasting accuracy for GDP.

The LSTN and ARIMA models underwent training with US quarterly GDP data covering the period from 1947 to 2022, acquired from the Federal Reserve Economic Data. Table 1 illustrates examples of the dataset used. The primary aim was to utilize historical data for forecasting future quarterly GDP values.

Table 1. GDP Data of US [33]

5.1 Arima model result

Stationarity testing was conducted on the GDP time series. Initial examination of the data involved time plots, depicted in Figure 4, which visually illustrate the non-stationary nature of the US GDP series.

Two differences were taken to eliminate any seasonality and trend in the data. Subsequently, the augmented Dickey-Fuller test was applied to the differenced time series, revealing a p-value of 0.00. This result, as depicted in Figure 5, confirms the attainment of stationarity in the series.

Figure 6 presents the Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots for the GDP time series. The ACF coefficients signify the correlation between the time series and its various lags, while the PACF coefficients provide insights into the partial correlations relevant for ARIMA model identification. The selection of the most suitable model will be determined based on the criteria of minimizing the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC) values. The results of this model comparison and selection process will be presented in the following Table 2.

4.jpg

Figure 4. Plot for US GDP data

5.jpg

Figure 5. Time series plot of quarterly GDP -at second difference

6.jpg

Figure 6. ACF and PACF plot for the time series of the Quarterly GDP

Table 2. The AIC and BIC values for tentative ARIMA

Based on the results presented in the Table 2 above, it is evident that the ARIMA (1,2,1) model is the optimal choice, as it exhibits the lowest AIC and BIC values. Therefore, the ARIMA (1,2,1) model is selected for the purpose of forecasting future values in this study. Depending on the ARIMA model, model fitting is performed on the quarterly GDP data, and the results of the error estimation studies are presented in Table 3. To improve the accuracy of the ARIMA model's predictions, we will investigate optimization strategies involving the combination of different models.

Table 3. Validation metrics of the ARIMA mode

5.2 LSTM model result

In this phase of the study, we utilized a Long Short-Term Memory (LSTM) architecture based on time series GDP data, the model utilized the HE initialization technique to initialize the weights and Earlystopping regularization to prevent overfitting [34], The model incorporated GDP data from the preceding three quarters as input, with the GDP value of the current quarter serving as the target output. This sequential approach was consistently applied across the dataset, as outlined in Table 4. Subsequently, the model's performance was evaluated on a test set comprising 20% of the total dataset. Additionally, Table 5 presents the configuration parameters for the LSTM models developed in this study, providing insights into the model setup and optimization choices.

Table 4. Sequence dataset

Table 5. LSTM learning parameters

Training of the model was conducted using historical data spanning from 1947 to 2007, while validation of the model was performed on separate validation data spanning from 2007 to 2022, as illustrated in Figure 7. The LSTM network prediction is based on the principle of regression. This means that a predicted value at a given time (t) is the result of learning from all previous values up to time (t-1).

7.jpg

Figure 7. The learning process of LSTM model

The LSTM model demonstrated high accuracy in forecasting quarterly GDP values, achieving minimal error metrics on the test set with MSE=0.010, MAE=0.077, and RMSE=0.104. Its predictions closely aligned with actual GDP figures, as shown in Figure 8, underscoring its effectiveness and reliability in economic forecasting.

To evaluate the model's predictive capability, we calculated the coefficient of determination (R2). The R2 value quantifies the proportion of variance in the data that is accounted for by the model, was found to be 0.96. This indicates that our LSTM model accurately accounted for 96% of the variance in the quarterly GDP data, signifying that our predictions were remarkably precise and dependable.

8.jpg

Figure 8. Actual vs. predicted LSTM

5.3 Hybrid model result

The parameters utilized in the hybrid model were the same as those in the LSTM and ARIMA models that came before it. The components of trends, seasonality, and residual data from the time series dataset were separated. Using the seasonal decompose function offered by the statsmodels package, this decomposition was carried out. Then, instruction was given to each of these parts.

9.jpg

Figure 9. Actual vs. predicted hybrid

The combined predictions from each component are shown in Figure 9 as the final prediction result, the hybrid model, combining LSTM and ARIMA (1,2,1), produced highly accurate predictions as depicted in Figure 10, closely tracking the movement of the dataset. The hybrid model achieved a significantly lower error rate, measured by MSE=0.0018, MAE= 0.028 and RMSE=0.043. Furthermore, the coefficient of determination (R2) was found to be 0.99.

10.jpg

Figure 10. Actual vs. predicted