Assessing Tourism Prediction Models: A Comparative Study of SARIMA, Random Forest, and LSTM, Considering Nonlinear Trends and the Influence of COVID-19

Econometric models, rooted in established statistical and economic principles, are frequently chosen for domains with relatively stable relationships, providing robust and interpretable results. Notably, among these econometric models, ARIMA (Auto-Regressive Integrated Moving Average), conceived in the 1970 [12], has emerged as a particularly valuable tool for forecasting tourist arrivals [13, 14]. On the other hand, AI models, particularly ML and DL, excel in capturing complex trends and nonlinear patterns, making them well-suited for dynamic and unpredictable data.

Tovmasyan [15] employed the ARIMA model to forecast travel patterns in Armenia during the COVID-19 epidemic. Time series data from 2015 to 2020 were analyzed, and the results showed a significant drop in tourism from 1,894,377 in 2019 to 375,216 in 2020. The average annual growth rate for 2021–2023 is projected to be 12.81%, 13.42%, and 13.66%, respectively, indicating a positive trend in the recovery of tourism. Despite these optimistic forecasts, the research acknowledges several limitations. Notably, it does not include data from 2020 because of the sudden onset of the pandemic, which might distort the results if not appropriately adjusted for context. Although this research provides valuable insights into tourist forecasting, it also emphasizes the need for continuous data gathering and methodological improvement.

Makoni et al. [16] utilized the SARIMA model to project international tourist arrivals in Zimbabwe, yielding precise predictions with favorable RMSE and MAPE values. Nonetheless, the research has several limitations. The forecasts do not account for potential disruptions caused by global events such as COVID-19, which could significantly affect tourism trends. Additionally, the model assumes data stationarity, which may not fully capture the dynamic nature of tourism influenced by changing market conditions and external factors. The SARIMA approach, while effective, may not capture all the complexities of contemporary forecasting techniques, which could enhance accuracy and adaptability.

Qiu et al. [17] conducted a study on predicting tourist arrivals for 20 countries using various predictive models. They found that the SARIMA, ETS, and STL models were the most accurate among single models. While stacking models, which combine multiple methods, generally provide higher accuracy, the study found that combining five single models was optimal for their specific dataset. A key limitation of this research is that the optimal number of models for stacking may not be universally applicable, potentially affecting the generalizability of the results. Additionally, the study might not fully account for external factors which could impact the accuracy of predictions in real-world scenarios.

In a comparative study by Bouhaddour et al. [13], the effectiveness of the SARIMA and PROPHET models in forecasting tourism in Singapore was evaluated using historical data on arrivals. The first model is known for its ability to capture both seasonal and non-seasonal trends, whereas the second model is designed to handle changes in trends and seasonality. The study employed performance metrics such as the mean absolute error and root mean squared error to assess the models' forecasting accuracy. The results indicated that the PROPHET model provided more accurate predictions than the SARIMA model. These findings are significant for enhancing tourism demand forecasting and assisting policymakers and industry stakeholders in making better-informed decisions.

Wu et al. [18] introduced an innovative forecasting approach by merging Seasonal Autoregressive Integrated Moving Average (SARIMA) with Long Short-Term Memory (LSTM), resulting in a hybrid model that integrates both econometric principles and deep learning capabilities. This innovative method was created specifically to forecast daily visitor arrivals in Macau Special Administrative Region, China, quickly. Long-term patterns in time series data are efficiently captured by LSTM, an AI technique renowned for its nonlinearity. The hybrid model of SARIMA + LSTM outperformed other predictive algorithms by combining the predictive powers of SARIMA with the capacity of LSTM to reduce residuals. The tourist industry's best practices are advanced by this study, particularly with regard to making accurate daily arrival estimates.

Bouhaddour et al. [19] present the PROPHET-BGP-FNN model as a superior approach for forecasting daily tourist arrivals in Hawaii, outperforming traditional models such as SARIMAX, PROPHET, and LSTM in terms of predictive accuracy. By combining PROPHET’s trend and seasonality detection with the BGP-FNN’s ability to capture complex nonlinear patterns, the model achieves lower MAE and mean MAPE values, demonstrating its effectiveness in handling the nuances of tourism data. However, the study is limited by its focus on Hawaiian data, which constrains the model's applicability to other regions and longer time series. Despite these constraints, the PROPHET-BGP-FNN model holds promise for enhancing decision-making in tourism and other sectors reliant on accurate forecasting.

This study offers a novel and comprehensive approach to Time Series Forecasting (TSF) by integrating and comparing econometric, ML, and DL models using an extensive dataset from Singapore that spans from January 2008 to August 2023, including the COVID-19 pandemic period. Unlike many studies that exclude the COVID-19 period from their datasets to simplify predictions, this research retains this complex period in the analysis. By including the pandemic period, the study acknowledges its impact on tourism forecasting and aims to provide a more accurate and realistic evaluation of model performance. The study’s distinctiveness lies in its methodological rigor, particularly the decomposition of data into three distinct testing periods that reflect Singapore's evolving travel conditions during the pandemic. This approach allows for a nuanced evaluation of model performance across varying conditions, which is crucial for capturing the effects of unprecedented disruptions like COVID-19.

The research advances the field by demonstrating how different TSF models, including advanced DL models, can be evaluated under diverse conditions to enhance forecasting accuracy. By employing a robust set of evaluation metrics, such as the coefficient of determination (R²) and Root Mean Square Error (RMSE), the study provides new insights into the performance of these models across different phases of the pandemic. The empirical results reveal that DL models show particularly promising outcomes, suggesting that these models may offer superior forecasting accuracy in complex and dynamic environments. This contribution is significant as it highlights the effectiveness of integrating multiple TSF methodologies and adapting them to evolving conditions, while also emphasizing the importance of including all relevant periods in the dataset to avoid skewed predictions. This approach offers valuable guidance for improving tourism demand forecasts and aids decision-making for policymakers and industry stakeholders.

The manuscript adheres to a well-structured organization. Section 3 offers an elaborate elucidation of the methodology. Section 4 describes the procedure, including data preparation, model implementation, and a detailed analysis of the empirical results. The concluding observations are presented in Section 5.

4.1 Data repartition

In order to construct and refine the model, the dataset was divided into two segments: a training set and a testing set. This data partitioning was based on the observed trend in tourist arrivals, characterized by fluctuations reflecting both upward and downward movements. These fluctuations were particularly pronounced during the challenging period of the COVID-19 pandemic (Table 2).

4.2 Implementation of SARIMA

The initial step involves identifying the SARIMA model with suitable values for p, d, q, P, D, and Q. In time series modeling, it is uncommon for a statistical model to perfectly capture the data-generating process; some information is inevitably lost. The objective, however, is to minimize this loss of information as much as possible. Akaike's Information Criterion (AIC) quantifies the amount of information lost by a given model. The AIC is mathematically represented by Eq. (3):

$A I C=-2 * \ln (likelihood)+2 *(p+P+q+Q+d+D)$     (3)

The AIC penalizes models with more parameters by adding a term 2(p+P+q+Q+d+D), which encourages simpler models while ensuring a good fit to the data. The final step in the modelling process is to assess the residuals to confirm that the SARIMA model provides the best fit.

Based on the data presented in Table 3, the SARIMA model (3, 2, 0) (1, 0, 1) has the lowest AIC value. Therefore, it is the optimal model, offering the best fit for forecasting tourism receipts in Singapore.

The analysis of the residuals from the SARIMA model (Figure 2) provides valuable insights into the model's adequacy and the behavior of the residuals. It is evident from this analysis that the residuals show minimal temporal dependence, as indicated by the near-zero autocorrelation values across various lags. This suggests that the residuals are not exhibiting significant patterns of correlation over time, which is a positive indication that the model has captured the underlying dynamics of the data well.

Table 2. Data partitioning by period and rationale

Table 3. Akaike information criterion (AIC) for different SARIMA models

Additionally, the residuals fluctuate within a relatively narrow range, approximately between -150,000 and 150,000, which is noteworthy given that the original dataset's values range up to nearly 2,000,000. The limited variability in the residuals' magnitudes, relative to the data's scale, suggests that the model has effectively captured a significant portion of the variability in the observed data.

These findings indicate that the SARIMA model has effectively accounted for temporal dependencies in the data, resulting in residuals that are largely uncorrelated and within a reasonable magnitude range compared to the original dataset. This reinforces the adequacy of the SARIMA model for the given time series data.

2.png

Figure 2. Residuals from SARIM (3,2,0)(1,0,1)

Table 4. Evaluating SARIMA model performance with different data decompositions

Drawing on the findings presented in Table 4, it is clear that the adapted SARIMA model utilizing the second and third data compositions performed exceptionally well in predicting tourist arrivals, surpassing the performance of the first data composition. This outcome reaffirms the reliability of SARIMA as a robust model for time series forecasting. Specifically, the models trained on the second and third training sets achieved the highest R² = 0.92, demonstrating a close match between forecasted and actual values.

These results highlight the importance of training models with ample and diverse data, particularly data that accounts for unexpected events like the COVID-19 pandemic, which enhances model accuracy. As emphasized in the previous study [35], researchers should focus on developing predictive models capable of handling unforeseen circumstances. In conclusion, the SARIMA model trained on the dataset that includes the COVID-19 pandemic period emerges as the most effective predictor, based on the results presented.

4.3 Implementation of RF

In this part, we present an implementation utilizing the Random Forest machine learning model for making predictions on a tourist time series dataset. The dataset was preprocessed, and the Min-Max Scaler was applied for normalization. The dataset was then divided into training and testing sets as showed in Figure 2. The dataset was then divided into training and testing sets, with appropriate data formatting for training using a specified look-back value. Subsequently, the Random Forest model was constructed and trained with 100 estimators.

Table 5. Evaluating RF model performance with different data decompositions

Considering the results presented in Table 5, the first data decomposition, accounting for 76% of the dataset, exhibited a higher RMSE and a negative R2 of -0.27. This suggests that the model faced challenges in accurately predicting tourist arrivals during this phase. The negative R-squared value indicates that the model did not fit the data well in this decomposition, due to the abrupt changes and unpredictability brought about by the onset of the pandemic. In the second data decomposition, covering 78% of the dataset, the RMSE remained high, indicating notable prediction errors. The R2 value further decreased to -0.67, signifying a weaker fit of the model to the data. This decline in predictive performance attributed to the lingering impact of the pandemic, with tourism still struggling to recover fully. However, in the third data decomposition, encompassing 87% of the dataset, the model showcased significantly improved performance. The RMSE notably decreased, indicating more accurate predictions compared to the earlier decompositions. The R2 value of 0.90 revealed a strong fit of the model to the data. This indicates that the model captured the underlying patterns and trends effectively during a phase of recovery and stabilization in the tourism industry, possibly after the pandemic's peak impact. Overall, these results underscore the importance of considering the temporal dynamics of tourism data, especially during exceptional events like the COVID-19 pandemic. The variations in predictive performance across different data decomposition phases highlight the necessity of adapting forecasting models to changing patterns and behaviors in the tourism sector. As tourism evolves over time, models need to be recalibrated and refined to accommodate these changes effectively, ensuring accurate predictions and informed decision-making for stakeholders in the tourism industry.

4.4 Implementation of LSTM

In the third part of our analysis, we utilized the LSTM (Long Short-Term Memory) model, a type of recurrent neural network known for its effectiveness in capturing long-term dependencies in sequential data [32]. The LSTM architecture consisted of a key LSTM layer with 4 units, followed by a Dense layer with a single unit. This LSTM layer played a crucial role in enabling the model to understand and recognize patterns over different time intervals [36]. The choice of using 4 units was determined through empirical experimentation, and this parameter can be further adjusted based on the complexity and characteristics of the dataset.

To prepare the data for the model, we applied a scaling technique known as Min-Max scaling. This method transformed the original data values (X) into a normalized range of [0, 1], using the minimum and maximum values of the variable. This step is fundamental for enhancing the model’s convergence. The formula for Min-Max Scaling is given by Eq. (4):

$X_{\text {scaled}}=\frac{x-X_{\min}}{x_{\max}-X_{\min}}$      (4)

where,

• X: is the original value you want to scale;
• X_min: is the minimum value of the variable;
• X_max: is the maximum value of the variable;
• X_scaled: is the scaled value in the range [0, 1].

The dataset was carefully divided into separate training and testing sets, as shown in Table 2. These sets were then converted into sequential time series, where each data point was linked to its preceding value. This transformation was essential, as it allowed the model to capture temporal patterns inherent in the data.

The LSTM model, with its custom architecture and carefully chosen parameters, was designed to effectively understand the intricate temporal dependencies within the data. The model was trained over 40 epochs with a batch size of 1. For optimization, we used the Mean Squared Error (MSE) loss function and the Adam optimizer. After training, the model was tested on both the training and testing datasets. Table 6 presents a comparative overview of the LSTM model's performance, showcasing the RMSE (Root Mean Squared Error) and R² values for each data decomposition. The results reveal varying levels of predictive accuracy based on the data decomposition, illustrating the model's ability to capture subtle patterns in tourist arrivals.

Table 6. Evaluating LSTM model performance with different data decompositions

From the results presented in Table 6, the LSTM model's performance during the pandemic years is especially notable. The third data decomposition demonstrated outstanding accuracy, with the lowest RMSE of 89,125.37 and a high R² value of 0.96. This indicates that the LSTM model was able to effectively capture the intricate patterns in the data, even during the tumultuous period marked by the pandemic.

4.5 Discussion

Figures 3-5 provide a comparative overview of the predicted trajectories for tourist arrivals generated by the SARIMA, RF, and LSTM models, in contrast with the actual value trajectory. The analysis of the SARIMA, RF, and LSTM models offers a nuanced understanding of how these forecasting approaches handle temporal patterns, seasonality, and disruptions such as the COVID-19 pandemic. Each model has its own strengths and weaknesses, and their performances provide valuable insights for the tourism industry in Singapore.

3.png

Figure 3. Actual tourist arrivals and predictions of all models for the first data decomposition

4.png

Figure 4. Actual tourist arrivals and predictions of all models for the second data decomposition

5.png

Figure 5. Actual tourist arrivals and predictions of all models for the third data decomposition

Each trajectory corresponds to a specific data decomposition representing different phases of the dataset. The SARIMA model, representing the econometric approach, demonstrates notable accuracy across all data decompositions, effectively capturing the underlying patterns based on statistical analysis. SARIMA stands out for its ability to capture seasonal trends and temporal dependencies effectively. The model’s low RMSE and high R² values across different data decompositions indicate that it successfully models the inherent seasonality and regular patterns in tourist arrivals. The ACF and PACF analysis of the residuals supports this, showing that the residuals are largely uncorrelated, suggesting a good fit. However, SARIMA's performance during the pandemic highlights a limitation: while it can handle regular seasonal variations, it may struggle with abrupt, nonlinear changes. The model’s inability to fully account for the sudden shifts in tourist behavior due to the pandemic demonstrates its challenge in adapting to such extraordinary events. This suggests that while SARIMA is robust for general forecasting, it may require additional adjustments or complementary models to manage unpredictable disruptions effectively.

The RF model, representing a machine learning approach, showcases varying levels of accuracy across different data decompositions. It struggles to predict accurately during the initial phases, particularly in the first data decomposition, due to the abrupt changes and uncertainties introduced by the pandemic. However, it showcases improved performance in the later phases, aligning with the stabilization of the tourism industry. The RF model's ability to capture complex nonlinear relationships makes it a valuable tool in understanding tourism trends. However, its performance varied significantly across different data decompositions. The high RMSE and negative R² values during the pandemic years indicate that RF struggled with the sudden and unpredictable changes in tourist patterns, likely due to insufficient adaptation to the new data dynamics introduced by the pandemic. In contrast, the improved performance in the post-pandemic period reflects RF’s strength in modeling complex patterns once the data stabilizes. This underscores RF’s capability to handle intricate data structures and interactions but also its sensitivity to the quality and continuity of data.

Remarkably, the LSTM model, representing deep learning approaches, consistently performs well across all data decompositions, indicating its ability to comprehend and forecast tourist arrivals even during challenging times. Specifically, it excels in the third data decomposition, showcasing a high level of accuracy and effectively capturing the patterns during the recovery phase post-pandemic. The LSTM's capacity to model long-term dependencies and intricate patterns in the data makes it a powerful tool for time series forecasting, particularly in domains with evolving trends and uncertainties, such as the tourism industry. These findings underscore the resilience and adaptability of the LSTM model, alongside the complementary strengths of both econometric and machine learning approaches, in capturing and predicting tourist arrivals. This makes them promising tools for accurate and insightful forecasting in the volatile tourism sector. The LSTM excels across all data decompositions, demonstrating its robust performance even during the turbulent pandemic period. The low RMSE and high R² values, particularly in the post-pandemic recovery phase, illustrate LSTM's effectiveness in capturing long-term dependencies and adapting to evolving patterns. LSTM’s ability to model intricate temporal patterns and learn from sequential data makes it particularly suited for forecasting in volatile environments. This adaptability is crucial for managing the dynamic nature of tourism data, especially in the wake of unprecedented disruptions like the COVID-19 pandemic.

The implications for the tourism industry in Singapore, based on the performance of SARIMA, Random Forest (RF), and Long Short-Term Memory (LSTM) models, highlight the need for a nuanced approach to forecasting and planning. The SARIMA model proves effective in capturing seasonal trends and regular patterns in tourist arrivals, making it a valuable tool for understanding recurring seasonal variations. However, its performance during the pandemic illustrates its limitations in adapting to sudden, nonlinear disruptions. On the other hand, LSTM's ability to handle complex, dynamic changes makes it a superior choice for forecasting during volatile periods. Its consistent accuracy across different phases, including the post-pandemic recovery, underscores its effectiveness in managing evolving patterns and abrupt changes in tourist behavior.

In practice, integrating the strengths of both SARIMA and LSTM can offer a more robust forecasting framework. While SARIMA’s seasonal modeling capabilities provide a solid foundation for understanding long-term trends, LSTM can address the immediate and unpredictable shifts in data patterns, enhancing the model's responsiveness to extraordinary events. This combined approach can support more informed decision-making, enabling better resource allocation and strategic planning in the tourism sector.

Adapting to future disruptions will require models that not only account for routine fluctuations but also offer resilience against sudden changes. The LSTM model’s demonstrated capacity to handle abrupt disruptions, such as those caused by the COVID-19 pandemic, highlights its value in preparing for and managing future uncertainties. By leveraging the complementary strengths of different forecasting approaches, the tourism industry can develop a more comprehensive and flexible strategy, ensuring better preparedness and adaptability in the face of both regular and exceptional challenges.

Overall, the comparative analysis of SARIMA, RF, and LSTM models emphasizes the importance of a multifaceted forecasting approach. Utilizing these insights allows for more effective tourism management, particularly as the industry continues to recover and adapt to new challenges.