A Machine Learning Approach to Gold Price Prediction Using Financial Indicators

The prediction of gold prices is a crucial aspect of financial forecasting, given gold's role as a reliable investment and a key economic indicator. Traditional forecasting models have largely depended on macroeconomic indicators and historical price data.

However, with the emergence of digital currencies and advanced machine learning techniques, opportunities have arisen to enhance prediction accuracy by integrating diverse financial variables, including cryptocurrencies, stock indices, and commodity prices. This section reviews the literature on gold price prediction, emphasizing various methodologies, their comparative performance, and the growing incorporation of novel financial indicators.

Machine learning techniques have been extensively applied across diverse fields for predictive modeling, including financial and material science applications. For instance, previous studies have demonstrated the integration of machine learning with physical simulation tools like COMSOL Multiphysics to design sensors for water salinity prediction and diesel adulteration detection, leveraging genetic algorithms and neural networks for optimal parameter tuning [1, 2]. These applications underscore the potential of combining domain-specific simulations with machine learning to achieve high predictive accuracy and robust performance in dynamic environments. Similarly, hybrid methodologies incorporating neural networks and optimization algorithms have shown promise in enhancing predictive capabilities, particularly when addressing high-dimensional data as in our work [3, 4]. The integration of these advanced techniques serves as a foundation for furthering the scope of predictive modeling in financial and engineering domains.

Early studies in gold price prediction relied on conventional statistical and machine learning models. Weng et al. [5] introduced a genetic algorithm regularization online extreme learning machine (GA-ROSELM), which incorporated variables such as crude oil and silver prices. This model outperformed traditional approaches, including ARIMA, support vector machines (SVM), and extreme learning machines (ELM). Similarly, Chandar et al. [6] demonstrated the superiority of ELM over other models by integrating data from gold, silver, crude oil, and the S&P 500 index. Complementing these findings, studies comparing LSTM, random forest regression, and linear regression identified LSTM models as particularly effective for handling historical price data due to their ability to capture temporal dependencies [7].

Machine learning techniques have further diversified prediction methodologies. The association rules algorithm and the G,M(1,1) model effectively mined influential factors from recent data for precise predictions [8]. Livieris et al. [9] extended these efforts by developing a hybrid deep learning model combining convolutional neural networks (CNN) and LSTM layers. This hybrid model improved performance by effectively capturing both short and long term dependencies. Random forest regression has also emerged as a popular choice, with studies highlighting its superior accuracy over decision trees and linear regression [10].

Comparative analyses of machine learning models have provided valuable insights. Yang et al. [11] evaluated ANN, LSTM, and SVR models, finding that SVR excelled in incorporating cryptocurrency data into gold price prediction. Other research revealed that ARIMA outperformed linear regression and random forest regression for predicting gold prices [12]. The integration of silver prices and S&P 500 indices in random forest regression models has further enhanced prediction accuracy [13]. Advanced hybrid models, such as CNN-RNN combinations, have also been explored, with RNN demonstrating notable performance in gold price forecasting [14, 15]. Studies comparing machine learning models, including linear regression, random forest, and SVM, found SVM to perform best with an R² index near 0.99, emphasizing eigenvalues' impact on prediction accuracy [16]. A review highlighted XGBoost with SHAP values as highly accurate and interpretable, while ANN effectively forecasted gold price fluctuations [17-19]. Ensemble models, such as ARIMAX and Extra Tree Regressor, outperformed individual methods, demonstrating strong predictive accuracy [20, 21]. Sadorsky [22] employed tree-based classifiers, including bagging, stochastic gradient boosting, and random forests, to predict the price direction of gold and silver ETFs, with random forests demonstrating the highest accuracy. Additionally, various ensemble models, such as a hybrid bagging ensemble, were evaluated to forecast the future momentum of gold and silver stock prices, achieving notable prediction accuracy [23]. These results underscore the growing relevance of ensemble approaches in financial forecasting. In a separate study, Vrtagic et al. [24] explored the application of ensemble methods in financial forecasting, further highlighting their effectiveness in improving prediction accuracy and robustness.

The growing influence of cryptocurrencies has prompted researchers to investigate their impact on gold price prediction. Studies have identified dynamic relationships between gold and cryptocurrencies, particularly during periods of financial uncertainty. Adebola et al. [25] employed fractional integration and co-integration techniques to reveal a limited but notable equilibrium relationship between gold and cryptocurrencies. Similarly, research during the COVID-19 pandemic highlighted an increasing correlation between Bitcoin and gold, positioning Bitcoin as "digital gold" [26-28]. Ji et al. [29] further explored the interconnectedness between gold and cryptocurrencies, noting significant volatility spillovers during financial turbulence.

Advanced methodologies have continued to uncover intricate relationships between financial variables. GARCH and copula models have shown time-varying correlations between Bitcoin, gold, and indices such as the S&P 500 [27]. Hybrid models combining cryptocurrency data with traditional indicators have consistently demonstrated improved accuracy, underscoring the evolving role of digital assets in financial forecasting [30-37]. Pearson correlation analyses have also emphasized the predictive value of relationships between cryptocurrency prices and gold [38, 39].

Despite the progress achieved in gold price prediction, most studies have focused on isolated financial indicators or specific cryptocurrencies. Our research addresses this gap by integrating a comprehensive range of variables, including NASDAQ-100 index (^NDX), Bitcoin (BTC-USD), and gold futures (GC=F). The NASDAQ-100 index (^NDX), Bitcoin (BTC-USD), and gold futures (GC=F) are uniquely positioned as predictive indicators due to their ability to capture diverse market dynamics. The NASDAQ-100 reflects economic confidence and risk sentiment, Bitcoin represents the speculative behavior of emerging digital markets, and gold futures provide direct insights into the supply and demand for gold. Integrating diverse financial indicators bridges traditional and emerging markets, offering a holistic and reliable approach to financial forecasting by enhancing predictive accuracy and reducing noise. Our methodology leverages the interconnected dynamics of assets such as the NASDAQ-100, Bitcoin, and gold futures—an integration not previously explored in the context of gold price prediction. This unique combination reinforces the importance of holistic modeling in modern financial forecasting and offers valuable insights for investors and policymakers.

Furthermore, the application of advanced preprocessing techniques—such as the Box-Cox transformation and Principal Component Analysis (PCA)—addresses multicollinearity and ensures dimensionality reduction with minimal information loss. By employing a Genetic Algorithm-optimized Multi-Layer Perceptron (MLP), we deliver a robust and scalable framework that achieves superior predictive performance, paving the way for innovative applications in financial prediction.

The remainder of this paper is structured as follows: Section 2 describes the dataset and preprocessing steps, including Box-Cox transformation and PCA. Section 3 outlines the methodology, focusing on the GA-optimized MLP regression model. Section 4 presents the results, including performance metrics and insights from permutation importance analysis. Section 5 discusses the findings in relation to existing literature and identifies limitations and areas for improvement. Finally, Section 6 concludes the paper and proposes directions for future research.

2.png

Figure 2. Features distribution of input-output data

3.png

Figure 3. Four days and GC price correlation matrix

4.png

Figure 4. Skewness and kurtosis features and target variables

5.png

Figure 5. Skewness and kurtosis features after preprocessing

Figure 4 shows that several features exhibit positive skewness, particularly GC_price, which has a skewness of 0.96, indicating a concentration of lower values with a long tail toward higher values. Additionally, high kurtosis in GC_price (1.97) suggests heavy tails and the presence of potential outliers, which may affect model accuracy. In contrast, some features, such as Interval_X_BTC-USD_High, display low kurtosis, indicating a flatter distribution.

Overall, the data contains some skewed and kurtosis features, with notable correlations that may lead to redundancy. Addressing these through transformations, feature selection, and outlier management is advised to enhance model fit and prediction accuracy. To address this, highly correlated features could be either removed or combined, potentially enhancing model robustness.

The data preprocessing involved applying the Box-Cox transformation, which significantly reduced the skewness of the highly skewed feature GC_price, which had a skewness of 0.958 before the transformation. After applying the Box-Cox transformation, the skewness for this feature dropped to 0.0368, bringing it much closer to normal distribution. The Box-Cox transformation also improved the skewness of other features related to GC=F_High at different intervals, with the skewness values shifting from positive values to values closer to zero (for example, from 0.91 to -0.034 for Interval_1_GC=F_High). The transformation uses a lambda value that varies slightly across different features, indicating that each feature requires a different adjustment to correct for skewness. These lambda values (e.g., -1.68 for Interval_1_GC=F_High) reflect the optimal power transformation that best normalizes the data for each specific feature.

The Figure 5 demonstrates the impact of preprocessing on the distribution of features through Box-Cox transformations. After applying Box-Cox transformations, skewness values are minimized, bringing the distributions closer to symmetry, while kurtosis values move closer to zero, indicating a shift towards a more normal, bell-shaped distribution. This transformation standardizes the data, reducing outliers’ influence and enhancing data consistency, which is essential for model performance and reliability in subsequent analyses. The line represents the density curve or a smoothed representation of the distribution of skewness or kurtosis. It provides a continuous view of the probability density, helping to visualize the general shape and spread of the distribution. The blue bars represent the histogram of skewness or kurtosis values for the features. Each bar shows the count of features (on the y-axis) that fall into specific ranges of skewness or kurtosis values (on the x-axis).

The outlier detection method identified a significant number of outliers in the GC_price data, with all 1103 rows flagged as outliers using the interquartile range (IQR) method. Although Box-Cox normalization effectively reduces skewness, additional steps may be required to address outliers separately to enhance model accuracy.

Finally, after applying scaling (standardization), the skewness remains very low for the transformed data, particularly for GC_price, with a skewness value of 0.0457, which is almost negligible. The kurtosis value (0.367) also indicates that the data distribution is fairly close to normal, as typical kurtosis values for a normal distribution are close to 0. In summary, applying the Box-Cox transformation has successfully reduced skewness and improved the normality of several features, making the data more suitable for machine learning models, however, further work on handling outliers was done with the Principal Component Analysis (PCA). We found that instead of removing data, applying PCA with two components provides a concise representation of the data in the lower-dimensional space defined by the two principal components. The first principal component (PC1) accounts for approximately 78.88% of the variance in the data, while the second component (PC2) accounts for around 17.49%. Together, these two components explain about 96.37% of the total variance. This indicates that most of the information in the data can be captured by these two components, suggesting strong dimensionality reduction. The cumulative explained variance reaching 96.37% with just two components suggests that these two components are sufficient to represent the dataset with minimal information loss, making PCA an effective tool for reducing dimensionality in this case, as shown in the Figure 6. The data frame displays the transformed data in terms of the principal components (PC1 and PC2) along with the original target variable, GC_price. This transformation enables a clearer understanding of patterns and relationships in the data with fewer variables, which is especially valuable for visualization, feature selection, and model efficiency.

6.png

Figure 6. First vs. second principal component data variance visualization

The heatmap presented in Figure 7 illustrates the correlation between the principal compo-nents (PC1 and PC2) and the gold closing price (GC_price), with correlation values ranging from -1 to +1, where values closer to ±1 indicate stronger linear relationships. PC1 demonstrates a strong positive correlation with GC_price (r = 0.79), indicating that it captures a significant portion of the variance in the dataset that is directly related to gold price movements. This suggests that PC1 encompasses dominant underlying features that influence or reflect the behavior of gold prices. In contrast, PC2 exhibits a moderate positive correlation with GC_price (r = 0.57), implying that it provides additional but comparatively less influential information, potentially representing secondary dynamics in the data. As expected, the correlation between PC1 and PC2 is zero, reflecting their mathematical orthogonality inherent to principal component analysis, which ensures that each component encapsulates unique and uncorrelated aspects of the data. The observed relationships validate the effectiveness of PCA as a dimensionality reduction technique in this context, preserving critical information related to gold price prediction while reducing the feature space, thereby enhancing the interpretability and efficiency of subsequent modeling efforts.

7.png

Figure 7. First-second principal component vs gold price

While the Box-Cox transformation and PCA significantly improved data quality, their combined effect on model performance warrants further emphasis. The transformation of the skewed features enhanced the model's ability to learn meaningful patterns by reducing the impact of outliers and non-normal distributions. PCA’s dimensionality reduction allowed the model to focus on the most informative components, effectively addressing multicollinearity without discarding valuable data. These preprocessing steps not only led to the reduction of skewness and kurtosis but also facilitated a more stable learning process by simplifying the dataset while retaining critical variance. This combination of methods directly contributed to the model's high performance metrics, showcasing the critical role that thoughtful data preprocessing plays in enhancing prediction accuracy and robustness in complex datasets like the one used in this study.

8.png

Figure 8. MLP prediction on test data (30%)

The model explains approximately 98% of the variance in gold prices, suggesting that the chosen features are highly informative for predicting gold prices as can be seen in Figure 8. This is a substantial R² value, indicating a strong fit between the model and the actual data. This high R² suggests that the model is highly effective in capturing the patterns in the historical data of gold prices with respect to the selected financial assets. Mean Squared Error (MSE = 551.43) and Root Mean Squared Error (RMSE = 23.48) indicates a moderate average squared difference between predicted and actual values with a relatively moderate MSE of 551.43. RMSE, which is in the same unit as the target variable (gold prices in USD), suggests that the predictions are, on average, off by approximately 23.48 USD. These results point to a reasonable level of predictive accuracy, especially considering that gold prices can fluctuate in the range of tens of dollars per unit. The MAE value is lower than the RMSE, further supporting the notion that the model’s errors are relatively consistent and not excessively influenced by outliers. The MAE value indicates that, on average, the model’s predictions deviate by only about 17.38 USD from the true gold prices. However, the daily price high-low difference average change for the last three years is 26.08 $. This suggests that the model performs quite well for practical purposes, offering a small margin of error that could be acceptable for real-world financial applications.

Permutation importance results reveal that the model assigns the highest importance to the feature PC1 (mean importance of 1.3456), followed by PC2 (mean importance of 0.8107). This indicates that the model relies heavily on the PC1 as the most significant predictor of gold prices, with PC2 also playing a noteworthy role. This makes sense as principal component (PC1) which describes approximately 78.88% and considering that both the stock market and cryptocurrency can exhibit correlated movements with gold, especially during periods of financial instability or market volatility.

The predictions made by the model for the observations that are out of testing and trading closely match the actual gold prices, demonstrating the model's ability to generalize well to unseen data (Table 3). The observed discrepancies seen in Table 3 between predicted and actual values (e.g., predicted 1587.6 USD vs. actual 1604.2 USD for the first observation) are within a reasonable range, further supporting the model's effectiveness in forecasting gold prices. These relatively small prediction errors are consistent with the reported RMSE and MAE values, reinforcing the model’s reliability.

Table 3. Observed discrepancies on the unseen data

The findings of this study align with and expand upon existing research in the domain of gold price prediction. Weng et al. [5] introduced a GA-ROSELM model that leveraged crude oil and silver prices for enhanced predictive accuracy. In contrast, our study utilized Genetic Algorithms to optimize an MLP model, incorporating a more diverse set of indicators, including NASDAQ-100 and Bitcoin, which significantly broaden the model's scope and applicability. Chandar et al. [6] demonstrated the efficacy of ELM models by combining gold, silver, crude oil, and S&P 500 data. Building on this, our work integrates a wider range of financial variables and utilizes PCA for dimensionality reduction, further improving computational efficiency and model performance. While Livieris et al. [9] utilized a hybrid CNN-LSTM model to capture temporal dependencies, our PCA-based MLP offers an alternative approach, prioritizing dimensionality reduction to balance accuracy with reduced computational overhead. Additionally, compared to the tree-based models highlighted by Tripurana et al. [13], which favored Random Forests for gold price prediction, our Genetic Algorithm-optimized MLP achieved superior performance metrics (R² = 0.98), demonstrating its competitiveness. Finally, Ben Jabeur et al. [18] used XGBoost with SHAP values for interpretability and accuracy in forecasting. Our approach simplifies interpretability through the optimization of MLP parameters without compromising accuracy, offering a streamlined yet effective solution for gold price prediction.

The novelty of the proposed approach lies in its holistic integration of diverse financial indicators—namely the NASDAQ-100 index, Bitcoin, and gold futures—combined with advanced pre-processing techniques such as the Box-Cox transformation and Principal Component Analysis (PCA). While previous studies have explored gold price prediction using individual indicators or traditional models, this work distinguishes itself by bridging traditional and emerging financial markets within a unified machine learning framework. The use of the Box-Cox transformation effectively normalizes skewed data distributions, while PCA addresses multicollinearity and reduces dimensionality without significant information loss. Together, these techniques ensure the input data is both statistically robust and computationally efficient for modeling. The exceptional performance metrics validate this innovative approach. The R² score of 0.98 indicates that the model captures approximately 98% of the variance in gold prices, demonstrating the high explanatory power of the selected features after dimensionality reduction. Contextualizing the error metrics against actual gold prices (\$1,604-\$1,985) reveals that the RMSE of \$23.48 and MAE of \$17.38 represent prediction errors of merely 1.2-1.5% relative to the asset's value—particularly noteworthy considering these error margins are smaller than gold's average daily price fluctuation of \$26.08. Analysis of model predictions on unseen data further substantiates this accuracy, with prediction errors ranging from \$3.00 (0.17%) to $37.33 (1.88%), consistently remaining below 2% of actual values. This precision is especially significant given gold's reputation as a complex asset influenced by numerous macroeconomic factors, geopolitical tensions, and market sentiment. This layered preprocessing strategy, combined with the integration of cross-market financial signals, enables the model to uncover deeper relationships and achieve strong predictive performance that could potentially inform trading decisions within gold's natural volatility range. The findings emphasize the efficacy of this multifaceted approach and its contribution to advancing predictive modelling methodologies in financial forecasting.

This study presents a comprehensive machine learning approach to predicting gold prices using a combination of financial indicators, including the NASDAQ-100 index (^NDX), Bitcoin (BTC-USD), and gold futures (GC=F). By leveraging a four-day window of high values and applying data preprocessing techniques such as Box-Cox transformation and PCA, the model achieved significant dimensionality reduction and enhanced data normalization. These steps addressed issues of skewness, kurtosis, and multicollinearity, making the dataset suitable for advanced machine learning techniques.

The Genetic Algorithm-optimized Multi-Layer Perceptron (MLP) regression model demonstrated excellent predictive accuracy, explaining approximately 98% of the variance in gold prices with an R² score of 0.98. Key performance metrics, including a low RMSE (23.48 USD) and MAE (17.38 USD), highlight the model’s ability to generalize well and make accurate predictions even in a volatile market. Furthermore, the principal components PC1 and PC2 emerged as the most influential features, collectively capturing 96.37% of the variance in the dataset.

The findings underscore the potential of integrating diverse financial variables and employing robust preprocessing techniques to improve gold price prediction. This approach bridges the gap between traditional and emerging financial indicators, emphasizing the growing role of cryptocurrencies and stock indices in modern financial forecasting.

While the model performs exceptionally well, further improvements can be achieved by:

• Adding macroeconomic indicators (e.g., inflation rates, interest rates) and market volatility indices to capture broader market dynamics.

• Experimenting with ensemble methods like Random Forests or Gradient Boosting to refine predictions and potentially enhance performance.

• Developing advanced outlier handling methods to mitigate their residual impact on predictive accuracy.

This research lays the foundation for more holistic modeling approaches in financial forecasting, offering valuable insights for investors and policymakers navigating interconnected markets.