Integration of Technical Analysis and Machine Learning to Improve Stock Price Prediction Accuracy

2.1 Dataset description

Data source: The dataset used in this study was obtained from Yahoo Finance, a platform that provides comprehensive data on historical stock prices of public companies. This data includes daily information on the opening price (Open), closing price (Close), highest price (High), lowest price (Low), adjusted closing price (Adj Close), and trading volume (Volume). This research analyzes stocks from various companies listed on global stock exchanges, focusing on stocks with high volatility and large trading volumes.

The data was downloaded in historical format, then processed for further analysis. The time period used in this research is 2019. The selection of this period is based on economic stability before the COVID-19 pandemic, which provides stock price data that is not affected by global crises or extraordinary events. In addition, data from that year is generally complete and includes various trends and relevant economic events, thus providing a more representative picture of stock market dynamics in regular situations.

Data description: Columns contained in the dataset, such as Date, Open, High, Low, Close, Adj Close, and Volume.

The dataset contains historical stock price data with the following columns:

•Date: The date of the stock price data.

•Open: The opening price of the stock on that date.

•High: The highest price of the stock on that date.

•Low: The lowest price of the stock on that date.

•Close: The closing price of the stock on that date.

•Adj Close: The adjusted closing price of the stock on that date.

•Volume: The volume of stocks traded on that date.

The data was downloaded in historical format, then processed for further analysis, as shown in Table 1.

Table 1. First few rows of dataset

Table 1 presents important data needed to analyze stock price movements over a certain period. This study uses data from 2019, chosen because it reflects economic stability before the COVID-19 pandemic. Data from this period provides a picture of stock prices that are not affected by the global crisis, making it suitable for assessing general market conditions. Table 1 shows the first few rows of the dataset, which include daily opening, high, low, and closing prices, as well as trading volume. For example, on July 26, 2019, the trading volume was quite high with little change in the opening and closing prices, whereas on July 29, 2019, no trading activity was recorded. This data supports further analysis of price volatility and trading trends in this study.

2.2 Algorithm formula

Support Vector Machine (SVM) is used for classification by separating two classes using a maximized hyperplane.

Basic SVM formula

$F(x)=w^T x+b$          (1)

Random Forest is an ensemble algorithm consisting of many decision trees that are trained independently, and the result is determined by voting (for classification) or averaging (for regression).

The basic formula of Random Forest:

$1 \operatorname{split}(D)=\arg \max \Delta \mid(F)$           (2)

Neural Networks (NN) consist of layers of interconnected neurons. Each neuron performs a linear operation, followed by a non-linear activation function.

The basic formula of Neural Networks (NN):

$z=w^T x+b$            (3)

2.3 Technical indicators used in machine learning

Technical indicators are used as features in machine learning models to help analyze market movements.

Moving Average (MA)

$E M A(n)=\left(\frac{2}{n+1}\right) \times\left(P_t-E M A_{t-1}\right)+E M A_{t-1}$          (4)

Relative Strength Index (RSI) is a momentum indicator that measures speed and price changes.

$R S I(n)=\frac{100}{1+R S}$            (5)

Moving Average Convergence Divergence (MACD) is an indicator that measures the difference between two moving averages to capture market momentum.

$M A C D=E M A_{12}-E M A_{20}$              (6)

2.4 Data pre-processing

(1) Data cleansing

Handling of missing values and outliers [17]. There are no missing values in the dataset as shown in Table 2.

Table 2. Missing values of dataset

The basic statistics of the dataset provide insights into the range and distribution of the data as shown in Table 3. There are no duplicate rows in the dataset. Number of duplicate rows: 0.

The box plots visualize potential outliers in the stock prices and trading volume (refer to Figures 1 and 2).

Table 3 reveals significant variation across the stock prices and trading volumes, with a minimum price of 20.59 and a maximum of 19,100. This wide range suggests that the dataset covers stocks with varying levels of market activity and volatility, supporting a robust analysis of different market conditions. Additionally, the volume statistics reflect diverse trading activities, with an average trading volume of approximately 16.2 million shares and a peak volume of over 380 million shares. These descriptive statistics offer foundational insights for examining stock price behaviors and trends within the dataset.

Table 3. Basic statistics

1.png

Figure 1. Stock prices box plots diagram

2.png

Figure 2. Trading volume box plots diagram

2.5 Format conversion

The date and time format conversion has been successfully completed [18]. Here's a summary of the changes and the resulting dataset:

Date Conversion: The 'Date' column has been converted to a datetime format and set as the index of the DataFrame.

Dataset Information:

DatetimeIndex: 1214 entries, 2019-07-26 to 2024-07-26

Data columns (total 6 columns):

# Column Non-Null Count Dtype

0 Open 1214 non-null float64

1 High 1214 non-null float64

2 Low 1214 non-null float64

3 Close 1214 non-null float64

4 Adj Close 1214 non-null float64

5 Volume 1214 non-null int64

dtypes: float64(5), int64(1)

memory usage: 66.4 KB

The dataset has a DatetimeIndex with 1214 entries, ranging from 2019-07-26 to 2024-07-26 (refer to Table 4).

Figure 3 presents a time series plot of the closing prices over the observed period, offering a visual analysis of the pricing trends.

Table 4. First few row DatetimeIndex

3.png

Figure 3. Closing price over time

The date format has been standardized, and the index is now in chronological order. This will make it easier to perform time-based analyzes and visualizations. The closing price plot shows the trend of the stock price over the entire period covered by the dataset.

2.6 Engineering features

Creation of additional features such as moving averages and daily returns [19].

(1) Moving averages

We will create moving averages for the 20-day and 50-day periods.

(2) Daily returns

We will calculate daily returns based on the closing price.

(3) New features

•20-day Moving Average (MA_20)

•50-day Moving Average (MA_50)

•Daily Return

(4) Sample of the updated dataset

Table 5 shows an example of a dataset that has been updated with the addition of these features:

Table 5. Sample of the updated dataset

Table 5 shows that the moving averages (MA_20 and MA_50) start as for the first 20 and 50 days respectively, as they require that many previous data points to calculate. Figure 4 shows the closing price along with the 20-day and 50-day moving averages.

This chart visualizes how the stock price has moved in relation to its short-term (20-day) and medium-term (50-day) trends.

-The 20-day MA (red line) responds more quickly to price changes, while the 50-day MA (green line) shows a smoother, longer-term trend. Crossovers between these lines can be used as potential trading signals.

-Daily Returns: This shows the percentage change in closing price from one day to the next. It's useful for analyzing the stock's volatility and for calculating risk metrics.

4.png

Figure 4. Closing price moving averages

Then Figure 5 shows the trend and stationarity in stock prices through a scatter plot comparing today's closing price with the previous day's closing price. It appears that there is a strong correlation between the closing prices from one day to the next, which is evident from the linear pattern on the graph. This suggests that the stock price tends to follow a trend and may be non-stationary, as the price movement does not fluctuate around a constant average, but rather follows a continuous pattern.

5.png

Figure 5. Trend and stationarity

Figure 6 illustrates the autocorrelation and partial autocorrelation (ACF and PACF) for the closing stock price. At the top, the time series graph of the closing price provides an overview of the trend and pattern of price movements over time. Meanwhile, the ACF and PACF graphs at the bottom show the correlation between the current price and prices at various lags. The slowly decreasing ACF pattern suggests an element of dependence or persistence in the stock price, which is often found in financial data.

6.png

Figure 6. Autocorrelation and partial autocorrelation

The next visualization displays the seasonal decomposition of the time series data (Figure 7), which divides the data into the main components of trend, seasonality, and residual (random variation). This decomposition makes it possible to see underlying patterns in the data. The trend component reflects long-term price movements, while the seasonal component shows patterns that repeat at certain intervals. The remaining fluctuations are shown as the residual component, which reflects random variation. This analysis helps identify the presence of seasonal patterns or cycles in stock price movements.

7.png

Figure 7. Seasonal decomposition

2.7 Data sharing

How data is divided for model training and testing [20]. The dataset is divided into training and testing subsets, with 80% (971 rows) allocated for training and 20% (243 rows) for testing. This split is conducted chronologically, where the training set comprises earlier data and the testing set includes later data. Such a chronological division is imperative for time series data to avoid data leakage and to replicate a real-world scenario where future data is unavailable during the training phase of the model.

2.8 Model selection

Description of the prediction model used, such as ARIMA, LSTM, or other models [21]. Based on the data analysis that has been carried out, the following is the model interpretation and recommendations:

The closing price plot shows a clear uptrend and some fluctuations. The data is not stationary, which is confirmed by the Augmented Dickey-Fuller test:

ADF Statistics: −1.1477−1.1477 p-value: 0.69570.6957

Since the p-value > 0.05, we cannot reject the null hypothesis, which means the data is not stationary.

ACF shows a slow decline, indicating non-stationary data. PACF shows significant correlation at several lags.

Figure 8 shows the presence of trends and seasonal components in the data.

8.png

Figure 8. Decomposition

The scatter plot shows a fairly linear relationship between today's and yesterday's closing prices, but there are some non-linear patterns visible. Based on this analysis, the following model recommendations are made:

SARIMA (Seasonal ARIMA):

The data shows clear trends and seasonal components.

SARIMA can handle both components.

It is necessary to do different things to make data stationery.

Prophet:

Suitable for data with strong trends and seasonality.

Easy to use and can handle missing values.

LSTM (Long Short-Term Memory):

Can handle non-linear patterns seen in the data.

Suitable for complex time series data.

Main recommendations:

SARIMA: As a baseline model. Easy to interpret and suitable for data with trends and seasonality.

Prophet: As an easy-to-use and comparable alternative to SARIMA.

LSTM: If you want to capture more complex non-linear patterns and have enough data for training.

2.9 Model parameters

Parameter settings are important in the model. Based on the results of the analysis that has just been done, we can determine the important parameters for the SARIMA (Seasonal ARIMA) model. Let's discuss the results and determine the appropriate parameters:

Differentiating (d parameter):

ADF Statistics (First Difference): −7.4749

p-value (First Difference): 4.9418×10⁻¹¹

After first differencing, the p-value is very small (< 0.05), which indicates that the data has become stationary. Therefore, we will use d=1 for the SARIMA model.

AR and MA terms (p and q parameters) (refer to Figure 9)

AR and MA terms, represented by the parameters p and q respectively (refer to Figure 9).

9.png

Figure 9. p and q parameters

Based on the ACF and PACF plots:

-ACF shows a rapid decline after some lag.

-PACF shows significant cut-off after some lags.

Based on this, we can try several combinations:

-p=1 or 22 (AR terms)

-q=1 or 22 (MA terms)

Seasonal Components (P, D, Q, m):

From the previous analysis, we see that there is a seasonal component.

For daily stock price data, we can try weekly (m=5) or monthly (m=30) seasonal periods.

For seasonal differencing, we can start with D=1.

For seasonal AR and MA terms, we can try P=1 and Q=1.

Based on this analysis, here are some combinations of SARIMA parameters that we can try:

1. SARIMA (1,1,1) (1,1,1,5) - weekly model

2. SARIMA (2,1,1) (1,1,1,5) - alternative weekly model

3. SARIMA (1,1,1) (1,1,1,30) - monthly model

4. SARIMA (2,1,2) (1,1,1,30) - alternative monthly model

For implementation, we will use grid search technique to try various combinations of these parameters and select the best performing model based on criteria such as AIC (Akaike Information Criterion) or BIC (Bayesian Information Criterion) [22].

2.10 Training and testing process

How the model is trained with training data. The SARIMA model has been successfully trained on the training data. Figure 10 shows a summary of the model and its diagnostics.

10.png

Figure 10. Model summary

Covariance matrix calculated using the outer product of gradients (complex-step)

AIC (Akaike Information Criterion): 13719.045

BIC (Bayesian Information Criterion): 13743.369

These criteria help in model selection, with lower values indicating a better fit.

The diagnostics plot (refer to Figure 11) is utilized to evaluate the model's fit by examining the residuals for normality, autocorrelation, and heteroscedasticity. A convergence warning was issued during the optimization process, indicating that the model might not have achieved the optimal solution. This issue could potentially be addressed by modifying the model parameters or employing an alternative optimization technique.

11a.png

11b.png

Figure 11. Model diagnostics

2.11 Model evaluation

The evaluation methods used include RMSE, MAE, or MAPE [23]. The model evaluation has been completed using the testing set. Here are the results:

Root Mean Square Error (RMSE): RMSE: 570.04

Mean Absolute Error (MAE): MAE: 467.69

Mean Absolute Percentage Error (MAPE): MAPE: %

The MAPE calculation resulted in, which may occur if there are zero values in the actual data. We can address this by filtering out zero values or using a different metric.

Figure 12 shows the comparison between actual and predicted stock prices over the testing period. The RMSE and MAE provide a sense of the average error magnitude, with lower values indicating better model performance.

12.png

Figure 12. Actual vs. predicted plot