Deenadayalan Venkatraman*
| Vaishnavi Pitchaipillai
© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
In recent years, renewable energy forecasting has gained increasing attention due to its potential to minimize energy resource usage and maximize the security of power plant operation. Deep learning models have emerged as a promising tool for renewable energy prediction. However, the application of these techniques for renewable energy forecasting remains sparse. In this work, we introduce a deep belief network-based auto-LSTM approach that utilizes a wireless sensor network (WSN) for energy forecasting in solar power plants. We also compare this approach with other deep learning techniques, including long short-term memory (LSTM) and artificial neural networks, as well as traditional models such as Multilayer Perception and physical prediction models. We evaluate the performance of these methods using data from 26 solar plants. Our experimental analysis shows that the proposed auto-LSTM-based deep learning approach outperforms the other models in terms of prediction accuracy, demonstrating its efficiency and effectiveness for renewable energy forecasting in solar power plants. By introducing the inventive auto-LSTM model and assessing its effectiveness in comparison to diverse established techniques, our study not only adds to the current discussions on predictive modeling but also tackles the tangible challenges linked with the integration of renewable energy.
long short-term memory (LSTM), deep learning, deep belief network (DBN), renewable energy, forecasting
The research related to expanding the renewable energy is increasing tremendously due to the fact that the integration of renewable energy for producing electricity has become more popular in the recent years. The generation of renewable energy from solar can be affected by several factors such as wind speed, weather condition, humidity, and temperature. The generation of wind energy is purely based on the speed of the wind and other physical characteristics. Many researchers have proposed the forecasting of renewable energy [1-3]. The research on existing forecasting methods can be categorized into three different ways such as short term, midterm, and long-term forecasting. Machine learning based approaches and deep learning-based approaches are the existing computer aided forecasting methods. Specifically, artificial neural networks created a big impression recently in deep learning since the prediction accuracy of this approach is high [4]. Deep Neural Network based techniques for forecasting also produced a tremendous success in learning and forecasting. Since they can handle complex data analysis, they are used in various applications such as signal processing, forecasting, medical diagnosis, image processing, robotics, and patter recognition. However, ANN based approaches should be properly trained to increase the prediction with maximum speed. To fulfill the electricity requirements of the household, smart micro grid is growing rapidly in addition to power plants. This smart grid utilizes renewable energy that is then transformed in to electrical energy to supply electricity to the household. The energy services provided by smart grid are supported with machine learning technologies to handle the enormous amount of energy data from Bigdata. However, the learning model should be accurate so that the renewable energy distribution can be improved. Yet, the data structure's complexity may escalate as the smart grid system frequently exhibits non-linear and uncertain characteristics. Consequently, basic machine learning methods might face challenges in executing the learning process effectively. There are also many research papers evolved that utilized hybrid machine learning models for the forecasting of renewable energy. To support highly complex data and non-linearity, deep learning techniques can be widely utilized for the learning process. Figure 1 shows the classification of different forecasting methods.
By predicting the information regarding the generation of renewable energy in future, market risks can be minimized [5]. In this paper, we proposed a deep belief network-based auto-LSTM approach for forecasting and the proposed model compared is with physical model, multilayer perception model and ANN to conduct the performance evaluation. These models are compared in terms of accuracy for the energy output of 26 solar plants. The primary objective is to introduce a forecasting approach for solar power plants, utilizing a deep belief network-based auto-LSTM model. The primary goal is to improve the precision of renewable energy predictions through the application of this innovative hybrid model. The research specifically involves assessing the performance of the proposed auto-LSTM approach in contrast to alternative deep learning techniques and traditional models. The evaluation focuses on optimizing energy resource utilization and enhancing the security of power plant operations.
Figure 1. Classification of different forecasting approaches
The remaining sections are organized in to five sections. Section 2 discuss the method used in this study, elucidating the use of a German solar farm dataset and the application of normalization techniques. Section 3 illustrates the pros and cons of existing models. Sections 4 introduces the proposed model. The results are discussed in section 5 and the work is concluded in section 6.
The aim of this study is to perform solar energy forecasting efficiently using deep learning strategies. German solar farm dataset consist of 26PV ranges from 100KW to 8500KW is used in the work. Min max normalization is used to normalize the timeseries dataset between 0 and 1. The patterns of solar irradiance will be differed based on the data available in the dataset. The details of the dataset can be found at https://github.com/fchollet/keras, 2015. To evaluate the forecasting process, the methods used in this research are MLP (Multilayer perceptron), DBN (deep belief network) and Auto LSTM. The performance metrics used in the work are MAPE (Mean Absolute Percentage Error), RMSE (Root Mean Square Error), Bias, and coefficient of determination. The proposed model is implemented in 4 stages. First, the model is defined and trained, then fit the auto LSTM model. Finally, the proposed model is used for forecasting.
The authors [6] described a forecasting model for solar energy using deep learning framework. This work uses a timer series model for energy forecasting based on the data collected from NASA. Both single and multiple locations are considered for investigation against accuracy and other standard metrics. The work proves that the proposed model is better than the other previous models for the accurate prediction of solar irradiance. However, the black box characteristics of the proposed model is very difficult to understand and the model focused on specific location of India without considering the weather condition.
Authors of the work [7] proposed a coefficient of model accuracy (CMA) based model for the classification. This work considers both linear and nonlinear parameters considering the model alternative to R2 for the deviation of the proposed model. However, R2 model does not guarantee the regression for classification and not performs well for model residuals.
The authors of the work [8] use LSTM for training the dataset to perform the prediction of weather forecasting. The proposed model is compared against ANN, linear regression, and BPNN (Back propagation neural network). The work shows that the performance is 18% than the abovementioned models in terms of RMSE for the prediction of solar irradiance. However, the work does not focus on error analysis that might be incurred on weather forecasting.
Authors of the work [9] proposed an idea of for improving the performance of deep learning models in terms of its accuracy. The paper provides a in depth review of all the deep learning frameworks to find the efficiency of their forecasting methods for renewable energy prediction. The paper also considers the important challenges in improving the prediction accuracy. But the paper does not provide the details about the role of statistical model and the physical condition for improving the forecasting is not addressed properly.
The authors of the work [10] proposed a novel method for the prediction of renewable energy at 1h ahead. The work uses multimode decomposition model for the highly dynamic data. The work also supports both linear and non-linear data model to perform forecasting of solar energy. With different decomposition techniques such as empirical mode decomposition, wavelet decomposition, and multilinear decomposition, the work achieved better accuracy. However, the work does not focus on the weather condition and horizon of the forecasting. In the literature studies, we learned that hybrid machine learning models can be helpful in improving the accuracy and efficiency. However, time complexity for training the algorithms and for the processing of hybrid models is increased. On the other hand, deep learning models perform well for high dimensional data. But these models require huge volume of data and will take long time for executing the models based on the complexity of data. To overcome the difficulties in the existing works, the work introduces a hybrid deep learning framework that combines ANN, DBN, LSTM, and MLP for the evaluation of the proposed work.
In this section, the work introduces a hybrid deep learning approach, specifically incorporating an auto-LSTM model and a deep belief network that are used for forecasting the renewable energy output. The methodology is organized into three key phases: clustering, training, and prediction. This hybrid model achieves better performance with the help of the proposed auto LSTM and deep belief network. The work is partitioned in to 3 parts; clustering, training and prediction. To choose the relevant historical data, correlation analysis is performed in clustering part. In training part, a deep belief network, LSTM, and WSN are combined to create a hybrid model for the renewable energy forecasting. The proper training model is fixed in testing part based on the test data.
4.1 Dataset
The dataset is collected from German solar farm dataset which consist of 26PV ranges from 100KW to 8500KW. Min max normalization is used to normalize the timeseries dataset between 0 and 1. The patterns of solar irradiance will be differed based on the data available in the dataset.
The dataset can be freely available at https://github.com/fchollet/keras, 2015. The proposed deep learning based architecture is shown in Figure 2.
Figure 2. The proposed deep learning based architecture
4.2 Clustering
First, clustering process is done and the data is divided into 4 sections and are training with the help of the deep learning algorithms used in the work for the improving the prediction accuracy. Training of the dataset is done with 80% of the dataset and 20% of the data is used for training. First 16 days of a whole month is selected that consist of 200 real time per day. Gaussian mixture model is used for clustering in this work to cluster the data into 4 clusters. These clusters are trained using DBN, LSTM, MLP and ANN. In this model, the weight corresponding to each layer and the input are normalized. The nearest distance between weight vector and input vector will be chosen as the elected neuron. Now the newly elected weight of neuron as shown in Eq. (1) and other nodes are updated.
$\mathrm{Wm}, \mathrm{n}(\mathrm{t}+1)=\mathrm{Wm}, \mathrm{n}(\mathrm{t})+\mathrm{h}(\mathrm{t}) \mathrm{Ld}, \mathrm{n}(\mathrm{t})(\mathrm{Xi}-\mathrm{Wm}, \mathrm{n}(\mathrm{t}))$ (1)
$L_{d, j}(t)=\left(\begin{array}{c}d^2 d n \\ 2 r^2\end{array}\right)$ (2)
$\mathrm{m}(\mathrm{t}+1)=\operatorname{int}((1(\mathrm{t})-1) \mathrm{x} 1-\mathrm{t}))+1$ (3)
$\mathrm{H}(\mathrm{t}+1)=\mathrm{h}(\mathrm{t})=\mathrm{H}(0) / \mathrm{T}$ (4)
where, the distance d is measure between m neuron and n neuron, and the radius in neighborhood is r(t) and the function int is used for rounding with T learning frequency.
4.3 Training
The clustering data are feeding as input to training model and time series method is followed for training. The training data is categorized as [x1, x2, x3…xn] where x is referred as data and n is referred as total data in each cluster. The timestamp m and x1, x2, x3…xm are considered as a new dataset. The these will be feed to the model to forecast the outcome that is closest to xi+1. Again, x2, x3, …. xi+1 is feed for training to predict the outcome that is closest to xi+2 and the same process is repeated till the final value is xn and the training process is concluded. The training process for the model is shown in Figure 3. It consists of three layers, input layer, hidden layer, and output layer. The proposed model uses hybrid approach for feature extraction from the raw data by combining the specified deep learning algorithms. Neural network is used to extract the features for time series data. The useful features extracted from second layer is feed into LSTM for prediction. This approach is useful in improving the prediction accuracy [11-13].
Figure 3. Process of the proposed model
4.4 Prediction
The proposed hybrid approach is useful to perform the forecasting using LSTM model. The CNN model is also employed to improve the prediction accuracy. This will be helpful in extracting the features from time series data (Figure 4). The features extracted from the CNN are feed to LSTM layer to perform the forecasting of renewable energy. Here, the work uses attention mechanism to assign the weights to the LSTM layer for the purpose of improving the prediction accuracy. WSN is used in attention mechanism to focus the target area while ignore the irrelevant area. LSTM is suitable for time series data and CNN can be adopted to support LSTM [14, 15]. The input of time series data can be referred as it and the units of hidden layer is referred as h and Ht is referred as time series data output. The proposed LSTM model expressed as follows:
$\mathrm{It}=\sigma(\mathrm{YtXi}+\mathrm{It}-1 \mathrm{Fxi}+\mathrm{ri})$ (5)
$\mathrm{Ot}=\sigma(\mathrm{YtXg}+\mathrm{It}-1 \mathrm{Fxi}+\mathrm{ri})$ (6)
$\mathrm{Pt}=\sigma(\mathrm{YtXo}+\mathrm{It}-1 \mathrm{Fxo}+\mathrm{ro})$ (7)
$\mathrm{Dt}=\tanh (\mathrm{Yt} \mathrm{Xig}+\mathrm{Fxi}+\mathrm{Gxf}+\mathrm{rc})$ (8)
$\mathrm{Cr}=\mathrm{Gt} \circ \mathrm{Dt}-1 \mathrm{Fxt} \mathrm{oDt}$ (9)
It $=\mathrm{Tt}$ o $\tan (\mathrm{Dt})$ (10)
In the above equations, the activation function works with in the ranges of 0 to 1 and-1 to 1 since they are all gates. The current input requirement is fulfilled by the input gate It and the previous state can be defined by overlook gate Ot and the connectivity to the external network is defined by output gate Pt. Based on the month, the training is performed with appropriate model.
Therefore, the work extracts the data from the PV system. Then, cleaning of data is performed to remove the outliers. After this process, original data is normalized. Finally, LSTM model is trained and tested for the quality of accuracy level to achieve the improved forecasting.
The work uses three architectures for developing the improved forecasting mode. A multi-layer perception mode (MLP) deep belief network, and Auto LSTM are used for improving prediction accuracy. MLP consists of fully connected layers and the training is done with the help of backpropagation algorithm [16].
Figure 4. Time series steps
The work uses deep belief network to improve the prediction with dimensionality reduction while performing features extraction and forecasting using auxiliary layer. The features of the data are reduced using unsupervised learning in the first training and another training is performed to train the auxiliary layers to perform prediction.
Auto encoder feature learning is employed along with LSTM for feature learning of Auto LSTM model. LSTM is used in the encoding part of auto encoder [17]. The time series steps are shown in Figure 4.
4.5 Activation function
Tanh activation function is used is in all the layers of the networks excluding output layers. ReLu activation function is used in output layer for forecasting. It has been expressed as:
$\operatorname{ReLU}(\mathrm{f})$ act $=\max (0, \mathrm{f})$ (11)
Tanh activation function is expressed as
$\operatorname{ReLU}(\mathrm{f})$ act $=\tanh (\mathrm{f})$ (12)
Figure 5. Forecasting power of LSTM
Figure 5 shows the forecasting power of LSTM model in PV power generation. Forecasting is done with activation functions such as tanh and ReLU.
The performance metrics for forecasting are Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). MAPE is used for accuracy prediction and offers the measure of model’s precision. RMSE helps to differentiate the predicted and actual values. MAE is used to directly measure the accuracy of the model.
The formula for the above metrics is given as follows.
MAPE $=\frac{1}{m} \sum_{i=1}^m\left|\frac{N_{\text {model }, i}-N_{\text {actual }, i}}{N_{\text {actual }, i}}\right|$ (13)
$\mathrm{RMSE}=\frac{1}{m} \sqrt{\sum_{i=1}^m\left(\mid N_{\text {model }, i}-N_{\text {actual }, i}\right)^2}$ (14)
$\operatorname{MAE}=\frac{1}{m} \sum_{i=1}^m\left|N_{\text {model }, i}-N_{\text {actual }, i}\right|$ (15)
The implementation of proposed model is done with the four steps.
(1) Model Definition and Training
(2) Auto LSTM Fitting
(3) Forecasting
(4) Performance Evaluation
LSTM model is defined in the first step. Then training the model and fit it. Finally, the model is used for forecasting. The parameters selected for training the models detailed in Table 1 were meticulously chosen to find a delicate equilibrium between the speed of convergence, overall model stability, and the ability to encompass diverse aspects within the intricate renewable energy dataset. A consistent learning rate of 1.10-3 was employed across Auto LSTM, deep belief network, and LSTM models to uphold stability throughout the training process. In the case of Auto LSTM and the deep belief network, iterative fine-tuning occurred through 300 and 700 iterations, respectively, aiming for optimal model performance. The Multilayer Perceptron (MLP) experimented with a learning rate range of [0.4, 0.004] to explore different convergence rates, coupled with 700 iterations for comprehensive training. Opting for 80 MLP models in training was intended to encompass a wide array of features and patterns. LSTM training involved a learning rate of 1.10-3 and 300 iterations, with 35 models trained to span a diverse spectrum of scenarios within the dataset. These parameter selections were strategically made to elevate the predictive prowess of the models, preventing overfitting, and ensuring a diverse representation of learned patterns [17-20].
The performance measures such as MAPE, RMSE, MAE are used to analyze the forecasting of energy. To check if the energy forecasting is predicting greater or lower values, bias from the error measures is analyzed. Higher deviation incurs when the RMSE is higher than the MAE and lower deviation incurs when the RMSE is approximately equal MAE values. Errors are calculated after validating the trained dataset. Least error from the validation is chosen to conclude with the final evaluation of the performance. The maximum number of iterations occurred for fine tuning the deep belief network with activation function is 200, and 700. Similarly, auto LSTM is trained by appending LSTM at its last layer. The maximum number of iterations occurred for fine tuning the auto LSTM is 300 and 700. The model is used for forecasting after the auto LSTM is constructed. The error rates of different models are given in Table 2.
The outstanding performance of the Auto LSTM model can be ascribed to a multitude of pivotal factors. Firstly, the incorporation of the deep belief network amplifies the model's capacity to extract pertinent features from intricate datasets, facilitating the capture of nuanced patterns in solar energy generation. The synergy of the autoencoder feature learning mechanism with LSTM further enhances the efficiency of feature extraction, culminating in heightened prediction. The results of Table 2 shows that the deep learning based Auto LSTM model outperforms the other state of the art model. The validation of RMSE and loss function is shown in Figure 6. LSTM is a good outfit for forecasting with less MAPE errors.
Table 1. Parameters used for training
|
Network Model |
Rate of Learning |
No. of Iterations |
Models Trained |
|
Auto LSTM |
1.10-3 |
300, 700 |
25 |
|
Deep belief network |
1.10-3 |
200, 700 |
30 |
|
Multilayer perceptron |
[0.4, 0.004] |
700 |
80 |
|
LSTM |
1.10-3 |
300 |
35 |
Table 2. Error rates of different models
|
Data |
Auto LSTM |
Deep Belief Network |
Multilayer Perceptron |
LSTM |
||||
|
Test |
Train |
Test |
Train |
Test |
Train |
Test |
Train |
|
|
MAPE |
0.0768 |
0.0628 |
0.0767 |
0.0677 |
0.0628 |
0.0617 |
0.0668 |
0.0617 |
|
RMSE |
0.0767 |
0.0624 |
0.0753 |
0.0654 |
0.0628 |
0.0617 |
0.0720 |
0.0690 |
|
MAE |
0.0457 |
0.0435 |
0.0285 |
0.0265 |
0.0315 |
0.0285 |
0.0357 |
0.0335 |
|
Bias |
-0.0028 |
-0.0035 |
-0.0038 |
-0.0045 |
-0.0018 |
-0.0025 |
-0.0078 |
-0.0045 |
|
Corr |
0.9373 |
0.9388 |
0.9372 |
0.9387 |
0.9352 |
0.9443 |
0.9434 |
0.9474 |
Figure 6. LSTM loss function in training
In summary, our study introduces a novel hybrid approach, Auto LSTM, for solar energy forecasting, leveraging a combination of LSTM, deep belief network (DBN), and attention mechanisms. The experimental process includes different deep learning based models and the combination of LSTM and deep belief network and the performance metrics used in the work are MAPE, MAE and RMSE. The work also uses ReLu activation function to ignore the error while performing prediction data. The training algorithm introduces in the work can effectively performs feature extraction through LSTM with attention mechanism. Through different deep learning models, the work increases the performance of forecasting than the other state of the art work. The experimental analysis proves that the work achieves higher prediction for renewable energy forecasting. In future, the work can be extended further with wavelet based LSTM using more generalized dataset.
[1] Singh, U., Rizwan, M., Alaraj, M., Alsaidan, I. (2021). A machine learning-based gradient boosting regression approach for wind power production forecasting: A step towards smart grid environments. Energies, 14(16): 5196. https://doi.org/10.3390/en14165196
[2] Nespoli, A., Niccolai, A., Ogliari, E., Perego, G., Collino, E., Ronzio, D. (2022). Machine learning techniques for solar irradiation nowcasting: Cloud type classification forecast through satellite data and imagery. Applied Energy, 305: 117834. https://doi.org/10.1016/j.apenergy.2021.117834
[3] Narvaez, G., Giraldo, L.F., Bressan, M., Pantoja, A. (2021). Machine learning for site-adaptation and solar radiation forecasting. Renewable Energy, 167: 333-342. https://doi.org/10.1016/j.renene.2020.11.089
[4] Musbah, H., Aly, H.H., Little, T.A. (2021). Energy management of hybrid energy system sources based on machine learning classification algorithms. Electric Power Systems Research, 199: 107436. https://doi.org/10.1016/j.epsr.2021.107436
[5] Stefenon, S.F., Ribeiro, M.H.D.M., Nied, A., Yow, K.C., Mariani, V.C., dos Santos Coelho, L., Seman, L.O. (2022). Time series forecasting using ensemble learning methods for emergency prevention in hydroelectric power plants with dam. Electric Power Systems Research, 202: 107584. https://doi.org/10.1016/j.epsr.2021.107584
[6] Sorkun, M.C., Paoli, C., Incel, Ö.D. (2017). Time series forecasting on solar irradiation using deep learning. In 2017 10th International Conference on Electrical and Electronics Engineering (ELECO), IEEE, pp. 151-155.
[7] Onyutha, C. (2020). From R-squared to coefficient of model accuracy for assessing "goodness-of-fits". Geoscientific Model Development Discussions, 1-25. https://doi.org/10.5194/gmd-2020-51
[8] Qing, X., Niu, Y. (2018). Hourly day-ahead solar irradiance prediction using weather forecasts by LSTM. Energy, 148: 461-468. https://doi.org/10.1016/j.energy.2018.01.177
[9] Wang, H., Lei, Z., Zhang, X., Zhou, B., Peng, J. (2019). A review of deep learning for renewable energy forecasting. Energy Conversion and Management, 198: 111799. https://doi.org/10.1016/j.enconman.2019.111799
[10] Monjoly, S., André, M., Calif, R., Soubdhan, T. (2017). Hourly forecasting of global solar radiation based on multiscale decomposition methods: A hybrid approach. Energy, 119: 288-298. https://doi.org/10.1016/j.energy.2016.11.061
[11] Qing, X. (2018). Statistical analysis of wind energy characteristics in Santiago Island, Cape Verde. Renewable Energy, 115: 448-461. https://doi.org/10.1016/j.renene.2017.08.077
[12] Lai, J.P., Chang, Y.M., Chen, C.H., Pai, P.F. (2020). A survey of machine learning models in renewable energy predictions. Applied Sciences, 10(17): 5975. https://doi.org/10.3390/app10175975
[13] Widodo, D.A., Iksan, N., Udayanti, E.D. (2021). Renewable energy power generation forecasting using deep learning method. In IOP Conference Series: Earth and Environmental Science. IOP Publishing, 700(1): 012026. https://doi.org/10.1088/1755-1315/700/1/012026
[14] Fentis, A., Bahatti, L., Tabaa, M., Mestari, M. (2019). Short-term nonlinear autoregressive photovoltaic power forecasting using statistical learning approaches and in-situ observations. International Journal of Energy and Environmental Engineering, 10(2): 189-206. https://doi.org/10.1007/s40095-018-0293-5
[15] Khan, Kamalakannan, K., Ahmed, N.S.S. (2023). Integrating machine learning and stochastic pattern analysis for the forecasting of time-series data. SN Computer Science, 4: 484. https://doi.org/10.1007/s42979-023-01981-0
[16] Keerthisinghe, C., Verbič, G., Chapman, A.C. (2016). A fast technique for smart home management: ADP with temporal difference learning. IEEE Transactions on Smart Grid, 9(4): 3291-3303. https://doi.org/10.1109/TSG.2016.2629470
[17] Harrou, F., Taghezouit, B., Sun, Y. (2019). Improved kNN-based monitoring schemes for detecting faults in PV systems. IEEE Journal of Photovoltaics, 9(3): 811-821. https://doi.org/10.1109/JPHOTOV.2019.2896652
[18] Kushwaha, V., Pindoriya, N.M. (2019). A SARIMA-RVFL hybrid model assisted by wavelet decomposition for very short-term solar PV power generation forecast. Renewable Energy, 140: 124-139. https://doi.org/10.1016/j.renene.2019.03.020
[19] Oksuz, I., Cruz, G., Clough, J., Bustin, A., Fuin, N., Botnar, R.M., Prieto, C., King, A.P., Schnabel, J.A. (2019). Magnetic resonance fingerprinting using recurrent neural networks. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, pp. 1537-1540. https://doi.org/10.1109/ISBI.2019.8759502
[20] Zhou, X., Hu, Y., Liang, W., Ma, J., Jin, Q. (2020). Variational LSTM enhanced anomaly detection for industrial big data. IEEE Transactions on Industrial Informatics, 17(5): 3469-3477. https://doi.org/10.1109/TII.2020.3022432
