Optimization of Convolutional Neural Networks for Classifying Power Quality Disturbances Using Wavelet Synchrosqueezed Transform

Power systems are supposed to perform under certain system frequencies and amplitudes, but these items are variable in the field. This situation affects the health, security, and efficiency of the power systems negatively. For this reason, monitoring and assessment of power quality momentarily have great importance. Some standardizations, like IEEE 1159, 1459, etc., propose the detailed properties of these effects and models [1-5]. The disturbances affecting negatively the power systems emerge alone or together in a window as flicker, harmonics, notch, sag, interruption, swell, etc. These occur in the range of a half-cycle to two hours in any interval and magnitude [1, 5]. Fourier transform (FT) is a basic signal processing method to analyze PQD, but it has some problems, like spectral leakage. For this reason, it is recommended to work with windows of 0.2 seconds in the standard of the Electrotechnical Commission, the IEC-61000-4-7 [3]. Thus, minimizing the spectral leakage in Fast Fourier Transform (FFT) for power systems with a fundamental system frequency of 50 Hz allows for working with a frequency resolution of 5 Hz and N = 64 (2⁶) samples per cycle. For this reason, as in some studies in studies [6-11], the sampling frequency is chosen as 3200 Hz in this study, too. The selection of machine learning algorithms’ (MLA) parameters seriously affects the performance of PQD D&C, and then the algorithm's accuracy with the optimum hyperparameters is increased [12].

Measurement of the fundamental frequency, which is sensitive to sag and harmonics, is essential and is estimated with a comb filter-based hybrid system [13]. The presence of interharmonics shows that subharmonics seriously damage the detection, measurement, and analysis of the harmonics [14]. Similarly, the measurement of the flicker is affected by not only low-frequency interharmonics (0.5–35 Hz) but also the subharmonics created by the interactions of these interharmonics with each other [8]. The standard IEC flickermeter cannot detect the flicker effects of high-frequency components that are present in the field. For this reason, the IEC flickermeter ought to be revised for the frequency range of 35 Hz and higher frequencies (from 450 Hz to a few KHz) [15]. The high-frequency interharmonics near the fundamental frequency also have a visible flicker effect but are not detectable by the standard flickermeter of IEC-61000-4-15. This problem was solved with a new flickermeter based on voltage peak detection and a curve compatible with this flickermeter at intervals of 5-95 Hz [6]. Those for only high frequency and the whole frequency, respectively, can be analyzed up to 950 Hz, too [7, 9]. Transient stability can be analyzed with the transient stability boundary, which separates the region as a secure and an unsecured one, by a sparse logistic classifier method. This method is better in comparison with other methods such as k-Nearest Neighbors (kNN), Support Vector Machines, and Linear Logistic Classifiers [16]. The envelope estimation, essential for all the features of the whole disturbance, can be analyzed with the signal geometric properties as well as FFT [17]. A comparative study with different MLAs based on feature selection showed that Random Forest with low-number features has more significant performance than the other MLAs [18]. This study also revealed a time-varying grid-noise effect on the signal of the experimental setup. Adaptive Process Noise Covariance Kalman A filter-based sag disturbance detection method can define the times of start and stop of a sag event and phase jump with the help of the estimated process noise [19]. A method for the detection and characterization of interruption and swell was introduced by using a dataset generated on MATLAB or Simulink [20]. Another signal processing method used for the detection of PQDs is wavelet transform (WT). Although this technique contains some filter banks that make it more robust than FFT, its enhanced versions are now available to increase its precision rate. For neighbor disturbances like sag and interruption, a detection method converting waveforms of PQ into 2D-binary vectors, Deep Learning (DL), and Maximal Overlap Discrete Wavelet Transform (MODWT), has been achieved robustly for a noisy field [21]. So far, the entire study is about analyzing each disturbance in detail. While all this process is applied to each disturbance focally and distinctly, that brings the computational load together. Instead of this long process, there are three-level processes such as detection, classification, and quantification. Detection and classification processes, which are important but time-consuming and hard, have motivated researchers recently.

The single and multiple power measurement parameters are calculated by using the Hilbert transform (HT) and the undecimated wavelet packet transform under the standard IEEE 1459-2010 [22]. This method is also used to evaluate transient and interharmonic disturbances [23]. A hybrid PQD-DC method of Rule-Based Decision Tree and Stockwell Transform (ST) is effective for low-number classes [24]. The improved version of ST, the Discrete Orthogonal ST method, is more capable than the others like the discrete wavelet transform (DWT), short-time Fourier transform (STFT), and ST [25]. A time-frequency-Scale transform method, a Hann window that can operate scaling and shifting, provides noise-immunity classification with a high accuracy rate and a 0.2-sec window length [26]. A method based on the fractional Fourier transform is somewhat better than the ST method for the classification of PQDs [27]. A hybrid method combining the features of ST and HT is effective for complex PQDs [28]. The Volterra series with the Type-2 Fuzzy Logic System with MLAs performs well for PQDs [29]. Another method using dual strong tracking filters and the rule-based Extreme Learning Machine achieves higher performance than other similar methods [30]. The best result of the space comprising some different pairs of signal processing algorithms and MLAs was obtained for kNN+HHT (Hilbert Huang Transform) [31]. The hybrid method of Kalman filter and fuzzy expert system has been studied in limited noisy environments and classes [32]. In a two-level method, Kalman filter-based generalized PQD recognition, fundamental frequency, and amplitudes of the harmonic measurement are made at the first level. In the second one, instantaneous total harmonic distortions and related statistical rates are obtained [33]. The method composed of sparse signal decomposition and decision trees classifies noiseless synthetic high-number classes with relatively low accuracy [34]. The hybrid model, including Multi-Objective Grey Wolf Optimizer, 2D-Riesz Transform, and kNN, has performed higher than the other similar MLA-based hybrid models [35]. The novel method using the reformation of Euler’s rotation hypothesis detailed the PQDs as 3D problems, and this has provided more achievement [36]. The Simple Gated Recurrent Network-based PQD algorithm has higher speed, accuracy, and lower complexity in comparison with the others [37]. Time-Dependent Spectral Featured and Adaptive kNN with Excluding Outliers-based PQD D&C method were proposed in study [38]. The hybrid method of classification of time-series features and the significant zero crossings of derivatives has competitive results on different popular datasets when compared to the widely used MLAs [39]. In most studies, the advantages of some time- and frequency-domain signal processing transform techniques and similar techniques compatible with MLAs were used for the detection and classification of PQDs. An evolving Gaussian fuzzy classification-based model is effective for low-number classes [40]. Another two-level hybrid model, including Variational Mode Decomposition and Detrended Fluctuation Analysis, is more robust than the compared techniques in the study [41]. It is underlined that real-time analysis of the PQD is hard and in a premature stage [33, 42].

In addition to the above, there are some studies based on wavelet transform and CNN for PQDs, as follows: A DWT-based effective feature extraction method for PQD was examined in study [43] in a noise-free environment. Optimum hybrid models, including base-wavelets and MLAs, were searched under noisy cases. It was understood that each MLA should pair a different base wavelet under different cases to gain maximum performance [44]. A 1D and 2D CNN-based hybrid method was performed for synthetic noise in study [45] and found to have more performance but nearly the same complexity when compared with the others. A hybrid model consisting of a temporal convolutional network and a CNN has been performed for different cases in study [46] under noiseless conditions. MODWT and the DL-based hybrid method have been proposed to detect differences between the sag and interruption. A spectrogram- and DL-based PQD D&C method was achieved in study [47] with a low-number class and narrow-range noise. Labeling methods based on DL are proposed, and the results of these studies are promising for low-number classes [48, 49]. An image-based deep learning study was carried out on a synthetic dataset with high accuracy [50]. The bagging LSTM method in study [51] performed with high accuracy for a synthetic high-number-classes dataset. In the study in study [52], a method using a CNN has a high accuracy rate for a noise-free synthetic dataset. Also, this study reveals that the Adam optimizer is more usable than the other optimizers for PQD classification. A hybrid method based on deep learning and 2D scalograms has high accuracy for a synthetic dataset generated in MATLAB or Simulink Simscape due to the IEEE 5 bus [51].

A comprehensive study states the requirement of time-frequency analysis to analyze non-stationary signals in real life. That study comparing time-frequency signal processing methods on biomedical signals shows that the most successful algorithm is Synchrosqueezed Transform (SST) + CNN among all considered methods like FFT, WT, HHT, etc. [53]. This method is declared to be enhanced and applied to other variational signals in different fields for future applications [53]. Power Quality Analysis needs robust time-frequency-based methods like SST and CNN since PQDs have similar characteristics to biomedical ones in terms of time variability, noisiness, and diversity, which require high robustness. For these reasons, an enhanced version of this algorithm, WSST and CNN, is selected to generate time-frequency images from PQD signals in the time domain, and then these images will be utilized to detect and classify them.

As can be seen, the algorithms for the detection and classification of PQDs have been carried out restrictedly with a synthetic dataset, low-number classes, a noiseless environment, pre-processing by data degradation leading to measurement error, etc. For all these boundaries, this paper proposes a PQD D&C method that is robust to noise, effective for high PQD classes, and reduces the process carried out with the optimization of a CNN and WSST. As a result, all of the requirements mentioned earlier will be met. Moreover, according to the best knowledge of the author and as mentioned above, although WSST and CNN are well-known methods, no study has been conducted to obtain the best parameters from the hybrid model created by these methods for the detection and classification of PQDs.

Objective

The primary objective of this research is to develop a hybrid methodology that facilitates rapid and highly accurate detection of PQDs within noisy and dynamically fluctuating power system signals. This approach leverages the combined strengths of CNN and WSST to achieve superior accuracy and robustness in PQD detection.

Contributions

This study makes several significant contributions to the field of power quality analysis:

(1) Data Generation and Preparation:

-Data for this research will be generated entirely at random, adhering to the IEEE 1149 standard.

-The dataset will incorporate noise variably added across a broad spectrum, ranging from 20 to 60 dB.

-A total of 21 distinct PQD classes will be delineated and analyzed.

(2) Innovative Approach:

-This study is pioneering in its use of WSST for the detection of power quality issues.

-It also marks the first instance of a hybrid PQD detection and classification (D&C) method that synergizes the optimization capabilities of both CNN and WSST.

Structure

Section 1 delineates the various PQD classes, elucidated through detailed equations and figures. Sections 2 and 3 provide an in-depth exploration of the WSST technique and CNN algorithms, including an examination of the relevant hyperparameters. In Section 5, the proposed PQD D&C method, which focuses on the Optimization of CNN with WSST, is defined and elaborated upon, incorporating models and block diagrams for clarity. Section 6 presents the numerical results of the study, featuring comparative analyses via figures and tables. Section 7 describes the experimental setup and the process flowchart employed to gather the dataset. Finally, Section 8 discusses the findings of the study and explores potential avenues for future research in this area.

1.png

Figure 1. PQD time-series waveform

2a.png

(a) Time-series signal

2b.png

(b) WSST scheme

Figure 2. WSST scheme with the time-series signal of C12

3.png

Figure 3. WSST schemes of time-series PQDs

$\begin{gathered}W_s(a, b)=\frac{1}{2 \pi} \int \hat{v}(\xi) a^{1 / 2} \overline{\hat{\psi}(a \xi)} e^{i b \xi} d \xi \\ W_s(a, b)=\frac{1}{i 4 \pi} \int[\delta(\xi-w)-\delta(\xi+w)] a^{1 / 2} \overline{\hat{\psi}(a \xi)} e^{i b \xi} d \xi\end{gathered}$                     (5)

Wavelet $\psi$ is concentrated in positive frequency and so, FFT of, $\hat{\psi}(\xi)=0$ for $\xi<0$.

$W_s(a, b)=\frac{1}{i 4 \pi} a^{1 / 2} \overline{\hat{\psi}(a \xi)} e^{i b w}$                        (6)

When $\overline{\hat{\psi}(a \xi)}$ is concentrated around $\xi=w_0, W_s(a, b)$ will be condensed around $a=w_0 / w$. By differentiating $W_s(a, b)$ with respect to $b$, the instantaneous frequency $w_{\text {inst }}(a, b)=2 \pi f_{\text {inst }}$ is obtained as:

$w_{\text {inst }}(a, b)=\frac{\frac{\partial W_s(a, b)}{\partial b}}{W_s(a, b)}$                      (7)

It is supposed for a pure signal that $w_{\text {inst }}(a, b)=w(a, b)=w$. The time-scale plane is transferred to the time-frequency plane according to $(a, b) \rightarrow\left(b, w_{i n s t}(a, b)\right)$. For all the frequencies, concentrated CWT meaning WSST is obtained by the scaling with coefficient $a$ and the shifting with the instantaneous frequency, $w_{i n s t}$, for $W_s(a, b)$ which is given for the continuous time by Eq. (8)

$T_{\text {inst }}(w, b)=\int W_s(a, b) a^{-3 / 2} \delta\left(w_{\text {inst }}(a, b)-w\right) d a$                    (8)

In this equation, the exponent of a can be changed to -1/2.

From 1D to the 2D conversion of the signal, i.e. from the time domain to the time-frequency domain, by using WSST for C12, one of the most complex problems, is shown with the time-series signal in Figure 2.

In this scheme, while harmonics are available in all timelines, sag emerges at 0.1 sec. and stops at 0.17 sec. While Figure 2(a). indicates the time domain information of the signal, Figure 2(b) shows frequency information as well as time. Hence, the time and frequency details of the signal can be analyzed easily. The WSST schemes of all PQDs excluding the C1 class of pure signal are given in Figure 3. In these schemes, light-colored lines with a heavy blue background are seen. From C6 to 21, intervals of disturbances can be seen explicitly because of the presence of variable character disturbances. The other disturbances, C2 to C5, have a smoother characteristic in which differences are observed when zoomed in. Although this situation seems like a disadvantage, both the amplitudes of the components leading to PQDs are condensed in a narrow band, and the unnecessary noise components are filtered. With these features, this method is more satisfactory than the classical methods like FFT, STFT, CWT, etc., as given in study [53]. For that reason, in this study, this method is selected to achieve higher classification performance.

The Eqs. (4) to (8) and Figure 2 show that the suggested WSST-based method is a good one that has a lot of potential. It is accurate, has a high level of resolution, and is robust when you look at the range of noise and the number of classes. Nevertheless, due to the nonlinear characteristics of the wavelets, the direct response of the WSST output is not available for each integer multiple of the frequency. Moreover, being a wavelet-based and 2D transformation method leads to computational load. In addition, this dataset is transmitted to CNN, which consists of a high-order matrix network. For all these reasons, one needs a higher-capacity computer or workstation than the other time- and time-frequency domain transformer-based hybrid models require.

12.png

Figure 12. Scheme of PQD detection and classification based on WSST & optimization of a CNN

13.png

Figure 13. Proposed CNN structure for PQD classification

Table 2. Hyperparameter space of CNN and best CNN model

Hyperparameters Search Space
Activations Functions	['relu', 'sigmoid', 'selu', 'softplus', 'softsign', 'LeakyReLU', 'tanh', 'elu', 'exponential']
Optimizer	['Adam', 'SGD', 'Adamax', 'Adadelta', 'Nesterov', 'Adagrad', 'RMSprop', 'Nadam', 'Ftrl']
Batch Size	[10, 20, 32, 64, 128, 256, 512, 1024]
Epoch	[50,75,100,125,150,200]
Learning Rate	[0.0001, 0.0003, 0.0006, 0.001, 0.003, 0.006, 0.01, 0.03, 0.06]
Layers	[convolutional layer, dropout, pooling layer, dense, fully connected layer]
Best CNN Model
Activations Functions	['linear', 'sigmoid']
Optimizer	['Adam']
Batch Size	[10]
Epoch	[100]
Learning Rate	[0.01]
Layers	['conv2d', 'max_pooling2d', 'conv2d_1', 'max_pooling2d_1', 'conv2d_2', 'max_pooling2d_2', 'flatten', 'dense', 'dense_1', 'dense_2']
Filters	[48]
Pool Size	[3]
Kernel Size	[3]

In this study, Tensorflow and Keras were used. TensorFlow is a library widely used for artificial intelligence tasks. In addition, Keras, an open-source library running on TensorFlow, was used. Keras was chosen for its overall ease of use, extensibility, and modularity.

6615 pre-processed images were given with a size of 875×656 in the type of RGB as input data to the selected CNN model. A total of 3 convolutions, 3 max-pooling, dense, and flattening layers are used in the model.

Figure 13 shows the summary of the proposed CNN structure. In the pooling process, max-pooling was chosen in the CNN model. Since the bright pixels in the WSST outputs are important in terms of frequency and location of the distortion, max-pooling was chosen to highlight these distinctive features in the picture. After the convolution and pooling processes, flattening was applied, and a dense layer was added. In the first and second of the last applied dense layers, the activation function was chosen as "linear." In the last layer, the sigmoid was applied as an activation function. Since there are 21 classes in the last layer, the last dense parameter is chosen as 21.

PQD signals are time-consuming due to their variational structure and are dangerous to obtain in the field. So, most studies are performed with experimental or only synthetic signals. While a synthetic one provides a wide range of signals and a high number of classes, experimental setups are more restricted in terms of types of equipment. In this study, a synthetic dataset generated in MATLAB for the training of the model is utilized, as mentioned before. Then, WSST is applied to all time signals. Finally, Python is utilized to build a CNN model with optimization through a training dataset, owing to the flexibility of Python in deep learning. Similarly, an experimental dataset is generated in an experimental setup for testing. The obtained dataset is processed with WSST in MATLAB. Eventually, the test dataset is performed on an optimized CNN model in Python for the evaluation of the proposed method. The experimental setup was established for the realization of the generation of the experimental dataset.

16.png

Figure 16. WT-based comparation of proposed method

This setup, consisting of an oscilloscope, two AATech 1032 Arbitrary Waveform Generators (AWG), a Rigol MSO 5204 Digital Storage Oscilloscope (DSO), a PC, 8 BNC connectors, and MATLAB and Python software, is used as figured out in Figure 17.

Initially, a 3.2 kHz (or more according to the specification of AWG) sampling frequency and 0.2 s window length providing a 5 Hz resolution are required in this setup, similar to some studies in studies [6-11]. In the first case of only one AWG usage, synchronization between AWGs and oscilloscopes was performed with a time-shifting arrangement. In the other case of two AWGs usage, both AWGs were tuned in phase, utilizing the AWGs’ consoles at first. Then, the obtained signals were arranged into the related window with the oscilloscope console, as in the first case. After the necessary preliminary preparations, AWGs are utilized to generate the signals based on variables and parameters with related intervals and scales in Table 1. These signals have time-varying noise, as illustrated in study [18], and therefore, noise addition is not required. BNC connectors transmit the signals to the DSO. After that, the transmitted signals are tuned into a 0.2-sec window. The whole signal at different channels is processed by different math functions to obtain the models in the table with the help of DSO's tool. As an example, the acquisition of a signal model is displayed in Figure 17(b) and clarified in the following paragraph, step by step. This method has been used for the generation of all event classes. The obtained 0.2-second signals are compiled in MATLAB and applied to signal processing with WSST on PC. After that, WSST images are used for testing on the optimized CNN model in Python.

C1 class, a pure signal, is seen having about 36 dB of spontaneous grid noise on the DSO monitor in Figure 18(a). While single classes like 5, 8, and 9 can be generated by only an AWG, lots of them need at least two AWGs and 8 BNC probes. One of the multiple disturbance events, C14, comprising the sag and transient signals, is given with all subcomponents in Figure 18(b). For the generation of this signal,

• Both pure signals and pure transients are generated periodically.
• These periodic signals are windowed by rectangular windows according to the related segments of sag and transients.
• The C14-event with sag- and transient-signals is created by the summation of the windowed signals.

17a.jpg

(a) An image from the setup

17b.png

(b) Flowchart of the test setup

Figure 17. PQD event generation to test proposed model

18a.jpg

(a) C1

18b.png

(b) C14

Figure 18. PQD events from the DSO’s monitor for different classes

All PQD classes are generated with about 36 dB of spontaneous grid noise under the mathematical models from AWG, as seen in the comprehensive waveforms in Figure 18(a). Further, since two different AWGs are needed, a phase shift is formed that must be coped with. Taking into account all these circumstances, it is seen that the generated signals to test the proposed model are disturbed, which is similar to field data. Herewith, the experimental dataset is obtained by this method.