Detection of Non-Technical Losses in Electric Distribution Network by Applying Machine Learning and Feature Engineering

In this section, we have discussed various causes of NTL, the economic effect of NTL, NTL proportion in different countries, and existed essential works of literature related to the detection of non-technical losses.

2.1 Causes of NTL

In NTL, there is an involvement of both types of the user either contractual or irregular. There are many reasons for the occurrence of NTL. It has been observed that many consumers try to bypass the meters with the wrong intention of fraud or deception. Due to any fault or damage in infrastructure, NTL also occurs. Yurtseven [9] and Lewis [10] shows that some consumers are so poor that they cannot pay the electricity bill. At the same time, some consumers can pay the electricity bill but are unwilling to do so. Therefore, another cause of the NTL is unpaid bills [2].

2.2 Economic effects of NTL

The economic effect of NTL reflects directly in the form of losses of income return for electric utilities, which leads to an increase in the additional charge for the regular customer. The indirect economic effect of NTL is the inspection cost because more NTL leads to more inspections in the consumer's premise. NTL also creates unreliability in the network. For maintaining the reliability of the system, the cost of maintenance can be used to settle the indirect effects. Linares and Rey [11] have explained that the extra cost of detection and prevention from NTL may be higher than the rate of return.

2.3 Variation of NTL

The proportion of the NTL in different countries is found different, which is shown in Table 1. The variation in the NTL depends upon the development status of that particular country.

Table 1. List of countries facing NTL and their proportions

2.4 Literature related to the detection of NTL

Negi et al. [15] group of authors have used the fuzzy logic expert system and SVM, where the data set the size of ~100K customers. The purpose of including expert human knowledge is to identify fraudulent behavior. In this work, test recall of 0.72 has been achieved. In Ref. [7], the following five features are being used, i.e., maximum consumption, average consumption, standard deviation, the average consumption of the residential area, and the number of inspections. Also, in this work, the fuzzy c-means clustering algorithm is used for grouping the customers into classes based on the features. Further, the classification of the customer into NTL and non-NTL is done by using the Euclidean distance measure based on the fuzzy membership values. This test has achieved 0.745 precision.

The Support Vector Machines (SVM) are supervised learning models used to analyze data for classification and regression. Nagi et al. [16] have used this technique for the detection of NTL. The data set used in this work is from Kuala Lumpur, Malaysia, where the size of the data is ~260K customers. In this data set, consumer's 25 months' meter readings are recorded, and the average daily consumption per month (used as a feature) is calculated. These features are normalized and used for training with a Gaussian kernel. Further, computation of the creditworthiness ranking (CWR) is also used as a feature. Calculations for CWR, along with its range, have already been discussed in the previous section. This work has achieved a recall value of 0.53. Paper [6], also discuss the related setting and achieved the test recall of 0.77 and accuracy of 0.86 on the different data set. According to Ramos et al. [17], 5K Brazilian Industrial Customer data set is used where each consumer profile has ten features like maximum demand, demand billed, installed power, etc. The test accuracy of SVM, neural network, and K-nearest neighbors (KNN) are 0.9628, 0.9448, and 0.9620, respectively.

According to Glauner et al. [18], massive data set of Brazilian customers around 700K and ~31 million monthly reading are inspected from January 2011 to January 2015. They have also used ~400K inspection data. In this work, the expert system has been used to optimize the fuzzy system parameters using stochastic gradient descent as explained by Bottou [19] to that data set. The performance of this fuzzy system is better than the Boolean system. Daily average consumption features are used for ~100K customers and compare the three different algorithms on the different proportions of the fraud. The area under the curve (AUC) shows performance evaluation. The test AUC of 0.55, 0.465, and 0.55 on an NTLproportion of 5% for the optimized fuzzy system, Boolean system, and SVM. For an NTL proportion of 20%, test AUC are0.545, 0.475, and 0.55 for the optimized fuzzy system, Boolean system, and SVM.

Nagi et al. [20] presented the data sets detail 6K high-voltage and 80K low-voltage customers in Malaysia. The detection of electricity theft is based on 30 days meter reading data at an interval of 30 minutes. The test recall is 0.55, which has been achieved in this work. This work uses fuzzy logic along with human expert knowledge for detection. Some of the supervised learning models like CNN and LSTM are used to detect non-technical losses and theft detection in the electric distribution network. In Ref. [21], convolutional neural network (CNN) and long short-term memory (LSTM) was used for classification. The class imbalance problem is also tackled by using the SMOTE algorithm. The data set size for simulation is 10,000 customers and achieved a test accuracy of 89%. In this technique, several hidden layers are used, first four layers performed the convolutional operation with 20 features by each layer, and the remaining layers performed the LSTM operation.

Viegas et al. [22] have introduced a technique for detecting NTL by applying fuzzy-based clustering known as fuzzy Gustafson Kessel Clustering. In this technique, the authors try to identify the user's consumption behavior to determine the prototype used to score the NTL. The AUC score of this technique is 0.741.

The proposed model is implemented using Python 3.6, and system configuration are Intel Core i3, CPU 3.4 GHz, and 4.0 GB RAM. The logistic regression, decision tree, SVM, and RF are programmed using sci-kit learn [23].

4.1 Performance metrics

NTL is detected as a discrete two-class classification task. Therefore, each consumer is divided into abnormal or normal categories. The output of classifier validation is called confusion matrices. Here, four confusion matrices true positive (TP), false negative (FN), false positive (FP), and true negative (TN) are used for NTL detection. These matrices are defined as the number of consumers that are classified correctly as normal, classified falsely as abnormal, classified falsely as normal, and classified correctly as abnormal, respectively. Another important evaluation metric is accuracy, which measures how accurately a classifier predicts the TP and TN values. Accuracy can be calculated using Eq. (7) which is as follows:

$\text { Accuracy }=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$     (7)

Some other essential evaluation metrics are used for performance measures of the proposed model, which are shown in Eqns. (8) to (12).

$\text { True positive rate }(T P R)=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$     (8)

$\text { True negative rate }(T N R)=\frac{\mathrm{TN}}{\mathrm{TN}+\mathrm{FP}}$     (9)

$\text { False positive rate }(F P R)=\frac{\text { FP }}{\text { TN + FP }}$     (10)

$\text { False negative rate }(F N R)=\frac{\mathrm{FN}}{\mathrm{TP}+\mathrm{FN}}$     (11)

$\text { Positive predictive value }(P P V)=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$     (12)

Recall or true positive rate (TPR) measures the total number of electricity theft consumers correctly classified by the classifier. If the value of TPR is higher, the detection of NTL becomes easier.

Tested values of the precision, recall, and F1 score are shown in Tables 2 to 5 for the LR, SVM, DT, and RF classifiers. Table 6 shows the comparative analysis among these classifiers based on precision, recall, and F1 score. We have also demonstrated its effectiveness by comparison in Figure 2. The precision, recall, and F1 score of the decision tree are 0.97, 0.97, and 0.97. The recall, precision, and F1 score of random forest are 0.98, 0.98, and 0.98, respectively, which are better than other existed works. This comparison is shown in Table 8. In Tables 2 to 5, we can see that both the classes (normal and defaulter consumers) have the approximately same value for precision, recall, and F1 score, which means that the proposed model solves the data-unbalancing problem.

Table 2. Classification score of Logistic Regression

Table 3. Classification score of SVM

Table 4. Classification score of Decision Tree

Table 5. Classification score of Random Forest

Table 6. Classification score of LR, SVM, DT, and RF

The harmonic mean is also used, which is calculated as follows:

$\text { Harmonic mean }=\frac{2 * \text { Precision } * \text { Recall }}{\text { Precision }+\text { Recall }}$     (13)

A high value of the harmonic mean shows the high value of precision and recall. The harmonic mean of this proposed model is 97% and 98% for Decision Tree and Random Forest, respectively. Arithmetic mean and Dominance parameters are also used, which are calculated as:

$\text { Arithmetic mean }=\frac{\text { Precision }+\text { Recall }}{2}$     (14)

$\text { Dominance }=T P R-T N R$     (15)

The range of this parameter is +1 to -1. The value of dominance is close to +1, showing the classifier's good accuracy for the positive class. And dominance value near to −1, depicting good accuracy of the classifier for the negative class. This proposed model's arithmetic mean and dominance is 98% and 0.006, respectively, shown in Table 7.

2.jpg

Figure 2. Comparison among different classifiers based on Precision, Recall, and F1 score

The area under curve (AUC) and receiver operating characteristic (ROC) curve are also used for the detection of NTL. The ROC curve of this proposed model is shown in Figure 5. Mathew's correlation coefficient (MCC) is a vital evaluation parameter [24], which is defined in Eq. (16). MCC can also be used for the evaluation parameter to detect non-technical losses based on machine learning. The proposed model has 0.94 and 0.96 MCC scores for Decision Tree and Random Forest, respectively.

$M=\frac{(\mathrm{TN} * \mathrm{TP})-(\mathrm{FN} * \mathrm{FP})}{\sqrt{(F P+T P)(T P+F N)(T N+F N)(T N+F P)}}$           (16)

Table 7. Complex metrics for LR, SVM, DT and RF

3.jpg

Figure 3. Dominance of LR, SVM, DT, and RF

4.jpg

Figure 4. Comparison among LR, SVM, DT, and RF based on TPR, TNR, and Harmonic Mean

5.jpg

Figure 5. ROC curves for LR, DT, and RF

Table 7 shows the comparative analysis of LR, SVM, DT, and random forest based on different metrics such as accuracy; arithmetic mean, harmonic mean, TPR, FPR, TNR, FNR, dominance, and AUC score. Figure 3 shows the comparative analysis of classifiers based on dominance metrics. Figure 4 shows a comparative analysis of the classifiers based on TPR, TNR, and harmonic mean. ROC curves of these three different classifiers (LR, SVM, RF) are shown in Figure 5. Figure 5 clearly shows that the performance of Random Forest is better than others. The proposed model's performance has been compared with existed works which are shown in Table 8. From Table 8, it is clear that the proposed model performs better than other existed works in terms of accuracy, recall, precision, and AUC score.

Table 8. Comparison between the proposed scheme and existed works