Dual-Scale Dataset-Based Intelligent Recognition of Power Equipment for Enhanced Digital Grid Planning

As industries and agriculture undergo automation, the demand for electricity has surged, exacerbating the challenge of efficiently allocating power resources. To ensure the stable operation of electrical equipment, effective power grid planning is paramount. Modern power grid planning aims to construct a resilient, intelligent power grid. Traditional planning methods rely heavily on manual labor, while digital planning, characterized by information-based, automated, and intelligent approaches, offers a promising alternative [1]. In recent years, digital technology has permeated various sectors of the economy and society. Notably, artificial intelligence and digital twins have emerged as crucial components for establishing a digital power grid [2]. A digital power grid necessitates the digitization of the entire process, encompassing all elements and business aspects of the power grid, to create a seamless virtual digital world mirroring the physical one. Leveraging digital information, novel business models and optimization methods can be devised to enhance the operation and management of the power grid [3].

The cornerstone technologies of a digital twin system include modeling, simulation, and data fusion-based digital threads. Establishing a corresponding "digital replica" of real power equipment along the physical power lines is a prerequisite for a digital twin power grid. Moreover, effective management and classification of this "digital replica" are essential for conducting precise simulations and optimizations. Intelligent recognition algorithms play a pivotal role in automatically tagging and classifying collected device image information. However, developing robust power equipment recognition algorithms poses numerous challenges. These algorithms must accurately detect various objects, ranging from large devices like wires, utility poles, and transformers to smaller components like insulators and crossbars. Disparities in scale among different power equipment components further complicate accurate detection. Additionally, complex environmental factors such as strong light and haze can distort object features, necessitating solutions to maintain high detection accuracy under varying conditions. Furthermore, for deployment on edge data acquisition devices such as robots and drones, detection algorithms must prioritize lightweight designs and meet real-time performance requirements.

Presently, research on power equipment target detection and recognition primarily revolves around traditional detection algorithms and deep learning-based approaches. Traditional algorithms rely on manually designed features for template matching, followed by feature classification and regression to achieve object recognition [4]. While these algorithms offer fast detection speeds, they often struggle with robustness in practical applications due to complex environmental factors, leading to issues like false positives and false negatives. In contrast, deep learning algorithms leverage hierarchical feature extraction and learning capabilities to achieve breakthroughs in object detection performance. These methods automatically learn advanced features from massive datasets, significantly improving recognition accuracy. Many studies have applied deep convolutional neural networks (CNNs) to power equipment detection, reflecting the general trend towards using intelligent detection methods to maintain power grid systems. For instance, Jiang et al. [5] propose a recognition model for substation equipment images based on Mask-RCNN [6] and Faster-RCNN [7], highlighting the need for enhancements to achieve better results in specific application scenarios. Similarly, Lin et al. [8] introduce a transmission line inspection image detection algorithm based on an improved Faster-RCNN, addressing challenges related to recognition accuracy and real-time detection. Despite these advancements, certain limitations persist, such as slow recognition speeds, large model sizes, and challenges in optimizing detection accuracy under various environmental conditions. Wan et al. [9] use the position information of largerscale power equipment to correct the probability of small components based on the position-related relationship between small and large components of transformers, and improve the recognition accuracy of the small objects, but a large number of conditional judgments need to be artificially set in the postprocessing process, and the generalization ability is weak. Chen et al. [10] improved the single-stage object detection algorithm, RFBNet [11], use a lightweight backbone network to improve the model’s detection speed, but the detection accuracy of small-scale equipment still needs to be further improved. Xiong et al. [12] add a de-fogging algorithm to the power equipment detection algorithm for data preprocessing to improve the recognition accuracy of the detection algorithm in foggy weather, but the detection accuracy in other complex environments such as strong light and night scenes still needs further optimization. Due to the slow speed and large model parameter size of two-stage object detection algorithms such as RCNN [13] and Fast RCNN [14], they cannot meet the real-time detection requirements.

In this paper, we present an improved single-stage multiscale object detection algorithm designed to address these challenges. Leveraging lightweight backbone networks, context mining, and feature fusion technology, our algorithm enhances detection accuracy and speed while significantly reducing model parameter size. We constructed a large-scale recognition dataset encompassing multiple categories of power equipment for training purposes. To tackle the issue of poor recognition caused by extreme differences in target scales, we divided device images into two scales and employed multi-stage training on data from different scales. Various data augmentation techniques were applied to enhance the algorithm's generalization ability and robustness against diverse weather and lighting conditions. During testing, the model was trained on both large-scale and small-scale device datasets, and the detection results were combined to establish an end-to-end multi-scale power equipment recognition pipeline. Our proposed method achieved a mAP of 92.13% and a recognition speed of 52.15 fps on the test set. Extensive experiments demonstrate the efficacy of our multiscale recognition method in power equipment detection, with potential applications in constructing digital twin power grid equipment tags.

3.1 Overview

Existing single-stage object detection algorithms (e.g., YOLO [25] and SSD) pre-set prior boxes with different aspect ratios on the feature maps. A prior box will be classified as an object by the network if the overlap ratio between the prior box and the ground truth box reaches 0.5. The network gradually fine-tunes the sizes of the prior boxes through training to fit the ground truth object. However, due to the scale of the small objects, which have a relatively small range on the feature map, the corresponding prior boxes will have a larger deviation from the ground truth boxes. Thus, it is difficult for the network to filter the small objects and take them into the training process, which will also lead to poor detection performance for small objects during testing. The scales of different power equipment vary greatly. Large-scale equipment includes transformers, utility poles, etc.; densely packed small-scale equipment includes crossbars and insulators. As shown in Table 1, the scale of large equipment is tens or hundreds of times that of small equipment. The image of equipment with different scales is shown in Figure 1. It can be seen that the overall contour of the large-scale equipment is clear. But it is difficult to see the details of the small-scale equipment's details are difficult to see. Simply increasing the resolution of small-scale equipment is not a suitable solution. Because the large-scale equipment in the image will only have local information, The input image resolution of the SSD algorithm is 300×300 or 512×512. Small-scale equipment will lose more details at this resolution, making detection difficult.

Table 1. The ratio of object area to image size

2.png

Figure 1. Scale comparison between large equipment and small equipment: the green box is the area of large equipment, and the red box is the area of small equipment

To solve the problem of low detection accuracy caused by differences in the scales of electric equipment, we divide the dataset into dual-scale datasets (large-scale and small-scale) based on the size of the equipment. In this paper, we propose a detection approach based on training on dual-scale datasets and an improved single-stage multi-scale object detection algorithm. The network is trained on both the large-scale and small-scale datasets separately. The outputs of the two models are merged during testing, followed by non-maximum suppression (NMS), to generate unified detection results. Figure 2 shows the complete pipeline of the proposed framework.

1.png

Figure 2. Overall pipeline

The detection algorithm is single-stage with a feature extraction module, a feature fusion network, and a detection head. The feature fusion network combines multi-scale features with global context for object detection. To facilitate model deployment on edge devices such as drones, our backbone network replaces VGGNet [26] with the more efficient and lightweight EfficientNet [27]. We improve the original feature pyramid structure with multiple levels of feature fusion to balance the network’s ability to extract semantic features and locate objects. Consequently, our network is adaptable to different sizes of equipment data in real-world scenarios. The detection neck also incorporates an attention-like global semantic information module, which enhances the network’s focus on the target by exploiting spatial domain context information and weakens interference from cluttered backgrounds.

3.2 Dual-scale dataset

We classify data based on the equipment's true scale. Large-scale data refers to large-scale power equipment such as transformers and utility poles; small-scale data consists of dense small equipment such as crossbars and insulators. To balance the algorithm’s recognition ability for objects of different scales, we build two datasets of large and small-scale power equipment, respectively, and train the network separately. This strategy solves the problem of insufficient training of small objects caused by mismatches between prior boxes and ground truth, thus improving the accuracy of small-scale device detection.

We use real images of power equipment on transmission lines as the data source, which covers multiple scales, angles, and lighting conditions. The dataset includes four types of equipment: utility poles, transformers, insulators, and crossbars. The resolution of the actual shooting image is 3000×4000, and the equipment scale ratio is determined based on the area ratio of the equipment in the image. We classify poles and transformers as large-scale equipment, and insulators and crossbars as small-scale equipment. The dataset is annotated using the LabelImg tool, and the annotation format follows the PASCAL VOC 2007 dataset format. To facilitate network training, the resolution of the images in the training set is fixed at 300×300. Since large-scale equipment occupies the entire image frame of the actual image, the actual shooting image is scaled to 225×300 and padded with zeros around the edges to obtain the large-scale training set. Actual shooting images containing small-scale equipment are selected, and a series of sub-images are generated by traversing the image using a sliding window with a resolution of 300×300 and a stride of 15×15. The sub-images are cropped and labeled to create a small-scale dataset. Table 2 shows the number of objects for each class in the dataset. The dataset is randomly sampled and divided into training and testing sets, with a ratio of 9:1 for both large and small-scale datasets.

Table 2. Statistics of various objects in the dataset

In actual outdoor scenarios, there are complex natural environmental influences, including noise, backlighting, changing weather conditions, as well as complex scale and angle changes. It is difficult to gain rich and realistic data with manual photography, which limits the detection accuracy of algorithms in extreme situations. To improve the generalization ability of the model, enhance the robustness of the recognition algorithm, and strengthen the performance under actual scenarios, we follow the data optimization method explored by Zoph et al. [28] and have applied various data augmentation techniques to both datasets.

3.png

Figure 3. The network structure of EfficientNet-b3

3.3 Single-stage object detection algorithm

To optimize detection speed, edge devices require lightweight and efficient detection algorithms. The SSD algorithm has demonstrated commendable detection performance on public datasets, owing to its single-stage structure, which facilitates seamless integration across diverse platforms. In this paper, we introduce an enhanced object detection approach that integrates multiscale features and global context into the SSD algorithm. The network architecture of our algorithm is depicted in Figure 3. We improved the backbone network with EfficientNet-b3. The VGG network is based on the traditional stacked convolutional layer structure. But the network depth of VGGNet is not enough, and it uses more parameters. The EfficientNet is constructed by neural architecture search (NAS), achieving better accuracy and efficiency while expanding the network scale. By searching for optimization, the depth and width of each convolutional layer are determined, and then the modules are stacked to maximize the feature extraction ability of the model with limited model parameters. Introducing EfficientNet-b3 in the feature extraction part can significantly reduce the number of parameters and improve the feature extraction ability, enabling the model to increase feature extraction performance and be greatly lightweight, so that the model can be deployed on edge devices with limited device storage space. EfficientNet b3 is composed of seven modules in series. Each module contains a series of convolutional, batch normalization, and activation layers. The network structure diagram of EfficientNet-b3 is shown in Figure 3. We select the output feature maps from the 7th, 17th, and 25th convolutional modules for object recognition and detection, and the scale of these feature maps is 38×38, 19×19, and 10×10, respectively. For predicting the position and size of detection boxes, we select shallow feature maps, which achieve good results.

4.png

Figure 4. The network structure of the proposed single-stage algorithm

Object detection is a multifaceted task, encompassing two crucial components: classification and localization. The classification aspect involves learning high-level semantic information to establish invariant feature representations. Meanwhile, localization aims to pinpoint changes in object location and scale, necessitating an understanding of the interplay between the object and its background, as well as the relationships between local and global features. Achieving this necessitates the acquisition of rich equivariant features within the lower-level feature maps. Consequently, an effective detector must simultaneously learn both high-level semantic features and shallow, locally equivariant features. CNNs naturally organize themselves into a hierarchical feature pyramid. However, in the case of Single Shot MultiBox Detector (SSD), it independently detects objects of varying scales on different branches of these feature maps, overlooking the correlations between feature maps at various levels. This leads to an imbalance that affects the detection performance, causing a disparity between semantic and detailed information across different levels of feature maps. To address this issue and ensure more efficient utilization of extracted features, it becomes essential to enhance the semantic content of the shallow feature maps. In this research paper, we propose a novel approach: concatenating the feature maps from different levels for feature fusion. This concatenation approach offers several advantages. Notably, it obviates the need for converting feature maps from different levels to the same channel, a requirement in some other fusion methods. Consequently, it enhances the flexibility of fusing feature maps while reducing computational resource consumption compared to element-wise summation.

In our approach, we incorporate three feature maps produced by the backbone network and apply upsampling to standardize the feature map size to 38×38 while preserving a channel size of 256. Subsequently, we interconnect these feature maps to create a unified 38×38×768 feature map, as illustrated in Figure 4. This novel methodology ensures the seamless integration of both high-level semantic information and shallow, locally equivariant features, resulting in enhanced object detection performance.

5.png

Figure 5. The network structure of GCBlock

The convolution operation utilizes local information to calculate the target pixel, and the receptive field is determined by the size of the convolution kernel. Due to the inability to introduce global information, traditional convolution has limitations in prediction. Currently, there are some straightforward methods to alleviate this problem, such as using larger convolution filters or building deeper networks. However, these methods do not significantly improve the results. The receptive fields of specific layers are still limited even with feature fusion, which increases the computational costs. The SSD algorithm does not consider the impact of environmental context on local objects, greatly limiting its ability to recognize small-scale, occluded, and low-resolution objects. There is a certain positional correlation between different pieces of power equipment. For example, in large-scale datasets, transformer equipment is usually installed between two utility poles; in small-scale datasets, there are adjacent positions between insulators and crossbars. Introducing context information into the model can implicitly model the positional correlation between different pieces of equipment and avoid missed or false detections. However, not all environmental context information is beneficial to improving object detection performance; adding meaningless background noise may even impair detection performance. Therefore, identifying useful context information is necessary. We introduce a global context block (GCBlock) for modeling context semantics at specific query positions. It can help the model assign different weights to remote positions of the input, extract critical information, and make more accurate judgments without causing significant computational and storage costs. As shallow feature maps contain more context information, the first three feature maps output by the backbone network are input into the GCBlock for context information extraction to obtain more accurate positional correlation modeling. Figure 5 shows the network structure of the global context block.

During training, if the intersection over union (IoU) between the prior box and the ground truth box is greater than 50%, it can be considered that the prior box matches the true target. The network simultaneously performs object classification and box regression on the matched prior boxes, and the total loss function is calculated as Eq. (1).

$L(x, c, l, g)=\frac{1}{N}\left(L_{c l s}(x, c)+\alpha L_{r e g}(x, l, g)\right)$,                  (1)

where, $L_{c l s}(x, c)$ is the classification loss, $L_{r e g}(x, l, g)$ is the regression loss, $N$ is the number of matched detection boxes, and $\alpha$ is a weighting factor, which is usually set to 1 . We use the cross-entropy loss for classification, as in Eq. (2).

$\begin{array}{r}L_{c l s}(x, c)=-\sum_{i \in P o s}^N x_{i j}^k \log \left(\varnothing\left(c_i^k\right)\right) -\sum_{i \in N e g}^N \log \left(\emptyset\left(c_i^0\right)\right)\end{array}$                      (2)

where, $x, k, j \in \{0,1\}$ is the true class label of the object, using one-hot encoding, and $k$ is the number of object classes. $\varnothing$ is the Softmax function, which gives the probability that the target belongs to a certain class. $L_{r e g}(x, l, g)$ regresses the detection frame position and size using smooth L1 loss to calculate the relative distance between the detection frame and the real frame calculated as Eq. (3).

$l L_{r e g}(x, l, g)=\sum_{i \in P o s}^N \sum_{m \in B o x} x_{i j}^k \varphi\left(l_i^m-\overline{g_j^m}\right)$,              (3)

$B o x=\{c x, c y, w, h\}$,               (4)

$\overline{g_j^{c x}}=\frac{g_j^{c x}-d_i^{c x}}{d_i^w}$,           (5)

$\overline{g_J^{c x}}=\frac{g_j^{c y}-d_i^{c y}}{d_i^h}$,              (6)

$\overline{g_j^w}=\log \left(\frac{g_j^w}{d_j^w}\right)$,             (7)

$g_j^h=\log \left(\frac{g_j^h}{d_j^h}\right)$,              (8)

where, $l_i^m$ is the relative offset of the $\mathrm{m}$-parameters of the $\mathrm{i}$-th detection frame, $\overline{g_j^m}$ is the offset between the $\mathrm{j}$-th real labeled box and the prior box, $g_m^j$ and $d_i^m$ correspond to the $\mathrm{m}$ parameters of the real labeled box and the prior box, respectively.

In the quest for a digitized, automated, and intelligent approach to power grid planning, the transformation of the power grid into a digital counterpart is imperative. This transformation necessitates the creation of a "digital twin" for physical power equipment, facilitating precise classification and management. Our research introduces a pioneering deep learning-based method for the object detection of power equipment, leveraging staged training on dual-scale datasets to accommodate the diverse scales of power equipment. This methodology divides the dataset into large-scale and small-scale categories, with the model trained on each dataset to address the challenges of scale disparity, complex data labeling, and the nuanced learning requirements of smaller objects.

We meticulously constructed a dataset specific to power equipment identification, encapsulating a broad spectrum of scales, angles, and lighting conditions, comprising four distinct types of equipment: utility poles, transformers, insulators, and crossbars, all meticulously labeled. Through strategic sample selection and sophisticated data augmentation techniques, we have achieved an optimal balance between model generalization and applicability to specific scenarios. Our single-stage object detection model, tailored for data acquisition devices such as drones, sets a new standard in recognition accuracy and processing speed while requiring significantly fewer parameters than existing solutions.

The algorithm presented in this paper not only achieves an exemplary mAP of 92.13% and a detection speed of 52.15 fps but also demonstrates robust performance across a variety of natural conditions, including occlusion, shadow, and diverse weather phenomena. It adeptly handles the complexities of multiple scales, angles, and rotations in image acquisition, showcasing remarkable adaptability to the dynamic and intricate environments encountered in practical inspection scenarios. Notably superior to other deep learning-based object detection algorithms, our proposed method meets the stringent requirements for low-latency, multi-angle, and multi-target recognition and positioning of power equipment in the domain of intelligent grid line identification.

This research not only marks a significant advancement in the field of power equipment recognition but also lays the groundwork for the digitization and intelligent management of power grids. By striking a balance between accuracy, efficiency, and model compactness, our algorithm paves the way for the seamless integration of digital simulations and data management tasks. This contributes substantially to the realization of an intelligent power grid planning system, emphasizing our method's potential to revolutionize energy management practices and underscore its vital role in promoting sustainable and efficient power grid operations. Through this work, we envision fostering a future where digital twin technologies are at the forefront of innovative power grid planning and management, driving progress towards a more sustainable, efficient, and intelligent energy infrastructure.