Keywords

1 Introduction

Transmission lines are important carriers for energy transmission in the transmission network, so they need to be regularly inspected. The threat of flying kites, balloons, and transmission lines has occurred all over the place. Traditional manual inspections are inefficient and subject to geographical constraints. In order to improve the efficiency of manual inspection, the intelligent inspection of transmission lines has achieved rapid development. The number of images acquired by the image pickup equipment mounted on the transmission tower is increasing, and if it is manually detected, it takes a lot of manpower. At the same time, the background of the image is complicated, and the precision of manual detection is low, which is easy to cause misjudgment. The application of deep learning technology to identify foreign objects in transmission lines can improve the accuracy of recognition, make full use of the accumulated image data resources for intelligent detection and identification. Therefore, it is necessary to study the intelligent detection technology to realize the automatic detection and identification of foreign objects in the transmission line.

2 Related Works and Solutions

The early detection methods of foreign bodies in transmission lines are mainly based on image processing techniques, For example, in the papers [1], the author based on image segmentation, filtering to reduce noise, using Canny edge detection, Hough transform for line detection is realized. In the papers [2, 3], on the basis of image segmentation, the author uses image feature matching to realize transmission line detection, For example, ASIFT feature, Harr feature, texture feature, gray level co-occurrence matrix, etc. In the paper [4], the author used a camera to acquire images and used the acquired images to detect ice covering wires. In addition, Wang et al. [5] designed an algorithm based on support vector machines (SVM) which can automatically analyze, characterize, and classify a fault based on its root cause.

However, due to the complex background texture of the captured images, it contains trees, fields, rivers, and roads, which results in a complex background and a large difference in the image acquisition, and brings great challenges to the foreign object recognition work of the transmission line. In recent years, object detection based on convolution neural networks has achieved good results in low-altitude targets, face detection, and image detection of drones. In the paper [6], Guo build a simple Convolutional neural network on image classification, and analyzed different methods of learning rate set and different optimization algorithm of solving the optimal parameters of the influence on image classification. In the papers [7, 8], the neural network is used to detect the components of the transmission line, and the transmission line fault detection algorithm is designed for the specific environment.

3 Introduction to the Faster R-CNN Model

In 2014, Girshick et al. proposed the Region-based Convolution Neural Network (R-CNN) [9] model, and proposed to perform target detection based on the selected search to extract region proposals and CNN for image feature extraction, which greatly improved the target detection accuracy. In 2016, Ren et al. proposed the Faster R-CNN [10] model based on R-CNN using a region proposal network (RPN) instead of a region search algorithm to extract region proposals. The four steps of feature extraction, extract region proposals, target detection and border regression are implemented in Faster R-CNN model. As shown in Fig. 1.

Fig. 1
figure 1

The model of faster R-CNN

Full size image

The functions of the four main parts are as follows: the convolutional layers uses a set of convolution operations for feature extraction. ROI Pooling extracts feature maps of the input image, and the extracted feature weights are followed by the RPN layer and the fully connection layer sharing. RPN is mainly used to extract region proposals. The essence of the output is the rectangular target object region proposals and the score of the target object. The ROI Pooling layer is mainly based on feature maps and region proposals, and is used for target recognition of subsequent fully connected layers. The role of the Classification layer is to calculate the category of the proposal using the feature maps, and the bounding box regression method calculates the final precise position of the detection frame.

4 Foreign Object Detection of Transmission Line Based on Faster R-CNN

4.1 Data Set and Experimental Environment

Due to the particularity of the power system, there is no large-scale professional data set, which brings great difficulties to the training of convolutional neural networks. The image used in this paper is from a transmission line image captured by a high-definition camera fixed on a transmission tower in different scenes and weather conditions. And a data set is constructed from the original collected image, and the resolution of the collected image is 1280 * 720. Randomly select 9/10 images for manual annotation as the training set, another 1/10 as a test set. At the same time, in order to ensure the accuracy and generalization ability of the Faster R-CNN model, the pre-training model based on ZFNet, VGG_CNN_M_1024 and VGG16 networks is used to initialization it. The produced data set contains a total of 12,000 images, and the number of channels of the image is 3.

The examples of dataset are shown in Fig. (2).

Fig. 2
figure 2

Data set examples

Full size image

The experimental hardware environment is Inter Core i5 9600K and GTX1070Ti 8G graphics card. The software environment is based on the Caffe framework for training and testing. The operating system chooses Ubuntu 1604.

4.2 Model Training and Testing

This paper trains the Faster R-CNN model using the classic 4-step iterative training strategy to achieve alternate optimization learning to share features. The main steps are: (1) Pre-training the RPN network. Different network models are used for initialization and training on the transmission line data set. (2) Train the Fast R-CNN target detection network separately using the region proposal obtained from the RPN trained in the first step. At this point, there is no shared convolution between the RPN and the Fast R-CNN detection network. (3) Fixed the parameters of the convolutional layer and initialized the RPN network with the feature extractor of CNN. (4) Keep the shared convolutional layer parameters unchanged and precisely adjust the unique layer parameters of Faster R-CNN.

The overall frame of the foreign body detection of the transmission line can be divided into four steps, as shown in Fig. (3).

Fig. 3
figure 3

Data set examples based on the faster R-CNN foreign object detection framework

Full size image

The trained Faster R-CNN model is used to extract features of each region proposal and classify them. Based on the similarity of confidence for each region proposals with a given foreign object category, the region above the confidence level is used as a positive sample. When the confidence threshold is set to 0.7, the actual detection effect is shown in Fig. (4). In order, they are kite, bird, balloon and plastic bag.

Fig. 4
figure 4

Object detection results by faster R-CNN

Full size image

5 Experimental Results and Analysis

In the experiment, kites, balloons, etc. are used as the characteristics of foreign object recognition in the transmission line, and whether the test sample contains foreign matter as the sole criterion for identifying images. The experiment uses the commonly used mean average precision (mAP) in the object detection field as the main evaluation index of the foreign object recognition accuracy of the transmission line, and the average detection time is used as the auxiliary evaluation index. There are a total of 4 categories for the data set, and the average precision value is used to represent the effect of foreign object recognition on the transmission line.

The experimental results are as follows (Table 1).

Table 1 Faster R-CNN model detection results based on different network structures
Full size table

The test set is tested using the Faster R-CNN model based on the ZFNet, VGG_CNN_M_1024 and VGG16 network models, respectively. It can be seen from the table that the mAP using the model based on the small ZENet network is 88.54%, the model mAP using the medium VGG_CNN_M_1024 network training can reach 90.23%, and the model using the large VGG16 network training can achieve 91.35% mAP. It can be concluded that the Faster R-CNN model based on VGG16 has higher detection accuracy than the other two models, and the average detection time is relatively reduced.

6 Conclusion

Aiming at the problem that the foreign object detection method of traditional transmission line is susceptible to illumination and complex background, this paper studies the foreign object identification method of transmission line based on Faster R-CNN model on the self-built transmission line dataset. The method uses the deep convolution network to automatically extract the target features, which effectively avoids the instability of the traditional manual selection target features. Contrast experiments were carried out based on the Faster R-CNN model of three convolutional neural networks. The experimental results show that the detection model based on VGG16 network is better than the model based on ZFNet and VGG_1024_M network. Compared with the traditional transmission line detection method, the Faster R-CNN model trained in this paper has improved detection accuracy and speed, and can achieve ideal detection results for multiple targets, occlusion, blur, illumination changes, complex background and other situations.