Keywords

1 Introduction

Power line detection (PLD) have attracted increasing attentions in recent years. This is mainly because PLD is of great importance and has many important applications in both military and civil fields. For example, PLD is vital for flight safety of helicopter and UAV etc. The United States army reports that they have lost more helicopters to power lines than against enemies in combat [1]. Besides, PLD can be applied in automatic surveillance of electrical power infrastructure [11, 13] and power line inspection [19]. This is very helpful to locate the points of faults on power lines and to diagnose the components equipped on power lines. Also, this helps to save a huge amount of money which is spent on manual power line surveillance.

In past decades, with the continuous increasing of camera resolution, detecting power lines from aerial images directly have become possible, which is not possible previously. Since then, many methods based on traditional image processing techniques have been proposed to detect or recognise power lines, for example using the edge detection and Hough transform. Some of these works are based on visible images, while some others utilise infrared images.

However, there are some shortcomings of traditional image processing-based methods. Firstly, featured used to detect lines are designed manually, which may not be very good, and this process is time-consuming and labour-intensive. Secondly, the frequently-used Hough transform is mainly designed for straight line detection. However, power lines may not always be straight lines. Thirdly, these algorithms may be computationally expensive since pre-processing and time-consuming feature extraction are needed. Also, many prior knowledge about power lines is needed in designing traditional algorithms, however this is labour-intensive to implement.

Consequently, a more robust, accurate, cheap and intelligent PLD method is necessary. Considering the great success of deep learning in various fields, in this work we propose to deal with PLD problem using deep learning.

The following parts of this paper are organised as follows. Section 2 gives some background information about PLD, then Sect. 3 presents a review of PLD approaches based on image processing. Section 4 will present the proposed deep learning framework for PLD and experimental work will be presented in Sect. 5. Then, Sect. 6 will give the results. Finally, Sect. 7 concludes the paper.

2 Background

2.1 Power Line Detection

PLD is an important yet challenging problem. One factor making PLD challenging is the complex or cluttered background with cloud, trees, grasses etc. Figure 1 presents some images containing power lines. As it can be seen, the background varies from human eyes. Another factor is the features of power lines themselves.

Fig. 1
figure 1

Taken from public dataset (https://data.mendeley.com/datasets/n6wrv4ry6v/7)

Some power line instances with various backgrounds.

Full size image

2.1.1 Features of Power Lines

Power lines in aerial images normally have some features compared to other objects [13, 27]. Firstly, power lines are very small and thin. Secondly, power lines can be approximated reasonably well by straight lines from overhead view. Thirdly, a power line is made of specific metal and has larger light reflection, thus from downward view it is usually brighter than its surroundings. Besides, a power line usually has uniform brightness. Finally, power lines are approximately parallel to each other. This is because the intersection of two power lines usually occurs far out of the image due to the limited size of images, and the intersecting angle of two lines is usually very small, as shown in Fig. 1.

On one hand, these features make the PLD problem even more challenging. On the other hand, these knowledge can be used to guide designing PLD algorithms, like presented by Li et al. [13].

2.1.2 Warning Sphere for Power Lines

Considering the features of power lines, some measures have been taken to help pilots to recognise power lines. One of the most frequently-used methods is the warning sphere. Normally, warning spheres are installed in places near airport as shown in Fig. 2 which illustrates some warning spheres near Shanghai Pudong International Airport in China. Although warning spheres can help to recognise power lines to some extent, it is not quite enough. It is difficult and time- consuming to install warning spheres. What’s more, it is not possible to install warning spheres everywhere, image to image, and sometimes it is very difficult to distinguish power lines from background even with thus only a very small part of power lines can have convert these spheres. In addition, sometimes it is possible that these warning spheres fall down, which is dangerous for people or houses beneath these power lines.

Fig. 2
figure 2

Warning spheres on power lines near Shanghai Pudong International Airport

Full size image

2.2 Machine Learning and Deep Learning

In recent years, machine learning and deep learning have attracted great interests in both academic field and industries. The fast development of them is mainly driven by the greatly increased amount of data and computing power.

Machine learning and deep learning have been applied in many fields and have provided significantly better performance than traditional methods. For example, in computer vision [25], robotics [12], medical imaging [15] and even Go [21, 22]. Compared to traditional machine learning algorithms, in past years deep leaning have exhibited better performance in various areas due to its advantages. For instance, it can automatically learning effective features, which has been well demonstrated in many computer vision problems. In addition, although it takes time to train the algorithm, it works quite effectively in detection once trained.

From the literature it can be found that some researchers have started to consider using machine learning to solve PLD problem. However, to the best of our knowledge, there is still no work introducing deep learning to this field. In the present work, we will utilise deep learning to solve PLD problem.

3 Related Work

Generally speaking, there are three main kinds of PLD methods. The first kind is laser-based method. For example, Avizonis and Barron [1] present the very early research on PLD using laser-based method. The second kind is radar-based method as introduced in some studies [18, 20]. The last kind is image- based methods, which are the focus of this work.

3.1 Power Line Detection Based on Traditional Image Processing Method

3.1.1 Detecting Power Lines as Straight Lines

As presented in Sect. 2, power lines can be approximated using straight lines in aerial images. Therefore, one can utilise straight line detection methods to detect those power lines, for instance the Hough transform [9] and Radon transform [26].

3.1.1.1 Hough Transform

The Hough transform is a classical straight line detection method, which is insensitive to noise and has been frequently applied to detect power lines. The main principle of Hough transform is to convert a line in the Cartesian coordinate into a point in the parametric space (Hough space), as illustrated in Fig. 3.

Fig. 3
figure 3

Mapping of one line in Cartesian space to a point in the Hough space

Full size image

Li et al. [13, 14] proposed a power line inspection system which consists of three parts. The first part was the Pulse-Coupled Neural Network (PCNN) filter which was used for removing background noise. The second part of the Hough transform was used to detect straight lines. The third part was knowledge based line clustering employed to refine detection results. Specifically, K-means clustering algorithm was utilised. Similarly, the method proposed by Wu et al. [27] consists of three steps. Compared to work of Li et al., the edges of power lines were extracted using the SWIFTS algorithm before the Hough transform was employed to detect power lines. Besides, the Nearest Neighbourhood (NN) algorithm was implemented to distinguish different lines. However, although experiments have shown that the algorithm presented by Wu et al. [27] is able to detect power lines, it is very time-consuming thus it may not be used in real applications.

Zhang et al. [34] performed study on power line detection and tracking for a UAV-based inspection. In that work, the Hough transform was used to detect lines, followed by a K-means algorithm to cluster and filter straight lines. Then, a Kalman filter was employed to track power lines. Liu et al. [16] also explored to detect power lines through Hough transform, but before applying the Hough transform, they did some pre-processing to obtain clearer images and used morphological operator for edge detection.

Apart from traditional Hough transform, researchers have also proposed some improved Hough transform and applied to PLD problem. For instance, Ji et al. [10] proposed an improved Hough transform and applied it to line detection. In that work, a local operator in the parameter space was defined to help to find peak and avoid parameter tuning needed in traditional Hough transform. Fernandes et al. [8] presented an improved Hough transform, aiming at achieve real-time performance. Specifically, they presented an improved voting scheme in the Hough transform that allows real-time performance for relatively large images. Candamo et al. [3] presented the method to detect thin lines in urban settings using a windowed Hough transform, which can avoid incidental edge alignment produced by noisy edges.

3.1.1.2 Radon Transform

There are also some works that conduct PLD based on Radon transform, which is another well-known line detection method. For example, Yan et al. [28] conducted PLD based on Radon transform. They firstly extract line segments using Radon transform, and then use grouping method and the Kalman filter to obtain an entire line. They claimed that their method can extract power lines successfully regardless of background complexity.

Researchers have also tried to improve Radon transform to obtain better performance. Chen et al. [7] presented a Cluster Radon Transform (CRT) PLD method based on improved Radon transform to extract linear feature from satellite image and can reduce/avoid alarm. Cao et al. [5] proposed a Boundary Search Radon Transform (BSRT) by assuming that the initial point of an integral line is on one of the image boundaries.

3.1.1.3 Other Methods

Some other methods have also been applied in detecting power cables as lines in images. For example, Ceron et al. [6] proposed a PLD method using a Circle Based Search (CBS) method. Yetgin et al. [29] proposed a method based on the EDlines method. Yetgin et al. [33] gave a comparison between three image-based PLD methods, i.e. EDLines, LSD and Hough Transform. They claimed that EDLines method can provide higher accuracy among these three approaches. Song et al. [24] presented an image-based local to global PLD method. Firstly, morphological filter is used to detect a line segment pool, and then these line segments are grouped into whole power line.

3.1.2 Detecting Power Line as an Object

In works presented above, power lines are considered as lines in images, rather than objects. Therefore, line detection method are utilised in those studies. However, it is also possible to detect power lines by recognising them as objects, in order to obtain better detection performance. Candamo et al. [4] presented very early research on this direction and proposed a method by considering the line and its surroundings, using the Gaussian model. Although linear patterns are searched in that work to find wires, the surroundings around lines are also considered to discriminate wires from other visually similar linear patterns. In this case, this method actually consider power lines as objects. Results in that work showed the method can improve detection performance in the condition of highly cluttered urban backgrounds, moderate rain and mist. Luo et al. [17] developed an object-aware PLD method which utilises the object properties of power lines. In that study, apart from RGB images, near-infrared (NIR) images are also employed to assist the detection of power lines.

3.2 Power Line Detection Based on Machine Learning

In the past, most works treated the power line as a straight line, and use method like Hough transform to detect it. However, the power line may not always be straight line, and may not always start from the edge of images. Consequently, the traditional image- based methods may not work well. To overcome these problems, one can turn to machine learning technique. Actually, with the increasing development of machine learning technique, some researchers have tried to conduct PLD problem with machine learning. When considering using machine learning to solve PLD problem, one can conduct in two ways. The first one is formulating the PLD problem as a binary classification problem, i.e. to judge whether the scene contains power lines or not. At the moment, most works are following this way [30, 31]. The second one is to formulate PLD as a detection problem.

PLD method based on machine learning normally consists of several stages. Firstly, one needs to do pre-processing for images, for instance to resize images and remove noise. Then, features should be extracted and selected in proper way. Finally, the features are fed into classifiers to do classification. The flow chart illustrating the whole process is given in Fig. 4.

Fig. 4
figure 4

The flow chart of PLD based on machine learning

Full size image

Yetgin et al. [30] conducted a study using binary classification to perform power line scene detection. In that work, new feature extraction/selection strategies based on Discrete Cosine Transform (DCT) have been applied to scenes obtained from aircraft-based cameras. The accuracy of their work on a public dataset was around 89.5% when using visible images.

Note that apart from formulating power line detection as a binary classification problem, in some of the above-mentioned works, some machine learning algorithms have been applied as a part of the power line detection method. For instance, in the work presented by Li et al. [13, 14], K-means was utilised. In the work of Wu et al. [27], the Nearest Neighbourhood (NN) algorithm was chosen. In addition to these studies, Bhola et al. [2] proposed a method to detect power lines in UAV remote sensed images using spectral-spatial methods, where the K-means algorithm have also been employed.

4 Method

Unlike traditional PLD methods based on image processing and machine learning, we propose to apply deep leaning to power line detection. Specifically, the convolutional neural network which is famous for handling images is utilised in this study. Besides, we treat the power line detection problem as a binary classification problem, namely we recognise whether the image contains power line or not. In real applications, this method can find the scene containing power lines and then give an alarm to remind pilots. This is in contrast to some studies mentioned above, which detect and locate lines directly in images.

The deep learning framework utilised in this work is presented in Fig. 5. The network structure is very similar to the famous VGG-16 network [23], containing 13 convolutional layers, pooling layers and 3 fully-connected layers. The convolutional kernel size is 3 × 3. To adapt to PLD problem, the last fully-connected layer is changed to only have two units. In this work, all weights and bias in the network are trained and optimised using labelled training images. Note that the input image should be resized to 224 × 224 in order to be processed by the deep learning model.

Fig. 5
figure 5

Power line detection based on convolutional neural networks

Full size image

5 Experimental Work

5.1 Dataset

Thanks to the work of Yetgin et al. [32], there is a public power line dataset available online. In the present study, this dataset is utilised to show the effectiveness of the proposed method in finding power lines from images. This dataset contains 4000 visible and 4000 infrared images captured all over Turkey with various backgrounds, temperatures and weather conditions. Among each kind, there are 2000 images with power line and 2000 images without power line. In the present study, only visible images are utilised.

5.2 Training of the CNN Model

Before training the network, all images are resized to the size of 224 × 224 to fit into the requirement of the CNN model. Main parameters of the train process are listed in Table 1. Note that among 4000 visible images in the dataset, 3000 images are used in training set, 1000 are in the test set.

Table 1 Main parameters in the training process
Full size table

The computer used to train the model is a desktop with an AMD Ryzen 5 1400 Quad-Core Processor (3.20 GHz) and 8 GB RAM. Besides, a NVIDIA Geforce GTX 1070 GPU is equipped. The training time for 6000 epochs is around 7 h. The loss and accuracy during the training process are given in Figs. 6 and 7, respectively. As it can be seen, the loss has been significantly reduced while the accuracy has been greatly enhanced after training, which indicates the effectiveness of the training process.

Fig. 6
figure 6

Loss function value during training process

Full size image
Fig. 7
figure 7

Accuracy value during training process

Full size image

6 Results and Discussions

6.1 Results

Two tests have been performed using the trained model to evaluate its performance. In Test 1, 100 images containing power lines from the public dataset are fed into the trained model. In Test 2, 100 images do not contain power lines from the public dataset as well are utilised. Note that these images have not been used in the training process. The test results are listed in Table 2. As it can be seen, 95% of images which include power lines can be correctly recognised, although a part of images which do not have power lines are classified having.

Table 2 Test performance using the trained model
Full size table

Note that although it takes several hours to train the model, it is very fast to perform testing. This can actually satisfy the requirement of real applications, i.e. the model can work in real time.

6.2 Discussions

As illustrated in Table 2, the proposed deep learning-based method can recognise power lines from various images with the accuracy of 95%. This is significantly higher than the accuracy of traditional image processing methods for the same dataset, which is only 89.5% [30]. To the best of our knowledge, this is the first study that apply deep learning into PLD problem. The significantly enhanced accuracy clearly demonstrate the strength and potential of deep learning in PLD.

However, the precision of above tests is around 72.0%, which means that the trained model will incorrectly find power lines from images which do not include them. In practical applications, this will give some false alarm to pilots.

There are several factors that may affect the performance of the network, which are listed as follows:

  • Network architecture. It is well-known that the network structure, for instance the number of units, layers and size of kernels, have huge affection on performance of the model.

  • Number of images in the dataset. Another very important factor in deep learning is the amount of labelled data. Generally speaking, the more data used, the better performance obtained. In this study, we utilised the public power line dataset to train the model. However, 4000 images are not really enough for a network to be trained from scratch, therefore, it can be expected that if more labelled data are available, better performance can be achieved. Alternatively, if advanced techniques which can efficiently utilise limited data were employed, better performance would also be achieved.

  • Hyperparameters in the CNN model. The hyperparameters in the CNN model, for instance the learning rate, also have affection on the network performance. In this work, we chose these parameters based on experience and trials.

To sum up, compared to traditional image processing-based and machine learning-based methods, deep learning can automatically extract and select effective features through learning from training set. Consequently, deep learning shows good performance in PLD problem although there is still much space to be improved.

7 Conclusion

In this paper, deep learning technique is applied to the power line detection (PLD) problem. Specifically, the convolutional neural network (CNN) is employed. To the best of our knowledge, this is the first public work attempting applying deep learning in PLD problem. Labelled images from public power line dataset are used to train the CNN model and test the feasibility of the proposed method. Experimental results clearly demonstrate that the proposed deep learning method can correctly recognise 95% of images containing power lines, which is much better than traditional image process or machine learning-based approaches.