Keywords

1 Introduction

Pavement distress detection [1] plays a key role in highway maintenance and rehabilitation due to the fact that surface crack is the most common distress of all pavement distress. Therefore, the detection of crack is necessary for the highway especially high-grade highway maintenance. Traditionally, pavement crack detections are accomplished by manual approaches, which are not only time-consuming, labor-consuming, and dangerous, but costly. More importantly, besides the fact that ongoing detections affect traffic, the quality of detection is determined subjectively. Thus, the automatic pavement crack detection methods should be applied to overcome the shortcoming of conventional visual and manual method.

There have been many thoughtful studies on cracking auto-recognition [2, 3]. In general form, crack image detection is composed of preprocessing, description and identification. Pavement surface image has two characteristics: small scale of gray values, and small difference between background and objective within original image. In addition, there could be many noises in the image due to the diversity of image acquisition and the difference of surface materials. Many researchers have paid great attention to crack image processing in recent years, and the study centralized on the image enhancement and segmentation [4, 5].

In recent years, the advanced convolutional neural networks can automatically learn feature and generalized, which was applied to the pattern recognition tasks, and received a good response, such as image classifications [6], target detection [7], vehicle detection [8], and so on. But now crack classification is rare using convolutional neural networks.

The remainder of this paper is organized as follows. Section 2 presents the crack detection method. Section 3 describes the crack reorganization using convolutional neural networks. Section 4 demonstrates extensive experimental comparison results. Section 5 finally draws the conclusions.

2 The Detection of the Crack

2.1 Preprocessing

Gamma correction is an important nonlinear transformation. It is actually a transformation of the input image gray value, and then correct the brightness deviation, which usually is applied to expand the details of the dark light. In general, when Gamma correction value is greater than 1, the image highlights parts will be compressed and dark color will be expanding. When Gamma correction value is less than 1, highlights part is expanding and dark part of the image is compressed. By definition the following formula:

$$ V_{out} = AV_{in}^{\gamma } $$
(1)

where A is constant, \( V_{in} \) is the input image before correction, \( V_{out} \) the rectified image, γ is gamma correction parameter. In this experiment, after repeated tests, when γ = 0.75 for crack images, the results will be the best. Gamma can be used to enhance the crack fractures.

Gaussian filter is a linear smoothing filtering, which is applicable to eliminate the Gaussian noise. It is widely used in image processing of the noise reduction. Popular speaking, Gaussian filtering is a process to the weighted average of the whole image, each pixel values, all by itself and other neighborhood pixels after the weighted average (Fig. 1).

Fig. 1.
figure 1

Image preprocessing result: (a) original image, (b) Gamma correction and (c) filtering result

Full size image

2.2 The Segmentation of the Crack

In [9], Lankton et al. proposed a natural framework that allows any region-based segmentation energy to be re-formulated in a local way. For example, we choose the global region-based energy that uses mean intensities is the one proposed by Yezzi et al. [10] which we refer to as mean separation energy:

$$ E_{MS} = \int_{{\Omega _{y} }} {(u - v)^{2} } $$
(2)

where \( u \) and \( v \) represents the global mean intensities of the interior and exterior regions, respectively. Optimizing this energy causes that the interior and exterior means have the largest difference possible (Fig. 2).

Fig. 2.
figure 2

The segmentation result

Full size image

\( B(x,y) \) is introduced to mask local regions. This function \( B(x,y) \) will be 1 when the point \( y \) is within a ball of radius \( r \) centered at \( x \), and 0 otherwise.

Accordingly, the corresponding \( F \) is formed by localizing the global energy with local mean equivalents as shown:

$$ F_{MS} = (u_{x} - v_{x} )^{2} $$
(3)

We can get the following local region-based flow:

$$ \begin{array}{*{20}l} {\frac{\partial \phi }{\partial t} = \delta \phi (x)\int_{{\Omega _{y} }} {B(x,y)\delta \phi (y) \cdot (\frac{{(I(y) - u_{x} ))^{2} }}{{A_{u} }} - \frac{{(I(y) - v_{x} ))^{2} }}{{A_{v} }})dy} } \hfill \\ {\,\,\,\,\,\,\,\,\,\, + \;\lambda \delta \phi (x)div(\frac{\nabla \phi (x)}{{\left| {\phi (x)} \right|}})} \hfill \\ \end{array} $$
(4)

where \( A_{u} \) and \( A_{v} \) are the areas of the local interior and local exterior regions respectively given by

$$ A_{u} = \int_{{\Omega _{y} }} {B(x,y) \cdot H\phi (y)dy} $$
(5)
$$ A_{v} = \int_{{\Omega _{y} }} {B(x,y) \cdot H(1 - \phi (y))dy} $$
(6)

2.3 The Location of the Crack Based on OpenCV

After the image preprocessing and segmentation module, we can locate the crack in the original image. The specific method is morphologic closing operation to enhance regional characteristics of crack. Due to the segmentation of pavement cracks is a certain connected components of the image, by image contour detection function, to search for each connected component contour matrix, to judge the size of the contour by scanning in turn, to select the largest contour matrix which is selected as the largest connected component. Also it is available through intercepting the area by Region of interest function (Fig. 3).

Fig. 3.
figure 3

The diagram of the locating algorithm

Full size image

The results of the locating algorithm are shown in Fig. 4.

Fig. 4.
figure 4

The running samples of locating algorithm of pavement cracks

Full size image

3 The Recognition of the Crack

In recent years, deep convolutional neural network has been very popular. It can learn features by itself and shows excellent generalization ability. And it has been widely used in the areas of computer vision and pattern recognition. However, at present, it is rare to apply convolutional neural networks to this issue. This paper aims at solving aircraft classification problem in optical remote sensing images using convolutional neural networks.

For crack classification problems, this convolution neural network structure is shown in Fig. 5, the network has 5 layers, the first 4 are a convolution layer, the final layer is for all connections.

Fig. 5.
figure 5

The structure of convolutional neural network

Full size image

The input of Network are the 256 × 256 size of the three channels (RGB) images. To ensure the network depth, to reduce the parameter, to improve network generalization, the convolution kernels using in the first three volumes of the network layer are smaller, they are 3 × 3. Finally a convolution kernel size is 15 × 15. All the convolution kernel effect simultaneously on all the features to the previous one, and the same different characteristics of convolution kernel to the former figure weight is not consistent. Four layers of convolution type of convolution kernel used is 64, 96, 96, 216, layer-by-layer abstraction to extract different features.

In the first three volumes are all connected to the pooling layer the pooling method is the largest pooling. Images are not expanded in the pooling, the window size of pooling are 2 × 2, 3 × 3, 4 × 4, sliding windows step are 2, 2, 4. After each pooling layer are nonlinear activation through the LeRU.

The final layer are all connection layer, a total of 8 nerve cells. The final convolution and all connection layer are added by random dropout layer to improve network generalization, the percentage is 0.5. Finally softmax function is used for output.

4 Experiments

The experiments are implemented with VC++ code run on a Dell Dimension 4600 PC, with Pentium 4 processor, 2.80 GHz, 1 GB RAM, on Windows XP.

They are downsampled as 8 * 8 images, which are composed of 128 dimensional vector as input layer, the hidden layer of 3 layers and 128 dimensions, the output layer of 10 dimensions. The images are classified to four categories (transverse crack, longitudinal crack, reticular crack sets, excess road information set), each type of 50 crack images are for training, and crack classifier is exported. After experiment and testing, and the crack can be accurately identified. Furthermore, high recognition rate.

The results of different crack images recognition are shown in Fig. 6. We further list the testing accuracy between the proposed method and the classic SVM (support vector machine) method (Table 1).

Fig. 6.
figure 6

The results of different fracture images recognition

Full size image
Table 1. Comparison with different methods
Full size table

5 Conclusions

In this paper we present a novel convolutional neural network based cracking identification method. Image preprocessing is introduced first as consisting of Gamma correction, spatial filtering, and binary processing. Then by segmenting pavement crack images using active contour model and locating crack, we locate the cracks efficiently, and use them to train our neural network. After convolutional neural network converges and identifies crack from pavement surface image. We finally revise convolutional network outputs to remove isolated crack points and connect crack for better identification accuracy. Our experiment indicates that crack could be identified by the convolutional network based identifier correctly and automatically.