A Novel Method of Automatic Crack Detection Utilizing Non-linear Relaxation Method and Crack Geometric Features for Safety Evaluation of Concrete Structure

Abstract

Traditionally, surface cracks of concrete structures are manually measured and recorded by experienced inspectors. Recently, many crack detection techniques utilizing image processing have been proposed to detect cracks in 2D piecewise constant image. However, crack images in real applications such as building safety evaluation and bridge maintenance are usually 2D multiple-phase piecewise constant images. Furthermore, these methods require the user to choose appropriate parameters, especially the threshold in image segmentation step, for a certain set of images with similar contrast. This paper presents a novel automatic method for the efficient detection of cracks in 2D multiple-phase piecewise constant crack image without predefined thresholds in image binarization step. Firstly, cracks are enhanced utilizing 2 geometric features of crack including tubular and symmetric properties. In the second step, we proposed a crack-feature based non-linear relaxation method to detect crack from the enhanced images. Finally, we proposed a crack probability based binarization method to extract the binary crack map. The robustness of our proposed method are demonstrated in various crack images of building concrete structures.

1 Introduction

Image-based crack detection is highly desirable for automatic and objective crack assessment in building safety evaluation or maintenance of concrete structures, which suffer high loads such as columns of buildings and bridges. Traditionally, experienced humans detect and measure cracks using conventional visual and manual approaches, which are very costly, time-consuming, labor-intensive, and subjective [1]. Recently, many image processing techniques have been proposed to achieve automatic crack detection. However, accurate detection and localization of cracks from real images of building or bridge is difficult because these images usually contain various blebs, irregularities in crack shape and size, and other unintended objects of non-concrete structures.

Available techniques of automatic crack detection in still images typically work by preprocessing the image to enhance the contrast between cracks and background before a binarization step (e.g. thresholding) is applied. Hutchinson and Chen [2] used the Canny filter and wavelet transformation for crack detection. Ito et al. [3] combined some image processing techniques such as wavelet transform, shading correction, and binarization to detect cracks. Yamaguchi et al. [4] proposed a crack-detection method based on the percolation model in which binarization is included after the percolation processing. Abdel-Qader et al. [5] compared the effectiveness of crack detection among wavelet transform, Fourier transform, Sobel filter, and Canny filter. All of these methods require the user to choose appropriate binarization thresholds for each set of images.

Lee [6] proposed a system for automatic inspection of bridges but including a post-processing supervised manipulation. This method is claimed to be able to detect crack automatically by using some techniques such as median filter, morphological image processing. On the other hand, those techniques can only be applied for two-phase piecewise constant image and are not applicable for crack image with clear step edge.

Shan et al. [7] used stereo-vision camera to detect edges of cracks. This method proposed a novel Canny-Zernike combination algorithm, which can obtain the image coordinates of crack edge in the image accurately. However, this method can not discriminate between edge of cracks and edge of other non-crack objects in the images.

Chen et al. [8] developed a crack detection system using multi-temporal images by utilizing the first derivative of a Gaussian filter. This method can compute the crack width in a controlled environment but requiring users to put nodes along the crack, and thus is not automatic.

Fujita and Hamamoto [9] used Frangi’s filter [11] and probabilistic relaxation to automatically segment cracks from noisy real concrete surface images. However, Frangi’s filter also enhances the step edges in the image so this method failed to detect crack from 2D multiple-phase images that have clear step edges.

Wang et al. [10] proposed an algorithm that integrates morphological approach, percolation approach and practical technique. This technique can detect crack accurately but needs user intervention. Nguyen et al. [12] used B-spline level set method to detect crack in 2D images but it takes a lot of computing resources for the level set model to converge.

In this paper, we proposed a novel automatic crack detection method for localizing cracks from images of buildings, bridges or other civil engineering infrastructure that contains concrete structures and unintended objects. To classify between step edges and cracks in images, we assumed that cracks possesses two geometric properties including tubular and symmetric properties. Generally, we proposed 3 main steps for automatic crack detection. In the first step, we need to enhance crack and remove other unintended objects in the images by utilizing 2 geometric features of crack. In the second step, we adapt a non-linear relaxation method to segment crack automatically from the enhanced crack images. In the final step, a proposed crack probability based binarization method is used to extract the binary crack map.

2 Proposed Method

2.1 Two and Multiple Phases Piece-Wise Constant Image

Images of surface concrete structures, which are demonstrated in available automatic crack detection techniques [1,2,3,4,5,6,7,8,9,10], are usually 2D piecewise constant images. However, in our work, we try to detect cracks from 2D multiple-phase piecewise constant images of civil infrastructures such as building, bridges… that contain surface cracks. To understand the differences between two-phase piecewise constant image and multiple-phase piece-wise constant image, we can look at the Fig. 1. The right image is the 2D multiple-phase piecewise constant image, which has more than two phases of intensities including “bright phase” of the background, “dark phase” of crack and some phases of other objects in building. The left image, which is called as a two-phase piecewise constant image, contains only two phases, “bright phase” and “dark phase”.

Fig. 1.

Examples of two-phase and multiple phase piece-wise constant images

2.2 Block Diagram of the Proposed Algorithm

The proposed algorithm is shown in Fig. 2. Generally, we proposed 3 main steps for crack enhancement and detection.

Fig. 2.

Overall block diagram of proposed method

Let $I({\mathbf{x}})$ be the input crack image. In the first step, we enhance crack and remove other unintended objects in the image by computing the tubular and symmetric feature. In the second step, we adapt a non-linear relaxation method to segment crack from the crack feature image to obtain the crack map, which is a binary image where “1” pixel represent the crack pixel and “0” pixel belongs to background. In the third step, we extract the binary crack map using crack probability-based local thresholding.

2.3 Crack Feature Image

In our previous work [12], we consider cracks are elongated dark objects which are tubular (line-like). In addition, crack also has a symmetric property since the pixel intensity profile of the cross-section across the crack center-line resembles an inversed Gaussian curve as shown in Fig. 2. In this paper, we utilize the crack tubular and symmetric features to enhance crack and remove unintended objects. Let $I({\mathbf{x}})$ be the input image, the crack feature image $C_{TS} ({\mathbf{x}})$, which represents the possibility of a pixel ${\mathbf{x}}(x,y)$ to be inside a crack, is defined as:

$$ C_{TS} ({\mathbf{x}}) = C_{T} ({\mathbf{x}}) \cdot C_{S} ({\mathbf{x}}) $$

(1)

where $C_{T} ({\mathbf{x}})$ and $C_{s} ({\mathbf{x}})$ are tubular and symmetric feature values at pixel ${\mathbf{x}}(x,y)$, respectively. We aim to divide crack image into regions of high crack feature value and a region of low crack feature value (background and other non-crack objects) (Fig. 3).

Fig. 3.

Pixel intensity profile of a cross-section across the crack-centerline

2.3.1 Tubular Feature

Frangi et al. [10] proposed to compute the vesselness measure of vessels using a multi-scale method. We assumed that the tubular feature of cracks is equal with vesselness measure of vessels and utilize the Frangi’s filter to obtain tubular feature of cracks.

The two eigenvectors $e_{1} ,e_{2}$ and their corresponding eigenvalues $\lambda_{1} ,\lambda_{2}$ of the Hessian matrix $\nabla^{2} I({\mathbf{x}})$ can be used as shape directors for $I({\mathbf{x}})$. The tubular feature $C_{T,\sigma } ({\mathbf{x}},\sigma )$ for a point ${\mathbf{x}}$ at a single scale $\sigma$ is computed as [10]

$$ C_{T,\sigma } ({\mathbf{x}},\sigma ) = \left\{ {\begin{array}{*{20}l} {0,\quad \quad \;{\text{if}}\quad \lambda_{2} \le 0} \hfill \\ {e^{{\frac{{ - A_{\alpha }^{2} }}{{2\alpha^{2} }}}} \left( {1 - e^{{\frac{{ - S^{2} }}{{2b^{2} }}}} } \right),\quad \;otherwise} \hfill \\ \end{array} } \right. $$

(2)

where $A_{\alpha } = \left| {\frac{{\lambda_{1} }}{{\lambda_{2} }}} \right|$; $S = \left( {\lambda_{1}^{2} + \lambda_{2}^{2} } \right)^{1/2}$; $\alpha$, $b$ are the thresholds that control the sensitivity of $C_{T,\sigma } ({\mathbf{x}},\sigma )$ to the measures $A_{\alpha }$, $S$. The normalized derivatives proposed by Lindeberg [13] were used to compute the derivatives in the Hessian matrix.

From (2), we attain the tubular feature for 1 single scale $\sigma$. For a given set of scales $\Omega _{\sigma }$, the maximum tubular feature of different scales $\Omega _{\sigma }$ was chosen to be the unique multi-scale tubular feature of the image:

$$ C_{T} ({\mathbf{x}}) = \mathop {\hbox{max} }\nolimits_{{\sigma \in\Omega _{\sigma } }} \left\{ {C_{T,\sigma } ({\mathbf{x}},\sigma )} \right\} $$

(3)

where $\hbox{max} \left\{ {.} \right\}$ is the maximum value of a set of $C_{T,\sigma } ({\mathbf{x}},\sigma )$.

2.3.2 Symmetric Feature

The symmetric feature can be extracted using the local phase information [14,15,16]. We combine the phase symmetry concept [11] and the monogenic signal framework [17] to compute the symmetric feature of cracks. Let $LG({\mathbf{x}})$ be the isotropic log-Gabor filter given as

$$ LG({\mathbf{x}}) = F^{ - 1} \left\{ {\exp \left( { - \frac{{\left( {\log \left( {\theta /\theta_{0} } \right)} \right)^{2} }}{{2\left( {\log \left( {k/\theta_{0} } \right)} \right)^{2} }}} \right)} \right\}\quad \quad $$

(4)

where $\theta$ is the spherical co-ordinate of the image in the frequency domain, $\theta_{0}$ is the center frequency of each sale. We choose 6 scales so $\theta_{0} = 1/2*2.1^{h - 1}$, $h = \{ 1,2,3,4,5,6\}$, $k/\theta_{0} = 0.65$.

From the log-Gabor filter, we can build the monogenic signal $G({\mathbf{x}})$ at ${\mathbf{x}} = (x,y)$ pixel as a three dimensional vector [17]:

$$ G({\mathbf{x}}) = [G^{1} ({\mathbf{x}}),G^{2} ({\mathbf{x}}),G^{3} ({\mathbf{x}})] $$

(5)

$$ G^{1} ({\mathbf{x}}) = I({\mathbf{x}})*LG({\mathbf{x}}{\mathbf{)}}*m_{1} ({\mathbf{x}}) $$

(6)

$$ G^{2} ({\mathbf{x}}) = I({\mathbf{x}})*LG({\mathbf{x}}{\mathbf{)}}*m_{2} ({\mathbf{x}}) $$

(7)

$$ G^{3} ({\mathbf{x}}) = I({\mathbf{x}})*LG({\mathbf{x}}{\mathbf{)}} $$

(8)

where $m_{1} ({\mathbf{x}})$ and $m_{2} ({\mathbf{x}})$ are two monogenic signal filters. $G_{odd} ({\mathbf{x}})$ and $G_{even} ({\mathbf{x}})$ are two even symmetric and odd symmetric filter responses, which can be represented by the components of the monogenic signal as

$$ G_{odd} ({\mathbf{x}}) = \sqrt {\left( {G^{1} ({\mathbf{x}})} \right)^{2} + \left( {G^{2} ({\mathbf{x}})} \right)^{2} } $$

(9)

$$ G_{even} ({\mathbf{x}}) = G^{3} ({\mathbf{x}}) $$

(10)

The phase symmetry then can be formulated from the monogenic signal as

$$ P_{S} ({\mathbf{x}}) = \frac{{Max\left\{ {\sum\limits_{h} {\left( { - G_{even}^{h} ({\mathbf{x}}) - \left| {G_{odd}^{h} ({\mathbf{x}})} \right|} \right) - T_{noise} ,0} } \right\}}}{{\sum\limits_{h} {\sqrt {\left( {G_{even}^{h} ({\mathbf{x}})} \right)^{2} + \left( {G_{odd}^{h} ({\mathbf{x}})} \right)^{2} } + \xi } }}\quad \quad $$

(11)

where $T_{noise}$ is a noise threshold, $\zeta$ is a small constant to prevent division by zero.

The shortcoming of phase symmetry in crack detection application is that it also has high values for round dots. To overcome this, we used the symmetric feature as the average value of the phase symmetry along the crack direction as follows [12]:

$$ C_{S} ({\mathbf{x}}) = \mathop {\text{max}}\limits_{o} \left\{ {\frac{{\sum\limits_{p = 1}^{M} {P_{S} ({\mathbf{x}}_{p}^{o} )} }}{{N_{{}} }}} \right\} $$

(12)

where $P_{S} ({\mathbf{x}}_{p}^{o} )$ is the phase symmetry of the $p^{th}$ pixel on the virtual line segment passing through ${\mathbf{x}}$ in $o^{th}$ direction. $N$ is the total number of each virtual line segment passing through ${\mathbf{x}}$ and $M$ is the total number of pixels on each virtual line segment.

2.4 Nonlinear Relaxation Method for Crack Detection

From the crack feature image $C({\mathbf{x}})$, we can apply B-spline level set method to extract crack regions. This method requires to build a matrix of B-spline radial basis functions that is half the size of the input images. For image of sizes lager than 1000 × 1000, a lot of computing resources and memory must be used for computation of the convolution task between this matrix and the input image until the convergence. Therefore, we utilize Nonlinear relaxation method, which does not requires convolution between large matrixes for crack detection. Nonlinear relaxation labeling is mainly used to label objects from noisy data [9, 18]. For a focal pixel, the probability of ${\mathbf{x}}$ to belong to a crack segment is defined as

$$ P_{C} ({\mathbf{x}}) = \frac{{\log (C({\mathbf{x}}) + 1)}}{{\log (C_{\hbox{max} } ({\mathbf{x}}) + 1)}} $$

(13)

where $C_{\hbox{max} } ({\mathbf{x}})$ is the maximum crack feature value. On the other hand, the probability of ${\mathbf{x}}$ to belong to background is $P_{B} ({\mathbf{x}}) = 1 - P_{C} ({\mathbf{x}})$. $P_{C} ({\mathbf{x}})$ can be updated by the rule proposed by Peleg [19] as

$$ P^{\prime}_{C} ({\mathbf{x}}) = \frac{{P_{C} ({\mathbf{x}})Q_{C} ({\mathbf{x}})}}{{P_{C} ({\mathbf{x}})Q_{C} ({\mathbf{x}}) + P_{B} ({\mathbf{x}})Q_{B} ({\mathbf{x}})}} $$

(14)

where $Q_{C} ({\mathbf{x}})$, $Q_{B} ({\mathbf{x}})$ are the strengths of support that ${\mathbf{x}}$ belonging to crack and background, respectively, from all the neighbor pixels ${\mathbf{x}}_{i}$ belongs to $R_{{\mathbf{x}}}$

$$ Q_{C} ({\mathbf{x}}) = \frac{1}{{N(R_{{\mathbf{x}}} )}}\sum\limits_{{{\mathbf{x}} \in R_{{\mathbf{x}}} }} {P_{C} ({\mathbf{x}})} $$

(15)

where $N(R_{{\mathbf{x}}} )$ is the total number of pixels in the supporting region $R_{{\mathbf{x}}}$. In this paper, we observe that the neighboring crack pixels of ${\mathbf{x}}$, which have high crack feature values, should lie in a line segment passing through ${\mathbf{x}}$. Therefore, we choose $R_{{\mathbf{x}}}$ to be one of a set of line segments passing through ${\mathbf{x}}$ defined as $\left\{ {R_{1} ,R_{2} , \ldots ,R_{o} } \right\}$. In this paper, we choose 4 regions as 4 line segments along 4 angular directions $\left\{ {0,\pi /4,\pi /2,3\pi /4} \right\}$. The chosen region is the one that gives the maximum $P_{C} ({\mathbf{x}})$. We update $P_{C} ({\mathbf{x}})$ by the following rule:

$$ P^{\prime}_{C} ({\mathbf{x}}) = \mathop {\hbox{max} }\limits_{i} \frac{{P_{C} ({\mathbf{x}})Q_{C}^{i} ({\mathbf{x}})}}{{P_{C} ({\mathbf{x}})Q_{C}^{i} ({\mathbf{x}}) + P_{B} ({\mathbf{x}})Q_{B}^{i} ({\mathbf{x}})}} $$

(16)

$$ Q_{C}^{i} ({\mathbf{x}}) = \frac{1}{{N(R_{i} )}}\sum\limits_{{{\mathbf{x}} \in R_{i} }} {P_{C} ({\mathbf{x}})} $$

(17)

The updating process for all pixels is repeated until $P_{C} ({\mathbf{x}})$ converges or stabilizes. The converged or stabilized $P_{C} ({\mathbf{x}})$ is called as $\bar{P}_{C} ({\mathbf{x}})$.

2.5 Crack Probability Based Binarization Method for Crack Localization

From the converged probability $P_{C} ({\mathbf{x}})$, we proposed to use crack probability based binarization method to localize crack:

• Step 1: Apply locally thresholding to $C({\mathbf{x}})$ to achieve $TH({\mathbf{x}})$.

• Step 2: Choose the binary crack map as

  $$ B({\mathbf{x}}) = \left\{ {\begin{array}{*{20}l} {1,} \hfill & {\bar{P}({\mathbf{x}})_{C} > 0.5} \hfill \\ {0,} \hfill & {otherwise} \hfill \\ \end{array} } \right. $$

  (18)

• Step 3: Create a new binary crack map $B({\mathbf{x}})$ by setting all the neighbor pixels of ‘1’ pixels in $B({\mathbf{x}})$ to be ‘1’.

• Step 4: Create a new binary crack map $B({\mathbf{x}})$ by implementing an “AND” operation between $B({\mathbf{x}})$ and $TH({\mathbf{x}})$

  $$ B({\mathbf{x}}) = B({\mathbf{x}})\,\& \,TH({\mathbf{x}}) $$

  (19)

• Step 5: Repeat step 3 and 4 until $B({\mathbf{x}})$ converges or stabilizes. The converged $B({\mathbf{x}})$ is the final result.

3 Results

The proposed method was implemented in Matlab 7.10 and tested on a computer with Intel Core I3 2.1 GHz CPU and 8 GB RAM. The following settings are applied:

• The image has an eight-bit dynamics and 2 dimensions.

• $\alpha = 0.5,b = 15$, $\Omega _{\sigma }$ = {0.5, 1, 1.5, 2, 2.5, 3}.

• $C_{S} ({\mathbf{x}})$ are computed over 6 scales with following parameters $K/\theta_{0} = 0.65$, $\zeta$ = 0.0001, $M = 15$, $N = 12$. These parameters are empirically tuned based on our experiments on a dataset of 50 crack images.

Figure 4 shows the results of automatic crack detection in 2D multiple-phase image of a concrete column. Although there’s a human hand in the crack image, our method can successfully detect the crack. To further demonstrate the capability of automatic crack detection in a real scene of building, Fig. 5 shows another successful example of detecting crack in an image of wall and ground. These two examples proved that our proposed method is effective in automatically detecting crack in 2D multiple-phase image.

Fig. 4.

Automatic crack detection in a concrete column. Left image is the input crack image, right image is the detected binary crack map

Fig. 5.

Automatic crack detection in a image of wall and ground. Left image is the input crack image, right image is the detected binary crack map

Figure 6 shows another experiment in which we detect crack from image of a scene in building. There’s a steel tube and some other non-crack objects in the 2D multiple-phase image. However, the proposed method can accurately detect the crack while ignoring all the non-crack objects.

Fig. 6.

Automatic crack detection in an image of a scene in building. Left image is the input crack image, right image is the detected binary crack map

Fig. 7.

Automatic crack detection in an image of different materials. Left image is the input crack image, right image is the detected binary crack map

In the next experiment, we make a comparison between our proposed method and Hutchinson’s method [20], Fujita’s method [9] which is claimed to be able to detect crack in noisy images of concrete surface. We use the robust set of parameters for Fujita’s method as described in [9]. Parameters for Hutchinson’s method is referenced from [20]. From the results in Fig. 8, we can observe that our proposed method detect crack correctly in the images of buildings. In contrast to our method, Fujita’s method, have considered many step edges as crack. Hutchinson’s method failed to detect crack as it only can work in two phase piece wise constant images.

Fig. 8.

Comparison between proposed method, Fujita’s method [9] and Hutchinson’s method [20]. From first to fourth row: input image, binary image result from the proposed method, result of Fujita’s method, result of Hutchinson’s method.

To build a metrics for comparing our method and other methods, similar with [21], we need to build the ground truth of images in our experiment by asking the volunteers to observe and select the cracked regions in images. Adapting the formulation of precision and recall concepts from [21], we use two following metrics to compute the accuracy of all the methods:

$$ Precision = \frac{{\left| {GA_{conn} } \right|}}{{\left| {CD} \right|}} $$

(20)

$$ Recall = \frac{{\left| {GA \cap CD} \right|}}{{\left| {GA} \right|}} $$

(21)

where $GA$ are the pixels volunteers chose to be inside the cracks. $CD$ are the pixels detected by the level set models. $GA_{conn}$ are those pixels detected in $CD$ which are connected to cracks in $GT$. High value of Recall proves that a large portion of pixels inside crack have been detected. Concurrently, high value of Precision shows that a large number of pixels detected by the level set model really belong to true cracks. As Hutchinson method cannot converge to detect crack in image with many clear step edge as in our experiments shown in Fig. 8, in the Table 1, we only make a comparison between our method and Fujita’s method. We can see that our method have high precisions and recalls in all three experiments while Fujita’s method only can obtain very low precisions as it detect many non-crack objects in the image. This shows that our method is more precise than Fujita’s method in detecting crack in images with many clear step edges.

Table 1. Comparing precision and recall of our method and Fujita’s method

In Fig. 7, we add 2 more experiments to demonstrate the effectiveness of our method in the surfaces of other materials. Our method can extract crack effectively although there are many clear step edges in the image. This show the high probability for the proposed method to be applied in crack detection in multiple-phase intensity inhomogeneous images.

4 Conclusions

We have proposed an automatic method for automatic crack detection from image of scenes in the building without human intervention. Unlike available methods that detect crack from images containing only concrete surface and cracks (2D piece-wise constant images), our method localize directly and automatically the cracks in 2D multiple-phases images of scenes inside and outside building. Especially, we used only one set of parameters for images of various scenes in building.

Our method can assist the civil engineering experts to localize crack automatically. The output of this advanced method can be used as the input for automatic method of measuring widths of cracks on the surface of concrete structure. A number of experiments have been given to demonstrate the robustness of the proposed method.

In the near future, we will extend our method to be capable of measuring crack width accurately from images containing concrete structures and other non-crack objects.