Keywords

1 Introduction

China is a country with frequent geological hazards, and the serious hazard of highway slope is related to economic development, traffic, production and the safety of people’s life and property. The occurrence of slope disasters not only depends on the rock quality and environment of the slope, but also depends on the cracks in the rock mass to a great extent. Cracks are the early symptoms of most slope disasters, and the increase, expansion and evolution of cracks are the direct reflection of slope instability. By capturing the change of cracks, this small change can be used as an important reference for real-time monitoring of slope state. How to capture the change of fracture is an urgent problem to be solved, and the accurate calculation of the actual area change of fracture is the key to solve the problem.

At present, the surface cracks of highway slope are mainly observed by artificial high-power telescope. Although this observation method is convenient to operate, it has shortcomings such as incomplete detection and careless observation. In order to make the detection method more safe, efficient and comprehensive, and to meet the needs of highway slope crack detection, some scholars proposed a slope crack detection method based on computer vision, which made up for the shortcomings of traditional detection methods. The research on slope crack detection algorithm based on computer vision has become a hot topic of academic attention. In [1], Weixing Wang et al. studied a crack extraction method based on the standard deviation of local window gray scale and the standard deviation of connected area distance. This method has a good detection effect on pavement cracks, and the detection accuracy can reach 96%. In [3], Zhang Haichuan put forward a kind of based on UNet++ and conditions against network (CGAN) road crack detection of image segmentation method, including the generator and the discriminant, through repeated game of generator and the discriminant training, the crack detection and segmentation algorithm makes the generator to generate highly close to the real image segmentation, and generator to achieve the effect of the crack detection. In [4], Zhang Yuefei et al. proposed an improved pavement crack detection algorithm of Mask RCNN, which adopted an adaptive loss function with weight and paid more attention to the characteristics of cracks. In [5], Youfa Cai et al. proposed a crack detection method based on computer vision technology and coordinate mapping. The crack measurement system integrated a high-power mirror image acquisition system, a two-dimensional electric pan-head device and a laser ranging system, which could automatically locate the measured crack within 16 s. In [7], Yana Xiao et al. proposed a crack detection method based on the fusion of penetration algorithm and adaptive Canny operator, which extracted more abundant edge information. In [9], Sadia Mubashshira et al. proposed a pavement threshold detection method based on road color histogram analysis, which achieved satisfactory results when detecting and locating cracks in the image by the algorithm. It can effectively remove noise and retain edges, which is very useful for obtaining better accuracy.

The calculation of crack area is an important link in slope disaster monitoring, but how to calculate the actual crack area from the pixel area of the crack is still a difficult problem. In [2], Wang Lei proposed the threshold segmentation based on the maximum inter-class variance method to extract cracks, calculate the crack area of highway, and realize the calculation of crack area. In [6], X. Jiang et al. proposed an improved phased array ultrasonic absolute time of arrival (AATT) technique, which used the absolute position of tip diffraction echo and angular reflection echo in the S-scan image of ultrasonic phased array to estimate the height and dip Angle of the crack. In [8], X. Xi et al. selected deep granite in a gold mine and studied the sample with pre-crack thickness of 0.3 mm by acoustic emission (AE) sensor and digital image correlation system, and finally obtained the change law of crack morphology and crack length through numerical simulation. In [10], Cherryl O. Tayo et al. used digital image processing to analyze the image, and the experimental results showed that the calculation accuracy of the developed crack width calculation device reached 96.93%. In [11], Diana A A et al. proposed a computer vision method based on ripple changes to detect concrete hot cracks, and used ripple changes to detect hot cracks of different concrete grades and different temperature profile durations. In [12], Zhao et al., based on the Timoshenko beam model, proposed the expression of the transfer matrix method to solve the natural frequency and mode shape of vibration, simulated various types of cracks in the cantilever beam with the finite element method, and compared the solution method with the transfer matrix method. The difference between the results can be used to detect the size and location of the crack.

In view of this, this paper proposes a method of slope crack detection and crack area quantification based on computer vision. Firstly, FsNet network was designed to segment the cracks at pixel level. Based on the generated binary segmentation mask, the chessboard mapping method is proposed to calculate the actual crack area, and the effectiveness of the proposed method is fully verified by taking the self-made data set of highway crack image as an example.

2 Crack Image Preprocessing

Highway slope is generally in the outskirts of barren mountains, slope images often have some shadows, strong light, dark light, etc., which will cause a large noise in the crack detection of highway slope, affecting the effect of crack detection of highway slope. The existence of noise will lead to the characteristics of the highway slope crack detection is not obvious, which will affect the calculation of the crack area, so it is necessary to pre-process the highway slope crack image, that is, image grayscale processing, image equalization processing. The original image of the fracture is shown in Fig. 1.

Fig. 1.
figure 1

Original image of fracture

Full size image

The grayscale of the image is to change the color image of the RGB three-channel data into the grayscale image of the single-channel data. The grayscale processing of the image can make the image containing only brightness information, without color information, which is convenient for further processing of the image. The image gray conversion formula is as follows:

$$ Gray = \sqrt[{2.2}]{{\frac{{R^{2.2} + \left( {1.5G} \right)^{2.2} + \left( {0.6B} \right)^{2.2} }}{{1 + 1.5^{2.2} + 0.6^{2.2} }}}} $$
(1)

Where, Gray represents the grayscale value in the grayscale image. R, G, and B represent the component values of the basic colors of red, green, and blue in the original color image.

Grayscale image of cracks is shown in Fig. 2.

Fig. 2.
figure 2

Grayscale image of fracture

Full size image

Image equalization is used to enhance the image. After image equalization correction, the gray distance of the image can be widened and the gray distribution can be uniform, so as to increase the contrast, make the image details clear, and achieve the purpose of enhancement. The equalized image is shown in Fig. 3, and it can be seen that the image becomes clearly clearer.

Fig. 3.
figure 3

Image equalization processing

Full size image

3 Crack Image Segmentation

The main purpose of crack image segmentation is to facilitate further image analysis and processing. The crack of highway slope has the characteristics of crisscross direction and difficult to locate and identify. Therefore, it is necessary to design an algorithm with high identification accuracy and a crack identification model that is easy to deploy and apply. At present, image segmentation methods are mainly divided into traditional methods and deep learning-based segmentation methods. With the continuous optimization and improvement of deep learning algorithms, image segmentation based on deep learning is applied more and more, which has the advantages of obvious feature extraction and high accuracy. In view of the characteristics of cracks and the development of deep learning technology, an image segmentation network FSNET based on deep learning is designed in this paper to achieve the segmentation of crack images. The image segmentation framework is shown in Fig. 4.

Fig. 4.
figure 4

Image segmentation framework

Full size image

After the crack image is preprocessed, the image segmentation network FSNET is used to train the preprocessed image. In the process of training, increasing the network depth can capture richer and more complex features. However, the greater the network depth is, the better it is. Excessive network depth will cause the problem of gradient disappearance. The crack output image after segmentation is shown in Fig. 5.

Fig. 5.
figure 5

Fracture output image

Full size image

4 Actual Fracture Area Calculation

4.1 Crack Pixel Area Calculation

The result of FsNet network segmentation is a relatively ideal binary image (white represents the crack and black represents the background). The number of white pixels in the image is counted by digital image processing technology to calculate the pixel area of the crack automatically. The specific process is as follows:

  1. (1)

    Image preprocessing: first, the combined operation of morphological corrosion and expansion is used to eliminate the noise such as isolated points in the output binary image, and then the discontinuous crack fragments are Mosaic processed through the connected domain operation, and then the threshold of the connected domain area is set to determine the number of continuous and complete cracks in the image \({N}_{c}\).

    $$ f \ominus b = \left\{ {x,y|\left( b \right)_{xy} \subseteq f} \right\} $$
    (2)
    $$ f \oplus b = \left\{ {x,y|\left( b \right)_{xy} \cap f \ne \emptyset } \right\} $$
    (3)

Where: f is the set of image pixels to be operated (etched or inflated); B is structural element; (x,y) is the position coordinate of any pixel point in set F.

  1. (2)

    Calculation of crack pixel area: The pixel statistical method is used to calculate the coverage area of cracks in the image. The digital matrix of the binary image is traverse with a pixel point as a unit. The total number of pixels with a statistical value of 1 is the crack area \({S}_{c}\). If there are \({N}_{c}>1\) cracks in the image, to get the area \({S}_{c}^{n}\) of the N th crack, it is necessary to conduct pixel traversal statistics in the N th marked connected domain. The pixel area in the crack output image in Fig. 5 is 6256 pixels.

4.2 Actual Fracture Area Calculation

The calculation of the actual crack area is an important part of slope disaster monitoring, but it is difficult to measure the actual crack area from an image. The chessboard mapping method is proposed in this paper. The chessboard is used as the calibration plate, and the actual length and width of the chessboard are known according to the principle of keyhole imaging, and the crack length is measured by length ratio and known length of the reference object.

Firstly, a checkerboard is placed on the plane where the test plane is. When the system is in the condition of small distortion, the checkerboard plane is taken as the X plane in the world coordinate system, then the ZW axis of the world coordinate system is perpendicular to the checkerboard plane, that is, the ZW coordinates of all feature points on the plane are 0.

$$ \left[ {\begin{array}{*{20}c} \mu \\ \nu \\ 1 \\ \end{array} } \right] = K\left[ {\begin{array}{*{20}c} {x_{w} } \\ {y_{w} } \\ 1 \\ \end{array} } \right] $$
(4)

Where, K is the homography matrix between the world coordinates on the plane and the pixel coordinate system; \((\mu ,\nu )\) is the ideal pixel coordinate without distortion, and \(({x}_{w},{y}_{w})\) is the actual coordinate of the plane where the checkerboard is located.

The specific steps are as follows:

  1. (1)

    Use a professional camera to photograph M rows and N columns of alternating black and white checkerboard. And record the height, distance, camera elevation when shooting. As shown in Fig. 6, this checkerboard is 1 cm by 1 cm.

Fig. 6.
figure 6

Checkerboard.

Full size image
  1. (2)

    Binary processing was carried out on the captured checkerboard image, and the combined operation of morphology etching and expansion was used to eliminate the noise such as isolated points in the output binary image, and then the discontinuous crack fragments were stitched through the connected domain operation. The checkerboard preprocessed image as shown in Fig. 7 is obtained. The pixel area of the checkerboard preprocessed image is 1764 pixels.

Fig. 7.
figure 7

Checkerboard preprocessing.

Full size image
  1. (3)

    Will Fig. 6 checkerboard placed in the cracks of the image above, using distance, height and elevation of the same camera checkerboard, if cracks of the pixel fall within one of checkerboard grid, known fracture pixel area, checkerboard pixel area and the actual area of checkerboard, according to the laws of the map to the actual area of the cracks.

5 The Example Analysis

Please note that the first paragraph of a section or subsection is not indented. The first paragraphs that follows a table, figure, equation etc. does not have an indent, either. The above image processing method was used to obtain the crack area of the highway slope, and the above image processing method was verified by taking the highway crack as an example. A certain section of pavement with cracks was selected. The experimental data were 300 highway crack images collected by professional cameras, and the image pixel size was 3000 * 4000px. Ten images with cracks were selected to calculate the pixel area of cracks. At the same time, the crack area of the road surface was measured by the crack area observation instrument and compared with the calculated actual crack area. Absolute error refers to the error between the measured value x and the true value x, which is the difference between the measured value and the true value. Relative error refers to the ratio between the absolute error \(\Delta x\) and the true value X, which can reflect the reliability of the measurement.

The calculation formula of absolute error is as follows:

$$ \Delta x = x - X $$
(5)

The relative error is calculated by the following formula

$$ E_{r} = \frac{\Delta x}{X} \times 100\% $$
(6)

The comparison results are shown in Table 1. It can be seen from Table 1 that the recognition accuracy of this image detection method is high.

Table 1. Correct rate of slope crack image area detection
Full size table

The test data verify the validity of the calculation of the actual fracture area. It can be seen from Table 1 that the identification accuracy of the actual area calculation of the fracture is good, and the relative errors of each test image are all less than 15%. The method adopted satisfies the calculation requirements of the fracture variation trend.

6 Conclusion

According to the characteristics of slope cracks, this paper realized the pixel-level detection of slope cracks based on deep learning and digital image processing technology, showing that deep learning has great advantages in crack detection. At the same time, the chessboard mapping method is proposed, and combined with the self-made data set of highway crack, the simulation scene is constructed and the comparative experiment is set, and the actual area of slope crack is calculated. The results show that the method is feasible for the detection of crack area. Although board mapping method can well solve the problem of actual slope crack area computation, but there is still a homemade checkerboard recognition accuracy, but the next step will continue to improve the method for calculating the actual highway slope cracks, improve the detection of highway slope fracture area as a result, improve the reliability and accuracy of the algorithm.