Keywords

55.1 Introduction

Subway networks play a key role in the development of an urban area, as one type of its high-capacity public transport. Unfortunately, the facilities in the subway networks experience continuous deteriorations, considering the severe environmental conditions and constant heavy traffic loads that the subway networks experience. In addition, the huge amount of underground water might infiltrate into the subway infrastructures and lead to corrosions of reinforcing steel bars, concrete delamination, spalling, etc. As a result, it is important to provide periodic structure inspection and assessment to keep the subway networks operational in service.

The current subway assessment process is conducted manually based on visual inspection. The manual visual inspection might produce important ideas about the actual infrastructure conditions. However, the inspection results are not always consistent. They are always subjective to the inspectors’ knowledge and expertise in nature. Also, the inspection process is time-consuming and labor-intensive, especially considering the fact that some parts or areas in the subway networks are difficult to access. Therefore many distresses are not diagnosed until they progress and become serious.

The limitations associated with the manual inspection highlighted the necessity for automated inspection process with the support of machine vision techniques (Radopoulou and Brilakis 2015). So far, several research studies have been proposed towards this end, and they have been tested in different types of concrete infrastructures. For example, Koch and Brilakis relied on image thresholding, morphological thinning, elliptic regression and texture extraction to detect potholes in highways (Koch and Brilakis 2011). Similarly, Li et al. (2016) designed an integrated framework for the detection and measurement of potholes on the basis of 2D images with the combination of Ground Penetrating Radar (GPR) data. In addition to the potholes, German et al. (2012) retrieved the spalling properties from concrete column images in an attempt to assess the safety of post-earthquake concrete structures. Adhikari et al. (2013) simulated on-site visual inspection by creating 3D model of a de facto bridge, followed by overlaying 2D digital images on the model to get bridge condition index. Atef et al. (2015) utilized a multi-tier technology to locate water pipes and detect leaks based on the image processing of captured Infrared (IR) and GPR images.

Compared with previous research studies, this paper proposed a novel method to detect and quantify spalling distress in subway networks. The method combined several image processing techniques with regression analysis. Under the framework, the image filtering is first used to identify and extract the spalling regions in an automatic manner. Then, a novel vision-based detector with regression analysis is created to support the 3D scene reconstruction for estimating the spalling depth. The effectiveness of the method has been tested in inspecting and assessing the subway networks in Montreal. The test results showed that the method could complement the current visual inspection practice. It helps to address the subjective nature. Also, it is expected to save the inspection time and cost.

55.2 Related Work

As for the detection and assessment of concrete distress in subway networks, many efforts have been made by researchers. For example, Semaan (2011) proposed a framework for the evaluation of subway stations using the Leveraged Analytic Hierarchy Process (AHP), Multi-Attribute Utility Theory (MAUT) and Weibull reliability function to predict the performance of subway infrastructure components. The framework was developed only for assessing subway stations, and it is difficult to investigate the severity of the surface defects on other structures in the subway networks. Therefore, it was still characterized as subjective. Similarly, Kepaptsoglou et al. (2012) presented a model to evaluate the functional conditions of subway stations with the Fuzzy AHP and MAUT to produce the Metro Condition Index (MCI). Again, the model was developed solely for the condition assessment of subway stations, hence it failed in evaluating the entire subway network. Gkountis and Zayed (2013) further enhanced the model of Kepaptsoglou et al. (2012) with the Analytic Network Process (ANP). This way, the interdependencies of different components in the subway network could be established. In 2014, Abouhamad integrated the fuzzy risk index model and risk-based budget allocation model to develop a risk-based asset management framework for subway networks (Abouhamad 2014). This study prioritized rehabilitating the metro stations, but it failed to be validated using an extensive and real life data set.

On the other hand, the use of machine vision techniques for the detection and evaluation of concrete infrastructures, such as bridges, highways, etc., has gained a lot of research interests. For example, in the field of bridge inspections, La et al. (2014) presented a crack detection and mapping algorithm for detecting cracks on the bridge decks. Adhikari et al. (2013) proposed a model for quantifying the cracks in bridges with the support of artificial neural networks and 3D visualization. Abudayyeh et al. (2004) developed an automated bridge inspection method based on digital images and the method was integrated with Bridge Management Systems PONTIS. In the field of pavement defects detection, Li et al. (2016) designed an integrated framework for the detection and measurement of potholes on the basis of 2D images and Ground Penetrating Radar (GPR) data. Yu and Salari (2011) introduced a laser-based detector and classifier for pavement defect inspection. Koch and Brilakis (2011) presented a pothole detection method which involved image thresholding, morphological thinning, elliptic regression and texture extraction. In addition, Atef et al. (2015) utilized a multi-tier technology to locate water pipes and detect leaks based on the image processing of captured Infrared (IR) and GPR images. Guo et al. (Guo et al. 2009) presented a condition assessment approach of wastewater systems grounded in visual pattern recognition techniques. Yu et al. (2006) created a system for inspecting and measuring tunnel cracks. Jahanshahi and Masri (2011) incorporated depth perception, image processing, and pattern recognition for the purpose of crack detection and quantification. Hutchinson and Chen (2006) proposed a statistical procedure for the detection of cracks through utilizing Bayesian Decision Theory. German et al. (2012) retrieved major properties of spalled regions on concrete columns subsequent to detecting the defect in an attempt to assess the safety of post-earthquake buildings. The following two sub-sections briefly describe the technical backgrounds that this research tries to build upon.

55.2.1 Image Filtering

Image filtering techniques are commonly used for defects detection and quantification. Existing techniques could be mainly classified into two categories: (1) filtering in the spatial domain and (2) filtering in the frequency domain. Spatial filters are directly applied in an image. For example, low-pass filters, such as median filter, could be used to minimize image deviations to smooth images. Another example of filtering in the spatial domain is image segmentation, which divides an image into regions or objects of interest. One of the fastest and most applicable segmentation methods is image thresholding. In addition, histogram equalization is often performed to enhance the contrast of the image and enhance the visual quality of the image.

Image filtering in the frequency domain is required to perform the Fourier transform of an image first. The Fourier transform converts the image from the spatial domain to the frequency domain. Then, a series of image processing steps could be conducted in the frequency domain of the image. The results are further converted from the frequency domain back to the spatial domain through the inverse Fourier transform. Examples of the filters in the frequency domain include Gaussian low-pass filters, Butterworth low-pass filters, etc.

55.2.2 Regression Analysis

Regression analysis is a statistical approach which analyzes and models the relationships between two or more variables. The simplest form of this relationship is postulated as a linear way (Kutner et al. 2003), where the value of one dependent variable is predicted from the value of an explanatory variable. In the case when there are more than one explanatory variable, the form could be expressed in a more generic way. Suppose given a dependent variable y and a number of explanatory variables X 1, X 2,…, X p that may be related to y. The regression analysis is to model the relationship between y and X 1, X 2,…, X p to quantify the strength of X 1, X 2,…, X p with corresponding regression coefficients. Also, it would decide which explanatory variable might have no relationship with y at all and which one might contain redundant information about y.

Typically, the regression coefficients in the model are unknown and they are estimated from sampling data through the least squares approach. In addition, there are several methods currently available to check the forms of linearity by observing curvatures in the plots. For example, they check the scatterplot of residuals as opposed to the fitted values or the scatterplot of residuals against each predictor. If the scatterplot proposes a curvilinear relationship, it means the form could be expressed in a polynomial way, as suggested by Kutner et al. (2003).

55.3 Research Objective and Methodology

The main objective of this paper is to create a novel method for detecting and assessing concrete spalling in subway networks. So far, most vision-based methods have been focused on detecting and assessing the cracks and potholes. Few of them have been developed to investigate the detection and quantification of the spalling, although it is one of common defects that could be found in concrete structures.

The proposed methodology includes three steps starting from the image data collection to the modeling of concrete spalling using regression analysis. The overall framework of the proposed methodology has been illustrate in Fig. 55.1. Under the framework, 2D images are first captured for different elements in subway networks using a digital camera. Then, the captured images are processed to remove noise and enhance image features related to concrete spalling. The images are further filtered both in the spatial domain and the frequency domain to detect the spalling. Finally, the spalling attributes (i.e. depth in this study) are estimated through the supervised machine learning combined with regression analysis.

Fig. 55.1
figure 1

Overall framework

Full size image

55.3.1 Image Acquisition

A myriad of images of subway facilities are always captured with a digital camera. The image data demonstrate all the visible defects and damage on the surfaces of structures, however this research is only focused on spalling regions at concrete surfaces. In order to optimize the quality of image analysis later, the following essential rules need to be taken into account during the process of capturing the digital images. First, it is necessary to make sure that the images are acquired orthogonal to the surface with suitable overlapping to reduce the error of projection and pledge better area coverage. Second, if necessary, artificial lighting could be used to guarantee consistent image qualities. Moreover, close-range images are capable to deliver detailed information of the physical attributes of the objects.

55.3.2 Image Processing for Spalling Detection

A hybrid method with various spatial and frequency domain filters is proposed to enable the detection of spalling regions on concrete surfaces. The method starts by reading, adjusting, and displaying the images. Consider the acquired images are RGB images. It is necessary to explore the color space and split the RGB image into three different channels. An appropriate color channel is then selected for image enhancement and detection.

Subsequently, an image smoothing technique is applied to decrease the variance among the intensities of each pixel without considerably affecting the information regarding the regions of interest (ROI). The smoothing is expected to improve the isolation of distress from the image background by retraining significant distress patterns and meanwhile suppressing the image noise. In this research study, an anisotropic diffusion method proposed by Perona and Malik (1990) is adopted to smooth image data. The method relies on partial differential equations, local gradient function, and implements sequential iterations to limit the smoothing operations in the internal ROI without crossing boundaries between sharp pixel intensities.

The smoothed image is further thresholded to segment the spalling defects from the image background. The main idea behind the thresholding is to select an appropriate threshold value. Here, the selection is based on the trial-and-error strategy in the image histogram, which describes the intensity distribution of all the pixels in the image. Then, a Gaussian 3D filter is established using stacks and hyper-stacks to enhance image resolutions. The process continues with the application of a couple of colored filters, image scaling, calibration, edge detection, etc. Finally, the image surface graph is analyzed and an interactive 3D scene presentation is plotted. These illustrations display pixel intensities in a three dimensional plot to allow a de facto visualization.

55.3.3 Regression Analysis for Depth Estimation

The regression analysis starts by the selection of the predictor (i.e. input) and response (i.e. output) variables. Here, the input variables comprise pixels intensities generated from the image processing step, and the output variable includes the spalling depths taken from in situ measurements. The diagnostic test with a scatter plot is conducted to find the relationship between the input and output variables. The examples of the relationship, such as linear, quadratic, cubic, etc., are checked. When an appropriate relationship is found, its effectiveness is further tested using four criteria, including R-square (R2), adjusted R-square (Adj R2), sum of squares due to error (SSE), and root mean squared error (RMSE).

55.4 Implementation and Results

The proposed methodology was implemented using different software packages, including MATLAB® R2013b and ImageJ. In order to test the effectiveness of the methodology, the images of one subway station in in Montreal’s subway network were taken with a digital camera, Canon EOS Rebel XS. The station was opened to the public in 1980. It belongs to one of the 31 stations along the Orange Line of Montreal’s subway system. Signs of deteriorations and defects could be found in multiple structural elements. Figure 55.2 shows the image acquisition process as well as examples of the images used for the test. The resolution of the test images is 2592 by 3888 pixels.

Fig. 55.2
figure 2

Image acquisition for tests

Full size image

The images were processed on a desktop PC. The technical specification of the desktop PC is Intel Core i3-3220 CPU @3.30 GHz. The operating system is 64-bit. Figure 55.3 shows the main steps involved in the image processing. In the figure, the surface image was thresholded based on the histogram. The thresholding split the image into two parts, where the spalling region in the image was isolated. Then, the boundaries of the spalling region were detected. Moreover, the spalling region was represented in 3D based on the intensity values of the image pixels in the spalling region. Table 55.1 summarized the detection performance of the spalling regions from the images. True positive is the number of correctly detected spalling regions; false positive is the number of incorrectly detected spalling regions; and false negative indicates the number of the spalling regions that were not detected. Recall is the ratio of true positive over the sum of true positive and false negative, while precision is measured as the ratio of true positive over the sum of true positive and false positive.

Fig. 55.3
figure 3

Example of image processing steps

Full size image
Table 55.1 Spalling region detection performance
Full size table

The intensity values were correlated with the depth information. The three relationships (i.e. linear, quadratic, and cubic) relationships were checked. The corresponding functions were estimated (Eqs. 55.1–55.3). Moreover, the effectiveness of the functions was evaluated. The evaluation results were summarized in Table 55.2.

Table 55.2 Statistical diagnostic tests
Full size table
$$ y = - 0.093x + 17 $$
(55.1)
$$ y = 0.00043x^{2} - 0.22x + 26 $$
(55.2)
$$ y = 0.000033x^{3} - 0.014x^{2} + 2x - 83 $$
(55.3)

55.5 Conclusions

This research presented an integrated framework for the detection and quantification of concrete spalling distress from the digital images. The framework includes a hybrid algorithm for the detection concrete spalling regions, interactive 3D presentation, and regression analysis to estimate the relationship between spalling intensity and depth. The method has been tested for inspecting one metro station in Montreal. The results showed that the performance of detecting concrete spalling regions was promising. It could reach 91.7% in recall and 94.8% in precision.

Also, the interactive 3D visualization simulated the de facto scene of spalling through the incorporation of depth perception. The schematic representations are helpful in envisioning the level and severity of the spalling on the integrity of structural elements. Also, the regression analysis was conducted to find that the intensities of the spalling regions in the images were correlated with their depth information. Three relationships (linear, quadratic, and cubic) were proposed and tested. The results showed that the cubic relationship fitted better than the other two.