Keywords

1 Introduction

Accurate pavement condition assessment of a road network is vital for managing maintenance and rehabilitation (M&R) efforts to ensure its continued functionality. Early detection and repair of pavement defects can prevent further degradation and reduce the overall road M&R expenditure. As such, efficient and timely road inspection is one of the key elements of a successful pavement management system. Nevertheless, periodic road condition surveys tend to be costly and time-consuming if being carried out in the conventional fashion, i.e., by human visual inspection.

Good road conditions are important in supporting the proper functions of a country’s economy, and many leading civil engineering organizations across the world, such as the American Society of Civil Engineers (ASCE), continue to emphasize the importance of maintaining the good conditions of the roads and other civil infrastructures. However, the existing situation is not optimistic. According to [1], approximately half of the major roads in the U.S. are currently considered as in poor conditions.

Pavement surface condition assessment has historically been conducted manually by state and/or local transportation agencies, which is still the predominant practice in most developing countries. This resource-demanding and time-consuming process has motivated the development of various automatic road condition assessment techniques and systems. With the growing interest in automating the pavement condition survey and rapid advancement in computer vision and deep learning, the past decade has witnessed an emerging popularity of automatic image-based road surface distress assessment systems. However, many existing automatic image-based systems still suffer from variation in environmental conditions, such as lighting condition, shadow cast by nearby objects, rainy weather, etc., and reliability issues associated with underlying image-based cracks detection algorithms, ranging from simple pixel level filtering to complex deep convolutional neural networks [2].

Automating the detection and classification of pavement surface cracks has been a challenging task due to multifaceted crack patterns arising from a multitude of factors underlying different crack initiation and propagation mechanisms, such as traffic loading spectrums, weather-induced material properties, pavement design, etc. Many local agencies still rely on the traditional visual inspection method for assessment of their local road networks.

In this study, we aim to leverage the availability of pavement surface images collected as part of the pavement survey process and apply unsupervised learning techniques (e.g., PCA and cluster analysis) to cluster similar crack types/patterns. Being able to identify different classes of cracks in an unsupervised fashion helps to understand the local dominant crack types and potential contributing factors, and to infer proper treatment strategies. For conceptual proof of this idea, the surface images collected by a survey vehicle from three local roads in Athens, Georgia were utilized.

2 Literature Review

Automated detection of distresses from pavement images is a challenging problem that has been studied by the computer vision research community for more than three decades. The challenges associated with 2D pavement image-based method include variations in image sources (digital cameras, smartphones, unmanned aerial vehicles, etc.), non-uniformity of cracks, surface texture, lack of sufficient background illumination, and the presence of other features such as joints, among others.

Traditional image analysis approaches rely on hand-crafted features. For example, a novel local binary pattern (LBP) based operator for pavement crack detection was introduced in [3] and a crack detection method using Gabor filter was presented in [4]. An automatic crack detection method referred to as CrackTree is developed in [5]. An integrated model for crack detection and characterization is presented in [6] and a variety of image processing algorithms aimed at detecting and characterizing road surface crack distresses are described in [7].

The recent advancement in machine learning (ML), especially in deep learning (DL) with vision tasks (e.g., object detection and classification), has reignited the research in image-based pavement distress detection and classification. In a recent study, Hoang et al. [8] developed a multiclass support vector machine model optimized by artificial bee colony algorithm to achieve an over 96% overall accuracy in classifying longitudinal crack, transverse crack, alligator crack, and no crack. A review on deep learning applications in pavement image analysis and distress detection was conducted by [9]. Specifically, Zhang et al. [10] developed a crack detection method where the discriminative features are learned from raw image patches using ConvNets. Likewise, [11] proposed a deep architecture of CNNs for detecting concrete cracks. More recently, Maeda et al. [12] evaluated Single Shot MultiBox Detector (SSD) with Inception V2 and MobileNet architectures, for detection eight different road damage types.

However, most previous studies on crack detection and classification relied on hand-crafted features or were conducted in supervised learning settings. In contrast, our study focuses on clustering pavement surface images that are congruous with the underlying pavement distress classes in an unsupervised fashion through cluster analysis in a low-dimensional feature space of processed images.

3 Data Set

Pavement surface images (800-by-600) were acquired through a dash camera mounted on a survey vehicle. The dataset used in our experiment consists of 1,125 pavement surface images collected from three local roads in Athens, GA, USA.

4 Analysis

Given the complexity of asphalt pavement images, which are typically endowed with various patterns of micro-texture, nonuniform color arising from the mixture of pavement materials, pavement markings, and shadows casted by nearby utility poles, vegetation, etc., preprocessing of the images is usually required to remove noises and highlight cracks. In this study, a series of image processing techniques, including logarithmic transformation, bilateral filters, Canny edge detector [13], and morphological operations, were applied, resulting in “cleaner” images with more distinct crack patterns. The resultant images were further processed by Principal Component Analysis (PCA) to reduce dimensionality of the feature space, followed by a K mean algorithm [14, 15] to cluster the images in the low-dimensional feature space.

4.1 Image Processing

The interference of the image taking with the dynamic environment often induces unwanted noises. Therefore, it is desirable to apply image preprocessing operations first to remove environmental interference as much as possible. In this study, the open-source computer vision library, OpenCV [16] was utilized, which implemented many popular and commonly used image processing and computer vision algorithms. In our experiment, the images were processed in several steps as described below.

First, the originally captured RGB images were resized from 800-by-600 to 100-by-100 to reduce unnecessary resolution as well as computational footprint and converted to grayscale images since colors do not contribute much to crack features (Fig. 1a). Then, the logarithmic transformation was applied, resulting in the range of darker pixels being expanded while brighter pixels being compressed (Fig. 1b).

Random noise is one of the main problems which may affect the results. Due to pavement physical characteristics, the naturally captured images are more or less affected by random noises (e.g., induced by uneven micro-texture) and/or uniform noise (e.g., a result of shadow), which need to be removed. A bilateral filter was applied to smooth images without losing edge precision. Figure 1c shows the result of the same image after applying the bilateral filtering. Finally, a canny edge detector was applied, followed by morphological operations (erosion and dilation) to highlight crack tracks (Fig. 1d).

Fig. 1.
figure 1

Image processing.

Full size image

4.2 Dimension Reduction

With the previously processed images, principal component analysis (PCA) was applied for dimension reduction and feature extraction. In PCA, the high-dimensional features are linearly projected to a low-dimensional space without losing much information. In other words, PCA aims to find a linear subspace of the original feature space that captures the most data variance. In PCA, the principal components (PCs) are created in a descending order of the amount of data variation and all PCs are orthogonal to one another. The eigenvectors of covariance matrix reveal the PC directions while the eigenvalues provide the relative magnitudes of variance explained by corresponding PCs. A limitation of PCA lies in its linearity, which can be addressed by extending the linear kernel to nonlinear kernels through the kernel trick, referred to as kernel PCA [17]. After applying PCA to our dataset, the cumulative variance explained was plotted against the number of components, as shown in Fig. 2. The top 700 PCs, which explained over 80% of the total variance, were retained for cluster analysis.

Fig. 2.
figure 2

Cumulative variance explained versus the number of principal components.

Full size image

4.3 Cluster Analysis

A variety of cluster analysis methods have been developed with K-means (Lloyd 1957; MacQueen 1967) being the most popular one that is flexible and works well in many settings. In K-means, the number of clusters, K, needs to be prespecified. The elbow method has been typically used to determine the K value that corresponds to the point of the maximum curvature in the inertia plot. Given a dataset, the goal of K-means is to find the K clusters with the minimal within-cluster variation. Using the Euclidean distance as the proximity measure, K-means algorithm can be expressed as an optimization problem in Eq. 1.

$$ \mathop {{\text{minimize}}}\limits_{{C_{1} , \ldots ,C_{k} }} \left\{ {\sum\nolimits_{k = 1}^{K} {\frac{1}{{\left| {C_{k} } \right|}}\sum\nolimits_{{i,i^{\prime} \in \,C_{k} }} {\sum\nolimits_{j = 1}^{p} {\left( {x_{ij} - x_{{i^{\prime}j}} } \right)^{2} } } } } \right\} $$
(1)

where, \(x_{ij}\) = the jth feature of observation i; p = the number of features; \(\left| {C_{k} } \right|\) = the number of observations in the kth cluster.

The within-cluster sum-of-squares in Eq. 1 is often referred to as inertia. Applying K-means to our dataset in the reduced feature space, the resulting inertia is plotted against the number of clusters (K) in Fig. 3. Per the elbow method, three clusters (K = 3) were chosen.

Fig. 3.
figure 3

Inertia versus number of clusters (K).

Full size image

For visualization purposes, the cluster results are plotted in the 2D plane formed by the first two principal components (see Fig. 4).

Fig. 4.
figure 4

The results of cluster analysis (K = 3).

Full size image

The clusters are indicated by colors in Fig. 4. Cluster 0 (blue) represents multi-directional cracks, Cluster 1 (orange) coincides with longitudinal crack, Cluster 2 (green) contains mainly images without cracks. Exemplar images from each cluster are shown in Fig. 5.

Fig. 5.
figure 5

Exemplar images in each cluster.

Full size image

5 Conclusions

As a popular nonintrusive technology, research in image-based pavement condition assessment has recently regained momentum by leveraging modern machine learning methods and techniques. This paper presents preliminary results from an ongoing study, demonstrating the use of simple unsupervised machine-learning methods (i.e., PCA for feature extraction and K-means for cluster analysis) for clustering pavement surface distresses. The results revealed a meaningful correlation between derived clusters and crack types. The simple approach could be used to help local jurisdictions understand their predominant pavement distress types and plan for M&R activities. The cluster outcome can also be used as a pre-step to assist with image annotation for developing supervised classification models. Given the limitation of PCA in its linear projection, further studies should consider nonlinearity in feature extraction, such as through kernel PCA, variational autoencoders, etc.

Besides the pavement surface images, modern pavement survey practices also tend to capture subsurface profiles using the ground penetrating radar (GPR) technology. Surface images emphasize functional failure while subsurface GPR scan images can reveal structural failure. For effective diagnosis of pavement structure, surface images and GPR scan images can be jointly used to enrich the feature space through feature engineering techniques to properly capture the complementary aspects of pavement structure.