Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Remote sensing is a science to retrieve the information about an object through analyzing the data acquired by a device that is not in physical contact with the object (Lillesand and Kiefer 2015). Infrared thermography is the product of remote sensing techniques and is usually used to record the surface temperature of the observed objects. Infrared thermography is widely used to identify defects presented in objects by observing the radiant heat pattern (Huang and Wu 2010). For a thermal camera, radiation is converted to temperature depending on the emissivity value of the specimen. Emissivity values are in the range of 0 and 1, and they are defined as the ratio of a body’s emission spectrum at a given temperature to that of a blackbody at the same temperature (Plotnikov and Winfree 1998). If the emissivity value is 1, it means that the test body is a black body; if it is 0, the test body completely reflects all the received energy. Infrared thermal cameras are designed to record the temperatures transformed from emissivity values. Changes in the recorded temperatures on thermal images reveal possible surface flaws (Maldague 2001). Furthermore, those areas in which the surface temperature information is different with their surrounding neighborhoods can be used to indicate the locations of the potential defects presented on objects.

Thermal segregation is a phenomenon caused by lack of temperature homogeneity in the hot mix asphalt (HMA) constituents of the in-place mat of such a magnitude that there is a reasonable expectation of accelerated pavement distresses (Stroup-Gardiner and Brown 2000); in another word, the surface temperature information illustrated on the mat cannot be distributed evenly. Thermal segregation can be classified into two groups: aggregate segregation and thermal segregation, and both of them can reduce pavement service life (Song et al. 2009). Aggregate segregation usually can be identified and observed by employing human visual inspection because of different textures shown on asphalt mats. As for thermal segregation, it could be caused by placing a cooler mass into the pavement or observing large temperature differentials because of insufficient compaction during pavement installation (Adams et al. 2001; Read 1996). It is difficult to locate those areas with large temperature differentials by employing human visual inspection. Thermography can accurately provide the surface temperature information of a large area. Recently, Texas Transportation Institute (TTI) has developed PAVE-IR to monitor the mat surface temperature of the HMA being placed. PAVE-IR is used to record the mat surface temperature in real-time such that those areas with lower surface temperature can be identified immediately and fixed by placing new HMA from the thermal profile generated by PAVE-IR. Texas Department of Transportation (TxDOT) applies the thermal profile in the construction specifications. However, those locations identified with thermal segregation still remain anomalies in the mat (Sebesta and Scullion 2012).

The paper focuses on analyzing the thermal infrared images taking after a mat had been rolled. The surface temperature information recorded in thermography provides an important clue to identifying the locations of thermal segregation. Image segmentation was employed to segment the collected thermal images such that regional boundaries of the segmented regions can be determined. Traditionally, a penetrating thermometer is employed to measure the surface temperature information on the few sampled points. However, the surface temperature information retrieved by employing a penetrating thermometer can only provide little information of thermal segregation occurring on HMA pavement construction because the limited points were measured by the thermometer. Thermography not only provides the surface temperature information of thermal segregation but also offers enough sampled points to reveal the cooling conditions of pavement construction. Several researchers have proposed different algorithms to analyze thermography by applying image processing techniques: Weritz et al. analyzed the thermal images with Fourier transform to obtain the phase image in the frequency domain (Weritz et al. 2005); Abdel-Qader et al. used segmentation techniques to isolate defects near a concrete bridge deck (Abdel-Qader et al. 2008); Huang et al. applied multilayer segmentation to group those pixels with similar surface temperature information such that the whole thermography can be divided into several regions and the temperature distributions in the segmented regions are homogeneous (Huang et al. 2014). Furthermore, the segmented regions not only quantitatively give the surface temperature information but also identify the regional boundaries such that the locations of uneven surface temperature distributions can be determined. This paper applies the image segmentation on RAC and HMA to identify those possible areas with thermal segregation, and their surface temperature changes during cooling status after pavement constructions.

The remainder of this paper is organized as follows. In the next section, the multilayer level set model is briefly introduced. In Sect. 3 illustrates the segmented results by employing the multilayer level set approach on real thermal infrared images. Eventually, some conclusions and related discussions are provided.

2 Segmented Thermography by Employing the Multilayer Segmentation

The fundamental idea of applying image segmentation on thermography is to partition a given thermal infrared image into a series of sub-regions such that each sub-region is homogeneous. In doing so, the surface temperature distributions can be identified and located in the thermal infrared image. Multilayer segmentation proposed by Chung and Vese segments the given thermal infrared image according to the pre-defined thresholds such that the pixel values shown in the segmented regions are grouped and centered by the pre-selected thresholds (Chung and Vese 2010). The multilayer segmentation is numerically implemented by level set functions proposed by Osher and Fedkiw (2002). In the multilayer approach, let \( {\text{I}}_{0} \) be a thermal image and the two level set functions are used to segment the image into a series of sub-regions with distinct level values \( \left\{ {l_{1} < l_{2} < \cdots < l_{m} } \right\} \) and \( \left\{ {k_{1} < k_{2} < \cdots < k_{n} } \right\} \) for \( {\upphi}_{1} \) and \( \phi_{2} \) the level set functions, respectively. Then the energy functions generated by level set functions for \( \phi_{1} \) and \( \phi_{2} \), pre-selected thresholds \( \left\{ {l_{1} < l_{2} < \cdots < l_{m} } \right\} \) and \( \left\{ {k_{1} < k_{2} < \cdots < k_{n} } \right\}, \) and the regional constants c (the average values of the segmented regions) can be defined as follows (Chung and Vese 2010):

$$ \begin{aligned} E(c,\phi ) & = \sum\limits_{,i,j = 1}^{m - 1,n - 1} {\int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} - c_{i,j} } \right|^{2} } \chi_{1} } dx + \sum\limits_{i = 1}^{m - 1} {\int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} { - }c_{\text{i,0}} } \right|^{2} \chi_{2} dx} } \\ & \quad + \sum\limits_{i = 1}^{m - 1} {\int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} { - }c_{\text{i,n}} } \right|^{2} \chi_{3} dx + } } \sum\limits_{j = 1}^{n - 1} {\int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} { - }c_{{ 0 , {\text{j}}}} } \right|^{2} \chi_{4} dx} } \\ & \quad + \sum\limits_{j = 1}^{n - 1} {\int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} { - }c_{\text{m,j}} } \right|^{2} \chi_{5} dx + } } \int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} { - }c_{ 0 , 0} } \right|^{2} \chi_{6} dx + } \int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} { - }c_{{ 0 , {\text{n}}}} } \right|^{2} \chi_{7} dx} \\ & \quad + \int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} { - }c_{\text{m,0}} } \right|^{2} \chi_{8} dx + \int\limits_{{\Omega} } {\left| {{\text{I}}_{ 0} { - }c_{\text{m,n}} } \right|^{2} \chi_{9} dx} } \\ & \quad + \mu \sum\limits_{i = 1}^{m} {\int\limits_{{\Omega} } {\left| {\nabla H(\phi_{1} - l_{i} )} \right|dx} } + \mu \sum\limits_{j = 1}^{n} {\int_{{\Omega} } {\left| {\nabla H(\phi_{2} (x) - k_{j} )} \right|dx} } \\ \end{aligned} $$
(1)

where H is the Heaviside function, \( \mu > 0 \) is a weight parameter, and \( c_{ij} \) is the sub-regional constant. The parameter \( \chi_{i} \) is the combinations of level set functions \( \phi_{1} \) and \( \phi_{2} \); for an example, \( \chi_{1} \) is defined as \( H\left( {\phi_{1} - l_{i} } \right)H\left( {l_{i + 1} - \phi_{1} } \right)H\left( {\phi_{2} - k_{j} } \right)H\left( {k_{j + 1} - \phi_{2} } \right). \) As for other parameters \( \chi_{i} \), those parameters can be referred the paper of Huang et al. (2014). The optimal approximation I derived from the segmented results can be shown as:

$$ \begin{aligned} {\text{I}} & = \sum\limits_{i = 1}^{m - 1} {\sum\limits_{j = 1}^{n - 1} {c_{ij} } } \chi_{1} + \sum\limits_{i = 1}^{m - 1} {c_{i0} \chi_{2} + } \sum\limits_{i = 1}^{m - 1} {c_{in} \chi_{3} + } \sum\limits_{j = 1}^{n - 1} {c_{0j} \chi_{4} } \\ & \quad + \sum\limits_{j = 1}^{n - 1} {c_{mj} \chi_{5} + } c_{00} \chi_{6} + c_{0n} \chi_{7} + c_{m0} \chi_{8} + c_{mn} \chi_{9}. \\ \end{aligned} $$
(2)

The level set functions \( \phi_{1} \) and \( \phi_{2} \) can be found by employing finite difference such that initial level set functions will change their shapes to meet the energy minimization steps by using iteration scheme (Chung and Vese 2010; Huang and Wu 2010; Huang et al. 2014).

3 Processed Results by Employing Multilayer Segmentation

Thermal infrared camera (Thermo Gear-G120, NEC) was immediately used to record a series of thermal infrared images such that the surface temperature information shown on the pavements right after the pavement was rolled. The thermal infrared camera offers 240 pixels × 320 pixels image sizes with a thermal resolution of 0.1 °C. For a RAC pavement, the location of 45K of Taiwan Provincial Highway No. 1 was selected, and a series of thermal images were recorded at 5-min intervals. The recorded date can be tracked back to May 19th, 2015, and the measured times were from 14:35 to 16:03. As for an AC pavement, a county road was chosen, similarly, and a series of thermal images were recorded at 5-min intervals. The recorded date was June 16th, 2015, and the measured times were from 11:03 to 12:26. Totally, there were 19 thermal infrared images for RAC, and 16 thermal infrared images for AC, respectively. In Fig. 1, two images were selected: one is for RAC, and another one is for AC. Then, the multilayer segmentation used in this paper was employed to analyze the given thermal images.

Fig. 1.
figure 1

The thermal images of RAC and HMA pavements (240 by 320 pixels), and their corresponding digital images (960 by 1280 pixels)

Full size image

The multilayer segmentation is implemented by employing two initial level set functions \( \phi_{1} \) and \( \phi_{2} \); the level set function \( \phi_{1} \) is defined as follows:

$$ \phi_{1} (x,y) = \sqrt {(x - x_{i} )^{2} + (y - y_{i} )^{2} } - r $$
(3)

where r is the specified radius. As for the level set function \( \phi_{2} \), it is generated by Eq. (3) with several pixels of the centers of the function \( \phi_{1} \). For simplicity, the pre-selected thresholds were chosen as the same values for the level set functions \( \phi_{1} \) and \( \phi_{2} \). The surface temperature information is transformed to the digital numbers between 0 and 255. In doing so, the thresholds are set as 0, 20, 50, 70, 100, 150, 200 and 230. The parameter \( \mu = 0.005 \times 256 \times 256 \) is selected. With employing those parameters into the multilayer segmentation, the segmented results are classified by five classes. The average temperature value of each segmented region is used to replace pixel values contained in the segmented regions. The same thresholds and parameters are applied on the RAC and HMA pavements.

The multilayer segmentation can divide thermography into a series of sub-regions such that the temperature distribution in each segmented region is homogeneous. With employing the multilayer segmentation on the RAC pavements, the segmented results of the first thermal infrared image were illustrated in Fig. 2; the average surface temperatures of the segmented regions are 90.59, 85.61, 78.39, 69.77 and 58.52 °C, respectively. Similarly, the 7th thermal image of the AC pavement are used to analyze the surface temperature distribution, and the segmented results are illustrated in Fig. 3; the average surface temperatures shown in the segmented regions are 81.54, 76.29, 69.15, 63.97 and 52.99 °C. Both results show that both sides of RAC and HMA pavements have the highest surface temperature information. In doing so, the surface temperature information collected at every interval can be used to describe the cooling behaviors of RAC and HMA pavement materials. In Fig. 4, it was found that the surface temperature of RAC will be decreased faster than HMA. However, the cooling phenomena observed by the analyzed thermal infrared images are more complicate than the thermal segregation observed by PAVE-IR. The largest differences of the segmented average surface temperatures are over 30 °C. However, the differences of the segmented average surface temperature are decreased while the cooling time is increased.

Fig. 2.
figure 2

The segmented results by employing the multilayer segmentation on the RAC pavement, and the thermal image can be grouped into five classes; the average surface temperatures of the segmented regions are 90.59, 85.61, 78.39, 69.77 and 58.52 °C

Full size image
Fig. 3.
figure 3

The segmented results by employing the multilayer segmentation on the HMA pavement, and the thermal image can be grouped into five classes; the average surface temperatures of the segmented regions are 81.54, 76.29, 69.15, 63.97 and 52.99 °C

Full size image
Fig. 4.
figure 4

The tendencies of the highest and lowest average surface temperatures of the RAC and HMA

Full size image

4 Conclusions

Thermography can be used to illustrating the surface temperature information; the multilayer segmentation can be used to analyze thermography and provide more quantitative descriptions for surface temperature distributions. From the segmented results, the thermal images can be divided into several groups by the surface temperature distributions. In doing so, the surface temperature differentials can be determined and located. PAVE-IR is employed to monitor the thermal segregation during pavement construction, and those areas with thermal segregation will be fixed. However, a thermal infrared camera can be immediately installed to monitor the surface temperature distributions of the rolled pavement. Image segmentation proposed in this paper can automatically group those pixels with the similar surface temperatures such that the surface temperature distributions can be identified. The distributions could be the clue to explain why those repaired areas with thermal segregation still have anomalies.