Keywords

1 Introduction

The current commercial 3D laser scanning solutions for the construction industries can collect millions of three-dimensional points in a short period of time accurately and safely under stationary conditions. However, the post-processing, which is point cloud registration process to combine each point cloud from different scan locations into one coordinate system, is still a labor-intensive and time-consuming process. First, multiple scans from different scan locations are manually gathered by the operator with targets which are marked correspondences in the scan area to find common points between point clouds. Second, it can cause errors in placing and relocating targets. Third, these collected data are registered with matching common features or targets manually. However, the operator does not know until the end of the registration process that the collected data contain imperfections due to incomplete scans, hindering structures, absence of targets and lack of common features for registration in many cases. This is because current laser scanning methods provide limited feedback to the operator during the scan process and registration process [1]. As like this, the current quality control methods at a job site rely heavily on manual inspection which is labor-intensive, tedious, and error-prone. Also, the raw data collected from construction sites are often unstructured and poorly organized which results in time and cost overheads in interpreting the data as well as delays in sharing the relevant information to stakeholders [2]. Therefore, robot technology has the potential to reduce construction and maintenance costs and improve productivity, quality, and safety. The automated data collection of construction site conditions by a mobile robot would provide an attractive alternative for the execution of routine work tasks at a job site [3, 4].

To be a practical solution for construction applications, an autonomous mobile robot must have the capability to safely move around an unstructured and cluttered environment with many spatiotemporal objects and obstacles which continuously change the environment conditions (i.e., uneven ground surface, equipment, materials, soil stockpiles, temporary structures). Robot’s obstacle recognition and avoidance mechanisms should be fast, robust and not solely dependent on the as-designed construction information of the environment because quickly changing as-is conditions of a construction job site are different from as-designed conditions in most cases. Several methods exist in the literature for robot navigation in other domains which work by observing an existing map and planning the robot’s every move in advance [5, 6]. However, the existing methods are not suitable for construction applications due to the aforementioned construction site’s unique characteristics.

This paper introduces an autonomous as-is 3D spatial data collection method by a mobile robot specifically aimed for construction job sites with many spatial uncertainties. The mobile robot system uses a simultaneous localization and mapping (SLAM) techniques to determine its navigation paths and construct a 3D point cloud of a construction site. While SLAM provides a robot’s current location, it does not detect obstacles or uneven surface conditions. The kinematic modeling of the mobile robot was analyzed, and a fuzzy control was developed for robot’s autonomous navigation in an unknown environment with obstacles. Additionally, obstacle detection algorithms were used to recognize spatiotemporal obstacles to avoid in real time. The proposed SLAM and object-recognition methods were implemented and tested with a custom-designed mobile robot platform, Ground Robot for Mapping Infrastructure (GRoMI), which uses multiple laser scanners along with infrared and sonar sensors to sense and build a 3D environment map. To evaluate the overall navigation and object recognition performance of GRoMI, two experiments were conducted: one for an indoor building site and the other for an outdoor construction site. The following sections will present an overview of the related work, methodology, results, conclusion, and discussion.

2 Literature Review

2.1 Mobile Robot Navigation

Path planning in static and dynamic unknown environments had been receiving considerable attention over the past years due to the fact that optimized path planning is the key subject in the navigating of the mobile robot. There are two different type of path planning algorithms, global and local. Whereas the global path planning requires complete information of the obstacles and environment, the local path planning needs to collect information through sensors as it moves around in unknown environments. Even though many researchers have developed various algorithms for the path planning typically including D*, Bug, Vector Field Histogram (VFH) and Dynamic Window Approach algorithm for static and dynamic unknown environment, still several particular issues, especially in construction sites, should be considered in the navigation of mobile robots based on the purpose and function of the mobile robot itself.

One of the widely used algorithms for mobile robot and autonomous vehicle navigation is D*, which has had many variations over the years. It draws a map into discrete areas called cells to finds the shortest path from its origin to destination with the assumption [5] that it does not have any obstacles in the unknown area. Each cell has a back-pointer, representing the optimal traveling direction in the cell’s area and costs for traveling to neighboring cells [6]. When it enters in an unknown environment or encounters obstacles, it adds new information to its map, and find a new shortest path from current location to the goal if necessary. Due to the limitations given by the cells, this has created a problem on the functionality of the D* in the setting of the map causing an out-of-bounds area. When a target is located on the out-of-bounds area, D* receives signals that the target is unreachable because there are no cells covering the area that is located on the outer part of the wall [6].

Another simple type of path planning algorithm is the Bug algorithm, where the robot moves in a straight line toward goal till it senses an obstacle. Then the robot avoids obstacles by deviating from the line while updating new important information such as new distance from the current position to the goal. Due to the simplicity of the algorithm, it requires less memory and is very competitive on total computation time compared to other algorithms. The most commonly used and referred in mobile robot path planning are Bug1 and Bug2 [7], which has weaknesses in that both algorithms do not find the optimal and shortest path in general and is not a good solution in complex areas due to its simplicity, especially in a construction site having many unpredictable objects in the environment.

Vector Field Histogram (VFH) was developed to avoid unexpected objects and easily enter narrow passages while simultaneously steering a mobile robot toward the target. A VFH-controlled mobile robot maneuvers quickly and without stopping among densely cluttered obstacles [8]. The VFH method uses a two-dimensional histogram grid for statistical representation of obstacles. The one-dimensional Polar Histogram is derived and reduced from the Histogram Grid which is updated continuously and represents the polar obstacle density around the robot in real-time. Consecutive sectors with low polar obstacle density are selected based on its near target direction. Using VFH method would not be efficient especially in construction sites requiring mapping large areas because the occupancy grid map needs a large amount of memory to be able to keep count of the values in each cell. In addition, a local minima problem, which traps the robot in another position far away from its destination causing it to travel a longer distance to reach its goal, sometimes cannot be avoided, and the method does not perform well if a narrow area has to be passed through [9].

The main idea of Dynamic Window Approach method is creating a valid search space and selecting an optimal solution from the search space. The search space is restricted to safe circular trajectories that can be reached within a short-time interval and are free from collisions [10]. The optimization goal is trying to minimize the travel time by moving fast as much as possible in the right direction. Regarding speed and safety of the mobile robot navigation in a construction site, reaching the right position efficiently to scan scenes and build a 3D environment map without any collisions is the highest priority. In addition, the local minima problem exists because of over-speed and may prevent the robot from real-time stability and non-optimal motion decision for obstacle avoidance due to not considering the size constraint of a mobile robot [11].

2.2 Object Recognition

Once 3D data of the construction environment has been collected, it is necessary to identify and localize objects in the scene that are relevant to the construction management team. Object recognition from point clouds of a construction site setting has been explored for multiple applications such as safety monitoring [12], as-built modeling [13, 14], surveying tasks [15], and performance monitoring [16]. Live construction sites are especially challenging for the object recognition task due to the unstructured nature of the site which contains many moving objects as well as articulated objects [17].

There have been several attempts to handle object recognition in construction settings from the literature. Dimitrov et al. [18] proposed a region-growing method for context-free segmentation of point cloud data. The method performed well on challenging point clouds of the built environment due to the ability to handle different sources of point cloud data variance. However, the method only extracted object contours from a scene and did not explicitly identify the types of objects. Wang et al. [19] used boundary detection methods to automatically extract Building Information Modelling (BIM) components from point cloud data. A region-growing plane segmentation step followed by a rule-based classification method was applied to identify walls, doors, windows, and roof elements from point cloud data. However, the method does not work well with incomplete data and can only be applied as a post-processing step after the whole site is scanned. Bosché et al. [20] studied a Hough transform-based method to detect Mechanical, Electrical, and Plumbing (MEP) components from laser-scan data. The method was able to account for objects that are not built in the planned location, as well as objects that may be incomplete. The work focused on the case of cylindrical MEP components and is difficult to generalize to the more general case of non-parametric objects. Chen et al. [21] proposed the use of a voxel grid-based 3D descriptor to classify different construction equipment in construction sites. A machine learning framework was used in conjunction with hand-crafted feature vectors to train a classifier from construction equipment Computer Aided Design (CAD) models. However, the work only considered outdoor environments with less clutter and only detected limited categories of construction equipment. Pu et al. [22] investigated object recognition on laser-scanned point clouds for road safety inspection. The classification algorithm uses geometric features such as size, shape, orientation and topological relationships of the point cloud segments assign detailed classes such as traffic signs, trees, building walls, and barriers. This work also only considered outdoor environments such as a roadway which has notably less obstruction than a construction site. Kim et al. [23] proposed the thermal-based object recognition. This paper introduces an innovative method for generating thermal-mapped point clouds of a robot’s work environment and performing automatic object recognition with the aid of thermal data fused to 3D point clouds. However, it was limited to heat source objects, indoor environments, and it depends on the light condition.

Overall, there exist many mobile robot navigation and path planning algorithms. However, the construction sites are outdoor complex dynamic environments. The state-of-the-art path planning algorithm can occur the problem of course of dimensionality, which arises when analyzing data in high-dimensional (often with thousands) spaces. Furthermore, the state-of-the-art object detection methods are limited to post data processing and are not suitable as a source of semantic feedback during the scanning process. Thus, this study proposes the use of object detection while the laser scan is carried out to achieve real-time scene understanding for the mobile robot. Based on that information, the local planner finds a local goal and optimal path within the scanned area.

3 Methodology

3.1 Mobile Robot 3D Data Collection and Mapping

A mobile robot with a hybrid LiDAR system, GRoMI, was used for this study. GRoMI is composed of two major parts, a hybrid laser scanning system and an autonomous mobile robot platform as shown in Fig. 1. The upper part is a hybrid laser scanning system, and the lower part is a mobile robot platform. The advantages of this robotic system are: (1) data acquisition is possible while the robot is moving; (2) robot navigation and data collection can be carried out remotely or autonomously. Figure 2 describes the flowchart of the proposed approach.

Fig. 1.
figure 1

Mobile robot platform with a hybrid 3D scanning system

Full size image
Fig. 2.
figure 2

Flowchart of the proposed framework

Full size image

First, GRoMI detects objects around it when it starts scanning. Second, the local goal position is determined by the nearest frontier algorithm [24] by using horizontally mounted laser scanners within the range of the currently built 2D map area. It is to choose next local goal as the nearest unscanned location within currently built 2D occupancy map. Then, an artificial potential field is constructed to find the best trajectory to the goal position by using the position and size of detected obstacles. While GRoMI is moving to the goal position, it calculates the current position and orientation from a currently estimated map and a previously built map of the environment with LiDAR scanners. If GRoMI reaches the local goal, it computes a new local goal position and moves toward it. This process is repeated until it reaches the global goal position.

The laser scan data from the horizontal LiDAR is used by the SLAM algorithm to calculate the position and orientation of the mobile robot on the horizontal plane. The Hector SLAM algorithm, developed by Kohlbrecher et al. [25], was adopted in this study to perform laser scan matching between a current LiDAR scan and an incrementally-built map to obtain the pose estimation and horizontal map of the surrounding environment. The SLAM algorithm is performed as shown in Fig. 3. Figure 3a shows the current raw scan data from the horizontal LiDAR. Then, scan matching is performed with a currently estimated map and a previously built map of the environment in order to obtain translation and rotation parameters, which is shown in Fig. 3b. This process is continuously repeated to compute both position and orientation of the robot at each step along its trajectory as well as the 2D map of the environment, which is shown in Fig. 3c.

Fig. 3.
figure 3

Scan matching with horizontal LiDAR

Full size image

The position and orientation information, obtained from the 2D Hector SLAM algorithm, can be utilized to generate a 3D map by adding vertical scanning information. The vertical scans are first registered in the local coordinate frame of the mobile robot based on the angular displacement of the revolving frame of the vertical LiDARs. Then, each scan is added to the global coordinate frame by the corresponding transformation and rotation parameters estimated by the Hector SLAM. Therefore, 3D maps are gradually built up by stacking multiple planar scans.

During the SLAM process, GRoMI gets a position update every 1/25 of a second (i.e., 25 Hz) with the horizontally mounted LiDAR. The received information contains positions for the robot, positions of all objects on the field and the goal position. The new goal position is updated continuously when it reaches a goal position. On the other hand, GRoMI also collects data from the vertically mounted laser scanner to avoid obstacles which have distance information at different vertical angles from the mobile robot to obstacles in a 3D environment. Figure 4 illustrates this process. The numerous vertical laser lines (example lines of 30, 15, 10° are shown) are emitted from the vertical LiDARs. If there are no obstacles near the mobile robot, each line will generate perfect circles with a certain radius. On the other hand, the circles will be distorted if there exist obstacles in the environment as shown in the left figure. By using this information, the mobile robot is able to avoid obstacles.

Fig. 4.
figure 4

Detecting obstacles by using vertical laser lines (–90° to 90°)

Full size image

3.2 Object Detection

The robot navigation process is aided by an object detection routine which gives a pathfinding algorithm for a basic semantic understanding of the environment. The semantic classes assigned to the scanned environment is divided into (i) building elements such as ceiling, floor, and walls which remain static and (ii) obstacles on the ground such as equipment, materials, and workers which could potentially move around. The static elements represent absolute barriers through which the robot cannot navigate whereas the dynamic elements represent temporary barriers which have to be avoided by the robot at the current time but could disappear in the future.

A simple rule-based classifier is used since it is computationally efficient and does not require point cloud segmentation. First, a voxel-grid representation is used to down-sample the raw point cloud such that each 10 cm × 10 cm × 10 cm voxel contains a unique point. The Principal Component Analysis is used to normalize the voxel-grid coordinates such that z is zero at the elevation where the laser scanners are placed, and the x-y axes are aligned with the surrounding walls. Next, a rule-based formulation is used to classify each voxel (at position x, y, z) into 4 classes as given in the equation below. The notation | P | indicates the total number of voxels in the point cloud which satisfies a given condition. For example, voxels at a particular elevation, z’, are classified into the ceiling region if the total number of voxels at that elevation exceeds the ceiling threshold. Note that this formulation allows the floor and ceiling regions to have multiple elevations since the robot may travel into different rooms with different ceiling and floor heights. In this study, the voxel thresholds are set to 50, 50, and 150 for ceiling, floor, and wall respectively

$$ C_{(x,y,z)} = \left\{ {\begin{array}{*{20}l} {ceiling,} \hfill & {z \in \left\{ {z^{\prime } ,\left| {P_{{z > 0\,\& \,z = z^{\prime } }} } \right| > threshold_{ceiling} } \right\}} \hfill \\ {floor,} \hfill & {z \in \left\{ {z^{\prime } ,\left| {P_{{z < 0\,\& \,z = z^{\prime } }} } \right| > threshold_{floor} } \right\}} \hfill \\ {wall,} \hfill & {x \in \left\{ {x^{\prime } ,\left| {P_{{x = x^{\prime } }} } \right| > threshold_{wall} } \right\}or} \hfill \\ {} \hfill & {y \in \left\{ {y^{\prime } ,\left| {P_{{y = y^{\prime } }} } \right| > threshold_{wall} } \right\}} \hfill \\ {obstacle,} \hfill & {\,\,\,\,\,otherwise} \hfill \\ \end{array} } \right. $$

A separate obstacle detection routine is simultaneously run to handle voxels that are detected as such. The obstacle detection routine is updated every second to take into account moving objects. Neighboring voxels are iteratively merged together to form clusters of objects. Each cluster is assigned a bounding box determined by the combined dimensions of its constituent voxels. Further, the Principal Component Analysis is used to obtain the orientation of the bounding box.

3.3 Obstacle Avoidance for Navigation

From the results of obstacle detection, an obstacle avoidance algorithm can be used to safely navigate the mobile robot. The artificial potential field method proposed by Khatib [26] uses the approach of creating virtual forces. The mobile robot, obstacles, and a goal position are three basic components of this method. The basic idea is that obstacles generate an artificial repulsive force and a goal position produces an artificial attractive force. Therefore, the goal position and obstacles will generate an optimization problem which is to find a feasible trajectory from the current mobile robot position to a goal position. The direction of the mobile robot in any position of the field is decided by the direction of total field intensity, which is the summation of the artificial forces.

Figure 5 shows an example of an artificial potential field with three obstacles and the resulting trajectory from a starting position to a goal position. The starting position is (0,0) which is located in the left part of Fig. 5, and the goal position is located in the right part of Fig. 5. As shown in Fig. 5, the potential value of starting position is the highest except obstacles. On the other hand, the potential value of the goal position is the lowest. Therefore, the mobile robot flows down naturally to the goal position while avoiding obstacles because every position in the field has a gradient which points the best direction to the goal position.

Fig. 5.
figure 5

Example of an artificial potential field and the resulting trajectory

Full size image

The artificial attractive potential field (\( {\text{V}}_{\text{g}} ) \) to a goal position from the current position of the mobile robot is defined as

$$ {{\rm V}}_{{\rm g}} = {{\rm K}}_{{\rm g}} {{\rm r}}_{{\rm g}} $$
(1)

where \( {\text{K}}_{\text{g}} \) is a design parameter and \( {\text{r}}_{\text{g}} \) is defined as

$$ {\text{r}}_{\text{g}} = \left( {x_{g} - x} \right)^{2} + \left( {y_{g} - y} \right)^{2} $$
(2)

which is the square of the distance between the mobile robot and the goal position. (\( x_{g} \),\( y_{g} \)) represents the center of goal position and (x,y) is the current position of the mobile robot. The minimum of the function is reached to zero when the mobile robot approaches to a goal position.

The repulsive potential field \( ({\text{V}}_{\text{o}} ) \) for an obstacle is defined as

$$ {\text{V}}_{\text{o}} = \frac{{{\text{K}}_{\text{o}} }}{{r_{o} }} $$
(3)

where \( {\text{K}}_{\text{o}} \) is a design parameter and \( r_{o} \) is defined as

$$ {\text{r}}_{\text{o}} = \left( {x_{o} - x} \right)^{2} + \left( {y_{o} - y} \right)^{2} - R^{2} $$
(4)

which is the square of the distance between the mobile robot and the obstacle position minus a square of radius R. The radius R is room to avoid the collision which is set as 0.5 m. (\( x_{o} \),\( y_{o} \)) represents the center of obstacle position and (x,y) is the current position of the mobile robot. In this method, each basic component is modeled as a point. However, the areas where the mobile robot should not be reached are needed to be represented as a boundary. Therefore, the radius R is included in the modeling equation, which information is given by the object detection described in the following section. Also, the Eq. (4) generates a pillar barrier with infinite height to move the mobile robot without crossing any of the boundaries. In this paper, the parameter \( {\text{K}}_{\text{g}} \) is fixed to 1 and \( {\text{K}}_{\text{o}} \) is changed while the mobile robot is moving. The parameter \( {\text{K}}_{\text{o}} \) determines the importance of obstacles, which is calculated by the distance between the mobile robot and the obstacles. The higher weighting obstacles, those that are close to the mobile robot, are taken into account as important obstacles when calculating potential field. The total potential field of the local scene is computed by adding individual potential field values as Eq. (5).

$$ {\text{V = V}}_{\text{g}} + \sum\limits_{i = 1}^{n} {V_{{o_{i} }} } $$
(5)

The distance the mobile robot moves toward or away from the obstacles is used to increase or decrease the parameter \( {\text{K}}_{\text{o}} \) and thus making the change of total potential field values.

$$ \begin{array}{*{20}c} {{\text{H}}_{\text{o}} = {\text{K}}_{\text{o}} + e^{ - a \cdot d} ,d = \sqrt {\left( {x - x_{{o_{i} }} } \right)^{2} + \left( {y - y_{{o_{i} }} } \right)^{2} < 0} } \\ {{\text{K}}_{\text{o}} = {\text{K}}_{\text{o}} - e^{ - a \cdot d} ,d = \sqrt {\left( {x - x_{{o_{i} }} } \right)^{2} + \left( {y - y_{{o_{i} }} } \right)^{2} > 0} } \\ \end{array} $$
(6)

where, a is a factor which is set to 2. The best range for the parameter \( {\text{K}}_{\text{o}} \) is between 0.05 and 1 because the parameter should be positive and not be too large. Therefore, the parameter should be set to 0.05 when it becomes smaller than 0.05 and set to 1 when it becomes larger than 1.

The trajectory should be determined by the best direction to the goal position for the current position of the mobile robot. The Newton direction method is used in this paper, which represents the optimal direction in which a step should be taken next. The Newton direction is defined as

$$ \vec{s} = - H_{V}^{ - 1} \cdot \nabla V $$
(7)

where \( \nabla V \) is the gradient which describes the first order derivative of the total potential field and \( H_{V}^{ - 1} \) is the inverse of the Hessian matrix which describes the second order derivative of the total potential field in Eq. (5). The first order derivative determines the steepest descent but it is not necessarily the goal direction. Therefore, the second derivative information is important due to finding optimal trajectory when the slope is tilting left or right. This is because the slope is not real but an artificial potential field.

4 Simulation and Experimental Results

Simulated data was first used to evaluate the artificial potential field method. As shown in Fig. 6, the laser scanning process was simulated by the yellow line which detects obstacles. In this simulation, the moving obstacles were used to demonstrate the robustness of this method. The black circle is the original position of obstacles, and the red circle is the current position of moving obstacles as times goes by. The light blue circle, near the red circle which is the current position of the mobile robot, represents the estimated position of obstacles sensed by LiDAR. Lastly, the light red circle indicates the current position of the mobile robot and the red line represents the optimal trajectory of mobile robot calculated by the artificial potential field. The contour line demonstrates the level of potential in every point in the field which has the pillar barrier obstacles and the local minimum at a goal position.

Fig. 6.
figure 6

Simulation combining the artificial potential field and odometry model (Color figure online)

Full size image

Based on this simulation result, the navigation and obstacle avoidance algorithm with artificial potential field method was implemented to GRoMI. The object detection and the navigation routine were evaluated through a series of laser scanning experiments conducted in an indoor environment. The building consists of a series of hallways, an example of which is shown in Fig. 7. Figure 8 shows the recognition result of building elements. The scan results are continuously updated in real time as the robot progresses through the environment.

Fig. 7.
figure 7

Image of the hallway where indoor laser scan experiment was carried out

Full size image
Fig. 8.
figure 8

Building element recognition

Full size image

Figure 9 shows the obstacle detection results, where the red boxes indicate individual obstacle instances that have to be avoided by the mobile robot, while the white points indicate the raw point cloud data, and the bold yellow line indicates the trajectory of the robot. The obstacle position and orientation are updated at every scan cycle to take into account possible movement of objects. The updated position and volume data of obstacles are provided to the navigation routine for constructing the obstacle barrier on the potential field. Only coarse object bounding boxes for detected obstacles are needed for navigation purposes.

Fig. 9.
figure 9

Obstacle detection in the indoor environment (Color figure online)

Full size image

With the same manner, an outdoor construction site was tested. Figure 10 shows the final 3D point clouds with fused RGB data which were generated by GRoMI. The bold yellow line indicates the trajectory path that GRoMI created. The 3D point clouds were built up by capturing laser scanner and camera data in static scan positions. Four separate scans were combined into a single point cloud automatically using the transformation matrices derived from the SLAM solution.

Fig. 10.
figure 10

An autonomously generated 3D point cloud of the unstructured outdoor construction site by GRoMI. A bold yellow line is the trajectory path created by GRoMI. (Color figure online)

Full size image

5 Conclusion and Discussion

This study proposed and implemented a custom-designed mobile robot platform for use in automated 3D data collection from construction sites. The mobile robot used Simultaneous Localization and Mapping (SLAM) coupled with object detection and obstacle avoidance algorithms to successfully navigate unstructured and cluttered environments. The SLAM result was used to localize the mobile robot in the scanning environment and determine the translation and rotation of each set of scans in the world coordinate system. Whereas, the object detection results were used to rapidly recognize static obstacles such as walls and dynamic obstacles such as equipment, materials, and workers. At the end of the automated scanning process, a complete and registered 3D point cloud of the scanning environment was obtained.

Experimental results showed that the proposed approach was an effective way of detecting and avoiding obstacles in real-time for a mobile robot to successfully collect as-is 3D geometry data at both indoor and outdoor construction sites. Also, it was identified that the outdoor application was much more challenging for the autonomous mobile robot compared to the indoor application due to uneven ground conditions, different soil types and conditions (e.g., loose, wet), and many scattered obstacles on the ground. Especially, dead-reckoning could not be applied at the outdoor site due to loose and slippery soil conditions, which is very common in construction sites. The 2D SLAM and 3D laser scanning for navigation, however, were sufficiently enough for estimating robot’s location and navigation paths. Overall, the point clouds collected by GRoMI were sufficiently accurate to be used for many construction management applications such as construction progress monitoring, safety hazard identification, defect detection, and material inventory management. For future study, the research team plans to investigate the collaborations between laborers and an autonomous mobile robot in close proximity with potential field construction applications. Moreover, machine learning techniques will be further explored to improve the current point cloud object detection accuracy.