Keywords

1 Introduction

Real-Time road detection is one of the most important issues in intelligent vehicle due to its essentiality and necessary in active safety, auto-driving and navigation. To solve this problem a lot of research have been done using visible image [1–3], infrared image [4, 5] and point set of light detection and ranging (Lidar) [6, 7]. Visible image is the most common perception data. A lot of research based on Visible image have done to deal with this problem. To detect road from a single color image, Hui et al. [1] proposed a vanishing point and Foedisch et al. [2] introduced a neural network. For stereo visible image, Son et al. [3] combined a posteriori probability and visual information for image segmentation. Visible image is widely applied for its high resolution, abundant texture information, and lower cost, but it is easier to be influenced by illumination, shadows, shield, etc. Infrared image is mainly used to detect pedestrian [4], but Fardi et al. [5] proposed a Hough transformation-based approach to detect road. The weakness of infrared type sensors is too sensitive to the temperature. To overcome those difficulties, Lidar sensor is a good choice for it stability of illumination, shadows, and temperature. Hence, some research has been done using this type of active sensor. For example, Kirchner and Heinrich [6] proposed a model-based approach and Wijesoma et al. [7] introduced a extend kalman filter. However, the computational complexities of those methods are high and cannot be used to achieve real-time road detection.

To solve the aforementioned problem, a new method based on line segments extraction is approached. For either the urban environment or the rural areas, the important feature of the road is that the road surface could be approximately represented by some planes. Hence, the Lidar’s scanning plane and the road surface intersect at a set of line segments, and a line segment means a road plane. To extract the line segments from these segments, a least mean square problem is proposed, which can be solved by an iterative line fitting approach. To eliminate the perception dead zone, some suitable historical data are adopted combining with the fresh data. To extract road from those data, an initial road range is given based on the hypothesis that the area under the vehicle is road. A lot of experiment results demonstrate the robustness and efficiency of the proposed approach for real-time auto-driving of the intelligent vehicles.

The rest of this paper is organized as follows. In Sect. 2, the model of road is built, and the point set segment method and the iterative line fitting approach are proposed. In Sect. 3 the perception dead zone elimination and the road detection are presented. In Sect. 4, experiments in the urban environment and campus are carried out to demonstrate the robustness and efficiency of the proposed approach for real-time auto-driving of the intelligent vehicles.

2 Road Model and Line Segment Extraction

2.1 Road Model

The most important feature of the road in the urban and highway environment is the well-paved surface and the distinct edge such as isolation belt and curb. For the rural areas in Fig. 1, the road is not so paved and the boundary is ambiguous too.

Fig. 1
figure 1

The actual road. a Urban environment. b Highway. c Rural environment

Full size image

Hence, a unified model is needed to present the road structure for all the above-mentioned environments. For either the urban environment or the rural areas, the important feature of the road is that the road surface could be approximately represented by some planes. The isolation belt and curb could also be treated as a plane perpendicular to the road surface. Hence, the Lidar’s scanning plane intersects with the road surface or edge at a set of line segments, and a line segment means a plane.

2.2 Point Set Segment

If the Lidar beams fire at the same object, the distance of the adjacent points is small, and it is clearly shown in Fig. 2. Otherwise a large distance means the adjacent points are belong to the different objects. Hence, the point set is segment according to the distances of each pair’s adjacent points, the unified road model is shown in Fig. 3 and Table 1 present the process of segmentation.

Fig. 2
figure 2

Configuration of the Lidar. a Location of Lidar on the intelligent vehicle. b Title angle and angle resolution of the Lidar

Full size image
Fig. 3
figure 3

Unified road model

Full size image
Table 1 The process of point set segment
Full size table

In the implementation, an average filtering is used after the segmentation to smooth the data in the same segment. The segments that have only one point are treated as the isolated point and those segments are removed, and the result of segmentation is shown in Fig. 4.

Fig. 4
figure 4

Result of segment and line segment extraction. a Result of segment. b Result of line segment extraction based on the result of segment

Full size image

2.3 Line Segment Extraction

The point set is segmented in the previous step, but it is not mean that one segment contain only one line segment. Hence, the problem how many line segments are contained and how to extract the line segments in each segment is proposed. To solve this problem, an iterative line fitting approach is adopted [8], and Table 2 presents the process of line fitting.

Table 2 The process of line segment extraction
Full size table

This iterative approach for extracting the line segments from the segments is terminated until all the segments are fitted to the proper line, and the computation complexity of this algorithm is O(log2(N)).

3 Road Detection

3.1 Eliminate Perception Dead Zone

A small vertical perception angle is the main disadvantage of the Lidar sensor, consequently, it is important to enlarge the Lidar’s scanning rang to eliminate the perception dead zone in front of the intelligent vehicle. Here we suppose that the vehicle is driving forward and its pose information is available at each moment. Therefore, the suitable historical data can be adopted to eliminate the perception dead zone.

The scanning data of Lidar could be easily corresponding with the pose information. To eliminate perception dead zone, the suitable historical Lidar data right corresponding to the dead zone are selected according to the pose information. The selected historical Lidar data is transformed to the current vehicle coordination according to the change of corresponding pose information.

Here, a four scan-level Lidar sensor is used and one frame historical scan is transformed to the current vehicle coordination to eliminate the dead zone, so we have eight scan data for road detection. In Fig. 5, the result of dead zone elimination is shown, and it is clear that the red historical data enlarge the perception range of the sensor.

Fig. 5
figure 5

Result of perception dead zone elimination. a Image of the real road environment. b Result of perception dead zone elimination, the blue data is the fresh data and the red data is the historical data

Full size image

3.2 Road Detection

Before road extraction, the point sets are segmented as the approach mentioned in part B of Sect. 2, and then the line segments are extracted from those segments using the method in part C of Sect. 2. To extract road from those line segments, we hypothesize that the area under the vehicle is road. Hence, the initial road range is given as the areas occupied by the car. Then the road detection is achieved for the scans from the close to the distance with a road range value that is updated by the previous scan. The process of road extraction is presented in Table 3.

Table 3 The process of road extraction
Full size table

Here we have eight scan data to detection road. To ensure the accuracy, all the eight scan data are computed as the above-mentioned steps. The line segment extraction based on least mean square algorithm is the most time consuming step. The computation complexity of each scan data is O(log2(N)), so the total computation complexity of the road detection is O(8log2(N)). It is not a calculation burden for the current computer while N is 220 in this paper. The result of road extraction is shown in Fig. 6.

Fig. 6
figure 6

Road extraction result of the urban environment without obstacle. a Image the real road environment. b Road extraction result, green is the road area

Full size image

4 Experiment Results

To demonstrate the robustness and efficiency of the proposed approach for real-time auto-driving of the intelligent vehicles, a lot of experiments in the real urban traffic environment and campus have been done. Here some experiment results of different traffic scenes are carried out.

The most common case in the real traffic scene is the obstacle which mainly is vehicle and pedestrian on the adjacent lane or in front of the vehicle, so it is significant to detect the road areas in those situations robustly. The road extraction results of those environments are given in Figs. 7 and 8, and it is clear that our approach works well in this kind of scene.

Fig. 7
figure 7

Road extraction result of the urban environment with vehicles. a Image of the real road environment. b Road extraction result, green is the road area

Full size image
Fig. 8
figure 8

Road extraction result of the urban environment with pedestrian. a Image of the real road environment. b Road extraction result, green is the road area

Full size image

The barrier is an important instrument on the road for it is essential to defend the vehicles turning back arbitrarily and the humans across the road. Consequently, it is important to detect road areas in this kind of situation and Fig. 9 illuminate that our approach is efficient.

Fig. 9
figure 9

Road extraction result of the urban environment with barrier and obstacle. a Image of the real road environment. b Road extraction result, green is the road area

Full size image

Cross is a complex traffic scene, it is essential to deal with this kind of scene for auto-driving of intelligent vehicle. The result is shown in Figs. 10 and 11, it is obvious that our approach can give out right road areas for vehicle driving through the cross.

Fig. 10
figure 10

Road extraction result in the cross of the urban environment without obstacle. a Image of the real road environment. b Road extraction result without obstacle, green is the road area

Full size image
Fig. 11
figure 11

Road extraction result in the cross of the urban environment with obstacle. a Image of the real road environment. b Road extraction result with obstacle, green is the road area

Full size image

In some particular situation such as the traffic accident and road works, the road is bounded by the traffic cones. Hence, it is necessary to detect the safe areas of the road. Figure 12 demonstrates that our approach is valid for this situation.

Fig. 12
figure 12

Road detection result of areas restricted by traffic cone. a Image of real environment. b Road extraction result, green is the road area

Full size image

5 Conclusion

This paper propose a real-time road detection approach using the Lidar data, meanwhile a lot of experiments in the real traffic scene and campus environment illuminate the efficiency and real-time capacity of the new approach. The feature work is to parameterize the present road.