Vehicle Brake and Indicator Detection for Autonomous Vehicles

Abstract

Automated detection of vehicle lights can be used as a part of the systems for forward collision avoidance and accidents. This paper presents automatic vehicle detection and tracking system using Haar-Cascade method. The threshold collection is HSV color space in the vehicle light representation and the segmentation selection of light area. The extracted vehicle brake and turn indicator signals are morphologically paired and vehicle light candidate information is extracted by identifying the Region of Interest (ROI). The canny edge light intensity of the vehicle is extracted and a novel feature is called Edge Block Intensity Vector (EBIV). In this experiment, traffic surveillance system is developed for recognition of moving vehicle lights in traffic scenes using SVM with polynomial and RBF (Radial Basis Function) kernel. The experiments are carried out on the real time data collection in traffic road environment. This approach gives an overall average higher accuracy 95.7 % of SVM with RBF kernel by using 36 EBIV features compared to SVM with Polynomial kernel by using 9, 16, 25, 36, 64 and 100 EBIV features and SVM with RBF kernel by using 9, 16, 25, 64 and 100 EBIV features for recognizing the vehicle light.

1 Introduction

Recognizing and classification of the vehicle lights is an important area for intelligent automated vehicles. The vehicle signals have been used as significant cue for driver assistance systems. Vehicle light detection and classification in videos has been receiving increasing attention of computer vision. Intellectual Vehicles with the human resources are appearing to be hopeful to avoid the accidents. However, involuntarily recognizing vehicle light detection in videos is difficult. The autonomous vehicle systems are suitable for all environmental conditions, during daytime and at night time. Although Recognizing and classification of the vehicle reduced the death in vehicle crashes, accident prediction and prevention would be the ultimate solution for maximizing driving safety. This paper focuses on vehicle detection, vehicle light detection and classification.

1.1 Related Work

Robust and efficient vehicle light detection (VLD) [1] algorithm is used for vehicle and vehicle light detection. Detection of automatic vehicle and tracking system using Haar-cascade which can be applied to traffic environment. Traffic surveillance system [2] is developed for detecting and tracking moving vehicles in night time traffic scenes using SVM classifier. Vehicle detection and classification [3] was done by locating their head lights and tail lights in the night time road environment using SVM classifiers. Detecting vehicles [4] in front of a camera-assisted vehicle during night time driving conditions in order to automatically change vehicle head lights between low beams and high beams avoiding glares for the drivers and vehicle classified using SVM classifiers. [5] proposed feature extraction and classification for rear-view vehicle detection. The appearance based hypothesis verification step verifies using Gabor features and SVM.

The rest of this paper is organized as follows: Sect. 2 describes a proposed approach. Section 3 discusses the Feature extraction. Section 4 presents the experimental results. Finally, Sect. 5 concludes the paper.

2 Proposed Work

The datasets are collected for 30 min approximately for experimental purpose. The input video is processed at 25 frames per second. The video series is converted into frames in .jpg format. The haar cascade method is used to detect the vehicle and vehicle extraction. The HSV color feature is used to detect the vehicle light. The Edge Block Intensity Vector (EBIV) feature is extracted and fed to SVM classifier. The overall block diagram of the proposed approach is shown in Fig. 1.

Fig. 1.

Block diagram of the proposed approach.

2.1 Vehicle Detection

The automatic vehicle detection and extraction [1, 6] has an significant importance in surveillance system. The traffic video sequences afford rich information regarding direction of the road, vehicle, pedestrian, traffic signs and lamps within its field of view as shown in Fig. 2(a). In this paper focal point is only vehicle and vehicle signals. First, the vehicle is detected using Haar cascade method [1]. Haar cascade [7] is used to find the vehicles shown in Fig. 2(b). The positive images contain vehicles and negative images did not contain vehicles. These set of samples are stored to XML files. Haar cascade XML file is used to detect the vehicles. Most of the detected vehicles are different size. Hence in order to minimize the computation, image size is reformed. The extracted vehicle regions are resized to image of size 240$\times$240. pixels as shown in Fig. 2(c).

Fig. 2.

Vehicle detection and extraction.

2.2 Vehicle Light Segmentation

The vehicle light segmentation is one of the mainly important features of vehicles. It is the major element which differs from other parts of the vehicles. The initial task of vehicle light detection is to convert RGB color image into HSV color image. The input RGB color format can represent any pattern color or intensity using a mixture of Red, Green, Blue components as shown in Fig. 3(a). RGB values could not represent the desired color range. In comparison, HSV is greatly improved than the RGB color at computer visualization and unproblematic to understand the color value. HSV is predictable and expedient color space. To adjust the parameter color region to real-world images convert it into the HSV color space as it is more sensitive to adjust and manipulate the threshold parameters better than RGB [8, 9].

Fig. 3.

Vehicle light segmentation.

Vehicle brake light, left and right indicator crave to be segmented out, while retaining the vehicles. The detected area is considered as yellow light only if the minimum and maximum values of HSV thresholds are in the range as mentioned in Table 1. The vehicle light is segmented using the HSV space vehicle light segmentation and small point of noise is affianced the image shown in Fig. 3(c). The operations of morphological erosion and dilation [10] used to remove the noises without changing meticulous light area. The morphological erosion course is to shrinks the pixels in the object boundary. The morphological dilation is to add the pixels in the object boundary as shown in Fig.  3(d).

Table 1. HSV space color values for vehicle brake and indicator lights.

2.3 Vehicle Light Detection

The prevalence of vehicle light detection systems for intelligent driver assistance system using video information is expanding rapidly in recent trends. Region of Interest (ROI) is used to detect the vehicle lights. ROI approach [11] is obtainable and it is utilized. The height and width of $ROI_{(light)}$ is calculated. ROI is identified as

$$\begin{aligned} {ROI_{(light)}=Height_{(t)}/Width_{(t)} } \end{aligned}$$

(1)

where, $Height_{(t)}$ is the $ROI_{(light)}$ of height at time ‘t’ and $Width_{(t)}$ is the $ROI_{(light)}$ of width at time ‘t’ for the video sequences.

Fig. 4.

Example frames for brake and indicator detection.

The height is calculated using,

$$\begin{aligned} {Height_{(t)}=H_{(t)}/H_{(max)} } \end{aligned}$$

(2)

where $H_{(t)}$ is the height of the frame at time ‘t’, $H_{(max)}$ is the maximum contour value at time ‘t’ in video sequences. Width of the bounding box is calculated using,

$$\begin{aligned} {Width_{(t)}=W_{(t)}/W_{(max)} } \end{aligned}$$

(3)

where $W_{(t)}$ is the width of the frame at time ‘t’, $W_{(max)}$ is the width of maximum contour at time ‘t’ in video sequence. Region of Interest (ROI) is found in the original frame which identifies all type of vehicle lights. Fig. 4 shows the located vehicle light using Region of Interest (ROI) for brake light, left and right indicator detection.

3 Feature Extraction

A novel feature called Edge Block Intensity Vector (EBIV) is proposed in this work. One of the most essential features of visual system is edge features, and using such edge orientation information is feasible to express shapes. For this reason, canny edge detection is utilized to identify the edges. The work is divided the vehicle light region into various size blocks such as 9, 16, 25, 36, 64 and 100 blocks shown in Fig. 5(b). In reason, vehicle lights are installed in centred position of the left and right corner. The vehicle light image is size of 240$\times$240. In order to minimize the computation, extraction of EBIV features as 3$\times$3, 4$\times$4, 5$\times$5, 6$\times$6, 8$\times$8 and 10$\times$10 blocks and each Block of size is 80$\times$80, 60$\times$60, 48$\times$48, 40$\times$40, 30$\times$30 and 24$\times$24 as shown in Fig. 5(c). The average intensity value of pixels in each block is calculated as Edge Block Intensity Vector (EBIV). In this work 9, 16, 25, 36, 64 and 100 dimension EBIV features are presented.

Fig. 5.

Edge Block Intensity Vector (EBIV) feature extraction.

4 Experimental Results

In this section, proposed method is evaluated using real collection road environment datasets. The experiments carried out in C++ with OpenCV 2.2 in Ubuntu 12.04 operating system on a computer with Intel $CORE^{TM}$ I5 Processor 2.30 GHz with 4 GB RAM. The obtained EBIV features are fed to SVM with Polynomial and RBF kernel for vehicle brake and indicator recognition. Each incident of the input vehicle light from an input video sequence is represented as a feature vector. Classically the dimension of the feature vector is between a few and several tens. In this experiment used 9, 16, 25, 36, 64 and 100 EBIV feature vectors for vehicle light recognition.

4.1 Dataset

Vehicle light recognition plays an essential role in traffic surveillance system. The dataset created during daylight in real road environment, is used for experimental purpose. The video is processed at 25 frames per second. The video is converted into frames in .jpg format. The sample images are shown in Fig. 6.

Fig. 6.

Example frames for real collection dataset.

The collected dataset is listed in Table 2. In this work, the images are taken into a training set at 30 min video and testing set at 16 min video.

Table 2. Dataset description.

4.2 Support Vector Machine

Support Vector Machine (SVM) is a popular technique for classification in visual pattern recognition [12]. The SVM is most widely used in kernel learning algorithm. It achieves reasonably vital pattern recognition performance in optimization theory [13, 14]. Classification tasks are typically involved with training and testing data. The training data are separated by $(x_1,y_1),(x_2,y_2),....(x_m,y_m)$ into two classes, where $x_i \in \mathfrak {R}^n$ contains n-dimensional feature vector and $y_i \in \{+1,-1\}$ are the class labels. The aim of SVM is to generate a model which predicts the target value from testing set. In binary classification the hyper plane $w_.x+b=0$, where $w \in \mathfrak {R}^n$, $b \in \mathfrak {R}$ is used to separate the two classes in some space Z [15]. The maximum margin is given by $M = 2 / ||w||$. The minimization problem is solved by using Lagrange multipliers $\alpha _i(i=1,....m)$ where w and b are optimal values obtained from Eq. 5.

The non-negative slack variables ${\xi }_{i}$ are used to maximize margin and minimize the training error. The soft margin classifier obtained by optimizing the Eqs. 6 and 7.

$$\begin{aligned} {\min _{w,b,\xi } ~~ \frac{1}{2}{{w}^{T}}w+C\sum \limits _{i=1}^{l}{{{\xi }_{i}}}} \end{aligned}$$

(4)

$$\begin{aligned} {{{y}_{i}}({{w}^{T}}\phi ({{x}_{i}})+b\ge 1-{{\xi }_{i}}, {{\xi }_{i}}\ge 0} \end{aligned}$$

(5)

If the training data is not linearly separable, the input space mapped into high dimensional space with kernel function $K(x_i, x_j) = \phi (x_i).\phi (x_j )$ is explained in [14].

4.3 Performance Evaluation

To compute F-measure,

$$\begin{aligned} {F-measure=2PR/(P + R) } \end{aligned}$$

(6)

where, P and R are precision and recall. The F-Measure computes some average of the information retrieval precision and recall metrics. Precision is calculated using following equation,

$$\begin{aligned} {Precision = TP/(TP + FP) } \end{aligned}$$

(7)

where, TP and FP are True Positive and False Positive. Recall is calculated using the equation,

$$\begin{aligned} {Recall = TP/(TP + FN) } \end{aligned}$$

(8)

where, TP and FP is True Positive and False Positive. TP is total number of correctly detected vehicles light. FP is total number of in-correctly detected vehicles light. False Negative (FN) represents the total number false detections.

4.4 Experimental Result for Vehicle Light Detection

The moving vehicle detection is very important for video surveillance. In this work, 3000 frames are taken into training set for each vehicle signals frame as 1000 and 1200 frames are taken into testing set for each vehicle signals frame as 400. In this experiment, the proposed approach utilized SVM with polynomial and RBF kernels. While, analyzing the performances of RBF kernels gives high accuracy compared with polynomial kernel. Table 4 shows the confusion matrix for vehicle brake and indicators using SVM Polynomial kernel with 36 dimension EBIV feature.

Table 3. Confusion Matrix for vehicle brake and indicators using SVM polynomial kernel with 36 EBIV feature (94.4 %).

Table 3 shows the confusion matrix for vehicle brake, left and right indicator using SVM RBF kernel with 36 EBIV feature.

Table 4. Confusion Matrix for vehicle brake and indicators using SVM RBF kernel with 36 EBIV feature (97.4 %).

Table 5 shows the Precision and Recall value for vehicle brake, left and right indicators using SVM Polynomial kernel with 36 EBIV feature.

Table 5. Precision and Recall value using SVM Polynomial kernel with 36 EBIV feature.

Table 6 shows the Precision and Recall value for vehicle brake, left and right indicators using SVM RBF kernel with 36 EBIV feature.

Table 6. Precision and Recall value using SVM RBF kernel with 36 EBIV feature.

Fig. 7 shows the overall performance of the F-measure using SVM Polynomial and RBF kernel with EBIV feature. SVM with RBF kernel gives better accuracy 97.4 % for vehicle brake and indicators recognition compared to SVM with Polynomial kernel by using 9, 16, 25, 36, 64 and 100 EBIV features and SVM with RBF kernel by using 9, 16, 25, 64 and 100 EBIV features.

Fig. 7.

The overall F-measure using SVM Polynomial and RBF kernel with EBIV feature.

Table 7. Comparative study of existing methods.

4.5 Comparative Study

Table 7 provides comparative study of various vehicle light detection methods using SVM classifiers and various types of datasets. The proposed brake and indicator light EBIV feature method is no directly similar to compare existing method.

5 Conclusion

Vehicle light detection and recognition systems have a wide applications in video surveillance. This paper proposed efficient vehicle tracking and recognition system for identifying and classifying moving vehicle lights for traffic surveillance using SVM classifier. The Edge Block Intensity Vector (EBIV) features are evaluated and fed to multiclass SVM with polynomial and RBF kernel. This approach gives a excellent classification accuracy of 97.4 % for 36 EBIV feature using SVM (RBF) kernel. Experimental results and comparison with existing methods are shown that the proposed system is effective and offers advantages for traffic surveillance in various environments. In future work, be going to improve the flexibility of this features are used in another classifier and recognize the vehicle brake and indicators.