Feature Based Multiple Vehicle License Plate Detection and Video Based Traffic Counting

Abstract

Traffic in the world is a vast problem which creates some inconveniences to the environment. Road traffic and traffic congestions have become a big mess in the society. So to monitor this issue video surveillance came into existence. With current technology, License Plate identification is in experimental from past few decades. Recognizing of license plate plays a noteworthy role in a few applications like Crime Investigation when the vehicles are stolen and when a vehicle met with some accident, for the Authorities to distinguish their own particular vehicle, for Security control, for Toll Enrolment, and identification of vehicle in parking slot. The proposed system accords with vehicle detection, traffic vehicle count and extracting of the license plate in an image which is obtained from a video sequence. The proposed method for license plate extraction copes with K-nearest neighbor classifier for training the input image and Contour Hierarchy technique is to identify and extract the license plate. This present strategy likewise it finds the closest match of the identified license plate number if the plate number already exists, then it produce the exact match of that license plate number. The Experimental outcomes license plate extraction creates exactness of the proposed work which can recognize around 99% times in case of 850 complex background images. The detection of vehicle is implemented using cascade classifier and background subtraction techniques and moment method is being used to count the vehicle in a motion video. In case of video detection and counting it produces 97.2% of accuracy.

1 Introduction

Intelligent Traffic System is one of the key parts. Traffic monitoring can pave a path to find solutions for people. A vision-based traffic analysis framework could comprise of numerous parts, for example, foreground segmentation, shadow removal, feature extraction, and tracking. For most traffic observation frameworks have fewer stages of traffic parameters, like vehicle location, counting, tracking, and classification. Every year, around million vehicle injuries happen on the road and more than a thousand is pushed to death. On monitoring traffic count and detecting vehicle and its license plate can reduce traffic congestion at peak hours and can avoid accidents. This monitoring helps in limiting the conceivable outcomes of folk activities in toll accumulation and rule violation. It is important to give better traffic surveillance to diminish the accidents. So the primary goal of this paper is to provide better traffic surveillance system.

The license plate extraction has four processes that include the following steps, image acquisition, locating license plate, fetching characters in the license plate and extracting the license plate. The main contribution of this research is to produce the clear view of the detected license plate number. In the case of crime the video extracted images plays and major role. The obtained images are blurred and not visible. In order to resolve this issue we have come out with this method to extract the license plate number and display them using an enhanced threshold technique. The rest of the paper is organized as follows. Section 1.1 describes related work. Section 2 elaborates proposed methods for real time vehicle detection, count and license plate extraction. Section 3 provides Experimental results and analysis. Section 4 gives the conclusion of the research work performed.

1.1 Related Work

In literature, there are lots of methods, which use specific features for detecting the license plate. In all countries there are standardized rules for the license plates, for their characters and colors. All the license plates are rectangular in shape and hence the aspect ratio is known, then it is possible to detect the plate using edge information. Multinational vehicle license plates have been detected based on rear-lights and Heuristic Energy Map of vertical edge information and using a unique histogram approach [1]. Using Sobel edge detector the license plates character and the border of the plate is detected [3]. Another method based on the Hough Transform which is only able to detect the straight plates [4]. Based on neural network a conversion of the image in the LAB color space to detect the license plate and it performs using level set methods to locate its contour. The license plate is detected in both day and night time using the line and clip functions in addition to Gaussian analysis [2]. A method for recognizing the vehicle number plate based on template matching using modified Otsu’s method algorithm for threshold partitioning with normalized cross correlation [5].

Real-time traffic flow parameter estimation based on KLT tracker, intersection point analysis using k-means clustering, traffic flow theory and connected graphs able to achieve count, density and speed of moving vehicles [6]. Shorter processing time traffic congestion method performed on non-adaptive traffic light scheduling algorithm with minimum destination distance first algorithm [7]. Vehicle detection and tracking based on UAV-based transportation traffic estimation is achieved by KLT feature and a particle filter [8, 9].

2 Proposed Method

2.1 License Plate Number Extraction (LPNE)

The proposed LPNE approach for multi-national and various vehicles works in two stages: (i) Training images using KNN classifier and Pre-processing of input images and (ii) Plate Recognition and Extraction of License Plate. The dataset holds different vehicle images with different brightening condition both on Day and Night time, the few pictures are not precisely the full picture of the vehicle, pictures that are taken through some other vehicle mirror and pictures that are acquired from video casings and pictures with shadow, pictures that are dim and pictures with more content are additionally incorporated into the preparation and testing. The challenging images from dataset outperforms well with the proposed method. Algorithm for License Plate Detection is as follows,

• Step 1: Image acquisition: Images of different vehicles are obtained as saved as a dataset.

• Step 2: Training all the input images using KNN classifier.

• Step 3: Fix different Aspect ratio based on image samples.

• Step 4: Pre-process the input image.

• Step 5: Apply Contour Hierarchy method for the input image.

• Step 6: Finds the Location of plate and possible characters of the license plate.

• Step 7: Find the extract or the nearest match of the detected license plate.

• Step 8: Extract the Plate and displays the number using a threshold.

Training and Pre-processing:

As first part detection of the license plate is to train images using K-nearest neighbor classifier and obtain the gray-level co-occurrence matrix (GLCM) features to classify the vehicles. After training with KNN based on the feature classification the values in the number plate find the exact match or the nearest match, this method is useful in some application of crime investigations of stolen vehicle or accidental vehicle when on a backend, some of the old list of vehicles in the database can be cross validated and produce the matches to find a solution for the case. After extracting the number of the license plates, the extracted number is passed as input to the KNN classifier to produce the best match relevant from the plate number given as input. In the proposed method the contour hierarchy function is used. The input passed to the contour hierarchy must be of black background and white text, to satisfy this condition the input image is pre-processed to a gray-scale and then adaptive threshold is applied.

Contour Hierarchy Method:

The hierarchy of the contour is a parent-child relationship. When few shapes are in different shapes, they are called as nested figures. In this situation, external read line is called as a parent and an internal red line is called as a child. Contour in the picture has some relationship to each other. It is associated with each other, as offspring of some other shape, or is it a parent and so on. The portrayal of this relationship is known as the Hierarchy.

In Fig. 1 Contours 0, 1, 2 are the external or outermost layers. The contour 2a, it is considered as a child of contour 2. Similarly, contour 3 is the child of contour 2. Finally, Contours 4, 5 are the children of contour 3a. In Fig. 2 Contour Hierarchy algorithm is explained.

Fig. 1.

(a) The contour hierarchy method, (b) the contour hierarchy method applied to the car. (Color figure online)

Fig. 2.

Contour Hierarchy algorithm

The Significance of Proposed Work:

The proposed technique distinguishes and extracts license plate number of any vehicle in complex background. The license plate is recognizable in different images like front, back, side perspectives and furthermore for crop images also creates the best outcome. Nearly three cars in a path are also able to detect and extract license plate number. Accident car image has been tested in the proposed method, which also out performs best and extracts the license plate number.

2.2 Video Based Detection and Counting

In this paper other technique is designed to detect the vehicles and count the number of vehicles in the motion video. To detect vehicle the method used is based on feature extraction and training based on KNN classifier. In order to classify the vehicle and detect them, it is mandatory to train with test images of different type of vehicles to detect vehicle correctly. While the video classifier worked decently well at detecting all cars during both day and night other vehicles like bus and trucks were also detected successfully. The video is captured with a digital camera and tested on the current method which provides a better result of detecting vehicle and counting them.

Real Time Background Subtraction:

In motion the pixels keep changing over time. In order to recognize the background and foreground, first identify the static object in the frame called as the background image and the foreground image is the one which keeps changing or in motion.

• Step 1: Estimate the background time proposition t,

• Step 2: Subtract the background image from each frame,

• Step 3: Apply threshold and obtain the foreground mask.

The Gaussian Mixture Model:

It is sufficient and flexible to deal with variation in multiple moving objects, lighting and other arbitrary objects. It utilizes a technique to display each background pixel by an optimized mixture of K Gaussian distribution in Eq. (1). The weights of the mixture speak to the time extents that every pixel is divided by its intensity in RGB color space. For every frame we must compute the probability, since it is multiple surface pixels is named as mixture model.

$$ P\left( {X_{t, } } \right) = \mathop \sum \limits_{i = 1}^{k} \omega_{i,t } .\eta \left( {X_{t} ,\mu_{i,t} , \sum\nolimits_{i,t} {} } \right) $$

(1)

X _t is the current pixel in frame t, K: the number of distributions in the Gaussian mixture, ω _i,t : the weight of the k^th distribution in frame t, μ _i,t : the mean of the k^th distribution in frame t, Σ _i,t : the standard deviation of the k^th distribution in frame t, Where η(X _t , μ _i,t , Σ _i,t ) is probability density function.

Vehicle Detection Using KNN:

The K Nearest Neighbor includes scanning for the nearest match of the test data in the component space of historical input data. Background mask using KNN when a larger region of pixel is detected, with and without motion the discern class produces the closest neighbor. The KNN mask works well on large area of motions like buses. The background mask using KNN gives the best balance between rapid response to changing of backgrounds, robustly recognizing vehicle motion, and not being triggered by external disturbance. The KNN classifier produces the best result of detecting vehicles.

Vehicle Counting:

The Vehicle counting is performed on Contour Extraction. The area and the perimeter for each frame are founded and based on the moments the counting output is produced. The active contour model is the famous method in computer vision, and snakes are colossally used in applications like object tacking, segmentation, edge detection, shape recognition and stereo matching. The active contour method is also called as snake model. The energy function is used in current method in Eq. (2). s(t) = x(t), y(t) where t is the parameter of the curve.

$$ E_{snake} = E_{snake} = \mathop \smallint \limits_{snake}^{ } E\left( {s\left( t \right)} \right)dt = \mathop \smallint \limits_{snake}^{ } [E_{\text{int}} \left( {s\left( t \right)} \right) + E_{ext} \left( {s\left( t \right)} \right)]dt $$

(2)

After the subtraction of the updates background of the current frame the edge information is obtained to calculate the external force of the active contour model. Since different noise influence from external sources can lead to disintegration of edge appropriation, on calculating the moments of the input can improve robustness. The Moment method of a snake with M is defined as,

$$ C = \sum_{i = 1 }^{M} \frac{{Z_{i} }}{M } $$

(3)

Where z_i = (x_i, y_i) is the 2D-coordinates of i contour. Figure 3 shows Vehicle detection and counting algorithm.

Fig. 3.

Vehicle detection and counting algorithm

As a result the current method successfully detects and counts the vehicle in motion. The video data consist of various video and the current method is tested with four different video and one video is captured using a digital camera is and pre-processed then tested with current method. The video data are bidirectional traffic flow, single lane view, the motion of the video can be on either left or right. The vehicles like truck, bus, van and car are detected. The current method using KNN classifier and Contour extraction provides the most exact detection, nearest detection. The error rate associated with each video is about minimum average and the accuracy is better than the related method. The following section shows the experiment result for both license plate and video based detection and counting of vehicles.

3 Experiment Result and Analysis

3.1 License Plate Number Extraction

The experiment for License Plate Number Extraction has been tested standard 850 challenging images with low quality, high quality and blurry images. Most of the vehicles are from different part of the country. The proposed method is compared with other latest and the classical method. Images with various backgrounds and various color plates are included in the dataset. The images that are taken in shadowed and dark are included to verify the adaptability of the proposed method.

In order to find the vehicle license plate number the current method is discussed with its output of images. In Fig. 4 the image (a) is taken as the test image, the next step as per algorithm is Preprocessing of image (b) image is converted into the grayscale image (c) The output of adaptive threshold (d) Execution of Contour hierarchy. The Fig. 5 is the trace of contour method and Fig. 6 is an output of license plate. The extraction of number is shown in Fig. 7 The multinational vehicle output is displayed in the next sections.

Fig. 4.

Vehicle license plate detection

Fig. 5.

Trace of contour method

Fig. 6.

The output of contour hierarchy method extracted license plate

Fig. 7.

Identifying the region of interest

The Fig. 8 shows, parking slot image where the extraction of a number plate is possible with the proposed method. Figure 9 shows the extraction of the number plate with very tilled license plate. Figure 10. Is the Image obtained through other vehicle’s mirror, and the extracted license plate number shows the exact match of the detected number.

Fig. 8.

(a) parking slot image. (b) Extracted license plate and threshold output.

Fig. 9.

(a) Image of a military vehicle produces best result in spite of tilled characters (b) extracted license plate and threshold output.

Fig. 10.

(a) Image obtained through other vehicles mirror, the input image finds exact match of the detected license plate (b) extracted license plate and threshold output.

The Fig. 11 shows the image of tourist bus with an extracted license plate number are nearest match. Figure 12 holds the Tamil Nadu government bus with extract license plate. Figure 13 proves the best of the proposed method by exactly locating the license plate and extracting it because the car also holds the similar number like a license plate number on the left most but the license plate number is correctly located and extracted.

Fig. 11.

Image of a tourist bus with relevant output of license plate number

Fig. 12.

Tamil Nadu Bus with its extracted license plate number

Fig. 13.

Ancient car, produce best result in spite of some number displayed on left of car

The Fig. 14 is most challenging images with headlight where the license plate is not much visible, But the novel algorithm produces the best output for this input image. The vehicle license plate is located accurately and the extracted plate is displayed (Fig. 15).

Fig. 14.

Poor lighting image with headlight

Fig. 15.

The image with headlight dark image

The accuracy rate is evaluated using statistical methods based on true and false rates. Each image is checked manually, if the detection and extraction of an LP of vehicle cover all areas, the output characters of plates are missing or partially detected, all the detection is effectively recognized. The images of video sequences are also tested, that can also able to extract license plate number.

3.2 Video Detection and Counting

To validate the accuracy video detection and counting of vehicles in a video of our proposed method is based on average speed per pixel by frames and vehicle count for each travel direction. The Figs. 16, 17 show the performance of current methods of video captured on a real time near University of Madras (Fig. 18).

Fig. 16.

(a) Output of bidirectional road (b) detection of single direction road.

Fig. 17.

Output of various vehicle detected in the video

Fig. 18.

Output of various vehicle counting in video

Proposed Method Output

The graph shows the number of vehicle’s counting by number of frames in Fig. 19. It portrays that number of count keep increasing as the number of frames increases. The video based detection and works well when compared to other recent method. The output of this method gives better accuracy.

Fig. 19.

The graph shows the count of the vehicle

4 Conclusion

The experimental results mainly focus on multinational license plate number extraction. In this paper, the attempt is made to detect and extract the number of the license plate of vehicle of various countries both on day and night. Thus, this paper use contour hierarchy method with KNN classifier tends to produce better output. The proposed method was tested on various vehicles like car, bus, van, bike, truck and auto. The proposed method is found to be robust and effective even under bad weather condition. The video based traffic count detection can detect and count various vehicles on bidirectional flow in the video. The current method works based on contour extraction and moment method. KNN is used to train the various images in order to detect multiple vehicles. The proposed method yields 98.4% for the vehicle license plate number extraction and 96.2% of accuracy for video based traffic vehicle count and detection.