Keywords

1 Introduction

According to the WHO, road traffic accident is the leading cause of death worldwide. Since 2007, more than 1.25 million people die and 50 million injuries occur every year by traffic accidents. Most of these deaths are in low and middle-income countries. The loss is approximately 3% of their GDP’s. Developing countries account more than 85% of the injuries. The problem is severe in Africa, specifically Sub-Saharan; they have only 4% of the global vehicles but account 10% of the global death. On the other hand, 60% of global vehicles are in the developed countries with only 14% of deaths. As can be seen in Fig. 1, the traffic accidents affect the economically active workforce group (15–50 age), in low and middle income countries [1].

Fig. 1.
figure 1

(Source WHO)

Top ten causes of death among people aged 15 to 29 years in 2012

Full size image

According to the report of Ethiopian Ministry of Transport, 1700 lives are lost and 7000 injuries occur each year. These are from the formal report of the police. However, there are many deaths and injuries which are not reported. Although the number of vehicles is very small in number, Ethiopia is one of the leading countries in traffic accident rate [2].

There are many reasons for the causes of traffic accidents, but the root causes are failing to observe the priority of pedestrians, passing the speed limit, human poor road network, absence of knowledge on road traffic safety, mixed traffic flow system, poor legislation and failure of enforcement, poor conditions of vehicles, lack of emergency medical services, poor road design, lack of automation or system computerization and monitoring [3,4,5]. World Bank warns that unless effective safety measures are taken, global road fatalities will increase by more than 65% between the years 2000 and 2020 [6]. The World Bank emphasized that traffic safety is the responsibility of governments, industries, business, non-governmental organizations and international agencies, with participation of people from all walks of life and disciplines [7].

Some of the problems associated with driving beyond speed limit, annual mechanical checkup verification, hit and run may be addressed by using ILPR systems.

In Ethiopia, Ministry of Transport carries out annual mechanical inspection of vehicles to ensure the technical competence of vehicles and their functionality. And if a vehicle has passed this test, a small sticker, which is valid for one year, will be posted to the vehicles windshield. A traffic police, on the road, undergoes a manual task of identifying vehicles that have not passed the annual mechanical inspection by stopping and checking whether it has the right sticker posted or not. However, there is no way to get information about any vehicle on the road using its LPN without stopping it. ILPR system will help computerize this manual task. Application of ILPR system is wide, for example in access control giving access only to vehicles of authorized personnel.

2 Previous Works

In traffic control, vehicles can be directed to different lanes for a better congestion control in busy urban communications during rush hours [8]. It is also used in tracking, automatic toll collection [9], automatic monitoring of traffic rules & maintaining law enforcement on public roads, surveillance, security, parking lots [10], monitoring vehicles passing through restricted areas [11], to mention few. The main parts of LPN recognition system are image acquisition, LP extraction, character segmentation, and character recognition [12]. In [13] cascade combination of adaptive boosting and Haar-like features are used to extract LP. The advantage of using Haar-like features is that irrespective of illumination variation, color, size, and position the LP can be detected. In [14], fast Vertical Edge Detection Algorithm (VEDA) was proposed, VEDA showed that it is faster than Sobel operator. Combination of edge detectors and mathematical morphology is used in [15] but, at the cost of computational time. Hence, [16] uses block-base algorithm for detection. Wavelet transform-based algorithm is also used in [17] for localization. Neural network based color extraction and template matching recognition of characters can be found in [18]. Genetic algorithm segmentation is used in [19] to extract the plate region. In [20] Gabor filter is applied for character recognition. As can be seen, it is possible to use many different features to find the location of the LPs, such as, shape, color, orientation and frequency. Edge detection is often used to help locate LPs. Systems detect vertical edges to find two parallel edges on each side of the LP, or to find characters [21]. Machine learning algorithms such as neural networks can be used to increase accuracy and computational speed [22]. Fuzzy algorithms like fuzzy C-means can also be used for plate detection [23]. Template matching based character recognition has been used in [27]. ANN have been adopted for increasing the speed of computation in [28] and Support Vector Machine (SVMs) [29] can be applied for classification.

While studying the literatures, we came across two papers which are studied in Ethiopia for the license plate recognition [25, 26]. Both of them are more or less similar. In the recognition part they both used template matching. But, ours differs, as will be discussed in detail under the “Proposed System”, the ILPR system uses ANN based character recognition and SMS reporting system. The recognized LPN is stored in a database for law enforcement, traffic control and facility management purpose. In this paper, we have used a still digital camera to acquire images of vehicles.

3 Proposed System

The proposed system basically consists of image acquisition, preprocessing and information retrieval (ILPR) components. Among current LP systems, the simplest example is that of European LPs. It contains black numerical and English characters on white background. The United States LP is complex because it contains figures, state names, different fonts and colors. Chinese or Japanese LP are similar to that of Europe. However, they contain many texts and complicated fonts [24]. In our system, the Ethiopian alphabets and numbers are more or less similar to the European LPs.

Figure 2, shows the process flow of our ILPR system.

Fig. 2.
figure 2

Flowchart of ILPR

Full size image

3.1 Capturing an Image and Preprocessing

Preprocessing is carried out on the image to improve its quality by removing noise. The following steps were followed for this phase.

  • Step1 : get picture of a vehicle with a still digital camera from the road

  • Step2 : Convert RGB to gray scale, as depicted in Fig.   3

    Fig. 3.
    figure 3

    RGB image (left) and gray image (right)

    Full size image

  • Step3 : Then filter the image using Gaussian filter.

3.2 License Plate Area Detection (Localization)

The second phase is plate localization. It is assumed that the LP is a rectangular area with increased occurrence of horizontal and vertical edges. The high density of horizontal and vertical edges on a small area is in many instances caused by an LP. This process can sometimes detect a wrong area that does not correspond to an LP. Because of this, we often detect several candidates for the plate, and then we choose the best one by a further heuristic analysis. Detection of an LP area consists of a series of operations. Modified snapshot is then projected into axes x and y. These projections are used to determine an area of an LP.

3.3 Image Segmentation

3.3.1 Edge Detection

This phase detects the presence of an edge within an image [30]. Edge detection simplifies processing speed, data and computational time [31]. Generally, an edge is characterized by an abrupt frequency or amplitude change in an image [32]. There are many different types of edge detection techniques. Here Sobel operator is used due to its high computational speed, accuracy and simplicity, see Table 1 and Fig. 4.

Table 1. Edge detection methods and their rate of success [33]
Full size table
Fig. 4.
figure 4

Detected edges

Full size image

3.3.2 Detecting and Cropping License Plate Area

We followed the following steps to detect, select and crop the LP area.

  • Step 1: Image Filling

    Fill holes, an area in the image surrounded by rounding lines, so that LP area can be identified to be isolated from the image. Output of this step is shown in Fig. 5 (Left).

    Fig. 5.
    figure 5

    LP detection and cropping. Image with LP candidates (left), Image with a true candidate (center), Image of a cropped LP (right)

    Full size image

  • Step 2: Scan and count each pixel in the following manner, where size of the LP area is supposed to be between x and y:

    If number of ‘white pixels’ < x;

           pixels become ‘black’

    Else; no change

    If number of ‘white pixels’ > y;

          pixels become ‘black’

    Else; no change

  • Step 3: Use Step 2 in both horizontal and vertical direction, changing x and y correspondingly.

  • Step 4: Check number of possible areas.

    If number of areas > 1,

          go to step 5

    Else; go to step 6

  • Step 5: Select true candidate for the LP. This function first calculates on aspect ratio; it then selects the candidate area with the most edges per area as the true LP. An example of the output of this stage is shown in Fig. 5 (Center).

  • Step 6: Logically AND with binary image obtained at “Edge Detection”

  • Step 7: Crop the LP

    This step determines position of the true candidate LP and uses that position to crop the LP from the original binary image. An example of the output of this stage can be seen in Fig. 5 (Right).

3.3.3 License Plate Normalization

The brightness and contrast characteristics of segmented characters vary due to different light conditions while the camera captures the image. The LP can sometimes be partially shadowed or non-uniformly illuminated. Because of this, it is necessary to normalize the LP. Histogram normalization, global, adaptive thresh-holding etc., can be used to enhance the brightness and contrast of an image. We used adaptive thresh-holding, since it computes threshold value for each pixel separately using its local neighborhood which increases the intensity and resolution of the LP.

3.4 Character Extraction

All the remaining steps depend on segmentation. If segmentation fails, a character can be improperly divided into two pieces, or two characters can be improperly merged. We can either use a horizontal projection of an LP for the segmentation or one of the more sophisticated methods, such as segmentation using the neural networks. If we assume only one-row plate, the segmentation is a process of finding horizontal boundaries between characters.

3.4.1 Segmentation of LP Using a Horizontal Projection

By detecting spaces in horizontal projection, the plate image can be segmented. After the adaptive thresh-holding, a horizontal projection of the LP is computed. This projection is used to determine horizontal boundaries between segmented characters.

3.5 LP Segmentation

Character segmentation is done using the following main steps:

  • Step 1: Connected components extraction.

    Step 2: Bounding-boxes detection.

    Aspect ratio and percentage area of the connected components are first computed. If the aspect ratio ‘r’ of the bounding box of a connected component is in an open interval, i.e., ]rMin, rMax[, and the percentage area pA, ]pAreaMin, pAreaMax[, then the character is segmented and its bounding box coordinates are stored in a vector R. Otherwise, the candidate is discarded. Note that, this process is run to every bounding box of connected components in the binary image obtained from “Detecting and Cropping License Plate Area.”

  • Step 3: Characters detection:

    The horizontal distance of each pair of nearest characters detected in the previous step is used as criterion in order to find new candidate characters.

Now, it is preferable to divide extracted LP into individual images, each containing one isolated character. Static bounds, vertical projection, and connected component are used for isolating. Here, connected component technique is applied. Bounding box is used to extract individual characters from the license.

3.6 Recognition of Characters Using ANN

Using the segmented images collected, as depicted in Table 2, as inputs we created and trained 3 feed forward neural networks which have two hidden layers each, for LP digits classification (digits 0 to 9), for LP region codes classification (AA, AM, OR, TG, ET) and for the LP identifier classification (digits 1 to 5). In all of the 3 artificial neural networks, we have used a gradient descent with momentum and adaptive learning rate back-propagation algorithm and log-sigmoid transfer function (Fig. 6).

Table 2. Extracted/Isolated characters
Full size table
Fig. 6.
figure 6

An example of a segmented neural network input character

Full size image

Figure 7 is an example showing input, output and hidden layer nodes for the first neural network using the input image shown in Fig. 8. We have resized the images of the characters to 7 × 5 pixels (height = 7 and width = 5), so as to have 35 input pixels to the neural network architecture.

Fig. 7.
figure 7

Feed forward back-propagation neural network for identifying digits 0 to 9.

Full size image

3.7 Notification

From the above steps we have acquired the LP identifier, region code and LP digits, which make up the LPN together. If an LPN was not found to be on the list of vehicles that have passed the annual mechanical inspection or if it was previously stored in a black list of LPs, a traffic police or a person in charge will be notified of the LPN. To facilitate this action, an android device has to be connected to a Personal Computer (PC), on which the MATLAB LPN recognition program is running, in USB debugging mode. The PC and the android device communicate through Android Debug Bridge (ADB) utility. Java program (which runs on the PC) is written to send an SMS of suspected LPNs, which it has accepted from the MATLAB program, using ADB through the connected android mobile device. SMS is used to send the LPN of the illegal car to the person in charge. Because of its reliability SMS based notification is chosen, it can even work in a very remote area where other access networks are hard to get.

4 Conclusion

It can be inefficient to manually keep track of vehicles on a busy road or parking lot. In order to overcome this problem, the ILPR is very helpful. Given an input image of a vehicle the ILPR should identify its LPN using ANN, undergo a comparison with previously stored database and send an SMS to a nearby traffic police if the detected LPN is found to be of a suspect vehicle. There may be commercial solutions available to automatic LP recognition which use different algorithms in other places, but the main purpose of our work is to develop a solution that also helps traffic police or any other concerned body identify a suspect vehicle automatically with a cheap or no price in order to minimize the death or injury which is reported every year.