Keywords

1 Introduction

Traffic accidents are among the leading causes of fatalities and injuries worldwide. Based on the third global status report on road safety, more than 1.2 million lives are lost annually due to traffic accidents. If no appropriate action is taken, it is estimated that the number will rise to nearly 1.9 million by 2020 [1].

Traffic signs are an important element of the road infrastructure which provides important information about the current state of the road. They are meant to regulate, warn and guide traffic and play a key role in ensuring road safety. Traffic signs are designed to be easily detectable as they sometimes communicate complex information at a glance. The Vienna Convention on Road Signs and Signals has been trying to standardize road signs around the world since 1968. Certain shapes and colors codes have been compiled to categorize different types of traffic signs [2].

One approach to minimize distracted driving is the development of Advanced Driver Assistance Systems (ADAS). ADAS are intended to support drivers in performing various driving tasks safely and to minimize the risk of traffic accidents due to driver negligence or erroneous decisions [3]. A subset of ADAS, traffic sign detection and recognition (TSDR) systems offer significant information to drivers about the surrounding environment and road restrictions in real time. Not limited to ADAS, traffic sign detection and recognition systems have also been exploited for other applications such as autonomous driving and sign inventory and maintenance [4, 5].

So far ADAS have been mainly limited to luxury vehicles. Such systems require several built-in sensors, cameras and controls. Unfortunately, a large number of vehicles are still not equipped with such systems. To make up for the lack of ADAS in conventional vehicles, smartphones have attracted the attention of researchers. The prevalence of smartphones and the recent advances in computational power and sensor accuracy of these mobile devices have transformed them into an appealing platform for the development of low-cost driver assistance applications.

We present in this work a low-cost driver assistance system that relies on image processing techniques from the OpenCV library for detecting traffic signs using Android smartphones.

2 Related Literature

OpenCV provides functions for image acquisition and processing which can be exploited for traffic sign detection. The application presented here follows standalone system architecture. It takes in live frames from the back camera of smartphones and uses the smartphone’s CPU to process the frames and detect traffic signs. The application does not require supplementary hardware, is user-friendly and cost-efficient.

TSDR has been an important and active research topic in the field of Intelligent Transport Systems (ITS) for the past 3 decades. The earliest study of TSDR is reported in Japan in 1984. Since that time several researchers have analyzed different methods to overcome certain difficulties and improve the performance of TSDR systems. In most real-time experiments, one or two cameras are mounted on the front of vehicles, a PC system is embedded in the car to capture the videos and a display is used to show detected signs. Only a few studies have implemented TSDR systems on smartphone platforms. This is due in part to the immaturity of smartphone camera and processor units in the recent past.

For example the authors in [6] developed a smartphone application to detect and recognize stop signs for inventory and assessment purposes. They utilized color-based detection techniques to identify the areas that differed in brightness and color from surrounding regions, so-called “blob regions”. The regions were then tested with trained classifiers to determine if they contained a stop sign.

In [7], still using smartphones, an Android application which received the images and reprocessed them to reduce their resolution and volume was proposed. The frames were then sent to a centralized server for processing. The authors claimed that since the computational power and battery life of smartphones are limited, a client-server approach could better perform the required heavy processing. They used the specific RGB color ratio features to detect the position of the road sign and extract the sign from the original image.

Several studies have compiled the difficulties and challenges of the TSDR process. Factors like variable lighting conditions, air pollution, weather conditions (e.g. fog, rain, sun, and haze), shadows, motion blur, car vibration, sign distortion, fade and partial occlusion can hinder the performance of a TSDR system significantly. Research has led to the development of various methods and strategies to minimize the effects of these influencing factors [8].

Road sign detection involves identifying a certain object from a generally unknown background. To achieve this, there are two main methods for object detection from that can be applied:

  • A Haar cascade classifier for traffic sign detection can be trained. This approach requires building a classifier for every traffic sign, which results in a time-consuming detection process as it uses a major part of the processor capacity.

  • The second method, color segmentation, offers better performance as it does not require any classifier.

In this work we rely on the second approach and apply color-and shape-based methods to achieve the highest possible precision in detection.

3 Technical Implementation

The developed smart phone application is intended for a smart device with an Android operating system that is equipped with a rear camera, GPS and accelerometer sensors so that the detection and recognition systems work only when the vehicle is moving.

To implement the code for image processing and detection using color-based segmentation method, the following steps were performed: image acquisition, region of interest calculation, color segmentation, blob detection, shape classification and plausible sign detection. We additionally designed a graphical user interface that included check-boxes, push-buttons and a camera display to enable or disable features easily.

In the first step, the mobile application captures a frame using the smartphone’s back camera that will be later processed. In order to reduce the detection time, an approximate area is calculated in which the traffic signs are most likely located. This area excludes 15% from the top and 35% from the bottom parts of the original frame. This total 50% of frame reduction significantly reduces the detection time in the search area. Figure 1 shows how this region of interest (ROI) is calculated.

Fig. 1.
figure 1

Calculation of the region of interest (Color figure online)

Full size image

We targeted in this work those signs which are dominated or bordered by the color red (e.g. stop or entry prohibition signs). To detect them within the ROI, we converted the reduced area from the Red, Green, Blue color model (RGB) to Hue-Saturation-Value (HSV) in order to extract the red color. We used HSV because it is more efficient handling lighting differences, and makes it possible to better discriminate the colors [7]. We then applied Gaussian Blur filters to reduce the noise from the processed frame and detect the red color. As a final step, we applied Binary Large OBject (BLOB) processing to detect the group of connected pixels in the binary image, differentiating between the “large” objects of a certain size that were relevant to this work (i.e. traffic signs) and the other “small” binary objects that were defined as noise. This method was selected due to its effectiveness in finding the targeted objects.

We developed a function to determine the pixels shape and area in order to find the biggest symmetrical target, a process which helps identify falsely-detected objects. Finally, if a red, symmetrical object such as a polygon, circle, triangle, etc. was detected, the application magnified the image and displayed it on the smartphone’s screen while triggering an alarm. Figure 2 depicts the implementation steps. Figure 3 illustrates the image processing steps from left to right and top to bottom.

Fig. 2.
figure 2

Overview of the implemented algorithm process.

Full size image
Fig. 3.
figure 3

Overview of the image processing steps from left to right and top to bottom. (Color figure online)

Full size image

4 Application Evaluation Results

To evaluate the performance of our implemented application regarding sign detection rate, accuracy and time, we performed several tests with a vehicle under daylight conditions and the optimal device position. Two smartphones with different hardware specifications were used in these tests:

  1. (A)

    Samsung Galaxy Fame (480p@25fps Camera = 5MP, RAM = 512 MB).

  2. (B)

    Samsung Galaxy A5 with higher image processing performance and better camera resolution (1080p@30fps Camera = 13 MP, RAM = 2 GB).

4.1 Traffic Sign Detection Rate

For the evaluation of the traffic sign detection algorithm, we performed 45 different tests with a minimum of one red sign per test with a detection range of up to 20 m. According to the device performance, the detection rate with device B was higher. 86.6% of all traffic signs were detected and the rate of false detections constituted 31.1%. A reduction in the detection time was observed depending on the RAM and CPU power. The detection time with device B ranged from 211 to 255 ms, the average detection time being 235.6 ms. The detection time with device A ranged from 35 to 45 ms, the average being 40.3 ms. Figures 4 and 5 illustrate the comparative results from both devices.

Fig. 4.
figure 4

Detection rates and time results for devices B (above) and A (below). (Color figure online)

Full size image
Fig. 5.
figure 5

Comparative results for the detection rates for devices B and A. (Color figure online)

Full size image

5 Conclusion and Future Work

In this paper, we proposed an application that uses a smartphone to monitor the road for traffic sign detection. The evaluation of the results showed a high object detection rate (86.6%) within a few milliseconds of time. As expected, the detection time differed on smartphones with different hardware specifications, the detection rate being higher for device B. Due to the higher resolution and number of pixels to process, a longer processing time was required for device B. Device A, although having a lower camera quality, detected the target objects as well. The number of false detections could be reduced by improving the detection algorithm in future work, for instance by adding a new filtration stage.