Keywords

1 Introduction

As we have recently entered a rapidly aging society, the number of elderly drivers has increased. Furthermore, according to a 2013 traffic accident statistics analysis [1], there is an increasing trend of accidents between vehicles, and between vehicles and pedestrians. Many techniques are being developed to prevent these types of traffic accidents, but such techniques are mostly aimed at avoiding inter-vehicle collisions, or at warning about lane departures [2, 3]. Safe-driving support technologies for vehicles are continuously being developed [4], and driver awareness is also being improved greatly. Under these conditions, car black boxes are being installed to store information on the surrounding circumstances of a vehicle in real time. The United States has exhibited a positive attitude to passing a bill for mandatory black box installation. In domestic major cities, company taxies are obligated to install black boxes. In public transportation and passenger ships, black box installation is strongly encouraged. The fundamental reason so many black boxes are being installed on means of transportation is that such boxes are used to determine accurately the causes of traffic accidents. In addition, black box installation has the advantage of discounts on car insurance, and of securing legal evidence. Presently, however, black boxes are used for situational assessments post-accident. In fact, black boxes have been categorized as vehicle IT equipment not helpful for safe-driving support. In other words, black boxes are not perceived as devices with built-in pre-processing functions for safe-driving support that can help drivers. Of the five senses, sight is the most used when driving. Thus, drivers operate vehicles using recognition and judgment based on visual information. However, because of the side effects of physical aging, farsightedness and the inability to react quickly are some of the causes of traffic accidents among elderly drivers. As technology aimed at counteracting such side effects, black boxes installed in most domestic vehicles can be more relevant for safe-driving support as a third-party visual information. To summarize, with the installation of image processors in black boxes, it is possible to provide drivers with preprocessed information.

2 Proposed Method

This study proposes a method for detecting the direction indicator region printed on road surfaces by inputting images from vehicle black boxes. Given that small memory, sound power, and real-time processes are required for small black boxes, a relatively simple, but effective, method was designed. The results from analyzing the characteristics of direction indicators marked on road surfaces indicate that corresponding indicators are made from the mono-color white, and they are located on a relatively black background. Thus, road indicators have the characteristic of appearing as a single color in a connected single range. Detecting connected pixels formed in bright mono-color in an input image is required. In order to do this, for the proposed method, the Maximally Stable Extremal Regions (MSER) are detected, and template matching is used to detect the surface indicator area. Figure 1 depicts the flowchart for the proposed method. Using video sequence input, the change in brightness of the road surface area is analyzed in each frame to determine whether the surface indicator is included. In addition, after undergoing pre-processing, such as noise removal and emphasizing pixel brightness in pertinent frames, a binary image is output using the MSER detection process. Subsequently, areas adjacent to the ROI region borderlines are removed through post-processing, and small areas are removed through connected component analysis. After matching the final binary rectangular areas with the templates of indicators marked on road surfaces, the most similar indicator area of the road surface is detected.

Fig. 1
figure 1

Flowchart for proposed method

Full size image

2.1 Histogram Analyzing

For the area that excludes the upper and lower section of an input image, change in pixel brightness at a specific area (240 × 810 pixel) is used to determine whether the indicator region is included in the pertinent frame. If various methods are used for pre-processing, better performance could be obtained, but difficulties might arise because of running time complexity. However, there is a relatively simple method: based on road surfaces with surface indicators as the background, intensity distribution can be used with the most differentiable colors. When brightness and histogram change exceed a given threshold, further processing is performed. If the brightness histogram change rate and average surface floor brightness change is greater than 30 %, we consider information similar to the road surface indicator to be included in the corresponding frame. The road surface indicator candidate ROI region is detected in the input image (Fig. 2).

Fig. 2
figure 2

Input and candidate ROI image

Full size image

2.2 Multilevel Thresholding

The most basic method for detecting the indicator area from the road surface indicator candidate ROI is binarization through thresholding. However, because of various changes in road environments, using the Otus adaptive thresholding method [5] could cause problems of image over-segmentation. Thus, for the proposed method, multi-thresholding is used to produce a binary image that can differentiate the road surface from the indicator area. Figure 3 is the intensity histogram of the candidate ROI. The dotted line is the threshold value 85 calculated using adaptive thresholding, and the solid lines are calculated two-level thresholding: the lower limit is 78 and the upper limit is 141. In the proposed method, instead of using adaptive thresholding, the two-level thresholding method is applied, and the upper limit is used as the threshold value for binarization. The results of our experiment indicate that, for road images, changes in the threshold values are required based on surrounding illumination. Thus, during daytime, road surface brightness is high overall because of sunlight reflection. In cloudy weather, road surface brightness is mostly low. Therefore, for the proposed method, the input frame average overall brightness is calculated, and level one and two threshold values are used selectively.

Fig. 3
figure 3

Intensity histogram of candidate ROI

Full size image

2.3 MSER Detection

For the MSER method, pixels with similar color information form groups based on color information differentiated from adjacent pixels, and each group region that satisfies the critical condition is detected [6]. At this stage, the size of the group region is set to minimum 300 and maximum 7000; the degree of change in brightness is selected at 0.6; and the change in threshold is selected at 2. Figure 4 is the result of detecting MSER in the road surface indicator candidate ROI image of Fig. 2, and each region is expressed in a different color. The brightness change in the specific MSER has a value between zero and one. A lower value indicates a pixel region with lower brightness change.

Fig. 4
figure 4

MSER detection results

Full size image

2.4 Post-processing and Image Dilation

Through post-processing, the detected multi-level threshold common region and MSER binary images are selected. From the selected common binary images, the indicator candidate regions are detected through the connected component analysis of each pixel. In addition, using pixel connected component analysis, areas with fewer than 100 pixels and where a traffic lane extends over the ROI region boundary line are removed. Moreover, the connection component of regions labeled as single character regions using morphology, and of regions that are discontinued because of brightness change, is reinforced. The morphology operation uses an arithmetic operation for lines with 15 pixels and 90-degree rotation. Binary images and connected areas derived from the morphology operation are shown in Fig. 5d. Figure 5 shows the post-processing results and the results of detecting the candidate region from the road surface indicator through the image dilation stage.

Fig. 5
figure 5

Results of post-processing and image dilation. a MSER detection. b Overlap region of (a) and thresholding image. c Labeling result. d Labeling result after image dilation. e Detected candidate regions

Full size image

2.5 Template Matching

For the final road surface indicator detection, the template with the Euclidean minimum distance is detected as the corresponding indicator using template matching for each indicator. Samples of road surface direction indicators are displayed in Fig. 6a, and the average image of the direction indicators is shown in Fig. 6b. For the indicator-matching template, the image in Fig. 6b is used to identify the region with the minimum Euclidean value as the corresponding indicator region.

Fig. 6
figure 6

Sample images for template matching. a Sample images of road surface indicators. b Average images of each road surface indicators

Full size image

3 Results and Future Works

In order to test the accuracy of the proposed method, the results of manually detecting the road surface indicators were compared to those obtained from the proposed method. From a total of 270 experiment images, 83 % showed detection accuracy, and the average processing time per frame was approximately 0.342 s. MSER detection and the template matching process required the longest processing time; for the remaining image processing, only the connected pixel areas in the 240 × 810-pixel binary images were processed, thus minimizing processing time. The results indicate that most detection errors occur because of low brightness levels or road surface contamination. In addition, there were cases where the characters marked on the road surface were misrecognized as direction indicators. By detecting the most appropriate indicator region through template matching, in the case where two or more direction indicators are the same, only one is recognized. The results of testing the proposed method with daytime road images showed relatively good performance, but in order to apply the method to various road environments (at night or in the rain), a more detailed experiment is required. However, for small memory and minimization of power consumption and heat generation in small car black boxes, research on the most effective processing is required. For research on the recognition of characters that exist on road surfaces, a study will be performed on deduction-based robust detection for environment changes in the future.