Keywords

1 Introduction

Recognition of traffic sign is a categorization problem having several classes of traffic signs. Even though, much work is done on the identification of road signs, but a complete benchmark datasets for identification of traffic signs are less considered. Hence, the comparison of the work on the recognition of traffic sign is a bit difficult. A small number of datasets are available as a benchmark for automatic recognition of traffic signs. Stallkamp et al. have contributed a German traffic sign dataset which has more than thirty thousand images of German traffic signs in forty three classes for public accessibility [1]. Larsson and Felsberg have contributed a Swedish traffic sign dataset and, have nearly twenty thousand frames with 20% being labeled [2]. Katholieke Universiteit of Leuven (KUL) Belgium Traffic Signs Dataset (BelgiumTS) is a dataset with 10000 + traffic signs. The material is captured in Belgium in urban environments [3]. Ruta et al. have focused on the use of discriminative feature selection algorithms for identification and tracking of traffic signs [4]. Mogelmose et al. have described various approaches used in the traffic sign recognition, especially in various stages like segmentation, feature extraction, and sign detection and, also explained the deficiency in the use of publicly available image datasets [5]. Recognition of traffic sign based on Eigen values was proposed by Fleyeh and Davami and was tested on database of traffic sign’s borders and speed limit pictograms [6]. Zhu et al. have created Tsinghua-Tencent 100 K benchmark dataset of road sign and database has 100000 Tencent Street View panoramas containing 30000 traffic-sign instances [7]. Yang et al. focused on the identification of traffic signs based on color probability model with convolutional neural network and also built a Chinese Traffic Sign Dataset (CTSD) with 1100 images [8].

Soilan et al. developed a publicly available Spanish traffic sign dataset. The dataset has approximately one thousand five hundred images with fifty to hundred classes of Spanish traffic signs [9]. The ensemble of bag-of visual- phrases and hierarchical deep models to recognize the traffic signs was discussed in the work [10]. Detection of traffic signs based on cascaded convolutional neural networks (CNN) was presented in the paper [11]. Detection of location of traffic signs based on hybrid region model and classification based on Fast R-CNN was presented in the work [12]. The use of off-line detector, online detector, and motion model predictor for simultaneously detecting and tracking road signs and a scale-based intra-frame fusion method for classification of the traffic signs was discussed in the work [13]. Recognition of traffic signs based on the use of kernel-based extreme learning machine with deep perceptual features was described in the paper [14]. The use of fuzzy rule based color segmentation method for the detection of traffic signs followed by the recognition of the traffic signs based on histogram oriented gradient features and speeded up robust features with artificial neural network was presented in the work [15]. Recognition of traffic signs using support vector machine with band of features including local binary patterns, histogram of oriented gradients, and Gabor features was described in the work [16]. Use of random forest and support vector machine classifiers with the feature descriptor obtained by the ensemble of histogram of oriented gradients features and local self-similarity feature to detect and recognize the traffic signs was focused in the work [17]. Use of convolutional neural network to detect and recognize the traffic sign was presented in the work [18]. Recognition of traffic sign based on extreme learning machine with histogram of oriented gradient variant feature was presented in the work [19]. Use of linear support vector machine to detect a traffic sign in a real time is described in the work [20]. Detection and recognition of traffic signs based on three dimensional point cloud models was described in the work [21].

But, not much work is done on the Indian traffic sign recognition benchmark. Hence, an Indian traffic sign recognition benchmark considering both rural and urban situations is presented in the work. The dataset consists of 13000 traffic sign images of 50 different classes of traffic signs taken at different times under different environmental conditions and at different places. Classification accuracy of k-nearest neighbor and support vector machine classifiers by extracting scale invariant feature transform followed by principal component analysis features is computed on the database. The proposed work also discusses the recognition of the unknown traffic sign using SVM with DSIFT.

2 Indian Traffic Sign Database

The Indian traffic sign database (INDTRDB) [22] is primarily designed as a benchmark for traffic sign recognition research keeping in view the systems for assisting the driver. The INDTRDB consists of 13000 traffic signs images of mandatory, cautionary and informatory signs taken under various conditions, such as during daytime, afternoon, night, and ideal images, partially occluded and distorted images. Out of 50 different classes of traffic sign images in a database, 22 images are of mandatory, 21 of cautionary and 07 of informatory sign images. The detailed annotation of the INDTRDB in terms of size, type, orientation, illumination and occlusion is given with a database. The design of the database consists of the following steps:

  1. 1.

    Data collection

  2. 2.

    Selection and cropping of traffic signs

  3. 3.

    Data organization and restructuring

  4. 4.

    Manual annotation.

3 Data Collection

The images of traffic signs are captured using Prosilica GX 1920c camera with automatic exposure control with frame rate of 112fps having resolution of 1936 × 1456. The images are taken at different roads of India considering both urban and rural areas. Figure 1 shows the picture having the traffic sign taken from some roadside.

Fig. 1.
figure 1

Picture containing traffic signs

Full size image

4 Selection and Cropping of Traffic Signs

Traffic signs are designed such that they are easily identified by human drivers. In this work the images containing the traffic signs are captured under various conditions like varying lighting conditions (daytime and night time), different seasons, at different places. After the frames are selected, the traffic signs are manually cropped. The signs are cropped in such a way that the images contain at least five pixels around the sign in order to maintain consistency across the images as the signs may vary with respect to orientations. The images are not necessarily in square and the size varies from 30 × 30 to 160 × 160. The dataset contains nearly 13000 images of 50 different classes. Figure 2 shows few samples of different classes of Indian traffic signs in a dataset. Every sign in the dataset varies with other sign either with respect to size, orientation, illumination, occlusion, background, contrast and quality. Figure 3 shows the sample images of mandatory traffic sign ‘NO PARKING’.

Fig. 2.
figure 2

Different classes of traffic signs in INDTRDB

Full size image
Fig. 3.
figure 3

Sample images of “NOPARKING” traffic signs in INDTRDB database

Full size image

5 Data Organization and Restructuring

The duplicate or similar images were removed from each class. Each image of a class is varied in size from 30 × 30 to 160 × 160. The images are resized in such a way that few images are square images and few are rectangular images. Different instances are taken for each class of traffic signs such as at different places, with different orientations, perfect signs, distorted signs, partially occluded signs. The contrast of each image is varied manually and considered as different instance of that sign. The dataset was split into two subsets with 50% images taken for training and remaining 50% for testing.

6 Manual Annotation

A manual annotation of every image for INDTRDB dataset is provided with the following attributes. The annotation details of the traffic sign “STOP” is shown in Fig. 4. XML description is used to store annotated information.

Fig. 4.
figure 4

XML annotation of STOP traffic sign

Full size image

∙ Size

In the range of 30 × 30 to 160 × 160

∙ Illumination

Good, Bad

∙ Orientation

Frontal, Left, Right

∙ Occlusion

Partial, No

∙ Contrast

Ideal, Dark, Bright

∙ Coordinate

(x, y) coordinate of the top left corner and bottom right corner.

∙ Category

Category of the traffic sign images.

∙ Folder name

The name of the folder which contains the required file

∙ File name

Name of the sign image in the folder

7 Database Complexity Analysis

The Indian traffic sign dataset INDTRDB has a wide range of variations in the traffic signs. Classification accuracy of KNN and SVM classifiers with PCA and DSIFT feature extractor is computed on the database. The experiment is implemented using MATLAB-2012b. The system divides the dataset created into two sets namely test set and training set. Various combinations of feature extractor and classifier algorithms are applied to get the best classification accuracy for the dataset created. The dataset shows good result for DSIFT with SVM. The classification accuracy of KNN and SVM classifiers on the INDTRDB, GTSRB, BelgiumTS database is shown Table 1 for various feature dimensions ranging from 20 to 80 with value of K taken as one. There is no much variation in the recognition rate by further increase of feature dimension. The comparison of INDTRDB dataset with GTSRB and BelgiumTS is shown in the Fig. 5. The result shows that both KNN and SVM methods has shown good classification accuracy on the INDTRDB database as compared to GTSRB and BelgiumTS database. The complexity of the dataset is better understood by the distribution of images in the dataset. For further comparison, a total of 600 images were selected from GTSRB, BelgiumTS, and INDTRDB with 100 images from six different classes and the Eigenvalue spread of the covariance matrix of the preferred signs is analyzed for all three datasets. The magnitude of Eigenvalue indicates the prominent directions along which the data have highest variations and it will be less, if the dataset has a smaller number of variations. Figure 6 shows the Eigenvalue range of GTSRB, BelgiumTS and INDTRDB. It is clear that, number of components needed to cover the space for INDTRDB and GTSRB is large compared to BelgiumTS dataset. This is mainly because of large variations of the signs in the dataset. Figures 7 and 8 shows the related mean signs and Eigenvectors for the same subset of datasets. The mean sign of INDTRDB dataset is more spread compared to BelgiumTS, but not as GTSRB database. The mean sign of BelgiumTS resembles the structure of traffic sign indicating less variation.

Table 1. Classification accuracy of three datasets.
Full size table
Fig. 5.
figure 5

Comparison of INSTRDB, GTSRB and BelgiumTS database

Full size image
Fig. 6.
figure 6

Eigen value spectrum of subset of INDTRDB, GTSRB, and Belgium TS

Full size image
Fig. 7.
figure 7

Eigen vectors (top 10) corresponding to subset of INDTRDB (top), GTSRB (middle), and BelgiumTS (bottom) traffic signs

Full size image
Fig. 8.
figure 8

Mean traffic signs of subset of databases with 6 different traffic signs from (left) INDTRDB, (middle) GTSRB and (right) BelgiumTS

Full size image

8 Recognition of Traffic Sign

In the recognition phase, the system is provided with an unknown test image to identify to which class the traffic sign image belongs. Road signs have a predetermined shape and color with some information and are distinguished by a fixed number of relatively constant colors such as white, red, and blue. To have a clear visibility, signs have enough color contrast with the background. In this work, detection of the road sign from the scene is instigated using color information. The work uses HSV color space to locate the signs from the scene. Hue and Saturation components are used to detect the red and blue colors. The images in the scene having the required colors are further classified based on shapes. Canny algorithm is used to find the required shapes in the image. The classification of the detected image is done using support vector machine with dense scale invariant feature transform feature extractor. After extracting the features, the image is classified using support vector machine, a supervised learning model. The method to recognize unknown traffic sign is given in Algorithm 1. The function takes SVM model, test image, the candidate image that needs to be predicted and will produce output, which is the prediction of the class with the highest probability. The summary of the traffic sign recognition system is shown in Fig. 9. Out of 100 unknown traffic signs 97 signs are correctly classified and 3 are misclassified due to higher occlusion, high distortion, etc. Illustration of four correctly classified and one misclassified traffic sign image is shown in Table 2. The sign is misclassified as “NO ENTRY” instead of “NO PARKING” as both signs have same shape with red color with a cross bar inside the sign. The only information that differentiates these signs is the presence of letter “P” in case of “NO PARKING” whereas an “Arrow Mark” in case of “NO ENTRY”.

Fig. 9.
figure 9

Recognition of traffic sign

Full size image
Table 2. Recognition of unknown traffic sign
Full size table
figure a

9 Conclusion

A new dataset called INDTRDB, an Indian traffic sign dataset is developed to provide a general reference for traffic sign recognition and for the related research area. The INDTRDB dataset has large numbers of images nearly 13000 sign images taken from 50 diverse classes which are taken at diverse illumination and environmental conditions with a detailed manual XML annotation. The classification accuracy of support vector machine on the INDTRDB, GTSRB, and BelgiumTS databases is found to be 98.67%, 93% and 86% respectively. The number of major principal components required to span the space for INDTRDB and GTSRB is large compared to BelgiumTS dataset, which shows the INDTRDB and GTSRB dataset has large variations of the signs in the dataset. The work also describes the recognition of the unknown traffic sign using DSIFT and SVM with a recognition accuracy of 98%. Out of 120 unknown traffic sign images 117 signs are correctly classified and, three are misclassified due to higher occlusion, high distortion, etc.