Optimized segmentation and multiscale emphasized feature extraction for traffic sign detection and recognition

Abstract

Traffic sign detection and recognition has been a topic of research for at least the last two decades. Efforts are being made to reliably detect candidate traffic signs in natural uncontrolled environment and to recognize their contents. For detection, a large proportion of relevant literature discusses color based segmentation by either sticking to a predefined color space (e.g., RGB, HSI, YCbCr etc.) or make use of empirically selected subset of eigen space to achieve partially data dependent segmentation. Since, the input RGB data for various color classes and the background is not linearly separable, none of the existing methods guarantee to achieve complete separation among pixels corresponding to traffic signs and the background objects. To tackle this problem, we propose a completely data driven segmentation technique that adaptively selects an optimized color space based on available training data. To recognize the contents of potential traffic signs, we present a hybrid spatio-frequency radial feature extraction technique with an emphasis on the regions containing useful information. We explore the energy compaction property of steerable discrete cosine transform for feature extraction and augment it with well known circular histogram of oriented gradients in a pyramid. Using our proposed method, experiments on (1) German Traffic Sign Detection Benchmark, (2) our self collected dataset and on a (3) hand crafted version of the combination of the two provide competitive performance compared to various latest and state of the art methods by achieving up to 0.978 precision and 0.98 recall values at an expense of only an insignificant additional computational cost. The method also obtained 0.81 precision on traffic signs partially occluded by other objects.

Keywords

traffic sign multiscale feature extraction optimized color space steered DCT pyramidal HOG

1. Introduction

Traffic sign detection and recognition (TSDR) is an active field of research to develop driver assistance systems with the objective of automatically detecting and reading various types of traffic signs under varying environmental conditions [1]. Another purpose of TSDR is inventory management [2] where traffic signs are extracted from videos captured during a field survey and are labeled according to their physical condition.

Both the subproblems of TSDR i.e., detection and recognition are challenging in real world uncontrolled situations due to problems like varying illumination, partial occlusions, and signs deterioration. [1 , 4].

Some methods take the task of detection by segmenting traffic signs on the basis of color because traffic signs, most of the times, possess high contrast edges which make them easily distinguishable in varying background conditions [5]. Color based segmentation is quite common and many initial systems used thresholding in various color spaces. The most common being HSI, because the hue component, an indicator of color, is invariant to illumination changes. Methods relying on statistical distribution of colors of interest in a selected color space constitute another set of segmentation techniques [6 –8].

For recognition, the first step is to extract features of the pictogram containing traffic sign. This is a one dimensional representation of the shape expected to be invariant to noise, illumination, and rotation etc. Various spatial and spectral methods are in use for this purpose. To extract features in spatial domain, the most common approach is to divide the image in various overlapping or non overlapping blocks and features are extracted in each block. The overall image descriptor is then obtained by concatenating all local features. An extension of block based approach is multiscale feature extraction [9]. In this technique, the process of dividing the input image in blocks is repeated at various spatial scales and finally the feature vectors computed at each level are concatenated to obtain the full image descriptor. This method is useful, especially, when some objects in a scene are small, others are large and the description of all the objects is required to be present in the final image descriptor [10]. The shape descriptor thus obtained is more detailed but the technique suffers due to (1) high computational cost and (2) irrelevant and redundant data. Shape descriptors, obtained by concatenating features computed at different spatial/spectral levels, are generally high dimensional [11] and hence training and testing models is time consuming. In literature, we find Principal component analysis and linear discriminant analysis [12] used to reduce dimensionality of descriptors.

We, in this paper, present a novel automatic traffic sign detection and recognition approach. To detect a potential traffic sign, we map to an optimized color space that ensures maximum statistical discrimination among traffic signs and the background while the data is not linearly separable. Super class is obtained by training a neural network using extreme learning machine [13]. Features from the potential traffic signs are computed using a hybrid spatio-frequency technique. This multilevel feature extraction approach is also effective to recognize partially occluded traffic signs with reasonably high accuracy. In an effort to enhance performance of the classifier, dimensionality of the image descriptor is reduced with the help of feature interaction among data features and the classes [14]. Finally, an SVM classifier [15, 16] is trained on the reduced subset of training data to classify test images.

Our contribution is summarized below:

We propose a fully data driven novel traffic sign detection method by working in a custom color space keeping in mind the non linearly separable nature of training data.

To recognize contents of traffic signs, we propose a hybrid radial multiscale feature extraction method. Features are extracted by working intelligently at multiple scales and by giving importance to the most informative portion of the pictogram.

To get rid of irrelevant and redundant attributes in long hybrid feature vector, we use an effective feature selection strategy based on feature interaction.

In addition to a benchmark dataset, the proposed traffic sign detection and recognition algorithm is tested on a self collected dataset obtained from the longest national highway in Pakistan (N5 highway) represented with deteriorations typical to the developing world.

The rest of the paper is organized as follows: Section 2 presents state of the art and recent work related to automatic detection and recognition of traffic signs (TSDR). Section 3 demonstrates, in detail, our proposed TSDR method along with suitable examples. Section 4 gives an overview of the datasets, explains experimental settings and the results compared with the state of the art methods. Section 5 discusses some important implementation details and finally, Section 6 presents conclusions and possible future directions.

2. Related work

A lot of early methods for traffic sign detection utilize color thresholding based segmentation in various color spaces e.g. RGB, HSI, and CIELab etc [2 , 18]. Fleyeh and Davami [19] performed segmentation using HSI color space by putting their mainstay on Hue, an illumination invariant, channel. Ellayani et al. [17] used thresholding in all three channels of HSI color space to segment traffic signs in complex natural scenes. Yang et al. [7] presented segmentation on the basis of maximum posterior probability of gray levels in color probability map of input RGB image. Likelihood and prior are pre-calculated on the training samples selected from the given database and the method was tested on GTSDB [20] dataset. A similar work by Tsai et al [6] reports an eigen analysis based segmentation of colors and found Ohta transform [7, 21] to be the best. Kim in [8] applied the idea of eigen color space presented by Ohta [21] for segmentation in HSI and YCbCr color spaces. Saliency is another technique used to segment traffic sign in real world images. Wang et al. [22] present a HOG based cascaded feature extraction on color saliency of traffic signs. Their proposed system works in multiple stages. In the first stage, a simple to train, linear SVM classifier rejects the windows of the image which do not contain a traffic sign, then in the preceding stages, more complex versions (with more complex kernel choices) are used to select only those windows which definitely contain a traffic sign in a coarse to fine manner. Barnas et al. present a radial symmetry based traffic sign segmentation strategy that doesn’t rely on colors [23]. This is useful when the sign is deteriorated and the colors are wiped away due to severe environmental conditions. Greenhalg and Mirmehdi [11] proposed to use maximally stable extremal region (MSER) on normalized red/ blue image transformed from RGB domain. This color and shape based technique looks for the most stable shapes under varying thresholds [24]. An achromatic approach to localize traffic signs is given by [9] and the superclass is identified on the basis of pyramidal feature extraction and Histogram of Oriented Gradients (HOG) [25] is computed at different spatial scales.

Extracting invariant features from the blobs containing candidate traffic signs is a very important step [26]. Ideally, these features are signature of shapes and remain unaffected of variations in illumination, partial occlusion and deterioration of traffic signs [1]. HOG [25] and its variants are by far the most widely used feature extraction techniques. Initially devised for pedestrian detection in images, Greenhalgh and Mirmehdi [11] first used this technique to recognize contents of traffic signs. Since then many variants of the technique e.g. HOG on HSI [17], signed and unsigned HOG [27] and HOG on occlusion maps have been proposed [28]. Bascon et al. [29] presented a distance to border technique to generate feature vectors from various shapes e.g. circular, triangular and hexagonal containing traffic signs. Recently Liu et al [30] presented a solution for occluded traffic signs using local binary pattern (LBP) based color cubic features. The method is based on automatic color thresholding on an integral image in various color spaces and was tested on a Chinese traffic signs database. A few efforts have been made to reduce the dimensionality of feature vector in order to feed only the most relevant and least redundant features to the classifier [12, 31].

Recently, authors in [12] have proposed extracting features on the basis of log polar transforms [32] claiming that the descriptors thus computed are invariant to scale and rotation. Authors in [33] have proposed a long descriptor to extract features of German traffic sign detection benchmark on the basis of bispectrum and gray level co-occurrence matrix [24]. Another recent technique inspired by biomedical analysis of images in conjunction with extreme learning machine was proposed by [31].

Detection and recognition of traffic signs with partial occlusions has rarely been touched in literature. Authors in [28] presented a technique based on HOG, LBP and SVM to detect traffic sign in complex outdoor scenes. Yawar et al. proposed a technique [34] to train a classifier only on those portions of a traffic sign that are unlikely to be occluded and called them the discriminative patches. Another recent technique [30] discussed detection of occluded traffic signs using color cubic LBP features and a cascade of tree classifiers. There is, however, a lot of gap in the field of traffic sign detection and recognition in the presence of partial occlusions.

3. Our proposed method

f (x, y) = [f₁ (x, y) f₂ (x, y) f₃ (x, y)] ^T is a tuple taken from an RGB image at coordinates (x, y) whereas f₁ (x, y), f₂ (x, y) and f₃ (x, y) are scalar values of red, green and blue channels respectively. g (x, y) is the corresponding tuple of channel values in the transformed color space at coordinates (x, y) obtained by executing a linear transformation g (x, y) = T [f (x, y)] = A × f (x, y). The transformation matrix A is of size 3 × 3 whose 9 elements (a₁₁ . . . a₃₃) are computed based on the available data. Given an RGB input, by adjusting different values of the elements of matrix A, one can map to another color spaces as shown in Table 1. By putting A equal to the identity matrix, we move nowhere but by setting any other chosen values of its elements, we can go anywhere (the last row in Table 1).

Table 1
Mapping on different color spaces using linear transformation

Contents of Matrix A Resulting Color Space

$[\begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}]$ RGB

$[\begin{matrix} \frac{1}{f_{1} + f_{2} + f_{3}} & 0 & 0 \\ 0 & \frac{1}{f_{1} + f_{2} + f_{3}} & 0 \\ 0 & 0 & \frac{1}{f_{1} + f_{2} + f_{3}} \end{matrix}]$ normalized RGB

$[\begin{matrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ 0.5 & 0 & - 2 \\ - .25 & .5 & .25 \end{matrix}]$ 3-channels Ohta

$[\begin{matrix} .299 & .587 & .114 \\ .5 & - .23 & - .27 \\ .202 & - .5 & .298 \end{matrix}]$ YIQ

$[\begin{matrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{matrix}]$ Any Values

Contents of Matrix A	Resulting Color Space
$[\begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}]$	RGB
$[\begin{matrix} \frac{1}{f_{1} + f_{2} + f_{3}} & 0 & 0 \\ 0 & \frac{1}{f_{1} + f_{2} + f_{3}} & 0 \\ 0 & 0 & \frac{1}{f_{1} + f_{2} + f_{3}} \end{matrix}]$	normalized RGB
$[\begin{matrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ 0.5 & 0 & - 2 \\ - .25 & .5 & .25 \end{matrix}]$	3-channels Ohta
$[\begin{matrix} .299 & .587 & .114 \\ .5 & - .23 & - .27 \\ .202 & - .5 & .298 \end{matrix}]$	YIQ
$[\begin{matrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{matrix}]$	Any Values

3.1. Detection

In this step, candidate traffic sign is segmented in a natural scene. For this purpose, as mentioned in Section 2, all previous work uses one or more of many well known color spaces either adopted experimentally or by mapping data on empirically chosen subset of eigen space of color data [6 –8]. Keeping in view that the color data is not linearly separable, this work proposes an optimum solution that finds a global maximum in transformed color space. This ensures maximum separation among different color classes by keeping spread of each class as concise as possible.

As mentioned above, given an RGB input, a pixel value in transformed domain is computed with the help of a linear transformation. An illustration of the concept is also given in Table 1 and it can be seen that the parameters of this transformation are different for different color spaces. In our proposed work, maximum discrimination among classes of different colors is achieved by obtaining parameters of the linear transformation from the solution of an optimization problem. In this way, we guarantee to have the optimum discrimination in a given situation. A conceptual description of the process is shown in Fig. 1 in which pixels belonging to different color classes in input RGB color space are mapped to another color space such that their inter centroid distance is increased by keeping the variance of each class as small as possible. The three clouds correspond to two traffic sign colors (i.e., red and blue) and background colors. A famous technique, linear discriminant analysis (LDA), reduces within class variance and increases between class variance but the data is mapped to a hyperplane.

On the other hand, we quickly solve an optimization problem with the help of a set of heuristics to map to another color space (not necessarily a hyperplane) and hence ensuring least loss of information. Since the data for various color classes is not linearly separable, it may not appear absolutely sparse in the transformed color space but is guaranteed to have the optimum discrimination with the available training stuff. It should be noted that with the help of a non linear transformation a better separation on training data can be achieved but that may lead to over fitting [35]. Authors in [6] and [7] adopted the work of Ohta [21] to achieve data dependent segmentation by mapping data to eigen space and chose two empirically best principal components whereas our proposed approach is linear and completely data driven.

Fig.1

An illustration of discrimination achieved using linear mapping.

Major steps in the detection process are summarized in Fig. 2. Training data comprises of RGB information of color pixels corresponding to traffic signs from training images. These values are collected manually and are fed to a parameter estimator module that computes elements of matrix A, mean (μ) vectors and covariance matrices ( Σ ) of colors corresponding to traffic signs in transformed color space. Once the model is trained, a test image is fed to obtain a processed image in which pixels corresponding to traffic sign take on higher gray values than the rest. This image is then subject to repeated thresholding method [11] whose output is a black and white image with pixels corresponding to the traffic sign white and the everything else appears black.

Fig.2

Block diagram of our proposed detection procedure.

The optimization problem to be solved is given in Equation 1. It is obvious that the Euclidean distance is large for large inter class distance and small variance of each color class. In order to obtain the maximum discrimination, the expression is to be maximized for every two color classes namely C_i and C_j. E_g (.) is a 3 × 1 vector representing expected value or center of a class in transformed color space whereas Σ _g (.) is the 3 × 3 covariance matrix. The subscript g represents image in transformed color space. We want to maximize Euclidean distance of the difference of ratios mentioned in Equation 1 between every two classes such that each pixel in the transformed color space takes value between 0 and 255. Since g (x, y) is a tuple containing three numbers for three different values of z, g (x, y, z) represents scalar value of a pixel in z plane. Moreover, there are three planes in both input and transformed color space which means the range of z is {0, 1, 2}. $maximize | | \frac{E_{g} (C_{i})}{tr (Σ_{g} (C_{i}))} - \frac{E_{g} (C_{j})}{tr (Σ_{g} (C_{j}))} | | \forall i \neq j$ (1) $\begin{matrix} s . t . g (x, y) = A \times f (x, y) \forall x, y \\ and g (x, y, z) \in {0 . . . 255} for z = {0, 1, 2} \end{matrix}$

Theorem 1. Assuming equal covariance for two non linearly separable color classes i and j, there exists a transformation matrix $\hat{A}$ that maximizes the expression given in Equation 1.

Proof. Expected value of class i in transformed color space is written in terms of probability distributions of transformed vector g (x, y) and the original vector f (x, y) in Equation 2. Assuming same covariance matrices for any two arbitrarily chosen classes, the difference mentioned in Equation 1 can be rewritten in terms of matrix $\hat{A}$ and the difference of expected values i.e., d_f (i, j) in input RGB color space as mentioned in Equation 4. This shows that there exists a non zero matrix $\hat{A}$ that maximizes the difference between centroids of any two color classes in transformed color space. $\begin{matrix} E_{g} (C_{i}) & = \sum_{x \in C_{i}} \sum_{y \in C_{i}} g (x, y) p (x, y) \\ = \sum_{x \in C_{i}} \sum_{y \in C_{i}} A \times f (x, y) p (x, y) \end{matrix}$ (2) $\sum_{x \in C_{i}} \sum_{y \in C_{i}} A \times f (x, y) p (x, y) = A \times E_{f} (C_{i})$ (3) $\begin{matrix} \max_{\forall i \neq j} & (| | E_{g} (C_{i}) - E_{g} (C_{j}) | |) \\ = \hat{A} \times | | E_{f} (C_{i}) - E_{f} (C_{j}) | | \\ = \hat{A} \times d_{f} (i, j) \end{matrix}$ (4) ■

Elements of matrix A that maximize the distance measure given in Equation 1 along with the constraints are to be selected as part of training the model. The presented problem is non convex and contains a lot of local maxima. We find the eigen space of the training data to be a good starting point and from there randomly jump to empirically chosen random locations in neighboring three dimensional space, $R^{3}$ . Local search based on gradient ascent is performed at each of these places and the maximum point is stored. The process of jumping randomly to neighboring points is repeated several times in an effort to reach to the global maximum. As mentioned earlier, since the proposed heuristic solution to this problem involves searching three dimensional space, for a given set of training data the solution can be non-exact but remains close to the desired three dimensional point g (x, y) in the transformed color space.

Let $G^{i} = {G_{1}^{i}, G_{2}^{i}, . . ., G_{M}^{i}}$ be the training set corresponding to color i. M shows the total number of images and the set of samples taken from m^th training image belonging to class i is $G_{m}^{i}$ where the matrix $G_{m}^{i} = {g_{m, 1}^{i}, g_{m, 2}^{i}, . . ., g_{m, N}^{i}}$ . Pixels taken from different images but belonging to the same class can be conveniently thought to be independent from each other whereas pixels from the same image can be correlated. To compute the maximum likelihood estimates of parameters, we assume, for simplicity, all components independent of each other. The situation is described using Fig. 3. Based on the above analysis, the maximum likelihood estimates of mean and covariance of each color class are calculated as given by Equations 5 and 6. In these Equations, $g_{m, n}^{i}$ represents n^th sample from m^th training image belonging to class i.

Fig.3

Collecting samples of a given color class from images in training dataset.

$μ_{i} = \frac{\sum_{m = 1}^{M} \sum_{n = 1}^{N} g_{m, n}^{i}}{MN}$ (5)

$Σ_{i} = \frac{\sum_{m = 1}^{M} \sum_{n = 1}^{N} (g_{m, n}^{i} - μ_{i}) (g_{m, n}^{i} - μ_{i})^{T}}{MN}$ (6)

Now, as mentioned in Equation 7, Bayes rule [35] can be used to compute the posterior probability of class C_i given g (x, y) i.e., a tuple in the transformed color space. The likelihood P (g (x, y) |C_i) is computed using $N (g (x, y); μ_{iitsc}, Σ_{iitsc})$ assuming a Gaussian distribution. The prior probability, however, is computed on the basis of proportional presence of each class in the training data.

$P (C_{i} | g (x, y) = \frac{P (g (x, y) | C_{i}) P (C_{i})}{P (g (x, y))}$ (7)

Applying Equation 7 on a test image generates a grayscale image with colors belonging to potential traffic signs represented with grayvalues higher than the background. The denominator of Equation 7 is neglected because it remains the same for all classes. The resultant suppressed background image is a good candidate to extract sign of interest. Maximally stable blobs [11] are then retained and are then fed to the superclass recognition step. Fig. 4 shows the processed images obtained as a result of various steps in the detection procedure. Fig. 4(a) represents an input image from GTSDB dataset [20]. Fig. 4(b) shows the traffic sign enhanced image computed using Equation 7 and Fig. 4(c) represents its intensity surface plot. It can be seen that in the surface plot the pixels corresponding to the traffic sign take significantly higher grayvalues than the background pixels which makes segmentation task easy. Fig. 4(d) shows the segmented image as a result of the search for maximally stable blobs.

Fig.4

(a) A sample image taken from GTSDB database. (b) Image corresponding to posterior probability as a result of applying Equation (7) after some post-processing. (c) Surface map of the image in b showing that pixels corresponding to the traffic sign take comparatively larger gray values. (d) Result of applying repeated thresholding and keeping track of the most stable shapes.

3.2. Recognition

This procedure includes identifying superclass of the blob taken from the output of segmentation process followed by recognizing its contents. Fig. 5 presents the internal blocks related to both above mentioned sub-procedures. The following subsections provide details of all sub-blocks. Recognizing super class i.e. circular, triangular etc. is the first recognition related step. A number of techniques have been reported in the literature. Block diagram of our proposed technique is shown in Fig. 6. A low resolution representation of the input blob is computed and is fed to an artificial neural network to detect the superclass i.e., triangle, inverted triangle or circle. Low resolution models of the shape used in literature make use of interpolation that incurs loss of information. But in this work, we make use of energy compaction property of discrete cosine transform (DCT) to ensure least information loss. To the best of authors’ knowledge, his is the first method to use discrete cosine transform for this purpose. The low resolution version, in our work, is obtained by applying discrete cosine transform on a rectangular grid [36 –38] and retaining only a few low frequency coefficients by scanning each cell in famous zig zag fashion [24] used for data compression.

Fig.5

Block diagram of our proposed recognition method.

Fig.6

Block diagram of super class identification procedure using low resolution representation of segmented blobs processed using artificial neural network trained using extreme learning machine.

Equation 8 mentions how discrete cosine transform (DCT) is computed on blob B of size X × Y. Image containing the whole shape is partitioned in grid_X × grid_Y cells and the transformation mentioned in Equation 8 is applied on every cell and the transformed image has range of coordinates 0 ≤ u ≤ X - 1, 0 ≤ v ≤ Y - 1. Finally, as mentioned in Equation 9, these features are concatenated and fed to the ANN classifier. $\begin{matrix} b (u, v) & = T_{dct} [B] = α_{u} α_{v} \sum_{x = 0}^{X - 1} \sum_{y = 0}^{Y - 1} b (x, y) \\ \cos \frac{π (2 x + 1) u}{2 X} \cos \frac{π (2 y + 1) v}{2 Y} \end{matrix}$ (8)

$d_{s} = {cat}_{grid} {T_{dct} [B]}$ (9)

Artificial neural networks are good at classifying low resolution digits. To classify the shapes of the traffic signs (i.e., super class), we use a network with one hidden layer and trained using extreme learning machine (ELM) technique [13] with RBF kernel. ELM is a smart technique in which the input weights are randomly assigned and the output weights are computed analytically without using a gradient descent algorithm [13]. The architecture of the network is shown in Fig. 6, number of input neurons is equal to the size of shape descriptor obtained as a result of the operation mentioned in Equation 9, number of hidden layer neurons are arbitrarily chosen and the output neurons are set equal to three. The blob declared as being a circle, triangle or inverted triangle with a certain minimum probability is passed to the next stage, otherwise it is considered as a background object and is neglected.

To extract contents, we propose a hybrid spatial and frequency domain descriptor based on circular HOG (CHOG) and steered DCT (SDCT). HOG is a famous shape descriptor presented by [25] for pedestrian detection initially and then was widely used in computer vision and biomedical image analysis [11]. HOG at multiple rectangular spatial scales was proposed by [3] and was applied by [4] for traffic signs superclass identification i.e. triangular or circular etc. In our work, CHOG is applied after dividing an image in various blocks each in a certain range of radius and angle assuming center of the image as origin. As shown in Fig. 7, we propose a unique strategy of applying pyramidal features in a grid. The radial distance from the center r and the angle φ are two variables, we first keep ranges of radial distance fixed and increase the sizes of blocks by increasing ranges of angles only, then we do vice versa and finally increase both. In our work, we tried five bins for r and eight bins for φ. Feature extraction algorithm is applied on each of these schemes and the feature vectors are concatenated.

Fig.7

An explanation of our proposed feature extraction approach, (a) Starting with an initial setting of r and φ (b) r is kept constant and no. of φ bins are increased (c) φ bins are constant whereas r bins are increased and finally (d) both bins are increased. The descriptors, computed at each level are finally concatenated.

Second part of the feature vector is computed by exploiting famous energy compact property of discrete cosine transform. This transform has been largely used for data compression in addition to its use as a feature extraction technique in various biomedical image processing applications [36, 38]. Numerical formula to compute steered discrete cosine transform on a two dimensional sign image S is given in Equation 10. The method is inspired by Gabor filters [39] and was used to obtain angular features at different orientations using DCT in a grid. Gabor filters have recently been used to recognize contents of traffic signs in conjunction with sparse learning [40]. In [41], the DCT based features are obtained by slightly modifying the basis function of the expression given in Equation 8 for the purpose of coding/compression. This is obtained by replacing x and y with their oriented versions as mentioned in Equation 10. We thus obtain features by putting orientation angles 0, π/8, π/4, 3π/8, π/2, 5π/8, 3π/4, 7π/8. To construct the SDCT feature descriptor, all oriented component features are finally concatenated. $\begin{matrix} s (u, v) & = SDCT [s (x, y)] = α_{u} α_{v} \sum_{x = 0}^{X - 1} \sum_{y = 0}^{Y - 1} s (x, y) \end{matrix}$

$\begin{matrix} \cos \frac{π (2 xcos θ + 2 ysin θ + 1) u}{X} \\ \cos \frac{π (- 2 xsin θ + 2 ycos θ + 1) v}{Y} \end{matrix}$ (10)

Feature vectors obtained as a result of applying SDCT and CHOG are finally concatenated in an interleaved style to form a hybrid multiscale circular feature descriptor as shown in Fig. 8. Applying SDCT and CHOG at multiple scales facilitates exploiting various features. Equation 11 describes that the shape descriptor d is obtained by concatenating features computed using SDCT and CHOG on image S. $d = {cat}_{multiscale} {SDCT [S], CHOG [S]}$ (11)

Fig.8

Graphical representation of how a shape descriptor is obtained as a result of concatenation performed at various spatial scales.

Since the portion outside the border containing the traffic sign is least important, therefore, the final shape descriptor (d_f) is obtained by applying a distance function to the core shape descriptor (d) computed using Equation 11. This emphasized descriptor, given in Equation 12, computes important distance radially outward from the center of the shape. As the magnitude of r increases, importance of the descriptor should decrease, therefore, we opted for a reversed and space shifted version of the sigmoid function sig (r, A, C) given by (1 + e^-A×(r-C)) ^-1. Here C is shifting and A is scaling parameter, and d is the basic descriptor before applying distance function.. The parameter C is adjusted empirically depending on the average area filled by the contents of the traffic sign. $d_{f} = T [d, r] = [1 - sig (r, A, C)] \times d$ (12)

An illustration of radial distances from the center of the shape is given in Fig. 9 for different values of shifting parameter C, the scaling parameter A is kept fixed. Center of the shape is considered as the origin; this ensures that background items are at a large radial distance and hence are given less importance. We see that the area around the center to be emphasized is adjusted by chosing an appropriate value of parameter C.

Fig.9

Distance function used to apply weights to the shape descriptor.

The hybrid descriptor obtained as a result of spatio-frequency feature extraction is detailed and contains both irrelevant and redundant information in addition to useful data. To obtain a subset of the most relevant and least redundant features, we use feature interaction based feature selection [14, 42]. To the best of our knowledge, this technique has never been used for traffic sign recognition applications before. The feature selection algorithm proceeds as follows:

Class relevance computed between a feature f and class c using mutual information is represented as I (f ; c). The process is repeated for all features in the dataset f and the features are then arranged in descending order of their class relevance.

Top ranked feature is transferred from f to the subset of selected features S without any comparison.

Feature with the next highest class relevance (f⁺) is temporarily added to the subset of selected features S and mutual information between this new subset {f⁺, S} and the class c is computed i.e., I (f⁺, S ; c).

Step 3 is repeated for all remaining features in f and the feature with the highest value of I (f⁺, S ; c) is added to the subset S permanently.

In order to add Z number of most relevant and least redundant features to the subset S, the process in steps 3 and 4 is repeated Z times.

An expression for exact computation of joint mutual information I (f⁺, S ; c) between candidate feature (f⁺) temporarily added to the already selected subset of features (S) and the target class (c) is given byI (f⁺, f₁, f₂, f₃ . . . f_S ; c). The subset (S) is now written as individual terms (f₁, f₂, f₃,. . . , f_S). Exact computation of this expression is expensive due to large joint probability terms. We, therefore, need to obtain an approximation to get the computations done quickly e.g. to take features from the selected subset one at a time to compute mutual information and then summing the results [43] i.e., ∑_{f_s∈S}I (f⁺, f_s ; c). In order to be more exact, we can take more than one features from the subset S at a time but that will make the computations more expensive [44].

Each of these member terms can be written in terms of joint entropy between f⁺, f_s and class c and marginal entropy of the class as given by Equation 13. $I (f^{+}, f_{s}; c) = H (f^{+}, f_{s}) + H (c) - H (f^{+}, f_{s}, c)$ (13)

It is noteworthy that since H (c), the entropy of class label, remains the same throughout the feature selection procedure, therefore it can be dropped and the remaining terms on the right hand side remain H (f⁺, f_s) - H (f⁺, f_s, c). This approximate formula was used to carry out all experiments.

4. Experiments and results

4.1. Datasets and test bench

German traffic sign detection benchmark (GTSDB) [20] is currently the most widely used dataset for testing and validation of TSDR algorithms [9 , 45]. The dataset contains 900 images from 43 classes of European traffic signs both with red rim and filled with blue color. We used 30 classes for the purpose of our experiments out of which some samples are given in Fig. 10(a).

Fig.10

Samples from GTSDB and our self collected datasets.

As a second dataset, we collected 1500 images corresponding to 17 different classes from N5 national highway in Pakistan [46]. It is an over 1800 km long road having traffic signs presented with a variety of deteriorations. The dataset is available online 1 [ObmZCTHFwZjA?usp=sharing] and a few samples from the dataset are shown in Fig. 10(b).

We randomly picked a number of samples from GTSDB and our self collected datasets and introduced occlusions manually. The percentage of occlusion varies from 5% to 50% and is placed randomly on the right, left, top, bottom or in the center of the board containing the sign. This synthetic dataset contains occluded images of 20 classes presented by 600 images.

Extreme Learning Machine (ELM) classifier was trained on low resolution versions of broad shapes of the targeted traffic signs. Input neurons were set equal to the number of pixels in the image being presented and the number of output neurons equal to the classes. A blob is considered as a background item if the score of all classes are below a certain threshold. To read contents of traffic signs, support vector machine (SVM) classifier was trained on the training data and was tested on images reserved for the purpose. Since SVM is a two class classifier by default, its implementation given in LibSVM [16] was used to carry out all experiments. LibSVM is originally written in C language and is a famous and widely used implementation of multi class SVM classifier in research literature [1 , 26].

4.2. Results

Since our proposed algorithm presents a novel method to detect traffic signs in natural scenes (the parameters were estimated solely on the basis of available training data), its comparison with other detection techniques is interesting. This is provided in Table 2 where the detection portion of our proposed algorithm is compared with four other methods. The first method to compare [3] uses thresholding in HSI color space, the second method [17] also uses HSI thresholding but with different limits in each channel, the third method [11] uses normalized RGB in which each channel in original RGB image is divided by the sum of R, G and B values and the last method [7] uses a variation of Ohta transform [21] obtained by dividing the color channels in Ohta space by the sum of R, G and B values. Table 2 shows that our proposed method achieves the highest detection rate for all three datasets. Normalized Ohta based detection [7] is the next closes technique whereas thresholding in HSI domain with fixed limits is the worst choice.

Table 2
Comparison of detection accuracies on all datasets.

Our Proposed Detection Method HSI thresh. [3] HSI thresh. [17] Norm. RGB [11] Norm. Ohta [7]

GTSDB 0.99 0.88 0.93 0.97 0.95

Self Collected 0.99 0.85 0.88 0.91 0.90

Occluded 0.85 0.69 0.77 0.80 0.80

	Our Proposed Detection Method	HSI thresh. [3]	HSI thresh. [17]	Norm. RGB [11]	Norm. Ohta [7]
GTSDB	0.99	0.88	0.93	0.97	0.95
Self Collected	0.99	0.85	0.88	0.91	0.90
Occluded	0.85	0.69	0.77	0.80	0.80

To carry out experiments for recognition, 30 classes of traffic signs were selected for GTSDB, 17 for our self collected and 20 classes for manually occluded datasets. 70% of the available data in each dataset was used for training and the remaining 30% was reserved for testing.

A comparison of precision and recall for various TSDR methods on GTSDB, our self collected and manually occluded datasets is given in Table 3. The first method to compare is a HOG and SVM method on color enhanced grayscale version of the input image presented in [11]. The method is state of the art and achieves fairly high values of precision and recall. The second method [9] is a pyramidal HOG based technique where HOG is applied in a predefined pyramid after some preprocessing, i.e., applying a suitable edge detection filter. Although this method was originally proposed to detect super class of the traffic sign, we used it for the purpose of identifying the contents. The third method [7] presents a color probability map technique classified with computationally expensive convolution neural network but claiming that their complete system runs faster than many similar methods. The method presents a low cost computational model and achieves a good detection accuracy. The last method in Table 3 proposes using HOG on Hue, Saturation, Intensity (HSI) color space [17]. HOG features are computed for each channel and are then concatenated. Table 3 shows that for GTSDB and our self collected datasets, our proposed method achieves the highest precision and recall values. For manually occluded dataset, it gets the highest recall.

Table 3

Precision and Recall values of all three datasets compared with four other techniques

	GTSDB		Self Collected		Occluded
	Precision	Recall	Precision	Recall	Precision	Recall
Proposed Method	0.98	0.98	0.98	0.94	0.81	0.80
Method in [11]	0.95	0.96	0.9	0.89	0.79	0.70
Method in [9]	0.96	0.97	0.92	0.89	0.82	0.79
Method in [7]	0.95	0.94	0.90	0.89	0.77	0.79
Method in [17]	0.93	0.93	0.88	0.87	0.75	0.76

Table 4 provides precision and recall values for individual signs taken from GTSDB dataset. The table provides comparison of our proposed method with two other techniques presented in [11] and [9]. The comparison includes 8 mandatory speed limit signs and 10 prohibitory signs. Our proposed method exhibits its superiority by achieving the highest precision and recall values for most of the signs. However, the methods given by [11] and [9] achieve the best recall values only for a few signs. Table 5 shows precision and recall values of all 17 types of traffic signs on our self collected dataset. Our proposed method outperforms the other two competing methods for all types of traffic signs.

Table 4

Recognition precision and recall on GTSDB [20] Dataset

Traffic Sign	Proposed Method		Method in [11]		Method in [9]
Traffic Sign	Precision	Recall	Precision	Recall	Precision	Recall
SL 20	0.960	0.982	0.905	0.915	0.913	0.913
SL 30	0.942	0.924	0.895	0.868	0.932	0.994
SL 50	0.980	0.974	0.885	0.976	0.944	0.913
SL 60	0.944	0.988	0.845	0.982	0.935	0.901
SL 70	0.920	0.983	0.905	0.795	0.943	0.991
SL 80	0.972	0.969	0.935	0.815	0.953	0.803
SL 100	0.982	0.949	0.905	0.968	0.953	0.862
SL 120	0.977	0.953	0.944	0.948	0.888	0.813
Right turn	0.924	0.943	0.934	0.943	0.900	0.947
Left turn	0.992	0.993	0.905	0.945	0.955	0.844
U turn	0.962	0.945	0.850	0.987	0.950	0.804
Give way	0.955	0.994	0.920	0.989	0.934	0.984
Heavy vehicles	0.974	0.955	0.944	0.753	0.953	0.954
Double bend	0.992	0.999	0.897	0.899	0.900	0.933
Speed bump	0.993	0.989	0.889	0.789	0.888	0.992
Slippery road	0.982	0.980	0.900	0.773	0.876	0.893
Road works	0.972	0.980	0.890	0.804	0.888	0.879
Traffic lights	0.985	0.979	0.744	0.774	0.903	0.808

Table 5

Recognition precision and recall on self collected real world dataset

Traffic Sign	Proposed Method		Method in [11]		Method in [9]
Traffic Sign	Precision	Recall	Precision	Recall	Precision	Recall
Bridge ahead	0.977	0.945	0.875	0.934	0.945	0.975
Left turn	0.966	0.929	0.886	0.823	0.945	0.907
Pedes. crossing	0.987	0.907	0.886	0.945	0.907	0.916
Right turn	0.987	0.987	0.934	0.794	0.897	0.987
Road narrow	0.966	0.897	0.907	0.835	0.897	0.845
Round about	0.987	0.934	0.954	0.813	0.886	0.954
School	0.987	0.907	0.929	0.934	0.945	0.854
Side road	0.977	0.916	0.934	0.834	0.963	0.987
SL 30	0.966	0.987	0.929	0.756	0.907	0.945
SL 50	0.987	0.963	0.907	0.975	0.934	0.807
SL 70	0.987	0.963	0.945	0.975	0.929	0.807
SL 90	0.987	0.963	0.886	0.975	0.907	0.897
SL 100	0.987	0.963	0.897	0.975	0.934	0.875
Slow	0.987	0.897	0.897	0.929	0.963	0.813
Speed bump	0.987	0.987	0.813	0.794	0.975	0.897
Split road	0.987	0.987	0.875	0.929	0.886	0.907
U turn	0.977	0.934	0.934	0.929	0.907	0.845

Figure 11 shows error rates of various methods plotted versus increasing sizes of subsets of feature vectors on GTSDB dataset. Subsets were computed using approximate joint mutual information maximization technique described in Section 3. The same technique was also applied on HOG+SVM method described in [11]. It can be seen that our proposed method achieves lower error rate with less number of features. Minimum error rate achieved by our proposed method is by using a subset of approximately 1500 featues and is valued around 0.03. The method presented in [11], however, exhibits the least error (around 0.04) using 1764 features of the training data. Fig. 11(b) describes the same comparison on our self collected dataset and our proposed method achieves the least error rate with around 1700 features.

Fig.11

Size of feature vector versus error rates.

On the third dataset, obtained by collecting samples from both GTSDB and our collected dataset, experiments were performed for manually crafted partial occlusions. Fig. 12 shows error rate of three different methods versus these partial occlusions on our third dataset. All three competing methods are claimed to be robust against partial occlusions. Our proposed method was compared with the method presented in [11] and was found better for signs occluded up to 45%. However, for occlusions beyond that the method in [11] performs slightly better. The other method presented by Fleyeh et al. [3] achieved the highest error rate for all tested occlusions.

Fig.12

Error rate of our proposed method in comparison with MSER+HOG+SVM [11] and HSI thresholding+Eigen+SVM [3] plotted versus partial occlusions.

Finally, in order to eliminate the doubt that the improvement in accuracy achieved, by applying our proposed feature selection method on long shape descriptor, was by chance, a statistical test, i.e., paired t-test was performed on all three datasets. Recognition accuracy on test sets of all three datasets before and after applying feature selection algorithm was determined separately. Difference of the recognition accuracies for each test sample was calculated and finally mean m_d and standard deviation s_d of difference array were determined. Test statistic was calculated using the formula $t = \frac{m_{d} - 0}{\frac{s_{d}}{N}}$ with the null hypothesis that there is no difference in the recognition accuracy values before and after applying the feature selection algorithm. Since we are working at 95% confidence level, null hypothesis is rejected if p-value is smaller than 0.05. Results in Table 6 show that the test on feature selection algorithm for all three datasets achieves p-value much smaller than 0.05. Therefore, we can confidently declare that the results are statistically significant.

Table 6

Statistical analysis of feature selection method

Dataset	p-value
GTSDB	0.0001
Self Collected	0.0002
Manually Crafted	0.0001

Finally, Fig. 13 shows examples of traffic signs successfully detected using our proposed technique taken from our self collected dataset. Fig. 13(a) show a bridge ahead warning sign found on N5 highway between two major cities whereas the sign shown in Fig. 13(b) is a an SL100 sign detected on the same highway.

Fig.13

Sample detected signs from our self collected dataset.

5. Discussion

A novel method for automatic detection and recognition of traffic signs was tested on three different datasets and was compared with a number of recent and famous methods. Our proposed algorithm involves a number of steps having high computational cost such as:

Adjusting coefficients of matrix A by solving heuristically an optimization problem

Adjusting maximum likelihood estimate based parameters of color classes of interest

Running joint mutual information based feature selection algorithm on long shape descriptor. The operation is time consuming because it involves a number of steps requiring computing joint probability between two and three variables.

Fortunately, all these steps are executed on training data only once. In other words, tuning of parameters and execution of feature selection algorithm, the time consuming processes, are complete before the first test instance is presented to the system. A comparison of time taken to train our proposed system and other state of the art methods on both GTSDB and our self collected datasets is shown in Fig. 14(a). Results are based on running the proposed system on Pentium 4, core i7 computer with 8GB RAM. It can be seen that the training time of our proposed method is the largest among all. This is because the parameters of our proposed color space were set by undergoing certain iterations. Moreover, since the data belonging to the shape descriptor consists of multivalued integers, applying feature selection algorithm on this data requires computing joint entropy terms. The formula to compute joint mutual information requires computing joint probability terms which are pretty complicated especially when the variables have more than two values [47, 48].

Fig.14

A comparison of execution speeds of various methods for detection and recognition.

A similar comparison of average prediction time taken by a single test instance is provided in Fig. 14(b). Our proposed method is found taking only 8 ms and 10 ms extra time compared to the fastest method i.e., [3] for the self collected and the GTSDB datasets respectively. That is not a great deal, because the proposed method offers at least more than 5% accuracy compared to the fastest method [3] in the competition for both datasets. Further, it is also mentioned in Fig. 14(b) that for the self collected dataset, the proposed method is not the slowest on test data. For both datasets, our proposed method is the most accurate; it achieved 3% higher accuracy by being less than 5ms slower compared to the second most accurate method i.e., [7]. To summarize, the proposed method offers better accuracy on test data by making an insignificant compromise on execution speed.

6. Conclusion

In this paper, we present a novel traffic sign detection and recognition method. Detection is based on segmenting a candidate traffic sign in uncontrolled natural environment. Optimum separation among classes of colors of interest and the background is achieved by quickly solving an optimization problem with the help of set heuristics. The most stable shapes are then accepted as potential traffic sign. Superclass is identified based on testing a pretrained ANN on low resolution versions of the desired shapes.

For recognition, we have proposed multiscale feature extraction, based on spatial and frequency analysis of the templates utilizing multiscale versions of circular HOG and steerable DCT. To reduce the large dimensional shape feature descriptor, joint mutual information based feature selection is proposed to obtain the most representative subset of features. A linear kernel SVM classifier is then trained on training data (70%) and is tested on the with-held instances (30%).

For experiments we used three datasets (1) German traffic sign detection benchmark (GTSDB) (2) our self collected dataset from N5 highway in Pakistan and (3) manually occluded samples from each of these two datasets. The results show that our proposed method was found by at least 5% more accurate compared to the other state of the art and recent methods at the cost of only an insignificant additional computational cost.

Footnotes

[1]

References

Mogelmose ,

M.M.

Trivedi and

T.B.

Moeslund , Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey, Intelligent Transportation Systems, IEEE Transactions on 13(4) (2012), 1484–1497.

Maldonado-Bascon ,

Lafuente-Arroyo ,

Gil-Jimenez ,

Gomez-Moreno and

Lopez-Ferreras , Road-sign detection and recognition based on support vector machines, IEEE Transactions on Intelligent Transportation Systems 8(2) (2007), 264–278.

Fleyeh and

Davami , Eigen-based traffic sign recognition, IET Intelligent Transport Systems 5(3) (2011), 190–196.

Yan ,

Yu ,

Shi and

Feng , The recognition of traffic speed limit sign in hazy weather, Journal of Intelligent & Fuzzy Systems 33(2) (2017), 873–883.

Liu ,

Chang ,

Chen and

Liu , Fast traffic sign recognition via high-contrast region extraction and extended sparse representation, IEEE Transactions on Intelligent Transportation Systems 17(1) (2016), 79–92.

L.-W.

Tsai ,

J.-W.

Hsieh ,

C.-H.

Chuang ,

Y.-J.

Tseng ,

K.-C.

Fan and

C.-C.

Lee , Road sign detection using eigen colour, IET Computer Vision 2(3) (2008), 164–177.

Yang ,

Luo ,

Xu and

Wu , Towards real-time traffic sign detection and classification, IEEE Transactions on Intelligent Transportation Systems 17(7) (2016), 2022–2031.

Kim , Detection of traffic signs based on eigen-color model and saliency model in driver assistance systems, International Journal of Automotive Technology 14(3) (2013), 429–439.

Li ,

Sun ,

Liu and

Wang , A novel traffic sign detection method via color segmentation and robust shape matching, Neurocomputing 169 (2015), 77–88.

10.

R.C.

Gonzalez , Digital image processing, Pearson Education India, 2009.

11.

Greenhalgh and

Mirmehdi , Real-time detection and recognition of road traffic signs, Intelligent Transportation Systems, IEEE Transactions on 13(4) (2012), 1498–1506.

12.

Gudigar ,

Chokkadi ,

Raghavendra and

U.R.

Acharya , Multiple thresholding and subspace based approach for detection and recognition of traffic sign, Multimedia Tools and Applications 76(5) (2017), 6973–6991.

13.

G.-B.

Huang ,

Q.-Y.

Zhu and

C.-K.

Siew , Extreme learning machine: Theory and applications, Neurocomputing 70(1-3) (2006), 489–501.

14.

Bennasar ,

Hicks and

Setchi , Feature selection using joint mutual information maximisation, Expert Systems with Applications 42(22) (2015), 8520–8532.

15.

Scholkopf and

A.J.

Smola , Learning with kernels: Support vector machines, regularization, optimization, and beyond, MIT press, 2002.

16.

C.-C.

Chang and

C.-J.

Lin , LIBSVM: A library for support vector machines, ACM Transactions on Intelligent Systems and Technology (TIST) 2(3) (2011), 27.

17.

Ellahyani ,

El Ansari and

El Jaafari , Traffic sign detection and recognition based on random forests, Applied Soft Computing 46 (2016), 805–815.

18.

De La Escalera ,

L.E.

Moreno ,

M.A.

Salichs and

J.M.

Armingol , Road traffic sign detection and classification, IEEE Transactions on Industrial Electronics 44(6) (1997), 848–859.

19.

Fleyeh and

Davami , Eigen-based traffic sign recognition, IET Intelligent Transport Systems 5(3) (2011), 190–196.

20.

Houben ,

Stallkamp ,

Salmen ,

Schlipsing and

Igel , Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark, in: Neural Networks (IJCNN), The 2013 International Joint Conference on, IEEE, 2013, pp. 1–8.

21.

Y.-I.

Ohta ,

Kanade and

Sakai , Color information for region segmentation, Computer Graphics and Image Processing 13(3) (1980), 222–241.

22.

Wang ,

Hou ,

Xu ,

Yue and

C.-L.

Liu , Traffic Sign Detection Using a Cascade Method With Fast Feature Extraction and Saliency Test, IEEE Transactions on Intelligent Transportation Systems 18(12) (2017), 3290–3302.

23.

Barnes ,

Zelinsky and

L.S.

Fletcher , Real-time speed sign detection using the radial symmetry detector, IEEE Transactions on Intelligent Transportation Systems 9(2) (2008), 322–332.

24.

R.C.

Gonzalez , Digital image processing, Pearson Education India, 2009.

25.

Dalal and

Triggs , Histograms of oriented gradients for human detection, in: Computer Vision and Pattern Recognition, 2005. CVPR2005. IEEE Computer Society Conference on, Vol. 1, IEEE, 2005, pp. 886–893.

26.

Gudigar ,

Chokkadi and

Raghavendra , A review on automatic detection and recognition of traffic sign, Multimedia Tools and Applications 75(1) (2016), 333–364.

27.

Huang ,

Yu ,

Gu and

Liu , An efficient method for traffic sign recognition based on extreme learning machine, IEEE Transactions on Cybernetics 47(4) (2017), 920–933.

28.

Y.-L.

Hou ,

Hao and

Chen , A cognitively motivated method for classification of occluded traffic signs, IEEE Transactions on Systems, Man, and Cybernetics: Systems 47(2) (2017), 255–262.

29.

Lafuente-Arroyo ,

Gil-Jimenez ,

Maldonado-Bascon ,

Lopez-Ferreras and

Maldonado-Bascon , Traffic sign shape classification evaluation I: SVM using distance to borders, in: Intelligent Vehicles Symposium, 2005. Proceedings. IEEE, IEEE, 2005, pp. 557–562.

30.

Liu ,

Chang and

Liu , Occlusion-robust traffic sign detection via cascaded colour cubic feature, IET Intelligent Transport Systems 10(5) (2016), 354–360.

31.

Gudigar ,

Chokkadi ,

Raghavendra and U.R. Acharya, An efficient traffic sign recognition based on graph embedding features, Neural Computing and Applications (2017), 1–13.

32.

Cyganek , Circular road signs recognition with soft classifiers, Integrated Computer-Aided Engineering 14(4) (2007), 323–343.

33.

Gudigar ,

Chokkadi ,

Raghavendra and

U.R.

Acharya , Local texture patterns for traffic sign recognition using higher order spectra, Pattern Recognition Letters 94 (2017), 202–210.

34.

Rehman ,

Riaz ,

Fan and

Shin , D-patches: Effective traffic sign detection with occlusion handling, IET Computer Vision 11(5) (2017), 368–377.

35.

Alpaydin , Introduction to machine learning, MIT press, 2014.

36.

X.-Y.

Jing and

Zhang , A face and palmprint recognition approach based on discriminant DCT feature extraction, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 34(6) (2004), 2405–2415.

37.

Noon ,

Javed ,

Mannan and

Babri , Recognizing traffic signs using flexible Discrete Cosine Transform (DCT) grid, Scientia Iranica. Transaction D, Computer Science & Engineering, Electrical 24(3) (2017), 1384–1394.

38.

M.P.

Dale ,

M.A.

Joshi and

Gilda , Texture based palmprint identification using DCT features, in: Advances in Pattern Recognition, 2009. ICAPR'09. Seventh International Conference on, IEEE, 2009, pp. 221–224.

39.

Yang ,

Liu ,

Jiang and

Fan , A modified Gabor filter design method for fingerprint image enhancement, Pattern Recognition Letters 24(12) (2003), 1805–1817.

40.

Wang ,

Kong ,

Ding ,

Wang and

Yang , A novel traffic sign recognition algorithm based on sparse representation and dictionary learning, Journal of Intelligent & Fuzzy Systems 32(5) (2017), 3775–3784.

41.

Fracastoro ,

S.M.

Fosson and

Magli , Steerable discrete cosine transform, IEEE Transactions on Image Processing 26(1) (2017), 303–314.

42.

Brown ,

Pocock ,

M.-J.

Zhao and

Lujan , Conditional likelihood maximisation: A unifying framework for information theoretic feature selection, Journal of Machine Learning Research 13 (2012), 27–66.

43.

Zeng ,

Zhang ,

Zhang and

Yin , A novel feature selection method considering feature interaction, Pattern Recognition 48(8) (2015), 2656–2666.

44.

Welch , Lower bounds on the maximum cross correlation of signals (Corresp.), IEEE Transactions on Information Theory (1974), 397–399.

45.

Y.-L.

Hou ,

Hao and

Chen , A Cognitively Motivated Method for Classification of Occluded Traffic Signs, IEEE Transactions on Systems, Man, and Cybernetics: Systems 47(2) (2017), 255–262.

46.

Shabbir ,

M.F.

Khokhar ,

Shaiganfar and

Wagner , Spatial variance and assessment of nitrogen dioxide pollution in major cities of Pakistan along N5-Highway, Journal of Environmental Sciences 43 (2016), 4–14.

47.

Peng ,

Long and

Ding , Feature selection based on mutual information criteria of max-dependency, max- relevance, and min-redundancy, IEEE Transactions on Pattern Analysis and Machine Intelligence 27(8) (2005), 1226–1238.

48.

Fleuret , Fast binary feature selection with conditional mutual information, The Journal of Machine Learning Research 5 (2004), 1531–1555.

Optimized segmentation and multiscale emphasized feature extraction for traffic sign detection and recognition

Abstract

Keywords

1. Introduction

2. Related work

3. Our proposed method

4.1. Datasets and test bench

Table 2 Comparison of detection accuracies on all datasets. Our Proposed Detection Method HSI thresh. [3] HSI thresh. [17] Norm. RGB [11] Norm. Ohta [7] GTSDB 0.99 0.88 0.93 0.97 0.95 Self Collected 0.99 0.85 0.88 0.91 0.90 Occluded 0.85 0.69 0.77 0.80 0.80

Footnotes

[1]

References

Table 2
Comparison of detection accuracies on all datasets.

Our Proposed Detection Method HSI thresh. [3] HSI thresh. [17] Norm. RGB [11] Norm. Ohta [7]

GTSDB 0.99 0.88 0.93 0.97 0.95

Self Collected 0.99 0.85 0.88 0.91 0.90

Occluded 0.85 0.69 0.77 0.80 0.80