Proceedings of the 19th World Congress

The International Federation of Automatic Control

Cape Town, South Africa. August 24-29, 2014

Traffic sign recognition application based on image processing techniques

Rubén Laguna*, Rubén Barrientos*, L. Felipe Blázquez*, Luis J. Miguel**



*Department of Electrical and Automatic Control and Systems Engineering, University of León, E.II.I.I.,
Campus de Vegazana s/n, 24071 León, Spain. (Tel: 34 987293471; e-mail:
{rlagug00,rbarrm00,diefbq}@unileon.es).
**Department of Automatic Control and Systems Engineering, University of Valladolid, E.I.I., Paseo del
Cauce s/n, 47011 Valladolid, Spain. (Tel: 34 983423545; e-mail: [email protected]).

Abstract: This paper describes a software application for traffic sign recognition (TSR). The application
works in four stages. First, an image preprocessing step and the detection of regions of interest (ROIs),
which involves a series of steps that include transforming the image to grayscale and applying edge
detection by the Laplacian of Gaussian (LOG) filter. Secondly, the potential traffic signs detection, where
the ROIs are compared with each shape pattern. Thirdly, a recognition stage using a cross-correlation
algorithm, where each potential traffic sign, if validated, is classified according to the data-base of traffic
signs. Finally, the previous stages can be managed and controlled by a graphical user interface, which has
been specially designed for this purpose. The results obtained show a good performance of the developed
application, taking into account acceptable conditions of size and contrast of the input image.
Keywords: Traffic sign recognition; image processing; image segmentation; visual pattern recognition;
boundary detection; cross correlation functions.

every traffic sign present in the scene (Ritter et. al., 1995). In
the last few years, the TSR systems have become important
1. INTRODUCTION components of advanced driver assistance systems (ADAS)
Digital image processing uses algorithms to process digital (Gil et al., 2008). The main difficulty that TSR systems face
images. Although it may seem recent technology, many of is the poor image quality due to low resolution, bad weather
the techniques were developed in the 1960s at the conditions or inadequate illumination.
Massachusetts Institute of Technology, University of
The application developed in this paper determines the area
Maryland and other research facilities (Chen et al., 1993). of the original image where the potential traffic sign is
However, due to the high costs of computers at this time, the located, and compares it with a database which contains
digital imaging process was too expensive for many to even
normalized traffic signs. The process starts with a
consider. That changed in the 1970s, when digital image
preprocessing stage for the input image, where the parameters
processing proliferated as cheaper computers and dedicated
of the image, such as resolution or contrast, are modified to
hardware became available. With the fast computers and guarantee that the filters and algorithms used later behave
signal processors available in the 2000s, images could be properly. When the parameters of the image have been
processed in real time and digital image processing has
adjusted, an edge detection algorithm is used in order to
become the most common form of image processing
determine the potential areas of the image where a possible
(González and Woods, 2007). Computer vision is
traffic sign may be located. The next step is to separate the
increasingly used in the field of intelligent transport
regions of interest (ROIs) found in the image and obtain the
(Mapanga and Ragavan, 2012), and traffic sign recognition is potential traffic signs. Every potential traffic sign is
a very important part of this. These systems are typically submitted to a recognition process, using a cross-correlation
based on detecting a region of interest (ROI), in which the
algorithm that compares each one with a database, which
traffic sign is located, using characteristics such as color and
contains patterns of traffic signs. This software includes a
geometric form. In computer vision, the ROI defines the
graphical user interface, which allows the user to control each
borders of an object under consideration and is commonly
stage of the application. The results obtained show a high
used in many application areas, such as medical imaging success rate, dependent on the environmental conditions of
(Dougherty, 2009). the input image and its resolution.
Video processing techniques for traffic applications have
This paper is organized as follows. In section 2, the
been an attractive field of research during the last two
preprocessing process and the ROI detection are described.
decades (Piccioli et al., 1996; Kastrinaki et al., 2003).
Section 3 shows the potential traffic sign detection stage.
Several techniques have been proposed to develop traffic sign
Section 4 details the traffic sign recognition process. Section
recognition (TSR) systems (Ruta et. al., 2010). The main 5 describes the graphical user interface. In section 6 the
goal of these systems is the detection and recognition of experiments carried out are described and the results obtained

978-3-902823-62-5/2014 © IFAC 104

19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

are presented. Section 7 contains the conclusions of the work The algorithm used to detect edges in the image is the
and finally some references are given. Laplacian of Gaussian (LOG), which takes a grayscale image
as input and produces another grayscale image as output,
where only the edges are shown. The LOG is a hybrid filter
2. IMAGE PREPROCESSING AND REGIONS OF
that contains a Gaussian smoothing filter and a Laplacian
INTEREST DETECTION
filter, and then convolves it with the image to achieve the
The first step of the preprocessing is to adjust the image size, required result (Sharifi et al., 2002). The function used,
reducing the space occupied when the resolution of the input centered on zero and with standard deviation σ, is given by
image is too high. If the image size were too high, it would (1) and has the form shown in Fig. 2, where (x,y) are the
slow down the execution of the algorithms to the point that it coordinates of each pixel.
would restrict the program execution. Next, a contrast limited
adaptive histogram equalization (Sepasian et al., 2008; Reza,
2004) is performed to enhance the contrast of the image by ( ) [ ] (1)
transforming the values obtained in the intensity image. This
method is chosen to prevent the over amplification of noise
that adaptive histogram equalization can give rise to (Pizer et
al., 1987). The contrast amplification in the vicinity of a pixel
is proportional to the slope of the neighborhood cumulative
distribution function and to the value of the histogram at that
pixel value. The next step is to transform the input image into
a gray scale image, so the edge detection algorithm can be
applied (Pei and Horng, 2000; Wan et al., 2007). In Fig. 1, a
typical traffic image taken as example and its corresponding
gray-scale image, are shown.

Fig. 2. Laplacian of Gaussian function.

The LOG operator calculates the second spatial derivative of

an image. In areas where the image has a constant intensity,
the gradient is zero and the response will be zero too. In the
vicinity of a change in intensity, the response will be positive
just to one side of the edge and negative just to the other side
of the edge. In the case of the image used as an example, the
result obtained is shown in Fig. 3.

Fig. 3. Edge detection.

Once the edge filter has been applied, the areas of the image
that do not exceed a certain size are eliminated to avoid
possible confusion with the environment. This is done
because traffic signs are quite big objects so small edges
Fig. 1. Traffic image and its corresponding grayscale image. correspond to objects that cannot be regions of interest.
Figure 4 shows the ROIs for the image taken as example. In
this figure, every ROI must be evaluated separately.

105
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

closest point to the upper right corner of the image, as Fig. 5

shows. Then the distance between the centroid and the
perimeter is calculated and the signature is obtained. This
signature is normalized, i.e., to fit its value between 0 and 1,
so it can be compared with the normalized signatures of the
shape patterns. Each normalized signature obtained is stored
in a vector of N=100 elements, where n=1,…,N. Using this
method, no matter the size of the object evaluated, its
signature will be the same. In Fig. 6, the normalized signature
Snorm obtained for the ROI of Fig. 5 is presented.

3. POTENTIAL TRAFFIC SIGN DETECTION

Once the ROIs have been identified, the normalized
signatures of the different shape patterns are loaded. In this
application, Ns=6 shape patterns are used: circle, square,
Fig. 4. Regions of interest. rhombus, octagon, triangle, and inverted triangle. The
normalized signatures for these six shape patterns are shown
in table 1 as Spattern. This application allows the user to add
new patterns, so special regional traffic signs could also be
recognized.

Table 1. Normalized signatures of the shape patterns

Traffic sign Normalized signature
patterns

Circle

Fig. 5. Contour of the ROI, its centroid and the starting point.
Square

Rhombus

Octagon

Fig. 6. Normalized signature of the ROI.

There are many types of traffic signs, but their shapes are
Triangle
very limited and characteristic, so knowing the shape allows
us to determine if an ROI contains a potential traffic sign or
not. In order to do so, the signature of every ROI must be
calculated and normalized in order to be compared with the
normalized signatures of the shape patterns, such as squares,
circles, etc. To obtain the signature, the developed application
calculates the contour of every ROI, its centroid and the Inverted triangle

106
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

For each ROI, its normalized signature is compared with the pattern k, and , k are the mean of all the elements in A and
normalized signature of each shape pattern by (2), where Bk respectively.
Snormi is each value of the signature of the ROI, Spatternij is
each value of the normalized signature of the pattern j, ∑ ∑ ( ̅ )( ̅ )
(4)
i=1,...,N with N=100 and j=1,…,Ns with Ns=6. √(∑ ∑ ( ̅ ) )(∑ ∑ ( ̅ ) )

∑ | | (2)
If Rk>thsign, where thsign is a threshold value, then the
potential traffic sign can be considered very similar to the
If min(Dj)<thshape, where thshape is a threshold value, then this traffic sign pattern. So the application would give the traffic
ROI is in fact a potential traffic sign, which has the same sign of the pattern k as the recognized sign. In this application
shape of the pattern j. In this case, the coordinates of this ROI thsign=0.51 and has been calculated empirically.
are saved in a cell array, along with the type of shape that it
contains. Then, the application continues with another ROI.
Table 3. Cross-correlation matrix for different patterns of
In this application, thshape=6 and has been calculated
circular traffic signs
empirically to minimize the error. On the other hand, if
min(Dj)≥ thshape, then the ROI is not a potential traffic sign
and the application continues with another ROI. When all Database
sign
γk Rk
ROIs have been checked and some potential traffic sign has
been found, the application continues with the traffic sign
recognition process.
Table 2 presents the values of Dj, obtained for the ROI of Fig.
0.5723
5, and these values show that the ROI is in fact a potential
traffic sign with a circular shape.

Table 2. Values of Dj

Spattern
0.3855
Dj 4.29 13.84 12.55 6.99 31.17 26.18

4. TRAFFIC SIGN RECOGNITION

Once the location and shape of all potential traffic signs in 0.3807
the image are known, each one of these potential traffic signs
must be compared with a database. The database contains a
certain number of traffic sign patterns, which are classified
according to their shape. Due to the fact that the comparison
process is iterative, the processing time of the application
increases when more patterns of traffic signs are added. In
this sense, the application allows the user to add new 0.4212
patterns. However, this would increase the processing time of
the whole process significantly.
Each potential traffic sign is compared with each traffic sign
pattern using the cross-correlation algorithm given by (3),
where f is the cell array corresponding to the potential traffic
sign and tk is the cell array of the traffic sign pattern k (Lewis, 0.5222
1995).

∑ [ ( ) ̅ ][ ( ) ̅ ]
( ) (3)
{∑ [ ( ) ̅ ] ∑ [ ( ) ̅ ] }

Table 3 presents the cross-correlation matrixes γk and their

The matrix γk obtained contains the correlation coefficients, corresponding correlation coefficient Rk obtained for the
whose values vary from -1.0 to 1.0. The peaks of the cross- potential traffic sign of Fig. 5 with k=1,…5. The values of the
correlation matrix occur where the images are best correlated. correlation coefficient of the table show that two values are
Both cell arrays are compared using the correlation higher than thsign, and in these cases, the highest is taken for
coefficient Rk of γk given by (4), where A is the traffic sign recognition purpose. So the application recognizes the
image resized to the size of the pattern, Bk is the image of the potential traffic sign as the traffic sign pattern shown in Fig. 7

107
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

and we can see that the traffic sign has been successfully input image, its histogram and the ROIs that are to be
recognized. evaluated. Through the button detect signs, the previously
described algorithms are executed and the results are
displayed. If more than one sign was detected, they would be
shown at the bottom right in the Signs recognized section.

Fig. 7. Potential traffic signal and traffic signal pattern.

If the potential traffic sign does not match any of the traffic
sign patterns, the object has the shape of a traffic sign, and
therefore its signature, but is not a traffic sign. It should be
noted that if the correlation coefficients of the cross- Fig. 9. Second interface.
correlation matrix are lower than the threshold, it could be
due to the fact that the signal pattern is not present in the 6. EXPERIMENTAL RESULTS
pattern database.
The application has been tested with 200 different images,
where 25 images do not contain any traffic sign, 141 images
5. GRAPHICAL USER INTERFACE DESIGN contain one traffic sign, 22 images contain two traffic signs
In order to improve visualization of the application and make and 12 images contain three traffic signs. The total number of
it interactive for a user, a graphical user interface has been shapes analyzed is 221 and the distribution is shown in Fig.
designed. It allows the user to see graphically the whole 10.
process executed by the application and to modify the input
parameters of the system. The code of the application is
structured and separated in functions, so it can be handled by
the dynamic interface. The graphical user interface actually
consists of two stages. The first interface, as Fig. 8 shows,
acts as a title page and allows the user to select dynamically
the directory where the images are stored, as well as their
extension. When the continue button is pressed, all the
patterns, both traffic sign shapes and traffic signs, are loaded
into the memory so they do not need to be loaded again,
significantly reducing the processing time of the application.

Fig. 10. Traffic sign shapes contained in the 200 test images.

Fig. 8. First interface.

The second interface, as Fig. 9 shows, contains a series of

buttons that allow the user to select the image to evaluate
from all the images contained in the previously selected Fig. 11. Shape recognition hit rate in 200 test images.
directory. The application allows the user to visualize the

108
19th IFAC World Congress
Cape Town, South Africa. August 24-29, 2014

The average success rate is 91.07%, the result being Lewis, J.P. (1995). Fast Normalized Cross-Correlation.
independent of the number of traffic signs that the image Vision Interface, 10, 120-123.
contains. The success rate for each type of shape is shown in Mapanga, K., and Ragavan, V. (2012). Machine Vision for
Fig. 11. The lower rate of success in octagon shaped signals intelligent Semi-autonomous Transport (MV-iSAT).
with 88.24%, is due to the fact that in certain images the Procedia Engineering, 41, 395-404.
signature of the octagon is recognized as a circle, so the Pei, S.-C., and Horng, J.-H. (2002). Design of FIR bilevel
detected sign is incorrect. Laplacian-of-Gaussian filter. Signal Processing, 82(4),
677-691.
There are recognition problems in low-resolution images as,
Piccioli, G., De Micheli, E., Parodi, P., and Campani, M.
due to the low pixel rate, the recognized the areas are joined
(1996). Robust method for road sign detection and
together, resulting in a detection of erroneous ROIs.
recognition. Image and Vision Computing, 14(3), 209-
Furthermore, very high resolution images, due to the weight
223.
in memory, must be previously treated, reducing the storage
Pizer, S.M., Amburn, E.P., Austin, J.D., Cromartie, R.,
weight before loading them, because the memory space is
Geselowitz, A., Greer, T., Romeny, B., Zimmerman, J.B.
problematic and cannot be properly processed by the
and Zuiderveld, K. (1987). Adaptive histogram
program. On the other hand, in pictures showing cuts in their
equalization and its variations. Computer Vision,
area or perimeter due to external agents such as vegetation,
Graphics, and Image Processing, 39(3), 355-368.
climate, etc., the application had difficulty recognizing the
Reza, A.M. (2004). Realization of the Contrast Limited
area and, in many cases, the area was ruled out due to the fact
Adaptive Histogram Equalization (CLAHE) for Real-
that it did not have a closed surface.
Time Image Enhancement. Journal of VLSI Signal
Processing Systems for signal, image and video
7. CONCLUSIONS technology, 38(1), 35-44.
Ritter, W., Stein, F., and Janssen, R. (1995). Traffic sign
A new traffic sign recognition system has been presented in recognition using colour informaion. Mathematical and
this paper. The application software developed in this work Computer Modelling, 22(4-7), 149-157, 159-161.
recognizes and classifies traffic signs from an input image. Ruta, A., Li, Y., and Liu, X. (2010). Real-time traffic sign
The image processing techniques used in this software recognition from video by class-specific discriminative
include a preprocessing stage, regions of interest detection, features. Pattern Recognition, 43(1), 416-430.
potential traffic sign detection, according to the traffic sign Sepasian, M., Balachandran, W., and Mares, C. (2008).
shape patterns, and finally, the recognition and classification Image Enhancement for Fingerprint Minutiae-Based
of these potential traffic signs according to a database of Algorithms Using CLAHE, Standard Deviation Analysis
traffic sign patterns. The performance of this application and Sliding Neighborhood. In International Association
depends on the quality of the input image, in relation to its of Engineers (Ed.), Proceedings of the World Congress
size, contrast and the way the signs appear in the image. With on Engineering and Computer Science WCECS 2008.
this consideration, the percentages of recognized signs for 1199-1203. San Francisco. USA.
this application are high. Sharifi, M., Fathy, M., and Tayefeh, M. (2002). A classified
As further work, a neural network could be implemented in and comparative study of edge detection algorithms. In.
order to obtain more accurately the empirical parameters used Proceedings of the International Conference on
in the application. Furthermore, the application could be Information Technology: Coding and Computing. 117-
optimized by implementing an embedded hardware for use in 120.
active applications. Wan, J., He, X., and Shi, P. (2007). An Iris Image Quality
Assessment Method Based on Laplacian of Gaussian
Operation, In Proceedings of the IAPR Conference on
REFERENCES Machine Vision Applications IAPR MVA 2007. 248-251.
Chen, C.H., Pau, L.F., and Wang, P.S.P. (1993). Handbook of Tokio, Japan.
Pattern Recognition and Computer Vision. World
Scientific Publishing, Singapore.
Dougherty, G. (2009). Digital Image Processing for Medical
Applications. Cambridge University Press, Cambridge.
Gil, P., Maldonado, S., Gómez, H., Lafuente, S., and López,
F. (2008). Traffic sign shape classification and
localization based on the normalized FFT of the
signature of blobs and 2D homographies. Signal
Processing, 88(12), 2943–2955.
González, R.C., and Woods, R.E. (2007). Digital image
processing, 3rd. Ed. Prentice Hall, New Jersey.
Kastrinaki, V., Zervakis, and M., Kalaitzakis, K. (2003). A
survey of video processing techniques for traffic
applications. Image and Vision Computing, 21(4), 359–
381.

109