A Robust License-Plate Extraction Method under Complex Image Conditions

Sunghoon Kim, Daechul Kim, Younbok Ryu, and Gyeonghwan Kim

Dept. of Electronic Engineering
Sogang University
CPO Box 1142, Seoul 100-611, Korea

doncopy, s950812, roy21, gkim@sogang.ac.kr

Abstract rectly affects system’s overall accuracy. To make the ex-

traction process successful, many difficult problems should
A robust approach for extracting car license plate from be dealt with, such as poor image quality due to various am-
images with complex background and relatively poor qual- bient lighting conditions and image distortion stems mainly
ity is presented in this paper. The approach focuses on deal- from various combination of visual angles between the cam-
ing with images taken under weak lighting condition. The era and the car. The image capturing process is largely
proposed method is divided into two steps: 1) searching depending on several aspects, including camera angle, dy-
candidate areas from the input image using gradient infor- namic range, ambient lighting condition, background com-
mation, and 2) determining the plate area among the candi- plexity, existence of reflection and plate fracture [2]. There-
dates and adjusting the boundary of the area by introducing fore, it is necessary to make the extraction process robust so
a plate template. A set of experiments has been performed that it can work on any situation in the real world environ-
to prove the robustness and accuracy of the approach. For ment.
many images collected from a large underground parking For extracting the plate region, techniques such as edge
place the result shows that 90% of them are correctly seg- extraction [1][7], Hough transform [8], histogram analy-
mented. sis [4] and morphological operators [10] have been applied.
An edge-based approach is normally simple and fast. How-
ever, it is too sensitive to the unwanted edges, which may
1. Introduction happen to appear in the front part of a car. Therefore, this
method cannot be used independently. Hough transform for
As the ITS(Intelligent Transport Systems) becomes get- line detection gives positive effect on images with a large
ting bigger attention, automatic identification of a vehicle plate region where it can be assumed the shape of license
has turned out to be an important research issue [5]. In plate is defined by lines. However, it needs a large mem-
general, there are two ways in identifying a vehicle: one ory space and considerable amount of computing time. The
approach is to use of electronic devices to identify each in- histogram based approach does not working properly on the
dividual car. The RF ID tag, which has been attached to a image with noises and the image with tilted plate. Morphol-
car for the purpose, is a popular example. The unique infor- ogy has been known to be strong to noise signals, but it is
mation from the tag, when microwave signal arrives from rarely used in real time system because of its slow opera-
an external antenna, is decoded and used for identifying the tion.
car. Another way is to retrieve the information on the li- The approach presented in this paper is to extract the
cense plate. In the approach, a camera captures the image region of the license plate from images taken from indoor
of a car and a computer processes the image and recognizes parking lots, which suffer from various real world problems.
the information on the plate by applying various image pro- We have encountered many images with various problems,
cessing and optical pattern recognition techniques [6]. including poor ambient lighting, diverse plate locations, tilt
Recognition-based approaches reported in previous re- of the plate, and influence from strong light beams. The
search generally consist of several procedures, including ex- proposed approach consists of two parts: 1) searching can-
tracting a license plate region, segmenting characters from didate areas using a combination of gradient features ob-
the region, and recognizing each character. Among the pro- tained from the input image, and 2) determining the plate
cedures, accurate extraction of the license plate region from area among the candidates and adjusting the boundary of
a variable of scenes is of crucial importance because it di- the area by introducing a plate template.

1051-4651/02 $17.00 (c) 2002 IEEE

 Figure 2. The variance is plotted based on

for a plate image

Figure 1. The overview of the algorithm

2. Outline of the approach

A two-stage search process is proposed in the approach:

the global search in which candidates of the plate region are
sought using the combination of the gradient features, and Figure 3. Feature images formed using  (left)
the local search where the correct plate region is selected and  (right) for two plate images
from the candidates and the boundary of the region is ad-
justed using a plate template. Details on each step shown in
Figure 1 are described in this section. from windows of size 1¢9 pixels using Eq.(2). The
window size is determined experimentally. Figure 2
2.1 Global search shows the changes of variance in a plate image.



 




In the beginning part of the stage, histogram stretching



  
       
  (2)
is performed as a pre-processing, using Eq.(1), in order to  
increase the contrast of the image and protect the vital edges
from subsequent operations. where  represents the gradient at

   and  is
the number of pixels involved while the variance is ob-


  

 (1) tained.
where,  represents the intensity value of the input image, ¯ Feature2 - the density of edges: Plate regions tend to
 is the variance of the image, and  is a constant (set to 30 have a high density of edges. The density is measured
in this case). in a block of size 1¢9 pixels by summing all edge pix-
Step 1: Obtaining three statistical features els using Eq.(3).
Conventional approaches, which rely only on the variance
of the gradient while ROIs being searched, could result in 

   


  

   (3)
 
  
limited success when the image under consideration con-
tains a tilted plate and/or complicated background. There-
fore three statistical features are introduced in the proposed where 

   represents the edge magnitude at

  ,
approach and combined by a simple neural network. and  is the number of pixels involved while the fea-
¯ Feature1 - the gradient variance: The gradient of the ture is obtained.
image is obtained using the Sobel operator. Based on Bottom left image in Figure 3 is formed using the fea-
the observation that gray scale changes are more fre- ture described above for the upper left image in the
quent in the plate area, local variances are obtained figure.

1051-4651/02 $17.00 (c) 2002 IEEE

 ¯ Feature3 - the density variance: The feature is based
on the concept that if a candidate area indeed contains
a plate, the foreground pixels are distributed evenly
comparing to the areas with simple structures [3].
Therefore, this feature can be used to discriminate text
regions from background regions. To obtain the fea-
ture, a block of size 9¢9, centered at

  , is divided
into nine equal-sized sub-blocks. For each sub-block

, let  denote the total number of edge points in the
sub-block. Then the feature, the density variance, is


defined by Eq.(4).

 
¿

   

 (4)


where 
 is the average of   s.
Bottom right image in Figure 3 is formed using the
feature described above for the upper right image in
the figure.
Those three features are combined with a neural network
to determine if a particular pixel under examination could
belong to the plate region. To train the network, around
300,000 patterns were collected from 50 images. The out-
put of the network forms a map, which is called the feature
map hereafter.
Figure 5. The search procedure
Step 2: Finding candidate areas
In the feature map, areas with higher variance are grouped
and labeled using chaincode-based contour follower [9]. After the merge is completed, the boxes with both the as-
While the processing is being performed, careful examina- pect ratio of less than 1/2 and horizontal length of less than
tion is carried out to apply certain limitation on the size of 200 pixels are used for the next step. A region of interest
candidates. The examination prevents from creating candi- is determined by another back-propagation neural network
dates with too big or small aspect ratio comparing to that of which inputs box diagonal indices, average value of inten-
the license plate. sities and box filling ratio.

Step 3: Selecting an ROI 2.2 Local search

Candidate areas are represented by bounding boxes and a
plate region may consist of one or more than one boxes as Once an ROI is selected, it is necessary to make sure
shown in Figure 4(left). To merge the boxes, a simple neural that the box contains the plate. The similarity transform is
network is employed and it determines whether two boxes performed to the plate template so that it fits into the ROI
under examination need to be merged or not. As inputs to selected. Then, projection profiles on both directions are ex-
the neural network, corner points of the boxes, aspect ratios, amined to judge whether the ROI contains the plate or not.
and box filling ratios are provided. When the parameters collected from the projection profiles
satisfy the conditions shown in Figure 5, the ROI is regarded
as the correct plate region. If the ROI does not satisfy either
of the conditions, the process is repeated until the selected
ROIs are consumed.
Figure 6 illustrates intermediate results of some steps ex-
plained in this section for an input image.

3 Experiments
Figure 4. Merging bounding boxes: before
(left) and after (right) Many number of images with various types of vehicles,
not only passenger cars but also vans and trucks, were col-

1051-4651/02 $17.00 (c) 2002 IEEE

 (a) (b)
Figure 7. The gradient variance method (left)
and the proposed method (right)

whether it contains the license plate by introducing a tem-

plate of the license plate.
(c) (d) Based on the experimental result, the performance of the
proposed approach is promising. Comparing to the conven-
tional edge-based approach, the overall processing speed
is a little bit slower, but the approach is more robust in
terms of accuracy. Ways to make the approach more ro-
bust and faster are being sought by combining a couple of
(e) (f) techniques, including the active contour algorithm.
Figure 6. Processing steps: (a) an input im-
age, (b) the feature map, (c) bounding boxes, References
(d) selected ROI, (e) initial setting for the local
search, and (f) the local search result [1] D. H. Ballard. Computer Vision. Prentice-Hall Inc., 1991.
[2] M. W. Burke. Image acquisition. Chapman & Hall, 1996.
[3] W.-Y. Chen and S.-Y. Chen. Adaptive page segmentation
for color technical journals’ cover images. Image and Vision
lected from a large indoor parking place with different an- Computing, 16(12-13):855–877, 1998.
[4] D. U. Cho and Y. H. Cho. Implementation of preprocessing
gles and different lightening conditions. The image acqui-
independent of environment and recognition of car number
sition system consists of a CCD camera, a photo sensor to plate using histogram and template matching. The Journal of
detect presence of a car, and a personal computer with a im-
age grabber in it. Size of the image is 640492. 1,000 of
the Korean Institute of Communication Sciences, 23(1):94–
100, 1998.
them were used for the evaluation purpose. [5] D. Gao and J. Zhou. Car License Plate Detection from Com-
The experimental result shows that the proposed algo- plex Scene. In Proceedings of International Conference on
rithm locates correctly the plate area for 90% of the images Signal Processing, pages 1409–1414, 2000.
used. Sources of the failure can be classified into three ma- [6] P. Hu, Y. Zhao, J. Zhu, and J. Wang. An Effective Automatic
License Plate Recognition System. In The proceedings of
jor categories: 1) existence of other text blocks, 2) bounding
CISST2000, pages 80–84, 2000.
box containing the plate is merged into other box(es), and [7] K. Kanayama, Y. Fujikawa, K. Fujimoto, and M. Horino.
(3) weak gradient information from the plate area. Development of vehicle-license number recognition sys-
As shown in Figure 7, by combining the features using tem using real-time image processing and its application to
the neural network the system becomes more robust against travel-time measurement. In Proceedings of IEEE Vehicular
the noise and tilt comparing to the other method [5]. Technology Conference, pages 798–804, 1991.
[8] K. M. Kim, B. J. Lee, K. Lyou, and G. T. Park. The auto-
matic recognition of the plate of vehicle using the correla-
4 Summary and Conclusion tion coefficient and Hough transform. Journal of Control,
Automation and System Engineering, 3(5):511–519, 1997.
[9] A. Rosenfeld and A. C. Kak. Digital Picture Processing.
A robust approach for extracting license plate from a car
Academic Press, 1982.
image is presented in this paper. The two-step approach is [10] M. Shridhar, J. W. Miller, G. Houle, and L. Bijnagte. Recog-
designed to deal with images taken under various real world nition of license plate images: Issues and perspectives.
conditions. In the first step, the whole image is searched and In Proceedings of International Conference on Document
candidate areas are located based on the gradient features. Analysis and Recognition, pages 17–20, 1999.
In the second stage, the candidates are examined to verify

1051-4651/02 $17.00 (c) 2002 IEEE