2020 International Conference on Computational Science and Computational Intelligence (CSCI)

Convolutional Neural Network-Based License

Plate Recognition Techniques: A Short Overview
Oladapo Ibitoye* Temitayo Ejidokun Olufemi Dada
Department of Electrical/Electronics Department of Electrical/Electronics Department of Electrical/Electronics
and Computer Engineering and Computer Engineering and Computer Engineering
Afe Babalola University Afe Babalola University Afe Babalola University
Ado Ekiti, Nigeria Ado Ekiti, Nigeria Ado Ekiti, Nigeria
[email protected] [email protected] [email protected]
*Corresponding author
Olusegun Omitola
Department of Electrical/Electronics
and Computer Engineering
Afe Babalola University
Ado Ekiti, Nigeria
[email protected]

Abstract— Identification of vehicle license plate has become

more practical within the last decade in many applications such potentials for solving problems related to detection, optical
as; storage and retrieval of vehicular inflow records, automatic character recognition, and classification [3]. CNN is
toll collection, parking fee payment, traffic monitoring, tracking considered as a special architecture of artificial neural
of moving vehicles, recovery of stolen vehicles, etc. A lot of networks and the top-performing model which is currently
researches have been carried out on license plate recognition being employed by researchers in license plate recognition
which has led to the development of new techniques and (LPR) systems [4].
modification of existing techniques. This study provides a
breakdown of some techniques used for the implementation of This paper provides a short overview of techniques for
license plate recognition systems, with significant emphasis on image acquisition, image preprocessing, license plate
Convolutional Neural Network (CNN). The strength and detection, license plate character segmentation, and license
weaknesses of the highlighted techniques were discussed. Also, plate character recognition. The paper also gives a breakdown
areas of further improvement of some of the selected CNN- of some CNN-based techniques for license plate recognition
based techniques were suggested. system.

Keywords— license plate, image processing, detection,

The remaining part of this paper is organized as follows.
recognition Section two provides a literature survey of all the stages from
image acquisition to license character recognition. Section
Short Papers three highlighted the strength of the reviewed techniques and
CSCI-ISCI possible areas of improvement were suggested. The last
section concluded the work and suggests areas of future
research.
I. INTRODUCTION
In recent years, automatic recognition of license plates has
II. REVIEW OF SOME CURRENT TECHNIQUES
constituted a significant part of intelligent transportation and
Surveillance System [1]. This is due to the tremendous A typical system for the recognition of characters in
advantages it has provided through its numerous applications license plates is presented in Fig. 1.
in our daily life. These include; instantaneous tracking of
traffic offenders, recovery of stolen vehicles, automated Image Pre- License Plate
record keeping of vehicular movements, and administration of processing Detection and
Acquisition Localization
parking tickets.
However, there are some challenges associated with
license plate detection and recognition, such as; plate
variations, poor illumination as a result of environmental
License Plate Character License Plate
lighting, inclination, and fonts variation [1]. In addition to the Character
above challenges, degradation and fading are equally issues Recognition
Segmentation
associated with license plate detection and recognition.
License plate detection remains a difficult task for a system of
Fig. 1: A typical block diagram for LPR system (crafted from process flow
license plate recognition. It is connected to and determines the
diagram). [5]
success of both segmentation and recognition of characters
[2]. A. Image Acquisition
There has been lots of research interest in recent years on An input of red, green, and blue (RGB) image captured by
Convolutional Neural Network (CNN), due to its numerous a camera was fed into the detector in [6]. Also, the camera was

978-1-7281-7624-6/20/$31.00 ©2020 IEEE 1529

DOI 10.1109/CSCI51800.2020.00283
integrated with electronic components such as ultrasonic mainly include color distortion and skew correction. In [14],
sensors and microcontrollers for effective image acquisition to get a good result in the character recognition phase, the
[6]. Affine transformation given in equation (1) was used to extracted license plates were made horizontal by correcting
simulate the distortion of the image resulting from facial the skew angle θ using equation (6).
imbalance and shooting angle of the camera in [8]. i i
cosθ sinθ
= (6)
(1 − ). . − . j −sinθ cosθ j
(1)
− . . + (1 − ). . where i and j represent horizontal and vertical respectively.
where: α is scale multiplied by cos angle, β is scale multiplied C. License Plate Detection and Localization
by sin angle.
After the preprocessing of the captured images of license
For a real-time deployment of a license plate recognition plates, it is, therefore, necessary to define certain unique
system, factors such as quality of the camera, its positioning, features of the license plate such as height, shape, color, etc.
quality of the plates to be detected and environmental For instance, license plates commonly have a rectangular
variations are key factors that determine the performance of shape. It is not impossible to have other objects within a
the system. vehicle image that also have rectangular shapes, examples are;
headlights and vehicle logos. Accurate detection thus helps
B. Image Pre-processing
reduce the number of false-positive errors and determines the
The captured images of the license plate need to be subsequent performance of the optical character recognition
subjected to various adaptations to ensure accurate detection engine [15].
and localization of the region of interest. Various methods
such as grayscale conversion, top-hat transformation, binary Color, edge, and region-based methods are commonly
morphological, Otsu threshold, and binary image projection used for license plate detection. License plate regions contain
were used for image preprocessing in [7]. Gray-Scale abundant edges when compared with the background, so,
Conversion is the transformation of the “RGB image” into a therefore, edge and region-based methods are preferred [15].
“Gray-scale image” as shown in equation (2) [8]. A deep learning-based technique for license plate
= 0.2 + 0.7 + 0.07 (2) recognition was proposed, the method uses two object
detection convolutional neural networks in [16]. The first
where Y is the converted gray. R, G, and B denote red, green, network was retrained for license plate detection and the
and blue respectively. As illustrated in equation (2), a resultant second network was also retrained for segmentation and
8-bit grayscale image was obtained from a 24-bit color image. recognition of characters based on the detected license plates.
This tends to simplify computation in license plate image The method was evaluated and an accuracy of 97.6 % was
enhancement and also increase the processing speed [6]. achieved.
Another aspect of image pre-processing is the conversion In [6], an algorithm for automatic license plate recognition
of the grayscale image into binary (white and black) [6]. In was developed using CNN. Overview for multiple pattern
[9], image binarization was employed using equation (3), the classification and detection methods were reported in [17].
pixels of the foreground colours were retained as white while The overview was accompanied by the comparisons of
all other pixels were converted to black. algorithms along with various datasets. [13] developed an
0 px ≤ 160 android based application for number plate recognition. The
B = (3) object detection was based on CNN using the You Only Look
225 otherwise
Once (YOLO) technique. 94.8% of accuracy was achieved
where px is the pixels during testing. While in [18], Tiny YOLOv3 was used for the
Also, noise removal needs to be carried out to eliminate detection of license plates.
unwanted signals, in the original image for enhanced CNN outputs were computed for the sub-regions of the
performance of the system [10]. Gaussian filtering was image. Furthermore, the location estimation of each license of
utilized in [11] for eliminating the noise in the images given the obtained result was estimated in [3] using equation (7).
by equation (4). The usage of Gaussian filtering does not only
eliminate the inherent noises but also concentrates the image − . 1 >
frequency [11]. = (7)
+ . ℎ
1 ( x +y
G(x, y) = e (4) where Sl and Sr are the output scores for left and right
2πa 2σ
neighbors respectively. w denotes the sub-region width The
where σ is the standard deviation of the distribution, the plate is set to the center when both Sl and Sr are 0, and offsetx
distribution is assumed to have a mean of 0. License plates is 0. Fig. 2 shows the CNN outputs for the two neighboring
are associated with salt and pepper noises [12]. To eliminate sub-regions.
the noises, a median filter expressed in the equation (5) was
engaged.
( ( , )= { ( , )}) (5)
where ( , ) is the filtered image, and ( , ) represents the
noisy image. In [13], the automatic number plate recognition
system utilized different image pre-processing techniques that (a) Sl = 0 and S = 1
made the images as readable as possible. These techniques

1530
 license plate thereby splitting the characters into different
segments [4].
E. License Plate Character Recognition
This is the last stage in the license plate recognition
process. In this stage, characters embedded in the images of
(b) Sl = 0:5 and S = 1
license plates are converted into texts. In [23], character
recognition with a high degree of accuracy was achieved by
using Convolution Neural Network for feature extraction. In
[5], a self-synthesized CNN was engaged for recognizing
optical characters with an accuracy of 90%. This was achieved
with a small amount of training dataset, hence the study has
(c) Sl = 1 and S = 1 proved that with an augmented dataset, a robust and effective
outcome can be achieved.
Fig. 2:”CNN output scores for two neighboring partitions” [3]
In [24], after extracting the localized license plate from
[4], employed a two-dimensional wavelet transform to vehicle images, CNN was applied to improve the character
extract the vertical edges of the acquired image of a car. Then recognition of blurred and obscure images. The results
the edges were computed to detect the possible area of the showed a better performance in comparison with a traditional
license plate. The verification of the plate and non-plate area license plate recognition system. A CNN classifier was used
was achieved using a CNN classifier. to recognize the characters of the segmented license plate
In [19], some existing CNN-related techniques for image characters with an accuracy of 98.9% in [4].
classification were organized, techniques that can be [25], utilized a pre-trained OCR engine for character
assembled into a single CNN were also highlighted. They recognition. A sstandard optical character recognition (OCR)
merged three different architectures, Inception, which brought pipeline was used in [15], and an accuracy of 93% was
new modules into the network with convolution layers. obtained. In [26], a tesseract training sheet was created using
ResNet, which used the idea of skip connection, and "prepcharsfortraining” utility program in Open ALPR, this is
DenseNet, which added dense feature connections to enhance also an arm of CNN for Optical Character Recognition.
the model performance. Methods that prevent overfitting by
increasing the training data through data augmentation
processes such as “AutoAugment” and “Mixup” were
engaged in [19]. The complexity of CNN was reduced using III. STRENGTH AND AREAS OF IMPROVEMENT OF SOME
“DropBlock” and Dropout. SELECTED CNN-BASED TECHNIQUES
Some CNN-based techniques of interest were examined
Region-Based CNN (R-CNN) was employed in [20]. The
and their strength and possible areas of improvement were
system used a “selective search (SS) approach to propose
analyzed as shown in table I.
around 2000 Regions of Interest (ROI)”. The regions served
as the input of the CNN which gives extracted features as TABLE I. STRENGTH AND POSSIBLE AREAS OF IMPROVEMENT ON
output. In [21], Region Proposal Network (RPN) was used in SOME SELECTED CNN-BASED TECHNIQUES
the Inception V2 architecture of Faster R-CNN. The RPN is
Ref. Techniques Strength Possible areas of improvement
positioned next to the final layer of the CNN. The proposals
[21], Faster R- The usage of Simultaneous usage of all the
from RPN were sent into the ROI pooling layer before feeding [20] CNN inception V2 networks to define the offsets of
in the Classifier and Bbox Regressor. The RPN is an end-to- as a module in the bounding boxes can help
end trained convolutional network that predicts object CNN shows increase the recall ratio.
boundaries and object scores for each detection [20]. almost perfect Also, considering RoI Align to
results. replace the RoI Pooling in the
The YOLO-based network was used in [22], the network Region Proposal Network will
is considered to be the fastest for object detection. A new improve the system localization
CNN-based network named Capsule Network was introduced accuracy.
[13], YOLO- Good at The usage of multi-model
in [1]. The network consists of three major building blocks [22], based CNN eliminating learning methods will improve
namely convolution, pooling, and fully connected layer. [2], false the system.
YOLO single convolutional neural network was used in [2] to [18] background Also, training the model on both
localize license plates. detections. crowdsourced and web-mined
images will improve the
D. License Plate Character Segmentation performance.
[5] Self- Shows perfect The model can further be made
This involves the isolation of individual characters in the
synthesized results for robust by using deeper
extracted license plate. Various techniques can be used for LP CNN high architecture and training
character segmentation, such as; morphological operations, resolution and procedures.
bounding box, CNN, improved blob detection algorithm, good quality
connected component analysis, etc. images.
[26], CNN-based Works best The addition of a recurrent layer
In [15], the segmentation process was carried out in three [15] OCR using with clear in the model will improve the
phases. The first phase is the character height estimation using Tesseract images performance.
lower and upper boundaries. The second phase is character Also, some careful and
painstaking parameters tuning at
width estimation, and the third phase is character labeling the image post-processing stage
using a block extraction algorithm. The method is based on will improve the performance.
the separation of empty spaces of individual characters on the

1531
IV. CONCLUSION AND FUTURE RESEARCH AREAS Journal of Advanced Research in Computer and
In this paper, an overview of some advances in CNN- Communication Engineering, vol. 4, no. 11, 2015.
based license plate recognition system was presented. The [12] I. B. Oluchi, A. E. Adewale and M. B. Mua’zu,
study analyzed the individual stage of the license plate "Development of a Nigeria Vehicle License Plate
recognition system presented in related works. The strength Detection System," APPLICATIONS OF
and possible areas of improvement on some of the selected MODELLING AND SIMULATION, 2019.
CNN-techniques were discussed. Despite the current
developments, further improvements can still be achieved, [13] S. Larsson and F. Mellqvist, "Automatic Number
such as; deeper learning frameworks for object detection and Plate Recognition for Android," The author(s) and
recognition, unsupervised object detection, and recognition, Karlstad University, 2019.
fast coding programming frameworks, universal framework [14] T. Ibrahim and K. Kirami, "License Plate Recognition
for general CNN, deep CNN models with the capacity to System Using Artificial Neural Networks," ETRI
recognize objects from partly observed data and reliable Journal , 2017.
interpretation of objects in complex conditions. [15] U. Farman, A. Hafeez and S. Iram, "Barrier Access
Control Using Sensors Platform and Vehicles License
ACKNOWLEDGEMENT Plate Characters Recognition," Sensors MDPI, 2019.
This work was supported by the management of Afe Babalola [16] H. Jorgensen, "Automatic License Plate Recognition,"
University, Ado Ekiti, Nigeria. Norwegian University of Science and Technology,
2017.
REFERENCES
[17] Avinash, Bhujbal; Deepak, Mane, "A Survey On Deep
Learning Approaches For Vehicle And Number Plate
[1] B. S. K, V. S and V. V, "Perspective Vehicle License Detection," INTERNATIONAL JOURNAL OF
Plate Transformation using Deep Neural Network on SCIENTIFIC & TECHNOLOGY RESEARCH, 2019.
Genesis of CPNet," in Elsevier, 2020.
[18] M. F. I. Diogo, P. A. F. Antonyus, R. M. Heitor and
[2] J. Yonten, R. Panomkhawn and W. Rattapoom, "Real- N. d. S. B. Edna N. da, "An embedded automatic
time Bhutanese license plate localization using license plate recognition system using deep learning,"
YOLO," Elsevier, 2020. Springer, November 2019.
[3] D. K. Francisco and M. Rodrigo, [19] L. Jungkyu, W. Taeryun, K. L. Tae, L. Hyemin, G.
"CONVOLUTIONAL NEURAL NETWORKS FOR Geonmo and H. Kiho, "Compounding the
LICENSE PLATE DETECTION IN IMAGES," in Performance Improvements of Assembled Techniques
IEEE, 2017. in a Convolutional Neural Network," in IEEE, 2020.
[4] S. Ibtissam, Z. Abdelmoghit, A. O. Wahban, A. Issam [20] R. Shaoqing, H. Kaiming, G. Ross and S. Jian ,
and H. Abdellatif, "An automated license plate "Faster R-CNN: Towards Real-Time Object Detection
detection and recognition system based on wavelet with Region Proposal Networks," in IEEE, 2016.
decomposition and CNN," Elsevier, September 2020.
[21] A. Derry and F. Muhammad, "Faster R-CNN with
[5] Madhusree, Mondal; Parmita, Mondal, "Automatic Inception V2 for Fingertip Detection in Homogenous
Number Plate Recognition Using CNN Based Self Background Image," Journal of Physics: Conference
Synthesized Feature Learning," in IEEE, 2017. Series, 2019.
[6] Surajit, Das; Joydeep, Mukherjee, "Automatic License [22] M. S. Sergio and R. J. Claudio, "Real-time license
Plate Recognition Technique using Convolutional plate detection and recognition using deep
Neural Network," International Journal of Computer convolutional neural networks," Elsevier, March
Applications, vol. 169, 2017. 2020.
[7] T. Joseph and R. D. Nadiab, "Plate Recognition Using [23] W. Chuin-Mu and L. Jian-Hong, "License Plate
Backpropagation Neural Network and Genetic Recognition System," in IEEE, 2015.
Algorithm," in Elsevier Procedia Computer Science,
2017. [24] C. Hung and Y. Sin, "An Efficient License Plate
Recognition System Using Convolution Neural
[8] K. Taranbir and K. S. Ravneet, "Performance Networks," in IEEE, 2018.
Evaluation of Fuzzy and Histogram Based Color
Image Enhancement," in Elsevier Procedia, 2015. [25] A. Parul, C. Kritika and K. Mohd, "Implementing
ALPR for detection of traffic violations: a step
[9] B. R , S. L. N. Vidya, S. Vivek and C. A, "A New towards sustainability," in Elsevier, 2018.
Methodology for License Plate Recognition," IETE
Journal of Research, 2015. [26] H. L. Nay, L. A. Yan and K. K. Win, "Automatic
Vehicle License Plate Recognition System for Smart
[10] I. Turkyılmaz and K. Kacan, "License Plate Transportation," in IEEE, 2018.
Recognition System Using Artificial Neural
Networks," ETRI Journal, vol. 39, 2017.
[11] D. G. Amusan, O. T. Arulogun and A. S. Falohun,
"Nigerian Vehicle License Plate Recognition System
using Artificial Neural Network," International

1532