Real Time Bangladeshi License Plate Detection &

Recognition
A project
Submitted in partial fulfillment of the requirements for the Degree of
Bachelor of Science in Computer Science and Engineering

Submitted by

Homaira Alam 160204102

Md. Saimom Islam 160204011
Md. Yeasir Arafat 160204093
Tauhidul Islam 160204031

Supervised by

Qamrun Nahar Eity

Department of Computer Science and Engineering

Ahsanullah University of Science and Technology

Dhaka, Bangladesh

December 2020
 Page |2

ABSTRACT
A number plate is a unique identification of a vehicle and is attached to a motor
vehicle for official recognition purposes. Automatic Number Plate Recognition
(ANPR) is an automatic system which capable of reading vehicle license plates
without human interference through the use of high speed image, detection of
characters within the given images, verification of the character patterns as being
those from a vehicle license plate, character recognition to convert an image to text.
All countries require license plates for road vehicles. Bangladesh Road Transport
Authority (BRTA) created a common standard for vehicle license plate. The size and
aspect ratio of all license plates are same. At 1st we took a video of vehicles. From
the video, detecting the vehicles and license plates is the first step of our system. We
have used 4 different models for this and compared the results.
 Page |3

Contents
1 Introduction 4

1.1 Study of Bangladeshi vehicle license plate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.2.1 Importance in reduction of vehicle theft & identifying stolen vehicles. . . . . . . . . . . . . . . . . 7

1.2.2 Importance in toll collection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.2.3 Importance in traffic management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.2.4 Importance in parking management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Literature Review 11

2.1 Related Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Methodology 12

3.1 Proposed work flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.2 Models used to train. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.2.1 Faster R-CNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.2.2 SSD (Single Shot Multibox Detector) Mobilenet V1 COCO . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.2.3 SSD (Single Shot Multibox Detector) Mobilenet V1 FPN COCO . . . . . . . . . . . . . . . . . . . . . 13

3.2.4 SSDlite Mobilenet V2 COCO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4 Data Collection and Processing 15

4.1 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.2 Data processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5 Experimental Result 17

6 Difficulties and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.1 Difficulties we have faced . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.2 Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

7 Future Works and Conclusion 23

7.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

7.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

References 24
 Page |4

Chapter 1

Introduction

1.1 Study of Bangladeshi vehicle license plate

Bangladesh Road Transport Authority which is known as BRTA, issues vehicle license plates for
motor vehicles in Bangladesh. The license plates in Bangladesh use the Bengali alphabet and
numerals. The general format of license plates in Bangladesh is "city - vehicle class letter and
number - vehicle number". For example:

"ঢাকা মেট্রা- গ ১১-৯৯৯৯”. Here, ঢাকা = Dhaka (city name), গ = registration letter (vehicle class),
১১ = registration number (vehicle class), ৯৯৯৯ = vehicle number.

Fig 1: A sample of BRTA standard License Plate

The license plates are placed in both the front and back of the vehicle. The letters used in the
license plate are:

অইউএকখগঘঙচছজঝতথঢডটঠদধনপফবভেযরলশসহ
 Page |5

The numerals that are permitted in the vehicle license plate are:

Each of the letters carries the identities of different vehicles. Such as:

অ Medium private goods (3.5 to 7.5 tons)

ই Agricultural vehicle – power tiller, tractor

Heavy private goods (7.5 to 22 tons) – bottle carrier, cargo truck (closed or open),
উ cargo van, dump truck, flatbed truck, low bed truck, pole carrier, refrigerated van,
tipper, panel van, demountable truck

এ Motorcycle (small, up to 50 cc)

ক Motor car (small, up to 1000 cc)

খ Motor car (medium, 1001 to 1300 cc)

গ Motor car (medium, 1301 to 2000 cc)

ঘ Private passenger (Crossover, SUV)

ঙ Private three wheeler tempo

চ Microbus for private service – microbus, human hauler, school bus

ছ Health service vehicle – ambulance, mobile dispensary

জ Minibus for public service – minibus, human hauler, touring coach, luxury coach,
school bus

ঝ Minibus for private service – minibus, human hauler, touring coach, luxury coach,
school bus

ট Heavy public goods (7.5 to 22 tons)

ঠ Vehicles for dual-purpose– pick-up with double cabin, pick-up with single cabin,
passenger van, security van, prison van, standard van, panel van, troop carriage,

ড Medium public goods

 Page |6

ঢ Private articulated vehicle

থ 4 stroke CNG Auto rickshaws

দ Private CNG Auto rickshaws

ন Light public goods (up to 3.5 tons)

প Taxi

ফ Public Auto tempo

ব Minibus public Service– small (31-45), medium (46-60), large(61-80), extra-large (81-
99), minibus (double-decker)

ভ Motor car (extra-large, 2001 cc & above)

Light private goods (up to 3.5 tons)

ে
Delivery vehicle (up to 2.5 tons) – delivery van, mini-truck, 3-wheel van, 3-wheel truck

য Any vehicle of Prime Minister's office

র Any vehicle of President's office

ল Motorcycle (large, over 125 cc) – moped, scooter, motor cycle

Vehicles for special purpose – cleaning vehicle, crash tender, concrete mixer/pumper,
শ earth digger/remover, foam tender, invalid carriage, ladder carrier, forklift, grander,
hearse, mobile crane, pay loader, refuse vehicle, tow truck

স Minibus for private service – small (31-45), medium (46-60), large (61-80), extra-large
(81-99), minibus (double-decker)

হ Motorcycle (medium, 51 to 125 cc)

 Page |7

1.2 Motivation

In recent years, automatic license plate detection has become more and more important due to
the growing number of vehicles on the roads. In the context of Bangladesh, ANPR can play a vital
role. Many problems like vehicle theft, traffic management, parking management, identifying
stolen vehicle etc. can be solved with ANPR system.

1.2.1 Importance in reduction of vehicle theft & identifying

stolen vehicles
As stated in the crime data of Dhaka Metropolitan Police (DMP), about 102 cases have been filed
at police stations over car theft in last three months. Twenty-three cars were stolen in July while
30 cars were stolen in August. In September, the gangs stole 49 cars. Sources at the Detective
Branch (DB) of police said around 20 car theft gangs are active in the capital and they steal 20-25
cars every month. Recently, instances of car theft have increased alarmingly. To reduce this
occurrence, if the vehicle is lost or stolen, the owner can notify the police and give the license
plate number to the them. Then the stolen vehicle can be detected using license plate detection
technique with their database images.

Fig 2: Detecting stolen vehicle

 Page |8

1.2.2 Importance in toll collection

The most challenging work faced by the travelers, is waiting in the toll plaza since manual work
can take more time and payment methods are not easier. It leads to wastage of fuel by waiting as
well as valuable time to a great scale which leads to traffic congestion on the express way paving
a way for air pollution. If the waiting time is long, it often results in drivers getting irritated which
can engage them in a petty quarrel over people and toll attendants. Collecting the tolls and
maintaining the records of different vehicles and transaction of money is a laborious process. By
using ANPR, the delay on taking tolls can be eliminated by cashless tolling and it is rapidly
becoming the most inventive technology for the travelers who pass through the toll plaza. The
cash payment is more difficult for collecting, transferring, recording and managing purpose since
fraudulent and Burglary are serious scenarios of manual payment methods.

Fig 6: ANPR for automatic toll collection

1.2.3 Importance in traffic management

In Bangladesh, the traffic condition is disorganized. As traffic jam causes a huge loss financially, it
is important to improve the traffic management system. ANPR can help to ensure mobility. It can
measure vehicle speed. It can accurately determine the average speed of vehicles travelling on a
 Page |9

section of road by identifying their license plate at both ends of a journey. When vehicles pass
through different road signs, the license plates are displayed with a safety message.

Fig 3: Safety message with license plate if a vehicle is over speed

If ANPR is used, it will simplify the manual labor and reduce the error opportunity. It can recognize
the license plate from vehicle in real time and this process is automatic and full time. So the need
of manpower for this purpose will reduce. If a vehicle is unregistered, it is not possible for the
traffic police to know that. So by using ANPR, it can be easily checked in database if the vehicle is
registered or not.

Fig 4: Detecting unregistered vehicle

Sometimes it becomes very difficult to identify for the traffic police to identify vehicle owner who
has violated the traffic rules or drive too fast. Therefore, it becomes impossible to catch and punish
them because the traffic police might not be able to retrieve license number from the moving
vehicle because of the speed. Therefore, Automatic Number Plate Recognition (ANPR) system can
be implemented as one of the solutions to this kind of problems.
 P a g e | 10

1.2.4 Importance in parking management

ANPR can be used in parking management. It can detect the license plate of vehicles entering and
exiting a car park. So the security for both parking operators and users can be improved. The traffic
flow during peak hours can also be improved by using ANPR.

Fig 5: ANPR in parking management

 P a g e | 11

Chapter 2

Literature Review

2.1 Related Work

In [1] for segmentation they used R-CNN model. For skewed license plate in [1] they generated
the Shi-Tomasi Corners reconstructed from Harris Corners of each license plate instance. YOLOv3
network to detect and localize license plate in [2] and [3]. In [2] they employed YOLOv3 by which
they detected and classified 10 Bangla digits and a Bangla character which is located at the end of
the first line in the license plate. In [3] they developed a CNN network based on ResNet-20
architecture for character recognition and also trained the network from scratch to classify 32
Bangla characters available in the license plate using our Bangla scene character dataset
(Character recognition). In [3] using YOLO, objects in an image simultaneously detected and
localized using only one convolutional network. Using this property, segmentation of words and
characters is done and classified at the same time if a license plate’s image is given to the network
(segmentation and Character recognition). In [4] the image is converted to grayscale and acted
upon with median filtering to reduce small noises. Template matching is used to recognize the
first line and second line characters (Character recognition). In [5] they used faster R-CNN and SSD
mobile net network to find number plate and to recognize the letter same way. In [6] CNN is used
for recognizing the number plate. In [7] and [11] contour algorithm is used to detect plate region.
In [8] Sobel method is used for edge detection, Haar feature based classifier is used in [10] to
detect license plate. HSI color model is used in [12] for license plate detection. Bounding Box
Parameters is used in [7] and [9], MATLAB function bwlabel to label used in [8], Haar feature based
classifier is used in [10], Otsu’s binaraization method is used in [11], Geometrical properties
(Bounding Box) used in [12]. CNN is used in [7] and [10], MATLAB Neural Network is used in [8],
Database is used in [9], Template matching is used in [11]. Intensity Histogram used in [12] for
candidate region verification.
 P a g e | 12

Chapter 3

Methodology

3.1 Proposed Work Flow

Fig 7: Work Flow

 P a g e | 13

3.2 Models Used to Train

We used four different models and compared each of their accuracy and speed.

3.2.1 Faster R-CNN

The accuracy of faster R-CNN is very high but the speed of it is very slow. So, in real time detection
and recognition it will not give desired output.

Training Parameter for Faster R-CNN:

batch_size: 1, initial Iearning rate: 0.0002, Steps: 100000.

3.2.2 SSD (Single Shot Multibox Detector) Mobilenet V1 COCO

The accuracy and speed of SSD Mobilenet V1 COCO is high. The accuracy may not be as high as faster
R-CNN but, it can be used for real time detection because of its faster speed.

Training Parameter for SSD (Single Shot Multibox Detector) Mobilenet V1 COCO:

batch_size: 24, initial Iearning rate: 0.004, Steps: 20000.

3.2.3 SSD (Single Shot Multibox Detector) Mobilenet V1 FPN COCO

As like faster R-CNN, the accuracy of SSD Mobilenet V1 FPN COCO is very high but it takes a lot of
time to detect an object which means the speed of it is very slow. In real time detection and
recognition, it will not give desired output. Because of this problem, it cannot be used for real time
detection.

Training Parameter for SSD (Single Shot Multibox Detector) Mobilenet V1 FPN COCO:

batch_size: 24, Iearning rate base: 0.4, Steps: 10000.

 P a g e | 14

3.2.4 SSDlite Mobilenet V2 COCO

The SSDlite Mobilenet V2 COCO generates multiple box for the same object after 10000 steps. This
problem was solved after 20000 steps. The speed of it is very high but the accuracy is a bit low
than the SSD Mobilenet V1 COCO. Because of this problem, it cannot be used for real time
detection.

Training Parameter for SSDlite Mobilenet V2 COCO:

batch_size: 24, initial Iearning rate: 0.004, Steps: 20000.

 P a g e | 15

Chapter 4

Data Collection and Processing

4.1 Data Collection

More than 500 sample photos of cars and bikes were taken randomly from different angels of the
camera on running vehicles on the road to create the experiment dataset and test the models. As
we could not collect much image of buses, trucks etc. we did train our model only with cars, bikes
and their license plates. After deleting some blurry, rustic and vague samples, total 462 photos
were chosen to train the model. Fig. 8 shows some sample image data that were used to train the
model.

Fig 8: Some Screenshots from the videos we have taken

 P a g e | 16

4.2 Data Processing

To label and annotate the training and test images, LabelImg annotation tool was used which is
an open-source graphical image annotation tool. We did the labeling of the objects from images
as car, bike, license plate. The annotations were saved as XML files. The XML files were then
converted into CSV files. The CSV files were converted into tfrecord format. Finally, the tfrecord
files were used as input data to train the models. The proposed model was tested using few videos
and still images which were taken randomly from different angels of the camera on running
vehicles on the road. The model could be tested with some other test samples, but due to having
its good performance in each test in the license plate detection and the recognition of characters
and numbers on the license plate further tests were not done.
 P a g e | 17

Chapter 5

Experimental Result

In this chapter, we have added classification and localization loss graphs for each models that are
used to train the system. The classification graph shows the loss occurred while not being able to
detect each class. The localization graph shows loss occurred when the object was not detected.

Faster R-CNN:

Fig 9: Classification loss graph for Faster R-CNN

 P a g e | 18

Fig 10: Localization loss graph for Faster R-CNN

SSD (Single Shot Multibox Detector) Mobilenet V1 COCO:

Fig 11: Classification loss graph for SSD Mobilenet V1 COCO

 P a g e | 19

Fig 12: Localization loss graph for SSD Mobilenet V1 COCO

SSD (Single Shot Multibox Detector) Mobilenet V1 FPN COCO:

Fig 13: Classification loss graph for SSD Mobilenet V1 FPN COCO
 P a g e | 20

Fig 14: Localizaion loss graph for SSD Mobilenet V1 FPN COCO

SSDlite Mobilenet V2 COCO:

Fig 15: Classification loss graph for SSDlite Mobilenet V2 COCO

 P a g e | 21

Fig 16: Localization loss graph for SSDlite Mobilenet V2 COCO

After observing all the output of the models, we have decided to use SSD Mobilenet V1 COCO for
further work.
 P a g e | 22

Chapter 6

Difficulties and Limitations

6.1 Difficulties

We captured the images and videos using one camera. At first we tried to capture image and
videos in free road. But, due to the high speed, we were unable to capture license plate clearly.
So we had to go to traffic signals, speed breakers where vehicles slow down. Some of the vehicles
were missed because of other vehicles passing in front of them because we had only one camera.
We could not find dataset or collect enough images of buses, trucks etc.

6.1 Limitations

The video frame rate should be 25fps or less and the resolution of the video should be 720px. If it
is more than that, the image may get clearer, but the processing takes a lot of time which is not
feasible to use in real time. As we could not find datasets for bus, trucks, we could not train our
system with those. So for now, our system can detect cars and bikes and their license plates.
 P a g e | 23

Chapter 7

Future Works and Conclusion

In this final chapter, we attempt to present some of our plans that we intend to do in future. It is of course
far from being a complete system for ready to be used.

7.1 Future Works

After detecting and recognizing the license plate of the vehicles we want to implement few of its
applications if possible. We want to add stolen vehicle detection and the automatic toll collection feature.
For stolen vehicle detection, the owner of the vehicle will need to inform the police about the occurrence
and give the license plate number. If the vehicle with that license plate is detected anywhere, an alarm will
ring and the person who monitors all the cameras will get to know it. The roads, bridges or flyovers where
toll is needed, with the help of cameras the license plate will be detected and recognized. From the
database, the owner of the vehicle will be searched and will get notified about the toll amount and the last
date to pay the toll.

7.2 Conclusion

In this report, we have discussed how an ANPR system can be applied and how it can be made useful to
people. We have present a procedure for detecting and recognizing the license plate number for Bangla
license plates. As there is no public dataset to our knowledge about the Bangla license plate, we created
our own dataset consisting of license plates for vehicles. As deep neural network models mostly depend on
data, we believe that, a more diverse dataset for training our model will produce a better result for our test
dataset. Also, by using a diverse dataset, we will be able to classify more types of vehicles in the future
without much alteration of any part in our model.
 P a g e 24

References

[1] K. Roy et al., "An Analytical Approach for Enhancing the Automatic Detection and Recognition of
Skewed Bangla License Plates," 2019 International Conference on Bangla Speech and Language
Processing (ICBSLP), Sylhet, Bangladesh, 2019, pp. 1-4, doi: 10.1109/ICBSLP47725.2019.201528.

[2] S. Abdullah, M. Mahedi Hasan and S. Muhammad Saiful Islam, "YOLO-Based Three-Stage Network
for Bangla License Plate Recognition in Dhaka Metropolitan City," 2018 International Conference on
Bangla Speech and Language Processing (ICBSLP), Sylhet, 2018, pp. 1-6, doi:
10.1109/ICBSLP.2018.8554668.

[3] N. Saif et al., "Automatic License Plate Recognition System for Bangla License Plates using
Convolutional Neural Network," TENCON 2019 - 2019 IEEE Region 10 Conference (TENCON), Kochi,
India, 2019, pp. 925-930, doi: 10.1109/TENCON.2019.8929280.
[4] R. A. Baten, Z. Omair and U. Sikder, "Bangla license plate reader for metropolitan cities of Bangladesh
using template matching," 8th International Conference on Electrical and Computer Engineering,
Dhaka, 2014, pp. 776-779, doi: 10.1109/ICECE.2014.7026925.

[5] T. Islam and R. I. Rasel, "Real-Time Bangla License Plate Recognition System using Faster R-CNN and
SSD: A Deep Learning Application," 2019 IEEE International Conference on Robotics, Automation,
Artificial-intelligence and Internet-of-Things (RAAICON), Dhaka, Bangladesh, 2019, pp. 108-111, doi:
10.1109/RAAICON48939.2019.45.

[6] M. M. Shaifur Rahman, M. Mostakim, M. S. Nasrin and M. Z. Alom, "Bangla License Plate Recognition
Using Convolutional Neural Networks (CNN)," 2019 22nd International Conference on Computer and
Information Technology (ICCIT), Dhaka, Bangladesh, 2019, pp. 1-6, doi:
10.1109/ICCIT48885.2019.9038597.

[7] Abedin, M. Z., Nath, A. C., Dhar, P., Deb, K., & Hossain, M. S. (2017). License plate recognition system
based on contour properties and deep learning model. 2017 IEEE Region 10 Humanitarian
Technology Conference (R10-HTC). doi:10.1109/r10-htc.2017.8289029

[8] Mashuk, M. S., Majid, M. A., Basher, N., & Rahman, T. R. (2010). Automatic Detection of Bangla
Characters in Bangladeshi Car Registration Plates. 2010 Second International Conference on
Computational Intelligence, Modelling and Simulation. doi:10.1109/cimsim.2010.73

[9] Shahed, M. T., Udoy, M. R. I., Saha, B., Khan, A. I., & Subrina, S. (2017). Automatic Bengali number
plate reader. TENCON 2017 - 2017 IEEE Region 10 Conference. doi:10.1109/tencon.2017.8228070
 P a g e | 25

[10] Atikuzzaman, M., Asaduzzaman, M., & Islam, M. Z. (2019). Vehicle Number Plate Detection and
Categorization Using CNNs. 2019 International Conference on Sustainable Technologies for Industry
4.0 (STI). doi:10.1109/sti47673.2019.9068049

[11] Roy, A. C., Hossen, M. K., & Nag, D. (2016). License plate detection and character recognition system
for commercial vehicles based on morphological approach and template matching. 2016 3rd
International Conference on Electrical Engineering and Information Communication Technology
(ICEEICT). doi:10.1109/ceeict.2016.7873098

[12] Deb, K., Hossen, M. K., Khan, M. I., & Alam, M. R. (2012). Bangladeshi Vehicle License Plate Detection
method based on HSI color model and geometrical properties. 2012 7th International Forum on
Strategic Technology (IFOST). doi:10.1109/ifost.2012.6357650