Smart Control of Trafficlight Using Artificial Intellegence-Gandhi2020
Smart Control of Trafficlight Using Artificial Intellegence-Gandhi2020
Abstract—Traffic congestion is becoming one of the critical allotted for a required area to control traffic. The
issues with increasing population and automobiles in cities. traffic police carry signboard, sign light, and whistle
Traffic jams not only cause extra delay and stress for the to control the traffic.
drivers, but also increase fuel consumption and air pollution.
Although it seems to pervade everywhere, megacities are the 2) Conventional traffic lights with static timers: These
ones most affected by it. And its ever-increasing nature makes it are controlled by fixed timers. A constant numerical
necessary to calculate the road traffic density in real-time for value is loaded in the timer. The lights are
better signal control and effective traffic management. The automatically switching to red and green based on the
traffic controller is one of the critical factors affecting traffic timer value.
flow. Therefore, the need for optimizing traffic control to better
accommodate this increasing demand arises. Our proposed 3) Electronic Sensors: Another advanced method is
system aims to utilize live images from the cameras at traffic placing some loop detectors or proximity sensors on
junctions for traffic density calculation using image processing the road. This sensor gives data about the traffic on
and AI. It also focuses on the algorithm for switching the traffic the road. According to the sensor data, the traffic
lights based on the vehicle density to reduce congestion, thereby signals are controlled.
providing faster transit to people and reducing pollution.
These conventional methods face certain drawbacks. The
Keywords—Traffic control, Traffic light system, Traffic manual controlling system requires a large amount of
management, Intelligent transport systems, Smart surveillance, manpower. As there is poor strength of traffic police, we
Computer Vision, Machine Learning, Object detection, YOLO. cannot have them controlling traffic manually in all areas of a
city or town. So a better system to control the traffic is needed.
I. INTRODUCTION Static traffic controlling uses a traffic light with a timer for
With the increasing number of vehicles in urban areas, every phase, which is fixed and does not adapt according to
many road networks are facing problems with the capacity the real-time traffic on that road. While using electronic
drop of roads and the corresponding Level of Service. Many sensors i.e., proximity sensors or loop detectors, the accuracy
traffic-related issues occur because of traffic control systems and coverage are often in conflict because the collection of
on intersections that use fixed signal timers. They repeat the high-quality information is usually based on sophisticated and
same phase sequence and its duration with no changes. expensive technologies, and thus limited budget will reduce
Increased demand for road capacity also increases the need for the number of facilities. Moreover, due to the limited effective
new solutions for traffic control that can be found in the field range of most sensors, the total coverage on a network of
of Intelligent Transport Systems. facilities usually requires a lot of sensors.
Let us take the case study of Mumbai and Bangalore. In recent years, video monitoring and surveillance systems
Traffic flow in Bangalore is the worst in the world while have been extensively used in traffic management for security,
Mumbai is close behind in fourth position, according to a ramp metering, and providing information and updates to
report detailing the traffic situation in 416 cities across 57 travelers in real-time. The traffic density estimation and
countries. In Bangalore, a journey during rush-hour takes 71% vehicle classification can also be achieved using video
longer. In Mumbai, it is 65% longer [1]. monitoring systems, which can then be used to control the
timers of the traffic signals so as to optimize traffic flow and
There are three standard methods for traffic control that minimize congestion. Our proposed system aims to design a
are being used currently: traffic light controller based on Computer Vision that can
adapt to the current traffic situation. It uses live images from
1) Manual Controlling: As the name suggests, it requires
the CCTV cameras at traffic junctions for real-time traffic
manpower to control the traffic. The traffic police are
density calculation by detecting the number of vehicles at the
Authorized licensed use limited to: Rutgers University. Downloaded on May 17,2021 at 07:59:27 UTC from IEEE Xplore. Restrictions apply.
signal and setting the green signal time accordingly. The fails to prioritize the authorized emergency vehicles nor to
vehicles are classified as a car, bike, bus/truck, or rickshaw to detect accidents on the intersection.
obtain an accurate estimate of the green signal time. It uses
YOLO in order to detect the number of vehicles and then set Reference [8] reviews various techniques used for traffic
the timer of the traffic signal according to vehicle density in light management system. This paper observes that each
the corresponding direction. This helps to optimize the green technique has a common architecture: choose input data,
signal times, and traffic is cleared at a much faster rate than a acquire traffic parameters from input data, process it,
static system, thus reducing the unwanted delays, congestion, determine density, and update parameters.
and waiting time, which in turn will reduce the fuel • In the first method, VANETS are used to get
consumption and pollution. information and location of every vehicle, which in
turn is passed on to the nearest Intelligent Traffic light
II. LITRATURE REVIEW with the help of installed GPS Further, these ITLs will
Reference [2] proposes a solution using video processing. update the statistics and sent it to nearby vehicles. In
The video from the live feed is processed before being sent to case of accidents, the information would be sent to
the servers where a C++ based algorithm is used to generate drivers to choose an alternate route to avoid
the results. Hard code and Dynamic coded methodologies are congestion. However, this technique is not feasible as
compared, in which the dynamic algorithm showed an its deployment is quite expensive.
improvement of 35%.
• In the second method, infrared sensor-based
Reference [3] proposes an Arduino-UNO based system microcontrollers are used, which capture the unique ID
that aims to reduce traffic congestion and waiting time. This of every car using transmitter and receiver. In case of
system acquires images through the camera and then an emergency situation, vehicle’s radio frequency tags
processes the image in MATLAB, where the image is can be used to identify them and let other vehicles
converted to a threshold image by removing saturation and move. This method detects red light violations.
hues, and traffic density is calculated. Arduino and MATLAB However, this technique is not flexible due to the fact
are connected using USB and simulation packages, which are that infrared sensors need to be in sight.
preinstalled. Depending on traffic count and traffic density,
the Arduino sets the duration of green light for each lane. But • In the third method, fuzzy logic technique is used in
this method has several flaws. The cars often overlap with which two fuzzy logic controllers are used – one is to
each other and it is difficult to get a proper count of how many optimize the signal and the other controller is used to
vehicles are on the road. Moreover, different objects interfered extend the green phase of a road in an intersection. The
with the detection as they too were converted to black and sensors used to collect input data are video cameras
white and there was no way of making a distinction between that are placed at incoming and outgoing lines. The
regular objects like billboards, poles, and trees with vehicles. controller then utilizes the information collected
through these sensors to make optimal decisions and
Reference [4] proposes a fuzzy logic-controlled traffic minimize the goal function.
light that can be adapt to the current traffic situations. This
system makes use of two fuzzy controllers with 3 inputs and • In the fourth method, fuzzy logic is used, and the
one output for primary and secondary driveways. A system takes in the number of vehicles and the average
simulation was done using VISSIM and MATLAB and for speed of traffic flow in each direction as the input
low traffic density, it improved traffic conditions. parameters. The number of vehicles and the average
speed of traffic flow can be determined using sensors
Reference [5] proposes a smart traffic light system using placed on the road.
ANN and fuzzy controller. This system makes use of images
captured from cameras installed at traffic site. The image is • In the fifth method, photoelectric sensors are used,
first converted to a grayscale image before further which are set at some distance apart, that capture data
normalization. Then, segmentation is performed using sliding and send it to the traffic cabinet, which calculates the
window technique to count the cars irrespective of size and weight of each road and then set the traffic light
ANN is run through the segmented image, the output of which accordingly. However, the maintenance cost is quite
is used in fuzzy controller to set timers for red and green light high.
using crisp output. Results had an average error of 2% with • In the sixth method, video imaging is used to capture
execution time of 1.5 seconds. the data. Dynamic background subtraction and various
Reference [6] makes use of a support vector machine morphological operations are performed to capture a
algorithm along with image processing techniques. From live clear image of the vehicle. Every time a new vehicle
video. images in small frames are captured and the algorithm enters the area of interest, a new rectangle is drawn and
is applied. Image processing is done using OpenCV and the vehicle count is incremented. The algorithm is easy to
images are converted to grayscale images before SVM is implement but does not handle occlusion and shadow
applied. This system not only detects traffic density but also overlapping.
detects red light violations.
III. PROPOSED SYSTEM
Reference [7] proposes the use of adaptive light timer
control using image processing techniques and traffic density. A. Proposed System Overview
This system consists of microcontroller-controlled traffic light Our proposed system takes an image from the CCTV
timer, high image sensing devices, MATLAB and cameras at traffic junctions as input for real-time traffic
transmission using UART principles. However, this system density calculation using image processing and object
detection. As shown in Fig. 1, this image is passed on to the
Authorized licensed use limited to: Rutgers University. Downloaded on May 17,2021 at 07:59:27 UTC from IEEE Xplore. Restrictions apply.
vehicle detection algorithm, which uses YOLO. The number 1 accuracy and 91.2% top-5 accuracy on ImageNet. Darknet
of vehicles of each class, such as car, bike, bus, and truck, is uses mostly 3 × 3 filters to extract features and 1 × 1 filters to
detected, which is to calculate the density of traffic. The signal reduce output channels. It also uses global average pooling to
switching algorithm uses this density, among some other make predictions [10].
factors, to set the green signal timer for each lane. The red
signal times are updated accordingly. The green signal time is The dataset for training the model was prepared by
restricted to a maximum and minimum value in order to avoid scraping images from google and labelling them manually
starvation of a particular lane. A simulation is also developed using LabelIMG, a graphical image annotation tool [12]. Then
to demonstrate the system’s effectiveness and compare it with the model was trained using the pre-trained weights
the existing static system. downloaded from the YOLO website. The configuration of
the .cfg file used for training was changed in accordance with
B. Vehicle Detection Module the specifications of our model. The number of output neurons
The proposed system uses YOLO (You only look in the last layer was set equal to the number of classes the
once) for vehicle detection, which provides the desired model is supposed to detect by changing the 'classes' variable.
accuracy and processing time. A custom YOLO model was In our system, this was 4 viz. Car, Bike, Bus/Truck, and
trained for vehicle detection, which can detect vehicles of Rickshaw. The number of filters also needs to be changed by
different classes like cars, bikes, heavy vehicles (buses and the formula 5*(5+number of classes), i.e., 45 in our case. After
trucks), and rickshaws. making these configuration changes, the model was trained
until the loss was significantly less and no longer seemed to
reduce. This marked the end of the training, and the weights
were now updated according to our requirements. These
weights were then imported in code and used for vehicle
detection with the help of OpenCV library. A threshold is set
as the minimum confidence required for successful detection.
After the model is loaded and an image is fed to the model, it
gives the result in a JSON format i.e., in the form of key-value
pairs, in which labels are keys, and their confidence and
coordinates are values. Again, OpenCV can be used to draw
the bounding boxes on the images from the labels and
coordinates received.
Fig. 2 shows test images on which our vehicle detection
model was applied. The left side of the figure shows the
original image and the right side is the output after the vehicle
detection model is applied on the image, with bounding boxes
and corressponding labels
Authorized licensed use limited to: Rutgers University. Downloaded on May 17,2021 at 07:59:27 UTC from IEEE Xplore. Restrictions apply.
The algorithm takes the information about the vehicles that where:
were detected from the detection module, as explained in the
previous section, as input. This is in JSON format, with the • GST is green signal time
label of the object detected as the key and the confidence and • noOfVehiclesOfClass is the number of vehicles of
coordinates as the values. This input is then parsed to calculate each class of vehicle at the signal as detected by the
the total number of vehicles of each class. After this, the green vehicle detection module,
signal time for the signal is calculated and assigned to it, and
the red signal times of other signals are adjusted accordingly. • averageTimeOfClass is the average time the vehicles
The algorithm can be scaled up or down to any number of of that class take to cross an intersection, and
signals at an intersection.
• noOfLanes is the number of lanes at the intersection.
The following factors were considered while developing the
The average time each class of vehicle takes to cross an
algorithm:
intersection can be set according to the location, i.e., region-
1) The processing time of the algorithm to calculate wise, city-wise, locality-wise, or even intersection-wise based
traffic density and then the green light duration – this on the characteristics of the intersection, to make traffic
decides at what time the image needs to be acquired management more effective. Data from the respective
transport authorities can be analyzed for this.
2) Number of lanes
The signals switch in a cyclic fashion and not according to
3) Total count of vehicles of each class like cars, trucks, the densest direction first. This is in accordance with the
motorcycles, etc. current system where the signals turn green one after the other
4) Traffic density calculated using the above factors in a fixed pattern and does not need the people to alter their
ways or cause any confusion. The order of signals is also the
5) Time added due to lag each vehicle suffers during same as the current system, and the yellow signals have been
start-up and the non-linear increase in lag suffered by accounted for as well.
the vehicles which are at the back [13]
Order of signals: Red → Green → Yellow → Red
6) The average speed of each class of vehicle when the
green light starts i.e. the average time required to cross D. Simulation Module
the signal by each class of vehicle [14] A simulation was developed from scratch using Pygame
7) The minimum and maximum time limit for the green to simulate real-life traffic. It assists in visualizing the system
light duration - to prevent starvation and comparing it with the existing static system. It contains a
4-way intersection with 4 traffic signals. Each signal has a
When the algorithm is first run, the default time is set for timer on top of it, which shows the time remaining for the
the first signal of the first cycle and the times for all other signal to switch from green to yellow, yellow to red, or red to
signals of the first cycle and all signals of the subsequent green. Each signal also has the number of vehicles that have
cycles are set by the algorithm. A separate thread is started crossed the intersection displayed beside it. Vehicles such as
which handles the detection of vehicles for each direction and cars, bikes, buses, trucks, and rickshaws come in from all
the main thread handles the timer of the current signal. When directions. In order to make the simulation more realistic,
the green light timer of the current signal (or the red light timer some of the vehicles in the rightmost lane turn to cross the
of the next green signal) reaches 5 seconds, the detection intersection. Whether a vehicle will turn or not is also set using
threads take the snapshot of the next direction. The result is random numbers when the vehicle is generated. It also
then parsed and the timer of the next green signal is set. All contains a timer that displays the time elapsed since the start
this happens in the background while the main thread is of the simulation. Fig. 3 shows a snapshot of the final output
counting down the timer of the current green signal. This of the simulation.
allows the assignment of the timer to be seamless and hence
prevents any lag. Once the green timer of the current signal Pygame is a cross-platform set of Python modules
becomes zero, the next signal becomes green for the amount designed for writing video games. It includes computer
of time set by the algorithm. graphics and sound libraries designed to be used with the
Python programming language. Pygame adds functionality on
The image is captured when the time of the signal that is
to turn green next is 5 seconds. This gives the system a total
of 10 seconds to process the image, to detect the number of
vehicles of each class present in the image, calculate the green
signal time, and accordingly set the times of this signal as well
as the red signal time of the next signal. To find the optimum
green signal time based on the number of vehicles of each
class at a signal, the average speeds of vehicles at startup and
their acceleration times were used, from which an estimate of
the average time each class of vehicle takes to cross an
intersection was found [14]. The green signal time is then
calculated using (1).
Authorized licensed use limited to: Rutgers University. Downloaded on May 17,2021 at 07:59:27 UTC from IEEE Xplore. Restrictions apply.
top of the excellent SDL library. This allows users to create TABLE II. SIMULATION RESULTS OF PROPOSED ADAPTIVE SYSTEM
fully featured games and multimedia programs in the python No. Distribution Lane1 Lane 2 Lane 3 Lane 4 Total
language. Pygame is highly portable and runs on nearly every
platform and operating system. It is free and licensed under 1 [300,600,800,1000] 87 109 41 50 287
LGPL [15]. 2 [500,700,900,1000] 128 55 49 25 257
10 [400,500,900,1000] 81 29 88 37 235
11 [200,400,600,1000] 42 47 54 86 229
12 [250,500,950,1000] 39 52 93 22 206
13 [850,900,950,1000] 74 10 13 17 114
14 [350,500,850,1000] 49 46 69 50 214
15 [350,700,850,1000] 51 64 37 43 195
Fig. 4. Comparison of current static system and proposed adaptive system
Authorized licensed use limited to: Rutgers University. Downloaded on May 17,2021 at 07:59:27 UTC from IEEE Xplore. Restrictions apply.
With all simulation conditions same i.e distribution of intersections offers tremendous benefits of saving
traffic, speeds of vehicles, probability of vehicles turning, the properties and lives and minimizing congestion and
gap between vehicles, and so on, the simulations were run for delay. This can be achieved by identifying the
a total period of 1 hour 15 minutes, with 300 seconds i.e. 5 vehicles that remain stationary for a long time in an
minutes for each distribution and it was found out that the inappropriate position such as in the middle of the
proposed system, on an average, increased the performance by road, so that parked vehicles are not included in this.
about 23% as compared to the current system with fixed times.
This implies a reduction in idle green signal time as well as 3) Synchronization of traffic signals across multiple
the waiting time of the vehicles. intersections: Synchronizing signals along a street can
benefit the commuters as once a vehicle enters the
On comparing these results with some alternative adaptive street, it may continue with minimal stopping. [16].
system, it was found that the proposed system performs better
than some of those. For example, [2] gives an accuracy of 70% 4) Adapting to emergency vehicles: Emergency vehicles
as compared to 80% of the proposed system. Reference [3] such as an ambulance need to be given quicker
achieves an average performance improvement of 12% as passage through the traffic signals. The model can be
compared to static systems while the proposed system trained to detect not just vehicles but also be able to
recognize that it is an emergency vehicle and
achieves 23% improvement.
accordingly adapt the timers such that the emergency
V. CONCLUSION AND FUTURE WORK vehicle is given priority and is able to cross the signal
at the earliest.
In conclusion, the proposed system sets the green signal
time adaptively according to the traffic density at the signal REFERENCES
and ensures that the direction with more traffic is allotted a [1] TomTom.com, 'Tom Tom World Traffic Index', 2019. [Online].
green signal for a longer duration of time as compared to the Available: https://www.tomtom.com/en_gb/traffic-index/ranking/
direction with lesser traffic. This will lower the unwanted [2] Khushi, "Smart Control of Traffic Light System using Image
delays and reduce congestion and waiting time, which in turn Processing," 2017 International Conference on Current Trends in
will reduce fuel consumption and pollution. Computer, Electrical, Electronics and Communication (CTCEEC),
Mysore, 2017, pp. 99-103, doi: 10.1109/CTCEEC.2017.8454966.
According to simulation results, the system shows about [3] A. Vogel, I. Oremović, R. Šimić and E. Ivanjko, "Improving Traffic
23% improvement over the current system in terms of the Light Control by Means of Fuzzy Logic," 2018 International
number of vehicles crossing the intersection, which is a Symposium ELMAR, Zadar, 2018, pp. 51-56, doi:
significant improvement. With further calibration using real- 10.23919/ELMAR.2018.8534692.
life CCTV data for training the model, this system can be [4] A. A. Zaid, Y. Suhweil and M. A. Yaman, "Smart controlling for traffic
light time," 2017 IEEE Jordan Conference on Applied Electrical
improved to perform even better. Engineering and Computing Technologies (AEECT), Aqaba, 2017, pp.
Moreover, the proposed system possesses certain 1-5, doi: 10.1109/AEECT.2017.8257768.
advantages over the existing intelligent traffic control systems [5] Renjith Soman "Traffic Light Control and Violation Detection Using
Image Processing”.” IOSR Journal of Engineering (IOSRJEN), vol. 08,
prevalent such as Pressure Mats and Infrared Sensors. The no. 4, 2018, pp. 23-27
cost required to deploy the system is negligible as footage [6] A. Kanungo, A. Sharma and C. Singla, "Smart traffic lights switching
from CCTV cameras from traffic signals is used, which and traffic density calculation using video processing," 2014 Recent
requires no additional hardware in most cases, as intersections Advances in Engineering and Computational Sciences (RAECS),
with heavy traffic are already equipped with such cameras. Chandigarh, 2014, pp. 1-6, doi: 10.1109/RAECS.2014.6799542.
Only minor alignment may need to be performed. The [7] Siddharth Srivastava, Subhadeep Chakraborty, Raj Kamal, Rahil,
maintenance cost also goes down as compared to other traffic Minocha, “Adaptive traffic light timer controller” , IIT KANPUR,
NERD MAGAZINE
monitoring systems such as pressure mats that normally suffer
wear and tear due to their placement on roads where they are [8] Ms. Saili Shinde, Prof. Sheetal Jagtap, Vishwakarma Institute Of
Technology, Intelligent traffic management system:a Review, IJIRST
subjected to immense pressure constantly. Thus, the proposed 2016
system can thus be integrated with the CCTV cameras in [9] Open Data Science, ‘Overview of the YOLO Object Detection
major cities in order to facilitate better management of traffic. Algorithm’, 2018. [Online]. Available: https://medium.com/@ODSC/
overview-of-the-yolo-object-detection-algorithm-7b52a745d3e0
The project can be further expanded to include the
[10] J. Hui, ‘Real-time Object Detection with YOLO, YOLOv2 and now
following functionalities to enhance traffic management and YOLOv3’, 2018. [Online]. Available: https://medium.com/ @jonathan
bring down congestion: _hui/ real-time-object-detection-with-yolo-yolov2-28b1b93e2088
1) Identification of vehicles violating traffic rules: The [11] J. Redmon, ‘Darknet: Open Source Neural Networks in C’, 2016.
[Online]. Available: https://pjreddie.com/darknet/
vehicles running red lights can be identified in an
[12] Tzutalin, ‘LabelImg Annotation Tool’, 2015. [Online]. Available:
image or a video stream by defining a violation line https://github.com/tzutalin/labelImg
and capturing the number plate of the image if that [13] Li, Z., Wang, B., and Zhang, J. “Comparative analysis of drivers' start‐
line is crossed when the signal is red. Lane changing up time of the first two vehicles at signalized intersections”, 2016 J.
can also be identified similarly. These can be achieved Adv. Transp., 50: 228– 239. doi: 10.1002/atr.1318
by background subtraction or image processing [14] Arkatkar, Shriniwas & Mitra, Sudeshna & Mathew, Tom. “India” in
techniques. Global Practices on Road Traffic Signal Control, ch.12, pp.217-242
[15] ‘Pygame Library’, 2019. [Online]. Available:
2) Accident or breakdown detection: Intersections also https://www.pygame.org/wiki/about
tend to experience severe crashes due to the fact that [16] ‘Traffic Signal Synchronization’. [Online]. Available:
several types of injurious crashes, such as angle and https://www.cityofirvine.org/signal-operations-maintenance/traffic-
left-turn collisions, commonly occur there. Therefore, signal-synchronization
accurate and prompt detection of accidents at
Authorized licensed use limited to: Rutgers University. Downloaded on May 17,2021 at 07:59:27 UTC from IEEE Xplore. Restrictions apply.