Optimizing Urban Traffic Flow Using Reinforcement

Learning Techniques

Presented by Supervised by

MD Zunaine Nashid Shakila Shafiq

DU Reg: 1574 Lecturer
MST. Rima Begum Dept. of CSE
NITER
DU Reg: 1578
MD Nasir Uddin
DU Reg: 1598

1
National Institute of Textile Engineering and Research (NITER)
 Presentation Outline
• Introduction
• Motivation
• Problem Statement
• Objective
• Related Works
• Proposed Methods
• Expected Outcome
• Limitations and Future Works
• Conclusion
• Reference 2

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
 Introduction
• Traffic Congestion is a growing problem in urban cities.
Reinforcement Learning (RL) offers a data-driven
solution to optimize traffic signals and improve efficiency.
• The Goal is to reduce waiting time, fuel consumption,
emissions and traffic congestion.

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
 Motivation
• In Bangladesh, Traffic congestion problem is a major
problem and it wastes most of our valuable times. For
this, we face big economy loss
• Traffic signals are totally inefficient for the roads of
Bangladesh
• We think we need an adaptive traffic control system to
optimize those issues

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
 Problem Statement
Introducing an AI model using RL algorithm for finding the
optimized way of controlling traffic with best efficiency and
low amount of computational cost with the aim of solving
the problem of static traffic control system which causes
congestion, emission, waiting time

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
 Objectives
• To optimize traffic signal control using RL
• To minimize key metrics: waiting time, queue lengths,
and congestion
• To compare RL results with traditional methods like
Fixed-Time and Actuated Control

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
 Literature Review
Serial Paper Title Techniques used Findings Limitations
No

Using a deep The traffic micro- Reduced average Struggles to scale

reinforcement learning simulator SUMO, cumulative delay effectively in large urban
1 agent for traffic signal CITYFLOW. by 82%, average networks.
control [1] queue length by
66% and average
travel time by
20%.

2 Multi-Agent Reinforcement Learning, The proposed MARL models often

Reinforcement Learning Multi-Agent approach can require large amounts of
for Traffic Signal Control: Reinforcement Learning, reduce fuel high-quality data for
ACooperative Approach Deep Q-Learning, Policy consumption by training, complex and
[2] Gradient Algorithm 11% and average computationally intensive.
travel time by
13%.

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
 Literature Review (Cont…)
Serial Paper Title Techniques used Findings Limitations
No

3 Deep Deterministic Deep Deterministic Policy The proposed The traffic demand has
Policy Gradient for Gradient algorithm, Deep approach can been created in a random
Urban Traffic Light Network Architecture reduce fuel manner, but ensuring
Control [3] consumption by enough vehicles are
10% and average present and trying to
travel time by collapse some of the
11%. sections of the network

4 Intelligent Traffic Signal Machine learning Works better than Can’t deal with unknown
Control System Using algorithms the traditional certainty and unexpected
Machine Learning traffic control environment
Techniques [4] system and
actuated traffic
control system

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
 Proposed Method
Data Collection
Simulated data (SUMO or CityFlow) or real-world sensor
data

State and Action Design

State: Vehicle counts, queue lengths, signal phases
Action: Adjust signal phase or duration

RL Algorithm
Use DQN or PPO (proximal policy optimization) or Multi-
Agent RL to train the model
9

National Institute of Textile Engineering and Research (NITER)

 Proposed Method

10

National
National
Institute
Institute
of Textile
of Textile
Engineering
Engineering
and and
Research
Research
(NITER)
(NITER)
 Expected Outcome
Result analysis of evaluation metrics such as
✓ Average Waiting Time
✓ Queue Lengths
✓ Vehicle Throughput

Comparative analysis of RL-based control with

baseline methods such as
✓ Fixed-Time Control
✓ Actuated Control

11

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
Limitations and Future Work
Current Limitations
• High computational requirements for RL training.
• Dependency on simulated data.

Future Work
• Multi-Agent RL for multiple intersections.
• Real-time integration with computer vision.

12

National Institute of Textile Engineering and Research (NITER)

 Conclusion

• RL can dynamically optimize traffic signals to reduce

congestion.
• It shows significant improvements over traditional
methods in reducing waiting time and congestion.
• Future work will focus on scalability and deployment in
real-world systems.

13

National Institute of Textile Engineering and Research (NITER)

 References
[1] Genders, W., & Razavi, S. (2016). Using a deep
reinforcement learning agent for traffic signal control. arXiv
preprint arXiv:1611.01142.
[2] Kolat, M., Kővári, B., Bécsi, T., & Aradi, S. (2023). Multi-agent
reinforcement learning for traffic signal control: A cooperative
approach. Sustainability, 15(4), 3479.
[3] Casas, N. (2017). Deep deterministic policy gradient for
urban traffic light control. arXiv preprint arXiv:1703.09035.
[4] Ali, M., Lavanya Devi, G., & Neelapu, R. (2021). Intelligent
traffic signal control system using machine learning techniques.
In Microelectronics, Electromagnetics and Telecommunications:
Proceedings of the Fifth ICMEET 2019 (pp. 611-619). Springer
Singapore.

14

National
NationalInstitute
InstituteofofTextile
TextileEngineering
Engineeringand
andResearch
Research(NITER)
(NITER)
 THANK YOU
Do you have any question?

15

National Institute of Textile Engineering and Research (NITER)