EE538: Computing and Software for

Systems Engineers

Lecture 8: BFS and

Shortest Distance Algorithms

University of Southern California

Instructor: Arash Saifhashemi

USC, Copyright: Arash Saifhashemi, All rights reserved

 Breadth First Search

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Breadth First Search
● An algorithm to iterate or search through vertices in a
directed graph 2 5
1
● Algorithm Overview:
○ Start from an arbitrary vertex v (call it root) 3
○ Mark v as visited
○ Visit each neighbor of v that is not marked visited at the current
depth, before moving to the next depth.
4 6
■ It is different than topological sort!

Observations: ● 5, 2, 1
● We start from an empty set of marked nodes and keep adding to it.
● We visit all children of v before moving to v’s grandchildren,
hence the name! (Different than DFS)
● We mark each node only once. We use each edge only once.
USC, Copyright: Arash Saifhashemi, All rights reserved
 void MapSetGraph::BFS(int root) {
BFS std::map<int, int> marks;
std::queue<int> q;
● Algorithm Overview: q.push(root);
○ Start from an arbitrary vertex v (call
marks[root] = 1;
it root)
○ Mark v as visited while (!q.empty()) {
○ Visit each neighbor of v that is not int cur = q.front();
marked visited at the current depth, q.pop();
before moving to the next depth.
for (auto &n : edge_map_[cur]) {
■ It is different than topological
sort! if (!marks[n]) {

● Use a Queue marks[n] = 1;

○ Track which child should visit first q.push(n);
● Notes: }
○ marks can be a map/vector/set/... }
}

USC, Copyright: Arash Saifhashemi, All rights reserved }

 void MapSetGraph::BFS(int root) {
std::map<int, int> marks;
std::queue<int> q;
q.push(root);
Initialize queue with root
and mark it as visited marks[root] = 1;
while (!q.empty()) {

Pop from queue int cur = q.front();

q.pop();
for (auto &n : edge_map_[cur]) {
if (!marks[n]) {
Enqueue unvisited marks[n] = 1;
children q.push(n);
}
}
}
}

USC, Copyright: Arash Saifhashemi, All rights reserved

 Marked Queue
Breadth First Search
0 0

0 0 1 2 5 5 2 1

0 1 2 5 3 3 5 2

2 5 0 1 2 5 3 3 5
1

3 0 1 2 5 3 3

0 1 2 5 3 4 6 6 4
4 6
0 1 2 5 3 4 6 6

0 1 2 5 3 4 6
USC, Copyright: Arash Saifhashemi, All rights reserved
 Breadth First Search 0

Observations:
1
● When we visit root’s children, we are 2
1
1 1 5
discovering all nodes at distance 1.
● When we visit children of a node at distance i, 3
2
we discover all nodes at distance i+1.
0: 0
1: 1
2: 1
3: 2 4 6
4: 3
5: 1
3 3
6: 3

In a graph where all edges have equal weights, BFS finds the shortest distance!

USC, Copyright: Arash Saifhashemi, All rights reserved

 void MapSetGraph::BFS(int root) {
Homework std::map<int, int> marks;
std::queue<int> q;
● 1. Enhance this code to calculate q.push(root);
the shortest distance for each marks[root] = 1;
node while (!q.empty()) {
● 2. Enhance this code to find the int cur = q.front();
actual shortest path from root to q.pop();
each node. for (auto &n : edge_map_[cur]) {
map(int, int) distance if (!marks[n]) {
distance[root] =0
marks[n] = 1;
distance[n] = distance[cur] + 1
q.push(n);
prev[n] = cur; }
}
}

USC, Copyright: Arash Saifhashemi, All rights reserved }

 0
BFS, More Observations
● Runtime is: m + n
● Can we get stuck in a cycle? 2 5
1
○ Marking saves us (Similar to DFS)
● It’s an iterative algorithm 3

● We find closer vertices first

● What is the maximum size of the queue?
● Applications: 4 6
○ Simply iterating the Graph
○ Find if a node is in the graph
○ Find if there is a path between two vertices
○ Finds shortest path in unweighted graphs (or when weights are all
equal)

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
DFS VS BFS
BFS DFS
2 5
Data Structure Queue Stack 1

3
Runtime O(m+n) O(m+n)

General usage Can be used for Can be used for

search and iteration search and iteration
4 6
Special Application Can find single Can be used for
source shortest path topological sort and
detecting cycles

Memory size Max memory: queue Max memory size:

size, i.e. max stack size, i.e. the
numbers in a level maximum distance
between root and
other nodes
USC, Copyright: Arash Saifhashemi, All rights reserved
 Grid Representation

USC, Copyright: Arash Saifhashemi, All rights reserved

 Grids and Graphs std::vector<std::vector<int>> grid = {
{1, 1, 1, 1},
{1, 0, 1, 1},
{1, 1, 0, 1},
{1, 0, 1, 1},
Start 1 1 1
};

1 0 1 1

1 1 0 1

1 0 1 Finish

Assuming you can only move vertical or horizontal (No diagonal).

Is there a path between Start and Finish?

Do we have a graph? If so, what is it?

USC, Copyright: Arash Saifhashemi, All rights reserved

 Grids and Graphs

● ● ● ●

● ● ● ●

V: { g[0][0], g[0][1], …, g[3][3]}

● ● ● ●
E: { {g[0][0], g[0][1]}, {g[0][0], g[1][0]}, … }

● ● ● ●

Each node has a maximum of four neighbors

USC, Copyright: Arash Saifhashemi, All rights reserved

 Grids and Graphs std::vector<std::vector<int>> grid = {
{1, 1, 1, 1},
{1, 0, 1, 1},
{1, 1, 0, 1},
{1, 0, 1, 1},
Start 1 1 1
};

1 0 1 1

1 1 0 1

1 0 1 Finish

Each node has a maximum of four neighbors

USC, Copyright: Arash Saifhashemi, All rights reserved

 Grids and Graphs std::vector<std::vector<int>> grid = {
{1, 1, 1, 1},
{1, 0, 1, 1},
{1, 1, 0, 1},
{1, 0, 1, 1},
Start 1 1 1
};

1 0 1 1

1 1 0 1

1 0 1 Finish

Assume the edges between nodes with 1 and nodes with 0

don’t exist.

USC, Copyright: Arash Saifhashemi, All rights reserved

 Grids and Graphs std::vector<std::vector<int>> grid = {
{1, 1, 1, 1},
{1, 0, 1, 1},
{1, 1, 0, 1},
{1, 0, 1, 1},
Start 1 1 1
};

1 0 1 1

1 1 0 1

1 0 1 Finish

Homework: Now, given the above graph (implicitly

represented by a grid), how can we find if a path
from Start to Finish exists?

USC, Copyright: Arash Saifhashemi, All rights reserved

 Shortest Path Algorithms

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Weighted Graph 1
3 10
● Weighted Graph: 6
● For each edge 𝑒 in E we associated a real number 2 5
1
1
𝑤(𝑒), called its weight.
1
● Unweighted Graph: 3
● Can be converted to a weighted graph with all 4 1
1
weights equal to 1.
4 6

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Shortest Path/Distance Algorithms 1
3 10
● The Shortest Distance problem (SDP): find the 6
shortest distance between nodes. 2 5
1

1 3

1 4 1
● The Shortest Path Problem (SPP): find the actual
shortest path between nodes. 4 6
○ Usually we can find path with minor enhancements in
SDP

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Path and Cycles 1
3 10
● A Path: 6
● A sequence of adjacent vertices 2 5
1
1
○ P=(v1, v2, …, vn) such that vi is adjacent to vi+1 for 1 ≤ i<n.
1
● A Cycle: A path where v1 = vn 3
● A Simple Path / Cycle: A Path with no repeated 4 1
1
vertices
● Except for v1 = vn in a simple cycle 4 6

Important:

● In the Shortest Distance/Path problem the path can be

either simple or non-simple.

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Negative Edge and Cycles -1
3 10
● Shortest path is meaningless if there is a negative -6
cycle 2 5
1
1
○ More precisely, a negative cycle is reachable from source
1
● We ignore the case where there is a zero cycle 3
● But we can still define shortest path if there are 4 1
1
negative edges
4 6

Remember:

● Negative edges (mostly) OK, negative cycles, not OK!

USC, Copyright: Arash Saifhashemi, All rights reserved

 -1 5 -1
Negative Edge and Cycles
1 2 3 4

● If there is no negative cycles: 3 3 3

○ Shortest path problem is really shortest No negative cycle, but negative edges exist
Shortest simple path: 1->2->5->3->4
simple path problem
Shortest path: 1->2->5->3->4
● Else:
○ There is NO shortest path. However, there
-10 5
-10
is a shortest simple path: 1->2->5->3->4
3 3
● Finding shortest simple path is
NP-Hard 1 2 3 4
● Max length of a simple path in a graph 3
=n-1 A negative cycle exists:
Shortest simple path: 1->2->5->3->4
Remember: Shortest path: Doesn’t exist

● If there is no negative cycles, there is no point to include a

cycle in the shortest path.
USC, Copyright: Arash Saifhashemi, All rights reserved
 0
Algorithms We Cover 1
3 10
6
Shortest Path 2 5
Algorithms
1
1
1 3

Single Source All Pairs 1 4 1

● Dijkstra ● Floyd–Warshall 4 6

● Bellman-Ford

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Floyd-Warshall 1
3 10
● Given that we are doing All Pair analysis, what is a
6
good data structure? 1
2 5
1
0 1 2 3 4 5 6 1 3
0 0 3 10 ∞ ∞ 1 ∞
1 4 1
1 3 0 6 1 ∞ ∞ ∞
2 10 6 0 1 ∞ ∞ ∞
3 ∞ 1 1 0 1 ∞ 4 4 6
4 ∞ ∞ ∞ 1 0 ∞ ∞
5 1 ∞ ∞ ∞ ∞ 0 1
6 ∞ ∞ ∞ 4 ∞ 1 0

We set the distance of each node

to itself to 0

USC, Copyright: Arash Saifhashemi, All rights reserved

 1 1
0 1
3 3

i j
9

d(i,j)= 9

d(i,j)= min (9, d(i,0) + d(0,j)) = min (9, 3 + 3) = 6

d(i,j)= min (6, d(i,1) + d(1,j)) = min (6, 4 + 1) = 5

USC, Copyright: Arash Saifhashemi, All rights reserved

 1 1
0 1
3 3

i j
9

We assume we can use {} as intermediate nodes

We assume we can use {0} as intermediate nodes

We assume we can use {0,1} as intermediate nodes

We assume we can use {0,1, …, n-1} as intermediate nodes

USC, Copyright: Arash Saifhashemi, All rights reserved

 1 1
0 1
3 3

i j
9

d(i,j,{})= 9

d(i,j,{0})= min(9, d(i,0,{}) + d(0,j,{})) = min(9, 3 + 3)= 6

d(i,j,{0,1})= min(6, d(i,1,{0}) + d(1,j,{0}))= min(6,4 + 1) = 5

USC, Copyright: Arash Saifhashemi, All rights reserved

 d(i,j,k):
Shortest distance from i to j
using nodes 0 to k, i.e {0, …k}.

i j

d(i,j,k)

USC, Copyright: Arash Saifhashemi, All rights reserved

 The moment that we allow the kth node:

1) k d(k
, k - ,j,
k
d (i, k-1
)

i j
d(i,j,k-1)

d(i,j,k) = min(d(i,j,k-1),

d(i,k,k-1)+ d(k,j,k-1))
USC, Copyright: Arash Saifhashemi, All rights reserved
 0
Example: 1
3 10
6
0 1 2 3 4 5 6 2 5
1
0 0 3 10 ∞ ∞ 1 ∞ 1
1 3 0 6 1 ∞ ∞ ∞ 1 3

2 10 6 0 1 ∞ ∞ ∞ 1 4 1
3 ∞ 1 1 0 1 ∞ 4
4 6
4 ∞ ∞ ∞ 1 0 ∞ ∞
5 1 ∞ ∞ ∞ ∞ 0 1
6 ∞ ∞ ∞ 4 ∞ 1 0

● If there is no edge between i, j, set the distance to ∞

● We set the distance of each node to itself to 0
USC, Copyright: Arash Saifhashemi, All rights reserved
 0
Floyd-Warshall 1
3 10
Base Case: shortestPath(i,j, k<0) = w(i,j)
Recursive Case: shortestPath(i,j, k) =
6
2 5
1
Min(shortestPath(i,j, k-1), 1
shortestPath(i,k, k-1) + shortestPath(k,j, k-1) 1 3

1 4 1
)

0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6 4 6
0 0 3 10 ∞ ∞ 1 ∞ 0 0 3 10 ∞ ∞ 1 ∞ 0 0 3 9 4 ∞ 1 ∞
1 3 0 6 1 ∞ ∞ ∞ 1 3 0 6 1 ∞ 4 ∞ 1 3 0 6 1 ∞ 4 ∞
2 10 6 0 1 ∞ ∞ ∞ 2 10 6 0 1 ∞ 11 ∞ 2 9 6 0 1 ∞ 10 ∞
3 ∞ 1 1 0 1 ∞ 4 3 ∞ 1 1 0 1 ∞ 4 3 4 1 1 0 1 5 4
4 ∞ ∞ ∞ 1 0 ∞ ∞ 4 ∞ ∞ ∞ 1 0 ∞ ∞ 4 ∞ ∞ ∞ 1 0 ∞ ∞
5 1 ∞ ∞ ∞ ∞ 0 1 5 1 4 11 ∞ ∞ 0 1 5 1 4 10 5 ∞ 0 1
6 ∞ ∞ ∞ 4 ∞ 1 0 6 ∞ ∞ ∞ 4 ∞ 1 0 6 ∞ ∞ ∞ 4 ∞ 1 0

k=-1 k=0 k=1

(use no nodes)
USC, Copyright: Arash Saifhashemi, All rights reserved(use node 0) (use nodes {0, 1})
 0
Floyd-Warshall 1
● Intuition: 3 10
○ Find shortest path from node i to j using first k nodes. 6
○ Recursively update 2 5
1
■ shortestPath(i,j, -1): Use no nodes: {} 1
■ shortestPath(i,j, 0): Use {0} 3
1
■ shortestPath(i,j, 1): Use {0,1}
■ shortestPath(i,j, 2): Use {0,1,2} 1 4 1
■ shortestPath(i,j, n-1): Use {0,1, …, n-1}
● This is what we eventually want! 4 6

Base Case: shortestPath(i,j, -1) = w(i,j)

(k=-1 means shortest path from i to j using no other nodes)
Recursive Case: shortestPath (i,j,k) =
Min(shortestPath(i,j,k-1),

shortestPath (i,k,k-1)+shortestPath(k,j,k-1)
USC, Copyright: Arash Saifhashemi, All rights reserved
 Floyd-Warshall
// Calculates shortest distance between i,j using nodes 0 to k
long FloydWarshallRecursiveHelper(int i, int j, int k) {
long distance;
if (k < 0) {
return weight_[i][j];
} else {
return std::min(FloydWarshallRecursiveHelper(i, j, k - 1),
FloydWarshallRecursiveHelper(i, k, k - 1) +
FloydWarshallRecursiveHelper(k, j, k - 1));
}
}

USC, Copyright: Arash Saifhashemi, All rights reserved

 Floyd-Warshall (Recursive)
std::vector<std::vector<long>> FloydWarshallRecursive() {
std::vector<std::vector<long>> d(weight_.size(),
std::vector<long>(weight_.size()));
// Find all pair distances
for (int i = 0; i < weight_.size(); ++i) {
for (int j = 0; j < weight_.size(); ++j) {
d[i][j] = FloydWarshallRecursiveHelper(i, j, weight_.size() - 1);
}
}
return d;
}

USC, Copyright: Arash Saifhashemi, All rights reserved

 // Calculates shortest distance between i,j using nodes 0 to k

Floyd-Warshall long DistMatrixGraph::FloydWarshallRecursiveHelper(int i, int j, int k)

{
long distance;

● The Recursion Tree if (k < 0) {

return weight_[i][j];
○ i, j, k can each change
} else {
between 0 to N-1
return std::min(FloydWarshallRecursiveHelper(i, j, k - 1),
○ We can use dynamic
FloydWarshallRecursiveHelper(i, k, k - 1) +
programming! FloydWarshallRecursiveHelper(k, j, k - 1));
}
}

i, j, k

i, j, k-1 i, k, k-1 k, j, k-1

USC, Copyright: Arash Saifhashemi, All rights reserved

 Dynamic Programming...
Fib(n) = Fib(n-2) + Fib(n-1)
f(5)

f(3) f(4)

f(1) f(2) f(2) f(3)

f(1) f(0) f(1) f(0)

f(1) f(2)

f(1) f(0)

Overlapping Subproblems:

Subproblems are reused several times (Usually recursive functions)

USC, Copyright: Arash Saifhashemi, All rights reserved
 // Calculates shortest distance between i,j using nodes 0 to k

Floyd-Warshall long DistMatrixGraph::FloydWarshallRecursiveHelper(int i, int j, int k)

{
long distance;

● The Recursion Tree if (k < 0) {

return weight_[i][j];
○ i, j, k can each change
} else {
between 0 to N-1
return std::min(FloydWarshallRecursiveHelper(i, j, k - 1),
○ We can use dynamic
FloydWarshallRecursiveHelper(i, k, k - 1) +
programming! FloydWarshallRecursiveHelper(k, j, k - 1));
}
}

i, j, k=n-1

i, j, k-1 i, k, k-1 k, j, k-1

1,2,4 ... 1,2,4

... 1,2,4 ...

USC, Copyright: Arash Saifhashemi, All rights reserved

 Floyd-Warshall (Memo)
● Dynamic Programming (Memo):
long DistMatrixGraph::FloydWarshallRecursiveHelperMemo(
int i, int j, int k, std::map<std::vector<int>, long> &memo) {
if (k < 0) {
return weight_[i][j];
} else {
if (memo.count({i, j, k}) > 0) {
return memo[{i, j, k}];
}
memo[{i, j, k}] =
std::min(FloydWarshallRecursiveHelperMemo(i, j, k - 1, memo),
FloydWarshallRecursiveHelperMemo(i, k, k - 1, memo) +
FloydWarshallRecursiveHelperMemo(k, j, k - 1, memo));
return memo[{i, j, k}];
}
}

USC, Copyright: Arash Saifhashemi, All rights reserved

 Floyd-Warshall (Non-Recursive)
● Non-Recursive Version of F.W. is an instance of
○ Dynamic Programming (Tabulation Method)
std::vector<std::vector<long>> DistMatrixGraph::FloydWarshall() {
std::vector<std::vector<long>> d(weight_.size(),
std::vector<long>(weight_.size()));
// To generate the base case (using no nodes) d is first initialized to weights_
for (int i = 0; i < weight_.size(); ++i) {
for (int j = 0; j < weight_.size(); ++j) {
// 0 if i==j, and ∞ if no edge between them
d[i][j] = weight_[i][j];
}
}

for (int k = 0; k < weight_.size(); k++) {

for (int i = 0; i < weight_.size(); ++i) {
for (int j = 0; j < weight_.size(); ++j) {
d[i][j] = std::min(d[i][j], d[i][k] + d[k][j]);
}
}
}
return d;
}

USC, Copyright: Arash Saifhashemi, All rights reserved

 Floyd-Warshall (Negative Cycles)
● Remember that we set the distance of each node to
0 1 2 3 4 5 6
itself to 0. 0 0 3 10 ∞ ∞ 1 ∞
● As we iterate through k, what would happen to the 1 3 0 6 1 ∞ ∞ ∞
2 10 6 0 1 ∞ ∞ ∞
distance of a node to itself if there exists a negative 3 ∞ 1 1 0 1 ∞ 4
cycle? (homework) 4 ∞ ∞ ∞ 1 0 ∞ ∞
5 1 ∞ ∞ ∞ ∞ 0 1
6 ∞ ∞ ∞ 4 ∞ 1 0

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
More About Floyd-Warshall 1
3 10
6
Shortest Path 2 5
Algorithms
1
1
1 3
Single Source
All Pairs 1 4 1
All other nodes

● Dijkstra ● Floyd–Warshall 4 6

CAN handle negative edges

CAN report negative cycles
● Bellman-Ford Runtime: O(n^3)

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
Update
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1
Update
4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2
u=1 Visited={0,5,6} 0 3 10 6 ∞ 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2
u=1 Visited={0,5,6} 0 3 10 6 ∞ 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1
Update
4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2
u=1 Visited={0,5,6} 0 3 10 6 ∞ 1 2
u=3 Visited={0,5,6,1} 0 3 9 4 ∞ 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2
u=1 Visited={0,5,6} 0 3 10 6 ∞ 1 2
u=3 Visited={0,5,6,1} 0 3 9 4 ∞ 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

Update
4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2
u=1 Visited={0,5,6} 0 3 10 6 ∞ 1 2
u=3 Visited={0,5,6,1} 0 3 9 4 ∞ 1 2
u=2 Visited={0,5,6,1,3} 0 3 5 4 5 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2
u=1 Visited={0,5,6} 0 3 10 6 ∞ 1 2
u=3 Visited={0,5,6,1} 0 3 9 4 ∞ 1 2
u=2 Visited={0,5,6,1,3} 0 3 5 4 5 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2
u=1 Visited={0,5,6} 0 3 10 6 ∞ 1 2
u=3 Visited={0,5,6,1} 0 3 9 4 ∞ 1 2
u=2 Visited={0,5,6,1,3} 0 3 5 4 5 1 2
u=4 Visited={0,5,6,1,3,2} 0 3 5 4 5 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 0
Dijkstra 1
3 10
● Intuition: Grow a list of visited nodes:
6
○ Find the unvisited node u with min distance 2 5
1
○ Relax the distance of unvisited neighbors of u 1
○ Add u to the visited list and repeat
1 3
● Greedy Algorithm
○ Making the locally optimal choice at each stage 1 4 1

4 6
0 1 2 3 4 5 6
u=5 Visited={0} 0 3 10 ∞ ∞ 1 ∞
u=6 Visited={0,5} 0 3 10 ∞ ∞ 1 2
u=1 Visited={0,5,6} 0 3 10 6 ∞ 1 2
u=3 Visited={0,5,6,1} 0 3 9 4 ∞ 1 2
u=2 Visited={0,5,6,1,3} 0 3 5 4 5 1 2
u=4 Visited={0,5,6,1,3,2} 0 3 5 4 5 1 2
Visited={0,5,6,1,3,2,4} 0 3 5 4 5 1 2

USC, Copyright: Arash Saifhashemi, All rights reserved

 std::vector<long> DistMatrixGraph::Dijkstra(int source) {
std::unordered_map<int, bool> visited;
int visited_size = 0;
std::vector<long> d(weight_.size());

for (int i = 0; i < weight_.size(); i++) {

d[i] = weight_[source][i];
}

visited[source] = true;
visited_size++;

while (visited_size < weight_.size()) {

int u = FindMinInDButNotInVisited(d, visited);

visited[u] = true;
visited_size++;

for (int v = 0; v < weight_.size(); v++) {

if (weight_[u][v] != INT_MAX && visited[v] == false) {
d[v] = std::min(d[v], d[u] + weight_[u][v]);
}
}
}
return d;
}
USC, Copyright: Arash Saifhashemi, All rights reserved
 Dijkstra Algorithm
Unvisited nodes

Source: Wikipedia

USC, Copyright: Arash Saifhashemi, All rights reserved

 Dijkstra Algorithm
n times
O(n) with array
O(log n) with heap

Total of m times (accounting

for both loops at lines 11
and 16)

Total runtime= O(m + n^2) with array

O(mlong + nlogn) with heap

N(v1)+N(v2)+...N(vn)= m
Source: Wikipedia

USC, Copyright: Arash Saifhashemi, All rights reserved

 Dijkstra: Negative edges
● Greedy Approach:
○ Dijkstra visits the closest node and then locks it inside visited
2 1
nodes.
○ It expects the distances to grow. Therefore, it cannot handle
s -4
negative edges.
int u = FindMinInDButNotInVisited(d, visited);
3 2
At this moment in time,
which node do we select for u?
s 1 2
... ... ...
u=? Visited={0} 0 2 3
Visited={0, 1} 0 2 -2

-1

Shortest path to 1 is through 2

USC, Copyright: Arash Saifhashemi, All rights reserved
 0
More About Dijkstra 1
3 10
Shortest Path
Algorithms
6
2 5
1
1
Single Source
All Pairs 3
All other nodes 1
● Dijkstra 1 4 1
● Floyd–Warshall
CANNOT handle negative edges
CANNOT handle negative cycles CAN handle negative edges 4 6
O(m + n^2) = O(n^2) OR CAN report negative cycles
O((m+n)log(n)) with heap Runtime: O(n^3)
● Bellman-Ford

USC, Copyright: Arash Saifhashemi, All rights reserved

 Bellman-Ford Algorithm

USC, Copyright: Arash Saifhashemi, All rights reserved

 Find shortest path using only 1 edge

Find shortest path using only 2 edges

Find shortest path using only 3 edges

Find shortest path using only n-1 edges

USC, Copyright: Arash Saifhashemi, All rights reserved

 7 Spend 1 edge
1 3 here
3 0 1
Source 3
1 1 1
S 4 6 5 2
1
9

i=1: d(2)= 9 (Distance of 2 using at most 1 edge)

i=2: d(2)= min(9, 7+1, 3+3, ∞+1) = 6

i=3: d(2)= min(6, 4+1, 6, ∞+1) = 5

i=4: d(2)= min(5, 5, 6, 3+1) = 4

USC, Copyright: Arash Saifhashemi, All rights reserved
 d(i, v): Shortest distance from s to v with at most i edges

Using d(i-1,v), how can we get to d(i, v) ?

i-1 + 1 = i

u 1)
1,
d (i- u1

d(i-1, u2)

w(
u2 w(

u1
u

,v
,v d(i, v) is the

)
2
s )
minimum of these
d(i-1, uk)
uk w( paths
u,
k v
)

d(i-1, v) v

USC, Copyright: Arash Saifhashemi, All rights reserved

 d(i, v) is the
u 1) u1 minimum of these
-1,
d(i w( paths
) w(
Bellman-Ford (Recursive) 1 , u2 u2 u
d(i- u
2 ,v
1 ,v
)
s d(i-1, uk) )

uk w(
u,
k v
)

Distance of s to
d(i-1, v) v
v using at most i
edges

if i=0, v=s
0 ;
(i.e. s to itself)

if i=0, v≠s
d(i,v) = ∞ ; (i.e. s to other nodes with
path of size 0)

min( d(i-1,v), d(i-1,u) + w(u,v));

(For edges (u,v) in E)

USC, Copyright: Arash Saifhashemi, All rights reserved

 2

s
2
t v
-10

f(1,v) = 2

f(2,v) = min(2, f(1,t) -10)= -8

USC, Copyright: Arash Saifhashemi, All rights reserved

 Bellman-Ford (Recursive)
long DistMatrixGraph::BellmanFordRecursiveHelper(int s, int i, int v) {
if (i == 0) {
return (v == s) ? 0 : INT_MAX;
} else {
auto pre = GetPredecessors(); // Map of node to its predecessors

long d = INT_MAX;
for (auto u : pre[v]) {
d = std::min(d, BellmanFordRecursiveHelper(s, i - 1, u) + weight_[u][v]);
}
return std::min(BellmanFordRecursiveHelper(s, i - 1, v), d);
}
}

USC, Copyright: Arash Saifhashemi, All rights reserved

 Bellman-Ford (Recursion Tree)
if i=0, v=s
0 ;

d(i,v) =
if i=0, v≠s
∞ ;

min( d(i-1,v), d(i-1,u) + w(u,v));

(For edges (u,v) in E)

i, v

i-1, v i - 1, u1 … i - 1, uk

USC, Copyright: Arash Saifhashemi, All rights reserved

 We calculate this
too many times!

u 1)
1,
d (i- u1 v1

d(i-1, u2)

w(
u2 w(

u1
u

,v
,v v2

)
2
s )
d(i-1, uk)
uk w(
u,
k v
)

d(i-1, v) vj

USC, Copyright: Arash Saifhashemi, All rights reserved

 Bellman-Ford (Homework: Add memo to this)
long DistMatrixGraph::BellmanFordRecursiveHelper(int s, int i, int v) {
if (i == 0) {
return (v == s) ? 0 : INT_MAX;
} else {
auto pre = GetPredecessors(); // Map of node to its predecessors

long d = INT_MAX;
for (auto u : pre[v]) {
d = std::min(d, BellmanFordRecursiveHelper(s, i - 1, u) + weight_[u][v]);
}
return std::min(BellmanFordRecursiveHelper(s, i - 1, v), d);
}
}

USC, Copyright: Arash Saifhashemi, All rights reserved

 Idea:

Instead of recalculating d(i,v) we create a 2D table:

If there is no negative cycles,
● i = 0 to n-1 shortest path is at most of size n-1
● v∈V

USC, Copyright: Arash Saifhashemi, All rights reserved

 Bellman-Ford Iterative with 2D Table
std::vector<long> DistMatrixGraph::BellmanFord2D(int source) {
std::vector<std::vector<long>> d(weight_.size() - 1, std::vector<long>(weight_.size()));
auto pre = GetPredecessors(); // Map of node to its predecessors
for (int i = 0; i < weight_.size() - 1; i++) {
for (int v = 0; v < weight_.size(); v++) {
if(i==0){ // Base case
d[0][v] = (v == source)? 0: INT_MAX;
continue;
}
d[i][v] = INT_MAX;
for (int u : pre[v]) {
std::vector<long> compareList = {d[i][v], d[i - 1][v], d[i - 1][u] + weight_[u][v]};
d[i][v] = *std::min_element(compareList.begin(), compareList.end());
}
}
}
return d[weight_.size() - 2];
USC,
} Copyright: Arash Saifhashemi, All rights reserved
 for (int i = 0; i < weight_.size() - 1; i++) {
for (int v = 0; v < weight_.size(); v++) {
// (Base case not shown)
d[i][v] = INT_MAX;
for (int u : pre[v]) {
std::vector<long> compareList = {d[i][v], d[i - 1][v], d[i - 1][u] + weight_[u][v]};
d[i][v] = *std::min_element(compareList.begin(), compareList.end());
}
}
}
return d[weight_.size() - 2]; // row n-1

USC, Copyright: Arash Saifhashemi, All rights reserved

 i=1
i=1

i=2
i=2

... ...

i=n-1
i=n-1

Overwrite
2D Table
a = a + 1 1D Table
USC, Copyright: Arash Saifhashemi, All rights reserved
 Using 1D Table
std::vector<long> DistMatrixGraph::BellmanFord(int source) {
std::vector<long> d(weight_.size(), INT_MAX);
auto pre = GetPredecessors(); // Map of node to its predecessors

d[source] = 0;

for (int i = 0; i < weight_.size() - 1; i++) {

for (int v = 0; v < weight_.size(); v++) {
for (int u : pre[v]) {
std::vector<long> compareList = {d[v], d[u] + weight_[u][v]};
d[v] = *std::min_element(compareList.begin(), compareList.end());
}
}
}
return d;
}
USC, Copyright: Arash Saifhashemi, All rights reserved
 Using 1D Table We are iterating all incoming
edges, so runtime is O(m)
Total runtime is O(n*m)

Source: Wikipedia
USC, Copyright: Arash Saifhashemi, All rights reserved
 Summary
Shortest Path
Algorithms

Single Source
All Pairs
All other nodes All weights equal

● Dijkstra
● Floyd–Warshall
CANNOT handle negative edges ● BFS
CANNOT handle negative cycles CAN handle negative edges CANNOT handle
O(n^2) OR O((m+n)log(n)) with heap CAN report negative cycles negative edges
Runtime: O(n^3) CANNOT report
● Bellman-Ford negative cycles
Runtime: O(m+n)
CAN handle negative edges
CAN report negative cycles
Runtime: O(m*n)

USC, Copyright: Arash Saifhashemi, All rights reserved