P T H I N K F A ST S
Competitive Programming
From Problem 2 Solution in O(1)
Graph Theory
Bellman-Ford Algorithm
Mostafa Saad Ibrahim
PhD Student @ Simon Fraser University
Recall: Dijkstra
◼ Solves Single-source shortest-paths (SSSP)
problem
◼ From one source s, find Shortest Path to all other nodes
◼ Dijkstra
◼ Greedy + nonnegative weighted graph
◼ 1st step: Pick non visited node with minimum cost
1
5 -4 ● Dijkstra pick shortest(0, 2) = [0, 2] = 4, WRONG
● shortest(0, 2) = [0, 1, 3, 2] = 5-4+1 = 2
0 3
4 2 1
Bellman-Ford Algorithm
◼ Solves SSSP, but graph can have negative
weights
◼ Why we may need -ve weights?
◼ Money transactions: -10$ = money you have to pay
◼ Games: -5 = lost 5 points for moving between states
◼ Some algorithms, need a -ve weight SSSP due to its nature
(e.g. Max Flow)
◼ If source can reach -ve cycle?
◼ All nodes affected by the cycle has no path from src
Cycles
2
1 -4
2 No cycles.
0 1 3 Bellman works
4 4 1
2
1 3
2 Positive cycle...reachable from 0
0 1 3 Bellman works
4 4 1
2
1 -4
2 Positive cycle...reachable from 0
0 1 3 Bellman works
4 4 5
Cycles
2
1 -9
2 Negative cycle of cost -5...reachable from 0
0 1 3 NO algorithms can work
2 4 1
2
1 -9
2 Negative cycle of cost -5...NOT reachable from 0
0 1 3 Bellman works
Cost for (1, 2, 3, 4) = OO
2 4 1
If added ege (7, 3) = -1, then -ve cycle is reachable
3
4 -6
5 6 7
Cycles
2
1 -9
2 Negative cycle of cost -5...reachable from 0
0 1 3 Bellman has no path to (1, 2, 3, 4)
But has to (5, 6, 7)
2 4 1
3
4 -6
5 6 7
Bellman-Ford Algorithm
◼ Fact: Simple path is at most n − 1 edges
◼ There are 2 popular ways to outline bellman
◼ Think in bellman as contrast to Dijsktra
◼ Relax ALL edges n-1 times (vs outgoing edges of node)
◼ See Introduction to Algorithms book
◼ Advantage: Minimize needed background to explain
◼ As dynamic programming solution
◼ FindPath(from, at most edges) recurrence
◼ See Algorithm Design book
◼ This is simpler idea, easier to prove, but more tricky to get
the algoithm optimized
Bellman-Ford Algorithm
3 6 5
0 1 2 3 What are all possible shortest paths from 0
with at most 2 edges?
3 {0, 1} = 3
2 {0, 1, 2} = 9
1 {0, 1, 4} = 4
Facts:
For each reachable node, expand it with
Expansions can be at most n-1 times every possible one edge:
To expand for k edges, you need k-1 edges results {0, 1, 2} + (2, 3, 5) => {0, 1, 2, 3} = 14
SP(S, X) uses 1 <= X <= n-1 edges {0, 1, 4} + (4, 3, 2) => {0, 1, 4, 3} = 6
We can think of that recursively or iteratively {0, 1, 4} + (4, 2, 3) => {0, 1, 4, 2} = 7
which is better than {0, 1, 2} = 9
Bellman-Ford Algorithm: Rec
Bellman-Ford Algorithm: improvements
◼ Order: O(n^3) time and O(N^2) memory
◼ Switch to Adjacency list,
◼ The node N^2 is replaced with M
◼ O(NM) time and O(N^2) memory
◼ Write code using table method
◼ Using rolling table technique in DP
◼ Now O(N) memory
◼ Or you can directly prove next code as it is
◼ Use the idea of edge expansion iteratively
Bellman-Ford Algorithm: iterative
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (0, 1) = 3 dist[0] = 0
(1, 2) = 2 dist[1] = OO
(2, 3) = 5 dist[2] = OO
dist[3] = OO
max_edge = 0, j = 0
ne = {0, 1, 3}
dist[1] > dist[0] + 3
OO > 0 + 3 => YES
Relax using this info
dist[1] = 3
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (0, 1) = 3 dist[0] = 0
(1, 2) = 2 dist[1] = 3
(2, 3) = 5 dist[2] = OO
dist[3] = OO
max_edge = 0, j = 1
ne = {1, 2, 2}
dist[2] > dist[1] + 2
OO > 3 + 2 => YES
Relax using this info
dist[2] = 5
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (0, 1) = 3 dist[0] = 0
(1, 2) = 2 dist[1] = 3
(2, 3) = 5 dist[2] = 5
dist[3] = OO
max_edge = 0, j = 2
ne = {2, 3, 5}
dist[3] > dist[2] + 5
OO > 5 + 5 => YES
Relax using this info
dist[3] = 10
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (0, 1) = 3 dist[0] = 0
(1, 2) = 2 dist[1] = 3
(2, 3) = 5 dist[2] = 5
dist[3] = 10
max_edge = 1, j = 0
ne = {0, 1, 3}
dist[1] > dist[0] + 3
3 > 3 + 0 => NO
And every next iteration will be zero
let’s try different edges ordering
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (1, 2) = 2 dist[0] = 0
(0, 1) = 3 dist[1] = OO
(2, 3) = 5 dist[2] = OO
dist[3] = OO
max_edge = 0, j = 0
ne = {1, 2, 2}
dist[2] > dist[1] + 2
OO > OO + 3 => NO
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (1, 2) = 2 dist[0] = 0
(0, 1) = 3 dist[1] = OO
(2, 3) = 5 dist[2] = OO
dist[3] = OO
max_edge = 0, j = 1
ne = {0, 1, 3}
dist[1] > dist[0] + 3
OO > 0 + 3 => YES
Relax using this info
dist[1] = 3
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (1, 2) = 2 dist[0] = 0
(0, 1) = 3 dist[1] = 3
(2, 3) = 5 dist[2] = OO
dist[3] = OO
max_edge = 0, j = 2
ne = {2, 3, 5}
dist[3] > dist[2] + 5
OO > OO + 5 => NO
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (1, 2) = 2 dist[0] = 0
(0, 1) = 3 dist[1] = 3
(2, 3) = 5 dist[2] = OO
dist[3] = OO
max_edge = 1, j = 0
ne = {1, 2, 2}
dist[2] > dist[1] + 2
OO > 3 + 2 => Yes
Relax using this info
dist[2] = 5
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (1, 2) = 2 dist[0] = 0
(0, 1) = 3 dist[1] = 3
(2, 3) = 5 dist[2] = 5
dist[3] = OO
max_edge = 1, j = 1
ne = {0, 1, 3}
dist[1] > dist[0] + 3
3 > 0 + 3 => No
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (1, 2) = 2 dist[0] = 0
(0, 1) = 3 dist[1] = 3
(2, 3) = 5 dist[2] = 5
dist[3] = OO
max_edge = 1, j = 2
ne = {2, 3, 5}
dist[3] > dist[2] + 5
OO > 5 + 5 => YES
Relax using this info
dist[3] = 10
Bellman-Ford Tracing
Assume edges order: Distance arr:
3 2 5
0 1 2 3 (2, 3) = 5 dist[0] = 0
(1, 2) = 2 dist[1] = OO
(0, 1) = 3 dist[2] = OO
dist[3] = OO
After max_edge = 0 After max_edge = 1 After max_edge = 2
Distance arr: Distance arr: Distance arr:
dist[0] = 0 dist[0] = 0 dist[0] = 0
dist[1] = 3 dist[1] = 3 dist[1] = 3
dist[2] = OO dist[2] = 5 dist[2] = 5
dist[3] = OO dist[3] = OO dist[3] = 10
Bellman-Ford Algorithm: behaviour
◼ Bellman-ford is pull-based algorithm
◼ It can only make use of neighbour info
◼ E.g. when edges were totally reversed, it only made use of
first edge 0-1
◼ In 2nd iteration, it could only use 1-2
◼ In 3rd iteration, it used 2-3
◼ So it expands knowledge based on its dist[]
◼ In worst case, n-1 is enough for any path
◼ In ith iteration, Shortest Paths of at most i-
edges are found
Bellman-Ford Tracing
Assume edges order
(B,E)
(D,B)
(B,D)
(A,B)
(A,C)
(D,C)
(B,C)
(E,D).
Pull-Based?
Only A is reachable => Either edge A-B or A-C will be first relaxation!
Src: http://www.geeksforgeeks.org/dynamic-programming-set-23-bellman-ford-algorithm/
Bellman-Ford Tracing
Assume edges order
(B,E)
(D,B)
(B,D)
(A,B)
(A,C)
(D,C)
(B,C)
(E,D).
Pull-Based?
A is reachable => Active edges {A-B, A-C}
B is reachable => Active edges {B-C, B-D, B-E}
C is reachable => Active edges {}
Src: http://www.geeksforgeeks.org/dynamic-programming-set-23-bellman-ford-algorithm/
Bellman-Ford Tracing
Assume edges order
(B,E)
(D,B)
(B,D)
(A,B)
(A,C)
(D,C)
(B,C)
(E,D).
Src: http://www.geeksforgeeks.org/dynamic-programming-set-23-bellman-ford-algorithm/
Bellman-Ford Algorithm: improvement
◼ Assume N = 1000. In 50th step, the internal
loop if condition is not activated
◼ Does it worth iterating more? No
◼ Improvement: If an iteration has no update,
next won’t have update..just break
Bellman-Ford Algorithm: improvement
Bellman-Ford Algorithm: cycle detection
◼ So far we compute shortest path
◼ What if there is -ve cycle, how to detec?
◼ Simple trick
◼ A path is at most n-1 edges
◼ Can’t be relaxed more
◼ If in n-th iteration a path is relaxed, this path has n edges
◼ So not simple path
◼ Then -ve cycle
Bellman-Ford Algorithm: cycle detection
Bellman-Ford Algorithm: get path
Bellman-Ford Algorithm: More
◼ We can know all nodes affected by -ve cycle
◼ After bellman finishes, saves its distance array
◼ Run bellman on updated array (not sure if 1 iter enough)
◼ Compare with new dist arr, Different values = Node Cycle
◼ Find a cycle
◼ Start from any affected node, say node A
◼ it is either in the cycle...or cycle reach it
◼ Go back (prev array), n steps
◼ Now, you must end at cycle..say node B
◼ Go back again, till you see B again..this a cycle
◼ Find positive cycle? Multiple graph with -1
ﺗﻢ ﺑﺤﻤﺪ ﷲ
ﻋﻠﻤﻜﻢ ﷲ ﻣﺎ ﯾﻨﻔﻌﻜﻢ
وﻧﻔﻌﻜﻢ ﺑﻤﺎ ﺗﻌﻠﻤﺘﻢ
وزادﻛﻢ ﻋﻠﻤﺎ ً
