CS 260 - Data Structures

All Pairs
Shortest Path
Yusuf Osmanlioglu
Department of Computer Science
Drexel University
Single Source Shortest Paths
● Given a graph G=(V,E), we want to find a shortest path from a source “s” to every
vertex v in V.
● Each edge has a weight w(a, b) = x means, the edge from a → b has a weight of x.
● A path is an order list of nodes:
○ P = (a, c, b, d)
○ means path P, starts at a, takes edges a → c, c → b, b → d, and ends at d.
● The weight of a path is the sum of all edges along the path.
○ w(P) = ∑(u,v ) in Pw(u, v) = w(a, c) + w(c, b) + w(b, d)
● The shortest path from X to Y is the path with the smallest total weight, or infinity if
there is no such path in G.

2
All Pairs Shortest Path Problem
● Single-Source Shortest Paths:
○ Compute shortest paths from a given source to all vertices in the graph.
● All-Pairs Shortest Paths:
○ Given a graph G = (V,E), |V|=n, and a weight function w on the edges, compute the shortest
paths between all pairs of vertices.
● The problem is not well defined in the presence of a negative weight cycle in the graph.

3
All Pairs Shortest Path Algorithms
● We can solve an all-pairs shortest-paths problem by running a single source
shortest-paths algorithm n times, once for each vertex as the source.
● Then the running time would be:
○ O(V ⨉ Elog V) if we use the Dijkstra’s algorithm (assuming no negative edge weights).
■ O(V3 log V) if the graph is dense
■ Note that, if Dijkstra’s is implemented using Fibonacci heaps, running it V times will
become O(V2 logV). [not covered in this class]
○ O(V ⨉ VE) if we use Bellman-Ford algorithm (to account for negative edge weights)
■ O(V4) if the graph is dense.

4
 All Pairs Shortest Path: Floyd-Warshall
Algorithm
● Instead we will give a direct approach to finding the shortest paths between
all pairs of vertices.
● We assume that negative weights exist, but no negative weight cycle.

Starting distance matrix b Shortest distance matrix

3 4

a b c d e a c a b c d e
2 8
a 0 3 8 ∞ -4 a 0 1 -3 2 -4
b ∞ 0 ∞ 1 7 1 b 3 0 -4 1 -1
D0 = -4 -5
c ∞ 4 0 ∞ ∞ 7 D5 = c 7 4 0 5 3
d 2 ∞ -5 0 ∞ d 2 -1 -5 0 -2
e d
e ∞ ∞ ∞ 6 0 6 e 8 5 1 6 0

5
Optimal Substructure
● Label vertices of the graph 1,2,...,n (edges are not shown in the picture)
● Sub-problem: For all node pairs u,v, find the cost of the shortest path from u to v, such
that all the internal vertices on that path are in {1,…,k-1}.

k ● Let D(k-1)[u,v] be the solution to this

k+1 subproblem, that is:
○ D(k-1)[u,v] = shortest distance between u
u 1 2 and v by using vertices 1,2,...,k-1 as
intermediary nodes
v
3 k-1
... ● Given D(k-1)[u,v], how can we calculate
D(k)[u,v]?
n
6
 Calculating D(k)[u,v] using D(k-1)[u,v]
● D(k)[u,v] is the cost of shortest path between u and v by using vertices 1,2,...,k as
intermediary nodes
● Given D(k-1)[u,v], how can we calculate Dk[u,v]?

k ● Case 1: we don’t need vertex k in the

k+1 shortest path between u,v
u 1 Path show in red was the shortest before, so
2
it is still the shortest now: Dk[u,v] = D(k-1)[u,v]
v
3
● Case 2: we need vertex k in the shortest
k-1
... path between u,v

n How to handle this case?

7
 Calculating D(k)[u,v] using D(k-1)[u,v]
● Suppose there are no negative cycles.
● If the path passes through k, it will consist of two sub-parts as follows

● Path shown in green is the shortest path

k
from u to k through {1,2,...,k-1}
k+1
○ Remember: sub-paths of shortest paths
are shortest paths!
u 1 2
● Path shown in purple is the shortest path
v
3 k-1
from k to v through {1,2,...,k-1}
...

n
D(k)[u,v] = D(k-1)[u,k] + D(k-1)[k,v]
8
Calculating D(k)[u,v] using D(k-1)[u,v]
● Case 1:
○ D(k)[u,v] = D(k-1)[u,v]
○ Cost of shortest path through {1,2,...,k-1}
● Case 2:
○ D(k)[u,v] = D(k-1)[u,k] + D(k-1)[k,v]
○ Cost of shortest path from u to k, and then from k to v through {1,2,...,k-1}
● Combining the two cases:
○ D(k)[u,v] = min(D(k-1)[u,v], D(k-1)[u,k] + D(k-1)[k,v])
○ Cost of shortest path through {1,2,...,k-1, k}
● Optimal Substructure:
○ We can solve the big problem using smaller problems.
● Overlapping sub-problems:
○ D(k-1)[k,v] can be used to help compute D(k)[u,v] for many u’s.
9
Floyd-Warshall: example
D(k)[u,v] = min(D(k-1)[u,v], D(k-1)[u,k] +
D(k-1)[k,v])
b a b c d e a b c d e a b c d e
3 4
a 0 3 8 ∞ -4 a 0 3 8 ∞ -4 a 0 3 8 4 -4
a 8 c b ∞ 0 ∞ 1 7 b ∞ 0 ∞ 1 7 b ∞ 0 ∞ 1 7
2 D0 = c ∞ 4 0 ∞ 1 2
∞ D = c ∞ 4 0 ∞ ∞ D = c ∞ 4 0 5 11
1 d 2 ∞ -5 0 ∞ d 2 5 -5 0 -2 d 2 -1 -5 0 -2
-4 -5
7 e ∞ ∞ ∞ 6 0 e ∞ ∞ ∞ 6 0 e ∞ ∞ ∞ 6 0

e d
6
a b c d e a b c d e a b c d e

a 0 3 8 4 -4 a 0 3 -1 4 -4 a 0 3 -3 2 -4
b ∞ 0 ∞ 1 7 b 3 0 -4 1 -1 b 3 0 -4 1 -1
3 4 5
D = c ∞ 4 0 5 11 D = c 7 4 0 5 3 D = c 7 4 0 5 3
d 2 -1 -5 0 -2 d 2 -1 -5 0 -2 d 2 -1 -5 0 -2
e ∞ ∞ ∞ 6 0 e 8 5 1 6 0 e 8 5 1 6 0
10
 Floyd-Warshall Algorithm
function Floyd-Warshall(G,w)
n = |G.V| ● Runtime:
let D(0) be an n⨉n matrix
for i=1 to n do ● Initialization:
for j=1 to n do
D(0) [i,j] = ∞ O(V2) ○ O(V2 + E + V)
end for
end for ● Updating distance:
for each e=(u,v) in G.E do
D(0) [u,v] = w(u,v) O(E) ○ O(V3)
end for
for each v in G.V do ● Overall:
D(0) [v,v] = 0 O(V) ○ O(V3)
end for

for k=1 to n do
let D(k) be a new n⨉n matrix
for i=1 to n do
for j=1 to n do
D(k) [i,j] = min(D(k-1) [i,j], D(k-1) [i,k] + O(V3)
D(k-1) [k,j])
end for
end for
end for
end function
11
Floyd-Warshall Algorithm: Negative Cycles
● As in Bellman-Ford algorithm, Floyd-Warshall can detect negative cycles.
● A negative cycle means there is a path from a vertex to itself with a cost less than 0.
● In the context of Floyd-Warshall:
○ For some vertex v, D(n)[v,v]<0 must be the case, if there exist a negative cycle involving v
● So, check for the distance matrix D(n) at the end of the algorithm.
○ Existence of a negative value indicates a negative cycle in the graph!

12
Runtime Comparison
● To calculate shortest path between all node pairs:
● Running Dijkstra’s V times would lead to:
○ O(V3 log V), if the graph is dense (otherwise, O(VElogV))
● Running Bellman-Ford V times would lead to:
○ O(V4), if the graph is dense (otherwise, O(V2E))
● Floyd-Warshall takes O(V3)!
○ Faster than Bellman-Ford and Dijkstra
■ Slower than Dijkstra’s Fibonacci heap implementation though (i.e., O(V2logV))
○ Straightforward to implement
○ Can handle negative edges!
○ Requires storage to keep two V⨉V matrices along with the input graph.

13
Dynamic Programming
● Floyd-Warshall algorithm is an example of an algorithm design paradigm called
dynamic programming.
● So far, we have seen examples of divide-and-conquer algorithm design paradigm
○ For example, merge-sort or quick-sort.
● We also saw an example of dynamic programming
○ Bellman-Ford algorithm.

14
Dynamic Programming: Bellman-Ford
Algorithm
function Bellman-Ford(G, w, s)
for each v in G.V do ● After |V|-1 iterations, the d[v] represent the
v.d = ∞ shortest path between s and each vertex v:
v.p = NIL
end for
○ Initially: d[s]=0
s.d = 0 ○ Let s→v1→v2→ ... →vk denote the shortest
for i=1 to |G.V|-1 do path between s and vk. Then:
for each edge (u,v) in G.E do
■ After 1st iteration d[v1] is correct, since
if v.d > u.d + w(u,v) then
v.d = u.d + w(u,v) d[v1] = d[s]+w(s,v1).
v.p = u ■ After 2nd iteration d[v2] is correct, since
end if d[v2] = d[v2]+w(v1,v2).
end for
end for ■ ...
for each edge (u,v) in G.E do ■ After kth iteration d[vk] is correct, since
if v.d > u.d + w(u,v) then d[vk] = d[vk-1]+w(vk-1,vk).
return false
return true ○ This holds for all vertices, since the longest
end function path in the graph has length |V|-1. 15
Dynamic Programming
● Optimal substructure:
○ the optimal solution of the big problem can be expressed in terms of optimal
solutions of smaller sub-problems
● Overlapping sub-problems:
○ sub-problems show up again and again
○ allowing to reuse their solution rather than recalculating from scratch
● Using these properties, we can design dynamic programming algorithms:
○ Keep a table of solutions to the smaller problems
○ Use the solutions in the table to solve bigger problems
○ At the end, we can use information in the table to find the optimal solution
■ Such as, reconstructing the shortest path in addition to the cost of the
shortest path in case of Floyd-Warshall!

16