SHORTEST PATH ALGORITHM

CS425 DESIGN ANALYSIS AND ALGORITHM

Edsger Wybe Dijkstra
- May 11, 1930 – August 6, 2002

- Received the 1972 A. M. Turing Award, widely considered the most prestigious
award in computer science.

- The Schlumberger Centennial Chair of Computer Sciences at The University of

Texas at Austin from 1984 until 2000

- Made a strong case against use of the GOTO statement in programming languages
and helped lead to its deprecation.

- Known for his many essays on programming.

Single-Source Shortest Path Problem

Single-Source Shortest Path Problem - The problem of finding shortest paths from
a source vertex v to all other vertices in the graph.
Dijkstra's algorithm
Dijkstra's algorithm - is a solution to the single-source shortest path problem in
graph theory.

Works on both directed and undirected graphs. However, all edges must have
nonnegative weights.

Approach: Greedy

Input: Weighted graph G={E,V} and source vertex v∈V, such that all edge weights
are nonnegative

Output: Lengths of shortest paths (or the shortest paths themselves) from a given
source vertex v∈V to all other vertices
Dijkstra's algorithm - Pseudocode

dist[s] ←0 (distance to source vertex is zero)

for all v ∈ V–{s}
do dist[v] ←∞ (set all other distances to infinity)
S←∅ (S, the set of visited vertices is initially empty)
Q←V (Q, the queue initially contains all vertices)
while Q ≠∅ (while the queue is not empty)
do u ← mindistance(Q,dist) (select the element of Q with the min. distance)
S←S∪{u} (add u to list of visited vertices)
for all v ∈ neighbors[u]
do if dist[v] > dist[u] + w(u, v) (if new shortest path found)
then d[v] ←d[u] + w(u, v) (set new value of shortest path)
(if desired, add traceback code)
return dist
Dijkstra Animated Example
Dijkstra Animated Example
Dijkstra Animated Example
Dijkstra Animated Example
Dijkstra Animated Example
Dijkstra Animated Example
Dijkstra Animated Example
Dijkstra Animated Example
Dijkstra Animated Example
Dijkstra Animated Example
Implementations and Running Times
The simplest implementation is to store vertices in an array or linked list. This will
produce a running time of

O(|V|^2 + |E|)

For sparse graphs, or graphs with very few edges and many nodes, it can be
implemented more efficiently storing the graph in an adjacency list using a binary
heap or priority queue. This will produce a running time of

O((|E|+|V|) log |V|)

Dijkstra's Algorithm - Why It Works

 As with all greedy algorithms, we need to make sure that it is a correct algorithm
(e.g., it always returns the right solution if it is given correct input).

 A formalproof would take longer than this presentation, but we can understand
how the argument works intuitively.

If you can’t sleep unless you see a proof, see the second reference or ask us
where you can find it.
DIJKSTRA'S ALGORITHM - WHY IT WORKS
 To understand how it works, we’ll go over the previous example
again. However, we need two mathematical results first:

 Lemma 1: Triangle inequality

If δ(u,v) is the shortest path length between u and v,
δ(u,v) ≤ δ(u,x) + δ(x,v)
 Lemma 2:
The subpath of any shortest path is itself a shortest path.

 The key is to understand why we can claim that anytime we put a new
vertex in S, we can say that we already know the shortest path to it.
 Now, back to the example…
DIJKSTRA'S ALGORITHM - WHY USE IT?

 As mentioned, Dijkstra’s algorithm calculates the

shortest path to every vertex.
 However, it is about as computationally expensive to
calculate the shortest path from vertex u to every
vertex using Dijkstra’s as it is to calculate the shortest
path to some particular vertex v.
 Therefore, anytime we want to know the optimal path
to some other vertex from a determined origin, we can
use Dijkstra’s algorithm.
Applications of Dijkstra's Algorithm
- Traffic Information Systems are most prominent use
- Mapping (Map Quest, Google Maps)
- Routing Systems
Applications of Dijkstra's
Algorithm
 One particularly relevant this
week: epidemiology
 Prof. Lauren Meyers (Biology
Dept.) uses networks to model the
spread of infectious diseases and
design prevention and response
strategies.
 Vertices represent individuals,
and edges their possible contacts.
It is useful to calculate how a
particular individual is connected to
others.
 Knowing the shortest path
lengths to other individuals can be
a relevant indicator of the potential
of a particular individual to infect
others.
Shortest paths
 What is the shortest path from a to d?

B D

C E
Shortest paths
 BFS

B D

C E
Shortest paths
 What is the shortest path from a to d?

2
B D
3
3
A 1 2

1 C E
4
Dijkstra’s algorithm
Dijkstra’s algorithm
Dijkstra’s algorithm
prev keeps track of
the shortest path
Dijkstra’s algorithm
Dijkstra’s algorithm
Dijkstra’s algorithm
Single source shortest paths

 All of the shortest path algorithms we’ll look at today

are call “single source shortest paths” algorithms
 Why?
 B 3 D
3
1
A 1 2

1 C E
4
  
B 3 D
3

1
A 1 2

1 C E 
4
 Heap

A 0
B 
  C 
B 3 D D 
3 E 
0
1
A 1 2

1 C E 
4
 Heap

B 
C 
  D 
B 3 D E 
3
0
1
A 1 2

1 C E 
4
 Heap

B 
C 
  D 
B 3 D E 
3
0
1
A 1 2

1 C E 
4
 Heap

C 1
B 
  D 
B 3 D E 
3
0
1
A 1 2
1
1 C E 
4
 Heap

C 1
B 
  D 
B 3 D E 
3
0
1
A 1 2
1
1 C E 
4
 Heap

C 1
B 3
3  D 
B 3 D E 
3
0
1
A 1 2
1
1 C E 
4
 Heap

C 1
B 3
3  D 
B 3 D E 
3
0
1
A 1 2
1
1 C E 
4
 Heap

B 3
D 
3  E 
B 3 D
3
0
1
A 1 2
1
1 C E 
4
 Heap

B 3
D 
3  E 
B 3 D
3
0
1
A 1 2
1
1 C E 
4
 Heap

B 3
D 
3  E 
B 3 D
3
0
1
A 1 2
1
1 C E 
4
 Heap

B 2
D 
2  E 
B 3 D
3
0
1
A 1 2
1
1 C E 
4
 Heap

B 2
D 
2  E 
B 3 D
3
0
1
A 1 2
1
1 C E 
4
 Heap

B 2
E 5
2  D 
B 3 D
3
0
1
A 1 2
1
1 C E 5
4
 Heap

B 2
E 5
2  D 
B 3 D
3
0
1
A 1 2 Frontier?
1
1 C E 5
4
 Heap

B 2
E 5
2  D 
B 3 D
3
0 All nodes reachable from
1
A 1 2 starting node within a
1 given distance
1 C E 5
4
 Heap

E 3
D 5
2 5
B 3 D
3
0
1
A 1 2
1
1 C E 3
4
 Heap

D 5

2 5
B 3 D
3
0
1
A 1 2
1
1 C E 3
4
 Heap

2 5
B 3 D
3
0
1
A 1 2
1
1 C E 3
4
 Heap

2 5
3
B D
0
1
A 1
1
1 C E 3
Is Dijkstra’s algorithm
correct?
 Invariant:
Is Dijkstra’s algorithm
correct?
 Invariant: For every vertex removed from the heap,
dist[v] is the actual shortest distance from s to v
Is Dijkstra’s algorithm
correct?
 Invariant: For every vertex removed from the heap, dist[v] is the actual
shortest distance from s to v
 The only time a vertex gets visited is when the distance from s to that vertex is
smaller than the distance to any remaining vertex
 Therefore, there cannot be any other path that hasn’t been visited already
that would result in a shorter path
Running time?
Running time?

1 call to MakeHeap
Running time?

|V| iterations
Running time?

|V| calls
Running time?

O(|E|) calls
 Running time?
 Depends on the heap implementation

1 MakeHeap |V| ExtractMin|E| Total

DecreaseKey
Array O(|V| 2
O(|V| ) O(|V|2)
O(|E|)
)

Bin heap O(|V| O(|V| log O(|E| log O((|V|+|E|) log

) |V|) |V|) |V|)
O(|E| log
|V|)
 Running time?
 Depends on the heap implementation

1 MakeHeap |V| ExtractMin|E| Total

DecreaseKey
Array O(|V| 2
O(|V| ) O(|V|2)
O(|E|)
)

Bin heap O(|V| O(|V| log O(|E| log O((|V|+|E|) log

) |V|) |V|) |V|)
O(|E| log
|V|)

Is this an improvement? If |E| < |V|2 / log |V|

 Running time?
 Depends on the heap implementation

1 MakeHeap |V| ExtractMin|E| Total

DecreaseKey
Array O(|V| 2
O(|V| ) O(|V|2)
O(|E|)
)

Bin heap O(|V| O(|V| log O(|E| log O((|V|+|E|) log

) |V|) |V|) |V|)
O(|E| log
|V|)

Fib heap O(|V| O(|V| log O(|E|) O(|V| log |V| +

) |V|) |E|)
What about Dijkstra’s on…?

1
1 B D

A 5
10 -10
C E
What about Dijkstra’s on…?

1
1 B D

A 5
10 -10
C E
What about Dijkstra’s on…?

Dijkstra’s algorithm only works

for positive edge weights

1
1 B D

A 5
10
C E
Bounding the distance
 Another invariant: For each vertex v, dist[v] is an upper bound on the
actual shortest distance
 start of at 
 only update the value if we find a shorter distance
 An update procedure

dist[v]  min{dist[v], dist[u]  w(u, v)}

dist[v]  min{dist[v], dist[u]  w(u, v)}

 Can we ever go wrong applying this update rule?

 We can apply this rule as many times as we want and will
never underestimate dist[v]
 When will dist[v] be right?
 If u is along the shortest path to v and dist[u] is correct
 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to v and dist[u] is correct
 Consider the shortest path from s to v

s p1 p2 p3 pk v
 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to v and dist[u] is correct
 What happens if we update all of the vertices with the above update?

s p1 p2 p3 pk v
 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to v and dist[u] is correct
 What happens if we update all of the vertices with the above update?

s p1 p2 p3 pk v

correct
 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to v and dist[u] is correct
 What happens if we update all of the vertices with the above update?

s p1 p2 p3 pk v

correct correct
 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to v and dist[u] is correct
 Does the order that we update the vertices matter?

s p1 p2 p3 pk v

correct correct
 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to

v and dist[u] is correct
 How many times do we have to do this for vertex pi
to have the correct shortest path from s?
 i times

s p1 p2 p3 pk v
 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to

v and dist[u] is correct
 How many times do we have to do this for vertex pi
to have the correct shortest path from s?
 i times

s p1 p2 p3 pk v

correct correct
 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to

v and dist[u] is correct
 How many times do we have to do this for vertex pi
to have the correct shortest path from s?
 i times

s p1 p2 p3 pk v

correct correct correct

 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to

v and dist[u] is correct
 How many times do we have to do this for vertex pi
to have the correct shortest path from s?
 i times

s p1 p2 p3 pk v

correct correct correct correct

 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to

v and dist[u] is correct
 How many times do we have to do this for vertex pi
to have the correct shortest path from s?
 i times

s p1 p2 p3 pk v

correct correct correct correct …

 dist[v]  min{dist[v], dist[u]  w(u, v)}

 dist[v] will be right if u is along the shortest path to v and dist[u] is correct
 What is the longest (vetex-wise) the path from s to any node v can be?
 |V| - 1 edges/vertices

s p1 p2 p3 pk v

correct correct correct correct …

Bellman-Ford algorithm
Bellman-Ford algorithm

Initialize all the

distances
Bellman-Ford algorithm

iterate over all

edges/vertices and
apply update rule
Bellman-Ford algorithm
Bellman-Ford algorithm

check for negative

cycles
Negative cycles

What is the shortest path from

a to e?

1
1 B D

A 5
10 -10
C E
3
Bellman-Ford algorithm
Bellman-Ford algorithm

1 How many edges is

S A
0 the shortest path from
8 1 s to:

G -4 B A:
2
1 1

F -2 C

-1 3
E D
-1
Bellman-Ford algorithm

1 How many edges is

S A
0 the shortest path from
8 1 s to:

G -4 B A: 3
2
1 1

F -2 C

-1 3
E D
-1
Bellman-Ford algorithm

1 How many edges is

S A
0 the shortest path from
8 1 s to:

G -4 B A: 3
2
1 1
-2 C B:
F

-1 3
E D
-1
Bellman-Ford algorithm

1 How many edges is

S A
0 the shortest path from
8 1 s to:

G -4 B A: 3
2
1 1
-2 C B: 5
F

-1 3
E D
-1
Bellman-Ford algorithm

1 How many edges is

S A
0 the shortest path from
8 1 s to:

G -4 B A: 3
2
1 1
-2 C B: 5
F

-1 3
E D D:
-1
Bellman-Ford algorithm

1 How many edges is

S A
0 the shortest path from
8 1 s to:

G -4 B A: 3
2
1 1
-2 C B: 5
F

-1 3
E D D: 7
-1
Bellman-Ford algorithm
0 
S 1 A
0 Iteration: 0
8 1

 G -4 B
2
1 1

F -2 C 

-1 3
E D
-1
 
Bellman-Ford algorithm
0 1
S 1 A0
0 Iteration: 1
8 1

8 G -4 B
2
1 1

F -2 C 

-1 3
E D
-1
 
Bellman-Ford algorithm
0 1
S 1 A0
0 Iteration: 2
8 1

8 G -4 B
2
1 1

F -2 C 
9
-1 3
E D
-1
1 
2
Bellman-Ford algorithm
0 5
S 1 A
0 Iteration: 3
8 1
1
8 G -4 B0 A has the correct
2 distance and path
1 1

F -2 C 
9
-1 3
E D
-1
8 
Bellman-Ford algorithm
0 5
S 1 A
0 Iteration: 4
8 1
6
8 G -4 B
2
1 1

F -2 C 11
9
-1 3
E D
-1
7 
Bellman-Ford algorithm
0 5
S 1 A
0 Iteration: 5
8 1
5
8 G -4 B B has the correct
2 distance and path
1 1

F -2 C 7
9
-1 3
E D
-1
7 1
4
Bellman-Ford algorithm
0 5
S 1 A
0 Iteration: 6
8 1
5
8 G -4 B
2
1 1

F -2 C 6
9
-1 3
E D
-1
7 1
0
Bellman-Ford algorithm
0 5
S 1 A
0 Iteration: 7
8 1
5
8 G -4 B D (and all other
2 nodes) have the
1 correct distance and
1
path
F -2 C 6
9
-1 3
E D
-1
7 9
Correctness of Bellman-Ford

 Loop invariant:
Correctness of Bellman-Ford

 Loop invariant: After iteration i, all vertices with

shortest paths from s of length i edges or less have
correct distances
Runtime of Bellman-Ford

O(|V| |E|)
Runtime of Bellman-Ford

Can you modify the algorithm to run faster

(in some circumstances)?
All pairs shortest paths

 Simple approach
 Call Bellman-Ford |V| times
 O(|V|2 |E|)
 Floyd-Warshall – Θ(|V|3)
 Johnson’s algorithm – O(|V|2 log |V| + |V| |E|)
Minimum spanning trees
 What is the lowest weight set of edges that connects all vertices of an
undirected graph with positive weights

 Input: An undirected, positive weight graph, G=(V,E)

 Output: A tree T=(V,E’) where E’  E that minimizes

weight (T )   we
eE '
 MST example

1
A C E

3 4
4 4 2 5

B D F
4 6 1
A C E

2 4 5
4

B D F
MSTs
 Can an MST have a cycle?

1
A C E

4
4 2 5

B D F
4
MSTs
 Can an MST have a cycle?

1
A C E

4
4 2 5

B D F
Applications?

 Connectivity
 Networks (e.g. communications)
 Circuit desing/wiring
 hub/spoke models (e.g. flights, transportation)
 Traveling salesman problem?
 Cuts

 A cut is a partitioning of the vertices into two sets S

and V-S
 An edges “crosses” the cut if it connects a vertex
uV and vV-S

1
A C E

3 4
4 4 2 5

B D F
4 6
Minimum cut property
 Given a partion S, let edge e be the minimum
cost edge that crosses the partition. Every
minimum spanning tree contains edge e.
S V-S

e’

Consider an MST with edge e’ that is not the minimum edge

Minimum cut property
 Given a partion S, let edge e be the minimum
cost edge that crosses the partition. Every
minimum spanning tree contains edge e.
S V-S

e’
Using e instead of e’, still connects the graph, but
produces a tree with smaller weights
Algorithm ideas?

 Given a partion S, let edge e be the minimum cost

edge that crosses the partition. Every minimum
spanning tree contains edge e.
Kruskal’s algorithm
Add smallest edge that connects two
sets not already connected

1
A C E

3 4 5
4 4 2
MST
B D F
4 6 A C E
G

B D F
Kruskal’s algorithm
Add smallest edge that connects two
sets not already connected

1
A C E

3 4 5
4 4 2
MST
B D F
4 6 1
A C E
G

B D F
Kruskal’s algorithm
Add smallest edge that connects two
sets not already connected

1
A C E

3 4 5
4 4 2
MST
B D F
4 6 1
A C E
G
2

B D F
Kruskal’s algorithm
Add smallest edge that connects two
sets not already connected

1
A C E

3 4 5
4 4 2
MST
B D F
4 6 1
A C E
G
4 2

B D F
Kruskal’s algorithm
Add smallest edge that connects two
sets not already connected

1
A C E

3 4
4 4 2 5
MST
B D F
4 6 1
A C E
G 4
4 2

B D F
Kruskal’s algorithm
Add smallest edge that connects two
sets not already connected

1
A C E

3 4
4 4 2 5
MST
B D F
4 6 1
A C E
G 4
4 2 5

B D F
 Correctness of Kruskal’s
 Never adds an edge that connects already
connected vertices
 Always adds lowest cost edge to connect two sets.
By min cut property, that edge must be part of the
MST
Running time of Kruskal’s

|V| calls to MakeSet

Running time of Kruskal’s

O(|E| log |E|)

Running time of Kruskal’s

2 |E| calls to FindSet

Running time of Kruskal’s

|V| calls to Union

 Running time of Kruskal’s
 Disjoint set data structure

O(|E| log |E|)

+
MakeSet FindSet Union Total
|E| calls |V| calls

Linked lists |V| O(|V| |V O(|V||E| + |E| log

|E|) | |E|)
O(|V| |E|)

Linked lists |V| O(|E| log |V O(|E| log |V|+ |E| log
+ |V|) | |E|)
O(|E| log |E| )
heuristics
Prim’s algorithm
Prim’s algorithm
Prim’s algorithm
Prim’s algorithm
 Start at some root node and build out the MST by
adding the lowest weighted edge at the frontier
Prim’s

A 1 C E

3 4 5
4 4 2 MST
B D F
4 6 A C E

B D F
Prim’s

A 1 C E

3 4 5
4 4 2 MST
  
B D F
4 6 A C E

B D F
  0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
 4 5
B D F
4 6 A C E

B D F
 6 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
 4 5
B D F
4 6 A C E

B D F
 6 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
1 4 5
B D F
4 6 A C E

B D F
4 2 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
1 4 5
B D F
4 6 A C E

B D F
4 2 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
1 4 5
B D F
4 6 A C E

B D F
4 2 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
1 4 5
B D F
4 6 A C E

B D F
4 2 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
1 4 5
B D F
4 6 A C E

B D F
4 2 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
1 4 5
B D F
4 6 A C E

B D F
4 2 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
1 4 5
B D F
4 6 A C E

B D F
4 2 0
Prim’s

A 1 C E

3 4 5
4 4 2 MST
1 4 5
B D F
4 6 A C E

B D F
4 2 0
Correctness of Prim’s?

 Can we use the min-cut property?

 Given a partion S, let edge e be the minimum cost edge
that crosses the partition. Every minimum spanning tree
contains edge e.
 Let S be the set of vertices visited so far
 The only time we add a new edge is if it’s the lowest
weight edge from S to V-S
Running time of Prim’s

Θ(|V|)
Running time of Prim’s

Θ(|V|)
Running time of Prim’s

|V| calls to Extract-Min

Running time of Prim’s

|E| calls to Decrease-Key

 Running time of Prim’s
 Same as Dijksta’s algorithm

1 MakeHeap |V| ExtractMin|E| Total

DecreaseKey
Array O(|V| 2
O(|V| ) O(|V|2)
O(|E|)
)

Bin heap O(|V| O(|V| log O(|E| log O((|V|+|E|) log

) |V|) |V|) |V|)
O(|E| log
|V|)

Fib heap O(|V| O(|V| log O(|E|) O(|V| log |V| +

) |V|) |E|) log |E| )
Kruskal’s: O(|E|
References

 Dijkstra’s original paper:

E. W. Dijkstra. (1959) A Note on Two Problems in
Connection with Graphs. Numerische Mathematik, 1. 269-
271.
 MIT OpenCourseware, 6.046J Introduction to Algorithms.
< http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-
Computer-Science/6-046JFall-2005/CourseHome/>
Accessed 4/25/09
 Meyers, L.A. (2007) Contact network epidemiology: Bond
percolation applied to infectious disease prediction and
control. Bulletin of the American Mathematical Society
44: 63-86.
 Department of Mathematics, University of Melbourne.
Dijkstra’s Algorithm.
<http://www.ms.unimelb.edu.au/~moshe/620-
261/dijkstra/dijkstra.html > Accessed 4/25/09