Algorithm Design & Analysis

SE 208
Unit 3-2 Greedy Method
Graph Shortest Path and Minimum Spanning Tree
Divyashikha Sethia
[email protected]
DTU
Contents

• Shortest Path
• Minimum Spanning Tree

Divyashikha Sethia (DTU) 2

 Shortest paths

• What is the shortest path from a to d?

B D

C E

Divyashikha Sethia (DTU) 3

 Shortest paths for un weighted graph

• BFS

B D

C E

Divyashikha Sethia (DTU) 4

 Shortest paths

• What is the shortest path from a to d?

2
B D
3
3
1 2
A

1
C E
4

Divyashikha Sethia (DTU) 5

Dijkstra’s algorithm

Divyashikha Sethia (DTU) 6

Dijkstra’s algorithm

Divyashikha Sethia (DTU) 7

Dijkstra’s algorithm
prev keeps track of
the shortest path

Divyashikha Sethia (DTU) 8

Dijkstra’s algorithm

Divyashikha Sethia (DTU) 9

Dijkstra’s algorithm

Divyashikha Sethia (DTU) 10

Dijkstra’s algorithm

Divyashikha Sethia (DTU) 11

 3
B D
3
1
1 2
A

1
C E Divyashikha Sethia (DTU) 12
4
  
3
B D
3

1
1 2
A

1
C E 
Divyashikha Sethia (DTU) 13
4
 Heap

A 0
B 
  C 
3
B D D 
3 E 
0
1
1 2
A

1
C E 
Divyashikha Sethia (DTU) 14
4
 Heap

B 
C 
  D 
3
B D E 
3
0
1
1 2
A

1
C E 
Divyashikha Sethia (DTU) 15
4
 Heap

B 
C 
  D 
3
B D E 
3
0
1
1 2
A

1
C E 
Divyashikha Sethia (DTU) 16
4
 Heap

C 1
B 
  D 
3
B D E 
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 17
4
 Heap

C 1
B 
  D 
3
B D E 
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 18
4
 Heap

C 1
B 3
3  D 
3
B D E 
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 19
4
 Heap

C 1
B 3
3  D 
3
B D E 
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 20
4
 Heap

B 3
D 
3  E 
3
B D
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 21
4
 Heap

B 3
D 
3  E 
3
B D
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 22
4
 Heap

B 3
D 
3  E 
3
B D
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 23
4
 Heap

B 2
D 
2  E 
3
B D
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 24
4
 Heap

B 2
D 
2  E 
3
B D
3
0
1
1 2
A
1
1
C E 
Divyashikha Sethia (DTU) 25
4
 Heap

B 2
E 5
2  D 
3
B D
3
0
1
1 2
A
1
1 5
C E Divyashikha Sethia (DTU) 26
4
 Heap

B 2
E 5
2  D 
3
B D
3
0
1
1 2
A
1
1 5
C E Divyashikha Sethia (DTU) 27
4
 Heap

B 2
E 5
2  D 
3
B D
3
0 All nodes reachable from
1
1 2
A starting node within a
1 given distance
1 5
C E Divyashikha Sethia (DTU) 28
4
 Heap

E 3
D 5
2 5
3
B D
3
0
1
1 2
A
1
1 3
C E Divyashikha Sethia (DTU) 29
4
 Heap

D 5

2 5
3
B D
3
0
1
1 2
A
1
1 3
C E Divyashikha Sethia (DTU) 30
4
 Heap

2 5
3
B D
3
0
1
1 2
A
1
1 3
C E Divyashikha Sethia (DTU) 31
4
 Heap

2 5
3
B D
0
1
A 1

1
1 3
C E Divyashikha Sethia (DTU) 32
 Is Dijkstra’s algorithm correct?

• Invariant:

Divyashikha Sethia (DTU) 33

 Is Dijkstra’s algorithm correct?

• Invariant: For every vertex removed from the heap, dist[v]

is the actual shortest distance from s to v

Divyashikha Sethia (DTU) 34

 Is Dijkstra’s algorithm correct?

• Invariant: For every vertex removed from the heap, dist[v] is the actual
shortest distance from s to v
• The only time a vertex gets visited is when the distance from s to that vertex is
smaller than the distance to any remaining vertex
• Therefore, there cannot be any other path that hasn’t been visited already that
would result in a shorter path

Divyashikha Sethia (DTU) 35

Running time?

Divyashikha Sethia (DTU) 36

Running time?

1 call to MakeHeap

Divyashikha Sethia (DTU) 37

Running time?

|V| iterations

Divyashikha Sethia (DTU) 38

Running time?

|V| calls

Divyashikha Sethia (DTU) 39

Running time?

O(|E|) calls

Divyashikha Sethia (DTU) 40

 Running time?

• Depends on the heap implementation

1 MakeHeap |V| ExtractMin |E| DecreaseKey Total

Array O(|V|) O(|V|2) O(|E|) O(|V|2)

Bin heap O(|V|) O(|V| log O(|E| log |V|) O((|V|+|E|) log |V|)
|V|)
= O(|E| log |V|)
if all vertices are
reachable from the
source

Divyashikha Sethia (DTU) 41

 Running time?

• Depends on the heap implementation

1 MakeHeap |V| ExtractMin |E| DecreaseKey Total

Array O(|V|) O(|V|2) O(|E|) O(|V|2)

Bin heap O(|V|) O(|V| log O(|E| log |V|) O((|V|+|E|) log |V|)
|V|)
O(|E| log |V|)

Divyashikha Sethia (DTU) 42

DIJKSTRA'S ALGORITHM - WHY USE IT?

• As mentioned, Dijkstra’s algorithm calculates the shortest path to

every vertex.
• However, it is about as computationally expensive to calculate
the shortest path from vertex u to every vertex using Dijkstra’s as
it is to calculate the shortest path to some particular vertex v.
• Therefore, anytime we want to know the optimal path to some
other vertex from a determined origin, we can use Dijkstra’s
algorithm.

Divyashikha Sethia (DTU) 43

Applications of Dijkstra's Algorithm
- Traffic Information Systems are most prominent use
- Mapping (Map Quest, Google Maps)
- Routing Systems

Divyashikha Sethia (DTU) 44

Applications of Dijkstra's Algorithm

 One particularly relevant this

week: epidemiology
 Prof. Lauren Meyers (Biology
Dept.) uses networks to model the
spread of infectious diseases and
design prevention and response
strategies.
 Vertices represent individuals,
and edges their possible contacts. It
is useful to calculate how a
particular individual is connected to
others.
 Knowing the shortest path
lengths to other individuals can be a
relevant indicator of the potential of
a particular individual to infect
others.
Divyashikha Sethia (DTU) 45
What about Dijkstra’s on…?

1
1 B D

5
A
10 -10

C E

Divyashikha Sethia (DTU) 46

What about Dijkstra’s on…?

1
1 B D

5
A
10 -10

C E

Divyashikha Sethia (DTU) 47

What about Dijkstra’s on…?

Dijkstra’s algorithm only works

for positive edge weights

1
1 B D

5
A
10

C E

Divyashikha Sethia (DTU) 48

 Differences
➢ Negative link weight: The Bellman-Ford algorithm works; Dijkstra’s
algorithm doesn’t.

➢ Distributed implementation: The Bellman-Ford algorithm can be easily

implemented in a distributed way. Dijkstra’s algorithm cannot.

➢ Time complexity: The Bellman-Ford algorithm is higher than Dijkstra’s

algorithm.

Divyashikha Sethia (DTU) 49

Negative cycles

What is the shortest path

from a to e?

1
1 B D

5
A
10 -10

C E
3

Divyashikha Sethia (DTU) 50

The Bellman-Ford Algorithm

t x
5
6 ,nil -2 ,nil
-3
0 8
s -4 7
2
7
,nil 9 ,nil
y z

Divyashikha Sethia (DTU) 51

 The Bellman-Ford Algorithm
t x t x
5 5
6 ,nil ,nil 6 6,s -2 ,nil
-2
-3 -3
0 8 0 8
s -4 7 s -4 7
2 2
7 7
,nil ,nil 7,s 9 ,nil
9
y Initialization z y After pass 1 z
t x t x
5 5
6 6,s -2 4,y 6 2,x 4,y
-2
-3 -3
0 8 0 8
s -4 7 s -4 7
2 2
7 7
7,s 9 2,t 7,s 2,t
9
y After pass 2 z
y After pass 3 z
The order of edges examined in each pass:
(t, x), (t, z), (x, t), (y, x), (y, t), (y, z), (z, x), (z, s), (s, t), (s, y)

Divyashikha Sethia (DTU) 52

The Bellman-Ford Algorithm

t x
5
6 2,x -2 4,y
-3
0 8
s -4 7
2
7
7,s 9 -2,t
y z

After pass 4

The order of edges examined in each pass:

(t, x), (t, z), (x, t), (y, x), (y, t), (y, z), (z, x), (z, s), (s, t), (s, y)

Divyashikha Sethia (DTU) 53

 The Bellman-Ford Algorithm
Bellman-Ford(G, w, s)
1. Initialize-Single-Source(G, s)
2. for i := 1 to |V| - 1 do
3. for each edge (u, v)  E do
4. Relax(u, v, w)
5. for each vertex v  u.adj do
6. if d[v] > d[u] + w(u, v)
7. then return False // there is a negative cycle
8. return True
Relax(u, v, w)
if d[v] > d[u] + w(u, v)
then d[v] := d[u] + w(u, v)
parent[v] := u

Divyashikha Sethia (DTU) 54

 Time Complexity
Bellman-Ford(G, w, s)
1. Initialize-Single-Source(G, s) O(|V|)
2. for i := 1 to |V| - 1 do
3. for each edge (u, v)  E do
O(|V||E|)
4. Relax(u, v, w)
5. for each vertex v  u.adj do O(|E|)
6. if d[v] > d[u] + w(u, v)
7. then return False // there is a negative cycle
8. return True

Time complexity: O(|V||E|)

Divyashikha Sethia (DTU) 55

 Minimum spanning trees

• What is the lowest weight set of edges that connects all vertices of an
undirected graph with positive weights

• Input: An undirected, positive weight graph, G=(V,E)

• Output: A tree T=(V,E’) where E’  E that minimizes

weight (T ) =  we
eE '

Divyashikha Sethia (DTU) 56

 MST example

1
A C E

3 4
4 4 2 5

B D F
4 6 1
A C E

4
4 2 5

B D F
Divyashikha Sethia (DTU) 57
 MSTs

• Can an MST have a cycle?

1
A C E

4
4 2 5

B D F
4

Divyashikha Sethia (DTU) 58

 MSTs

• Can an MST have a cycle?

1
A C E

4
4 2 5

B D F

Divyashikha Sethia (DTU) 59

Applications?

• Connectivity
• Networks (e.g. communications)
• Circuit desing/wiring
• hub/spoke models (e.g. flights, transportation)
• Traveling salesman problem?

Divyashikha Sethia (DTU) 60

 Cuts

• A cut is a partitioning of the vertices into two sets S and V-

S
• An edges “crosses” the cut if it connects a vertex uV and
vV-S

1
A C E

3 4
4 4 2 5

B D F
4 6
Divyashikha Sethia (DTU) 61
Minimum cut property
• Given a partion S, let edge e be the minimum cost
edge that crosses the partition. Every minimum
spanning tree contains edge e.

S V-S

e’

Consider an MST with edge Divyashikha

e’ that is(DTU)
Sethia not the minimum edge 62
Minimum cut property
• Given a partion S, let edge e be the minimum cost
edge that crosses the partition. Every minimum
spanning tree contains edge e.

S V-S

e’

Using e instead of e’, still connects the graph, but

produces a tree with Divyashikha
smaller Sethiaweights
(DTU) 63
 Algorithm ideas?

• Given a partion S, let edge e be the minimum cost edge

that crosses the partition. Every minimum spanning tree
contains edge e.

Divyashikha Sethia (DTU) 64

Kruskal’s algorithm
Add smallest edge that connects
two sets not already connected

1
A C E

3 4 5
4 4 2

MST
B D F
4 6
A C E
G

B D F
Divyashikha Sethia (DTU) 65
Kruskal’s algorithm
Add smallest edge that connects
two sets not already connected

1
A C E

3 4 5
4 4 2

MST
B D F
4 6 1
A C E
G

B D F
Divyashikha Sethia (DTU) 66
Kruskal’s algorithm
Add smallest edge that connects
two sets not already connected

1
A C E

3 4 5
4 4 2

MST
B D F
4 6 1
A C E
G 2

B D F
Divyashikha Sethia (DTU) 67
Kruskal’s algorithm
Add smallest edge that connects
two sets not already connected

1
A C E

3 4 5
4 4 2

MST
B D F
4 6 1
A C E
G 2
4

B D F
Divyashikha Sethia (DTU) 68
Kruskal’s algorithm
Add smallest edge that connects
two sets not already connected

1
A C E

3 4
4 4 2 5

MST
B D F
4 6 1
A C E
G 2
4
4

B D F
Divyashikha Sethia (DTU) 69
Kruskal’s algorithm
Add smallest edge that connects
two sets not already connected

1
A C E

3 4
4 4 2 5

MST
B D F
4 6 1
A C E
G 2
4
5
4

B D F
Divyashikha Sethia (DTU) 70
 Correctness of Kruskal’s
• Never adds an edge that connects already connected
vertices
• Always adds lowest cost edge to connect two sets. By min
cut property, that edge must be part of the MST

Divyashikha Sethia (DTU) 71

Running time of Kruskal’s

|V| calls to MakeSet

Divyashikha Sethia (DTU) 72

Running time of Kruskal’s

O(|E| log |E|)

Divyashikha Sethia (DTU) 73

Running time of Kruskal’s

2 |E| calls to FindSet

Divyashikha Sethia (DTU) 74

Running time of Kruskal’s

|V| calls to Union

Divyashikha Sethia (DTU) 75

 Running time of Kruskal’s

• Disjoint set data structure

Sorting: O(|E| log |E|) +

MakeSet FindSet Union Total

|E| calls |V| calls

Linked lists |V| O(|V| |E|) |V| O(|V||E| + |E| log |E|)

|E| >= |V | - 1 then complexity : |E| log |E|),

If |E| < |V|2 then lg |E| = O(lg V), complexity : |E| log |V|)

Divyashikha Sethia (DTU) 76

Prim’s algorithm

Divyashikha Sethia (DTU) 77

Prim’s algorithm

Divyashikha Sethia (DTU) 78

Prim’s algorithm

Divyashikha Sethia (DTU) 79

 Prim’s algorithm

• Start at some root node and build out the MST by adding
the lowest weighted edge at the frontier

Divyashikha Sethia (DTU) 80

Prim’s

1
A C E

3 4 5
4 4 2
MST
B D F
4 6 A C E

B D F
Divyashikha Sethia (DTU) 81
Prim’s

1
A C E

3 4 5
4 4 2
MST
  
B D F
4 6 A C E

B D F

Divyashikha Sethia (DTU)
 0
82
Prim’s

1
A C E

3 4 5
4 4 2
MST
 4 5
B D F
4 6 A C E

B D F

Divyashikha Sethia (DTU)
6 0 83
Prim’s

1
A C E

3 4 5
4 4 2
MST
 4 5
B D F
4 6 A C E

B D F

Divyashikha Sethia (DTU)
6 0 84
Prim’s

1
A C E

3 4 5
4 4 2
MST
1 4 5
B D F
4 6 A C E

B D F
4
Divyashikha Sethia (DTU)
2 0 85
Prim’s

1
A C E

3 4 5
4 4 2
MST
1 4 5
B D F
4 6 A C E

B D F
4
Divyashikha Sethia (DTU)
2 0 86
Prim’s

1
A C E

3 4 5
4 4 2
MST
1 4 5
B D F
4 6 A C E

B D F
4
Divyashikha Sethia (DTU)
2 0 87
Prim’s

1
A C E

3 4 5
4 4 2
MST
1 4 5
B D F
4 6 A C E

B D F
4
Divyashikha Sethia (DTU)
2 0 88
Prim’s

1
A C E

3 4 5
4 4 2
MST
1 4 5
B D F
4 6 A C E

B D F
4
Divyashikha Sethia (DTU)
2 0 89
Prim’s

1
A C E

3 4 5
4 4 2
MST
1 4 5
B D F
4 6 A C E

B D F
4
Divyashikha Sethia (DTU)
2 0 90
Prim’s

1
A C E

3 4 5
4 4 2
MST
1 4 5
B D F
4 6 A C E

B D F
4
Divyashikha Sethia (DTU)
2 0 91
Prim’s

1
A C E

3 4 5
4 4 2
MST
1 4 5
B D F
4 6 A C E

B D F
4
Divyashikha Sethia (DTU)
2 0 92
Correctness of Prim’s?

• Can we use the min-cut property?

• Given a partion S, let edge e be the minimum cost edge that crosses the
partition. Every minimum spanning tree contains edge e.
• Let S be the set of vertices visited so far
• The only time we add a new edge is if it’s the lowest weight edge
from S to V-S

Divyashikha Sethia (DTU) 93

Running time of Prim’s

Θ(|V|)

Divyashikha Sethia (DTU) 94

Running time of Prim’s

Θ(|V|)

Divyashikha Sethia (DTU) 95

Running time of Prim’s

|V| calls to Extract-Min

Divyashikha Sethia (DTU) 96

Running time of Prim’s

|E| calls to Decrease-Key

Divyashikha Sethia (DTU) 97

 Running time of Prim’s

• Same as Dijksta’s algorithm

1 MakeHeap |V| ExtractMin |E| DecreaseKey Total

Array O(|V|) O(|V|2) O(|E|) O(|V|2)

Bin heap O(|V|) O(|V| log O(|E| log |V|) O((|V|+|E|) log |V|)
|V|)
O(|E| log |V|)

Divyashikha Sethia (DTU) 98

References

• David Kauchak cs161 Summer 2009

- Neil Tang 03/11/2010 CS223 Advanced Data Structures
and Algorithms
- Cormen chapter 23,24

Divyashikha Sethia (DTU) 99