Algorithm

Single-Source Shortest Path

 Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6 4
Q = V[G]; 5 9
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
p[r] = NULL;
while (Q not empty) 3 8
u = ExtractMin(Q); Run on example graph
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
  
p[r] = NULL;
while (Q not empty) 3  8
u = ExtractMin(Q); Run on example graph
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
r 0  
p[r] = NULL;
while (Q not empty) 3  8
u = ExtractMin(Q); Pick a start vertex r
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
u 0  
p[r] = NULL;
while (Q not empty) 3  8
u = ExtractMin(Q); Red vertices have been removed from Q
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
u 0  
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q); Red arrows indicate parent pointers
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
14  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
u 0  
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
14  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0  
p[r] = NULL;
while (Q not empty) 3 3 8
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
14  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 
p[r] = NULL;
while (Q not empty) 3 3 8
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
10  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 
p[r] = NULL;
while (Q not empty) 3 3 8
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
10  
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 
p[r] = NULL;
while (Q not empty) 3 3 8
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
10 2 
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 
p[r] = NULL;
while (Q not empty) 3 3 8
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
Q = V[G]; 5 9
10 2 
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Prim’s Algorithm
MST-Prim(G, w, r) 6  4
u
Q = V[G]; 5 9
10 2 
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6  4
u
Q = V[G]; 5 9
10 2 9
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6 4 4
u
Q = V[G]; 5 9
10 2 9
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6 4 4
u
Q = V[G]; 5 9
5 2 9
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
u
MST-Prim(G, w, r) 6 4 4
Q = V[G]; 5 9
5 2 9
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r) u 6 4 4
Q = V[G]; 5 9
5 2 9
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
u
MST-Prim(G, w, r) 6 4 4
Q = V[G]; 5 9
5 2 9
for each u  Q
key[u] = ; 14 2
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);

David Luebke 21 11/05/24

 Review: Prim’s Algorithm
MST-Prim(G, w, r) 6 4 4
Q = V[G]; 5 9
5 2 9
for each u  Q
key[u] = ; 14 2 u
10
key[r] = 0; 15
0 8 15
p[r] = NULL;
while (Q not empty) 3 3 8
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
What is the hidden cost in this code?
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
 Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
How often is ExtractMin() called?
key[r] = 0;
p[r] = NULL;
How often is DecreaseKey() called?
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
 Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q What will be the running time?
key[u] = ; A: Depends on queue
key[r] = 0;
binary heap: O(E lg V)
p[r] = NULL;
while (Q not empty)
Fibonacci heap: O(V lg V + E)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
 Single-Source Shortest Path
● Problem: given a weighted directed graph G,
find the minimum-weight path from a given
source vertex s to another vertex v
■ “Shortest-path” = minimum weight
■ Weight of path is sum of edges
■ E.g., a road map: what is the shortest path from
Chapel Hill to Charlottesville?
 Shortest Path Properties
● Again, we have optimal substructure: the shortest
path consists of shortest subpaths:

■ Proof: suppose some subpath is not a shortest path

○ There must then exist a shorter subpath
○ Could substitute the shorter subpath for a shorter path
○ But then overall path is not shortest path. Contradiction
 Shortest Path Properties
● Define (u,v) to be the weight of the shortest
path from u to v
● Shortest paths satisfy the triangle inequality:
(u,v)  (u,x) + (x,v)
● “Proof”: x

u v

This path is no longer than any other path

 Shortest Path Properties
● In graphs with negative weight cycles, some
shortest paths will not exist (Why?):

<0
 Relaxation
● A key technique in shortest path algorithms is
relaxation
■ Idea: for all v, maintain upper bound d[v] on (s,v)
Relax(u,v,w) {
if (d[v] > d[u]+w) then d[v]=d[u]+w;
}
2 2
5 9 5 6

Relax Relax
2 2
5 7 5 6
 Bellman-Ford Algorithm
BellmanFord()
for each v  V Initialize d[], which
d[v] = ; will converge to
shortest-path value 
d[s] = 0;
for i=1 to |V|-1
Relaxation:
for each edge (u,v)  E
Make |V|-1 passes,
Relax(u,v, w(u,v)); relaxing each edge
for each edge (u,v)  E
if (d[v] > d[u] + w(u,v)) Test for solution
Under what condition
return “no solution”;
do we get a solution?

Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w

 Bellman-Ford Algorithm
BellmanFord()
for each v  V What will be the
running time?
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
for each edge (u,v)  E
if (d[v] > d[u] + w(u,v))
return “no solution”;

Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w

 Bellman-Ford Algorithm
BellmanFord()
for each v  V What will be the
running time?
d[v] = ;
d[s] = 0; A: O(VE)
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
for each edge (u,v)  E
if (d[v] > d[u] + w(u,v))
return “no solution”;

Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w

 Bellman-Ford Algorithm
BellmanFord()
for each v  V B
s -1 2
d[v] = ;
d[s] = 0; A 2 E
3
for i=1 to |V|-1
for each edge (u,v)  E 1
4 -3
Relax(u,v, w(u,v));
for each edge (u,v)  E
C D
if (d[v] > d[u] + w(u,v)) 5
return “no solution”;
Ex: work on board

Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w

 Bellman-Ford
● Note that order in which edges are processed affects how
quickly it converges
● Correctness: show d[v] = (s,v) after |V|-1 passes
■ Lemma: d[v]  (s,v) always
○ Initially true
○ Let v be first vertex for which d[v] < (s,v)
○ Let u be the vertex that caused d[v] to change:
d[v] = d[u] + w(u,v)
○ Then d[v] < (s,v)
(s,v)  (s,u) + w(u,v) (Why?)
(s,u) + w(u,v)  d[u] + w(u,v) (Why?)
○ So d[v] < d[u] + w(u,v). Contradiction.
 Bellman-Ford
● Prove: after |V|-1 passes, all d values correct
■ Consider shortest path from s to v:
s  v1  v2  v3  v4  v
○ Initially, d[s] = 0 is correct, and doesn’t change (Why?)
○ After 1 pass through edges, d[v1] is correct (Why?) and
doesn’t change
○ After 2 passes, d[v2] is correct and doesn’t change
○…
○ Terminates in |V| - 1 passes: (Why?)
○ What if it doesn’t?
 DAG Shortest Paths
● Problem: finding shortest paths in DAG
■ Bellman-Ford takes O(VE) time.
■ How can we do better?
■ Idea: use topological sort
○ If were lucky and processes vertices on each shortest path from
left to right, would be done in one pass
○ Every path in a dag is subsequence of topologically sorted
vertex order, so processing verts in that order, we will do each
path in forward order (will never relax edges out of vert before
doing all edges into vert).
○ Thus: just one pass. What will be the running time?
 Dijkstra’s Algorithm
● If no negative edge weights, we can beat BF
● Similar to breadth-first search
■ Grow a tree gradually, advancing from vertices
taken from a queue
● Also similar to Prim’s algorithm for MST
■ Use a priority queue keyed on d[v]
 Dijkstra’s Algorithm
Dijkstra(G) B
10 2
for each v  V
A 4 3 D
d[v] = ;
d[s] = 0; S = ; Q = V; 5 1
C
while (Q  )
u = ExtractMin(Q); Ex: run the algorithm
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
Relaxation
Note: this d[v] = d[u]+w(u,v); Step
is really a
call to Q->DecreaseKey()
 Dijkstra’s Algorithm
Dijkstra(G)
for each v  V How many times is
ExtractMin() called?
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  ) How many times is
u = ExtractMin(Q); DecraseKey() called?
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
What will be the total running time?
 Dijkstra’s Algorithm
Dijkstra(G)
for each v  V How many times is
ExtractMin() called?
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  ) How many times is
u = ExtractMin(Q); DecraseKey() called?
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
A: O(E lg V) using binary heap for Q
Can acheive O(V lg V + E) with Fibonacci heaps
 Dijkstra’s Algorithm
Dijkstra(G)
for each v  V
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U{u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
Correctness: we must show that when u is
removed from Q, it has already converged
 Correctness Of Dijkstra's Algorithm

p2
u
s
y
x
p2

● Note that d[v]  (s,v) v

● Let u be first vertex picked s.t.  shorter path than d[u] d[u] > (s,u)
● Let y be first vertex V-S on actual shortest path from su  d[y] = (s,y)
■ Because d[x] is set correctly for y's predecessor x  S on the shortest path, and
■ When we put x into S, we relaxed (x,y), giving d[y] the correct value
 Correctness Of Dijkstra's Algorithm

p2
u
s
y
x
p2

● Note that d[v]  (s,v) v

● Let u be first vertex picked s.t.  shorter path than d[u] d[u] > (s,u)
● Let y be first vertex V-S on actual shortest path from su  d[y] = (s,y)
● d[u] > (s,u)
= (s,y) + (y,u) (Why?)
= d[y] + (y,u)
 d[y] But if d[u] > d[y], wouldn't have chosen u. Contradiction.
 Disjoint-Set Union Problem
● Want a data structure to support disjoint sets
■ Collection of disjoint sets S = {Si}, Si ∩ Sj = 

● Need to support following operations:

■ MakeSet(x): S = S U {{x}}
■ Union(Si, Sj): S = S - {Si, Sj} U {Si U Sj}
■ FindSet(X): return Si  S such that x  Si

● Before discussing implementation details, we

look at example application: MSTs
 Kruskal’s Algorithm
Kruskal()
{
T = ;
for each v  V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1?
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2? 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5?
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8? 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9?
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13? 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14? 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17?
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19?
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21? 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25?
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm
Kruskal() Run the algorithm:
2 19
{
9
T = ; 14 17
8 25
for each v  V 5
MakeSet(v); 21 13 1
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Correctness Of Kruskal’s Algorithm
● Sketch of a proof that this algorithm produces an
MST for T:
■ Assume algorithm is wrong: result is not an MST
■ Then algorithm adds a wrong edge at some point
■ If it adds a wrong edge, there must be a lower weight edge
(cut and paste argument)
■ But algorithm chooses lowest weight edge at each step.
Contradiction
● Again, important to be comfortable with cut and paste
arguments
 Kruskal’s Algorithm

Kruskal() What will affect the running time?

{
T = ;
for each v  V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
 Kruskal’s Algorithm

Kruskal() What will affect the running time?

{ 1 Sort
T = ; O(V) MakeSet() calls
for each v  V
O(E) FindSet() calls
O(V) Union() calls
MakeSet(v);
(Exactly how many Union()s?)
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Kruskal’s Algorithm: Running Time
● To summarize:
■ Sort edges: O(E lg E)
■ O(V) MakeSet()’s
■ O(E) FindSet()’s
■ O(V) Union()’s
● Upshot:
■ Best disjoint-set union algorithm makes above 3
operations take O(E(E,V)),  almost constant
■ Overall thus O(E lg E), almost linear w/o sorting
 Disjoint Set Union
● So how do we implement disjoint-set union?
■ Naïve implementation: use a linked list to
represent each set:

○ MakeSet(): ??? time

○ FindSet(): ??? time
○ Union(A,B): “copy” elements of A into B: ??? time
 Disjoint Set Union
● So how do we implement disjoint-set union?
■ Naïve implementation: use a linked list to represent each
set:

○ MakeSet(): O(1) time

○ FindSet(): O(1) time
○ Union(A,B): “copy” elements of A into B: O(A) time
■ How long can a single Union() take?
■ How long will n Union()’s take?
 Disjoint Set Union: Analysis
● Worst-case analysis: O(n2) time for n Union’s
Union(S1, S2) “copy” 1 element
Union(S2, S3) “copy” 2 elements
…
Union(Sn-1, Sn) “copy” n-1 elements
O(n2)
● Improvement: always copy smaller into larger
■ Why will this make things better?
■ What is the worst-case time of Union()?
● But now n Union’s take only O(n lg n) time!
Amortized Analysis of Disjoint Sets
● Amortized analysis computes average times without
using probability
● With our new Union(), any individual element is
copied at most lg n times when forming the complete
set from 1-element sets
■ Worst case: Each time copied, element in smaller set
1st time resulting set size 2
2nd time 4
…
(lg n)th time n
Amortized Analysis of Disjoint Sets
● Since we have n elements each copied at most lg
n times, n Union()’s takes O(n lg n) time
● We say that each Union() takes O(lg n) amortized
time
■ Financial term: imagine paying $(lg n) per Union
■ At first we are overpaying; initial Union $O(1)
■ But we accumulate enough $ in bank to pay for later
expensive O(n) operation.
■ Important: amount in bank never goes negative
The End