CS 316: ALGORITHMS

Lecture 8: Minimum Spanning

Tree (MST)
Prepared by: Assoc. Prof. Ensaf Hussein
Presented by:
Dr. Mohammed El-Said
Dr. Mai Hamdalla
PROBLEM: LAYING TELEPHONE WIRE

Central office
WIRING: NAÏVE APPROACH

Central office

Expensive!
WIRING: BETTER APPROACH

Central office

Minimize the total length of wire connecting the customers

PROBLEM 7
8
b c d
4 9
2
a 11 i 4 14 e

8 7 6
10
h g f
1 2
A town has a set of houses and a set of roads
A road connects 2 and only 2 houses
A road connecting houses u and v has a repair cost
w(u, v)
PROBLEM 7
8
b c d
4 9
2
a 11 i 4 14 e

8 7 6
10
h g f
1 2
Goal: Repair enough (and no more) roads such that:
1. Everyone stays connected
i.e., can reach every house from all other
houses
2. Total repair cost is minimum
MINIMUM SPANNING TREES
A connected, undirected graph:

• Vertices = houses, Edges = roads

A weight w(u, v) on each edge (u, v)  E
8 7
8 7 b c d
b c d 4 9
4 9 2
2 a 11 i 14 e
4
a 11 i 4 14 e 7 6
6 8 10
7
8 10 h g f
1 2
g g 2
f
1

Find T  E such that:

1. T connects all vertices
2. w(T) = Σ(u,v)T w(u, v) is minimized
MINIMUM SPANNING TREES
Spanning Tree
• A tree (i.e., connected, acyclic graph) which
contains all the vertices of the graph
Minimum Spanning Tree
• Spanning tree with the minimum sum of
weights 8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
g g 2
f
1
Spanning forest
• If a graph is not connected, then there is a
spanning tree for each connected component of
the graph
MINIMUM SPANNING TREE (MST)

Let’s think about simple MSTs on this

graph:
1
a b
5
2 4

c 3
d
MINIMUM SPANNING TREE (MST)

Black edges and nodes are in T

Is T a minimum cost spanning tree?
1
a b
5
2 4

c 3
d

Not spanning  d is not in T.

MINIMUM SPANNING TREE (MST)

Black edges and nodes are in T

Is T a minimum cost spanning tree?
1
a b
5
2 4

c 3
d

Not a tree  has a cycle.

MINIMUM SPANNING TREE (MST)

Black edges and nodes are in T

Is T a minimum cost spanning tree?
1
a b
5
2 4

c 3
d

Not minimum cost  can swap edges 4

and 2.
MINIMUM SPANNING TREE (MST)

Which edges form a MST?

1 1
a b a b
3 3
4 2 4 2

c 3
d c 3
d
Minimum spanning tree is not unique
Number of edges in a MST: |V| - 1
 APPLICATIONS OF MST
Any time you want to visit all vertices in a
graph at minimum cost

• In the design of electronic circuitry, it is often

necessary to make a set of pins electrically
equivalent by wiring them together

• Find the least expensive way to connect a set of

cities, terminals, computers, etc.

• Provides a heuristic for traveling salesman

problems. The optimum traveling salesman tour is
at most twice the length of the minimum spanning
tree (why??)
GROWING A MST(GENERIC
ALGORITHM)
GENERIC_MST(G, w)
1 A:={ } -- A is a subset of MST
2 while A does not form a spanning tree
do
3 find an edge (u,v) that is safe for A
4 A:=A∪{(u,v)}
5 return A
HOW TO FIND A SAFE EDGE
7
b c d 9
4
2
S↑ a 11 i e ↑S
4
7 6
8 10
V-S↓ h g f ↓ V-S
1 2

We need some definitions and a theorem:

• A cut (S,V-S) of an undirected graph G=(V,E) is
a partition of V.
HOW TO FIND A SAFE EDGE
7
b c d 9
4
2
S↑ a 11 i e ↑S
4
7 6
8 10
V-S↓ h g f ↓ V-S
1 2

We need some definitions and a theorem:

• An edge crosses the cut (S,V-S) if one of its
endpoints is in S and the other is in V-S.
• An edge is a light edge crossing a cut if its
weight is the minimum of any edge crossing
the cut.
THE ALGORITHMS OF MST
Kruskal and Prim algorithms are
elaborations of the generic algorithm.

Each of them uses a specific rule to

determine a safe edge of GENERIC_MST.
 PRIM ALGORITHM

 starts with one node.

 one by one, adds a node

 each time selecting the node whose connecting

edge is safe edge
 PRIM’S ALGORITHM
The edges in set A always form a single tree

Starts from an arbitrary “root”: VA = {a}

At each step: 8 7
b c d
4 9
• Find a light edge crossing (VA, V - VA) 2
a 11 i 4 14 e
• Add this edge to A 7 6
8 10
• Repeat until the tree spans all vertices
h
1
g 2
f
 HOW TO FIND LIGHT EDGES QUICKLY?
Use a priority queue Q contains vertices not yet included in
the tree, i.e., (V – VA)
• EX: VA = {a, b}, Q = {c, d, e, f, g, h, i}
We associate a key with each vertex v:
key[v] = minimum weight of any edge (u, v)
connecting v to VA
8 7
b c d
4 9
2
a i 4 14 e
11
7 6
8 10
Key[h]=min(8, 11) h g f
1 2
 HOW TO FIND LIGHT EDGES QUICKLY?
Initially  Key of v is  if v is not adjacent to any
vertices in VA

After adding a new node to VA  We update the

weights of all the nodes adjacent to it
EX: after adding a to the tree, k[b]=4 and k[h]=8

8 7
b c d
4 9
2
a i 4 14 e
11
7 6
8 10
h g f
1 2
EXAMPLE
  
8
b c
7
d 0  
4 9
 2  Q = {a, b, c, d, e, f, g, h,
a 11 e
i 4 14 i}
7 6
8 10
VA = 
h g 2
f
1
  
Extract-MIN(Q)  a
4
   key [b] = 4  [b] = a
8 7
b c d key [h] = 8  [h] = a
4 9
 2 
a 11 i 14 e
4 4 8
7 6
8 10 Q = {b, c, d, e, f, g, h, i} VA =
h g 2
f
1
   {a}
8
 = {a, #, #, #, #,#, a,#}
Extract-MIN(Q)  b
 EXAMPLE
4 8
  key [c] = 8  [c] = b
8 7
b c d key [h] = 8  [h] = a 
4 9
 2  unchanged
a 11 i 4 14 e 8 8
7 6
8 10 Q = {c, d, e, f, g, h, i} VA =
h g f
1

2

{a, b}
8
4 8 
7  = {b, #, #, #,#, a,#}
8 7 key [d] = 7
Extract-MIN(Q) [d]
c =c
b c d
4 2 2 9 key [f] = 4  [f] = c
 
a 11 i 4 14 e key [i] = 2  [i] = c
8
7 6
10 7 4 8 2
h
1
g 2
f Q = {d, e, f, g, h, i} VA =
8  
4 {a, b, c}
 = { c, #, c ,#, a, c}
 EXAMPLE key [h] = 7  [h] = i
4 8
8
7 key [g] = 6  [g] = i
7
b c d 7 46 8
4 9
2 2 
11
Q = {d, e, f, g, h} VA = {a,
a i 4 14 e
7 6 b, c, i}
8 10
h g f  = { c, #, c ,i, i}
1 2
87 
6 4 Extract-MIN(Q)  f
4 8 7
key [g] = 2  [g] = f
b
8
c
7
d
key [d] = 7  [d] = c
4 9 10 unchanged
2 2 
11
key [e] = 10  [e] = f
a i 4 14 e
8
7 6 7 10 2 8
10
h g 2
f Q = {d, e, g, h} VA = {a, b,
1
7 6
2 4 c, i, f}
 = { c, f ,f, i}
 EXAMPLE
4 8 7 key [h] = 1  [h] = g
8 7
b c d 7 10 1
4 9
2 2 10 Q = {d, e, h} VA = {a, b, c,
a 11 i 14 e
6
4 i, f, g}
7
8 10
 = { c, f ,g}
h g 2
f
1
2 4
Extract-MIN(Q)  h
7
1
4 8 7 7 10
8 7
b c d Q = {d, e} VA = {a, b, c, i,
4 9
2 2 10 f, g, h}
a 11 i 14 e
4  = { c, f}
7 6
8 10 Extract-MIN(Q)  d
h g 2
f
1
1 2 4
 8 7
EXAMPLE 4
b c d
9
2
a 11 i 4 14 e
7 6
8 10
4 8 7 h g 2
f
1
8 7
b c d
4 9 9
2 2 10
key [e] = 9  [e] = d
a 11 i 4 14 e
7 6 9
8 10
h g f Q = {e} VA = {a, b, c, i, f, g,
1 2
1 2 4 h, d}
 = { d}
Extract-MIN(Q)  e
Q =  VA = {a, b, c, i, f, g, h,
d, e}
 PRIM(V, E, W, R) For the
Preferred to be
graph
implemented as a
1. Q ←  binary heap
2. for each u  V
A binary heap is a complete
3. do key[u]
binary tree which ← satisfies
∞
4. π[u] ← NIL
the heap ordering property.
In Q change the key
of r to be 0
5. INSERT(Q, u)
Thekey[r]
6. min-heap
← 0 ►property: the
DECREASE-KEY(Q, r, 0)
value of each node is greater
7. while Q  
than or equal to the value of
8.
its parent.do u ← node
Root EXTRACT-MIN(Q)
hold
9.
the smallest for
keyeach v  Adj[u]
element.
10. do if v  Q and w(u, v)<key[v]
A heap with N nodes then
11. alwaysπ[v] ← u
has O(log N) height.
12. DECREASE-KEY(Q, v,
w(u, v))
RUNNING TIME ANALYSIS

• Depending on the heap implementation, running time is :

Extract_Mi Decrease_K
n ey
Binary heap O(lg V) O(lg V)
 PRIM(V, E, W, R)
1. Q← 
2. for each u  V Total time: O(VlgV + ElgV) = O(ElgV)

3. do key[u] ← ∞ O(V) if Q is implemented

4. π[u] ← NIL as a min-heap

5. INSERT(Q, u)
6. key[r] ← 0 ► DECREASE-KEY(Q, r, 0) O(lgV)
Min-heap
7. while Q   Executed |V| times
operations:
8. do u ← EXTRACT-MIN(Q) Takes O(lgV) O(VlgV)
9. for each v  Adj[u] Executed O(E) times total
10. do if v  Q and w(u, v)<key[v] Constant O(ElgV)

11. then π[v] ← u Takes O(lgV)

12. DECREASE-KEY(Q, v,
w(u, v))
 PRIM’S ALGORITHM
Prim’s algorithm is a “greedy” algorithm

• Greedy algorithms find solutions based on a

sequence of choices which are “locally” optimal
at each step.
Nevertheless, Prim’s greedy strategy produces a
globally optimum solution!
 KRUSKAL’S ALGORITHM

 Creates a forest of trees. Initially the forest

consists of n single node trees (and no edges).
 At each step, we add one safe edge (the
cheapest one) so that it joins two trees together.
 KRUSKAL’S ALGORITHM
How is it different from Prim’s algorithm?
• Prim’s algorithm grows one
tree all the time
• Kruskal’s algorithm grows
tree1
multiple trees (i.e., a forest)
at the same time.
• Trees are merged together u
using safe edges
• Since an MST has exactly |V| - 1 v
edges, after |V| - 1 merges, tree2

we would have only one component

 EXAMPLE
Initially we have 9 set of
vertices:
b 8 c 7 d {a}, {b}, {c}, {d}, {e},
4 9 {f}, {g}, {h}, {i}
2
a 11 i 4 14 e
7 6 Edges arranged in decreasing
8 10 order according to their
h g f
1 2 weights:
1: (h, g) 8: (a, h), (b, c) 1: (h, g) 2: (c, i), (g, f)
2: (c, i), (g, f) 9: (d, e) 4: (a, b), (c, f) 6: (i, g)
4: (a, b), (c, f)10: (e, f)
7: (c, d), (i, h) 8: (a, h), (b,
6: (i, g) 11: (b, h)
c)
7: (c, d), (i, h)14: (d, f)
9: (d, e) 10: (e, f)
{b}, {c}, {d}, {e}, {f}, {g}, {h}, {i} 11: (b, h) 14: (d, f)
 1. Add (h, g)
EXAMPLE 2. Add (c, i)
3. Add (g, f)
8 7 d 4. Add (a, b)
b c
4 9 5. Add (c, f)
2
a 11 i 4 14 e 6. Ignore (i, g)
7 6
8 10 7. Add (c, d)
h g f
1 2 8. Ignore (i, h)
1: (h, g) 8: (a, h), (b, 9. Add (a, h)
2: (c, i), (g, c) 10. Ignore (b,
f) 9: (d, e) c)
4: (a, b), (c, 10: (e, f) 11. Add (d, e)
f) 11: (b, h) 12. Ignore (e,
6: (i, g) 14: (d, f) f)
{a},
{a,
7: (c, {b},
b},
b,d), (i,{c},
c,{c,
i, {c, f,i},
f,i, g, {d},
h,{d},
g, d},{e},
h,
d,
h}, {e},
d},
e}
{e}{e}
{d}, 13.Ignore (b,
 IMPLEMENTATION OF KRUSKAL’S ALGORITHM
• Uses a disjoint-set data structure to
determine whether an edge connects
vertices in different components.
• Disjoint-set: is a data structure that tracks a
set of elements partitioned into a number
of disjoint (non-overlapping) subsets.
• A disjoint-set forest consists of a number of
elements each of which stores a parent
pointer.
 IMPLEMENTATION OF KRUSKAL’S ALGORITHM
• If an element's parent pointer points to no
other element, then the element is the root
of a tree and is the representative member
of its set.
• Each element in the set that has a
particular property representative member.
OPERATIONS ON DISJOINT DATA SETS
MAKE-SET(u) – creates a new set whose only member
is u
FIND-SET(u) – returns a representative element from
the set that contains u
• E.g.: Su = {r, s, t, u}
FIND-SET(u) = r FIND-SET(s) = r
• FIND-SET has to return the same value for the
elements of the same set
OPERATIONS ON DISJOINT DATA SETS
UNION(u, v) – unites the dynamic sets that contain
u and v, say Su and Sv

• E.g.: Su = {r, s, t, u}, Sv = {v, x, y}

UNION (u, v) = {r, s, t, u, v, x, y}

Running time for FIND-SET and UNION

depends on implementation. Can be O(lg
n).
KRUSKAL(V, E, W)
1. A ← 
2. for each vertex v  V
3. do MAKE-SET(v)
4. sort E into non-decreasing order by w
5. for each (u, v) taken from the sorted list
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)}
8. UNION(u, v)
9. return A
KRUSKAL(V, E, W)
1. A ← 
2. for each vertex v  V
3. do MAKE-SET(v)
O(V)
4. sort E into non-decreasing order by w O(ElgE)
5. for each (u, v) taken from the sorted listO(E)
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)}
O(lgV) O(ElgV)
8. UNION(u, v)
9. return A

Running time: O(V+ElgE+ElgV)

RUNNING TIME

- Running time: O(V+ElgE+ElgV)

- Since E= O(V2),
we have lgE=O(2lgV)=O(lgV)

The total time for Kruskal's algorithm

is O(E lg V), which is asymptotically
the same as of Prim's algorithm.
KRUSKAL’S ALGORITHM

Kruskal’s algorithm is a “greedy” algorithm

Kruskal’s greedy strategy produces a globally

optimum solution
S

u y

v
V-S
REFERENCES

Introduction to Algorithms, Second Edition by

Thomas H. Cormen, Charles E. Leiserson, Ronald L.
Rivest and Clifford Stein, The MIT Press © 2001
Chapter 23.2