Minimum Spanning Trees (MST)

Minimum Spanning Trees

• Spanning Tree
– A tree (i.e., connected, acyclic graph) which contains
all the vertices of the graph
• Minimum Spanning Tree
– Spanning tree with the minimum sum of weights
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
g g f
• Spanning forest 1 2

– If a graph is not connected, then there is a spanning

tree for each connected component of the graph
2
 Applications of MST
– Find the least expensive way to connect a set of
cities, terminals, computers, etc.

3
 Example
8
Problem b c
7
d
4 9
• A town has a set of houses 2
and a set of roads a 11 i 4 14 e
7 6
• A road connects 2 and only 8 10
h g f
2 houses 1 2

• A road connecting houses u and v has a repair

cost w(u, v)
Goal: Repair enough (and no more) roads such
that:
1. Everyone stays connected
i.e., can reach every house from all other houses
2. Total repair cost is minimum
4
 Minimum Spanning Trees
• A connected, undirected graph:
– Vertices = houses, Edges = roads
• A weight w(u, v) on each edge (u, v)  E

8 7
Find T  E such that: b c d
4 9
2
1. T connects all vertices a 11 i 14 e
4
7 6
2. w(T) = Σ(u,v)T w(u, v) is 8 10
h g 2
f
1
minimized

5
Properties of Minimum Spanning Trees

• Minimum spanning tree is not unique

• MST has no cycles – see why:

– We can take out an edge of a cycle, and still have
the vertices connected while reducing the cost

• # of edges in a MST:
– |V| - 1

6
 Growing a MST – Generic Approach

• Grow a set A of edges (initially

empty)
• Incrementally add edges to A 8 7
b c d
such that they would belong 4
2
9

a 11 i 14 e
to a MST 4
7 6
8 10
h g f
Idea: add only “safe” edges 1 2

– An edge (u, v) is safe for A if and

only if A  {(u, v)} is also a subset
of some MST
7
 Generic MST algorithm

1. A ← 
2. while A is not a spanning tree
3. do find an edge (u, v) that is safe for A
4. A ← A  {(u, v)} b
8
c
7
d
4 9
2
5. return A a 11 i 4 14 e
7 6
8 10
h g 2
f
1
• How do we find safe edges?

8
 Finding Safe Edges
8 7 V-S
b c d
• Let’s look at edge (h, g) 4
2
9

a 11 i 14 e
– Is it safe for A initially? 4
7 6
8 10
S
• Later on: h
1
g 2
f

– Let S  V be any set of vertices that includes h but not

g (so that g is in V - S)
– In any MST, there has to be one edge (at least) that
connects S with V - S
– Why not choose the edge with minimum weight
(h,g)?
9
 Definitions
8 7
b c d
4 9
2
• A cut (S, V - S) S a 11 i 14 e
4 S
7 6
is a partition of vertices V- S  8 10
 V- S
h g 2
f
1
into disjoint sets S and V - S
• An edge crosses the cut
(S, V - S) if one endpoint is in S
and the other in V – S

10
 Prim’s Algorithm
• The edges in set A always form a single tree

• Starts from an arbitrary “root”: VA = {a}

• At each step: 8 7
b c d
4 9
– Find a light edge crossing (VA, V - VA) 2
a 11 i 4 14 e
– Add this edge to A
7 6
8 10
– Repeat until the tree spans all vertices
h g 2
f
1

11
 How to Find Light Edges Quickly?
Use a priority queue Q: 8 7
b c d
• Contains vertices not yet 4 9
2
11
included in the tree, i.e., (V – VA) a i 4 14 e
7 6
8 10
– VA = {a}, Q = {b, c, d, e, f, g, h, i}
h g 2
f
1
• We associate a key with each vertex v:
key[v] = minimum weight of any edge (u, v)
connecting v to VA

Key[a]=min(w1,w2) w1

w2
a

12
 How to Find Light Edges Quickly?
(cont.)
• After adding a new node to VA we update the weights of all
the nodes adjacent to it
e.g., after adding a to the tree, k[b]=4 and k[h]=8

• Key of v is  if v is not adjacent to any vertices in VA

8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g 2
f
1

13
 Example
  
8
b c
7
d 0  
4 9
 2  Q = {a, b, c, d, e, f, g, h, i}
a 11 i 14 e
4 VA = 
7 6
8 10
Extract-MIN(Q)  a
h g 2
f
1
  

4
  
8 7 key [b] = 4  [b] = a
b c d
4 9

key [h] = 8  [h] = a
 2
a 11 i 4 14 e
7 6
8 10 4 8
h g f
1 2 Q = {b, c, d, e, f, g, h, i} VA = {a}
  
8
Extract-MIN(Q)  b
14
 Example
4 8
  key [c] = 8  [c] = b
8 7
b c d key [h] = 8  [h] = a -
4 9
 2  unchanged
11
a i 4 14 e 8 8
7 6
8 10 Q = {c, d, e, f, g, h, i} VA = {a, b}
h g 2
f
1
8   Extract-MIN(Q)  c
key [d] = 7  [d] = c
4 8 
7
8 7 key [f] = 4  [f] = c
b c d
4 2 2

9

key [i] = 2  [i] = c
a 11 i 4 14 e
7 6
8 10 7 4 8 2
h g f
1 2 Q = {d, e, f, g, h, i} VA = {a, b, c}
8  
4
Extract-MIN(Q)  i 15
 Example
4 8 7 key [h] = 7  [h] = i
8 7
b c d key [g] = 6  [g] = i
4 9
2 2  7 46 8
a 11 i 4 14 e
6
Q = {d, e, f, g, h} VA = {a, b, c, i}
7
8 10
h g f Extract-MIN(Q)  f
1 2
87 
6 4
4 8 7 key [g] = 2  [g] = f
8 7
b c d key [d] = 7  [d] = c
4 9
10

unchanged
2 2
a 11 i 4 14 e key [e] = 10  [e] = f
7 6
8 10 7 10 2 8
h g 2
f Q = {d, e, g, h} VA = {a, b, c, i, f}
1
7 6
2 4
Extract-MIN(Q)  g 16
 Example
4 8 7 key [h] = 1  [h] = g
8 7
b c d 7 10 1
4 9
2 2 10 Q = {d, e, h} VA = {a, b, c, i, f, g}
a 11 i 4 14 e
7 6 Extract-MIN(Q)  h
8 10
h g 2
f
1
7
1 2 4 7 10
4 8 7
8 7
Q = {d, e} VA = {a, b, c, i, f, g, h}
b c d
4 9 Extract-MIN(Q)  d
2 2 10
a 11 i 4 14 e
7 6
8 10
h g 2
f
1
1 2 4
17
 Example

4 8 7
8 7
b c d key [e] = 9  [e] = f
4 9 9
2 2 10
9
a 11 i 4 14 e
7 6 Q = {e} VA = {a, b, c, i, f, g, h, d}
8 10
h
1
g 2
f Extract-MIN(Q)  e
1 2 4 Q =  VA = {a, b, c, i, f, g, h, d, e}

18
 PRIM(V, E, w, r)
1. Q← 
Total time: O(VlgV + ElgV) = O(ElgV)
2. for each u  V
3. do key[u] ← ∞ O(V) if Q is implemented
as a min-heap
4. π[u] ← NIL
5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0 O(lgV)
Min-heap
7. while Q   Executed |V| times
operations:
8. do u ← EXTRACT-MIN(Q) Takes O(lgV) O(VlgV)

9. for each v  Adj[u] Executed O(E) times total

10. do if v  Q and w(u, v) < key[v] Constant O(ElgV)

11. then π[v] ← u Takes O(lgV)

12. DECREASE-KEY(Q, v, w(u, v))

19
 Prim’s Algorithm
• Prim’s algorithm is a “greedy” algorithm
– Greedy algorithms find solutions based on a sequence
of choices which are “locally” optimal at each step.

• Nevertheless, Prim’s greedy strategy produces a

globally optimum solution!

20
 Kruskal’s Algorithm
• How is it different from Prim’s algorithm?
– Prim’s algorithm grows one
tree all the time
– Kruskal’s algorithm grows tree1

multiple trees (i.e., a forest)

at the same time.
u
– Trees are merged together
using safe edges
v
– Since an MST has exactly |V| - 1 tree2
edges, after |V| - 1 merges,
we would have only one component
21
 Kruskal’s Algorithm
8 7
• Start with each vertex being its b c d
4 9
own component 2
a 11 i 4 14 e
• Repeatedly merge two 7 6
8 10
components into one by h g f
1 2
choosing the light edge that
We would add
connects them edge (c, f)
• Which components to consider
at each iteration?
– Scan the set of edges in
monotonically increasing order by
weight

22
 Example
1. Add (h, g) {g, h}, {a}, {b}, {c}, {d}, {e}, {f}, {i}
8 7
b c d 2. Add (c, i) {g, h}, {c, i}, {a}, {b}, {d}, {e}, {f}
4 9
2 3. Add (g, f) {g, h, f}, {c, i}, {a}, {b}, {d}, {e}
a 11 i 4 14 e 4. Add (a, b) {g, h, f}, {c, i}, {a, b}, {d}, {e}
7 6
8 10 5. Add (c, f) {g, h, f, c, i}, {a, b}, {d}, {e}
h g 2
f 6. Ignore (i, g) {g, h, f, c, i}, {a, b}, {d}, {e}
1
7. Add (c, d) {g, h, f, c, i, d}, {a, b}, {e}
1: (h, g) 8: (a, h), (b, c) 8. Ignore (i, h) {g, h, f, c, i, d}, {a, b}, {e}
2: (c, i), (g, f) 9: (d, e) 9. Add (a, h) {g, h, f, c, i, d, a, b}, {e}
4: (a, b), (c, f) 10: (e, f) 10. Ignore (b, c) {g, h, f, c, i, d, a, b}, {e}
6: (i, g) 11: (b, h) 11. Add (d, e) {g, h, f, c, i, d, a, b, e}
7: (c, d), (i, h) 14: (d, f) 12. Ignore (e, f) {g, h, f, c, i, d, a, b, e}
{a}, {b}, {c}, {d}, {e}, {f}, {g}, {h}, {i} 13. Ignore (b, h) {g, h, f, c, i, d, a, b, e}
14. Ignore (d, f) {g, h, f, c, i, d, a, b, e}
23
 Implementation of Kruskal’s Algorithm

8 7
• Uses a disjoint-set data 4
b c d
9
2
structure to determine a 11 i 4 14 e
7 6
whether an edge 8 10
h g 2
f
1
connects vertices in
We would add
different components edge (c, f)

24
 KRUSKAL(V, E, w)
1. A← 
2. for each vertex v  V
O(V)
3. do MAKE-SET(v)
4. sort E into non-decreasing order by w O(ElgE)
5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)} O(lgV)
8. UNION(u, v)
9. return A
Running time: O(V+ElgE+ElgV)=O(ElgE) – dependent on
the implementation of the disjoint-set data structure
25
 KRUSKAL(V, E, w) (cont.)
1. A← 
2. for each vertex v  V
O(V)
3. do MAKE-SET(v)
4. sort E into non-decreasing order by w O(ElgE)
5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)} O(lgV)
8. UNION(u, v)
9. return A
- Running time: O(V+ElgE+ElgV)=O(ElgE) O(ElgV)
- Since E=O(V2), we have lgE=O(2lgV)=O(lgV) 26
 Kruskal’s Algorithm
• Kruskal’s algorithm is a “greedy” algorithm
• Kruskal’s greedy strategy produces a globally
optimum solution
S

u y

v
V-S

27