0% found this document useful (0 votes)
25 views9 pages

Minimum Spanning Trees

The document discusses minimum spanning trees (MST), which are spanning trees of weighted graphs that have minimum total edge weight. A spanning tree is a tree that connects all vertices of a graph without cycles. An MST provides a way to connect all vertices with minimum total cost. The document defines MSTs and spanning trees, provides properties of MSTs such as being unique and acyclic, and presents a generic algorithm to grow an MST by incrementally adding safe edges.

Uploaded by

bradmonjohn
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
25 views9 pages

Minimum Spanning Trees

The document discusses minimum spanning trees (MST), which are spanning trees of weighted graphs that have minimum total edge weight. A spanning tree is a tree that connects all vertices of a graph without cycles. An MST provides a way to connect all vertices with minimum total cost. The document defines MSTs and spanning trees, provides properties of MSTs such as being unique and acyclic, and presents a generic algorithm to grow an MST by incrementally adding safe edges.

Uploaded by

bradmonjohn
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Minimum Spanning Trees

• Spanning Tree
– A tree (i.e., connected, acyclic graph) which contains
Design and Analysis of all the vertices of the graph
• Minimum Spanning Tree
Algorithms – Spanning tree with the minimum sum of weights
8 7
b c d
4 9
2
Minimum Spanning Trees (MST) 11
a i 4 14 e
7 6
8 10
Chapter 23
• Spanning forest g
1
g 2
f

Thanks to George Bebis – If a graph is not connected, then there is a spanning


tree for each connected component of the graph

1 2

Applications of MST Example


8
– Find the least expensive way to connect a set of Problem b c
7
d
9
cities, terminals, computers, etc. • A town has a set of houses 4
2
and a set of roads a 11 i 4 14 e
7 6
• A road connects 2 and only 8 10
h g f
2 houses 1 2
• A road connecting houses u and v has a repair
cost w(u, v)
Goal: Repair enough (and no more) roads such
that:
1. Everyone stays connected
i.e., can reach every house from all other houses
2. Total repair cost is minimum

3 4

3 4

Minimum Spanning Trees Properties of Minimum Spanning Trees


• A connected, undirected graph: • Minimum spanning tree is not unique
– Vertices = houses, Edges = roads

• A weight w(u, v) on each edge (u, v)  E

Find T  E such that: b


8
c
7
d
• MST has no cycles – see why:
4 9
1. T connects all vertices 2 – We can take out an edge of a cycle, and still have
a 11 i 4 14 e
the vertices connected while reducing the cost
2. w(T) = Σ(u,v)T w(u, v) is 8
7 6
10
h
1
g 2
f • # of edges in a MST:
minimized
– |V| - 1

5 6

5 6

1
Growing a MST – Generic Approach Generic MST algorithm
• Grow a set A of edges (initially 1. A ← 
empty) 2. while A is not a spanning tree
• Incrementally add edges to A
do find an edge (u, v) that is safe for A
8 7
b c d 3.
such that they would belong 4 9
2
to a MST a 11 i 4 14 e 4. A ← A  {(u, v)} b
8
c
7
d
7 6 4 9
8 10 2
h g f 5. return A
Idea: add only “safe” edges 1 2 a 11 i 4 14 e
7 6
8 10
– An edge (u, v) is safe for A if and h
1
g 2
f

only if A  {(u, v)} is also a subset • How do we find safe edges?


of some MST
7 8

7 8

Finding Safe Edges Definitions


8 7 V-S 8 7
Let’s look at edge (h, g)
b c d b c d
• 4 9 4 9
2 2
– Is it safe for A initially? a 11 i 4 14 e • A cut (S, V - S) S a 11 i 4 14 e S
7 6 7 6
8 10 is a partition of vertices V- S  8 10
 V- S
• Later on: S h
1
g 2
f h
1
g 2
f
into disjoint sets S and V - S
– Let S  V be any set of vertices that includes h but not
g (so that g is in V - S)
• An edge crosses the cut

– In any MST, there has to be one edge (at least) that (S, V - S) if one endpoint is in S
connects S with V - S and the other in V – S
– Why not choose the edge with minimum weight (h,g)?

9 10

9 10

Definitions (cont’d) Theorem


• A cut respects a set A b
8
c
7
d • Let A be a subset of some MST (i.e., T), (S, V - S) be a
4
2
9 cut that respects A, and (u, v) be a light edge crossing
of edges  no edge S a 11 i 4 14 e S (S, V-S). Then (u, v) is safe for A .
in A crosses the cut
7 6
V- S  8 10 Proof:
 V- S
h
1
g 2
f • Let T be an MST that includes A
• An edge is a light edge – edges in A are shaded
S

crossing a cut  its weight is minimum over all • Case1: If T includes (u,v), then
edges crossing the cut it would be safe for A
u
• Case2: Suppose T does not include
– Note that for a given cut, there can be > 1 light
the edge (u, v)
edges crossing it • Idea: construct another MST T’ v
V-S
that includes A  {(u, v)}
11 12

11 12

2
Theorem - Proof Theorem – Proof (cont.)
• T contains a unique path p between u and v T’ = T - {(x, y)}  {(u, v)}
• Path p must cross the S
Have to show that T’ is an MST: S

x
cut (S, V - S) at least x • (u, v) is a light edge
once: let (x, y) be that edge  w(u, v) ≤ w(x, y) u y
u y
p
• Let’s remove (x,y)  breaks p • w(T ’) = w(T) - w(x, y) + w(u, v)
v
T into two components. v ≤ w(T) V-S
V-S
• Adding (u, v) reconnects the components • Since T is a spanning tree
T’ = T - {(x, y)}  {(u, v)} w(T) ≤ w(T ’)  T’ must be an MST as well
13 14

13 14

Theorem – Proof (cont.) Prim’s Algorithm


Need to show that (u, v) is safe for A: • The edges in set A always form a single tree
• Starts from an arbitrary “root”: VA = {a}
i.e., (u, v) can be a part of an MST
• At each step:
• A  T and (x, y)  T 
8 7
b c d
S – Find a light edge crossing (VA, V - VA) 4 9
2
(x, y)  A  A T’ x – Add this edge to A a 11 i 4 14 e
7 6
– Repeat until the tree spans all vertices 8 10
• A  {(u, v)}  T’ u y h
1
g 2
f

p
• Since T’ is an MST
v
 (u, v) is safe for A V-S

15 16

15 16

How to Find Light Edges Quickly?


How to Find Light Edges Quickly? (cont.)
Use a priority queue Q: 8 7
• After adding a new node to VA we update the weights of all
b c d
• Contains vertices not yet 4 9 the nodes adjacent to it
2
e.g., after adding a to the tree, k[b]=4 and k[h]=8
included in the tree, i.e., (V – VA) a 11 i 4 14 e
– VA = {a}, Q = {b, c, d, e, f, g, h, i} 8
7 6
10 • Key of v is  if v is not adjacent to any vertices in VA
h g f
• We associate a key with each vertex v: 1 2
8 7
key[v] = minimum weight of any edge (u, v) b c d
4 9
connecting v to VA 2
a 11 i 4 14 e
7 6
8 10
Key[a]=min(w1,w2) w1
h g 2
f
w2 1
a

17 18

17 18

3
PRIM(V, E, w, r)
1. Q← 
2. for each u  V
3. do key[u] ← ∞
4. π[u] ← NIL
5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0

Prim’s Algorithm
7. while Q  
8. do u ← EXTRACT-MIN(Q)
9. for each v  Adj[u]
10. do if v  Q and w(u, v) < key[v]
11. then π[v] ← u
12. DECREASE-KEY(Q, v, w(u, v))

19 20

19 20

Example
  
b
8
c
7
d 0  
4 9
 2  Q = {a, b, c, d, e, f, g, h, i}
a 11 i 4 14 e VA = 
7 6
8 10 Extract-MIN(Q)  a
h g 2
f
1
  

4
  
 [b] = a
Example/Dry Run 4
b
8

 2
c
7
d
9

key [b] = 4
key [h] = 8  [h] = a
a 11 i 4 14 e
8
7 6
10 4 8
h g 2
f Q = {b, c, d, e, f, g, h, i} VA = {a}
1
8  
Extract-MIN(Q)  b
21 22

21 22

Example Example
4 
8  key [c] = 8  [c] = b 4 8 7 key [h] = 7  [h] = i
8 7 8
b c d key [h] = 8  [h] = a - unchanged b c
7
d key [g] = 6  [g] = i
4 9 9
 2  4
8 8 2 2  7 46 7
a 11 i 4 14 e 11
a i 4 14 e
7 6 Q = {c, d, e, f, g, h, i} VA = {a, b} 6
Q = {d, e, f, g, h} VA = {a, b, c, i}
8 10 7
Extract-MIN(Q)  f
8
h g f Extract-MIN(Q)  c h g f
10
1 2
8   1 2
key [d] = 7  [d] = c 87 
6 4
4 8 
7 key [g] = 2  [g] = f
8 7 key [f] = 4  [f] = c 4 8 7

4
b c d
9 key [i] = 2  [i] = c b
8
c
7
d key [d] = 7  [d] = c unchanged
2 2
  9
11 14
4
2 2 
10 key [e] = 10  [e] = f
a i 4 e
a 11 i 14 e
8
7 6
7 4 8 2
4 7 10 2 7
10 7 6
8 10
h g 2
f Q = {d, e, f, g, h, i} VA = {a, b, c} Q = {d, e, g, h} VA = {a, b, c, i, f}
1 h g f
 
8 4
Extract-MIN(Q)  i
1
6
2
4
Extract-MIN(Q)  g
23
7 2 24

23 24

4
Example Example
4 8 7 key [h] = 1  [h] = g
8 7
b c d 7 10 1
4 9
2 2 10 Q = {d, e, h} VA = {a, b, c, i, f, g} 4 8 7
a 11 i 14 e
4
Extract-MIN(Q)  h 8 7
8
7 6
10
b c d
9 9
key [e] = 9  [e] = f
4
2 2 10
h
1
g 2
f 9
a 11 i 4 14 e
7 2 4 7 10
1
7 6 Q = {e} VA = {a, b, c, i, f, g, h, d}
4 8 7 8 10
8 7
Q = {d, e} VA = {a, b, c, i, f, g, h} h g f Extract-MIN(Q)  e
b c d 1 2
4 9 Extract-MIN(Q)  d 1 2 4 Q =  VA = {a, b, c, i, f, g, h, d, e}
2 2 10
a 11 i 4 14 e
7 6
8 10
h g 2
f
1
1 2 4
25 26

25 26

PRIM(V, E, w, r)
1. Q← 
Total time: O(VlgV + ElgV) = O(ElgV)
2. for each u  V
3. do key[u] ← ∞ O(V) if Q is implemented
as a min-heap
4. π[u] ← NIL
5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0 O(lgV)

7. while Q   Min-heap

Analysis 8. do u ← EXTRACT-MIN(Q)
Executed |V| times
Takes O(lgV)
operations:
O(VlgV)
9. for each v  Adj[u] Executed O(E) times total
10. do if v  Q and w(u, v) < key[v] Constant O(ElgV)

11. then π[v] ← u Takes O(lgV)


12. DECREASE-KEY(Q, v, w(u, v))

27 28

27 28

Using Fibonacci Heaps Prim’s Algorithm


• Prim’s algorithm is a “greedy” algorithm
• Depending on the heap implementation, running time
could be improved! – Greedy algorithms find solutions based on a sequence
of choices which are “locally” optimal at each step.

• Nevertheless, Prim’s greedy strategy produces a


globally optimum solution!
– See proof for generic approach (i.e., slides 12-15)

29 30

29 30

5
A different instance of the
generic approach
S
(instance 1)

(instance 2)
u

tree1
v

Kruskal’s Algorithm • A is a forest containing connected


V-S

u
components
– Initially, each component is a single
vertex v
tree2
• Any safe edge merges two of
these components into one
31 – Each component is a tree 32

31 32

Kruskal’s Algorithm Kruskal’s Algorithm


• How is it different from Prim’s algorithm? • Start with each vertex being its b
8
c
7
d
9
– Prim’s algorithm grows one own component
4
2
a 11 i 14 e
tree all the time • Repeatedly merge two 4
7 6
– Kruskal’s algorithm grows tree1 components into one by
8 10
h g 2
f
multiple trees (i.e., a forest) 1
choosing the light edge that
connects them We would add
at the same time. edge (c, f)
u
– Trees are merged together • Which components to consider
using safe edges at each iteration?
v
– Since an MST has exactly |V| - 1 tree2
– Scan the set of edges in
monotonically increasing order by
edges, after |V| - 1 merges,
weight
we would have only one component
33 34

33 34

KRUSKAL(V, E, w) Example
1. A←  1. Add (h, g) {g, h}, {a}, {b}, {c}, {d}, {e}, {f}, {i}
for each vertex v  V
8 7
2. b c d
9
2. Add (c, i) {g, h}, {c, i}, {a}, {b}, {d}, {e}, {f}
4
2 3. Add (g, f) {g, h, f}, {c, i}, {a}, {b}, {d}, {e}
3. do MAKE-SET(v) a 11 14 e 4.
i 4 Add (a, b) {g, h, f}, {c, i}, {a, b}, {d}, {e}
4. sort E into non-decreasing order by w 8
7 6
10 5. Add (c, f) {g, h, f, c, i}, {a, b}, {d}, {e}

5. for each (u, v) taken from the sorted list h


1
g 2
f 6. Ignore (i, g) {g, h, f, c, i}, {a, b}, {d}, {e}
7. Add (c, d) {g, h, f, c, i, d}, {a, b}, {e}
6. do if FIND-SET(u)  FIND-SET(v) 1: (h, g) 8: (a, h), (b, c) 8. Ignore (i, h) {g, h, f, c, i, d}, {a, b}, {e}
7. then A ← A  {(u, v)} 2: (c, i), (g, f) 9: (d, e) 9. Add (a, h) {g, h, f, c, i, d, a, b}, {e}
4: (a, b), (c, f) 10: (e, f)
UNION(u, v)
10. Ignore (b, c) {g, h, f, c, i, d, a, b}, {e}
8. 6: (i, g) 11: (b, h) 11. Add (d, e) {g, h, f, c, i, d, a, b, e}
9. return A 7: (c, d), (i, h) 14: (d, f) 12. Ignore (e, f) {g, h, f, c, i, d, a, b, e}
{a}, {b}, {c}, {d}, {e}, {f}, {g}, {h}, {i} 13. Ignore (b, h) {g, h, f, c, i, d, a, b, e}
14. Ignore (d, f) {g, h, f, c, i, d, a, b, e}
35 36

35 36

6
Implementation of Kruskal’s Algorithm Operations on Disjoint Data Sets
8 • MAKE-SET(u) – creates a new set whose only
• Uses a disjoint-set data
7
b c d
4
2
9 member is u
structure (see Chapter a 11 i 4 14 e • FIND-SET(u) – returns a representative element
7 6
21) to determine whether 8 10
from the set that contains u
h g 2
f
1
an edge connects – Any of the elements of the set that has a particular
We would add
vertices in different edge (c, f) property
components – E.g.: Su = {r, s, t, u}, the property is that the element be
the first one alphabetically
FIND-SET(u) = r FIND-SET(s) = r
– FIND-SET has to return the same value for a given set
37 38

37 38

Operations on Disjoint Data Sets KRUSKAL(V, E, w)


1. A← 
• UNION(u, v) – unites the dynamic sets that
2. for each vertex v  V
contain u and v, say Su and Sv O(V)
3. do MAKE-SET(v)
– E.g.: Su = {r, s, t, u}, Sv = {v, x, y} 4. sort E into non-decreasing order by w O(ElgE)
UNION (u, v) = {r, s, t, u, v, x, y} 5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
• Running time for FIND-SET and UNION
7. then A ← A  {(u, v)} O(lgV)
depends on implementation. 8. UNION(u, v)
• Can be shown to be α(n)=O(lgn) where α() is a 9. return A
Running time: O(V+ElgE+ElgV)=O(ElgE) – dependent on
very slowly growing function (see Chapter 21) the implementation of the disjoint-set data structure
39 40

39 40

KRUSKAL(V, E, w) (cont.) Kruskal’s Algorithm


1. A← 
• Kruskal’s algorithm is a “greedy” algorithm
2. for each vertex v  V
O(V)
3. do MAKE-SET(v) • Kruskal’s greedy strategy produces a globally
4. sort E into non-decreasing order by w O(ElgE) optimum solution
5. for each (u, v) taken from the sorted list O(E) • Proof for generic approach
S

6. do if FIND-SET(u)  FIND-SET(v) x
applies to Kruskal’s
7. then A ← A  {(u, v)} O(lgV) u y
8. UNION(u, v) algorithm too
9. return A
v
- Running time: O(V+ElgE+ElgV)=O(ElgE) O(ElgV) V-S
- Since E=O(V2), we have lgE=O(2lgV)=O(lgV) 41 42

41 42

7
Problem 1 Problem 1 (cont.)
• (Exercise 23.2-3, page 573) Compare Prim’s algorithm
with and Kruskal’s algorithm assuming:
(b) dense graphs
In this case, E=O(V2)
(a) sparse graphs:
In this case, E=O(V)
Kruskal:
Kruskal:

O(ElgE)=O(V2lgV2)=O(2V2lgV)=O(V2lgV)
O(ElgE)=O(VlgV)
Prim:
Prim: - binary heap: O(ElgV)=O(V2lgV)
- binary heap: O(ElgV)=O(VlgV) - Fibonacci heap: O(VlgV+E)=O(VlgV+V2)=O(V2)
- Fibonacci heap: O(VlgV+E)=O(VlgV)
43 44

43 44

Problem 2 Problem 2 (cont.)


1. A← 
2. for each vertex v  V
(Exercise 23.2-4, page 574): Analyze the O(V)
running time of Kruskal’s algorithm when 3. do MAKE-SET(v)
weights are in the range [1 … V] 4. sort E into non-decreasing order by w O(ElgE)
5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)} O(lgV)
8. UNION(u, v)
9. return A
- Sorting can be done in O(E) time (e.g., using counting sort)
45 - However, overall running time will not change, i.e, O(ElgV)46

45 46

Problem 3 Problem 4
• Suppose that some of the weights in a connected • (Exercise 23.2-2, page 573) Analyze Prim’s
graph G are negative. Will Prim’s algorithm still algorithm assuming:
work? What about Kruskal’s algorithm? Justify
your answers. (a) an adjacency-list representation of G
– Yes, both algorithms will work with negative weights.
Review the proof of the generic approach; there is no O(ElgV)
assumption in the proof about the weights being
positive.
(b) an adjacency-matrix representation of G
O(ElgV+V2)

47 48

47 48

8
PRIM(V, E, w, r) PRIM(V, E, w, r)
1. Q←  1. Q← 
Total time: O(VlgV + ElgV) = O(ElgV) Total time: O(VlgV + ElgV+V2) = O(ElgV+V2)
2. for each u  V 2. for each u  V
3. do key[u] ← ∞ O(V) if Q is implemented 3. do key[u] ← ∞ O(V) if Q is implemented
as a min-heap as a min-heap
4. π[u] ← NIL 4. π[u] ← NIL
5. INSERT(Q, u) 5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0 O(lgV) 6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0 O(lgV)

7. while Q   Executed |V| times


Min-heap 7. while Q   Executed |V| times
Min-heap
operations: operations:
8. do u ← EXTRACT-MIN(Q) Takes O(lgV) O(VlgV) 8. do u ← EXTRACT-MIN(Q) Takes O(lgV) O(VlgV)
9. for each v  Adj[u] Executed O(E) times 9. for (j=0; j<|V|; j++) Executed O(V2) times total
10. do if v  Q and w(u, v) < key[v] Constant O(ElgV) 10. if (A[u][j]=1) Constant
11. then π[v] ← u Takes O(lgV) 11. if v  Q and w(u, v) < key[v]
12. DECREASE-KEY(Q, v, w(u, v)) 12. then π[v] ← u Takes O(lgV) O(ElgV)
49
13. DECREASE-KEY(Q, v, w(u, v)) 50

49 50

Problem 5 Problem 6
• Find an algorithm for the “maximum” spanning • (Exercise 23.1-8, page 567) Let T be a MST of
tree. That is, given an undirected weighted a graph G, and let L be the sorted list of the
graph G, find a spanning tree of G of maximum edge weights of T. Show that for any other MST
cost. Prove the correctness of your algorithm. T’ of G, the list L is also the sorted list of the
– Consider choosing the “heaviest” edge (i.e., the edge edge weights of T’
associated with the largest weight) in a cut. The
generic proof can be modified easily to show that this
approach will work.
– Alternatively, multiply the weights by -1 and apply
either Prim’s or Kruskal’s algorithms without any
modification at all! T, L={1,2} T’, L={1,2}

51 52

51 52

You might also like