EECS 3101

Week 10 - Monday
Ilir Dema

1
Logistics
• Assignment A3 due Sunday, July 24

2
Minimum Spanning Tree

3
The Graph of interest today
A connected undirected weighted graph

G = (V, E) with weights w(e) for each e ∈ E

8
2

10 5
3

5 12

4
It has the smallest total weight

It covers all vertices in G

Minimum Spanning Tree

of graph G

It’s a connected,
acyclic subgraph

5
A Minimum Spanning
Tree

8
2

10 5
3

5 12

May NOT be unique

6
Applications of MST
Build a road network that connects all
towns and with the minimum cost.

7
Applications of MST
Connect all components with the least
amount of wiring.

8
Other applications
➔ Cluster analysis
➔ Approximation algorithms for the “travelling
salesman problem”
➔ ...

9
 In order to understand
minimum spanning tree
we need to first understand

tree

10
Tree:
undirected connected acyclic graph

A tree T with n vertices has

n-1
exactly _________ edges.

Removing one edge from T

disconnect the tree
will ________________________.

Adding one edge to T will

create a cycle
______________________.

11
The MST of a connected graph G = (V, E)
|V|
has________________ vertices.

because “spanning”

The MST of a connected graph G = (V, E)

|V| - 1
has________________ edges.

because “tree”
12
Now we are ready to talk about
algorithms

13
Idea #1
Start with T = G.E, then keep deleting edges until
an MST remains.

Which sounds more efficient

in terms of worst-case runtime?

Idea #2
Start with empty T, then keep adding edges
until an MST is built.
14
Hint

A undirected simple graph G with

n vertices can have at most
___________ edges.

15
 Note: Here T is an edge set
Idea #1
Start with T = G.E, then keep deleting edges until
an MST remains.

In worst-case, need to delete

O(|V|²) edges (n choose 2) - (n-1)

In worst-case, need to
Idea #2 add O(|V|) edges
Start with empty T, then keep adding edges
until an MST is built.
This is more efficient! 16
So, let’s explore more of Idea #2,
i.e.,
building an MST by adding edges
one by one
i.e.,
we “grow” a tree

17
The generic growing algorithm

GENERIC-MST(G=(V, E, w)):
|T| < |V|-1
T ← ∅
while T is not a spanning tree:
find a “safe” edge e
T ← T ∪ {e}
return T

What is a “safe” edge?

18
 “Safe” means it keeps the hope of
T growing into an MST.

“Safe” edge e for T

Loop Invariant:
Initially and after each iteration of the loop
T ⊆ some MST

If we make sure T
is always a subset GENERIC-MST(G=(V, E, w)):

of some MST while T ← ∅

we grow it, then

while T is not a spanning tree:
find a “safe” edge e
eventually T will T ← T ∪ {e}
become an MST. return T
19
Intuition

20
The generic growing algorithm
GENERIC-MST(G=(V, E, w)):
|T| < |V|-1
T ← ∅
while T is not a spanning tree:
find a “safe” edge e
T ← T ∪ {e}
return T

How to find a “safe” edge?

21
Two major algorithms we’ll learn

➔ Kruskal’s algorithm

➔ Prim’s algorithm

They are both based on

one theorem...

22
Theorem (Corollary 23.1 in CLRS)
● Let G(V, E) be a connected, weighted, undirected graph.
● Let T be a subset of E that is included in some MST for G
● let C be a connected component (tree) in the forest GT(V, T)
● Let (u, v) be a min-weight edge crossing C and some other
component in GT.
● Then, edge (u, v) is safe for T.

The greedy-choice property of MST

23
Example:

8
a b 2

10 5
3 c
5 12
e d

24
Two things that need to be worried about when
actually implementing the algorithm

➔ How to keep track of the connected

components?
➔ How to efficiently find the minimum
weighted edge?

Kruskal’s and Prim’s basically use different

data structures to do these two things.

25
Overview: Prim’s and Kruskal’s
Keep track of
Find minimum
connected
weight edge
components

Keep “one tree

use priority
Prim’s plus isolated
queue
vertices”

Sort all edges

Kruskal’s use “disjoint set” according to
weight

26
 Prim’s Kruskal’s

https://trendsofcode.files.wordpress.com/2014/09/p
rim-algorithm-animation-2.gif
https://trendsofcode.files.wordpress.com/2014/09/kruskal_
algorithm.gif 27
Prim’s MST algorithm

28
Prim’s algorithm: Idea
➔ Start from an arbitrary vertex as root
➔ Focus on growing one tree, add one edge at a time.
The tree is one component, each of the other
(isolated) vertices is a component.
➔ Add which edge? Among all edges that are incident
to the current tree (edges crossing components),
pick one with the minimum weight.
➔ How to get that minimum? Store all candidate
vertices in a Min-Priority Queue whose key is the
weight of the crossing edge (incident to tree).
29
PRIM-MST(G=(V, E, w)):
key[v] keeps the “shortest distance”
1 T ← {} between v and the current tree
2 for all v in V:
3 key[v] ← ∞ pi[v] keeps who, in the tree, is v
4 pi[v] ← NIL connected to via lightest edge.

5 Initialize priority queue Q with all v in V

6 pick arbitrary vertex r as root
7 key[r] ← 0
u is the next vertex to add to
8 while Q is not empty: current tree
9 u ← EXTRACT-MIN(Q) add edge, pi[u] is lightest
10 if pi[u] != NIL: vertex to connect to, “safe”
11 T ← T ∪ {(pi[u], u)}
12 for each neighbour v of u:
13 if v in Q and w(u, v) < key[v]:
14 DECREASE-KEY(Q, v, w(u, v))
15 pi[v] ← u all u’s neighbours’ distances to the
current tree need update 30
Trace an example!
Pick “a” as root
Q key pi

a 0 NIL
8
a b 2
b ∞ NIL
3 5
10 c
c ∞ NIL
5 12
e d
d ∞ NIL

Next, ExtractMin ! e ∞ NIL

31
ExtractMin (#1)
then update neighbours’ keys
a: 0, NIL

8
Q key pi
a b 2
b ∞ →8 NIL
5 →a
3 10 c
c ∞ NIL
5 12
e d
d ∞ NIL

e ∞ →3 NIL
→a 32
ExtractMin (#2)
then update neighbours’ keys
e: 3, a

8
Q key pi
a b 2
b 8→5 a →e
3 5
10 c
c ∞ NIL
5 12
e d
d ∞ →5 NIL
→e

33
ExtractMin (#3)
then update neighbours’ keys
b: 5, e

8
Q key pi
a b 2
c ∞ →2 NIL
5 →b
3 10 c
d 5 e
5 12
e d
Could also have extracted d
since its key is also 5 (min)
34
ExtractMin (#4)
then update neighbours’ keys
c: 2, b

8
Q key pi
a b 2
d 5 e
3 5
10 c
5 12
e d

35
ExtractMin (#4)
then update neighbours’ keys
d: 5, e

8
Q key pi
a b 2

3 5 Q is empty now.
10 c
5 12
e d

MST grown!
36
Correctness of Prim’s
The added edge is always a “safe” edge, i.e., the
minimum weight edge crossing components (vertices in
the tree and the rest) because of ExtractMin.

8
a b 2

3 5
10 c
5 12
e d
37
Runtime analysis: Prim’s
➔ Assume we use binary min heap to
implement the priority queue.
➔ Each ExtractMin take O(log V)
➔ In total V ExtractMin’s
➔ In total, check at most O(E) neighbours, each
check neighbour could lead to a
DecreaseKey which takes O(log V)

➔TOTAL: O( (V+E)log V ) = O(E log V)

38
In a connected graph G = (V, E)

|V| is in O(|E|) because…

|E| has to be at least |V|-1

Also, log |E| is in O(log |V|) because …

E is at most V²,
so log E is at most log V² = 2 log V, which is
in O(log V)

39
Kruskal’s MST
algorithm

40
Kruskal’s algorithm: idea
➔ Sort all edges according to weight, then start
adding to MST from the lightest one.
➔ Constraint: added edge must NOT cause a cycle
◆ In other words, the two endpoints of the edge must
belong to two different trees (components).
➔ The whole process is like unioning small trees into
a big tree.

41
 m = |E|
Pseudocode
KRUSKAL-MST(G(V, E, w)):
1 T ← {}
2 sort edges so that w(e1)≤w(e2)≤...≤w(em)
3 for i ← 1 to m:
4 # let (ui, vi) = ei
5 if ui and vi in different components:
6 T ← T ∪ {ei}

42
Example

6
a b 2

3 5
10 c
9 12
e d

43
Add (b, c), the lightest edge

6
a b 2

3 5
10 c
9 12
e d

44
Add (a, e), the 2nd lightest

6
a b 2

3 5
10 c
9 12
e d

45
Add (b, e), the 3rd lightest

6
a b 2

3 5
10 c
9 12
e d

46
Add (a, b), the 4th lightest ...
No! a, b are in the same component
Add (d, e) instead!
6
a b 2

3 5
10 c
9 12
e d

47
 MST grown!
Add (d, e) ...

6
a b 2

3 5
10 c
9 12
e d

48
Correctness of Kruskal’s
The added edge is always a “safe” edge, because
it is the minimum weight edge among all edges
that cross components

6
a b 2

3 5
10 c
9 12
e d
49
 m = |E|
Runtime ...
sorting takes O(E log E)
KRUSKAL-MST(G(V, E, w)):
1 T ← {}
2 sort edges so that w(e1)≤w(e2)≤...≤w(em)
3 for i ← 1 to m:
4 # let (ui, vi) = ei
5 if ui and vi in different components:
6 T ← T ∪ {ei}

How exactly do we do this two lines?

50
We need the Disjoint Set ADT
stores a collections of nonempty disjoint sets S1,
S2, …, Sk, each has a “representative”.

and supports the following operations

➔ MakeSet(x): create a new set {x}
➔ FindSet(x): return the representative of the
set that x belongs to
➔ Union(x, y): union the two sets that contain x
and y, if different

You’ll learn this in EECS 4101.

51
 m = |E|
Real Pseudocode
KRUSKAL-MST(G(V, E, w)):
1 T ← {}
2 sort edges so that w(e1)≤w(e2)≤...≤w(em)
3 for each v in V:
4 MakeSet(v)
5 for i ← 1 to m:
6 # let (ui, vi) = ei
7 if FindSet(ui) != FindSet(vi):
8 Union(ui, vi)
9 T ← T ∪ {ei}

52
Next
Exercises for BFS, DFS, MST

53