CSC2211: Algorithms

Greedy Graph Algorithm

Sajib Hasan
[email protected]
American International University Bangladesh
 Spanning Tree
• A spanning tree of G is a subgraph which
– is a tree
– contains all vertices of G

 How many edges

are there in a
spanning tree, if
there are V
vertices?

CSC2211: Algorithms Sajib Hasan Graph Algorithm2

 Minimum Spanning Trees
• Undirected, connected graph
G = (V,E)
• Weight function W: E  
(assigning cost or length or other
values to edges)

 Spanning tree: tree that connects all vertices

 Minimum spanning tree (MST): spanning tree T
that minimizes w(T )  
( u ,v )T
w(u , v)

CSC2211: Algorithms Sajib Hasan Graph Algorithm3

 Generic MST Algorithm
Generic-MST
Generic-MST(G, (G, w)w)
11 AA := 
:= // Contains edges
// Contains edges that
that belong
belong to
to aa MST
MST
22 while A does not form a spanning
while A does not form a spanning tree do tree do
33 Find
Find an an edge
edge (u,v)
(u,v) that
that is
is safe
safe for
for AA
44 := AA 
AA := {(u,v)}
{(u,v)}
55 return
return A A

A safe edge is an edge that does not destroy A’s property.

MoreSpecific-MST(G,
MoreSpecific-MST(G, w) w)
11 AA := //
:= // Contains edges
Contains edges that
that belong
belong toto aa MST
MST
22 while
while AA does
does not
not form
form aa spanning
spanning tree
tree do
do
3.1
3.1 Make
Make aa cut
cut (S,
(S, V-S)
V-S) of
of GG that
that respects
respects AA
3.2
3.2 Take
Take thethe min-weight
min-weight edge
edge (u,v)
(u,v) connecting
connecting SS toto V-S
V-S
44 := AA {(u,v)}
AA := {(u,v)}
55 return
return AA

CSC2211: Algorithms Sajib Hasan Graph Algorithm4

CSC2211: Algorithms Sajib Hasan Graph Algorithm5
 Optimal Substructure
MST(G ) = T MST(G’ ) = T – (u,v )
u ”u +v”
v

• Rationale:
– If G’ would have a cheaper ST T’, then we would get
a cheaper ST of G: T’ + (u, v)

CSC2211: Algorithms Sajib Hasan Graph Algorithm6

 Idea for an Algorithm
• We have to make V–1 choices (edges of the
MST) to arrive at the optimization goal.
• After each choice we have a sub-problem that
is one vertex smaller than the original problem.
– A dynamic programming algorithm would consider all
possible choices (edges) at each vertex.
– Goal: at each vertex cheaply determine an edge that
definitely belongs to an MST.

CSC2211: Algorithms Sajib Hasan Graph Algorithm7

 Greedy Choice
• Greedy choice property: locally optimal
(greedy) choice yields a globally optimal
solution.
• Theorem on Greedy choice for MST
– Let G=(V, E) and S V
– S is a cut of G (it splits G into parts S and V-S)
– (u,v) is a light edge if it is a min-weight edge of G
that connects S and V-S
– Then (u,v) belongs to a MST T of G.

CSC2211: Algorithms Sajib Hasan Graph Algorithm8

 ...Greedy Choice
• Proof
– Suppose (u,v) is light but (u,v)  any MST
– look at path from u to v in some MST T
– Let (x, y) be the first edge on a path from u to v in T that crosses
from S to V–S. Swap (x, y) with (u,v) in T.
– this improves cost of T  contradiction (T is supposed to be an
MST)

V-S
S
y
x
u v

CSC2211: Algorithms Sajib Hasan Graph Algorithm9

 Prim-Jarnik Algorithm
• Vertex based algorithm
• Grows a single MST T one vertex at a time
• The set A covers the portion of T that was already computed
• Annotate all vertices v outside of the set A with v.key the minimum weight of
an edge that connects v to a vertex in A
(v.key =  if no such edge exists)

CSC2211: Algorithms Sajib Hasan Graph Algorithm10

 ...Prim-Jarnik Algorithm
MST-Prim(G, s)
01 for each vertex u  G.V
02 u.key := 
03 u.pred := NIL
04 s.key := 0
05 init(Q, G.V) // Q is a priority queue
06 while not isEmpty(Q)
07 u := extractMin(Q) // add u to T
08 for each v  u.adj do
09 if v  Q and w(u,v) < v.key then
10 v.key := w(u,v)
11 modifyKey(Q,v)
12 v.pred := u updating
keys

CSC2211: Algorithms Sajib Hasan Graph Algorithm11

 Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B A NIL 0
3 4 13 9
A 12 B NIL 
6 I
8 H 10 C NIL 
F E
5 3 D NIL 
MST-Prim(Graph,A) 1
G E NIL 

F NIL 
A Node
G NIL 
Pred
Key H NIL 

I NIL 

CSC2211: Algorithms Sajib Hasan Graph Algorithm12

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B B A 4
3 4 13 9
A 12 H A 8
6 I
8 H 10 C NIL 
F E
5 3
D NIL 
1
G
E NIL 

F NIL 
A Node A

Pred NIL G NIL 

Key 0
I NIL 

CSC2211: Algorithms Sajib Hasan Graph Algorithm13

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B H A 8
3 4 13 9
A 12 C B 8
6 I
8 H 10 F NIL 
F E
5 3
E NIL 
1
G
F NIL 

G NIL 
A Node A B

Pred NIL A I NIL 

Key 0 4

CSC2211: Algorithms Sajib Hasan Graph Algorithm14

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B G H 1
3 4 13 9
A 12 I H 6
6 I
8 H 10 C B 8
F E
5 3
D NIL 
1
G
E NIL 

F NIL 
A Node A B H

Pred NIL A A

Key 0 4 8

CSC2211: Algorithms Sajib Hasan Graph Algorithm15

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B F G 3
3 4 13 9
A 12 I G 5
6 I
8 H 10 C B 8
F E
5 3
D NIL 
1
G
E NIL 

A Node A B H G

Pred NIL A A H

Key 0 4 8 1

CSC2211: Algorithms Sajib Hasan Graph Algorithm16

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B C F 4
3 4 13 9
A 12 I G 5
6 I
8 H 10 E F 10
F E
5 3
D F 13
1
G

A Node A B H G F

Pred NIL A A H G

Key 0 4 8 1 3

CSC2211: Algorithms Sajib Hasan Graph Algorithm17

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B I C 3
3 4 13 9
A 12 D C 6
6 I
8 H 10 E F 10
F E
5 3
1
G

A Node A B H G F C

Pred NIL A A H G F

Key 0 4 8 1 3 4

CSC2211: Algorithms Sajib Hasan Graph Algorithm18

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B D C 6
3 4 13 9
A 12 E F 10
6 I
8 H 10
F E
5 3
1
G

A Node A B H G F C I

Pred NIL A A H G F C

Key 0 4 8 1 3 4 3

CSC2211: Algorithms Sajib Hasan Graph Algorithm19

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B E D 9
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

A Node A B H G F C I D

Pred NIL A A H G F C C

Key 0 4 8 1 3 4 3 6

CSC2211: Algorithms Sajib Hasan Graph Algorithm20

 ...Prim-Jarnik’s Example
Q is empty
Node Pred key

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

A Node A B H G F C I D E

Pred NIL A A H G F C C D

Key 0 4 8 1 3 4 3 6 9

CSC2211: Algorithms Sajib Hasan Graph Algorithm21

 ...Prim-Jarnik’s Example
Q is empty
Node Pred Key

6
C D
4 B
3 4 9
A
I
8 H
F E
3
1
G

A Node A B H G F C I D E Total
Cost
Pred NIL A A H G F C C D

Key 0 4 8 1 3 4 3 6 9 38

CSC2211: Algorithms Sajib Hasan Graph Algorithm22

 Implementation Issues
MST-Prim(G,r)
01 for each vertex u  G.V
02 u.key := 
03 u.parent := NIL
04 r.key := 0
05 init(Q, G.V) // Q is a priority queue
06 while not isEmpty(Q)
07 u := extractMin(Q) // add u to T
08 for each v  u.adj do
09 if v  Q and w(u,v) < v.key then
10 v.key := w(u,v)
11 modifyKey(Q,v)
12 v.parent := u

CSC2211: Algorithms Sajib Hasan Graph Algorithm23

 Priority Queues
• A priority queue maintains a set S of elements, each with
an associated key value.
• We need PQ to support the following operations
– init( Q: PriorityQueue, S: VertexSet)
– extractMin( Q: PriorityQueue): Vertex
– modifyKey( Q: PriorityQueue, v: Vertex)

• To choose how to implement a PQ, we need to count

how many times the operations are performed:
– init is performed just once and runs in O(n)

CSC2211: Algorithms Sajib Hasan Graph Algorithm24

CSC2211: Algorithms Sajib Hasan Graph Algorithm25
CSC2211: Algorithms Sajib Hasan Graph Algorithm26
 Prim-Jarnik’s Running Time
• Time = |V|*T(extractMin) + O(E)*T(modifyKey)

Q T(extractMin) T(modifyKey) Total

array O(V) O(1) O(V2)
binary heap O(log V) O(log V) O(E logV)

• E  V-1, E < V2, E = O(V2)

• Binary heap implementation:
– Time = O(V logV + E logV) = O(V2 logV) = O(E logV)

CSC2211: Algorithms Sajib Hasan Graph Algorithm27

 Kruskal's Algorithm
• Edge based algorithm
• Add edges one at a time in increasing
weight order.
• The algorithm maintains A: a forest of
trees.
• An edge is accepted if it connects
vertices of distinct trees (the cut
respects A).

CSC2211: Algorithms Sajib Hasan Graph Algorithm28

 ...Kruskal's Algorithm
• The algorithm keeps adding the cheapest edge that connects two
trees of the forest
MST-Kruskal(G)
01 A := 
02 init(S) // Init disjoint-set
03 for each vertex v  G.V
04 addSingletonSet(S,v)
05 sort the edges of G.E by non-decreasing w(u,v)
06 for each edge (u,v)  E in sorted order do
07 if findSet(S,u)  findSet(S,v) then
08 A := A  {(u,v)}
09 unionSets(S,u,v)
10 return A
CSC2211: Algorithms Sajib Hasan Graph Algorithm29
 Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, C, D, E, F, G,
G, H,
H, I }
E’ = { HG-1,
HG-1, CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm30

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, C, D, E, F, GH, I }
E’ = { CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm31

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, CI, D, E, F, GH }
E’ = { GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm32

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, CI, D, E, FGH }
E’ = { CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm33

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, CFGHI, D, E }
E’ = { AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm34

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { AB, CFGHI, D, E }
E’ = { HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm35

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { AB, CDFGHI, E }
E’ = { BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm36

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { ABCDFGHI, E }
E’ = { AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm37

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { ABCDEFGHI }
E’ = { EF-10, BH-12, DF-13 }

CSC2211: Algorithms Sajib Hasan Graph Algorithm38

 ...Kruskal’s Example

6
8 C D
4 B
3 4 9
A
I
H E
3 F
1
G

S = { ABCDEFGHI }
E’ = { HG-1, CI-3, GF-3, CF-4, AB-4, CD-6, BC-8, DE-9 }; Total Cost := 38

CSC2211: Algorithms Sajib Hasan Graph Algorithm39

 Implementation Factors:
Disjoint Sets
• We need to maintain a disjoint partitioning of a
set, i.e., a collection S of disjoint sets.
Operations:
– addSingeltonSet(S:Set, x:Vertex)
• S := S  {{x}}
– findSet(S:Set, x:Vertex): Set
• returns X  S such that x  X
– unionSets(S:Set, x:Vertex, y:Vertex)
• X := findSet(S:Set, x)
• Y := findSet(S:Set, y)
• S := S – {X, Y}  {X Y}

CSC2211: Algorithms Sajib Hasan Graph Algorithm40

CSC2211: Algorithms Sajib Hasan Graph Algorithm41
CSC2211: Algorithms Sajib Hasan Graph Algorithm42
 Disjoint-set forests

CSC2211: Algorithms Sajib Hasan Graph Algorithm43

 Implementation Factors:
Disjoint Sets as Lists
• Each set is a list of elements identified by the first
element, all elements in the list point to the first element
• UnionSets: add a shorter list to a longer one, O(min{|
C(u)|, |C(v)|})
• MakeSet/FindSet: O(1)

1 2 3 4 A B C
 

1 2 3 4 A B C


CSC2211: Algorithms Sajib Hasan Graph Algorithm44

 Kruskal Running Time
• Initialization O(V) time
• Sorting the edges (E log E) = (E log V)
• O(E) calls to FindSet
• Union costs
– Let t(v) be the number of times v is moved to a new cluster
– Each time a vertex is moved to a new cluster the size of the
cluster containing the vertex at least doubles: t(v) log V
– Total time spent doing Union  t (v)  V log V
vV

• Total time: O(E log V)

CSC2211: Algorithms Sajib Hasan Graph Algorithm45