Greedy

Greedy Algorithms
Algorithms

Comp 122, Spring 2004

 Overview
 Like dynamic programming, used to solve optimization
problems.
 Problems exhibit optimal substructure (like DP).
 Problems also exhibit the greedy-choice property.
» When we have a choice to make, make the one that looks
best right now.
» Make a locally optimal choice in hope of getting a globally
optimal solution.

eedy - 2 Comp 122, Fall 2003 Lin / Devi

 Greedy Strategy
 The choice that seems best at the moment is the one we
go with.
» Prove that when there is a choice to make, one of the optimal
choices is the greedy choice. Therefore, it’s always safe to
make the greedy choice.
» Show that all but one of the subproblems resulting from the
greedy choice are empty.

eedy - 3 Comp 122, Fall 2003 Lin / Devi

 Activity-selection Problem
 Input: Set S of n activities, a1, a2, …, an.
» si = start time of activity i.
» fi = finish time of activity i.
 Output: Subset A of maximum number of compatible
activities.
» Two activities are compatible, if their intervals don’t overlap.

Example: Activities in each line

are compatible.

eedy - 4 Comp 122, Fall 2003 Lin / Devi

 Optimal Substructure
 Assume activities are sorted by finishing times.
» f 1  f2  …  fn .
 Suppose an optimal solution includes activity ak.
» This generates two subproblems.
» Selecting from a1, …, ak-1, activities compatible with one
another, and that finish before ak starts (compatible with ak).
» Selecting from ak+1, …, an, activities compatible with one
another, and that start after ak finishes.
» The solutions to the two subproblems must be optimal.
• Prove using the cut-and-paste approach.

eedy - 5 Comp 122, Fall 2003 Lin / Devi

 Recursive Solution
 Let Sij = subset of activities in S that start after ai
finishes and finish before aj starts.
 Subproblems: Selecting maximum number of mutually
compatible activities from Sij.
 Let c[i, j] = size of maximum-size subset of mutually
compatible activities in Sij.

 0 if Sij  
Recursive c[i, j ] 
max{c[i, k ]  c[k , j ]  1} if Sij  
Solution:  ik  j

eedy - 6 Comp 122, Fall 2003 Lin / Devi

 Greedy-choice Property
 The problem also exhibits the greedy-choice property.
» There is an optimal solution to the subproblem Sij, that
includes the activity with the smallest finish time in set Sij.
» Can be proved easily.
 Hence, there is an optimal solution to S that includes
a1.
 Therefore, make this greedy choice without solving
subproblems first and evaluating them.
 Solve the subproblem that ensues as a result of making
this greedy choice.
 Combine the greedy choice and the solution to the
subproblem.

eedy - 7 Comp 122, Fall 2003 Lin / Devi

 Recursive Algorithm
Recursive-Activity-Selector
Recursive-Activity-Selector(s, (s,f,f,i,i,j)j)
1.1. mm i+1
i+1
2.2. while
whilemm<<jjand andssmm<<ffi i
3.3. domm
do m+1
m+1
4.4. ifif mm<<jj
5.5. then
thenreturn
return{a{amm}}

Recursive-Activity-Selector(s,
Recursive-Activity-Selector(s,f,f,m,
m,j)j)
6.6. else return
elsereturn

Initial Call: Recursive-Activity-Selector (s, f, 0, n+1)

Complexity: (n)
Straightforward to convert the algorithm to an iterative one.
See the text.
eedy - 8 Comp 122, Fall 2003 Lin / Devi
 Typical Steps
 Cast the optimization problem as one in which we
make a choice and are left with one subproblem to
solve.
 Prove that there’s always an optimal solution that
makes the greedy choice, so that the greedy choice is
always safe.
 Show that greedy choice and optimal solution to
subproblem  optimal solution to the problem.
 Make the greedy choice and solve top-down.
 May have to preprocess input to put it into greedy
order.
» Example: Sorting activities by finish time.

eedy - 9 Comp 122, Fall 2003 Lin / Devi

 Elements of Greedy Algorithms
 Greedy-choice Property.
» A globally optimal solution can be arrived at by making a
locally optimal (greedy) choice.
 Optimal Substructure.

eedy - 10 Comp 122, Fall 2003 Lin / Devi

Minimum
Minimum Spanning
Spanning Trees
Trees

Comp 122, Spring 2004

 Minimum Spanning Trees
• Given: Connected, undirected, weighted graph, G
• Find: Minimum - weight spanning tree, T
• Example:
b 7 c
5 Acyclic subset of edges(E) that connects
a 1
all vertices of G.
3 -3
11
d e f
0 2 b c
5

a 1
3 -3

d e f
0

eedy - 12 Comp 122, Fall 2003 Lin / Devi

 Generic Algorithm
“Grows” a set A.

A is subset of some MST.

Edge is “safe” if it can be added to A without destroying this

invariant.

:=;
AA:= ;
while
whileAAnot
notcomplete
completetree
treedo
do
find
findaasafe
safeedge
edge(u,
(u,v);
v);
AA:=:=AA{(u,
{(u,v)}
v)}
od
od

eedy - 13 Comp 122, Fall 2003 Lin / Devi

 Definitions
no edge in the set crosses the cut
cut respects the edge set {(a, b), (b, c)}

a
5
b 7 c a light edge crossing cut
(could be more than one)
1 -3
11 3
cut partitions vertices into
d
0
e
2
f disjoint sets, S and V – S.

this edge crosses the cut

one endpoint is in S and the other is in V – S.

eedy - 14 Comp 122, Fall 2003 Lin / Devi

 Theorem 23.1
Theorem
Theorem23.1:
23.1:Let
Let(S,
(S,V-S)
V-S)be
beany
anycut
cutthat
thatrespects
respectsA, A,and
andlet
let(u,
(u,v)
v)
be
beaalight
lightedge
edgecrossing
crossing(S,
(S,V-S).
V-S).Then,
Then,(u,
(u,v)
v)isissafe
safefor
forA.
A.
Proof:
Let T be a MST that includes A.
Case: (u, v) in T. We’re done.
Case: (u, v) not in T. We have the following:
edge in A
(x, y) crosses cut.
x Let T´ = T - {(x, y)}  {(u, v)}.
y cut Because (u, v) is light for cut,
u
w(u, v)  w(x, y). Thus,
w(T´) = w(T) - w(x, y) + w(u, v)  w(T).
shows edges Hence, T´ is also a MST.
in T v So, (u, v) is safe for A.
eedy - 15 Comp 122, Fall 2003 Lin / Devi
 Corollary

In general, A will consist of several connected

components.

Corollary:
Corollary:IfIf(u,
(u,v)
v)isisaalight
lightedge
edgeconnecting
connectingoneoneCC
CCin
in(V,
(V,A)
A)
to
toanother
anotherCC
CCin in(V,
(V,A),
A),then
then(u,
(u,v)
v)isissafe
safefor
forA.
A.

eedy - 16 Comp 122, Fall 2003 Lin / Devi

 Kruskal’s Algorithm
 Starts with each vertex in its own component.
 Repeatedly merges two components into one by choosing a light
edge that connects them (i.e., a light edge crossing the cut
between them).
 Scans the set of edges in monotonically increasing order by
weight.
 Uses a disjoint-set data structure to determine whether an edge
connects vertices in different components.

eedy - 17 Comp 122, Fall 2003 Lin / Devi

 Prim’s Algorithm
 Builds one tree, so A is always a tree.
 Starts from an arbitrary “root” r .
 At each step, adds a light edge crossing cut (VA, V - VA) to A.
» VA = vertices that A is incident on.

eedy - 18 Comp 122, Fall 2003 Lin / Devi

 Prim’s Algorithm
 Uses a priority queue Q to find a light edge quickly.
 Each object in Q is a vertex in V - VA.
 Key of v is minimum weight of any edge (u, v), where u  VA.
 Then the vertex returned by Extract-Min is v such that there exists
u  VA and (u, v) is light edge crossing (VA, V - VA).
 Key of v is  if v is not adjacent to any vertex in VA.

eedy - 19 Comp 122, Fall 2003 Lin / Devi

 Prim’s Algorithm
QQ:=
:=V[G];
V[G];
for eachuuQQdo
foreach do Complexity:
key[u]
key[u]:= := Using binary heaps: O(E lg V).
od;
od; Initialization – O(V).
key[r]
key[r]:=
:=0;0; Building initial queue – O(V).
[r]
[r]:=
:=NIL;
NIL; V Extract-Min’s – O(V lgV).
whileQQ
while dodo E Decrease-Key’s – O(E lg V).
uu:=
:=Extract
Extract--Min(Q);
Min(Q);
for eachvvAdj[u]
foreach Adj[u]do do Using Fibonacci heaps: O(E + V lg V).
ififvvQQw(u,
w(u,v)v)<<key[v] then (see book)
key[v]then
[v]
[v]:=
:=u;u;
key[v]
key[v]:=
:=w(u,
w(u,v)
v)  decrease-key operation
fifi
od
od
od
od

Note: A = {(v, [v]) : v  v - {r} - Q}.

eedy - 20 Comp 122, Fall 2003 Lin / Devi
 Example of Prim’s Algorithm

Not in tree
a/0 5 b/ 7 c/

11 3 1 -3 Q=a b c d e f
0  
d/ e/ f/
0 2

eedy - 21 Comp 122, Fall 2003 Lin / Devi

 Example of Prim’s Algorithm

a/0 5 b/5 7 c/

11 3 1 -3 Q=b d c e f
5 11  
d/11 e/ f/
0 2

eedy - 22 Comp 122, Fall 2003 Lin / Devi

 Example of Prim’s Algorithm

a/0 5 b/5 7 c/7

11 3 1 -3 Q=e c d f
3 7 11 
d/11 e/3 f/
0 2

eedy - 23 Comp 122, Fall 2003 Lin / Devi

 Example of Prim’s Algorithm

a/0 5 b/5 7 c/1

11 3 1 -3 Q=d c f
0 1 2
d/0 e/3 f/2
0 2

eedy - 24 Comp 122, Fall 2003 Lin / Devi

 Example of Prim’s Algorithm

a/0 5 b/5 7 c/1

11 3 1 -3 Q=c f
1 2
d/0 e/3 f/2
0 2

eedy - 25 Comp 122, Fall 2003 Lin / Devi

 Example of Prim’s Algorithm

a/0 5 b/5 7 c/1

11 3 1 -3 Q=f
-3
d/0 e/3 f/-3
0 2

eedy - 26 Comp 122, Fall 2003 Lin / Devi

 Example of Prim’s Algorithm

a/0 5 b/5 7 c/1

11 3 1 -3 Q=
d/0 e/3 f/-3
0 2

eedy - 27 Comp 122, Fall 2003 Lin / Devi

 Example of Prim’s Algorithm

a/0 5 b/5 c/1

3 1 -3

d/0 e/3 f/-3

0

eedy - 28 Comp 122, Fall 2003 Lin / Devi