Algorithm: Design and Analysis

Greedy Algorithm
Dr. Khaled W. Mahmoud

xxx

1
Agenda
 Introduction
 Problems

1. Knapsack
2. Activity Selection

2
Introduction to Greedy Algorithm
 Simple approach
 Used for optimization problems
 Idea:

◦ When we have a (Greedy) choice to make, make the one that

looks best right now
◦ Make a locally optimal choice in the hope of getting a globally
optimal solution
◦ This means that the algorithm picks the best solution at the
moment without regard for consequences.
 does not consider the big picture

3
Introduction to Greedy Algorithm
 Top down solution, problems decrease in size
 Greedy algorithms don’t always yield an

optimal solution

Greedy-choice property: A globally

optimal solution can be arrived at by
making a locally optimal (greedy)
choice.

4
Introduction: Example
Making Change Example
 Problem: Find the minimum # of quarters(25), dimes

(10), nickels(5), and pennies(1) that total to a given

amount.
 Greedy Choice:

◦ Start by grabbing quarters until exceeds amount, then dimes,

then nickels, then pennies.

5
Introduction: Example
Amount = 92¢
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
1¢ 1¢

 Does this lead to an optimal # of coins?

6
Introduction: Example
 Problem:
◦ Find the minimum # of 4, 3, and 1 cent coins to make up 6
cents.

 Greedy Choice:
◦ Start by grabbing a 4 coin.

 Consequences:
◦ 4+1+1 = 6 solution but not optimal
◦ 3+3=6 better solution

 Greedy Algorithm does not work!

Problems
 The fractional knapsack problem
 The 0-1 knapsack problem
 Activity Selection problem

8
Fractional knapsack problem
 Greedy property: Pick the item with maximum value
per weight
v1 v2 vn
  ...  30
w1 w2 wn --- JD80
Item 3 20
+
Item 2
50 20 JD100
Item 1 30
20 +
10 10 JD60

JD 60 JD100 JD120 JD 240

JD6/KG JD5/KG JD 4/KG

This value is Optimal

9
Fractional Knapsack Problem: Alg
Fractional-Knapsack (w, v, n, C)
Sort w and v by v[i]/w[i] in decreasing order
// objects are ordered so that v(i)/w(i)  v(i+1)/w(i+1)
Load= empty
For i=1 to n and load < C
if w[i]<=C-load
Take all of item i w: array of weights
load+=w[i] v: array of values
Profit= profit+v[i] n: number of items
else C: Knapsack capacity.
Take C-load of item i
profit= profit+(c-load)*v[i]/w[i]
load=C

10
Fractional Knapsack Problem: Alg
 Running Time:
◦ O(n lg n) for sorting items by value per pound
◦ O(n) cost for “For Loop”
◦ Overall cost = O(n lg n)
Fractional Knapsack - Example
Item Weight Value v/w Weight limit W = 10
i w v
1 2 2 1

2 4 3 0.75

3 3 3 1

4 5 6 1.2

5 2 4 2

6 6 9 1.5
Fractional Knapsack - Example
item Weight Value v/w  Weight limit: 10
i w v ◦ Load=0, profit=0
5 2 4 2  Take item 5
6 6 9 1.5 ◦ 2 Kg, jd 4
 Take item 6
4 5 6 1.2
◦ 8 Kg, jd 13 (4+6*1.5)
1 2 2 1  Take 2 Kg of item 4
3 3 3 1 ◦ 10 Kg, jd 15.4
(13+2*1.2)
2 4 3 0.75
Problems
 The Fractional knapsack problem
 The 0-1 knapsack problem
 Activity Selection problem

14
0-1 knapsack problem: Greedy Sol
Greedy property: Pick the item with maximum value per weight

Item 3

Item 2

30 50 50
Item 1 20 jd100
20
10 +
JD100 JD120 10 jd60 30 jd120
JD 60

JD6/KG JD5/KG JD 4/KG jd160 50 +

This value is not Optimal

20 jd100

This greedy does not give the optimal

solution for this instance. jd220

This value is Optimal

15
0-1 knapsack problem: Greedy Sol
 Greedy property: Pick the item with maximum value

Item 3
20 jd100
Item 2
+
30 50
Item 1
20
10 30 jd120

JD 60 JD100 JD120

JD6/KG JD5/KG JD 4/KG Jd 220

 This greedy property gives the optimal solution for this instance with
value equal to jd 220.

16
0-1 knapsack problem: Greedy Sol
 Greedy property: Pick the item with maximum value

Item 3

Item 2
10 jd60
50 50 50
Item 1 50
+
20 jd120
10 20 jd100

JD 60 JD100 JD120
jd120 jd160
JD6/KG JD5/KG JD 4/KG

 This greedy property does not gives the optimal solution for this
instance.

17
Problems
 The fractional knapsack problem
 The 0-1 knapsack problem
 Activity Selection problem

18
Activity Selection Problem
 Problem: Schedule n activities that require exclusive
use of a common resource
◦ find max # of non-overlapping activities
 Problem Description
◦ S = {a1, . . . , an} – set of activities
◦ ai needs resource during period [si , fi)
◦ si = start time and fi = finish time of activity ai
◦ 0  si < f i < 
◦ Activities ai and aj are compatible if the intervals [si , fi) and
[sj, fj) do not overlap

19
Activity Selection Problem: Example
 Possible Greedy strategy is select event with minimum
duration i 1 2 3 4 5 6 7 8 9
si 2 2 4 1 5 8 9 11 13
fi 3 5 7 8 9 10 11 14 16
Di 1 3 3 7 4 2 2 3 3

 Solution is S= {a1,a6,a3,a8} |S|=4.

◦ It is an optimal Solution, BUT Minimum duration is not optimal for all
instances. i 1 2 3
 Counter example si 1 3 4
◦ Greedy Solution ={a2}
fi 4 5 8
◦ Optimal Solution ={a1,a3}
Di 3 2 4
20
Activity Selection Problem: Example
 Possible Greedy strategy is select event with The
Earliest Starting Time
 Not working: Counter Example

i 1 2 3 4 5 6 7 8 9
si 2 2 4 1 5 8 9 11 13
fi 3 5 7 17 9 10 11 14 16

21
Activity Selection Problem: Example
 Possible Greedy strategy is select event Conflicting
with the Fewest Other Events
 Not working: Counter Example

22
Activity Selection Problem : Example
 Possible Greedy strategy is select event with minimum finish
time i 1 2 3 4 5 6 7 8 9
si 2 2 4 1 5 8 9 11 13
fi 3 5 7 8 9 10 11 14 16

◦ Solution is S= {a1,a3,a6,a8} |S|=4.

 By symmetry : The maximum start time also gives the

optimal solution for all instances.
◦ Solution is S= {a9,a7,a5,a2} |S|=4. Different combination!

23
Activity Selection Problem
 Greedy strategies:
• Minimum Duration: Not Optimal
• Minimum Finish Time: Optimal
• Maximum Start Time: Optimal
• Minimum Start Time : Not Optimal

24
Activity Selection Problem: Alg
// Using minimum finish time greedy algorithm strategy.
Activity selection(s, f, n)
Sort activities in both arrays by minimum finish time.
A = A U {a1}
k=1 s: array of start time.
f: array of finish time.
for i=2……..n n: number of the activities.
if s[i] >= f[k] A: Solution Set contains the selected activities
k: the index of the last taken activity.
A = A U {ai}
k=i

Return A

25
Activity Selection Problem: Alg
 It takes the start and finish times of the
activities , represented as arrays s and f
 The variable k indexes the most recent

addition to A
 Since the activities are sorted by finish time,

◦ fk is always the maximum finish time of any activity

in A
◦ fk = max {fi : ai  A}
Activity Selection Problem: Alg
 Let n be the total number of activities
◦ The algorithm first sorts the activities by finish time
taking: O(n lg n)
◦ Then the algorithm visits each activity exactly once,
doing a constant amount of work each time. This
takes O(n)
◦ Thus total time is O(n lg n)
Correctness
 This algorithm is Optimal
 Proof:
 Note that if ordered by fi, activity 1 has earliest finish
 Show that there exists an optimal solution that begins with a greedy choice
(activity 1).
 Let A   S be an optimal solution. If the first activity in A is  k!=1  (not greedy),
then there exists another optimal solution B that begins with 1.
 B = A - {k}    {1}
 Because  f 1≤ fk , activity 1 is still compatible with A
 |B|  =  |A| , so B is also optimal.
 Thus there exists an optimal solution that begins with a greedy choice.

28
 Now prove optimal substructure.
◦ Once we make the first greedy choice, now the problem is S’ = {i S: si
≥ f1) , with optimal solution A' = A - {1}.
◦ This solution must be optimal.
◦ If B' solves S' with more activities than A', adding activity 1 to B' makes
it bigger than A, which is a contradiction.
◦ Therefore, greedy choice yields an optimal solution.

29
When does Greedy Work?
Two key ingredients:
1. Optimal sub-structure
◦ An optimal solution to the entire problem
contains within it optimal solutions to sub-
problems (this is also true of dynamic
programming)
2. Greedy choice property

Greedy choice + Optimal sub-structure

establish the correctness of the greedy
algorithm