Giải thuật tham lam

Greedy algorithms
 Giải thuật tham lam (Greedy
algorithms)
• Mỗi thời điểm chỉ chọn một cách
• Không quay lui
• Hi vọng lựa chọn là hợp lý nhất

Sử dụng một phần tài liệu bài giảng CS161 Stanford University

2
Nội dung
• Phản ví dụ:
• Knapsack
• 3 ví dụ greedy algorithms:
• Activity Selection (lựa chọn hành động)
• Job Scheduling (lập lịch)
• Mã hóa Huffman

3
 Item:
Weight: 6 2 4 3 11
Capacity: 10 Value: 20 8 14 13 35

• Unbounded Knapsack:
• Suppose I have infinite copies of all items.
• What’s the most valuable way to fill the knapsack?
Total weight: 10
Total value: 42

• “Greedy” algorithm for unbounded knapsack:

• Tacos have the best Value/Weight ratio!
• Keep grabbing tacos!

Total weight: 9
Total value: 39
4
 Example where greedy works
Activity selection CS 161 Class

Math 51 Class
CS 161 Sleep
Section
Frisbee Practice
You can only do
CS 161 Office one activity at a time, and you want to
maximize
Hours the number of activities that youTheory
do. Seminar
Orchestra
Programming
team meeting
What to choose?
CS 166 Class
Underwater basket
CS110 weaving class CS161 study
Class group Swimming
lessons
Theory Lunch
Combinatorics
Seminar Social activity

5
time
Activity selection
• Input: ai
• Activities a1, a2, …, an
• Start times s1, s2, …, sn time
• Finish times f1, f2, …, fn si fi

• Output:
• A way to maximize the number of activities you can do
today.
In what order should you
greedily add activities?

Think-share!
6
1 minute think; (wait) 1 minute share
 Greedy Algorithm

a5
a1 a3
a7
a4
a2 a6
time

• Pick activity you can add with the smallest finish time.
• Repeat.

7
 Greedy Algorithm

a5
a1 a3
a7
a4
a2 a6
time

• Pick activity you can add with the smallest finish time.
• Repeat.

8
 Greedy Algorithm

a5
a1 a3
a7
a4
a2 a6
time

• Pick activity you can add with the smallest finish time.
• Repeat.

9
 Greedy Algorithm

a5
a1 a3
a7
a4
a2 a6
time

• Pick activity you can add with the smallest finish time.
• Repeat.

10
 Greedy Algorithm

a5
a1 a3
a7
a4
a2 a6
time

• Pick activity you can add with the smallest finish time.
• Repeat.

11
 Greedy Algorithm

a5
a1 a3
a7
a4
a2 a6
time

• Pick activity you can add with the smallest finish time.
• Repeat.

12
 Greedy Algorithm

a5
a1 a3
a7
a4
a2 a6
time

• Pick activity you can add with the smallest finish time.
• Repeat.

13
 Greedy Algorithm

a5
a1 a3
a7
a4
a2 a6
time

• Pick activity you can add with the smallest finish time.
• Repeat.

14
At least it’s fast

• Running time:
• O(n) if the activities are already sorted by finish time.
• Otherwise, O(nlog(n)) if you have to sort them first.

15
What makes it greedy?

• At each step in the algorithm, make a choice.

• Hey, I can increase my activity set by one,
• And leave lots of room for future choices,
• Let’s do that and hope for the best!!!

• Hope that at the end of the day, this results in a

globally optimal solution.

16
Optimal sub-structure
in greedy algorithms
• Our greedy activity selection algorithm exploited a natural
sub-problem structure:
A[i] = number of activities you can do after the end of activity i

• How does this substructure relate to that of divide-and-

conquer or DP? A[i] = solution to
this sub-problem

ak
ai a3
a7
aj
a2 a6
17
time
Let’s see a few more examples

18
Another example:
Scheduling
CS161 HW

Personal hygiene

Math HW

Administrative stuff for student club

Econ HW

Do laundry
Meditate
Practice musical instrument

Read lecture notes

Have a social life

Sleep 19
Scheduling
• n tasks
• Task i takes ti hours
• For every hour that passes until task i is done, pay ci

10 hours

Cost: 2 units per

CS161 HW hour until it’s done.

Sleep Cost: 3 units per

hour until it’s done.

8 hours

• CS161 HW, then Sleep: costs 10 2 + (10 + 8) 3 = 74 units

• Sleep, then CS161 HW: costs 8 3 + (10 + 8) 2 = 60 units
20
 Optimal substructure
• This problem breaks up nicely into sub-problems:

Suppose this is the optimal schedule:

Job A Job B Job C Job D

Then this must be the optimal Why?

schedule on just jobs B,C,D.

Think-share
1 minute think
(wait) 1 minute share 21
 Optimal substructure
• Seems amenable to a greedy algorithm:
Take the best job first Then solve this problem

Job A Job B Job C Job D

Take the best job first Then solve this problem

Job C Job B Job D

Take the best job first Then solve this problem

Job D Job B

(That one’s easy  ) 22

 AB is better than BA when:
What does “best”
mean?
Note: here we are defining x, y, z, and w. (We use ci and ti for these in
the general problem, but we are changing notation for just this thought
experiment to save on subscripts.)

• Of these two jobs, which should we do first?

x hours

Cost: z units per

Job A hour until it’s done.

Job B Cost: w units per

hour until it’s done.

y hours What matters is the ratio:

• Cost( A then B ) = x z + (x + y) w
• Cost( B then A ) = y w + (x + y) z “Best” means
biggest ratio.23
Idea for greedy algorithm

• Choose the job with the biggest ratio.

24
 Greedy Scheduling Solution
• scheduleJobs( JOBS ):
• Sort JOBS in decreasing order by the ratio:
𝒄𝒊
• 𝒊 𝒕𝒊
• Return JOBS

Running time: O(nlog(n))

Now you can go about your schedule

peacefully, in the optimal
25 way.
One more example
Huffman coding
• everyday english sentence
• 01100101 01110110 01100101 01110010 01111001 01100100 01100001
01111001 00100000 01100101 01101110 01100111 01101100 01101001
01110011 01101000 00100000 01110011 01100101 01101110 01110100
01100101 01101110 01100011 01100101

• qwertyui_opasdfg+hjklzxcv
• 01110001 01110111 01100101 01110010 01110100 01111001 01110101
01101001 01011111 01101111 01110000 01100001 01110011 01100100
01100110 01100111 00101011 01101000 01101010 01101011 01101100
01111010 01111000 01100011 01110110

26
One more example
ASCII is pretty wasteful for
English sentences. If e shows
up so often, we should have a
Huffman coding shorter way of representing it!

• everyday english sentence

• 01100101 01110110 01100101 01110010 01111001 01100100 01100001
01111001 00100000 01100101 01101110 01100111 01101100 01101001
01110011 01101000 00100000 01110011 01100101 01101110 01110100
01100101 01101110 01100011 01100101

• qwertyui_opasdfg+hjklzxcv
• 01110001 01110111 01100101 01110010 01110100 01111001 01110101
01101001 01011111 01101111 01110000 01100001 01110011 01100100
01100110 01100111 00101011 01101000 01101010 01101011 01101100
01111010 01111000 01100011 01110110

27
Suppose we have some
distribution on characters

28
Suppose we have some For simplicity,
let’s go with this
distribution on characters made-up example

How to encode them as

45
Percentage

efficiently as possible?

16

13
12
9
5
A B C D E F
Letter

29
Try 0 • Every letter is assigned a binary string
(like ASCII) of three bits.

Wasteful!
• 110 and 111 are never used.
45
Percentage

• We should have a shorter way of

representing A.

16

13
12
9
5
A B C D E F

000 001 010 011 100 101 Letter

30
Try 1 • Every letter is assigned a binary string
of one or two bits.
• The more frequent letters get the
shorter strings.
45 • Problem:
Percentage

• Does 000 mean AAA or BA or AB?

16

13
12
9
5
A B C D E F

00 01 1 10 11 Letter
0
31
 Confusingly, “prefix-free codes” are also sometimes
called “prefix codes” (e.g. in CLRS).

Try 2: prefix-free coding

• Every letter is assigned a binary string.
• More frequent letters get shorter strings.
45
• No encoded string is a prefix of any other.
Percentage

10010101

16

13
12
9
5
A B C D E F

101 110 00 111 100 Letter

01
32
 Confusingly, “prefix-free codes” are also sometimes
called “prefix codes” (including in CLRS).

Try 2: prefix-free coding

• Every letter is assigned a binary string.
• More frequent letters get shorter strings.
45
• No encoded string is a prefix of any other.
Percentage

10010101 F

16

13
12
9
5
A B C D E F

101 110 00 111 100 Letter

01
33
 Confusingly, “prefix-free codes” are also sometimes
called “prefix codes” (including in CLRS).

Try 2: prefix-free coding

• Every letter is assigned a binary string.
• More frequent letters get shorter strings.
45
• No encoded string is a prefix of any other.
Percentage

10010101 FB

16

13
12
9
5
A B C D E F

101 110 00 111 100 Letter

01
34
 Confusingly, “prefix-free codes” are also sometimes
called “prefix codes” (including in CLRS).

Try 2: prefix-free coding

• Every letter is assigned a binary string.
• More frequent letters get shorter strings.
45
• No encoded string is a prefix of any other.
Percentage

10010101 FBA
Question: What is the most efficient
16 way to do prefix-free coding?
That is, how can we use as few bits
as possible in expectation?
13
12 (This is not it).
9
5
A B C D E F

101 110 00 111 100 Letter

01
35
A prefix-free code is a tree
B:13 below means that ‘B’
makes up 13% of the
1 characters that ever appear.
0

0 1
1 0

A: 45 D: 16

00 01 0 1 0 1

F:5 B:13 C:12 E:9

As long as all the letters
show up as leaves, this 100 101 110 111
code is prefix-free.
36
How good is a tree?
• Imagine choosing a letter at random from the language.
• Not uniformly random, but according to our histogram!
• The cost of a tree is the expected length of the encoding of a random letter.
The depth in the
Cost = tree is the length
of the encoding
0 1

P(x) is the
probability
of letter x
0 1
1 0

A: 45 D: 16

00 01 0 1 0 1

F:5 B:13 C:12 E:9

100 101 110 111
Expected cost of encoding a letter with this tree:
37
Question
• Given a distribution P on letters, find the lowest-
cost tree, where

P(x) is the The depth in the

probability tree is the length
of letter x of the encoding

38
Greedy algorithm
• Greedily build sub-trees from the bottom up.
• Greedy goal: less frequent letters should be further
down the tree.

39
Solution
greedily build subtrees, starting with the infrequent letters

14

0 1

A: 45 B:13 C:12 D: 16 E:9 F:5

40
Solution
greedily build subtrees, starting with the infrequent letters

25 14

0 1
0 1

A: 45 B:13 C:12 D: 16 E:9 F:5

41
Solution
greedily build subtrees, starting with the infrequent letters

30

25 14
0
0 1
0 1

A: 45 B:13 C:12 D: 16 E:9 F:5

42
Solution
greedily build subtrees, starting with the infrequent letters
55
1

30
0
1

25 14
0
0 1
0 1

A: 45 B:13 C:12 D: 16 E:9 F:5

43
Solution 100
1
greedily build subtrees, starting with the infrequent letters
55
1

30
0 0
1

25 14
0
0 1
0 1

A: 45 B:13 C:12 D: 16 E:9 F:5

44
Solution
greedily build subtrees, starting with the infrequent letters
100 Expected cost of encoding a letter:

0 1

55
A: 45 1
0
0
30
25
0 1
0 1

C:12 D: 16 14
B:13
101 110 0 1
100

E:9 F:5
This is called Huffman Coding
1110 1111 45
Tổng kết
• 3 ví dụ về giải thuật tham lam
• Dạng thuật toán dễ hiểu, dễ cài đặt
• Không phải lúc nào cũng đưa về được giải thuật
tham lam
• Dạng bài toán phù hợp là:
• Có cấu trúc con tối ưu dạng tuyến tính
• Lời giải của bài toán con chỉ phụ thuộc vào một bài toán con
nhỏ hơn

46
Recap II
• Often easy to write down
• The natural greedy algorithm may not always be
correct.
• A problem is a good candidate for a greedy
algorithm if:
• it has optimal substructure
• that optimal substructure is REALLY NICE
• solutions depend on just one other sub-problem.

48