Fundamentals of Algorithms

Lecture 11
Today’s Outline
• Greedy Algorithms
• Knapsack Problem
1. 0-1 Knapsack Problem
2. Fractional Knapsack Problem
• Huffman Coding
Greedy Algorithms
• This approach is used to solve the optimization problems
• An optimization problem is one in which you want to find not just a
solution but the best solution.
• Example: You want to travel from point A to point B
• There may be more than one solution
1. By walk
2. By bike
3. By car
4. By train
5. By flight
Greedy Algorithms (cont.)
• But, there is a constraint in the problem that you have to cover the
journey within 12 hours.
• So, you can travel either by train or by flight to satisfy the constraint.
• Those solutions that satisfy the constraint given in the problem are
called feasible solutions.
• Now, if you want to cover this journey in minimum cost then choose
one solution from the feasible solutions. This solution will be called
optimal solution.
• For any problem, their can be only one optimal solution.
• Hence, there can be multiple solutions of a problem and there can be
multiple feasible solution but there is only one optimal solution.
Greedy Method
ALGORITHM GREEDY (P, n) //P is the problem of size n
for i = 1 to n
x = select(P[i]) //select one input from a problem
if x is a feasible solution
solution = solution + x //add all the feasible solutions

In, greedy method, a problem is solved in stages. In each stage, we

consider one input from a given problem. If that input is feasible then
we will include it in the solution. By adding all the feasible solutions we
get an optimal solution.
Greedy Method – Example 1
• Suppose, you want to buy a best smartphone in terms of features.
How do you choose the best smartphone?
1. You should look at all the brands and models available in the world or at
least in your city. Then you can say, that I have checked all the smartphones
and I select the best one.
• This method will be very much time consuming
2. Sort out all the available brands and filter the desired brands of your
choice. Then, choose the top models of the selected brands. Further,
choose the latest release smartphone from the selected models. Finally,
you have choose the best smartphone.
• This method is the greedy method
Greedy Method – Example 2
• Suppose, you want to hire an employee for your company. One
thousand candidates have applied for the job. How you select the
best one?
1. You should take the interview of each candidate to choose the best one.
• This method will be very much time consuming
2. Take the written test for each candidate. Then, filter the candidates that
meet the criteria defined. Then, take some sort of technical test and filter
more candidates. Finally, you have few candidates and you can choose the
best candidate for your company.
• This method is the greedy method
Knapsack Problem – Greedy Method
• In a knapsack problem, we are given a set of n items, where each item
i is specified by a weight Wi and a profit Pi, We are also given the size
of the knapsack.
• We have to find the items to place in the knapsack such that the
profit is maximum and the sum of all weights does not exceed the
maximum capacity of the knapsack.
• There are two versions of the knapsack problem
1. 0-1 knapsack problem
2. Fractional knapsack problem
Fractional Knapsack Problem
Objects (O) Profit (P) Weight (W)
1 10 2
2 5 3
3 15 5
71 approach:7 select the items having maximum profit
st
4
62 approach:
1 select the items having minimum weight
nd
5
3rd approach: select the items having maximum profit/weight ratio
6 18 4
7 3 1

Here, size of the knapsack is 15

Fractional Knapsack Problem – 1st
Approach
Objects (O) Profit (P) Weight (W) Remaining Weight
6 18 4 15-4=11
3 15 5 11-5=6
1 10 2 6-2=4
4 7*4/7 7-3=4 4-4=0

In this approach, total profit is 47

Fractional Knapsack Problem – 2nd
Approach
Objects (O) Profit (P) Weight (W) Remaining Weight
5 6 1 15-1=14
7 3 1 14-1=13
1 10 2 13-2=11
2 5 3 11-3=8
6 18 4 8-4=4
3 15*4/5 5-1=4 4-4=0

In this approach, total profit is 54

Fractional Knapsack Problem – 3rd
Approach
In this approach, first calculate the Profit/Weight ratio for each
object. Objects (O) Profit (P) Weight (W) P/W Ratio
1 10 2 10/2=5
2 5 3 5/3=1.6
3 15 5 15/5=3
4 7 7 7/7=1
5 6 1 6/1=6
6 18 4 18/4=4.5
7 3 1 3/1=3
Fractional Knapsack Problem – 3rd
Approach
Objects (O) Profit (P) Weight (W) Remaining Weight
5 6 1 15-1=14
1 10 2 14-2=12
6 18 4 12-4=8
3 15 5 8-5=3
7 3 1 3-1=2
2 5*2/3 3-1=2 2-2=0

In this approach, total profit is 55.3

Fractional Knapsack Problem – Summary
• In the 1st approach, total profit is 47
• In the 2nd approach, total profit is 54
• In the 3rd approach, total profit is 55.3

• So, the 3rd approach is the best in which the maximum profit per kg is
considered.

• In 0-1 knapsack problem, object weights cannot be divisible, it means

you cannot take an object in fraction.
Huffman Coding – Greedy Method
• Huffman coding is a compression technique used to reduce the size of
the data or message.
• Suppose you want to store the data in a file. So you can store the data
in compress form to reduce the size of the file.
• Example: Sending data over the network
• We will discuss two methods
1. Fixed length coding
2. Variable length coding

• Cost of the data/message will be measured in bits

Huffman Coding – Example
• Message: BCCABBDDAECCBBAEDDCC
• Length of the message is: 20
• In computer and any electronic device, ASCII code is used to
represent any character or alphabet. ASCII code is an 8-bit code.
• We have used 5 alphabets in our message, i.e. A-E
Alphabet Decimal Binary
A 65 01000001
B 66 01000010
C 67 01000011
D 68 01000100
E 69 01000101
Huffman Coding – Example (cont.)
• Message: BCCABBDDAECCBBAEDDCC
• Length of the message is: 20
• As, we have seen 8 bits are used to represent an alphabet and we
have 20 alphabets in the message.
• So, the size/cost of the message is: 20*8 = 160 bits
• Note: This is the simple message sending without any encoding.
• Can we use our own code instead of ASCII code to reduce the cost?
• We are not using all English alphabets and characters on the
keyboard. We are using only 5 alphabets so we can use our own
coding scheme instead of ASCII.
Huffman Coding – Fixed Length Coding
• In fixed-length coding, we don’t use ASCII code. We will use own code
to represent the alphabets.
• So, we can represent A-E alphabets by using 3-bits only.
Alphabet Code
A 000
B 001
C 010
D 011
E 100

• So, the size/cost of the message is: 20*3 = 60 bits

Huffman Coding – Fixed Length Coding
(cont.)
• As, we have used own coding scheme then how the receiver will
decode the message.
• We should have to send the table/chart along with the data.
• Now, we will calculate the size of the table/chart.
• As, we have 5 alphabets and each alphabet is an 8-bit ASCII code so,
5*8 = 40 bits (for alphabets)
• We have 5 alphabets and represent each alphabet using 3-bit code so,
5*3 = 15 bits (for codes)
• So, the total size/cost of the message is: 60 + 40 +15 = 115 bits.
• The size is reduced from 160 bits to 115 bits.
Huffman Coding – Variable Length
Coding
• In variable-length coding, we will not use fixed bit-size for each alphabet.
• In variable-length coding, 1st step is to count the frequency of each
alphabet.
Alphabet Frequency
A 3
B 5
C 6
D 4
E 2
• In the 2nd step, arrange the alphabets in increasing order and make
Huffman tree.
Huffman Coding – Variable Length Coding
(cont.)

2/E 3/A 4/D 5/B 6/C

Huffman Coding – Variable Length Coding
(cont.)

2/E 3/A 4/D 5/B 6/C

Huffman Coding – Variable Length Coding
(cont.)

2/E 3/A 4/D 5/B 6/C

Huffman Coding – Variable Length Coding
(cont.)

5 11

2/E 3/A 4/D 5/B 6/C

Huffman Coding – Variable Length Coding
(cont.)
20

5 11

2/E 3/A 4/D 5/B 6/C

Huffman Coding – Variable Length Coding
(cont.)
20

1
9
0

5 1 11

0 1 0 1

2/E 3/A 4/D 5/B 6/C

Huffman Coding – Variable Length Coding
(cont.)
• In 3rd step, trace the code from Huffman tree.
Alphabet Frequency Code
A 3 001
B 5 10
C 6 11
D 4 01
E 2 000

• We can see from the table, some alphabets are represents by 3-bits
and some are represented by 2-bits. This is called variable-length
coding.
• Now, what will be the cost/size of the message?
Huffman Coding – Variable Length Coding
(cont.)
Alphabet Frequency Code Size
A 3 001 3*3=9
B 5 10 5*2=10
C 6 11 6*2=12
D 4 01 4*2=8
E 2 000 2*3=6

• Size of the message is: 45-bits but we have to send chart/table along
with the data for decoding at receiver end. So, what will be the size of
the chart?
• 5*8 = 40 bits (for alphabets) and 12 bits (for codes), total are 52-bits
for chart or table.
• So, the total size/cost of the message is: 45 + 52 = 97 bits.