Analysis and Design of Algorithms

(ADA)
GTU # 3150703

Unit-5
Greedy Algorithms

Rupesh G. Vaishnav
Computer Engineering Department
Darshan Institute of Engineering & Technology, Rajkot
[email protected]
9428037452
 Outline
Looping
 General Characteristics of greedy algorithms
 Elements of Greedy Strategy
 Make change Problem
 Minimum Spanning trees (Kruskal’s algorithm, Prim’s
algorithm)
 The Knapsack Problem
 Job Scheduling Problem
 Huffman code
Introduction
Characteristics of Greedy Algorithms
 Greedy algorithms are characterized by the following features.
1. Greedy approach forms a set or list of candidates 𝑪.
2. Once a candidate is selected in the solution, it is there forever: once a candidate is excluded from the
solution, it is never reconsidered.
3. To construct the solution in an optimal way, Greedy Algorithm maintains two sets.
4. One set contains candidates that have already been considered and chosen, while the other set contains
candidates that have been considered but rejected.

 The greedy algorithm consists of four functions.

i. Solution Function:- A function that checks whether chosen set of items provides a solution.
ii. Feasible Function:- A function that checks the feasibility of a set.
iii. Selection Function:- The selection function tells which of the candidates is the most promising.
iv. Objective Function:- An objective function, which does not appear explicitly, but gives the value of a
solution.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 4

Make a Change Problem
Problem Definition
 Suppose following coins are available with unlimited quantity:
1. 10
2. 5
3. 2
4. 1
5. 50 paisa

 Our problem is to devise an algorithm for paying a given amount to a customer using the
smallest possible number of coins.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 6

Make Change – Greedy Solution
 If suppose, we need to pay an amount of 28/- using the available coins.
 Here we have a candidate (coins) set
 The greedy solution is,

Selected coins are Amount Total required coins = 5

Selected coins = {10, 5, 2,
coins quantity 20
28
27
25
0 1}

10 2
5 1
2 1
1 1
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 7
Make Change - Algorithm
# Input: C = {10, 5, 2, 1, 0.5} //C is a candidate set
# Output: S: set of selected coins
Function make-change(n): set of coins
S ← Ø {S is a set that will hold the solution}
sum ← 0 {sum of the items in solution set S}
while sum ≠ n do
x ← the largest item in C such that sum + x ≤ n
if there is no such item then
return "no solution found"
S ← S U {a coin of value x}
sum ← sum + x
return S
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 8
Make Change – The Greedy Property
 The algorithm is greedy because,
 At every step it chooses the largest available coin, without worrying whether this will prove to be a
correct decision later.
 It never changes the decision, i.e., once a coin has been included in the solution, it is there forever.
 Examples:
1. Some coins with denominations 50, 20, 10, 5, 1 are available.
 How many minimum coins required to make change for 37 cents? 5
 How many minimum coins required to make change for 91 cents? 4

2. Denominations: d1=6, d2=4, d3=1. Make a change of 8. 3

The minimum coins required are 2

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 9

Minimum Spanning Tree
Introduction to Minimum Spanning Tree (MST)
 Let be a connected, undirected graph where,
1. is the set of nodes and
2. is the set of edges.
 Each edge has a given positive length or weight.
 A spanning tree of a graph is a sub-graph which is basically a tree and it contains all the
vertices of but does not contain cycle.
 A minimum spanning tree (MST) of a weighted connected graph is a spanning tree with
minimum or smallest weight of edges.
 Two Algorithms for constructing minimum spanning tree are,
1. Kruskal’s Algorithm
2. Prim’s Algorithm

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 11

Spanning Tree Examples
Graph A Spanning Tree A
B C B C
D E F G D E F G

H H

Graph Spanning Tree

A
A
B C E B C E

D F D F

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 12

Kruskal’s Algorithm for MST – Example 1
A Step 2: Taking next Step 4: Taking next
4 5
6 min edge (B,C) min edge (A,B)
6
B 3
E B A
4
5 7 2
2
C D B E
C D 1 3
1
2
Step 3: Taking next

Step 1: Taking min

min edge (B,E) C D
edge (C,D) B E 1
3
So, we obtained a
C D 2
1 C D minimum
1 spanning tree of cost:
4 + 2 + 1 + 3 = 10
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 13
Kruskal’s Algorithm for MST – Example Step:1
Sort the edges in increasing order of their
2 weight.
Edges Weight

1 2 {1, 2} 1
1 2 3 {2, 3} 2
{4, 5} 3
4 6 4 5 6
{6, 7} 3
{1, 4} 4
4 3 5 8 6 {2, 5} 4
{4, 7} 4
7 {3, 5) 5
4 3
{2, 4} 6
{3, 6} 6
7
{5, 7} 7
{5, 6} 8
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 14
Kruskal’s Algorithm for MST – Example Step:2
Select the minimum weight edge but no cycle.
2
Edges Weight

1 2 {1, 2} 1 
1 2 3 {2, 3} 2 
{4, 5} 3 
4 6 4 5 6
{6, 7} 3 
{1, 4} 4 
4 3 5 8 6 {2, 5} 4
{4, 7} 4 
7 {3, 5) 5
4 3
{2, 4} 6
{3, 6} 6
7
{5, 7} 7
{5, 6} 8
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 15
Kruskal’s Algorithm for MST – Example Step:3
The minimum spanning tree for the given
2 graph.
Edges Weight

1 2 {1, 2} 1 
1 2 3 {2, 3} 2 
{4, 5} 3 
4
{6, 7} 3 
{1, 4} 4 
4 3 5 6 {4, 7} 4 

4 3 Total Cost = 17

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 16

Kruskal’s Algorithm – Example 2
Step Edges considered - Connected Components Edges Weight
{u, v}
Init. - {1} {2} {3} {4} {5} {6} {7} {1, 2} 1
1 {1,2} {1,2} (3} {4} {5} {6} {7} {2, 3} 2
2 {2,3} {1,2,3} {4} {5} {6} {7} {4, 5} 3
3 {4,5} {1,2,3} {4,5} {6} {7} {6, 7} 3
4 {6,7} {1,2,3} {4,5} {6,7} {1, 4} 4
5 {1,4} {1,2,3,4,5} {6,7} {4, 7} 4
6 {2,5} Rejected Total Cost = 17
7 {4,7} {1,2,3,4,5,6,7}

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 17

Kruskal’s Algorithm for MST
Function Kruskal(G = (N, A))
Sort A by increasing length
n ← the number of nodes in N
T ← Ø {edges of the minimum spanning tree}
Define n sets, containing a different element of set N
repeat
e ← {u, v} //e is the shortest edge not yet considered
ucomp ← find(u)
vcomp ← find(v) find(u) tells in which connected component a node is found
if ucomp ≠ vcomp then merge(ucomp, vcomp)
T ← T U {e} merge(ucomp, vcomp) is used to merge two connected components.
until T contains n - 1 edges
return T
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 18
Exercises – Home Work
 The complexity for the Kruskal’s algorithm is in where is total number of edges and is the
total number of nodes in the graph .

 Write the kruskal’s Algorithm to find out Minimum Spanning Tree. Apply the same and find
MST for the graph given below.
𝟏. 𝟐.
10
3
3 A 1
F C
A 4 4 3 7
B 2 4
C 5
8
5 6
4 B D D E F G
4
H 2 3 2 8
1 9
3
G 3
E H

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 19

Prim’s Algorithm
 In Prim's algorithm, the minimum spanning tree grows in a natural way, starting from an
arbitrary root.
 At each stage we add a new branch to the tree already constructed; the algorithm stops when
all the nodes have been reached.
 The complexity for the Prim’s algorithm is where is the total number of nodes in the graph .

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 20

 Step:1 Select an arbitrary node.
Prim’s Algorithm for MST – Example 1
Node - Set B Edges

1 2
1 2 3 1

4 6 4 5 6

4 3 5 8 6
7
4 3

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 21

 Step:2 Find an edge with minimum weight.
Prim’s Algorithm for MST – Example 1
Node - Set B Edges

1 2
1 2 3 1 {1, 2}, {1,
4}
4 6 1, 2 {1, 4}, {2, 3} {2, 4}, {2, 5}
6 4 5
1, 2, 3 {1,4}, {2,4}, {2,5}, {3,5},
4 3 5 8 6 {3,6}
1, 2, 3, 4 {2,4} {2,5} {3,5} {3,6}
7 {4,5} {4,7}
4 3 1, 2, 3, 4, 5 {2,4} {2,5} {3,5} {3,6}
{4,7} {5,6} {5,7}
7 1, 2, 3, 4, 5, 7 {2,4} {2,5} {3,5} {3,6}
{5,6} {5,7} {6,7}
1, 2, 3, 4, 5, 6, 7
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 22
 Step:3
Prim’s Algorithm for MST – Example 1
The minimum spanning tree for the given
graph.
Node Edges
1 2
1 2 3 1
1, 2 {1, 2}
4
1, 2, 3 {2, 3}
1, 2, 3, 4 {1, 4}
4 3 5 6 1, 2, 3, 4, 5 {4, 5}
1, 2, 3, 4, 5, 7 {4, 7}
4 3 1, 2, 3, 4, 5, 6, 7 {6, 7}

Total Cost = 17
7

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 23

 Cost =
Prim’s Algorithm – Example 1

17

Step Edge Set B Edges Considered

Selected
{u, v}
Init. - {1} --
1 {1, 2} {1,2} {1,2} {1,4}
2 {2, 3} {1,2,3} {1,4} {2,3} {2,4} {2,5}
3 {1, 4} {1,2,3,4} {1,4} {2,4} {2,5} {3,5} {3,6}
4 {4, 5} {1,2,3,4,5} {2,4} {2,5} {3,5} {3,6} {4,5} {4,7}
5 {4, 7} {1,2,3,4,5,7} {2,4} {2,5} {3,5} {3,6} {4,7} {5,6}
{5,7}
6 {6,7} {1,2,3,4,5,6,7} {2,4} {2,5} {3,5} {3,6} {5,6} {5,7}
Prim’s Algorithm
Function Prim(G = (N, A): graph; length: A — R+): set of edges
T ← Ø
B ← {an arbitrary member of N}
while B ≠ N do
find e = {u, v} of minimum length such that
u B and v ∈ N \ B
T ← T U {e}
B ← B U {v}
return T

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 25

Exercises – Home Work
 Write the Prim’s Algorithm to find out Minimum Spanning Tree. Apply the same and find
MST for the graph given below.

𝟏. 𝟐.
10 F
3
3 A 1
C
A 4 4 3 7
B 2 4
C 5
8
5 6
4 B D D E F G
4
H 2 3 2 8
1 9
3
G 3
E H

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 26

Single Source Shortest Path –
Dijkstra’s Algorithm
Introduction
 Consider now a directed graph where is the set of nodes and is the set of directed edges of
graph .
 Each edge has a positive length.
 One of the nodes is designated as the source node.
 The problem is to determine the length of the shortest path from the source to each of the
other nodes of the graph.
 Dijkstra’s Algorithm is for finding the shortest paths between the nodes in a graph.
 For a given source node, the algorithm finds the shortest path between the source node and
every other node.
 The algorithm maintains a matrix which gives the length of each directed edge:

if the edge and

otherwise.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 28

Dijkstra’s Algorithm - Example

1 Single source shortest path algorithm

10 50

5 2
100 30
Source node = 1
10 5 Step v C 2 3 4 5
20
{2, 3, 4,
Init. - 50 30 100 10
4 3 5}
50 1 5 {2, 3, 4} 50 30 20 10

Is there path from 1 - 5 - Compare cost of 1–5–

Is
Isthere
therepath
path from
from11 --55 -- No
Yes
2 4 (20) and 1–4 (100)
4 3
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 29
Dijkstra’s Algorithm - Example

1 Single source shortest path algorithm

10 50

5 2
100 30
Source node = 1
10 5 Step v C 2 3 4 5
20
{2, 3, 4,
Init. - 50 30 100 10
4 3 5}
50 1 5 {2, 3, 4} 50 30 20 10
2 4 {2, 3} 40 30 20 10

Is there path from 1 - 4 - Compare cost of 1–4–

IsIsthere
therepath
path2from
from11--44-- Yes
No 32 (70)
(40) and
and 1–3
1–2(50)
(30)
5 3
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 30
Dijkstra’s Algorithm - Example

1 Single source shortest path algorithm

10 50

5 2
100 30
Source node = 1
10 5 Step v C 2 3 4 5
20
{2, 3, 4,
Init. - 50 30 100 10
4 3 5}
50 1 5 {2, 3, 4} 50 30 20 10
2 4 {2, 3} 40 30 20 10
3 3 {2} 35 30 20 10

Is there path from 1 - 3 - Compare cost of

IsIsthere
therepath
path2from
from11--33-- Yes
No
1–3–2 and 1–2
5 4
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 31
Exercises – Home Work
 Write Dijkstra’s Algorithm for shortest path. Use the algorithm to find the shortest path from
the following graph.

1. 2.
2
2 A B
B D
10 4 1 3 10

1 4 8 7 9 2 2
A C D E

3 2 5 8 4 6
C E
1
F G

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 32

Dijkstra’s Algorithm
Function Dijkstra(L[1 .. n, 1 .. n]): array [2..n]
array D[2.. n]
C {2,3,…, n}
{S = N \ C exists only implicitly}
for i 2 to n do
D[i] L[1, i]
repeat n - 2 times
v some element of C minimizing D[v]
C C \ {v} {and implicitly S S U {v}}
for each w C do
D[w] min(D[w], D[v] + L[v, w])
return D
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 33
Knapsack Problem
Knapsack Problem

Rupesh G. Vaishnavv #3150703 (ADA)  Unit 5 – Greedy Algorithms 35

Fractional Knapsack Problem
Introduction
 We are given objects and a knapsack.
 Object has a positive weight and a positive value for
 The knapsack can carry a weight not exceeding .
 Our aim is to fill the knapsack in a way that maximizes the value of the included objects,
while respecting the capacity constraint.
 In a fractional knapsack problem, we assume that the objects can be broken into smaller
pieces.
 So we may decide to carry only a fraction of object , where
 In this case, object contribute to the total weight in the knapsack, and to the value of the
load.
 Symbolic Representation of the problem can be given as follows:
maximize subject to
Where, and for .
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 37
Fractional Knapsack Problem - Example
 We are given objects and the weight carrying capacity of knapsack is .
 For each object, weight and value are given in the following table.

Object

 Fill the knapsack with given objects such that the total value of knapsack is maximized.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 38

Fractional Knapsack Problem - Greedy Solution
 Three Selection Functions can be defined as,
1. Sort the items in descending order of their values and select the items till weight criteria is satisfied.
2. Sort the items in ascending order of their weight and select the items till weight criteria is satisfied.
3. To calculate the ratio value/weight for each item and sort the item on basis of this ratio. Then take the
item with the highest ratio and add it.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 39

Fractional Knapsack Problem - Greedy Solution
Object

2.0 1.5 2.2 1.0 1.2

Selection Objects Value Weight

1 2 3 4 5 Capacity 100
0 0 1 0.5 1 146 30 50 20
1 1 1 1 0 156 10 20 30 40
1 1 1 0 0.8 164 30 10 20 40

Profit = 66 + 60 + (40 * 0.5) =

Profit 66 + 30
Profit = 20 20 ++ 66
30 ++ 40
48 == 156
164
146
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 40
Fractional Knapsack Problem - Algorithm
Algorithm: Greedy-Fractional-Knapsack (w[1..n], p[1..n], W)
for i = 1 to n do
x[i] 0 ; weight 0
While weight < W do
i the best remaining object
if weight + w[i] ≤ W then W = 100 and Current weight in knapsack=
x[i] 1 60
Object weight = 50
weight weight + w[i] The fraction of object to be included will be
(100 – 60) / 50 = 0.8
else
x[i] (W - weight) / w[i]
weight W
return x
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 41
Exercises – Home Work
1. Consider Knapsack capacity , and find the maximum profit using greedy approach.
2. Consider Knapsack capacity , and . Find the maximum profit using greedy approach.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 42

Activity Selection Problem
Introduction
 The Activity Selection Problem is an optimization problem which deals with the selection of
non-overlapping activities that needs to be executed by a single person or a machine in a
given time duration.
 An activity-selection can also be applicable for scheduling a resource among several
competing activities.
 We are given a set of activities with start time and finish time , of an activity. Find the
maximum size set of mutually compatible activities.
 Activities andare compatible if the half-open internal and do not overlap, that is, and are
compatible if or .

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 44

 Example: 11 activities are given as,
Activity Selection Problem - Example
Solution:
Sr. Activity (, )
1 P (1, 4)
Step 1:
2 Q (3, 5) Sort the activities of set 𝑺 as per increasing
3 R (0, 6) finish time to directly identify mutually
4 S (5, 7) compatible activities by comparing finish time
5 T (3, 8) of first activity and start time of next activity
6 U (5, 9)
7 V (6, 10)
8 W (8, 11)
9 X (8, 12)
10 Y (2, 13)
11 Z (12, 14)

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 45

 Example: 11 activities are given as,
Activity Selection Problem - Example
Solution:
Sr. Activity (, )
1 P (1, 4)
Step 2:
2 Q (3, 5) 1. A = {P}
3 R (0, 6) 2. A = {P, S}
4 S (5, 7) 3. A = {P, S, W}
5 T (3, 8) 4. A = {P, S, W, Z}
6 U (5, 9)
Answer: A = {P, S, W, Z}
7 V (6, 10)
8 W (8, 11)
9 X (8, 12)
10 Y (2, 13)
11 Z (12, 14)

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 46

Activity Selection - Algorithm
Algorithm: Activity Selection
Step I: Sort the input activities by increasing finishing time. f1 ≤
f2 ≤ . . . ≤ fn

Step II: Call GREEDY-ACTIVITY-SELECTOR (s, f)

n = length [s]
A = {i}
j = 1
for i = 2 to n
do if si ≥ f j
then A = A U {i}
j = i
return set A
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 47
Exercise - HW
 Given arrival and departure times of all trains that reach a railway station, find the minimum
number of platforms required for the railway station so that no train waits. We are given two
arrays which represent arrival and departure times of trains that stop. Arr[ ] = {9:00, 9:40,
9:50, 11:00, 15:00, 18:00} dep[ ] = {9:10, 12:00, 11:20, 11:30, 19:00, 20:00}

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 48

Job Scheduling with Deadlines
Introduction
 We have set of jobs to execute, each of which takes unit time.
 At any point of time we can execute only one job.
 Jobearns profit if and only if it is executed no later than its deadline
 We have to find an optimal sequence of jobs such that our total profit is maximized.
 Feasible jobs: A set of job is feasible if there exits at least one sequence that allows all the
jobs in the set to be executed no later than their respective deadlines.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 50

Job Scheduling with Deadlines - Example
 Using greedy algorithm find an optimal schedule for following jobs with .
 Profits: &
 Deadline:
Solution:
Step 1: Sort the jobs in decreasing order of their profit.

Job
Profit
Deadline

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 51

Job Scheduling with Deadlines - Example
Job
Profit
Deadline
Step 2: Find total position
Here,
P 1 2 3
Job selected 0 0 0

Step 3: : assign job to position

P 1 2 3
Job selected 0 0 J1
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 52
Job Scheduling with Deadlines - Example
Job
Profit
Deadline

Step 4: assign job to position

P 1 2 3
Job selected J2 0 J1

Step 5: : assign job to position

But position 1 is already occupied and two

jobs can not be executed in parallel, so reject
job 3
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 53
Job Scheduling with Deadlines - Example
Job
Profit
Deadline
Step 6: assign job to position as, position is not free but position is free.

P 1 2 3
Job selected J2 J4 J1

Now no more free position is left so no more jobs can be scheduled.

The final optimal sequence:
Execute the job in order 2, 4, 1 with total profit value 42.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 54

Exercises – Home Work
1. Using greedy algorithm find an optimal schedule for following jobs with 𝒏=4.
Profits: (a, b, c, d) = (20,10,40,30) &
Deadline: (d1, d2, d3, d4) = (4, 1, 1, 1)
2. Using greedy algorithm find an optimal schedule for following jobs with 𝒏=5.
Profits: (a, b, c, d, e) = (100,19,27,25,15) &
Deadline: (d1, d2, d3, d4, d5) = (2, 1, 2, 1, 3)

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 55

Job Scheduling with Deadlines - Algorithm
Algorithm: Job-Scheduling (P[1..n], D[1..n])
1. Sort all the n jobs in decreasing order of their profit.
2. Let total position P = min(n, max(d i))
3. Each position 0, 1, 2…, P is in different set and T({i}) = i, for 0 ≤ i ≤
P.
4. Find the set that contains d, let this set be K. if T(K) = 0 reject the
job; otherwise:
1. Assign the new job to position T(K).
2. Find the set that contains T(K) – 1. Call this set L.
3. Merge K and L. the value for this new set is the old value of T(L).

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 56

Huffman Codes
Prefix Code
 Prefix code is used for encoding(compression) and Decoding(Decompression).
 Prefix Code: Any code that is not prefix of another code is called prefix code.

Characters Frequency Code Bits

a 45 000 135
b 13 111 39
c 12 101 36
d 16 110 48
e 9 011 27
f 5 001 5
Total bits 290

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 58

Huffman code Introduction
 Huffman invented a greedy algorithm that constructs an optimal prefix code called a Huffman
code.
 Huffman coding is a lossless data compression algorithm.
 It assigns variable-length codes to input characters.
 Lengths of the assigned codes are based on the frequencies of corresponding characters.
 The most frequent character gets the smallest code and the least frequent character gets the
largest code.
 The variable-length codes assigned to input characters are Prefix Codes.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 59

Huffman Codes
 In Prefix codes, the codes are assigned in such a way that the code assigned to one character
is not a prefix of code assigned to any other character.
 For example,

a = 01, b = 010 and c = 11 Not a prefix code

 This is how Huffman Coding makes sure that there is no ambiguity when decoding the
generated bit stream.
 There are mainly two major parts in Huffman Coding
1. Build a Huffman Tree from input characters.
2. Traverse the Huffman Tree and assign codes to characters.

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 60

Huffman Codes - Example
 Find the Huffman codes for the following characters.

Characters a b c d e f
Frequency (in thousand) 45 13 12 16 9 5

Step 1: Arrange the characters in the Ascending order of their frequency.

f:5 e:9 c:12 b:13 d:16 a:45

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 61

Huffman Codes - Example
Step 2:
 Extract two nodes with the minimum frequency.
 Create a new internal node with frequency equal to the sum of the two nodes
frequencies.
 Make the first extracted node as its left child and the other extracted node as its right
child.

14

f:5 e:9 c:12 b:13 d:16 a:45

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 62

Huffman Codes - Example
Step 3:
 Rearrange the tree in ascending order.
 Assign to the left branch and to the right branch.
 Repeat the process to complete the tree.

14

f:5 e:9 c:12 b:13 d:16 a:45

0 1

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 63

Huffman Codes - Example
Step 4: 25

c:12 b:13 14 d:16 a:45

0 1

f:5 e:9

14 d:16 25 a:45
0 1 0 1

f:5 e:9 c:12 b:13

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 64
Huffman Codes - Example
Step 5: 30

14 d:16 25 a:45
0 1 0 1

f:5 e:9 c:12 b:13

25 30 a:45
0 1 0 1

c:12 b:13 14 d:16

0 1
f:5 e:9
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 65
Huffman Codes - Example
Step 6:

55
0 1

25 30 a:45
0 1
0 1

c:12 b:13 14 d:16

0 1
f:5 e:9

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 66

Huffman Codes - Example
Step 7: 100
0 1

a:45 55
0 1

25 30
0 1
0 1

c:12 b:13 14 d:16

0 1
f:5 e:9
Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 67
Huffman Codes - Example
Characters a b c d e f
Step 8:
Frequency (in thousand) 45 13 12 16 9 5
0 101 100 111 1101 1100

Total bits: 224

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 68

Huffman Codes - Algorithm
Algorithm: HUFFMAN (C)
n = |C|
Q = C
for i = 1 to n-1
allocate a new node z
z.left = x = EXTRACT-MIN(Q)
z.right = y = EXTRACT-MIN(Q)
z.freq = x.freq + y.freq
INSERT(Q,z)
return EXTRACT-MIN(Q) // return the root of the tree

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 69

Exercises – Home Work
 Find an optimal Huffman code for the following set of frequency.
1. a : 50, b : 20, c : 15, d : 30.
2. Frequency
Characters A B C D E F
Frequency (in thousand) 24 12 10 8 8 5

3. Frequency

Characters a b c d e f g
Frequency (in thousand) 37 28 29 13 30 17 6

Rupesh G. Vaishnav #3150703 (ADA)  Unit 5 – Greedy Algorithms 70

Thank You!