Filière : ILSI

Ingénierie Logicielle et Systèmes Intelligents

Design and Analysis of Algorithms

Greedy Algorithms

EL Moukhtar ZEMMOURI
ENSAM – Meknès

Example 1: Scheduling
Activity selection problem
 Scheduling : activity selection problem

• ! = #! , #" , … , ## activities that require exclusive use of a common resource R.

o !! has start time "! and finish time #! with 0 ≤ "! < #! < ∞
o !! and !" are compatible if the intervals ["! , #! [ and ["" , #" [ do not overlap.
• è !! ≤ #" or !" ≤ #!

• Activity selection problem:

o The goal is to select a maximum number of mutually compatible activities
o è Maximum number of non overlapping activities.
• Example :

o !# , !$ , !% , !## and !& , !$ , !' , !## are possible solutions.

E. Zemmouri, ENSAM - Meknès 3

DP approach
 The optimal substructure

• *!" = the set of activities starting after !! finishes and finishing before !" starts.

o $!" = &# | !! ≤ ## &() !# ≤ #"

• +!" = the maximum size set of mutually compatible activities in *!"

• For a given activity !( ∈ +!" we have :
o $!$ and $$" are two subproblems of $!"

o *!" = *!$ ∪ &$ ∪ *$"

o *!" = *!$ + *$" + 1

o That is an optimal solution *!" must also include optimal solutions to the two subproblems for $!$
and $$" .

• è Activity selection problem verifies the optimal substructure property

o Can be solved using DP

E. Zemmouri, ENSAM - Meknès 5

The optimal substructure

• &[(, )] = the size of an optimal solution for the set !$% .

o - ., / = -!12 +!"

• We have :
o - ., / = 0 if *!" = ∅

o - ., / = max - ., 7 + - 7, / + 1 #:1 !( ∈ *!" if *!" ≠ ∅

• With this we can implement a DP solution.

E. Zemmouri, ENSAM - Meknès 6

 Greedy Algorithm

Making the greedy choice

• Another characteristic that we can use to avoid solving all subproblems !

o Choose an activity to add to an optimal solution without having to first solve all the
subproblems è the greedy choice.

• What is the greedy choice in this case?

o Intuition : Choose the activity !! in * with the earliest finish time.
o è It leaves the resource R available for as many other activities as possible.

• So :
o Sort the activities such that ## ≤ #& ≤ ⋯ ≤ #)
o Choose !#
o Then you have one subproblem to solve: set of activities that start after !# finishes.

E. Zemmouri, ENSAM - Meknès 8

 Making the greedy choice

• Question : is our intuition correct?

o Is the greedy choice (first activity to finish) always part of an optimal solution?
• !! = #" ∈ ! | &" ≥ (! the set of activities that start after activity #! finishes.
• ## the greedy choice for the subproblem !!
o We should prove that !* ∈ an optimal solution of #( .
• )! a maximum size subset of mutually compatible activities in !! .
• #$ the activity in )! that finishes first.
• If #$ = ##
o Then !* ∈ an optimal solution of #( . Done.
• If #$ ≠ ##
o Then we consider $+( = $( − !" ∪ !* an optimal solution of #( and contains !*. Done.

E. Zemmouri, ENSAM - Meknès 9

Greedy algorithm

activity_selection_greedy (a, s, f, n)

sort (a, s, f) in increasing order according to f

A = {a[1]} // The set of activities of optimal solution

k = 1
for m = 2 to n
if s[m] >= f[k]:
A = A ∪ {a[m]}
k = m
return A

E. Zemmouri, ENSAM - Meknès 10

 To s u m u p
• Greedy algorithm
o obtains an optimal solution to a problem by making a sequence of choices.
o At each decision point, the algorithm makes the choice that seems best at the moment (local
optimal) with the hope of finding an optimal solution (global optimal).
• This strategy does not always produce an optimal solution, but sometimes it does!

• Process for designing a greedy algorithm :

o Define the problem and identify the goal (maximize, minimize, …) and the constraints to be
satisfied.
o Cast the optimization problem as one in which you make a choice and are left with one
subproblem to solve.
o Determine the greedy choice and prove that there is always an optimal solution to the original
problem that makes the greedy choice.
o Demonstrate optimal substructure by showing that, having made the greedy choice, what remains
is a subproblem with the property that if you combine an optimal solution to the subproblem with
the greedy choice you have made, you arrive at an optimal solution to the original problem.

E. Zemmouri, ENSAM - Meknès 11

Exercises

• Give and implement in your favorite programming language a dynamic

programming algorithm for the activity selection problem. Compare the running
time of this algorithm to the running time of the previous greedy algorithm.

• Suppose that instead of selecting the first activity to finish, you instead select the
last activity to start that is compatible with all previously selected activities.
Does this greedy choice produce an optimal solution?

E. Zemmouri, ENSAM - Meknès 12

 Example 2: Data compression
Huffman codes

Data compression

• A binary code assigns for each character of a string a codeword that

consists of bits.
• A constant-length code è the length of each codeword is the same. We
need =(log & B ) bits to represent n characters.
Constant-length code
o Example :
o The string AABACDACA is represented by 00 00 01 00 10 11 00 10 00
• Using a variable-length code we can compress the string : give frequent
characters short codewords and infrequent characters long codewords.
o Example :
Prefix-free code
o The string AABACDACA is represented by 0 0 110 0 10 111 0 10 0
• An optimal code produces a compressed string that is as short as possible.
o In addition, the optimal code should be prefix-free : no codeword is also a
prefix of some other codeword è this simplifies decoding

Not valid !
E. Zemmouri, ENSAM - Meknès 14
 Huffman Coding

• Huffman proposed (in 1952) a greedy algorithm that constructs an optimal prefix-
free code called a Huffman code.
• The algorithm builds a binary tree based on the frequencies of the characters
o Each character’s codeword can be read by following a path from the root to the
corresponding node.
o A move to the left corresponds to bit 0, and a move to the right corresponds to bit 1.
• Algorithm :
o Initially, each character is represented by a node whose weight is the frequency of this
character.
o Then at each step two nodes with minimum weights are combined by creating a new
node whose weight is the sum of the weights of the original nodes.
o The process continues until all nodes have been combined.

E. Zemmouri, ENSAM - Meknès 15

Huffman Coding: example

E. Zemmouri, ENSAM - Meknès 16

 Huffman Coding: algorithm

huffman (characters)
// characters : a hash table containing (char, frequency) pairs

queue = MinPriorityQueue () // a min priority queue

for k in characters.keys
node = Node (k, characters[k]) // a node have 2 attributes : char and frequency
queue.insert (node) // insert the node into the min priority queue

n = charcters.size
for i = 1 à n-1
x = queue.extract() // extract a first node with min frequency
y = queue.extract() // extract a second node with min frequency
z = Node ('_’ , x.frequency + y.frequency) // create new node with sum frequencies
z.left = x
z.right = y
queue.insert (z) // insert the new node into the queue

root = queue.extract()
return root // return the root of the constructed binary tree

E. Zemmouri, ENSAM - Meknès 17

Huffman Coding: exercise

• Propose an implementation of the Huffman coding and decoding

algorithm

E. Zemmouri, ENSAM - Meknès 18