CSE 1205: Object Oriented Programming

Greedy Techniques
Md Mehrab Hossain Opi
 Definition 2
The greedy method is a problem-solving strategy where decisions are
made by selecting the best option available at the moment, without
considering future consequences.

CSE 1205: Object Oriented Programming December 21, 2024

3

Activity Selection
Problem
Problem Statement 4
• Imagine that you are in charge of scheduling the KUET central field.

• Several groups have requested to reserve the field, but you can only
accommodate one group on the field at a time.

• Your objective is to maximize the number of non-overlapping activities,

ensuring that as many groups as possible get a chance to use the field.
Problem Statement 5
• You are given a list of activities, each requesting a time slot
to reserve the central field.
• Each request has a start time and an end time.
• Your task is to select the maximum number of non-
overlapping time slots so that the field is used as efficiently
as possible.
Problem Input 6
• A list of n activities, each with a time slot (
• ​is the start time requested by team ,
• ​is the end time when their game will finish.
Objective 7
• Maximize the number of activities by selecting the
maximum number of non-overlapping time slots.
• Activities and are non-overlapping (compatible)
if or .
Optimal Substructure of the Problem 8

• Let be the set of activities that start after activity finishes and
that finish before game starts.
• Let be the maximum set of non-overlapping activities in .
• Suppose activity is in .
• Then we get two subproblems.
• find and .
• Let and .
• Thus, we have .
Optimal Substructure of the Problem 9
• The maximum value of can be computed by choosing the
value of that maximizes the value .
• This problem can be solved using dynamic programming.
• We will see this later in our course.
 Making The Greedy Choice 10

• What is the greedy choice for the activity selection problem?

• You would want to select the activity that leaves the field for as
many other activity as possible.
• Of the activities you end up choosing, one of them must be the
first one to finish.
• Intuition says, choose the activity with the earliest finish time.
• That leaves the field available for as many of the games that
follow it as possible.
Making The Greedy Choice 11
• Once you make the greedy choice, you have only one
remaining subproblem to solve.
• Finding activities that start after the selected activity is
finished.
• There are no activities that started before the
selected one.
Making the Greedy Choice 12

• But is this intuition correct?

• Is the greedy choice- in which you the first game to
finish- always part of some optimal solution?
Theorem 1 13

Consider any non-empty subproblem and let be an

activity in with the earliest finish time. Then is
included in some maximum-size subset of non-
overlapping activites of .
Theorem 1 - Proof 14

• Let be a maximum-size subset of mutually compatible activities in , and let

be the activity with the earliest finish time.
• If , we are done, since we have shown that belongs to some maximum-size
subset .
• If let the set .
• The activities in are compatible, which follows because the activities in are
compatible, is the first activity in to finish, and .
• Since , we conclude that is a maximum-size subset of and it includes
A Recursive Greedy Algorithm 15
• Let’s see a straightforward, recursive procedure.
• It takes the start and finish times of the activities,
represented as arrays s and f, the index k that defines the
subproblem it is to solve, and the size n of the original
problem.
• It returns a maximum-size set of mutually compatible
activities in .
A Recursive Greedy Algorithm 16
• The procedure assumes that the n activities are already
sorted by monotonically increasing finish time.
• If not, you need to first sort them in this order.
• In order to start, add the fictious activity with .
• The initial call, which solves the entire problem, is
RECURSIVE-ACTIVITY-SELECTOR(s,f,0,n)
 A Recursive Greedy Algorithm 17

RECURSIVE-ACTIVITY-SELECTOR(s,f,k,n)
1) m = k + 1
2) while and s[m] < f[k]
3) m = m+1
4) if
5) return RECURSIVE-ACTIVITY-SELECTOR(s,f,m,n)
6) else return
An Iterative Greedy Algorithm 18
• The recursive procedure can be converted to an iterative
one because the procedure is almost tail recursive.
• It ends with a recursive call to itself followed by a union
operation.
• It is usually a straightforward task to transform a tail-
recursive procedure to an iterative form.
An Iterative Greedy Algorithm 19

GREEDY-ACTIVITY-SELECTOR (s, f, n)
1) A =
2) k = 1
3) for m = 2 to n
4) if
5)
6) k=m
7) return A
Example 20
• Consider the following set of activities sorted by finish time
in monotonically increasing order

1 2 3 4 5 6 7 8 9 10 11
1 3 0 5 3 5 6 7 8 2 12
4 5 6 7 9 9 10 11 12 14 16
Example Solution 21

0 1 2 3 4 5 6 7 8 9 10 11 1 13 14 1 16 time
2 5
The General Method 22
• Most, though not all, of these problems have n inputs and
require us to obtain a subset that satisfies some
constraints.
• Any subset that satisfies these constraints is called a
feasible solution.
• We need to find a feasible solution that either maximizes or
minimizes a given objective function.
• A feasible solution that does this is called an optimal
solution.
 The General Method 23

• The greedy method suggests that one can devise an algorithm that works in
stages, considering one input at a time.
• At each stage, a decision is made regarding whether a particular input is in
an optimal solution.
• Inputs are considered in an order determined by some selection procedure.
• If the inclusion of the next input into the partially constructed optimal
solution will result in an infeasible solution, then this input is not added to
the partial solution.
• Otherwise, it is added.
The General Method 24
• The Selection procedure itself is based on some
optimization measure.
• This measure may be the objective function.
• This version of the greedy technique is called the subset
paradigm.
The General Method 25
• Some algorithms do not need selection of an optimal
subset but make decisions by looking at the inputs in some
order.
• Each decision is made by using an optimization criterion
that is computed using the decisions made so far.
• This version of the greedy technique is called the ordering
paradigm.
Elements of the Greedy Strategy 26

• How can you tell whether a greedy algorithm will solve a

particular optimization problem?
• No way works all the time.
• But the greedy-choice property and optimal substructure are
the two key ingredients.
• If you can demonstrate that the problem has these properties,
then you are well on the way to developing a greedy algorithm
for it.
Greedy-choice Property 27
• You can assemble a globally optimal solution by making
locally optimal (greedy) choices.
• When you are considering which choice to make, you make
the choice that looks best in the current problem, without
considering results from subproblems.
Greedy-choice Property 28
• You make whatever choice seems best at the moment and
then solve the subproblem that re-mains.
• The choice made by a greedy algorithm may depend on
choices so far, but it cannot depend on any future choices
or on the solutions to subproblems.
Greedy-choice Property 29
• You need to prove that a greedy choice at each step yields
a globally optimal solution.
• Typically, the proof examines a globally optimal solution to
some subproblem.
• It then shows how to modify the solution to substitute the
greedy choice for some other choice, resulting in one
similar, but smaller, subproblem.
Greedy-choice Property 30
• By preprocessing the input or by using an appropriate data
structure (often a priority queue), you often can make
greedy choices quickly, thus yielding an efficient algorithm.
Optimal Substructure 31
• A problem exhibits optimal substructure if an optimal
solution to the problem contains within its optimal solutions
to subproblems.
• assume that you arrived at a subproblem by having made
the greedy choice in the original problem.
• All you really need to do is argue that an optimal solution to
the subproblem, combined with the greedy choice already
made, yields an optimal solution to the original problem.
 Designing a Greedy Algorithm 32
• Cast the optimization problem as one in which you make a choice and
are left with one subproblem to solve.
• Prove that there is always an optimal solution to the original problem
that makes the greedy choice, so that greedy choice is always safe.
• Demonstrate optimal substructure by showing that, having made the
greedy choice, what remains is a subproblem with the property that if
you combine an optimal solution to the subproblem with the greedy
choice you have made, you arrive at an optimal solution to the original
problem.

CSE 1205: Object Oriented Programming December 21, 2024

CSE 1205: Object Oriented Programming December 21, 2024 33

Huffman Coding
 Huffman Coding 34
• A technique of compressing data to reduce its size without losing any
of the details.
• Data arrives as a sequence of characters.
• Algorithm finds an optimal way of representing each character as a
binary string.

CSE 1205: Object Oriented Programming December 21, 2024

 Huffman Codes 35
• Suppose you have a 100,000- character data file.
• There are 6-distinct character.

• The frequency of each character is shown in the table

A B C D E F
Frequency 45 13 12 16 9 5
(in thousands)

• We want to represent each character by a binary string (codeword).

CSE 1205: Object Oriented Programming December 21, 2024

 Fixed Length Code 36
• If we want to represent each character by a fixed-length code we need
bits to represent characters.
• For 6 characters we need 3 bits.
A B C D E F
Fixed – Length 000 001 010 011 100 101
Codeword

• We need 300,000 bits to encode entire file.

• Can we do better?

CSE 1205: Object Oriented Programming December 21, 2024

 Variable – Length Codeword 37
• We will assign short codewords for frequent characters, and long
codewords for infrequent characters.

A B C D E F
Frequency 45 13 12 16 9 5
(in thousands)
Variable-length 0 101 100 111 1101 1100
codeword

• Now we require, bits.

• We saved around 25% memory.

• But how do we generate this code?

CSE 1205: Object Oriented Programming December 21, 2024

 Prefix-free Codes 38
• We can’t just randomly assign codes.
• Suppose, we assign A = 0, B = 1, and C = 10.
• I give you the encoded message 101.
• What will be the decoded message? – CB or BAB?

• To remove this ambiguity, we will use prefix-free codes.

• No codeword is also a prefix of some other codeword.
• This achieve the optimal data compression among any character code.

CSE 1205: Object Oriented Programming December 21, 2024

 Prefix-free Codes 39
• To encode a message, we convert each character to its codeword and
concatenate them.
A B C D E F
Variable-length 0 101 100 111 1101 1100
codeword

• The encoded message of the word ‘face’ will be –

1100.0.100.1101 = 110001001101

CSE 1205: Object Oriented Programming December 21, 2024

 Prefix-free Codes 40
• To decode a message, we first detect the first codeword.
• Each codeword is unique and unambiguous.

• Then we translate the first codeword to initial character.

• And we repeat the decoding process on the remainder.

CSE 1205: Object Oriented Programming December 21, 2024

 Constructing a Huffman Code 41
• We will use a greedy algorithm to construct an optimal-prefix free code.
• Let’s call the procedure Huffman.
• Let C be the set of n characters.
• and each character c has attribute c.freq giving its frequency.

• The algorithm builds the tree T in a bottom-up manner.

• It begins with a set of |C| leaves and performs a sequence of |C| - 1
merging operation to create the final tree.

CSE 1205: Object Oriented Programming December 21, 2024

 Constructing a Huffman Code 42

CSE 1205: Object Oriented Programming December 21, 2024

 Codeword Construction 43
• Let’s consider the previous example.
• We had the following frequency table.
A B C D E F
Frequency 45 13 12 16 9 5
(in thousands)

• We will first create 6 leaf nodes.

A:45 B:13 C:12 D:16 E:9 F:5

CSE 1205: Object Oriented Programming December 21, 2024

 Codeword Construction 44
• We will insert these nodes in a priority queue.
• Nodes will be sorted based on their frequency (freq) value.

F:5 E:9 C:12 B:13 D:16 A:45

• Now we will repeatedly

• take the two nodes with lowest frequency,
• merge them
• and put the merged node back in the priority queue.

CSE 1205: Object Oriented Programming December 21, 2024

 Codeword Construction 45

F:5 E:9 C:12 B:13 D:16 A:45

14
0 1

F:5 E:9

14
C:12 B:13 0 1 D:16 A:45

F:5 E:9

CSE 1205: Object Oriented Programming December 21, 2024

 Codeword Construction 46
14
C:12 B:13 D:16 A:45

F:5 E:9

25

C:12 14 B:13 25
D:16 A:45
0 1 0 1
0
F:5 E:9 C:12 B:13

CSE 1205: Object Oriented Programming December 21, 2024

 Codeword Construction 47

14 25
D:16 A:45
0 1 0 1

F:5 E:9 C:12 B:13

30
0 1
D:1
14 30
25 06 1
0 1
0 1 14 D:16 A:45
F:5 E:9 0 1
C:12 B:13
F:5 E:9

CSE 1205: Object Oriented Programming December 21, 2024

 Codeword Construction 48

25 30
0 1
0 1 14 D:16 A:45
0 1
C:12 B:13
F:5 E:9

55

25 30
0 1
0 1
14 D:16

C:12 B:13 0 1

F:5 E:9

CSE 1205: Object Oriented Programming December 21, 2024

 Codeword Construction 49

55

25 30 A:45
0 1
0 1
14 D:16
C:12 B:13 0 1
F:5 E:9
1

100
0 1

A:45 55

25 30
0 1
0 1
14 D:16
C:12 B:13 0 1
F:5 E:9
1

CSE 1205: Object Oriented Programming December 21, 2024

 Codeword Construction 50
10
0 0 1
A:4 55
5
0 1

25 30
0 1 1 1
B:1
C:12 14 D:16
3
0 1

F:5 E:9

CSE 1205: Object Oriented Programming December 21, 2024

 Huffman Coding Complexity 51
• The time complexity to encode each unique character is .
• Extracting minimum frequency from the priority queue takes place 2*(n-1)
times.
• And its complexity is .

CSE 1205: Object Oriented Programming December 21, 2024

CSE 1205: Object Oriented Programming December 21, 2024 52

Job Sequencing Problem

 Problem Statement 53
• We are given a set of n jobs.
• Each job has a deadline and associated profit if the job is finished
before the deadline.
• Every job takes a single unit of time.
• Maximize the total profit if only one job can be scheduled at a time.

CSE 1205: Object Oriented Programming December 21, 2024

 Naïve Approach 54
• We can generate all possible subset of jobs.
• Check if the set is feasible.
• Keep track of maximum profit among all feasible solution.

CSE 1205: Object Oriented Programming December 21, 2024

 Greedy Approach 55
• We must formulate a optimization measure to determine how the next
job is chosen.
• We can choose the objective function as our optimization measure.
• Consider jobs in non-increasing order of the ’s.
• Keep taking jobs as long they create a feasible solution.

• How do we know if a set of job feasible?

• Obviously, we can try all possible solution.
• If we have k jobs we will get k! way.
• Is there a better option?

CSE 1205: Object Oriented Programming December 21, 2024

 Greedy Approach 56
Theorem: Let J be a set of K jobs and a permutation of jobs in J such
that . Then J is feasible solution iff the jobs in J can be processed in the
order without violating any deadline.

Theorem: The greedy method described earlier always obtains an

optimal solution.

CSE 1205: Object Oriented Programming December 21, 2024

 High-level Description 57

1.

CSE 1205: Object Oriented Programming December 21, 2024

 Greedy Approach 58
• Instead of trying all possible permutation we can do the following for
feasibility test.
• Sort all jobs in non-increasing order of profit.
• Iterate on jobs in non-increasing order of profit. For each job, do the
following:
• Find a time slot i, such that slot is empty and i<deadline and i is greatest.
• Put the job in this slot and mark this slot filled.
• If no such i exists, then ignore this job.

CSE 1205: Object Oriented Programming December 21, 2024

 Example 59
• Consider the following list of jobs

JobID A B C D E
Deadline 2 1 2 1 3
Profit 100 19 27 25 15

CSE 1205: Object Oriented Programming December 21, 2024

 Complexity Analysis 60
• Time Complexity:
• Auxiliary Space:

CSE 1205: Object Oriented Programming December 21, 2024

 References 61
• Huffman Coding Algorithm (programiz.com)
• Job Sequencing Problem – GeeksforGeeks
• Job Sequencing Problem using Disjoint Set – GeeksforGeeks
• Introduction to Algorithm
• Computer Algorithms

CSE 1205: Object Oriented Programming December 21, 2024

CSE 1205: Object Oriented Programming December 21, 2024 62

Thank You.