Greedy Algorithms

Dr. Awnish Kumar

Assistant Professor
Computer Science and Engineering Department
National Institute of Technology Agartala
Introduction
• Greedy algorithms work in stages.
• In each stage, a decision is made that is good at that point, without
bothering about the future.
• This means that some local best is chosen. It assumes that a local
good selection makes for a global optimal solution.
• In the game of Chess, every time we make a decision about a move,
we have to also think about the future consequences.
• Whereas, in the game of Tennis (or Volleyball), our action is based on
the immediate situation.
Elements of Greedy Algorithms
• The two basic properties of optimal Greedy algorithms are:
1) Greedy choice property
2) Optimal substructure
• Greedy choice property says that the globally optimal solution can be obtained by
making a locally optimal solution (Greedy).
• The choice made by a Greedy algorithm may depend on earlier choices but not
on the future.
• It iteratively makes one Greedy choice after another and reduces the given
problem to a smaller one.
• A problem exhibits optimal substructure if an optimal solution to the problem
contains optimal solutions to the subproblems.
• That means we can solve subproblems and build up the solutions to solve larger
problems.
Does Greedy Always Work?
• Making locally optimal choices does not always work.
• Hence, Greedy algorithms will not always give the best solutions.
• We will see particular examples in the Problems section and in the
Dynamic Programming chapter.
Advantages and Disadvantages of Greedy
Method
• The main advantage of the Greedy method is that it is
straightforward, easy to understand and easy to code.
• In Greedy algorithms, once we make a decision, we do not have to
spend time reexamining the already computed values.
• Its main disadvantage is that for many problems there is no greedy
algorithm.
• That means, in many cases there is no guarantee that making locally
optimal improvements in a locally optimal solution gives the optimal
global solution.
Greedy Applications
• Sorting: Selection sort, Topological sort
• Priority Queues: Heap sort
• Huffman coding compression algorithm
• Prim’s and Kruskal’s algorithms
• Shortest path in Weighted Graph [Dijkstra’s]
• Fractional Knapsack problem
• Disjoint sets-UNION by size and UNION by height (or rank)
• Job scheduling algorithm
Huffman Coding Algorithm
• Given a set of n characters from the alphabet A [each
character c ∈ A] and their associated frequency freq(c),
find a binary code for each character c ∈ A, such that
∑c ∈ A freq(c)|binarycode(c)| is minimum, where
binarycode(c)| represents the length of binary code of
character c.
• That means the sum of the lengths of all character codes should be
minimum [the sum of each character’s frequency multiplied by the
number of bits in the representation].
• The idea behind the character coding is to give longer binary codes for
less frequent characters and groups of characters.
• Also, the character coding is constructed in such a way that no two
character codes are prefixes of each other.
Huffman Coding Algorithm - Example
• Let’s assume that after scanning a file we find the following character
frequencies:
• Given this, create a binary tree for each character that also stores the
frequency with which it occurs (as shown below).
Huffman Coding Algorithm - Example
• The algorithm works as follows: In the list, find the two binary trees
that store minimum frequencies at their nodes.
• Connect these two nodes at a newly created common node that will
store no character but will store the sum of the frequencies of all the
nodes connected below it. So our picture looks like this:
Huffman Coding Algorithm - Example
• Repeat this process until only one tree is left:

10
Huffman Coding Algorithm - Example

11
Huffman Coding Algorithm - Example

12
Huffman Coding Algorithm - Example
• Once the tree is built, each leaf node corresponds to a letter with a code.
• To determine the code for a particular node, traverse from the root to the leaf node.
• For each move to the left, append a 0 to the code, and for each move to the right, append a 1.
• As a result, for the above generated tree, we get the following codes:

•
Calculating Bits Saved
• let’s assume each character is stored with a three bit code. Since there are 133
such characters (multiply total frequencies by 3), the total number of bits used is
3 * 133 = 399.
• Using the Huffman coding frequencies we can calculate the new total number of
bits used:

• Thus, we saved 399 – 238 = 161 bits, or nearly 40% of the storage space
Problem
• Given an array F with size n. Assume the array content F[i] indicates the length of the ith file and
we want to merge all these files into one single file. Provide an algorithm with minimum merging
cost.
• Approach 1: Merge the files contiguously. That means select the first two files and merge them.
Then select the output of the previous merge and merge with the third file, and keep going.
• let us consider the following file sizes array. F = {10,5,100,50,20,15}
• As per the above algorithm, we need to merge the first two files (10 and 5 size files), and as a
result we get the following list of files. In the list below, 15 indicates the cost of merging two files
with sizes 10 and 5. {15,100,50,20,15}
• Similarly, merging 15 with the next file 100 produces: {115,50,20,15}. For the subsequent steps
the list becomes {165,20,15}, {185,15}
• Finally, {200}
• The total cost of merging = Cost of all merging operations = 15 + 115 + 165 + 185 + 200 = 680.
Problem - 1
• Given an array F with size n. Assume the array content F[i] indicates the length of the ith
file and we want to merge all these files into one single file. Provide an algorithm with
minimum merging cost. F = {10,5,100,50,20,15}
• Approach 2: Merge the files in pairs. That means after the first step, the algorithm
produces the n/2 intermediate files. For the next step, we need to consider these
intermediate files and merge them in pairs and keep going.
• Note: Sometimes this algorithm is called 2-way merging. Instead of two files at a time, if
we merge K files at a time then we call it K-way merging.
• we need to merge the first pair of files (10 and 5 size files), the second pair of files (100
and 50) and the third pair of files (20 and 15). As a result we get the following list of files.
{15,150,35}
• Similarly, merge the output in pairs and this step produces [below, the third element
does not have a pair element, so keep it the same]: {165,35} Finally, {185}
• The total cost of merging = Cost of all merging operations = 15 + 150 + 35 + 165 + 185 =
550.
Problem - 1
• Given an array F with size n. Assume the array content F[i] indicates the length of the ith
file and we want to merge all these files into one single file. Provide an algorithm with
minimum merging cost. F = {10,5,100,50,20,15}
• Approach 3: As per the above algorithm, after sorting the list it becomes:
{5,10,15,20,50,100}. We need to merge the two smallest files (5 and 10 size files) and as
a result we get the following list of files. In the list below, 15 indicates the cost of merging
two files with sizes 10 and 5. {15,15,20,50,100}
• Similarly, merging the two smallest elements (15 and 15) produces: {20,30,50,100}. For
the subsequent steps the list becomes {50,50,100} // merging 20 and 30
• {100,100} // merging 20 and 30
• Finally, {200}
• The total cost of merging = Cost of all merging operations = 15 + 30 + 50 + 100 + 200 =
395. So, this algorithm is producing the optimal solution for this merging problem.
Interval Scheduling Algorithm
• Given a set of n intervals S = {(starti, endj) for 1 ≤ i≤ n. Let us assume
that we want to find a maximum subset S′ of S such that no pair of
intervals in S′ overlaps. Provide an algorithm to schedule maximum
number of possible events.
Interval Scheduling Algorithm
• Approach 1:

• This algorithm does not solve the problem of finding a maximum

subset of non- overlapping intervals. Consider the following intervals.
The optimal solution is {M,O,N,K}. However, the interval that overlaps
with the fewest others is C, and the given algorithm will select C first.
Interval Scheduling Algorithm
• Approach 2: Select the event that starts first.
• No. It will not give the optimal solution. Let us consider the example
below. It can be seen that the optimal solution is 4 whereas the given
algorithm gives 1.
Interval Scheduling Algorithm
• Approach 3: Select the event with shortest interval.
• No. It will not give the optimal solution. Let us consider the example
below. It can be seen that the optimal solution is 2 whereas the
algorithm gives 1.
Interval Scheduling Algorithm
• Approach 4:

• Time complexity = Time for sorting + Time for scanning = O(nlogn + n) =

O(nlogn).
Making change problem
• Consider the making change problem in the country of India. The input to this
problem is an integer M. The output should be the minimum number of coins to
make M rupees of change.
• In India, assume the available coins are 1,5,10,20,25,50 rupees. Assume that we
have an unlimited number of coins of each type.
• Solution: Take as many coins as possible from the highest denominations. So for
example, to make change for 234 rupees the greedy algorithm would take four 50
rupee coins, one 25 rupee coin, one 5 rupee coin, and four 1 rupee coins.
• The greedy algorithm is not optimal for the problem of making change with the
minimum number of coins when the denominations are 1,5,10,20,25, and 50. In
order to make 40 rupees, the greedy algorithm would use three coins of 25,10,
and 5 rupees. The optimal solution is to use two 20-shilling coins.
Fractional Knapsack Problem
• Given the weights and profits of N items, in the form of {profit,
weight} put these items in a knapsack of capacity W to get the
maximum total profit in the knapsack. In Fractional Knapsack, we can
break items for maximizing the total value of the knapsack.
• Input: arr[] = {{60, 10}, {100, 20}, {120, 30}}, W = 50
Output: 240
Explanation: By taking items of weight 10 and 20 kg and 2/3
fraction of 30 kg. Hence total price will be 60+100+(2/3)(120) = 240
• Input: arr[] = {{500, 30}}, W = 10
Output: 166.667
Fractional Knapsack Problem
• Approach 1: Try all possible subsets with all different fractions.
• Time Complexity: O(2N) Auxiliary Space: O(N)
• Approach 2: The basic idea of the greedy approach is to calculate the
ratio profit/weight for each item and sort the item on the basis of this
ratio. Then take the item with the highest ratio and add them as much
as we can (can be the whole element or a fraction of it).
• This will always give the maximum profit because, in each step it adds
an element such that this is the maximum possible profit for that
much weight.
Fractional Knapsack Problem
• Consider the example: arr[] = {{100, 20}, {60, 10}, {120, 30}}, W = 50.
• Sorting: Initially sort the array based on the profit/weight ratio. The sorted
array will be {{60, 10}, {100, 20}, {120, 30}}.
• Iteration:
• For i = 0, weight = 10 which is less than W. So add this element in the
knapsack. profit = 60 and remaining W = 50 – 10 = 40.
• For i = 1, weight = 20 which is less than W. So add this element too. profit =
60 + 100 = 160 and remaining W = 40 – 20 = 20.
• For i = 2, weight = 30 is greater than W. So add 20/30 fraction
= 2/3 fraction of the element. Therefore profit = 2/3 * 120 + 160 = 80 + 160
= 240 and remaining W becomes 0.
• So the final profit becomes 240 for W = 50.
Additional Problems
• Number of railway-platforms: At a railway station, we have a time-
table with the trains’ arrivals and departures. We need to find the
minimum number of platforms so that all the trains can be
accommodated as per their schedule.
• Example: The timetable is as given below, the answer is 3. Otherwise,
the railway station will not be able to accommodate all the trains.
Thank You.