Scribd
Upload
36 views

Algorithm Design Paradigm-3 - 1

The document discusses greedy algorithms and their application to optimization problems like scheduling, Huffman coding, knapsack problems, and set covering. It describes how greedy algorithms work by making locally optimal choices at each step in the hope of finding a global optimum. Examples are provided of applying greedy strategies to fractional knapsack problems and set covering problems.

Uploaded by

Yukti Satheesh
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
36 views

Algorithm Design Paradigm-3 - 1

The document discusses greedy algorithms and their application to optimization problems like scheduling, Huffman coding, knapsack problems, and set covering. It describes how greedy algorithms work by making locally optimal choices at each step in the hope of finding a global optimum. Examples are provided of applying greedy strategies to fractional knapsack problems and set covering problems.

Uploaded by

Yukti Satheesh
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

Algorithm Design

Paradigm – 3_1

Dr. Sandeep Kumar Satapathy


School of Computer Sc. & Engg. (SCOPE)
VIT University, Chennai

Source: Introduction to Algorithm by CORMEN, LEISERSON, RIVEST and STEIN (3 rd Edition)


Algorithm Design and Application by GOODRICH and TAMASSIA (Wiley Publication)
Greedy Algorithms: Introduction
• An optimization problem is one in which you want to find, not just a
solution, but the best solution
• A “greedy algorithm” sometimes works well for optimization problems
• A greedy algorithm works in phases. At each phase:
• You take the best you can get right now, without regard for future
consequences
• You hope that by choosing a local optimum at each step, you will end
up at a global optimum
• Greedy algorithms do not always yield optimal solutions, but for many
problems they do.
Greedy Algorithms: Strategic Steps
1. Determine the optimal substructure of the problem.
2. Develop a recursive solution. (For the activity-selection problem, the
formulated recurrence has been bypassed developing a recursive
algorithm based on this recurrence.)
3. Show that if we make the greedy choice, then only one subproblem
remains.
4. Prove that it is always safe to make the greedy choice. (Steps 3 and 4
can occur in either order.)
5. Develop a recursive algorithm that implements the greedy strategy.
6. Convert the recursive algorithm to an iterative algorithm.
Greedy Algorithms: Introduction
• Algorithms to be solved by using Greedy Strategy
• Scheduling Algorithm
• Huffman Code
• Knapsack Problem
• Greedy Set Cover
Knapsack Problem: Introduction
• The knapsack problem is a problem in
combinatorial optimization:
• Given a set of items, each with a
weight and a value,
• determine the number of each item to
include in a collection so that the total
weight is less than or equal to a given
limit and the total value is as large as
possible.

• 0-1 Knapsack Problem


• Fractional Knapsack Problem
Knapsack Problem: Mathematical Definition
• A thief robbing a store finds 𝑛 items. The 𝑖𝑡ℎ item is worth 𝑣𝑖 dollars and
weighs 𝑤𝑖 pounds, where 𝑣𝑖 and 𝑤𝑖 are integers.
• The thief wants to take as valuable a load as possible, but he can carry at
most 𝑊 pounds in his knapsack, for some integer 𝑊.
• Which items should he take?
• In 0-1 Knapsack Problem, for each item, the thief must either take it or
leave it behind; he cannot take a fractional amount of an item or take an
item more than once
• In Fractional Knapsack Problem, the setup is the same, but the thief can
take fractions of items, rather than having to make a binary (0-1) choice for
each item.
You can think of an item in the 0-1 knapsack problem as being like a gold bars
and an item in the fractional knapsack problem as more like gold dust
Knapsack Problem: Mathematical Definition
• A thief robbing a store finds 𝑛 items. The 𝑖𝑡ℎ item is worth 𝑣𝑖 dollars and
weighs 𝑤𝑖 pounds, where 𝑣𝑖 and 𝑤𝑖 are integers.
• The thief wants to take as valuable a load as possible, but he can carry at
most 𝑊 pounds in his knapsack, for some integer 𝑊.
• Which items should he take?
• In 0-1 Knapsack Problem, for each item, the thief must either take it or
leave it behind; he cannot take a fractional amount of an item or take an
item more than once
• In Fractional Knapsack Problem, the setup is the same, but the thief can
take fractions of items, rather than having to make a binary (0-1) choice for
each item.
Knapsack Problem: Greedy Strategy
• Fractional Knapsack Problem, is suitable for applying greedy strategy but
0-1 knapsack problem can not be solved by the greedy approach
• Consider the fractional knapsack problem, where we are given a set 𝑆 of 𝑛
items, such that each item 𝑖 has a positive benefit 𝑣𝑖 and a positive weight
𝑤𝑖, and we wish to find the maximum-benefit subset that does not exceed
a given weight 𝑊.
Knapsack Problem: Greedy Strategy
Knapsack Problem: Greedy Strategy
Set Cover Problem: Introduction
• In the set cover problem, we are given a universe 𝑈, such that |𝑈| = 𝑛,
and sets 𝑆1, … , 𝑆𝑘 ⊆ 𝑈.
• A set cover is a collection 𝐶 of some of the sets from 𝑆1, … , 𝑆𝑘 whose
union is the entire universe 𝑈.
• Formally, 𝐶 is a set cover if ‫𝑈 = 𝑖𝑆 𝐶∈𝑖𝑆ڂ‬. We would like to minimize |𝐶|.
Set Cover Problem: Example
• Input: Let 𝑈 = {1,2,3,4,5} and 𝑆 = {𝑆1, 𝑆2, 𝑆3}
Let 𝑆1 = {4,1,3}, 𝐶𝑜𝑠𝑡(𝑆1) = 5
𝑆2 = {2,5}, 𝐶𝑜𝑠𝑡(𝑆2) = 10
𝑆3 = {1,4,3,2}, 𝐶𝑜𝑠𝑡(𝑆3) = 3
• Output: Minimum cost of set cover is 13 and set cover is {𝑆2, 𝑆3}
• There are two possible set covers {𝑆1, 𝑆2} with cost 15 and {𝑆2, 𝑆3} with
cost 13.
Set Cover Problem: Greedy Strategy
• Let 𝑈 be the universe of elements, {𝑆1, 𝑆2, … , 𝑆𝑚} be collection of subsets
of 𝑈 and 𝐶𝑜𝑠𝑡 𝑆1 , 𝐶𝑜𝑠𝑡 𝑆2 , … , 𝐶𝑜𝑠𝑡(𝑆𝑚) be costs of subsets.
• Algorithm:
1) Let 𝐼 represents set of elements included so far. Initialize 𝐼 = {}
2) Do following while 𝐼 is not same as 𝑈.
a) Find the set 𝑆𝑖 in {𝑆1, 𝑆2, … , 𝑆𝑚} whose cost effectiveness is
smallest, i.e., the ratio of cost 𝐶𝑜𝑠𝑡(𝑆𝑖) and number of newly
added elements is minimum. Basically we pick the set for which
following value is minimum.
𝐶𝑜𝑠𝑡(𝑆𝑖) / |𝑆𝑖 − 𝐼|
b) Add elements of above picked 𝑆𝑖 to 𝐼, i.e., 𝐼 = 𝐼 𝑈 𝑆𝑖
Greedy Set Cover: Example
• First Iteration:
𝐼 = {}
Per new element cost for 𝑆1 = 𝐶𝑜𝑠𝑡(𝑆1)/|𝑆1 – 𝐼| = 5/3
Per new element cost for 𝑆2 = 𝐶𝑜𝑠𝑡(𝑆2)/|𝑆2 – 𝐼| = 10/2
Per new element cost for 𝑆3 = 𝐶𝑜𝑠𝑡(𝑆3)/|𝑆3 – 𝐼| = 3/4
Since 𝑆3 has minimum value 𝑆3 is added, 𝐼 becomes {1,4,3,2}.
Greedy Set Cover: Example

• Second Iteration:
𝐼 = {1,4,3,2}
Per new element cost for 𝑆1 = 𝐶𝑜𝑠𝑡(𝑆1)/|𝑆1 – 𝐼| = 5/0
Note that 𝑆1 doesn’t add any new element to 𝐼.
Per new element cost for 𝑆2 = 𝐶𝑜𝑠𝑡(𝑆2)/|𝑆2 – 𝐼| = 10/1
Note that 𝑆2 adds only 5 to 𝐼.

• The greedy algorithm provides the optimal solution for above example,
but it may not provide optimal solution all the time.
Greedy Set Cover: Example No Optimal Sol
• Example (Not Giving Optimal Solution):
𝑆1 = {1, 2}
𝑆2 = {2, 3, 4, 5}
𝑆3 = {6, 7, 8, 9, 10, 11, 12, 13}
𝑆4 = {1, 3, 5, 7, 9, 11, 13}
𝑆5 = {2, 4, 6, 8, 10, 12, 13}
• Let the cost of every set be same.
• The greedy algorithm produces result as {𝑆3, 𝑆2, 𝑆1}
• The optimal solution is {𝑆4, 𝑆5}

You might also like