Scribd
Upload
2 views

Week10_Optimization

The document discusses optimization problems and the use of Greedy Algorithms and Dynamic Programming (DP) to solve them. It explains the characteristics, use cases, and examples of both approaches, highlighting their differences in terms of speed, memory usage, and solution quality. Additionally, it provides a case study comparing the 0/1 Knapsack and Fractional Knapsack problems, demonstrating how the choice of algorithm affects the optimal solution.

Uploaded by

Muskan Aneja
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

Week10_Optimization

The document discusses optimization problems and the use of Greedy Algorithms and Dynamic Programming (DP) to solve them. It explains the characteristics, use cases, and examples of both approaches, highlighting their differences in terms of speed, memory usage, and solution quality. Additionally, it provides a case study comparing the 0/1 Knapsack and Fractional Knapsack problems, demonstrating how the choice of algorithm affects the optimal solution.

Uploaded by

Muskan Aneja
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Optimization Using Greedy Algorithms and Dynamic Programming

1. What is an Optimization Problem?


An optimization problem aims to maximize or minimize a particular objective,
such as:
• Maximizing profit
• Minimizing cost, distance, or time
• Finding the most efficient way to perform a task
To solve these problems efficiently, we commonly use algorithmic strategies
like Greedy Algorithms and Dynamic Programming (DP).

2. Greedy Algorithms
Definition
A Greedy algorithm makes a sequence of choices, each of which is the best at
that moment (locally optimal), in the hope that the final result is globally
optimal.
When to Use Greedy Approach
• The problem exhibits the Greedy Choice Property (a global optimum can
be obtained by choosing local optimums).
• The problem has Optimal Substructure (an optimal solution to the
problem contains optimal solutions to its subproblems).
Characteristics
• Simple and fast
• No backtracking
• No need to solve overlapping subproblems
Examples
Problem Description
Activity Selection Select the maximum number of non-overlapping
activities
Huffman Coding Build a binary tree with minimum weighted path
length
Dijkstra’s Algorithm Find shortest path from a source (non-negative
weights)
Fractional Knapsack Choose items with maximum value/weight ratio
until full
Prim's & Kruskal's Construct Minimum Spanning Tree (MST) by
Algorithms greedy edge selection

3. Dynamic Programming (DP)


Definition
Dynamic Programming solves problems by breaking them into overlapping
subproblems, solving each subproblem only once, and storing the results.
When to Use DP
• The problem has Overlapping Subproblems (same subproblem appears
multiple times).
• The problem has Optimal Substructure (solution to the main problem
depends on optimal solutions to subproblems).
Characteristics
• Uses memoization (top-down) or tabulation (bottom-up)
• More complex but always gives optimal result
• Higher memory usage due to storing results
Examples
Problem Description
0/1 Knapsack Select items without exceeding weight to maximize
value
Fibonacci Numbers Calculate nth number using stored results of
previous numbers
Longest Common Find the longest sequence present in both strings in
Subsequence (LCS) order
Matrix Chain Determine parenthesis placement to minimize
Multiplication multiplication operations
Edit Distance Find the minimum operations to convert one string
to another
4. Greedy vs. Dynamic Programming - Comparison Table
Feature Greedy Algorithm Dynamic Programming
Approach Locally optimal choices Solve and store
subproblems
Subproblem Not required Required
Overlap
Optimal Required Required
Substructure
Solution Quality May not be globally Always optimal
optimal
Speed Usually faster Usually slower
Memory Usage Low High
Common Activity Selection, 0/1 Knapsack, LCS, Edit
Examples Huffman, Dijkstra Distance

5. Choosing Between Greedy and DP


When faced with an optimization problem:
• If it has Greedy Choice Property and Optimal Substructure, and no
overlapping subproblems → Try Greedy.
• If it has Overlapping Subproblems and Optimal Substructure → Use
Dynamic Programming.

6. Case Study: 0/1 Knapsack vs. Fractional Knapsack


• 0/1 Knapsack
o You must either take an item or leave it.
o Greedy fails to give the optimal result.
o Use Dynamic Programming.
• Fractional Knapsack
o You can take a part of an item.
o Greedy (based on value/weight ratio) gives optimal result.
o Use Greedy Algorithm.
Fractional Knapsack

The knapsack problem states that − given a set of items, holding weights and
profit values, one must determine the subset of the items to be added in a
knapsack such that, the total weight of the items must not exceed the limit of
the knapsack and its total profit value is maximum.

It is one of the most popular problems that take greedy approach to be solved.
It is called as the Fractional Knapsack Problem.

To explain this problem a little easier, consider a test with 12 questions, 10


marks each, out of which only 10 should be attempted to get the maximum
mark of 100. The test taker now must calculate the highest profitable
questions the one that hes confident in to achieve the maximum mark.
However, he cannot attempt all the 12 questions since there will not be any
extra marks awarded for those attempted answers. This is the most basic real-
world application of the knapsack problem.

Knapsack Algorithm
The weights (Wi) and profit values (Pi) of the items to be added in the knapsack
are taken as an input for the fractional knapsack algorithm and the subset of
the items added in the knapsack without exceeding the limit and with
maximum profit is achieved as the output.

Algorithm steps:
1. Consider all the items with their weights and profits mentioned
respectively.
2. Calculate Pi/Wi of all the items and sort the items in descending order
based on their Pi/Wi values.
3. Without exceeding the limit, add the items into the knapsack.
4. If the knapsack can still store some weight, but the weights of other
items exceed the limit, the fractional part of the next time can be added.
5. Hence, giving it the name fractional knapsack problem.
Examples
For the given set of items and the knapsack capacity of 10 kg, find the subset
of the items to be added in the knapsack such that the profit is maximum.

Items 1 2 3 4 5
Weights (in kg) 3 3 2 5 1
Profits 10 15 10 12 8

Solution

1. Considering Weights Only (Choosing the Lightest Items First)

If we prioritize items with the smallest weight first (greedy approach), we choose
items in ascending order of weight:

• Item 5 (Weight = 1, Profit = 8)


• Item 3 (Weight = 2, Profit = 10)
• Item 1 (Weight = 3, Profit = 10)
• Item 2 (Weight = 3, Profit = 15)
• Item 4 (Weight = 1(as per the knapsack capacity), Profit = 12/5=2.4)

Total weight = 1 + 2 + 3 + 3 + 1 = 10 kg (fits within the 10 kg capacity)


Total profit = 8 + 10 + 10 + 15 + 2.4= 45.4

2. Considering Profits Only (Choosing the Highest Profit Items


First)

If we prioritize the highest-profit items first:

• Item 2 (Profit = 15, Weight = 3)


• Item 1 (Profit = 10, Weight = 3)
• Item 3 (Profit = 10, Weight = 2)
• Item 4 (Profit = 4.8, Weight = 2) (cannot be included as it exceeds
capacity)
Total weight = 3 + 3 + 2 + 2 = 10 kg (fits within the 10 kg capacity)
Total profit = 15 + 10 + 10 + 4.8 = 39.8

3. Considering Profits/Weights ratio (Pi/Wi)

Step 1
Given, n = 5
Wi = {3, 3, 2, 5, 1}
Pi = {10, 15, 10, 12, 8}
Calculate Pi/Wi for all the items
Items 1 2 3 4 5
Weights (in kg) 3 3 2 5 1
Profits 10 15 10 20 8
Pi/Wi 3.3 5 5 4 8

Step 2
Arrange all the items in descending order based on Pi/Wi
Items 5 2 3 4 1
Weights (in kg) 1 3 2 5 3
Profits 8 15 10 20 10
Pi/Wi 8 5 5 4 3.3

Step 3
Without exceeding the knapsack capacity, insert the items in the knapsack
with maximum profit.

Knapsack = {5, 2, 3}
However, the knapsack can still hold 4 kg weight, but the next item having 5
kg weight will exceed the capacity. Therefore, only 4 kg weight of the 5 kg will
be added in the knapsack.

Items 5 2 3 4 1
Weights (in kg) 1 3 2 5 3
Profits 8 15 10 20 10
Knapsack 1 1 1 4/5 0

Hence, the knapsack holds the weights = [(1 * 1) + (1 * 3) + (1 * 2) + (4/5 * 5)]


= 10, with maximum profit of [(1 * 8) + (1 * 15) + (1 * 10) + (4/5 * 20)] = 49.

Problem 2:

Solution:

You might also like