Scribd
Upload
2 views18 pages

Lecture 3

The document discusses the design and analysis of algorithms, focusing on the Divide and Conquer strategy and its application in solving problems like the Coin Change Problem. It explains how to analyze these algorithms using recurrence relations and highlights the importance of recognizing overlapping subproblems in optimization problems. Additionally, it outlines the optimal substructure property, which is essential for dynamic programming solutions.

Uploaded by

mokshimani123
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
2 views18 pages

Lecture 3

The document discusses the design and analysis of algorithms, focusing on the Divide and Conquer strategy and its application in solving problems like the Coin Change Problem. It explains how to analyze these algorithms using recurrence relations and highlights the importance of recognizing overlapping subproblems in optimization problems. Additionally, it outlines the optimal substructure property, which is essential for dynamic programming solutions.

Uploaded by

mokshimani123
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 18

BITS, PILANI – K. K.

BIRLA GOA CAMPUS

Design & Analysis of Algorithms


(CS F364)

Lecture No. 3

1
Review - Algorithm Design Strategy
• Divide & Conquer
- binary search
- merge sort
- quick sort
- Matrix Multiplication (Strassen Algorithm)
Many More
Many standard iterative algorithms can be written
as Divide & Conquer Algorithms.
Example: Sum/Max of n numbers
Question: Any advantage in doing so?
Divide & Conquer
Divide-and-conquer algorithms:
1. Dividing the problem into smaller sub-
problems
(independent sub-problems)
2. Solving those sub-problems
3. Combining the solutions for those smaller
sub-problems to solve the original problem
Divide & Conquer
How to analyze Divide & Conquer Algorithms
Generally, using Recurrence Relations
• Let T(n) be the number of operations required to solve
the problem of size n.
Then T(n) = a T(n/b) + c(n) where
- each sub-problem is of size n/b
- There are a such sub-problems
- c(n) extra operations are required to combine the
solutions of sub-problems into a solution of the
original problem
Divide & Conquer
• How to solve Recurrence Relations
- Substitution Method
- works in simple relations
- Masters Theorem
- See Text Book for details
Coin Change Problem
Coin Change Problem
Given a value N, if we want to make change for
N paisa, and we have infinite supply of each of C
= { 1, 5, 10, 25} valued coins, what is the
minimum number of coins to make the change?
Solution – Easy (We all know this)
Greedy Solution
Coin Change Problem
Key Observations
• Optimization Problem
• Making Change is possible
i.e. solution exists
• Multiple solution exists
For example: 60 - 25, 25, 10 OR 10, 10, six times
• Question has two parts
1. Minimum number of coins required
2. Which coins are part of the solution
• What is the guarantee that solution works?
Coin Change Problem
Key Observations
Suppose we add coin of denomination 20 to the set
i.e. C = { 1, 5, 10, 20, 25}
If N = 40
Then the greedy solution fails.
Because the greedy solution will give answer as
3 which is {25, 10, 5}
Whereas correct answer is {20, 20}
Course Objective
Main Question:
Given an problem (mostly optimization
problem)
1. How to decide whether to use
Greedy Strategy OR Dynamic Programming Strategy?
2. How to show that solution works?
We will explore these ideas in the initial few
lectures.
Coin Change Problem
Suppose we want to compute the minimum
number of coins with values
d[1], d[2], …,d[n] where each d[i]>0
& where coin of denomination i has value d[i]
Let c[i][j] be minimum number of coins required
to pay an amount of j units 0<=j<=N using only
coins of denominations 1 to i, 1<=i<=n
C[n][N] is the solution to the problem
Coin Change Problem
In calculating c[i][j], notice that:
• Suppose we do not use the coin with value
d[i] in the solution of the (i,j)-problem,
then c[i][j] = c[i-1][j]
• Suppose we use the coin with value d[i] in the
solution of the (i,j)-problem,
then c[i][j] = 1 + c[i][ j-d[i]]
Since we want to minimize the number of coins,
we choose whichever is the better alternative
Coin Change Problem – Recurrence
Therefore
c[i][j] = min{c[i-1][j], 1 + c[i][ j-d[i]]}
&
c[i][0] = 0 for every i
Alternative 1
Recursive algorithm
Overlapping Subproblems
When a recursive algorithm revisits the same
problem over and over again, we say that the
optimization problem has overlapping
subproblems.
How to observe/prove that problem has
overlapping subproblems.
Answer – Draw Computation tree and observe
Overlapping Subproblems
Computation Tree

Dynamic-programming algorithms typically take advantage of overlapping


subproblems by solving each subproblem once and then storing the solution in a
table where it can be looked up when needed, using constant time per lookup.
Overlapping Subproblems- Fibonacci numbers
• Fibonacci numbers:
F(n) = F(n-1) + F(n-2)
F(0) = 0 & F(1) = 1
Computing the n Fibonacci number recursively (top-
th

down):
F(n)

F(n-1) + F(n-2)

F(n-2) + F(n-3) F(n-3) + F(n-4)


Subproblems share subsubproblems
...
Coin Change Problem
• Example
We have to pay 8 units with coins worth 1,4 & 6 units
For example c[2][6] is obtained in this case as the smaller of c[1][6]
and 1+ c[2][6-d[2]] = 1+c[2][2]
The other entries of the table are obtained similarly
0 1 2 3 4 5 6 7 8
d[1]=1 0 1 2 3 4 5 6 7 8
d[2]=4 0 1 2 3 1 2 3 4 2
d[3]=6 0 1 2 3 1 2 1 2 2

The table gives the solution to our problem for all the instances
involving a payment of 8 units or less
Analysis
Time Complexity
We have to compute n(N+1) entries
Each entry takes constant time to compute
Running time – O(nN)
Optimal Substructure Property
• Why does the solution work?
Optimal Substructure Property/ Principle of
Optimality
• The optimal solution to the original problem
incorporates optimal solutions to the subproblems.
This is a hallmark of problems amenable to dynamic
programming.
• Not all problems have this property.

You might also like