Dynamic Programming

Zahra Farahai
[email protected]
Dynamic Programming
• Divide and Conquered  top-down
• Dynamic Programming  bottom-up

• In dynamic programming we solve each problem just for once and

save it, to use in the future even for more than once
First approach

Second approach
Fibonacci Numbers
• How to implement?
• Implementing directly from the definition:
Fibonacci Numbers
• Computing b(5):
Fibonacci Numbers
How to improve?
Start from zero and keep in memory the last two numbers of the sequence
Fibonacci Numbers
• Concepts to recall:
• Dividing a problem in subproblems of the same type
• Calculating the same subproblem just once
Number Pyramid
"Classic" problem from the 1994 International Olympiad in informatics
Compute the path, starting on the top of the pyramid and ending on the
base, with the biggest sum. In each step we can go diagonally down and
left or down and right.
Number Pyramid
Two possible paths:
Number Pyramid
• How to solve the problem?
• Idea: Exhaustive search (aka "Brute Force")
• I Evaluate all the paths and choose the best one.
• How much time does this take? How many paths exist?
• Analysing the temporal complexity:
• In each line we can take one of two decisions: left or right
• Let n be the height of the pyramid. A path corresponds to... n - 1 decisions!
• There are diferent paths
• A program to compute all possible paths has therefore complexity :
exponential!
Number Pyramid
• When we are at the top we have two possible choices (left or right):

In each case, we need to have in account all possible paths of the respective
subpyramid.
Number Pyramid
• But what do we really need to know about these subpyramids?
• The only thing that matters is the best internal path, which is a smaller
instance of the same problem!
• For the example, the solution is 7 plus the maximum between the
value of the best paths in each subpyramid
Number Pyramid
• This problem can then be solved recursively
• Let P[i][j] be the j-th number of the i-th line
• Let Max(I, j) be the best we can do from position (i,j)
Number Pyramid

• To solve the problem we just need to call... Max(1,1)

 Number Pyramid
We still have exponential growth!

We are evaluating the same problem several times...

Number Pyramid
• We need to reuse what we have already computed
• Compute only once each subproblem.
• Idea: create a table with the value we got for each subproblem
• Matrix M[i][j]
• Is there an order to fill the table so that when we need a value we
have already computed it?
 Number Pyramid
• We can start from the end! (pyramid base)
Number Pyramid
• Having in mind the way we fill the table, we can even reuse P[i][j]:

With this the solution is in... P[1][1]

Now the time needed to solve the problem only grows polynomially and we
have an admissible solution for the problem
Number Pyramid
• What if we need to know what are the numbers in the best path?
• We can use the computed table!
Number Pyramid
To solve the number pyramid number we used...
Dynamic Programming

Algorithmic Technique: general method for solving problem that have

some common characteristics
Optimization Problem: find the "best" solution among all possible
solutions, according to a certain criteria (goal function). Normally it
means finding a minimum or a maximum.
Dynamic Programming
• What are then the characteristics that a problem must
present so that it can be solved using DP?
Optimal substructure
Overlapping subproblems
 Dynamic Programming - Characteristics

If a problem presents this characteristic, we say that it respects the optimality

principle.
Dynamic Programming - Methodology
• What steps should we then follow to solve a problem with DP?
 Dynamic Programming - Methodology

• Really understand the problem

• Verify if an algorithm that verifies all solutions (brute force) is
not enough
• Try to generalize the problem (it takes practice to understand how
to correctly generalize)
• Try to divide the problem in subproblems of the same type
• Verify if the problem obeys the optimality principle
• Verify if there are overlapping subproblems
 Dynamic Programming - Methodology

• Recursively dene the optimal solution value, exactly and with

rigour, from the solutions of subproblems of the same type
• Imagine that the values of optimal solutions are already available
when we need them
• No need to code. You can just mathematically define the
recursion
 Dynamic Programming - Methodology

• Find the order in which the subproblems are needed, from the smaller
subproblem until we reach the global problem and implement, using a
table
• Usually this order is the inverse to the normal order of the recursive
function that solves the problem
Dynamic Programming - Methodology

• There is a technique, known as "memoization", that allows us to

solve the problem by the normal order.
• Just use the recursive function directly obtained from the
definition of the solution and keep a table with the results
already computed.
• When we need to access a value for the first time we need to
compute it, and from then on we just need to see the already
computed result.
 Dynamic Programming - Methodology

It may (or may not) be needed, given what the problem asks for
two alternatives:
Directly from the subproblems table
New table that stores the decisions in each step
If we do not need to know the solution in itself, we can
eventually save some space
Longest Increasing Subsequence (LIS)
• Given a number sequence:
7, 6, 10, 3, 4, 1, 8, 9, 5, 2
• Compute the longest increasing subsequence (not necessarily
contiguous)
 Longest Increasing Subsequence (LIS)

• Let n be the size of the sequence and num[i] the i-th number
• "Brute force", how many options? Exponential!
• Generalize and solve with subproblems of the same type:

Let best(i) be the size of the best subsequence starting from the i-th position
Base case: the best subsequence from the last position has size... 1!
 General case: for a given i, we can continue to all numbers from i + 1 to n, as long as
they are... bigger or equal

For those numbers, we only need to know the best starting from them! (optimality principle)
 The best, starting from a position, is necessary for computing all the positions of lower index!
(overlapping subproblems)
Longest Increasing Subsequence (LIS)

 n - sequence size
 num[i] - number in position i
 best(i) - size of best sequence starting in position i
Longest Increasing Subsequence (LIS)

Let best[] the table to save the values of best()

 Longest Increasing Subsequence (LIS)

• We will exemplify with an auxiliary table that stores the decisions

• Let next[i] be the next position in order to obtain the best solution from position i
('X' if it is the last position of the solution).
Longest Increasing Subsequence (LIS)

We can change the second loop and transform it into binary search
Longest Increasing Subsequence (LIS)
• For each number in nums, we perform the following steps:
If the number is greater than the last element of the last bucket (i.e., the largest
element in the current subsequence), we append the number to the end of the
list. This indicates that we have found a longer subsequence.
Otherwise, we perform a binary search on the list of buckets to find the
smallest element that is greater than or equal to the current number. This step
helps us maintain the property of increasing elements in the buckets.
Once we find the position to update, we replace that element with the current
number. This keeps the buckets sorted and ensures that we have the potential
for a longer subsequence in the future.
0-1 Knapsack
0-1 Knapsack
• A greedy solution will not work on this integer case

Choosing max ratio first will result in {1,2,7} with a value of 35

Choosing max value first will also result in {1,2,7} with a value of 35
Choosing min weight first will result in {1,2,5} with a value of 25
...
None of these is optimal: we could get a value of 40 with {3, 4}
0-1 Knapsack

• "Brute force", how many options? Exponential (2n)!

• Let's first consider the unbounded case
(no limit on number of items of each type)
• In this unbounded case we could generalize in the following way:
• Let best(i) be the best value we can get for capacity i
• Base case: best(0) = 0 (obviously)
• General case: for a given i, we can simple see all possible items and get the
best if we insert that item:

• But how can we limit the amount of items of each type?

0-1 Knapsack

• Let's add more information to our DP state

• Let best(i, j) be the best value we can get for capacity j using only
the first i materials
• For computing the values of a given best(i , j) we can now simply
use the values of previously computed best(i – 1, k)
• Let's put all the pieces into place...
0-1 Knapsack

 n - number of materials
 wi - weight of material i
 vi - value of material of material i
 C - capacity of backpack
 best(i,j) - maximum possible value for capacity j using the first i materials

The desired result is: best(n,C)

0-1 Knapsack
• Let's see a table of results to better understand the DP formulation:
0-1 Knapsack

Let best[][] be the matrix that stores the values of the DP states
 0-1 Knapsack

• If needed we could store for each position how we obtained its value:
• We either used current item or we did not use it
 0-1 Knapsack

• Yes: each row of the table only need the values of another row
• We could use O(C) memory by simply just storing previous row
• In fact, if we carefully consider the order in which we compute the
values, we can simply just store one row:
 Let best[] be the array that stores the current row
0-1 Knapsack variants
• There are many possible variants for knapsack problem.
• Subset sum
• Given a set S of integers and value K, verify if we can obtain a subset of S that sums
to K
• Ex: S = {1, 3, 5, 10}
• K = 8 has answer "yes" because 3 + 5 = 8
• K = 7 has answer "no" because no subset of S has sum 7
• It's the "same" as knapsack if we disregard values and just consider if a certain
weight is achievable:
si = i-th element of set S
sum(i , j) = is sum j achievable using the first i items? (T/F)
sum(0, 0) = T
sum(0, j) = F for
sum(i , j) = sum(i – 1, j) OR AND ) for ,
Edit Distance
Let's look at another problem, this time with strings:
Edit Distance

Let ed(a,b) be the edit distance between a and b

Let "" be the empty word
Are there any simple cases?

 Clearly ed("","") is zero

 ed("",b), for any word b? It is the size of word b (we need to make insertions)
 ed(a,""), for any word a? It is the size of word a (we need to make removals)

And in the other cases? We must try dividing the problem in subproblems,
where we can decide based on the solution of the subproblems.
Edit Distance
• None of the words is empty
• How can equalize the end of both words?
• Let be the last letter of a and a’ the remaining letters of a
• Let be the last letter of b and b’ the remaining letters of b
• If , then all that is left is to find the edit distance between a’ and b’ (a smaller
instance of the same problem!)
• Otherwise, we have three possible movements:
• Substitute with . We spend 1 transformation and now we need the edit distance
between a’ and b’.
• Remove. We spend 1 transformation and now we need the edit distance between a’
and b.
• Insert at the end of a. We spend 1 transformation and now we need the edit distance
between a and b’.
Edit Distance
• [a] and [b] - size (length) of words a and b
• a[i] and b[i] - letter on position i of words a and b
• ed(i,j) - edit distance between the first i letters of a and the first j
letters of b
Edit Distance
Edit Distance
• Let's see the table for the edit distance between "gotas" and "afoga":
Edit Distance
• If we needed to reconstruct the solution
Edit Distance Variants
• There are many possible variants for the edit distance problem. Here is
perhaps the most common (and classical one):
• Longest Common Subsequence Problem (LCS)
• Given two strings, find the length of the longest subsequence (not necessarily
contiguous) common to both strings
• Ex: LCS("ABAZDC", "BACBAD") = 4 [corresponding to "ABAD"]
• It's the "same" as edit distance if no swapping is allowed (only additions and
deletions). Suppose that the strings are a and b:
Matrix Chain Multiplication
• Given the dimension of a sequence of matrices in an array arr[], where
the dimension of the ith matrix is (arr[i-1] * arr[i]), the task is to find
the most efficient way to multiply these matrices together such that the
total number of element multiplications is minimum.
 Matrix Chain Multiplication
• input: arr[] = {40, 20, 30, 10, 30}
Output: 26000
Explanation: There are 4 matrices of dimensions 40×20, 20×30, 30×10, 10×30.
Let the input 4 matrices be A, B, C and D.
The minimum number of multiplications are obtained by putting parenthesis in following way (A(BC))D.
The minimum is 20*30*10 + 40*20*10 + 40*10*30

• Input: arr[] = {1, 2, 3, 4, 3}

Output: 30
Explanation: There are 4 matrices of dimensions 1×2, 2×3, 3×4, 4×3.
Let the input 4 matrices be A, B, C and D.
The minimum number of multiplications are obtained by putting parenthesis in following way ((AB)C)D.
The minimum number is 1*2*3 + 1*3*4 + 1*4*3 = 30
Matrix Chain Multiplication
• Input: arr[] = {10, 20, 30}
Output: 6000
Explanation: There are only two matrices of dimensions 10×20 and
20×30.
So there is only one way to multiply the matrices, cost of which is
10*20*30
 Matrix Chain Multiplication using Recursion:
• Two matrices of size m*n and n*p when multiplied, they generate a matrix of size m*p and
the number of multiplications performed are m*n*p.

• Now, for a given chain of N matrices, the first partition can be done in N-1 ways. For
example, sequence of matrices A, B, C and D can be grouped as (A)(BCD), (AB)(CD) or
(ABC)(D) in these 3 ways.

• So a range [i, j] can be broken into two groups like {[i, i+1], [i+1, j]}, {[i, i+2], [i+2, j]}, . . . ,
{[i, j-1], [j-1, j]}.

• Each of the groups can be further partitioned into smaller groups and we can find the total
required multiplications by solving for each of the groups.

• The minimum number of multiplications among all the first partitions is the required answer.
Matrix Chain Multiplication using Recursion:
• Follow the steps mentioned below to implement the approach:
• Create a recursive function that takes i and j as parameters that
determines the range of a group.
• Iterate from k = i to j to partition the given range into two groups.
• Call the recursive function for these groups.
• Return the minimum value among all the partitions as the required minimum
number of multiplications to multiply all the matrices of this group.
• The minimum value returned for the range 0 to N-1 is the required
answer.
The time complexity of the solution is exponential
Recursive Programming Solution for Matrix Chain
Dynamic Programming Solution for Matrix Chain
• 1) Optimal Substructure: In the above case, we are breaking the
bigger groups into smaller subgroups and solving them to finally find
the minimum number of multiplications. Therefore, it can be said that
the problem has optimal substructure property.
• 2) Overlapping Subproblems: We can see in the recursion tree that
the same subproblems are called again and again and this problem has
the Overlapping Subproblems property.
Dynamic Programming Solution for Matrix Chain
Follow the below steps to solve the problem:
• Build a matrix dp[][] of size N*N for memorization purposes.
• Use the same recursive call as done in the above approach:
• When we find a range (i, j) for which the value is already calculated, return
the minimum value for that range (i.e., dp[i][j]).
• Otherwise, perform the recursive calls as mentioned earlier.
• The value stored at dp[0][N-1] is the required answer.

Time Complexity: O(N3 )

Auxiliary Space: O(N2) ignoring recursion stack space
Dynamic Programming Solution for Matrix Chain
Dynamic Programming Solution for Matrix Chain
• Example: We are given the sequence {4, 10, 3, 12, 20, and 7}. The matrices
have size . We need to compute M[i,j], 0 ≤ i, j≤ 5. We know M [i, i] = 0 for all
i.

Let us proceed with working away from the diagonal. We compute the optimal
solution for the product of 2 matrices.
Calculation of Product of 2 Matrices:
Calculation of Product of 3 Matrices:
Calculation of Product of 3 Matrices:
Calculation of Product of 4 Matrices:

There are three cases by which we can solve this multiplication:

Calculation of Product of 4 Matrices:

There are three cases by which we can solve this multiplication:

Calculation of Product of 4 Matrices:
Calculation of Product of 5 Matrices:

There are five cases by which we can solve this multiplication:

Calculation of Product of 5 Matrices: