Dynamic Programming

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Overview
● Powerful design technique
● Invented by Richard Bellman
●
Intended for optimization problems
● e.g 0-1 knapsack
●
Can be thought of as an
● Exhaustive search done intelligently

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 The idea
● Divide the problem into subproblems
● Can be overlapping
●
Solve subproblems
●
Reuse the solutions to the subproblems
● Store intermediate results
● Solve original problem

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Nth Fibonacci Number
●
Find nth Fibonacci Number
●
Usual Recursive Solution
● recfib(n)
{
if ( n==0 )
return 0;
if ( n<=2 )
return 1;
return (recfib(n-1) + recfib(n-2) );
}
● T(n) = T(n-1)+T(n-2) + 2// does unnecessary extra work
● Exponential cost
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Nth Fibonacci Number
●
Find nth Fibonacci Number
●
Usual Recursive Solution
● recfib(n)
{
if ( n==0 )
return 0;
if ( n<=2 )
return 1;
return (recfib(n-1) + recfib(n-2) );
}
● T(n) = T(n-1)+T(n-2) + 2// does unnecessary extra work
● Exponential cost
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Flaw in Recursion
● Same subproblem is being recomputed
repeatedly
● Can be avoided by storing results of
subproblems and reusing it

Src: http://www.ics.uci.edu/~eppstein/161/960109.html
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Memoized Solution
●
Memoization: Store intermediate results
so as to reuse
●
Memoized Algorithm
memo = {};
mfib(n):
if n in memo: return memo[n]
if n<=2: f=1
else: f=mfib(n-1)+mfib(n-2)
memo[n]=f
return f

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Analysis
● The recursion call invoked atmost n
times
●
Non recursive work per call Q(1)
●
Cost Q(n)

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Bottom-up Approach
● Use an iterative method
dynamic_fib(n){
int f[n+1];
f[1]=f[2]=1;
for (int i=3; i<=n;i++)
f[i]=f[i-1]+f[i-2];
return f[n];
}
Cost is O(n) (both time and space)

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Requirements
● Simple Subproblem : Must be able to divide
problems into simple subproblems each similar to
the original
● Usually there are polynomial number of
subproblems
● Subproblem Optimality: optimal solution to global
problem must be composition of optimal
subproblem solutions using simple combining
operation
● Subproblem Overlap: Optimal solutions to unrelated
subproblems can contain subproblems in common

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 5 Easy Steps
● Define subproblems
● Guess part of the solution
●
Relate subproblems
●
Recurse and Memoize or build DP table
●
Solve final problem

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
Matrix Chain Multiplication
● Given n two dimensional matrices whose
product we wish to compute
● A = A0.A1.A2....An-1
● Ai is di X di+1 matrix, for i = 0, 1, 2 ..n-1
● Use standard matrix multiplication algorithm
● Matrix multiplication associative
● B.(C.D) = (B.C).D
● We can paranthesize A anyway we wish

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
Impact of Parenthesization
●
Parenthesization impacts the number of scalar
multiplications done
●
E.g B is a 2 x 10 matrix, C is a 10 x 50 matrix, D
a 50 x 20 matrix
●
B.(C.D) requires 2.10.20 + 10.50.20 = 10400
●
(B.C).D requires 2.10.50 + 2.50.20 = 3000
●
Paranthezation matters!!!!
●
What is the optimal paranthization that
minimizes total number of scalar
multiplications?

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Brute Force
● Enumerate all parenthization and
choose best
● Calculate multiplications done by each
●
Complexity
● Number of all parenthizations is set of all
binary trees that have n external nodes
● Exponential in n
● nth Catalan Number, =

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 DP Strategy
● Identify Subproblem
● Compute best parenthization for some sub-
expression
– Ai.Ai+1......Aj
● Ni,j, the minimum number of multiplications
needed to compute subexpression
● Original problem characterized as finding
N0,n-1
●
Number of subproblems Q(n2)

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 DP Strategy
● Guess: Outermost or last multiplication
to be performed
● (Ai...Ak).(Ak+1....Aj)
● Relate subproblems
● Try all possibilities for k to get best solution
– Number of choices = O(j-i+1) = O(n)
● Irrespective of value of k
– (Ai...Ak) and (Ak+1....Aj) must be solved optimally
● Recursion

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Recursive Algorithm
● RecursiveMCP(int d[], int i, int j){
if (i == j) return 0;
int min = Integer.MAX_VALUE;
for (int k=i; k<j; k++) {
int count = RecursiveMCP(d, i, k) +
RecursiveMCP(d, k+1, j) +
d[i]*d[k+1]*d[j+1];
if (count < min)
min = count;
}
return min;
}
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 DP Strategy
● Recursion results in unnecessary extra
computations
● Memoization can help

Src: http://www.geeksforgeeks.org/dynamic-programming-set-8-matrix-chain-
multiplication/
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 DP Strategy
●
Memoize
● Create a 2d matrix m to store intermediate
results
● N[i,j] represents the cost of multiplying the
maxtrices Ai..Aj
●
Solve final problem
●
Entry N[0,n-1] gives the total cost of multiplying
the matrices
●
Optimal parenthesization can be determined by
backtracking through table

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
Recursion with Memoization
● MemoizedMCP(int d[], int i, int j){
if N[i,j] != null return N[i,j];
if (i == j) return N[i,i] = 0;
N[i,j] = infinity;
for (int k=i; k<j; k++) {
int count = MemoizedMCP(d, i, k) +
MemoizedMCP(d, k+1, j) +
d[i]*d[k+1]*d[j+1];
if (count < N[i,j])
N[i,j] = count; }
return N[i,j]; }
● Complexity O(n3)
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Bottom-up Approach
Initialize Ni,i to 0 for i ← 0 to n-1
for b ← 1 to n-1
for i ← 0 to n-b-1
Set j ← i+b
Set Ni,j to +infinity
for k ← i to j-1 do
N i,j=min {N i,j ,N i,k +N k+1,j +d i d k+1 d j+1 }

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Example: CLR pg 276
● N=6 and A1:30x35, A2:35x15, A3:15x5, A4:5x10,
A5:10x20, A6:20x25

{ }
N 2,2 +N 3,5 +d 1 d 2 d 5 =0+ 2500+35 . 15 . 20=13,000
N 2,5 =min N 2,3 +N 4,5 +d 1 d 3 d 5 =2625+ 1000+35 . 5x . 20=7125
N 2,4 +N 5,5 +d 1 d 4 d 5 =4375+ 0+35 . 10 . 20=11,375
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
Bottom-up solution outline
● First compute Ni,i = 0 for i = 1 , 2 , . . . . , n
● Then compute Ni,i+1 for i = 1 , 2 , . . . . , n-1 and so on
● The Ni,j computed depends on Ni,k and Nk+1,j already
computed
● Using this layout, the minimum cost Ni,j for
multiplying a subchain AiAi+1. . . Aj of matrices
● Use intersection of lines running northeast from Ai and
northwest from Aj
● Each entry si,j records a value of k such that an
optimal parenthesization of AiAi+1. . . Aj splits the
product between Ak and Ak+1

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Generalization
●
Given a linear sequence of objects, an associative
binary operation on those objects, and a way to
compute the cost of performing that operation on
any two given objects (as well as all partial
results), compute the minimum cost way to group
the objects to apply the operation over the
sequence
●
Subproblems in Strings
●
Prefix A[:i]
●
Postfix A[i:]
●
Substring A[i:j]

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Problem
● Find an optimal parenthesization of a
matrix-chain product whose sequence of
dimensions is
〈 5, 10, 3, 12, 5, 50 ,6 〉

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
Longest Common Subsequence
●
Given a string X of size n, a subsequence of X is
any string
●
sequence of characters that are not necessarily
contiguous but taken in order from X
– e.g AAAG is subsequence of CGATAATTGAGA
● of form X[i1]X[i2]...X[ik], ij<ij+1 for j=1,...,k
● Longest Common Subsequence (LCS) problem
●
Given two character strings, X of size n and Y of size
m, over some alphabet find
●
A longest sequence S that is a subsequence of both X
and Y

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Brute Force
●
Enumerate all possible common
subsequences of X
● Take the longest one that is also a subsequence
of Y
● Potentially 2n different subsequences of X, each
of which requires O(m) time to determine if it is
a subsequence of Y
●
Exponential algorithm
●
O(2nm)
● Use dynamic programming method

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
Dynamic Programming Strategy
● Identifying subproblem
●
Computing the length of longest common
subsequence of prefix of X and prefix of Y
– i.e, X[0,...i] and Y[0,...j] as L[i,j]
– Q(n2) subproblems
● Idea
●
Either X[i] or Y[j] or neither are part of LCS
● Guess
● Choose X[i] or Y[j] or neither as part of the LCS

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 DP Strategy
● Relate Subproblems
● Case 1: X[i] = Y[j] = c
– LCS of X[0,...i] and Y[0,...j] ends with c
– Let some LCS be X[i1]X[i2]...X[ik] =
Y[j1]Y[j2]...Y[jk]
● If X[ik] or Y[jk] =c, we get same sequence by setting
ik=i, jk =j
● If X[ik] ≠ c , we can get longer LCS by adding c to the
end
– Hence L[i,j] = L[i-1,j-1]+1 if X[i] = Y[j]

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 DP Strategy
● Relate Subproblems
● Case 2: X[i] ≠ Y[j]
– Cannot have common subsequence that
includes both X[i] and Y[j]
● LCS can end with X[i] or Y[j] or neither but not both
– Hence L[i,j] = max{L[i-1,j],L[i,j-1]} if X[i] ≠ Y[j]

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Recursive LCS
● LCS(X,Y,i,j)
if X[i] =='\0' or Y[j] =='\0': return 0;
else if X[i] == Y[j] return 1 + lcs(X, Y, i-1, j-
1);
else return max(lcs(X, Y, i, j-1), lcs(X, Y, i-
1, j));
lcs("ABET", "ACTS")
/ \
lcs("ABE" "ACTS") lcs("ABET" "ACT")
/ \ / \
lcs("AB","ACTS") lcs("ABE","ACT") lcs("ABE","ACT") lcs("ABET","AC")

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
Recursion with Memoization
● Memoize
● Store cost of longest subsequence between X[0:i]
and Y[0:j] to L[i,j]
● LCS(X,Y, i, j){
if (L[i,j] < 0) {
if (X[i] == '\0' || Y[j] == '\0') L[i,j] = 0;
else if (A[i] == B[j]) L[i,j] = 1+ LCS(i-1, j-1);
else L[i,j] = max(LCS(X,Y,i-1, j), LCS(X,Y,i, j-
1));}
return L[i,j];
}
https://www.ics.uci.edu/~eppstein/161/960229.htm
l
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Analysis
● Call to subproblem – constant
● Called only when an entry to the table is
made
● O(m+1)(n+1) entries
●
Cost atmost 2(m+1)(n+1)+1 ie O(mn)

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Iterative LCS
●
Build Table bottom-up
●
IterativeLCS(X,Y,m,n)
for i ← -1 to n-1 do L[i,-1] ← 0
for j ← 0 to m-1 do L[-1,j] ← 0
for i ← 0 to n-1 do
for j ← 0 to m-1 do
if X[i]=Y[j] then
L[i,j] ← L[i-1,j-1]+1
Else
L[i,j] ← max{L[i-1,j],L[i,j-1]}

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Example

https://s3.amazo
naws.com/hr-
challenge-
images/8677/14
33916387-
bae1920043-
LCS.png

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Exercise
● Show the longest common subsequence
table L for the following strings
X = MULTIDIMENSIONAL
Y = MANGROVEFOREST

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
The 0-1 Knapsack Problem

Determine which items to take so that it fits the sack, and total
utility is maximized (an item must be taken completely or dropped)

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
The optimization problem
●
Given set S of n items, where each item i has
● Benefit bi >= 0 and Weight wi
●
We can take an item completely or drop it
●
We have knapsack that can carry weight
atmost W
●
Goal – Maximize total benefit s.t

●
Objective function to maximize

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 The DP Solution
● Subproblem Definition:
● Let Sk be a subset of items from the original
set S
● Define B[k,w] as maximum total benefit of
subset Sk, whose weight is w, considering
item k
●
Guess:
● Should item k be included in the solution?

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 The DP Solution
● Relating sub-problems
● Best subset of Sk that has total weight w is
either best subset Sk-1 with weight w, or
best subset Sk-1 with weight w-wk plus item
k
● Item k can be part of the knapsack or not

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
0-1 Knapsack: Recursive
●
int RecKnapsack(int k, int w){
if(k == 0 || w == 0)
return 0;
if(w[k] > w)
return RecKnapsack(k - 1, w);
else
return max(RecKnapsack(k - 1, w), Rec
Knapsack(k - 1, w - w[k]) + b[k]);
}

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Need for Memoization

KP(4,10)

KP(3,10) KP(3,5)

KP(2,10) KP(2,6)

KP(1,10) KP(1,8)

KP(0,10) KP(0,8) KP(0,8) KP(1,6)

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Memoized
●
int MemKnapsack(int k, int w){
if(B[k][w] != -1)
return B[k][w];
if(w[k] > w)
B[k][w] = MemKnapsack(k - 1, w);
else
B[k][w] = max(MemKnapsack(k-1, w),
MemKnapsack(k - 1, w - w[k]) + B[k]);
}
●
Compexity O(nW)

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Bottom-up Approach
● for w = 0 to W
B[0,w] = 0
for i = 1 to n
B[i,0] = 0
for i = 1 to n
for w = 0 to W
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
Finding the Actual Knapsack
● B[n,W] is the maximal value of items
that can be placed in the Knapsack.
● Let i=n and k=W
if B[i,k] != B[i-1,k] then
mark the ith item as in the knapsack
i = i-1, k = k-wi
else
i = i-1 // Assume the ith item is not in the
knapsack

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Coin Change Problem
●
You are given some amount N and a set of
denominations C={c1,c2,..,cd}, and each
denominations has ni coins.
●
The goal is to make change for the
amount n using the given denominations.
● Minimize the total number of coins returned
for a particular quantity of change.

d
N= ∑ x k c k
k=1
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
Dynamic Programming Solution
● Subproblem: C[p] is the minimum
number of coins of denominations c1, c2,
. . . , cd needed for some value p
● Guess: The solution must start with
some coin ci
● remaining coins in the optimal solution
must themselves be the optimal solution to
making change for p−ci
● C[p] = 1+C[p−ci];

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham
Dynamic Programming Solution
●
Relate Subproblems
● Find the denomination for which C[p] = 1+C[p−ci] is
minimum
●
Recursion

C [ p ]=
{ 0 ifp=0
mini:c ≼ p 1+c [ p−c i ] ifp>0
i
}
●
Find minimum coins for n=1,n=2, etc till final solution
reached
●
e.g consider denominations 1,2,5, and choose the
denominations in order
●
C[1] =1; C[2] = min(1+C[1], 1); C[5]= min(1+C[4], 1+C[3], 1)
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Recursion Tree

Src: http://ice-
web.cc.gatech.edu/ce21/1/static/audio/static/pythonds/_images/callTre
e.png
CSE211: Design and Amrita School of Engineering
Analysis of Algorithms Amrita Vishwa Vidyapeetham
 Bottom-up DP Solution
Change(C, d, N)
C[0] = 0
for p = 1 to n
min = 1
for i = 1 to d
if c[i] ⩽ p then
if 1 + C[p − c[i]] < min then
min = 1 + C[p − c[i]]
coin = i
C[p] = min
S[p] = coin
return C and S

CSE211: Design and Amrita School of Engineering

Analysis of Algorithms Amrita Vishwa Vidyapeetham