Design and Analysis of

Algorithms

Dynamic Programming
 Dynamic Programming
• An algorithm design technique (like divide and
conquer)

• Divide and conquer

– Partition the problem into independent subproblems

– Solve the subproblems recursively

– Combine the solutions to solve the original problem

2
 Dynamic Programming
• Applicable when subproblems are not independent
– Subproblems share subsubproblems
– A divide and conquer approach would repeatedly solve the
common subproblems
– Dynamic programming solves every subproblem just once and
stores the answer in a table

• Used for optimization problems

– A set of choices must be made to get an optimal solution
– Find a solution with the optimal value (minimum or maximum)

3
 Dynamic Programming
Also known as dynamic optimization is a method
for solving a complex problem by breaking it down
into a collection of simpler subproblems,
solving each of those subproblems just once,
and storing their solutions. The next time the
same subproblem occurs, instead of recomputing
its solution, one simply looks up the previously
computed solution.

4
 Dynamic Programming Algorithm
1. Characterize the structure of an optimal
solution
2. Recursively define the value of an optimal
solution
3. Compute the value of an optimal solution in a
bottom-up fashion
4. Construct an optimal solution from computed
information

5
 Assembly Line Scheduling
• Automobile factory with two assembly lines
– Each line has n stations: S1,1, . . . , S1,n and S2,1, . . . , S2,n
– Corresponding stations S1, j and S2, j perform the same function
but can take different amounts of time a1, j and a2, j
– Entry times e1 and e2 and exit times x1 and x2

6
 Assembly Line
• After going through a station, can either:
– stay on same line at no cost, or
– transfer to other line: cost after Si,j is ti,j , j = 1, . . . , n - 1

7
 Assembly Line Scheduling
• Problem:
what stations should be chosen from line 1 and which
from line 2 in order to minimize the total time through the
factory for one car?

8
 One Solution
• Brute force
– Enumerate all possibilities of selecting stations
– Compute how long it takes in each case and choose
the best one
• Problem: 1 2 3 4 n

1 0 0 1 1

0 if choosing line 2 1 if choosing line 1

at step j (= 3) at step j (= n)

– There are 2n possible ways to choose stations

– Infeasible when n is large
9
1. Structure of the Optimal Solution
• Let’s consider the fastest way possible to get
from the starting point through station S1,j
– If j = 1: determine how long it takes to get through S1,1
– If j ≥ 2, have two choices of how to get to S1, j:
• Through S1, j - 1, then directly to S1, j
• Through S2, j - 1, then transfer over to S1, j

10
1. Structure of the Optimal Solution
• Suppose that the fastest way through S1, j is
through S1, j – 1
– We must have taken a fastest way from entry through S1, j – 1
– If there were a faster way through S1, j - 1, we would use it instead
• Similarly for S2, j – 1

11
 Optimal Substructure
• Generalization: an optimal solution to the
problem find the fastest way through S1, j contains
within it an optimal solution to subproblems: find
the fastest way through S1, j - 1 or S2, j - 1.

• This is referred to as the optimal substructure

property

• We use this property to construct an optimal

solution to a problem from optimal solutions to
subproblems
12
 2. A Recursive Solution
• Define the value of an optimal solution in terms of the optimal
solution to subproblems
• Assembly line subproblems
– Finding the fastest way through station j on both lines, j = 1, 2, …, n

13
 2. A Recursive Solution (cont.)
• fi[j] = the fastest time to get from the starting point
through station Si,j
• j = 1 (getting through station 1)
f1[1] = e1 + a1,1
f2[1] = e2 + a2,1

14
 2. A Recursive Solution (cont.)
• Compute fi[j] for j = 2, 3, …,n, and i = 1, 2
• Fastest way through S1, j is either:
– fastest way through S1, j - 1 then directly through S1, j, or
f1[j] = f1[j - 1] + a1,j
– fastest way through S2, j - 1, transfer from line 2 to line 1, then
through S1, j
f1[j] = f2[j -1] + t2,j-1 + a1,j S1,j-1 S1,j

a1,j-1 a1,j

t2,j-1
f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)
a2,j-1
S2,j-1
15
 2. A Recursive Solution (cont.)
• f* = the fastest time to get through the entire factory
f* = min (f1[n] + x1, f2[n] + x2)

16
 2. A Recursive Solution (cont.)

e1 + a1,1 if j = 1
f1[j] =
min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j) if j ≥ 2

e2 + a2,1 if j = 1
f2[j] =
min(f2[j - 1] + a2,j ,f1[j -1] + t1,j-1 + a2,j) if j ≥ 2

17
3. Computing the Optimal Solution
f* = min (f1[n] + x1, f2[n] + x2)
f1[j] = min(f2[j - 1] + a2,j ,f1[j -1] + t1,j-1 + a2,j)

1 2 3 4 5
f1[j] f1(1) f1(2) f1(3) f1(4) f1(5)

f2[j] f2(1) f2(2) f2(3) f2(4) f2(5)

4 times 2 times
• Solving top-down would result in exponential
running time

18
 3. Computing the Optimal Solution
• For j ≥ 2, each value fi[j] depends only on the values of
f1[j – 1] and f2[j - 1]
• Compute the values of fi[j]
– in increasing order of j
increasing j

1 2 3 4 5
f1[j]
f2[j]

• Bottom-up approach
– First find optimal solutions to subproblems
– Find an optimal solution to the problem from the subproblems
19
 Additional Information
• To construct the optimal solution we need the sequence
of what line has been used at each station:
– li[j] – the line number (1, 2) whose station (j - 1) has been used
to get in fastest time through Si,j, j = 2, 3, …, n
– l* - the line whose station n is used to get in the fastest way
through the entire factory
increasing j
2 3 4 5

l1[j]
l2[j]
20
Example

e1 + a1,1, if j = 1
f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j) if j ≥ 2
1 2 3 4 5
f1[j] 9 18[1] 20[2] 24[1] 32[1]
f* = 35[1]
f2[j] 12 16[1] 22[2] 25[1] 30[2]
21
 4. Construct an Optimal Solution
Alg.: PRINT-STATIONS(l, n)
line 1, station 5
• i ← l*
line 1, station 4
• print “line ” i “, station ” n
line 1, station 3
• for j ← n downto 2
line 2, station 2
• do i ←li[j]
• print “line ” i “, station ” j - 1 line 1, station 1

1 2 3 4 5
f1[j]/l1[j] 9 18[1] 20[2] 24[1] 32[1]
l* = 1
f2[j]/l2[j] 12 16[1] 22[2] 25[1] 30[2]

24
 Dynamic Programming Algorithm
1. Characterize the structure of an optimal solution
– Fastest time through a station depends on the fastest time on
previous stations

2. Recursively define the value of an optimal solution

– f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)

3. Compute the value of an optimal solution in a bottom-up

fashion
– Fill in the fastest time table in increasing order of j (station #)

4. Construct an optimal solution from computed information

– Use an additional table to help reconstruct the optimal solution

25
 Matrix-Chain Multiplication
Problem: given a sequence A1, A2, …, An,
compute the product:
A1  A2  An

• Matrix compatibility:
C=AB
colA = rowB
rowC = rowA
colC = colB
A1  A2  Ai  Ai+1  An
coli = rowi+1
26
 Matrix-Chain Multiplication
• In what order should we multiply the matrices?
A1  A2  An
• Parenthesize the product to get the order in which
matrices are multiplied
• E.g.: A1  A2  A3 = ((A1  A2)  A3)
= (A1  (A2  A3))
• Which one of these orderings should we choose?
– The order in which we multiply the matrices has a
significant impact on the cost of evaluating the product
27
 MATRIX-MULTIPLY(A, B)
if columns[A]  rows[B]
then error “incompatible dimensions”
else for i  1 to rows[A]
do for j  1 to columns[B] rows[A]  cols[A]  cols[B]
do C[i, j] = 0 multiplications

for k  1 to columns[A]
k do C[i, j]  C[i, j] + A[i, k] B[k, j]
j cols[B]
j cols[B]

i = i
* k
A B C
rows[A]
rows[A] 28
 Example
A1  A2  A3
• A1: 10 x 100
• A2: 100 x 5
• A3: 5 x 50
1. ((A1  A2)  A3): A1  A2 = 10 x 100 x 5 = 5,000 (10 x 5)
((A1  A2)  A3) = 10 x 5 x 50 = 2,500
Total: 7,500 scalar multiplications
2. (A1  (A2  A3)): A2  A3 = 100 x 5 x 50 = 25,000 (100 x 50)
(A1  (A2  A3)) = 10 x 100 x 50 = 50,000
Total: 75,000 scalar multiplications
one order of magnitude difference!!
29
 Matrix-Chain Multiplication
• Given a chain of matrices A1, A2, …, An, where
for i = 1, 2, …, n matrix Ai has dimensions pi-1x pi,
fully parenthesize the product A1  A2  An in a
way that minimizes the number of scalar
multiplications.
A1  A2  Ai  Ai+1  An
p0 x p1 p1 x p2 pi-1 x pi pi x pi+1 pn-1 x pn

30
 1. The Structure of an Optimal
Parenthesization
• Notation:
Ai…j = Ai Ai+1  Aj, i  j

• For i < j:
Ai…j = Ai Ai+1  Aj
= Ai Ai+1  Ak Ak+1  Aj
= Ai…k Ak+1…j

• Suppose that an optimal parenthesization of Ai…j

splits the product between Ak and Ak+1, where
ik<j
31
 Optimal Substructure
Ai…j = Ai…k Ak+1…j

• The parenthesization of the “prefix” Ai…k must be an

optimal parentesization
• If there were a less costly way to parenthesize Ai…k, we
could substitute that one in the parenthesization of Ai…j
and produce a parenthesization with a lower cost than
the optimum  contradiction!
• An optimal solution to an instance of the matrix-chain
multiplication contains within it optimal solutions to
subproblems

32
 2. A Recursive Solution
• Subproblem:

determine the minimum cost of parenthesizing

Ai…j = Ai Ai+1  Aj for 1  i  j  n

• Let m[i, j] = the minimum number of

multiplications needed to compute Ai…j
– Full problem (A1..n): m[1, n]

33
 2. A Recursive Solution (cont.)
• Define m recursively:

– i = j: Ai…i = Ai  m[i, i] = 0, for i = 1, 2, …, n

– i < j: assume that the optimal parenthesization splits

the product Ai Ai+1  Aj between Ak and Ak+1, i  k < j

Ai…j = Ai…k Ak+1…j

m[i, j] = m[i, k] + m[k+1, j] + pi-1pkpj

Ai…k Ak+1…j Ai…kAk+1…j
34
 2. A Recursive Solution (cont.)
• Consider the subproblem of parenthesizing
Ai…j = Ai Ai+1  Aj for 1  i  j  n
pi-1pkpj
= Ai…k Ak+1…j for i  k < j
m[i, k] m[k+1,j]
• m[i, j] = the minimum number of multiplications
needed to compute the product Ai…j

m[i, j] = m[i, k] + m[k+1, j] + pi-1pkpj

min # of multiplications min # of multiplications # of multiplications

to compute Ai…k to compute Ak+1…j to compute Ai…kAk…j
35
 2. A Recursive Solution (cont.)
m[i, j] = m[i, k] + m[k+1, j] + pi-1pkpj
• We do not know the value of k
– There are j – i possible values for k: k = i, i+1, …, j-1
• Minimizing the cost of parenthesizing the product
Ai Ai+1  Aj becomes:
0 if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj} if i < j
ik<j

36
 Reconstructing the Optimal Solution

• Additional information to maintain:

• s[i, j] = a value of k at which we can split

the product Ai Ai+1  Aj in order

to obtain an optimal parenthesization

37
 3. Computing the Optimal Costs
0 if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj} if i < j
ik<j

• How many subproblems do we have?

– Parenthesize Ai…j for 1  i  j  n
– One problem for each choice of i and j  (n2)
• A recurrent algorithm may encounter each
subproblem many times in different branches of
the recursion  overlapping subproblems
• Compute a solution using a tabular bottom-up
approach
38
3. Computing the Optimal Costs (cont.)
0 if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj} if i < j
ik<j

• How do we fill in the tables m[1..n, 1..n] and s[1..n, 1..n] ?

– Determine which entries of the table are used in computing m[i, j]
– m[i, j] = cost of computing a product of j – i – 1 matrices
– m[i, j] depends only on costs for products of fewer than j – i – 1
matrices
Ai…j = Ai…k Ak+1…j
– Fill in m such that it corresponds to solving problems of increasing
length

39
3. Computing the Optimal Costs (cont.)
0 if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj} if i < j
ik<j

• Length = 0: i = j, i = 1, 2, …, n
• Length = 1: j = i + 1, i = 1, 2, …, n-1
1 2 3 n
n
m[1, n] gives the optimal
solution to the problem

j
Compute rows from bottom to top 3
and from left to right 2
In a similar matrix s we keep the
1
optimal values of k
i 40
Example: min {m[i, k] + m[k+1, j] + pi-1pkpj}
m[2, 2] + m[3, 5] + p1p2p5 k=2

m[2, 5] = min m[2, 3] + m[4, 5] + p1p3p5 k=3

m[2, 4] + m[5, 5] + p1p4p5 k=4

1 2 3 4 5 6
6
5
• Values m[i, j] depend only
4
j on values that have been
3
previously computed
2
1

i
41
 Example
1 2 3
Compute A1  A2  A3 2
3 2
7500 25000 0
• A1: 10 x 100 (p0 x p1)
1
2 0
• A2: 100 x 5 (p1 x p2) 5000

• A3: 5 x 50 (p2 x p3) 1 0

m[i, i] = 0 for i = 1, 2, 3
m[1, 2] = m[1, 1] + m[2, 2] + p0p1p2 (A1A2)
= 0 + 0 + 10 *100* 5 = 5,000
m[2, 3] = m[2, 2] + m[3, 3] + p1p2p3 (A2A3)
= 0 + 0 + 100 * 5 * 50 = 25,000
m[1, 3] = min m[1, 1] + m[2, 3] + p0p1p3 = 75,000 (A1(A2A3))
m[1, 2] + m[3, 3] + p0p2p3 = 7,500 ((A1A2)A3)
42
 l =3
l=2

35*15*5= 10*20*25
2625 =5000

m[3,4]+m[5,5] + 15*10*20
=750 + 0 + 3000 = 3750
m[3,5] = min
m[3,3]+m[4,5] + 15*5*20
=0 + 1000 + 1500 = 2500

43
 4. Construct the Optimal Solution
• Store the optimal choice made at each
subproblem
• s[i, j] = a value of k such that an optimal
parenthesization of Ai..j splits the product
between Ak and Ak+1
• s[1, n] is associated with the entire product A1..n
– The final matrix multiplication will be split at k = s[1, n]
A1..n = A1..s[1, n]  As[1, n]+1..n
– For each subproduct recursively find the
corresponding value of k that results in an optimal
parenthesization

45
4. Construct the Optimal Solution (cont.)
• s[i, j] = value of k such that the optimal
parenthesization of Ai Ai+1  Aj splits the
product between Ak and Ak+1
1 2 3 4 5 6 A1..n = A1..s[1, n]  As[1, n]+1..n
6 3 3 3 5 5 -
• s[1, n] = 3  A1..6 = A1..3 A4..6
5 3 3 3 4 -
• s[1, 3] = 1  A1..3 = A1..1 A2..3
4 3 3 3 -
• s[4, 6] = 5  A4..6 = A4..5 A6..6
3 1 2 -
j
2 1 -
1 -
i
46
 4. Construct the Optimal Solution (cont.)

PRINT-OPT-PARENS(s, i, j)
1 2 3 4 5 6
if i = j 6 3 3 3 5 5 -
then print “A”i 5 3 3 3 4 -
else print “(” 4 3 3 3 -
j
3 1 2 -
PRINT-OPT-PARENS(s, i, s[i, j])
2 1 -
PRINT-OPT-PARENS(s, s[i, j] + 1, j)
1 -
print “)”
i

47
Example: A1  A6 ( ( A1 ( A2 A3 ) ) ( ( A4 A5 ) A6 ) )

PRINT-OPT-PARENS(s, i, j) s[1..6, 1..6] 1 2 3 4 5 6

if i = j 6 3 3 3 5 5 -
then print “A”i 5 3 3 3 4 -
else print “(”
4 3 3 3 -
PRINT-OPT-PARENS(s, i, s[i, j]) j
PRINT-OPT-PARENS(s, s[i, j] + 1, j) 3 1 2 -
print “)” 2 1 -
P-O-P(s, 1, 6) s[1, 6] = 3 1 -
i = 1, j = 6 “(“ P-O-P (s, 1, 3) s[1, 3] = 1 i
i = 1, j = 3 “(“ P-O-P(s, 1, 1)  “A1”
P-O-P(s, 2, 3) s[2, 3] = 2
i = 2, j = 3 “(“ P-O-P (s, 2, 2)  “A2”
P-O-P (s, 3, 3)  “A3”
“)”
“)” … 48
Elements of Dynamic Programming
• Optimal Substructure
– An optimal solution to a problem contains within it an
optimal solution to subproblems
– Optimal solution to the entire problem is built in a
bottom-up manner from optimal solutions to
subproblems
• Overlapping Subproblems
– If a recursive algorithm revisits the same subproblems
over and over  the problem has overlapping
subproblems
49
 Optimal Substructure - Examples
• Assembly line
– Fastest way of going through a station j contains:
the fastest way of going through station j-1 on either
line

• Matrix multiplication
– Optimal parenthesization of Ai  Ai+1  Aj that splits
the product between Ak and Ak+1 contains:
an optimal solution to the problem of parenthesizing
Ai..k and Ak+1..j

50
 Discovering Optimal Substructure
1. Show that a solution to a problem consists of making a
choice that leaves one or more similar problems to be
solved
2. Suppose that for a given problem you are given the
choice that leads to an optimal solution
3. Given this choice determine which subproblems result
4. Show that the solutions to the subproblems used within
the optimal solution must themselves be optimal
• Cut-and-paste approach

51
 Parameters of Optimal Substructure

• How many subproblems are used in an optimal

solution for the original problem
– Assembly line: One subproblem (the line that gives best time)
– Matrix multiplication: Two subproblems (subproducts Ai..k, Ak+1..j)

• How many choices we have in determining

which subproblems to use in an optimal solution
– Assembly line: Two choices (line 1 or line 2)
– Matrix multiplication: j - i choices for k (splitting the product)

52
 Parameters of Optimal Substructure
• Intuitively, the running time of a dynamic
programming algorithm depends on two factors:
– Number of subproblems overall
– How many choices we look at for each subproblem
• Assembly line
– (n) subproblems (n stations) (n) overall
– 2 choices for each subproblem
• Matrix multiplication:
– (n2) subproblems (1  i  j  n)
(n3) overall
– At most n-1 choices
53
 Readings
• Chapter 15

54