Chapter 7: Dynamic Programming

¨ Introduction
¨ Shortest Path Problem
¨ Knapsack Problem
¨ Extended Knapsack Problem
¨ Key Reference: Chapter 12, Taha’s book
 1. Dynamic Programming
¨ Dynamic Programming is a technique that can
be used to solve many optimization problems.
In most applications, dynamic programming
obtains solutions by working backward from
the end of problem toward the beginning, the
breaking up a large, unwieldy problem into a
series of smaller, more tractable problems.
4 Steps in Dynamic Programming
¨ Divide the Problem into stages
¨ Solve the last stage first
¨ Work backwards solving stages
¨ Optimal solution obtained from all solved
stages
 TYPES OF DP PROBLEMS
4

Two types of DP:

¨ Network:

Ex: Shortest path problem

¨ Non – network:

Ex: Knapsack problem.

 2. The shortest path
¨ To find a shortest path in a multi-stage graph
3 2 7

1 4
S A B 5
T

5 6

¨ Apply the greedy method :

the shortest path from S to T :
1+2+5=8
The shortest path in multistage graphs
4
A D
¨ e.g. 1
11 9
18

2 5 13
S B E T
16 2

5
C 2
F

¨ The greedy method can not be applied to this case:

(S, A, D, T) 1+4+18 = 23.
¨ The real shortest path is:
(S, C, F, T) 5+2+2 = 9.
 Shortest path construction:

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

 Shortest path construction: 4th stage
¨ d(J, H) = 3

¨ d(J, I) = 4

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

 Shortest path construction: 3rd stage
d(E,J) =d(E,H )+ d(J, H) = 1+3=4*
d(E,J) = d(E, I) +d(J, I) = 4+4=8

d(F,J )=d(F,H )+ d(J, H) = 6+3=9

d(F,J )= d(F, I) +d(J, I) = 3+4=7*

d(G,J )= d(G,H )+ d(J, H) = 3+3=6*

d(G,J )= d(G, I) +d(J, I) = 3+4=7
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
 Shortest path construction: 2nd stage
d(B,J)=d(B,E )+ d(E,J) = 7+4=11*
d(B,J)= d(B, F) + d(F,J)= 4+7=11*
d(B,J)=d(B, G) + d(G,J)=6+6=12

d(C,J)=d(C,E )+ d(E, J) = 3+4=7*

d(C,J)=d(C,F)+ d(F, J) = 2+7=9
d(C,J)= d(C, G) +d(G, J) = 4+6=10

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

d(D,J)= d(D, E) +d(E, J) = 4+4=8*
d(D,J)= d(D, F) +d(F, J) = 1+7=8*
d(D,J)=d(D,G )+ d(G, J) = 5+6=11
 Shortest path construction: 1st stage

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

d(A,J)=d(A,B )+ d(B,J) = 2+11=13 A-C-E-H-J

d(A,J)= d(A, C) + d(C,J)= 4+7=11*
d(A,J)= d(A, D) + d(D,J)= 3+8=11*
A-D-F-I-J
A-D-E-H-J
 The shortest path

11 4

11 7 7 0

8 6

Route A-C-E-H-J: 4+3+1+3=11

A-D-F-I-J : 11
 3. KNAPSACK PROBLEM
¨ Objective function of Knapsack problem can be:
- Minimization or
- Maximization

¨ Example:
A knapsack can hold some limited items. Given a set of
items, each with a weight and a value, determine the
count of each item to include in a collection so that the
total weight is less than or equal to a given limit and the
total value is as large as possible
 Mathematical Model

¨ Wi = weight of item i = 1, 2, …, n.
¨ Vi = value of item i = 1, 2, …, n.
¨ C = capacity of knapsack
¨ Xi = 1 if item i is selected;
𝑛 0, otherwise
¨ Model: 𝑀𝑎𝑥 𝑍=∑ 𝑉 𝑖 𝑋 𝑖
𝑖=1
Subject to:
={0,1}
 KNAPSACK PROBLEM CLASSIFY
¨ There are many types of Knapsack problem in practice.
¨ Examples:
- Choosing goods to carry on for delivery.
- Some scheduling problems, for example, there are some
tasks that need to be completed in the next 2 weeks, the
2-week duration can be considered as knapsack, the
tasks must be scheduled so that profit can be maximized
or the cost can be minimized; the constraints are the
working days, or hours in this 2 weeks.
 Formula for Sub-problems

¨ f(g) = the highest total value that can be achieved from a

knapsack with capacity; g is remaining capacity
¨ Forward recursive formula for sub problems: for item k
f
f(0,g)=0; f(k,0)=0

No add k
Add k
 Example 1: Knapsack Problem
Given some items, pack the knapsack to get the
maximum total value. Each item has some weight and
some value. Total weight that we can carry is no more
than some fixed number C = 8. So we must consider
weights of items as well as their values.

Item # Weight Value

1 1 15
2 5 10
3 3 9
4 4 5
 General Steps f(s) g

¨ Stage: i is number of item, i =1,2,.., n s

¨ Alternatives at stage i: xi = 0, 1. selected item
¨ State si : total weight assigned/consumed to stages i, i+1,…,n
¨ fi(si) = the maximum value at stages i, i +1, .., n given si
¨ Backward recursive formula: start from item i =n.
The weight used at stage i is: ()=.
Therefore:
fi
fn+1(sn+1)=0
Note: s ~ g in previous method
 Example 1 Solution
¨ Stage i = 4: s4 = w4x4 +s5=4x4+0 ≤ C = 8
f4(s4) =max[v4x4+f5(s5)]=max[5x4+0]; f5(s5) =0;
s4 5x4+f5(s4-4x4)] Optimal Solution
x4 = 0 x4 = 1 f4(s4) x*4 Item W V

0 0 - 0 0
1 1 15
1 0 - 0 0 2 5 10
2 0 - 0 0 3 3 9
3 0 - 0 0
4 4 5
4 0 5+0 5 1
C=8
5 0 5+0 5 1

6 0 5+0 5 1

7 0 5+0 5 1

0 5+0 5
 Example 1 Solution
¨ Stage i = 3:
f3(s3) =max[v3x3+f4(s4)]=max[9x3+ f4(s3-3x3)];
s3 9x3+ f4(s3-3x3) Optimal Solution
x3=0 x3=1 f3(s3) x*3 Item W V

0 0+0 - 0 0
1 1 15
1 0+0 - 0 0 2 5 10
2 0+0 - 0 0 3 3 9
3 0+0 9+0 9 1
4 4 5
4 0+5 9+0 9 1
C=8
5 0+5 9+0 9 1

6 0+5 9+0 9 1

7 0+5 9+5 14 1

0+5 9+5 14
 Example 1 Solution
¨ Stage i = 2:
f2(s2) =max[v2x2+f3(s3)]=max[10x2+ f3(s2-5x2)];
s2 10x2+ f3(s2-5x2) Optimal Solution
x2 = 0 x2 = 1 f2(s2) x*2 Item W V

0 0+0 - 0 0
1 1 15
1 0+0 - 0 0 2 5 10
2 0+0 - 0 0 3 3 9
3 0+9 - 9 0
4 4 5
4 0+9 - 9 0
C=8
5 0+9 10+0 10 1

6 0+9 10+0 10 1

7 0+14 10+0 14 0

0+14 10+9 19
 Example 1 Solution
¨ Stage i = 1:
f1(s1) =max[v1x1+f2(s2)]=max[15x1+ f2(s1-x1)];
s1 15x1+ f2(s1-x1) Optimal Solution
x1 = 0 x1= 1 f1(s1) x*1 Item W V

0 0+0 - 0 0
1 1 15
1 0+0 15+0 15 1 2 5 10
2 0+0 15+0 15 1 3 3 9
3 0+9 15+0 15 1
4 4 5
4 0+9 15+9 24 1
C=8
5 0+10 15+9 24 1

6 0+10 15+10 25 1

7 0+14 15+10 25 1

0+19 15+14 29
 Example 1 Solution
¨ Optimal Solution: given C =8
¨ At Stage 1, f1(s1) =29, s1=8, x*1 = 1  f2(s2)
= 14 ; s2=s1-x1=8-1=7
¨ At stage 2, f2(s2) = 14 at s2= 7, x*2 = 0 f3(s3)
=14 ;s3=7
¨ At stage 3, f3(s3) =14 at s3=7, x*3 =1
 f4(s4) = 5 ;s4=4
¨ At stage 4, f4(s4) = 5 at s4 =4, x*4 =1.
¨ Optimal solution is (1,0,1,1) Objective
 4. Extended Model

¨ Wi = weight of item i = 1, 2, …, n.
¨ Vi = value of item i = 1, 2, …, n.
¨ C = capacity of knapsack
¨ Xi 0 and integers 𝑛
¨ Model: 𝑀𝑎𝑥 𝑍=∑ 𝑉 𝑖 𝑋 𝑖
𝑖=1
Subject to:
0, and integers
 Solution
¨ Stage: i is number of item, i =1,2,.., n
¨ Alternatives at stage i: xi =0,1,… number of selected items
¨ Stage si : total weight assigned/consumed to stages i, i+1,…,n
¨ fi(si) = the maximum value at stages i, i +1, .., n given si
¨ Backward recursive formula: start from item i =n.
The weight used at stage i is: ()=.
Therefore:
f
fn+1(sn+1)=0
Note: s ~ g in previous method
Example 2: Extended Knapsack Problem
A vessel has capacity C = 4 tons. So we must consider
to load among 3 items as the following table

Item Weight Value

1 2 31
2 3 47
3 1 14
Determine number of units of each item that
maximize total value
 Example 2 Solution
¨ Stage i = 3: s3 = w3x3 =x3 ≤ C = 4
f3(s3) =max[v3x3+f4(s4)]=max[14x3]; f4(s4)=0;
s3 14x3+f4(s3-x3)] Optimal
Solution Item W V
x3=0 x3=1 x3=2 x3=3 x3=4 f3(s3) x*3
1 2 31
0 0 - - - - 0 0
2 3 47
1 0 14+0 - - - 14 1
3 1 14
2 0 14+0 28 - - 28 2

0 14+0 28 42 - 42 C=4
3 3

4 0 14+0 28 42 56 56 4
 Example 2 Solution
¨ Stage i = 2: x2 =={0,1}
f2(s2) =max[v2x2+f3(s3)]= max[47x2+ f3(s2-3x2)];

s2 [47x2+ f3(s2-3x2)] Optimal Solution Item W V

x2=0 x2=1 f2(s2) x*2 1 2 31
0+0 - 0
0 0 2 3 47
1 0 + 14 - 14 0
3 1 14
2 0 + 28 - 28 0
C=4
3 0 + 42 47 + 0 47 1

4 0 + 56 47 + 14 61 1
 Example 2 Solution
¨ Stage i = 1: x1 =={0,1,2}
f1(s1) =max[v1x1+f2(s2)]= max[31x1+ f2(s1-2x1)];

s1 [31x1+ f2(s1-2x1)] Optimal Solution Item W V

x1=0 x1=1 x1=2 f1(s1) x*1 1 2 31
0+0 - - 0
0 0 2 3 47
1 0 + 14 - - 14 0
3 1 14
2 0 + 28 31 + 0 - 31 1
C=4
3 0 + 47 31 + 14 - 47 0

4 0 + 61 31 + 28 62 + 0 62 2
 Example 2 Solution
¨ Optimal Solution: given C =4
¨ At Stage 1, f1(s1) =62, s1=4, x*1 = 2  f2(s2)
= 0;s2=0=s1-2*x1
¨ At stage 2, f2(s2) = 0 at s2= 0, x*2 = 0 f3(s3)
=0 ;s3=0
¨ At stage 3, f3(s3) =0 at s3=0, x*3 =0.
¨ Optimal solution is (2,0,0) Objective value
= 62
 Practice
¨ Let’s run our algorithm on the following data:
n = 4 (# of elements)
W = 5 (max weight)
¨ Elements (weight, benefit): (2,3), (3,4), (4,5), (5,6)
¨ How to pack the knapsack to achieve maximum
total value of packed items?
 Practice Solution
¨ Stage i = 4: s4 = w4x4 =5x4 ≤ C = 5
f4(s4) =max[v4x4+f5(s5)]=max[6x4]; f5(s5)=0;
s4 6x4+f5(s4-5x4)] Optimal
Solution Item W V
x4=0 x4=1 f4(s4) x*4
1 2 3
0 0 - 0 0
2 3 4
1 0 - 0 0
3 4 5
2 0 - 0 0

0 - 0 4 5 6
3 0

4 0 - 0 0 C=5
5 0 6 6 1
 Practice Solution
¨ Stage i = 3: s3 = w3x3 =4x3 ≤ C = 5
f3(s3) =max[v3x3+f4(s4)]=max[5x3+f4(s3-4x3)]=;
s3 5x4+f4(s3-4x3)] Optimal
Solution Item W V
x3=0 x3=1 f3(s3) x*3
1 2 3
0 0+0 - 0 0
2 3 4
1 0+0 - 0 0
3 4 5
2 0+0 - 0 0

0+0 - 0 4 5 6
3 0

4 0+0 5+0 5 1 C=5

5 0+6 5+0 6 0
 Practice Solution
¨ Stage i = 2: s2 = w2x2 =3x2 ≤ C = 5
f2(s2) =max[v2x2+f3(s3)]=max[4x2+f3(s2-3x2)]=;
s2 4x2+f3(s2-3x2)] Optimal
Solution Item W V
x2=0 x2=1 f2(s2) x*2
1 2 3
0 0+0 - 0 0
2 3 4
1 0+0 - 0 0
3 4 5
2 0+0 - 0 0

0+0 4+0 4 4 5 6
3 1

4 0+5 4+0 5 0 C=5

5 0+6 4+0 6 0
 Practice Solution
¨ Stage i = 1: s1 = w1x1 =2x1 ≤ C = 5
f1(s1) =max[v1x1+f1(s1)]=max[3x1+f2(s1-2x1)]
s1 3x1+f2(s1-2x1)] Optimal
Solution Item W V
x1=0 x1=1 x1=2 f1(s1) x*1
1 2 3
0 0+0 - - 0 0
2 3 4
1 0+0 - - 0 0
3 4 5
2 0+0 3+0 - 3 1

0+0 3+0 - 3 4 5 6
3 1

4 0+5 3+0 6+0 6 2 C=5

5 0+6 3+4 6+0 7 1
X1=1,x2=1, x3=x4=0 V* =7
 Practice 2
¨ Let’s run our algorithm on the following data:
Item P s h C EOQ
1 1.7 5 1 2 7.07
2 1.6 13 1 3 11.4
3 1.3 5 1 2 7.07
4 1.1 8 1 1 8.94
5 1.2 9 1 2 9.48
6 1.7 10 1 1 10
7 1.1 14 1 3 11.8
D=5
 Practice Solution
¨ Stage i = 4: s4 = w4x4 =5x4 ≤ C = 5
f4(s4) =max[v4x4+f5(s5)]=max[6x4]; f5(s5)=0;
s4 6x4+f5(s4-5x4)] Optimal
Solution Item W V
x4=0 x4=1 f4(s4) x*4
1 2 3
0 0 - 0 0
2 3 4
1 0 - 0 0
3 4 5
2 0 - 0 0

0 - 0 4 5 6
3 0

4 0 - 0 0 C=5
5 0 6 6 1
 Practice Solution
¨ Stage i = 7:Q7 = 3x7 ≤ C = 5
f7(s7) =min[v7x7+f8(s8)]=max[5x3+f4(s3-4x3)]=;
s3 5x4+f4(s3-4x3)] Optimal
Solution Item W V
x3=0 x3=1 f3(s3) x*3
1 2 3
0 0+0 - 0 0
2 3 4
1 0+0 - 0 0
3 4 5
2 0+0 - 0 0

0+0 - 0 4 5 6
3 0

4 0+0 5+0 5 1 C=5

5 0+6 5+0 6 0
 Practice Solution
¨ Stage i = 2: s2 = w2x2 =3x2 ≤ C = 5
f2(s2) =max[v2x2+f3(s3)]=max[4x2+f3(s2-3x2)]=;
s2 4x2+f3(s2-3x2)] Optimal
Solution Item W V
x2=0 x2=1 f2(s2) x*2
1 2 3
0 0+0 - 0 0
2 3 4
1 0+0 - 0 0
3 4 5
2 0+0 - 0 0

0+0 4+0 0 4 5 6
3 0

4 0+5 4+0 5 0 C=5

5 0+6 4+0 6 0
 Practice Solution
¨ Stage i = 1: s1 = w1x1 =2x1 ≤ C = 5
f1(s1) =max[v1x1+f1(s1)]=max[3x1+f2(s1-2x1)]
s1 3x1+f2(s1-2x1)] Optimal
Solution Item W V
x1=0 x1=1 x1=2 f1(s1) x*1
1 2 3
0 0+0 - - 0 0
2 3 4
1 0+0 - - 0 0
3 4 5
2 0+0 3+0 - 3 1

0+0 3+0 - 3 4 5 6
3 1

4 0+5 3+0 6+0 6 2 C=5

5 0+6 3+0 6+0 6 0,2
X1=2,x2=x3=x4=0 V* =6
X1=x2=x3=0;x4=1  V*=6