1
DYNAMIC PROGRAMMING
24/05/2023
24/05/2023
The shortest path
• To find a shortest path in a multi-stage graph
3 2 7
1 4
S A B 5
T
5 6
• Apply the greedy method :
the shortest path from S to T :
1+2+5=8
24/05/2023
The shortest path in multistage graphs
4
A D
1 18
11 9
• e.g.
2 5 13
S B E T
16 2
5
C 2
F
• The greedy method can not be applied to this
case: (S, A, D, T) 1+4+18 = 23.
• The real shortest path is:
(S, C, F, T) 5+2+2 = 9.
24/05/2023
Dynamic Programming
• Dynamic Programming is a technique that can
be used to solve many optimization problems. In
most applications, dynamic programming
obtains solutions by working backward from the
end of problem toward the beginning, the
breaking up a large, unwieldy problem into a
series of smaller, more tractable problems.
5
Compare DP & LP
24/05/2023
LP DP
Algorithm Can programming Cannot programming.
(Simplex)
Solution One stage Multi-stage
6
24/05/2023
4 Steps in Dynamic Programming
• Divide the Problem into stages
• Solve the last stage first
• Work backwards solving stages
• Optimal solution obtained from all solved stages
7
24/05/2023
TYPES OF DP PROBLEMS
2 types of DP:
- Network:
Ex: Shortest path problem
- Non – network:
Ex: Knapsack problem.
8
Shortest path construction:
24/05/2023
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
9
24/05/2023
Shortest path construction: 4th stage
• d(J, H) = 3
• d(J, I) = 4
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
10
24/05/2023
Shortest path construction: 3rd stage
d(E,J) =d(E,H )+ d(J, H) = 1+3=4*
d(E,J) = d(E, I) +d(J, I) = 4+4=8
d(F,J )=d(F,H )+ d(J, H) = 6+3=9
d(F,J )= d(F, I) +d(J, I) = 3+4=7*
d(G,J )= d(G,H )+ d(J, H) = 3+3=6*
d(G,J )= d(G, I) +d(J, I) = 3+4=7
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
11
24/05/2023
Shortest path construction: 2nd stage
d(B,J)=d(B,E )+ d(E,J) = 7+4=11*
d(B,J)= d(B, F) + d(F,J)= 7+4=11*
d(B,J)=d(B, G) + d(G,J)=6+6=12
d(C,J)=d(C,F )+ d(F, J) = 6+3=9
d(C,J)=d(C,E)+ d(E, J) =3+4=7*
d(C,J)= d(C, G) +d(G, J) = 4+6=10
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
d(D,J)= d(D, E) +d(E, J) = 4+4=8*
d(D,J)= d(D, F) +d(F, J) = 1+7=8*
d(D,J)=d(D,G )+ d(G, J) = 5+6=11
12
24/05/2023
Shortest path construction: 1st stage
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
d(A,J)=d(A,B )+ d(B,J) = 2+11=13
d(A,J)= d(A, C) + d(C,J)= 4+7=11*
d(A,J)= d(A, D) + d(D,J)= 3+8=11*
13
The shortest path
24/05/2023
11 4
11 7 7 0
8 6
Route A-C-E-H-J: 4+3+1+3=11
14
24/05/2023
KNAPSACK PROBLEM
Objective function of Knapsack problem can be:
- Minimization or
- Maximization
Ex:
- A knapsack can hold some limited items. Given a set
of items, each with a weight and a value, determine
the count of each item to include in a collection so
that the total weight is less than or equal to a given
limit and the total value is as large as possible
15
24/05/2023
KNAPSACK PROBLEM CLASSIFY
• There are many types of Knapsack problem in practice.
Example:
- Choosing goods to carry on NASA space ships.
- Some scheduling problems, for example, there are
some tasks that need to be completed in the next 2
weeks, the 2-week duration can be considered as
knapsack, the tasks must be scheduled so that profit
can be maximized or the cost can be minimized; the
constraints are the working days, or hours in this 2
weeks.
15
16
24/05/2023
Example : Knapsack problem
Given some items, pack the knapsack to get the
maximum total value. Each item has some weight and
some value. Total weight that we can carry is no more
than some fixed number W. So we must consider
weights of items as well as their values.
Item # Weight Value
1 1 8
2 3 6
3 5 5
17
24/05/2023
Formula for subproblems
Recursive formula for subproblems:
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w],V [i 1, w wi ] bi } else
It means, that the best subset of Si that has total weight
w is:
1) the best subset of Si-1 that has total weight w, or
2) the best subset of Si-1 that has total weight w-wi
plus the item i
Recursive Formula
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
• The best subset of Si that has the total weight w, either
contains item i or not.
• First case: wi>w. Item k can’t be part of the solution,
since if it was, the total weight would be > w, which is
unacceptable.
• Second case: wi w. Then the item k can be in the
solution, and we choose the case with greater value.
19
24/05/2023
Example
Let’s run our algorithm on the following data:
n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3),
(3,4),
(4,5),
(5,6)
How to pack the knapsack to achieve maximum total value of packed items?
20
24/05/2023
Example (3)
i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0
21
24/05/2023
Items:
Example (4) 1: (2,3)
i\W 2: (3,4)
0 1 2 3 4 5
0 0 0 0 0 0 0
3: (4,5)
i=1 4: (5,6)
1 0 0 bi=3
2 0
wi=2
3 0
w=1
4 0
w-wi =-1
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [1,1] V [1 1,1] 0
22
Items:
24/05/2023
1: (2,3)
2: (3,4)
Example (5) 3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3
wi=2
2 0
3 0 w=2
4 0 w-wi =0
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [1, 2] max{V [1 1, 2], V [1 1, 2 2] 3}
3
23
24/05/2023
Items:
1: (2,3)
Example (6) 2: (3,4)
i\W 0 1 2 3 4 5 3: (4,5)
0 0 0 0 0 0 0 i=1 4: (5,6)
1 0 0 3 3 bi=3
2 0 wi=2
3 0 w=3
4 0 w-wi =1
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [1,3] max{V [1 1,3], V [1 1,3 2] 3}
max{0,3} 3
24
Items:
24/05/2023
1: (2,3)
Example (7) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3 3
wi=2
2 0
3 0 w=4
4 0 w-wi =2
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [1, 4] max{V [1 1, 4], V [1 1, 4 2] 3}
max{0,3} 3
25
Items:
24/05/2023
Example (8) 1: (2,3)
2: (3,4)
i\W 0 1 2 3 4 5 i=1 3: (4,5)
0 0 0 0 0 0 0
bi=3 4: (5,6)
1 0 0 3 3 3 3
wi=2
2 0
3 0 w=5
4 0 w-wi =3
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [1,5] max{V [1 1,5], V [1 1,5 2] 3}
max{0,3} 3
26
24/05/2023
Items:
Example (9) 1: (2,3)
2: (3,4)
i\W 0 1 2 3 4 5 3: (4,5)
i=2
0 0 0 0 0 0 0 4: (5,6)
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0
3 0 w=1
4 0 w-wi =-2
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [2,1] V [2 1,1] =0
27
24/05/2023
Items:
Example (10) 1: (2,3)
i\W 0 1 2 3 4 5 i=2 2: (3,4)
0 0 0 0 0 0 0 3: (4,5)
bi=4
1 0 0 3 3 3 3 4: (5,6)
wi=3
2 0 0 3
3 0 w=2
4 0 w-wi =-1
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [1,5] max{V [1 1,5], V [1 1,5 2] 3}
max{0,3} 3
28
24/05/2023
Items:
Example (11) 1: (2,3)
2: (3,4)
i\W 0 1 2 3 4 5 3: (4,5)
i=2
0 0 0 0 0 0 0
bi=4 4: (5,6)
1 0 0 3 3 3 3
wi=3
2 0 0 3 4
3 0 w=3
4 0 w-wi =0
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [2,3] max{V [2 1,3], V [2 1,3 3] 4}
max{0, 4} 4
29
Items:
24/05/2023
1: (2,3)
2: (3,4)
Example (12) 3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4
3 0 w=4
4 0 w-wi =1
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [2, 4] max{V [2 1, 4], V [2 1, 4 3] 4}
max{0, 4} 4
30
Items:
24/05/2023
1: (2,3)
Example (13) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4 7
3 0 w=5
4 0 w-wi =2
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [2,5] max{V [2 1,5], V [2 1,5 3] 4}
max{0,3 4} 7
31
Items:
24/05/2023
1: (2,3)
Example (14) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 w= 1..3
4 0
32
24/05/2023
Items:
Example (15) 1: (2,3)
2: (3,4)
i\W 0 1 2 3 4 5 3: (4,5)
i=3
0 0 0 0 0 0 0 4: (5,6)
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 w= 4
4 0 w- wi=0
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w], V [i 1, w wi ] bi } else
V [3, 4] max{V [3 1, 4], V [3 1, 4 4] 5}
max{3, 5} 5
33
Items:
24/05/2023
1: (2,3)
Example (16) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 5
4 0 w- wi=1
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w],V [i 1, w wi ] bi } else
V [3,5] max{V [3 1,5], V [3 1,5 4] 5}
max{7, 5} 7
34
Items:
24/05/2023
1: (2,3)
Example (17) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 1..4
4 0 0 3 4 5
35
Items:
24/05/2023
1: (2,3)
Example (18) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 5
4 0 0 3 4 5 7 w- wi=0
V [i 1, w] if wi w
V [i, w]
max{V [i 1, w],V [i 1, w wi ] bi } else
V [4,5] max{V [4 1,5], V [4 1,5 5] 6}
max{7, 6} 7
How to find actual Knapsack Items
• All of the information we need is in the table.
• V[n,W] is the maximal value of items that can be
placed in the Knapsack.
• Let i=n and k=W
if V[i,k] V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 // Assume the ith item is not in the knapsack
// Could it be in the optimally packed
knapsack?
Items:
1: (2,3)
Finding the Items 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k] V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
Items:
1: (2,3)
Finding the Items (2) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k] V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
Items:
1: (2,3)
Finding the Items (3) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=5
2 0 0 3 4 4 7 wi=4
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k] V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
Items:
1: (2,3)
Finding the Items (4) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=4
2 0 0 3 4 4 7 wi=3
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =3
i=n, k=W
k wi=2
while i,k > 0
if V[i,k] V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
Items:
1: (2,3)
Finding the Items (5) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 k= 2
1 0 0 3 3 3 3 bi=3
2 0 0 3 4 4 7 wi=2
3 0 0 3 4 5 7 V[i,k] = 3
4 0 0 3 4 5 7 V[i1,k] =0
i=n, k=W
k wi=0
while i,k > 0
if V[i,k] V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
Items:
1: (2,3)
2: (3,4)
Finding the Items (6) 3: (4,5)
i\W 0 1 2 3 4 5 i=0 4: (5,6)
0 0 0 0 0 0 0 k= 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
knapsack
4 0 0 3 4 5 7
should contain
i=n, k=W {1, 2}
while i,k > 0
if V[i,k] V[i1,k] then
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1
Items:
Finding the Items (7) 1: (2,3)
i\W 2: (3,4)
0 1 2 3 4 5
0 0 0 0 0 0 0
3: (4,5)
4: (5,6)
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
knapsack
4 0 0 3 4 5 7
should contain
i=n, k=W {1, 2}
while i,k > 0
if V[i,k] V[i1,k] then
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1