1527250826E textofChapter8Module2
1527250826E textofChapter8Module2
Chapter 8
Dynamic Programming
Department of Mathematics
Jadavpur University
Kolkata, India
Email: [email protected]
MODULE - 2: DP Model III, Solution of
Discrete DP Problem and Solution of LPP
by DP
sn = yn yn−1 · · · y2 y1 ≥ p
sn−1 = sn /yn = yn−1 yn−2 · · · y2 y1
··· ······ ··· ··· ··· ··· ···
s2 = s3 /y3 = y2 y1
s1 = s2 /y2 = y1
These state variables are stage transformation of the type sj−1 = Tj (sj , yj ).
Let F(sn ) be the minimum value of the objective function for any feasible sn . Thus,
proceeding as earlier, we obtain the recursion formula
2
Example 2.1: Use Bellman’s principle of optimality to minimize z = y1 + y2 + · · · + yn ,
subject to y1 y2 · · · yn = d, yj ≥ 0 for j = 1, 2, · · · , n
Solution: Let fn (d) denote the minimum attainable sum y1 + y2 + y3 + · · · + yn when the
quantity d is factorized into n factors.
For n = 1, d is factorized into one factor only. So f1 (d) = min {y1 } = d.
y1 =d
For n = 2, d is factorized into two factors y1 and y2 .
If y1 = x and y2 = d/x, then
√ {
} √
f3 (d) = min {x + f2 (d/x)} = min x + 2 d/x = d + 2 (d/d 1/3 ) = 3d 1/3
1/3
0<x≤d 0<x≤d
Therefore, the result which is assumed to be true for n = m is found to be true for
n = m + 1. Hence the result is true for all values of n. Thus the required optimal policy
is
(d 1/n , d 1/n , · · · , d 1/n ) with fn (d) = nd 1/n .
Store
1 2 3 4
0 0 0 0 0
1 4 2 6 2
2 6 4 8 3
Number of boxes 3 7 6 8 4
4 7 8 8 4
5 7 9 8 4
6 7 10 8 4
For administrative reasons, the owner does not wish to split crates between stores.
However, he is willing to distribute zero crates to any of his stores. Find the allocation
of six crates to your stores so as to maximize the expected profit.
Solution: Let the four stores be considered as four stages in a dynamic programming
formulation. The decision variables xj (j = 1, 2, 3, 4) denote the number of crates allo-
cated at the jth stage.
Now, let Pj (xj ) be the expected profit from allocation of xj crates to store j. Then, the
problem can be formulated as an L.P.P as follows :
When xj is allocated at stage j, let s be the number of crates still available at this
stage. Define fj (xj ) as the value of the optimal allocation for stores 1 to 4. If fj (s, xj )
be the profit associated with the optimum solution fj∗ (s) (j = 1, 2, 3, 4) then f4∗ (s) =
max0<x4 ≤s f4 (s, x4 ) = max0<x4 ≤s P4 (x4 ). We have the recurrence relation
∗
fj (s, xj ) = Pj (xj ) + fj+1 (s − xj ) for j = 3, 2, 1
and fj∗ (s) = ∗
max {Pj (xj ) + fj+1 (s − xj )}
0<xj ≤s
The solution to this problem starts with f4∗ (s) and is completed when f1∗ (s) is obtained.
The computations for the first stage (i.e., for j = 4) are as follows :
The computations for the second stage (i.e. for j = 3) are as follows :
From the above computations, it is observed that the maximum profit of Rs. 18 can be
obtained by choosing the following eight alternative solutions:
Example 2.3 (Cargo Load Problem): A vessel is to be loaded with stocks of 3 items.
Each unit of item i has a weight wi and value ri . The maximum cargo weight the vessel
can take is 5 and the details of the three items are as follows:
i wi ri
1 1 30
2 3 80
3 2 65
Develop the recursive equation for the above case and find the most valuable cargo
load without exceeding the maximum cargo weight by using dynamic programming.
where pj (xj ) denotes the expected value obtained from allocation of xj units of weight
to the item j.
Now, for one-stage problem, we have f1∗ = maxx1 {30x1 }, where the largest value of x1
is [5/1] = 5. We then have the following tabular computations:
For 2-stage problem, the largest value of x2 is [5/3] = 1 and f2∗ = maxx2 {80x2 +f1 (s−3x2 )}
For 3-stage problem, the largest value of x3 is [5/2] = 2 and f3∗ (s) = maxx3 {65x3 + f2∗ (s −
2x3 )}
Value of 65x3 + f2∗ (s − 2x3 ) Optimum solution
x3 0 1 2 f3∗ (s) x3∗
s
0 0 + 0 =0 0 0
1 0 +30 =30 30 0
2 0 +60 =60 65+0 =65 65 1
3 0 +90 =90 65+30 =95 95 1
4 0 +120=120 65+60=125 130+0=130 130 2
5 0 +150=150 65+90=155 130+30=160 160 2
From the above table, we see that x3∗ = 2. Then we have s = 1. Therefore, from the
previous table, we get x2∗ = 0 and finally for s = 1 we get from the first table x1∗ = 1.
Thus the required solution is x3∗ = 2, x2∗ = 0, x1∗ = 1 with the maximum value of load =
160.
Maximize z = c1 x1 + c2 x2 + ... + cn xn
subject to ai1 x1 + ai2 x2 + ... + ain xn ≤ bi , i = 1, 2, ..., m
xj ≥ 0, j = 1, 2, ..., n (2.1)
Let the above general LPP be considered as a multi-stage problem with each ac-
tivity j (j = 1, 2, ..., n) as individual stage. Then, this is an n−stage problem and the
decision variables are the levels of activities xj (≥ 0) at stage j.
Here the constraints represent the limitations of different resources. Therefore, the
state must be represented by an m−component vector s = (b1 , b2 , ..., bm ).
Let fn (b1 , b2 , ..., bm ) be the maximum value of the general LPP defined above for
stages x1 , x2 , ..., xm for states b1 , b2 , ..., bm .
Using forward computational procedure, the recursive equation is given by
fj (b1 , b2 , ..., bm ) = max {cj xj + fj−1 (b1 − a1j xj , b2 − a2j xj , ..., bm − amj xj )}
0≤xj ≤b
The maximum value of b that xj can assume is
b1 b2 b
b = Min{ , ,··· , m }
a1j a2j amj
Solution: The problem consists of three resources and two decision variables. There-
fore, the states of the equivalent dynamic programming problem are b1 = 4, b2 = 6, b3 =
18.
For the first stage, we have f1 (b1 , b2 , b3 ) = max0≤x1 ≤b {3x1 }, where maximum value of b
3 } = 4.
that x1 can assume is b = min{ 41 , 60 , 18
Therefore, f1 (4, 6, 18) = max0≤x1 ≤4 {3x1 } = 3Min.{4, 18−2x
3 }
2
For the two-stage problem, we have f2 (b1 , b2 , b3 ) = max0≤x2 ≤b {5x2 + 3x1 }, where maxi-
2 } = 6.
mum value of b that x2 can assume is b = min{ 40 , 61 , 18
Therefore, f2 (4, 6, 18) = max0≤x2 ≤6 {5x2 + 3Min(4, 18−2x 2
3 )}
( 18−2x )
4, if 0 ≤ x2 ≤ 3
Now, min 4, 3 2 =
18−2x2 , if 3 < x2 ≤ 6
3
( 18−2x )
5x2 + 12, if 0 ≤ x2 ≤ 3
Therefore, 5x2 + 3 min 4, 3 2 =
18 + 3x2 , if 3 < x2 ≤ 6
Hence the maximum value of 5x2 +12 is 27 at x2 = 3 and the maximum value of 18+3x2
is 36 which occurs at x2 = 6. Thus the optimal value of f2 (4, 6, 18) is 36.
Hence, the required solution is max z = 36 with x2∗ = 6, x1∗ = Min{4, 18−2x
3 } = 2.
2