Dynamic Programming

7 -1
 Dynamic Programming
 Dynamic Programming is an algorithm design
method that can be used when the solution
to a problem may be viewed as the result of a
sequence of decisions
 Break problems into subproblems and
combine their solutions into solutions to
larger problems.
 In contrast to divide-and-conquer, there may
be relationships across subproblems.

7 -2
 DYNAMIC PROGRAMMING NOTATION AND
MATHEMATICAL DESCRIPTION

Stage Stage Stage

7 -3
f *n ( x n )  opt{ f n  f n 1  ...  f1 }

7 -4
A recursive relationship is always used to
related the optimal policy at stage n to to
the (n-1) stages that follow. The
relationship is given by

f *n ( x n )  opt{rn (d n )  f *n 1 ( x n  d n1 }

7 -5
 Backbone od DP:
f n ( x n )  {rn (d n )  f *n 1 ( x n  d n1 }

f *n ( x n )  opt{ f n ( x n ) }

7 -6
 The shortest path
 To find a shortest path in a multi-stage graph
3 2 7

1 4
S A B 5
T

5 6
 Apply the greedy method :
the shortest path from S to T :
1+2+5=8

7 -7
 The shortest path in
multistage graphs
 e.g. A
4
D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F
 The greedy method can not be applied to this case:
(S, A, D, T) 1+4+18 = 23.
 The real shortest path is:
(S, C, F, T) 5+2+2 = 9.

7 -8
 Dynamic programming approach
 Dynamic programming approach (forward approach):

A
4
D 1 A
1 d(A, T)
18
11 9

2 d(B, T)
S
2
B
5
E
13
T S B T
16 2

5 d(C, T)
5
C 2
F C

 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}

7 -9
 d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}
= min{9+18, 5+13, 16+2} = 18.
4
A D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F
 d(C, T) = min{ 2+d(F, T) } = 2+2 = 4
 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
 The above way of reasoning is called
backward reasoning.

7 -10
 Backward approach
(forward reasoning)
4
A D
1 18
11 9

2 5 13
S B E T
16
 d(S, A) = 1 2

5
d(S, B) = 2 C F
2
d(S, C) = 5
 d(S,D)=min{d(S,A)+d(A,D), d(S,B)+d(B,D)}
= min{ 1+4, 2+9 } = 5
d(S,E)=min{d(S,A)+d(A,E), d(S,B)+d(B,E)}
= min{ 1+11, 2+5 } = 7
d(S,F)=min{d(S,B)+d(B,F), d(S,C)+d(C,F)}
= min{ 2+16, 5+2 } = 7
7 -11
 d(S,T) = min{d(S, D)+d(D, T), d(S,E)+
d(E,T), d(S, F)+d(F, T)}
= min{ 5+18, 7+13, 7+2 }
=9 4
A D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F

7 -12
7 -13
Stage 1 Stage 2 Stage 3 Stage 4

7 -14
Formulation

7 -15
 Thus,
* *
f ( s )  min f n ( s, x n )  f n ( s, x )
n n
xn

7 -16
 Stage 4
S Alternative rn f4 f*4 Optimal
8 8-10 8 8 8 8-10
9 9-10 4 4 4 9-10

Recursive equation

* *
f ( s )  min f ( s,10)
4 4
xn

7 -17
 Stage 3
S Alternative rn f3 f*3 Optimal
5 5-8 4 4+8=12 12 5-8
5-9 8 8+4= 12 5-9
6 6-8 9 9+8=17 10 6-9
6-9 6 6+4=10
7 7-8 5 5+8 =13 9 7-9
7-9 4 5+4 =9

Recursive equation
* *
f ( s )  min (r3  f )
3 4
xn
7 -18
 Stage 2
S Alternative rn f3 f*3 Optimal
2 2-5 10 10+12=22 19 2-6
2-6 9 9+10=19
3 3-5 6 6+12=18 17 3-6
3-6 7 7+10=17
3-7 10 10+9=19
4 4-6 3 3+10=13 13 4-6
4-7 8 8+9=17

Recursive equation

* *
f ( s )  min (r2  f )
2 3
xn
7 -19
 Stage 1
S Alternative rn f3 f*3 Optimal
1 1-2 4 4+22=26 16 1-4
1-3 2 2+17=19
1-4 3 3+13=16

Recursive equation

* *
f ( s )  min (r3  f )
3 2
xn
7 -20
 Optimal Solution:
Total Distance =16
Best Route(path)
=1 4 6 9 10

7 -21
 Principle of optimality
 Principle of optimality: Suppose that in solving a
problem, we have to make a sequence of decisions
D1, D2, …, Dn. If this sequence is optimal, then the
last k decisions, 1  k  n must be optimal.
 e.g. the shortest path problem
If i, i1, i2, …, j is a shortest path from i to j, then i1,
i2, …, j must be a shortest path from i1 to j
 In summary, if a problem can be described by a
multistage graph, then it can be solved by dynamic
programming.

7 -22
 Dynamic programming
 Forward approach and backward approach:
 Note that if the recurrence relations are formulated
using the forward approach then the relations are
solved backwards . i.e., beginning with the last
decision
 On the other hand if the relations are formulated
using the backward approach, they are solved
forwards.
 To solve a problem by using dynamic
programming:
 Find out the recurrence relations.
 Represent the problem by a multistage graph.

7 -23
7 -24
7 -25
7 -26
7 -27
7 -28
Example #3

7 -29
7 -30
7 -31
7 -32
Example # 4

7 -33
7 -34
7 -35