DEPARTMENT OF COMPUTER SCIENCE

UNIVERSITY OF JOENSUU
JOENSUU, FINLAND

Design and Analysis of Algorithms

Lecture

Dynamic programming
Alexander Kolesnikov

17.10.205
List of sample problems
Shortest path in trellis graph;
Optimal allocation of constrained resource;
Optimal sequence partition (k-link shortest path).
to be continued ...
Shortest path in trellis graph
Stagecoach problem

A traveler wishes to minimize the length of a journey

from town A to J.
Greedy algorithm

The length of the route A-B-F-I-J: 2+4+3+4=13.

Can we find shorter route?

Exhaustive search: try all

Route A-D-F-I-J: 3+1+3+4=11

The total number of routes to be tested: 3321=18
Can we avoid exhaustive search?
Shortest path construction: 1st stage (B)

?
Shortest path construction: 1st stage

S(A,B)=2
S(A,C)=4
S(A,D)=3
Shortest path construction: 2nd stage (E)

? 1. (A..B)-E: 2+7=9
2. (A..C)-E: 4+3=7
3. (A..D)-E: 3+4=7
----------------------
(A..C)-E: 7
Shortest path construction: 2nd stage (E)

1. (A..B)-E: 2+7=9
2. (A..C)-E: 4+3=7 *)
3. (A..D)-E: 3+4=7
----------------------
(A..C)-E: 7
Shortest path construction: 2nd stage (F)

1. (A..B)-F: 2+4=6
2. (A..C)-F: 4+2=6
3. (A..D)-F: 3+1=4 *)
?
----------------------
(A..C)-F: 4
Shortest path construction: 2nd stage (F)

1. (A..B)-F: 2+4=6
2. (A..C)-F: 4+2=6
3. (A..D)-F: 3+1=4 *)
----------------------
(A..D)-F: 4
Shortest path construction: 2nd stage (G)

1. (A..B)-G: 2+6=8 *)
2. (A..C)-G: 4+6=10
3. (A..D)-G: 3+5=8
----------------------
(A..B)-G: 8
?
Shortest path construction: 2nd stage (G)

1. (A..B)-G: 2+6=8 *)
2. (A..C)-G: 4+6=10
3. (A..D)-G: 3+5=8
----------------------
(A..B)-G: 8
Shortest path construction: 3rd stage (H)

1. (A..E)-H: 7+1=8 *)
2. (A..F)-H: 4+6=10
3. (A..G)-H: 5+3=8
----------------------
(A..E)-H: 5
Shortest path construction: 3rd stage (H)

1. (A..E)-H: 7+1=8 *)
2. (A..F)-H: 4+6=10
3. (A..G)-H: 5+3=8
----------------------
(A..E)-H: 5
Shortest path construction: 3rd stage (I)

1. (A..E)-I: 7+4=11
2. (A..F)-I: 4+3=7 *)
3. (A..G)-I: 5+3=8
----------------------
(A..F)-I: 7
Shortest path construction: 3rd stage (I)

1. (A..E)-I: 7+4=11
2. (A..F)-I: 4+3=7 *)
3. (A..G)-I: 5+3=8
----------------------
(A..F)-I: 7
Shortest path construction: 4th stage (J)

1. (A..H)-J: 8+3=11 *)
2. (A..I) -J: 7+4=11
----------------------
(A..H)-J: 11
Shortest path construction: 4th stage (J)

1. (A..H)-J: 8+3=11 *)
2. (A..I) -J: 7+4=11
----------------------
(A..H)-J: 11
Backtrack the shortest path

2 7

0 4 4 11

3 8
The shortest path

2 7

0 4 4 11

3 8

Route A-C-E-H-J: 4+3+1+3=11

Trellis graph

1 2 3 4 K-1 K

ff ((kk 11,,ii),),((kk,, jj))

Trellis graph
Distance (weight) from point i1 at stage (j1) to point i2
at stage j:

The total value of cost function:

L f ( j 1, i j 1 ), ( j , i j ) min
K

j 1
Principle of optimality of Bellman
An optimal path has the property that whatever the initial
conditions and control variables (choices) over some
initial period, the control (or decision variables) chosen
over the remaining period must be optimal for the
remaining problem, with the state resulting from the
early decisions taken to be the initial condition.
Dynamic programming
Cost function:
k
Gk (n) min f p j 1 (i j 1 ), p j (i j )
j 1

Recursive eqution:
Gk (n) min Gk (i ) f (k 1, i ), (k , n)
i

Initialization: G0 (n) 0
Complexity
Exhaustive search: O(nK)
Dynamic programming algorithm: O(Kn2)
where K is the number of stages,
n is the number of points in a stage
Optimal allocation
of constrained resource
Problem formulation
N units of a resource;
This resource must be distributed among K activities;
Functions fk(x) - profit for allocated resource;

Allocate N units of resource to K activities with given

return functions so that the total profit is maximal:

K
P f j (n j ) max
j 1
K
subject to: n
j 1
j N nj 0
Dynamic programming formulation
Optimal value function:
k
Gk (n) max f j n j F GK (N )
n1 ... n j n
j 1
Recursive equation:

Gk (n) max Gk 1 (m) f k n m ; 0 n N .

0 m n

Initialization:

G0 (n) 0; 0 n N
Allocate 3 mln euros into four projects
Profit fk(x), K=3, N=3.
x Proj #1 Proj #2 Proj #3 Proj #4
0 0 0 0 0
1 2 5 4 2
2 8 6 4 4
3 9 7 4 5
4
P f j (n j ) max
j 1
4

n
j 1
j 3
Trellis graph
Solution

*)
#1: 2; f1(2)=8
#2: 1; f2(1)=5
#3: 0; f3(0)=0
#4: 0; f4(0)=0
----------------------
N=3; G4(3)=13
 Search in the state space
GK(N)
K

Gk(n)
fk(n-j)
k
k-1
*
Gk-1(j)
1
0
Start state 0 j n N

Gk(n) = max{Gk(0) + fk(n),

Gk(1) + fk(n-1),
...
Gk(j-1)+ fk(j), Ak(n)=jopt
...
Gk(n) + fk(0)}
Optimal partition of
data sequence
Problem formulation
Given a sequence of data X={x1, x2, ,xN}
Do partition of the sequence X into to K groups with
given cost functions f(xi,xj) so that the total value of
the cost function is minimal:

f ( x1 , xi1 ) f ( xi1 , xi2 ) ... f ( xiK 1 , xN ) min

f (x
j 1
i j 1 , xi j ) min
Partition into groups: Example
Data: x 0= < x1 <... < xj < ... < xN
Partition indices: i0= 0 < i1 <... < ij < ... < iM =N.
Groups: #1 #2 #3

... ( ]( ]( ]
(x0=) x1 x2 x3 x4 x5 x6 x7x8 x9 x10 x11 x12 x13 x14 xN

(i0=0) i1=4 i2=10 iK =N=15

P f 0,4 f 4,10 f 10,15

K=3
Problem formulation

Cost function:

Gk (n) f i j 1 , i j ; i j n .
k

j 1

Recursive equation:
Gk (n) min Gk 1 ( j ) f j , n ; 0 n N .
k 1 j n

Initialization:

G0 (n) 0; 0 n N .
 Search in the state space
GK(N)
K b
State space

Gk (n)
f(j,n)
k
k-1
*
Gk-1(j)

0
Start state 1 j n N

Gk(n) = min{Gk(k) + f(k, n],

Gk(k) + f(k+1,n],
...
Gk(j-1) + f(j, n]*),
Ak(n)=jopt
...
Gk(n-1) + f(n, n]}
Scheme of the DP algorithm
// Initialization
FOR n = 1 TO N DO G1(n)= (n) f(1,n]
// Minimum search
FOR k = 2 TO K DO
FOR n = k TO N DO
dmin
FOR j= k-1 TO n-1 DO
c Gk-1(j) + f(j,n]
IF(c < cmin)
cmin c;
jmin j
ENDIF
ENDFOR
Gk (n) dmin Complexity:
Complexity: O(KN
O(KN2))
2

Ak (n) jmin
ENDFOR
ENDFOR
 Backtrack in the state space
AK(N)
K b
State space

0
Start state 1 j n N
S(M+1)=
S(M+1)=NN N=22, K=8: S={22,18,14,12,9,6,4,3,1}
FOR
FORm m==K+1
K+1 TO
TO 22DO
DO (x0,x3], (x3,x4], (x4,x6], (x6,x9], (x9,x12],
S(m1)
S(m1)==A(S(m),
A(S(m),m))
m)) (x12,x14], (x14,x18], (x18,x22]
PP==GGK(N)
K(N) K