CH3.

Dynamic Programming(DP)
Mergesort, Quicksort, Binary Search
No subproblem overlapping
D & C is good!!
However, if subproblem overlapping, D.P.
Strategy
Establish a recursive property
Solve in a bottom-up fashion programming
Step1. Solving smaller instances first
Step2. Save them in an array
Step3. Use them later
nth Fibonacci Term
Recursive Solution : O(2
n
)
int fib (int n) {
if (n <= 1)
return n;
else
return fib(n-1) + fib(n-2);
}
Iterative Solution : O(n) with extra space O(n)
int fib2 (int n) {
index i; int f[0..n];
f[0] = 0;
if (n > 0) {
f[1] = 1;
for (i = 2; i <= n; i++)
f[i] = f[i-1] + f[i-2];}
return f[n];
}
3.1. Binomial Coefficient
Binomial Coefficient








int bin(int n,k)
{
if k=0 or k=n then bin=1;
else bin(n,k)= bin(n-1,k-1)+bin(n-1,k);
}
= =
< <
-
+
-
-
=
-
= =
n k k
n k
k
n
k
n
k
n
k n k
n
C
k
n
k n
or 0 1
0
1
1
1
)! ( !
!
Binomial Coefficient
Too much overlapping -> exponential algo
Dynamic Programming(D.P.)
1.Establish a recursive property


2.Solve in a bottom-up fashion
= =
< < - + - -
=
i or j j
i j j i B j i B
j i B
0 1
0 ], , 1 [ ] 1 , 1 [
] , [
0 1 2 3 4 j k
0
1
2
3
4

i
n
1
1
1
1
1

1
1
1
2
3
4
1
3
6
1
4
1
B[i-1,j-1] B[i-1,j]
B[i,j]
Binomial Coefficient
int bin2(int n,k)
{
index i,j;
int B[0..n,0..k];
for (i=0; in; i++)
for (j=0; j min(i,k); j++)
if (j=0 or j=i) B[i,j]=1;
else B[i,j] = B[i-1,j-1] + B[i-1,j];
return B[n,k];
}

Time Complexity? Space Complexity?
3.4. Chained Matrix Multiplication
Multiplying A = (a
ij
)
pq
B = (b
ij
)
qr
Standard method : p,q,r multiplications

Multiplying chained matrices?
e.g. A B C D
20 2 2 30 30 12 12 8

Note : Associativity
A(B(CD)) = 30 12 8 + 2 30 8 + 20 2 8 = 3,680
(AB)(CD) = 20 2 30 + 30 12 8 + 20 30 8 = 8,880
A((BC)D) = 2 30 12 + 2 12 8 + 20 2 8 = 1,232
((AB)C)D = 20 2 30 + 20 30 12 + 20 12 8 = 10,320
(A(BC))D = 2 30 12 + 20 2 12 + 20 12 8 = 3,120


Chained Matrix Multiplication
Problem : Find an optimal order for mult. n mat

Naive Algo : consider all possible orders
t
n
: # different orders for A
1
, A
2
, , A
n
t
n
t
n-1
+ t
n-1
= 2 t
n-1

A
1
(A
2
, , A
n
) or (A
1
, A
2
, , )A
n

t
n
2
n-2


Chained Matrix Multiplication
M[i,j] = min # mult. for A
i
x A
i+1
xxA
j
M[i,i]=0
(e.g.) A
1
x A
2
x A
3
x A
4
x A
5
x A
6
5 x 2 x 3

x 4 x 6

x 7 x 8
d
0
x d
1
x d
2
x d
3
x d
4
x d
5
x d
6

(A
4
A
5
) A
6
: 4 6 7 + 4 7 8 = 392
A
4
(A
5
A
6
) : 6 7 8 + 4 6 8 = 528
M[4,6] = min(392, 528) = 392
Chained Matrix Multiplication
Optimal order? : one of the followings
0 ] , [
) ] , 1 [ ] , [ ( min ] , [
) ] 6 , 1 [ ] , 1 [ ( min ] 6 , 1 [
] 6 , 1 [ ] , 1 [
] 6 , 6 [ ] 5 , 1 [ 5.
4.
3.
] 6 , 3 [ ] 2 , 1 [ 2.
] 6 , 2 [ ] 1 , 1 [ . 1
1
1
5 1
=
< + + + =
+ + + =
+ + +
+ +
+ +
+ +
-
-

i i M
j i if d d d j k M k i M j i M
k M k M M
k M k M
M M
M M
d
0
d
1
d
6
M M
A
1
(A
2
A
3
...A
6
)
j
k i
j k i
k
d
0
d
2
d
6
d
0
d
k
d
6
d
0
d
5
d
6
(A
1
A
2
)(A
3
...A
6
)
(A
1
...

A
5
)

A
6
(A
1
A
2
A
3
)(A
4
A
5
A
6
)
(A
1
...

A
4
)(A
5
A
6
)
d
0
d
k
d
6
Chained Matrix Multiplication
(e.g.) A
1
A
2
A
3
A
4
A
5
A
6
5 2 3 4 6 7 8

0 6
) 5 ( 336 0 5
) 5 ( 392 ) 4 ( 168 0 4
) 5 ( 336 ) 4 ( 198 ) 3 ( 72 0 3
) 5 ( 268 ) 4 ( 156 ) 2 ( 72 ) 2 ( 24 0 2
) 1 ( 348 ) 1 ( 226 ) 1 ( 132 ) 1 ( 64 ) 1 ( 30 0 1
6 5 4 3 2 1
132 ) 6 4 5 0 64 , 6 3 5 72 30 , 6 2 5 72 0 min( ] 4 , 1 [
64 ) 4 3 5 0 30 , 4 2 5 24 0 ( min
) ] 3 , 3 [ ] 2 , 1 [ , ] 3 , 2 [ ] 1 , 1 [ ( min ] 3 , 1 [
3 2 0 3 1 0
= + + + + + + =
= + + + + =
+ + + + =
M
d d d M M d d d M M M
(j)
(i)
Chained Matrix Multiplication
minmult (n, d, P) // Algorithm 3.6
1. for i = 1 to n
2. M[ i, i] = 0
3. for diagonal = 1 to n -1
4. for i = 1 to n - diagonal
5. j = i + diagonal
6. M[ i, j ] = min ( M[i ,k ] + M[k+1, j ] +d[i-1]*d[k]*d[j])


i s k s j -1
7. P[ i, j ] = "k" from 6
8. return M[1,n]


Time Complexity???
compute each
diagonal
Compute each
element in the
diagonal
k has to run
"diagonal number"
of elements
3.2. Floyds Algorithm for Shortest Paths








Weighted, directed graph(digraph)
G=(V,E)
v
5
v
4
v
3
v
2
v
1
3
5
1
9
1 2
2
4
3
Vertex, edge, path
[v
1
,v
2
,v
3
,v
4
]
Simple path
Cycle
Cyclic, Acyclic
Length[v
1
,v
2
,v
3
]=1+3=4
Length[v
1
,v
2
,v
4
,v
3
]=1+2+2=5
Shortest path from v
1
to v
3

[v
1
,v
4
,v
3
]: S.P.
3
Floyds Algorithm for Shortest Paths
All pairs shortest problem
Given any pair (u,v), find a S.P. from u to v
Naive Algorithm
Compute all paths and determine a S.P.
Worse than exponential time

-
(n-2)! paths from a vertex
More efficient algorithm?
Yes. Use D.P.
Graph representation!
Adjacency list
Adjacency matrix : W
=


=
j i if 0
E j) if(i,

E j) if(i, ] , [
] , [
j i c
j i W
Floyds Algorithm for Shortest Paths
0 1

1 5
9 0 3 2


0 4


2 0 3
3

0
0 1 3 1 4
8 0 3 2 5
10 11 0 4 7
6 7 2 0 3
3 4 6 4 0
1 2 3 4 5 1 2 3 4 5
1
2
3
4
5
1
2
3
4
5
Adjacency matrix W Distance matrix D
D[i,j]=length of S.P. from i to j
Goal : D[1..n,1..n], P[1..n,1..n]

Assumption : no negative weight cycle!
Floyds Algorithm for Shortest Paths
D
(k)
[i,j]=length of S.P.

Only {v
1
, v
2
, , v
k
}

IDEA: D
(0)
= W D
(1)
D
(2)
D
(n)
=

D
(e.g.) D
(0)
[2,5]=
D
(1)
[2,5]= min(length[v
2
,v
5
], length[v
2
,v
1
,v
5
])= min(,14)=14
D
(2)
[2,5]=14
D
(3)
[2,5]=14
D
(4)
[2,5]= min(14,5,13,10)=5
D
(5)
[2,5]= 5 = D[2,5]
D
(0)
[i,j]= W[i,j] D
(n)
[i,j]=D[i,j]
v
i
v
j
Floyds Algorithm for Shortest Paths
Strategy (Approach)
1. Establish a recursive property : D
(k-1)
D
(k)
2. Solve bottom-up for k=1,2,,n
W = D
(0)
D
(1)
D
(2 )
D
(n )
=

D

void floyd(int n, const number W[][], number D[][]) {
int i, j, k;
D = W;
for(k=1; k <= n; k++)
for(i=1; i <= n; i++)
for(j=1; j <= n; j++)
D[i][j] = min(D[i][j], D[i][k]+D[k][j]);
}

Time Complexity ? Space Complexity ?
3.3. D.P. & Optimization Problems
Principle of optimality?
All subsolutions of an optimal sol. are optimal
e.g. shortest path s.p form v
i
to v
j



recursive property O
bottom-up construction (smaller larger)
v
i
v
k
v
j
S.P S.P
3.6. Traveling Salesperson Problem
Euler cycle : visit each edge once
Hamiltonian cycle: visit each node once (tour)
The T.S.P. problem;
Determine a shortest route(H.C.) (find an optimal tour)






Naive Algo.: consider all possible tours (n-1)! Tours
Principle of Optimality : v
1
v
k

D.P can be used.
v
1
v
4
v
3
v
2
2
6
1
6
4
8
7
3
9
Length[1,2,3,4,1]=22
Length[1,3,2,4,1]=26
Length[1,3,4,2,1]=21

optimal optimal
Traveling Salesperson Problem
Input: Graph Adjacency Matrix W
weight of (v
i
,v
j
) if (v
i
,v
j
) E
W[i,j] = if (v
i
,v
j
) e E
0 if i=j

V: the set of all the vertices {v
1
,v
n
}
A: A V
D[v
i
,A] = length of an S.P
passing through each of A exactly once

v
i
v
1
Traveling Salesperson Problem
(e.g.) A={v
3
}: D[v
2
,A] = length[v
2
,v
3
,v
1
] =
A={v
3
,v
4
}: D[v
2
,A] = min(length[2,3,4,1],length[2,4,3,1])
= min(20,) = 20

Length of an optimal tour = min(W[1,j] + D[v
j
,V-{v
1
,v
j
}])
2jn

D[v
i
,A] = min(W[i,j] + D[v
j
,A-{v
j
}]) if A C
v
j
A
D[v
i
,C] = W[i,1]


Traveling Salesperson Problem
void travel(int n, const number W[], number& minlength)
{
index i,j,k; number D[1..n, subset of V-{v1}];
for i=2 to n
D[i,0] = W[i,1];
for k=1 to n-2
for all subsets A V-{v
i
} s.t. |A|=k
for i s.t. i1 & v
i
e A {
D[i,A] = min(W[i,j] + D[j,A-{v
j
}]);}
j:v
j
A
D[1,V-{v1}] = min(W[1,j] + D[j,V-{v
1
,v
j
}]);
2jn
minlength = D[1,V-{v
1
}];
}



Traveling Salesperson Problem
Time Complexity: O(n
2
2
n
) Space Complexity: O(n2
n
)
n = 20
Brute-force algo : 19! ms = 3,857 years
D.P. algo : (20-1)(20-2)2
20-3
ms = 45 sec
2020
20
= 20,971,520 array slots
How about n 30?
Just give up & go to chapter 9(NP-hard problem)