Dynamic
Programming
Dynamic Programming
Dynamic
Programming is an algorithm design
method that can be used when the solution to
a problem may be viewed as the result of a
sequence of decisions
Example Fibonacci Numbers
using recursion
Function f(n)
if n = 0
output 0
else
if n = 1
output 1
else
output f(n-1) + f(n-2)
Example Fibonacci Numbers
using recursion
Run
time: T(n) = T(n-1) + T(n-2)
This function grows as n grows
The run time doubles as n grows and is order
O(2n).
This is a bad algorithm as there are
numerous repetitive calculations.
Example Fibonacci Numbers
using Dynamic programming
f(5)
f(4)
f(3)
f(2)
f(3)
f(2)
f(1)
f(2)
f(1)
Example Fibonacci Numbers
using Dynamic programming
Dynamic
Programming calculates
bottom to top
Values are stored for later use.
This reduces repetitive calculation.
from
Greedy vs. Dynamic Programming :
Both techniques are optimization techniques, and both
build solutions from a collection of choices of individual
elements.
The greedy method computes its solution by making its
choices in a serial forward fashion, never looking back or
revising previous choices.
Dynamic programming computes its solution bottom up by
synthesizing them from smaller sub-solutions, and by trying
many possibilities and choices before it arrives at the
optimal set of choices.
There is no priori test by which one can tell if the Greedy
method will lead to an optimal solution.
By contrast, there is a test for Dynamic Programming,
called the principle of optimality
Statement of Principle of Optimality
Principle of optimality: Suppose that in solving a
problem, we have to make a sequence of
decisions D1, D2, , Dn. If this sequence is optimal,
then the last k decisions, 1 k n must be optimal.
Divide and Conquer
vs.
Dynamic Programming:
Both techniques split their input into parts, find
subsolutions to the parts, and synthesize larger
solutions from smaller ones.
Divide and Conquer splits its input at pre-specified
deterministic points (e.g., always in the middle)
Dynamic Programming splits its input at every
possible split points rather than at a pre-specified
points. After trying all split points, it determines which
split point is optimal.
Multistage Graphs
To find a shortest path in a multi-stage graph
Apply the greedy method :
the shortest path from S to T :
1+2+5=8
The shortest path in multistage
A
D
graphs
e.g.
4
11
18
16
13
The greedy method can not be applied to this
case:
(S, A, D, T) 1+4+18 = 23.
The real shortest path is:
(S, C, F, T) 5+2+2 = 9.
Dynamic programming approach
Dynamic programming approach (forward approach):
A
1
11
d(A, T)
18
16
B
5
d(B, T)
13
d(C, T)
d(S, T) =
4
11
D
E
d(D, T)
d(E, T)
d(A,T) = min{4+d(D,T), 11+d(E,T)}
= min{4+18, 11+13} = 22.
Dynamic programming
4
1
11
18
16
13
d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}
= min{9+18, 5+13, 16+2} = 18.
d(C, T) = min{ 2+d(F, T) } = 2+2 = 4
d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
The above way of reasoning is called backward
reasoning.
Backward approach
(forward reasoning)
d(S, A) = 1
d(S, B) = 2
d(S, C) = 5
1
11
18
16
13
d(S,D)=
= min{ 1+4, 2+9 } = 5
d(S,E)=min{d(S, A)+d(A, E),d(S, B)+d(B, E)}
= min{ 1+11, 2+5 } = 7
d(S,F)=min{d(S, A)+d(A, F),d(S, B)+d(B, F)}
= min{ 2+16, 5+2 } = 7
d(S,T) = min{d(S, D)+d(D, T),d(S,E)+d(E,T),
d(S, F)+d(F, T)}
= min{ 5+18, 7+13, 7+2 }
=9
Principle of optimality-
the shortest
path problem
If i, i1, i2, , j is a shortest path from i to j, then i1, i2,
, j must be a shortest path from i1 to j
In summary, if a problem can be described by a
multistage graph, then it can be solved by dynamic
programming.
Dynamic programming
Forward approach and backward approach:
Note: If the recurrence relations are formulated
using the forward approach then the relations are
solved backwards . i.e., beginning with the last
decision
On the other hand if the relations are formulated
using the backward approach, they are solved
forwards.
To solve a problem by using dynamic
programming:
Find out the recurrence relations.
Represent the problem by a multistage graph.
0-1 Knapsack problem
Given a knapsack with maximum capacity
W, and a set S consisting of n items
Each item i has some weight wi and benefit
value bi (all wi , bi and W are integer values)
Problem: How to pack the knapsack to
achieve maximum total value of packed
items?
Using only item1 to itemi and knapsack
weight at most w
The Traveling Salesperson
Problem
Given
a list of cities and their pairwise
distances, the task is to find the shortest
possible route that visits each city exactly
once.
Salesman wishes to travel around a given
set of cities, and return to the beginning,
covering the smallest total distance
The Traveling Salesperson Problem
Instance: n vertices
(cities), distance
between every pair of
vertices
Question: Find
shortest (simple)
cycle that visits every
city
2
3
1
2
4
3
5
3
2
2
4
3
2
5
2
3
Applications of TSP
Collection
and delivery problems
Robotics
Board
drilling
The Traveling Salesperson Problem
Assumption: A tour starts and ends at vertex 1
The principle of optimality holds
g(i,S) = length of a shortest path starting at vertex i, going through all
vertices in S, and terminating at vertex 1
g(1,V-{1}) =
g(i,S) =
Solving the recurrence relation
g(i,) = c , 1in
i1
Then obtain g(i,S) for all S of size 1
Then obtain g(i,S) for all S of size 2
Then obtain g(i,S) for all S of size 3
Finally obtain g(1,V-{1})
The Traveling Salesperson Problem
Example
For |S|=0
g(2,)=c =5
21
g(3,)=c31=6
g(4,)=c41=8
For |S|=1
g(2,{3})=c +g(3,)=15
23
g(2,{4})=c24+g(4,)=18
g(3,{2})=c32+g(2,)=18
g(3,{4})=c34+g(4,)=20
g(4,{2})=c42+g(2,)=13
g(4,{3})=c43+g(3,)=15
3
(a)
(b)
The Traveling Salesperson Problem
Example
For |S|=2
g(2,{3,4}) = min {c +g(3,{4}), c +g(4,{3})} = 25
23
24
g(3,{2,4}) = min {c32+g(2,{4}), c34+g(4,{2})} = 25
g(4,{2,3}) = min {c42+g(2,{3}), c43+g(3,{2})} = 23
For |S|=3
g(1,{2,3,4} = min {c +g(2,{3,4}), c +g(3,{2,4}),
12
13
c14+g(4,{2,3})} =
= min {35, 40, 43} = 35
g(1,V-{1}) =
Example 2
The Traveling Salesperson Problem
Time complexity
Let N be the number of g(i,S) that have to be
computed before it can be used to compute g(1,V-{1})
N=
Total time = O(n22n)
This is better than enumerating all n! different tours
Optimal Binary Search
Tree
Optimal Binary Search Tree
We now want to focus on the construction of
binary search trees for a static set of identifiers.
And only searches are performed.
To find an optimal binary search tree for a given
static file, a cost measure must be determined
for search trees.
Its reasonable to use the level number of a node
as the cost.
Binary Search Tree Example
for
for
while
do
return
if
4 comparisons
in worst case
return
do
if
3 comparisons
in worst case
while
Extended Binary Tree
Example
for
for
while
do
return
return
do
if
while
if
(b)
(a)
External Path Length and Internal
Path Length
External path length of a binary tree is the sum
over all external nodes of the lengths of the
paths from the root to those nodes.
Internal path length is the sum over all internal
nodes of the lengths of the paths from the root to
those nodes.
Let the internal path length be I and external
path length E, then the binary tree of (a) has I =
0+1+1+2+3 = 7, E = 2+2+4+4+3+2 = 17.
Binary Search Tree
Containing A Symbol Table
Lets look at the problem of representing a symbol
table as a binary search tree.
If a binary search tree contains the identifiers a1, a2,
, an with a1 < a2 < < an, and the probability of
searching for each ai is pi, then the total cost of any
binary search tree is
when only successful searches are made.
Binary Search Tree Containing A
Symbol Table
For unsuccessful searches, lets partitioned the identifiers
not in the binary search tree into n+1 classes Ei, 0 i n. If
qi is the probability that the identifier being sought is in Ei,
then the cost of the failure node is
Therefore, the total cost of a binary search tree is
Binary Search Tree With Three
Identifiers Example
do
if
while
if
do
if
while
while
do
(c)
(b)
(a)
while
do
do
while
if
(d)
if
(e)
Cost of Binary Search Tree In The
Example
With equal probabilities, pi = qj = 1/7 for all i and j, we
have
cost(tree a) = 12/7; cost(tree b) = 10/7
cost(tree c) = 12/7; cost(tree d) = 12/7
cost(tree e) = 12/7
Tree b is optimal.
With p1=0.5, p2=0.1, p3=0.05, q0=0.15, q1=0.1, q2=0.05,
and q3=0.05 we have
cost(tree a) = 2.65; cost(tree b) = 1.9
cost(tree c) = 1.5; cost(tree d) = 2.05
cost(tree e) = 1.6
Tree c is optimal.
Determine Optimal Binary
Search Tree
So to determine which is the optimal binary search, it is not
practical to follow the above brute force approach since the
complexity is O(n4n/n3/2).
Now lets take another approach. Let Tij denote an optimal
binary search tree for ai+1, , aj, i<j.
Let cij be the cost of the search tree Tij.
Let rij be the root of Tij and let wij be the weight of Tij, where
Therefore, by definition rii=0, wii=qi, 0 i n. T0n is an optimal
binary search tree for a1, , an. Its cost function is e0n, it
weight w0n, and it root is r0n.
Determine Optimal Binary
Search Tree (Cont.)
If Tij is an optimal binary search tree for ai+1, , aj,
and rij =k, then i< k <j. Tij has two subtrees L and R.
L contains ai+1, , ak-1, and R contains ak+1, , aj. So
the cost cij of Tij is
cij = cost(L) + cost(R) + weight(L) + weight(R)
cij = ci,k-1+ ckj + wi,k-1+ wkj
=min i<k<=j { ci,k-1+ ckj }+ wij
OPTIMALBST(p,q,n)
Example:
i
pi
qi
0.05
0.15
0.10
0.05
0.10
0.20
0.10
0.05
0.05
0.05
0.10
e
1
5
3
0.45
0.05
1.75
0.90
1
0
1.20
0.70
0.10
0.05
1.30
0.60
0.25
0.40
2.00
0.90
0.05
0.55
0.45
0.30
0.10
0.05
0.80
0.50
0.35
0.25
0.10
1.00
0.70
0.50
0.30
0.05
2.75
1.25
0.60
0.30
0.15
0.05
0.20
0.05
0.05
3
2
2
2
2
4
2
2
1
2
3
5
4
5
4
4
0.35
root
5
0.50
5
5
The tables e[i,j], w[i,j], and root [i,j]computed by Optimal-BST
0.10
k2
k1
d0
k5
d1
d5
k4
i
d4
k3
pi
0.15
0.10
0.05
0.10
0.20
d2
qi
0.05
0.10
0.05
0.05
0.05
0.10
d3
Example: key
A B C D
probability 0.1 0.2 0.4 0.3
The tables below are filled diagonal by diagonal: the left one is filled
using the recurrence and the right one, for trees roots, records ks
values giving the minima
.1
.4
1.1 1.7
A
optimal BST
.2
.8 1.4
Example
Let n=4, (a1, a2, a3, a4) = (do, if return, while). Let (p1, p2, p3,
p4)=(3,3,1,1) and (q0, q1, q2, q3, q4)=(2,3,1,1,1). wii = qii, cii=0,
and rii=0, 0 i 4.
Example
0
3
4
0
w00=2
c00=0
r00=0
w01=8
c01=8
r01=1
w02=12
c02=19
r02=1
w03=14
c03=25
r03=2
w04=16
c04=32
r04=2
1
w11=3
c11=0
r11=0
w12=7
c12=7
r12=2
w13=9
c13=12
r13=2
w14=11
c14=19
r14=2
2
w22=1
c22=0
r22=0
w23=3
c23=3
r23=3
w24=5
c24=8
r24=3
3
w33=1
c33=0
r33=0
w34=3
c34=3
r34=4
4
w44=1
c44=0
r44=0
Computation Complexity of
Optimal Binary Search Tree
To evaluate the optimal binary tree we need to
compute cij for (j-i)=1, 2, ,n in that order. When ji=m, there are n-m+1 cijs to compute.
The computation of each cijs can be computed in
time O(m).
The total time for all cijs with j-i=m is therefore
O(nm-m2). The total time to evaluate all the cijs
and rijs is
All pairs shortest path
Floyds Algorithm
All pairs shortest path
In
the all pairs shortest path problem, we want to
find the shortest path from every possible source
to every possible destination.
All pairs shortest path
The problem: find the shortest path between every pair
of vertices of a graph
The graph: may contain negative edges but no
negative cycles
A representation: a weight matrix where
W(i,j)=0 if i=j.
W(i,j)= if there is no edge between i and j.
W(i,j)=weight of edge
The weight matrix and the
graph
1
v1
3
5
v5
3
v2
9
v4
2
2
4
v3
The subproblems
How can we define the shortest distance di,j in
terms of smaller problems?
One way is to restrict the paths to only include
vertices from a restricted subset.
Initially, the subset is empty.
Then, it is incrementally increased until it
includes all the vertices.
The sub-problems
Let
D(k)[i,j]=weight of a shortest path from vi
to vj using only vertices from {v1,v2,,vk}
as intermediate vertices in the path
D(0)=W
D(n)=D which is the goal matrix
How
Floyds Algorithm 95
do we compute D(k) from D(k-1) ?
The Recursive Definition:
Case 1: A shortest path from vi to vj restricted to using only
vertices from {v1,v2,,vk} as intermediate vertices does not use
vk.
Then D(k)[i,j]= D(k-1)[i,j].
Case 2: A shortest path from vi to vj restricted to using only
vertices from {v1,v2,,vk} as intermediate vertices does use vk.
Then D(k)[i,j]= D(k-1)[i,k]+ D(k-1)[k,j].
Vk
Vi
Shortest path using intermediate vertices
{V1, . . . Vk }
Vj
Shortest Path using intermediate vertices { V1, . . . Vk -1 }
The recursive definition
Since
D(k)[i,j]= D(k-1)[i,j] or
D(k)[i,j]= D(k-1)[i,k]+ D(k-1)[k,j].
We conclude:
D(k)[i,j]= min{ D(k-1)[i,j], D(k-1)[i,k]+ D(k-1)[k,j] }.
Shortest path using intermediate vertices
{V1, . . . Vk }
Vk
Vi
Vj
Shortest Path using intermediate vertices { V1, . . . Vk -1 }
The pointer array P
Used to enable finding a shortest path
Initially the array contains 0
Each time that a shorter path from i to j is found the k
that provided the minimum is saved (highest index
node on the path from i to j)
To print the intermediate nodes on the shortest path a
recursive procedure that print the shortest paths from
i and k, and from k to j can be used
Floyd's Algorithm
//Computes shortest distance between all pairs of
//nodes, and saves P to enable finding shortest paths
1. D0 W // initialize D array to W [ ]
2. P 0 // initialize P array to [0]
3. for k 1 to n
4.
do for i 1 to n
5.
do for j 1 to n
6.
if (Dk-1[ i, j ] > Dk-1 [ i, k ] + Dk-1 [ k, j ] )
7.
then Dk[ i, j ] Dk-1 [ i, k ] + Dk-1 [ k, j ]
8.
P[ i, j ] k;
9.
else Dk[ i, j ] Dk-1 [ i, j ]
Floyd//Computes shortest distance between all pairs
of nodes
1. D0 W // initialize D array to W [ ]
2. P 0 // initialize P array to [0]
3. for k 1 to n
4.
do for i 1 to n
5.
do for j 1 to n
6.
Dk[i, j ] min(Dk-1[i, k] +Dk-1[k,j ])
Example
W=D =
0
1
4
-3
3
-3
P=
5
2
-3
D1 =
P=
D =
0
-3
-3
k=1
Vertex 1 can be
intermediate
node
D1[2,3] = min( D0[2,3], D0[2,1]+D0[1,3] )
= min (, 7)
=7
D1[3,2] = min( D0[3,2], D0[3,1]+D0[1,2] )
= min (-3,)
= -3
5
2
-3
D2 =
P=
D1 = 1
-3
-1
-3
k=2
Vertices 1, 2 can
be intermediate
D2[1,3] = min( D1[1,3], D1[1,2]+D1[2,3] )
= min (5, 4+7)
=5
D2[3,1] = min( D1[3,1], D1[3,2]+D1[2,1] )
= min (, -3+2)
= -1
2
-3
D =
3
P=
-1
-3
D2 =
1
3
-1
-3
k=3
Vertices 1, 2, 3
can be
intermediate
D3[1,2] = min(D2[1,2], D2[1,3]+D2[3,2] )
= min (4, 5+(-3))
=2
D3[2,1] = min(D2[2,1], D2[2,3]+D2[3,1] )
= min (2, 7+ (-1))
=2
Apply the Floyds algorithm to the following problem
instance:
0 3
(0)
D = 2 0
7 0 1
6 0
Time Complexity
FLOYD(G)
for i,j in [1..n]
d[i,j]=w(ui,uj)
for k in [1..n]
for i in [1..n]
for j in [1..n]
d[i,j]=min(d[i,j],d[i,k]+d[k,j])
Time complexity: O(n3)
Advantages of Dynamic
programming
A
recursive procedure does not have memory
Dynamic programming stores previous
values to avoid multiple calculations
Space and Time
The
space can be reduced to order of
O(min{m,n})
It is enough to keep only two rows j and j-1
After calculating the entries for row j move
that row up to row j-1 and delete row j-1 and
get the new entries for row j.
The time cannot be reduced
Space reduction
To
compare for a smaller area w, where w is
the window size, the matrix size will reduce.
S1: a b c d a c e
S2: b a d c a b e
3
w
3
w
Space reduction
Specifying
the window size reduces the
number of calculation
The runtime of this algorithm is:
O(2w * min{m,n})
It is a linear time algorithm and the space
required: O(w)
