0% found this document useful (0 votes)
10 views

Dynamic Programming Class

Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
10 views

Dynamic Programming Class

Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 33

Dynamic Programming

7 -1
Dynamic Programming
 Dynamic Programming is an
algorithm design method that can
be used when the solution to a
problem may be viewed as the
result of a sequence of decisions

7 -2
Dynamic Programming (DP)
 Dynamic programming (DP) applies when a problem has both of
these properties:
1. Optimal substructure: “optimal solutions to a problem

incorporate optimal solutions to related subproblems, which we


may solve independently”.
2. Overlapping subproblems: “a recursive algorithm revisits the

same problem repeatedly”.

 Dynamic programming is typically used to:


 Solve optimization problems that have the above properties.
 Solve counting problems –e.g. Matrix Traversal.
 Speed up existing recursive implementations of problems that
have overlapping subproblems (property 2) – e.g. Fibonacci.

3
Iterative or Bottom-Up
Dynamic Programming
• Main type of solution for DP problems
• We can define the problems size and solve problems
from size 0 going up to the size we need.
• Iterative – because it uses a loop
• Bottom-up because you solve problems from the
bottom (the smallest problem size) up to the original
problem size.

4
Bottom-Up vs. Top Down
• There are two versions of dynamic programming.
• Bottom-up.
• Top-down (or memoization).

• Bottom-up:
• Iterative, solves problems in sequence, from smaller to bigger.
• Top-down:
• Recursive, start from the larger problem, solve smaller problems as
needed.
• For any problem that we solve, store the solution, so we never have
to compute the same solution twice.
• This approach is also called memoization.

5
Top-Down Dynamic Programming
( Memoization )

• Maintain an array/table where solutions to problems


can be saved.
• To solve a problem P:
• See if the solution has already been stored in the
array.
• If yes, return the solution.
• Else:
• Issue recursive calls to solve whatever smaller
problems we need to solve.
• Using those solutions obtain the solution to
problem P.
• Store the solution in the solutions array.
• Return the solution.

6
Fibonacci Numbers

7
Fibonacci Numbers
• Generate Fibonacci numbers
– 3 solutions: inefficient recursive, memoization (top-down
dynamic programming (DP)), bottom-up DP.
– Not an optimization problem but it has overlapping
subproblems => DP eliminates recomputing the same
problem over and over again.

8
Fibonacci Numbers
• Fibonacci(0) = 0
• Fibonacci(1) = 1
• If N >= 2:
Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• How can we write a function that computes Fibonacci
numbers?

9
Fibonacci Numbers
• Fibonacci(0) = 0
• Fibonacci(1) = 1
• If N >= 2: Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• Consider this function: what is its running time?

int Fib(int i)
{
if (i < 1) return 0;
if (i == 1) return 1;
return Fib(i-1) + Fib(i-2);
} 10
Fibonacci Numbers
• Fibonacci(0) = 0
• Fibonacci(1) = 1
• If N >= 2: Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• Consider this function: what is its running time?
– g(N) = g(N-1) + g(N-2) + constant
 g(N) ≥ Fibonacci(N) => g(N) = Ω(Fibonacci(N))
 Also g(N) ≤ 2g(N-1)+constant => g(N) ≤ c2N => g(N) = O(2N)
=> g(N) is exponential
– We cannot compute Fibonacci(40) in a reasonable amount of time
(with this implementation). int Fib(int i)
{
– See how many times this function is executed. if (i < 1) return 0;
if (i == 1) return 1;
return Fib(i-1) + Fib(i-2);
– Draw the tree
}
11
Fibonacci sequence

• Fibonacci sequence: 0 , 1 , 1 , 2 , 3 , 5 , 8 , 13 , 21 , …
Fi = i if i  1
Fi = Fi-1 + Fi-2 if i  2
• Solved by a recursive program:
f5

f4 f3

f3 f2 f2 f1

f2 f1 f1 f0 f1 f0

f1 f0

• Much replicated computation is done.


• It should be solved by a simple loop. 7 -12
Fibonacci Numbers
• Fibonacci(0) = 0
• Fibonacci(1) = 1
• If N >= 2: Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• Alternative to inefficient recursion: compute from small to
large and store data in an array.

Notice the mapping/correspondence of the mathematical expression and code.


linear version (Iterative, bottom-up ):
int Fib_iter (int i) {
exponential version:
int F[i+1];
F[0] = 0; F[1] = 1; int Fib(int i) {
int k; if (i < 1) return 0;
for (k = 2; k <= i; k++) F[k] = F[k-1] + F[k-2]; if (i == 1) return 1;
return F[i]; return Fib(i-1) + Fib(i-2);
} }
13
Applied scenario
• F(N) = F(N-1)+F(N-2), F(0) = 0, F(1) = 1,
• Consider a webserver where clients can ask what the value of a
certain Fibonacci number, F(N) is, and the server answers it.
How would you do that? (the back end, not the front end)
(Assume a uniform distribution of F(N) requests over time most F(N) will be asked.)
• Constraints:
– Each loop iteration or function call costs you 1cent.
– Each loop iteration or function call costs the client 0.001seconds wait time
– Memory is cheap
• How would you charge for the service? (flat fee/function calls/loop
iterations?)
• Think of some scenarios of requests that you could get. Think of it
with focus on:
– “good sequence of requests”
– “bad sequence of requests”
14
– Is it clear what good and bad refer to here?

Fibonacci Numbers
Fibonacci(0) = 0 , Fibonacci(1) = 1
• If N >= 2: Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• Alternative: remember values we have already computed.

memoized version:
int Fib_mem_wrap(int i) {
int sol[i+1];
if (i<=1) return i;
sol[0] = 0; sol[1] = 1;
for(int k=2; k<=i; k++) sol[k]=-1;
Fib_mem(i,sol);
return sol[i];
}
exponential version:
int Fib_mem (int i, int[] sol) {
if (sol[i]!=-1) return sol[i]; int Fib(int i) {
int res = Fib_mem(i-1, sol) + Fib_mem(i-2, sol); if (i < 1) return 0;
sol[i] = res; if (i == 1) return 1;
return res; return Fib(i-1) + Fib(i-2);
} } 15
The shortest path
 To find a shortest path in a multi-stage
graph 3 2 7

1 4
S A B 5
T

5 6

 Apply the greedy method :


the shortest path from S to T :
1+2+5=8
7 -16
The shortest path in
multistage graphs
 e.g. A
4
D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F
 The greedy method can not be applied to
this case: (S, A, D, T) 1+4+18 = 23.
 The real shortest path is:
(S, C, F, T) 5+2+2 = 9.

7 -17
Dynamic programming
approach
 Dynamic programming approach (forward
approach):

A
4
D 1 A
1 18 d(A, T)
11 9

2 d(B, T)
S
2
B
5
E
13
T S B T
16 2

5 d(C, T)
5
C 2
F C

 d(S,=
d(A,T) T)min{4+d(D,T),
= min{1+d(A, T), 2+d(B, T),4 5+d(C, T)}
A D
11+d(E,T)} d(D, T)

= min{4+18, 11+13} = 22. T


11
E d(E,
7 T)
-18
 d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}
= min{9+18, 5+13, 16+2} = 18.
4
A D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F
 d(C, T) = min{ 2+d(F, T) } = 2+2 = 4
 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
 The above way of reasoning is called
backward reasoning.

7 -19
Backward approach
(forward reasoning)
4
A D
1 18
11 9

2 5 13
S B E T
16 2
 d(S, A) = 1
5
d(S, B) = 2 C 2
F
d(S, C) = 5
 d(S,D)=min{d(S,A)+d(A,D), d(S,B)+d(B,D)}
= min{ 1+4, 2+9 } = 5
d(S,E)=min{d(S,A)+d(A,E), d(S,B)+d(B,E)}
= min{ 1+11, 2+5 } = 7
d(S,F)=min{d(S,B)+d(B,F), d(S,C)+d(C,F)}
= min{ 2+16, 5+2 } = 7
7 -20
 d(S,T) = min{d(S, D)+d(D, T), d(S,E)+
d(E,T), d(S, F)+d(F, T)}
= min{ 5+18, 7+13, 7+2 }
=9 4
A D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F

7 -21
Principle of optimality
 Principle of optimality: Suppose that in solving a
problem, we have to make a sequence of
decisions D1, D2, …, Dn. If this sequence is
optimal, then the last k decisions, 1  k  n must
be optimal.
 e.g. the shortest path problem
If i, i1, i2, …, j is a shortest path from i to j, then
i1, i2, …, j must be a shortest path from i1 to j
 In summary, if a problem can be described by a
multistage graph, then it can be solved by
dynamic programming.

7 -22
Example 2

7 -23
Example 3
Dynamic programming
 Forward approach and backward
approach:
 Note that if the recurrence relations are
formulated using the forward approach then
the relations are solved backwards . i.e.,
beginning with the last decision
 On the other hand if the relations are
formulated using the backward approach, they
are solved forwards.
 To solve a problem by using dynamic
programming:
 Find out the recurrence relations.
 Represent the problem by a multistage graph.

7 -27
Travelling Salesperson Problem
The traveling salesperson problem

 Figure Directed graph and edge length matrix c


Example 2
 For the given TSP find a tour of minimum cost

A B C D

A 0 16 11 6

B 8 0 13 16

C 4 7 0 9

D 5 12 2 0

7 -33

You might also like