Introduction to

Dynamic Programming

Prof Rajan Kumar

4/8/2024 1
 Approaches for solving DP Problems

Greedy DP Brute Force

- typically not optimal - Optimal solution - Optimal solution
solution (for DP-type - Write math function, sol, that - Produce all possible combinations,
problems) captures the dependency of solution [check if valid], and keep the best.
- Build solution to current pb on solutions to smaller - Time: exponential
- Use a criterion for picking problems - Space: depends on
- Commit to a choice and - Can be implemented in any of the implementation
do not look back following: iterative, memoized,
- It may be hard to generate all
recursive
possible combinations

Iterative (bottom-up) - BEST Memoized Recursive

- Optimal solution - Optimal solution - Optimal solution
- sol is an array (1D or 2D). Size: N+1 - Combines recursion and - Time: exponential
- Fill in sol from 0 to N usage of sol array. (typically) =>
- Time: polynomial (or pseudo- - sol is an array (1D or 2D) - DO NOT USE
polynomial for some problems) - Fill in sol from 0 to n - Space: depends on
- Space: polynomial (or pseudo- - Time: same as iterative implementation (code). E.g.
polynomial version (typically) store all combinations, or
- To recover the choices that gave the - Space: same as iterative generate, evaluate on the fly
optimal answer, must backtrace => version (typically) + space for and keep best seen so far.
must keep picked array (1D or 2D). frame stack. (Frame stack - Easy to code given math
depth is typically smaller function
than the size of the sol array)

Improve space usage DP can solve:

- Improves the iterative solution
- some type of counting problems (e.g. stair climbing)
- Saves space
- If used, cannot recover the choices - some type of optimization problems (e.g. Knapsack)
(gives the optimal value, but not the - some type of recursively defined pbs (e.g. Fibonacci)
choices)
SOME DP solutions have pseudo polynomial time
 Dynamic Programming (DP) - CLRS
• Dynamic programming (DP) applies when a problem has
both of these properties:
1. Optimal substructure: “optimal solutions to a problem
incorporate optimal solutions to related subproblems, which we
may solve independently”.
2. Overlapping subproblems: “a recursive algorithm revisits the
same problem repeatedly”.
• Dynamic programming is typically used to:
• Solve optimization problems that have the above properties.
• Solve counting problems –e.g. Stair Climbing or Matrix Traversal.
• Speed up existing recursive implementations of problems that
have overlapping subproblems (property 2) – e.g. Fibonacci.
• Compare dynamic programming with divide and conquer.
3
Iterative or Bottom-Up
Dynamic Programming
• Main type of solution for DP problems
• We can define the problems size and solve problems from size 0
going up to the size we need.
• Iterative – because it uses a loop
• Bottom-up because you solve problems from the bottom (the
smallest problem size) up to the original problem size.

4
Bottom-Up vs. Top Down
• There are two versions of dynamic programming.
• Bottom-up. (tabular approaches)
• Top-down (or memorization).

• Bottom-up:
• Iterative, solves problems in sequence, from smaller to bigger.
• Top-down:
• Recursive, start from the larger problem, solve smaller problems as needed.
• For any problem that we solve, store the solution, so we never have to compute the same
solution twice.
• This approach is also called memoization.

5
Top-Down Dynamic Programming
( Memoization )
• Maintain an array/table where solutions to problems
can be saved.
• To solve a problem P:
• See if the solution has already been stored in the array.
• If yes, return the solution.
• Else:
• Issue recursive calls to solve whatever smaller problems we need
to solve.
• Using those solutions obtain the solution to problem P.
• Store the solution in the solutions array.
• Return the solution.

6
Example on Fibonacci series calculation
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55,…………..
Number 0 1 1 3 5 8 13
Array index 0 1 2 3 4 5 6

{
0 if n=0

Fib(n) = 1 if n=1

Fib(n-2)+ Fib(n-1) if n>1

Example on Fibonacci series calculation
If somebody wants to find Fib(5), then we can do using the dynamic
programming

Fib(5)

Fib(3) Fib(4)

Fib(2) Fib(3)
Fib(1) Fib(2)

Fib(0) Fib(1) Fib(0) Fib(1) Fib(1) Fib(2)

Fib(0) Fib(1)
Example on Fibonacci series calculation
If somebody wants to find Fib(5), then we can do using the dynamic
programming

Pseudo code for this problem

Int fib(int n)
{
If (n<=1)
return n;
Else
Return fib(n-2)+fib(n-1);
}
0/1 Knapsack problem
The 0/1 knapsack problem is a classic optimization problem in
computer science and combinatorial optimization. It's called "0/1"
because for each item, you can either include it in the knapsack
(take it) or exclude it (leave it), but you cannot take a fraction of an
item.

You are given a set of items, each with a weight Wi and a

value vi, and a knapsack that can carry a maximum weight
W. The goal is to maximize the total value of items you can
carry in the knapsack without exceeding its weight capacity.
0/1 Knapsack problem
Given data
Total weight limit = 7
Item Price Weight
A 1 1
B 4 3
C 5 4
D 7 5
0/1 Knapsack problem
Given data
Total weight limit = 7
Weight
Item Price Weight 0 1 2 3 4 5 6 7

0 0 0 0 0 0 0 0 0 0 0
A 1 1
B 4 3
C 5 4
D 7 5
0/1 Knapsack problem solution
Given data
Total weight limit = 7
Weight
Item Price Weight 0 1 2 3 4 5 6 7

0 0 0 0 0 0 0 0 0 0 0
A 1 1 0 1 1 1 1 1 1 1
B 4 3 0 1 1 4 5 5 5 5
C 5 4 0 1 1 4 5 6 6 9
D 7 5 0 1 1 4 5 7 8 9
0/1 Knapsack problem practice problem
Given data
Total weight limit = 8
Weight
Item Price Weight 0 1 2 3 4 5 6 7 8

A 2 3
B 3 4
C 4 5
D 1 6
0/1 Knapsack problem practice problem solution
Given data, Total weight limit = 8
Weight(w)
Item Price Weight i 0 1 2 3 4 5 6 7 8

0 0 0 0
A 2 3 1
B 3 4 2
C 4 5 3
D 1 6 4
0/1 Knapsack problem practice problem solution
Given data, Total weight limit = 8
Weight(w)
Item Price Weight i 0 1 2 3 4 5 6 7 8

0 0 0 0 0 0 0 0 0 0 0 0 0
A 2 3 1 0 0 0 2 2 2 2 2 2
B 3 4 2 0 0 0 2 3 3 3 5 5
C 4 5 3 0 0 0 2 3 4 4 5 6
D 1 6 4 0 0 0 2 3 4 4 5 6

m(i, w) = Max {(m[i-1, w], m[i-1, w-w(i)]) + p(i)}

Multistage graph using dynamic programming

2 6 9

1 3 7 10 12
End
start
4

5 8 11
Multistage graph using dynamic programming
L-2 L-3 L-4
L-1 L-5
4 6
2 6 9
9 2 2 5 4
4

7 3 2
1 3 7 7 10 12
3 11 End
start

2 4 11 5
1 5

6
8
5 8 11
Multistage graph using dynamic programming
Vertex 1 2 3 4 5 6 7 8 9 10 11 12
Cost 0
d 12

L-1 L-2 L-3 L-4 L-5

2 4 6 6 9
9 2 2 5 4
4
1 1
1 7 3 7 3 2
7 0 2
3 11 End
start
4
2 11
1 5
6 1
5 8 8
1
Multistage graph using dynamic programming
Vertex 1 2 3 4 5 6 7 8 9 10 11 12
Cost 4 2 5 0
d 12 12 12 12

L-1 L-2 L-3 L-4 L-5

2 4 6 9
6

9 2 2 5 4
4

1 7 3 7 3 10 2 12
7
3 11 End
start
4
2 11
1 5

6
5 8 8 11

Cost of 4th stage 9th node = C(4th stage, 9 node)= C(4,9) = 4

Level-4 Cost of 4th stage 10th node = C(4th stage, 10 node)= C(4,10) = 2
Cost of 4th stage 11th node = C(4th stage, 11 node)= C(4,11) = 5
Multistage graph using dynamic programming
Vertex 1 2 3 4 5 6 7 8 9 10 11 12
Cost 7 5 7 4 2 5 0
d 10 10 10 12 12 12 12

L-1 L-2 L-3 L-4 L-5

2 4 6 9
6

9 2 2 5 4
4

1 7 3 7 3 10 2 12
7
3 11 End
start
4
2 11
1 5

6
Level-3 5 8 8 11

Cost of 3rd stage 6th node = C(3rd stage, 6 node)= min {[C(6,9) + C(4th,9)], [C(6,10)+C(4th ,10)] = min {6+4, 5+2} = 7
Cost of 3rd stage 7th node = C(3rd stage, 7 node)= min {[C(7,9) + C(4th,9)], [C(7,10)+C(4th ,10)] = min {4+4, 3+2} = 5
Cost of 3rd stage 8th node = C(4th stage, 8 node)= min {[C(8,10) + C(4th,10)], [C(8,11)+C(4th ,11)] = min {5+2, 6+5} =7
Multistage graph using dynamic programming

Cost (i, j) = min { cost(j, l) +cost (i+1,l)}

J, l belongs to vertices
Stage or level l belongs to vertice in the next level connected to j

Vertex no.
Multistage graph using dynamic programming
Vertex 1 2 3 4 5 6 7 8 9 10 11 12
Cost 7 5 7 4 2 5 0
d 10 10 10 12 12 12 12

L-1 L-2 L-3 L-4 L-5

2 4 6 9
6

9 2 2 5 4
4

1 7 3 7 3 10 2 12
7
3 11 End
start
4
2 11
1 5

6
Level-3 5 8 8 11

Cost of 3rd stage 6th node = C(3rd stage, 6 node)= min {[C(6,9) + C(4th,9)], [C(6,10)+C(4th ,10)] = min {6+4, 5+2} = 7
Cost of 3rd stage 7th node = C(3rd stage, 7 node)= min {[C(7,9) + C(4th,9)], [C(7,10)+C(4th ,10)] = min {4+4, 3+2} = 5
Cost of 3rd stage 8th node = C(4th stage, 8 node)= min {[C(8,10) + C(4th,10)], [C(8,11)+C(4th ,11)] = min {5+2, 6+5} =7
Multistage graph using dynamic programming
Vertex 1 2 3 4 5 6 7 8 9 10 11 12
Cost 16 7 9 18 15 7 5 7 4 2 5 0
d 2 or 3 7 6 8 8 10 10 10 12 12 12 12

L-1 L-2 L-3 L-4 L-5

2 4 6 9
6

9 2 2 5 4
4

1 7 3 7 3 10 2 12
7
3 11 End
start
4
2 11
1 5

6
Level-3 5 8 8 11

Cost of 3rd stage 6th node = C(3rd stage, 6 node)= min {[C(6,9) + C(4th,9)], [C(6,10)+C(4th ,10)] = min {6+4, 5+2} = 7
Cost of 3rd stage 7th node = C(3rd stage, 7 node)= min {[C(7,9) + C(4th,9)], [C(7,10)+C(4th ,10)] = min {4+4, 3+2} = 5
Cost of 3rd stage 8th node = C(4th stage, 8 node)= min {[C(8,10) + C(4th,10)], [C(8,11)+C(4th ,11)] = min {5+2, 6+5} =7