Dynamic Programming

Dynamic programming is a name, coined by Richard Bellman in 1955. Dynamic

programming, as greedy method, is a powerful algorithm design technique that can
be used when the solution to the problem may be viewed as the result of a sequence
of decisions. In the greedy method we make irrevocable decisions one at a time,
using a greedy criterion. However, in dynamic programming we examine the decision
sequence to see whether an optimal decision sequence contains optimal decision
subsequence.

When optimal decision sequences contain optimal decision subsequences, we can

establish recurrence equations, called dynamic-programming recurrence equations,
that enable us to solve the problem in an efficient way.

Dynamic programming is based on the principle of optimality (also coined by

Bellman). The principle of optimality states that no matter whatever the initial state
and initial decision are, the remaining decision sequence must constitute an optimal
decision sequence with regard to the state resulting from the first decision. The
principle implies that an optimal decision sequence is comprised of optimal decision
subsequences. Since the principle of optimality may not hold for some formulations
of some problems, it is necessary to verify that it does hold for the problem being
solved. Dynamic programming cannot be applied when this principle does not hold.

The steps in a dynamic programming solution are:

 Verify that the principle of optimality holds

 Set up the dynamic-programming recurrence equations

 Solve the dynamic-programming recurrence equations for the value of the

optimal solution.

 Perform a trace back step in which the solution itself is constructed.

Dynamic programming differs from the greedy method since the greedy method
produces only one feasible solution, which may or may not be optimal, while dynamic
programming produces all possible sub-problems at most once, one of which
guaranteed to be optimal. Optimal solutions to sub-problems are retained in a table,
thereby avoiding the work of recomputing the answer every time a sub-problem is
encountered

The divide and conquer principle solve a large problem, by breaking it up into smaller
problems which can be solved independently. In dynamic programming this principle
is carried to an extreme: when we don't know exactly which smaller problems to
solve, we simply solve them all, then store the answers away in a table to be used
later in solving larger problems. Care is to be taken to avoid recomputing previously
computed values, otherwise the recursive program will have prohibitive complexity.
In some cases, the solution can be improved and in other cases, the dynamic
programming technique is the best approach.
Two difficulties may arise in any application of dynamic programming:
1. It may not always be possible to combine the solutions of smaller problems to
form the solution of a larger one.
2. The number of small problems to solve may be un-acceptably large.
There is no characterized precisely which problems can be effectively solved with
dynamic programming; there are many hard problems for which it does not seen to
be applicable, as well as many easy problems for which it is less efficient than
standard algorithms.

MULTI STAGE GRAPHS:

A multistage graph G = (V, E) is a directed graph in which the vertices are

partitioned into k > 2 disjoint sets Vi, 1 < i < k. In addition, if <u, v> is an edge in E,
then u  Vi and v  Vi+1 for some i, 1 < i < k.

Let the vertex ‘s’ is the source, and ‘t’ the sink. Let c (i, j) be the cost of edge <i, j>.
The cost of a path from ‘s’ to ‘t’ is the sum of the costs of the edges on the path. The
multistage graph problem is to find a minimum cost path from ‘s’ to ‘t’. Each set V i
defines a stage in the graph. Because of the constraints on E, every path from ‘s’ to
‘t’ starts in stage 1, goes to stage 2, then to stage 3, then to stage 4, and so on, and
eventually terminates in stage k.

A dynamic programming formulation for a k-stage graph problem is obtained by first

noticing that every s to t path is the result of a sequence of k – 2 decisions. The ith
decision involves determining which vertex in v i+1, 1 < i < k - 2, is to be on the
path. Let c (i, j) be the cost of the path from source to destination. Then using the
forward approach, we obtain:

cost (i, j) = min {c (j, l) + cost (i + 1, l)}

l  Vi + 1
<j, l>  E

ALGORITHM:

Algorithm Fgraph (G, k, n, p)

// The input is a k-stage graph G = (V, E) with n vertices
// indexed in order or stages. E is a set of edges and c [i, j]
// is the cost of (i, j). p [1 : k] is a minimum cost path.
{
cost [n] := 0.0;
for j:= n - 1 to 1 step – 1 do
{ // compute cost [j]
let r be a vertex such that (j, r) is an edge
of G and c [j, r] + cost [r] is minimum;
cost [j] := c [j, r] + cost [r];
d [j] := r:
}
p [1] := 1; p [k] := n; // Find a minimum cost path.
for j := 2 to k - 1 do p [j] := d [p [j - 1]];
}
The multistage graph problem can also be solved using the backward approach.
Let bp(i, j) be a minimum cost path from vertex s to j vertex in V i. Let Bcost(i, j) be
the cost of bp(i, j). From the backward approach we obtain:
Bcost (i, j) = min { Bcost (i –1, l) + c (l,
j)} l  Vi - 1
<l, j>  E

Algorithm Bgraph (G, k, n, p)

// Same function as Fgraph
{
Bcost [1] := 0.0;
for j := 2 to n do
{ // Compute Bcost [j].
Let r be such that (r, j) is an edge of
G and Bcost [r] + c [r, j] is minimum;
Bcost [j] := Bcost [r] + c [r, j];
D [j] := r;
} //find a minimum cost path
p [1] := 1; p [k] := n;
for j:= k - 1 to 2 do p [j] := d [p [j + 1]];
}

Complexity Analysis:

The complexity analysis of the algorithm is fairly straightforward. Here, if G has E
edges, then the time for the first for loop is  (V +E).

EXAMPLE:

Find the minimum cost path from s to t in the multistage graph of five stages shown
below. Do this first using forward approach and then using backward approach.

s 0 2 t

4 11

1
11

FORWARD APPROACH:

We use the following equation to find the minimum cost path from s to t:

cost (i, j) = min {c (j, l) + cost (i + 1, l)}

l  Vi + 1
<j, l>  E
cost (1, 1) = min {c (1, 2) + cost (2, 2), c (1, 3) + cost (2, 3), c (1, 4) + cost (2, 4),
c (1, 5) + cost (2, 5)}
= min {9 + cost (2, 2), 7 + cost (2, 3), 3 + cost (2, 4), 2 + cost (2, 5)}
Now first starting with,

cost (2, 2) = min{c (2, 6) + cost (3, 6), c (2, 7) + cost (3, 7), c (2, 8) + cost (3, 8)}
= min {4 + cost (3, 6), 2 + cost (3, 7), 1 + cost (3, 8)}

cost (3, 6) = min {c (6, 9) + cost (4, 9), c (6, 10) + cost (4, 10)}
= min {6 + cost (4, 9), 5 + cost (4, 10)}

cost (4, 9) = min {c (9, 12) + cost (5, 12)} = min {4 + 0) = 4

cost (4, 10) = min {c (10, 12) + cost (5, 12)} = 2

Therefore, cost (3, 6) = min {6 + 4, 5 + 2} = 7

cost (3, 7) = min {c (7, 9) + cost (4, 9) , c (7, 10) + cost (4, 10)}
= min {4 + cost (4, 9), 3 + cost (4, 10)}

cost (4, 9) = min {c (9, 12) + cost (5, 12)} = min {4 + 0} = 4

Cost (4, 10) = min {c (10, 2) + cost (5, 12)} = min {2 + 0} = 2

Therefore, cost (3, 7) = min {4 + 4, 3 + 2} = min {8, 5} = 5

cost (3, 8) = min {c (8, 10) + cost (4, 10), c (8, 11) + cost (4, 11)}
= min {5 + cost (4, 10), 6 + cost (4 + 11)}

cost (4, 11) = min {c (11, 12) + cost (5, 12)} = 5

Therefore, cost (3, 8) = min {5 + 2, 6 + 5} = min {7, 11} = 7

Therefore, cost (2, 2) = min {4 + 7, 2 + 5, 1 + 7} = min {11, 7, 8} = 7

Therefore, cost (2, 3) = min {c (3, 6) + cost (3, 6), c (3, 7) + cost (3, 7)}
= min {2 + cost (3, 6), 7 + cost (3, 7)}
= min {2 + 7, 7 + 5} = min {9, 12} = 9

cost (2, 4) = min {c (4, 8) + cost (3, 8)} = min {11 + 7} = 18

cost (2, 5) = min {c (5, 7) + cost (3, 7), c (5, 8) + cost (3, 8)}
= min {11 + 5, 8 + 7} = min {16, 15} = 15

Therefore, cost (1, 1) = min {9 + 7, 7 + 9, 3 + 18, 2 + 15}

= min {16, 16, 21, 17} = 16

The minimum cost path is 16.

The path is 1 2 7 10 12

or
1 3 6 10 12

BACKWARD APPROACH:

We use the following equation to find the minimum cost path from t to s:

Bcost (i, J) = min {Bcost (i – 1, l) + c (l, J)}

l  vi – 1
<l, j>  E

Bcost (5, 12) = min {Bcost (4, 9) + c (9, 12), Bcost (4, 10) + c (10, 12),
Bcost (4, 11) + c (11, 12)}
= min {Bcost (4, 9) + 4, Bcost (4, 10) + 2, Bcost (4, 11) + 5}

Bcost (4, 9) = min {Bcost (3, 6) + c (6, 9), Bcost (3, 7) + c (7,9)}
= min {Bcost (3, 6) + 6, Bcost (3, 7) + 4}

Bcost (3, 6) = min {Bcost (2, 2) + c (2, 6), Bcost (2, 3) + c (3,6)}
= min {Bcost (2, 2) + 4, Bcost (2, 3) + 2}

Bcost (2, 2) = min {Bcost (1, 1) + c (1, 2)} = min {0 + 9} = 9

Bcost (2, 3) = min {Bcost (1, 1) + c (1, 3)} = min {0 + 7} = 7

Bcost (3, 6) = min {9 + 4, 7 + 2} = min {13, 9} =9

Bcost (3, 7) = min {Bcost (2, 2) + c (2, 7), Bcost (2, 3) + c (3, 7),
Bcost (2, 5) + c (5, 7)}

Bcost (2, 5) = min {Bcost (1, 1) + c (1, 5)} = 2

Bcost (3, 7) = min {9 + 2, 7 + 7, 2 + 11} = min {11, 14, 13} =11

Bcost (4, 9) = min {9 + 6, 11 + 4} = min {15, 15} = 15

Bcost (4, 10) = min {Bcost (3, 6) + c (6, 10), Bcost (3, 7) + c (7,10),
Bcost (3, 8) + c (8, 10)}

Bcost (3, 8) = min {Bcost (2, 2) + c (2, 8), Bcost (2, 4) + c (4, 8),
Bcost (2, 5) + c (5, 8)}
Bcost (2, 4) = min {Bcost (1, 1) + c (1, 4)} = 3

Bcost (3, 8) = min {9 + 1, 3 + 11, 2 + 8} = min {10, 14, 10} =10

Bcost (4, 10) = min {9 + 5, 11 + 3, 10 + 5} = min {14, 14, 15) = 14

Bcost (4, 11) = min {Bcost (3, 8) + c (8, 11)} = min {Bcost (3, 8) + 6}
= min {10 + 6} = 16
Bcost (5, 12) = min {15 + 4, 14 + 2, 16 + 5} = min {19, 16, 21} = 16.

EXAMPLE 2:

Find the minimum cost path from s to t in the multistage graph of five stages shown
below. Do this first using forward approach and then using backward approach.

2 4 7

s 1 5 2 t

SOLUTION:

FORWARD APPROACH:

cost (i, J) = min {c (j, l) + cost (i + 1, l)}

l  Vi + 1
<J, l> E

cost (1, 1) = min {c (1, 2) + cost (2, 2), c (1, 3) + cost (2, 3)}
= min {5 + cost (2, 2), 2 + cost (2, 3)}

cost (2, 2) = min {c (2, 4) + cost (3, 4), c (2, 6) + cost (3, 6)}
= min {3+ cost (3, 4), 3 + cost (3, 6)}

cost (3, 4) = min {c (4, 7) + cost (4, 7), c (4, 8) + cost (4, 8)}
= min {(1 + cost (4, 7), 4 + cost (4, 8)}

cost (4, 7) = min {c (7, 9) + cost (5, 9)} = min {7 + 0) = 7

cost (4, 8) = min {c (8, 9) + cost (5, 9)} = 3

Therefore, cost (3, 4) = min {8, 7} = 7

cost (3, 6) = min {c (6, 7) + cost (4, 7), c (6, 8) + cost (4, 8)}
= min {6 + cost (4, 7), 2 + cost (4, 8)} = min {6 + 7, 2 + 3} =5

Therefore, cost (2, 2) = min {10, 8} = 8

cost (2, 3) = min {c (3, 4) + cost (3, 4), c (3, 5) + cost (3, 5), c (3, 6) + cost
(3,6)}

cost (3, 5) = min {c (5, 7) + cost (4, 7), c (5, 8) + cost (4, 8)}= min {6 + 7, 2 + 3}
=5

Therefore, cost (2, 3) = min {13, 10, 13} = 10

cost (1, 1) = min {5 + 8, 2 + 10} = min {13, 12} = 12

 BACKWARD APPROACH:

Bcost (i, J) = min {Bcost (i – 1, l) = c (l, J)}

l  vi – 1
<l ,j> E

Bcost (5, 9) = min {Bcost (4, 7) + c (7, 9), Bcost (4, 8) + c (8, 9)}
= min {Bcost (4, 7) + 7, Bcost (4, 8) + 3}

Bcost (4, 7) = min {Bcost (3, 4) + c (4, 7), Bcost (3, 5) + c (5, 7),
Bcost (3, 6) + c (6, 7)}
= min {Bcost (3, 4) + 1, Bcost (3, 5) + 6, Bcost (3, 6) + 6}

Bcost (3, 4) = min {Bcost (2, 2) + c (2, 4), Bcost (2, 3) + c (3, 4)}
= min {Bcost (2, 2) + 3, Bcost (2, 3) + 6}

Bcost (2, 2) = min {Bcost (1, 1) + c (1, 2)} = min {0 + 5} = 5

Bcost (2, 3) = min (Bcost (1, 1) + c (1, 3)} = min {0 + 2} = 2

Therefore, Bcost (3, 4) = min {5 + 3, 2 + 6} = min {8, 8} = 8

Bcost (3, 5) = min {Bcost (2, 3) + c (3, 5)} = min {2 + 5} = 7

Bcost (3, 6) = min {Bcost (2, 2) + c (2, 6), Bcost (2, 3) + c (3, 6)}
= min {5 + 5, 2 + 8} = 10

Therefore, Bcost (4, 7) = min {8 + 1, 7 + 6, 10 + 6} = 9

Bcost (4, 8) = min {Bcost (3, 4) + c (4, 8), Bcost (3, 5) + c (5, 8),
Bcost (3, 6) + c (6, 8)}
= min {8 + 4, 7 + 2, 10 + 2} = 9

Therefore, Bcost (5, 9) = min {9 + 7, 9 + 3} = 12

All pairs shortest paths:

In the all pairs shortest path problem, we are to find a shortest path between every
pair of vertices in a directed graph G. That is, for every pair of vertices (i, j), we are
to find a shortest path from i to j as well as one from j to i. These two paths are the
same when G is undirected.

When no edge has a negative length, the all-pairs shortest path problem may be
solved by using Dijkstra’s greedy single source algorithm n times, once with each of
the n vertices as the source vertex.

The all pairs shortest path problem is to determine a matrix A such that A (i, j) is the
length of a shortest path from i to j. The matrix A can be obtained by solving n
single-source problems using the algorithm shortest Paths. Since each application of
this procedure requires O (n2) time, the matrix A can be obtained in O (n3) time.
 The dynamic programming solution, called Floyd’s algorithm, runs in O (n3) time.
Floyd’s algorithm works even when the graph has negative length edges (provided
there are no negative length cycles).

The shortest i to j path in G, i ≠ j originates at vertex i and goes through some

intermediate vertices (possibly none) and terminates at vertex j. If k is an
intermediate vertex on this shortest path, then the subpaths from i to k and from k
to j must be shortest paths from i to k and k to j, respectively. Otherwise, the i to j
path is not of minimum length. So, the principle of optimality holds. Let A k (i, j)
represent the length of a shortest path from i to j going through no vertex of index
greater than k, we obtain:

Ak (i, j) = {min {min {Ak-1 (i, k) + Ak-1 (k, j)}, c (i, j)}
1<k<n

Algorithm All Paths (Cost, A, n)

// cost [1:n, 1:n] is the cost adjacency matrix of a graph which
// n vertices; A [I, j] is the cost of a shortest path from vertex
// i to vertex j. cost [i, i] = 0.0, for 1 < i < n.
{
for i := 1 to n do
for j:= 1 to n do
A [i, j] := cost [i, j]; // copy cost into A.
for k := 1 to n do
for i := 1 to n do
for j := 1 to n do
A [i, j] := min (A [i, j], A [i, k] + A [k, j]);
}

Complexity Analysis: A Dynamic programming algorithm based on this recurrence

involves in calculating n+1 matrices, each of size n x n. Therefore, the algorithm has
a complexity of O (n3).

Example 1:

Given a weighted digraph G = (V, E) with weight. Determine the length of the
shortest path between all pairs of vertices in G. Here we assume that there are no
cycles with zero or negative cost.

6
1 2 0 4 11
4
0  
1 1 2
Cost adjacency matrix (A ) = 6 0 2 
3
3  0 



General formula: min {Ak-1 (i, k) + Ak-1 (k, j)}, c (i, j)}
1<k<n

Solve the problem for different values of k = 1, 2 and 3

Step 1: Solving the equation for, k = 1;

A1 (1, 1) = min {(Ao (1, 1) + Ao (1, 1)), c (1, 1)} = min {0 + 0, 0} = 0
A1 (1, 2) = min {(Ao (1, 1) + Ao (1, 2)), c (1, 2)} = min {(0 + 4), 4} = 4
A1 (1, 3) = min {(Ao (1, 1) + Ao (1, 3)), c (1, 3)} = min {(0 + 11), 11} = 11
A1 (2, 1) = min {(Ao (2, 1) + Ao (1, 1)), c (2, 1)} = min {(6 + 0), 6} = 6
A1 (2, 2) = min {(Ao (2, 1) + Ao (1, 2)), c (2, 2)} = min {(6 + 4), 0)} = 0
A1 (2, 3) = min {(Ao (2, 1) + Ao (1, 3)), c (2, 3)} = min {(6 + 11), 2} =2
A1 (3, 1) = min {(Ao (3, 1) + Ao (1, 1)), c (3, 1)} = min {(3 + 0), 3} = 3
A1 (3, 2) = min {(Ao (3, 1) + Ao (1, 2)), c (3, 2)} = min {(3 + 4), } = 7
A1 (3, 3) = min {(Ao (3, 1) + Ao (1, 3)), c (3, 3)} = min {(3 + 11), 0} = 0

0 4 11
 
A(1) = 2 
6 0
3 7 0 

Step 2: Solving the equation for, K = 2;

A2 (1, 1) = min {(A1(1, 2) + A1 (2, 1), c (1, 1)} = min {(4 + 6), 0} = 0
A2 (1, 2) = min {(A1(1, 2) + A1 (2, 2), c (1, 2)} = min {(4 + 0), 4} = 4
A2 (1, 3) = min {(A1 (1, 2) + A1 (2, 3), c (1, 3)} = min {(4 + 2), 11} = 6
A2 (2, 1) = min {(A (2, 2) + A (2, 1), c (2, 1)} = min {(0 + 6), 6} = 6
A2 (2, 2) = min {(A (2, 2) + A (2, 2), c (2, 2)} = min {(0 + 0), 0} = 0
A2 (2, 3) = min {(A (2, 2) + A (2, 3), c (2, 3)} = min {(0 + 2), 2} = 2
A2 (3, 1) = min {(A (3, 2) + A (2, 1), c (3, 1)} = min {(7 + 6), 3} = 3
A2 (3, 2) = min {(A (3, 2) + A (2, 2), c (3, 2)} = min {(7 + 0), 7} = 7
A2 (3, 3) = min {(A (3, 2) + A (2, 3), c (3, 3)} = min {(7 + 2), 0} = 0

0 4 6 
 
A(2) = 2 
6 0
3 7 0 

Step 3: Solving the equation for, k = 3;

A3 (1, 1) = min {A2 (1, 3) + A2 (3, 1), c (1, 1)} = min {(6 +3), 0} = 0
A3 (1, 2) = min {A2 (1, 3) + A2 (3, 2), c (1, 2)} = min {(6 +7), 4} = 4
A3 (1, 3) = min {A2 (1, 3) + A2 (3, 3), c (1, 3)} = min {(6 +0), 6} = 6
A3 (2, 1) = min {A2 (2, 3) + A2 (3, 1), c (2, 1)} = min {(2 +3), 6} = 5
A3 (2, 2) = min {A2 (2, 3) + A2 (3, 2), c (2, 2)} = min {(2 +7), 0} = 0
A3 (2, 3) = min {A2 (2, 3) + A2 (3, 3), c (2, 3)} = min {(2 +0), 2} = 2
A3 (3, 1) = min {A2 (3, 3) + A2 (3, 1), c (3, 1)} = min {(0 +3), 3} = 3
A3 (3, 2) = min {A2 (3, 3) + A2 (3, 2), c (3, 2)} = min {(0 +7), 7} = 7
 A3 (3, 3) = min {A2 (3, 3) + A2 (3, 3), c (3, 3)} = min {(0 + 0), 0} = 0

0 4 6 
 
A(3) = 2 
5 0 
3 7 0 

TRAVELLING SALESPERSON PROBLEM:

Let G = (V, E) be a directed graph with edge costs Cij. The variable cij is defined such
that cij > 0 for all I and j and cij =  if < i, j>  E. Let |V| = n and assume n > 1. A
tour of G is a directed simple cycle that includes every vertex in V. The cost of a tour
is the sum of the cost of the edges on the tour. The traveling sales person problem is
to find a tour of minimum cost. The tour is to be a simple path that starts and ends
at vertex 1.

Let g (i, S) be the length of shortest path starting at vertex i, going through all
vertices in S, and terminating at vertex 1. The function g (1, V – {1}) is the length of
an optimal salesperson tour. From the principal of optimality it follows that:

g1, V -  1  min  c1k  g  k, V   1, k -- 1

2kn

Generalizing equation 1, we obtain (for i  S)

g i, S   minci j  g i, S  j  -- 2

j S

The Equation can be solved for g (1, V – 1}) if we know g (k, V – {1, k}) for all
choices of k.

Example :

For the following graph find minimum cost tour for the traveling salesperson
problem:

0 20
5 10 15 
The cost adjacency matrix =  10 
0 9 
6 12
13 0 

8 8 9 0 
Let us start the tour from vertex 1:

g (1, V – {1}) = min {c1k + g (k, V – {1, K})} - (1)

2<k<n

More generally writing:

g (i, s) = min {cij + g (J, s – {J})} - (2)

Clearly, g (i, ) = ci1 , 1 ≤ i ≤ n. So,

g (2, ) = C21 = 5

g (3, ) = C31 = 6

g (4, ) = C41 = 8

Using equation – (2) we obtain:

g (1, {2, 3, 4}) = min {c12 + g (2, {3, 4}, c13 + g (3, {2, 4}), c14 + g (4, {2, 3})}

g (2, {3, 4}) = min {c23 + g (3, {4}), c24 + g (4, {3})}
= min {9 + g (3, {4}), 10 + g (4, {3})}

g (3, {4}) = min {c34 + g (4, )} = 12 + 8 = 20

g (4, {3}) = min {c43 + g (3, )} = 9 + 6 = 15

Therefore, g (2, {3, 4}) = min {9 + 20, 10 + 15} = min {29, 25} = 25

g (3, {2, 4}) = min {(c32 + g (2, {4}), (c34 + g (4,{2})}

g (2, {4}) = min {c24 + g (4, )} = 10 + 8 = 18

g (4, {2}) = min {c42 + g (2, )} = 8 + 5 = 13

Therefore, g (3, {2, 4}) = min {13 + 18, 12 + 13} = min {41, 25} = 25

g (4, {2, 3}) = min {c42 + g (2, {3}), c43 + g (3, {2})}

g (2, {3}) = min {c23 + g (3, } = 9 + 6 = 15

g (3, {2}) = min {c32 + g (2, } = 13 + 5 = 18

Therefore, g (4, {2, 3}) = min {8 + 15, 9 + 18} = min {23, 27} =23

g (1, {2, 3, 4}) = min {c12 + g (2, {3, 4}), c13 + g (3, {2, 4}), c14 + g (4, {2, 3})}
= min {10 + 25, 15 + 25, 20 + 23} = min {35, 40, 43} = 35

The optimal tour for the graph has length = 35

The optimal tour is: 1, 2, 4, 3, 1.

0/1 – KNAPSACK:

We are given n objects and a knapsack. Each object i has a positive weight w i and a
positive value Vi. The knapsack can carry a weight not exceeding W. Fill the knapsack
so that the value of objects in the knapsack is optimized.

A solution to the knapsack problem can be obtained by making a sequence of

decisions on the variables x1, x2, . . . . , xn. A decision on variable xi involves
determining which of the values 0 or 1 is to be assigned to it. Let us assume that

decisions on the xi are made in the order xn, xn-1, x1. Following a decision on xn,
we may be in one of two possible states: the capacity remaining in m – wn and a
profit of pn has accrued. It is clear that the remaining decisions xn-1, , x1 must be
optimal with respect to the problem state resulting from the decision on xn.
Otherwise, xn, , x1 will not be optimal. Hence, the principal of optimality holds.

Fn (m) = max {fn-1 (m), fn-1 (m - wn) + pn} -- 1

For arbitrary fi (y), i > 0, this equation generalizes to:

Fi (y) = max {fi-1 (y), fi-1 (y - wi) + pi} -- 2

Equation-2 can be solved for fn (m) by beginning with the knowledge fo (y) = 0 for all
y and fi (y) = - , y < 0. Then f1, f2, . . . fn can be successively computed using
equation–2.

When the wi’s are integer, we need to compute fi (y) for integer y, 0 < y < m. Since fi
(y) = -  for y < 0, these function values need not be computed explicitly. Since
each fi can be computed from fi - 1 in Θ (m) time, it takes Θ (m n) time to compute
fn. When the wi’s are real numbers, fi (y) is needed for real numbers y such that 0 <
y < m. So, fi cannot be explicitly computed for all y in this range. Even when the w i’s
are integer, the explicit Θ (m n) computation of fn may not be the most efficient
computation. So, we explore an alternative method for both cases.
The fi (y) is an ascending step function; i.e., there are a finite number of y’s, 0 = y1
< y2 < . . . . < yk, such that fi (y1) < fi (y2) < . . . . . < fi (yk); fi (y) = -  , y < y1; fi
(y) = f (yk), y > yk; and fi (y) = fi (yj), yj < y < yj+1. So, we need to compute only fi
(yj), 1 < j < k. We use the ordered set Si = {(f (yj), yj) | 1 < j < k} to represent fi
(y). Each number of Si is a pair (P, W), where P = fi (yj) and W = yj. Notice that S0 =
{(0, 0)}. We can compute Si+1 from Si by first computing:

Si1= {(P, W) | (P – p ,i W – w i)  Si}

Now, Si+1 can be computed by merging the pairs in Si and Si to1gether. Note that if
Si+1 contains two pairs (Pj, Wj) and (Pk, Wk) with the property that Pj < Pk and Wj >
Wk, then the pair (Pj, Wj) can be discarded because of equation-2. Discarding or
purging rules such as this one are also known as dominance rules. Dominated tuples
get purged. In the above, (Pk, Wk) dominates (Pj, Wj).

Example 1:

Consider the knapsack instance n = 3, (w1, w2, w3) = (2, 3, 4), (P1, P2, P3) = (1,2,
5) and M = 6.
Solution:

Initially, fo (x) = 0, for all x and fi (x) = -  if x < 0.

Fn (M) = max {fn-1 (M), fn-1 (M - wn) + pn}

F3 (6) = max (f2 (6), f2 (6 – 4) + 5} = max {f2 (6), f2 (2) + 5}

F2 (6) = max (f1 (6), f1 (6 – 3) + 2} = max {f1 (6), f1 (3) + 2}

F1 (6) = max (f0 (6), f0 (6 – 2) + 1} = max {0, 0 + 1} = 1

F1 (3) = max (f0 (3), f0 (3 – 2) + 1} = max {0, 0 + 1} = 1

Therefore, F2 (6) = max (1, 1 + 2} = 3

F2 (2) = max (f1 (2), f1 (2 – 3) + 2} = max {f1 (2), -  + 2}

F1 (2) = max (f0 (2), f0 (2 – 2) + 1} = max {0, 0 + 1} = 1

F2 (2) = max {1, -  + 2} = 1

Finally, f3 (6) = max {3, 1 + 5} = 6

Other Solution:

For the given data we have:

S0 = {(0, 0)}; S0 = {(11, 2)}

S1 = (S0 U S01) = {(0, 0), (1, 2)}

X - 2 = 0 => x = 2. y – 3 = 0 => y = 3
X - 2 = 1 => x = 3. y – 3 = 2 => y = 5

S11 = {(2, 3), (3, 5)}

S2 = (S1 U S1 1) = {(0, 0), (1, 2), (2, 3), (3, 5)}

X – 5 = 0 => x = 5. y–4 =0 => y = 4

X – 5 = 1 => x = 6. y–4 =2 => y = 6
X – 5 = 2 => x = 7. y–4 =3 => y = 7
X – 5 = 3 => x = 8. y–4 =5 => y = 9

S21 = {(5, 4), (6, 6), (7, 7), (8, 9)}

S3 = (S2 U S21) = {(0, 0), (1, 2), (2, 3), (3, 5), (5, 4), (6, 6), (7, 7), (8, 9)}

By applying Dominance rule,

S3 = (S2 U S21) = {(0, 0), (1, 2), (2, 3), (5, 4), (6, 6)}

From (6, 6) we can infer that the maximum Profit  pi xi = 6 and weight  xi wi = 6
Reliability Design:

The problem is to design a system that is composed of several devices connected in

series. Let ri be the reliability of device Di (that is ri is the probability that device i
will function properly) then the reliability of the entire system is  ri. Even if the
individual devices are very reliable (the ri’s are very close to one), the reliability of
the system may not be very good. For example, if n = 10 and ri = 0.99, i < i < 10,
then  ri = .904. Hence, it is desirable to duplicate devices. Multiply copies of the
same device type are connected in parallel.

If stage i contains mi mcopies of device Di. Then the probability that all mi h amv e a
malfunction is (1 - r) i. Hence the reliability of stage i becomes 1 – (1 - r) i.
i i

The reliability of stage ‘i’ is given by a function i (mi).

Our problem is to use device duplication. This maximization is to be carried out under
a cost constraint. Let ci be the cost of each unit of device i and let c be the maximum
allowable cost of the system being designed.

We wish to solve:

Maximize 
1 i  n 
i mi 

Subject to C m C
1 i  n
i i

mi > 1 and interger, 1 < i < n

Assume each Ci > 0, each mi must be in the range 1 < mi < ui, where
  n  
ui   C  Ci C J 
Ci 
  1  

The upper bound ui follows from the observation that mj > 1

An optimal solution m1, m2 ................mn is the result of a sequence of decisions, one
decision for each mi.

Let fi (x) represent the maximum value of   mJ

1  j i

Subject to the constrains:

 C
1ji
J mJ  x and 1 < mj < uJ, 1 < j < i
Example :

Design a three stage system with device types D1, D2 and D3. The costs are $30, $15
and $20 respectively. The Cost of the system is to be no more than $105. The
reliability of each device is 0.9, 0.8 and 0.5 respectively.

Solution:

We assume that if if stage I has mi devices of type i in parallel, then  i (mi) =1 – (1-
ri)mi

Since, we can assume each ci > 0, each mi must be in the range 1 ≤ mi ≤ ui. Where:
  n  
ui   C  Ci 
 CJ  Ci 
  1  

Using the above equation compute u1, u2 and u3.

105 30 3015  20
u1   70
30 30  2
10515 3015  20 55
u2   3
15 15
105 20 3015  20 60
u3    3
20 20

We useSij  i:stage number and J: no. of devices in stage i  mi

So  fo (x), x initially fo x  1 and x  0, so, So  1,0

Compute S1, S2 and S3 as follows:

S1 = depends on u1 value, as u1 = 2, so
S1  S , S 
1 1

1 2

S2 = depends on u2 value, as u2 = 3, so
 S2  S 2
, S2 , S2 
1 2 3

S3 = depends on u3 value, as u3 = 3, so

S3  S , S
3 3
, S3 
1 2 3

Now find,S11  f (x), x 

1

f1 x   1 (1) fo  , 1 (2) f 0 ()} With devices m1 = 1 and m2 = 2

Compute 1 (1) and 1 (2) using the formula: i mi)  1 (1  ri )mi

1 1  1 1  r1m 1 = 1 – (1 – 0.9)1 = 0.9

1 2  1 1 0.92  0.99

S1   f1 x, x     0.9, 30
1

S12   0.99 , 30  30    0.99, 60

Therefore, S1 = {(0.9, 30), (0.99, 60)}

Next findS2  f (x), x 
1 2

f2 (x)  {2 1 * f1 , 2 2 * f1  , 2 3 * f1  }

2 1  1  1  rImi = 1 – (1 – 0.81) = 1 – 0.2 = 0.8

2 2  1  1  0.8 2  0.96

2 3  1  1  0.8 3  0.992

S12  {(0.8(0.9),30  15), (0.8(0.99),60  15)} = {(0.72, 45), (0.792, 75)}

S22  {(0.96(0.9),30  15 15) , (0.96(0.99),60  15  15)}

= {(0.864, 60), (0.9504, 90)}

S32  {(0.992(0.9),30  15 1515) , (0.992(0.99),60  15  1515)}

= {(0.8928, 75), (0.98208, 105)}
S2  S 2
, S2 , S2 
1 2 3

By applying Dominance rule to S2:

Therefore, S2 = {(0.72, 45), (0.864, 60), (0.8928, 75)}

 Dominance Rule:
If Si contains two pairs (f1, x1) and (f2, x2) with the property that f1 ≥ f2 and x1 ≤ x2,
then (f1, x1) dominates (f2, x2), hence by dominance rule (f2, x2) can be discarded.
Discarding or pruning rules such as the one above is known as dominance rule.
Dominating tuples will be present in Si and Dominated tuples has to be discarded
from Si.

Case 1: if f1 ≤ f2 and x1 > x2 then discard (f1, x1)

Case 2: if f1 > f2 and x1 < x2 the discard (f2, x2)

Case 3: otherwise simply write (f1, x1)

S2 = {(0.72, 45), (0.864, 60), (0.8928, 75)}

 3 1  1  1  rI  mi = 1 – (1 – 0.5)1 = 1 – 0.5 = 0.5

 3 2  1  1  0.5 2  0.75

 3 3  1  1  0.5 3  0.875

S13  0.5 (0.72), 45  20, 0.5 (0.864), 60  20, 0.5 (0.8928), 75  20

S13  0.36, 65, 0.437, 80, 0.4464, 95

S23 {0.75 (0.72), 45  20  20, 0.75 (0.864), 60  20  20,

0.75 (0.8928), 75  20  20}

= {(0.54, 85), (0.648, 100), (0.6696, 115)}

S33   0.875 (0.72), 45  20  20  20, 0.875 (0.864), 60  20  20  20,

0.875 (0.8928), 75  20  20  20 

S33   (0.63, 105), 1.756, 120, 0.7812, 135

If cost exceeds 105, remove that tuples

S3 = {(0.36, 65), (0.437, 80), (0.54, 85), (0.648, 100)}

The best design has a reliability of 0.648 and a cost of 100. Tracing back forthe
solution through Si ‘s we can determine that m3 = 2, m2 = 2 and m1 = 1.
Optimal Binary Search Tree:
In computer science, an optimal binary search tree (Optimal BST), sometimes called a weight-balanced
binary tree,[1] is a binary search tree which provides the smallest possible search time (or expected search
time) for a given sequence of accesses (or access probabilities).

The no of external nodes are same in both trees.

The C (i, J) can be computed as:

C (i, J) = min {C (i, k-1) + C (k, J) + P (K) + w (i, K-1) + w (K,J)}

i<k<J

= min {C (i, K-1) + C (K, J)} + w (i, J) -- (1)

i<k<J
Where W (i, J) = P (J) + Q (J) + w (i, J-1) -- (2)

Initially C (i, i) = 0 and w (i, i) = Q (i) for 0 < i < n.

C (i, J) is the cost of the optimal binary search tree 'Tij' during computation we record
the root R (i, J) of each tree 'Tij'. Then an optimal binary search tree may be
constructed from these R (i, J). R (i, J) is the value of 'K' that minimizes equation (1).

We solve the problem by knowing W (i, i+1), C (i, i+1) and R (i, i+1), 0 ≤ i < 4;
Knowing W (i, i+2), C (i, i+2) and R (i, i+2), 0 ≤ i < 3 and repeating until W (0, n),
C (0, n) and R (0, n) are obtained.
Matrix chainmultiplication

The problem

Given a sequence of matrices A1, A2, A3, ..., An, find the best way (using the minimal number of
multiplications) to compute their product.

• Isn’t there only one way? ((· · · ((A1 · A2) · A3 ) · · ·) · An)

• No, matrix multiplication is associative.
e.g. A1 · (A2 · (A3 · (· · · (An−1 · An) · · ·))) yields the same matrix.
• Different multiplication orders do not cost the same:
– Multiplying p × q matrix A and q × r matrix B takes p · q · r multiplications; result is a
p × r matrix.
– Consider multiplying 10 × 100 matrix A1 with 100 × 5 matrix A2 and 5 × 50 matrix A3.
– (A1 · A2) · A3 takes 10 · 100 · 5 + 10 · 5 · 50 = 7500 multiplications.
– A1 · (A2 · A3) takes 100 · 5 · 50 + 10 · 50 · 100 = 75000 multiplications.

Notation

• In general, let Ai be pi−1 × pi matrix.

• Let m(i, j) denote minimal number of multiplications needed to compute Ai · Ai+1 · · · Aj

• We want to compute m(1, n).

Recursive algorithm

• Assume that someone tells us the position of the last product, say k. Then we have to
compute recursively the best way to multiply the chain from i to k, and from k + 1 to j, and
add the cost of the final product. This means that

m(i, j) = m(i, k) + m(k + 1, j) + pi−1 · pk · pj

• If noone tells us k, then we have to try all possible values of k and pick the best solution.
• Recursive formulation of m(i, j):
.
0 If i = j
m(i, j) =
mini≤k<j{m(i,k) +m(k +1,j) +pi−1 · pk · pj} If i < j
• To go from the recursive formulation above to a program is pretty straightforward:

23
 Matrix-chain(i, j)
IF i = j THEN return 0

m=∞
FOR k = i TO j − 1 DO
q = Matrix-chain(i, k) + Matrix-chain(k + 1, j) +pi−1 · pk · pj
IF q < m THEN m = q
OD
Return m
END Matrix-chain

Return Matrix-chain(1, n)

• Running time:
n
Σ−1
T (n) = (T (k) + T (n− k) + O(1))
k=1
n−1
Σ
= 2· T (k) + O(n)
k=1

≥ 2 · T (n − 1)
≥ 2 · 2 · T (n − 2)
≥ 2· 2· 2...
= 2n

• Exponential is ... SLOW!

• Problem is that we compute the same result over and over again.
– Example: Recursion tree for Matrix-chain(1, 4)
1,4

1,1 2,4 1,2 3,4 1,3 4,4

2,2 3,4 2,3 4,4 1,1 2,2 3,3 4,4 1,1 2,3 1,2 3,3

3,3 4,4 2,2 3,3 2,2 3,3 1,1 2,2

24
 For example, we compute Matrix-chain(3, 4) twice.

Dynamic programming with a table and recursion

• Solution is to “remember” the values we have already computed in a table. This is called
memoization. We’ll have a table T[1..n][1..n] such that T[i][j] stores the solution to problem
Matrix-CHAIN(i,j). Initially all entries will be set to ∞.
FOR i = 1 to n DO
FOR j = i to n DO
T [i][j] = ∞
OD
OD
• The code for MATRIX-CHAIN(i,j) stays the same, except that it now uses the table. The first
thing MATRIX-CHAIN(i,j) does is to check the table to see if T [i][j] is already computed.
Is so, it returns it, otherwise, it computes it and writes it in the table. Below is the updated
code.
Matrix-chain(i, j)

IF T [i][j] < ∞ THEN return T [i][j]

IF i = j THEN T [i][j] = 0, return 0

m=∞
FOR k = i to j − 1 DO
q = Matrix-chain(i, k) + Matrix-chain(k + 1, j)+pi−1 · pk · pj
IF q < m THEN m = q
OD
T [i][j] = m
return m
END Matrix-chain

return Matrix-chain(1, n)

• The table will prevent a subproblem MATRIX-CHAIN(i,j) to be computed more than once.
• Running time:
– Θ(n2) different calls to matrix-chain(i, j).
– The first time a call is made it takes O(n) time, not counting recursive calls.
– When a call has been made once it costs O(1) time to make it again.
⇓
O(n3) time
– Another way of thinking about it: Θ(n2) total entries to fill, it takes O(n) to fill one.

25
26