DYNAMIC PROGRAMMING

The General Method

Dynamic programming is an algorithm design method suitable for problems in
which solutions can be viewed as the result of a sequence of decisions.

Example [Knapsack] :
The solution to the knapsack problem can be viewed as the result of a sequence of
decisions.

The decision has to be made on the values of xi, 1 ≤ i ≤ n.

First, a decision is made on x1, then on x2, then on x3, and so on.

An optimal sequence of decisions maximizes the objective function Σ p ixi and also
satisfies the constraints Σ wixi ≤ m and 0 ≤ xi ≤ 1.

Example [Shortest path] :

To find a shortest path from vertex i to vertex j in a directed graph G a sequence of
decisions have to be made.

First, decide which vertex should be the second vertex, which the third, which the
fourth, and so on, until vertex j is reached.

An optimal sequence of decisions is one that results in a path of least length.

For some problems, it is not possible to make a sequence of stepwise decisions
leading to an optimal decision sequence.
Here, there is a need to try all possible decision sequences and then pick out the
best. But the time and space requirements may be prohibitive.

Dynamic programming drastically reduces the amount of enumeration by avoiding

the enumeration of some decision sequences that cannot possibly be optimal.

In dynamic programming an optimal sequence of decisions is obtained by making

explicit appeal to the principle of optimality.

Principle of optimality :
This principle states that an optimal sequence of decisions has the property that
whatever the initial state and decision are, the remaining decisions must constitute
an optimal decision sequence with regard to the state resulting from the first
decision.

The essential difference between the greedy method and dynamic programming is
that :
- In the greedy method only one decision sequence is ever generated.
- In dynamic programming, many decision sequences may be generated.

In dynamic programming, sequences containing suboptimal sub-sequences cannot

be optimal and so will not be generated.
By using the principle of optimality, decision sequences containing sub-sequences
that are suboptimal are not considered.

Although the total number of different decision sequences is exponential in the

number of decisions (if there are d choices for each of the n decisions to be made
then there are dn possible decision sequences), dynamic programming algorithms
often have a polynomial complexity.

Another important feature of the dynamic programming approach is that optimal

solutions to sub-problems are retained and tabulated to avoid recomputing their
values.

The use of these tabulated values makes it natural to recast the recursive equations
into an iterative algorithm.
 Multi-Stage Graphs
A multistage graph G = (V, E) is a directed graph, with the vertices partitioned into k
> 2 disjoint sets Vi, 1 ≤ i ≤ k.

In this graph:
- If <u, v> is an edge in E, then u  Vi and v  Vi+1 for some i, 1 ≤ i < k.
- The sets V1 and Vk are such that │V1│ = │Vk│ = 1.
- Let s and t, respectively, be the vertices in V1 and Vk, then the vertex s is the
source and t the sink.
- Let c(i, j) be the cost of the edge <i, j>.
- The cost of a path from s to t is the sum of the costs of the edges on the path.

The multistage graph problem is to find a minimum-cost path from s to t.

In this graph, each set Vi defines a stage.
Here, every path from s to t
- starts in stage 1,
- goes to stage 2,
- then to stage 3,
- then to stage 4, and so on
- eventually terminates in stage k.
The following figure shows a five-stage graph.

A minimum-cost path from s to t is indicated by the broken edges.

Let us consider a resource allocation problem, in which n units of resource are to be
allocated to r projects.

Let j, 0 ≤ j ≤ n units of the resource are allocated to project i and the resulting net
profit is N(i, j).

Now, the problem is to allocate the resource to the r projects such that the total net
profit is maximized.

The above problem can be formulated as an r + 1 stage graph problem as:

- Stage i, 1 ≤ i ≤ r, represents project i.
- There are n + 1 vertices V(i, j), 0 ≤ j ≤ n, associated with stage i, 2 ≤ i ≤ r.
- Stages 1 and r + 1 each have one vertex, V(1, 0) = s and V(r + 1, n) = t
respectively.
- Vertex V(i, j), 2 ≤ i ≤ r, represents the state in which a total of j units of resource
have been allocated to projects 1, 2, …, i – 1.
- The edges in G are of the form <V(i, j), V(i+1, l)> for all j ≤ l and 1 ≤ i < r.
- The edge <V(i, j), V(i+1, l)>, j ≤ l, is assigned a weight or cost of N(i, l-j) and
corresponds to allocating l-j units of resource to project i, 1 ≤ i < r.

In addition, G has edges of the type <V(r, j), V(r+1, n)> and each such edge is
assigned a weight of max0 ≤ p ≤ n-j {N(r, p)}.

The resulting graph for a three project problem with n = 4 is:

A dynamic programming formulation for a k-stage graph problem is obtained by the
fact that every s to t path is the result of a sequence of k – 2 decisions.

The ith decision involves determining a vertex Vi+1, 1 ≤ i ≤ k-2, is to be on the path.
Hence, the principal of optimality holds.

Let p(i, j) be a minimum-cost path from vertex j in V 1 to vertex t.

Let cost(i, j) be the cost of this path.

Then, the following equation can be obtained

cost(i, j) = minlVi+1, <j,l>E {c(j,l) + cost(i+1, l)}

Since, cost(k-1, j) = c(j, t) if <j, t>  E and cost(k-1, j) = ∞ if <j, t>  E, the above
equation may be solved for cost(1, s) by
- first computing cost(k-2, j) for all j  Vk-2
- then computing cost(k-3, j) for all j  Vk-3 and so on
- finally compute cost(1, s)

Let us apply the above equation on the following graph to obtain:

cost(4, 9) = 4
cost(4,10) = 2
cost(4, 11) = 5
cost(3, 6) = min {6 + cost(4,9), 5 + cost(4, 10)}
= min {6 + 4, 5 + 2}
= min {10, 7}
=7
cost(3, 7) = min{4 + cost(4, 9), 3 + cost(4, 10)} = 5
cost(3, 8) = min{5 + cost(4, 10), 6 + cost(4, 11)} = 7
cost(2, 2) = min{4 + cost(3, 6), 2 + cost(3, 7), 1 + cost(3, 8)} = 7
cost(2, 3) = min{2 + cost(3, 6), 7 + cost(3, 7)} = 9
cost(2, 4) = min{11 + cost(3, 8)} = 18
cost(2, 5) = min{11 + cost(3, 7), 8 + cost(3, 8)} = 15
cost(1, 1) = min{9 + cost(2, 2), 7 + cost(2, 3), 3 + cost(2, 4), 2 + cost(2, 5)}
= 16
Note: To calculate the current stage values, the next stage values are reused, there
by avoiding recomputation.

From the above calculations, the following values are obtained.

d(3, 6) = 10 d(3, 7) = 10 d(3, 8) = 10
d(2, 2) = 7 d(2, 3) = 6 d(2, 4) = 8 d(2, 5) = 8
d(1, 1) = 2
Let the minimum-cost path be 1, V2, V3, V4, 12.
V2 = d(1, 1) = 2.
V3 = d(2, d(1, 1)) = d(2, 2) = 7.
V4 = d(3, d(2, d(1, 1))) = d(3, 7) = 10.
So, the minimum-cost path is 1, 2, 7, 10, 12.

In order to write an algorithm for a general k-stage graph, the ordering on n vertices
need to be decided.
Indices are assigned in order of stages:
- First, s is assigned index 1.
- Then vertices in V2 are assigned indices.
- Then vertices from V3 and so on.
- Finally vertex t has index n.

The algorithm FGraph is:

Algorithm FGraph(G, k, n, p)
// The input is a k-stage graph G = (V, E) with n vertices indexed in order of stages
// E is a set of edges and c[i, j] is the cost of <i, j>. P[1 : k] is a minimum-cost
// path.
{
cost[n] = 0.0;
for j := n-1 to 1 step -1 do
{ // Compute cost[j]
Let r be a vertex such that <j, r> is an edge of G and
c[j, r] + cost[r] is minimum;
cost[j] := c[j, r] + cost[r];
d[j] := r;
} // Find a minimum-cost path
p[1] := 1; p[k] := n;
for j := 2 to k-1 do p[j] := d[p[j-1]];
}
The above problem can also be solved using the backward approach.
Let bp(i, j) be a minimum-cost path from vertex s to a vertex j in V i.
Let bcost(i, j) be the cost of bp(i, j).
From the backward approach, the following equation is obtained:
bcost(i, j) = minlVi-1, <l,j>E {c(l, j) + bcost(i-1, l)}

Since bcost(2, j) = c(1, j) if <1, j>  E and bcost(2, j) = ∞ if <1, j>  E, the above
equation can be used to compute bcost(i, j) by
- first computing for i = 3,
- then for i = 4 and so on.

For the above graph,

bcost(2, 2) = 9
bcost(2, 3) = 7
bcost(2, 4) = 3
bcost(2, 5) = 2
bcost(3, 6) = min {bcost(2, 2) + c(2, 6), bcost(2, 3) + c(3, 6)}
= min {9 + 4, 7 + 2}
=9
bcost(3, 7) = 11
bcost(3, 8) = 10
bcost(4, 9) = 15
bcost(4, 10) = 14
bcost(4, 11) = 16
bcost(5, 12) = 16

The algorithm used to obtain a minimum-cost s – t path is:

Algorithm Bgraph(G, k, n, p)
// Same function as Fgraph
{
bcost[1] = 0.0;
for j := 2 to n do
{ // Compute bcost[j]
Let r be such that <r, j> is an edge of G and bcost[r] + c[r, j] is
minimum;
bcost[j] := bcost[r] + c[r, j];
d[j] := r;
}
// Find a minimum-cost path
p[1] := 1; p[k] := n;
for j := k – 1 to 2 do p[j] := d[p[j + 1]];
}
 Matrix Chain Multiplication
Given some matrices to multiply, determine the best order to multiply them so that
the number of single element multiplications can be minimized.
i.e. Determine the way the matrices are parenthesized.

First, it should be noted that matrix multiplication is associative, but not

commutative.
Since it is associative, we can always have:
((AB)(CD)) = (A(B(CD))), or any other grouping as long as the matrices are in the
same consecutive order.

But Not : ((AB)(CD)) = ((BA)(DC))

It may appear that the amount of work done won’t change if you change the
parenthesization of the expression, but can be proved that it is not the case!
Let us consider the following example:
- Let A be a 2x10 matrix
- Let B be a 10x50 matrix
- Let C be a 50x20 matrix

FIRST, let’s review some matrix multiplication rules …

We will show that the way the matrices grouped, when multiplying A, B, C matters:

• Consider computing A(BC):

- # multiplications for (BC) = 10x50x20 = 10000, creating a 10x20 answer Matrix
- # multiplications for A(BC) = 2x10x20 = 400
- Total multiplications = 10000 + 400 = 10400.

• Consider computing (AB)C:

- # multiplications for (AB) = 2x10x50 = 1000, creating a 2x50 answer matrix
- # multiplications for (AB)C = 2x50x20 = 2000,
- Total multiplications = 1000 + 2000 = 3000

So our problem is :
Given a chain of matrices to multiply, determine the fewest number of
multiplications necessary to compute the product.

Formal Definition of the problem:

- Let A = A0 · A1 · ... An-1
- Let Ni,j denote the minimal number of multiplications necessary to find
the product:
Ai · Ai+1 · ... Aj
- And let di X di+1 denote the dimensions of matrix A i.
The aim is to determine the minimal number of multiplications necessary(N 0,n-1) to
find A,
– assuming that each matrix multiplication is carried out in the standard method.

The key to solve this problem is noticing the sub-problem optimality condition:
– If a particular parenthesization of the whole product is optimal, then any sub-
parenthesization in that product is optimal as well.

Means :
– If (A (B ((CD) (EF)) ) ) is optimal
– Then (B ((CD) (EF)) ) is optimal as well

Assume that we are calculating ABCDEF and that the following parenthesization is
optimal:
• (A(B((CD) (EF))))
Then it is necessarily the case that
• (B((CD) (EF))) is the optimal parenthesization of BCDEF.

Why is this?
– Because if it wasn't, and say ( ((BC) (DE)) F) was better, then it would also follow
that (A ( ((BC) (DE)) F) ) is better than (A (B ((CD) (EF)) ) ),
– contradicting its optimality!
Our final multiplication will always be of the form
(A0 · A1 · ... Ak) · (Ak+1 · Ak+2 · ... · An-1)

In essence, there is exactly one value of k for which we should "split" our work into
two separate cases so that we get an optimal result.
Here is a list of the cases to choose from:
(A0) · (A1 · … · Ak+2 · ... · An-1)
(A0 · A1) · (A2 · … · Ak+2 · ... · An-1)
(A0 · A1 · A2) · (A3 · … · Ak+2 · ... · An-1)
·
·
·
(A0 · A1 · ... · An-3) · (An-2 · An-1)
(A0 · A1 · ... · An-2) · (An-1)

Basically, count the number of multiplications in each of these choices and pick the
minimum.

Consider the case of multiplying these 4 matrices:

- A: 2x4
- B: 4x2
- C: 2x3
- D: 3x1
1. (A)(BCD) - This is a 2x4 multiplied by a 4x1,
So 2x4x1 = 8 multiplications, plus whatever work it will take to multiply (BCD).

2. (AB)(CD) - This is a 2x2 multiplied by a 2x1,

So 2x2x1 = 4 multiplications, plus whatever work it will take to multiply (AB) and
(CD).

3. (ABC)(D) - This is a 2x3 multiplied by a 3x1,

So 2x3x1 = 6 multiplications, plus whatever work it will take to multiply (ABC).

Now, the recursive formula:

Ni,j = min value of Ni,k + Nk+1,j + didk+1dj+1, over all valid values of k.

Now let’s turn this recursive formula into a dynamic programming solution
Which sub-problems are necessary to solve first?

Clearly it's necessary to solve the smaller problems before the larger ones.
• In particular, there is a need to know Ni,i+1, the number of multiplications to
multiply any adjacent pair of matrices before we move onto larger tasks.

• Similarly, the next task to be solved is finding all the values of the form N i,i+2, then
Ni,i+3, etc.
Algorithm:
Initialize N[i][i] = 0, and all other entries in N to .
for i=1 to n-1 do the following
for j=0 to n-1-i do
for k=j to j+i-1
if (N[j][j+i-1] > N[j][k]+N[k+1][j+i-1]+djdk+1di+j)
N[j][j+i-1]= N[j][k]+N[k+1][j+i-1]+djdk+1di+j

Basically, checking for different places to “split” matrices by checking different

values of k and seeing whether they can improve current minimum value.
 Optimal Binary Search Trees
For a given set of identifiers, create a binary search tree organization.
Different binary search trees for the same identifier set can be expected to have
different performance characteristics.

The tree of Figure (a), in the worst case, requires four comparisons to find an
identifier.
The tree of Figure (b), requires only three comparisons in the worst case.
In the case of tree (a), it takes 1,2,2, 3, and 4 comparisons, respectively, to find the
identifiers for, do. while, int, and if.
In the case of tree (b), it takes 1,2,2, 3, and 3 comparisons, respectively, to find the
identifiers for, do. while, int, and if.

On the average the two trees need 12/5 and 11/5 comparisons, respectively.
In a general situation, different identifiers are to be searched for with different
frequencies (or probabilities).
In addition, there will be unsuccessful searches also.

Let us assume that the given set of identifiers is {a 1, a2,... ,an} such that
a1 < a2 < … < an.
Let p(i) be the probability with which an identifier a i is searched.
Let q(i) be the probability with which an identifier x being searched, a i < x < ai+1,
0 ≤ i ≤ n. Let us assume that a0 = - and an + 1 = +.

So,
Σ1 ≤ i ≤ n p(i) is the probability of a successful search.
Σ0 ≤ i ≤ n q(i) is the probability of an unsuccessful search.
Clearly,
Σ1 ≤ i ≤ n p(i) + Σ0 ≤ i ≤ n q(i) = 1.
Based on the above data, an optimal binary search tree has to constructed for
{a1, a2, ..., an}.
To obtain a cost function for binary search trees, it is required to add external nodes,
as shown below.

For a binary search tree representing n identifiers, there will be exactly n internal
nodes and n + 1 external nodes.

Every internal node represents termination of a successful search and every

external node represents termination of an unsuccessful search.
The iterative algorithm to search a binary search tree is as given below :

Algorithm Search(x)
{
found := false;
t := tree;
while ((t ≠ 0) and not found) do
{
if (x = (t  data)) then found := true;
else if (x < (t  data)) then t := (t  lchild);
else t := (t  rchild);
}
if (not found) then return 0;
else return t;
}

If a successful search terminates at an internal node at level i, then i iterations of

the while loop are needed.

Hence, the expected cost contribution from the internal node for a i is
p(i) * level (ai).
Unsuccessful searches terminate with t = 0 at external nodes in the above
algorithm.

The identifiers not in the binary search tree can be partitioned into n + 1
equivalence classes Ei, 0 ≤ i ≤ n.
The class E0 contains all identifiers x such that x < a1.
The class Ei contains all identifiers x such that ai < x < ai + 1, 1  i < n.
The class En contains all identifiers x, x > an.

For all identifiers in the same class Ei, the search terminates at the same external
node.
If the failure node for Ei is at level l, then only l — 1 iterations of the while loop are
made.
Hence, the cost contribution of this node is q(i) * (level(E i) — 1).
Now, the expected cost of a binary search tree is given by :

Our aim to construct an optimal binary search tree for the identifier set
{a1, a2,... , an} for which (1) is minimum.

Example :
The possible binary search trees for the identifier set {a 1, a2, a3} = {do, if, while}
With equal probabilities p(i) = q(i) = 1/7 for all i. The costs are :
cost(tree a) = 6/7 + 9/7 = 15/7
cost(tree b) = 5/7 + 8/7 = 13/7
cost(tree c) = 6/7 + 9/7 = 15/7
cost(tree d) = 6/7 + 9/7 = 15/7
cost(tree e) = 6/7 + 9/7 = 15/7
The tree b is optimal.

With p(1) = .5, p(2) = .1, p(3) = .05,

q(0) = .15, q(1) = .1, q(2) = .05 and q(3) = .05.
The costs are :
cost(tree a) = 2.65 cost(tree b) = 1.9 cost(tree c) = 1.5
cost(tree d) = 2.05 cost(tree e) = 1.6
The tree c is optimal.

Apply dynamic programming technique now for this problem to obtain an optimal
binary search tree, as the result of a sequence of decisions.

The principle of optimality should hold when applied to the problem state resulting
from a decision.
First, a decision has to made about which of the a i's should be assigned as the root
node of the tree. Let it be ak.
Now, the left subtree l of the root node contains :
- a1, a2, …, ak – 1 as internal nodes and
- External nodes from the classes E0, E1, …, Ek – 1.
The right subtree r of the root node contains :
- ak + 1, ak + 2, …, an as internal nodes and
- External nodes from the classes Ek, Ek + 1, …, En.

Let us define

and

In both cases, the level is measured with regard to the root, at level 1.
let us use w(i, j) to represent the sum

The expected cost of the binary search tree shown above is :

p(k) + cost(l) + cost(r) + w(0, k - 1) + w(k, n) - - - (2)
The equation (2) must be minimum for the tree to be optimal.

Hence, cost(l) and cost(r) must be minimum over all binary search trees.
Let us use c(i, j) to represent the cost of an optimal binary search tree t ij containing
ai+1, ..., aj and Ei, ..., Ej.

For the optimality of the tree,

- Cost(l) = c(0, k – 1).
- Cost(r) = c(k, n).
The value of k must be selected such that
p(k) + c(0, k -1) + c(k, n) + w(0, k - 1) + w(k, n) is minimum.

So, c(0, n) can be obtained from

--- (3)
The equation (3) can be generalized for c(i, j) as

} --- (4)

Initially, c(i, i) = 0 and w(i, i) = q(i), 0 ≤ i ≤ n.

Equation (4) can be solved for c(0, n) by:

First computing all c(i, j) such that j — i = 1,
Next, computing all c(i, j) such that j — i = 2,
Later all c(i,j) with j — i = 3, and so on.

During the above computation, the root r(i, j) is recorded for each tree t i, j.
After that an optimal binary search tree is constructed from these r(i, j).
This r(i, j) is the value of k that minimizes (4).

Example :
Let n = 4 and (a1, a2, a3, a4) = (do, if, int, while).
Let p(1 : 4) = (3, 3, 1, 1) and q(0 : 4) = (2, 3, 1, 1, 1).
The p's and q's have been multiplied by 16 for convenience.
Solution :
Initially, w(i, i) = q(i), c(i, i) = 0 and r(i, i) = 0, 0 ≤ i ≤ 4.
Using Equation (4) and the observation w(i,j) = P(j) + q(j) + w(i, j - 1), the following
are obtained.

w(0, 1) = p(1) + q(1) + w(0,0) = 8

c(0, 1) = w(0, 1) + min{c(0, 0) + c(1, 1)} = 8
r(0, 1) = 1

w(1, 2) = p(2) + q(2) + w(1, 1) = 7

c(1, 2) = w(1, 2) + min{c(1, 1) + c(2,2)} = 7
r(1, 2) = 2

w(2, 3) = p(3) + q(3) + w(2,2) = 3

c(2, 3) = w(2, 3) + min{c(2,2) + c(3, 3)} = 3
r(2, 3) = 3

w(3, 4) = p(4) + q(4) + w(3,3) = 3

c(3, 4) = w(3, 4) + min{c(3, 3) + c(4, 4)} = 3
r(3, 4) = 4
After calculating w(i, i + 1), c(i, i + 1), and r(i, i + 1), 0 ≤ i < 4 using equation (4),
next is to calculate w(i, i + 2), c(i, i + 2), and r(i, i + 2) and so on, till w(0, 4),
c(0, 4) and r(0, 4) are obtained.
 All-Pairs Shortest Paths
Let G = (V, E) be a directed graph with n vertices.
Let cost be a cost adjacency matrix for G such that :
- cost(i,i) = 0, 1 ≤ i ≤ n.
- cost(i,j) is the length (or cost) of edge <i, j> if <i, j>  E(G).
- cost(i,j) =  if <i, j>  E(G).

The all-pairs shortest-path problem is to determine a matrix A such that A(i, j) is the
length of a shortest path from i to j.
The matrix A can be obtained by solving n single-source problems using the
algorithm ShortestPaths.
Since each application of this procedure requires O(n2) time, the matrix A can be
obtained in O(n3) time.

An alternate solution to this problem using the principle of optimality, requires a

weaker restriction on edge costs compared to ShortestPaths.

In this solution, instead of requiring cost(i,j) ≥ 0, for every edge (i,j), it requires that
G have no cycles with negative length.

If G is allowed to contain a cycle of negative length, then the shortest path between
any two vertices on this cycle has length -.
Let us examine a shortest i to j path in G, i ≠ j.
This path originates at vertex i and goes through some intermediate vertices and
terminates at vertex j.

Let us assume that this path contains no cycles, if there is a cycle, then this can be
deleted without increasing the path length (no cycle has negative length).
If k is an intermediate vertex on this shortest path, then the subpaths from i to k
and from k to j must be shortest paths from i to k and k to j, respectively.

Otherwise, the i to j path is not of minimum length.

So, the principle of optimality holds. This helps us to the prospect of using dynamic
programming.

If k is the intermediate vertex with highest index, then the i to k path is a shortest i
to k path in G going through no vertex with index greater than k - 1.
Similarly the k to j path is a shortest k to j path in G going through no vertex of
index greater than k - 1.

We can regard the construction of a shortest i to j path as first requiring a decision

as to which is the highest indexed intermediate vertex k.
Once this decision has been made, we need to find two shortest paths, one from i to
k and the other from k to j.
Neither of these may go through a vertex with index greater than k — 1.
Using Ak(i, j) to represent the length of a shortest path from i to j going through no
vertex of index greater than k, we obtain
A(i, j) = min {min {Ak - 1(i, k) + Ak – 1(k, j)}, cost(i, j)}, 1 ≤ k ≤ n. - (1)

Clearly, A0(i,j) = cost(i, j), 1 ≤ i ≤ n, 1 ≤ j ≤ n.

We can obtain a recurrence for Ak(i,j) using an argument similar to that used before.

A shortest path from i to j going through no vertex higher than k either goes through
vertex k or it does not.

If it does, Ak(i,j) = Ak - 1(i, k) + Ak - 1(k, j). If it does not, then no intermediate vertex
has index greater than k—1.

Hence Ak(i, j) = Ak - 1(i, j). Combining, we get

Ak(i, j) = min {Ak - 1(i, j), Ak - 1(i, k) + Ak - 1(k, j)}, k ≥ 1 - (2)

The above equation is not true for graphs with cycles of negative length. The
following example illustrates this.
Example :
The following figure shows a digraph together with its matrix A 0.

For this graph A2(1,3) ≠ min{A1(1, 3), A1(1, 2) + A1(2, 3)} = 2.

Instead, A2(1,3) = -.
The path is:
1, 2, 1, 2, 1, 2, ..., 1, 2, 3.
This is so because of the presence of the cycle 1 2 1 which has a length of -1.

Recurrence (2) can be solved for An by first computing A1, then A2, then A3, and so
on.
Since there is no vertex in G with index greater than n, A(i,j) = A n(i,j).

The algorithm AllPaths computes An(i, j).

Algorithm AllPaths(cost, A, n)
// cost[1 : n, 1 : n] is the cost adjacency matrix of a graph with n vertices.
// A[i, j] is the cost of the shortest path from vertex i to vertex j.
// cost[i, i] = 0.0, for 1 ≤ i ≤ n.
{
for i := 1 to n do
for j := 1 to n do
A[i, j] := cost[i, j];
for k := 1 to n do
for i := 1 to n do
for j := 1 to n do
A[i, j] := min(A[i, j], A[i, k] + A[k, j]);
}

The time complexity of the above algorithm is θ(n3).

Example :
In the following figure, the graph (a) has the cost matrix of (b). The initial
A matrix, A0, and its values after three iterations A1, A2, and A3 are as
shown below
 The Traveling Salesperson
Problem
This problem is a permutation problem.
The permutation problems usually are much harder to solve than subset problems
because:
- The permutation problems have n! different permutations of n objects.
- The subset problems have only 2n different subsets of n objects (n! > 2n).

Let G = (V, E) be a directed graph with n vertices.

Let c be a cost adjacency matrix for G such that:
- c(i, i) = 0, 1 ≤ i ≤ n.
- c(i, j) is the length (or cost) of edge <i, j> if <i, j>  E(G).
- c(i, j) =  if <i, j>  E(G).

Let |V| = n and assume n > 1.

A tour of G is a directed simple cycle that includes every vertex in V.
The cost of a tour is the sum of the cost of the edges on the tour.
The traveling salesperson problem is to find a tour of minimum cost
 The traveling salesperson problem finds application in a variety of
situations.

Application 1 :
 A postal van has to be routed to pick up mail from mail boxes located at n
different sites.
 For this situation, an n + 1 vertex graph can be used.
 In this, one vertex represents the post office from which the postal van
starts and returns. The remaining n vertices represent n mail boxes.
 Edge <i, j> is assigned a cost equal to the distance from site i to site j.
 The route taken by the postal van is a tour, and it should be of minimum
length.

Application 2 :
 A robot arm has to tighten the nuts on some piece of machinery on an
assembly line.
 The arm will start from its initial position, successively move to each of the
nuts, and return to the initial position.
 The path of the arm is clearly a tour on a graph in which vertices
represent the nuts.
 A minimum-cost tour will minimize the time needed for the arm to
complete its task.
 Application 3 :
 Let us consider a production environment in which several commodities are
manufactured on the same set of machines. The manufacture proceeds in cycles.
 In each production cycle, n different commodities are produced.
 When the machines are changed from production of commodity i to commodity j, a
change over cost cij is incurred.
 It is desired to find a sequence in which to manufacture these commodities with the
least change over costs in a cycle.
 So, this problem can be viewed as a traveling salesperson problem on an n vertex
graph with edge cost cij 's being the changeover cost.

Let us assume that a tour be a simple path that starts and ends at vertex 1.
Every tour consists of an edge <1, k> for some k  V - {1} and a path from vertex k
to vertex 1.
So, the path from vertex k to vertex 1 goes through each vertex in V - {1, k} exactly
once.
For the optimal tour, the path from k to 1 must be a shortest k to 1 path going
through all vertices in V - {1,k}. So, the principle of optimality holds.

Let g(i,S) be the length of a shortest path starting at vertex i, going through all
vertices in S, and terminating at vertex 1.
So, the function g(1, V - {1}) is the length of an optimal salesperson tour.
From the principal of optimality it follows that

The equation (1) is generalized for i  S, to get

Equation (1) can be solved for g(1, V - {1}), if g(k, V - {1, k}) is known for all
values of k.
The g values can be obtained from equation (2).
So, g(i, Ф) = ci, 1, 1 < i ≤ n.

Now, equation (2) is used to obtain g(i, S) for all S of size 1.

Later, g(i, S) is obtained for S with |S| = 2, and so on.
When |S| < n — 1, the values of i and S for which g(i, S) is needed are such that i
≠ 1, 1  S, and i  S.
Example :
Consider the following directed graph of Figure (a). The edge lengths are given by
matrix c of Figure (b).

Thus,
g(2,Ф) = c2, 1 = 5,
g(3,Ф) = c3, 1 = 6, and
g(4,Ф) = c4, 1 = 8.
Using equation(2), compute g(i, S) for |S| = 1, i ≠ 1, 1  S, and i  S.
g(2, {3}) = c2, 3 + g(3, Ф) = 9 + 6 = 15
g(2, {4}) = c2, 4 + g(4, Ф) = 10 + 8 = 18
g(3, {2}) = c3, 2 + g(2, Ф) = 13 + 5 = 18
g(3, {4}) = c3, 4 + g(4, Ф) = 12 + 8 = 20
g(4, {2}) = c4, 2 + g(2, Ф) = 8 + 5 = 13
g(4, {3}) = c4, 3 + g(3, Ф) = 9 + 6 = 15

Later, compute g(i, S) for |S| = 2, i ≠ 1, 1  S, and i  S.

g(2, {3, 4}) = min {c2, 3 + g(3, {4}), c2, 4 + g(4, {3})}
= min {9 + 20, 10 + 15} = min {29, 25} = 25

g(3, {2, 4}) = min {c3, 2 + g(2, {4}), c3, 4 + g(4, {2})}
= min {13 + 18, 12 + 13} = min {41, 25} = 25

g(4, {2, 3}) = min {c4, 2 + g(2, {3}), c4, 3 + g(3, {2})}
= min {8 + 15, 9 + 18} = min {23, 27} = 23

Finally, from equation (1)

g(1, {2, 3, 4}) = min {c1,2 + g(2,{3,4}), c1,3 + g(3,{2,4}), c1,4 + g(4,{2,3})}
= min {10 + 25, 15 + 25, 20 + 23}
= min {35, 40, 43} = 35
The optimal tour of the graph in figure (a) is 1, 2, 4, 3, 1 and its length is 35.

Let N be the number of g(i, ,S)'s that have to be computed before equation (1) can
be used to compute g(1, V — {1}).
For each value of |S| there are n — 1 choices for i.
The number of distinct sets S of size k not including 1 and i is .

So,

The most serious drawback of this dynamic programming solution is:

- The amount of space required is O(n2n)
- The amount of time required is θ(n22n).
 Reliability Design
In this, dynamic programming technique is used to solve a problem with a
multiplicative optimization function.

The problem is to design a system that is composed of several devices connected in

series.

Let the device Di has a reliability as ri (that is, ri is the probability that device Di will
function properly).
Then, the reliability of the entire system is given by ri.

Even though, the individual devices are very reliable, i.e., the r i's are very close to
one), the reliability of the entire system may not be very good.
For example, if n = 10 and ri = .99, 1 ≤ i ≤ 10, then ri = .904.

So, it is desirable to have multiple copies of the same device type connected in
parallel through the use of switching circuits as shown below.
The switching circuits determine which devices in any given group are functioning
properly and one such device at each stage is used.

Let stage i contains mi copies of device Di.

Now, the probability that all mi have a malfunction is (1 — ri)mi.
So, the reliability of stage i becomes 1 - (1 — ri)mi.

Since, the switching circuits are not fully reliable, practically, the stage reliability is
little less than 1 - (1 — ri)mi.

Let the reliability of a stage i is given by the function Фi(mi).

Now, the reliability of the entire system having n stages is
Now, the problem is to use device duplication to achieve maximize reliability, under
a cost constraint.
Let ci be the cost of each unit of device Di and let c be the maximum cost allowed
for the entire system.

So, the given problem is maximization (reliability) problem:

mi ≥ 1 and integer.

Each ci > 0 and each mi should be in the range 1 ≤ mi ≤ ui, where the upper bound
ui is given by

An optimal solution m1, m2, …, mn is obtained by a sequence of decisions, one

decision for each mi.
Let fi(x) represent the maximum value of 1 ≤ j ≤ i Ф(mj)
subject to the constraints Σ1 ≤ j ≤ i cj mj ≤ x, and 1 ≤ mj ≤ uj.
Then, the value of an optimal solution is fn(c)

The last decision is to choose mn from {1, 2, 3, …, un}.

After the selection of mn, the remaining decisions must have used the remaining
funds c – cnmn in an optimal way and the principle of optimality holds.

The above equation can be generalized for any fi (x), i ≥ 1 as

Clearly f0 (x) = 1 for all x in the range 1 to c.

Let Si consist of tuples of the form (f, x), where f = fi(x).

There is at most one tuple for each different x that results from a sequence of
decisions on m1, m2, …, mn.
The dominance rule (f1, x1) dominates (f2, x2) iff f1 ≥ f2 and x1 ≤ x2 holds for this
problem.
Hence, the dominated tuples can be discarded from Si.
Example :
Design a three stage system with device types D1, D2 and D3 with the costs 30, 15,
20 and the reliabilities .9, .8, .5 respectively.
The cost of the entire system can not be more than 105.

Solution :
The stage i is having mi devices of type Di in parallel
Фi(mi) = 1 - (1 — ri)mi.
c1 = 30 c2 = 15 c3 = 20 c = 105
r1 = .9 r2 = .8 r3 = .5
u1 = 2 u2 = 3 u3 = 3

Si is used to denote set of undominated tuples (f, x) resulting from the decision
sequence for m1, m2, …, mi.
So, f(x) = fi(x).

Begin with S0 = {(1, 0)} and obtain each Si from Si – 1 by trying out all possible values
for mi and combining the resulting tuples together.
Sij is used to represent all tuples obtainable from Si – 1 by choosing mi = j.

Now
S11 = {(.9, 30)} and S12 = {(.99, 60)}
So, S1 = {(.9, 30), (.99, 60)}
S21 = {(.72,45), (.792, 75)} and S22 = {(.864, 60)}.
The tuple (.9504, 90) is eliminated because it leaves only the cost 15 not sufficient
for D3.
S23 = {(.8928,75)}.
So, S2 = {(.72,45), (.864, 60), (.8928, 75)}

S31 = {(.36, 65), (.432, 80), (.4464, 95)}.

S32 = {(.54, 85), (.648,100)}.
S33 = {(.63,105)}.
So, S3 = {(.36,65), (.432, 80), (.54, 85), (.648,100)}.

The best design has a reliability of .648 and a cost of 100.

Tracing back through the Si ’s, m1 = 1, m2 = 2, and m3 = 2.