DYNAMIC

PROGRAMMING
DYNAMIC PROGRAMMING
 Dynamic programming is an approach to problem
solving that decomposes a large problem that may
be difficult to solve into a number of smaller
problems that are usually much easier to solve.
 Moreover, the dynamic programming approach
allows us to break up a large problem in such a way
that once all the smaller problems have been
solved, we have an optimal solution to the large
problem.
 We shall see that each of the smaller problems is identified
with a stage of the dynamic programming solution
procedure.
 As a consequence, the technique has been applied to
decision problems that are multistage in nature.
 Often, multiple stages are created because a sequence of
decisions must be made over time. For example, a problem
of determining an optimal decision over a one-year horizon
might be broken into 12 smaller stages, where each stage
requires an optimal decision over a one-month horizon.
 A SHORTEST-ROUTE PROBLEM
 Let us illustrate the dynamic programming
approach by using it to solve a shortest-route
problem.
 Consider the network presented in Figure 21.1.
Assuming that the numbers above each arc
denote the direct distance in miles between
two nodes, find the shortest route from node 1
to node 10.
 Before attempting to solve this problem, let us
consider an important characteristic of all shortest-
route problems.
 This characteristic is a restatement of Richard
Bellman’s famous principle of optimality as it
applies to the shortest-route problem.
 We will work backward by starting at the terminal node, node 10, and
calculating the shortest route from each node to node 10 until we reach the
origin, node 1.
 At this point, we will have solved the original problem of finding the shortest
route from node 1 to node 10.
 In the shortest-route problem for the network in Figure 21.1, the smaller
problems that we will create define a four-stage dynamic programming
problem.
 The first stage begins with nodes that are exactly one arc away from the
destination, and ends at the destination node.
 Note from Figure 21.1 that only nodes 8 and 9 are exactly one arc away from
node 10. In dynamic programming terminology, nodes 8 and 9 are considered
to be the input nodes for stage 1, and node 10 is considered to be the output
node for stage 1.
 The second stage begins with all nodes that are exactly two arcs away from the
destination and ends with all nodes that are exactly one arc away.
 Hence, nodes 5, 6, and 7 are the input nodes for stage 2, and nodes 8 and 9 are
the output nodes for stage 2.
 Note that the output nodes for stage 2 are the input nodes for stage 1. The input
nodes for the third-stage problem are all nodes that are exactly three arcs away
from the destination—that is, nodes 2, 3, and 4.
 The output nodes for stage 3, all of which are one arc closer to the destination,
are nodes 5, 6, and 7.
 Finally, the input node for stage 4 is node 1, and the output nodes are 2, 3, and
4. The decision problem we shall want to solve at each stage is, Which arc is best
to travel over in moving from each particular input node to an output node?
 We arbitrarily begin the stage 1 calculations with node 9. Because only one
way affords travel from node 9 to node 10, this route is obviously shortest
and requires us to travel a distance of 2 miles.
 Similarly, only one path goes from node 8 to node 10. The shortest route
from node 8 to the end is thus the length of that route, or 5 miles.
 The stage 1 decision problem is solved. For each input node, we have
identified an optimal decision—that is, the best arc to travel over to reach
the output node. The stage 1 results are summarized here:
 To begin the solution to the stage 2 problem, we move to node 7. (We could have
selected node 5 or 6; the order of the nodes selected at any stage is arbitrary.)
Two arcs leave node 7 and are connected to input nodes for stage 1: arc 7–8,
which has a length of 8 miles, and arc 7–9, which has a length of 10 miles.
 If we select arc 7–8, we will have a distance from node 7 to node 10 of 13 miles,
that is, the length of arc 7–8, 8 miles, plus the shortest distance to node 10 from
node 8, 5 miles.
 Thus, the decision to select arc 7–8 has a total associated distance of 8+5=13
miles. With a distance of 10 miles for arc 7–9 and stage 1 results showing a
distance of 2 miles from node 9 to node 10, the decision to select arc 7–9 has an
associated distance of 10+2=12 miles. Thus, given we are at node 7, we should
select arc 7–9 because it is on the path that will reach node 10 in the shortest
distance (12 miles). By performing similar calculations for nodes 5 and 6, we can
generate the following stage 2 results:
 In Figure 21.2 the number in the square above each node
considered so far indicates the length of the shortest route
from that node to the end.
 We have completed the solution to the first two subproblems
(stages 1 and 2). We now know the shortest route from nodes
5, 6, 7, 8, and 9 to node 10.
 Thus, the shortest route is through nodes 1–3–5–8–10 with a distance of
5+6+3+5=19 miles.
 Figure 21.3 illustrates the final network calculations. Note that in working
back through the stages we have now determined the shortest route from
every node to node 10.
 Dynamic programming, while enumerating or evaluating several paths at
each stage, does not require us to enumerate all possible paths from node
1 to node 10. Returning to the stage 4 calculations, we consider three
alternatives for leaving node 1.
 The complete route associated with each of these alternatives is presented
as follows: