Dynamic Programming

• Sequence of decisions.
• Problem state.
• Principle of optimality.
• Dynamic Programming Recurrence
Equations.
• Solution of recurrence equations.
 Sequence Of Decisions

• As in the greedy method, the solution to a

problem is viewed as the result of a
sequence of decisions.
• Unlike the greedy method, decisions are not
made in a greedy and binding manner.
 0/1 Knapsack Problem
Let xi = 1 when item i is selected and let xi = 0 when item i is not selected.

n
maximize pi xi
i=1
n
subject to wi xi <= c
i=1
and xi = 0 or 1 for all i
All profits and weights are positive.
 Sequence Of Decisions

• Decide the xi values in the order x1, x2, x3, …, xn.

• Decide the xi values in the order xn, xn-1, xn-2, …,
x1.
• Decide the xi values in the order x1, xn, x2, xn-1, …
• Or any other order.
 Problem State
• The state of the 0/1 knapsack problem is given by
 the weights and profits of the available items
 the capacity of the knapsack
• When a decision on one of the xi values is made,
the problem state changes.
 item i is no longer available
 the remaining knapsack capacity may be less
 Problem State
• Suppose that decisions are made in the order x1, x2, x3,
…, xn.
• The initial state of the problem is described by the pair
(1, c).
 Items 1 through n are available (the weights, profits and n are
implicit).
 The available knapsack capacity is c.
• Following the first decision the state becomes one of the
following:
 (2, c) … when the decision is to set x1= 0.
 (2, c-w1) … when the decision is to set x1= 1.
 Problem State
• Suppose that decisions are made in the order xn, xn-1, xn-2,
…, x1.
• The initial state of the problem is described by the pair
(n, c).
 Items 1 through n are available (the weights, profits and first
item index are implicit).
 The available knapsack capacity is c.
• Following the first decision the state becomes one of the
following:
 (n-1, c) … when the decision is to set xn= 0.
 (n-1, c-wn) … when the decision is to set xn= 1.
 Principle Of Optimality
• An optimal solution satisfies the following
property:
 No matter what the first decision, the remaining
decisions are optimal with respect to the state that
results from this decision.
• Dynamic programming may be used only when
the principle of optimality holds.
 0/1 Knapsack Problem
• Suppose that decisions are made in the order x1,
x2, x3, …, xn.
• Let x1= a1, x2 = a2, x3 = a3, …, xn = an be an optimal
solution.
• If a1 = 0, then following the first decision the state
is (2, c).
• a2, a3, …, an must be an optimal solution to the
knapsack instance given by the state (2,c).
 x1 = a 1 = 0
n
maximize pi xi
i=2
n
subject to wi xi <= c
i=2
and xi = 0 or 1 for all i

• If not, this instance has a better solution b2, b3,

…, bn.
n n
pi bi > pi a i
i=2 i=2
 x1 = a 1 = 1
n
maximize pi xi
i=2
n
subject to wi xi <= c- w1
i=2
and xi = 0 or 1 for all i

• If not, this instance has a better solution b2, b3,

…, bn.
n n
pi bi > pi a i
i=2 i=2
 0/1 Knapsack Problem
• Therefore, no matter what the first decision, the
remaining decisions are optimal with respect to
the state that results from this decision.
• The principle of optimality holds and dynamic
programming may be applied.
Dynamic Programming Recurrence
• Let f(i,y) be the profit value of the optimal solution to
the knapsack instance defined by the state (i,y).
 Items i through n are available.
 Available capacity is y.
• For the time being assume that we wish to determine
only the value of the best solution.
 Later we will worry about determining the xis that yield this
maximum value.
• Under this assumption, our task is to determine f(1,c).
Dynamic Programming Recurrence
• f(n,y) is the value of the optimal solution to the
knapsack instance defined by the state (n,y).
 Only item n is available.
 Available capacity is y.
• If wn <= y, f(n,y) = pn.
• If wn > y, f(n,y) = 0.
Dynamic Programming Recurrence
• Suppose that i < n.
• f(i,y) is the value of the optimal solution to the
knapsack instance defined by the state (i,y).
 Items i through n are available.
 Available capacity is y.
• Suppose that in the optimal solution for the state
(i,y), the first decision is to set xi= 0.
• From the principle of optimality (we have
shown that this principle holds for the knapsack
problem), it follows that f(i,y) = f(i+1,y).
Dynamic Programming Recurrence
• The only other possibility for the first decision
is xi= 1.
• The case xi= 1 can arise only when y >= wi.
• From the principle of optimality, it follows that
f(i,y) = f(i+1,y-wi) + pi.
• Combining the two cases, we get
 f(i,y) = f(i+1,y) whenever y < wi.
 f(i,y) = max{f(i+1,y), f(i+1,y-wi) + pi}, y >= wi.
 Recursive Code
/** @return f(i,y) */
private static int f(int i, int y)
{
if (i == n) return (y < w[n]) ? 0 : p[n];
if (y < w[i]) return f(i + 1, y);
return Math.max(f(i + 1, y),
f(i + 1, y - w[i]) + p[i]);
}
 Recursion Tree

f(1,c)

f(2,c) f(2,c-w1)

f(3,c) f(3,c-w2) f(3,c-w1) f(3,c-w1 –w2)

f(4,c) f(4,c-w3) f(4,c-w2) f(4,c-w1 –w3)

f(5,c)
f(5,c-w1 –w3 –w4)
 Time Complexity
• Let t(n) be the time required when n items are
available.
• t(0) = t(1) = a, where a is a constant.
• When t > 1,
t(n) <= 2t(n-1) + b,
where b is a constant.
• t(n) = O(2n).

Solving dynamic programming recurrences

recursively can be hazardous to run time.
 Reducing Run Time

f(1,c)

f(2,c) f(2,c-w1)

f(3,c) f(3,c-w2) f(3,c-w1) f(3,c-w1 –w2)

f(4,c) f(4,c-w3) f(4,c-w2) f(4,c-w1 –w3)

f(5,c)
f(5,c-w1 –w3 –w4)
 Integer Weights Dictionary

• Use an array fArray[][] as the dictionary.

• fArray[1:n][0:c]
• fArray[i][y] = -1 iff f(i,y) not yet computed.
• This initialization is done before the recursive method
is invoked.
• The initialization takes O(cn) time.
 No Recomputation Code
private static int f(int i, int y)
{
if (fArray[i][y] >= 0) return fArray[i][y];
if (i == n) {fArray[i][y] = (y < w[n]) ? 0 : p[n];
return fArray[i][y];}
if (y < w[i]) fArray[i][y] = f(i + 1, y);
else fArray[i][y] = Math.max(f(i + 1, y),
f(i + 1, y - w[i]) + p[i]);
return fArray[i][y];
}
 Time Complexity
• t(n) = O(cn).
• Analysis done in text.
• Good when cn is small relative to 2n.
• n = 3, c = 1010101
w = [100102, 1000321, 6327]
p = [102, 505, 5]
• 2n = 8
• cn = 3030303