Today’s Outline

CS 362, Lecture 10
• Fractional Knapsack Wrapup
Jared Saia • Amortized Analysis
University of New Mexico

Proof Proof

• Assume the objects are sorted in order of cost per pound.

Let vi be the value for item i and let wi be its weight.
• Let k be the smallest index with xk < 1
• Let xi be the fraction of object i selected by greedy and let
• Note that for i ≤ k, xi = 1 and for i > k, xi = 0
V be the total value obtained by greedy
• You will show that for all i,
• Consider some arbitrary solution, B, and let x0i be the fraction
v v
of object i taken in B and let V 0 be the total profit obtained (xi − x0i) i ≥ (xi − x0i) k
wi wk
by B
• We want to show that V 0 ≤ V or that V − V 0 ≥ 0

2 3
 Proof Proof

n n
vk
V −V0 = x0ivi
X X
xi v i − (1) • Note that the last step follows because w is positive and
k
i=1 i=1 because:
n
(xi − x0i) ∗ vi n n n
X
= (2)
(xi − x0i) ∗ wi = x0i ∗ wi
X X X
i=1
xi w i − (7)
n ! i=1 i=1 i=1
vi
(xi − x0i) ∗ wi = W − W0
X
= (3) (8)
i=1 wi
n ! ≥ 0. (9)
X
0 vk
≥ (xi − xi) ∗ wi (4) • Where W is the total weight taken by greedy and W 0 is the
i=1 ! wk
vk n total weight for the strategy B
(xi − x0i) ∗ wi
X
≥ ∗ (5) • We know that W ≥ W 0
wk i=1
≥ 0 (6)

4 5

In-Class Exercise Q1

v v
(xi − x0i) i ≥ (xi − x0i) k
Consider the inequality: wi wk
v v
(xi − x0i) i ≥ (xi − x0i) k • Q1: Show that the inequality is true for i < k
wi wk
• For i < k, (xi − x0i) ≥ 0
• Q1: Show this inequality is true for i < k • If (xi − x0i) = 0, trivially true. Otherwise, can divide both
• Q2: Show it’s true for i = k sides of the inequality by xi − x0i to get
• Q3: Show it’s true for i > k vi v
≥ k.
wi wk
• This is true since the items are sorted by profit per weight

6 7
 Q2 Q3

v v
(xi − x0i) i ≥ (xi − x0i) k
wi wk

v v
(xi − x0i) i ≥ (xi − x0i) k • Q3: Show that the inequality is true for i > k
wi wk • For i < k, (xi − x0i) ≤ 0
• If (xi − x0i) = 0, trivially true. Otherwise can divide both sides
• Q2: Show that the inequality is true for i = k
of the inequality by xi − x0i to get
• When i = k, we have
vi v
v v ≤ k.
(xk − x0k ) k ≥ (xk − x0k ) k wi wk
wk wk
• This is obviously true since the items are sorted by profit per
• Which is true since the left side equals the right side
weight
• Note that the direction of the inequality changed when we
divided by (xi − x0i), since it is negative

8 9

Amortized Analysis Amortized analysis

“I will gladly pay you Tuesday for a hamburger today” - Welling-

ton Wimpy
Amortized analysis is not average case analysis.

• In amortized analysis, time required to perform a sequence of

• Average Case Analysis: the expected cost of each operation
data structure operations is averaged over all the operations
• Amortized analysis: the average cost of each operation in
performed
the worst case
• Typically used to show that the average cost of an operation
• Probability is not involved in amortized analysis
is small for a sequence of operations, even though a single
operation can cost a lot

10 11
 Types of Amortized Analysis Aggregate Analysis

• Aggregate Analysis
• Accounting or Taxation Method • We get an upperbound T (n) on the total cost of a sequence
• Potential method of n operations. The average cost per operation is then
• We’ll see each method used for 1) a stack with the additional T (n)/n, which is also the amortized cost per operation
operation MULTIPOP and 2) a binary counter

12 13

Stack with Multipop Multipop

• Recall that a standard stack has the operations PUSH and

POP
• Q: What is the running time of Multipop(S,k) on a stack of
• Each of these operations runs in O(1) time, so let’s say the
s objects?
cost of each is 1
• A: The cost is min(s,k) pop operations
• Now for a stack S and number k, let’s add the operation
• If there are n stack operations, in the worst case, a single
MULTIPOP which removes the top k objects on the stack
Multipop can take O(n) time
• Multipop just calls Pop either k times or until the stack is
empty

14 15
 Multipop Analysis The Problem

• Let’s analyze a sequence of n push, pop, and multipop op-

• This analysis is technically correct, but overly pessimistic
erations on an initially empty stack
• While some of the multipop operations can take O(n) time,
• The worst case cost of a multipop operation is O(n) since
not all of them can
the stack size is at most n, so the worst case time for any
• We need some way to average over the entire sequence of n
operation is O(n)
operations
• Hence a sequence of n operations costs O(n2)

16 17

Aggregate Analysis Aggregate Analysis

• In fact, the total cost of n operations on an initially empty

stack is O(n) • Hence for any value of n, any sequence of n Push, Pop, and
• Why? Because each object can be popped at most once for Multipop operations on an initially empty stack takes O(n)
each time that it is pushed time
• Hence the number of times POP (including calls within Mul- • The average cost of an operation is thus O(n)/n = O(1)
tipop) can be called on a nonempty stack is at most the • Thus all stack operations have an amortized cost of O(1)
number of Push operations which is O(n)

18 19
 Another Example Binary Counter

Another example where we can use aggregate analysis:

• Initially x = 0 so A[i] = 0 for all i = 0, 1, . . . , k − 1

• Consider the problem of creating a k bit binary counter that
• To add 1 to the counter, we use a simple procedure which
counts upward from 0
scans the bits from right to left, zeroing out 1’s until it finally
• We use an array A[0..k − 1] of bits as the counter
find a zero bit which it flips to a 1
• A binary number x that is stored in A has its lowest-order bit
Pk−1
in A[0] and highest order bit in A[k − 1] (x = i=0 A[i] ∗ 2i)

20 21

Increment Analysis

Increment(A){
i = 0;
• It’s not hard to see that in the worst case, the increment
while(i<k && A[i]=1){
procedure takes time Θ(k)
A[i] = 0;
• Thus a sequence of n increments takes time O(nk) in the
i++;
worst case
}
• Note that again this bound is correct but overly pessimistic
if (i<k)
- not all bits flip each time increment is called!
A[i] = 1;
}

22 23
 Aggregate Analysis Aggregate Analysis

j k
• In general, for i = 0, . . . blog nc, bit A[i] flips n/2i times in a
• In fact, we can show that a sequence of n calls to Increment sequence of n calls to Increment on an initially zero counter
has a worst case time of O(n) • For i > blog nc, bit A[i] never flips
• A[0] flips every time Increment is called, A[1] flips over every • Total number of flips in the sequence of n calls is thus
other time, A[2] flips over every fourth time, . . .
blog
Xnc  n  ∞
• Thus if there are n calls to increment, A[0] flips n times, A[1] X 1
< n (10)
flips bn/2c times, A[2] flips bn/4c times i=0 2i i=0 2i

= 2n (11)

24 25

Aggregate Analysis Accounting or Taxation Method

• The second method is called the accounting method in the

• Thus the worst-case time for a sequence of n Increment
book, but a better name might be the taxation method
operations on an initially empty counter is O(n)
• Suppose it costs us a dollar to do a Push or Pop
• The average cost of each operation in the worst case then
• We can then measure the run time of our algorithm in dollars
is O(n)/n = O(1)
(Time is money!)

26 27
 Taxation Method for Multipop Taxation Method

• Instead of paying for each Push and Pop operation when they
• Like any good government (ha ha) we need to make sure
occur, let’s tax the pushes to pay for the pops
that: 1) our taxes are low and 2) we can use our taxes to
• I.e. we tax the push operation 2 dollars, and the pop and
pay for all our costs
multipop operations 0 dollars
• We already know that our taxes for n operations are no more
• Then each time we do a push, we spend one dollar of the
than 2n dollars
tax to pay for the push and then save the other dollar of the
• We now want to show that we can use the 2 dollars we collect
tax to pay for the inevitable pop or multipop of that item
for each push to pay for all the push, pop and multipop
• Note that if we do n operations, the total amount of taxes
operations
we collect is then 2n

28 29

Taxation Method Taxation Method

• This is easy to show. When we do a push, we use 1 dollar

of the tax to pay for the push and then store the extra dollar
with the item that was just pushed on the stack • We’ve shown that we can use the 2 tax on each item pushed
• Then all items on the stack will have one dollar stored with to pay for the cost of all pops, pushes and multipops.
them • Moreover we know that this taxation scheme collects at most
• Whenever we do a Pop, we can use the dollar stored with 2n dollars in taxes over n stack operations
the item popped to pay for the cost of that Pop • Hence we’ve shown that the amortized cost per operation is
• Moreover, whenever we do a Multipop, for each item that O(1)
we pop off in the Multipop, we can use the dollar stored with
that item to pay for the cost of popping that item

30 31
 Taxation Method for Binary Counter
Taxation Scheme

• Let’s tax the algorithm 2 dollars to set a bit to 1

• Let’s now use the taxation method to show that the amor- • Now we need to show that: 1) this scheme has low total
tized cost of the Increment algorithm is O(1) taxes and 2) we will collect enough taxes to pay for all of
• Let’s say that it costs us 1 dollar to flip a bit the bit flips
• What is a good taxation scheme to ensure that we can pay • Showing overall taxes are low is easy: Each time Increment
for the costs of all flips but that we keep taxes low? is called, it sets at most one bit to a 1
• So we collect exactly 2 dollars in taxes each time increment
is called
• Thus over n calls to Increment, we collect 2n dollars in taxes

32
33

Taxation Scheme Binary Counter

• We now need to show that our taxation scheme has enough

money to pay for the costs of all operations • We’ve shown that we can use the 2 tax each time a bit is
• This is easy: Each time we set a bit to a 1, we collect 2 set to a 1 to pay for all operations which flip a bit
dollars in tax. We use one dollar to pay for the cost of • Moreover we know that this taxation scheme collects 2n dol-
setting the bit to a 1, then we store the extra dollar on that lars in taxes over n calls to Increment
bit • Hence we’ve shown that the amortized cost per call to In-
• When the bit gets flipped back from a 1 to a 0, we use the crement is O(1)
dollar already on that bit to pay for the cost of the flip!
 In Class Exercise In Class Exercise

• A sequence of Pushes and Pops is performed on a stack • A sequence of Pushes and Pops is performed on a stack
whose size never exceeds k whose size never exceeds k
• After every k operations, a copy of the entire stack is made • After every k operations, a copy of the entire stack is made
for backup purposes for backup purposes
• Show that the cost of n stack operations, including copying • Show that the cost of n stack operations, including copying
the stack, is O(n) the stack, is O(n)

36 37

In Class Exercise

• Q1: What is your taxation scheme?

• Q2: What is the maximum amount of taxes this scheme
collects over n operations?
• Q3: Show that your taxation scheme can pay for the costs
of all operations