Amortized Analysis

Amortized Analysis
• Not just consider one operation, but a sequence
of operations on a given data structure.
• Average cost over a sequence of operations.
• Used to show that average cost of an operation is
small even though a single operation might be
expensive.
• Probabilistic analysis:
– Average case running time: average over all
possible inputs for one algorithm (operation).
– If using probability, called expected running
time.
• Amortized analysis:
– No involvement of probability
– Average performance on a sequence of
operations, even some operation is
expensive.
– Guarantee average performance of each
operation among the sequence in worst case.
 Three Methods of Amortized Analysis
• Aggregate analysis:
– Total cost of n operations/n,
• Accounting method:
– Assign each type of operation an (different) amortized
cost
– Overcharge some operations,
– Store the overcharge as credit on specific objects,
– Use the credit for compensation for some later
operations.
• Potential method:
– Same as accounting method
– But store the credit as “potential energy” and as a whole.
 The Aggregate Method

• Show that sequence of n operations takes

– Worst case time T(n) in total for all n
• The amortized cost (average cost in the worst
case) per operation is therefore T(n)n
• This amortized cost applies to each operation
– Even when there are several types of
operations in the sequence

5
 Example: Stack Operations
PUSH(S, x): pushed object x onto stack
POP(S): pops the top of the stack S and returns the
popped object
MULTIPOP(S, k): removes the k top objects of the
stack S or pops the entire stack if | S |  k
• PUSH and POP runs in (1) time
– The total cost of a sequence of n PUSH and
POP operations is therefore (n)
• The running time of MULTIPOP(S, k) is
– (min(s, k)) where s  | S |

6
Stack Operations: Multipop

MULTIPOP(S, k)
while not StackEmpty(S) and k  0 do
t  POP(S)
k  k 1
return Running time:
(min(s, k)) where s  | S |

7
 Cont.
• Consider a sequence of n POP, PUSH, and MULTIPOP
operations on an initially empty stack.
• Worst case of a MULTIPOP operation in the sequence is
O(n), since the stack size is at most n.
• Hence, a sequence of n operations costs O(n2).
– may have n MULTIPOP operations each costing O(n).
• The analysis is correct, however,
– Considering worst-case cost of each operation, it is not
tight.
• We can obtain a better bound by using aggregate method
of amortized analysis

8
 Cont.
• Aggregate method considers the entire sequence
of n operations
– Although a single MULTIPOP can be expensive
– Any sequence of n POP, PUSH, and MULTIPOP
operations on an initially empty sequence can cost at
most O(n).
• Each object can be POP only once (including in
MULTIPOP) for each time it is PUSHed.
#POPs is at most #PUSHs, which is at most n.
The amortized cost of an operation is the
average O(n)n  O(1).
9
 Incrementing a Binary Counter

• Implementing a k-bit binary counter that counts

upward from 0
• Use array A[0k1] of bits as the counter
where length[A]k
A[0] is the least significant bit
A[k1] is the most significant bit

k1
i.e., xi0A[i]2i

10
 Cont.
Initially x  0, i.e., A[i]  0 for i  0,1, , k1
To add 1 (mod 2k) to the counter
Essentially same as the one
implemented in hardware by a
INCREMENT(A, k) ripple-carry counter
i0  A single execution of
while i  k and A[i]  1 do
A[i]  0 increment takes (k) in the
i  i +1 worst case in which array A
if i  k then contains all 1’s
A[i]  1
 Thus, n increment operations
return
on an initially zero counter
takes O(kn) time in the worst
case. NOT TIGHT 11
 The Aggregate Method:
Incrementing a Binary Counter
Counter Incre cost Total cost
value
[7] [6] [5] [4] [3] [2] [1] [0]

0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 1 1 1
2 0 0 0 0 0 0 1 0 2 3
3 0 0 0 0 0 0 1 1 1 4
4 0 0 0 0 0 1 0 0 3 7
5 0 0 0 0 0 1 0 1 1 8 Bits that flip to
6 0 0 0 0 0 1 1 0 2 10 achieve the
7 0 0 0 0 0 1 1 1 1 11 next value are
8 0 0 0 0 1 0 0 0 4 15 shaded
9 0 0 0 0 1 0 0 1 1 16
10 0 0 0 0 1 0 1 0 2 18
11 0 0 0 0 1 0 1 1 1 19

12
 Cont.
• Note that, the running time of an increment operation is
proportional to the number of bits flipped
• However, all bits are not flipped at each INCREMENT
A[0] flips at each increment operation
A[1] flips at alternate increment operations
A[2] flips only once for 4 successive increment
operations
• In general, bit A[i] flips n/2i times in a sequence of n
INCREMENTs

13
 Cont.
• Therefore, the total number of flips in the
sequence is
lg n 

i0
n/2i  n 
i0
1/2i  2n

• The amortized cost of each operation is

O(n)n  O(1)

14
 Amortized Analysis: Accounting
Method
• Idea:
– Assign differing charges to different operations.
– The amount of the charge is called amortized cost.
– amortized cost is more or less than actual cost.
– When amortized cost > actual cost, the difference is
saved in specific objects as credits.
– The credits can be used by later operations whose
amortized cost < actual cost.
• As a comparison, in aggregate analysis, all operations
have same amortized costs.
 Cont.
• Conditions:
– suppose actual cost is ci for the ith operation in the
sequence, and amortized cost is ci',
– i=1n ci' i=1n ci should hold.
• Since we want to show the average cost per operation is
small using amortized cost, we need the total amortized
cost is an upper bound of total actual cost.
• Holds for all sequences of operations.
– Total credits is i=1n ci' - i=1n ci , which should be
nonnegative,
• Moreover, i=1t ci' - i=1t ci ≥0 for any t>0.
 The Accounting Method: Stack
Operations
Assign the following amortized costs:
Push: 2 Pop: 0 Multipop: 0

Notes:
• Amortized cost of multipop is a constant (0),
whereas the actual cost is variable
• All amortized costs are O(1), however, in general,
amortized costs of different operations
may differ asymptotically

Suppose we use $1 bill top represent each unit of cost 17

 Cont.
 Start with an empty stack of plates
 When we push a plate on the stack
• we use $1 to pay the actual cost of the push
operation
• we put a credit of $1 on top of the pushed plate
 At any time point, every plate on the stack has a $1 of
credit on it
 The $1 stored on the plate is a prepayment for the cost
of popping it
 In order to pop a plate from the stack
• we take $1 of credit off the plate
• and use it to pay the actual cost of the pop operation18
 Cont.
Thus by charging the push operation a little bit more we
don’t need to charge anything from the pop & multipop
operations
We have ensured that the amount of credits is always
nonnegative
• since each plate on the stack always has $1 of credit
• and the stack always has a nonnegative number of
plates

Thus, for any sequence of n push, pop, multipop

operations the total amortized cost is an upper bound
on the total actual cost 19
 Incrementing a binary counter
 Recall that, the running time of an increment
operation is proportional to the number of bits flipped
 Charge an amortized cost of $2 to set a bit to 1
When a bit is set
• we use $1 to pay for the actual setting of the bit and
• we place the other $1 on the bit as credit
 At any time point, every 1 in the counter has a $1 of
credit on it
 Hence, we don’t need to charge anything to reset a bit
to 0, we just pay for the reset with the $1 on it

20
 Cont.
• The amortized cost of increment can now be
determined the cost of resetting bits within the while
loop is paid by the dollars on the bits that are reset
• At most one bit is set to 1, in an increment operation
• Therefore, the amortized cost of an increment
operation is at most 2 dollars
• The number of 1’s in the counter is never negative,
thus the amount of credit is always nonnegative
• Thus, for n increment operations, the total amortized
cost is O(n), which bounds the actual cost

21
 The Potential Method
• Accounting method represents prepaid work as credit
stored with specific objects in the data structure
• Potential method represents the prepaid work as
potential energy or just potential that can be released to
pay for the future operations
• The potential is associated with the data structure as a
whole rather than with specific objects within the data
structure

22
 Cont.
We start with an initial data structure D0 on which we
perform n operations
For each i 1, 2, …, n, let
Ci: the actual cost of the i-th operation
Di: data structure that results after applying i-th
operation to Di1
 : potential function that maps each data structure Di
to a real number  (Di)
 (Di): the potential associated with data structure Di
Ĉ i: amortized cost of the i-th operation w.r.t. function
 23
 Cont.
Cˆ i  Ci   ( Di )   ( Di 1 )
actual increase in potential
cost due to the operation

The total amortized cost of n operations is

n n

 i  i
Cˆ  (C   ( D )   ( D ))
i 1 i 1
i i 1

n
  Ci   ( Dn )   ( D0 )
i 1

24
 Cont.
 If we can ensure that  (Di)   (D0) then
n
the total amortized cost  Cˆ i is an upper bound on the
i 1
total actual cost
 However,  (Dn)   (D0) should hold for all possible
n since, in practice, we do not always know n in
advance
 Hence, if we require that  (Di)   (D0), for all i, then
we ensure that we pay in advance (as in the accounting
method) 25
 Cont.
If  (Di)  (Di1) > 0, then the amortized cost Ĉ i represents
• an overcharge to the i-th operation and
• the potential of the data structure increases
If  (Di)   (Di1) < 0, then the amortized cost Ĉ i represents
• an undercharge to the i-th operation and
• the actual cost of the operation is paid by the decrease in
potential
Different potential functions may yield different amortized costs
which are still upper bounds for the actual costs
The best potential fn. to use depends on the desired time bounds

26
 The Potential Method:
Stack Operations

• Define  (S)| S |, the number of objects in the stack

• For the initial empty stack, we have  (D0)  0
• Since |S|  0, stack Di that results after ith operation
has nonnegative potential for all i, that is
 (Di)  0   (D0) for all i
• Total amortized cost is an upper bound on total actual cost
• Let us compute the amortized costs of stack operations where
ith operation is performed on a stack with s objects

27
 The Potential Method:
Stack Operations
PUSH (S):  (Di)   (Di1)  (s 1)  (s)  1
Cˆi  Ci   (Di)  (Di1)  1  1  2

MULTIPOP(S, k):  (Di)  (Di1)   k'   min{s, k}

Cˆi  Ci   (Di)  (Di1)  k'  k'  0
POP (S): Cˆ 0, similarly
i

• The amortized cost of each operation is O(1), and thus

the total amortized cost of a sequence of n operations
is O(n)
28
 The Potential Method: Incrementing
a Binary Counter
• Define  (Di) bi, number of 1s in the counter after the ith
operation
• Compute the amortized cost of an INCREMENT operation wrt 
• Suppose that ith INCREMENT resets ti bits then,
ti  Ci  ti 1
• The number of 1s in the counter after the ith operation is
bi1 ti  bi  bi1 ti 1  bi  bi1  1 ti
• The amortized cost is therefore
Cˆi  Ci   (Di)  (Di1)  (ti 1)  (1 ti)  2

29
 The Potential Method: Incrementing
a Binary Counter
• If the counter starts at zero, then  (D0)  0, the
number of 1s in the counter after the ith operation
• Since  (Di)  0 for all i the total amortized cost is an
upper bound on the total actual cost
• Hence, the worst-case cost of n operations is O(n)

30
 The Potential Method:
Incrementing a Binary Counter
• Assume that the counter does not start at zero, i.,e., b00
• Then, after n INCREMENT operations the number of 1s is bn,
where 0  b0, bn  k
n n
 C i
i1 i1
C  n (D )   (D )   2bnb0
i n 0

ˆ
i  1 2nb b
n 0
• Since b0 k, if we execute at least n  (k) INCREMENT
operations the total actual cost is O(n)
• No matter what initial value the counter contains

31