Amortized Analysis

In an amortized analysis, the time required to perform a sequence of

data-structure operations is averaged over all the operations
performed. Amortized analysis can be used to show that the average
cost of an operation is small, if one averages over a sequence of
operations, even though a single operation might be expensive.
Amortized analysis differs from average-case analysis in that
probability is not involved; an amortized analysis guarantees
the average performance of each operation in the worst case.
Starts with the aggregate method, in which we determine an upper
bound T(n) on the total cost of a sequence of n operations. The
amortized cost per operation is then T(n)/n.

The aggregate method

In the aggregate method of amortized analysis, we show that for all n,
a sequence of n operations takes worst-case time T(n) in total. In the
worst case, the average cost, or amortized cost, per operation is
therefore T(n) / n. Note that this amortized cost applies to each
operation, even when there are several types of operations in the
sequence. The other two methods we shall study in this chapter, the
accounting method and the potential method, may assign different
amortized costs to different types of operations.

Stack operations
In our first example of the aggregate method, we analyze stacks that
have been augmented with a new operation., each of which takes O(1)
time:

PUSH(S, x) pushes object x onto stack S.

POP(S) pops the top of stack S and returns the popped object.

Since each of these operations runs in O(1) time, let us consider the
cost of each to be 1. The total cost of a sequence
of n PUSH and POP operations is therefore n, and the actual running
time for n operations is therefore (n).
The situation becomes more interesting if we add the stack
operation MULTIPOP(S, k), which removes the k top objects of
stack S, or pops the entire stack if it contains less than k objects. In the
following pseudocode, the operation STACK-EMPTY returns TRUE if
there are no objects currently on the stack, and FALSE otherwise.
MULTIPOP(S,k)
1 while not STACK-EMPTY(S) and k 0
2 do POP(S)
3 k k - 1

example of MULTIPOP.

What is the running time of MULTIPOP(S, k) on a stack of s objects?

The actual running time is linear in the number of POP operations
actually executed, and thus it suffices to analyze MULTIPOP in terms
of the abstract costs of 1 each for PUSH and POP. The number of
iterations of the while loop is the number min(s, k) of objects popped
off the stack. For each iteration of the loop, one call is made to
POP in line 2. Thus, the total cost of MULTIPOP is min(s, k), and the
actual running time is a linear function of this cost.

The action of MULTIPOP on a stack S, shown initially in (a). The top 4

objects are popped by MULTIPOP(S, 4), whose result is shown in (b).
The next operation is MULTIPOP(S, 7), which empties the stack
shown in (c) since there were fewer than 7 objects remaining.

Let us analyze a sequence of n PUSH, POP,

and MULTIPOP operations on an initially empty stack. The worst-
case cost of a MULTIPOP operation in the sequence is O(n), since the
stack size is at most n. The worst-case time of any stack operation is
therefore O(n), and hence a sequence of n operations costs O(n2),
since we may have O(n) MULTIPOP operations costing O(n) each.
Although this analysis is correct, the O(n2) result, obtained by
considering the worst-case cost of each operation individually, is not
tight.

Using the aggregate method of amortized analysis, we can obtain a

better upper bound that considers the entire sequence of n operations.
In fact, although a single MULTIPOP operation can be expensive, any
sequence of n PUSH, POP, and MULTIPOP operations on an initially
empty stack can cost at most O(n). Why? Each object can be popped
at most once for each time it is pushed. Therefore, the number of
times that POP can be called on a nonempty stack, including calls
within MULTIPOP, is at most the number of PUSH operations, which
is at most n. For any value of n, any sequence of n PUSH, POP,
and MULTIPOP operations takes a total of O(n) time. The amortized
cost of an operation is the average: O(n)/n = O(1).

We emphasize again that although we have just shown that the

average cost, and hence running time, of a stack operation is O(1), no
probabilistic reasoning was involved. We actually showed a worst-
case bound of O(n) on a sequence of n operations. Dividing this total
cost by n yielded the average cost per operation, or the amortized
cost.

Incrementing a binary counter

As another example of the aggregate method, consider the problem of
implementing a k-bit binary counter that counts upward from 0. We
use an array A[0 . . k - 1] of bits, where length[A] = k, as the counter.
A binary number x that is stored in the counter has its lowest-order bit
in A[0] and its highest-order bit in A[k - 1], so that .
Initially, x = 0, and thus A[i] = 0 for i = 0, 1, . . . , k - 1. To add 1
(modulo 2k) to the value in the counter, we use the following
procedure.
An 8-bit binary counter as its value goes from 0 to 16 by a sequence of
16 INCREMENT operations. Bits that flip to achieve the next value are
shaded. The running cost for flipping bits is shown at the right. Notice
that the total cost is never more than twice the total number
of INCREMENT operations.

INCREMENT(A)
1 i 0
2 while i< length[A] and A[i] = 1
3 do A[i] 0
4 i i + 1
5 if i < length[A]
6 then A[i] 1

As Figure shows, A[0] does flip each time INCREMENT is called.

The next-highest-order bit, A[1], flips only every other time: a
sequence of n INCREMENT operations on an initially zero counter
causes A[1] to flip n/2 times. Similarly, bit A[2] flips only every
fourth time, or n/4 times in a sequence of n INCREMENT'S. In
general, for i = 0, 1, . . . , lg n , bit A[i] flips n/2i times in a sequence
of n INCREMENT operations on an initially zero counter. For i > lg n
, bit A[i] never flips at all. The total number of flips in the sequence is
thus
The accounting method
In the accounting method of amortized analysis, we assign differing
charges to different operations, with some operations charged more or
less than they actually cost. The amount we charge an operation is
called its amortized cost. When an operation's amortized cost exceeds
its actual cost, the difference is assigned to specific objects in the data
structure as credit. Credit can be used later on to help pay for
operations whose amortized cost is less than their actual cost. Thus,
one can view the amortized cost of an operation as being split
between its actual cost and credit that is either deposited or used up.
This is very different from the aggregate method, in which all
operations have the same amortized cost.

Stack operations
To illustrate the accounting method of amortized analysis, let us
return to the stack example. Recall that the actual costs of the
operations were
PUSH 1 ,
POP 1 ,
MULTIPOP min(k,s) ,

where k is the argument supplied to MULTIPOP and s is the stack size

when it is called. Let us assign the following amortized costs:
PUSH 2 ,
POP 0 ,
MULTIPOP 0 .
The potential method
Instead of representing prepaid work as credit stored with specific
objects in the data structure, the potential method of amortized
analysis represents the prepaid work as "potential energy,"or just
"potential," that can be released to pay for future operations. The
potential is associated with the data structure as a whole rather than
with specific objects within the data structure.
The potential method works as follows. We start with an initial data
structure D0 on which n operations are performed. For each i = 1, 2, . .
. , n, we let ci be the actual cost of the ith operation and Di be the data
structure that results after applying the ith operation to data
structure Di - l. A potential function maps each data structure Di to a
real number (Di), which is the potential associated with data
structure Di. The amortized cost of the ith operation with respect to
potential function is defined by

The amortized cost of each operation is therefore its actual cost plus
the increase in potential due to the operation. By equation (18.1), the
total amortized cost of the n operations is

If we can define a potential function so that (Dn) (D0), then the

total amortized cost is an upper bound on the total actual cost.
In practice, we do not always know how many operations might be
performed. Therefore, if we require that (Di) (D0) for all i, then
we guarantee, as in the accounting method, that we pay in advance. It
is often convenient to define (D0) to be 0 and then to show that
(Di) 0 for all i. Intuitively, if the potential difference (Di) - (Di - 1)
of the ith operation is positive, then the amortized cost represents
an overcharge to the ith operation, and the potential of the data
structure increases. If the potential difference is negative, then the
amortized cost represents an undercharge to the ith operation, and the
actual cost of the operation is paid by the decrease in the potential.

The amortized costs defined by equations (18.1) and (18.2) depend on

the choice of the potential function . Different potential functions
may yield different amortized costs yet still be upper bounds on the
actual costs. There are often trade-offs that can be made in choosing a
potential function; the best potential function to use depends on the
desired time bounds.

Stack operations
To illustrate the potential method, we return once again to the
example of the stack operations PUSH, POP, and MULTIPOP. We
define the potential function on a stack to be the number of objects
in the stack. For the empty stack D0 with which we start, we have
(D0) = 0. Since the number of objects in the stack is never negative,
the stack Di that results after the ith operation has nonnegative
potential, and thus
(Di) 0 = (D0).

The total amortized cost of n operations with respect to therefore

represents an upper bound on the actual cost.

Let us now compute the amortized costs of the various stack

operations. If the ith operation on a stack containing s objects is
a PUSH operation, then the potential difference is
(Di) - (Di - 1) = (s + 1 ) - s
= 1 .

By equation (18.1), the amortized cost of this PUSH operation is

Suppose that the ith operation on the stack is MULTIPOP(S,k) and

that k' = min(k,s) objects are popped off the stack. The actual cost of
the operation is k', and the potential difference is
 (Di) - (Di-1) = -k'.

Thus, the amortized cost of the MULTIPOP operation is

Similarly, the amortized cost of an ordinary POP operation is 0.

The amortized cost of each of the three operations is O(1), and thus
the total amortized cost of a sequence of n operations is O(n). Since
we have already argued that (Di) (D0), the total amortized cost
of n operations is an upper bound on the total actual cost. The worst-
case cost of n operations is therefore O(n).