Amortized Analysis

IT Department RCOEM 1
Analyzing Calls to a Data Structure
• Some algorithms involve repeated calls to one or more
data structures
• Example: Heap sort
– repeatedly insert keys into a priority queue (heap)
– repeatedly remove the smallest key from the heap
• When analyzing the running time of the overall
algorithm, we need to sum up the time spent in all the
calls to the data structure
• When different calls take different times, how can we
accurately calculate the total time?

IT Department RCOEM 2
 Heap sort Example
• Each of the n calls to insert into the heap operates on
a heap with at most n elements
• Inserting into a heap with n elements takes O(log n)
time
• So total time spent doing the insertions is O(n log n)
time
• But maybe this is an over-estimate!
– different insertions take different amounts of time
– many of the insertions are on significantly smaller heaps

IT Department RCOEM 3
 Amortized Analysis
• Purpose is to accurately compute the total time spent
in executing a sequence of operations on a data
structure
Basic Idea:

In amortized analysis, the time required to perform a sequence

of operations is averaged over all the operations performed.

 No involvement of probability
 Average performance on a sequence of operations,
even if some operation is expensive
 Guarantee average performance of each operation
among the sequence in worst case

IT Department RCOEM 4
 Methods of Amortized Analysis
• Aggregate Method: we determine an upper
bound T(n) on the total sequence of n operations.
The cost of each will then be T(n)/n.
• Accounting Method: we charge the operations.
We overcharge some operations early and use
them to as prepaid charge later for other
operations.
• Potential Method: we maintain credit as
potential energy associated with the structure as
a whole.
IT Department RCOEM 5
 Aggregate method
 Show that for all n, a sequence of n operations
take worst-case time T(n) in total

 In the worst case, the average cost, or

amortized cost , per operation is T(n)/n.

 The amortized cost applies to each operation,

even when there are several types of operations
in the sequence.

IT Department RCOEM 6
Example #1: Stack operations
Operations are:
– Push(S,x)
– Pop(S)
– Multipop(S,k) - pop the top k elements
Implement with either array or linked list
– time for Push is O(1)
– time for Pop is O(1)
– time for Multipop is O(min(|S|,k))

IT Department RCOEM 7
Sequence of n push, pop, Multipop
operations
Worst-case cost of Multipop is O(n)
Have n operations
Therefore, worst-case cost of sequence is
O(n2)

IT Department RCOEM 8
Observations
Each object can be popped only once per time
that it’s pushed
We have <= n pushes --> <= n pops, including
those in Multipop
Therefore total cost of sequence = O(n)
Average over n operations => O(1) per operation
on average

Notice that no probability involved

IT Department RCOEM 9
Example #2: k-Bit Counter A
• k-bit Binary Counter: A[0..k1]
K-1 ……… 0

x   ik01 A [ i ]  2 i
INCREMENT(A)
1. i  0
2. while i < length[A] and A[i] = 1
3. do A[i]  0 reset a bit
4. ii+1
5. if i < length[A]
6. then A[i]  1 set a bit

IT Department RCOEM 10
 Aggregate Method for k-Bit Counter
• Worst-case time for an increment is O(k),
occurs when all k bits are flipped
• But in a sequence of n operations, not all of
them will cause all k bits to flip
– bit 0 flips with every increment
– bit 1 flips with every 2nd increment
– bit 2 flips with every 4th increment …
– bit k flips with every 2k-th increment

IT Department RCOEM 11
 Aggregate Method for k-Bit Counter
• Total number of bit flips in n increment
operations is
– n + n/2 + n/4 + … + n/2k < 2n

Cost of n increments

• So total cost of the sequence is O(n).

• Amortized cost per operation is O(n)/n = O(1).

IT Department RCOEM 12
 Accounting Method
• Assign a cost, called the "amortized cost", to
each operation
• Assignment must ensure that the sum of all
the amortized costs in a sequence is at least
the sum of all the actual costs
– remember, we want an upper bound on the total
cost of the sequence
• How to ensure this property?

IT Department RCOEM 13
 Accounting Method
• Every operation is assigned an amortized cost
• For each operation in the sequence:
– if amortized cost > actual cost then store extra as a credit with
an object in the data structure
– if amortized cost < actual cost then use the stored credits to
make up the difference
• If ci is the actual cost and ci^ is the amortized cost then,
we must ensure that for all n :

• Thus, the total amortized costs provide an upper bound

on the total true costs.

IT Department RCOEM 14
 Accounting Method for Stack
• Assign the amortized costs:
– Push - 2
– Pop - 0
– Multipop - 0
• For Push, actual cost is 1. Store the extra 1 as a
credit, associated with the pushed element
• Pay for each popped element (either from Pop or
Multipop) using the associated credit

IT Department RCOEM 15
Accounting Method: Stack Example

3 ops:

Push(S,x) Pop(S) Multi-pop(S,k)

•Assigned
2 0 0
cost:
•Actual cost: 1 1 min(|S|,k)

Note: Push(S,x) pays for possible later pop of x.

IT Department RCOEM 16
 Accounting Method for Stack
• When pushing an object, pay Rs.2
Rs.1 pays for the push
Rs.1 is prepayment for it being popped by
either pop or Multipop
Since each object has Rs.1, which is credit,
the credit can never go negative
Therefore, total amortized cost = O(n), is an
upper bound on total actual cost

IT Department RCOEM 17
 Accounting Method for k-Bit Counter
• Charge an amortized cost of Rs.2 every time a
bit is set from 0 to 1
• Rs.1 pays for the actual bit setting.
• Rs.1 is stored for later re-setting (from 1 to 0)
• At any point, every 1 bit in the counter has
Rs.1 on it… that pays for resetting it.
(reset is “free”)

IT Department RCOEM 18
Accounting Method for k-Bit Counter
1

0 0 0 0 0 0 0 1 0 0

1 1 1

0 0 0 0 1 0 0 1 0 1

1 1 1

0 0 0 1 0 0 0 1 1 0

1 1 1 1 1

0 0 0 1 1 0 0 1 1 1

CPSC 411, Fall 2008: Set 6 19

 When Incrementing,
• Amortized cost for line 3 = Rs.0
• Amortized cost for line 6 = Rs.2
 Amortized cost for INCREMENT(A) = Rs.2
 Amortized cost for n INCREMENT(A) = Rs.2n =O(n)

IT Department RCOEM 20
 Accounting Method vs. Aggregate
Method
• Aggregate method:
– first analyze entire sequence
– then calculate amortized cost per operation
• Accounting method:
– first assign amortized cost per operation
– then compute cost of entire sequence of
operations, this gives per operation cost

IT Department RCOEM 21
 Potential Method
• Similar to accounting method
• Amortized costs are assigned in a more complicated
way
– based on a potential function
– and the current state of the data structure
• Must ensure that sum of amortized costs of all
operations in the sequence is at least the sum of the
actual costs of all operations in the sequence.

IT Department RCOEM 22
 Potential Method
• Define potential function  which maps any
state of the data structure to a real number
• Notation:
– D0 - initial state of data structure
– Di - state of data structure after i-th operation
– ci - actual cost of i-th operation
– mi - amortized cost of i-th operation

IT Department RCOEM 23
 Potential Method
• Define amortized cost of i-th operation:
mi = ci + (Di) – (Di–1)
• This is the actual cost plus the change in the
potential from previous state to current state
• Sum of all the amortized costs is sum of all the
actual costs plus (Dn) — (D0)
– must ensure this last term is nonnegative

IT Department RCOEM 24
 Potential Function
• Usually  is defined so that
– (D0) = 0 and
– (Di) ≥ 0
• so it is easy to see that (Dn) — (D0) is
nonnegative

IT Department RCOEM 25
 Potential Method for Stack
• Define (Di) to be number of elements in the
stack after the i-th operation
• Check:
– (D0) = 0, since stack is initially empty
– (Di) ≥ 0, since can't have a negative number of
elements in the stack
• Next calculate amortized cost of each
operation…

IT Department RCOEM 26
Potential Method for Augmented Stack
• If i-th operation is a Push and stack has s elements:
– mi = ci + (Di) — (Di-1)
= 1 + (s+1) — s
=2
• If i-th operation is a pop and stack has s elements:
– mi = ci + (Di) — (Di-1)
= 1 + (s–1) — s
=0

IT Department RCOEM 27
Potential Method for Augmented Stack
• If i-th operation is a Multipop(k) and stack has s
elements:
– Let x = min(s,k)
– mi = ci + (Di) — (Di-1)
= x + (s–x) — s
=0
• All operations have O(1) amortized time
• So cost of entire sequence is O(n)

IT Department RCOEM 28
Potential Method for k-Bit Counter
• Define (Di) to be number of 1's in the
counter after the i-th operation
• Check:
– (D0) = 0, since counter is initially all 0's
– (Di) ≥ 0, since can't have a negative number of
1's in the counter
• Next calculate amortized cost of the
increment operation…

IT Department RCOEM 29
Potential Method for k-Bit Counter
• Let b = number of 1's just before i-th op
• Let x = number of 1's that are changed to 0 in
i-th op
• mi = ci + (Di) — (Di-1)
= (x+1) + (b–x+1) – b x 1's are changed to 0
=2 and one 0 is changed to 1

• All ops have O(1) amortized time

• So total cost of sequence is O(n)

IT Department RCOEM 30