CMSC 341

Amortized Analysis
What is amortized analysis?

 Consider a sequence of operations on a

dynamic data structure
 Insert or delete in any (valid) order
 Worst case analysis asks: What is the most
expensive any single operation can be?
 Amortized analysis asks: What is an upper
bound on the average per-operation cost
over the entire sequence?

UMBC CMSC 341 Amortized

10/6/2008 Analysis 2
Nature of the amortized
bound
 Amortized bounds are hard bounds
 They do not mean “on average (or most of
the time) the bound holds”
 They do mean “for any sequence of n
operations, the bound holds over that
sequence”

UMBC CMSC 341 Amortized

10/6/2008 Analysis 3
Three methods

 Aggregate method
 T(n) = upper bound on total cost of n operations
 Amortized cost is T(n)/n
 Some operations may cost more, a lot more, than
T(n)/n
 If so, some operations must cost less
 But the average cost over the sequence will never
exceed T(n)/n

UMBC CMSC 341 Amortized

10/6/2008 Analysis 4
Three methods

 Accounting method
 Each operation pays a “fee” (cost of operation)
 Overcharge some operations and store extra as
pre-payment for later operations
 Amortized cost is (total of fees paid)/n
 Must ensure bank account never negative,
otherwise fee was not high enough and bound
does not hold
 Overpayment stored with specific objects in data
structure (e.g., nodes in a BST)
UMBC CMSC 341 Amortized
10/6/2008 Analysis 5
Three methods (cont.)

 Potential method
 Like accounting method
 Overpayment stored as “potential energy” of
entire data structure (not specific objects)
 Must ensure that potential energy never falls
below zero

UMBC CMSC 341 Amortized

10/6/2008 Analysis 6
Increment a binary counter
A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 0 0

 K-bit value stored in array

 To increment value, flip bits right-to-left until
you turn a 0 into a 1
 Each bit flip costs O(1)
 What is the amortized cost of counting from 0
to n?
UMBC CMSC 341 Amortized
10/6/2008 Analysis 7
Increment a binary counter
A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 0 0

 Worst case
 Flip k bits per increment
 Do that n times to count to n
 O(kn)
 But, most of the time we don’t flip many bits

UMBC CMSC 341 Amortized

10/6/2008 Analysis 8
Increment a binary counter
A(k-1) A(k-2) A(2) A(1) A(0) Cost
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 1 1
0 0 0 0 0 0 0 0 1 0 2
0 0 0 0 0 0 0 0 1 1 1
0 0 0 0 0 0 0 1 0 0 3
0 0 0 0 0 0 0 1 0 1 1
0 0 0 0 0 0 0 1 1 0 2
0 0 0 0 0 0 0 1 1 1 1
0 0 0 0 0 0 1 0 0 0 4

Total: 15

UMBC CMSC 341 Amortized

10/6/2008 Analysis 9
Aggregate method

 A(0) flips every time, or n times

 A(1) flips every 2nd time, or n/2 times
 A(2) flips every 4th time, or n/4 times
 A(i) flips n/2i times

 Total cost is Σi=0,k-1n/2i ≤ Σi=0,∞n/2i = 2n = O(n)

 So amortized cost is O(n)/n = O(1)

UMBC CMSC 341 Amortized

10/6/2008 Analysis 10
Accounting method

 Flipping a bit costs $1 (one unit of

computational work)
 Pay $2 to change a 0 to a 1
 Use $1 to pay for flipping the bit to 1
 Leave $1 there to pay when/if the bit gets flipped
back to 0
 Since only one bit gets flipped to 1 per
increment, total cost is $2n = O(n)

UMBC CMSC 341 Amortized

10/6/2008 Analysis 11
Accounting method
A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 0 0
$0 $0 $0 $0 $0 $0 $0 $0 $0 $0

 Flip A(0) to 1 and pay $1 for flip and leave $1

with that bit (total fee of $2)

A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 0 1
$0 $0 $0 $0 $0 $0 $0 $0 $0 $1

UMBC CMSC 341 Amortized

10/6/2008 Analysis 12
Accounting method
A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 0 1
$0 $0 $0 $0 $0 $0 $0 $0 $0 $1

 Flip A(0) to 0 and pay with the $1 that was there already
 Flip A(1) to 1 and pay $1 for flip and leave $1 with that
bit (total fee of $2)

A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 1 0
$0 $0 $0 $0 $0 $0 $0 $0 $1 $0

UMBC CMSC 341 Amortized

10/6/2008 Analysis 13
Accounting method
A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 1 0
$0 $0 $0 $0 $0 $0 $0 $0 $1 $0

 Flip A(0) to 1 and pay $1 for flip and leave $1

with that bit (total fee of $2)

A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 1 1
$0 $0 $0 $0 $0 $0 $0 $0 $1 $1

UMBC CMSC 341 Amortized

10/6/2008 Analysis 14
Accounting method
A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 0 1 1
$0 $0 $0 $0 $0 $0 $0 $0 $1 $1

 Flip A(0) and A(1) to 0 and pay with the $$ that were there already
 Flip A(2) to 1 and pay $1 for flip and leave $1 with that bit (total fee
of $2)

A(k-1) A(k-2) A(2) A(1) A(0)

0 0 0 0 0 0 0 1 0 0
$0 $0 $0 $0 $0 $0 $0 $1 $0 $0

UMBC CMSC 341 Amortized

10/6/2008 Analysis 15
Accounting method

 … and so on
 We “overpay” by $1 for flipping each 0 to 1
 Use the extra $1 to pay for the cost of flipping
it back to a zero
 Because a $2 fee for each increment ensures
that we have enough money stored to
complete that increment, amortized cost is $2
= O(1) per operation

UMBC CMSC 341 Amortized

10/6/2008 Analysis 16
Potential method

 Record overpayments as “potential energy”

(or just “potential”) of entire data structure
 Contrast with accounting method where
overpayments stored with specific parts of
data structure (e.g., array cells)

UMBC CMSC 341 Amortized

10/6/2008 Analysis 17
Potential method

 Initial data structure is D0

 Perform operations i = 1, 2, 3, …, n
 The actual cost of operation i is ci
 The ith operation yields data structure Di

 Φ(Di) = potential of Di, or stored overpayment

 Amortized cost of the ith operation is
 xi = ci + Φ(Di) - Φ(Di-1)
UMBC CMSC 341 Amortized
10/6/2008 Analysis 18
Potential method
 If Φ(Di) - Φ(Di-1) > 0 then xi is an overcharge
to ith operation
 We paid more than the actual cost of the
operation

 If Φ(Di) - Φ(Di-1) < 0 then xi is an undercharge

to the ith operation
 We paid less than the actual cost of the operation,
but covered the difference by spending potential

UMBC CMSC 341 Amortized

10/6/2008 Analysis 19
Potential method

 Total amortized cost:

 Σxi = Σ(ci + Φ(Di) - Φ(Di-1)) = Σci + Φ(Dn) - Φ(D0)
 Sum of actual costs plus whatever potential we
added but didn’t use

 Require that Φ(Di) ≥ 0 so we always “pay in

advance”

UMBC CMSC 341 Amortized

10/6/2008 Analysis 20
Potential method: Binary
counter
 Need to choose potential function Φ(Di)
 Want to make xi = ci + Φ(Di) - Φ(Di-1) small
 Usually have to be lucky or clever!
 Let Φ(Di) = bi, the number of ones in the
counter after the ith operation
 Note that Φ(Di) ≥ 0 so we’re OK
 Recall $1 stored with each 1 in the array when using
the accounting method

UMBC CMSC 341 Amortized

10/6/2008 Analysis 21
Potential method: Binary
counter
 Operation i resets (zeroes) ti bits
 True cost of operation i is ti + 1
 The +1 is for setting a single bit to 1
 Number of ones in counter after ith operation
is therefore bi = bi-1 – ti + 1

UMBC CMSC 341 Amortized

10/6/2008 Analysis 22
Potential method: Binary
counter
 Number of ones in counter after ith operation
is bi = bi-1 – ti + 1
 Potential difference is
 Φ(Di) - Φ(Di-1) = bi – bi-1 = (bi-1–ti+1) – bi-1 = 1 – ti
 Amortized cost is
 xi = ci + Φ(Di) - Φ(Di-1) = (ti + 1) + (1 – ti) = 2
 If we pay just $2 per operation, we always have
enough potential to cover our actual costs per
operation
UMBC CMSC 341 Amortized
10/6/2008 Analysis 23
Amortized analysis of splay
trees
 Use the accounting method
 Store $$ with nodes in tree
 First, some definitions
 Let nx be the number of nodes in the subtree
rooted by node x
 Let rx = floor(log(nx))
 Called the rank of x

UMBC CMSC 341 Amortized

10/6/2008 Analysis 24
What we’ll show

 If every node x always has rx credits then

splay operations require amortized O(lgn)
time
 This is called the “credit invariant”
 For each operation (find, insert, delete) we’ll
have to show that we can maintain the credit
invariant and pay for the true cost of the
operation with O(lgn) $$ per operation

UMBC CMSC 341 Amortized

10/6/2008 Analysis 25
First things first

 Consider a single splay step

 Single rotation (no grandparent), zig-zig, or zig-
zag
 Nodes x, y = parent(x), z = parent(y)
 rx, ry, and rz are ranks before splay step
 r’x, r’y,and r’z are ranks after splay step

UMBC CMSC 341 Amortized

10/6/2008 Analysis 26
For example (zig-zig case)

rz r’x
z x
ry r’y
y D A y
rx r’z
x C B z

A B C D

UMBC CMSC 341 Amortized

10/6/2008 Analysis 27
What does a single splay step
cost?
 To pay for rotations (true cost of step) and
maintain credit invariant
 3(r’x – rx) + 1 credits suffice for single rotation
 3(r’x – rx) credits suffice for zig-zig and zig-zag

UMBC CMSC 341 Amortized

10/6/2008 Analysis 28
What does a sequence of
splay steps cost?
 As node x moves up the tree, sum costs of
individual steps
 r’x for one step becomes rx for next step
 Summing over all steps to the root telescopes to
become 3(rv – rx) + 1 where v is the root node
 3(r’x – rx) + 3(r’’x – r’x) + 3(r’’’x – r’’x) … + 1
 Note +1 only required (sometimes) for last step

UMBC CMSC 341 Amortized

10/6/2008 Analysis 29
The punch line!

 3(rv – rx) + 1 = O(logn)

 v is root node of tree with n nodes
 rv = floor(logn)
 We can splay any node to the root in O(logn)
time

UMBC CMSC 341 Amortized

10/6/2008 Analysis 30
Single rotation
ry r’x
y x
rx r’y
x C A y

A B B C

To maintain credit invariant at all nodes need to add $Δ

•Only rx and ry can change
•Δ = (r’x – rx) + (r’y – ry)
•Note that r’x = ry so …
• Δ = r’y – rx
•Note that r’x ≥ r’y so …
•Δ = r’y – rx ≤ r’x - rx
UMBC CMSC 341 Amortized
10/6/2008 Analysis 31
Single rotation
ry r’x
y x
rx r’y
x C A y

A B B C

To maintain credit invariant at all nodes it suffices to pay $(r’x – rx)

•Still need to pay O(1) for the rotation
•We allocated 3(r’x – rx) + 1 credits
•If r’x > rx we’ve still got 2(r’x – rx) > 1 credits to pay for the rotation
•The +1 is there in case r’x = rx
• When can that happen?

UMBC CMSC 341 Amortized

10/6/2008 Analysis 32
Zig-zig rz r’x
z x
ry r’y
y D A y
rx r’z
x C B z

A B C D

To maintain credit invariant at all nodes need to add $Δ

•Δ = (r’x – rx) + (r’y – ry) + (r’z – rz)
•Note that r’x = rz so …
•Δ = r’y + r’z – rx – ry
•Note that r’x ≥ r’y and r’x ≥ r’z and rx ≤ ry so …
•Δ = r’y + r’z – rx – ry ≤ r’x + r’x – rx – rx = 2(r’x – rx)

UMBC CMSC 341 Amortized

10/6/2008 Analysis 33
Zig-zig rz r’x
z x
ry r’y
y D A y
rx r’z
x C B z

A B C D

To maintain credit invariant at all nodes it suffices to pay $2(r’ x – rx)

•Still need to pay O(1) for the rotations
•If r’x > rx we can use r’x – rx ≥ 1 credits to pay for the two rotations for
a total of $3(r’x – rx)
•Otherwise, r’x = rx so r’x = rx = ry = rz
• Why?
•In this case, we can show that maintaining the invariant frees one or
more credits that can be used to pay for the rotations
UMBC CMSC 341 Amortized
10/6/2008 Analysis 34
Zig-zag

z
x

A y z y

x D
A B C D

B C

• Analysis analogous to zig-zig step

• At most $3(r’x – rx) required to maintain invariant and
pay for rotations

UMBC CMSC 341 Amortized

10/6/2008 Analysis 35