Scribd
Upload
29 views26 pages

CS 332: Algorithms: Amortized Analysis

The document discusses the analysis of disjoint set union operations like union and find. It explains that a naive implementation using linked lists would have worst case O(n^2) time for n union operations. An improved implementation is to always copy the smaller set into the larger set. This improves the worst case time of union to O(n log n) using amortized analysis. Specifically, each element can be copied at most log n times, so n union operations take O(n log n) time. Therefore, the amortized time of each union operation is O(log n).

Uploaded by

Anonymous niE5VQOH
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
29 views26 pages

CS 332: Algorithms: Amortized Analysis

The document discusses the analysis of disjoint set union operations like union and find. It explains that a naive implementation using linked lists would have worst case O(n^2) time for n union operations. An improved implementation is to always copy the smaller set into the larger set. This improves the worst case time of union to O(n log n) using amortized analysis. Specifically, each element can be copied at most log n times, so n union operations take O(n log n) time. Therefore, the amortized time of each union operation is O(log n).

Uploaded by

Anonymous niE5VQOH
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 26

CS 332: Algorithms

Amortized Analysis

Review: Running Time of


Kruskals Algorithm
Expensive operations:
Sort edges: O(E lg E)
O(V) MakeSet()s
O(E) FindSet()s
O(V) Union()s

Upshot:
Comes down to efficiency of disjoint-set

operations, particularly Union()

Review: Disjoint Set Union


So how do we represent disjoint sets?
Nave implementation: use a linked list to represent

elements, with pointers back to set:

MakeSet(): O(1)
FindSet(): O(1)
Union(A,B): Copy elements of A into set B by adjusting

elements of A to point to B: O(A)


How long could n Union()s take?

Review:
Disjoint Set Union Analysis
Worst-case analysis: O(n2) time for n Unions
Union(S1, S2) copy

1 element

Union(S2, S3) copy 2 elements

Union(Sn-1, Sn) copy n-1 elements


O(n2)

Improvement: always copy smaller into larger


How long would above sequence of Unions take?
Worst case: n Unions take O(n lg n) time
Proof uses amortized analysis

Review:
Amortized Analysis of Disjoint Sets
If elements are copied from the smaller set into

the larger set, an element can be copied at most


lg n times
Worst case: Each time copied, element in smaller set
1st time
resulting set size
2
2nd time
4

(lg n)th time


n

Review:
Amortized Analysis of Disjoint Sets
Since we have n elements each copied at most

lg n times, n Union()s takes O(n lg n) time


Therefore we say the amortized cost of a
Union() operation is O(lg n)
This is the aggregate method of amortized
analysis:
n operations take time T(n)
Average cost of an operation = T(n)/n

Amortized Analysis:
Accounting Method
Accounting method
Charge each operation an amortized cost
Amount not used stored in bank
Later operations can used stored money
Balance must not go negative

Book also discusses potential method


But we wont worry about it here

Accounting Method Example:


Dynamic Tables
Implementing a table (e.g., hash table) for

dynamic data, want to make it small as possible


Problem: if too many items inserted, table may
be too small
Idea: allocate more memory as needed

Dynamic Tables
1. Init table size m = 1
2. Insert elements until number n > m
3. Generate new table of size 2m
4. Reinsert old elements into new table
5. (back to step 2)
What is the worst-case cost of an insert?
One insert can be costly, but the total?

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)

Table Size
1

Cost
1

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)
Insert(2)

Table Size
1
2

Cost
1
1 + 1

1
2

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)
Insert(2)
Insert(3)

Table Size
1
2
4

Cost
1
1 + 1
1 + 2

1
2
3

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)
Insert(2)
Insert(3)
Insert(4)

Table Size
1
2
4
4

Cost
1
1 + 1
1 + 2
1

1
2
3
4

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)
Insert(2)
Insert(3)
Insert(4)
Insert(5)

Table Size
1
2
4
4
8

Cost
1
1 + 1
1 + 2
1
1 + 4

1
2
3
4
5

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)
Insert(2)
Insert(3)
Insert(4)
Insert(5)
Insert(6)

Table Size
1
2
4
4
8
8

Cost
1
1 + 1
1 + 2
1
1 + 4
1

1
2
3
4
5
6

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)
Insert(2)
Insert(3)
Insert(4)
Insert(5)
Insert(6)
Insert(7)

Table Size
1
2
4
4
8
8
8

Cost
1
1 + 1
1 + 2
1
1 + 4
1
1

1
2
3
4
5
6
7

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)
Insert(2)
Insert(3)
Insert(4)
Insert(5)
Insert(6)
Insert(7)
Insert(8)

Table Size
1
2
4
4
8
8
8
8

Cost
1
1 + 1
1 + 2
1
1 + 4
1
1
1

1
2
3
4
5
6
7
8

Analysis Of Dynamic Tables


Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Operation
Insert(1)
Insert(2)
Insert(3)
Insert(4)
Insert(5)
Insert(6)
Insert(7)
Insert(8)
Insert(9)

Table Size
1
2
4
4
8
8
8
8
16

Cost
1
1
1
1
1
1
1
1
1

+ 1
+ 2
+ 4

+ 8

1
2
3
4
5
6
7
1
8
2
9

Aggregate Analysis
n Insert() operations cost
n

c
i 1

lg n

n 2 n (2n 1) 3n
j

j 0

Average cost of operation

= (total cost)/(# operations) < 3


Asymptotically, then, a dynamic table costs the
same as a fixed-size table
Both O(1) per Insert operation

Accounting Analysis
Charge each operation $3 amortized cost
Use $1 to perform immediate Insert()
Store $2

When table doubles


$1 reinserts old item, $1 reinserts another old item
Point is, weve already paid these costs
Upshot: constant (amortized) cost per operation

Disjoint Set Union


So how do we implement disjoint-set union?
Nave implementation: use a linked list to

represent each set:

MakeSet(): ??? time


FindSet(): ??? time
Union(A,B): copy elements of A into B: ??? time

Disjoint Set Union


So how do we implement disjoint-set union?
Nave implementation: use a linked list to represent each

set:

MakeSet(): O(1) time


FindSet(): O(1) time
Union(A,B): copy elements of A into B: O(A) time

How long can a single Union() take?


How long will n Union()s take?

Disjoint Set Union: Analysis


Worst-case analysis: O(n2) time for n Unions
Union(S1, S2)

copy

1 element

Union(S2, S3)

copy

2 elements

Union(Sn-1, Sn) copy

n-1 elements

O(n2)

Improvement: always copy smaller into larger


Why will this make things better?
What is the worst-case time of Union()?

But now n Unions take only O(n lg n) time!

Amortized Analysis of Disjoint Sets


Amortized analysis computes average times without

using probability
With our new Union(), any individual element is copied
at most lg n times when forming the complete set from
1-element sets
Worst case: Each time copied, element in smaller set

1st time
resulting set size
2nd time
4

(lg n)th time n

Amortized Analysis of Disjoint Sets


Since we have n elements each copied at most lg n

times, n Union()s takes O(n lg n) time


We say that each Union() takes O(lg n) amortized
time
Financial term: imagine paying $(lg n) per Union
At first we are overpaying; initial Union $O(1)
But we accumulate enough $ in bank to pay for later

expensive O(n) operation.


Important: amount in bank never goes negative

The End

You might also like