Beyond Worst Case Analysis

Average Case Analysis Worst-case analysis.

■ Analyze running time as function of worst input of a given size.

Average case analysis.

■ Analyze average running time over some distribution of inputs.
■ Ex: quicksort.

Amortized analysis.
■ Worst-case bound on sequence of operations.
■ Ex: splay trees, union-find.

Competitive analysis.
■ Make quantitative statements about online algorithms.
■ Ex: paging, load balancing.

Princeton University • COS 423 • Theory of Algorithms • Spring 2001 • Kevin Wayne 2

Average Case Analysis Quicksort

Average case analysis. Quicksort.
■ Analyze average running time over some distribution of inputs. ■ Assume all elements are unique.
■ Ex: quicksort. ■ Assume input is a random permutation of inputs.
– O(N log N) if input is assumed to be in random order. ■ Denote ith largest element by i.
– leads to randomized algorithm with O(N log N) expected running
time, independent of input
■ Major disadvantage: hard to quantify what input distributions will quicksort.c
look like in practice. void quicksort(Item a[], int left, int right) {
int m;
if (right > left) {
m = partition(a, left, right);
quicksort(a, left, m - 1);
quicksort(a, m + 1, right);
}
}

3 4
 Quicksort: BST Representation of Pivots Quicksort: Average Case Analysis
7 6 12 3 11 8 7 1 15 13 17 5 16 14 9 4 10 Probability that i = 2 and j = 7 get compared.
7 6 4 3 9 8 2 1 5 10 17 15 16 14 11 12 13 ■ Let x be pivot element that separates i and j.
1 2 4 3 5 8 6 7 9 10 12 11 13 14 15 17 16 ■ Case 1: x ∈ {3, 4, 5, 6} ⇒ i and j not compared.
1 2 3 4 5 8 6 7 9 10 11 12 13 14 15 16 17
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

10 pivot elements 10

5 13 5 13

3 9 11 16 3 9 11 16

2 4 7 12 15 17 2 4 7 12 15 17

1 6 8 14 5
1 6 8 14 6

Quicksort: Average Case Analysis Quicksort: Average Case Analysis

Probability that i = 2 and j = 7 get compared. Probability that i = 2 and j = 7 get compared.
■ Let x be pivot element that separates i and j. ■ Let x be pivot element that separates i and j.
■ Case 1: x ∈ {3, 4, 5, 6} ⇒ i and j not compared. ■ Case 1: x ∈ {3, 4, 5, 6} ⇒ i and j not compared.
■ Case 2: x ∈ {2, 7} ⇒ i and j are compared. ■ Case 2: x ∈ {2, 7} ⇒ i and j are compared.

2
10 Pr[ i and j compared ( i < j ) ] =
j −i +1
7 13

3 9 11 16

2 5 8 12 15 17

1 4 6 14 7 8
 Quicksort: Average Case Analysis Comparison Based Sorting Lower Bound
Quicksort: average case analysis.
a1 < a2

2 YES NO
Pr[ i and j compared ( i < j ) ] =
j −i +1
a2 < a3 a1 < a3
N i YES NO YES NO
2 1
Expected # of comparisons = ∑ = 2∑ ∑
j−i+1
■

1≤ i < j ≤ N i =1 j = 2 j
N 1 1, 2, 3 a1 < a3 2, 1, 3 a2 < a3
≤ 2N ∑
j =1 j
YES NO YES NO
1
≈ 2N ∫
j 1, 3, 2 3, 1, 2 2, 3, 1 3, 2, 1
Remark: same analysis if we choose partition
= 2 N ln N
element uniformly at random.
■ Running time independent of input.

Decision Tree of Program

9 10

Comparison Based Sorting Lower Bound

Lower bound = Ω(N log2N): applies to comparison-based algorithms.
■ Tree height h determines worst case running time.
■ N! different input orderings.
■ One (or more) leaves corresponds to each ordering.
■ Binary tree with N! leaves has height:

h ≥ log 2 ( N ! )
≥ log 2 ( N / e ) N Stirling’s formula
= N log 2 N − N log 2 e
= Ω ( N log 2 N)

11