CS4311
Design and Analysis of
Algorithms
Lecture 8: Order Statistics
1
About this lecture
•Finding max, min in an unsorted array
(upper bound and lower bound)
•Finding both max and min (upper bound)
•Selecting the kth smallest element
kth smallest element kth order statistics
2
Finding Maximum
in unsorted array
3
Finding Maximum (Method I)
• Let S denote the input set of n items
• To find the maximum of S, we can:
Step 1: Set max = item 1
Step 2: for k = 2, 3, …, n
if (item k is larger than max)
Update max = item k;
Step 3: return max;
# comparisons = n –1
4
Finding Maximum (Method II)
• Define a function Find-Max as follows:
Find-Max(R, k) /* R is a set with k items */
1. if (k 2) return maximum of R;
2. Partition items of R into bk/2c pairs;
3. Delete smaller item from R in each pair;
4. return Find-Max(R, k - bk/2c);
Calling Find-Max(S,n) gives the maximum of S
5
Finding Maximum (Method II)
Let T(n) = # comparisons for Find-Max with
problem size n
So, T(n) = T(n -bn/2c) + bn/2c for n ¸ 3
T(2) = 1
Solving the recurrence (by substitution),
we get T(n) = n - 1
6
Lower Bound
Question: Can we find the maximum using
fewer than n –1 comparisons?
Answer: No ! To ensure that an item x is
not the maximum, there must be at least
one comparison in which x is the smaller
of the compared items
So, we need to ensure n-1 items not max
at least n –1 comparisons are needed
7
Finding Both Max and Min
in unsorted array
8
Finding Both Max and Min
Can we find both max and min quickly?
Solution 1:
First, find max with n –1 comparisons
Then, find min with n –1 comparisons
Total = 2n –2 comparisons
Is there a better solution ??
9
Finding Both Max and Min
Better Solution: (Case 1: if n is even)
First, partition items into n/2 pairs;
Next, compare items within each pair;
…
= larger = smaller
10
Finding Both Max and Min
Then, max = Find-Max in larger items
min = Find-Min in smaller items
Find-Max Find-Min
# comparisons = 3n/2 –2
11
Finding Both Max and Min
Better Solution: (Case 2: if n is odd)
We find max and min of first n - 1 items;
if (last item is larger than max)
Update max = last item;
if (last item is smaller than min )
Update min = last item;
# comparisons = 3(n-1)/2
12
Finding Both Max and Min
Conclusion:
To find both max and min:
if n is odd: 3(n-1)/2 comparisons
if n is even: 3n/2 –2 comparisons
Combining: at most 3bn/2c comparisons
better than finding max and min separately
13
Lower Bound
Textbook Ex 9.1-2 (Very challenging):
• Show that we need at least
d3n/2e–2 comparisons
to find both max and min in worst-case
Hint: Consider how many numbers may be
max or min (or both). Investigate how a
comparison affects these counts
14
Selecting kth smallest item
in unsorted array
15
Selection in Linear Time
• In next slides, we describe a recursive call
Select(S,k)
which supports finding the kth smallest
element in S
• Recursion is used for two purposes:
(1) selecting a good pivot (as in Quicksort)
(2) solving a smaller sub-problem
16
Select(S, k)
/* First,find a good pivot */
1. Partition S into d|S|/5e groups, each
group has five items (one group may
have fewer items);
2. Sort each group separately;
3. Collect median of each group into S’;
4. Find median m of S’
:
m = Select(S’
,dd|S|/5e/2e);
17
4. Let q = # items of S smaller than m;
5. If (k == q + 1)
return m;
/* Partition with pivot */
6. Else partition S into X and Y
X = {items smaller than m}
Y = {items larger than m}
/* Next,form a sub-problem */
7. If (k q + 1)
return Select(X, k)
8. Else
return Select(Y, k– (q+1));
18
Selection in Linear Time
Questions:
1. Why is the previous algorithm correct?
(Prove by Induction)
2. What is its running time?
19
Running Time
• In our selection algorithm, we chose m,
which is the median of medians, to be a
pivot and partition S into two sets X and Y
• In fact, if we choose any other item as the
pivot, the algorithm is still correct
• Why don’ t we just pick an arbitrary pivot
so that we can save some time ??
20
Running Time
• A closer look reviews that the worst-case
running time depends on |X| and |Y|
• Precisely, if T(|S|) denote the worst-case
running time of the algorithm on S, then
T(|S|) = T(d|S|/5e) + (|S|)
+ max {T(|X|),T(|Y|) }
21
Running Time
• Later, we show that if we choose m, the
“median of medians”, as the pivot,
both |X| and |Y| will be at most 3|S|/4
• Consequently,
T(n) = T(dn/5e) + (n) + T(3n/4)
T(n) = (n) (obtained by substitution)
22
Median of Medians
• Let’
s begin with dn/5e sorted groups, each
has 5 items (one group may have fewer)
= larger = median = smaller
23
Median of Medians
• Then, we obtain the median of medians, m
Groups with median Groups with median
smaller than m =m larger than m 24
Median of Medians
Then, we know that all items marked with
X have value at most m
X X
X
X X X X
X X
X X
X
Groups with median
smaller than m =m X=“
value m”
25
Median of Medians
The number of items with value at most m
is at least
3(ddn/5e/2e–1) - 2
one group may have
each full group has min # of only 1 ‘
crossed’
item
3‘crossed’ items groups
number of items: at least 3n/10 –5
26
Median of Medians
Previous page implies that at most
7n/10 + 5 items
are greater than m
For large enough n (say, n 100)
7n/10 + 5 3n/4
|Y| is at most 3n/4 for large enough n
27
Median of Medians
Similarly, we can show that at most
7n/10 + 5 items are smaller than m
|X| is at most 3n/4 for large enough n
Conclusion:
The “ median of medians”helps us control
the worst-case size of the sub-problem
without it, the algorithm runs in (n2)
time in the worst-case
28
