Medians and Order Statistics

Prof. Prateek Vishnoi

August 29, 2024

Medians and Order Statistics

Definition and Terminology

Let X ⊆ N and |X | = n

Prof. Prateek Vishnoi Medians and Order Statistics

Medians and Order Statistics

Definition and Terminology

Let X ⊆ N and |X | = n
ith order statistics of X

Prof. Prateek Vishnoi Medians and Order Statistics

Medians and Order Statistics

Definition and Terminology

Let X ⊆ N and |X | = n
ith order statistics of X
ith smallest element of the set.

Prof. Prateek Vishnoi Medians and Order Statistics

Medians and Order Statistics

Definition and Terminology

Let X ⊆ N and |X | = n
ith order statistics of X
ith smallest element of the set.
Minimum of X is first order statistic.

Prof. Prateek Vishnoi Medians and Order Statistics

Medians and Order Statistics

Definition and Terminology

Let X ⊆ N and |X | = n
ith order statistics of X
ith smallest element of the set.
Minimum of X is first order statistic.
Maximum of X is called nth order statistic.

Prof. Prateek Vishnoi Medians and Order Statistics

Medians and Order Statistics

Definition and Terminology

Let X ⊆ N and |X | = n
ith order statistics of X
ith smallest element of the set.
Minimum of X is first order statistic.
Maximum of X is called nth order statistic.
Median of X :
$ %th
n+1
order statistic of X
2

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach
Sort the X and return the i th element.

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach
Sort the X and return the i th element.
Time Complexity = Θ(n log n)

Second Approach

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach
Sort the X and return the i th element.
Time Complexity = Θ(n log n)

Second Approach
Divide and conquer approach.

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach
Sort the X and return the i th element.
Time Complexity = Θ(n log n)

Second Approach
Divide and conquer approach.
Apply the RANDOMISED PARTITION from quick sort on the
array by selecting a pivot.

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach
Sort the X and return the i th element.
Time Complexity = Θ(n log n)

Second Approach
Divide and conquer approach.
Apply the RANDOMISED PARTITION from quick sort on the
array by selecting a pivot.
Once the partition ends the array is divided into two parts
with (k − 1) elements on left array and (n − k) on right array.

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach
Sort the X and return the i th element.
Time Complexity = Θ(n log n)

Second Approach
Divide and conquer approach.
Apply the RANDOMISED PARTITION from quick sort on the
array by selecting a pivot.
Once the partition ends the array is divided into two parts
with (k − 1) elements on left array and (n − k) on right array.
If k= i, then RETURN.

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach
Sort the X and return the i th element.
Time Complexity = Θ(n log n)

Second Approach
Divide and conquer approach.
Apply the RANDOMISED PARTITION from quick sort on the
array by selecting a pivot.
Once the partition ends the array is divided into two parts
with (k − 1) elements on left array and (n − k) on right array.
If k= i, then RETURN.
If k < i, recursively call the RANDOMISED PARTITION on
right subarray.

Prof. Prateek Vishnoi Medians and Order Statistics

Find the i th order statistic of X , where 1 ≤ i ≤ n

First approach
Sort the X and return the i th element.
Time Complexity = Θ(n log n)

Second Approach
Divide and conquer approach.
Apply the RANDOMISED PARTITION from quick sort on the
array by selecting a pivot.
Once the partition ends the array is divided into two parts
with (k − 1) elements on left array and (n − k) on right array.
If k= i, then RETURN.
If k < i, recursively call the RANDOMISED PARTITION on
right subarray.
otherwise call on left subarray.
Prof. Prateek Vishnoi Medians and Order Statistics
Complexity Analysis

Best Case

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Best Case
Best Case occurs when the pivot selected is the i th order
statistic.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Best Case
Best Case occurs when the pivot selected is the i th order
statistic.
Recurrence Relation

T (n) = O(1) + Θ(n)

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Best Case
Best Case occurs when the pivot selected is the i th order
statistic.
Recurrence Relation

T (n) = O(1) + Θ(n)

Θ(n) requires for the partitioning.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Best Case
Best Case occurs when the pivot selected is the i th order
statistic.
Recurrence Relation

T (n) = O(1) + Θ(n)

Θ(n) requires for the partitioning.

O(1) requires for the returning the element.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Best Case
Best Case occurs when the pivot selected is the i th order
statistic.
Recurrence Relation

T (n) = O(1) + Θ(n)

Θ(n) requires for the partitioning.

O(1) requires for the returning the element.
Time Complexity = Θ(n)

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Worst Case occurs when the pivot selected divides array into
two parts of size (n − 1) and 0 and pivot is not the i th order
statistic.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Worst Case occurs when the pivot selected divides array into
two parts of size (n − 1) and 0 and pivot is not the i th order
statistic.
Recurrence Relation

T (n) = T (n − 1) + Θ(n)

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Worst Case occurs when the pivot selected divides array into
two parts of size (n − 1) and 0 and pivot is not the i th order
statistic.
Recurrence Relation

T (n) = T (n − 1) + Θ(n)

Θ(n) requires for the partitioning.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Worst Case occurs when the pivot selected divides array into
two parts of size (n − 1) and 0 and pivot is not the i th order
statistic.
Recurrence Relation

T (n) = T (n − 1) + Θ(n)

Θ(n) requires for the partitioning.

T (n − 1) requires for the recursive call.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Worst Case occurs when the pivot selected divides array into
two parts of size (n − 1) and 0 and pivot is not the i th order
statistic.
Recurrence Relation

T (n) = T (n − 1) + Θ(n)

Θ(n) requires for the partitioning.

T (n − 1) requires for the recursive call.
Time Complexity = O(n2 )

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

T (n) = (n − 1) + T (X )

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

T (n) = (n − 1) + T (X )
where X is a random variable s.t, 0 ≤ X ≤ (n − 1)

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

T (n) = (n − 1) + T (X )
where X is a random variable s.t, 0 ≤ X ≤ (n − 1)

T (n) = (n − 1) + E [T (X )]

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

T (n) = (n − 1) + T (X )
where X is a random variable s.t, 0 ≤ X ≤ (n − 1)

T (n) = (n − 1) + E [T (X )]
Possible splits of array :

(0, n−1), (1, n−2), (2, n−3) . . . (n/2−2, n/2+1), (n/2−1, n/2)

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

T (n) = (n − 1) + T (X )
where X is a random variable s.t, 0 ≤ X ≤ (n − 1)

T (n) = (n − 1) + E [T (X )]
Possible splits of array :

(0, n−1), (1, n−2), (2, n−3) . . . (n/2−2, n/2+1), (n/2−1, n/2)

Expected size of larger array= 3n/4

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

T (n) = (n − 1) + T (X )
where X is a random variable s.t, 0 ≤ X ≤ (n − 1)

T (n) = (n − 1) + E [T (X )]
Possible splits of array :

(0, n−1), (1, n−2), (2, n−3) . . . (n/2−2, n/2+1), (n/2−1, n/2)

Expected size of larger array= 3n/4

E [T (X )] ≤ 12 T ( 3n 1 1 3n 1
4 ) + 2 T (n − 1) ≤ 2 T ( 4 ) + 2 T (n)

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

T (n) = (n − 1) + T (X )
where X is a random variable s.t, 0 ≤ X ≤ (n − 1)

T (n) = (n − 1) + E [T (X )]
Possible splits of array :

(0, n−1), (1, n−2), (2, n−3) . . . (n/2−2, n/2+1), (n/2−1, n/2)

Expected size of larger array= 3n/4

E [T (X )] ≤ 12 T ( 3n 1 1 3n 1
4 ) + 2 T (n − 1) ≤ 2 T ( 4 ) + 2 T (n)
Place the bound on the upper equation.

Prof. Prateek Vishnoi Medians and Order Statistics

Average Case
Recurrence Relation

T (n) = (n − 1) + T (X )
where X is a random variable s.t, 0 ≤ X ≤ (n − 1)

T (n) = (n − 1) + E [T (X )]
Possible splits of array :

(0, n−1), (1, n−2), (2, n−3) . . . (n/2−2, n/2+1), (n/2−1, n/2)

Expected size of larger array= 3n/4

E [T (X )] ≤ 12 T ( 3n 1 1 3n 1
4 ) + 2 T (n − 1) ≤ 2 T ( 4 ) + 2 T (n)
Place the bound on the upper equation.
T (n) ≤ T ( 3n
4 ) + 2(n − 1) = O(n)
Prof. Prateek Vishnoi Medians and Order Statistics
Deterministic Algorithm for Selection

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Turing Award for this Paper

In 1972, a deterministic linear time algorithm was developed by
Manuel Blum, Bob Floyd, Vaughan Pratt, Ron Rivest, and Bob
Tarjan

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.
Recursively find the true median of the medians. Call this p.

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.
Recursively find the true median of the medians. Call this p.
Use p as a pivot to partition the array.

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.
Recursively find the true median of the medians. Call this p.
Use p as a pivot to partition the array.
Recurse on the appropriate piece in the same way as previous.

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.
Recursively find the true median of the medians. Call this p.
Use p as a pivot to partition the array.
Recurse on the appropriate piece in the same way as previous.

Example
A = {1,2,3,10,11,4,5,6,12,13,7,8,9,14,15}

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.
Recursively find the true median of the medians. Call this p.
Use p as a pivot to partition the array.
Recurse on the appropriate piece in the same way as previous.

Example
A = {1,2,3,10,11,4,5,6,12,13,7,8,9,14,15}
{ 1,2,3,10,11} , {4,5,6,12,13}, { 7,8,9,14,15}

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.
Recursively find the true median of the medians. Call this p.
Use p as a pivot to partition the array.
Recurse on the appropriate piece in the same way as previous.

Example
A = {1,2,3,10,11,4,5,6,12,13,7,8,9,14,15}
{ 1,2,3,10,11} , {4,5,6,12,13}, { 7,8,9,14,15}
{ 3,6,9}

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.
Recursively find the true median of the medians. Call this p.
Use p as a pivot to partition the array.
Recurse on the appropriate piece in the same way as previous.

Example
A = {1,2,3,10,11,4,5,6,12,13,7,8,9,14,15}
{ 1,2,3,10,11} , {4,5,6,12,13}, { 7,8,9,14,15}
{ 3,6,9}
p ={6}

Prof. Prateek Vishnoi Medians and Order Statistics

Deterministic Algorithm for Selection

Algorithm
I/P : Array A of size n and positive integer k ≤ n
Group the array into n/5 groups and find the median of each
group.
Recursively find the true median of the medians. Call this p.
Use p as a pivot to partition the array.
Recurse on the appropriate piece in the same way as previous.

Example
A = {1,2,3,10,11,4,5,6,12,13,7,8,9,14,15}
{ 1,2,3,10,11} , {4,5,6,12,13}, { 7,8,9,14,15}
{ 3,6,9}
p ={6}
5 elements at left and 9 elements at right.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Step 1 takes O(n) time.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Step 1 takes O(n) time.
Step 2 takes atmost T (n/5) time.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Step 1 takes O(n) time.
Step 2 takes atmost T (n/5) time.
Step 3(partitioning) takes O(n) time.

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Step 1 takes O(n) time.
Step 2 takes atmost T (n/5) time.
Step 3(partitioning) takes O(n) time.
Assume for now, that pivot divides the array of size atmost
7n/10
T (n) ≤ cn + T (n/5) + T (7n/10)

Prof. Prateek Vishnoi Medians and Order Statistics

Complexity Analysis

Worst Case
Step 1 takes O(n) time.
Step 2 takes atmost T (n/5) time.
Step 3(partitioning) takes O(n) time.
Assume for now, that pivot divides the array of size atmost
7n/10
T (n) ≤ cn + T (n/5) + T (7n/10)

T (n) = O(n)

Prof. Prateek Vishnoi Medians and Order Statistics

Justification for assumption

Prof. Prateek Vishnoi Medians and Order Statistics

Justification for assumption

Let there are n elements in an array.

Prof. Prateek Vishnoi Medians and Order Statistics

Justification for assumption

Let there are n elements in an array.

Let g = n/5

Prof. Prateek Vishnoi Medians and Order Statistics

Justification for assumption

Let there are n elements in an array.

Let g = n/5
Atleast ⌊g /2⌋ of them(groups having median ≤ p ), atleast 3
of 5 elements are ≤ p.

Prof. Prateek Vishnoi Medians and Order Statistics

Justification for assumption

Let there are n elements in an array.

Let g = n/5
Atleast ⌊g /2⌋ of them(groups having median ≤ p ), atleast 3
of 5 elements are ≤ p.
Total number of elements ≤ p is atleast 3n/10

Prof. Prateek Vishnoi Medians and Order Statistics