Advanced Algorithms

Median Findings and Order Statistics

Dr. Rubi Quiñones
Computer Science Department
Southern Illinois University Edwardsville
 Course Timeline
• Introduction to algorithms

ALGORITH

ANALYSIS
MIC
• Median findings and order statistics
• Time complexity
STRATEGIES • Activity selection problems
GREEDY

• Water connection problem, Egyptian fraction

• Huffman (de)coding
• shelves, mice, and policemen problem
•
CONQUER

Max subarray sum and Nearest neighbor

DIVIDE
AND

• Newton’s and bisection algorithm

• Matrix multiplication, skyline and hanoi
• Fibonacci and path counting
PROGRAMMING

• Coin row and collecting problem

DYNAMIC

• Matrix chain multiplication and longest common subsequence

• Knapsack and optimal binary trees
• Floyd Warshall Algorithm and A*
Concep

•
Advanc

Algorithm intractability
ed

ts

• Randomized algorithms 2
 Median Findings
• General Selection Problem
• Minimum
• Minimum and Maximum

3
 General Selection Problem
• We are starting off learning the basic concepts – Finding a value.
• The General Selection Problem:
• If I want to find the ith element from a set of n distinct numbers, how can we do that?

4
 General Selection Problem
• We are starting off learning the basic concepts - Finding a value.
• The General Selection Problem:
• If I want to find the ith element from a set of n distinct numbers, how can we do that?
1. One Way: Iterating through your set of n distinct numbers until you find the ith element.
1. Time complexity: ?

5
 General Selection Problem
• We are starting off learning the basic concepts - Finding a value.
• The General Selection Problem:
• If I want to find the ith element from a set of n distinct numbers, how can we do that?
1. One Way: Iterating through your set of n distinct numbers until you find the ith element.
1. Time complexity: ?
2. Second Way: Sort the set of n distinct numbers, find the median number, see if it the median value is less
than or greater than the ith element, and that will determine which chunk you’ll look at and keep going
from there. (Divide and conquer)
1. Time complexity: ?

6
 ith Order Statistics
• Let’s find some ith order statistics!
• Given an array A of n distinct numbers, the ith order statistic of A is its ith smallest element
• 𝑖 = 1 ⟹ 𝑚𝑖𝑛𝑖𝑚𝑢𝑚
• 𝑖 = 𝑛 ⟹ 𝑚𝑎𝑥𝑖𝑚𝑢𝑚
𝑛+1
• If n is odd,median is 𝑖 =
2
𝑛+1 𝑛 𝑛+1 𝑛
• If n is even, median is = (lower median) and = + 1 (upper median)
2 2 2 2

7
 Minimum(A)
Given an array A of n elements and each element is an integer, write pseudocode to find
the minimum element. Do not rearrange A.

8
 Minimum(A)
Given an array A of n elements and each element is an integer, write pseudocode to find
the minimum element. Do not rearrange A.

9
 Efficiency of Minimum(A)
How many times is the loop executed?

10
 Efficiency of Minimum(A)
How many times is the loop executed? n-1 comparisons
Can we do better?

11
 Efficiency of Minimum(A)
How many times is the loop executed? n-1 comparisons
Can we do better? NO
Lower bound: any algorithm finding minimum of n elements will need at least n-1
comparisons
• Proof of this comes from fact that no element of A can be considered for elimination as
the minimum until it's been shown to be greater than at least one other element
• Imagine that all elements are still eligible to the smallest are in a bucket, and are
removed only after it is shown to be > some other elements
• Since each comparison removes at most one element from the bucket, at least n-1
comparisons are needed to remove all but one from the bucket

12
 Maximum(A)
Given an array A of n elements and each element is an integer, write pseudocode to find the maximum
element. Do not rearrange A.

• Produce pseudocode.
• Determine its efficiency and number of iterations.

13
 Simultaneous Minimum and Maximum
Given array A with n elements, find both its minimum and maximum

• What is the obvious algorithm?

• What is its (non-asymptotic) time complexity?
• Can we do better?

14
Simultaneous Minimum and Maximum

15
 Explanation of Minimum and Maximum
• Idea: for each pair of values examined in the loop, compare them directly
• For each such pair, compare the smaller one to small and the larger one to large
• Try it with: A = [8, 5, 3, 10, 4, 12, 6]

16
 Explanation of Minimum and Maximum
• Idea: for each pair of values examined in the loop, compare them directly
• For each such pair, compare the smaller one to small and the larger one to large
• Try it with: A = [8, 5, 3, 10, 4, 12, 6]

• Initialization: large=8, small=5

• Compare 3 to 10: large=max(8,10)=10
small=min(5,3) = 3
• Compare 4 to 12: large=max(10,12)=12
small=min(3,4) = 3
• Final: large=max(12,6)=12
small=min(3,6)=3

17
 Efficiency of Minimum and Maximum
• How many comparisons does MinAndMax make?

18
 Efficiency of Minimum and Maximum
• How many comparisons does MinAndMax make?
• Initialization in Line 1 and line 2 requires two comparisons , for a total of 2
• Each iteration in the loop requires one comparison between A[2i-1] and A[2i] and then one comparison to
each of the large and small, for a total of 4
• Lines 8 and 9 require a comparison, total 2
• Total is at most (<=):
𝑛
• 2+4 − 1 + 2 = 3n/2 = 2n (if n is odd)
2
𝑛−1
• 2+4 + 2 = 2𝑛 − 2 (if n is even)
2

19
 Efficiency of Minimum and Maximum
• How many comparisons does MinAndMax make?
• Initialization in Line 1 and line 2 requires two comparisons , for a total of 2
• Each iteration in the loop requires one comparison between A[2i-1] and A[2i] and then one comparison to each of the large and small, for a
total of 4
• Lines 8 and 9 require a comparison, total 2
• Total is at most (<=):
𝑛
• 2+4 2
− 1 + 2 = 3n/2 = 2n (if n is odd)
𝑛−1
• 2+4 2
+ 2 = 2𝑛 − 2 (if n is even)

• Considering if we were to do min and max individually

• 𝑚𝑖𝑛𝑖𝑚𝑢𝑚 𝑛 − 1 + 𝑚𝑎𝑥𝑖𝑚𝑢𝑚 𝑛 − 1 = 2𝑛 − 2
• -1 for the worse case scenario (even)
• What's the total for finding minimum and maximum separately?
• 2𝑛 − 3

20
 Efficiency of Minimum and Maximum

What is the time and space complexity?

Which approach would be best if we had 10

samples or 100 samples?

21
 Course Timeline
• Introduction to algorithms

ALGORITH

ANALYSIS
MIC
• Median findings and order statistics
• Time complexity
STRATEGIES • Activity selection problems
GREEDY

• Water connection problem, Egyptian fraction

• Huffman (de)coding
• shelves, mice, and policemen problem
•
CONQUER

Max subarray sum and Nearest neighbor

DIVIDE
AND

• Newton’s and bisection algorithm

• Matrix multiplication, skyline and hanoi
• Fibonacci and path counting
PROGRAMMING

• Coin row and collecting problem

DYNAMIC

• Matrix chain multiplication and longest common subsequence

• Knapsack and optimal binary trees
• Floyd Warshall Algorithm and A*
Concep

•
Advanc

Algorithm intractability
ed

ts

• Randomized algorithms 22