Randomized Quicksort

Instructor: Ashok Singh Sairam

Lecture Plan
• Algorithm Randomized QuicksortRan
• Analysis of Randomized Quicksort
§ Random variables
§ Expectation
§ Indicator random variables

MA512: Data Structures and Algorithms

2
Recap - 1
• Apply Quick sort on a given sequence 7 11 14 6 9 4 3 12.
What is the sequence after first phase, pivot is first
element?
A) 6 4 3 7 11 9 14 12
B) 6 3 4 7 9 14 11 12
C) 7 6 14 11 9 4 3 12
D) 7 6 4 3 9 14 11 12

MA512: Data Structures and Algorithms

3
Recap - 2
• Consider the Quicksort algorithm which sorts elements in
ascending order using the first element as pivot. Then
which of the following input sequence will require a
maximum number of comparisons ?
A) 22 25 56 67 89
B) 52 25 76 67 89
C) 22 25 76 67 50
D) 52 25 89 67 76

How do you ensure average #comparisons?

MA512: Data Structures and Algorithms

4
Randomization!
• How do we always ensure a good split?
§ Add randomization to obtain good average performance
• Randomly permute the input array before sorting
• OR ... modify the PARTITION procedure
§ At each step use a randomly chosen element from
A[p…r] as pivot
§ It can be implemented by exchanging pivot element
A[r] with a random element from A[p…r]

MA512: Data Structures and Algorithms

5
Randomized Quick sort Algorithm
• Implement the swap before actual partitioning

• The randomized algorithm calls randomized-

partition

MA512: Data Structures and Algorithms

6
Analysis of Randomized Quicksort
• The maximum time spent by quick sort is in the
partition procedure
• For analysis, we need to answer the following
question
1. What is the maximum #times procedure
partition is called?
2. What is the maximum time spent in the
procedure during each call?

MA512: Data Structures and Algorithms

7
Analysis of Randomized Quicksort
RANDOMIZED-QUICKSORT(A, p, r)
The running time of Quicksort is
if p < r
dominated by PARTITION !!
then q ← RANDOMIZED-PARTITION(A, p, r)

RANDOMIZED-QUICKSORT(A, p, q - 1)

RANDOMIZED-QUICKSORT(A, q + 1, r)
PARTITION is called at most n times, each call takes
O(1) plus the number of comparisons
(at each call a pivot is selected and never again included
in future calls) 8
Analysis of Randomized Quicksort
•

MA512: Data Structures and Algorithms

9
Time spent in procedure partition
PARTITION(A, p, r)
x ← A[r] O(1) - constant
i←p-1
for j ← p to r - 1
# of comparisons: X
do if A[ j ] ≤ x
between the pivot and
then i ← i + 1
the other elements
exchange A[i] ↔ A[j]

exchange A[i + 1] ↔ A[r]

return i + 1 O(1) - constant

MA512: Data Structures and Algorithms

10
•

MA512: Data Structures and Algorithms

11
Notation
• z2 z9 z8 z3 z5 z4 z1 z6 z10 z7
A 2 9 8 3 5 4 1 6 10 7

MA512: Data Structures and Algorithms

12
Random Variables
(Discrete) random variable X: a function from a
sample space S to the real numbers.
§ It associates a real number with each possible
outcome of an experiment.

X(j)

MA512: Data Structures and Algorithms

13
Random Variable: example
• Experiment: Toss a coin three times
• X represents the number of times head comes up

MA512: Data Structures and Algorithms

14
Expectation
• Expected value (expectation, mean) of a discrete
random variable X is:
E[X] = Σx x Pr{X = x}
§ “Average” over all possible values of random variable X

MA512: Data Structures and Algorithms

15
 Event X

ttt 0

tth 1

tht 1

thh 2

htt 1

hth 2

hht 2

hhh 3

MA512: Data Structures and Algorithms

16
Indicator Random Variables
• Given a sample space S and an event A, the
indicator random variable I{A} associated with A:
§ I{A} =
1 if A occurs
0 if A does not occur
• The expected value of an indicator random
variable XA=I{A} is:
E[XA] = Pr {A}
• E[XA] = E[I{A}] = 1  Pr{A} + 0  Pr{Ā}

MA512: Data Structures and Algorithms

17
Indicator Random Variables:
number of comparisons in
partition()
• Define Xij = I {zi is compared to zj }
• Total number of comparisons X performed by the
algorithm
§ Each pair is compared at most once

i n-1

i+1 n

MA512: Data Structures and Algorithms

18
Expected No. of Total Comparisons

by linearity indicator
of expectation random variable

the expectation of Xij is equal to the probability of the

event “zi is compared to zj”

[Linearity of expectation: Exp. of sum = Sum of Exp.]

MA512: Data Structures and Algorithms

19
Comparisons in PARTITION :
Observation 1
• Each pair of elements is compared at most once
during the entire execution of the algorithm
§ Elements are compared only to the pivot point!
§ Pivot point is excluded from future calls to PARTITION

MA512: Data Structures and Algorithms

20
Comparisons in PARTITION:
Observation 2
• Only the pivot is compared with elements in both
partitions!
z2 z9 z8 z3 z5 z4 z1 z6 z10 z7
2 9 8 3 5 4 1 6 10 7

Z1,6= {1, 2, 3, 4, 5, 6} {7} Z8,9 = {8, 9, 10}

pivot

Elements between different partitions

are never compared!

MA512: Data Structures and Algorithms

21
Comparisons in PARTITION
• Case 1: pivot chosen such as: zi < x < zj
§ zi and zj will never be compared
• Case 2: zi or zj is the pivot
§ zi and zj will be compared
§ only if one of them is chosen as pivot before any other
element in range zi to zj
z2 z9 z8 z3 z5 z4 z1 z6 z10 z7
2 9 8 3 5 4 1 6 10 7

Z1,6= {1, 2, 3, 4, 5, 6} {7} Z8,10 = {8, 9, 10}

MA512: Data Structures and Algorithms

22
Probability of comparing zi with zj

• There are j – i + 1 elements between zi and zj

– Pivot is chosen randomly and independently
– The probability that any particular element is the first
one chosen is 1/( j - i + 1)

MA512: Data Structures and Algorithms

23
Number of Comparisons in PARTITION
• Expected number of comparisons in PARTITION:

(set k=j-i) (harmonic series)

 Expected running time of Quicksort using

RANDOMIZED-PARTITION is O(nlgn)
MA512: Data Structures and Algorithms
24
Comparison Merge sort, Quicksort
Criteria Merge sort Quicksort
Dataset Works well on any Work well on small
dataset, large or dataset
small
In-place No, requires auxiliary Yes
sorting space for merging
Locality of Does NOT exhibit Exhibits good locality
reference good locality of of reference, thus
reference runs faster in virtual
memory environment

MA512: Data Structures and Algorithms

25
Acknowledgement
• Dr George Bebis, Foundation Professor, Dept of
Computer Science and Engineering, University of
Nevada Reno

MA512: Data Structures and Algorithms

26
Hoare’s Partition
• Hoare’s strategy: “Burn the candle from both ends!”
• Move items bigger than pivot to the end
• Move items smaller than pivot to beginning
• Items still to be examined are in the middle
• Keep two indexes into array to separate the 3 sections
§ These indexes move towards each other. Done when they meet .
A[p…i]  x x  A[j…r]

i j
MA512: Data Structures and Algorithms
27
Hoare’s Partition: pseudo code

MA512: Data Structures and Algorithms

28
Home work
• Demonstrate the operation of HOARE-PARTITION on the
array A ={13, 19, 9, 5, 12, 8, 7, 4, 11, 2, 6, 21}, showing the
values of the array and auxiliary values after each iteration
of the while loop in lines 4–13.

MA512: Data Structures and Algorithms

29