Divide-and-Conquer:

Quick Sort
Alexander S. Kulikov
Steklov Institute of Mathematics at St. Petersburg
Russian Academy of Sciences

Algorithmic Toolbox
Data Structures and Algorithms
 Outline
1 Overview

2 Algorithm

3 Random Pivot

4 Running Time Analysis

5 Equal Elements

6 Final Remarks
 Quick Sort

comparison based algorithm

running time: O(n log n) (on average)
efficient in practice
Example: quick sort

6 4 8 2 9 3 9 4 7 6 1
Example: quick sort

6 4 8 2 9 3 9 4 7 6 1
partition with respect to x = A[1]
in particular, x is in its final position
1 4 2 3 4 6 6 9 7 8 9
≤6 >6
Example: quick sort

6 4 8 2 9 3 9 4 7 6 1
partition with respect to x = A[1]
in particular, x is in its final position
1 4 2 3 4 6 6 9 7 8 9
sort the two parts recursively
1 2 3 4 4 6 6 7 8 9 9
 Outline
1 Overview

2 Algorithm

3 Random Pivot

4 Running Time Analysis

5 Equal Elements

6 Final Remarks
QuickSort(A, ℓ, r )
if ℓ ≥ r :
return
m ← Partition(A, ℓ, r )
{A[m] is in the final position}
QuickSort(A, ℓ, m − 1)
QuickSort(A, m + 1, r )
 ℓ r
A
 ℓ r
A

m ← Partition(A, ℓ, r )

A ≤x x >x
m
 ℓ r
A

m ← Partition(A, ℓ, r )

A ≤x x >x
m
QuickSort(A, ℓ, m − 1) QuickSort(A, m + 1, r )

A x
sorted
Partitioning: example
the pivot is x = A[ℓ]
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 9 8 9 4 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 9 8 9 4 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 9 8 9 4 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 9 8 9 4 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 9 8 9 4 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 8 9 9 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 8 9 9 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 8 9 9 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 8 9 9 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 8 9 9 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 8 9 9 7 6 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 6 9 9 7 8 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 6 9 9 7 8 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 6 9 9 7 8 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 6 9 9 7 8 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i

ℓ r
6 4 2 3 4 6 9 9 7 8 1
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i
in the end, move A[ℓ] to its final place

ℓ r
6 4 2 3 4 6 1 9 7 8 9
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i
in the end, move A[ℓ] to its final place

ℓ r
6 4 2 3 4 6 1 9 7 8 9
j i
Partitioning: example
the pivot is x = A[ℓ]
move i from ℓ + 1 to r maintaining the
following invariant:
A[k] ≤ x for all ℓ + 1 ≤ k ≤ j
A[k] > x for all j + 1 ≤ k ≤ i
in the end, move A[ℓ] to its final place

ℓ r
1 4 2 3 4 6 6 9 7 8 9
j i
Partition(A, ℓ, r )
x ← A[ℓ] {pivot}
j ←ℓ
for i from ℓ + 1 to r :
if A[i] ≤ x:
j ←j +1
swap A[j] and A[i]
{A[ℓ + 1 . . . j] ≤ x, A[j + 1 . . . i] > x}
swap A[ℓ] and A[j]
return j
 Outline
1 Overview

2 Algorithm

3 Random Pivot

4 Running Time Analysis

5 Equal Elements

6 Final Remarks
 Unbalanced Partitions

T (n) = n + T (n − 1):

T (n) = n+(n−1)+(n−2)+· · · = Θ(n2)

 Unbalanced Partitions

T (n) = n + T (n − 1):

T (n) = n+(n−1)+(n−2)+· · · = Θ(n2)

T (n) = n + T (n − 5) + T (4):

T (n) ≥ n+(n−5)+(n−10)+· · · = Θ(n2)

 Balanced Partitions

T (n) = 2T (n/2) + n:

T (n) = Θ(n log n)

 Balanced Partitions

T (n) = 2T (n/2) + n:

T (n) = Θ(n log n)

T (n) = T (n/10) + T (9n/10) + n:

T (n) = Θ(n log n)

 Balanced Partitions
T (n) = T (n/10) + T (9n/10) + O(n)
 Balanced Partitions
T (n) = T (n/10) + T (9n/10) + O(n)

log10 n
log10/9 n
 Balanced Partitions
T (n) = T (n/10) + T (9n/10) + O(n)

log10 n
log10/9 n

T (n) = O(n log n)

 Random Pivot
RandomizedQuickSort(A, ℓ, r )
if ℓ ≥ r :
return
k ← random number between ℓ and r
swap A[ℓ] and A[k]
m ← Partition(A, ℓ, r )
{A[m] is in the final position}
RandomizedQuickSort(A, ℓ, m − 1)
RandomizedQuickSort(A, m + 1, r )
 Why Random?
half of the elements of A guarantees a
balanced partition:

sorted A
n/4 n/2 n/4
Theorem
Assume that all the elements of A[1 . . . n] are
pairwise different. Then the average running
time of RandomizedQuickSort(A) is
O(n log n) while the worst case running time
is O(n2).
Theorem
Assume that all the elements of A[1 . . . n] are
pairwise different. Then the average running
time of RandomizedQuickSort(A) is
O(n log n) while the worst case running time
is O(n2).

Remark
Averaging is over random numbers used by
the algorithm, but not over the inputs.
 Outline
1 Overview

2 Algorithm

3 Random Pivot

4 Running Time Analysis

5 Equal Elements

6 Final Remarks
Proof Ideas: Comparisons
the running time is proportional to the
number of comparisons made
Proof Ideas: Comparisons
the running time is proportional to the
number of comparisons made
balanced partition are better since they
reduce the number of comparisons
needed:
5 1 2 4 7 3 6

1 5 4 3 6 7 2 3 1 2 4 6 5 7
1 is min 3, 1, 2 < 6, 5, 7
Proof Ideas: Probability
A 5 1 8 9 2 4 7 3 6
A′ 1 2 3 4 5 6 7 8 9
Proof Ideas: Probability
A 5 1 8 9 2 4 7 3 6
A′ 1 2 3 4 5 6 7 8 9

Prob (1 and 9 are compared) =

Proof Ideas: Probability
A 5 1 8 9 2 4 7 3 6
A′ 1 2 3 4 5 6 7 8 9

2
Prob (1 and 9 are compared) =
9
Proof Ideas: Probability
A 5 1 8 9 2 4 7 3 6
A′ 1 2 3 4 5 6 7 8 9

2
Prob (1 and 9 are compared) =
9

Prob (3 and 4 are compared) =

Proof Ideas: Probability
A 5 1 8 9 2 4 7 3 6
A′ 1 2 3 4 5 6 7 8 9

2
Prob (1 and 9 are compared) =
9

Prob (3 and 4 are compared) = 1

 Proof
let, for i < j,
{︃
1 A′[i] and A′[j] are compared
𝜒ij =
0 otherwise
 Proof
let, for i < j,
{︃
1 A′[i] and A′[j] are compared
𝜒ij =
0 otherwise

for all i < j, A′[i] and A′[j] are either

compared exactly once or not compared
at all (as we compare with a pivot)
 Proof
let, for i < j,
{︃
1 A′[i] and A′[j] are compared
𝜒ij =
0 otherwise

for all i < j, A′[i] and A′[j] are either

compared exactly once or not compared
at all (as we compare with a pivot)
this, in particular, implies that the worst
case running time is O(n2)
 Proof (continued)

crucial observation: 𝜒ij = 1 iff the first

selected pivot in A′[i . . . j] is A′[i] or
A′[j]
 Proof (continued)

crucial observation: 𝜒ij = 1 iff the first

selected pivot in A′[i . . . j] is A′[i] or
A′[j]
2
then Prob(𝜒ij ) = j−i+1 and
2
E(𝜒ij ) = j−i+1
 Proof (continued)
Then (the expected value of) the running
time is
n ∑︁
∑︁ n n ∑︁
∑︁ n
E 𝜒ij = E(𝜒ij )
i=1 j=i+1 i=1 j=i+1
∑︁ 2
=
j −i +1
i<j
(︂ )︂
1 1 1
≤ 2n · + + ... +
2 3 n
= Θ(n log n)
 Outline
1 Overview

2 Algorithm

3 Random Pivot

4 Running Time Analysis

5 Equal Elements

6 Final Remarks
 Equal Elements

what if all the elements of the given

array are equal to each other?
 Equal Elements

what if all the elements of the given

array are equal to each other?
quick sort visualization
 Equal Elements

what if all the elements of the given

array are equal to each other?
quick sort visualization
the array is always split into two parts of
size 0 and n − 1
 Equal Elements

what if all the elements of the given

array are equal to each other?
quick sort visualization
the array is always split into two parts of
size 0 and n − 1
T (n) = n + T (n − 1) + T (0) and hence
T (n) = Θ(n2)!
To handle equal elements, we replace the line

m ← Partition(A, ℓ, r )

with the line

(m1, m2) ← Partition3(A, ℓ, r )

such that
for all ℓ ≤ k ≤ m1 − 1, A[k] < x
for all m1 ≤ k ≤ m2, A[k] = x
for all m2 + 1 ≤ k ≤ r , A[k] > x
 ℓ r
A

(m1, m2) ← Partition3(A, ℓ, r )

ℓ r
A <x =x >x
m1 m2
RandomizedQuickSort(A, ℓ, r )
if ℓ ≥ r :
return
k ← random number between ℓ and r
swap A[ℓ] and A[k]
(m1, m2) ← Partition3(A, ℓ, r )
{A[m1 . . . m2] is in final position}
RandomizedQuickSort(A, ℓ, m1 − 1)
RandomizedQuickSort(A, m2 + 1, r )
 Outline
1 Overview

2 Algorithm

3 Random Pivot

4 Running Time Analysis

5 Equal Elements

6 Final Remarks
 Tail Recursion Elimination

QuickSort(A, ℓ, r )
while ℓ < r :
m ← Partition(A, ℓ, r )
QuickSort(A, ℓ, m − 1)
ℓ←m+1
QuickSort(A, ℓ, r )
while ℓ < r :
m ← Partition(A, ℓ, r )
if (m − ℓ) < (r − m):
QuickSort(A, ℓ, m − 1)
ℓ←m+1
else:
QuickSort(A, m + 1, r )
r ←m−1
QuickSort(A, ℓ, r )
while ℓ < r :
m ← Partition(A, ℓ, r )
if (m − ℓ) < (r − m):
QuickSort(A, ℓ, m − 1)
ℓ←m+1
else:
QuickSort(A, m + 1, r )
r ←m−1
Worst-case space requirement: O(log n)
 Intro Sort
runs quick sort with a simple
deterministic pivot selection heuristic
(say, median of the first, middle, and
last element)
 Intro Sort
runs quick sort with a simple
deterministic pivot selection heuristic
(say, median of the first, middle, and
last element)
if the recursion depth exceeds a certain
threshold c log n the algorithm switches
to heap sort
 Intro Sort
runs quick sort with a simple
deterministic pivot selection heuristic
(say, median of the first, middle, and
last element)
if the recursion depth exceeds a certain
threshold c log n the algorithm switches
to heap sort
the running time is O(n log n) in the
worst case
 Conclusion

Quick sort is a comparison based

algorithm
 Conclusion

Quick sort is a comparison based

algorithm
Running time: O(n log n) on average,
O(n2) in the worst case
 Conclusion

Quick sort is a comparison based

algorithm
Running time: O(n log n) on average,
O(n2) in the worst case
Efficient in practice