Quicksort

• Proposed by C.A.R. Hoare in 1962.

• Divide-and-conquer algorithm.
• Sorts “in place” (like insertion sort, but not
like merge sort).
• Very practical (with tuning).

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.2
 Divide and conquer
Quicksort an n-element array:
1. Divide: Partition the array into two subarrays
around a pivot x such that elements in lower
subarray ≤ x ≤ elements in upper subarray.
≤x x ≥x
2. Conquer: Recursively sort the two subarrays.
3. Combine: Trivial.
Key: Linear-time partitioning subroutine.
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.3
 Partitioning subroutine
PARTITION(A, p, q) ⊳ A[ p . . q]
x ← A[ p] ⊳ pivot = A[ p] Running time
i←p = O(n) for n
for j ← p + 1 to q elements.
do if A[ j] ≤ x
then i ← i + 1
exchange A[i] ↔ A[ j]
exchange A[ p] ↔ A[i]
return i
Invariant: x ≤x ≥x ?
p i j q
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.4
 Example of partitioning

6 10 13 5 8 3 2 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.5
 Example of partitioning

6 10 13 5 8 3 2 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.6
 Example of partitioning

6 10 13 5 8 3 2 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.7
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.8
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.9
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.10
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.11
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.12
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11

6 5 3 2 8 13 10 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.13
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11

6 5 3 2 8 13 10 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.14
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11

6 5 3 2 8 13 10 11
i j

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.15
 Example of partitioning

6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11

6 5 3 2 8 13 10 11

2 5 3 6 8 13 10 11
i
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.16
 Pseudocode for quicksort
QUICKSORT(A, p, r)
if p < r
then q ← PARTITION(A, p, r)
QUICKSORT(A, p, q–1)
QUICKSORT(A, q+1, r)

Initial call: QUICKSORT(A, 1, n)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.17
 Analysis of quicksort

• Assume all input elements are distinct.

• In practice, there are better partitioning
algorithms for when duplicate input
elements may exist.
• Let T(n) = worst-case running time on
an array of n elements.

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.18
 Worst-case of quicksort
• Input sorted or reverse sorted.
• Partition around min or max element.
• One side of partition always has no elements.
T (n) = T (0) + T (n − 1) + Θ(n)
= Θ(1) + T (n − 1) + Θ(n)
= T (n − 1) + Θ(n)
= Θ( n 2 ) (arithmetic series)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.19
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.20
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
T(n)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.21
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn
T(0) T(n–1)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.22
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn
T(0) c(n–1)
T(0) T(n–2)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.23
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn
T(0) c(n–1)
T(0) c(n–2)
T(0) 

Θ(1)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.24
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn  n 
T(0) c(n–1) Θ ∑ k  = Θ(n 2 )
 k =1 
T(0) c(n–2)
T(0) 

Θ(1)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.25
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn  n 
Θ(1) c(n–1) Θ ∑ k  = Θ(n 2 )
 k =1 
Θ(1) c(n–2)
h=n T(n) = Θ(n) + Θ(n2)
Θ(1)  = Θ(n2)

Θ(1)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.26
 Best-case analysis
(For intuition only!)
If we’re lucky, PARTITION splits the array evenly:
T(n) = 2T(n/2) + Θ(n)
= Θ(n lg n) (same as merge sort)

1 9
What if the split is always 10 : 10 ?
T (n) = T (101 n ) + T (109 n ) + Θ(n)
What is the solution to this recurrence?

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.27
 Analysis of “almost-best” case
T (n)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.28
 Analysis of “almost-best” case
cn
T (101 n ) T (109 n )

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.29
 Analysis of “almost-best” case
cn
1
10
cn 9
10
cn

T (100
1
n ) T (100
9
n ) T (100
9
n )T (100
81
n)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.30
 Analysis of “almost-best” case
cn cn
1
10
cn 9
10
cn cn
log10/9n
1 9 9 81
100
cn 100
cn 100
cn 100
cn cn

…
Θ(1) O(n) leaves
Θ(1)

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.31
 Analysis of “almost-best” case
cn cn
1
10
cn 9
10
cn cn
log10 log10/9n
n 1 cn 9
cn 9
cn 81
cn cn
100 100 100 100

…
Θ(1) O(n) leaves

Θ(n lg n) Θ(1)
Lucky! cn log10n ≤ T(n) ≤ cn log10/9n + Ο(n)
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.32
 More intuition
Suppose we alternate lucky, unlucky,
lucky, unlucky, lucky, ….
L(n) = 2U(n/2) + Θ(n) lucky
U(n) = L(n – 1) + Θ(n) unlucky
Solving:
L(n) = 2(L(n/2 – 1) + Θ(n/2)) + Θ(n)
= 2L(n/2 – 1) + Θ(n)
= Θ(n lg n) Lucky!
How can we make sure we are usually lucky?
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.33
 Randomized quicksort
IDEA: Partition around a random element.
• Running time is independent of the input
order.
• No assumptions need to be made about
the input distribution.
• No specific input elicits the worst-case
behavior.
• The worst case is determined only by the
output of a random-number generator.
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.34
 Randomized quicksort
analysis
Let T(n) = the random variable for the running
time of randomized quicksort on an input of size
n, assuming random numbers are independent.
For k = 0, 1, …, n–1, define the indicator
random variable
1 if PARTITION generates a k : n–k–1 split,
Xk =
0 otherwise.
E[Xk] = Pr{Xk = 1} = 1/n, since all splits are
equally likely, assuming elements are distinct.
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.35
 Analysis (continued)
T(0) + T(n–1) + Θ(n) if 0 : n–1 split,
T(1) + T(n–2) + Θ(n) if 1 : n–2 split,
T(n) =

T(n–1) + T(0) + Θ(n) if n–1 : 0 split,
n −1
= ∑ X k (T (k ) + T (n − k − 1) + Θ(n) )
k =0

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.36
 Calculating expectation
 n −1 
E[T (n)] = E  ∑ X k (T (k ) + T (n − k − 1) + Θ(n) )
k =0 

Take expectations of both sides.

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.37
 Calculating expectation
 n −1 
E[T ( n)] = E  ∑ X k (T (k ) + T (n − k − 1) + Θ(n) )
k = 0 
n −1
= ∑ E[ X k (T (k ) + T (n − k − 1) + Θ(n) )]
k =0

Linearity of expectation.

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.38
 Calculating expectation
 n −1 
E[T ( n)] = E  ∑ X k (T (k ) + T (n − k − 1) + Θ(n) )
k =0 
n −1
= ∑ E[ X k (T (k ) + T (n − k − 1) + Θ(n) )]
k =0
n −1
= ∑ E[ X k ] ⋅ E[T (k ) + T (n − k − 1) + Θ(n)]
k =0

Independence of Xk from other random

choices.

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.39
 Calculating expectation
 n −1 
E[T (n)] = E  ∑ X k (T (k ) + T (n − k − 1) + Θ(n) )
k =0 
n −1
= ∑ E[ X k (T (k ) + T (n − k − 1) + Θ(n) )]
k =0
n −1
= ∑ E[ X k ] ⋅ E[T (k ) + T (n − k − 1) + Θ(n)]
k =0
n −1 n −1 n −1
= 1 ∑ E [T (k )] + 1 ∑ E [T (n − k − 1)] + 1 ∑ Θ(n)
n k =0 n k =0 n k =0

Linearity of expectation; E[Xk] = 1/n .

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.40
 Calculating expectation
 n −1 
E[T ( n)] = E  ∑ X k (T ( k ) + T ( n − k − 1) + Θ( n) )
k =0 
n −1
= ∑ E[ X k (T (k ) + T (n − k − 1) + Θ(n) )]
k =0
n −1
= ∑ E[ X k ] ⋅ E[T (k ) + T (n − k − 1) + Θ(n)]
k =0
n −1 n −1 n −1
= 1 ∑ E [T (k )] + 1 ∑ E [T (n − k − 1)] + 1 ∑ Θ(n)
n k =0 n k =0 n k =0
n −1
= 2 ∑ E [T (k )] + Θ(n) Summations have
n k =1
identical terms.
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.41
 Hairy recurrence
n −1
E[T (n)] = 2 ∑ E [T (k )] + Θ(n)
n k =2
(The k = 0, 1 terms can be absorbed in the Θ(n).)
Prove: E[T(n)] ≤ a n lg n for constant a > 0 .
• Choose a large enough so that a n lg n
dominates E[T(n)] for sufficiently small n ≥ 2.
n −1
Use fact: ∑ k lg k ≤ 1 n 2 lg n − 1n 2
2 8 (exercise).
k =2
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.42
 Substitution method
n −1
E [T (n)] ≤ 2 ∑ ak lg k + Θ(n)
n k =2
Substitute inductive hypothesis.

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.43
 Substitution method
n −1
E [T (n)] ≤ 2 ∑ ak lg k + Θ(n)
n k =2

≤ 2a  1 n 2 lg n − 1 n 2  + Θ(n)
n 2 8 
Use fact.

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.44
 Substitution method
n −1
E [T (n)] ≤ 2 ∑ ak lg k + Θ(n)
n k =2

≤ 2a  1 n 2 lg n − 1 n 2  + Θ(n)
n 2 8 
= an lg n −  an − Θ(n) 
 4 
Express as desired – residual.

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.45
 Substitution method
n −1
E [T (n)] ≤ 2 ∑ ak lg k + Θ(n)
n k =2

= 2a  1 n 2 lg n − 1 n 2  + Θ(n)
n 2 8 
= an lg n −  an − Θ(n) 
 4 
≤ an lg n ,
if a is chosen large enough so that
an/4 dominates the Θ(n).
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.46
 Quicksort in practice

• Quicksort is a great general-purpose

sorting algorithm.
• Quicksort is typically over twice as fast
as merge sort.
• Quicksort can benefit substantially from
code tuning.
• Quicksort behaves well even with
caching and virtual memory.

September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L4.47