CS 161, Lecture 5 Quicksort

Scribe: Sam Keller (2015), Seth Hildick-Smith (2016), G. Valiant (2017) Date: January 25,
2017

1 Introduction
Today we’ll study another sorting algorithm. Quicksort was invented in 1959 by Tony Hoare. You may
wonder why we want to study a new sorting algorithm. We have already studied MergeSort, which we
showed to perform significantly better than the trivial 𝑂(𝑛2 ) algorithm. While MergeSort achieves an
𝑂(𝑛 log 𝑛) worst-case asymptotic bound, in practice, there are a number of implementation details about
MergeSort that make it tricky to achieve high performance. Quicksort is an alternative algorithm, which
is simpler to implement in practice. Quicksort will also use a divide and conquer strategy but will use
randomization to improve the performance of the algortithm in expectation. Java, Unix, and C stdlib all
have implementations of Quicksort as one of their built-in sorting routines.

2 Quicksort Overview
As in all sorting algorithms, we start with an array 𝐴 of 𝑛 numbers; again we assume without loss of
generality that the numbers are distinct. Quicksort is very similar to the Select algorithm we studied last
lecture. The description of Quicksort is the following:
(1) If length of 𝐴 ≤ 1: return A (it is trivially sorted).
(2) Pick some element 𝑥 ← 𝐴[𝑖]. We call 𝑥 the pivot.
(3) Split the rest of 𝐴 into 𝐴< (the elements less than 𝑥) and 𝐴> (the elements greater than 𝑥).

(4) Rearrange 𝐴 into [𝐴< , 𝑥, 𝐴> ].

(5) Recurse on 𝐴< , 𝐴> .
Steps 1-3 define a “partition” function on 𝐴. The partition function of Quicksort can vary depending on
how the pivot is chosen and also on the implementation. Quicksort is often used in practice because we can
implement this step in linear time, and with very small constant factors. In addition, the rearrangement
(Step 3) can be done in-place, rather than making several copies of the array (as is done in MergeSort). In
these notes we will not describe the details of an in-place implementation, but the pseudocode can be found
in CLRS.

3 Speculation on the Runtime

The performance of Quicksort depends on which element is chosen as the pivot. Assume that we choose the
𝑘 th smallest element; then |𝐴< | = 𝑘 − 1, |𝐴> | = 𝑛 − 𝑘.
This allows us to write a recurrence; let 𝑇 (𝑛) be the runtime of Quicksort on an 𝑛-element array. We
know the partition step takes 𝑂(𝑛) time; therefore the recurrence is

𝑇 (𝑛) ≤ 𝑐𝑛 + 𝑇 (𝑘 − 1) + 𝑇 (𝑛 − 𝑘).

For the worst pivot choice (the maximum or minimum element in the array), the runtime will resolve to
𝑇 (𝑛) = 𝑇 (𝑛 − 1) + 𝑂(𝑛) = 𝑂(𝑛2 ).

1
 One way that seems optimal to define the partition function is to pick the median as the pivot. In the
above recurrence this would mean that 𝑘 = ⌈ 𝑛2 ⌉. We showed in the previous lecture that the algorithm
Select can find the median element in linear time. Therefore the recurrence becomes 𝑇 (𝑛) ≤ 𝑐𝑛 + 2𝑇 ( 𝑛2 ).
This is exactly the same recurrence as MergeSort, which means that this algorithm is guaranteed to run in
𝑂(𝑛 log 𝑛) time.
Unfortunately, the median selection algorithm is not practical; while it runs in linear time, it has much
larger constant factors than we would like. To improve it, we will explore some alternative methods for
choosing a pivot.
We leave the proof of correctness of Quicksort as an exercise to the reader (a proof by induction is
suggested).

3.1 Random Pivot Selection

One method of “defending” against adversarial input is to choose a random element as the pivot. We observe
that it is unlikely that the random element will be either the median (best-case) or the maximum or minimum
(worst-case). Note that we have a uniform distribution over the 𝑛 order statistics of the array (that is, for
every 1 ≤ 𝑖 ≤ 𝑛 we pick the 𝑖th -highest element with the same probability 𝑛1 ). However, for the first time,
we encounter a randomized algorithm, and the analysis now becomes more complex. How do we analyze a
randomized algorithm?

4 Worst-Case Analysis
In this section we will derive a bound on the worst-case running time of Quicksort. If we consider the worst
random choice of pivot at each step, the running time will be Θ(𝑛2 ). We are thus interested in what is the
running time of Quicksort on average over all possible choices of the pivots. Note that we still consider the
running time for a worst-case input, and average only over the random choices of the algorithm (which is
different from averaging over all possible inputs). We formalize the idea of averaging over the random choices
of the algorithm by considering the running time of the algorithm on an input 𝐼 as a random variable and
bounding the expectation of that random variable.
Claim 1. For every input array of size 𝑛, the expected running time of Quicksort is 𝑂(𝑛 log 𝑛).
Recall that a random variable (often abbreviated as RV) is a function that maps every element in the
sample space to a real number. In the case of rolling a die, the real number (or value of the point in the
sample space) would be the number on the top of the die. Here the sample space is the set of all possible
choices of pivots, and an example for a random variable can be the running time of Quicksort on a specific
input 𝐼.
Denote by 𝑧𝑖 the 𝑖𝑡ℎ element in the sorted array. For each 𝑖, 𝑗, we define a random variable 𝑋𝑖,𝑗 (𝜎) to
be the number of times 𝑧𝑖 and 𝑧𝑗 are compared for a given series of pivot choices 𝜎. What are the possible
values for 𝑋𝑖,𝑗 (𝜎)? It can be 0 if 𝑧𝑖 and 𝑧𝑗 are not compared. Note that all comparisons are with the pivot,
and that the pivot is not included in the elements of the arrays in the recursive calls. Thus, no two elements
are compared twice. Therefore, 𝑋𝑖,𝑗 (𝜎) ∈ 0, 1.
Our goal is to compute the expected number of comparisons that Quicksort makes. Recall the definition
of expectation: ∑︁ ∑︁
𝐸[𝑋] = 𝑃 (𝜎)𝑋(𝜎) = 𝑘𝑃 (𝑥 = 𝑘).
𝜎 𝑘

An important property of expectation is the linearity of expectation. For any random variables 𝑋1 , . . . , 𝑋𝑛 :
[︃ 𝑛 ]︃ 𝑛
∑︁ ∑︁
𝐸 𝑋𝑖 = 𝐸[𝑋𝑖 ].
𝑖=1 𝑖=1

2
 We start with computing the expected value of 𝑋𝑖,𝑗 . These variables are indicator random variables,
which take the value 1 if some event happens, and 0 otherwise. The expected value is

𝐸[𝑋𝑖,𝑗 ] = 𝑃 (𝑋𝑖,𝑗 = 1) · 1 + 𝑃 (𝑋𝑖,𝑗 = 0) · 0

= 𝑃 (𝑋𝑖,𝑗 = 1)

Let 𝐶(𝜎) be the total number of comparisons made by Quicksort for a given set of pivot choices 𝜎:
𝑛
∑︁ 𝑛
∑︁
𝐶(𝜎) = 𝑋𝑖,𝑗 (𝜎).
𝑖=1 𝑗=𝑖+1

We wish to compute 𝐸[𝐶] to get the expected number of comparisons made by Quicksort for an input array
of size 𝑛.
⎡ ⎤
𝑛
∑︁ 𝑛
∑︁
𝐸[𝐶] = 𝐸 ⎣ 𝑋𝑖,𝑗 (𝜎)⎦
𝑖=1 𝑗=𝑖+1
⎡ ⎤
𝑛
∑︁ 𝑛
∑︁
= 𝐸⎣ 𝑋𝑖,𝑗 (𝜎)⎦
𝑖=1 𝑗=𝑖+1
𝑛
∑︁ 𝑛
∑︁
= 𝑃 (𝑧𝑖 , 𝑧𝑗 are compared)
𝑖=1 𝑗=𝑖+1

Now we find 𝑃 (𝑧𝑖 , 𝑧𝑗 are compared). Note that each element in the array except the pivot is compared
to the pivot at each level of the recurrence. To analyze 𝑃 (𝑧𝑖 , 𝑧𝑗 are compared), we need to examine the
portion of the array [𝑧𝑖 ...., 𝑧𝑗 ]. After the array is split using a pivot from [𝑧𝑖 ...., 𝑧𝑗 ], 𝑧𝑖 and 𝑧𝑗 can no longer
be compared. Hence, 𝑧𝑖 and 𝑧𝑗 are compared only when from [𝑧𝑖 ...., 𝑧𝑗 ], either 𝑧𝑖 or 𝑧𝑗 is the first one picked
as the pivot. So,

𝑃 (𝑧𝑖 , 𝑧𝑗 compared) = 𝑃 (𝑧𝑖 or 𝑧𝑗 is the first pivot picked from [𝑧𝑖 ...., 𝑧𝑗 ])
1 1
= +
𝑗−𝑖+1 𝑗−𝑖+1
2
=
𝑗−𝑖+1
We return to the expected value of 𝐶:
𝑛
∑︁ 𝑛
∑︁
𝐸[𝐶] = 𝑃 (𝑧𝑖 , 𝑧𝑗 are compared)
𝑖=1 𝑗=𝑖+1
𝑛 𝑛
∑︁ ∑︁ 2
=
𝑖=1 𝑗=𝑖+1
𝑗−𝑖+1

Note that for a fixed 𝑖,

𝑛
∑︁ 1 1 1 1
= + + ... +
𝑗=𝑖+1
𝑗 − 𝑖 + 1 2 3 𝑛 − 𝑖+1
1 1 1
≤ + + ... +
2 3 𝑛

3
 ∑︀𝑛 1
And using 𝑘=2 𝑘 ≤ ln 𝑛, we get that
⎡ ⎤
𝑛
∑︁ 𝑛
∑︁
𝐸[𝐶] = 𝐸 ⎣ 𝑋𝑖,𝑗 (𝜎)⎦
𝑖=1 𝑗=𝑖+1
𝑛 𝑛
∑︁ ∑︁ 2
=
𝑖=1 𝑗=𝑖+1
𝑗 − 𝑖+1
≤ 2𝑛 ln 𝑛

Thus, the expected number of comparisons made by Quicksort is no greater than 2𝑛 ln 𝑛 = 𝑂(𝑛 log 𝑛). To
complete the proof, we have to show that the running time is dominated by the number of comparisons. Note
that in each recursive call to Quicksort on an array of size 𝑘, the algorithm performs 𝑘 − 1 comparisons in
order to split the array, and the amount of work done is 𝑂(𝑘). In addition, Quicksort will be called on single-
element arrays at most once for each element in the original array, so the total running time of Quicksort is
𝑂(𝐶 + 𝑛). In conclusion, the expected running time of Quicksort on worst-case input is 𝑂(𝑛 log 𝑛).

4.1 Alternative Proof

Here we provide an alternative method for bounding the expected number of comparisons. Let 𝑇 (𝑛) be the
expected number of comparisons performed by Quicksort on an input of size 𝑛. In general, if the pivot is
chosen to be the 𝑖𝑡ℎ order statistic of the input array,

𝑇 (𝑛) = 𝑛 − 1 + 𝑇 (𝑖 − 1) + 𝑇 (𝑛 − 𝑖).

where we define 𝑇 (0) = 0. Each of the 𝑛 possible choices of 𝑖 are equally likely. Thus, the expected number
of comparisons is:
𝑛 (︂ )︂
1 ∑︁
𝑇 (𝑛) = 𝑛 − 1 + 𝑇 (𝑖 − 1) + 𝑇 (𝑛 − 𝑖)
𝑛 𝑖=1
𝑛−1 (︂ )︂
2 ∑︁
=𝑛−1+ 𝑇 (𝑖)
𝑛 𝑖=1

We use two facts:

∑︀𝑛−1 ∫︀ 𝑛
1. 𝑖=1 𝑓 (𝑖) ≤ 1 𝑓 (𝑥)𝑑𝑥 for an increasing function 𝑓
2
2. 2𝑥 ln 𝑥𝑑𝑥 = 𝑥2 ln 𝑥 − 𝑥2 + 𝐶
∫︀

Now we show that 𝑇 (𝑛) ≤ 2𝑛 ln 𝑛 by induction.

Base case: An array of size 1 is trivially sorted and requires no comparisons. Thus, 𝑇 (1) = 0.
Inductive hypothesis: 𝑇 (𝑖) ≤ 2𝑖 ln 𝑖 for all 𝑖 < 𝑛

4
Inductive step: We bound 𝑇 (𝑛):
𝑛−1
2 ∑︁
𝑇 (𝑛) = 𝑛 − 1 + 𝑇 (𝑖)
𝑛 𝑖=1
𝑛−1
2 ∑︁
≤𝑛−1+ 2𝑖 ln 𝑖
𝑛 𝑖=1
2 𝑛
∫︁
≤𝑛−1+ (2𝑥 ln 𝑥)𝑑𝑥
𝑛 1
𝑛2
[︂ ]︂
2 2 1
=𝑛−1+ 𝑛 ln 𝑛 − +
𝑛 2 2
1
= 2𝑛 ln 𝑛 + 𝑛 − 1 − 𝑛 +
𝑛
1
= 2𝑛 ln 𝑛 − 1 +
𝑛
≤ 2𝑛 ln 𝑛