Design & Analysis of Algorithm (CSC-321)

Lecture 8
Computer Sciences Department
Bahria University (Karachi Campus)
 Sorting Review
• Insertion Sort
– T(n) = Q(n2)
• Merge Sort
• T(n) = Q(n lg(n))
• Selection Sort
• T(n) = Q(n2) Seems pretty good.
Can we do better?
• Heap Sort
– T(n) = Q(n lg(n))
Quick Sort
 Quicksort Overview
• To sort a[left...right]:
1. if left < right:
1.1. Partition a[left...right] such that:
all a[left...p-1] are less than a[p],
and
all a[p+1...right] are >= a[p]
1.2. Quicksort a[left...p-1]
1.3. Quicksort a[p+1...right]
2. Terminate
 Partitioning in Quicksort
• A key step in the Quicksort algorithm is
partitioning the array
– We choose some (any) number p in the array to
use as a pivot
– We partition the array into three parts:

numbers less p numbers greater than or

than p equal to p
Partitioning at various levels
 Partitioning
• Choose an array value (say, the first) to use as the
pivot
• Starting from the left end, find the first element
that is greater than or equal to the pivot
• Searching backward from the right end, find the
first element that is less than the pivot
• Interchange (swap) these two elements
• Repeat, searching from where we left off, until
done
 Pseudocode
Quicksort(A, p, r)
if p < r then Partition(A, p, r)
q := Partition(A, p, r); x, i := A[r], p – 1;
Quicksort(A, p, q – 1); for j := p to r – 1 do
Quicksort(A, q + 1, r) if A[j]  x then
i := i + 1;
A[p..r] A[i]  A[j]
A[i + 1]  A[r];
5 return i + 1

A[p..q – 1] A[q+1..r]
Partition
5

5 5
 Example
p r
initially: 2 5 8 3 9 4 1 7 10 6 note: pivot (x) = 6
i j

next iteration: 2 5 8 3 9 4 1 7 10 6
i j Partition(A, p, r)
x, i := A[r], p – 1;
next iteration: 2 5 8 3 9 4 1 7 10 6 for j := p to r – 1 do
i j if A[j]  x then
i := i + 1;
next iteration: 2 5 8 3 9 4 1 7 10 6 A[i]  A[j]
i j A[i + 1]  A[r];
return i + 1
next iteration: 2 5 3 8 9 4 1 7 10 6
i j
 Example (Continued)
next iteration: 2 5 3 8 9 4 1 7 10 6
i j
next iteration: 2 5 3 8 9 4 1 7 10 6
i j
next iteration: 2 5 3 4 9 8 1 7 10 6 Partition(A, p, r)
i j x, i := A[r], p – 1;
next iteration: 2 5 3 4 1 8 9 7 10 6 for j := p to r – 1 do
i j if A[j]  x then
next iteration: 2 5 3 4 1 8 9 7 10 6 i := i + 1;
i j A[i]  A[j]
next iteration: 2 5 3 4 1 8 9 7 10 6 A[i + 1]  A[r];
i j return i + 1
after final swap: 2 5 3 4 1 6 9 7 10 8
i j
 Partitioning
• Select the last element A[r] in the subarray A[p..r] as
the pivot – the element around which to partition.
• As the procedure executes, the array is partitioned into
four (possibly empty) regions.
1. A[p..i ] — All entries in this region are < pivot.
2. A[i+1..j – 1] — All entries in this region are > pivot.
3. A[r] = pivot.
4. A[j..r – 1] — Not known how they compare to pivot.
• The above hold before each iteration of the for loop,
and constitute a loop invariant. (4 is not part of the loopi.)
 Quicksort Analysis
• Assumptions:
– A random pivot (no median-of-three partitioning)
– No cutoff for small arrays
• Running time
– pivot selection: constant time, i.e. O(1)
– partitioning: linear time, i.e. O(N)
– running time of the two recursive calls
• T(N)=T(i)+T(N-i-1)+cN where c is a constant
– i: number of elements in S1
 Best Case Analysis
• We cut the array size in half each time
• So the depth of the recursion in log2n
• At each level of the recursion, all the partitions
at that level do work that is linear in n
• O(log2n) * O(n) = O(n log2n)
• Hence in the best case, quicksort has time
complexity O(n log2n)
• What about the worst case?
Worst case partitioning
 Worst case
• In the worst case, partitioning always divides
the size n array into these three parts:
– A length one part, containing the pivot itself
– A length zero part, and
– A length n-1 part, containing everything else
• We don’t recur on the zero-length part
• Recurring on the length n-1 part requires (in
the worst case) recurring to depth n-1
 Worst case for quicksort
• In the worst case, recursion may be n levels deep (for
an array of size n)
• But the partitioning work done at each level is still n
• O(n) * O(n) = O(n2)
• So worst case for Quicksort is O(n2)
• When does this happen?
– There are many arrangements that could make this happen
– Here are two common cases:
• When the array is already sorted
• When the array is inversely sorted (sorted in the opposite order)
 Worst-Case Analysis
• What will be the worst case?
– The pivot is the smallest element, all the time
– Partition is always unbalanced
 Best-case Analysis
• What will be the best case?
– Partition is perfectly balanced.
– Pivot is always in the middle (median of the
array)
 Average-Case Analysis
• Assume
– Each of the sizes for S1 is equally likely
• This assumption is valid for our pivoting
(median-of-three) strategy
• On average, the running time is O(N log N)
(covered in comp271)
 Quicksort is ‘faster’ than Mergesort
• Both quicksort and mergesort take O(N log N) in the
average case.
• Why is quicksort faster than mergesort?
– The inner loop consists of an increment/decrement (by 1, which is
fast), a test and a jump.
– There is no extra juggling as in mergesort.

inner loop
 Mergesort vs Quicksort
• Both run in O(n lgn)
– Mergesort – always.
– Quicksort – on average
• Compared with Quicksort, Mergesort has less
number of comparisons but larger number of
moving elements
• In Java, an element comparison is expensive but
moving elements is cheap. Therefore, Mergesort is
used in the standard Java library for generic sorting
 Mergesort vs Quicksort
In C++, copying objects can be expensive while
comparing objects often is relatively cheap.
Therefore, quicksort is the sorting routine
commonly used in C++ libraries

Note these last two rules are not really language

specific, but rather how the language is
typically used.