Lectures 11-12.

Counting Sort

1
Sorting So Far (1/4)
¢ Insertion Sort
u Easy to code
u Fast on small inputs (less than ~ 50 elements)
u Fast on nearly-sorted inputs
u 𝑂(𝑛2) worst case
u 𝑂(𝑛2) average (equally-likely inputs) case
u 𝑂(𝑛2) reverse-sorted case

2
Sorting So Far (2/4)
¢ Merge Sort
u Divide-and-Conquer
ê Split array in half
ê Recursively sort subarrays
ê Linear-time merge step
u 𝑂(𝑛𝑙𝑔𝑛) worst case
u It does not sort in place

3
Sorting So Far (3/4)
¢ Heap Sort
u It uses the very useful heap data structure
ê Complete binary tree
ê Heap property: parent key > children’s keys
u 𝑂(𝑛𝑙𝑔𝑛) worst case
u It sorts in place

4
Sorting So Far (4/4)
¢ Quick Sort
u Divide-and-Conquer
ê Partition array into two subarrays, recursively sort
ê All of first subarray < all of second subarray
ê No merge step needed !
u 𝑂(𝑛𝑙𝑔𝑛) average case
u Fast in practice
u 𝑂(𝑛2) worst case
ê Naïve implementation: worst case on sorted input
ê Address this with randomized quicksort

5
How Fast Can We Sort?

¢ We will provide a lower bound, then beat it

6
Decision Trees
5 7 3

5 3
7 7
3

3
5 3
5 7

7
Decision Trees
¢ Decision trees provide an abstraction of comparison
sorts
u A decision tree represents the comparisons made by a
comparison sort.
¢ What do the leaves represent?
¢ How many leaves must there be?

8
Practice Problems
¢ What is the smallest possible depth of a leaf in a
decision tree for a comparison sort?
u The absolute best case happens when we just check every
element and see that the data is already sorted.
u This will result in 𝑛 − 1 comparisons and thus the leaf will have
a depth of 𝑛 − 1.

9
Decision Trees
¢ Decision trees can model comparison sorts
For a given algorithm:
u One tree for each 𝑛
u Tree paths are all possible execution traces
u What’s the longest path in a decision tree for insertion sort?
For merge sort?
¢ What is the asymptotic height of any decision tree for
sorting n elements?
¢ Answer: W(𝑛𝑙𝑔𝑛) (Now let’s prove it…)

10
A Lower Bound for Comparison Sorting
¢ Theorem: Any decision tree that sorts 𝑛 elements has
height W(𝑛𝑙𝑔𝑛)
¢ What’s the minimum number of leaves?
u n!
¢ What’s the maximum number of leaves of a binary tree
of height ℎ?
u 2h

¢ Clearly the minimum number of leaves is less than or

11
A Lower Bound for Comparison Sorting
¢ So, we have:
𝑛! £ 2ℎ
¢ Taking logarithms:
lg(𝑛!) £ ℎ
¢ Stirling’s approximation tells us:
n
ænö
n! > ç ÷
èeø
¢ Thus:
n
ænö
h ³ lgç ÷
èeø

12
A Lower Bound for Comparison Sorting
¢ So, we have:
n
ænö
h ³ lgç ÷
èeø
= n lg n - n lg e
= W(n lg n )

¢ Thus, the minimum height of a decision tree is W(𝑛lg 𝑛)

13
A Lower Bound for Comparison Sorts
¢ Thus, the time to comparison sort 𝑛 elements is
W(𝑛lg 𝑛)
¢ Corollary: Heapsort and Merge sort are asymptotically
optimal comparison sorts
¢ But the name of this lecture is “Sorting in linear time”!
u How can we do better than W(𝑛lg 𝑛)?

14
Sorting in Linear Time

¢ Counting Sort
u No comparisons between elements !
u But…depends on assumption about the numbers being sorted
¢ Basic Idea
u For each input element x,
determine the number of elements less than or equal to x
u Assume that each element is an integer in the range 0 to k, for each
integer i (0 £ i £ k), count how many elements whose values are i
ê Then we know how many elements are less than or equal to i
¢ Algorithm Storage
u A[1..n]: input elements
u B[1..n]: sorted elements
u C[0..k]: hold the number of elements less than or equal to i

15
Counting Sort

16
 Counting Sort Illustration
(Range from 0 to 5)

17
Implementation

18
Counting Sort

Q(𝑘)

Q(𝑛)

Q(𝑘)

Q(𝑛)

Q(𝑛 + 𝑘)
19
Evaluation
¢ Total time: 𝑂(𝑛 + 𝑘)
u Usually, 𝑘 = 𝑂(𝑛)
u Thus, counting sort runs in 𝑂(𝑛) time
¢ But sorting is W(𝑛 lg 𝑛)!
u No contradiction
ê This is not a comparison sort
ê In fact, there are no comparisons at all !
u Notice that this algorithm is stable

20
Evaluation
¢ Advantage
u Simple
u O(n)
¢ Disavantage
u Input needs to be integers
u Difficult for big integers

21
 Sorting of ASCII characters
Which sorting algorithms is most efficient to sort string
consisting of ASCII characters?
a) Quick sort
b) Heap sort
c) Merge sort
d) Counting sort

ASCII characters are in range 0 to 255, so, by using counting sort,

22