CSE-209

Algorithms-I

Lecture 5

Sorting: Counting Sort

Md. Rafsan Jani

Assistant Professor
Department of Computer Science and Engineering
Jahangirnagar University
 How fast can we sort?

All the sorting algorithms we have seen so far

are comparison sorts: only use comparisons to
determine the relative order of elements.
•E.g., insertion sort, merge sort, quicksort,
heapsort.
The best worst-case running time that we’ve
seen for comparison sorting is O(n lg n) .
Is O(nlgn) the best we can do?
Decision trees can help us answer this question.
 Decision-tree example

Sort 〈a1, a2, …, an〉 1 :

1:
2 :3 2 1 :
2: 3 2 1:
123
123 213
213 33
1 : 2 :
1: 2:
132
132 33 312
312 231
231 33 321
321

Each internal node is labeled i:j for i, j ∈ {1, 2,…, n}.

•The left subtree shows subsequent comparisons if ai ≤ aj.
•The right subtree shows subsequent comparisons if ai ≥ aj.

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.3
 Decision-tree example

Sort 〈a1, a2, a3〉 1 : 9≥4

= 〈 9, 4, 6 〉: 1:
2 : 2 1 :
2: 2 1:
123
123 33 213 33
1 : 2 :
1: 2:
132 312213
13231233 231 321 231 3
3 321

Each internal node is labeled i:j for i, j ∈ {1, 2,…, n}.

•The left subtree shows subsequent comparisons if ai ≤ aj.
•The right subtree shows subsequent comparisons if ai ≥ aj.
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.4
 Decision-tree example

Sort 〈a1, a2, a3〉 1 :

= 〈 9, 4, 6 〉: 1:
2 : 2
2 1 : 9≥6
2: 1:
123
123 33 213
213 33
1 : 2 :
1: 2:
132
13231233 231
312 321 231 33 321

Each internal node is labeled i:j for i, j ∈ {1, 2,…, n}.

•The left subtree shows subsequent comparisons if ai ≤ aj.
•The right subtree shows subsequent comparisons if ai ≥ aj.
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.5
 Decision-tree example

Sort 〈a1, a2, a3〉 1 :

= 〈 9, 4, 6 〉: 1:
2 : 2
1 :
2: 2
1:
123
123 33 213 3
1 : 213 4 3≤ 6 2 :
1: 2:
132
132 3312 312 231
231 3321 321
3 3

Each internal node is labeled i:j for i, j ∈ {1, 2,…, n}.

•The left subtree shows subsequent comparisons if ai ≤ aj.
•The right subtree shows subsequent comparisons if ai ≥ aj.

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.6
 Decision-tree example

Sort 〈a1, a2, a3〉 1 :

= 〈 9, 4, 6 〉: 1:
2 : 2 1 :
2: 2 1:
123
123 33 213
213 33
1 : 2 :
1: 2:
132
132 33 312
312 231
231 33 321
321
4≤6≤
9
Each leaf contains a permutation 〈π(1), π(2),…, π(n)〉 to
indicate that the ordering aπ(1) ≤ aπ(2) ≤ • ≤ aπ(n) has been
established.
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. LeisersonL5.7
 Decision-tree model

A decision tree can model the execution of

any comparison sort:
•One tree for each input size n.
•View the algorithm as splitting whenever
it compares two elements.
•The tree contains the comparisons along
all possible instruction traces.
•The running time of the algorithm = the
length of the path taken.
L5.8
 Lower bound for
decision- tree sorting
Theorem. Any decision tree that can sort n
elements must have height Ω(n lg n) .
Proof. The tree must contain ≥ n! leaves,
since there are n! possible permutations. A
height-h binary tree has ≤ 2h leaves. Thus, n! ≤
2∴h h. ≥ lg(n!) (lg is mono. increasing)
≥ lg ((n/e)n) (Stirling’s formula)
= n lg n – n lg e
= Ω(n lg n) .
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.9
 Lower bound for
comparison sorting

Corollary.Heapsort and merge sort are

asymptotically optimal comparison sorting
algorithms.

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.10
 Sorting in linear
time
Counting sort: No comparisons between elements.
•Input: A[1 . . n], where A[ j]∈{1, 2, …, k} .
•Output: B[1 . . n], sorted.
•Auxiliary storage: C[1 . . k] .

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.11
 Counting sort

for i ← 1 to k
do C[i] ← 0
for j ← 1 to n
do C[A[ j]] ← C[A[ j]] + ⊳ C[i] = |{key = i}|
1
⊳ C[i] = |{key ≤ i}|
for i ← 2 to k
do C[i] ← C[i] + C[i–1]
for j ← n downto 1
do B[C[A[ j]]] ← A[ j]
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.12
 Counting-sort example

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C:

B:

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.13
 Loop 1

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 00 00 00 00

B:

for i ← 1 to k
do C[i] ← 0

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.14
 Loop 2

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 00 00 00 11

B:

for j ← 1 to n
do C[A[ j]] ← C[A[ j]] + ⊳ C[i] = |{key = i}|
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.15
 Loop 2

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 00 00 11

B:

for j ← 1 to n
do C[A[ j]] ← C[A[ j]] + ⊳ C[i] = |{key = i}|
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.16
 Loop 2

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 00 11 11

B:

for j ← 1 to n
do C[A[ j]] ← C[A[ j]] + ⊳ C[i] = |{key = i}|
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.17
 Loop 2

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 00 11 22

B:

for j ← 1 to n
do C[A[ j]] ← C[A[ j]] + ⊳ C[i] = |{key = i}|
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.18
 Loop 2

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 00 22 22

B:

for j ← 1 to n
do C[A[ j]] ← C[A[ j]] + ⊳ C[i] = |{key = i}|
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.19
 Loop 3

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 00 22 22

B: C': 11 11 22 22

for i ← 2 to k
do C[i] ← C[i] + C[i–1] ⊳ C[i] = |{key ≤ i}|

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.20
 Loop 3

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 00 22 22

B: C': 11 11 33 22

for i ← 2 to k
do C[i] ← C[i] + C[i–1] ⊳ C[i] = |{key ≤ i}|

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.21
 Loop 3

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 00 22 22

B: C': 11 11 33 55

for i ← 2 to k
do C[i] ← C[i] + C[i–1] ⊳ C[i] = |{key ≤ i}|

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.22
 Loop 4

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 11 33 55

B: C': 33 11 11 22 55

for j ← n downto 1
do B[C[A[ j]]] ← A[ j]
C[A[ j]] ← C[A[ j]] –
September 26, 2005 1 © 2001-5 Erik D. Demaine and Charles E. Leiserson
Copyright L5.23
 Loop 4

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 11 22 55

B: C': 33 44 11 11 22 44

for j ← n downto 1
do B[C[A[ j]]] ← A[ j]
C[A[ j]] ← C[A[ j]] – 1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.24
 Loop 4

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 11 22 44

B: C': 33 33 44 11 11 11 44

for j ← n downto 1
do B[C[A[ j]]] ← A[ j]
C[A[ j]] ← C[A[ j]] – 1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.25
 Loop 4

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 11 11 11 44

B: C':11 33 33 44 00 11 11 44

for j ← n downto 1
do B[C[A[ j]]] ← A[ j]
C[A[ j]] ← C[A[ j]] – 1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.26
 Loop 4

1 2 3 4 5 1 2 3 4

A: 44 11 33 44 33 C: 00 11 11 44

B: C':11 33 33 44 44 00 11 11 33

for j ← n downto 1
do B[C[A[ j]]] ← A[ j]
C[A[ j]] ← C[A[ j]] – 1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.27
 Analysis

for i ← 1 to k
Θ(k)
do C[i] ← 0
for j ← 1 to n
Θ(n)
do C[A[ j]] ← C[A[ j]] +
1
Θ(k)
for i ← 2 to k
do C[i] ← C[i] + C[i–1]
Θ(n) for j ← n downto 1
do B[C[A[ j]]] ← A[ j]
Θ(n + k) C[A[ j]] ← C[A[ j]] –
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.28
 Running time

If k = O(n), then counting sort takes Θ(n) time.

•But, sorting takes Ω(n lg n) time!
•Where’s the fallacy?
Answer:
•Comparison sorting takes Ω(n lg n) time.
•Counting sort is not a comparison sort.
•In fact, not a single comparison between
elements occurs!
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.29
 Stable sorting

Counting sort is a stable sort: it preserves

the input order among equal elements.

A: 44 11 33 44 33

11 33 33 44 44
B:

Exercise: What other sorts have this property?

September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.30