8.

Sorting in linear time

Hsu, Lih-Hsing
 Computer Theory Lab.

8.1 Lower bound for sorting

The decision tree model
a1:a2

 

a2:a3 a1:a3

   

<1,2,3> a1:a3 <2,1,3> a2:a3

   

<1,3,2> <3,1,2> <2,3,1> <3.2,1>

Chapter 8 P.2
 Computer Theory Lab.

Theorem 9.1. Any decision tree that sorts n elements has

height ( n log n ) .
Proof:
n! l  2h ,
h  log(n!)  (n lg n).

Corollary 9.2 Heapsort and merge sort are asymptotically

optimal comparisons.

Chapter 8 P.3
 Computer Theory Lab.

8.2 Counting sort

Assume that each of the n input elements is an

integer in the range 1 to k for some integer k.
COUNTING_SORT(A,B,k)
1 for i  1 to k 6 for i  2 to k
2 do c [ i ]  0 7 do c [ i ]  c [ i ]  c [ i 1 ]
3 for j  1 to length[A] 8 ► c[i] now contains the number of
4 do c [ A[ j ]]  c [ A[ j ]] 1 elements less than or equal to i
5 ► c[i] now contains the number 9 for j  length [ A ] downto 1

of elements equal to i 10 do B [ c [ A[ j ]]]  A[ j ]

11 c [ A[ j ]]  c [ A[ j ]] 1

Chapter 8 P.4
 Computer Theory Lab.

 A n a l y s i s : O ( k  n )
 S p e c i a l c a s e : O ( n ) w h e n k  O ( n ) .

 P r o p e r t y : s t a b l e ( n u m b e r w i t h t h e s a m e v a l u e a p p e a r i n t h e
o u tp u t a r r a y in th e s a m e o r d e r a s th e y d o in th e in p u t a r r a y .)

Chapter 8 P.5
 Computer Theory Lab.

The operation of Counting-sort on an input array A[1..8]

Chapter 8 P.6
 Computer Theory Lab.

8.3 Radix sort

Used by the card-sorting machines you can now fi
nd only in computer museum.

RADIX_SORT(A,d)
1 for i  1 to d
2 do use a stable sort to sort array A on digit i

Chapter 8 P.7
 Computer Theory Lab.

Analysis: O( d ( n  k ))  O( dn  dk )
NOTE: not in-place (in-place: only constant number of elements
of the input array are ever stored outside the array.)

Lemma 8.3
Given n d-digit numbers in which each digit can take on up to k
possible values, RADIX-SORT correctly sorts these number in
(d(n + k)) time.

Chapter 8 P.8
 Computer Theory Lab.

Lemma 8.4
Given n b-bit numbers and any positive integer r ≤ b, RADIX- SORT
correctly sorts these numbers in ((b/r)(n+2r)) time.

Proof : Choose d = b/r.

IF b < lg n, choose r = b.

  ((b/r)(n+2r)) = (bn/logn)

Chapter 8 P.9
 Computer Theory Lab.

8.4 Bucket sort

B U C K E T _ S O R T (A )
1 n  le n g th [ A ]
2 fo r i  1 to n
3 d o in s e rt A [ i ] in to
l i s t B n A [ i ] 
4 fo r i  1 to n -1
5 d o s o rt lis t B [ i ] w ith in s e rtio n s o rt
6 c o n c a te n a te B [ 0 ], B [1 ],..., B [ n  1 ] to g e th e r in
o rd e r

Chapter 8 P.10
 Computer Theory Lab.

Analysis
The running time of bucket sort is
n 1
T (n)  (n)   O (ni2 ).
i 0
taking expectations of both sides and using linearity of
expectation, we have
 n1 
E[T (n)]  E (n)   O(ni2 )
 i 0 
n 1
　 　 　 　 　 (n)   E[O(ni2 )]
i 0
n 1
　 　 　 　 　 (n)   O[ E (ni2 )]
i 0
Chapter 8 P.11
 Computer Theory Lab.

We claim that
E[ni2 ]  2  1 / n

We define indicator random variables

Xij = I {A[j] falls in bucket i}
for i = 0, 1, …, n-1 and j = 1, 2,…,n. thus,
n
ni   X ij .
j 1

Chapter 8 P.12
 Computer Theory Lab.

 n 2

E[ni2 ]  E   X ij  
 j 1  
 
n n 
　 　 　  E    X ij X ik 
 j 1k 1 
 
n
 
　 　 　  E   X ij2    X ij X ik 
 j 1 1 j  n 1 k n

 k j 
n
　 　 　   E[ X ij2 ]    E[ X ij X ik ],
j 1 1 j  n 1k n
k j

Chapter 8 P.13
 Computer Theory Lab.

Indicator random variable Xij is 1 with probability 1/n and 0

otherwise, and therefore
1  1
E[ X ij2 ]  1   0  1  
n  n
1

n
When k  j, the variables Xij and Xik are independent, and hence
E[ X ij X ik ]  E[ X ij ]E[ X ik ]
1 1
　 　 　 　 　 
n n
1
　 　 　 　 　 2 .
Chapter 8
n P.14
 Computer Theory Lab.

n 1 1
E[ni2 ]  n   2

j 1 1 j  n 1 k  n n
k j

1 1
　 　 　  n   n(n  1)  2
n n
n 1
　 　 　 1 
n
1
　 　 　 2 
n
We can conclude that the expected time for bucket sort is
(n)+n·O(2-1/n)= (n).
Chapter 8 P.15
 Computer Theory Lab.

Another analysis:

n 1 2 n 1 2 n 1
 O( E [ ni ])  O(  E [ ni ])  O(  (1 ))  O( n )
i 0 i 0 i 0

Because

E( ni2 )  Var [ ni ]  E 2 [ ni ]

1
E [ ni ]  np  1 where p=
n

1
Var [ ni ]  np(1  p )  1 
n

1 2 1
E( ni2 )  1   1  2   (1 )
n n

(Basic Probability Theory)

Chapter 8 P.16