0% found this document useful (0 votes)
48 views17 pages

Announcements: Weekly Reading Assignment: Chapter 9 (CLRS) (Not On This Exam)

The document discusses various sorting algorithms including counting sort, radix sort, and bucket sort. It provides analysis of their time complexities and proofs of correctness. Counting sort runs in linear time when elements are integers between 1 and n. Radix sort generalizes counting sort to multiple digits and also runs in linear time. Bucket sort handles floating point numbers by distributing them into buckets that are then sorted individually.

Uploaded by

Ibrahim Hawari
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
48 views17 pages

Announcements: Weekly Reading Assignment: Chapter 9 (CLRS) (Not On This Exam)

The document discusses various sorting algorithms including counting sort, radix sort, and bucket sort. It provides analysis of their time complexities and proofs of correctness. Counting sort runs in linear time when elements are integers between 1 and n. Radix sort generalizes counting sort to multiple digits and also runs in linear time. Bucket sort handles floating point numbers by distributing them into buckets that are then sorted individually.

Uploaded by

Ibrahim Hawari
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

Announcements

Weekly Reading Assignment:


Chapter 9 (CLRS) [not on this exam]

Homework #3 due on Th. 9/19/13
Exam on sorting and its analysis on
Th. 9/26/13.


M. C. Lin
Decision Tree
Each internal node is annotated by a
i
: a
j

for some i and j in range 1 i,j n. Each
leaf is annotated by a permutation (i).
Lower Bound for Worst Case
Any decision tree that sorts n elements has
height (n lg n).

Proof: There are n! permutations of n elements, each
permutation representing a distinct sorted order, the
tree must have at least n! leaves. Since a binary tree
of height h has no more than 2
h
leaves, we have

n! 2
h
h lg(n!)
By Stirlings approximation: n! > (n/e)
n
h lg(n!) lg(n/e)
n
= n lg n - n lg e = (n lg n)


Counting-Sort (A, B, k)
1. for i 1 to k /* entries are in [1,k]
2. do C[i] 0
3. for j 1 to length[A] /* count # of instances of
each possible entry
4. do C[A[j]] C[A[j]] + 1
5. for i 2 to k /* count # cumulative instances up
to each possible entry

6. do C[i] C[i] + C[i-1]
7. for j length[A] downto 1
8. do B[C[A[ j ]]] A[j] /* move results into B
9. C[A[j]] C[A[j]] - 1

Counting Sort
Assuming each of n input elements is an integer
ranging 1 to k, when k = O(n) sort runs in O(n) time.








Algorithm Analysis
The overall time is O(n+k). When we have
k=O(n), the worst case is O(n).
for-loop of lines 1-2 takes time O(k)
for-loop of lines 3-4 takes time O(n)
for-loop of lines 5-6 takes time O(k)
for-loop of lines 7-9 takes time O(n)

Stable, but not in place.

No comparisons made: it uses actual values
of the elements to index into an array.
Radix Sort
It was used by the card-sorting machines
to read the punch cards.

The key is sort the least significant digit
first and the remaining digits in sequential
order. The sorting method used to sort
each digit must be stable.
If we start with the most significant
digit, well need extra storage.
An Example
392 631 928 356
356 392 631 392
446 532 532 446
928 495 446 495
631 356 356 532
532 446 392 631
495 928 495 928

Radix-Sort(A, d)
1. for i 1 to d
2. do use a stable sort to sort array A on digit i

** To prove the correctness of this algorithm by
induction on the column being sorted:
Proof: Assuming that radix sort works for d-1 digits, well
show that it works for d digits.
Radix sort sorts each digit separately, starting from digit 1.
Thus radix sort of d digits is equivalent to radix sort of
the low-order d -1 digits followed by a sort on digit d .
Correctness of Radix Sort
By our induction hypothesis, the sort of the low-order d-1
digits works, so just before the sort on digit d , the elements
are in order according to their low-order d-1 digits. The
sort on digit d will order the elements by their dth digit.
Consider two elements, a and b, with dth digits a
d
and b
d
:
If a
d
< b
d
, the sort will put a before b, since a < b regardless
of the low-order digits.
If a
d
> b
d
, the sort will put a after b, since a > b regardless
of the low-order digits.
If a
d
= b
d
, the sort will leave a and b in the same order,
since the sort is stable. But that order is already correct,
since the correct order of is determined by the low-order
digits when their dth digits are equal.
Algorithm Analysis, Assuming
Digit Sort Done by Counting
Each pass over n d-digit numbers then takes
time (n+k).

There are d passes, so the total time for radix
sort is (d n+ d k).

When d is a constant and k = O(n), radix sort
runs in linear time.

Radix sort, if it uses counting sort as the
intermediate stable sort, does not sort in place.
If primary memory storage is an issue, quicksort or other
sorting methods may be preferable.
Bucket Sort
Counting sort and radix sort are good for integers.
For floating point numbers, try bucket sort or other
comparison-based methods.

Assume the input is generated by a random process
that distributes the elements uniformly over [0,1).
(Other ranges can be scaled accordingly.)

The basic idea is to divide the interval into n equal-
sized subintervals, or buckets, then insert the n
input numbers into the buckets. The elements in
each bucket are then sorted; lists from all buckets are
concatenated in sequential order to generate output.
An Example
Bucket-Sort (A)
1. n length[A]
2. for i 1 to n
3. do insert A[i] into list B[ nA[i] ]
4. for i 0 to n-1
5. do sort list B[i] with insertion sort
6. Concatenate the lists B[i]s together in order
Algorithm Analysis
All lines except line 5 take O(n) time in the worst
case. Total time to examine all buckets in line 5
is O(n), without the sorting time.
To analyze sorting time, let n
i
be a random
variable denoting the number of elements placed
in bucket B[i]. The avg. total time to sort is

i = 0 to n-1
O(E[n
i
2
]) =

O(
i = 0 to n-1
E[n
i
2
] ) = O(n)

E[n
i
2
] = Var[n
i
] + E
2
[n
i
]
= n p (1 - p) + 1
2
= 1 - (1/n) + 1
= 2 - 1/n = (1)
Review: Binomial Distribution
Given n independent trials, each trial has two
possible outcomes. Such trials are called
Bernoulli trials. If p is the probability of
getting a head, then the probability of getting
k heads in n tosses is given by (CLRS p.1203)
P(X=k) = (n!/(k!(n-k)!)) p
k
(1-p)
n-k
= b(k;n,p)

This probability distribution is called the
binomial distribution. p
k
is the probability of
tossing k heads and (1-p)
n-k
is the probability
of tossing n-k tails. (n!/(k!(n-k)!)) is the total
number of different ways that the k heads
could be distributed among n tosses.
Review: Binomial Distribution
See p. 1203-1205 for the derivations.

E[x] = n p

Var[x] = E[X
2
] - E
2
[X] = n p (1-p)

E[X
2
] = Var[X] + E
2
[X]
= n p (1-p) + (n p)
2
= 1(1-p) + 1
2

You might also like