Analysis of Algorithms

Lecture 9
Today’s Outline
Linear Time Sorting Algorithms
◦ Counting sort Algorithm
◦ Working of Counting sort
◦ Analysis of Counting sort
◦ Radix sort Algorithm
◦ Working of Radix sort
◦ Analysis of Radix sort
Linear Time Sorting Algorithms
There are sorting algorithms that run faster than O(nlogn) time but they require
special assumptions about the input sequence.
They have the time complexity in linear time i.e. O(n)
Examples of these sorting algorithms that run in linear time are:
◦ Counting sort
◦ Radix sort
◦ Bucket sort
Counting Sort
Counting sort is a non-comparison sorting algorithm.
In this algorithm, array elements will be sort using key value. Where, Key value is
the highest element in the input array.
Consider the input array: 4 1 0 5 2 3 2. Then n=7 and key=5
Counting sort cannot applied on floating point values…

The algorithm uses three array:

Input array: A[0……n] store input elements
Output array: B[0…….n] store the sorted elements
Temporary array: C[0…..k] store data temporarily
Counting Sort Process
Suppose we have an input array A of size n
Step 1: Take a temporary array C of size k+1 and initialize all the array elements
with 0
Step 2: Count the number of distinct elements (frequency) from input array A
and save them in array C
Step 3: Find the actual position of the elements in the output array B by adding
the current and previous element in array C
Step 4: Place the elements in sorted order in array B
Step 5: Finally, copy the elements from array B to array A
 Counting Sort Algorithm
COUNTING-SORT (A, B, k)
Let C[0…..k] be a new array

for i = 0 to k

C[i] = 0 Step 1

for i = 0 to A.length – 1

C[A[i]] = C[A[i]] + 1 Step 2

for i = 1 to k

C[i] = C[i] + C[i – 1] Step 3

for i = A.length – 1 downto 0

C[A[i]] = C[A[i]] – 1

B[C[A[i]]] = A[i] Step 4

for i = 0 to A.length – 1

A[i] = B [i] Step 5

Working of Counting Sort
Input Array A 2 1 1 0 2 5 4 0 2 8 7 7 9 2 0 1 9
In the input array maximum element is 9, so k = 9
Create a temporary array C of size k+1 i.e. 10
Array C will be after step 2: 3 3 4 0 1 1 0 2 1 2
Array C will be after step 3: 3 6 10 10 11 12 12 14 15 17

Array B after step 4: 0 0 0 1 1 1 2 2 2 2 4 5 7 7 8 9 9

Finally, copy array B into array A in step 5
 Analysis of Counting Sort
COUNTING-SORT (A, B, k)
Let C[0…..k] be a new array

for i = 0 to k --------------------------- k

C[i] = 0

for i = 0 to A.length – 1-------------- n

C[A[i]] = C[A[i]] + 1

for i = 1 to k --------------------------- k

C[i] = C[i] + C[i – 1]

for i = A.length – 1 downto 0 ------- n

C[A[i]] = C[A[i]] – 1

B[C[A[i]]] = A[i]

for i = 0 to A.length – 1 --------------- n

A[i] = B [i]
Analysis of Counting Sort (cont.)
Adding all the values, f(n) = 3n + 2k
By ignoring the constants: f(n) = n + k
Therefore, Time complexity of counting sort is: O(n + k)
If k is supposed to be a constant then the complexity will be: O(n)
That is why it’s called linear time sorting algorithm
Radix Sort
Counting sort is the prerequisite of radix sort because we will use counting sort
as a sub routine in radix sort.
Radix sort is also a non-comparison sorting algorithm.
In this algorithm, the sorting will be done in d passes. Here, d is the highest digit
in the input array.
Consider the input array: 432, 8, 530, 90, 88, 231, 11, 45. Here the maximum
digit is 530 which is 3 digit number. So, counting sort will be applied 3 times.
Radix Sort Process
Suppose we have an input array A of size n
Step 1: Find the highest digit in the array and make all the elements d-digit
number. Suppose, if the highest element is a 3-digit number then make all the
elements 3-digit.
Step 2: Apply the counting sort d-times from Least Significant Digit (LSD) to Most
Significant Digit (MSD). If the highest element is a 3-digit number then counting
sort will be applied 3 time.
Working of Radix Sort
Input Array A 432 8 530 90 88 231 11 45 677 199
Step 1: Find the highest digit in the array i.e. 677 which is 3-digit number. Now
make all the elements 3-digit number.
432 008 530 090 088 231 011 045 677 199
Step 2: Call the Counting sort algorithm 3 times from LSD to MSD
After 1st pass: 530 090 231 011 432 045 677 008 088 199
After 2nd pass: 008 011 530 231 432 045 677 088 090 199
After 3rd pass: 008 011 045 088 090 199 231 432 530 677
Finally, the array is sorted.
Radix Sort – Example 2
Radix Sort Algorithm
RADIX-SORT (A, d)
for i = 1 to d

COUNTING-SORT (A)
Radix Sort Time Complexity
RADIX-SORT (A, d)
for i = 1 to d

COUNTING-SORT (A)

We have already examine the time complexity of counting sort i.e. O(n+k)
How many times the counting sort will be called i.e. number of digits time. So, the
time complexity of the radix sort is: O(d(n+k))
When d is constant and k = n, we can make radix sort run in linear time i.e. O(n)
Practice Assignment
1. Illustrate the operation of COUNTING-SORT on the array A = [6; 0; 2; 0; 1; 3;
4; 6; 1; 3; 2].
2. Illustrate the operation of RADIX-SORT on the following list of English words:
COW, DOG, SEA, RUG, ROW, MOB, BOX, TAB, BAR, EAR, TAR, DIG, BIG, TEA,
NOW, FOX.