Advanced Algorithms Analysis

Dr. Ehsan Ullah Munir

1
Comparison & Non-Comparison Sorting
• Comparison sorting algorithms are the most used type of sorting algorithm.
They compare the values in the data set to each other and use this
comparison to sort the data. The most common comparison sorting
algorithms include bubble sort, selection sort, insertion sort, quicksort,
mergesort, and heapsort.
• Non-comparison sorting algorithms sort data without comparing the values
in the data set to each other. Instead, they use specific properties of the data
to sort it. The most common non-comparison sorting algorithms include
counting sort, radix sort, and bucket sort.
Linear Time Sorting
• Linear time sorting is a subset of sorting algorithms with a significant
advantage: they can sort a given set of elements in linear time [O(n)]; the
runtime increases linearly with the input size.
• Computational sorting is particularly efficient when the range of input
elements is known and relatively small. This eliminates the need to compare
elements, the main time-consuming operation in many other sorting
algorithms.
• The two main types are count-based algorithms and radix-based algorithms.

3
Linear Time Sorting - Advantages
• Efficient for large input sizes: The time complexity of such algorithms is O(n), which
means that the running time increases linearly with the input size. This makes them
very efficient for large data sets.
• No comparison operations: Such algorithms do not rely on elementary comparison.
• Suitability to specific input properties: Linear-time sorting algorithms often have
specific requirements or assumptions about the input elements. For example, to
calculate a sort order, you need to know the range of input elements in advance.
• Stable sort: Many linear-time sorting algorithms, including counting and radix sort,
are inherently stable.
• Ease of use: Linear-time sorting algorithms are easier to understand and debug,
making them suitable for situations where simplicity and clarity are desired.

4
Linear Time Sorting – Disadvantages
• Constraining input requirements: Linear time sorting algorithms often have
specific requirements or assumptions about the input elements. For example,
to calculate a sort order, you need to know the range of input elements in
advance.
• Additional space requirements: Some linear time sorting algorithms, such as
counting sort, require additional space to store other arrays or data
structures.
• Lack of Versatility: Linear time sorting algorithms are specialized algorithms
designed for specific scenarios or constraints.
• Inefficient for small ranges or sparse data: Such algorithms are most efficient
when the range of input elements is small and densely distributed.
• Limited to specific data types: Such algorithms, such as counting sort, are
primarily designed to sort non-negative integers or key-value objects.
5
Counting Sort
• Counting sort is a non-comparative sorting algorithm.
• It counts the occurrence of each particular element in the input array and
uses this information to determine the correct position of each element in
the sorted output array.
• Input size n
• Range k (1-5)

6
Counting Sort
• Counting sort is a non-comparative sorting algorithm.
• It counts the occurrence of each particular element in the input array and
uses this information to determine the correct position of each element in
the sorted output array.
• Input size n 2 1 2 3 1 4 2 4
• Range k (1-5)
• Create an extra array of the range size
• Advantage linear time O (n + k)
• Disadvantage
• 2 10000 3 6 7

7
Counting Sort

8
Counting Sort

9
Counting Sort

Range 0-5

10
Counting Sort - Algorithm
// Function to sort the array using counting sort
void countingSort(int A[], int B[], int k, int n)
{
int C[k + 1];
// initialize count array with zeros
for (int i = 0; i <= k; i = i + 1)
C[i] = 0;

// store the count of each element in the count array

for (int i = 0; i < n; i = i + 1)
C[A[i]] = C[A[i]] + 1 ;

// store the cumulative count of each element in the count array

for (int i = 1; i <= k; i = i + 1)
C[i] = C[i] + C[i - 1];

// place the elements in the sorted array

for (int i = n - 1; i >= 0; i = i - 1)
{
B[C[A[i]]] = A[i];
c[A[i]] = c[A[i]] - 1;
}
} 11
Counting Sort - Demo

12
Counting Sort - Complexity
// Function to sort the array using counting sort
void countingSort(int A[], int B[], int k, int n)
{
int C[k + 1];
// initialize count array with zeros
for (int i = 0; i <= k; i = i + 1) takes O(k) time
C[i] = 0;

// store the count of each element in the count array

for (int i = 0; i < n; i = i + 1) takes O(n) time
C[A[i]] = C[A[i]] + 1 ;

// store the cumulative count of each element in the count array

for (int i = 1; i <= k; i = i + 1) takes O(k) time
C[i] = C[i] + C[i - 1];

// place the elements in the sorted array

for (int i = n - 1; i >= 0; i = i - 1) takes O(n) time
{
B[C[A[i]]] = A[i];
c[A[i]] = c[A[i]] - 1; The overall complexity = O(k) + O(n) + O(k) + O(n) = O(k+n)
} if k = n, then the complexity = O(n)
} 13
Radix Sort
• Radix sort is the linear sorting algorithm that is used for integers.
• The key idea behind Radix Sort is to exploit the concept of place value.
• It assumes that sorting numbers digit by digit will eventually result in a fully sorted
list.
• Radix Sort can be performed using different variations, such as Least Significant
Digit (LSD) Radix Sort or Most Significant Digit (MSD) Radix Sort.
• 904
• 46
• 5
• 74
• 62
• 1
14
Radix Sort

15
Radix Sort

16
Radix Sort – Demo (LSD–Lest Significant Digit)
• Suppose, we have an array of 7
elements.
• First, we will sort elements based on the
value of the unit place.
• Then, we will sort elements based on
the value of the tenth place.
• This process goes on until the last
significant place (i.e. Most Significant
Digit (MSD)).
• Let the initial array be:

121 432 564 23 1 45 788

17
Radix Sort - Complexity

it requires O(n) on each iteration

complexity of O(d * (n + b)) We usually omit the base, as it is most of the

where d is the number of digits, time base-10. So,
n is the number of elements, The complexity = O(dn)
and b is the base of the number system being used
18
Bucket Sort
• The bucket sort algorithm is an advanced sorting technique, in contrast to
others.
• Bucket sort can be considered as a collective framework built using a variety
of sorting algorithms.
• Bucket sort, also known as bin sort, is a sorting algorithm that divides an
array's elements into several buckets.
• The buckets are then sorted one at a time, either using a different sorting
algorithm or by recursively applying the bucket sorting algorithm.

19
Bucket Sort - Demo
• To apply bucket sort on the input array
[0.78, 0.17, 0.39, 0.26, 0.72, 0.94, 0.21, 0.12, 0.23, 0.68]
• we follow these steps:
• Step 1: Create an array of size 10, where each slot represents a bucket.

20
Bucket Sort - Demo
• Step 2: Insert elements into the buckets from the input array based on their
range.

21
Bucket Sort - Demo
• Step 3: Sort the elements within each bucket. Use any stable sorting
algorithm to sort the elements within each bucket.

22
Bucket Sort - Demo
• Step 4: Gather the elements from each bucket and put them back into the
original array.

23
Bucket Sort - Demo
• Step 5: The original array now contains the sorted elements. Return the
sorted array.

24
Bucket Sort - Demo

25
Bucket Sort - Demo

26
Bucket Sort - Complexity

Steps Description Time

Create an array of size 10, where each slot represents a
1 O(1)
bucket.
Insert elements into the buckets from the input array
2 O(n)
based on their range.
Sort the elements within each bucket. Use any stable
3 sorting algorithm to sort the elements within each O(n)
bucket.
Gather the elements from each bucket and put them
4 O(n)
back into the original array.
Overall complexity O(n)
27