CENG 205 – Data Structures and Algorithms

Week 13-a – Sorting

Dr. Yücel Tekin

Chapter 14 : Seaching and Sorting

 The two commonly used orders are numerical order and alphabetical order.
 In this chapter, we will discuss
 Searching Algoritms:
⁻ Linear Search
⁻ Binary Search
⁻ Interpolation Search
⁻ Jump Search
 Sorting Algoritms:
⁻ Buble Sort
⁻ Insertion Sort
⁻ Selection Sort
⁻ Merge Sort
⁻ Quick Sort
⁻ Radix Sort
⁻ Heap Sort
⁻ Shell Sort
⁻ Tree Sort
14.6 INTRODUCTION TO SORTING

Sorting means arranging the elements of an array in some relevant order (ascending or descending)

if A is an array, then the elements of A are arranged in a sorted order (ascending order) in such a way that
A[0] < A[1] < A[2] < ...... < A[N]

Example
Unsorted initialized array : A[] = {21, 34, 11, 9, 1, 0, 22};
Sorted array (ascending order) : A[] = {0, 1, 9, 11, 21, 22, 34};
Sorted array (descending order) : A[] = {34, 22, 21, 11, 9, 1, 0};
14.6 INTRODUCTION TO SORTING

 There are two types of sorting:

 Internal sorting - sorting the data stored in the computer’s memory
 External sorting - sorting the data stored in files.

 External sorting is applied when there is voluminous data that cannot be stored in the memory.
BUBBLE SORT

Bubble sort is a very simple method that sorts the array elements by repeatedly moving the largest element to the
highest index position of the array segment (in case of arranging elements in ascending order).

In bubble sorting, consecutive adjacent pairs of elements in the array are compared with each other.

If the element at the lower index is greater than the element at the higher index, the two elements are interchanged
so that the element is placed before the bigger one.

This process will continue till the entired list is sorted.

Note that at the end of the first pass, the largest element in the list will be placed at its proper position (i.e., at the
end of the list).
BUBBLE SORT - Technique

The basic methodology of the working of bubble sort is given as follows:

In Pass 1 : A[0] > A[1]  A[1] >A[2]  A[2] > A[3]… A[N–2] > A[N–1] n–1 comparisons
In Pass 2 : A[0] > A[1]  A[1] >A[2]  A[2] > A[3]… A[N–3] > A[N–2] n–2 comparisons
In Pass 3 : A[0] > A[1]  A[1] >A[2]  A[2] > A[3]… A[N–4] > A[N–3] n–3 comparisons
.
.
.
In Pass n–1: A[0] and A[1] n–(n-1) = 1 comparisons

After this step, all the elements of the array are arranged in ascending order.
 BUBBLE SORT – Technique & Algorithm

The basic methodology of the working of bubble sort is given as follows:

In Pass 1 : A[0] > A[1]  A[1] >A[2]  A[2] > A[3]… A[N–2] > A[N–1] n–1 comparisons
In Pass 2 : A[0] > A[1]  A[1] >A[2]  A[2] > A[3]… A[N–3] > A[N–2] n–2 comparisons
In Pass 3 : A[0] > A[1]  A[1] >A[2]  A[2] > A[3]… A[N–4] > A[N–3] n–3 comparisons
.
.
.
In Pass n–1: A[0] and A[1] n–(n-1) = 1 comparisons

After this step, all the elements of the array are arranged in ascending order.

Algorithm for bubble sort

BUBBLE SORT - Example
A[] = {30, 52, 29, 87, 63, 27, 19, 54}

Pass 1:
1.Compare 30 and 52. Since 30 < 52, no swapping is done.
2.Compare 52 and 29. Since 52 > 29, swapping is done.
30, 29, 52, 87, 63, 27, 19, 54
3.Compare 52 and 87. Since 52 < 87, no swapping is done.
4.Compare 87 and 63. Since 87 > 63, swapping is done.
30, 29, 52, 63, 87, 27, 19, 54
5.Compare 87 and 27. Since 87 > 27, swapping is done.
30, 29, 52, 63, 27, 87, 19, 54
6.Compare 87 and 19. Since 87 > 19, swapping is done.
30, 29, 52, 63, 27, 19, 87, 54
7.Compare 87 and 54. Since 87 > 54, swapping is done.
30, 29, 52, 63, 27, 19, 54, 87

After Pass 1 : 30, 29, 52, 63, 27, 19, 54, 87

After Pass 2 : 29, 30, 52, 27, 19, 54, 63, 87
After Pass 3 : 29, 30, 27, 19, 52, 54, 63, 87
After Pass 4: 29, 27, 19, 30, 52, 54, 63, 87
After Pass 5: 27, 19, 29, 30, 52, 54, 63, 87
Algorithm for bubble sort
After Pass 6: 19, 27, 29, 30, 52, 54, 63, 87
After Pass 7: 19, 27, 29, 30, 52, 54, 63, 87
BUBBLE SORT - Complexity

The complexity of any sorting algorithm depends upon the number of comparisons.
to compute the complexity of bubble sort, we need to calculate the total number of comparisons.

It can be given as:

f(n) = (n – 1) + (n – 2) + (n – 3) + ..... + 3 + 2 + 1
f(n) = n (n – 1)/2
f(n) = n2/2 + O(n) = O(n2)

all cases (Best-Avarage-Worst) : O(n2)

BUBBLE SORT - Optimization

 Consider a case when the array is already sorted.

 In this situation no swapping is done but we still have to continue with all n–1 passes.

 We may even have an array that will be sorted in 2 or 3 passes but we still have to continue with rest of
the passes.

 So once we have detected that the array is sorted, the algorithm must not be executed further.

 This is the optimization over the original bubble sort algorithm.

BUBBLE SORT - Optimization
 In order to stop the execution of further passes after the array is sorted,
 Use a variable flag which is set to TRUE before each pass
 made FALSE when a swapping is performed

Optimized bubble sort code:

void bubble_sort(int *arr, int n)

{
int i, j, temp, flag = 0;
for(i=0; i<n; i++)
{
flag = 0
for(j=0; j<n–i–1; j++)
{
if(arr[j]>arr[j+1])
{
flag = 1;
temp = arr[j+1];
arr[j+1] = arr[j];
arr[j] = temp;
}
}
if(flag == 0) // array is sorted
return;
}
}
BUBBLE SORT - Optimized Bubble Sort Algorithm Complexity

the best case : O(n) (when the array is already sorted)

the worst case : O(n2)

when all the passes are performed, the algorithm will perform slower
than the original algorithm.

The average case also, the performance will see an improvement.

Compare it with the complexity of original bubble sort algorithm

which takes O(n2) in all the cases.
INSERTION SORT

Insertion sort is a very simple sorting algorithm in which the sorted array (or list) is built one
element at a time.

The main idea behind insertion sort is that it inserts each item into its proper place in the final
list.

Insertion sort is less efficient as compared to other more advanced algorithms such as quick sort, heap
sort, and merge sort.
INSERTION SORT- Technique

 The array of values to be sorted is divided into two sets:

 Sorted values
 Unsorted values

 The sorting algorithm will proceed until there are elements in the unsorted set.

 Suppose there are n elements in the array.

 Initially, the element with index 0 is in the sorted set.
 Rest of the elements are in the unsorted set.
 The first element of the unsorted partition has array index 1 (if Lower bound = 0).
 During each iteration of the algorithm, the first element in the unsorted set is picked up and
inserted into the correct position in the sorted set.
INSERTION SORT - Example

Consider an array of integers given below. We will sort the values in the array using insertion sort.

Solution
INSERTION SORT - Algorithm

Algorithm for insertion sort

INSERTION SORT - Complexity
The best case : O(n) (the array is already sorted)
The worst case: O(n2) (the array is sorted in the reverse order)
The Avarage case: O(n2) (at least (K–1)/2 comparisons)

 In the worst case, the first element of the unsorted set has to be compared with almost every element in the sorted set.
 Furthermore, every iteration of the inner loop will have to shift the elements of the sorted set of the array before inserting
the next element.
 Even in the average case, the insertion sort algorithm will have to make at least (K–1)/2 comparisons.
INSERTION SORT - Advantages

 easy to implement and efficient to use on small sets of data.

 It can be efficiently implemented on data sets that are already substantially sorted.
 It is over twice as fast as the bubble sort
 almost 40 per cent faster than the selection sort
 it requires less memory space (only O(1) of additional memory space).
 It is said to be online, as it can sort a list as and when it receives new elements.
SELECTION SORT
 Selection sort is a sorting algorithm that has a quadratic running time complexity of O(n2)
 it is not efficient to be used on large lists
 Although selection sort performs worse than insertion sort algorithm, it is noted for its simplicity and also has
performance advantages over more complicated algorithms in certain situations.
 Selection sort is generally used for sorting files with very large objects (records) and small keys.
SELECTION SORT - Technique
Consider an array ARR with N elements.

Selection sort works as follows:

First find the smallest value in the array and place it in the first position.
Then, find the second smallest value in the array and place it in the second position.
Repeat this procedure until the entire array is sorted.

In Pass 1 : find POS of the smallest value in array  swap ARR[POS]  ARR[0] ARR[0] is sorted
In Pass 2 : find POS of the smallest value in sub-array (N–1)  swap ARR[POS]  ARR[1] ARR[0], ARR[1] is sorted.

In Pass N–1: find POS of the smaller of the elements ARR[N–2] and ARR[N–1]  Swap ARR[POS]  ARR[N–2]

so that ARR[0], ARR[1], ..., ARR[N–1] is sorted.

SELECTION SORT - Example

Sort the array given below using selection sort.

Solution
SELECTION SORT - Algorithm

 In the algorithm, during the Kth pass, we need to find the position POS of the smallest elements from ARR[K],
ARR[K+1], ..., ARR[N].
 To find the smallest element, we use a variable SMALL to hold the smallest value in the sub-array ranging from ARR[K]
to ARR[N].
 Then, swap ARR[K] with ARR[POS].
 This procedure is repeated until all the elements in the array are sorted.

Algorithm for insertion sort

SELECTION SORT - Complexity
all cases (Best-Avarage-Worst) : O(n2)

Selection sort is a sorting algorithm that is independent of the original order of elements in the array.
In Pass 1, selecting the element with the smallest value calls for scanning all n elements; (n–1 comparisons)
In Pass 2, selecting the second smallest value requires scanning the remaining n – 1 elements and so on. (n-2 comparisons)
Therefore,
(n – 1) + (n – 2) + ... + 2 + 1 = n(n – 1) / 2 = O(n2) comparisons
SELECTION SORT - Advantages
 It is simple and easy to implement.
 It can be used for small data sets.
 It is 60 percent more efficient than bubble sort.

 However, in case of large data sets, the efficiency of selection sort drops as compared to insertion sort.
MERGE SORT
Merge sort is a sorting algorithm that uses the divide, conquer, and combine algorithmic paradigm.
Divide means partitioning the n-element array to be sorted into two sub-arrays of n/2 elements.
If A is an array containing zero or one element, then it is already sorted. However, if there are more
elements in the array, divide A into two sub-arrays, A1 and A2, each containing about half of the
elements of A.
Conquer means sorting the two sub-arrays recursively using merge sort.
Combine means merging the two sorted sub-arrays of size n/2 to produce the sorted array of n
elements.
MERGE SORT - Technique
Merge sort algorithm focuses on two main concepts to improve its performance (running time):
 A smaller list takes fewer steps and thus less time to sort than a large list.
 As number of steps is relatively less, thus less time is needed to create a sorted list from two sorted lists rather
than creating it using two unsorted lists.

The basic steps of a merge sort algorithm are as follows:

 If the array is of length 0 or 1, then it is already sorted.
 Otherwise, divide the unsorted array into two sub-arrays of about half the size.
 Use merge sort algorithm recursively to sort each sub-array.
 Merge the two sub-arrays to form a single sorted list.
MERGE SORT - Example

Sort the array given below using merge sort.

Solution
MERGE SORT - Algorithm

 The merge sort algorithm uses a function merge which

combines the sub-arrays to form a sorted array.
 While the merge sort algorithm recursively divides the list
into smaller lists, the merge algorithm conquers the list to sort
the elements in individual lists.
 Finally, the smaller lists are merged to form one list

Algorithm for merge sort

Merge function
MERGE SORT - Complexity

all cases (Best-Avarage-Worst) : O(n log n)

Although merge sort has an optimal time complexity, it needs an additional space of O(n) for the temporary array TEMP.
Time & Space Complexity Table of Sorting Algoritms
Next Lesson
Chapter15: Hashing and Collision