INTRODUCTION TO SORTING

Sorting is nothing but storage of data in sorted order, it can be in ascending or descending order.
The term Sorting comes into picture with the term Searching. There are so many things in our
real life that we need to search, like a particular record in database, roll numbers in merit list, a
particular telephone number, any particular page in a book etc.

Sorting arranges data in a sequence which makes searching easier. Every record which is going
to be sorted will contain one key. Based on the key the record will be sorted.

5.3 TYPES OF SORTING

• An internal sort is any data sorting process that takes place entirely within the main
memory of a computer. This is possible whenever the data to be sorted is small enough to
all be held in the main memory.
• External sorting is a term for a class of sorting algorithms that can handle massive
amounts of data. External sorting is required when the data being sorted do not fit into the
main memory of a computing device (usually RAM) and instead they must reside in
the slower external memory (usually a hard drive). External sorting typically uses a
hybrid sort-merge strategy. In the sorting phase, chunks of data small enough to fit in
main memory are read, sorted, and written out to a temporary file. In the merge phase, the
sorted sub files are combined into a single larger file.
• We can say a sorting algorithm is stable if two objects with equal keys appear in the same
order in sorted output as they appear in the input unsorted array.
5.3.1 Insertion sort
It is a simple sorting algorithm that builds the final sorted array (or list) one item at a time. This
algorithm is less efficient on large lists than more advanced algorithms such as quicksort,
heapsort, or merge sort. However, insertion sort provides several advantages:

· Simple implementation
· Efficient for small data sets
· Stable; i.e., does not change the relative order of elements with equal keys
· In-place; i.e., only requires a constant amount O(1) of additional memory space.

How Insertion Sort Works

Insertion Sort Algorithm

This algorithm sorts the array A with N elements.

1. Set A[0]=-12345(infinity i.e. Any large no)

2. Repeat step 3 to 5 for k=2 to n

3. Set key=A[k] And j=k-1

4. Repeat while key<A[j]

A) Set A[j+1]=A[j]

b) j=j-1

5. Set A[j+1]=key

6. Return

//CODE
int A[6] = {5, 1, 6, 2, 4, 3};
int i, j, key;
for(i=1; i<6; i++)
{
key = A[i];
j = i-1;
while(j>=0 && key < A[j])
{
A[j+1] = A[j];
j--;
}
A[j+1] = key;
}
Complexity of Insertion Sort

The number f(n) of comparisons in the insertion sort algorithm can be easily computed. First of
all, the worst case occurs when the array A is in reverse order and the inner loop must use the
maximum number K-1 of comparisons. Hence

2
F(n)= 1+2+3+……………………………….+(n-1)=n(n-1)/2= O(n )
Furthermore, One can show that, on the average, there will be approximately (K-1)/2
comparisons in the inner loop. Accordingly, for the average case.
2
F(n)=O(n )
Thus the insertion sort algorithm is a very slow algorithm when n is very large.

Algorithm Worst Case Average Case Best Case

2 2
Insertion Sort n(n-1)/2 = O(n ) n(n-1)/4 = O(n ) O(n)

5.3.2 Selection Sort

Selection sorting is conceptually the simplest sorting algorithm. This algorithm first finds the
smallest element in the array and exchanges it with the element in the first position, then find the
second smallest element and exchange it with the element in the second position, and continues
in this way until the entire array is sorted
How Selection Sort works

In the first pass, the smallest element found is 1, so it is placed at the first position, then leaving
first element, smallest element is searched from the rest of the elements, 3 is the smallest, so it is
then placed at the second position. Then we leave 1 and 3, from the rest of the elements, we
search for the smallest and put it at third position and keep doing this, until array is sorted
Selection Sort Algorithm

1. Repeat For J = 0 to N-1

2. Set MIN = J
3. Repeat For K = J+1 to N
4. If (A[K] < A[MIN]) Then
5. Set MIN = K
6. Interchange A[J] and A[MIN]
[End of Step 1 For Loop]
7. Exit
//CODE
void selectionSort(int a[], int size)

{
int i, j, min, temp;
for(i=0; i < size-1; i++ )
{
min = i; //setting min as i
 for(j=i+1; j < size; j++)
{
if(a[j] < a[min]) //if element at j is less than element at min position
{
min = j; //then set min as j
}
}
temp = a[i];
a[i] = a[min];
a[min] = temp;
}
}

Complexity of Selection Sort Algorithm

The number of comparison in the selection sort algorithm is independent of the original order of

the element. That is there are n-1 comparison during PASS 1 to find the smallest element, there
are n-2 comparisons during PASS 2 to find the second smallest element, and so on. Accordingly

2
F(n)=(n-1)+(n-2)+…………………………+2+1=n(n-1)/2 = O(n )

Algorithm Worst Case Average Case Best Case

2 2 2
Selection Sort n(n-1)/2 = O(n ) n(n-1)/2 = O(n ) O(n )

Bubble Sort

Bubble Sort is an algorithm which is used to sort N elements that are given in a memory for eg:
an Array with N number of elements. Bubble Sort compares all the element one by one and sort
them based on their values.

It is called Bubble sort, because with each iteration the smaller element in the list bubbles up
towards the first place, just like a water bubble rises up to the water surface.
Sorting takes place by stepping through all the data items one-by-one in pairs and comparing
adjacent data items and swapping each pair that is out of order.

How Bubble Sort Works

Let us take the array of numbers "5 1 4 2 8", and sort the array from lowest number to greatest
number using bubble sort. In each step, elements written in bold are being compared. Three
passes will be required.

First Pass:

(51428) ( 1 5 4 2 8 ), Here, algorithm compares the first two elements, and swaps since
5 > 1.
(15428) ( 1 4 5 2 8 ), Swap since 5 > 4

(14528) ( 1 4 2 5 8 ), Swap since 5 > 2

(14258) ( 1 4 2 5 8 ), Now, since these elements are already in order (8 > 5),
algorithm does not swap them.

Second Pass:

(14258) (14258)

(14258) ( 1 2 4 5 8 ), Swap since 4 > 2

(12458) (12458)

(12458) (12458)
Now, the array is already sorted, but our algorithm does not know if it is completed. The
algorithm needs one whole pass without any swap to know it is sorted.
Third Pass:
(12458) (12458)
(12458) (12458)
(12458) (12458)
(12458) (12458)

Bubble Sort Algorithm

1. Repeat Step 2 and 3 for k=1 to n

2. Set ptr=1

3. Repeat while ptr<n-k

a) If (A[ptr] > A[ptr+1]) Then

Interchange A[ptr] and A[ptr+1]

[End of If]

b) ptr=ptr+1

[end of step 3 loop]

[end of step 1 loop]

4. Exit /

/CODE

Let's consider an array with values {5, 1, 6, 2, 4, 3}

int a[6] = {5, 1, 6, 2, 4, 3}; int i, j, temp;

for(i=0; i<6, i++)

{
for(j=0; j<6-i-1; j++)
{
if( a[j] > a[j+1])
{
temp = a[j];
a[j] = a[j+1];
a[j+1] = temp;
}
}
}
Above is the algorithm, to sort an array using Bubble Sort. Although the above logic will sort
and unsorted array, still the above algorithm isn't efficient and can be enhanced further. Because
as per the above logic, the for loop will keep going for six iterations even if the array gets sorted
after the second iteration.

Hence we can insert a flag and can keep checking whether swapping of elements is taking place
or not. If no swapping is taking place that means the array is sorted and we can jump out of the
for loop.

int a[6] = {5, 1, 6, 2, 4, 3};

int i, j, temp;
for(i=0; i<6, i++)
{
for(j=0; j<6-i-1; j++)
{
int flag = 0; //taking a flag variable
if( a[j] > a[j+1])
{
temp = a[j];
a[j] = a[j+1];
a[j+1] = temp;
flag = 1; //setting flag as 1, if swapping occurs
}
}
if(!flag) //breaking out of for loop if no swapping takes place
{
break;
}
}
In the above code, if in a complete single cycle of j iteration(inner for loop), no swapping takes
place, and flag remains 0, then we will break out of the for loops, because the array has already
been sorted.

Complexity of Bubble Sort Algorithm

In Bubble Sort, n-1 comparisons will be done in 1st pass, n-2 in 2nd pass, n-3 in 3rd pass and so
on. So the total number of comparisons will be

2
F(n)=(n-1)+(n-2)+…………………………+2+1=n(n-1)/2 = O(n )

Algorithm Worst Case Average Case Best Case

2 2
Bubble Sort n(n-1)/2 = O(n ) n(n-1)/2 = O(n ) O(n)

Quick Sort

Quick Sort, as the name suggests, sorts any list very quickly. Quick sort is not stable search, but
it is very fast and requires very less additional space. It is based on the rule of Divide and
Conquer (also called partition-exchange sort). This algorithm divides the list into three main
parts

Elements less than the Pivot element

Pivot element
Elements greater than the pivot element

How Quick Sort Works

In the list of elements, mentioned in below example, we have taken 25 as pivot. So after the first
pass, the list will be changed like this.

6 8 17 14 25 63 37 52
Hence after the first pass, pivot will be set at its position, with all the elements smaller to it on its
left and all the elements larger than it on the right. Now 6 8 17 14 and 63 37 52 are considered as
two separate lists, and same logic is applied on them, and we keep doing this until the complete
list is sorted.

Quick Sort Algorithm

QUICKSORT (A, p, r)

1 if p < r

2 then q ← PARTITION (A, p, r)3 QUICKSORT (A, p, q - 1)

4 QUICKSORT (A, q + 1, r)

The key to the algorithm is the PARTITION procedure, which rearranges the subarray A[p r] in
place.

PARTITION (A, p, r)

1 x ← A[r]

2i←p-1
3 for j ← p to r - 1

4 do if A[j] ≤ x

5 then i ← i + 1

6 exchange A[i] ↔ A[j]

7 exchange A[i + 1] ↔ A[r]

8 return i + 1

//CODE

/* Sorting using Quick Sort Algorithm

a[] is the array, p is starting index, that is 0,
and r is the last index of array. */

void quicksort(int a[], int p, int r)

{
if(p < r)
{
int q;
q = partition(a, p, r);
quicksort(a, p, q-1);
quicksort(a, q+1, r);
}
}

int partition (int a[], int p, int r)

{
int i, j, pivot, temp;
pivot = a[p];
i = p;
j = r;
while(1)
{
while(a[i] < pivot && a[i] != pivot)
i++;
while(a[j] > pivot && a[j] != pivot)
j--;
if(i < j)
{
temp = a[i];
a[i] = a[j];
a[j] = temp;
 }
else
{
Return j;
}}}
Complexity of Quick Sort Algorithm

The Worst Case occurs when the list is sorted. Then the first element will require n comparisons
to recognize that it remains in the first position. Furthermore, the first sublist will be empty, but
the second sublist will have n-1 elements. Accordingly the second element require n-1
comparisons to recognize that it remains in the second position and so on.
2
F(n)= n+(n-1)+(n-2)+…………………………+2+1=n(n+1)/2 = O(n )

Algorithm Worst Case Average Case Best Case

2
Quick Sort n(n+1)/2 = O(n ) O(n logn) O(n logn)

Merge Sort

Merge Sort follows the rule of Divide and Conquer. But it doesn't divide the list into two halves.
In merge sort the unsorted list is divided into N sub lists, each having one element, because a list
of one element is considered sorted. Then, it repeatedly merge these sub lists, to produce new
sorted sub lists, and at lasts one sorted list is produced.
Merge Sort is quite fast, and has a time complexity of O(n log n). It is also a stable sort, which
means the equal elements are ordered in the same order in the sorted list.

How Merge Sort Works

Suppose the array A contains 8 elements, each pass of the merge-sort algorithm will start at the
beginning of the array A and merge pairs of sorted subarrays as follows.
PASS 1. Merge each pair of elements to obtain the list of sorted pairs.

PASS 2. Merge each pair of pairs to obtain the list of sorted quadruplets.

PASS 3. Merge each pair of sorted quadruplets to obtain the two sorted subarrays.

PASS 4. Merge the two sorted subarrays to obtain the single sorted array.
Merge Sort Algorithm

/* Sorting using Merge Sort Algorithm

a[] is the array, p is starting index, that is 0,
and r is the last index of array. */

Lets take a[5] = {32, 45, 67, 2, 7} as the array to be sorted.

void mergesort(int a[], int p, int r)

{
int q;
if(p < r)
{
q = floor( (p+r) / 2);
mergesort(a, p, q);
mergesort(a, q+1, r);
merge(a, p, q, r);
}
}

void merge (int a[], int p, int q, int r)

{
int b[5]; //same size of a[]
int i, j, k;
k = 0;
i = p;
j = q+1;
while(i <= q && j <= r)
{
if(a[i] < a[j])
{
b[k++] = a[i++]; // same as b[k]=a[i]; k++; i++;
 }
else
{
b[k++] = a[j++];
}
}

while(i <= q)
{
b[k++] = a[i++];
}

while(j <= r)
{
b[k++] = a[j++];
}
for(i=r; i >= p; i--)
{
a[i] = b[--k]; // copying back the sorted list to a[]
}
}
Complexity of Merge Sort Algorithm

Let f(n) denote the number of comparisons needed to sort an n-element array A using merge-sort
algorithm. The algorithm requires at most logn passes. Each pass merges a total of n elements
and each pass require at most n comparisons. Thus for both the worst and average case
F(n) ≤ n logn

Thus the time complexity of Merge Sort is O(n Log n) in all 3 cases (worst, average and best) as
merge sort always divides the array in two halves and take linear time to merge two halves.

Algorithm Worst Case Average Case Best Case

Merge Sort O(n logn) O(n logn) O(n logn)

Heap Sort

Heap Sort is one of the best sorting methods being in -place and with no quadratic worst-case
scenarios. Heap sort algorithm is divided into two basic parts Creating a Heap of the unsorted
list.
Then a sorted array is created by repeatedly removing the largest/smallest element from the heap,
and inserting it into the array. The heap is reconstructed after each removal. What is a Heap?

Heap is a special tree-based data structure that satisfies the following special heap properties
Shape Property: Heap data structure is always a Complete Binary Tree, which means all levels
of the tree are fully filled.
Heap Property: All nodes are either greater than or equal to or less than or equal to each of its
children. If the parent nodes are greater than their children, heap is called a Max-Heap, and if the
parent nodes are smaller than their child nodes, heap is called Min-Heap.

How Heap Sort Works

Initially on receiving an unsorted list, the first step in heap sort is to create a Heap data structure
(Max-Heap or Min-Heap). Once heap is built, the first element of the Heap is either largest or
smallest (depending upon Max-Heap or Min-Heap), so we put the first element of the heap in our
array. Then we again make heap using the remaining elements, to again pick the first element of
the heap and put it into the array. We keep on doing the same repeatedly until we have the
complete sorted list in our array.
Heap Sort Algorithm

• HEAPSORT(A)
1. BUILD-MAX-HEAP(A)
2. for i ← length[A] downto 2
3. do exchange A[1] ↔ A[i ]
4. heap-size[A] ← heap-size[A] – 1
5. MAX-HEAPIFY(A, 1)

• BUILD-MAX-HEAP(A)
1. heap-size[A] ← length[A]
2. for i ← length[A]/2 downto 1
3. do MAX-HEAPIFY(A, i )

• MAX-HEAPIFY(A, i )
1. l ← LEFT(i )
2. r ← RIGHT(i )
3. if l ≤ heap-size[A] and A[l] > A[i ]
4. then largest ←l
5. else largest ←i
6. if r ≤ heap-size[A] and A[r] > A[largest]
7. then largest ←r
8. if largest = i
9. then exchange A[i ] ↔ A[largest]
10. MAX-HEAPIFY(A, largest)
//CODE

In the below algorithm, initially heapsort() function is called, which calls buildmaxheap() to
build heap, which inturn uses maxheap() to build the heap.

void heapsort(int[], int);

void buildmaxheap(int [], int);
void maxheap(int [], int, int);

void main()
{
int a[10], i, size;
printf("Enter size of list"); // less than 10, because max size of array is 10
scanf(“%d”,&size);
printf( "Enter" elements");
for( i=0; i < size; i++)
{
Scanf(“%d”,&a[i]);
}
heapsort(a, size);
getch();
}

void heapsort (int a[], int length)

{
buildmaxheap(a, length);
int heapsize, i, temp;
heapsize = length - 1;
for( i=heapsize; i >= 0; i--)
{
temp = a[0];
a[0] = a[heapsize];
a[heapsize] = temp;
heapsize--;
maxheap(a, 0, heapsize);
}
for( i=0; i < length; i++)
{
Printf("\t%d" ,a[i]);
}
}

void buildmaxheap (int a[], int length)

{
int i, heapsize;
heapsize = length - 1;
 for( i=(length/2); i >= 0; i--)
{
maxheap(a, i, heapsize);
}
}

void maxheap(int a[], int i, int heapsize)

{
int l, r, largest, temp;
l = 2*i;
r = 2*i + 1;
if(l <= heapsize && a[l] > a[i])
{
largest = l;
}
else
{
largest = i;
}
if( r <= heapsize && a[r] > a[largest])
{
largest = r;
}
if(largest != i)
{
temp = a[i];
a[i] = a[largest];
a[largest] = temp;
maxheap(a, largest, heapsize);
}
}

Complexity of Heap Sort Algorithm

The heap sort algorithm is applied to an array A with n elements. The algorithm has two phases,
and we analyze the complexity of each phae separately.
Phase 1. Suppose H is a heap. The number of comparisons to find the appropriate place of a new
element item in H cannot exceed the depth of H. Since H is complete tree, its depth is bounded
by log2m where m is the number of elements in H. Accordingly, the total number g(n) of
comparisons to insert the n elements of A into H is bounded as
g(n) ≤ n log2n
Phase 2. If H is a complete tree with m elements, the left and right subtrees of H are heaps and L
is the root of H Reheaping uses 4 comparisons to move the node L one step down the tree H.
Since the depth cannot exceeds log2m , it uses 4log2m comparisons to find the appropriate place
of L in the tree H.
h(n)≤4nlog2n
Thus each phase requires time proportional to nlog2n, the running time to sort n elements array
A would be nlog2n

Algorithm Worst Case Average Case Best Case

Heap Sort O(n logn) O(n logn) O(n logn)

Radix Sort

The idea is to consider the key one character at a time and to divide the entries, not into two sub
lists, but into as many sub lists as there are possibilities for the given character from the key. If
our keys, for example, are words or other alphabetic strings, then we divide the list into 26 sub
lists at each stage. That is, we set up a table of 26 lists and distribute the entries into the lists
according to one of the characters in the key.

How Radix Sort Works

A person sorting words by this method might first distribute the words into 26 lists according to
the initial letter (or distribute punched cards into 12 piles), then divide each of these sub lists into
further sub lists according to the second letter, and so on. The following idea eliminates this
multiplicity of sub lists: Partition the items into the table of sub lists first by the least significant
position, not the most significant. After this first partition, the sub lists from the table are put
back together as a single list, in the order given by the character in the least significant position.
The list is then partitioned into the table according to the second least significant position and
recombined as one list. When, after repetition of these steps, the list has been partitioned by the
most significant place and recombined, it will be completely sorted. This process is illustrated by
sorting the list of nine three-letter words below.
Radix Sort Algorithm

Radixsort(A,d)

1. For i←1 to d
2. Do use a stable sort to sort array A on digit i

Complexity of Radix Sort Algorithm

The list A of n elements A1, A2,……………An is given. Let d denote the radix(e.g d=10 for
decimal digits, d=26 for letters and d=2 for bits) and each item Ai is represented by means of s
of the digits:
Ai = di1 di2………………. dis
The radix sort require s passes, the number of digits in each item . Pass K will compare each
dik with each of the d digits. Hence

C(n)≤ d*s*n

Algorithm Worst Case Average Case Best Case

2
Radix Sort O(n ) d*s*n O(n logn)