1.3 Sorting
1.3 Sorting
Sorting:
4.1 Insertion Sort:
1 25 21
1st iteration 2 21 21 < 25 25
3 15 15
4 30 30
5 9 9
1 21 21 15
2nd iteration 2 25 15 15 < 21 21
3 15 15 < 25 25 25
4 30 30 30
5 9 9 9
1 15 15
3rd iteration 2 21 21
3 25 25
4 30 30 > 25 30
5 9 9
1 15 15 15 15 9
4th iteration 2 21 21 21 9 9 < 15 15
3 25 25 9 9 < 21 21 21
4 30 9 9 < 25 25 25 25
5 9 9 < 30 30 30 30 30
4.1.1 Algorithm:
INSERTION-SORT (A, N)
Where A is an array of N numbers.
15 | P a g e
Shashank Yadav (IT-dept KIET)
6. c6
∑
7. c7
∑
8. c8 n-1
Running time of insertion sort: The running time of the algorithm is the sum of running times for
each statement executed.
Best Case:
In best case, the array is already sorted. For each j=2, 3. . . n, we then find A[i] <=key in line 5 when i
has its initial value of j-1. Thus tj=1 for j=2, 3. . . n and best case running time is
This running time can be expressed as (an + b) for constants a and b. It is a linear function of n.
T(n) = O (n)
Worst Case:
In worst case, the array is reverse sorted. We must compare each element A[j] with each element in
the entire sorted sub-array A[1…j-1], so tj= j for j= 2, 3, …, n.
This worst-case running time can be expressed as an2 + bn + c for constants a, b, and c. This is a
quadratic function.
T(n) = O(n2)
Best Case (when numbers are already sorted) Time Complexity: T(n)= O(n)
Worst Case (when numbers are reversed sorted) Time Complexity: T(n)= O(n2)
#include<stdio.h>
#include<conio.h>
void main()
{
16 | P a g e
Shashank Yadav (IT-dept KIET)
int A[30], n, key, i, j;
clrscr();
printf("How many numbers do you want to sort by Insertion Sort:");
scanf("%d",&n);
printf("\n Enter %d numbers: ",n);
for(i=0;i<n;i++)
scanf("%d",&A[i]);
for(j=1;j<n;j++)
{
key=A[j];
i=j-1;
while(i>-1 && A[i]>key)
{
A[i+1]=A[i];
i=i-1;
}
A[i+1]=key;
}
printf("\n Sorted array is \n");
for(i=0;i<n;i++)
printf("%d ",A[i]);
getch();
}
1 9 9 9 9
2nd iteration 2 25 25 > 21 21 21 < 30 21 21 > 15 15
3 21 25 25 25
4 30 30 30 30
5 15 15 15 21
1 9 9 9
3rd iteration 2 15 15 15
3 25 25 < 30 25 25 > 21 21
4 30 30 30
5 21 21 25
1 9 9
4th iteration 2 15 15
3 21 21
4 30 30 > 25 25
5 25 30
17 | P a g e
Shashank Yadav (IT-dept KIET)
4.2.1 Algorithm
SELECTION-SORT(A, N)
Where A is an array of N numbers.
1. for I= 1 to N-1 do
a. for J= I+1 to N do
i. if(A[I] > A[J]) then
a) Exchange (A[I]A[J])
Outer loop will run N times and inner loop is dependent on the value of I, therefore it will run
(N+ (N-1)+ (N-2)+ . . . + 1)times i.e N(N+1)/2 times. And if block will run maximum
N(N-1)/2 times. So overall time complexity is dependent on inner loop.
T(n)= O(n2)
Selection sort time complexity does not depend on the order of elements in the list, therefore
in every case, the time complexity is O(n2).
#include<stdio.h>
#include<conio.h>
void main()
{
int A[50], n, i, j, c;
clrscr();
printf("Enter the number of elements of array ");
scanf("%d",&n);
printf("\nEnter %d numbers ",n);
for(i=0;i<n;i++)
scanf("%d",&A[i]);
for(i=0;i<n-1;i++)
{
for(j=i+1;j<n;j++)
{
if(A[i]> A[j])
{
c=A[i];
A[i]=A[j];
A[j]=c;
}
}
}
18 | P a g e
Shashank Yadav (IT-dept KIET)
for(i=0;i<n;i++)
printf("%d ",A[i]);
getch();
}
1 21 21 > 15 15 15 15
2nd iteration 2 15 21 21 < 25 21 21
3 25 25 25 25 > 9 9
4 9 9 9 25
5 30 30 30 30
1 15 15 < 21 15 15
3rd iteration 2 21 21 21 > 9 9
3 9 9 21
4 25 25 25
5 30 30 30
1 15 15 > 9 9
4th iteration 2 9 15
3 21 21
4 25 25
5 30 30
4.3.1 Algorithm:
BUBBLE-SORT(A, N)
Where A is an array of N numbers.
Outer loop in algorithm will run N times, and inner loop is dependent on the value of I.
Therefore it will run (N+ (N-1)+ (N-2)+ . . . + 1) times i.e. N(N+1)/2 times. So overall time
complexity of algorithm is dependent on the inner loop.
T(n)= O(n2)
19 | P a g e
Shashank Yadav (IT-dept KIET)
This Bubble sort algorithm does not depend on the order of the elements. Therefore, in all
case, time complexity will be O(n2).
#include<stdio.h>
#include<conio.h>
void main()
{
int A[50], n, i, j, c;
clrscr();
printf("Enter the number of elements of array ");
scanf("%d",&n);
printf("\nEnter %d numbers ",n);
for(i=0;i<n;i++)
scanf("%d",&A[i]);
for(i=0;i<n-1;i++)
{
for(j=0;j<n-i-1;j++)
{
if(A[j]> A[j+1])
{
c=A[j];
A[j]=A[j+1];
A[j+1]=c;
}
}
}
getch();
}
20 | P a g e
Shashank Yadav (IT-dept KIET)
4.3.5 Analysis of Modified-Bubble-Sort:
If data is already sorted, there will be no exchange after first iteration. Outer loop will run
only one times and inner loop will run only N times. Therefore the time complexity is
T(n)= O(n)
In this case outer loop will run N times and inner loop will run N(N+1)/2 times. Therefore
overall time complexity is
T(n)= O(n2)
i p, j r
a. 2 8 7 1 3 5 6 4 2< 4, i= i+1, Exchange A[i], A[j]
p, i J r j= j+1
b. 2 8 7 1 3 5 6 4 8> 4
p, i j r j=j+1
c. 2 8 7 1 3 5 6 4 7> 4
p, i J r j=j+1
d. 2 8 7 1 3 5 6 4 1< 4 i= i+1, Exchange A[i], A[j]
p I j r j=j+1
e. 2 1 7 8 3 5 6 4 3< 4 i= i+1, Exchange A[i], A[j]
p i j r j=j+1
f. 2 1 3 8 7 5 6 4 5> 4
p i j r j=j+1
g. 2 1 3 8 7 5 6 4 6> 4
p i r j=j+1
h. 2 1 3 8 7 5 6 4 Exchange A[i+1], A[r]
p i r
i. 2 1 3 4 7 5 6 8
4.4.1 Algorithm:
QUICK-SORT(A, P, R)
Where A is a array of with lower bound P and upper bound R.
1. If (P< R) then
a. Q= PARTITION(A, P, R)
b. QUICK-SORT(A, P, Q-1)
c. QUICK-SORT(A, Q+1, R)
PARTITION(A, P, R)
1. X= A[R]
2. I= P-1
3. For (J= P to R-1) do
a. If (A[J] <= X) then
21 | P a g e
Shashank Yadav (IT-dept KIET)
i. I= I+1
ii. Exchange (A[I], A[J])
4. Exchange (A[I+1], A[R])
5. Return (I+1)
The running time of quick-sort depends on whether the partition of array is balanced or
unbalanced.
T(n)<= 2T(n/2) + cn
Worst Case (when Partition produces only one sub-problem with n-1 elements):
T(n)= T(n-1) + cn
#include<stdio.h>
#include<conio.h>
void Quicksort(int*,int,int);
int Partition(int*,int,int);
void main()
{
int A[50], n, i;
clrscr();
printf("Enter number of elements in the array: ");
scanf("%d",&n);
printf("\nEnter %d numbers\n",n);
for(i=0;i<n;i++)
scanf("%d",&A[i]);
Quicksort(A,0,n-1);
printf("\nSorted array is \n");
for(i=0;i<n;i++)
printf("%d ",A[i]);
getch();
}
22 | P a g e
Shashank Yadav (IT-dept KIET)
if(p<r)
{
q=Partition(A,p,r);
Quicksort(A,p,q-1);
Quicksort(A,q+1,r);
}
}
To improve complexity in worst case of Quick Sort, a Randomized Quick Sort is there. By
this algorithm worst case can be converted into average case and average case complexity is
O(n lg n).
RANDOMIZED-PARTITION(A, p, r)
1. i RANDOM(p, r)
2. Exchange(A[r] A[i])
3. return PARTITION(A, p, r)
RANDOMIZED-QUICKSORT(A, p, r)
1. if (p<r)
2. then q RANDOMIZED-PARTITION(A, p, r)
3. RANDOMIZED-QUICKSORT(A, p, q-1)
4. RANDOMIZED-QUICKSORT(A, q+1, r)
23 | P a g e
Shashank Yadav (IT-dept KIET)
4.5 Heapsort:
4.5.1 Max-Heap:
A max-heap is a complete binary tree in which the value in each internal node is
greater than or equal to the values in the children of that node.
The position of left child of any node(i) is (2*i).
The position of right child of any node(i) is (2*i +1).
The position of parent of any node (i) is floor value of (i/2).
24 | P a g e
Shashank Yadav (IT-dept KIET)
25 | P a g e
Shashank Yadav (IT-dept KIET)
4.5.2.2 Heap sort process example.
4.5.3 Algorithm:
MAX-HEAPIFY(A, I)
1. L= left(I)
2. R= right(I)
3. If(L<= heapsize[A] and A[L] > A[I])
a. Largest= L
4. Else
a. Largest= I
5. If(R<= heapsize[A] and A[R] > A[Largest])
a. Largest= R
6. If(Largest != I)
26 | P a g e
Shashank Yadav (IT-dept KIET)
a. Exchange (A[I], A[Largest])
b. MAX-HEAPIFY(A, Largest)
BUILD-MAX-HEAP(A)
1. Heapsize[A]= Length[A]
2. For(I= Length[A]/ 2 down to 1) do
a. MAX-HEAPIFY(A, I)
HEAPSORT(A)
1. BUILD-MAX-HEAP(A)
2. For( I= Length[A] down to 2) do
a. Exchange (A[1], A[I])
b. Heapsize[A]= Heapsize[A]- 1
c. MAX-HEAPIFY(A, 1)
Time complexity of Heap sort depends on the time complexity of BUILD-MAX-HEAP and
MAX-HEAPIFY.
The running time of Max-Heapify depends on the height of the tree. The height of
complete binary tree is log2n, therefore time complexity of this algorithm is O(lg n).
Initially we apply Max-Heapify algorithm on binary tree of n nodes, after then we go
to left or right sub-tree. The left or right sub-tree can contain at most 2n/3 nodes. So
we apply this algorithm on at most 2n/3 nodes only.
T(n) ≤ T(2n/3) + θ(1)
Therefore T(n)= O(lg n).
27 | P a g e
Shashank Yadav (IT-dept KIET)
Time complexity of Heap Sort:
The HeapSort procedure takes time O(n lg n), since the call to Build-Max-Heap takes
time O(n) and each of the n-1 calls to Max-Heapify takes time O(lg n).
T(n) = O(n lg n)
int heapsize;
28 | P a g e
Shashank Yadav (IT-dept KIET)
int i;
heapsize=n;
for(i=(n/2)-1; i>=0; i--)
Max_Heapify(A, i);
void main()
{
int A[50], n, i;
clrscr();
printf("Enter the number of elements in the array: ");
scanf("%d",&n);
printf("\nEnter %d numbers\n",n);
for(i=0;i<n;i++)
scanf("%d",&A[i]);
Heapsort(A,n);
29 | P a g e
Shashank Yadav (IT-dept KIET)
4.6 Merge Sort:
4.6.1 Algorithm:
MERGE-SORT(A, p, r)
Where A is an array with P lower bound and R upper bound.
1. if p<r then
a. q = (p + r)/2
b. MERGE-SORT(A, p, q)
c. MERGE-SORT(A, q + 1, r)
d. MERGE(A, p, q, r)
MERGE(A, p, q, r)
1. N1 = q- p + 1
2. N2 = r- q
3. create arrays L[1 . . N1] and R[1 . . N2]
4. for i = 1 to N1 do
a. L[i] = A[p+ i − 1]
5. for j = 1 to N2 do
a. R[j] = A[q+j]
6. i = 1
7. j = 1
8. for k = p to r do
a. if(i<= N1 and L[i] <= R[j] ) or j >N2) then
i. A[k] = L[i]
30 | P a g e
Shashank Yadav (IT-dept KIET)
ii. i= i + 1
b. else if((j<= N2 and R[j]< L[i]) or i > N1) then
i. A[k] = R[j]
ii. j= j + 1
{
( )
{
( )
T(n)= θ(n lg n)
#include<stdio.h>
#include<conio.h>
n1= q-p+1;
n2= r-q;
L=(int*)malloc(n1*sizeof(int));
R=(int*)malloc(n2*sizeof(int));
for(i=0;i<n1;i++)
L[i]= A[p+i];
for(j=0;j<n2;j++)
R[j]= A[q+j+1];
i=0;
j=0;
for(k=p;k<=r;k++)
31 | P a g e
Shashank Yadav (IT-dept KIET)
{
if((i<n1 && L[i]<= R[j]) || j= =n2)
{
A[k]= L[i];
i++;
}
else if((j<n2 && R[j]< L[i]) || i= =n1)
{
A[k]= R[j];
j++;
}
}
}
void main()
{
int A[50], n, i;
clrscr();
printf("Enter number of elements in the array: ");
scanf("%d",&n);
MergeSort(A, 0, n-1);
32 | P a g e
Shashank Yadav (IT-dept KIET)
4.7 Shell Sort:
Shell sort is named by Donald Shell who first published an account of this sorting technique.
Shell sort is also called the diminishing increment sort, and is a variation on the basic
insertion sort algorithm.
In insertion sort, comparison is always done between adjacent elements (Ai, Ai+1), i<n, and a
swap occurs if Ai+1 < Ai.
The variation used in shell sort, is to avoid comparing adjacent elements except the last step
of the algorithm. So, the last step of shell sort is effectively the insertion sort algorithm, but
by the time we reach the last step, the list is either already sorted, or mostly sorted, so
insertion sort runs in almost linear time.
The running time of shell sort is highly dependent on the selection of an increment sequence
hi, hi-1, . . . , h1, that effectively partition the original list into a set of sub-lists, which are
sorted after one pass through the entire list.
By Knuth
hi = 1
hi = hi * 3 + 1 calculate this equation until hi > n
4.7.1
Example: Illustrate the operation of shell sort on the array A= {A, S, O, R, T, I, N, G, E, X, A,
M, P, L, E}
Step:1
1. since h<=n, h= 3 * 1 + 1= 4
2. since h<=n, h= 3 * 4 + 1= 13
3. since h<=n, h= 3 * 13 + 1= 40
4. since h> n, stop
1. h=(40-1)/3= 13
2. h=(13-1)/3= 4
3. h=(4-1)/3= 1
Step 2:
When increment h is 13, the first element will be compared with (h+1= 14th) element and second
element will be compared with 15th element.
1 2 3 4 5 6 7 8 9 10 11 12 13
A S O R T I N G E X A M P
L E
14 15
33 | P a g e
Shashank Yadav (IT-dept KIET)
Compare {A[1], A[14]}, {A[2], A[15]} and exchange if first element is greater than second.
1 2 3 4 5 6 7 8 9 10 11 12 13
A E O R T I N G E X A M P
L S
14 15
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
A E O R T I N G E X A M P L S
Step 3:
Now h is 4, then (1st, 5th, 9th, 13th) elements will be compared for sorting, (2nd, 6th, 10th, 14th) elements
will be compared for sorting and so on.
1 2 3 4
A E O R
5 6 7 8
T I N G
9 10 11 12
E X A M
13 14 15
P L S
1 2 3 4
A E A G
5 6 7 8
E I N M
9 10 11 12
P L O R
13 14 15
T X S
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
A E A G E I N M P L O R T X S
Step 4:
34 | P a g e
Shashank Yadav (IT-dept KIET)
When h is 1, then there is only one list. The first element will be compared with 2 nd, 2nd element will
be compared with 3rd and so on.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
A A E E G I L M N O P R S T X
4.7.2 Algorithm:
Shell-Sort(A, n)
Where A is an array of n elements.
1. h=1
2. while (h<=n)
3. h=3*h+1
4. do
5. h=(h-1)/3
6. c=1
7. for(j=h+c; j<=n; j=j+h)
8. key=A[j]
9. i=j-h
10. while(i>0 and A[i] > key)
11. A[i+h]=A[i]
12. i=i-h
13. A[i+h]= key
14. if((j+h)> n and c<h)
15. c= c+1
16. j= h+c
17. while(h!=1)
Shell sort is a rather curious algorithm, quite different from other fast sorting algorithms. It is actually
so different that it even isn’t an O(n lg n) algorithm like the others, but instead it’s something between
O(n lg2 n) and O(n1.5) depending on implementation details.
#include<stdio.h>
#include<conio.h>
#include<alloc.h>
do
{
h=(h-1)/3;
35 | P a g e
Shashank Yadav (IT-dept KIET)
c=1;
for(j=h+c;j<=n;j=j+h)
{
key=A[j];
i=j-h;
while(i>0 && A[i]>key)
{
A[i+h]=A[i];
i=i-h;
}
A[i+h]=key;
if((j+h)>n && c<h)
{
c++;
j=h+c;
}
}
}while(h!=1);
}
void main()
{
int *A, n, i;
clrscr();
printf("\nEnter number of elements in array: ");
scanf("%d",&n);
A=(int*)malloc((n+1)*sizeof(int));
printf("\nEnter %d numbers\n",n);
for(i=1;i<=n;i++)
scanf("%d",&A[i]);
ShellSort(A,n);
free(A);
getch();
}
36 | P a g e
Shashank Yadav (IT-dept KIET)