SORTING TECHNIQUES

https://visualgo.net/en/sorting
 Syllabus
Sorting: Insertion Sort, Selection, Bubble Sort, Quick Sort,
Merge Sort, Heap Sort and Radix Sort.
 Introduction to Sorting
Sorting is nothing but arranging the data in ascending or
descending order. The term sorting came into picture, as
humans realized the importance of searching quickly.
There are so many things in our real life that we need to
search for, like a particular record in database, roll numbers
in merit list, a particular telephone number in telephone
directory, a particular page in a book etc. All this would
have been a mess if the data was kept unordered and
unsorted, but fortunately the concept of sorting came into
existence, making it easier for everyone to arrange data in
an order, hence making it easier to search.
Sorting arranges data in a sequence which makes searching
easier.
 Sorting Efficiency
Since the beginning of the programming age, computer
scientists have been working on solving the problem of
sorting by coming up with various different algorithms to
sort data.
The two main criteria's to judge which algorithm is better
than the other have been:
Time taken to sort the given data.
Memory Space required to do so.
 Types of Sorting Algorithms

•If the data sorting process If the entire data won’t fit in the
RAM, rest of the data is normally
takes place entirely within the
held on some larger, but slower
RAM of a computer, it’s called
medium, like a hard disk then it is
internal sorting.
called external sorting.
Ex- Bubble Sort, Insertion
Ex- Merge sort, Tape sort etc.
Sort, Quick Sort etc.
Stable Sorting Algorithms: Bubble Sort, Insertion Sort, Merge
Sort, Count Sort
Unstable Sorting Algorithms: Quick sort, Heap sort, Selection sort
In-place algorithms:
An in-place algorithm transforms the input without using any
extra memory.
As the algorithm executes, the input is usually overwritten by
the output, and no additional space is needed for this
operation.
Ex- insertion sort, selection sort, quick sort, bubble sort, heap
sort.
Out-of-place algorithms:
An algorithm that is not in-place is called a not-in-place or
out-of-place algorithm.
Unlike an in-place algorithm, the extra space used by an out-
of-place algorithm depends on the input size.
Ex- merge sort
 Different Sorting Algorithms
There are many different techniques available for sorting,
differentiated by their efficiency and space requirements.
Following are some sorting techniques which we will be
covering in next few tutorials.
Bubble Sort
Insertion Sort
Selection Sort
Quick Sort
Merge Sort
Heap Sort
Radix Sort
Counting Sort
 Bubble Sort Algorithm
 Bubble Sort is a simple algorithm which is used to sort a given set
of n elements provided in form of an array with n number of elements.
Bubble Sort compares all the element one by one and sort them based on
their values.
 If the given array has to be sorted in ascending order, then bubble sort will
start by comparing the first element of the array with the second element,
if the first element is greater than the second element, it will swap both the
elements, and then move on to compare the second and the third element,
and so on.
 If we have total n elements, then we need to repeat this process for n-
1 times.
 It is known as bubble sort, because with every complete iteration the
largest element in the given array, bubbles up towards the last place or the
highest index, just like a water bubble rises up to the water surface.
 Sorting takes place by stepping through all the elements one-by-one and
comparing it with the adjacent element and swapping them if required.
 Example
Let's consider an array with values {5, 1, 6, 2, 4, 3}
 Complexity Analysis of Bubble Sort
 In Bubble Sort, n-1 comparisons will be done in the 1st pass, n-2 in 2nd
pass, n-3 in 3rd pass and so on. So the total number of comparisons will be,
(n-1) + (n-2) + (n-3) + ..... + 3 + 2 + 1
Sum = n(n-1)/2 i.e O(n2)
 Hence the time complexity of Bubble Sort is O(n2).
 The space complexity for Bubble Sort is O(1), because only a single
additional memory space is required i.e. for temp variable.
 Also, the best case time complexity will be O(n), it is when the list is
already sorted.
 Following are the Time and Space complexity for the Bubble Sort
algorithm.
 Worst Case Time Complexity [ Big-O ]: O(n2)
 Best Case Time Complexity [Big-omega]: Ω(n)
 Average Time Complexity [Big-theta]: θ (n2)
 Space Complexity: O(1)
 Advantage
The main advantage of Bubble Sort is the simplicity of
the algorithm.
 UGC NET CS 2015 Jun - II
Assume that we use Bubble Sort to sort n distinct elements
in ascending order. When does the best case of Bubble
Sort occur?
When elements are sorted in ascending order
When elements are sorted in descending order
When elements are not sorted by any order
There is no best case for Bubble Sort. It always takes
O(n*n) time
 Insertion Sort Algorithm
Insertion sort works as sorting a deck of cards. It starts from the
index 1(not 0), and each index starting from index 1 is like a new
card, that you have to place at the right position in the sorted sub array
on the left.
Following are some of the important characteristics of Insertion Sort:
 It is efficient for smaller data sets, but very inefficient for larger lists.
 Insertion Sort is adaptive, that means it reduces its total number of
steps if a partially sorted array is provided as input, making it
efficient.
 It is better than Selection Sort and Bubble Sort algorithms.
 Its space complexity is less. Like bubble Sort, insertion sort also
requires a single additional memory space.
 It is a stable sorting technique, as it does not change the relative order
of elements which are equal.
 Pseudo code
INSERTION-SORT(A)
for i = 1 to n-1
key ← A [i]
j←i–1
while j > = 0 and A[j] > key
A[j+1] ← A[j]
j←j–1
End while
A[j+1] ← key
End for
 Complexity Analysis of Insertion Sort
 As we mentioned above that insertion sort is an efficient
sorting algorithm, as it does not run on preset conditions
using for loops, but instead it uses one while loop, which
avoids extra steps once the array gets sorted.
 Even though insertion sort is efficient, still, if we provide an
already sorted array to the insertion sort algorithm, it will still
execute the outer for loop, thereby requiring n steps to sort an
already sorted array of n elements, which makes its best case
time complexity a linear function of n.
 Worst Case Time Complexity [ Big-O ]: O(n2)
 Best Case Time Complexity [Big-omega]: Ω(n)
 Average Time Complexity [Big-theta]: θ (n2)
 Space Complexity: O(1)
 GATE-CS-2003
In a permutation a1.....an of n distinct integers, an inversion
is a pair (ai, aj) such that i < j and ai > aj. What would be
the worst case time complexity of the Insertion Sort
algorithm, if the inputs are restricted to permutations of
1.....n with at most n inversions?
Θ (n2)
Θ (n log n)
Θ (n1.5)
Θ (n)
 GATE CS 1999
Consider the array A[]= {6,4,8,1,3} apply the insertion
sort to sort the array . Consider the cost associated with
each sort is 25 rupees , what is the total cost of the
insertion sort when element 1 reaches the first position of
the array?
50
25
75
100
 Shell Sort
It is a sorting algorithm that is an extended version of
insertion sort. Shell sort has improved the average time
complexity of insertion sort. In 1959, Donald Shell
published the first version of the shell sort algorithm.
As similar to insertion sort, it is a comparison-based and in-
place sorting algorithm. Shell sort is efficient for medium-
sized data sets.
This algorithm first sorts the elements that are far away from
each other, then it subsequently reduces the gap between
them. This gap is called as interval. Various intervals have
been proposed:
Donald Shell: H1=n/2, Hk+1=Hk/2, produces series n/2, n/4,
n/8…..1
Hibbard: Hi=2i-1, produces series 1, 3, 7, 15….
Algorithm
1. h=1
2. while(h<=n)
a. h=3*h+1
3. do
a. h=(h-1)/3
b. for j=h to n-1
a. key=A[j]
b. i=j-h
c. while i>=0 && A[i]>key
a. A[i+h]=A[i]
b. i=i-h
d. A[i+h]=key
4. while(h!=1)
At the interval of 4, the sublists are {33, 12}, {31, 17}, {40, 25}, {8, 42}.
In this step (interval=1), shell sort uses insertion sort to sort the array
elements.
Shell sort complexity
 Selection Sort
Selection sort is conceptually the most simplest sorting
algorithm. This algorithm will first find the smallest element
in the array and swap it with the element in the first position,
then it will find the second smallest element and swap it
with the element in the second position, and it will keep on
doing this until the entire array is sorted.
It is called selection sort because it repeatedly selects the
next-smallest element and swaps it into the right place.
 Example
Let's consider an array with values {3, 6, 1, 8, 4, 5}
 Complexity Analysis of Selection Sort
Selection Sort requires two nested for loops to complete
itself.
Hence for a given input size of n, following will be the time
and space complexity for selection sort algorithm:
Worst Case Time Complexity [ Big-O ]: O(n2)
Best Case Time Complexity [Big-omega]: Ω (n2)
Average Time Complexity [Big-theta]: θ(n2)
Space Complexity: O(1)
 ISRO CS 2008
How many comparisons are needed to sort an array of
length 5 if a straight selection sort is used and array is
already in the opposite order?
1
5
10
15
 ISRO CS 2007
Which of the following algorithm design technique is
used in Selection sort?
Greedy method
Backtracking
Dynamic programming
Divide and Conquer

Greedy method
 Quick Sort Algorithm
Quick Sort is based on the concept of Divide and Conquer
approach. In quick sort all the heavy lifting(major work) is
done while dividing the array into two sub-arrays. In case of
quick sort, the combine step does absolutely nothing.
It is also called partition-exchange sort. This algorithm
divides the list into three main parts:
Elements less than the Pivot element
Pivot element(Central element)
Elements greater than the pivot element
 Pivot element
 Pivot element can be any element from the array, it can be the
first element, the last element or any random element. In this
tutorial, we will take the rightmost element or the last element
as pivot.
 For example: In the array {52, 37, 63, 14, 17, 8, 6, 25}, we
take 25 as pivot. So after the first pass, the list will be changed
like this.
{6 8 17 14 25 63 37 52}
 Hence after the first pass, pivot will be set at its position, with
all the elements smaller to it on its left and all the
elements larger than to its right. Now 6 8 17 14 and 63 37
52 are considered as two separate sub-arrays, and same
recursive logic will be applied on them, and we will keep doing
this until the complete array is sorted.
 How Quick Sorting Works?
Following are the steps involved in quick sort algorithm:
After selecting an element as pivot, which is the last index of
the array in our case, we divide the array for the first time.
In quick sort, we call this partitioning. It is not simple breaking
down of array into 2 sub-arrays, but in case of partitioning, the
array elements are so positioned that all the elements smaller
than the pivot will be on the left side of the pivot and all the
elements greater than the pivot will be on the right side of it.
And the pivot element will be at its final sorted position.
The elements to the left and right, may not be sorted.
Then we pick sub-arrays, elements on the left of pivot and
elements on the right of pivot, and we perform partitioning on
them by choosing a pivot in the sub-arrays.
Let's consider an array with values {9, 7, 5, 11, 12, 2, 14, 3,
10, 6}
 Complexity Analysis of Quick Sort
For an array, in which partitioning leads to unbalanced sub-
arrays, to an extent where on the left side there are no elements,
with all the elements greater than the pivot, hence on the right
side.
And if keep on getting unbalanced sub-arrays, then the running
time is the worst case, which is O(n2)
Where as if partitioning leads to almost equal sub-arrays, then
the running time is the best, with time complexity as O(n*log n).
Worst Case Time Complexity [ Big-O ]: O(n2)
Best Case Time Complexity [Big-omega]: Ω(n*log n)
Average Time Complexity [Big-theta]: θ(n*log n)
 GATE CS 2008
Consider the Quick-sort algorithm. Suppose there is a
procedure for finding a pivot element which splits the list
into two sub-lists each of which contains at least one-fifth
of the elements. Let T(n) be the number of comparisons
required to sort n elements. Then
T(n) <= 2T(n/5) + n
T(n) <= T(n/5) + T(4n/5) + n
T(n) <= 2T(4n/5) + n
T(n) <= 2T(n/2) + n
 GATE-CS-2014-(Set-1)
Let P be a QuickSort Program to sort numbers in ascending
order using the first element as pivot. Let t1 and t2 be the
number of comparisons made by P for the inputs {1, 2, 3,
4, 5} and {4, 1, 5, 3, 2} respectively. Which one of the
following holds?
t1 = 5
t1 < t2
T1 = t2
t1 > t2
 GATE CS 1996 and ISRO CS
2017
Quicksort is run on two inputs shown below to sort in
ascending order taking first element as pivot,
(i) 1, 2, 3,......., n
(ii) n, n-1, n-2,......, 2, 1
Let C1 and C2 be the number of comparisons made for the
inputs (i) and (ii) respectively. Then,
 C1 < C2
 C1 = C2
 C1 > C2
 We cannot say anything for arbitrary n
 GATE-CS-2003
You have an array of n elements. Suppose you
implement quick sort by always choosing the central
element of the array as the pivot. Then the tightest upper
bound for the worst case performance is:
O(n2)
O(nLogn)
Θ(nLogn)
O(n3)
 Merge Sort Algorithm
Merge Sort follows the rule of Divide and Conquer to sort
a given set of numbers/elements, recursively, hence
consuming less time.
Merge sort runs in O(n*log n) time in all the cases.
Before jumping on to, how merge sort works and it's
implementation, first lets understand what is the rule
of Divide and Conquer?
 Divide and Conquer
 If we can break a single big problem into smaller sub-problems,
solve the smaller sub-problems and combine their solutions to find
the solution for the original big problem, it becomes easier to solve
the whole problem.
 In Merge Sort, the given unsorted array with n elements, is
divided into n sub-arrays, each having one element, because a
single element is always sorted in itself. Then, it repeatedly merges
these sub-arrays, to produce new sorted sub-arrays, and in the end,
one complete sorted array is produced.
 The concept of Divide and Conquer involves three steps:
 Divide the problem into multiple small problems.
 Conquer the subproblems by solving them. The idea is to break down
the problem into atomic subproblems, where they are actually solved.
 Combine the solutions of the subproblems to find the solution of the
actual problem.
Pseudo Code
 Complexity Analysis of Merge Sort
 Merge Sort is quite fast, and has a time complexity of O(n*log n). It is also a stable
sort, which means the "equal" elements are ordered in the same order in the sorted list.
 In this section we will understand why the running time for merge sort is O(n*log n).
 As we have already learned in Binary Search that whenever we divide a number into
half in every step, it can be represented using a logarithmic function, which is log
n and the number of steps can be represented by log n + 1(at most)
 Also, we perform a single step operation to find out the middle of any subarray,
i.e. O(1).
 And to merge the sub-arrays, made by dividing the original array of n elements, a
running time of O(n) will be required.
 Hence the total time for merge-Sort function will become n(log n + 1), which gives us
a time complexity of O(n*log n).
 Worst Case Time Complexity [ Big-O ]: O(n*log n)
 Best Case Time Complexity [Big-omega]: Ω(n*log n)
 Average Time Complexity [Big-theta]: θ(n*log n)
 Space Complexity: O(n)
 It is the best Sorting technique used for sorting Linked Lists.
 GATE-CS-2015 (Set 3)
Assume that a merge sort algorithm in the worst case takes
30 seconds for an input of size 64. Which of the
following most closely approximates the maximum input
size of a problem that can be solved in 6 minutes?
256
512
1024
2048
 GATE-CS-2016 (Set 1)
The worst case running times of Insertion sort, Merge sort
and Quick sort, respectively, are:
Θ(n log n), Θ(n log n) and Θ(n2)
Θ(n2), Θ(n2) and Θ(n Log n)
Θ(n2), Θ(n log n) and Θ(n log n)
Θ(n2), Θ(n log n) and Θ(n2)
 ISRO CS 2011
Which of the following algorithm design technique is used
in merge sort?
Greedy method
Backtracking
Dynamic programming
Divide and Conquer
 ISRO CS 2007
The average case and worst case complexities for Merge
sort algorithm are:
O ( n2 ), O ( n2 )
O ( n2 ), O ( n log2n )
O ( n log2n ), O ( n2)
O ( n log2n ), O ( n log2n )
 Radix sort
Radix sort is a non-comparative integer sorting algorithm
that sorts data with integer keys by grouping keys by the
individual digits which share the same significant position
and value
sort by the least significant digit first (counting sort)
Numbers with the same digit go to same bin
reorder all the numbers: the numbers in bin 0 precede the
numbers in bin 1, which precede the numbers in bin 2, and so
on
sort by the next least significant digit
continue this process until the numbers have been sorted on all
k digits
 Algorithm
Every integer can be represented by at most k digits
d1,d2…dk, where di are digits in base r
d1: most significant digit and dk: least significant digit
Example: 275, 087, 426, 061, 509, 170, 677, 503
 Complexity Analysis
k passes over the numbers (i.e. k counting sorts, with range
being 0..r)
each pass takes O(N+r)
total: O(Nk+rk)
r and k are constants: O(N)
 UGC NET CS 2015 Jun - II
Which of the following algorithms sort n integers, having
the range 0 to (n2 - 1), in ascending order in O(n) time ?
Selection Sort
Bubble Sort
Radix Sort
Insertion Sort
 GATE 2017 Mock
Which of the below given sorting techniques has highest
best-case runtime complexity.
Quick sort
Selection Sort
Insertion Sort
Bubble Sort
 GATE CS 1999
A sorting technique is called stable if:
It takes O(nlog n)time
It maintains the relative order of occurrence of non-distinct
elements
It uses divide and conquer paradigm
It takes O(n) space
 UGC NET CS 2017 Jan - II
Which of the following is true for computation time in
insertion, deletion and finding maximum and minimum
element in a sorted array?
 Insertion – 0(1), Deletion – 0(1), Maximum – 0(1), Minimum – 0(l)
 Insertion – 0(1), Deletion – 0(1), Maximum – 0(n), Minimum – 0(n)
 Insertion – 0(n), Deletion – 0(n), Maximum – 0(1), Minimum – 0(1)
 Insertion – 0(n), Deletion – 0(n), Maximum – 0(n), Minimum – 0(n)
 ISRO CS 2014
Consider the following sorting algorithms.
I. Quicksort
II. Heapsort
III. Mergesort
Which of them perform in least time in the worst case?
I and II only
II and III only
III only
I, II and III
 Heap Sort Algorithm
Heap Sort is one of the best sorting methods being in-
place and with no quadratic worst-case running time.
Heap sort involves building a Heap data structure from
the given array and then utilizing the Heap to sort the
array.
 What is a Heap ?
Heap is a special tree-based data structure, that satisfies
the following special heap properties:
Shape Property: Heap data structure is always a
Complete Binary Tree, which means all levels of the tree
are fully filled.
Heap Property: All nodes are either greater than or
equal to or less than or equal to each of its children. If
the parent nodes are greater than their child nodes, heap is
called a Max-Heap, and if the parent nodes are smaller
than their child nodes, heap is called Min-Heap.
 Working of Heap-Sort
 Heap sort algorithm is divided into two basic parts:
 Creating a Heap of the unsorted list/array.
 Then a sorted array is created by repeatedly removing the
largest/smallest element from the heap, and inserting it into the
array. The heap is reconstructed after each removal.
 Initially on receiving an unsorted list, the first step in heap sort is
to create a Heap data structure(Max-Heap or Min-Heap). Once
heap is built, the first element of the Heap is either largest or
smallest(depending upon Max-Heap or Min-Heap), so we put the
first element of the heap in our array. Then we again make heap
using the remaining elements, to again pick the first element of
the heap and put it into the array. We keep on doing the same
repeatedly untill we have the complete sorted list in our array.
 In the below algorithm, initially heapsort() function is called,
which calls heapify() to build the heap.
Example
 BUILD MAX HEAP
Build_Max_Heap(A)
1. heap_size(A) = length(A)
2. for i = length(A)/2 to 1
3. Max_Heapify(A,i)
 MAX HEAPIFY
Max_Heapify(A,i)
1. L = left(i)
2. R = right(i)
3. if L <= heap_size[A] and A[L] > A[i]
largest = L
4. else
largest = i
5. if R <= heap_size[A] and A[R] > A[largest]
largest = R
6. if largest != i
swap A[i] with A[largest]
Max_Heapify(A,largest)
 HEAP SORT
Heap_Sort(A)
1. Build_Max_Heap(A)
2. for i = length(A) to 2
3. swap A[1] with A[i]
4. heap_size[A] = heap_size[A] - 1
5. MaxHeapify(A,1)
 Complexity Analysis of Heap Sort
Worst Case Time Complexity: O(n*log n)
Best Case Time Complexity: O(n*log n)
Average Time Complexity: O(n*log n)
Space Complexity : O(1)
Heap sort is not a Stable sort, and requires a constant space
for sorting a list.
Heap Sort is very fast and is widely used for sorting
 ISRO 2016
Suppose we are sorting an array of eight integers using
heapsort, and we have just finished some heapify (either
maxheapify or minheapify) operations. The array now
looks like this: 16 14 15 10 12 27 28 How many heapify
operations have been performed on root of heap?
1
2
3 or 4
5 or 6
 GATE-CS-2014-(Set-2)
A priority queue is implemented as a Max-Heap. Initially,
it has 5 elements. The level-order traversal of the heap is:
10, 8, 5, 3, 2. Two new elements 1 and 7 are inserted into
the heap in that order. The level-order traversal of the
heap after the insertion of the elements is:
10, 8, 7, 3, 2, 1, 5
10, 8, 7, 2, 3, 1, 5
10, 8, 7, 1, 2, 3, 5
10, 8, 7, 5, 3, 2, 1
 GATE-CS-2001
Consider any array representation of an n element binary
heap where the elements are stored from index 1 to index
n of the array. For the element stored at index i of the
array (i <= n), the index of the parent is:
i – 1
floor(i/2)
ceiling(i/2)
(i+1)/2
 GATE-CS-2015 (Set 1)
Consider a max heap, represented by the array: 40, 30, 20,
10, 15, 16, 17, 8, 4. Now consider that a value 35 is
inserted into this heap. After insertion, the new heap is:
40, 30, 20, 10, 15, 16, 17, 8, 4, 35
40, 35, 20, 10, 30, 16, 17, 8, 4, 15
40, 30, 20, 10, 35, 16, 17, 8, 4, 15
40, 35, 20, 10, 15, 16, 17, 8, 4, 30
 GATE-CS-2015 (Set 3)
Consider the following array of elements. 〈89, 19, 50, 17,
12, 15, 2, 5, 7, 11, 6, 9, 100〉. The minimum number of
interchanges needed to convert it into a max-heap is:
2
3
4
5
 GATE-CS-2016 (Set 2)
A complete binary min-heap is made by including each
integer in [1, 1023] exactly once. The depth of a node in
the heap is the length of the path from the root of the
heap to that node. Thus, the root is at depth 0. The
maximum depth at which integer 9 can appear is:
6
7
8
9
 AKTU QUESTIONS
Write an algorithm for Heap Sort. Use Heap sort
algorithm, sort the following sequence: 18, 25, 45,
34, 36, 51, 43, and 24. [2022]
Examine the minimum number of interchanges
needed to convert the array 90, 20, 41,18, 13, 11, 3, 6,
8,12, 7, 71, 99 into a maximum heap. [2020]
Use the heap sort algorithm to sort the following
elements in ascending order. 13, 16, 10, 11, 4, 12, 6,
7. What is the time and space complexity of heap
sort. [2018]