Data Structures and Algorithms (CS-2001)

KALINGA INSTITUTE OF INDUSTRIAL

TECHNOLOGY

School of Computer Engineering

4 Credit Lecture Note

 Chapter Contents
2

Sr Major and Detailed Coverage Area Hrs

#

7 Sorting 4
Bubble Sort, Insertion Sort, Selection Sort,
Quick Sort, Merge Sort, Heap Sort, Radix
Sort

School of Computer Engineering

 Introduction
3

Sorting refers to arranging data in a particular format. Sorting algorithm specifies the way to arrange data in a particular
order. Most common orders are in numerical or lexicographical order. The importance of sorting lies in the fact that data
searching can be optimized to a very high level, if data is stored in a sorted manner. Sorting is also used to represent data in
more readable formats. Following are some of the examples of sorting in real-life scenarios:
 Telephone Directory − The telephone directory stores the telephone numbers of people sorted by their names, so
that the names can be searched easily.
 Dictionary − The dictionary stores words in an alphabetical order so that searching of any word becomes easy.
Important Terms
 Increasing order − A sequence of values is said to be in increasing order, if the successive element is greater than the
previous one. For example, 1, 3, 4, 6, 8, 9 are in increasing order, as every next element is greater than the previous
element.
 Decreasing Order − A sequence of values is said to be in decreasing order, if the successive element is less than the
current one. For example, 9, 8, 6, 4, 3, 1 are in decreasing order, as every next element is less than the previous element.
 Non-increasing order − A sequence of values is said to be in non-increasing order, if the successive element is less
than or equal to its previous element in the sequence. This order occurs when the sequence contains duplicate values.
For example, 9, 8, 6, 3, 3, 1 are in non-increasing order, as every next element is less than or equal to (in case of 3) but
not greater than any previous element.
 Non-decreasing Order − A sequence of values is said to be in non-decreasing order, if the successive element is
greater than or equal to its previous element in the sequence. This order occurs when the sequence contains duplicate
values. For example, 1, 3, 3, 6, 8, 9 are in non-decreasing order, as every next element is greater than or equal to (in case
of 3) but not less than the previous one.

School of Computer Engineering

 Introduction cont…
4

n-place Sorting and Not-in-place Sorting

Sorting algorithms may require some extra space for comparison and temporary storage of few data
elements. These algorithms do not require any extra space and sorting is said to happen in-place, or
for example, within the array itself. This is called in-place sorting. Bubble sort is an example of in-
place sorting. However, in some sorting algorithms, the program requires space which is more than
or equal to the elements being sorted. Sorting which uses equal or more space is called not-in-place
sorting. Merge-sort is an example of not-in-place sorting.
Sorting Algorithm

Sr# Traditional Sorting Sorting based on Divide & Sorting based on Trees
Conquer
1 Bubble Sort Merge sort Heap Sort
2 Insertion sort Quick sort
3 Selection sort
4 Radix Sort

School of Computer Engineering

Bubble Sort
5

Bubble sort is a simple sorting algorithm. This sorting algorithm is comparison based algorithm in which
each pair of adjacent elements is compared and elements are swapped if they are not in order. This
algorithm is not suitable for large data sets as its average and worst case complexity is of O(n2) where n are
no. of items.

How bubble sort works?

Take an unsorted array

Bubble sort starts with very first two elements, comparing them to check which one
is greater.
In this case, 33 is greater than 14, so it is already in sorted locations. Next, it
compares 27 with 33.

We find that 27 is smaller than 33 and these two values must be swapped.

Post to swapping, the new array should looks like

School of Computer Engineering

Bubble Sort cont…
6

Next it compares 35 with 33 and both are already in sorted positions.

Then we move to next two values, 35 and 10.

We know than 10 is smaller 35. Hence they are not sorted.

We swap these values. We find that we reach at the end of the array. After one
iteration the array should look like this

To be precise, we are now showing that how array should look like after each
iteration. After second iteration, it should look like this

Notice that after each iteration, at least one value moves at the end

When there's no swap required, bubble sorts learns that array is completely sorted.

School of Computer Engineering

Bubble Sort cont...
7

Algorithm C Code Snippet

Suppose LA is an array with n integer elements and int a[6] = {5, 1, 6, 2, 4, 3};
swap function is to exchange the values for the array
int i, j, temp;
elements. Below is the Bubble Sort algorithm to
arrange the elements in ascending order
for(i=0; i<6; i++)
{
1. Start
for(j=0; j<6-i-1; j++)
2. Set i=0
{
3. Set j = 0
4. Repeat steps 5 till 9 while i < n if( a[j] > a[j+1])
5. Set j = 0 {
6. Repeat steps 7 till 8 while j < n – i - 1 temp = a[j]; a[j] = a[j+1]; a[j+1] = temp;
7. If (LA[j] > LA[j+1]) then swap (LA[j], LA[j+1]) }
8. Set j = j + 1 }
9. Set i = i + 1
}
10. Stop

School of Computer Engineering

Bubble Sort cont…
8

Time Complexity Space Complexity

n-1 comparisons will be done in 1st pass, n-2 in 2nd Space complexity is O(1) since only single additional
pass, n-3 in 3rd pass and so on. So the total number memory space is required for temp variable
of comparisons will be

(n-1)+(n-2)+(n-3)+.....+3+2+1
Sum = n * (n-1)/2
i.e O(n2)

Hence the time complexity of Bubble Sort is

 O(n2) in worst case
 θ(n2) in average case
 Ω(n) in best case

Animation
https://cathyatseneca.github.io/DSAnim/web/bubble.html

School of Computer Engineering

Insertion Sort
9

Insertion sort is in-place comparison based sorting algorithm. A sub-list is maintained which is always
sorted. For example, the lower part of an array is maintained to be sorted. A element which is to be inserted
in the sorted sub-list, has to find its appropriate place and insert it there. Hence the name insertion sort.

The array is searched sequentially and unsorted items are moved and inserted into sorted sub-list (in the
same array). This algorithm is not suitable for large data sets as its average and worst case complexity are
of Ο(n2) where n are no. of items.

How Insertion sort works?

Take an unsorted array for the example

Insertion sort compares the first two elements

It finds that both 14 and 33 are already in ascending

order. For now, 14 is in sorted sub-list.

School of Computer Engineering

Insertion Sort cont…
10

Insertion sort moves ahead and compares 33 with 27

And finds that 33 is not in correct position

It swaps 33 with 27. Also it checks with all the

elements of sorted sublist. Here we see that sorted
sub-list has only one element 14 and 27 is greater than
14. Hence sorted sub-list remain sorted after swapping.

By now 14 and 27 in the sorted sub list. Next it

compares 33 with 10.

These values are not in sorted order

So swap them

School of Computer Engineering

Insertion Sort cont…
11

But swapping makes 27 and 10 unsorted

So we swap them too

Again we find 14 and 10 in unsorted order

And we swap them. By the end of third iteration we

have a sorted sub list of 4 items.

This process goes until all the unsorted values are

covered in sorted sub list.

School of Computer Engineering

Insertion Sort cont...
12

Algorith C Code Snippet

m
Suppose LA is an array with n integer elements and int a[6] = {5, 1, 6, 2, 4, 3};
below is the Insertion Sort algorithm to arrange the
int i, j, key;
elements in ascending order
for(i=1; i<6; i++)
1. Start {
2. Set i=1, j = 0, key = 0
key = a[i];
4. Repeat steps 5 till 10 while i < n
j = i-1;
5. Set key = LA[i]
while(j>=0 && key < a[j])
6. Set j = i - 1
7. While j >= 0 AND key < LA[j] repeat 8 and 9 {
8. LA[j + 1] = LA [j] a[j+1] = a[j];
9. Set j = j – 1 j--;
10. LA[j + 1] = key }
11. Stop a[j+1] = key;
}

School of Computer Engineering

Insertion Sort cont…
13

Time Complexity

Time complexity of Insertion Sort is

 O(n2) in worst case
 Ω(n) in best case
 θ(n2) in average case

Space Complexity

Space complexity is O(1)

Animation
https://courses.cs.vt.edu/csonline/Algorithms/Lessons/InsertionCardSort/insertioncardsort.swf

School of Computer Engineering

Selection Sort
14

Selection sort is a in-place comparison based algorithm in which the list is divided into two parts, sorted
part at left end and unsorted part at right end. Initially sorted part is empty and unsorted part is entire list.

Smallest element is selected from the unsorted array and swapped with the leftmost element and that
element becomes part of sorted array. This process continues moving unsorted array boundary by one
element to the right. This algorithm is not suitable for large data sets as its average and worst case
complexity are of O(n2) where n are no. of items.

How Selection sort works?

Take an unsorted array for the example

For the first position in the sorted list, the whole list is
scanned sequentially. The first position where 14 is
stored presently, we search the whole list and find that
10 is the lowest value.

School of Computer Engineering

Selection Sort cont…
15

So we replace 14 with 10. After one iteration 10, which

happens to be the minimum value in the list, appears in
the first position of sorted list

For the second position, where 33 is residing, we start

scanning the rest of the list in linear manner.

We find that 14 is the second lowest value in the list

and it should appear at the second place. We swap
these values.

After two iterations, two least values are positioned at

the beginning in the sorted manner.

School of Computer Engineering

Selection Sort cont…
16

The same process is applied on the rest of the items in the array and pictorial depiction of entire
sorting process is

School of Computer Engineering

Selection Sort cont...
17

C Code Pseudo code

Snippet procedure selection sort
int a[6] = {5, 1, 6, 2, 4, 3};
list : array of items
int i, j, min, temp;
n : size of list
for(i=0; i<6; i++)
{ for i = 1 to n - 1
min= i; min = i /* set current element as minimum*/
for(j=i+1; i<6; j++) for j = i+1 to n /* check the element to be minimum */
{ if list[j] < list[min] then
if (a[j] < a[min]) min = j;
end if
{
end for
min = j;
/* swap the minimum element with the current element*/
} if indexMin != i then
} swap list[min] and list[i]
temp = a[i]; end if
a[i] = a[min]; end for
a[min] = temp; end procedure
}
School of Computer Engineering
Selection Sort cont...
18

Time Complexity

Time complexity of Selection Sort is

 O(n2) in worst case
 Ω(n) in best case
 θ(n2) in average case

Space Complexity

Space complexity is O(1)

Animation
http://www.algostructure.com/sorting/selectionsort.php

School of Computer Engineering

Divide & Conquer
19

In divide and conquer approach, the problem in hand, is divided into smaller sub-problems and then each
problem is solved independently. When we keep on dividing the sub-problems into even smaller sub-
problems, we may eventually reach at a stage where no more dividation is possible. Those "atomic" smallest
possible sub-problem (fractions) are solved. The solution of all sub-problems is finally merged in order to
obtain the solution of original problem.

School of Computer Engineering

Divide & Conquer cont…
20

Broadly, we can understand divide-and-conquer approach as three step process.

 Divide/Break: This step involves breaking the problem into smaller sub-problems. Sub-problems
should represent as a part of original problem. This step generally takes recursive approach to divide
the problem until no sub-problem is further dividable. At this stage, sub-problems become atomic in
nature but still represents some part of actual problem.
 Conquer/Solve: This step receives lot of smaller sub-problem to be solved. Generally at this level,

problems are considered 'solved' on their own.

 Merge/Combine: When the smaller sub-problems are solved, this stage recursively combines them

until they formulate solution of the original problem.

The following computer algorithms are based on divide-and-conquer programming approach
 Merge Sort
 Quick Sort

This algorithmic approach works recursively and conquer & merge steps works so close that they appear
as one.

School of Computer Engineering

Quick Sort
21

Quick sort is based on partitioning of array of data into smaller arrays. A large array is
partitioned into two arrays one of which holds values smaller than specified value say pivot
based on which the partition is made and another array holds values greater than pivot
value. The quick sort partitions an array and then calls itself recursively twice to sort the
resulting two sub-array.
How Quick Sort works?
Quick Sort Algorithm Quick Sort Pseudo code
Step 1 − Start procedure quickSort(left, right)
Step 2 − Make the right-most index value pivot if right-left <= 0
Step 3 − partition the array using pivot value return
Step 4 − quicksort left partition recursively else
Step 5 − quicksort right partition recursively pivot = A[right]
Step 6 − Stop partition = partitionFunc(left, right, pivot)
quickSort(left,partition-1)
quickSort(partition+1,right)
end if
end procedure

School of Computer Engineering

Quick Sort cont…
22

Quick sort Pivot Algorithm Quick sort Pivot Pseudo code

Step 1 − Choose the highest index value has function partitionFunc(left, right, pivot)
leftPointer = left -1
pivot rightPointer = right
Step 2 − Take two variables to point left and
while true do
right of the list excluding pivot while A[++leftPointer] < pivot do
Step 3 − left points to the low index //do-nothing
end while
Step 4 − right points to the high
Step 5 − while value at left is less than pivot while rightPointer > 0 && A[--rightPointer] > pivot do
//do-nothing
move right end while
Step 6 − while value at right is greater than
if leftPointer >= rightPointer
pivot move left break
Step 7 − if both step 5 and step 6 does not else
swap leftPointer,rightPointer
match swap left and right end if
Step 8 − if left ≥ right, we stop. The point
end while
where they met is new pivot. swap the pivot
value with the split point. swap leftPointer,right
return leftPointer
end function

School of Computer Engineering

Quick Sort
23

Quick sort is a divide and conquer algorithm. Its divided large list in mainly three parts:

1. Elements less than (i.e. LT) pivot element.

2. Pivot element.
3. Elements greater than (i.e. GT) pivot element.

Elements in Green represents LT pivot element

Post to the Partition, Recursively sort the sub-list of Elements in Red represents GT pivot element
lesser elements and the sub-list of greater elements. Element with no color represents pivot element

Partition in Quick sort

School of Computer Engineering

Quick Sort Code Snippet & Complexity Analysis
24

int a[6] = {5, 1, 6, 2, 4, 3}; //continuation of program

int partition(int left, int right, int pivot) void quickSort(int left, int right) Time Complexity
{ {
int leftPointer = left; if(right-left <= 0) Time complexity of Selection Sort is
int rightPointer = right-1; {  O(n2) in worst case
while(true) return;
 Ω(n log n) in best case
{ }
while(a[leftPointer++] < pivot) else  θ(n log n) in average case
{ {
//do nothing
}
int pivot = a[right];
int partitionPoint = partition(left, right, pivot);
Space Complexity
while(rightPointer > 0 && a[rightPointer--] > pivot) quickSort(left, partitionPoint-1);
{ quickSort(partitionPoint+1, right);
Space complexity is O(log n)
//do nothing }
} }
if(leftPointer >= rightPointer) void swap(int num1, int num2)
{ {
break; int temp = a[num1];
} a[num1] = a[num2];
else a[num2] = temp;
{ }
swap(leftPointer, rightPointer); int main()
} {
} quickSort(0,5);
swap(leftPointer, right); return 0;
return leftPointer; }
}

School of Computer Engineering

Merge Sort
25

Merge sort is a sorting technique based on divide and conquer technique. With worst-case time
complexity being Ο(n log n), it is one of the most respected algorithms.

Merge sort first divides the array into equal halves and then combines them in a sorted manner.

How Quick sort works?

Take an unsorted array

Merge sort first divides the whole array iteratively into

equal halves unless the atomic values are achieved.
Here the array of 8 items is divided into two arrays of
size 4.

This does not change the sequence of appearance of

items in the original. Now these two arrays are divided
into halves.

School of Computer Engineering

Merge Sort cont…
26

We further divide these arrays and we

achieve atomic value which can no more be
divided.

Now, we combine them in exactly same manner they

were broken down. Please note the color codes given to
these lists. We first compare the element for each list
and then combine them into another list in sorted
manner. We see that 14 and 33 are in sorted positions.
We compare 27 and 10 and in the target list of 2 values
we put 10 first, followed by 27. We change the order 19
and 35. 42 and 44 are placed sequentially.

In next iteration of combining phase, we compare lists

of two data values, and merge them into a list of found
data values placing all in sorted order.

After final merging, the list should look like this

School of Computer Engineering

Merge Sort cont...
27

Algorithm Pseudo code

procedure mergesort( var a as array ) procedure merge( var a as array, var b as array )
Step 1 - Start n = length of a var c as array
Step 2 − if it is only one element in the list it if ( n == 1 ) return a
while ( a and b have elements )
is already sorted, return. var l1 as array = a[0] ... a[n/2] if ( a[0] > b[0] )
var l2 as array = a[n/2+1] ... a[n] add b[0] to the end of c
Step 3 − divide the list recursively into two remove b[0] from b
halves until it can no more be divided. l1 = mergesort( l1 ) else
l2 = mergesort( l2 ) add a[0] to the end of c
Step 4 − merge the smaller lists into new list remove a[0] from a
in sorted order. return merge( l1, l2 ) end if
end procedure end while
Step 5 - Stop
while ( a has elements )
Time Complexity add a[0] to the end of c
remove a[0] from a
 O(n log n) in worst case end while
 Ω(n log n) in best case
while ( b has elements )
 θ(n log n) in average case add b[0] to the end of c
remove b[0] from b

Space Complexity end while

return c
Space complexity is O(n) end procedure

School of Computer Engineering

Merge Sort C Code
28

#define max 10 void sort(int low, int high) int main()

{ {
int a[10] = { 10, 14, 19, 26, 27, 31, 33, 35, 42, 44 }; int mid; int i;
int b[10];
if(low < high) printf("List before sorting\n");
void merging(int low, int mid, int high) {
{ mid = (low + high) / 2; for(i = 0; i < max; i++)
int l1, l2, i; sort(low, mid); printf("%d ", a[i]);
sort(mid+1, high);
for(l1 = low, l2 = mid + 1, i = low; l1 <= mid && l2 <= high; i++) merging(low, mid, high); sort(0, max-1);
{ }
if(a[l1] <= a[l2]) else printf("\nList after sorting\n");
b[i] = a[l1++]; {
else return; for(i = 0; i < max; i++)
b[i] = a[l2++]; } printf("%d ", a[i]);
} } }

while(l1 <= mid)

b[i++] = a[l1++];

while(l2 <= high)

b[i++] = a[l2++];

for(i = low; i <= high; i++)

a[i] = b[i];
}
School of Computer Engineering
Heap Sort
29

Heap sort is a comparison based sorting technique based on Binary Heap data structure. Complete
Binary Tree: A binary tree T with n levels is complete if all levels except possibly the last are completely
full, and the last level has all its nodes to the left side. A Binary Heap: It is a Complete Binary Tree where
items are stored in a special order such that value in a parent node is greater(or smaller) than the values
in its two children nodes. The former is called as max heap and the latter is called min heap.

Max Heap Min Heap

Since a Binary Heap is a Complete Binary Tree, it can be easily represented as array and array based
representation is space efficient. If the parent node is stored at index I, the left child can be calculated by 2
* I + 1 and right child by 2 * I + 2 (assuming the indexing starts at 0).

School of Computer Engineering

How Heap Sort Works?
30

 In this algorithm we first build a heap using the given elements

 If we want to sort the elements in ascending order, we create a Min Heap
 If we want to sort the elements in descending order, we create a Max Heap
 Once the heap is created, we delete the root node from the heap and put the last node in

the root position and repeat the process till we have covered all the elements
Pointes to Remember for ascending order Pointes to Remember for descending order
sorting sorting
1. Build Heap 1. Build Heap
2. Transform the heap into Min Heap 2. Transform the heap into Max Heap
3. Delete the root node 3. Delete the root node
4. Put the last node of the heap in root 4. Put the last node of the heap in root
position position
5. Repeat from step 2 till all nodes are 5. Repeat from step 2 till all nodes are
covered covered

School of Computer Engineering

Heap Sort Example
31

School of Computer Engineering

Heap Sort Code Snippet & Complexity Analysis
32

Time Complexity Space Complexity

Time complexity of Selection Sort is
Space complexity is O(1)
 O(n log n) in worst case
 Ω(n log n) in best case
 θ(n log n) in average case

C Code

Max Heap

School of Computer Engineering

Radix Sort
33

Radix Sort is a clever and intuitive little sorting algorithm. The idea of Radix Sort is to do digit
by digit sort starting from least significant digit to most significant digit.
Example:
Original, unsorted list: 170, 45, 75, 90, 802, 24, 2, 66
Sorting by least significant digit (1s place) gives: 170, 90, 802, 2, 24, 45, 75, 66
Sorting by next digit (10s place) gives: 802, 2, 24, 45, 66, 170, 75, 90
Sorting by most significant digit (100s place) gives: 2, 24, 45, 66, 75, 90, 170, 802
Time Complexity Space Complexity
Time complexity of Radix Sort is Space complexity is O(w+n)
 O(w*n) in worst case
 Ω(w*n) in best case
 θ(w*n) in average case
Note: w is the word size and n is the number of elements

School of Computer Engineering

Radix Sort C Code
34

Radix Sort

School of Computer Engineering

Class Work
35

Sr # Algorithm Time Complexity Space

Complexi
Best Averag Worst ty
e
1 Bubble Sort Ω(n) Θ(n2) O(n2) O(1)
2 Insertion Sort
3 Selection Sort
4 Quick Sort
5 Merge Sort
6 Heap Sort
7 Radix Sort

School of Computer Engineering

Assignments
36

1. Let A[1], A[2], . . . , A[n] be an array containing n very large positive integers.
Describe an efficient algorithm to find the minimum positive difference
between any two integers in the array. What is the complexity of your
algorithm? Explain.

2. Design an efficient algorithm to sort 5 distinct keys.

3. Let A = A[1], . . . , A[n] be an array of n distinct positive integers. An inversion is

a pair of indices i and j such that i < j but A[i] > A [j]. For example in the array
[30000, 80000, 20000, 40000, 10000], the pair i = 1 and j = 3 is an inversion
because A [1] = 30000 is greater than A[3] = 20000. On the other hand, the pair
i = 1 and j = 2 is not an inversion because A [1] = 30000 is smaller than A [2] =
80000. In this array there are 7 inversions and 3 non-inversions. Describe an
efficient algorithm that counts the number of inversions in any array.
What is the running time of your algorithm?

School of Computer Engineering

37

School of Computer Engineering

Home Work
38

Experiment with the C implementations of the various sorting algorithms:

 Bubble Sort
 Insertion Sort
 Selection Sort
 Quick Sort
 Merge Sort
 Heap Sort
 Radix Sort

School of Computer Engineering

Supplementary Reading
39

 http://www.geeksforgeeks.org/sorting-algorithms/
 https://www.tutorialspoint.com/data_structures_algorithms/sorting_algorithms.htm
 http://www.studytonight.com/data-structures/introduction-to-sorting
 http://nptel.ac.in/courses/Webcourse-contents/IIT-%20Guwahati/data_str_algo/Module_5/binder1.pdf
 http://nptel.ac.in/courses/106105164/

School of Computer Engineering

FAQ
40

 Bubble Sort : Exchange two adjacent elements if they are out of order. Repeat until array is
sorted.
 Insertion sort : Scan successive elements for an out-of-order item, then insert the item in the
proper place.
 Selection sort : Find the smallest element in the array, and put it in the proper place. Swap it
with the value in the first position. Repeat until array is sorted.
 Quick sort : Partition the array into two segments. In the first segment, all elements are less
than or equal to the pivot value. In the second segment, all elements are greater than or equal
to the pivot value. Finally, sort the two segments recursively.
 Merge sort : Divide the list of elements in two parts, sort the two parts individually and then
merge it.
 Heap sort : The first maximum element is searched and place it at the end and the process is
repeated for remaining elements.
 Radix sort : Sorts data with integer keys by grouping keys by the individual digits which
share the same significant position and value.

School of Computer Engineering