4 Credit

Data Structures and Algorithms (C

KALINGA INSTITUTE OF IND

TECHNOLOGY

School of Computer Engin

Lecture Note
 Chapter Contents
2

Sr Major and Detailed Coverage Area Hr

# s
7 Sorting 4
Bubble Sort, Insertion Sort, Selection Sort, Quick
Sort, Merge Sort, Heap Sort, Radix Sort
School of Computer Engineering
 3

Sorting refers to arranging data in a particular format. Sorting

Introduction
algorithm specifies the way to arrange data in a particular order.
Most common orders are in numerical or lexicographical order.
Sorting is also used to represent data in more readable formats.
Following are some of the examples of sorting in real-life
scenarios:
 Telephone Directory − The telephone directory stores the
School of Computer Engineering

Sorting Algorithm
telephone numbers of people sorted by their names, so that the
names
Sr# can be
Traditional searched
Sorting easily.
Sorting based on Divide & Conquer Sorting based on Trees

 1Dictionary
Bubble Sort Merge sortstores wordsHeap
− The dictionary in an Sort alphabetical
2order so that searching
Insertion sort Quickof any word becomes easy.
sort
3 Selection sort
4 Radix Sort
 4

Bubble sort is a simple sorting algorithm. This sorting algorithm is comparison based algorithm in which

Bubble Sort
each pair of adjacent elements is compared and elements are swapped if they are not in order. This
algorithm is not suitable for large data sets as its average and worst case complexity is of O(n2) where n
are no. of items.

How bubble sort works?

Take an unsorted array
School of Computer Engineering

Bubble sort starts with very first two elements, comparing them to check which
one is greater.
In this case, 33 is greater than 14, so it is already in sorted locations. Next, it
compares 27 with 33.

We find that 27 is smaller than 33 and these two values must be swapped.

Post to swapping, the new array should looks like

 Bubble Sort cont…
5

Next it compares 35 with 33 and both are already in sorted positions.

Then we move to next two values, 35 and 10.

We know than 10 is smaller 35. Hence they are not sorted.

We swap these values. We find that we reach at the end of the array. After one
School of Computer Engineering

iteration the array should look like this

To be precise, we are now showing that how array should look like after each
iteration. After second iteration, it should look like this

Notice that after each iteration, at least one value moves at the end

When there's no swap required, bubble sorts learns that array is completely sorted.
 Bubble Sort cont...
6

Algorithm C Code Snippet

Suppose LA is an array with n integer elements and
swap function is to exchange the values for the
int a[6] = {5, 1, 6, 2, 4, 3};
array elements. Below is the Bubble Sort algorithm int i, j, temp;
to arrange the elements in ascending order
for(i=0; i<6; i++)
1. Start
2. Set i=0 {
3. Set j = 0 for(j=0; j<6-i-1; j++)
School of Computer Engineering

4. Repeat steps 5 till 9 while i < n

5. Set j = 0 {
6. Repeat steps 7 till 8 while j < n – i - 1 if( a[j] > a[j+1])
7. If (LA[j] > LA[j+1]) then swap (LA[j], LA[j+1])
8. Set j = j + 1 {
9. Set i = i + 1 temp = a[j]; a[j] = a[j+1]; a[j+1] = temp;
10. Stop
}
}
 Bubble Sort cont…
7

Time Complexity Space Complexity

n-1 comparisons will be done in 1st pass, n-2 in Auxiliary Space is the extra space or temporary
2nd pass, n-3 in 3rd pass and so on. So the
space used by an algorithm. if we want to
total number of comparisons will be
compare standard sorting algorithms on the basis
of space, then auxiliary space would be a better
(n-1)+(n-2)+(n-3)+.....+3+2+1
criteria than space complexity. Space complexity
Sum = n * (n-1)/2
i.e O(n2)
is O(1) since only single additional memory space
School of Computer Engineering

is required for temp variable

Hence the time complexity of Bubble Sort is
 O(n2) in worst case
 θ(n2) in average case
 Ω(n) in best case

Animation
https://cathyatseneca.github.io/DSAnim/web/bubble.html
 8

Insertion sort is in-place comparison based sorting algorithm. A sub-list is maintained which is always

Insertion Sort
sorted. For example, the lower part of an array is maintained to be sorted. A element which is to be
inserted in the sorted sub-list, has to find its appropriate place and insert it there. Hence the name
insertion sort.

The array is searched sequentially and unsorted items are moved and inserted into sorted sub-list (in the
same array). This algorithm is not suitable for large data sets as its average and worst case complexity
are of Ο(n2) where n are no. of items.
School of Computer Engineering

ow Insertion sort works?

Take an unsorted array for the example

Insertion sort compares the first two elements

It finds that both 14 and 33 are already in ascending

order. For now, 14 is in sorted sub-list.
 Insertion Sort cont…
9

Insertion sort moves ahead and compares 33 with 27

And finds that 33 is not in correct position

It swaps 33 with 27. Also it checks with all the

elements of sorted sublist. Here we see that sorted
sub-list has only one element 14 and 27 is greater
than 14. Hence sorted sub-list remain sorted after
School of Computer Engineering

swapping.

By now 14 and 27 in the sorted sub list. Next it

compares 33 with 10.

These values are not in sorted order

So swap them
 Insertion Sort cont…
10

But swapping makes 27 and 10 unsorted

So we swap them too

Again we find 14 and 10 in unsorted order

School of Computer Engineering

And we swap them. By the end of third iteration we

have a sorted sub list of 4 items.

This process goes until all the unsorted values are

covered in sorted sub list.
 Insertion Sort cont...
11

Algorit C Code Snippet

hm
Suppose LA is an array with n integer elements and
int a[6] = {5, 1, 6, 2, 4, 3};
below is the Insertion Sort algorithm to arrange the
elements in ascending order int i, j, key;
1. Start for(i=1; i<6; i++)
2. Set i=1, j = 0, key = 0
4. Repeat steps 5 till 10 while i < n {
5. Set key = LA[i] key = a[i];
School of Computer Engineering

6. Set j = i - 1
7. While j >= 0 AND key < LA[j] repeat 8 and 9 j = i-1;
8. LA[j + 1] = LA [j] while(j>=0 && key < a[j])
9. Set j = j – 1
10. LA[j + 1] = key {
11. Stop a[j+1] = a[j];
j--;
}
 Insertion Sort cont…
12

Time Complexity
Time complexity of Insertion Sort is
 O(n2) in worst case
 θ(n2) in average case
 Ω(n) in best case
School of Computer Engineering

Space Complexity
Space complexity is O(1)
Animation
https://courses.cs.vt.edu/csonline/Algorithms/Lessons/InsertionCardSort/insertioncardsort.swf
 13

Selection sort is a in-place comparison based algorithm in which the list is divided into two parts, sorted

Selection Sort
part at left end and unsorted part at right end. Initially sorted part is empty and unsorted part is entire list.

Smallest element is selected from the unsorted array and swapped with the leftmost element and that
element becomes part of sorted array. This process continues moving unsorted array boundary by one
element to the right. This algorithm is not suitable for large data sets as its average and worst case
complexity are of O(n2) where n are no. of items.
School of Computer Engineering

ow Selection sort works?

Take an unsorted array for the example

For the first position in the sorted list, the whole list
is scanned sequentially. The first position where 14
is stored presently, we search the whole list and
find that 10 is the lowest value.
 Selection Sort cont…
14

So we replace 14 with 10. After one iteration 10,

which happens to be the minimum value in the list,
appears in the first position of sorted list

For the second position, where 33 is residing, we

start scanning the rest of the list in linear manner.
School of Computer Engineering

We find that 14 is the second lowest value in the list

and it should appear at the second place. We swap
these values.

After two iterations, two least values are positioned

at the beginning in the sorted manner.
 Selection Sort cont…
15

The same process is applied on the rest of the items in the array and pictorial depiction of entire
sorting process is

School of Computer Engineering

 Selection Sort cont...
16

C Code Pseudo
Snippet
int a[6] = {5, 1, 6, 2, 4, 3}; procedure selection sort
code
list : array of items
int i, j, min, temp;
n : size of list
for(i=0; i<6; i++)
{ for i = 1 to n - 1
min= i; min = i /* set current element as minimum*/
for(j=i+1; j<6; j++) for j = i+1 to n /* check the element to be minimum */
School of Computer Engineering

{ if list[j] < list[min] then

if (a[j] < a[min]) min = j;
end if
{
end for
min = j;
/* swap the minimum element with the current element*/
} if (indexMin != i) then
} swap list[min] and list[i]
temp = a[i]; end if
a[i] = a[min]; end for
a[min] = temp; end procedure
}
 Selection Sort cont...
17

Time Complexity
Time complexity of Selection Sort is
 O(n2) in worst case
 θ(n2) in average case
 Ω(n2) in best case
School of Computer Engineering

Space
Complexity
Space complexity is O(1)
Animation
http://www.algostructure.com/sorting/selectionsort.php
18

Radix Sort is a clever and intuitive little sorting algorithm. The idea
of Radix Sort is to do digit by digit sort starting from least

Radix Sort
significant digit to most significant digit.
Example:
Original, unsorted list: 170, 45, 75, 90, 802, 24, 2, 66
School of Computer Engineering

Sorting by least significant digit (1s place)

Time Complexity gives: 170, 90, 802, 2,
Space
24,
Time45, 75, 66
complexity of Complexity
Space complexity is
Sorting
Radixby next
Sort is digit (10s place) gives:O(w+n)
802, 2, 24, 45, 66, 170, 75,
90
 O(w*n) in worst
Sorting
casewby
Note: is most significant
the word digit
size and n is(100s place) gives:
the number of 2, 24, 45, 66,
75, 90, 170,in802
elements
 θ(w*n) average
case
 19

School of Computer Engineering

Radix Sort C Code

 Divide & Conquer
20

In divide and conquer approach, the problem in hand, is divided into smaller sub-problems and then each
problem is solved independently. When we keep on dividing the sub-problems into even smaller sub-
problems, we may eventually reach at a stage where no more dividation is possible. Those "atomic" smallest
possible sub-problem (fractions) are solved. The solution of all sub-problems is finally merged in order to
obtain the solution of original problem.

School of Computer Engineering

 Divide & Conquer
21

Broadly, we can understand divide-and-conquer approach as three step process.

 Divide/Break: This step involves breaking the problem into smaller sub-problems. Sub-problems
should represent as a part of original problem. This step generally takes recursive approach to
divide the problem until no sub-problem is further dividable. At this stage, sub-problems become
atomic in nature but still represents some part of actual problem.
 Conquer/Solve: This step receives lot of smaller sub-problem to be solved. Generally at this level,
problems are considered 'solved' on their own.
 Merge/Combine: When the smaller sub-problems are solved, this stage recursively combines
School of Computer Engineering

them until they formulate solution of the original problem.

The following computer algorithms are based on divide-and-conquer programming approach
 Merge Sort
 Quick Sort

This algorithmic approach works recursively and conquer & merge steps works so close that they
appear as one.
 22

Merge sort is a sorting technique based on divide and conquer technique. With worst-case time
complexity being Ο(n log n), it is one of the most respected algorithms.

Merge Sort
Merge sort first divides the array into equal halves and then combines them in a sorted manner.

How Merge sort works?

Take an unsorted array
School of Computer Engineering

Merge sort first divides the whole array iteratively

into equal halves unless the atomic values are
achieved. Here the array of 8 items is divided into
two arrays of size 4.

This does not change the sequence of appearance of

items in the original. Now these two arrays are
divided into halves.
 Merge Sort cont…
23

We further divide these arrays and we

achieve atomic value which can no more
be divided.

Now, we combine them in exactly same manner they

were broken down. Please note the color codes given
to these lists. We first compare the element for each
list and then combine them into another list in sorted
manner. We see that 14 and 33 are in sorted
School of Computer Engineering

positions. We compare 27 and 10 and in the target

list of 2 values we put 10 first, followed by 27. We
change the order 19 and 35. 42 and 44 are placed
sequentially.

In next iteration of combining phase, we compare

lists of two data values, and merge them into a list
of found data values placing all in sorted order.

After final merging, the list should look like this

 Merge Sort cont...
24

Algorithm Pseudo code

procedure mergesort( var a as array ) procedure merge( var a as array, var b as array )
Step 1 - Start n = length of a var c as array
Step 2 − if it is only one element in the list if ( n == 1 ) return a
while ( a and b have elements )
it is already sorted, return. var l1 as array = a[0] ... a[n/2] if ( a[0] > b[0] )
add b[0] to the end of c
Step 3 − divide the list recursively into two var l2 as array = a[n/2+1] ... a[n]
remove b[0] from b
halves until it can no more be divided. l1 = mergesort( l1 ) else
l2 = mergesort( l2 ) add a[0] to the end of c
Step 4 − merge the smaller lists into new remove a[0] from a
list in sorted order. end if
School of Computer Engineering

return merge( l1, l2 )

end procedure end while
Step 5 - Stop
while ( a has elements )
Time Complexity add a[0] to the end of c
remove a[0] from a
 O(n log n) in worst end while

case while ( b has elements )

add b[0] to the end of c
remove b[0] from b
 θ(n log n) in end while
Space Complexity
average
Space case is O(n)
complexity
return c
end procedure

 Ω(n log n) in best

 Merge Sort C Code
25

#define max 10 void sort(int low, int high) int main()

{ {
int a[max] = { 10, 14, 19, 26, 27, 31, 33, 35, 42, 44 }; int mid; int i;
int b[max];
if(low < high) printf("List before sorting\n");
void merge(int low, int mid, int high) {
{ mid = (low + high) / 2; for(i = 0; i < max; i++)
int l1, l2, i; sort(low, mid); printf("%d ", a[i]);
sort(mid+1, high);
for(l1 = low, l2 = mid + 1, i = low; l1 <= mid && l2 <= high; i++) merge(low, mid, high); sort(0, max-1);
{ }
if(a[l1] <= a[l2]) else printf("\nList after sorting\n");
b[i] = a[l1++]; {
School of Computer Engineering

else return; for(i = 0; i < max; i++)

b[i] = a[l2++]; } printf("%d ", a[i]);
} } }

while(l1 <= mid)

b[i++] = a[l1++];

while(l2 <= high)

b[i++] = a[l2++];

for(i = low; i <= high; i++)

a[i] = b[i];
}
26

Quick sort is based on partitioning of array of data into smaller

arrays. A large array is partitioned into two arrays one of which

Quick Sort
holds values smaller than specified value say pivot based on
which the partition is made and another array holds values
How Quick
greater thanSort works?
pivot value. The quick sort partitions an array and
Quickcalls
then Sortitself
Algorithm
recursively twiceQuick
to sortSortthePseudo code two sub-
resulting
School of Computer Engineering

Step 1 − Start procedure quickSort(left, right)

if right-left <= 0
array.
Step 2 − Make the right-most index value pivot
return
Step 3 − partition the array using pivot value
Step 4 − quicksort left partition recursively else
Step 5 − quicksort right partition recursively pivot = A[right]
Step 6 − Stop partition = partitionFunc(left, right, pivot)
quickSort(left,partition-1)
quickSort(partition+1,right)
end if
end procedure
 Quick Sort cont…
27

Quick sort is a divide and conquer algorithm. Its divided large list in mainly three parts:

1. Elements less than (i.e. LT) pivot element.

2. Pivot element.
3. Elements greater than (i.e. GT) pivot element.

Elements in Green represents LT pivot element

Post to the Partition, Recursively sort the sub-list of Elements in Red represents GT pivot element
Element with no color represents pivot element
lesser elements and the sub-list of greater elements.
School of Computer Engineering

Quick sort Pivot Algorithm Partition in Quick sort

Step 1 − Choose the highest index value has pivot
Step 2 − Take two variables to point left and right of the list
excluding pivot
Step 3 − left points to the low index
Step 4 − right points to the high
Step 5 − while value at left is less than pivot move right
Step 6 − while value at right is greater than pivot move left
Step 7 − if both step 5 and step 6 does not match swap left and
right
Step 8 − if left ≥ right, we stop else goto step 5. The point where
they met is new pivot. swap the pivot value with the split point.
 Quick Sort Code Snippet & C
28

int a[6] = {5, 1, 6, 2, 4, 3}; //continuation of program

int partition(int left, int right, int pivot)
{
void quickSort(int left, int right) Time Complexity
{
int leftPointer = left; if(right-left <= 0) Time complexity of Selection Sort is
int rightPointer = right-1; {  O(n2) in worst case
while(true) return;
{
 Ω(n log n) in best case
}
while(a[leftPointer++] < pivot) else  θ(n log n) in average case
{ {
//do nothing int pivot = a[right];
} int partitionPoint = partition(left, right, pivot); Space
while(rightPointer > 0 && a[rightPointer--] > pivot) quickSort(left, partitionPoint-1);
{ quickSort(partitionPoint+1, right); Complexity
Space complexity is O(log n)
School of Computer Engineering

//do nothing }
} }
if(leftPointer >= rightPointer) void swap(int num1, int num2)
{ {
break; int temp = a[num1];
} a[num1] = a[num2];
else a[num2] = temp;
{ }
swap(leftPointer, rightPointer); int main()
} {
} quickSort(0,5);
swap(leftPointer, right); return 0;
return leftPointer; }
}
29

Heap sort is a comparison based sorting technique based on Binary Heap data structure.
Complete Binary Tree: A binary tree T with n levels is complete if all levels except possibly the last
are completely full, and the last level has all its nodes to the left side.

Heap Sort
A Binary Heap: It is a Complete Binary Tree where items are stored in a special order such that value
in a parent node is greater(or smaller) than the values in its two children nodes. The former is called
as max heap and the latter is called min heap.

Max Heap Min Heap

School of Computer Engineering

Since a Binary Heap is a Complete Binary Tree, it can be easily represented as array and array based
representation is space efficient. If the parent node is stored at index I, the left child can be calculated
by 2 * I + 1 and right child by 2 * I + 2 (assuming the indexing starts at 0).
 How Heap Sort Wo
30

 In this algorithm we first build a heap using the given elements

 If we want to sort the elements in ascending order, we create a
Min Heap
 If we want to sort the elements in descending order, we create
a Max
Pointes
sorting
Heap
to Remember for ascending order Pointes to Remember for descending order
sorting
Build the
1. Once
 Heapheap is created, we delete 1. Build Heapnode from the heap
the root
School of Computer Engineering

Transform
2. and put thethe lastheap
nodeinto Transform
Minroot2.position
in the the heap
and repeat theinto Max
process
Heap
till we have covered all the elements Heap
3. Delete the root node 3. Delete the root node
4. Put the last node of the heap 4. Put the last node of the heap
in root position in root position
5. Repeat from step 2 till all 5. Repeat from step 2 till all
 31

School of Computer Engineering

Heap Sort Exampl

 Heap Sort Code Snippet & Co
32

Time Complexity Space

Time complexity of Selection Sort is
 O(n log n) in worst case
Complexity
Space complexity is O(1)

 Ω(n log n) in best case

 θ(n log n) in average case

C Code School of Computer Engineering

33

Sr Algorithm Time Complexity Space

Class Work
# Best Average Worst Comple
xity

1 Bubble Sort Ω(n) Θ(n2) O(n2) O(1)

School of Computer Engineering

2 Insertion Sort
3 Selection Sort
4 Quick Sort
5 Merge Sort
6 Heap Sort
7 Radix Sort
34

Assignments
1. Let A[1], A[2], . . . , A[n] be an array containing very large positive integers.
Describe an efficient algorithm to find the minimum positive difference
between any two integers in the array. What is the complexity of your
algorithm? Explain.

2. Design an efficient algorithm to sort 5 distinct keys.

3. Let A = A[1], . . . , A[n] be an array of n distinct positive integers. An

School of Computer Engineering

inversion is a pair of indices i and j such that i < j but A[i] > A [j]. For
example in the array [30000, 80000, 20000, 40000, 10000], the pair i = 1
and j = 3 is an inversion because A [1] = 30000 is greater than A[3] =
20000. On the other hand, the pair i = 1 and j = 2 is not an inversion
because A [1] = 30000 is smaller than A [2] = 80000. In this array there are
7 inversions and 3 non-inversions. Describe an efficient algorithm that
counts the number of inversions in any array. What is the running
time of your algorithm?
 35

School of Computer Engineering

36

Experiment with the C implementations of the various sorting algorithms:

Home Work
 Bubble Sort
 Insertion Sort
 Selection Sort
 Quick Sort
 Merge Sort
 Heap Sort
School of Computer Engineering

 Radix Sort
 Supplementary Rea
37

 http://www.geeksforgeeks.org/sorting-algorithms/
 https://www.tutorialspoint.com/data_structures_algorithms/sorting_algorithms.htm
 http://www.studytonight.com/data-structures/introduction-to-sorting
 http://nptel.ac.in/courses/Webcourse-contents/IIT-%20Guwahati/data_str_algo/Module_5/binder1.pdf
 http://nptel.ac.in/courses/106105164/

School of Computer Engineering

38

 Bubble Sort : Exchange two adjacent elements if they are out of order. Repeat until array
is sorted.
 Insertion sort : Scan successive elements for an out-of-order item, then insert the item in
the proper place.
 Selection sort : Find the smallest element in the array, and put it in the proper place.
Swap it with the value in the first position. Repeat until array is sorted.
 Quick sort : Partition the array into two segments. In the first segment, all elements are
less than or equal to the pivot value. In the second segment, all elements are greater than

FAQ
School of Computer Engineering

or equal to the pivot value. Finally, sort the two segments recursively.
 Merge sort : Divide the list of elements in two parts, sort the two parts individually and
then merge it.
 Heap sort : The first maximum element is searched and place it at the end and the
process is repeated for remaining elements.
 Radix sort : Sorts data with integer keys by grouping keys by the individual digits which
share the same significant position and value.