Sorting Algorithms

Assistant Professor
IT-ICT Department, LJIET
Mr. Prayag Patel [email protected]
 Problems solved by Algorithms
Sorting Combinatorial
Problem.

Searching
Problems Graph
solved by Problem
Algorithms

Numerical FA
Problem String Processing Problem
Mr. Prayag Patel Department: IT/ICT-LJIET
 Applications of Sorting
1. Phone Bill: the calls made are date wise sorted.
2. Bank statement or Credit card Bill: transactions made
are date wise sorted.
3. Filling forms online: “select country” drop down box
will have the name of countries sorted in Alphabetical
order.
4. Online shopping: the items can be sorted price wise,
date wise or relevance wise.
5. Files or folders on your desktop are sorted date wise.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Types of Sorting Algorithms
1. Bubble Sort.
2. Selection Sort.
3. Insertion Sort.
4. Heap Sort.
5. Counting Sort.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Stable Sorting algorithm
Stable sorting algorithms maintain the relative order of records
with equal keys (i.e. values).

Mr. Prayag Patel Department: IT/ICT-LJIET

 Stable Sorting algorithm

Example: Bubble sort, Insertion Sort, Merge sort, Counting sort.

Mr. Prayag Patel Department: IT/ICT-LJIET

 In place Sorting algorithm
In-place sort algorithm : A sort algorithm in which the sorted
items occupy the same storage as the original ones.
In-place means that the algorithm does not use extra space
for manipulating the input.

Mr. Prayag Patel Department: IT/ICT-LJIET

 In place Sorting algorithm

Example: Bubble sort, Selection sort, Insertion Sort, Heap sort, Shell sort, Quick sort.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Bubble Sort

Assistant Professor
IT-ICT Department, LJIET
Mr. Prayag Patel [email protected]
 Bubble Sort
▪ It is a simple sorting algorithm that works by
• Comparing each pair of adjacent items and swapping them if they are in
the wrong order.
• The pass through the list is repeated until no swaps are needed, which indicates
that the list is sorted.
▪ As it only uses comparisons to operate on elements, it is a
comparison sort.
▪ Although the algorithm is simple, it is too slow for practical use.
▪ It is stable sorting algorithm.
▪ It is In place sorting algorithm.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Bubble Sort
Sort the following array in Ascending order
45 34 56 23 12

Pass 1 :

34
45 34 34 34

swap
34
45 45 45 45
56 56 56
23 23

swap
23 23 23
56 56
12

swap
12 12 12 12
56

if(A[j] > A[j+1])

swap(A[j],A[j+1])

Mr. Prayag Patel Department: IT/ICT-LJIET

 Bubble Sort
Pass 2 : Pass 3 : Pass 4 :

34 34 34 23
34 23 12
23

swap
swap
45 45
23 23 23
34 34
12 12
23

swap
swap
23 23
45 45
12 12 12
34 34

swap
12 12 12
45 45 45 45
56 56 56 56 56 56

if(A[j] > A[j+1])

swap(A[j],A[j+1])

Mr. Prayag Patel Department: IT/ICT-LJIET

 Bubble Sort Algorithm
# Input: Array A n=5
# Output: Sorted array A 0 34
i = 1,2,3,4 1 45
Algorithm: Bubble_Sort(A) 2 23
j = 0,1,2,3
3 12
for i ← 1 to n-1 do 4 56
for j ← 0 to n-1-i do
if A[j] > A[j+1] then
thenswap(A[j],A[j+1])
temp← ←A[j]
temp A[j]
A[j]← ←A[j+1]
A[j] A[j+1]
A[j+1]← ←temp
A[j+1] temp

The time complexity of bubble sort is

Mr. Prayag Patel Department: IT/ICT-LJIET

 Bubble Sort Analysis
Algorithm: Bubble_Sort(A) Cost Times
for i ← 1 to n-1 do
for j ← 0 to n-1-i do 1

if A[j] > A[j+1] 1

then temp ← A[j] 1

A[j] ← A[j+1]
A[j+1] ← temp
1 1 1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Bubble Sort Analysis
Equation used in analysis
1) 1 = 1 + 1 + 1 +……+ 1 = n

2) 1 = 1 + 1 + 1 +……+ 1 = n – 1

3) i = 1 + 2 + 3 +……+ n = n(n+1) /2

4) i = 1 + 2 + 3 +……+ (n – 1) = n(n -1) /2

Mr. Prayag Patel Department: IT/ICT-LJIET

 Bubble Sort Analysis

T(n) = C1n+ C2[ (n+1) - i ] + C3 [ n - i ] + C4 [ n - i]

= C1n+C2[(n+1) 1 - i ] + C3[n 1- i ] +C4[n 1- i]

= C1n + C2 (n+1) (n-1) - C2 n(n-1)/2 + C3 n(n-1) - C3n(n-1)/2 + C4 n(n-1) -

C4 n(n-1)/2
= C1n + C2 n2- C2 - + C2 n2/2 + C2n/2 + C3 n2- C3 n - C3 n2/2 + C3n/2 + C4 n2-
C4 n - C4 n2/2 + C4n/2
= n2(C2 – C2/2 + C3 – C3/2 + C4 – C4 /2 ) + n(C1 + C2/2 – C3 + C3/2 – C4
+ C4 /2) + (– C2)
= an2 + bn + c = O (n2) For Worst case and Average case

Mr. Prayag Patel Department: IT/ICT-LJIET

 Bubble Sort: Best Case Analysis
Pass 1 : i=1

int flag=1; 12 j=0

for(i = 0; i < n-1; i++) 23 j=1
34
for(j = 0; j < =n-1-i; j++) j=2
45
if(A[j] > A[j+1]) Condition never 59 j=3
becomes true
flag=0;
swap(A[j],A[j+1])
if(flag == 1)
cout<<"already sorted"<<endl
break;

Only one iteration takes place from 0 to n-2 (n-1-1), i.e. Pass 1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort

Assistant Professor
IT-ICT Department, LJIET
Mr. Prayag Patel [email protected]
 Selection Sort
• Selection sort divides the array or list into two parts,
1. The sorted part at the left end
2. and the unsorted part at the right end.
• Initially, the sorted part is empty and the unsorted part is the entire list.
• The smallest element is selected from the unsorted array and swapped with the
leftmost element, and that element becomes a part of the sorted array.
• Then it finds the second smallest element and exchanges it with the element in the
second leftmost position.
• This process continues until the entire array is sorted.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort
Sort the following elements in Ascending order
5 1 12 -5 16 2 12 14

Step 1 :
Unsorted Array
5 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

Step 2 :
Unsorted Array (elements 2 to 8) i = 1, Value A[i] = 5
Find the smallest value
-5
5 1 12 -5
5 16 2 12 14 from the entire Unsorted
1 2 3 4 5 6 7 8 array
Min = 4, value A[min] = -5
Swap

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort
Step 3 : i = 2, Value A[i] = 1
Unsorted Array (elements 3 to 8)
Find min value from
-5 1 12 5 16 2 12 14 Remaining unsorted array
1 2 3 4 5 6 7 8 min = 2, value A[min] = 1

No Swapping as min value is already at right place

Step 4 :
Unsorted Array
(elements 4 to 8) i = 3, Value A[i] = 12

Find min value from

-5 1 12
2 5 16 12
2 12 14 Remaining unsorted array
1 2 3 4 5 6 7 8 min = 6, value A[min] = 2

Swap

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort
Step 5 :
Unsorted Array i = 4, value A[i] = 5
(elements 5 to 8)
Find min value from
-5 1 2 5 16 12 12 14 Remaining unsorted array
min= 4, value A[min] = 5
1 2 3 4 5 6 7 8

No Swapping as min value is already at right place

Step 6 :
Unsorted Array
(elements 6 to 8)
i = 5, value A[i] = 16

-5 1 2 5 12
16 16
12 12 14 Find min value from
1 2 3 4 5 6 7 8 Remaining unsorted array
min = 6, value A[min] = 12
Swap

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort
Step 7 :
Unsorted Array i = 6, value A[i] = 16
(elements 7 to 8)
Find min value from
-5 1 2 5 12 12
16 16
12 14 Remaining unsorted array
min = 7, value A[min] = 12
1 2 3 4 5 6 7 8

Swap

Step 8 :
Unsorted Array
(element 8)
i = 7, Value A[i] = 16

-5 1 2 5 12 12 14
16 16
14 Find min value from
1 2 3 4 5 6 7 8 Remaining unsorted array
min = 8, value A[min]= 14
Swap

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort Analysis
# Input: Array A
# Output: Sorted array A
Algorithm: Selection_Sort(A)
for i ← 1 to n-1 do
{
min ← i;
for j ← i + 1 to n do
{
if (A[j] < A[min]) then
min ← j;
}
temp ← A[i];
A[i] ← A[min];
A[min] ← temp;
}

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort Algorithm
Algorithm: Selection_Sort(A)
Pass 1 :
for i ← 1 to n-1 do
min ← i; i = 1
for j ← i + 1 to n do
Min ← 12
if A[j] < A[min] then
min ← j ; A[min]← 34
45
temp ← A[i];
A[i] ← A[min]; j = 2 3
A[min] ← temp; No Change

Sort in Ascending order

45 34 56 23 12
1 2 3 4 5

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort Algorithm
Algorithm: Selection_Sort(A) Pass 1 :
for i ← 1 to n-1 do
min ← i; i = 1
for j ← i + 1 to n do Min ← 51
2
4
if A[j] < A[min] then
min ← j ; A[min] ← 12
34
23
45
temp ← A[i];
j = 2 3 4 5
A[i] ← A[min];
A[min] ← temp;

Sort in Ascending order Unsorted Array

45 34 56 23 12 45
12 34 56 23 12
45
1 2 3 4 5 1 2 3 4 5
45
12 23
34 34
56 34
23
45
56 45
56

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort Analysis
# Input: Array A
# Output: Sorted array A
Algorithm: Selection_Sort(A)
for i ← 1 to n-1 do
{
min ← i;
for j ← i + 1 to n do
{
if (A[j] < A[min]) then
min ← j;
}
temp ← A[i];
A[i] ← A[min];
A[min] ← temp;
}
The time complexity of Selection sort is
Mr. Prayag Patel Department: IT/ICT-LJIET
 Selection Sort Analysis
Algorithm: Selection_Sort(A) Cost Times
for i ← 1 to n-1 do
min ← i ;
for j ← i + 1 to n do

if A[j] < A[min] then

C3

min ← j ;

temp ← A[i]; C4

A[i] ← A[min]; C5

A[min] ← temp; C6

Mr. Prayag Patel Department: IT/ICT-LJIET

 Selection Sort Analysis
T(n) = C1n+ C2(n-1)+ C3 n(n+1)/2+ C4(n-1) + C5(n-1 ) + C6(n-1)

= C1n+ C2n- C2+ C3 n2/2 + C3 n/2 + C4n- C4 + C5n- C5+C6n- C6

= n2 C3/2 + n(C1+ C2+ C3/2 + C4 + C5 + C6) + (-C2 –C4 –C6)

= an2 + bn + c

= O (n2)

Time Complexity of Selection sort algorithm is O (n2)

Mr. Prayag Patel Department: IT/ICT-LJIET

 Insertion Sort

Assistant Professor
IT-ICT Department, LJIET
Mr. Prayag Patel [email protected]
 Insertion Sort
Sort the following elements in Ascending order
5 1 12 -5 16 2 12 14

Step 1 :
Unsorted Array
5 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8
Step 2 :

15 1 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

Shift down

Mr. Prayag Patel Department: IT/ICT-LJIET

 Insertion Sort
Step 3 :

1 5 12 -5 16 2 12 14
1 2 3 4 5 6 7 8

No Shift will take place

Step 4 :

1
-5 5 12 -5 16 2 12 14
1 2 3 4 5 6 7 8
Shift down
Shift down
Shift down

Mr. Prayag Patel Department: IT/ICT-LJIET

 Insertion Sort
Step 5 :

-5 1 5 12 16 2 12 14
1 2 3 4 5 6 7 8

No Shift will take place

Step 6 :

-5 1 5
2 12 16 2 12 14
1 2 3 4 5 6 7 8

Shift down Shift down

Shift down

Mr. Prayag Patel Department: IT/ICT-LJIET

 Insertion Sort
Step 7 :

-5 1 2 12 12 14
5 12 16
1 2 3 4 5 6 7 8
Shift down

Step 8 :

-5 1 2 14 14
5 12 12 16
1 2 3 4 5 6 7 8

Shift down

Sorted List -5 1 2 5 12 12 14 16

Mr. Prayag Patel Department: IT/ICT-LJIET

 Insertion Sort Algorithm
# Input: Array T
# Output: Sorted array T
Algorithm: Insertion_Sort(T[1,…,n])
for i ← 2 to n do
x ← T[i];
j ← i – 1;
while x < T[j] and j > 0 do
T[j+1] ← T[j];
j ← j – 1;
T[j+1] ← x;

Mr. Prayag Patel Department: IT/ICT-LJIET

 Insertion Sort Analysis
# Input: Array T
# Output: Sorted array T
Algorithm: Insertion_Sort(T[1,…,n])
for i ← 2 to n do
x ← T[i];
j ← i – 1;
while x < T[j] and j > 0 do
T[j+1] ← T[j];
j ← j – 1;
T[j+1] ← x;

The time complexity of Insertion sort is

Mr. Prayag Patel Department: IT/ICT-LJIET

 Insertion Sort Best case Analysis
Pass 1 :
# Input: Array T
12
# Output: Sorted array T
23 i=2 T[j]=12
Algorithm: Insertion_Sort(T[1,…,n]) 34 i=3 T[j]=23
for i ← 2 to n do 45 i=4 T[j]=34
x ← T[i]; 59 i=5 T[j]=45

j ← i - 1;
while x < T[j] and j > 0 do
T[j+1] ← T[j];
j ← j – 1;
T[j+1] ← x;
The best case time complexity of Insertion sort is

Mr. Prayag Patel Department: IT/ICT-LJIET

 Insertion Sort - Example
• Sort given elements in descending order using insertion
sort: 3, 9, 6, 5, 23, 14
Step 1 : Step 5:

3 9 6 5 23 14 23
9 6 5 53 23 14

Step 2 :
Step 6 :
39 93 6 5 23 14
23 9
14 6 5 3 14
Step 3 :

39 36 63 5 23 14
Step 4 :

39 36 35 53 23 14

Mr. Prayag Patel Department: IT/ICT-LJIET

 HEAP SORT

Assistant Professor
IT/ICT Department, LJIET
Mr. Prayag Patel [email protected]
 Heap Sort
▪ A heap data structure is a binary tree with the following two
properties.
1. It is a complete binary tree: Each level of the tree is completely filled,
except possibly the bottom level. At this level it is filled from left to right.

a a

b c b c

d e f d e f

Binary Tree Complete Binary Tree - Heap

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort
2. It satisfies the heap order property: the data item stored in each node
is greater than or equal to the data item stored in its children node.

9 9

6 7 6 7

2 4 8 2 4 1

Not a Heap Heap

Mr. Prayag Patel Department: IT/ICT-LJIET

 Types of Heap Sort
1. Max-Heap − Where the value of the root 9
node is greater than or equal to either of
its children.
6 7

2 4 1
1
2. Min-Heap − Where the value of the root
node is less than or equal to either of its
2 4 children.

6 7 9

Mr. Prayag Patel Department: IT/ICT-LJIET

 Array Representation of Heap
heap
16

14 10

8 7 9 3

2 4 1
Array representation of heap

16 14 10 8 7 9 3 2 4 1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Steps of Heap Sort
1. Build the complete binary tree using given elements.
2. Create Max-heap to sort in ascending order.
3. Once the heap is created, swap the last node with the
root node and delete the last node from the heap.
4. Repeat step 2 and 3 until the heap is empty.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Example
▪ Sort given element in ascending order using heap sort.
19, 7, 16, 1, 14, 17
1 2 3 4 5 6
19 7 16 1 14 17
Step 1: Create complete binary tree

1
19

2 3 16
7

4 1 5 14 17
6

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Example
▪ Sort given element in ascending order using heap sort. 19, 7, 16,
1, 14, 17 1 2 3 4 5 6
19 7 16 1 14 17

Step 2: Create Max-heap Step 2: Create Max-heap

Step-3
1 1
19 19
Step-1
Step-2
2 7 3 16 3 17
2 14

4 1 5 14 17 1 5 7 16
6 4 6

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Example
Step 3: swap the last node with the root node and delete the
last node from the heap

1 2 3 4 5 6
19 14 17 1 7 16

1 19

Swap &
remove
2 14 3 17 the last
element
6
4 1 5 7 16

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Example
Step 2: Create Max-heap

1 2 3 4 5 6
1
16 14 17 1 7 19 16

2 14 17
3

4 5
1 7

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Example
Step 3: swap the last node with the Step 2: Create Max-heap
root node and delete the last node
from the heap

1 2 3 4 5 6 1 2 3 4 5 6
17 14 16 1 7 19 7 14 16 1 17 19

1 17 1 7
Swap &
remove
2 14 3 16 the last 14 3
16
2
element

4 1 5 7 4 1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Example
Step 3: swap the last node with the Step 2: Create Max-heap
root node and delete the last node
from the heap

1 2 3 4 5 6 1 2 3 4 5 6
16 14 7 1 17 19 1 14 7 16 17 19

1 16 1 1
Swap &
remove
14 3 7 14 3 7
2 the last 2
element

4 1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Example
Step 3: swap the last node with the Step 2: Create Max-heap
root node and delete the last node
from the heap

1 2 3 4 5 6 1 2 3 4 5 6
14 1 7 16 17 19 7 1 14 16 17 19

1 1 7
14 Swap &
remove
the last
element 2 1
2 1 3 7

Already a Max-heap
Mr. Prayag Patel Department: IT/ICT-LJIET
 Heap Sort Example
Step 3: swap the last node with the Step 2: Create Max-heap
root node and delete the last node
from the heap 1 2 3 4 5 6
1 7 14 16 17 19
1 2 3 4 5 6
1
7 1 14 16 17 19 1

1 7 Swap &
Remove the last element
remove
the last
element
2 1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Example

The final array is sorted

1 2 3 4 5 6
1 7 14 16 17 19

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Algorithm

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Algorithm
Algorithm: BUILD-MAX-HEAP(A) 4 1 3 2 9 7
heap-size[A] ← length[A] 1
4
for i ← ⌊length[A]/2⌋ downto 1
2 3
do MAX-HEAPIFY(A, i, n) 1 3
4 5 6
heap-size[A] = 6 2 9 7

4 1 7 2 9 3 4 9 7 2 1 3 9 4 7 2 1 3
i=3 1 i=2 1 i=1 1
4 4 94
2 3 2 3 2 3
1 37 9
1 7 49 7
4 5 6 4 5 6 4 5 6
2 9 3
7 2 19 3 2 1 3

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Algorithm
9 4 7 2 1 3
1
9
2 3
4 7

4 5 6
2 1 3

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Algorithm
3 4 7 2 1 9
1
3
2 3
4 7

4 5 6
2 1 9

Mr. Prayag Patel Department: IT/ICT-LJIET

 Max-Heapify - Algorithm
1
3
1 2 3
4 7
Yes
4 5
2 1

Yes

Yes

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Algorithm
1
7
2 3
4 3

4 5
2 1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Heap Sort Algorithm

Mr. Prayag Patel Department: IT/ICT-LJIET

 COUNTING SORT

Assistant Professor
IT/ICT Department, LJIET
Mr. Prayag Patel [email protected]
 Counting Sort
• Sort the following elements in an ascending order.

3 6 4 1 3 4 1 4 2

Step 1

Index
Elements 3 6 4 1 3 4 1 4 2

Step 2

Index
Elements 0 0 0 0 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Counting Sort

3 6 4 1 3 4 1 4 2

Step 3
Index
Elements 2 1 2 3 0 1

+ +
Step 4

Index
Elements 2 3 5 8 8 9

Mr. Prayag Patel Department: IT/ICT-LJIET

 Counting Sort

Input Array A 3 6 4 1 3 4 1 4 2

Temporary Array C 23
2 5 87 8 9

Output Array B 1 1 2 3 3 4 4 4 6

Mr. Prayag Patel Department: IT/ICT-LJIET

 Counting Sort

Input Array A 3 6 4 1 3 4 1 4 2

Temporary Array C 6
21 2
3 5 78 8 9

Output Array B 1 1 2 3 3 4 4 4 6

Mr. Prayag Patel Department: IT/ICT-LJIET

 Counting Sort

Input Array A 3 6 4 1 3 4 1 4 2

Temporary Array C 43 658

21
0 2
3 5 8 98

Output Array B 1 1 2 3 3 4 4 4 6
The final array is sorted

Mr. Prayag Patel Department: IT/ICT-LJIET

 Counting Sort Procedure

Mr. Prayag Patel Department: IT/ICT-LJIET

 Counting Sort Algorithm
# Input: Array A
# Output: Sorted array A
Algorithm: Counting-Sort(A[1,…,n], B[1,…,n], k)
for i ← 1 to k do
c[i] ← 0
for j ← 1 to n do
c[A[j]] ← c[A[j]] + 1
for i ← 2 to k do
c[i] ← c[i] + c[i-1]
for j ← n downto 1 do
B[c[A[j]]] ← A[j]
c[A[j]] ← c[A[j]] - 1

Mr. Prayag Patel Department: IT/ICT-LJIET

 THANK YOU FOR
WATCHING!

Mr. Prayag Patel [email protected]