Complexity of conventional Sorting

Algorithms
Course Code: CSC2211 Course Title: Algorithms

Dept. of Computer Science

Faculty of Science and Technology

Lecturer No: 03 Week No: 03 Semester: Fall 24-25

Lecturer: Dr. Mohammad Rabiul Islam. Email : [email protected]
Lecture Outline

1. Sorting Algorithms
 Merge Sort
Divide and Conquer

Recursive in structure

 Divide the problem into independent sub-problems that are

similar to the original but smaller in size
 Conquer the sub-problems by solving them recursively. If they
are small enough, just solve them in a straightforward manner.
This can be done by reducing the problem until it reaches the
base case, which is the solution.
 Combine the solutions of the sub-problems to create a solution
to the original problem
Merge Sort

Sorting Problem: Sort a sequence of n elements into non-

decreasing order.
 Divide: Divide the n-element sequence to be sorted into
two subsequences of n/2 elements each
 Conquer: Sort the two subsequences recursively using
merge sort.
 Combine: Merge the two sorted subsequences to produce
the sorted answer.
Execution Example

 Partition
7 2 9 43 8 6 1

6
Execution Example (cont.)

 Recursive call, partition

7 2 9 43 8 6 1

7 29 4

7
Execution Example (cont.)

 Recursive call, partition

7 2 9 43 8 6 1

7 29 4

72

8
Execution Example (cont.)

 Recursive call, base case

7 2 9 43 8 6 1

7 29 4

72

77

9
Execution Example (cont.)

 Recursive call, base case

7 2 9 43 8 6 1

7 29 4

72

77 22

10
Execution Example (cont.)

 Merge
7 2 9 43 8 6 1

7 29 4

722 7

77 22

11
Execution Example (cont.)

 Recursive call, …, base case, merge

7 2 9 43 8 6 1

7 29 4

722 7 9 4  4 9

77 22 99 44

12
Execution Example (cont.)

 Merge
7 2 9 43 8 6 1

7 29 4 2 4 7 9

722 7 9 4  4 9

77 22 99 44

13
Execution Example (cont.)

 Recursive call, …, merge, merge

7 2 9 43 8 6 1

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

14
Execution Example (cont.)

 Merge
7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

15
 7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

16
 Merge Sort Algorithm
Step 1 − if it is only one element in the list it is already sorted, return.
Step 2 − divide the list recursively into two halves until it can no more be divided.
Step 3 − merge the smaller lists into new list in sorted order.
Merge Sort Algorithm – Simulation understanding in Lab
Merge Sort Algorithm
Merge(L,R,A) {
nL = length(L)
Mergesort(A) nR = length(R)
{ i= j = k = 0
n = length(A) while(i< nL && j< nR) {
if(n<2) if(L[i] <= R[j]) {
return A[k] = L[i]
mid = n/2 i= i+1
L = array of size(mid) }
R = array of size(n-mid) else {
for(i=0; i < mid; i++) A[k] = R[j]
L[i] = A[i] j= j+1
for(i=mid; i < n; i++) }
R[i-mid] = A[i] k= k+1
Mergesort(L) }
Mergesort(R) while( i<nL) {
Merge(L , R , A) A[k] = L[i]; i=i+1;
} k=k+1;
}
while (j<nR) {
A[k] = R[j]; j=j+1;
k=k+1;
}
}
Merge(L,R,A) {
nL = length(L)
nR = length(R)
i= j = k = 0
while(i< nL && j< nR) {
if(L[i] <= R[j]) {
A[k] = L[i] n*c0+ n*c1+n*c2+ c4
}
i= i+1
=n*(c0+c1+c2)+ c4
else { =n*c3 + c4
A[k] = R[j]
j= j+1
=n (Linear time)
} Because of having no nested
k= k+1
} loops
while( i<nL) {
A[k] = L[i]; i=i+1;
k=k+1;
}
while (j<nR) {
A[k] = R[j]; j=j+1;
k=k+1;
}
}
nL = length(L)
nR = length(R)
i= j = k = 0

i=0 j=0

L 14 23 45 98 R 6 33 42 67
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=0 j=0

L 14 23 45 98 R 6 33 42 67

A
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=0 j=1

L 14 23 45 98 R 6 33 42 67

A 6

k=0
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=1 j=1

L 14 23 45 98 R 6 33 42 67

A 6 14

k=1
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=2 j=1

L 14 23 45 98 R 6 33 42 67

A 6 14 23

k=2
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1;
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=2 j=2

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33

k=3
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=2 j=3

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42

k=4
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=3 j=3

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45

k=5
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=3 j=4

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45 67

k=6
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=3 j=4

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45 67

k=6
while( i<nL) {
A[k] = L[i];
i=i+1;
k=k+1;
}
while (j<nR) {
A[k] = R[i];
j=j+1;
k=k+1;
}
i=3 j=4

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45 67

k=6
while( i<nL) {
A[k] = L[i];
i=i+1;
k=k+1;
}
while (j<nR) {
A[k] = R[i];
j=j+1;
k=k+1;
}
i=4 j=4

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45 67 98

k=7
 Exercise

98 23 45 14 6 67 33 42
Merge Sort Analysis for Time Complexity

1. The divide step takes constant time, regardless of the

subarray size. After all, the divide step just computes the
midpoint q of the indices p and r. Recall that in big-O
notation, we indicate constant time by O(1).

2. The conquer step, where we recursively sort two

subarrays of approximately n/2 elements each, takes
some amount of time, but we'll account for that time
when we consider the subproblems.

3. The combine step merges a total of n elements, taking

O(n) time.
Merge Sort Analysis
Substitution Method
Mergesort(A) T(n), to sort n
{
elements
n = length(A) c1
if(n<2) return
mid = n/2
L = array of size(mid)
R = array of size(n-mid)

for(i=0; i < mid; i++) c2* (n/2) + c2* (n/2)

L[i] = A[i] =c2( n/2 + n/2)
for(i=mid; i < n; i++) =c2 * n
R[i-mid] = A[i]

Mergesort(L) T(n/2)
Mergesort(R) T(n/2)
Merge(L , R , A) c3.n+c4
}

So, T(n) = O(1) when n = 1, and

2T(n/2) + (c2+c3)*n + (c1+c4) when n > 1
So

Skipping the insignificant parts,

[]
[k= ]

[= n]
 Merge Sort Analysis
Recursion Tree Method
Recursion-tree Method

 Show successive expansions of recurrences using

trees.
 Keep track of the time spent on the subproblems of a
divide and conquer algorithm.
 Help organize the algebraic bookkeeping necessary to
solve a recurrence.
Recursion-tree Method for Merge Sort

For the original problem, Each of the size n/2 problems

we have a cost of cn, plus has a cost of cn/2 plus two
two subproblems each of subproblems, each costing
size (n/2) and running time T(n/4).
T(n/2).
cn

cn
Cost of divide
and merge.
cn/2 cn/2

T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of sorting
subproblems.
Recursion Tree for
Merge Sort
Continue expanding until the problem size reduces to 1.
cn •Each level has total cost cn.

•Each time we go down one level, the number of

sub-problems doubles, but the cost per sub-
cn/2 cn/2
problem halves  cost per level remains the
same.

cn/4 cn/4 cn/4 cn/4 •There are lg n + 1 levels, height is lg n.

(Assuming n is a power of 2.)

•Total cost = sum of costs at each level = (lg n +

1)cn = cnlgn + cn = O(n lgn).
c c c c c c
Recursion-tree Method for Merge Sort
Elements to sort /
Recursion Tree for
Merge Sort
n

𝒏 𝒏 =c.n
𝟐 𝟐

𝒏 𝒏 𝒏 𝒏
𝟒 𝟒 𝟒 𝟒 =c.n

𝒏 𝒏 𝒏 𝒏 𝒏 𝒏 𝒏 𝒏
𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 =c.n
Recursion Tree for
Merge Sort
n=8

𝒏 𝒏
c.n
𝟐 𝟐

𝒏 𝒏 𝒏 𝒏
=c.n
𝟒 𝟒 𝟒 𝟒

𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖
=1 =1 =1 =1 =1 =1 =1 =1
𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 =c.n

Continue expanding until the problem size reduces to 1.

So , if n=8 we need 3cn
n=8 3 levels we need 3.cn

n=16 4 levels we need 4.cn

n=32 5 levels we need 5.cn

n=64 6 levels we need 6.cn

n=8 n= 3 we need
levels 3.cn
n=16 n= 4 we need
levels 4.cn
n=32 n= 5 we need
levels 5.cn
n=64 n= 6 we need
levels 6.cn
n=8 n= 3 levels we need 3.cn
n=16 n= 4 levels we need 4.cn
n=32 n= 5 levels we need 5.cn
n=64 n= 6 levels we need 6.cn
n n= levels . cn
Merge Sort Analysis
Master Method
The master method applies to recurrences of the form
T(n) = a T(n/b) + f (n) ,
where a ≥ 1, b > 1, and f is asymptotically positive.
Examples:
Examples:
Examples:

Þ and
Examples:
Examples:
Best Case Time Complexity: 𝛀(n*log n)
Worst Case Time Complexity: O(n*log n)
Average Time Complexity: ϴ(n*log n)

The time complexity of MergeSort is O(n*Log n) in all the 3

cases (worst, average and best) as the mergesort always
divides the array into two halves and takes linear time to
merge two halves.
Books

 Introduction to Algorithms, Thomas H. Cormen, Charle E.

Leiserson, Ronald L. Rivest, Clifford Stein (CLRS).

 Fundamental of Computer Algorithms, Ellis Horowitz, Sartaj

Sahni, Sanguthevar Rajasekaran (HSR)
References

 https://www.google.com/search?q=bubble+sort+step+by+step&sxsrf=AL
eKk01uxzgfT3Oy6k1Q3WxVnSpiIN8_4g:1587999728942&tbm=isch&sourc
e=iu&ictx=1&fir=vRwFsGwVfJ6pJM%253A%252CSzhhze6MPQr4cM%252C
_&vet=1&usg=AI4_-kSrEEXqwRL-PkHhVUtn7jNfF9dB6g&sa=X&ved=2ahU
KEwje0Pz974jpAhXRAnIKHWhMD2UQ_h0wAXoECAcQBg#imgrc=EN4Sdu7
veOWVoM&imgdii=eOqvCu85p9-eBM

 https://www.interviewcake.com/concept/java/counting-sort

 https://www.geeksforgeeks.org/counting-sort/

 https://www.hackerearth.com/practice/algorithms/sorting/quick-sort/tut
orial/