0% found this document useful (0 votes)

19 views626 pages

DAA Part 1

Uploaded by

anantdrive2103

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

19 views626 pages

DAA Part 1

Uploaded by

anantdrive2103

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 626

Design and Analysis of Algorithm

(KCS503)

Introduction to Algorithms and its

analysis mechanism

Lecture - 1
Overview
• Provide an overview of algorithms and analysis.
• Try to touching the distinguishing features and
the significance of analysis of algorithm.
• Start using frameworks for describing and
analysing algorithms.
• See how to describe algorithms in pseudo code in
the context of real world software development.
• Begin using asymptotic notation to express
running-time analysis with Examples.
What is an Algorithm ?
• An algorithm is a finite set of rules that gives a
sequence of operations for solving a specific
problem
• An algorithm is any well defined computational
procedure that takes some value or set of
values, as input and produces some value or set
of values as output.
• We can also view an algorithm as a tool for
solving a well specified computational problem.
Characteristics of an Algorithm
• Input: provided by the user.
• Output: produced by algorithm.
• Definiteness: clearly define.
• Finiteness: It has finite number of steps.
• Effectiveness: An algorithm must be
effective so that it’s output can be carried
out with the help of paper and pen.
Analysis of an Algorithm
• Loop Invariant technique was done in three
steps:
– Initialization
– Maintenance
– Termination
• It deals with predicting the resources that an
algorithm requires to its completion such as
memory and CPU time.
• To main measure for the efficiency of an algorithm
are Time and space.
Complexity of an Algorithm
• Complexity of an Algorithm is a function, f(n)
which gives the running time and storage
space requirement of the algorithm in terms
of size n of the input data.
• Space Complexity: Amount of memory
needed by an algorithm to run its
completion.
• Time complexity: Amount of time that it
needs to complete itself.
Cases in Complexity Theory

• Best Case: Minimum time

• Worst Case: Maximum amount of time
• Average Case: Expected / Average value
of the function f(n).
Example 1
#include <stdio.h>

int main()
{
printf("Hello PSIT");
return 0;
}
Complexity of Example 1
#include <stdio.h>

int main()
{
printf("Hello PSIT");
return 0;
}

𝑇(𝑛) = 𝑂(1)
Example 2
#include <stdio.h>
void main()
{
int i, n = 8;
for (i = 1; i <= n; i++) {
printf("Hello PSIT !!!\n");
}
}
Complexity of Example 2
#include <stdio.h>
void main()
{
int i, n = 8;
for (i = 1; i <= n; i++) {
printf("Hello PSIT !!!\n");
}
}

𝑇(𝑛) = 𝑂(𝑛)
Example 3
#include <stdio.h>
void main()
{
int i, n = 8;
for (i = 1; i <= n; i=i*2) {
printf("Hello PSIT !!!\n");
}
}
Complexity of Example 3
#include <stdio.h>
void main()
{
int i, n = 8;
for (i = 1; i <= n; i=i*2) {
printf("Hello PSIT !!!\n");
}
}

𝑇 𝑛 = log 2 𝑛
Example 4
#include <stdio.h>
#include <math.h>
void main()
{
int i, n = 8;
for (i = 2; i <= n; i=pow(i,2)) {
printf("Hello PSIT !!!\n");
}
}
Complexity of Example 4
#include <stdio.h>
#include <math.h>
void main()
{
int i, n = 8;
for (i = 2; i <= n; i=pow(i,2)) {
printf("Hello PSIT !!!\n");
}
}

𝑇 𝑛 = Ο(log 2 (log 2 𝑛))

Example 5
A()
{
int i;
for(i=1;i<=n;i++)
{
printf(“ABCD”);
}
}
Complexity of Example 5
A()
{
int i;
for(i=1;i<=n;i++)
{
printf(“ABCD”);
}
}

𝑇 𝑛 = 𝛰(𝑛)
Example 6
A()
{
int i=1 ,s=1;
scanf(“%d”, &n);
while(s<=n)
{
i++;
s=s+i;
printf(“abcd”);
}
}
Complexity of Example 6
A()
{
int i=1 ,s=1;
scanf(“%d”, &n);
while(s<=n)
{
i++;
s=s+i;
printf(“abcd”);
}
} 𝑇 𝑛 = 𝛰( 𝑛)
Example 7
A()
{
int i=1;
for(i=1; 𝑖 2 <=n; i++)
{
printf(“abcd”);
}
}
Complexity of Example 7
A()
{
int i=1;
for(i=1; 𝑖 2 <=n; i++)
{
printf(“abcd”);
}
}
𝑇 𝑛 = 𝛰( 𝑛)
Example 8
A()
{
int i=1;
for (i=1; i≤n; i++)
{
for (j=1; j≤ 𝑖 2 ; j++)
{
for (k=1; k≤n/2; k++)
{
printf(“abcd”);
}
}
}
}
Complexity of Example 8
𝑇 𝑛 = Ο 𝑛4

Explanation:
𝐼 = 1, 2, 3, 4, 5, … … … … … … … … … … … .
𝐽 = 1, 4, 9, 16, 25, … … … … … … … … … …
𝑛 4𝑛 9𝑛 16𝑛
𝐾= , , , , …………………………
2 2 2 2
Complexity of Example 8
𝑇 𝑛 = Ο 𝑛4

Explanation:
𝐼 = 1, 2, 3, 4, 5, … … … … … … … … … … … .
𝐽 = 1, 4, 9, 16, 25, … … … … … … … … … …
𝑛 4𝑛 9𝑛 16𝑛
𝐾= , , , , ………………………..
2 2 2 2

𝑺𝒖𝒎 𝒐𝒇 𝒔𝒒𝒖𝒂𝒓𝒆 𝒐𝒇 𝑵𝒂𝒕𝒖𝒓𝒂𝒍 𝑵𝒖𝒎𝒃𝒆𝒓.

= [𝒏(𝒏 + 𝟏)(𝟐𝒏 + 𝟏)]/𝟔
Design and Analysis of Algorithm
(KCS503)

Analysis of Linear Search

Lecture - 2
Searching

• Searching is a process of finding a

particular element among several given
elements.
• The search is successful if the required
element is found.
• Otherwise, the search is unsuccessful.
Searching
Searching Algorithms are a family of algorithms
used for the purpose of searching.

The searching of an element in the given array may

be carried out in the following two ways-

• Linear Searching
• Binary Searching
Linear Searching
To understand the working of linear search algorithm,
let's take an unsorted array. It will be easy to understand
the working of linear search with an example.

Let the elements of array are :

Let the element to be searched is K = 41

Linear Searching
Now, start from the first element and
compare K=41 with each element of the array.

The value of K=41, is

not matched with the
first element of the
array. So, move to the
next element. And
follow the same
process until the
respective element is
found.
Searching
Now, start from the first element and
compare K=41 with each element of the array.
Searching
Now, start from the first element and
compare K=41 with each element of the array.
Searching
Now, start from the first element and
compare K=41 with each element of the array.
Searching
Now, start from the first element and
compare K=41 with each element of the array.
Searching
Now, start from the first element and
compare K=41 with each element of the array.
Searching
Now, the element to be searched is found. So, algorithm
will return the index of the element matched.
Linear Searching
What Is Linear Search?

• A linear Search is the simplest searching

algorithm.
• It traverses the array sequentially to locate the
required element.
• It searches for an element by comparing it with
each element of the array one by one.
• So, it is also called as Sequential Search.
Linear Searching

When Linear Search Algorithm is applied-

• No information is given about the array.

• The given array is unsorted, or the elements are
unordered.
• The list of data items is smaller.
Linear Searching
Linear Search Algorithm –

Linear_Search (a , n , item , loc)

Begin
for i = 0 to (n - 1) by 1 do
if (a[i] = item) then
set loc = i
exit
endif
end for
set loc = -1
End
Linear Searching
Time Complexity Analysis- (Best case)

In the best possible case,

• The element being searched may be found at the

first position.
• In this case, the search terminates in success
with just one comparison.
• Thus in best case, linear search algorithm takes
O(1) operations.
Linear Searching
Time Complexity Analysis- (Worst Case)
In the worst possible case,

• The element being searched may be present at

the last position or not present in the array at all.
• In the former case, the search terminates in
success with n comparisons.
• In the later case, the search terminates in failure
with n comparisons.
• Thus in worst case, linear search algorithm takes
O(n) operations.
Linear Searching
Time Complexity Analysis- (Worst Case)
In the worst possible case,(Mathematically)
𝑇 𝑛 = 𝑂(1) + 𝑇(𝑛 − 1)
Linear Searching
Time Complexity Analysis- (Worst Case)
In the worst possible case,(Mathematically)
𝑇 𝑛 = 𝑂(1) + 𝑇(𝑛 − 1)
= 𝑂(1) + 𝑂(1) + 𝑇(𝑛 − 2)
Linear Searching
Time Complexity Analysis- (Worst Case)
In the worst possible case,(Mathematically)
𝑇 𝑛 = 𝑂(1) + 𝑇(𝑛 − 1)
= 𝑂(1) + 𝑂(1) + 𝑇(𝑛 − 2)
= 𝑂(1) + 𝑂(1) + 𝑂(1) + 𝑇(𝑛 − 3)
= 𝑂(1) + 𝑂(1) + 𝑂(1) + 𝑂(1) + 𝑇(𝑛 − 4)
after n times of iteration
= 𝑂 1 + 𝑂 1 + 𝑂 1 + 𝑂 1 + ⋯ +𝑂 1 + 𝑇(0)
=𝑂 𝑛 +𝑂 0
=𝑂 𝑛
Linear Searching
Time Complexity Analysis-

Thus, we can say that in general:

The time Complexity of Linear Search Algorithm is

O(n). where, n is the number of elements in the
linear array.
Design and Analysis of Algorithm
(KCS503)

Definition of Sorting problem through

exhaustive search and analysis of
Selection Sort through iteration Method

Lecture -3
A Sorting Problem
(Exhaustive Search Approach)

Input: A sequence of n numbers 𝑨𝟏, 𝑨𝟐, . . . , 𝑨𝒏.

Output: A permutation (reordering) 𝑨𝟏 , 𝑨𝟐 , . . . , 𝑨𝒏 of the

input sequence such that 𝑨𝟏 ≤ 𝑨𝟐 ≤ … ≤ 𝑨𝒏 .

The sequences are typically stored in arrays.

A Sorting Problem
(Exhaustive Search Approach)
Let there be a set of four digits and note that there are multiple
possible permutations for the four digits. They are:
1 2 34 2134 3124 41 23
1 2 43 2143 3142 41 32
1 3 24 2314 3214 42 13
1 3 42 2341 3241 42 31
1 4 23 2431 3412 43 12
1 4 32 2413 3421 43 21
• There are 24 different permutations possible. (as shown
above)
• Only one of these permutations meets our criteria.
(i.e. A1 ≤ A2 ≤ …≤ An) .
•
A Sorting Problem
(Exhaustive Search Approach)
How to generate the 24 permutation ?

2 3 4

3 4 2 4 2 3

4 3 4 2 3 2
A Sorting Problem
(Exhaustive Search Approach)
How to generate the 24 permutation ?

1 2

2 3 4 1 3 4

3 4 2 4 2 3 3 4 1 4 1 3

4 3 4 2 3 2 4 3 4 1 3 1
A Sorting Problem
(Exhaustive Search Approach)
How to generate the 24 permutation ?

3 4

1 2 4 1 3 2

2 4 1 4 2 1 3 2 1 2 1 3

4 2 4 1 1 2 2 3 2 1 3 1
A Sorting Problem
(Exhaustive Search Approach)
An in-depth look at the analysis of the 24 permutations of four digits
A Sorting Problem
(Exhaustive Search Approach)
Let there be a set of four digits and note that there are multiple
possible permutations for the four digits. They are:
1234 2134 3124 41 23
1243 2143 3142 41 32
1324 2314 3214 42 13
1342 2341 3241 42 31
1423 2431 3412 43 12
1432 2413 3421 43 21
• There are 24 different permutations possible. (as shown
above)
• Only one of these permutations meets our criteria.
(i.e. A1 ≤ A2 ≤ …≤ An) . (1 2 3 4)
A Sorting Problem
(Exhaustive Search Approach)
How we do this ?
Step 1: Generate all the permutation and store it.
Step 2: Check all the permutation one by one and find which
permutation is satisfying the required condition (i.e. a1 ≤ a2 ≤
・・・ ≤ an).
Step 3: Once we get it , we got the victory.
A Sorting Problem
(Exhaustive Search Approach)
How we do this ?
Step 1: Generate all the permutation and store it.
Step 2: Check all the permutation one by one and find which
permutation is satisfying the required condition (i.e. a1 ≤ a2 ≤
・・・ ≤ an).
Step 3: Once we get it , we got the victory.

How we do this in algo based?

For each permutation P ∈ set of n! permutations:
if (a1 ≤ a2 ≤ ・・・ ≤ an) == permutation set[p]:
print (permutation set[p])
A Sorting Problem
(Exhaustive Search Approach)
How we do this ?
Step 1: Generate all the permutation and store it.
Step 2: Check all the permutation one by one and find which
permutation is satisfying the required condition (i.e. a1 ≤ a2 ≤
・・・ ≤ an).
Step 3: Once we get it , we got the victory. Exhaustive
Search
How we do this in algo based?
For each permutation P ∈ set of n! permutations:
if (a1 ≤ a2 ≤ ・・・ ≤ an) == permutation set[p]:
print (permutation set[p])
A Sorting Problem
(Exhaustive Search Approach)
How we do this ?
Step 1: Generate all the permutation and store it.
Step 2: Check all the permutation one by one and find
which permutation is satisfying the required condition
(i.e. a1 ≤ a2 ≤ ・・・ ≤ an).
Step 3: Once we get it , we got the victory. Exhaustive
Search
How we do this in algo based?
For each permutation P ∈ set of n! permutations:
if (a1 ≤ a2 ≤ ・・・ ≤ an) == permutation set[p]:
print (permutation set[p])
𝑪𝒐𝒎𝒑𝒍𝒆𝒙𝒊𝒕𝒚
= 𝚶 𝒏! ∗ 𝒏 𝒕𝒊𝒎𝒆
A Sorting Problem
(Selection Sort Approach)
Selection sort is a simple and efficient sorting algorithm that works by
repeatedly selecting the smallest (or largest) element from the
unsorted portion of the list and moving it to the sorted portion of the
list.
A Sorting Problem
(Selection Sort Approach)
Selection sort is a simple and efficient sorting algorithm that works by
repeatedly selecting the smallest (or largest) element from the
unsorted portion of the list and moving it to the sorted portion of the
list.

Lets consider the following array as an example:

A [] = (7, 4, 3, 6, 5).
A Sorting Problem
(Selection Sort Approach)

Lets consider the following array as an example:

A [] = (7, 4, 3, 6, 5).

• For the first position in the sorted array, the whole array is traversed
from index 0 to 4 sequentially. After going through the entire array, it is
evident that 3 is the lowest value, with 7 being stored at the first
position.
• Thus, replace 7 with 3. At the end of the first iteration, the item with
the lowest value, in this case 3, at position 2 is most likely to be at the
top of the sorted list.
A Sorting Problem
(Selection Sort Approach)
Lets consider the following array as an example:
A [] = (7, 4, 3, 6, 5).

• 1st Iteration
A Sorting Problem
(Selection Sort Approach)

Lets consider the updated array as an example:

A [] = (3, 4, 7, 6, 5).

• For the second position, where 25 is present, again traverse the rest of
the array in a sequential manner.
• Using the traversal method, we determined that the value 12 is the
second-lowest in the array and thus should be placed in the second
position. So no need of swapping.
A Sorting Problem
(Selection Sort Approach)
Lets consider the following array as an example:
A [] = (7, 4, 3, 6, 5).

• 2nd Iteration
A Sorting Problem
(Selection Sort Approach)

Lets consider the updated array as an example:

A [] = (3, 4, 7, 6, 5).

• For the third position, where 7 is present, again traverse the rest of the
array in a sequential manner.
• Using the traversal method, we determined that the value 5 is the third-
lowest in the array and thus should be placed in the third position.
A Sorting Problem
(Selection Sort Approach)
Lets consider the following array as an example:
A [] = (7, 4, 3, 6, 5).

• 3rd Iteration
A Sorting Problem
(Selection Sort Approach)

Lets consider the updated array as an example:

A [] = (3, 4, 5, 6, 7).

• Similarly we execute it for fourth and fifth iteration and finally the sorted
array is looks like as below:
A Sorting Problem
(Selection Sort Algorithm)
SELECTION SORT(arr, n)

Step 1: Repeat Steps 2 and 3 for

i = 0 to n-1
Step 2: CALL SMALLEST(arr, i, n, SMALLEST (arr, i, n, pos)
pos) Step 1: [INITIALIZE] SET SMALL = arr[i]
Step 3: SWAP arr[i] with arr[pos] Step 2: [INITIALIZE] SET pos = i
[END OF LOOP] Step 3: Repeat for j = i+1 to n
Step 4: EXIT if (SMALL > arr[j])
SET SMALL = arr[j]
SET pos = j
[END OF if]
[END OF LOOP]
Step 4: RETURN pos
Use C programming language to convert the above into a programme
A Sorting Problem
(Selection Sort Complexity)
Input: Given n input elements.
Output: Number of steps incurred to sort a list.
Logic: If we are given n elements, then in the first pass, it will do n-
1 comparisons; in the second pass, it will do n-2; in the third pass, it will
do n-3 and so on. Thus, the total number of comparisons can be found by;
A Sorting Problem
(Selection Sort Complexity)

• Best Case Complexity: The selection sort algorithm has a best-case

time complexity of O(n2) for the already sorted array.
• Average Case Complexity: The average-case time complexity for the
selection sort algorithm is O(n2), in which the existing elements are in
jumbled ordered, i.e., neither in the ascending order nor in the
descending order.
• Worst Case Complexity: The worst-case time complexity is
also O(n2), which occurs when we sort the descending order of an array
into the ascending order.
Thank You
Design and Analysis of Algorithm
(KCS503)

Insertion Sort and its Analysis

Lecture - 4
A Sorting Problem (Incremental Approach)
Input: A sequence of n numbers a1, a2, . . . , an.
Output: A permutation (reordering) a1 , a2 , . . . , an of the input
sequence such that a1 ≤ a2 ≤ ・・・ ≤ an .

The sequences are typically stored in arrays.

We also refer to the numbers as keys. Along with each key may be
additional information, known as satellite data. (You might want to clarify
that .satellite data. does not necessarily come from a satellite!)

We will see several ways to solve the sorting problem. Each way will be
expressed as an algorithm: a well-defined computational procedure that
takes some value, or set of values, as input and produces some value, or
set of values, as output.
Insertion sort
• A good algorithm for sorting a small number of elements.

• It works the way you might sort a hand of playing cards:

– Start with an empty left hand and the cards face down on the
table.
– Then remove one card at a time from the table, and insert it
into the correct position in the left hand.
– To find the correct position for a card, compare it with each of
the cards already in the hand, from right to left.
– At all times, the cards held in the left hand are sorted, and
these cards were originally the top cards of the pile on the
table.
Insertion sort (Example)
Insertion sort (Example)

1st Iteration
Insertion sort (Example)

1st Iteration 2nd Iteration

Insertion sort (Example)

1st Iteration 2nd Iteration 3rd Iteration

Insertion sort (Example)

1st Iteration 2nd Iteration 3rd Iteration

4th Iteration
Insertion sort (Example)

1st Iteration 2nd Iteration 3rd Iteration

4th Iteration 5th Iteration

Insertion sort (Example)

1st Iteration 2nd Iteration 3rd Iteration

4th Iteration 5th Iteration 6th Iteration

Insertion sort (Example)
Insertion sort (Algorithm)
Insertion sort (Algorithm)
Correctness Proof of Insertion Sort
• Initialization: Just before the first iteration, j = 2. The sub array
A[1 . . j − 1] is the single element A[1], which is the element originally
in A[1], and it is trivially sorted.

• Maintenance: To be precise, we would need to state and prove a loop

invariant for the “inner” while loop. Rather than getting bogged down
in another loop invariant, we instead note that the body of the inner
while loop works by moving A[ j − 1], A[ j − 2], A[ j − 3], and so on, by
one position to the right until the proper position for key (which has the
value that started out in A[ j ]) is found. At that point, the value of key is
placed into this position.

• Termination: The outer for loop ends when j > n; this occurs when j
= n + 1. Therefore, j −1 = n. Plugging n in for j −1 in the loop invariant,
the sub array A[1 . . n] consists of the elements originally in A[1 . . n]
but in sorted order. In other words, the entire array is sorted!
How do we analyze an algorithm's running
time?
• Input size: Depends on the problem being studied.

– Usually, the number of items in the input. Like the size n of the
array being sorted.
– But could be something else. If multiplying two integers, could
be the total number of bits in the two integers.
– Could be described by more than one number. For example,
graph algorithm running times are usually expressed in terms of
the number of vertices and the number of edges in the input
graph.
• Running time: On a particular input, it is the number
of primitive operations (steps) executed.
– Want to define steps to be machine-independent.
– Figure that each line of pseudo code requires a constant
amount of time.
– One line may take a different amount of time than another,
but each execution of line i takes the same amount of time
ci .
– This is assuming that the line consists only of primitive
operations.
• If the line is a subroutine call, then the actual call takes
constant time, but the execution of the subroutine being
called might not.
• If the line specifies operations other than primitive ones,
then it might take
• more than constant time.
Analysis of insertion sort

• Assume that the i th line takes time ci , which is a

constant. (Since the third line is a comment, it
takes no time.)
• For j = 2, 3, . . . , n, let tj be the number of times
that the while loop test is executed for that value
of j .
• Note that when a for or while loop exits in the
usual way-due to the test in the loop header-the
test is executed one time more than the loop body.
Running Time of Insertion Sort
Home Assignment

• Write the algorithm of Bubble sort and

calculate the time complexity.
Design and Analysis of Algorithm

Growth of Functions

Lecture –5,6, and 7

Overview
• A way to describe behaviour of functions in the limit. We’re
studying Asymptotic efficiency.
• Describe growth of functions.(i.e. The order of growth of the
running time of an algorithm)
• Focus on what’s important by abstracting away low-order
terms and constant factors.
• How we indicate running times of algorithms.
• A way to compare “sizes” of functions through different
notations (i.e. Asymptotic Notations):
𝜪 ≈ ≤
𝜴 ≈ ≥
𝜣 ≈ =
𝝄 ≈ <
𝝎 ≈ >
Asymptotic notation (Big Oh )
Asymptotic notation (Big Oh )
Asymptotic notation (Big Oh )
Example 1
𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛 + 3 ∈ 𝑂(𝑛)
Asymptotic notation (Big Oh )
Example 1
𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛 + 3 ∈ 𝑂(𝑛)
⟹ 2𝑛 + 3 ≤ 𝑐𝑔 𝑛
⟹ 2𝑛 + 3 ≤ 𝑐 𝑛
⟹ 2𝑛 + 3 ≤ 5𝑛, 𝑛≥1
𝐻𝑒𝑛𝑐𝑒𝑓 𝑛 = 𝑂 𝑛
Asymptotic notation (Big Oh )
Example 1
𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛 + 3 ∈ 𝑂(𝑛)
⟹ 2𝑛 + 3 ≤ 𝑐𝑔 𝑛
⟹ 2𝑛 + 3 ≤ 𝑐 𝑛
⟹ 2𝑛 + 3 ≤ 5𝑛, 𝑛≥1
𝐻𝑒𝑛𝑐𝑒𝑓 𝑛 = 𝑂 𝑛

𝐹𝑜𝑟, 𝑓 𝑛 = 𝑂 𝑛2 𝑖𝑠 𝑎𝑙𝑠𝑜 𝑡𝑟𝑢𝑒

𝐹𝑜𝑟, 𝑓 𝑛 = 𝑂 2𝑛 𝑖𝑠 𝑎𝑙𝑠𝑜 𝑡𝑟𝑢𝑒
Asymptotic notation (Big Oh )
Example 1
𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛 + 3 ∈ 𝑂(𝑛)
⟹ 2𝑛 + 3 ≤ 𝑐𝑔 𝑛
⟹ 2𝑛 + 3 ≤ 𝑐 𝑛
⟹ 2𝑛 + 3 ≤ 5𝑛, 𝑛≥1
𝐻𝑒𝑛𝑐𝑒𝑓 𝑛 = 𝑂 𝑛
𝐹𝑜𝑟, 𝑓 𝑛 = 𝑂 𝑛2 𝑖𝑠 𝑎𝑙𝑠𝑜 𝑡𝑟𝑢𝑒
𝐹𝑜𝑟, 𝑓 𝑛 = 𝑂 2𝑛 𝑖𝑠 𝑎𝑙𝑠𝑜 𝑡𝑟𝑢𝑒

𝐵𝑢𝑡 𝑓𝑜𝑟, 𝑓 𝑛 = 𝑂 lg 𝑛 𝑖𝑠 𝑛𝑜𝑡 𝑡𝑟𝑢𝑒

Because
𝟏 < 𝒍𝒈 𝒏 < 𝒏 𝒏 < 𝒏𝟐 < 𝒏𝟑 < ⋯ < 𝟐𝒏 < 𝟑𝒏 < ⋯ < 𝒏𝒏
Asymptotic notation (Big Oh )
Example 1 (Second Method)
𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛 + 3 ∈ 𝑂(𝑛)
Hear 𝑓(𝑛) = 2𝑛 + 3 𝑎𝑛𝑑 𝑔(𝑛) = 𝑛
So as per the definition of Big Oh
If 𝑓 𝑛 = Ο 𝑔 𝑛 , 𝑡ℎ𝑒𝑛
𝑓(𝑛)
lim ≤ 𝐶 ∀ 𝑐 > 0, 𝑛0 ≥ 1
𝑛⟶∞ 𝑔(𝑛)
Asymptotic notation (Big Oh )
Example 1 (Second Method)
𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛 + 3 ∈ 𝑂(𝑛)
Hear 𝑓(𝑛) = 2𝑛 + 3 𝑎𝑛𝑑 𝑔(𝑛) = 𝑛
So as per the definition of Big Oh
If 𝑓 𝑛 = Ο 𝑔 𝑛 , 𝑡ℎ𝑒𝑛
𝑓(𝑛)
lim ≤ 𝐶 ∀ 𝑐 > 0, 𝑛0 ≥ 1
𝑛⟶∞ 𝑔(𝑛)
2𝑛 + 3
lim
𝑛⟶∞ 𝑛
3
𝑛(2+𝑛) 3 3
= lim = lim (2 + ) = lim (2 + ) = lim 2 + 0= lim 2
𝑛⟶∞ 𝑛 𝑛⟶∞ 𝑛 𝑛⟶∞ ∞ 𝑛⟶∞ 𝑛⟶∞
Is a constant, Hence Proved
Asymptotic notation (Big Oh )
Example 2

𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛2 + 3𝑛 + 4 ∈ 𝑂 𝑛2

Asymptotic notation (Big Oh )
Example 2

𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛2 + 3𝑛 + 4 ∈ 𝑂 𝑛2

⟹ 2𝑛2 + 3𝑛 + 4 ≤ 2𝑛2 + 3𝑛2 + 4𝑛2
⟹ 2𝑛2 + 3𝑛 + 4 ≤ 11𝑛2 , 𝑤ℎ𝑒𝑟𝑒 𝑐 = 11 𝑎𝑛𝑑 𝑛 ≥ 1
𝐻𝑒𝑛𝑐𝑒𝑓 𝑛 = 𝑂 𝑛2
Asymptotic notation (Big Oh )
Example 3
𝐼𝑓 𝑓 𝑛 = 2𝑛+1 𝑎𝑛𝑑 𝑔 𝑛 = 2𝑛 𝑡ℎ𝑒 𝑝𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 ∈ Ο(𝑔 𝑛 )
Asymptotic notation (Big Oh )
Example 3
𝐼𝑓 𝑓 𝑛 = 2𝑛+1 𝑎𝑛𝑑 𝑔 𝑛 = 2𝑛 𝑡ℎ𝑒 𝑝𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 ∈ Ο(𝑔 𝑛 )
⟹ 2𝑛+1 = 2𝑛 . 2
So, as per the definition of Big Oh
𝑓 𝑛 ≤ 𝑐𝑔(𝑛)
Hence
⟹ 2𝑛+1 ≤ 2𝑛 . 2
⟹ 2𝑛+1 ≤ 2. 2𝑛 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑛 ≥ 1 𝑎𝑛𝑑 𝑐 > 1
𝐻𝑒𝑛𝑐𝑒, 𝑓 𝑛 ∈ Ο(𝑔 𝑛 )
Asymptotic notation (Big Omega )
Asymptotic notation (Big Omega )
Asymptotic notation (Big Omega )

Example 4
𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛2 + 3𝑛 + 4 ∈ Ω 𝑛2
Asymptotic notation (Big Omega )

Example 4
𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 2𝑛2 + 3𝑛 + 4 ∈ Ω 𝑛2
⟹ 2𝑛2 + 3𝑛 + 4 ≥ 1 ∗ 𝑛2
𝐻𝑒𝑛𝑐𝑒 𝑓 𝑛 = 𝛺 𝑛2 𝑤ℎ𝑒𝑟𝑒 𝑐 = 1 𝑎𝑛𝑑 𝑛 ≥ 1
Asymptotic notation (Big Omega )

Example 5
𝐼𝑓 𝑓 𝑛 = 3𝑛 + 2, 𝑔 𝑛 = 𝑛2 𝑠ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 ∉ Ω(𝑔 𝑛 )
Asymptotic notation (Big Omega )

Example 5
𝐼𝑓 𝑓 𝑛 = 3𝑛 + 2, 𝑔 𝑛 = 𝑛2 𝑠ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 ∉ Ω(𝑔 𝑛 )
𝑓 𝑛
⟹ 𝑙𝑖𝑚 >0
𝑛→∞ 𝑔 𝑛
3𝑛 + 2
⟹ 𝑙𝑖𝑚 >0
𝑛→∞ 𝑛2
2
𝑛 3+
𝑛
⟹ 𝑙𝑖𝑚 >0
𝑛→∞ 𝑛2
2
3+
𝑛
⟹ 𝑙𝑖𝑚 >0
𝑛→∞ 𝑛
⟹ 0 > 0 𝑖𝑠 𝑓𝑎𝑙𝑠𝑒, 𝐻𝑒𝑛𝑐𝑒 𝑓 𝑛 ∉ Ω(𝑔 𝑛 )
Asymptotic notation (Big Omega )

Example 6

𝐼𝑓 𝑓 𝑛 = 2𝑛 + 𝑛2 𝑎𝑛𝑑 𝑔 𝑛 = 2𝑛 𝑠ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 ∈ Ω(𝑔 𝑛 )

Asymptotic notation (Big Omega )

Example 6

𝐼𝑓 𝑓 𝑛 = 2𝑛 + 𝑛2 𝑎𝑛𝑑 𝑔 𝑛 = 2𝑛 𝑠ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 ∈ Ω(𝑔 𝑛 )

⟹ 2𝑛 + 𝑛2 > 2𝑛 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑛 ≥ 1 𝑎𝑛𝑑 𝑐 = 1

𝐻𝑒𝑛𝑐𝑒, 𝑓 𝑛 ∈ Ω 𝑔 𝑛 𝑖𝑠 𝑡𝑟𝑢𝑒
Asymptotic notation (Theta)
Asymptotic notation (Theta)
Example 7
𝑆ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 10𝑛3 + 5𝑛2 + 17 ∈ Θ 𝑛3
Asymptotic notation (Theta)
Example 7
𝑆ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 10𝑛3 + 5𝑛2 + 17 ∈ Θ 𝑛3
As per the definition of 𝜃 notation 𝐶1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝐶2 𝑔 𝑛
⟹ 10𝑛3 ≤ 10𝑛3 + 5𝑛2 + 171 < 10𝑛3 + 5𝑛3 + 17𝑛3
⟹ 10𝑛3 ≤ 10𝑛3 + 5𝑛2 + 171 < 32𝑛3
𝑆𝑜, 𝐶1 = 10 and 𝐶2 = 32 for all n ≥ 1
𝐻𝑒𝑛𝑐𝑒, 𝑃𝑟𝑜𝑣𝑒𝑑
Asymptotic notation (Theta)
Example 8
𝑆ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 = (𝑛 + 𝑎)𝑏 ∈ Θ 𝑛𝑏
Asymptotic notation (Theta)
Example 8
𝑆ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 = (𝑛 + 𝑎)𝑏 ∈ Θ 𝑛𝑏
𝑓 𝑛
As per the definition of 𝜃 notation ⟹ 𝑙𝑖𝑚 = 𝑐 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑛 ≥ 1 𝑎𝑛𝑑 𝑐 > 0
𝑛→∞ 𝑔 𝑛
(𝑛 + 𝑎)𝑏
⟹ 𝑙𝑖𝑚
𝑛→∞ 𝑛𝑏
𝑏 𝑎 𝑏
𝑛 1+𝑛
⟹ 𝑙𝑖𝑚
𝑛→∞ 𝑛𝑏
𝑎 𝑏 𝑎
⟹ 𝑙𝑖𝑚 1 + ∴ =0
𝑛→∞ 𝑛 ∞
⟹ 1 𝑤ℎ𝑖𝑐ℎ 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
𝐻𝑒𝑛𝑐𝑒 , 𝑓 𝑛 = (𝑛 + 𝑎)𝑏 ∈ Θ 𝑛𝑏 is true
Asymptotic notation (Little Oh )
Asymptotic notation (Little Oh )

Example 9
𝐼𝑓 𝑓 𝑛 = 2𝑛, 𝑔 𝑛 = 𝑛2 𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 𝜊(𝑔 𝑛 )
Asymptotic notation (Little Oh )

Example 9
𝐼𝑓 𝑓 𝑛 = 2𝑛, 𝑔 𝑛 = 𝑛2 𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 = 𝜊(𝑔 𝑛 )
𝑓 𝑛
⟹ 𝑙𝑖𝑚 =0
𝑛→∞ 𝑔 𝑛

2𝑛
⟹ 𝑙𝑖𝑚 2
𝑛→∞ 𝑛
2
⟹ 𝑙𝑖𝑚
𝑛→∞ 𝑛

⟹ 0
𝑊ℎ𝑖𝑐ℎ 𝑖𝑠 𝑇𝑟𝑢𝑒, 𝐻𝑒𝑛𝑐𝑒 𝑓 𝑛 = 𝜊(𝑔 𝑛 )
Asymptotic notation (Little Oh )

Example 10
𝐼𝑓 𝑓 𝑛 = 2𝑛2 , 𝑔 𝑛 = 𝑛2 𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 ≠ 𝜊(𝑔 𝑛 )
Asymptotic notation (Little Oh )

Example 10
𝐼𝑓 𝑓 𝑛 = 2𝑛2 , 𝑔 𝑛 = 𝑛2 𝑃𝑟𝑜𝑣𝑒 𝑡ℎ𝑎𝑡 𝑓 𝑛 ≠ 𝜊(𝑔 𝑛 )
𝑓 𝑛
⟹ 𝑙𝑖𝑚 =0
𝑛→∞ 𝑔 𝑛

2𝑛2
⟹ 𝑙𝑖𝑚 2
𝑛→∞ 𝑛
⟹ 𝑙𝑖𝑚 2
𝑛→∞

⟹ 2≠0
𝑊ℎ𝑖𝑐ℎ 𝑖𝑠 𝑇𝑟𝑢𝑒, 𝐻𝑒𝑛𝑐𝑒 𝑓 𝑛 ≠ 𝜊(𝑔 𝑛 )
Asymptotic notation (Little omega )
Asymptotic notation (Little omega )

Example 11
𝐼𝑓 𝑓 𝑛 = 2𝑛2 + 16 𝑎𝑛𝑑 𝑔 𝑛 = 𝑛2 𝑠ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 ≠ 𝜔 𝑔 𝑛
Asymptotic notation (Little omega )

Example 11
𝐼𝑓 𝑓 𝑛 = 2𝑛2 + 16 𝑎𝑛𝑑 𝑔 𝑛 = 𝑛2 𝑠ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 ≠ 𝜔 𝑔 𝑛
𝑓 𝑛
⟹ 𝑙𝑖𝑚 =∞
𝑛→∞ 𝑔 𝑛
16
𝑛2 2 + 2
𝑛
⟹ 𝑙𝑖𝑚
𝑛→∞ 𝑛2
16
⟹ 𝑙𝑖𝑚 2 + 2
𝑛→∞ 𝑛
⟹ 𝑙𝑖𝑚 2 + 0
𝑛→∞
⟹ 𝑙𝑖𝑚 2
𝑛→∞
𝑆𝑜 2 ≠ ∞ 𝑖𝑠 𝑡𝑟𝑢𝑒 , 𝐻𝑒𝑛𝑐𝑒 𝑓 𝑛 ≠ 𝜔 𝑔 𝑛
Asymptotic notation (Little Oh omega )
Example 12
𝐼𝑓 𝑓 𝑛 = 𝑛2 𝑎𝑛𝑑 𝑔 𝑛 = log 𝑛 𝑠ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 ∈ 𝜔 𝑔 𝑛
Asymptotic notation (Little Oh omega )
Example 12
𝐼𝑓 𝑓 𝑛 = 𝑛2 𝑎𝑛𝑑 𝑔 𝑛 = log 𝑛 𝑠ℎ𝑜𝑤 𝑡ℎ𝑎𝑡 𝑓 𝑛 ∈ 𝜔 𝑔 𝑛
𝑓 𝑛
⟹ 𝑙𝑖𝑚 =∞
𝑛→∞ 𝑔 𝑛
𝑛2
⟹ 𝑙𝑖𝑚
𝑛→∞ log 𝑛

Apply L Hospital Rule

2𝑛
⟹ 𝑙𝑖𝑚
𝑛→∞ 1ൗ
𝑛
⟹ 𝑙𝑖𝑚 2𝑛2
𝑛→∞
⟹ 𝑙𝑖𝑚 ∞
𝑛→∞
𝑊ℎ𝑖𝑐ℎ 𝑖𝑠 𝑡𝑟𝑢𝑒 𝑎𝑠 𝑝𝑒𝑟 𝜔 − 𝑛𝑜𝑡𝑎𝑡𝑖𝑜𝑛, 𝐻𝑒𝑛𝑐𝑒 𝑓 𝑛 ∈ 𝜔 𝑔 𝑛
Growth of the Functions

• Basically, divided into following Categories:

1. Decrement Functions
2. Constant Functions
3. Logarithmic Functions
4. Polynomial Functions
5. Exponential Functions
Growth of the Functions

1. Decrement Functions (Smallest among all categories)

10 1 1 𝑛
𝑓 𝑛 = , , ,
𝑛 𝑛2 𝑛3 2𝑛

10 1 1 𝑛
> > >
𝑛 𝑛2 𝑛3 2𝑛
Growth of the Functions

2. Constant Functions (Second Smallest among all

categories)
𝑓 𝑛 = 𝐶𝑜𝑛𝑠𝑡𝑎𝑛𝑡

10
= 0 (𝐶𝑜𝑛𝑠𝑡𝑎𝑛𝑡) [𝑖𝑓 𝑛 → ∞]
𝑛
Growth of the Functions

3. Logarithmic Functions (Third Smallest among all

categories)
Example:
log 𝑛 > log log 𝑛 > log log log 𝑛
(log 𝑛)𝑘 > (log log 𝑛)𝑘 > (log log log 𝑛)𝑘
(log log log 𝑛) < (log log log 𝑛)𝑘
(log log 𝑛) < (log log 𝑛)𝑘
(log 𝑛) < (log 𝑛)𝑘
Combining above three inequalities, we have
(log log log 𝑛) < (log log log 𝑛)𝑘 < (log log 𝑛) < (log log 𝑛)𝑘 < (log 𝑛) < (log 𝑛)𝑘
for large value of n
Growth of the Functions

4. Polynomial Functions (Fourth Smallest among all

categories) 𝑛𝑘 : 𝑘 𝑖𝑠 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
Example:
𝑛1 , 𝑛2 , 𝑛3 ,…, 𝑛𝑘

𝑛1 < 𝑛2 < 𝑛3 <… < 𝑛𝑘

Growth of the Functions

5. Exponential Functions (Fifth Smallest among all

categories) 𝑎𝑛 : 𝑖𝑓 𝑎 > 1 𝑜𝑟 0 < 𝑎 < 1
Example:
2𝑛 , 3𝑛 ,5𝑛 ,…, 100𝑛

2𝑛 < 3𝑛 < 5𝑛 <…< 100𝑛

Growth of the Functions

Decrement < Constant < Logarithmic < Polynomial

<
Exponential
Function Function Function Function Function

Order of Functions

Example:

1
< 1 < lg 𝑛 < 𝑛 𝑛 < 𝑛2 < 𝑛3 < ⋯ < 2𝑛 < 3𝑛 < ⋯ < 𝑛𝑛
𝑛
Comparisons of functions
Standard notations and common functions
Design and Analysis of Algorithm
(KCS503)

Implementation and Analysis of

Selection Sort through Tail recursion
Approach

Lecture -8
Objective

• Able to learn and apply Tail recursive

approach to develop the recurrence equation
𝑻 𝒏 = 𝒏 + 𝑻 𝒏 − 𝟏 , 𝑻 𝟏 = 𝟏.
• Analyse and solve the recurrence by
substitution method.
• Finally, Show that selection sort can be solve
recursively by the recursive version of linear
search.
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
After 3 iteration
rd

5 elements 2 elements left

7 4 3 6 5 3 4 5 6 7

After 1st iteration After 4th iteration

4 elements left 1 elements left

3 4 7 6 5 3 4 5 6 7

After 2nd iteration After 5th iteration

3 elements left 0 elements left

3 4 7 6 5 3 4 5 6 7
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
SELECTION SORT(arr, i, n-1)
if (i==n-1)
return
j=minIndex(arr,i,n-1)
if I != j
swap(arr[i],arr[j])
SELECTION SORT(arr, i+1, n-1)
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
SELECTION SORT(arr, i, n-1) minIndex (arr, i, n-1)
if (i==n-1) min_indx=i
return min_val=arr[i]
j=minIndex(arr,i,n-1) for j=i+1 to n-1
if I != j if min_val>arr[j]
swap(arr[i],arr[j]) min_val=arr[j]
SELECTION SORT(arr, i+1, n-1) min_indx=j
return min_indx

T(n)=n+T(n-1)
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
SELECTION SORT(arr, i, n-1) minIndex (arr, i, n-1)
if (i==n-1) min_indx=i
return min_val=arr[i]
j=minIndex(arr,i,n-1) for j=i+1 to n-1
if I != j if min_val>arr[j]
swap(arr[i],arr[j]) min_val=arr[j]
SELECTION SORT(arr, i+1, n-1) min_indx=j
return min_indx

T(n)=n+T(n-1)
(n-1)+T(n-2)
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
SELECTION SORT(arr, i, n-1) minIndex (arr, i, n-1)
if (i==n-1) min_indx=i
return min_val=arr[i]
j=minIndex(arr,i,n-1) for j=i+1 to n-1
if I != j if min_val>arr[j]
swap(arr[i],arr[j]) min_val=arr[j]
SELECTION SORT(arr, i+1, n-1) min_indx=j
return min_indx

T(n)=n+T(n-1)
(n-1)+T(n-2)
(n-2)+T(n-3)
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
SELECTION SORT(arr, i, n-1) minIndex (arr, i, n-1)
if (i==n-1) min_indx=i
return min_val=arr[i]
j=minIndex(arr,i,n-1) for j=i+1 to n-1
if I != j if min_val>arr[j]
swap(arr[i],arr[j]) min_val=arr[j]
SELECTION SORT(arr, i+1, n-1) min_indx=j
return min_indx

T(n)=n+T(n-1)
(n-1)+T(n-2)

.....
(n-2)+T(n-3)
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
SELECTION SORT(arr, i, n-1) minIndex (arr, i, n-1)
if (i==n-1) min_indx=i
return min_val=arr[i]
j=minIndex(arr,i,n-1) for j=i+1 to n-1
if I != j if min_val>arr[j]
swap(arr[i],arr[j]) min_val=arr[j]
SELECTION SORT(arr, i+1, n-1) min_indx=j
return min_indx

T(n)=n+T(n-1)
(n-1)+T(n-2)

.....
(n-2)+T(n-3)

2+T(1)
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
SELECTION SORT(arr, i, n-1) minIndex (arr, i, n-1)
if (i==n-1) min_indx=i
return min_val=arr[i]
j=minIndex(arr,i,n-1) for j=i+1 to n-1
if I != j if min_val>arr[j]
swap(arr[i],arr[j]) min_val=arr[j]
SELECTION SORT(arr, i+1, n-1) min_indx=j
return min_indx

T(n)=n+T(n-1)
(n-1)+T(n-2)
i==n-1

.....
(n-2)+T(n-3)

2+T(1)
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
T(n)=n+T(n-1)
(n-1)+T(n-2)
i==n-1
.....
(n-2)+T(n-3)

2+T(1)
𝑇(𝑛) = 𝑛 + (𝑛 − 1) + (𝑛 − 2) + (𝑛 − 3) + ⋯ … … … . . +3 + 2 + 1
𝑛 𝑛+1
=
2
= Ο(𝑛2 )
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
T(n)=n+T(n-1)
(n-1)+T(n-2)
i==n-1
.....
(n-2)+T(n-3)

2+T(1)
𝑇(𝑛) = 𝑛 + (𝑛 − 1) + (𝑛 − 2) + (𝑛 − 3) + ⋯ … … … . . +3 + 2 + 1
𝑛 𝑛+1
=
2
= Ο(𝑛2 )
Hence, the recursive (Tail) method have develop the recurrence equation
𝑇 𝑛 = Ο 𝑛 + 𝑇 𝑛 − 1 ∈ Ο(𝑛2 )
A Sorting Problem
(Selection Sort Algorithm with
Tail Recursive Approach)
SELECTION SORT(arr, i, n-1) minIndex (arr, i, n-1)
if (i==n-1) min_indx=i
return min_val=arr[i]
j=minIndex(arr,i,n-1) for j=i+1 to n-1
if I != j if min_val>arr[j]
swap(arr[i],arr[j]) min_val=arr[j]
SELECTION SORT(arr, i+1, n-1) min_indx=j
return min_indx

Hence, the recursive (Tail) method have develop the recurrence equation
𝑇 𝑛 = Ο 𝑛 + 𝑇 𝑛 − 1 ∈ Ο(𝑛2 )
Thank You
Design and Analysis of Algorithm
(KCS503)

Implementation and Analysis of

Insertion Sort through Head recursion
Approach

Lecture -9
Objective

• Able to learn and apply the head recursive

approach

𝑻 𝒏 = 𝑻 𝒏 − 𝟏 + 𝚶(𝒏), 𝑻 𝟏 = 𝟏

of Insertion sort with analysis and analyse its

complexity.
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)

1st Iteration 2nd Iteration 3rd Iteration

4th Iteration 5th Iteration 6th Iteration

A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)
A = [ 5, 2, 4, 6, 1, 3], size = 6
insertion-sort(A, 0) insertion-sort(A, 3)
insert 5 insert 6
return A = [5] return A = [2, 4, 5, 6]

insertion-sort(A, 1) insertion-sort(A, 4)
insert 2 insert 1
return A = [2, 5] return A = [1, 2, 4, 5, 6]

insertion-sort(A, 2) insertion-sort(A, 5)
insert 4 insert 3
return A = [2, 4, 5] return A = [1, 2, 3, 4, 5, 6]
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)
It was observed that the concept is:

Insert an element in a previously sorted array named as ‘A’.

Solution Steps:
• Base Case: If array size is 1 or smaller,
return.
• Recursively sort first n-1 elements.
• Insert the last element at its correct
position in the sorted array.
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)

void insertion_sort(A, j) {
//Initially j=length(A)
// Base case
if (j <= 1)
return
// Sort first i-1 elements
insertion_sort( A, j-1 )
insert A[j-1] into A[0…j-2]

}
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)
0 1 2 3 4 5
5 2 4 6 1 3

5 2 4 6 1 3

5 2 4 6 1

5 2 4 6

5 2 4

5 2

5
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)
0 1 2 3 4 5 0 1 2 3 4 5
5 2 4 6 1 3 1 2 3 4 5 6

5 2 4 6 1 3 1 2 3 4 5 6

5 2 4 6 1 1 2 4 5 6

5 2 4 6 2 4 5 6

5 2 4 2 4 5

5 2 Bottom 2 5 UP
5 5
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)
void insertion_sort(A, j) {
//Initially j=length(A)
// Base case
if (j <= 1)
return
// Sort first n-1 elements
insertion_sort( A, j-1 )
val = A[j-1]
i = j-2
while (i >= 0 && A[i] > val) {
A[i+1] = A[i]
i=i-1
}
A[i+1] = val
}
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)
Complexity:
𝐓 𝐧 =𝐓 𝐧−𝟏 + 𝐧
𝐓 𝐧 =𝐓 𝐧−𝟐 + 𝐧−𝟏 + 𝐧
𝐓 𝐧 =𝐓 𝐧−𝟑 + 𝐧−𝟐 + 𝐧−𝟏 + 𝐧
…………………….…………………………………………..
…………………….…………….………………………………..
…………………………………………………………………………
𝐓 𝐧 = 𝟏+𝟐 +𝟑+⋯+⋯+ 𝐧 −𝟑 + 𝐧 −𝟐 + 𝐧 −𝟏 + 𝐧

𝐧 𝐧+𝟏
𝐇𝐞𝐧𝐜𝐞 𝐓 𝐧 = 𝐓 𝐧 = 𝚶(𝐧𝟐 )
𝟐
A Sorting Problem
(Insertion Sort Algorithm with Head Recursive Approach)
void insertion_sort(A, j) {
//Initially j=length(A)
// Base case
if (j <= 1)
return
// Sort first j-1 elements Hence, the recursive (Head) method have
insertion_sort( A, j-1 ) develop the recurrence equation
val = A[j-1] 𝑻 𝒏 = 𝑻 𝒏 − 𝟏 + 𝜪 𝒏 ∈ 𝜪(𝒏𝟐 )
i = j-2
while (i >= 0 && A[i] > val) {
A[i+1] = A[i]
i=i-1
}
A[i+1] = val
}
🙏🏻 Thank You 🙏🏻
Design and Analysis of Algorithm

Recurrence Equation
(Solving Recurrence using
Iteration Methods)

Lecture – 10 and 11
Overview
• A recurrence is a function is defined in terms of
– one or more base cases, and
– itself, with smaller arguments.
Examples:

Linear Decay Division

Changing Variable Decision Tree

Overview
• Many technical issues:
• Floors and ceilings
[Floors and ceilings can easily be removed and don’t affect
the solution to the recurrence. They are better left to a
discrete math course.]
• Exact vs. asymptotic functions
• Boundary conditions
Overview

In algorithm analysis, the recurrence and it’s solution are

expressed by the help of asymptotic notation.
• Example: 𝑇 (𝑛) = 2𝑇 (𝑛/2) + 𝛩(𝑛), with solution
𝑇 (𝑛) = 𝛩(𝑛 lg 𝑛).
• The boundary conditions are usually expressed as
𝑇 (𝑛) = 𝛰(1) for sufficiently small n..
• But when there is a desire of an exact, rather than
an asymptotic, solution, the need is to deal with
boundary conditions.
• In practice, just use asymptotics most of the time,
and ignore boundary conditions.
Recursive Function
• Example
𝐴(𝑛)
{
𝐼𝑓(𝑛 > 1)
𝑅𝑒𝑡𝑢𝑟𝑛(𝐴 𝑛 − 1 )
}
The relation is called recurrence relation
The Recurrence relation of given function is written as follows.
𝑇(𝑛) = 𝑇(𝑛 − 1) + 1
Recursive Function
• To solve the Recurrence relation the following methods
are used:
1. Iteration method
2. Recursion-Tree method
3. Master Method
4. Substitution Method
Iteration Method( Example 1)
• In Iteration method the basic idea is to expand the recurrence
and express it as a summation of terms dependent only on ‘n’
(i.e. the number of input) and the initial conditions.
Example 1:
Solve the following recurrence relation by using Iteration method.

𝑇 𝑛−1 +1 𝑖𝑓 𝑛 > 1
𝑇 𝑛 =ቊ
1 𝑖𝑓 𝑛 = 1
Iteration Method ( Example 1)
It means 𝑇 𝑛 = 𝑇 𝑛 − 1 + 1 𝑖𝑓 𝑛 > 1
𝑎𝑛𝑑 𝑇 𝑛 = 1 𝑤ℎ𝑒𝑛 𝑛 = 1 −−−−−−−−−−−− −(1)
𝑃𝑢𝑡 𝑛 = 𝑛 − 1 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
𝑇 𝑛−1 =𝑇 𝑛−2 +1
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇(𝑛 − 1) 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
𝑇 𝑛 = 𝑇 𝑛 − 2 + 2 −−−−−−−−−−−−−−−− −(2)
Iteration Method ( Example 1)
𝑃𝑢𝑡 𝑛 = 𝑛 − 2 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
𝑇 𝑛−2 =𝑇 𝑛−3 +1
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇(𝑛 − 2) 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 2, 𝑤𝑒 𝑔𝑒𝑡
𝑇 𝑛 = 𝑇 𝑛 − 3 + 3 −−−−−−−−−−−−−−−− −(3)
……….
𝑇 𝑛 = 𝑇 𝑛 − 𝑘 + 𝑘 −−−−−−−−−−−−−−−− −(𝑘)
Iteration Method ( Example 1)
𝐿𝑒𝑡 𝑇 𝑛 − 𝑘 = 𝑇 1 = 1
(𝐴𝑠 𝑝𝑒𝑟 𝑡ℎ𝑒 𝑏𝑎𝑠𝑒 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 𝑜𝑓 𝑟𝑒𝑐𝑢𝑟𝑟𝑒𝑛𝑐𝑒)
𝑆𝑜 𝑛 − 𝑘 = 1
⇒𝑘 =𝑛−1
𝑁𝑜𝑤 𝑝𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑘 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 𝑘
𝑇 𝑛 =𝑇 𝑛− 𝑛−1 +𝑛−1
𝑇 𝑛 =𝑇 1 +𝑛−1
𝑇 𝑛 =1+𝑛−1 [∴ 𝑇 1 = 1]
𝑇 𝑛 =𝑛
∴ 𝑇 𝑛 =Θ 𝑛
Iteration Method ( Example 2)
Example 2:
Solve the following recurrence relation by using Iteration method.

𝑛
2𝑇 + 3𝑛2 𝑖𝑓 𝑛 > 1
𝑇 𝑛 =ቐ 2
11 𝑖𝑓 𝑛 = 1
Iteration Method ( Example 2)
𝑛
𝐼𝑡 𝑚𝑒𝑎𝑛𝑠 𝑇 𝑛 = 2𝑇 + 3𝑛2 𝑖𝑓 𝑛 > 1 𝑎𝑛𝑑 𝑇 𝑛 = 11 𝑤ℎ𝑒𝑛 𝑛 = 1 − −(1)
2
𝑛
𝑃𝑢𝑡 𝑛 = 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
2
𝑛 𝑛 𝑛 2
𝑇 = 2𝑇 +3
2 4 2
𝑛 𝑛 𝑛 2
𝑇 = 2𝑇 2 + 3
2 2 2
𝑛
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
2
𝑛 𝑛 2
𝑇 𝑛 = 2 2𝑇 2 + 3 + 3𝑛2
2 2
𝑛 𝑛2
𝑇 𝑛 = 2 𝑇 2 + 2.3 + 3𝑛2
2
2 4
2
𝑛 𝑛
𝑇 𝑛 = 22 𝑇 2 + 3 + 3𝑛2 −−−−−−−−−−−−− −(2)
2 2
Iteration Method ( Example 2)
𝑛
𝑃𝑢𝑡 𝑛 = 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
4
𝑛 𝑛 𝑛 2
𝑇 = 2𝑇 +3
4 8 4
𝑛 𝑛 𝑛 2
𝑇 = 2𝑇 3 + 3
4 2 4
𝑛
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 2, 𝑤𝑒 𝑔𝑒𝑡
4
2 2
𝑛 𝑛 𝑛
𝑇 𝑛 = 22 2𝑇 +3 + 3 +3𝑛2
8 16 2
2
𝑛 𝑛 2 𝑛2
𝑇 𝑛 = 2 2𝑇 3 + 3 + 3 +3𝑛2
2 4 2
2 2
𝑛 𝑛 𝑛
𝑇 𝑛 = 23 𝑇 3 + 4.3 + 3 +3𝑛2
2 16 2
Iteration Method ( Example 2)
2 2
𝑛 𝑛 𝑛
𝑇 𝑛 = 23 𝑇 3 + 3 2 + 3 +3𝑛2 −−−−−−−−−−−−− −(3)
2 2 2
…….
𝑛 𝑛2 𝑛2
𝑇 𝑛 = 2 𝑇 𝑖 + ⋯ + ⋯ + ⋯ + 3 2 + 3 +3𝑛2 −−−−−−−− −(𝑖 𝑡ℎ 𝑡𝑒𝑟𝑚)
𝑖
2 2 2
𝑛
𝑎𝑛𝑑 𝑡ℎ𝑒 𝑠𝑒𝑟𝑖𝑒𝑠 𝑡𝑒𝑟𝑚𝑖𝑛𝑎𝑡𝑒 𝑤ℎ𝑒𝑛 𝑖 = 1
2
⇒ 𝑛 = 2𝑖
𝑇𝑎𝑘𝑖𝑛𝑔 log 𝑏𝑜𝑡ℎ 𝑠𝑖𝑑𝑒
⇒ log 2 𝑛 = 𝑖 log 2 2
⇒ 𝑖 = log 2 𝑛 (𝑏𝑒𝑐𝑎𝑢𝑠𝑒 log 2 2 = 1)
Iteration Method ( Example 2)
𝐻𝑒𝑛𝑐𝑒 𝑤𝑒 𝑐𝑎𝑛 𝑤𝑟𝑖𝑡𝑒 𝑡ℎ𝑒 𝑖 𝑡ℎ 𝑡𝑒𝑟𝑚 𝑎𝑠 𝑓𝑜𝑙𝑙𝑜𝑤𝑠
2 2
𝑛 𝑛 𝑛
⇒ 𝑇 𝑛 = 3𝑛2 + 3 + 3 2 + ⋯ + ⋯ + ⋯ + 2𝑖 𝑇 𝑖
2 2 2
2 2
𝑛 𝑛
⇒ 𝑇 𝑛 = 3𝑛2 + 3 + 3 2 + ⋯ + ⋯ + ⋯ + 2log2 𝑛 𝑇 1
2 2
𝑛2 𝑛2
⇒ 𝑇 𝑛 = 3𝑛 + 3 + 3 2 + ⋯ + ⋯ + ⋯ + 2log2 𝑛 . 11
2
2 2
2 2
𝑛 𝑛
⇒ 𝑇 𝑛 = 3𝑛2 + 3 + 3 2 + ⋯ + ⋯ + ⋯ + 𝑛log2 2 . 11 [ 𝐴𝑠 log 2 2 = 1]
2 2
2
𝑛2 𝑛2
⇒ 𝑇 𝑛 = 3𝑛 + 3 + 3 2 + ⋯ + ⋯ + ⋯ + 𝑛. 11
2 2
2 2
𝑛 𝑛
⇒ 𝑇 𝑛 = 3𝑛2 + 3 + 3 2 + ⋯ + ⋯ + ⋯ + 11. 𝑛
2 2
1 1
⇒ 𝑇 𝑛 = 3𝑛2 1 + + 2 + ⋯ + ⋯ + ⋯ + 11. 𝑛
2 2
Iteration Method ( Example 2)
𝐴𝑠 𝑤𝑒 𝑘𝑛𝑜𝑤 𝑡ℎ𝑎𝑡 𝑆𝑢𝑚 𝑜𝑓 𝑖𝑛𝑓𝑖𝑛𝑖𝑡𝑒 𝐺𝑒𝑜𝑚𝑒𝑡𝑟𝑖𝑐 𝑠𝑒𝑟𝑖𝑒𝑠 𝑖𝑠
∞
1 𝑎
= 𝑎 + 𝑎𝑟 + 𝑎𝑟 2 + ...+ 𝑎𝑟 (𝑛−1) = ෍ 𝑎𝑟 𝑖 =𝑎 =
1−𝑟 1−𝑟
𝑖=0

Hence,
1
⇒ 𝑇 𝑛 ≤ 3𝑛2 + 11 𝑛
1
1−
2
⇒ 𝑇 𝑛 ≤ 3𝑛2 . 2 + 11 𝑛
⇒ 𝑇 𝑛 ≤ 6𝑛2 + 11 𝑛
𝐻𝑒𝑛𝑐𝑒 𝑇 𝑛 = Ο(𝑛2 )
Iteration Method ( Example 3)
Example 3:
Solve the following recurrence relation by using Iteration method.
𝑛
8𝑇 + 𝑛2 𝑖𝑓 𝑛 > 1
𝑇 𝑛 =ቐ 2
1 𝑖𝑓 𝑛 = 1
Iteration Method ( Example 3)
𝑛
𝐼𝑡 𝑚𝑒𝑎𝑛𝑠 𝑇 𝑛 = 8𝑇 + 𝑛2 𝑖𝑓 𝑛 > 1 𝑎𝑛𝑑 𝑇 𝑛 = 1 𝑤ℎ𝑒𝑛 𝑛 = 1 −−− −(1)
2
𝑛
𝑃𝑢𝑡 𝑛 = 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
2
𝑛 𝑛 𝑛 2
𝑇 = 8𝑇 +
2 4 2
𝑛
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
2
𝑛 𝑛 2
𝑇 𝑛 = 8 8𝑇 + + 𝑛2
4 2
2
𝑛 𝑛2
𝑇 𝑛 =8 𝑇 + 8 + 𝑛2 −−−−−−−−−−−−− −(2)
4 4
𝑛
𝑃𝑢𝑡 𝑛 = 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
4
𝑛 𝑛 𝑛 2
𝑇 = 8𝑇 +
4 8 4
Iteration Method ( Example 3)
𝑛
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 2, 𝑤𝑒 𝑔𝑒𝑡
4
2
𝑛 𝑛 2 𝑛2
𝑇 𝑛 = 8 8𝑇 + + 8 + 𝑛2
8 4 4
𝑛 𝑛2 𝑛2
𝑇 𝑛 = 83 𝑇 8 + 82 42 + 84 + 𝑛2 −−−−−−−−−−−−− −(3)
……
𝑇 𝑛
𝑛 𝑛 2 𝑛 2 𝑛 2
= 8𝑘 𝑇 𝑘 + 8𝑘−1 𝑘−1 + ⋯ + ⋯ + ⋯ + 82 2 + 8 + 𝑛2 −−− −(𝑘 𝑡ℎ 𝑡𝑒𝑟𝑚)
2 4 4 4
𝑛 8𝑘−1 8𝑘−2 82 8
𝑇 𝑛 = 8𝑘 𝑇 𝑘 + 𝑛2 𝑘−1 + 𝑘−2 … + ⋯ + ⋯ + 2 + + 1
2 4 4 4 4
𝑛
𝑇 𝑛 = 8 𝑇 𝑘 + 𝑛2 2𝑘−1 + 2𝑘−2 … + ⋯ + ⋯ + 22 + 2 + 1 −−−− −(4)
𝑘
2
Iteration Method ( Example 3)
𝑛
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 2, 𝑤𝑒 𝑔𝑒𝑡
4
2
𝑛 𝑛 2 𝑛2
𝑇 𝑛 = 8 8𝑇 + + 8 + 𝑛2
8 4 4
𝑛 𝑛2 𝑛2
𝑇 𝑛 = 83 𝑇 8 + 82 42 + 84 + 𝑛2 −−−−−−−−−−−−− −(3)
……
𝑛 𝑛2 𝑛2 𝑛2
𝑇 𝑛 = 8𝑘 𝑇𝑘 +8𝑘−1
𝑘−1 + ⋯ + ⋯ + 8 2 + 8 + 𝑛2 −−− −(𝑘 𝑡ℎ 𝑡𝑒𝑟𝑚)
2
2 4 4 4
𝑛 8𝑘−1 8𝑘−2 82 8
𝑘
𝑇 𝑛 =8 𝑇 𝑘 + 𝑛 2 + …+ ⋯+ ⋯+ 2 + + 1
2 4𝑘−1 4𝑘−2 4 4
𝑛
𝑇 𝑛 = 8𝑘 𝑇 𝑘 + 𝑛2 2𝑘−1 + 2𝑘−2 … + ⋯ + ⋯ + 22 + 2 + 1 −−−− −(4)
2
Iteration Method ( Example 3)
𝑛
𝑇 𝑛 = 8 𝑇 𝑘 + 𝑛2 2𝑘−1 + 2𝑘−2 … + ⋯ + ⋯ + 22 + 2 + 1 −−− −(4)
𝑘
2
𝑛
𝑇 𝑛 = 8𝑘 𝑇 + 𝑛2 1 + 2 + 22 + ⋯ + ⋯ + 2𝑘−2 + 2𝑘−1 −−− − (5)
2𝑘
𝑛
𝑎𝑛𝑑 𝑡ℎ𝑒 𝑠𝑒𝑟𝑖𝑒𝑠 𝑡𝑒𝑟𝑚𝑖𝑛𝑎𝑡𝑒 𝑤ℎ𝑒𝑛 𝑘 = 1
2
⇒ 𝑛 = 2𝑘
𝑇𝑎𝑘𝑖𝑛𝑔 log 𝑏𝑜𝑡ℎ 𝑠𝑖𝑑𝑒
⇒ log 2 𝑛 = 𝑘 log 2 2
⇒ 𝑘 = log 2 𝑛 (𝑏𝑒𝑐𝑎𝑢𝑠𝑒 log 2 2 = 1)
𝑛
𝑁𝑜𝑤, 𝑎𝑝𝑝𝑙𝑦 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑘 = log 2 𝑛 𝑎𝑛𝑑 𝑘
= 1 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 5
2
Iteration Method ( Example 3)
𝑇 𝑛 = 8log2 𝑛 𝑇 1 + 𝑛2 1 + 2 + 22 + ⋯ + ⋯ + 2log2 𝑛−2 + 2log2 𝑛−1 −
(6)
= 𝑛log2 8 . 1 + 𝑛2 1 + 2 + 22 + ⋯ + ⋯ + 2log2 𝑛−2 + 2log2 𝑛−1
= 𝑛3 + 𝑛2 1 + 2 + 22 + ⋯ + ⋯ + 2log2 𝑛−2 + 2log2 𝑛−1

Is a G.P Series, but in this case no need of

evaluation. Because the highest order
polynomial is 𝑛3 . So no need to calculate 𝑛2 .

= 𝑛3
Hence the complexity is 𝑇(𝑛) = 𝑛3
Iteration Method ( Example 3)
𝑇 𝑛 = 8log2 𝑛 𝑇 1 + 𝑛2 1 + 2 + 22 + ⋯ + 2log2 𝑛−2 + 2log2 𝑛−1 − (6)
= 𝑛log2 8 . 1 + 𝑛2 1 + 2 + 22 + ⋯ + ⋯ + 2log2 𝑛−2 + 2log2 𝑛−1
= 𝑛3 + 𝑛2 1 + 2 + 22 + ⋯ + ⋯ + 2log2 𝑛−2 + 2log2 𝑛−1

Is a G.P Series, but in this case no need of

evaluation. Because the highest order
polynomial is 𝑛3 . So no need to calculate 𝑛2 .

= 𝑛3
Hence the complexity is 𝑇(𝑛) = 𝜪(𝑛3 )
𝑆𝑢𝑚 𝑜𝑓 𝑓𝑖𝑛𝑖𝑡𝑒 𝐺𝑒𝑜𝑚𝑒𝑡𝑟𝑖𝑐 𝑃𝑟𝑜𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑠𝑒𝑟𝑖𝑒𝑠 𝑖𝑠
𝑛 𝑛+1 −1
𝑛 𝑟
= 𝑎 + 𝑎𝑟 + 𝑎𝑟 2 + . . . + 𝑎𝑟 = ෍ 𝑎𝑟 𝑖 = 𝑎
𝑟−1
𝑖=0
Iteration Method ( Example 4)
Example 4:
Solve the following recurrence relation by using Iteration method.
𝑛
7𝑇 + 𝑛2 𝑖𝑓 𝑛 > 1
𝑇 𝑛 =ቐ 2
1 𝑖𝑓 𝑛 = 1
(𝑖. 𝑒. 𝑆𝑡𝑟𝑎𝑠𝑠𝑖𝑜𝑛 𝐴𝑙𝑔𝑜𝑟𝑖𝑡ℎ𝑚)
Iteration Method ( Example 4)
𝑛
𝐼𝑡 𝑚𝑒𝑎𝑛𝑠 𝑇 𝑛 = 7𝑇 + 𝑛2 𝑖𝑓 𝑛 > 1 𝑎𝑛𝑑 𝑇 𝑛 = 1 𝑤ℎ𝑒𝑛 𝑛 = 1 −−− −(1)
2
𝑛
𝑃𝑢𝑡 𝑛 = 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
2
𝑛 𝑛 𝑛 2
𝑇 = 7𝑇 +
2 4 2
𝑛
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
2
𝑛 𝑛 2
𝑇 𝑛 = 7 7𝑇 + + 𝑛2
4 2
2
𝑛 𝑛2
𝑇 𝑛 =7 𝑇 + 7 + 𝑛2 −−−−−−−−−−−−− −(2)
4 4
𝑛
𝑃𝑢𝑡 𝑛 = 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
4
𝑛 𝑛 𝑛 2
𝑇 = 7𝑇 +
4 8 4
Iteration Method ( Example 4)
𝑛
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 2, 𝑤𝑒 𝑔𝑒𝑡
4
2
𝑛 𝑛 2 𝑛2
𝑇 𝑛 = 7 7𝑇 + + 7 + 𝑛2
8 4 4
3
𝑛 𝑛2 𝑛2
𝑇 𝑛 =7 𝑇 + 7 2 + 7 + 𝑛2 −−−−−−−−−−−−− −(3)
2
8 4 4
……
2 2 2
𝑛 𝑛 𝑛 𝑛
𝑇 𝑛 = 7𝑘 𝑇 𝑘 + 7𝑘−1 𝑘−1 + ⋯ + ⋯ + ⋯ + 72 2 + 7 + 𝑛2 −− −(𝑘 𝑡ℎ 𝑡𝑒𝑟𝑚)
2 4 4 4
𝑘−1 𝑘−2 2
𝑛 7 7 7 7
𝑇 𝑛 = 7𝑘 𝑇 𝑘 + 𝑛2 𝑘−1 + 𝑘−2 … + ⋯ + ⋯ + 2 + + 1
2 4 4 4 4
𝑘−1 𝑖
𝑛 7
𝑇 𝑛 = 7𝑘 𝑇 𝑘 + 𝑛2 ෍ −−−− −(4)
2 4
𝑖=0
Iteration Method ( Example 4)
𝑛
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 2, 𝑤𝑒 𝑔𝑒𝑡
4
2 2
𝑛 𝑛 𝑛
𝑇 𝑛 = 72 7𝑇 + + 7 + 𝑛2
8 4 4
𝑛 𝑛 2 𝑛 2
𝑇 𝑛 = 73 𝑇 + 72 2 + 7 + 𝑛2 −−−−−−−−−−−−− −(3)
8 4 4
……
𝑘
𝑛 𝑘−1
𝑛2 2
𝑛2 𝑛2 2 −− −(𝑘 𝑡ℎ 𝑡𝑒𝑟𝑚)
𝑇 𝑛 =7 𝑇 𝑘 +7 + ⋯ + ⋯ + ⋯ + 7 + 7 + 𝑛
2 4𝑘−1 42 4
𝑘
𝑛 2
7𝑘−1 7𝑘−2 72 7
𝑇 𝑛 =7 𝑇 𝑘 + 𝑛 + …+ ⋯+ ⋯+ 2 + + 1
2 4𝑘−1 4𝑘−2 4 4
𝑘−1 𝑖
𝑘
𝑛 2
7
𝑇 𝑛 =7 𝑇 𝑘 + 𝑛 ෍ −−−− −(4)
2 4
𝑖=0
Iteration Method ( Example 4)
𝑘−1 𝑖
𝑛 7
𝑇 𝑛 = 7𝑘 𝑇 𝑘 + 𝑛2 ෍ −−−− − 4
2 4
𝑖=0
𝑛
𝑎𝑛𝑑 𝑡ℎ𝑒 𝑠𝑒𝑟𝑖𝑒𝑠 𝑡𝑒𝑟𝑚𝑖𝑛𝑎𝑡𝑒 𝑤ℎ𝑒𝑛 𝑘 = 1
2
⇒ 𝑛 = 2𝑘
𝑇𝑎𝑘𝑖𝑛𝑔 log 𝑏𝑜𝑡ℎ 𝑠𝑖𝑑𝑒
⇒ log 2 𝑛 = 𝑘 log 2 2
⇒ 𝑘 = log 2 𝑛 (𝑏𝑒𝑐𝑎𝑢𝑠𝑒 log 2 2 = 1)
𝑛
𝑁𝑜𝑤, 𝑎𝑝𝑝𝑙𝑦 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑘 = log 2 𝑛 𝑎𝑛𝑑 = 1 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 4
2𝑘
Iteration Method ( Example 4)
log2 𝑛−1 𝑖
7
𝑇 𝑛 = 7log2 𝑛 𝑇 1 + 𝑛2 ෍ −−−− − 5
4
𝑖=0
log2 𝑛−1 𝑖
7
= 𝑛log2 7 . 1 + 𝑛2 ෍
4
𝑖=0
log2 𝑛−1 𝑖
7
= 𝑛log2 7 . 1 + 𝑛2 ෍
4
𝑖=0
log2 𝑛−1 𝑖
7
= 𝑛2.8 + 𝑛2 ෍
4
𝑖=0

Is a G.P Series, but in this case no need of

evaluation. Because the highest order
polynomial is 𝑛3 . So no need to calculate 𝑛2 .
Iteration Method ( Example 4)
log2 𝑛−1 𝑖
7
= 𝑛2.8 + 𝑛2 ෍
4
𝑖=0

Is a G.P Series, but in this case no need of

evaluation. Because the highest order
polynomial is 𝑛2.8 . So no need to calculate 𝑛2 .

= 𝑛2.8
Hence the complexity is 𝑇(𝑛) = 𝑛2.8
Iteration Method ( Example 4)
log2 𝑛−1 𝑖
7
= 𝑛2.8 + 𝑛2 ෍
4
𝑖=0

Is a G.P Series, but in this case no need of

evaluation. Because the highest order
polynomial is 𝑛2.8 . So no need to calculate 𝑛2 .

= 𝑛2.8
Hence the complexity is 𝑇(𝑛) = 𝜪(𝑛2.8 )
𝑆𝑢𝑚 𝑜𝑓 𝑓𝑖𝑛𝑖𝑡𝑒 𝐺𝑒𝑜𝑚𝑒𝑡𝑟𝑖𝑐 𝑃𝑟𝑜𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑠𝑒𝑟𝑖𝑒𝑠 𝑖𝑠
𝑛
2 𝑛 𝑟 𝑛+1 −1
𝑖
= 𝑎 + 𝑎𝑟 + 𝑎𝑟 + . . . + 𝑎𝑟 = ෍ 𝑎𝑟 = 𝑎
𝑟−1
𝑖=0
Iteration Method ( Example 5)
Example 5:
Solve the following recurrence relation by using Iteration method.

𝑇 𝑛 − 1 + log 𝑛 𝑖𝑓 𝑛 > 1
𝑇 𝑛 =ቊ
1 𝑖𝑓 𝑛 = 1
Iteration Method ( Example 5)
𝐼𝑡 𝑚𝑒𝑎𝑛𝑠 𝑇 𝑛 = 𝑇 𝑛 − 1 + log 𝑛 𝑖𝑓 𝑛 > 1 𝑎𝑛𝑑 𝑇 𝑛 = 1 𝑤ℎ𝑒𝑛 𝑛 = 1 − −(1)
𝑃𝑢𝑡 𝑛 = 𝑛 − 1 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
𝑇 𝑛 − 1 = 𝑇 𝑛 − 2 + log(𝑛 − 1)
𝑃𝑢𝑡 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑇 𝑛 − 1 𝑖𝑛 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 1, 𝑤𝑒 𝑔𝑒𝑡
𝑇 𝑛 = 𝑇 𝑛 − 2 + log(𝑛 − 1) + log 𝑛
= 𝑇 𝑛 − 3 + log(𝑛 − 2) + log(𝑛 − 1) + log 𝑛
= T n − 4 + log (𝑛 − 3) + log(𝑛 − 2) + log(𝑛 − 1) + log 𝑛
…………….
…………….
Hence the 𝑘𝑡ℎ order is :
T n = T n − k + log n − k − 1 + ⋯ + log(𝑛 − 2) + log(𝑛 − 1) + log 𝑛
T n = T n − k + log n − 𝑘 + 1 + ⋯ + log(𝑛 − 2) + log(𝑛 − 1) + log 𝑛
Iteration Method ( Example 5)
Hence the 𝑘𝑡ℎ order is :
T n = T n − k + log n − 𝑘 + 1 + ⋯ + log(𝑛 − 2) + log(𝑛 − 1) + log 𝑛

𝐴𝑠 𝑝𝑒𝑟 𝑡ℎ𝑒 𝑎𝑠𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛 𝑛 − 𝑘 = 1

𝑆𝑜 𝑘 = 𝑛 − 1
The 𝑘 𝑡ℎ order can be written as:
T n = T 1 + log n − 𝑛 + 1 + 1 + ⋯ + log(𝑛 − 2) + log(𝑛 − 1) + log 𝑛
= 1 + log 2 + ⋯ + log(𝑛 − 2) + log(𝑛 − 1) + log 𝑛
= 1 + log 2 + log 3 + log(4) … + log(𝑛 − 2) + log(𝑛 − 1) + log 𝑛
= 1 + log 2. 3. 4. 5 … … … … . n
= 1 + log n!
𝑯𝒆𝒏𝒄𝒆 𝒕𝒉𝒆 𝒄𝒐𝒎𝒑𝒍𝒆𝒙𝒊𝒕𝒚 𝒊𝒔 ∶ 𝜪(𝒍𝒐𝒈𝒏!)
Iteration Method ( Practice)
7𝑛
𝑇 +𝑛 𝑖𝑓 𝑛 > 1
𝑄1. 𝑇 𝑛 = ൞ 10
1 𝑖𝑓 𝑛 = 1

𝑇 𝑛 − 1 + (𝑛 − 1) 𝑖𝑓 𝑛 > 1
𝑄2. 𝑇 𝑛 = ቊ
1 𝑖𝑓 𝑛 = 1

𝑇 𝑛 − 1 + 𝑛2 𝑖𝑓 𝑛 > 1
𝑄3. 𝑇 𝑛 = ቊ
1 𝑖𝑓 𝑛 = 1
Design and Analysis of Algorithm

Recurrence Equation
(Solving Recurrence using
Recursion Tree Methods)

Lecture – 12 and 13
Overview
• A recurrence is a function is defined in terms of
– one or more base cases, and
– itself, with smaller arguments.
Examples:

Linear Decay Division

Changing Variable Decision Tree

In algorithm analysis, the recurrence and it’s solution are

expressed by the help of asymptotic notation.
• Example: 𝑇 (𝑛) = 2𝑇 (𝑛/2) + 𝛩(𝑛), with solution
𝑇 (𝑛) = 𝛩(𝑛 lg 𝑛).
• The boundary conditions are usually expressed as
𝑇 (𝑛) = 𝛰(1) for sufficiently small n..
• But when there is a desire of an exact, rather than
an asymptotic, solution, the need is to deal with
boundary conditions.
• In practice, just use asymptotics most of the time,
and ignore boundary conditions.
Recursive Function
• Example
𝐴(𝑛)
{
𝐼𝑓(𝑛 > 1)
𝑛
𝑅𝑒𝑡𝑢𝑟𝑛 𝐴
2
}
The relation is called recurrence relation
The Recurrence relation of given function is written as follows.
𝑛
𝑇(𝑛) = 𝑇 +1
2
Recursive Function
• To solve the Recurrence relation the following methods
are used:
1. Iteration method
2. Recursion-Tree method
3. Master Method
4. Substitution Method
Recursion Tree Method
• Recursion Tree is another method for solving recurrence relations.
This method work on two steps. These are
• First, A set of pre level costs are obtained by sum the cost of each
level of the tree and the height of the tree.
• Second, to determine the total cost of all level of recursion, we sum
all the pre level cost.
• This method is best used for good guess.
• For generating good guess, we can ignore floors( 𝑥 ) and ceiling
( 𝑥 ) when solving the recurrences. Because they usually do not
affect the final guess.
Recursion Tree Method (Example 1)
Example 1

𝑛
Solve the recurrence 𝑇 𝑛 = 3𝑇 + Θ 𝑛2 by
4
using recursion tree method.
Recursion Tree Method (Example 1)
Answer:
We start by focusing on finding an upper bound for the
solution by using good guess. As we know that floors
and ceilings usually do not matter when solving the
recurrences, we drop the floor and write the recurrence
equation as follows:
𝑛
𝑇 𝑛 = 3𝑇 + 𝑐𝑛2 , 𝑐 > 0
4
The term 𝑐𝑛2 , at the root represent the costs incurred by
𝑛
the subproblems of size .
4
Recursion Tree Method (Example 1)
𝑛
𝑇 𝑛 = 3𝑇 + 𝑐𝑛2 , 𝑐 > 0 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
4
𝑇(𝑛)
Fig –(a)
Figure (a) shows T(n), which progressively expands in (b) and (d) to form recursion
tree.
𝑐 𝑛 2

𝑛 𝑛 𝑛
𝑇 𝑇 𝑇
4 4 4
Fig –(b)

L1.10
Recursion Tree Method (Example 1)
𝑛
𝑇 𝑛 = 3𝑇 + 𝑐𝑛2 , 𝑐 > 0 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
4
𝑐 𝑛 2

𝑐(𝑛/4)2 𝑐(𝑛/4)2 𝑐(𝑛/4)2

𝑛 𝑛 𝑛 𝑛 𝑛 𝑛 𝑛 𝑛 𝑛
𝑇 𝑇 𝑇 𝑇 𝑇 𝑇 𝑇 𝑇 𝑇
16 16 16 16 16 16 16 16 16

Fig –(c)

L1.11
Recursion Tree Method (Example 1)
𝑛
𝑇 𝑛 = 3𝑇 + 𝑐𝑛2 , 𝑐 > 0 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
4
2 2
𝑐 𝑛 𝑐 𝑛

3 2
𝑐(𝑛/4)2 𝑐(𝑛/4)2 𝑐(𝑛/4)2 𝑐 𝑛
16
log 4 𝑛

2
𝑛 2
𝑛 2 𝑛 2 𝑛 2 𝑛 2 𝑛 2
𝑛 2 𝑛 2 𝑛 2 3 2
𝑐
16
𝑐
16
𝑐
16
𝑐
16
𝑐
16
𝑐
16
𝑐
16
𝑐
16
𝑐
16 𝑐 𝑛
16

𝑖
3 2
𝑐 𝑛
16
𝑖
𝑇 1 𝑇 1 𝑇 1 ………………………..𝑇 1 3
Fig –(d)

L1.12
Recursion Tree Method (Example 1)

Analysis
First, we find the height of the recursion tree
𝑛
Observe that a node at depth ′𝑖′ reflects a subproblem of size .
4 𝑖
𝑛
i.e. the subproblem size hits 𝑛 = 1, when =1
4 𝑖
𝑛
So, if 4 𝑖
=1
⟹𝑛= 4 𝑖 𝐴𝑝𝑝𝑙𝑦 𝐿𝑜𝑔 𝑏𝑜𝑡ℎ 𝑠𝑖𝑑𝑒
⟹ log 𝑛 = log 4 𝑖
⟹ 𝑖 = log 4 𝑛
So the height of the tree is log 4 𝑛.
Recursion Tree Method (Example 1)
Second, we determine the cost of each level of the tree.
The number of nodes at depth ′𝑖′ is 3 𝑖 . It was observed that
2 3 𝑖
3 3 3 3
𝑇 𝑛 = 𝑐𝑛2 + 𝑐𝑛2 + 𝑐𝑛2 + 𝑐𝑛2 + ⋯ + 𝑐𝑛2 + (3)𝑖
16 16 16 16
𝑛 2
So, each node at depth ′𝑖 ′ 𝑖. 𝑒. 𝑖 = 0,1,2,3,4, … , log 4 𝑛 − 1 has cost 𝑐 .
4𝑖
𝑛 2
Hence the total cost at level ′𝑖′ is 3𝑖 𝑐 4𝑖
𝑖
𝑖
𝑛 2
𝑖
𝑛2 3𝑖 2
3
⟹ 3 . 𝑐. 𝑖 ⟹ 3 . 𝑐. 𝑖 ⟹ 𝑖 . 𝑐. 𝑛 ⟹ . 𝑐. 𝑛2
4 16 16 16
Recursion Tree Method (Example 1)

However, the bottom level is special. Each of the bottom node has
contribute cost = 𝑇(1)
Hence the cost of the bottom level is = 3𝑖
⟹ 3log4 𝑛 𝑎𝑠 𝑖 = log 4 𝑛 𝑡ℎ𝑒 ℎ𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑡𝑟𝑒𝑒
⟹ 𝑛log4 3
So, the total cost of entire tree is
𝑇 𝑛
2 3 𝑖
3 3 3 3
= 𝑐𝑛2 + 𝑐𝑛2 + 𝑐𝑛2 + 𝑐𝑛2 + ⋯ + ⋯ + 𝑐𝑛2 + Θ(𝑛log4 3 )
16 16 16 16
log4 𝑛 𝑖
3
𝑇 𝑛 = ෍ 𝑐𝑛2 + Θ(𝑛log4 3 )
16
𝑖=0
Recursion Tree Method (Example 1)
The left term is just a sum of Geometric series. So the value of 𝑇 𝑛 is as follows.

3ൗ log4 𝑛 − 1
𝑇 𝑛 = 16 𝑐𝑛2 + Θ(𝑛log4 3 )
3
−1
16
The above equation looks very complicated. So, we use an infinite geometric series as an upper
bound. Hence the new form of the equation is given below:
log4 𝑛−1 𝑖
3
𝑇 𝑛 = ෍ 𝑐𝑛2 + Θ(𝑛log4 3 )
𝑖=0
16 𝑆𝑢𝑚 𝑜𝑓 𝑓𝑖𝑛𝑖𝑡𝑒 𝐺𝑒𝑜𝑚𝑒𝑡𝑟𝑖𝑐 𝑃𝑟𝑜𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛
∞ 𝑛
3
𝑖 2
𝑆𝑛 = 𝑎 + 𝑎𝑟 + 𝑎𝑟 + . . . + 𝑎𝑟
𝑇 𝑛 ≤෍ 𝑐𝑛2 + Θ(𝑛log4 3 )
16
𝑖=0 𝑛
1 𝑟 𝑛+1 −1
2 log4 3
𝑇 𝑛 ≤ 𝑐𝑛 + Θ(𝑛 )
1−
3 𝑆𝑛 = ෍ 𝑎𝑟 𝑖 = 𝑎
16 𝑟−1
𝑇 𝑛 ≤
16 2
𝑐𝑛 + Θ(𝑛log4 3 )
𝑖=0
13
𝑇 𝑛 = Ο(𝑛2 )
Recursion Tree Method (Example 2)
Example 2
𝑛
Solve the recurrence 𝑇 𝑛 = 4𝑇 + 𝑛 by using recursion tree
2
method.
𝑛
𝑇 𝑛 = 4𝑇 + 𝑐𝑛 , 𝑐 > 0
2
The term 𝑐𝑛, at the root represent the costs incurred by the
𝑛
subproblems of size .
2

Construction of Recursion tree

𝑇(𝑛)

Fig –(a)

Figure (a) shows T(n), which progressively expands in (b) to form

recursion tree.
Recursion Tree Method (Example 2)
𝑛
𝑇 𝑛 = 4𝑇 + 𝑐𝑛 , 𝑐 > 0
2
𝑐𝑛 𝑐𝑛

𝑛 4
𝑛 𝑐 𝑐𝑛
𝑛 𝑐 2
𝑐
𝑛
2 2
𝑐
2 2
log 2 𝑛

2
4
𝑛 𝑛 𝑇
𝑛
𝑛 𝑛 𝑇
𝑛
𝑐𝑛
𝑇
4 𝑇
𝑛
𝑇
𝑛
4
𝑇
4 𝑇
𝑛
4 𝑇
𝑛
𝑇
𝑛 𝑇
𝑛
4 𝑇
𝑛
𝑛 𝑛 𝑇
𝑛
4 𝑇
4
𝑇
4
4
2
4 4 4 𝑇 4
4 𝑇
4 4 3
4
𝑐𝑛
2
𝑖
4
𝑐𝑛
2

𝑇 1 𝑇 1 𝑇 1 ………………………..𝑇 1 4 𝑖

Fig –(b)
Recursion Tree Method (Example 2)

Analysis
First, we find the height of the recursion tree
𝑛
Observe that a node at depth ′𝑖′ reflects a subproblem of size .
2 𝑖
𝑛
i.e. the subproblem size hits 𝑛 = 1, when =1
2 𝑖
𝑛
So, if =1
2 𝑖
⟹𝑛= 2 𝑖 𝐴𝑝𝑝𝑙𝑦 𝐿𝑜𝑔 𝑏𝑜𝑡ℎ 𝑠𝑖𝑑𝑒
⟹ log 𝑛 = log 2 𝑖
⟹ 𝑖 = log 2 𝑛
So the height of the tree is log 2 𝑛.
Recursion Tree Method (Example 2)
Second, we determine the cost of each level of the tree.
The number of nodes at depth ′𝑖′ is 4 𝑖 .
So, each node at depth ′𝑖 ′ 𝑖. 𝑒. 𝑖 = 0,1,2,3,4, … … , log 2 𝑛 − 1 has cost
𝑛
𝑐 2𝑖 .
𝑛
Hence the total cost at level ′𝑖′ is 4𝑖 𝑐 2𝑖
𝑖
𝑛
⟹ 4 . 𝑐. 𝑖
2
𝑖
𝑛
⟹ 4 . 𝑐. 𝑖
2
𝑖
4
⟹ 𝑖 . 𝑐𝑛
2
𝑖
4
⟹ . 𝑐𝑛
2
Recursion Tree Method (Example 2)
However, the bottom level is special. Each of the bottom node has contribute
cost = 𝑇(1)
Hence the cost of the bottom level is = 4𝑖
⟹ 4log2 𝑛 𝑎𝑠 𝑖 = log 2 𝑛 𝑡ℎ𝑒 ℎ𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑡𝑟𝑒𝑒
⟹ 𝑛log2 4

So, the total cost of entire tree is

2 3 𝑖
4 4 4 4
𝑇 𝑛 = 𝑐𝑛 + 𝑐𝑛 + 𝑐𝑛 + 𝑐𝑛 + ⋯ + ⋯ + 𝑐𝑛 + Θ(𝑛log2 4 )
2 2 2 2
log2 𝑛 𝑖
4
𝑇 𝑛 = ෍ 𝑐𝑛 + Θ(𝑛log2 4 )
2
𝑖=0
Recursion Tree Method (Example 2)
The left term is just a sum of Geometric series. So the value of 𝑇 𝑛 is as follows.
log2 𝑛
4
−1
2
𝑇 𝑛 = 𝑐𝑛 + Θ(𝑛log2 4 )
4
−1
2
log2 𝑛
2 −1
𝑇 𝑛 = 𝑐𝑛 + Θ(𝑛2 )
2−1
log2 2
𝑛 −1
𝑛 = 𝑐𝑛 + 𝑐𝑛2
2−1
𝑛−1
𝑇 𝑛 = 𝑐𝑛 + 𝑐𝑛2
1
𝑇 𝑛 = 𝑐𝑛2 − 𝑐𝑛 + 𝑐𝑛2
𝑇 𝑛 = 2𝑐𝑛2 − 𝑐𝑛
𝐻𝑒𝑛𝑐𝑒, 𝑇 𝑛 = Θ(𝑛2 )
Recursion Tree Method (Example 3)
Example 3
𝑛
Solve the recurrence 𝑇 𝑛 = 2𝑇 2 + Θ(𝑛) by using recursion tree
method.
𝑇(𝑛) = 2𝑇(𝑛/2) + 𝑐𝑛 , 𝑐 > 0
The term 𝑐𝑛, at the root represent the costs incurred by the
𝑛
subproblems of size 2 .
Construction of Recursion tree
𝑇(𝑛)
Fig –(a)
Figure (a) shows T(n), which progressively expands in (b) to form
recursion tree.
Recursion Tree Method (Example 3)
𝑇(𝑛) = 2𝑇(𝑛/2) + 𝑐𝑛 , 𝑐 > 0

𝑐𝑛 𝑛

𝑛 𝑛 𝑛
𝑐 𝑐 2 =𝑛
2 2 2
log 2 𝑛

𝑛 𝑛 𝑛 𝑛 𝑛
𝑇 𝑇 𝑇 𝑇 22 =𝑛
4 4 4 4 22

𝑛
2𝑖 =𝑛
2𝑖
𝑇 1 𝑇 1 𝑇 1 ………………………..𝑇 1
2𝑖
Fig –(b)
Recursion Tree Method (Example 3)
Analysis
First, we find the height of the recursion tree
𝑛
Observe that a node at depth ′𝑖′ reflects a subproblem of size .
2 𝑖
𝑛
i.e. the subproblem size hits 𝑛 = 1, when =1
2 𝑖
𝑛
So, if =1
2 𝑖
⟹𝑛= 2 𝑖 𝐴𝑝𝑝𝑙𝑦 𝐿𝑜𝑔 𝑏𝑜𝑡ℎ 𝑠𝑖𝑑𝑒
⟹ log 𝑛 = log 2 𝑖
⟹ 𝑖 = log 2 𝑛
So the height of the tree is log 2 𝑛.
Recursion Tree Method (Example 3)
Second, we determine the cost of each level of the tree.
The number of nodes at depth ′𝑖′ is 4 𝑖 .
𝑛
So, each node at depth ′𝑖 ′ 𝑖. 𝑒. 𝑖 = 0,1,2,3,4, … … , log 2 𝑛 − 1 has cost 𝑐 .
2𝑖
𝑛
Hence the total cost at level ′𝑖′ is 2𝑖 𝑐 2𝑖 = 𝑐𝑛
However, the bottom level is special. Each of the bottom node has contribute
cost = 𝑇(1)
Hence the cost of the bottom level is = 2𝑖
⟹ 2log2 𝑛 𝑎𝑠 𝑖 = log 2 𝑛 𝑡ℎ𝑒 ℎ𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑡𝑟𝑒𝑒
⟹ 𝑛log2 2
⟹𝑛
Recursion Tree Method (Example 3)

So, the total cost of entire tree is

𝑇 𝑛 = 𝑐𝑛 + 𝑐𝑛 + 𝑐𝑛 + 𝑐𝑛 + ⋯ + ⋯ + 𝑐𝑛

log 𝑛
𝑇 𝑛 = 𝑐𝑛 σ𝑖=02 1𝑖 .

𝑇 𝑛 = 𝑐𝑛 (log 2 𝑛 + 1)

𝑇 𝑛 = 𝑐𝑛 log 2 𝑛 + 𝑐𝑛

𝐻𝑒𝑛𝑐𝑒, 𝑇 𝑛 = Θ(𝑛 log 2 𝑛)

Recursion Tree Method (Example 4)
Example 4
𝑛 𝑛 𝑛
Solve the recurrence 𝑇 𝑛 = 𝑇 2 + 𝑇 4 + 𝑇 8 + Θ(𝑛) by using
recursion tree method.
𝑛 𝑛 𝑛
𝑇 𝑛 =𝑇 +𝑇 +𝑇 + 𝑐𝑛 , 𝑐 > 0
2 4 8
The term 𝑐𝑛, at the root represent the costs incurred by the subproblems
𝑛 𝑛 𝑛
of size 2 , 4 , 𝑎𝑛𝑑 8 .
Construction of Recursion tree
𝑇(𝑛)
Fig –(a)
Figure (a) shows T(n), which progressively expands in (b) to form
recursion tree.
Recursion Tree Method (Example 4)
𝑛 𝑛 𝑛
𝑇 𝑛 =𝑇 +𝑇 +𝑇 + 𝑐𝑛 , 𝑐 > 0
2 4 8
𝑐𝑛 𝑐𝑛

𝑛 𝑛
𝑛 𝑐 7
𝑐 𝑐 8
2 4 𝑐𝑛
8
log 2 𝑛

2
𝑇
𝑛
𝑇
𝑛
𝑇
𝑛
𝑇
𝑛
8
𝑇
𝑛
16
𝑇
𝑛
𝑇
𝑛
𝑇
𝑛
𝑇
𝑛
7
2 4 8 32 16 32 64
𝑐𝑛
8

𝑖
7
𝑐𝑛
8

𝑇 1 𝑇 1 𝑇 1 ………………………..𝑇 1 𝑐𝑛
Fig –(b)
Recursion Tree Method (Example 4)
Analysis
First, we find the height of the recursion tree

𝑛 𝑛 𝑛
Hear the problem divide into three subproblem of size 𝑖 , 𝑖 , 𝑎𝑛𝑑 𝑖 .
2 4 8
For calculating the height of the tree, we consider the longest path of the tree. It has
been observed that the node on the left-hand side is the longest path of the tree.
𝑛
Hence the node at depth ′𝑖′ reflects a subproblem of size .
2𝑖

𝑛
i.e. the subproblem size hits 𝑛 = 1, when =1
2𝑖
𝑛
So, if =1
2𝑖
⟹ 𝑛 = 2𝑖 𝐴𝑝𝑝𝑙𝑦 𝐿𝑜𝑔 𝑏𝑜𝑡ℎ 𝑠𝑖𝑑𝑒
⟹ log 𝑛 = log 2 𝑖
⟹ 𝑖 = log 2 𝑛
So the height of the tree is log 2 𝑛.
Recursion Tree Method (Example 4)
𝑛 𝑛 𝑛
Second, we determine the cost of the tree in level ′𝑖′ = + +
2𝑖 4𝑖 8𝑖
So, the total cost of the tree is:
2 3
7 7 7
𝑇 𝑛 = 𝑐𝑛 + 𝑐𝑛 + 𝑐𝑛 + 𝑐𝑛 + ⋯ + ⋯
8 8 8
For simplicity we take ∞ Geometric Series
2 3
7 7 7
𝑇 𝑛 = 𝑐𝑛 + 𝑐𝑛 + 𝑐𝑛 + 𝑐𝑛 + ⋯ + ∞
8 8 8
∞ 𝑖
7
𝑇 𝑛 ≤෍ 𝑐𝑛
8
𝑖=0
1
𝑇 𝑛 ≤ 𝑐𝑛
7
1−
8
8
𝑇 𝑛 ≤ 𝑐𝑛
1
𝑇 𝑛 ≤ 8𝑐𝑛
𝐻𝑒𝑛𝑐𝑒, 𝑇 𝑛 = Ο(𝑛)
Design and Analysis of Algorithm

(Heap Sort)

Lecture -14 - 17
Overview

• O(n lg n) worst case time complexity.

• Sorts in place-like insertion sort.

• O(n) is the tight bound analysis of Build Heap.

Heap data structure
Example
Example

• Given an array of size N. The task is to sort the

array elements by using Heap Sort.
– Input:
– N=10
– Arr[]:{16, 4, 10, 14, 7, 9, 3, 2, 8, 1}
– Output: 1 2 3 4 7 8 9 10 14 16
Example

• Given an array of size N. The task is to sort the

array elements by using Heap Sort.
– Input:
– N = 10
– arr[] = {10,9,8,7,6,5,4,3,2,1}
– Output:1 2 3 4 5 6 7 8 9 10
Heap property

• For max-heaps (largest element at root),

max-heap property: for all nodes i ,
excluding the root, A[PARENT(i )] ≥ A[i ].
• For min-heaps (smallest element at root),
min-heap property: for all nodes i ,
excluding the root, A[PARENT(i )] ≤ A[i ].
Maintaining the heap property
The Time Complexity
of MAX-HEAPIFY (A,i,n)
is Ο(log 𝑛).
[Because any element
insert in a tree
required maximum
log n time]
Building a heap
Example
Building a max-heap from the following unsorted array results in the
first heap example.
Correctness
Initialization:

Initialization: we know that each node n / 2 + 1, n / 2 + 2, . . . , n is a

leaf, which is the root of a trivial max-heap. Since i = n / 2 before the
first iteration of the for loop, the invariant is initially true.

Maintenance: Children of node i are indexed higher than i , so by the

loop invariant, they are both roots of max-heaps. Correctly assuming
that i+1, i+2, . . . , n are all roots of max-heaps, MAX-HEAPIFY makes
node i a max-heap root. Decrementing i re-establishes the loop
invariant at each iteration.

Termination: When i = 0, the loop terminates. By the loop invariant,

each node, notably node 1, is the root of a max-heap.
Analysis
• Simple bound: O(n) calls to MAX-HEAPIFY,
each of which takes O(lg n) time ⇒ O(n lg n).

•Tighter analysis observation:

An n element heap has height log 𝑛 and at
𝑛
most nodes of any height h.
2ℎ +1
Tighter analysis Proof

BUILD-MAX-HEAP(A,n) 9
𝒇𝒐𝒓 𝒊 ← 𝒏/𝟐 𝒅𝒐𝒘𝒏𝒕𝒐 𝟏
do MAX-HEAPIFY (A,i,n) 6 5

0 8 2 1
1 2 3 4 5 6 7 8
9 6 5 0 8 2 1 3
3
Tighter analysis Proof
• For easy understanding, Let us take a complete binary
Tree,

• The height of a node is the number of edges from the

node to the deepest leaf.
• The depth of a node is the no of edges from the root of
the node.
Tighter analysis Proof
• All the leaves are of height 0, therefore
there are 8 nodes at height 0.
• 4 numbers of nodes at height 1.
• 2 numbers of nodes at height 2.
• and one node at height 3.
Tighter analysis Proof
• Hence the question is how many nodes are
there at height ‘h’ in a complete binary tree?
• The answer is :
• If there are n nodes in tree, then at most
𝑛
nodes are available at height h.
2ℎ+1
Tighter analysis Proof
• Now if we apply MAX-HEAPIFY() on any node of any
level, then the time taken by MAX-HEAPIFY() is the
𝑛
height of the node.(i.e. ℎ+1 Ο(ℎ))
2
• Hence in case of root the time taken is log 𝒏.
• Hence
log 𝑛
𝑛
𝑇 𝑛 = ෍ Ο(ℎ)
2ℎ+1
ℎ=0
Tighter analysis Proof
log 𝑛
𝑛
𝑇 𝑛 = ෍ Ο(ℎ)
2ℎ+1
ℎ=0
ℎ
≤ Ο 𝑛 σ∞
ℎ=0 2ℎ 2
𝑛 ∞ ℎ
≤Ο σℎ=0 ℎ
2 2
𝑛 ∞ 1 ℎ
≤Ο σ ℎ 2
2 ℎ=0

𝑛 1Τ 𝑛
≤Ο 2
⟹ Ο(2 2 )
2 1−1Τ2 2
T(n) ⟹ Ο 𝑛
Hence the running time of BUILD-MAX-HEAP(A,n) is Ο 𝑛 in tight bound .
Tighter analysis Proof
∞
1
෍ 𝑥𝑘 = value of Infanite G P Series
1−𝑥
𝑘=0
∞

෍ 𝑥 𝑘 = (1 − 𝑥)−1
𝑘=0
Differentiate both side:
∞
1
෍ 𝑘. 𝑥 𝑘−1 = −1 1 − 𝑥 −2
−1 =
(1 − 𝑥)2
𝑘=0
Multiply x both side
∞
𝑥
෍ 𝑘. 𝑥 𝑘 ==
(1 − 𝑥)2
𝑘=0
The heapsort algorithm
Given an input array, the heapsort algorithm acts as
follows:
• Builds a max-heap from the array.
• Starting with the root (the maximum element), the
algorithm places the maximum element into the
correct place in the array by swapping it with the
element in the last position in the array.
• “Discard” this last node (knowing that it is in its
correct place) by decreasing the heap size, and calling
MAX-HEAPIFY on the new (possibly incorrectly-placed)
root.
• Repeat this “discarding” process until only one node
(the smallest element) remains, and therefore is in the
correct place in the array.
Example
Analysis
• BUILD-MAX-HEAP: O(n)
• for loop: n − 1 times
• exchange elements: O(1)
• MAX-HEAPIFY: O(lg n)

Total time: O(n lg n).

Heap implementation of
priority queue
• Heaps efficiently implement priority
queues. These notes will deal with max
priority queues implemented with max-
heaps. Min-priority queues are
implemented with min-heaps similarly.
• A heap gives a good compromise between
fast insertion but slow extraction and vice
versa. Both operations take O(lg n) time.
Priority queue
• Maintains a dynamic set S of elements.
• Each set element has a key-an associated value.
• Max-priority queue supports dynamic-set operations:
• INSERT(S, x): inserts element x into set S.
• MAXIMUM(S): returns element of S with largest key.
• EXTRACT-MAX(S): removes and returns element of S
with largest key.
• INCREASE-KEY(S, x, k): increases value of element x’s
key to k. Assume k ≥ x’s current key value.
• Example max-priority queue application: schedule jobs on
shared computer.
• Min-priority queue supports similar operations:
• INSERT(S, x): inserts element x into set S.
• MINIMUM(S): returns element of S with
smallest key.
• EXTRACT-MIN(S): removes and returns
element of S with smallest key.
• DECREASE-KEY(S, x, k): decreases value of
element x’s key to k. Assume k ≤ x’s current
key value.
• Example min-priority queue application:
event - driven simulator.
Finding the maximum element

Getting the maximum element is easy: it’s

the root.
HEAP-MAXIMUM(A)
return A[1]
Finding the maximum element

Getting the maximum element is easy: it’s

the root.
HEAP-MAXIMUM(A)
return A[1]
Time: 𝚶(1).
Extracting max element
Given the array A:
• Make sure heap is not empty.
• Make a copy of the maximum element (the root).
• Make the last node in the tree the new root.
• Re-heapify the heap, with one fewer node.
• Return the copy of the maximum element.
Extracting max element
Given the array A:
• Make sure heap is not empty.
• Make a copy of the maximum element (the root).
• Make the last node in the tree the new root.
• Re-heapify the heap, with one fewer node.
• Return the copy of the maximum element.
HEAP-EXTRACT-MAX(A, n)
if n < 1
then error .heap underflow.
max ← A[1]
A[1] ← A[n]
MAX-HEAPIFY(A, 1, n − 1) remakes heap
return max
Extracting max element
Given the array A:
• Make sure heap is not empty.
• Make a copy of the maximum element (the root).
• Make the last node in the tree the new root.
• Re-heapify the heap, with one fewer node.
• Return the copy of the maximum element.
HEAP-EXTRACT-MAX(A, n) Time Complexity
if n < 1
then error .heap underflow.
is Ο(log 𝑛)
max ← A[1]
A[1] ← A[n]
MAX-HEAPIFY(A, 1, n − 1) remakes heap
return max
• HEAP-INCREASE-KEY(A,i,key)
1. If key<A[i]
2. error” new key is smaller than the
current key”.
3. A[i]= key
4. While i>1 and A[parent(i)]<A[i]
5. swap(A[parent(i)], A[i])
6. i=parent(i)
• HEAP-INCREASE-KEY(A,i,key)
1. If key<A[i]
2. error” new key is smaller than the
current key”.
3. A[i]= key
4. While i>1 and A[parent(i)]<A[i]
5. swap(A[parent(i)], A[i])
6. i=parent(i)
The running time of HEAP-INCREASE-
KEY(A,i,key) is Ο(log 𝑛)
• Example(from own)
• MAX-HEAP-INSERT(A,key)
1. A.heap-size= A.heap-size+1
2. A[heap-size]=-∞
3. HEAP-INCREASE-KEY(A,heap-size,key)
• MAX-HEAP-INSERT(A,key)
1. A.heap-size= A.heap-size+1
2. A[heap-size]=-∞
3. HEAP-INCREASE-KEY(A,heap-size,key)
The running time of MAX-HEAP-INSERT(A,key) is
Ο(log 𝑛).
Design and Analysis of Algorithm

Linear Time Sorting

(Counting Sort)

Lecture -18
Overview

• Running time of counting sort is O(n+k).

• Required extra space for sorting.
• Is a stable sorting.
Counting Sort
• Counting sort is a type of sorting technique
which is based on keys between a specific
range.
• It works by counting the number of objects
having distinct key values (i.e. one kind of
hashing).
Counting Sort
• Consider the input set : 4, 1, 3, 4, 3. Then n=5 and
k=4.
• Counting sort determines for each input element 𝑥,
the number of elements less than 𝑥.
• This information is uses to place element 𝑥 directly
into its position in the output array.
• For example if there exits 17 elements less that x
then x is placed into the 18th position into the
output array.
Counting Sort
• Assumptions:
• n records
• Each record contains keys or data
• All keys are in the range of 0 to k, where k is
the highest key value of the array.
• Space:
For coding this algorithm uses three array:
• Input Array: A[1..n] store input data , where n is the
length of the array.
• Output Array: B[1..n] finally store the sorted data
• Temporary Array: C[0..k] store data temporarily
Counting Sort
• Let us illustrate the counting sort with an example.
Apply the concept of counting sort on the given array.
1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
Counting Sort
• Let us illustrate the counting sort with an example.
Apply the concept of counting sort on the given
array.
1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3

• First create a new array C[0…..k] , where k is the

highest key value. And initialize with 0(i.e. zero)

for i=0 to k 0 1 2 3 4 5
C 0 0 0 0 0 0
C[i]= 0;
Counting Sort
• Let us illustrate the counting sort with an example.
Apply the concept of counting sort on the given array.
1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3

• Find the frequencies of each object and store it in C

array.
for j=1 to A. length
C[ A[j] ] = C[ A[j] ] + 1; 0 1 2 3 4 5
C 2 0 2 3 0 1
Counting Sort
• Let us illustrate the counting sort with an example.
Apply the concept of counting sort on the given array.
1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3

• Find the frequencies of each object and store it in C

array.
for j=1 to A. length
0 1 2 3 4 5
C[ A[j] ] = C[ A[j] ] + 1; C 2 0 2 3 0 1

• And then cumulatively add C array.

for i=1 to k 0 1 2 3 4 5
C[i] = C[i] + C[i-1]; C 2 2 4 7 7 8
Counting Sort
for j=A. length down to 1
B[C[ A[j] ]] = A[j]; 1 2 3 4 5 6 7 8
C[ A[j] ] = C[ A[j] ] - 1; A 2 5 3 0 2 3 0 3
j

1 2 3 4 5 6 7 8 0 1 2 3 4 5
B 3 C 2 2 4 7 7 8
Counting Sort
for j=A. length down to 1
B[C[ A[j] ]] = A[j]; 1 2 3 4 5 6 7 8
C[ A[j] ] = C[ A[j] ] - 1; A 2 5 3 0 2 3 0 3
j

1 2 3 4 5 6 7 8 0 1 2 3 4 5
B 3 C 2 2 4 7 7 8