Amity School of Engineering &

Technology
Algorithm
Algorithm is a step-by-step procedure, which
defines a set of instructions to be executed
in a certain order to get the desired output.
Algorithms are generally created
independent of underlying languages, i.e. an
algorithm can be implemented in more than
one programming language.
 Amity School of Engineering &

Characteristics of an Algorithm Technology

• Unambiguous − Algorithm should be clear and

unambiguous. Each of its steps (or phases), and their
inputs/outputs should be clear and must lead to only one
meaning.
• Input − An algorithm should have 0 or more well-defined
inputs.
• Output − An algorithm should have 1 or more well-
defined outputs, and should match the desired output.
• Finiteness − Algorithms must terminate after a finite
number of steps.
• Feasibility − Should be feasible with the available
resources.
• Independent − An algorithm should have step-by-step
directions, which should be independent of any
programming code.
Amity School of Engineering &
Technology
 Amity School of Engineering &
Algorithm Complexity Technology

Suppose X is an algorithm and n is the size of input

data, the time and space used by the algorithm X
are the two main factors, which decide the
efficiency of X.
• Time Factor − Time is measured by counting the
number of key operations such as comparisons in
the sorting algorithm.
• Space Factor − Space is measured by counting
the maximum memory space required by the
algorithm.
The complexity of an algorithm f(n) gives the
running time and/or the storage space required by
the algorithm in terms of n as the size of input data.
 Amity School of Engineering &

Analysis of Algorithm
Technology

Analysis of algorithm is the process of

analyzing the problem-solving capability of
the algorithm in terms of the time and size
required (the size of memory for storage
while implementation). However, the main
concern of analysis of algorithms is the
required time or performance. Generally, we
perform the following types of analysis −
 Amity School of Engineering &
Technology

• Worst-case − The maximum number

of steps taken on any instance of
size n.
• Best-case − The minimum number of
steps taken on any instance of size n.
• Average case − An average number
of steps taken on any instance of
size n.
 Amity School of Engineering &
Technology
Asymptotic Notations
• Execution time of an algorithm depends on
the instruction set, processor speed, disk
I/O speed, etc. Hence, we estimate the
efficiency of an algorithm asymptotically.
• Time function of an algorithm is
represented by T(n), where n is the input
size.
 Amity School of Engineering &
Technology
Different types of asymptotic notations are
used to represent the complexity of an
algorithm. Following asymptotic notations
are used to calculate the running time
complexity of an algorithm.
• O − Big Oh
• Ω − Big omega
• θ − Theta
• o − Little Oh
• ω − Little omega
 Amity School of Engineering &
• Big O Notation: The Big O notation defines Technology

an upper bound of an algorithm, it bounds a

function only from above.

O(g(n)) = { f(n): there exist positive constants

c and n0 such that 0 <= f(n) <= c*g(n) for all n
>= n0}
 Amity School of Engineering &
Technology
• Ω Notation: Just as Big O notation provides an
asymptotic upper bound on a function, Ω notation
provides an asymptotic lower bound.

• Ω (g(n)) = {f(n): there exist positive constants c and

n0 such that 0 <= c*g(n) <= f(n) for all n >= n 0}.
 Amity School of Engineering &
• Θ Notation: The theta notation bounds a Technology

functions from above and below, so it

defines exact asymptotic behavior.

Θ(g(n)) = {f(n): there exist positive constants

c1, c2 and n0 such that 0 <= c1*g(n) <= f(n)
<= c2*g(n) for all n >= n0}
 Amity School of Engineering &

Little ο asymptotic notation

Technology

• Big-Ο is used as a tight upper-bound on

the growth of an algorithm’s effort (this
effort is described by the function f(n)),
even though, as written, it can also be a
loose upper-bound. “Little-ο” (ο()) notation
is used to describe an upper-bound that
cannot be tight.
 Amity School of Engineering &
Technology

• Let f(n) and g(n) be functions that map positive

integers to positive real numbers. We say that
f(n) is ο(g(n)) (or f(n) ο(g(n))) if for any
real constant c > 0, there exists an integer
constant n0 ≥ 1 such that 0 ≤ f(n) < c*g(n).
 Amity School of Engineering &
Technology

• Intuitively, in the o-notation, the

function f(n) becomes insignificant relative
to g(n) as n approaches infinity; that is,
 Amity School of Engineering &
Little ω asymptotic notation Technology

• Let f(n) and g(n) be functions that

map positive integers to positive real
numbers. We say that f(n) is ω(g(n))
(or f(n) ∈ ω(g(n))) if for any real
constant c > 0, there exists an integer
constant n0 ≥ 1 such that
0<=c*g(n)<f(n)for every integer n ≥ n0.
 Amity School of Engineering &
Technology

• That is, f(n) becomes arbitrarily large

relative to g(n) as n approaches infinity.
 Amity School of Engineering &

Control Structures Technology

1. Sequence logic or Sequential flow

2. Selection logic or Conditional flow

3. Iteration logic or Repetitive flow

 Amity School of Engineering &
Technology
Space Complexity

• S(P)= C + SP

Where , S(P) is space required by a

problem P,
C is a constant and SP is an instance
characteristic.
 Amity School of Engineering &
Technology

Time Complexity

• T(P)= Compile Time+ Run Time (tp)

Run time is denoted by tp

We count the number of program steps for

finding execution time.
 Amity School of Engineering &
Technology

The number of steps of any program depends

on the kind of statement. Different statements
have different counts. For example
1. Comments count as zero steps.
2. Assignment statements are counted as one,
as they do not call any other algorithm.
3. For iterative statements such as for, while
etc. we count the steps only for control part.
 Amity School of Engineering &
Analysis of Loops
Technology

1) O(1): Time complexity of a function (or

set of statements) is considered as O(1)
if it doesn’t contain loop, recursion and
call to any other non-constant time
function.
 Amity School of Engineering &
Technology

• A loop or recursion that runs a constant

number of times is also considered as
O(1). For example the following loop is
O(1).
// Here c is a constant
for (int i = 1; i <= c; i++)
{ // some O(1) expressions }
 Amity School of Engineering &
Technology
2) O(n): Time Complexity of a loop is
considered as O(n) if the loop variables is
incremented / decremented by a constant
amount. For example following functions
have O(n) time complexity.
// Here c is a positive integer constant
for (int i = 1; i <= n; i += c)
{ // some O(1) expressions }
for (int i = n; i > 0; i -= c)
{ // some O(1) expressions }
 Amity School of Engineering &
• 3) O(np): Time complexity of nested loops is
Technology

equal to the number of times the innermost

statement is executed. For example the
following sample loops have O(n2) time
complexity.
• for (int i = 1; i <=n; i += c)
{ for (int j = 1; j <=n; j += c)
{ // some O(1) expressions } }
for (int i = n; i > 0; i -= c)
{ for (int j = n; j >0; j -= c)
{ // some O(1) expressions }}
• 4) O(Logn) Time Complexity Amity School of Engineering &
of a loop is
Technology

considered as O(Logn) if the loop

variables is divided / multiplied by a
constant amount.
for (int i = 1; i <=n; i *= c)
{ // some O(1) expressions }

for (int i = n; i > 0; i /= c)

{ // some O(1) expressions }
 Amity School of Engineering &
• 5) O(LogLogn) Time Complexity of a loop Technology

is considered as O(LogLogn) if the loop

variables is reduced / increased
exponentially by a constant amount.
// Here c is a constant greater than 1
for (int i = 2; i <=n; i = pow(i, c))
{ // some O(1) expressions }
//Here fun is sqrt or cuberoot or any other
constant root
for (int i = n; i > 1; i = fun(i))
{ // some O(1) expressions }
 Amity School of Engineering &
Technology

• How to combine time

complexities of
consecutive loops?
 Amity School of Engineering &
Technology

• When there are consecutive loops, we

calculate time complexity as sum of time
complexities of individual loops.
 Amity School of Engineering &
Example Technology

for (int i = 1; i <=m; i += c)

{ // some O(1) expressions }

for (int i = 1; i <=n; i += c)

{ // some O(1) expressions }
Time complexity of above code is O(m) +
O(n) which is O(m+n)
If m == n, the time complexity becomes
O(2n) which is O(n).
 Amity School of Engineering &
Technology

• How to calculate time

complexity when there are
many if, else statements
inside loops?
• As we know,worst case time complexity
Amity School of Engineering &
Technology

is the most useful among best, average

and worst. Therefore we need to
consider worst case. We evaluate the
situation when values in if-else
conditions cause maximum number of
statements to be executed.
When the code is too complex to
consider all if-else cases, we can get
an upper bound by ignoring if else and
other complex control statements.
 Amity School of Engineering &
Technology

What is the time complexity of following function

int fun(int n)
{
for (int i = 1; i <= n; i++)
{
for (int j = 1; j < n; j +=
i)
{
// Some O(1) task
}
}
}
 Amity School of Engineering &
Answer Technology

• O(nlogn)
 Amity School of Engineering &
Technology
What is the time complexity of
following code:
int a = 0, b = 0;
for (i = 0; i < N; i++)
{
a = a + rand();
}
for (j = 0; j < M; j++)
{
b = b + rand();
}
 Amity School of Engineering &
Answer Technology

• O(max(N, M)) or O(N + M)

 Amity School of Engineering &
Technology

What is time complexity of fun()?

int fun(int n)
{
int count = 0;
for (int i = n; i > 0; i /= 2)
for (int j = 0; j < i; j++)
count += 1;
return count;
}
 Amity School of Engineering &
Answer Technology

• O(n)
 What is time complexity of fun()?
Amity School of Engineering &
Technology

int fun(int n)
{
int count = 0;
for (int i = 0; i < n; i++)
for (int j = i; j > 0; j--)
count = count + 1;
return count;
}
 Amity School of Engineering &
Answer Technology

• O( n2 )
 What is time complexity of fun()?
Amity School of Engineering &
Technology

void fun(int n, int arr[])

{
int i = 0, j = 0;
for(; i < n; ++i)
while(j < n && arr[i] < arr[j])
j++;
}
 Amity School of Engineering &
Answer Technology

• O(n)
 What is time complexity ()?
Amity School of Engineering &
Technology

int a = 0, i = N;
while (i > 0)
{
a += i;
i /= 2;
}
 Amity School of Engineering &
Answer Technology

• O(log N)
 Amity School of Engineering &
Recurrences Technology

• Many algorithms are recursive in

nature. When we analyze them, we get a
recurrence relation for time complexity. We
get running time on an input of size n as a
function of n and the running time on
inputs of smaller sizes.
 Amity School of Engineering &
Technology

• A recurrence is an equation or
inequality that describes a
function in terms of its value on
smaller inputs. Recurrences are
generally used in divide-and-
conquer paradigm.
 Amity School of Engineering &
Technology

• Let us consider T(n) to be the running

time on a problem of size n.
• If the problem size is small enough,
say n < c where c is a constant, the
straightforward solution takes constant
time, which is written as θ(1). If the
division of the problem yields a
number of sub-problems with size n/b.
 Amity School of Engineering &
Technology

• To solve the problem, the required time

is a.T(n/b). If we consider the time
required for division is D(n) and the time
required for combining the results of sub-
problems is C(n), the recurrence relation
can be represented as −
 Amity School of Engineering &
Technology

There are mainly four ways for solving

recurrences.
1. Substitution Method
2. Iteration Method
3. Recurrence Tree Method
4. Master Method
 Amity School of Engineering &
Substitution Method Technology

The Substitution Method Consists of two

main steps:
(i) Guess the Solution.
(ii)Use the mathematical induction to find the
boundary condition and shows that the
guess is correct.
 Amity School of Engineering &
Consider the Recurrence Technology
 Amity School of Engineering &
Technology

• We guess the solution is O (n (logn)).

Thus for constant 'c'.
T (n) ≤c n logn
Put this in given Recurrence Equation.
Now, T (n) ≤2c(n/2)log(n/2) +n ≤cnlogn-
cnlog2+n =cn logn-n (clog2-1) ≤cn logn for
(c≥1)
Thus T (n) = O (n logn).
 Amity School of Engineering &
Iteration Methods Technology

• Consider the Recurrence

• T (n) = 1 if n=1
= 2T (n-1) if n>1
 Amity School of Engineering &
Technology

• T (n) = 2T (n-1)
• = 2[2T (n-2)] = 22T (n-2)
• = 4[2T (n-3)] = 23T (n-3)
• = 8[2T (n-4)] = 24T (n-4) …………….(Eq.1)

• Repeat the procedure for i times

T (n) = 2i T (n-i)
 Amity School of Engineering &
Technology

• Put n-i=1 or i= n-1 in (Eq.1)

• T (n) = 2n-1 T (1) = 2n-1 .1 {T (1)
=1 .....given} = 2n-1
= O(2n )
 Amity School of Engineering &
Consider the Recurrence Technology

T (n) = T (n-1) +1 and T (1) = θ (1).

 Amity School of Engineering &
Technology

• T (n) = T (n-1) +1
• = (T (n-2) +1) +1
• = (T (n-3) +1) +1+1
• = T (n-4) +4
• = T (n-5) +1+4 = T (n-5) +5
• = T (n-k) + k
• Where k = n-1 ,T (n-k) = T (1) = θ (1)
• T (n) = θ (1) + (n-1) = 1+n-1=n= θ (n).
 Amity School of Engineering &
Recursion Tree Method Technology

1. Recursion Tree Method is a pictorial

representation of an iteration method which
is in the form of a tree where at each level
nodes are expanded.
2. In general, we consider the second term
in recurrence as root.
3. It is useful when the divide & Conquer
algorithm is used.
 Amity School of Engineering &
Technology
4.It is sometimes difficult to come up with a
good guess. In Recursion tree, each root and
child represents the cost of a single
subproblem.
5. We sum the costs within each of the levels
of the tree to obtain a set of per-level costs
and then sum all per-level costs to determine
the total cost of all levels of the recursion.
6. A Recursion Tree is best used to generate a
good guess, which can be verified by the
Substitution Method.
 Amity School of Engineering &
Technology

• Consider T (n) = 2T(n/2) + n2

Amity School of Engineering &
Technology
Amity School of Engineering &
Technology
Amity School of Engineering &
Technology
 Amity School of Engineering &
Answer Technology

• O(n2 )
 Amity School of Engineering &
Consider the following Technology

recurrence
T (n) = 4T(n/2) +n

Obtain the asymptotic bound using recursion

tree method.
 Amity School of Engineering &
Answer Technology

• O(n2 )
 Amity School of Engineering &
Consider the following Technology

recurrence
 Amity School of Engineering &
Answer Technology

• O(n log 3/2 n) or O(n log n)

 Amity School of Engineering &
Consider the following Technology

recurrence
• T(n)= 3 T(n/4) +n2
 Amity School of Engineering &
Answer Technology

• O(n2 )
 Amity School of Engineering &
Consider the following Technology

recurrence
• T(n) = 2T(n/2) + n
 Amity School of Engineering &
Answer Technology

• O(nlogn)
 Amity School of Engineering &
Consider the following Technology

recurrence
• T(n) = T(n/5) + T(4n/5) + n
 Amity School of Engineering &
Technology

• O(nlog5/4n)
 Amity School of Engineering &
Master Method Technology

• The Master Method is used for solving the

following types of recurrence
T(n) = a T(n/b)+ f (n) where a≥1 and b>1 are
constants and f(n) is an asymptotically
positive function.
 Amity School of Engineering &
In the function to the analysis of a Technology

recursive algorithm, the constants and

function take on the following
significance:

• n is the size of the problem.

• a is the number of subproblems in the

recursion.

•
 Amity School of Engineering &
Technology
• f (n) is the sum of the work done
outside the recursive calls, which
includes the sum of dividing the
problem and the sum of combining
the solutions to the subproblems.
• It is not possible always bound the
function according to the requirement,
so we make three cases which will tell
us what kind of bound we can apply
on the function.
Amity School of Engineering &
Technology
 Amity School of Engineering &
Technology

• T(n)=9 T(n/3)+n
 Amity School of Engineering &
Answer Technology

Θ(n2)
 Amity School of Engineering &
Technology

• T(n)=T(2n/3) +1
 Amity School of Engineering &
Answer Technology

• Θ(log2 n)
 Amity School of Engineering &
Technology

• T(n)=3T(n/4)+n log2 n
 Amity School of Engineering &
Answer Technology

• Θ (n log 2 n)
 Amity School of Engineering &
Solve following Technology

• T (n) = 8 T(n/2)+1000n2

• T (n) = 2 T(n/2) +10n

• T (n) = 2T(n/2)+ n2
 Amity School of Engineering &
Answer Technology

• Θ (n3)

• Θ (n log n)

• Θ(n2)
 Solve following using Master Method
Amity School of Engineering &
Technology

(i)T (n) = 3T (n/2) + n 2

(ii)T (n) = 4T (n/2) + n 2
(iii)T (n) = 2nT (n/2) + n n
(iv)T (n) = 16T (n/4) + n
(v)T (n) = 2T (n/4) + n 0.51
(vi)T(n)=0.5 T(n/2) +1/n
(vii)T(n)=16 T(n/4)+n!
(viii)T(n)=2 T(n/2)+log n
(ix)T(n)=3T(n/2)+n
(x)T(n)=64 T(n/8)-n2logn
 Answer Amity School of Engineering &
Technology

(i)T (n) = 3T (n/2) + n 2 => Θ(n 2 )

(ii)T (n) = 4T (n/2) + n 2 => Θ(n 2 log n )
(iii)T (n) = 2nT (n/2) + n n => Does not apply
(iv)T (n) = 16T (n/4) + n => Θ(n 2 )
(v)T (n) = 2T (n/4) + n 0.51 => Θ(n 0.51 )
(vi)T(n)=0.5 T(n/2) +1/n => Does not apply
(vii)T(n)=16 T(n/4)+n! => Θ(n!)
(viii)T(n)=2 T(n/2)+log n => Θ(
(ix)T(n)=3T(n/2)+n => Θ(n lg3 )
(x)T(n)=64 T(n/8)-n2logn => Does not apply