Time Complexity
Time Complexity
Time Complexity
Time Complexity
Dr. Viswanathan V.
Professor
School of Computer Science and Engineering
Vellore Institute of Technology, Chennai
Algorithm analysis
• Algorithm - Step by step procedure to solve the
problem/task
3
How can I measure the performance of an
algorithm?
4
Measuring the performance of an
algorithm …
5
Asymptotic notations
Asymptotic Analysis is to measure the performance of
an algorithm in terms of input size (without measuring
the actual running time) and calculate, how the time (or
space) taken by an algorithm increases with the input
size. This is also known as an algorithm’s growth rate.
Three types of asymptotic notations to represent the
growth of any algorithm
• Big Oh(O)
• Big Omega (Ω)
• Big Theta (Θ)
6
Big O notation
cg(n)
time
f(n)
n
no
Data input
f(n) = O (n)
Proof:
3n+2 c n
When c=4; 3n+2 4 n
n2
f(n) = O (n)
8
- notation
f(n)
time
cg(n)
n
no
Data input
f(n) = (g(n))
f(n) c g(n) , c>0 and no 1 , n no
f(n) = (g(n))
9
Example :
Let f(n) = 3n+2 g(n) = n
f(n) c g(n) , c>1 and no 1
f(n) = (n)
Proof:
3n+2 c n
When c=1; 3n+2 n
n1
f(n) = (n)
10
- notation
c2g(n)
f(n)
time
c1g(n)
f(n) = (g(n))
n
no
Data input
f(n) = (g(n))
11
Example :
Let f(n) = 3n+2 g(n) = n
f(n) c2g(n) , c2 4 and no 1
f(n) c1g(n) , c1 1 and no 1
f(n) = (n)
Proof:
3n+2 c n
When c=1; 3n+2 n
n1
When c=4; 3n+2 4 n
n1
f(n) = (n)
12
Common time complexities
13
Common time complexities ….
14
Common time complexities …
2
Quadratic time - O(n )
Ex: Duplicate elements in array
A function with a quadratic time complexity has a growth
2
rate of n . If the input is size 2, it will do four operations. If
the input is size 8, it will take 64, and so on.
c
Polynomial time - O(n ) c>1
c
Polynomial running is represented as O(n ), when c > 1.
n
Exponential time - O(2 )
Ex: Find all subsets
15
Common time complexities ….
BETTER
O(1) constant time
16
Types of Analysis
Worst case (O notation)
we calculate upper bound on running time on algorithm.
An absolute guarantee that the algorithm would not run
longer
No matter what the inputs are. (Maximum number of
operations to be executed to complete the task)
17
Types of Analysis ….
Best case ( notation)
In the best case analysis, we calculate lower bound on
running time on algorithm. Input is the one for which the
algorithm runs the fastest.
ie. Minimum number of operations to be executed to
complete the task.
18
Algorithm
20
Example 1:
Algorithm X()
{
int i
for i = 1 to n
print(“VIT”) basic operations
}
21
Example 2:
Algorithm X()
{
int i, j
for i = 1 to n
for j = 1 to n
print(“VIT”) basic operations
}
22
Example 3:
Algorithm X()
{
int i=1, j=1 i 1 2 3 4 5 6
while(j<=n)
{ j 1 3 6 10 15 21
i = i+1
j=j+i
print(“VIT”) basic operations
}
}
𝑘 (𝑘 + 1)
>𝑛
2
(𝑘 2 + 𝑘)
>𝑛
2
𝑘 = 𝑂( 𝑛)
23
Example 4:
Algorithm X() i 1 2 3 4 …. n
{
int i,j,k j 1 2 3 4 …. n
for i = 1 to n
{ k 100 200 300 400 …. n x 100
for j = 1 to i
{
for k= 1 to 100
print(“VIT”)
}
} 100+200+300+…….+n*100
}
100(1+2+3+…+n)
(𝑛2 + 𝑛)
100 ∗
2
i = 1 2 4 8 …….. n
20 21 22 23 2𝑘
𝒏 = 𝟐𝒌
log(𝒏) = 𝒌
25
Plan for Analysis of Recursive Algorithms
• Decide on a parameter indicating an input’s size.
• Identify the algorithm’s basic operation.
• Check whether the number of times the basic operation is
executed may vary on different inputs of the same size. (If
it may, the worst, average, and best cases must be
investigated separately.)
• Set up a recurrence relation with an appropriate initial
condition expressing the number of times the basic
operation is executed.
• Solve the recurrence (or, at the very least, establish its
solution’s order of growth) by backward substitutions or
another method.
26
Methods for analyzing recursive algorithm
• Substitution method
• Master Theorem
• Recursion - tree method
27
Example : Recursive evaluation of n!
Algorithm F(n)
//Computes n! recursively
//Input : A non negative integer ‘n’ M - multiplications
//Output : The value of n!
if n=0 return 1
else return F(n-1) * n
Size: n
Basic operation: multiplication
Recurrence relation: M(n) = M(n-1) + 1
M(0) = 0 28
Solving the recurrence for M(n)
29
Solve the recurrence relation using Recursion Tree
Example 1: T(n) = 2T(n/2)+n
n n
(n/2) (n/2) n
log2n
…… ……
…
n
1 1 1 1 1 1 1 1 1 1 1 1 1
Total: nlogn
Time complexity is O (n log n) 30
Solve the recurrence relation using Recursion Tree …
Example 2: T(n) = T(n/3)+T(2n/3) +n
n n
(n/3) (2n/3) n
log3/2n
log3n
…… ……
…
1 1 1 1 1 1 1 1 1 1 1 1 1 n
Total: n log n
Time complexity is O(nlogn ) 31
Master Theorem
Master theorem is used for calculate the time complexity of
recurrence relations (divide and conquer algorithms) in a
simple and quick way.
32
Master Theorem ….
Cases :
1. a < bk T(n) ∈ Θ(nk)
2. a = bk T(n) ∈ Θ(nk log n )
3. a > bk T(n) ∈ Θ(nlogb a)
• Examples:
– T(n) = T(n/2) + 1 Θ(log n)
– T(n) = 2T(n/2) + n Θ(nlog n)
– T(n) = 3T(n/2) + n Θ(nlog23)
– T(n) = T(n/2) + n Θ(n)
33