Asymptotic Analysis-1

Asymptotic Analysis-1
Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/1

Asymptotic Analysis-1
Topics
Objectives
Asymptotic Notation
Definitions ( Big-Oh, Big-Omega and Theta)
Asymptotic Notation Theorems

Asymptotic Analysis
Objectives
The purpose of asymptotic analysis is to examine the behavior of an algorithm for large
input size. More specifically, if T(n) is the running time for an input of size n , we would
want to know the behavior or growth rate of T(n) for very large values of n. An analysis of
algorithm for large input is referred to as asymptotic analysis.
The asymptotic behavior of an algorithm is often compared to some standard

mathematical function, such as n2, n lg n etc The relationship or similarity of behavior is
often expressed by a special notation which is called asymptotic notation.
The standard asymptotic notations commonly used in the analysis of algorithms are known
as O (Big Oh), Ω (Big Omega), and θ(Theta).
Sometimes, additional notations o( small-oh) and ω( small-omega) are also used to show
the growth rates of algorithms

Asymptotic Notation

O-Notation
Definition
If f(n) is running time of an algorithm, and g(n) is some standard growth function such that
for some positive constants c and integer n0 ,
f(n) ≤ c.g(n) for all n ≥ n0
then f(n) = O(g(n)) (Read f(n) is Big-Oh of g(n) )
¾ The behavior of f(n) and g(n) is portrayed in the diagram. It follows that for n<n0, ,f(n) may
lie above or below g(n), but for all n ≥ n0, f(n) falls consistently below g(n)..
Trend of running time

O-Notation
Asymptotic Upper Bound
If f(n) = O(g(n)), then the function g(n) is called asymptotic upper bound of f(n)
Since the worst-case running time of an algorithm is the maximum running time for any
input, it would follow that g(n) provides an upper bound on the worst running time
The notation O( g(n) ) does not imply that g(n) is the worse running time; it simply means
that worst running time would never exceed upper limit determined by g(n).
Asymptotic Upper Bound of f(n)

O-Notation
Basic Method
Example(1): Using basic definition, we show that 3n2 + 10n = O(n2)
Consider, 10 ≤ n for n ≥ 10 ( obvious !)
10n ≤ n2 for n ≥ 10 ( Multiplying both sides with n )
3n2+10n ≤ 3n2 + n2 for n ≥ 10 ( Adding 3n2 to both sides )
=4n2 (Simplifying )
3n2 + 10 n ≤ c.n2 for n ≥ n0 where c =4 and n0= 10 (Solution)

Therefore, it follows from the basic definition that
3n2 +10n = O(n2)
¾ The choice of constant c is not unique. However, for each different c there is a
corresponding value of n0 which satisfies the basic relation .This behavior is illustrated
by the next example

O-Notation
Basic Method
Example(2): In the preceding example it was shown that 3n2 + 10n ≤ c.n2 for c=4, n0=10. We
now show that the relation holds true for a different value of c and corresponding n0.
Consider n ≤ n2 for n ≥ 1 ( Obvious)
10n ≤ 10n2 for n≥ 1 (Multiplying both sides with 10 )
3n2 +10 n ≤ 3n2 + 10n2 for n≥ 1 ( Adding 3n2 on both sides)
3n2 +10n ≤ 13n2 for n ≥ 1 ( Simplifying )
Or, 3n2 +10n ≤ c.n2, for n ≥ n0 where c=13, n0=1 ( Solution )
Therefore, by basic definition,
3n2 + 10n = O(n2)

O-Notation Example
Comparison of Growth Rates
The results of analysis in the preceding examples are plotted in the diagram below.
It can be seen that the function cg(n) =4n2 ( c= 4) overshoots the function f(n)= 3n2 + 10n
for n0=10. Also, the function cg(n) =13n2 ( c =13 ) grows faster than f(n) = 3n2 + 10n for
n0=1
cg(n)=4n2
f(n)=3n2 + 10 n
cg(n)= 13n2
n0=1 n0=10
Growth of functions 13n2 and 4n2 versus the function 3n2 + 10 n

Set Builder O-Notation
Definition
Consider the functions , say f1(n), f2(n), f3(n), ..fk(n) for which g(n) is the asymptotic
upper bound. By definition,
f1(n) ≤ c1 g(n) for n ≥ n1
f2(n) ≤ c2 g(n) for n ≥ n2
f3(n) ≤ c3 g(n) for n ≥ n3

……………………
fk(n) ≤ ck g(n) for ≥ nk
where c1, c2, c3 ,.. ck are constants and n1 ,n2 ,n3,.. nk are positive integers. The functions f1(n),
f2(n), f3(n), .. fk(n) are said to belong to the class O( g(n)). In set notation, the relation is
denoted by
O( g(n) )= { f1(n) , f2(n), f3(n)…,fk(n)}
Alternatively, using set-builder notation, if
O(g(n)) ={ f(n): there exist positive constants c and n0, such that f(n) ≤ c.g(n), for all n ≥ n0 }
then, f(n) ∈ O(g(n))

Ω-Notation
Definition
If f(n) is running time of an algorithm, and g(n) is some standard growth function such that
for some positive constants c, positive integer n0 ,
c.g(n) ≤ f(n) for all n ≥ n0
then f(n) = Ω(g(n)) (Read f(n) is Big-Omega of g(n) )
¾ The behavior of f(n) and g(n) is portrayed in the graph. It follows that for n < n0, f(n) may
lie above or below g(n), but for all n ≥ n0, f(n) falls consistently above g(n). It also implies that
g(n) grows slower than f(n)
Trend of running time

Ω-Notation
Asymptotic Lower Bound
If f(n) = Ω(g(n)), then the function g(n) is called asymptotic lower bound of f(n)
Since the best-case running time of an algorithm is the minimum running time for any input,
it would follow that g(n) provides a lower bound on best running time
As before, the notation Ω( g(n) ) does not imply that g(n) is the best running time; it simply
means that best running time would never be lower than g(n).
Asymptotic lower bound of f(n)

Ω-Notation
Basic Method
Example(1) : Using basic definition, we show that n2 -10n = Ω(n2).
For, n ≥ n/2 for n ≥ 0 ( Obvious !)
n - 10 ≥ n / (2 x 10) for n ≥ 10 ( Dividing right side by 10, to maintain the relation )
= n / 20
n2 – 10n ≥ n2 / 20 for n ≥ 10 ( Multiplying both sides with n )
n2 – 10n ≥ c. n2 for n ≥ n0, where c=1 / 20 and n0=10 ( Solution )
Therefore, by definition ,
n2 -10n = Ω(n2).

Ω-Notation Example-1
The behavior of functions n2-10n and n2 / 20 is shown in the graph. For n > 10, the
function cg(n)= n2 / 20 (c=1/20) consistently fall below the function f(n) = n2-10n
f(n)= n2 - 10n
Asymptotic Lower bound

cg(n)=n2/20 n0
n
Growth of function n2 /20 versus the function f(n)= n2 - 10 n

Ω-Notation
Basic Method
Example(2): Next we show that 3n2 -25n = Ω (n2 ).
For, n ≥ n / 2 for n ≥ 0 ( Obvious !)
n- 25/3 ≥ 3 n / (2 x 25) for n ≥ 9 ( Dividing right side by 25/3 ≈ 8.3 to maintain

the inequality, since n >8.3 ie n≥9 )
= 3 n / 50 for n ≥ 9 (Simplifying )
3n2 – 25n ≥ 9n2 / 50 for n ≥ 9 ( Multiplying both sides with 3n to obtain

the desired function on the left hand side)
3n2 – 25 n≥ c. n2 for n ≥ n0, where c= 9 / 50 , n0=9 (Solution)
Therefore, by definition,
3n2 -25n = Ω(n2).

Ω-Notation Example-2
The behavior of functions 3n2-25n and 9n2 / 50 is shown in the graph. Observe that for
n≥9, the function 9n2 / 50 falls below the function 3n2-25n
f(n)=3n2 -25n
n0=9
Asymptotic Lower bound
cg(n)=9.n2/50
n
Growth of function 9 n2 /50 versus the function f(n)=3 n2 -25 n
Set Builder Ω-Notation
Definition
Consider the functions , say f1(n), f2(n), f3(n), ..fk(n) for which g(n) is the asymptotic
lower bound. By definition,
f1(n) ≥ c1 g(n) for n ≥ n1
f2(n) ≥ c2 g(n) for n ≥ n2
f3(n) ≥ c3 g(n) for n ≥ n3

……………………
fk(n) ≥ ck g(n) for ≥ nk
where c1, c2, c3 ,.. ck are constants and n1 ,n2 ,n3,.. nk are positive integers. The functions f1(n),
f2(n), f3(n), .. fk(n) are said to belong to the class Ω(g(n)). In set notation, the relation is
denoted by
Ω( g(n) )= { f1(n) , f2(n), f3(n)…,fk(n)}
Alternatively, using set-builder notation , if
Ω(g(n)) ={ f(n): there exist positive constants c and n0, such that f(n) ≥ c.g(n), for all n ≥ n0 }
then f(n) ∈ Ω(g(n))

θ-Notation
Definition
If f(n) is running time of an algorithm, and g(n) is some standard growth function
such that for some positive constants c1 , c2 and positive integer n0 ,
0 < c2.g(n) ≤ f(n) ≤ c1.g(n) for all n ≥ n0
then f(n) = θ(g(n)) (Read f(n) is theta of g(n) )
¾ The behavior of f(n) and g(n) is portrayed in the graph. It follows that for n < n0, f(n) may
be above or below g(n), but for all n ≥ n0, f(n) falls consistently between c1.g(n) and c2.g(n).
It also implies that g(n) grows as fast as f(n) The function g(n) is said to be the asymptotic
tight bound for f(n)
Asymptotic tight bound of f(n)

Set Builder θ-Notation
Definition
There can be a several functions for which the g(n) is asymptotic tight bound. All such
functions are said to belong to the class Ө(g(n))
Using set builder notation,, if
Ө(g(n)) = { f(n): there exit positive constants c1,c2 and positive integer n0 such that
c1 g(n) ≤ f(n) ≤ c2 g(n) }
then f(n) ∈ θ(g(n))

θ-Notation
Basic Method
Example: We show that 5n2 -19n = θ(n2).
First consider the upper bound . As explained before,
5n2 – 19n ≤ 5n2 for n ≥ 0
5n2 – 19 n ≤ c1. n2 for n ≥ n1, where c1=5 and n1=0 ………..(1)

Next, consider the lower bound,
n ≥ n / 2 for n ≥ 0 ( Obvious !)
n- 19/5 ≥ 5 n / (2 x 19) for n ≥ 4 ( Dividing right side by 19/5= 3.8)
= 5 n / 38 for n ≥ 4
5n2 – 19n ≥ 25n2 / 38 for n ≥ 4 ( Multiplying both sides with 5n )
5n2 – 19 n≥ c2. n2 for n ≥ n2, where c2=25 / 38 and n2=4 …….(2)
From (1) and (2) it follows ,

0 < c2.n2 ≤ 5n2 - 19n ≤ c1.n2 for n ≥ n0 , where n0=4, c1=5 and c2=25/38. (Solution)
Therefore, 5n2 -19n = θ(n2)

θ-Notation Example
The relation 5n2 -19n = θ(n2) is illustrated in the figure below. The graph shows the
asymptotic upper and lower bounds for the function f(n)= 5n2-19 n
Asymptotic Upper bound
cg(n)=5n2
f(n)=5n2 -19n
cg(n)=25n2/38
n0=4 AsymptoticLower bound
n
Growth of functions 5 n2 and 25n2/38 versus the function f(n)=5 n2 -19 n
Asymptotic Notation
Constant Running Time
If running time T(n)=c is a constant, i. e independent of input size, then by convention,
the asymptotic behavior is denoted by the notation
O(c) = O(1) , θ (c) = θ(1), Ω(c) = Ω(1)
The convention implies that the running time of an algorithm ,which does not depend
on the size of input, can be expressed in any of the above ways.
If c is constant then using basic definition it can be shown that

O( c.f(n) ) = O( f(n) )
θ( c.f(n) ) = θ( f(n) )
Ω( c.f(n) ) = Ω( f(n ))
Example: (i) O(1000n) = O(n),

(ii) θ(7lgn ) = θ (lg n),
(ii) Ω(100 n!) = Ω(n!)
¾ The above results imply that in asymptotic notation the multiplier constants in an
expression for the running time can be dropped

Asymptotic Notations
Relationships
Theorem: If f(n) = θ( g(n) ) then
f(n) = Ω( g(n) ), and f(n)= O( g(n) )
Conversely, if f(n) = Ω( g(n) ) and f(n) = O( g(n) )

then f(n) = θ( g(n) )
¾ The above relations can be established by using basic definitions
Example(1): Since, n(n-1)/2 = θ(n2 ), therefore it follows that

n(n-1)/2 = Ω(n2 )
n(n-1)/2 = O(n2 )
Example(2): It can be shown that

5n2+1 = Ω(n2 )
and 5n2+1 = O(n2 )
Therefore, 5n2 + 1 = θ(n2 )
Asymptotic Set Notations
Relationship
We have seen that if f(n) = O( g(n) ) and f(n) = Ω(g(n)) then f(n) = Ө( g(n) )
Using set notation we can express the relationship as follows:
if f(n) ∈O( g(n) ),

and f(n) ∈Ω (g(n)),
then f(n) ∈ Ө( g(n) ) ,
where Ө(g(n)) = O( g(n) ) ∩ Ω (g(n) )
¾ The above property is illustrated by the following Venn diagram
O(g(n)) Ө(g(n)) Ω(g(n))

Asymptotic Set Notation
Example
The relationship among the O ,Ω , and θ notations can be expressed using set notation
Consider, for example, the following sets of growth functions
O(n2) = {√n, n+5, lg n+4n, n1.5+n, √n+5n2, n2+5n, lg n+4n2, n1.5+3n2 }
Ω(n2) = { √n+5n2, n2+5n, lg n+4n2, n1.5+3n2 , 5n2+n3, n3 +n2+n, lg n+4n4, nlg n+3n4 }
θ(n2) = { √n+5n2 , n2+5n, lg n+4n2, n1.5+3n2 }
¾ It follows that θ(n2) = O(n2) ∩ Ω(n2)

Asymptotic Set Notation
Example
The following Venn diagram represents the preceding set relations
√n √n+5n2
√n+5n2 5n2+n3
n+5 n2+5n
n2+5n n3 +n2+n
lg n+4n lg n+4n2 2
lg n+4n lg n+4n4
n1.5+n n1.5+3n2
n1.5+3n2 nlg n+3n4
O(n2) Ω(n2)
√n √n+5n2 5n2+n3
n+5 n2+5n n3 +n2+n
lg n+4n lg n+4n2 lg n+4n4
n1.5+n n1.5+3n2 nlg n+3n4
θ(n2)

Asymptotic Notation
Order Theorem
Theorem: If f1(n) = O( g1(n) ) and f2(n) = O( g2(n) ) then
f1(n) + f2(n)= O( max( g1(n) , g2(n) )
Proof: By definition,
f1(n) ≤ c1. g1(n) for n ≥ n1
f2(n) ≤ c2. g2(n) for n ≥ n2
Let n0 = max( n1, n2) c3=max(c1, c2)
f1(n) ≤ c3. g1(n) for n ≥ n0
f2(n) ≤ c3 g2(n) for n ≥ n0
f1(n) + f2(n) ≤ c3.g1(n) + c3. g2(n) ) for n ≥ n0
Let h(n) = max( g1(n) , g2(n) )
f1(n) + f2(n) ≤ 2c3.h(n) = c. h(n) where c=2c3 for n ≥ n0
f1(n) + f2(n) ≤ c. h(n) = c. max( g1(n) , g2(n) ) for n ≥ n0
Therefore, f1(n) + f2(n) = O( max( g1(n) , g2(n) )
¾ The theorem also applies to θ and Ω notations

General Theorem
Theorem: If f1(n)= O(g1(n)) , f2(n) =O(g2(n)), f3(n)=O(g3(n))…., fk(n)=O(gk(n)) then
f1(n) + f2(n) + f3(n) ….+ fk(n) = O( max (g1 (n), g2 (n), g3 (n)…, gk(n) ) )
where max means fastest growing function
The theorem can be proved by using basic definition of Big-Oh
¾ It follows from the theorem that in an expression consisting of sum of several functions,
the comparatively slower growing functions can be discarded in favor of the fastest
growing function to obtain the Big oh notation for the whole expression This also true for
the Ө and Ω notations

Using Order Theorem
Example: Consider the summation f(n) consisting of basic functions :
f(n) = n +√n + n1.5+ lg n + n lg n n2
We have seen that
lg n < √n < n <n lg n <n1.5 < n2

which means that the function n2 grows faster than all other functions in the expression
Thus,
f(n) = n + √n + n1.5+ lg n + n lg n + n2
= O( max (n , √n , n1.5, lg n , n lg n , n2 ) )
= O(n2)

Asymptotic Analysis-1

Uploaded by

Copyright:

Available Formats

Asymptotic Analysis-1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Asymptotic Analysis-1

Uploaded by

Copyright:

Available Formats

Asymptotic Analysis-1

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/1

 Definitions ( Big-Oh, Big-Omega and Theta)

 Asymptotic Notation Theorems

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/2

 The asymptotic behavior of an algorithm is often compared to some standard

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/3

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/4

f(n) ≤ c.g(n) for all n ≥ n0

then f(n) = O(g(n)) (Read f(n) is Big-Oh of g(n) )

Trend of running time

Asymptotic Upper Bound

Asymptotic Upper Bound of f(n)

Consider, 10 ≤ n for n ≥ 10 ( obvious !)

10n ≤ n2 for n ≥ 10 ( Multiplying both sides with n )

3n2+10n ≤ 3n2 + n2 for n ≥ 10 ( Adding 3n2 to both sides )

3n2 + 10 n ≤ c.n2 for n ≥ n0 where c =4 and n0= 10 (Solution)

3n2 +10n = O(n2)

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/7

Consider n ≤ n2 for n ≥ 1 ( Obvious)

10n ≤ 10n2 for n≥ 1 (Multiplying both sides with 10 )

3n2 +10 n ≤ 3n2 + 10n2 for n≥ 1 ( Adding 3n2 on both sides)

3n2 +10n ≤ 13n2 for n ≥ 1 ( Simplifying )

Or, 3n2 +10n ≤ c.n2, for n ≥ n0 where c=13, n0=1 ( Solution )

Therefore, by basic definition,

3n2 + 10n = O(n2)

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/8

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/9

f3(n) ≤ c3 g(n) for n ≥ n3

 Alternatively, using set-builder notation, if

then, f(n) ∈ O(g(n))

c.g(n) ≤ f(n) for all n ≥ n0

then f(n) = Ω(g(n)) (Read f(n) is Big-Omega of g(n) )

Trend of running time

Asymptotic Lower Bound

Asymptotic lower bound of f(n)

For, n ≥ n/2 for n ≥ 0 ( Obvious !)

n - 10 ≥ n / (2 x 10) for n ≥ 10 ( Dividing right side by 10, to maintain the relation )

n2 – 10n ≥ n2 / 20 for n ≥ 10 ( Multiplying both sides with n )

n2 – 10n ≥ c. n2 for n ≥ n0, where c=1 / 20 and n0=10 ( Solution )

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/13

Asymptotic Lower bound

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/14

For, n ≥ n / 2 for n ≥ 0 ( Obvious !)

n- 25/3 ≥ 3 n / (2 x 25) for n ≥ 9 ( Dividing right side by 25/3 ≈ 8.3 to maintain

3n2 – 25n ≥ 9n2 / 50 for n ≥ 9 ( Multiplying both sides with 3n to obtain

3n2 – 25 n≥ c. n2 for n ≥ n0, where c= 9 / 50 , n0=9 (Solution)

3n2 -25n = Ω(n2).

Asymptotic Analysis-1/IIU 2008/Dr.A.Sattar/15

f3(n) ≥ c3 g(n) for n ≥ n3

 Alternatively, using set-builder notation , if

then f(n) ∈ Ω(g(n))

0 < c2.g(n) ≤ f(n) ≤ c1.g(n) for all n ≥ n0

then f(n) = θ(g(n)) (Read f(n) is theta of g(n) )

Asymptotic Upper Bound

Asymptotic Lower Bound

Asymptotic tight bound of f(n)

 Using set builder notation,, if

c1 g(n) ≤ f(n) ≤ c2 g(n) }

then f(n) ∈ θ(g(n))

Definitions ( Big-Oh, Big-Omega and Theta)

Asymptotic Notation Theorems

The asymptotic behavior of an algorithm is often compared to some standard

Alternatively, using set-builder notation, if

Alternatively, using set-builder notation , if

Using set builder notation,, if

If c is constant then using basic definition it can be shown that