Asymptotic Analysis

Asymptotic Analysis

 Asymptotic analysis of an algorithm refers to defining the

mathematical boundation of its run-time performance.

 Using asymptotic analysis, we can very well conclude the best

case, average case, and worst case scenario of an algorithm.

 Asymptotic analysis is input bound i.e.,

• If there is no input to the algorithm, it is concluded to

work in a constant time.
 Asymptotic Analysis
Contd..
 Other than the "input" all other factors are considered constant.

 Asymptotic analysis refers to computing the running time of

any operation in mathematical units of computation.

 For example, the running time of one operation is computed

as 𝑓(𝑛) and may be for another operation it is computed as 𝑔(𝑛2).
 Asymptotic Analysis
Contd..
 This means the first operation running time will increase
linearly with the increase in 𝒏

• and the running time of the second operation will

increase quadratically when 𝒏 increases.

 Similarly, the running time of both operations will be nearly the

same if 𝒏 is significantly small.
 Asymptotic Analysis
Contd..
 Usually, the time required by an algorithm falls under three types:

• Best Case: Minimum time required for program execution.

• Average Case: Average time required for program execution.

• Worst Case: Maximum time required for program execution.

 Asymptotic Analysis
Contd..
 The main idea of asymptotic analysis is to have a measure of the
efficiency of algorithms.

 Asymptotic notations are mathematical tools to represent the

time complexity of algorithms for asymptotic analysis.
 Asymptotic Notation

 There are mainly three asymptotic notations:

• Big-O Notation (𝑂-Notation)

• Big Omega Notation (Ω-Notation)

• Theta Notation (Θ-Notation)

 Asymptotic Notation Types (O-Notation)

Big-O Notation (O-notation):

 Big-O notation represents the upper bound of the running time

of an algorithm.

 Therefore, it gives the worst-case complexity of an algorithm.

 Asymptotic Notation Types (O-Notation)
Contd..
 If 𝑓(𝑛) describes the running time of an algorithm,

• 𝑓(𝑛) is 𝑂(𝑔(𝑛)) if there exist a positive constant 𝑐 and 𝑛0

such that,

𝟎 ≤ 𝒇 𝒏 ≤ 𝒄 ∗ 𝒈(𝒏) for all 𝒏 ≥ 𝒏𝟎

 It returns the highest possible complexity (Big-O) for a given

input.

 The execution time serves as an upper bound on the algorithm’s

time complexity.
Asymptotic Notation Types (O-Notation)
Contd..
 Asymptotic Notation Types (O-Notation)
Contd..
Mathematical Representation of Big-O Notation:

𝑂(𝑔(𝑛)) = { 𝑓(𝑛): there exist positive constants 𝑐 and 𝑛0 such that

𝟎 ≤ 𝒇 𝒏 ≤ 𝒄 ∗ 𝒈(𝒏) for all 𝒏 ≥ 𝒏𝟎 }
 Asymptotic Notation Types (O-Notation)
Contd..
 The Big-O notation is useful when we only have an upper
bound on the time complexity of an algorithm.

 Many times, we easily find an upper bound by simply looking at

the algorithm.
 Asymptotic Notation Types (O-Notation)
Contd..
Examples:

𝑈{100, log(2000), 104 } belongs to 𝑂(1).

𝑈{(𝑛/4), (2𝑛 + 3), (𝑛/100 + log(𝑛))} belongs to 𝑂(𝑛).

𝑈 {(𝑛2 + 𝑛), (2𝑛2 ), (𝑛2 + log(𝑛))} belongs to 𝑂 𝑛2 .

Note: Here, 𝑼 represents union, we can write it in these manner

because 𝑶 provides exact or upper bounds.
 Asymptotic Notation Types (Ω-Notation)

Big Omega Notation (Ω-Notation):

 Omega notation represents the lower bound of the running time

of an algorithm.

 Thus, it provides the best-case complexity of an algorithm.

 Asymptotic Notation Types (Ω-Notation)
Contd..
 The execution time serves as a lower bound on the algorithm’s
time complexity.

 It is defined as the condition that allows an algorithm to

complete statement execution in the shortest amount of time.
 Asymptotic Notation Types (Ω-Notation)
Contd..
 Let 𝑔 and 𝑓 be the function from the set of natural numbers to
itself.

 The function 𝑓 is said to be Ω(𝑔), if there is a constant 𝑐 > 0 and

a natural number 𝑛0 such that 𝒄 ∗ 𝒈(𝒏) ≤ 𝒇(𝒏) for all 𝒏 ≥ 𝒏𝟎 .
Asymptotic Notation Types (Ω-Notation)
Contd..
 Asymptotic Notation Types (Ω-Notation)
Contd..
Mathematical representation of Big Omega notation:

Ω(𝑔(𝑛)) = {𝑓(𝑛): there exist positive constants 𝑐 and 𝑛0 such that

𝟎 ≤ 𝒄 ∗ 𝒈(𝒏) ≤ 𝒇(𝒏) for all 𝑛 ≥ 𝑛0 }.
 Asymptotic Notation Types (Θ-Notation)

Theta Notation (Θ-Notation):

 Theta notation encloses the function from above and below.

 Since, it represents the upper and the lower bound of the

running time of an algorithm,

• It is used for analyzing the average-case complexity of an

algorithm.
 Asymptotic Notation Types (Θ-Notation)
Contd..
 Let 𝑔 and 𝑓 be the function from the set of natural numbers to
itself.

 The function 𝑓 is said to be Θ(𝑔),

• if there are constants 𝑐1 , 𝑐2 > 0 and

• a natural number 𝑛0 such that

𝒄𝟏 ∗ 𝒈 𝒏 ≤ 𝒇 𝒏 ≤ 𝒄𝟐 ∗ 𝒈(𝒏) for all 𝑛 ≥ 𝑛0
Asymptotic Notation Types (Θ-Notation)
Contd..
 Asymptotic Notation Types (Θ-Notation)
Contd..
Mathematical Representation of Theta notation:

Θ (𝑔(𝑛)) = {𝑓(𝑛): there exist positive constants 𝑐1 , 𝑐2 and 𝑛0 such

that 0 ≤ 𝑐1 ∗ 𝑔(𝑛) ≤ 𝑓(𝑛) ≤ 𝑐2 ∗ 𝑔(𝑛) for all 𝑛 ≥ 𝑛0 }
 Asymptotic Notation Types (Θ-Notation)
Contd..
 The above expression can be described as if 𝒇(𝒏) is theta of
𝒈(𝒏), then the value 𝑓(𝑛) is always between

𝑐1 ∗ 𝑔(𝑛) and 𝑐2 ∗ 𝑔(𝑛) for large values of 𝑛 ( 𝑖. 𝑒., 𝑛 ≥ 𝑛0 ).

 The definition of theta also requires that 𝒇(𝒏) must be non-

negative for values of 𝑛 greater than 𝑛0 .
 Asymptotic Notation Types (Θ-Notation)
Contd..
 The execution time serves as both a lower and upper bound on
the algorithm’s time complexity.

 It exist as both, most, and least boundaries for a given input

value.
 Asymptotic Notation Types (Θ-Notation)
Contd..
 A simple way to get the Theta notation of an expression is to
drop low-order terms and ignore leading constants.

 For example, Consider the expression

𝟑𝒏𝟑 + 𝟔𝒏𝟐 + 𝟔𝟎𝟎𝟎 = 𝚯(𝒏𝟑),

 Dropping lower order terms is always fine because

• there will always be a number 𝑛 after which Θ(𝑛3) has

higher values than Θ(𝑛2) irrespective of the constants
involved.
 Asymptotic Notation Properties

Properties of Asymptotic Notations:

General Property:

If 𝑓(𝑛) is 𝑂(𝑔(𝑛)) then 𝑎 ∗ 𝑓(𝑛) is also 𝑂(𝑔(𝑛)), where 𝒂 is a

constant.
 Asymptotic Notation Properties
Contd..
Example:

𝑓(𝑛) = 2𝑛² + 5 𝑖𝑠 𝑂(𝑛²)

then, 7 ∗ 𝑓(𝑛) = 7(2𝑛² + 5) = 14𝑛² + 35 is also 𝑂(𝑛²).

 Asymptotic Notation Properties
Contd..
 Similarly, this property satisfies both Θ and Ω notation.

 If 𝑓(𝑛) is Θ(𝑔(𝑛)) then 𝑎 ∗ 𝑓(𝑛) is also Θ(𝑔(𝑛)), where 𝑎 is a

constant.

 If 𝑓(𝑛) is Ω (𝑔(𝑛)) then 𝑎 ∗ 𝑓(𝑛) is also Ω (𝑔(𝑛)), where 𝑎 is a

constant.
 Asymptotic Notation Properties
(Transitive Property)

Transitive Properties:

 If 𝑓(𝑛) is 𝑂(𝑔(𝑛)) and 𝑔(𝑛) is 𝑂(ℎ(𝑛)) then 𝑓(𝑛) = 𝑂(ℎ(𝑛)).

Example:

 If 𝑓(𝑛) = 𝑛, 𝑔(𝑛) = 𝑛² and ℎ(𝑛) = 𝑛³

 If 𝑛 is 𝑂(𝑛²) and 𝑛² is 𝑂(𝑛³) then, 𝑛 is 𝑂(𝑛³).

 Asymptotic Notation Properties
(Transitive Property)
Contd..
 Similarly, this property satisfies both Θ and Ω notation.

 If 𝑓(𝑛) is Θ(𝑔(𝑛)) and 𝑔(𝑛) is Θ(ℎ(𝑛)) then 𝑓(𝑛) = Θ(ℎ(𝑛)).

 If 𝑓(𝑛) is Ω (𝑔(𝑛)) and 𝑔(𝑛) is Ω (ℎ(𝑛)) then 𝑓(𝑛) = Ω (ℎ(𝑛)).

 Asymptotic Notation Properties
(Reflexive Property)

Reflexive Properties:

 Reflexive properties are always easy to understand after

transitive.

 If 𝑓(𝑛) is given then 𝒇(𝒏) is 𝑶(𝒇(𝒏)).

 Since, maximum value of 𝑓(𝑛) will be 𝑓(𝑛) itself!

Hence 𝑥 = 𝑓(𝑛) and 𝑦 = 𝑂(𝑓(𝑛) tie themselves in reflexive relation

always.
 Asymptotic Notation Properties
(Reflexive Property)
Contd..
Example:

𝑓(𝑛) = 𝑛² ; 𝑂(𝑛²) i.e., 𝑓(𝑛) is 𝑂(𝑓(𝑛))

Similarly, this property satisfies both 𝚯 and 𝛀 notation.

 If 𝑓(𝑛) is given then 𝒇(𝒏) is 𝚯(𝒇(𝒏)).

 If 𝑓(𝑛) is given then 𝒇(𝒏) is 𝛀 (𝒇(𝒏)).

 Asymptotic Notation Properties
(Symmetric Property)

Symmetric Properties:
If 𝑓(𝑛) is Θ(𝑔(𝑛)) then 𝑔(𝑛) is Θ(𝑓(𝑛)).
Example:
If 𝑓(𝑛) = 𝑛² and 𝑔 𝑛 = 4 ∗ 𝑛²
then, 𝑓(𝑛) = Θ(𝑛²) and 𝑔(𝑛) = Θ(𝑛²)

Hence, 𝑓(𝑛) is Θ(𝑔(𝑛)) and 𝑔(𝑛) is Θ(𝑓(𝑛)).

This property only satisfies for Θ notation.

 Asymptotic Notation Properties
(Transpose Symmetric Property)

Transpose Symmetric Properties:

 If 𝒇(𝒏) is 𝑶(𝒈(𝒏)) then 𝒈(𝒏) is 𝛀(𝒇(𝒏)).

Example:

If 𝑓(𝑛) = 𝑛 , 𝑔(𝑛) = 𝑛²

then 𝑛 is 𝑂(𝑛²) and 𝑛² is Ω 𝑛 .

This property only satisfies 𝑶 and 𝛀 notations.

 Asymptotic Notation Properties
Contd..
Some More Properties:

 If 𝑓(𝑛) = 𝑂(𝑔(𝑛)) and 𝑓(𝑛) = Ω(𝑔(𝑛)) then 𝑓(𝑛) = Θ(𝑔(𝑛)).

 If 𝑓(𝑛) = 𝑂(𝑔(𝑛)) and 𝑑(𝑛) = 𝑂(𝑒(𝑛)) then

𝒇(𝒏) + 𝒅(𝒏) = 𝑶( 𝒎𝒂𝒙( 𝒈(𝒏), 𝒆(𝒏) ))

 Asymptotic Notation Properties
Contd..
Example:

𝑓(𝑛) = 𝑛 i.e., 𝑂(𝑛)

𝑑(𝑛) = 𝑛² i.e., 𝑂(𝑛²)

then 𝒇(𝒏) + 𝒅(𝒏) = 𝑶(𝒎𝒂𝒙( 𝒇(𝒏), 𝒅(𝒏))) i.e., 𝑂(𝑛²)

 Asymptotic Notation Properties
Contd..
If 𝑓(𝑛) = 𝑂(𝑔(𝑛)) and 𝑑(𝑛) = 𝑂(𝑒(𝑛)) then
𝒇(𝒏) ∗ 𝒅(𝒏) = 𝑶(𝒈(𝒏) ∗ 𝒆(𝒏))

Example:

𝑓(𝑛) = 𝑛 i.e., 𝑂(𝑛)

𝑑(𝑛) = 𝑛² i.e., 𝑂(𝑛²)

then 𝑓(𝑛) ∗ 𝑑(𝑛) = 𝑛 ∗ 𝑛² = 𝑛³ i.e., 𝑂(𝑛³)

 Asymptotic Notation Properties
Contd..

 There are two more notations called Little 𝒐 and Little Omega
(ω).

 Little o provides a strict upper bound (equality condition is

removed from 𝐵𝑖𝑔 𝑂).

 Little Omega (ω) provides strict lower bound (equality

condition is removed from 𝐵𝑖𝑔 Ω).
 Asymptotic Notation Properties
Contd..
 Note: If 𝑓(𝑛) = 𝑂(𝑔(𝑛)) then 𝑔 𝑛 = Ω 𝑓 𝑛 .
 Asymptotic Notation (Little o-Notation)

Little ο Asymptotic Notation:

 Big-Ο is used as a tight upper bound on the growth of an

algorithm’s effort

• this effort is described by the function 𝑓(𝑛)) , even

though, as written, it can also be a loose upper bound.
 Asymptotic Notation (Little o-Notation)
Contd..
 “Little-ο” (𝜊()) notation is used to describe an upper bound
that cannot be tight.

 Definition: Let 𝑓(𝑛) and 𝑔(𝑛) be functions that map positive

integers to positive real numbers.

 We say that 𝑓(𝑛) is 𝜊(𝑔(𝑛)) (or 𝑓(𝑛) ∈ 𝜊(𝑔(𝑛))) ,

 if for any real constant 𝑐 > 0, there exists

an integer constant 𝑛0 ≥ 1 such that 𝟎 ≤ 𝒇(𝒏) < 𝒄 ∗ 𝒈(𝒏).

 Asymptotic Notation (Little o-Notation)
Contd..
 Thus, Little 𝒐 means loose upper-bound of 𝑓(𝑛).

 Little o is a rough estimate of the maximum order of growth

whereas Big-Ο may be the actual order of growth.
 Asymptotic Notation (Little Omega (ω)-Notation)

Little omega (ω) asymptotic notation:

 Definition: Let 𝑓(𝑛) and 𝑔(𝑛) be functions that map positive

integers to positive real numbers.

 We say that 𝑓(𝑛) is 𝜔(𝑔(𝑛)) (or 𝑓(𝑛) ∈ 𝜔(𝑔(𝑛))) if for any real
constant 𝑐 > 0,

there exists an integer constant 𝑛0 ≥ 1

such that 𝒇(𝒏) > 𝒄 ∗ 𝒈(𝒏) ≥ 𝟎 for every integer 𝑛 ≥ 𝑛0 .

Asymptotic Notation (Little Omega (ω)-Notation)
Contd..

 𝒇(𝒏) has a higher growth rate than 𝒈(𝒏)

• so main difference between Big Omega (Ω) and Little omega

(ω) lies in their definitions.

 In the case of Big Omega, 𝑓(𝑛) = Ω(𝑔(𝑛)) and

the bound is 𝟎 <= 𝒄 ∗ 𝒈(𝒏) <= 𝒇(𝒏),

 But in case of Little omega (𝝎), it is true for

𝟎 <= 𝒄 ∗ 𝒈(𝒏) < 𝒇(𝒏).
Asymptotic Notation (Little Omega (ω)-Notation)
Contd..

 The relationship between Big Omega (Ω) and Little Omega (ω)
is similar to that of Big-Ο and Little o

• except that in case of Big Omega (Ω) and Little Omega (ω)
we look at the lower bounds.
Asymptotic Notation (Little Omega (ω)-Notation)
Contd..

 Little Omega (ω) is rough estimate of minimum order of the

growth whereas

• Big Omega (Ω) may represent exact order of growth.

 We use ω notation to denote a lower bound that is not

asymptotically tight.

• And, 𝑓(𝑛) ∈ 𝜔(𝑔(𝑛)) if and only if 𝑔(𝑛) ∈ 𝜊((𝑓(𝑛)).

 Asymptotic Notations
Common Asymptotic Notations:
 Time Complexity

Example:

func( )
{
int i, n;
for (i=1; i<=n; i++)
printf(“ABC”);
}

 Since 𝑖 equals 1 to 𝑛, so the above program will print 𝑨𝑩𝑪, 𝒏

number of times. Thus, the complexity will be 𝑶(𝒏).
 Time Complexity
Contd..
Example:

func( )
{
int i, j, n;
for (i=1; i<=n; i++)
{
for (j=1; j<=n; j++)
printf(“ABC");
}
}
 Time Complexity
Contd..
 In this case, firstly, the outer loop will run 𝒏 times, such that for
each time, the inner loop will also run 𝒏 times.

 Thus, the time complexity will be 𝑶(𝒏𝟐).