Recap

Problems that we faced

Solution
Asymptotic Bounds

Asymptotic notations

In the previous lecture we tried to look into the basics of algorithms

and what all features one desires in a good algorithm.

We also tried to gain insight into the high school algorithms for
I factorization of an integer.
I finding the GCD of two integers.
 Recap
Problems that we faced
Solution
Asymptotic Bounds

Even though we found (or at least we think that we found) a good

solution for finding the GCD of two integers, we can not be sure
that we have found the best solution.

For example, 50 years back everybody thought that the fastest way
to reach from Mumbai to Madras was to catch a train. But now we
have airplanes.

Similarly it is quite easy to see if the strategy/procedure that you

may devise is faster than the previously existing strategies or not.
But how can one be sure that the algorithm one comes up
with is the best possible algorithm?
 Recap
Problems that we faced
Solution
Asymptotic Bounds

Each algorithm for a computational task may take a different

number of clock cycles based on factors like the technique used, the
size of the input, etc. So how does one compare the number of
clock cycles taken by two algorithms that solve the same problem?

One possible way to check their performance would be to provide

the same inputs to both the algorithms and measure the time taken
to solve the problems.
 Recap
Problems that we faced
Solution
Asymptotic Bounds

If one algorithm (say A) is consistently faster than the other than

the other (say B) then we might claim that the algorithm A is faster
than the algorithm B.

But in order to make such a valid claim, we would have to check

the entire universe of inputs on both the algorithms A and B, which
may not be feasible/inconclusive. Let’s take an example.
 Recap
Problems that we faced
Solution
Asymptotic Bounds

Suppose there are two algorithms A and B which solve the same
problem. For an instance of size n
I A takes log2 n number of clock cycles.
I B takes 2100 number of clock cycles.

100
∀ inputs of size < 22 , we will observe that the algorithm A takes
less time compared to algorithm B and hence we may prematurely
conclude that A is the better algorithm, but as soon as the inputs
100
are of size greater than 22 , we may realize that sometimes B can
be better than A.
 Recap
Problems that we faced
Solution
Asymptotic Bounds

This gives us a clue as to what the problem is.

Right now we do not know how to compare two functions (and by
extension two algorithms) or to be specific we do not know how to
compare the rate of growth of two functions.

In this lecture we will take one step forward in the direction of

understanding efficient solutions by trying to understand how
functions grow.

In order to do that we have to look at a few mathematical concepts.

 Recap
Problems that we faced
Solution
Asymptotic Bounds

An asymptote provides a behavior in respect of other function for

varying value of input size.
An asymptote is a line or curve that a graph approaches but does
not intersect. An asymptote of a curve is a line in such a way that
distance between curve and line approaches zero towards large
values or infinity.

Based on asymptotes we define asymptotic bounds. First we shall

look at asymptotic upper bounds.
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

Asymptotic upper bounds:- Let f be a real/complex valued

function and g a real valued function. Let both functions be defined
on some unbounded subset of the positive real numbers, and g (x)
be strictly positive for all large enough values of x.

f (x) = O(g (x)) (pronounced ”big-oh of g of x” or sometimes just

”oh of g of x”) if there exists a positive real number M and a real
number x0 such that ∀x > x0 ,

f (x)
≤M
g (x)
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

Examples of Asymptotic upper bound

For sake of simplicity we assume that f takes non-negative integer

values

Let f (x) = 3x 2 − 15x + 10

Let g1 (x) = x 2

f (x) = 3x 2 − 15x + 1
≤ 3x 2 + 15x + 1
≤ 3x 2 + 15x 2 + x 2 = 19x 2

f (x)
Thus f (x) = O(g1 (x)), because g1 (x) ≤ M = 19
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

Examples of Asymptotic upper bound

Let g2 (x) = x 3

f (x) = 3x 2 − 15x + 1
≤ 3x 2 + 15x + 1
≤ 3x 3 + 15x 3 + x 3 = 19x 3

f (x)
Thus f (x) = O(g2 (x)), because g2 (x) ≤ 19
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

Examples of Asymptotic upper bound

Sometimes the deduction is not straightforward and one may have

to use L’ Hopital’s rule to find the limit
For example, lets take g3 = log2 x

f (x) f 0 (x) 6x + 15
lim = 0 = = lim 6x 2 + 15x = undefined
x→∞ g3 (x) g (x) 1/x x→∞

There exists no real value M such that M > 6x 2 + 15x for all values
of x. Thus f (x) 6= O(g3 (x))
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

Valuable insights

This process of finding a function g for a function f such that

f = O(g ), simplifies the analysis of an function’s growth process for
two reasons
1. The function f may have several terms, but the function g
usually has just one term.
2. Comparison of the growth rate of two functions f1 and f2 can
be done very easily.
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

Valuable insights

This also gives a hint that while analysing algorithms we should not
focus on smaller instances and focus on how an algorithm behaves
when the input extremely large.

Based on these insights we can say that the algorithm B is

better/faster than algorithm A (slide number 4.)
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

Omega Notation

Analogously we can define the Omega notation.

Asymptotic lower bounds:- Let f be a real/complex valued

function and g a real valued function. Let both functions be defined
on some unbounded subset of the positive real numbers, and g (x)
be strictly positive for all large enough values of x.

f (x) = Ω(g (x)) (pronounced ”Omega of g of x”) if there exists a

positive real number M and a real number x0 such that ∀x > x0 ,,

g (x)
≤M
f (x)
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

Theta Notation

For any two functions f and g , we have f (x) = Θ(g (x)) if and only
if
f (x) = O(g (x)) and
f (x) = Ω(g (x))

For example, if f (x) = 5x 2 + 3x + 2 and

g (x) = 29x 2 − 3000x + log x, we can say that f (x) = Θ(g (x))

In layman terms, this would mean that both the functions f and g
grow in a similar fashion.
 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

little o and little omega

The asymptotic upper bound provided by O-notation may or may

not be asymptotically tight. The bound 2n2 = O(n2 ) is
asymptotically tight, but the bound 2n = O(n2 ) is not.

We use the little o notation to denote an upper bound that is not

asymptotically tight. f (x) = o(g (x)) if

f (x)
lim =0
x→∞ g (x)

for example, 2n = o(n2 )

 Recap
Asymptotic upper bound
Problems that we faced
Asymptotic lower bound
Solution
Other useful notations
Asymptotic Bounds

little o and little omega

Analogously, the asymptotic lower bound provided by Ω-notation

may or may not be asymptotically tight. The bound 2n2 = Ω(n2 ) is
asymptotically tight, but the bound 2n2 = Ω(n) is not.

We use the little ω notation to denote an lower bound that is not

asymptotically tight. f (x) = ω(g (x)) if

g (x)
lim =0
x→∞ f (x)

for example, 5n3 = ω(n2 )