TOPIC 1.1.

Asymptotic
Asymptotic Notation
Notation

By: Er.Bhavneet Kaur

Master Subject Co-ordinator
CSE 2nd Year
Asymptotic Complexity

Running time of an algorithm as a function of input size n for large n.

 Expressed using only the highest-order term in the expression for the exact running time.
Instead of exact running time, say Q(n2).
Describes behavior of function in the limit.
Written using Asymptotic Notation.
Asymptotic Notation
Q, O, W, o, w
Defined for functions over the natural numbers.
◦ Ex: f(n) = Q(n2).
◦ Describes how f(n) grows in comparison to n2.

Define a set of functions; in practice used to compare

two function sizes.
The notations describe different rate-of-growth relations
between the defining function and the defined set of
functions.
 -notation
For function g(n), we define (g(n)), big-
Theta of n, as the set:

(g(n)) = {f(n) :
 positive constants c1, c2, and n0,
such that n  n0,
we have 0  c1g(n)  f(n)  c2g(n)

}
Intuitively: Set of all functions that
have the same rate of growth as g(n).
g(n) is an asymptotically tight bound for f(n).
 -notation
For function g(n), we define (g(n)),
big-Theta of n, as the set:

(g(n)) = {f(n) :
 positive constants c1, c2, and n0,
such that n  n0,
we have 0  c1g(n)  f(n)  c2g(n)

}
Technically, f(n)  (g(n)).
Older usage, f(n) = (g(n)).
I’ll accept either…
f(n) and g(n) are nonnegative, for large n.
 Exercise
Express functions in A in asymptotic notation using functions in B.

A B
A  (B)
1. 5n2 + 100n 3n2 + 2
A  (n2), n2  (B)  A  (B)
2. log3(n2) log2(n3) A  (B)
logba = logca / logcb; A = 2lgn / lg3, B = 3lgn, A/B =2/(3lg3)
A  (B)
3. nlg4 3lg n
alog b = blog a; B =3lg n=nlg 3; A/B =nlg(4/3)   as n
4. lg2n n1/2 A  o (B)
lim ( lga n / nb ) = 0 (here a = 2 and b = 1/2)  A  o (B)
n
Example
(g(n)) = {f(n) :  positive constants c1, c2, and n0, such
that n  n0, 0  c1g(n)  f(n)  c2g(n)}
What constants for n0, c1, and c2 will work?
10n2 - 3n = Q(n2)
Make c1 a little smaller than the leading coefficient, and c2 a little bigger.
To compare orders of growth, look at the leading term.

Practice question: Prove that n2/2-3n= Q(n2)

Example
(g(n)) = {f(n) :  positive constants c1, c2, and n0, such
that n  n0, 0  c1g(n)  f(n)  c2g(n)}
Is 3n3  Q(n4) ??
How about 22n Q(2n)??

Note: The above questions can be solved by taking

c1 and c2 values.
O-notation
For function g(n), we define O(g(n)),
big-O of n, as the set:

O(g(n)) = {f(n) :
 positive constants c and n0, such
that n  n0,
we have 0  f(n)  cg(n) }
Intuitively: Set of all functions whose rate of
growth is the same as or lower than that of g(n).
g(n) is an asymptotic upper bound for f(n).
f(n) = (g(n))  f(n) = O(g(n)).
(g(n))  O(g(n)).
 Examples

O(g(n)) = {f(n) :  positive constants c and n0, such

that n  n0, we have 0  f(n)  cg(n) }
Any linear function an + b is in O(n2). How?
Show that 3n3=O(n4) for appropriate c and n0.
  -notation
For function g(n), we define (g(n)), big-Omega of n, as the set:

(g(n)) = {f(n) :
 positive constants c and
n0, such that n  n0,
we have 0  cg(n)  f(n)}

Intuitively: Set of all functions whose

rate of growth is the same as or higher
than that of g(n).

g(n) is an asymptotic lower bound for f(n).

 Example

(g(n)) = {f(n) :  positive constants c and n0, such that

n  n0, we have 0  cg(n)  f(n)}

n = (lg n). Choose c and n0.

Relations Between Q, O, W
 Relations Between Q, W, O
Theorem
Theorem:: For
Forany
anytwo
twofunctions
functions g(n)
g(n)and
andf(n),
f(n),
f(n)==(g(n))
f(n) (g(n))iff
iff
f(n)
f(n)==O(g(n))
O(g(n))and f(n)==(g(n)).
andf(n) (g(n)).
I.e., (g(n)) = O(g(n)) Ç W(g(n))

In practice, asymptotically tight bounds are obtained

from asymptotic upper and lower bounds.
Running Times

“Running time is O(f(n))” Þ Worst case is O(f(n))

O(f(n)) bound on the worst-case running time  O(f(n)) bound on the
running time of every input.
Q(f(n)) bound on the worst-case running time  Q(f(n)) bound on the
running time of every input.
“Running time is W(f(n))” Þ Best case is W(f(n))
Can still say “Worst-case running time is W(f(n))”
◦ Means worst-case running time is given by some unspecified function g(n) Î
W(f(n)).
Example

Insertion sort takes Q(n2) in the worst case, so sorting (as a

problem) is O(n2).

Any sort algorithm must look at each item, so sorting is W(n).

In fact, using (e.g.) merge sort, sorting is Q(n lg n) in the worst

case.
Asymptotic Notation in Equations
Can use asymptotic notation in equations to replace
expressions containing lower-order terms.
For example,
4n3 + 3n2 + 2n + 1 = 4n3 + 3n2 + (n)
= 4n3 + (n2) = (n3).

In equations, (f(n)) always stands for an anonymous

function g(n) Î (f(n))
◦ In the example above, (n2) stands for
3n2 + 2n + 1.
 o-notation
For a given function g(n), the set little-o:
o(g(n)) = {f(n):  c > 0,  n0 > 0 such that
 n  n0, we have 0  f(n) < cg(n)}.
f(n) becomes insignificant relative to g(n) as n approaches
infinity:
lim [f(n) / g(n)] = 0
n

g(n) is an upper bound for f(n) that is not asymptotically tight.

 w -notation
For a given function g(n), the set little-omega:
w(g(n)) = {f(n):  c > 0,  n > 0 such that
0
 n  n0, we have 0  cg(n) < f(n)}.
f(n) becomes arbitrarily large relative to g(n) as n
approaches infinity:
lim [f(n) / g(n)] = .
n

g(n) is a lower bound for f(n) that is not asymptotically

tight.
Relationships between these notations:
Comparison of Functions
fg  ab

f (n) = O(g(n))  a  b
f (n) = (g(n))  a  b
f (n) = (g(n))  a = b
f (n) = o(g(n))  a < b
f (n) = w (g(n))  a > b
 Limits
lim [f(n) / g(n)] = 0 Þ f(n) Î o(g(n))
n

lim [f(n) / g(n)] <  Þ f(n) Î O(g(n))

n

0 < lim [f(n) / g(n)] <  Þ f(n) Î Q(g(n))

n

0 < lim [f(n) / g(n)] Þ f(n) Î W(g(n))

n

lim [f(n) / g(n)] =  Þ f(n) Î w(g(n))

n

lim [f(n) / g(n)] undefined Þ can’t say

n
Properties
Transitivity
f(n) = (g(n)) & g(n) = (h(n))  f(n) = (h(n))
f(n) = O(g(n)) & g(n) = O(h(n))  f(n) = O(h(n))
f(n) = (g(n)) & g(n) = (h(n))  f(n) = (h(n))
f(n) = o (g(n)) & g(n) = o (h(n))  f(n) = o (h(n))
f(n) = w(g(n)) & g(n) = w(h(n))  f(n) = w(h(n))

Reflexivity
f(n) = (f(n))
f(n) = O(f(n))
f(n) = (f(n))
Properties
Symmetry
f(n) = (g(n)) iff g(n) = (f(n))

Complementarity
f(n) = O(g(n)) iff g(n) = (f(n))
f(n) = o(g(n)) iff g(n) = w((f(n))
Practice Question
Common Functions :
Important Theorems
Monotonicity
f(n) is
◦ monotonically increasing if m  n  f(m)  f(n).
◦ monotonically decreasing if m  n  f(m)  f(n).
◦ strictly increasing if m < n  f(m) < f(n).
◦ strictly decreasing if m > n  f(m) > f(n).

◦ Monotonically increasing means when a function is said to be monotonically increasing if its graph is
only increasing with increasing values of the equation.
◦ If f′(x)>0 for all x in the interval, then the function f is strictly increasing. If f′(x)<0 for all x in the interval,
then the function f is strictly decreasing.
Exponentials
Useful Identities:

1
a 1 
a
( a m ) n  a mn
a m a n  a m n
Exponentials and polynomials

nb
lim n  0
n  a

 n b  o( a n )
Logarithms a  b log b a
log c ( ab)  log c a  log c b
x = logba is the
exponent for a = bx. log b a n  n log b a
log c a
log b a 
Natural log: ln a = log a e log c b
Binary log: lg a = log a2 log b (1 / a )   log b a
1
lg a = (lg a)
2 2 log b a 
lg lg a = lg (lg a)
log a b
a log b c  c log b a
 Logarithms and exponentials – Bases
If the base of a logarithm is changed from one constant to another, the value is altered by a constant factor.
◦ Ex: log10 n * log210 = log2 n.
◦ Base of logarithm is not an issue in asymptotic notation.

Exponentials with different bases differ by a exponential factor (not a constant factor).
◦ Ex: 2n = (2/3)n*3n.
Polylogarithms

Polylogarithms: An algorithm is said to run in polylogarithmic time

if T(n) = O((log n)k), for some constant k. For example: Matrix
chain ordering.
For eg, [O((log(n))^c)]

For a ³ 0, b > 0, lim n ( lga n / nb ) = 0,

so lga n = o(nb), and nb = w(lga n )

lg(n!) = (n lg n)