impomontne the alrvthma

We wn the adahite mnng tinne pvgruma

CHAPTER 2
Growth of Functions
NNNRmmd amadd owlnd the KaINdwn Aawe Mavhnne iRAM
a t The RAM Mocl of Gomputatkon
A undn dvwas MadeadbrvNnd RAM)iN
an wsimplitkvd vwrsm a mnputon
NROeeTON
wwists the ddAwime elbmoanta
Amattaqyattape
Akey
22NS AOR ASRINS RUNNNG ME
M
Am amg akaatdm we a e "RAM MML* wdd the dddwing
The stntims ane eutad aune ater amahr, wtd wRrwt Yerataa
e u y wnds have tdnd vmstani sim
 Direct/indirect memory access.
accesses
take the same amount of time 21
All memory Growth of Functions
(chars, ints, doubles, pointers, etc.) tak
unit
All "atomic"'data
Basic perations like memory
(conditional and

access,
unconditional

arithmet
(+,-*,1,%, spaceceilin
floor,
branch, subroutin call
.The order of growth can be estimated by taking the
the algorithm. Here we only specify how the running time
term
dominant
increases as the input increases
of runningtime o
rather than specifying the exact relation between an algorithm's input and its running
control
such instruction takes a constant amount of time (t.e., cost 1) called + and
measuring
Tetum, ta time. For example, if the running time for an algorithm is 9n', with input size n,
we say that it's running time scales as n2 times when we increase the input size n.
then
The unit cost measure
is independent of processor speed EXAMPLE 2.1. Let us consider the expression ofrunning time ofan algorithm is given b
"important" operations called the (n 20). Ifthe size
.We can focus on some has:.
(n) = an +b where a andb a r e constant and n is the input
some positive
algorithms, which dominate
the running time. operat ft input size, n is multiplied by factor k then the dominant term in T (n), an, is also multiplied
by k as:
For Example,
Tkn) = k (an) +b
In sorting: only count comparisons and data movements.
Hence we can say that the running time Tn) has a linear order of growth.
In matrix multiplication: only count the multiplication and addition EXAMPLE 2.2. Let us consider the running time expression ofan algorithm is given
In polynomial evaluation: only count the multiplication and additin
dition. by T (n) ana +bn + c where a, b andc a r e some positiveconstant and n is
=
the input
in T (n),
In Chess player ranking: only count matches. size (n 20). After multiplying the input size, n by k then the dominant term
an", is multiplied by k> as:
In recursive programs: only count recursive calls.
T (kn) = a (kn)2 + b (kn) + c k(an?) + k (bn)+ c =
2.2.2. How to Measure Running Time of an quadratic order
Algorithm? Hence we can say that the running time T(n) has a
ofgrowth.
Letc, is the execution time of the basic operation of an algorithm EXAMPLE 2.3. Let us consider the running time of an algorithm with logarithmie
of times this operation is to be executed. Then and f(n) be the
the running time T number
(n) of the algorithms running time expression T (n) algn + b where a and b a r e the positive
constant
be caleulated by the formula. =
and n is the input size (n 20). If the input size,n is multiplied by factor
k then the
dominant term in T (n), algn, does not modify, but the running time is modified by
T(n)= e *fn) adding a constant as:
Here the count ffn) is T (n) = algn+b
only, and does not containcomputed approximately because it relates to the basic
approximation. The above
any information about the
non-basic operations. The coperations
is alsoan T (kn) =
algn + algk +b;
formula
algorithm ifn is extremely large or gives a reasonable result for the running time ofa Hence the running time of algorithm has a logarithmic order of growth.
very small.
With this formula we
can also EXAMPLE 2.4. Let u s take a running time expression of an algorithm given by
the
algorithm run if we triple itsevaluate for the algorithm as "how much
longer Tn) =a3" +b where a and b a r e the positive constants and n is the input size (n 2
input size?", for example. ). Ifthe input size, n is multiplied by factor k then the T (n) c a n be epressed a s :
fn)nn-1)=n?-nn3
3

T (n) = a (3")* + b
Hence the running time has an exponential order of growth.
Now, T
(8n)
f3n)=3n)2
T n)
The growth of the running time of algorithms
given in the below table.
as the input grows of known functions is
c4 (Bn)2/
-
algorithm runs nine times c, 4
Hence the n2 =
9
functions (1) Unit cost log log
2.3. ORDERS longer if we triple its size.
OF input size n
GRoWTH
wom 5 1 3 5 15 25 32
To
analyze the
time increases efficiency of an 10 1 4 10 33 100 103
time of when the
input
algorithm we are interested
size inCreas erested in analyzing how the run 100 7 100 664 104 1030
their algorithm is not
Pat S1ze increases, but the anayis of the r
behaviour for the necessary. When two detailed anayrith
1000 10 1000 104 106 10300
large input algorithms compared with respect
resp 13 10000 105 108 103000
neasureare
sizes, 10000
a
useful measure is
is called order ofgrowu
cau
 h Growth of Functions 23
for searching
0 In this case the algorithm makes the largest number of key comparisons
an element in the array.
0f Best case running time:
T h e best case running time is the minimum amount of time that an algorithm
requires
taken on
for an input of size n. it is the function defined by minimum number of steps
40
any instance of size n.
an
The term best-case performance is used in computer science to describe the way
30 conditions. For example, the best case for a simple
algorithm behaves under optimal
linear search on a list when the desired element is the first element of the list.
occurs
The best-case efficiency of an algorithm is its efficiency for the best-case input of size n,
all
which is an input (or inputs) of size n for which the algorithm runs the fastest among
the inputs of that size.
.The analysis of the best case efficiency is not nearly as important as that of the worst
case efficiency. But it is not completely useless.
Average Case running time:
The average case time complexity is the function defined by the average number of
steps taken on any instance of size.
about
Fig. 2.2. .Neither the worst-case nor the best-case can answer the question the running
time of a typical input or a random input. This is given by the average case effñiciency of
24. WORST-CASE, BEST-CASE AND AVERAGE-CASE an algorithm.
In
RUNNING TIME Average-case running times are calculated by first arriving at an understanding of the
computer science, best, worst and
the resource average
usage is at least, at most and on
cases of a given
algorithm express t average nature of the input, and then performing a running-time analysis of the
average, algorithm for this configuration.
being considered is running time, respectively. Usually the resoure
Worst-case execution time is often memory
or other
resources. In real-time
of particular concern computing, the
2.5. ASYMPTOTIC NOTATION
much time might be needed in the worst since it is important to
finish on time. case to
guarantee that the know b In order to choose the best algorithm for a particular task, you need to be able to judge how
algorithm
will aiag
long a particular solution will take to run. Or, more accurately, you need to be able to judge
We have now
identified that it is a how long two solutions will take to run, and choose the better of the two. You don't need to
as a
function
of the good idea to
express the performance of
input size an augorn know how many minutes and seconds they will take, but you do need some way to compare
However, in some
a particular input. cases, the
efficiency of the algorithm also depends on the ciñisd
algorithms against one another.
Asymptotic means a line tends to converge to a curve, which may or may not eventually
Worst
case
sper touch the curve. It is a line that says within bounds.
running time:
The goodness of an algorithm is Time and space complexity is measured in terms of asymptotic notations. Asymptotie
time. most often notation can describe the time and space complexity accurately for large instance characteristics.
The worst-case expressed in terms or 1ts For a given algorithm, asymptotic notation will provide upper and /or lower timeand
space
n, efficiency of an bounds. Upper bound gives the maximum time / space required and lower bound gives the
all which algorithm
is
input (or inputs)
an is its
efficien
worst-case input
ofsie
minimum time/space required.
possible inputs of that
of size for
n
which the the longest amo
The
running time
size. algorithm ru A tool for analyzing time and space usage of algorithms.
algorithm becauseTit(n)bindsin the worst
case "mation aboutthe
Assumes input size is a variable, say n, and gives time and space bounds as a function
For a
simple
its
linear search, in runningt
provides ery important in
ing time from the
ver,
bound).
of n.
matching element above (an per Provides simple characterization ofan algorithm efficiency.
is in the the worst case there are no matching elements the
last are mate
or
position in the list. no
 DeSi9
24 additive
constants.
Growth of Functions
25
and
multiplicative
Ignores expressions of running time
me.
rates of two
Compare the growth e(n*).
the rate ofgrowth. EXAMPLE 2.5. Let us prove thatn-3n
=
Concerned only with
definition of
Notations Here we assume that g(n) =
n and c, C2, ng are positive constant then by
Asymptotic
2.5.1. Types of that we have to use in daed theta notation
notations
the asymptotic gn and analysis q
The following are
algorithm: cns 3n Sen
1. Big-Theta Notation
2. Big-Oh Notation
n
3. Big-Omega Notation
4. Smal-Oh Notation
.. right hand inequality
5. Small-Omega Notation
1. Theta notation:
c .n . . left-hand inequality
Definition: Let fn) and gin) be the functions that map positive integers to posit The right hand inequality will be hold for any value of
numbers. We denote fln) = ¬gln), (or fAn) e 0Mg (n), as the set: ositivered
n21 by choosing ca 2
egn)) = Mn):
The left-hand inequality can be hold for any value of
3 positive constants c, c, and ng, 8uch that vn 2
no
we have 0 S e,en) s fln) s ca gln)
n2 7 by
choosing C14
Therefore, we can say that by choosing c = C,= and n, = 7, we can verify that
Graphical representation of theta notation n-3n = 0(n*).
Figure 2.3 shows the graphical representation of theta notation.
EXAMPLE 2.6. Find notation for 1On + 5n +3.
Upper bound positive constant then by definition of 6-notation.
Solution: Let gln) =
n and c, cg, ng are
f(n) cn s10 n+5n +3 Sez n2
Function
(n) e s10+ s
Lower bound Now,
qs10 ... Left hand inequality
f(n)= (g (n)) and
Here the Fig. 2.3. o 2 10+ Right hand inequality
function g(n) is an
asymptotically tight bound for the
The 8-notation
A asymptotically bounds a function
funcuo"f the left hand inequality will be hold for any value n 21 by choosing c, = 10, i.e.,
it
functionf(n) belongs to the set 0 (gin),
from above and below.
such that 10s 10++will behold for n 2 1.
can be "Sandwiched" if there exitpositive
The fact that between c,g (n) and constan
c»g (n), for sufiñcienuy large n the right hand inequality will be hold for any value n 26 by choosing C, = 11 i.e.,
fn) =
g (n),
G.e., they are means
that fn) and o f growth
asymptotically equivalent). gn) have the sa prder
 26 27
Growth of Functions
for n 26
will be hold
11210++ n
1l and n 2 n,
= 6 cg (n)
= 10, c2 =
therefore, by choosingc, Upper bound
fAn) = Olg(n))
have
we
f
10n+ 6n +3 =e(n2) Function
a Note: Lemma: 2.1
exists, then function.
Let ffn) andg(n) be the
function such that
g(n) ction fin)
e
lgn) O
f(n) O (g (n)
im constant such that 0 <<c < oo,
I n 5 gtn) =Cfor some Fig. 2.4.
H e r e the function gin) is an asymptotic upper bound
for the function fln).
EXAMPLE 2.7. Is 3+3= (3")? use O-notation. Big oh notation
When we have only an asymptotic upper bound, we
Solution: Given that fin) = 3n *8 and gln) = 3"
We write f (n) O (g (n)) to indicate that a
provides an upper bound for the function.
=
function f (n) is a member of the set O (g (n).
then by applying Lemma 2.1 as
Upper bound or maximum level of order is termed as efficiency.
worst-case
or lower than that
m I t represent the set of all functions whose rate of growth is the same
n gln)Cror some constant
ofg n).
such that 0 <c< oo notation is a stronger notation
.Note that f(n)
than O-notation.
(g (n)
=
impliesf(n) =O (g (n), since
37+3 3".33 For the given-functiongin) we have egtn)) c Ogn).
3
lim (3) = 27 (constant)
EXAMPLE 2.8. Let us consider the function fn) 3n +2,
Solution: By using the definition of O-notation we have
=
find the O-notation.
n
.e.
An) e(gtn))
0s fn) Segn)
3+3 =
(3n) is true.
0s 3n+2 s c.n
0s 3n +2s 4n for all n 22
2. Big Oh notation c= 4, n =2 we can write fn) = O(n).
Definition: Let f (n) and
realnumbers. We denote g (n) be the functions that map positive integers to po
as the set: f(n) 0 (g (n)) =
(or f (n) e O
EXAMPLE 2.9. Find O-notation for the funetion fin) = 10000 n* + 1000 n - 12.
(g (n))), (pronounced g ofn'",
"bIg-0 Solution: Let g(n)
have
= n2 and c, n, are positive constant then by definition of O-notation we
Ogn)) = fn)
3 0s10000 n2 + 1000n-12 s c, n2
positive constants and c = 10000 n2 + 1000 n - 12 Sc n2
we have 0 ng, Such that Vn2
sfn) s ogtn)) no»
Graphical 10000-+00 Sc
Representation
Figure 2.4, shows the
of
Big-oh notation
n n
Above inequality will be hold for any values 2 1000 by choosing c,
graphical representation of n =
10001, ie.,
big-oh notation 10000+ 1000 1210001 will be hold
 Desigr an0 AIdy
28 29
n2 100 Growth of Functions
for 2 no 1000, we have
10001 and n
bychoosingC,
=
therefore
An) = Ogln)
-
12 = Otn?). Function n)
1000 n
10000n2+
Cgn
Lemma 2.2
a Note. Lower bound
function such that
be the
Let fn) and g(n)
(n)) if
exists, then function fn) Olg
e
lim
n gn)
o fn)=2 (9 (n))
lim = c, such that o Sc < * ie., including the case in which the limit ie o
Fig. 2.5.
.Here the functiong (n) is an asymptotic lower
bound for the function f (n).
EXAMPLE 2.10. Prove that 21000 is O(1).
-Notation provides asymptotic lower bound.
an
Solution: Given that fn) =21000 and gln) = 1 then by applying Lemma 2.2 as L o w e r bound or minimum level of order is termed as best case efficiency.
I t includes Set of all functions whose rate of growth is the same as or higher than that
lim =c such that 0 sc<oo ofg n).
n gin)
.For two functions f(n) and g (n) we have e(g (n))
f (n) = = f(n) =
(g (n)).
ie., including the case in which the limit is 0.
F o r given faunctiong (n) we have eg (n)) eQ g(n))
1000
lim
2000 (constant) EXAMPLE 2.11. Let us consider the funetion fln) =3n + 2 by using the definition of
n 1
2notation we have
An) Ogn)) = O(1).
0 Seg(n) sffn)
3. Omega notation 0 Se.n s 3n + 2
1. Hence, write fln) 2 (n) for
Definition: Let f (n) and g 3 and n, 2 =
we c a n
(n) be the functions that map positive integers to this inequality is hold for e =
(fn) e 2 (g(n))), (pronounced "big-omegapositive
real numbers. We denote e= 3 and n, 21.
fln) = 2 (g(n)) or
n"), as the set: ofg ot
EXAMPLE 2.12. Find the anotation for the function 8n+ 7n +6
gln)) ={n): n" and e, n, are positive constant then by definition of Q-notation,
Solution: Let gin) =
we
3 positive have
constants c and no, such that vn2
no» 0s en3 s Sn3 + 7n +6
we have 0 egn) s fin))
n s 8n3+ 7n +6
Graphical representation of
Figure 2.5 shows the Omega notation
graphical representation of
omega notation. above inequality will be hold for any value n 21 by choosing e, = 8 i.e.,
8s 8 t t w i l l be hold for n 21
therefore, by choosing ej = 8 and n 2 ng = 1, we have fAn) = 2(g(n), i.e.,
8n3+7n +6 n*)
 30
a Note.
Lemma 2.3
be the
functions such that Growth of Functions 31
Let fn) and gin)
egn)): functions that grow at the same rate as gln)
f(n) fn) e 2gln) if
im = exist then the function Og(n)): functions that grow no faster than gin).
n g(n)
Theorem 2.1
the case in which
lim =c, such that
0 <eSo i.e., including the limit is
tho
0 gln) if and only if fn) O g(n) and
For any two
fAn) = 2g(n).
functionsfn) and g(n), we have fn) = =
12*3" +n* 2M3")
EXAMPLE 2.13.Prove that
=
Proof: By the definitioins of O-notation and 2-notation, we have.
Solution: Given that fln) = 12*3 +n and gin) = 3 then by applyine T.on () fn) = O(gn)) ifthe following inequality is hold 0 sfin) Se, gn) for all n > no-
eema 2.3 as (ii) fln) =
Qgn)) ifthe followimg inequality is hold 0 s c2 gln) s An) for all n >0 from
lim =c such that 0 <cSo, i.e., including the case in which the limit equations (i) and (ii), w¿have:
mit is.
12 *3 +n
0s ch gn) s fn) s e 8n)for alln2 max
(ng n)
lim Now by the definition of 6-notation, we have
n 3
(ii) fn) = 0gn)) if the following inequality is hold.
n
lim
3"
0s
c2 gn) sfn) s cgln) for all n 2 ng
Now seperating the left and right inequality, we have:
0s c2 gn) s fn) for all n 2 n ie., An) = O (g(n)
lim 12 12(constant) 0s fin) s c,gn) for all n 2 ng i.e., An) = 2 (g(n))
Both sides of the implication are true so we can say that
fAn) Q (gtn)) =9 (3"). An) = O g(n)) and fAn) = Q (gln) A n ) e (g(n)
POINTSTO REMEMBER 4.0-notation
Always use O for upper bounds and 2 for lower The asymptotic upper bound (O-notation) may or not be asymptotically tight. For example,
Worst case bounds, i.e.,
running time of an algorithm andQ to bound the bound we use O to te
algorithm. best-case running thebound 2 nd =
On?) is asymptotically tighi but the bound 2n Otn) is not. =
time ota O-notation use to denote upper bound that is not
asymptotically tight.
Relations between 0, Definition: For the given functions fin) and gn) we denote o (g(n)) ("little -oh ofg ofn")
as the set olg(n)) = lfn): for any positive constant e > 03 a constant ng>0 such that
Let us consdier two functions fln) and
and 2 notation
in below Fig. 2.6. g{n)
then the osfn)<egln)for alln 2 n
is given
representation of functions In o-notation, the
Goln)
function fln) becomes insignificant relative to gln) asn o , ie.,
cgn)
lim )
n) n g{n)
9(n)
cgo) For example: 2n = o(n), but 2n2 o(n2)
EXAMPLE 2.14. Find o-notation for the function fin) = 5n3 +4n + 7.
Solution: Given that An)= 5n3+ 4n + 7
of(n)=e (g (n)) f(n)=O (e (n))o
Let gn) = ni
The f(n) (g(n)
relationship between the orders Fig. 2.6. then by definition of o-notation, we have
gln): functions of functions
that grow at
least as fast
is
illustrated in beL figure
as g{n)
 Growth of Functions 33
EXAMPLE 2.16. Find anotation for the function fin) = 100 n** 70?
Solution: Given that An)= 100 n3 +70
Let sn)= n2
then by definition of a-notation we have
im fin
Aa)ogtn)=otn)
EXAMPLE 215s =oS m 100+70
lim
Solution: Grenthst fa) =S and gin)=then by definition of o-notation .
we bare
im 0 im 100n+
ie., An)= ogn) = otn),
lim EXAMPLE 2.17.Is= o*)?
Solution: Given that fAn) = and gin) = 2*
then by definition of -notation, we have
im lima)
gln)
lim
i.e. An)= olgin)) An) ol3) =
3 o(3) is true.
5.notation
e-aotation use to denote a lower ie. An) = oXgtn)) = An) = o(2*)
boundthat is not
Definition: For the given functions asymptotically tight. i.e.. 22= o2) is true.
)as the set, fn) and gin) we denote olgtn) (Little omeg afgd
2.6. RELATIONSHIPS BETWEEN O, o, 0, a AND o NOTATIONS
o gn))= nk for any positive constant e > 03 a costant With the help of the definitions O, o, 6, Q and o notations we can draw the following graph
0s n>0 such tnar
Here we can also use
egtn)<An) for all n 2ng which visualizes the relationships between them.
limit to define the
o-notation as
effort
lim fa)
agln) fn)=)
ie, the funetion
fAn) becomes arbitrarily
For example, large relative to gin) as n *
2
on) but" 2
n n
Fig. 2.7.
 Design
and AnalySIs ot AlgoHHH
35
34 Growth of Functions
above graph, we get relationship
from
notations, w e c a n give
some
These all above properties holds for asymptotic comparison ot two
of two real numbers a and
b and asymptotic
o(g(n)) between the comparison
functions fand g.
ofgn) e(g(n) feg a b
Ngn)) fn)) = O gn)) a s 6 =
og(n)) 2 (gn)) a 2b
An) = =
fn)) =
(gn)) =a =b
An)) = o (gn)) = a <b
Fig. 2.8.
An)) = o (gln)) =a >b
smaller than g(n) iffAn) =
ogln)). and ftn) is asymptotically
Hence, we can say that We say thatfn) is asymptotically
is a sub setof2 (g(n). larger than gin) iffn)
o g(n).
=
o(gtn)) c n gln), ie., olg(n))
o g n ) c0 (gtn), ie., olgtn)) a
sub (gln)).
is setof O 2.8. COMPARING ORDERS OF GROWTH
o (gn) ue (gn)) ca (Gn), ie., union (gn) and e (gin)) is a subset ofQ(ait
of o the s a m e problem of size n one is the linear
time algorithm
There are two algorithm for solving
ogn) ue gtn)) cO (gn), ie., union ofo (gin) and e (gn)) is a subset of O (efn 101 +5 and other is the quadratic time algorithm
Awhose running time is given by 74(n)
=
n these running
+ 10. Let us suppose that both
0 eln), ie, intersection B whose running time is given by 7p(n) 5n
=
O (gtn) na en) =
of O(gr) and a (etn) is e (gin))
times are the number of milliseconds
taken on a particular computer on an input
of sizen.
shown in Fig. 2.9.
ASYMPTOTIC NOTATION PROPERTIES The graphs of the running times
2.7.
Let fn) and gln) are asymptotically positive functions; the relational
some
real numbers can also apply to asymptotic comparisons as well. These propertiesof 20,000
properties are given below.
asymptotic notation
15,000
Ten) =5n+10
.Reflexivity T(n) 10,000
Symmetry
Transitivity 5,000 - TA(n)=101n+5
Transpose symmetry
Reflexivity: 20 0
Input size n
80 100
fAn) = f n ) Running tirmes of a linear and a quadratic program
fn) = 0 fn)
Fig. 2.9.
fn) = 2fn)
B is
Symmetry: An) e (g(n))
From the above figure it is clear that for of
the inputs size less than
50,
50, program A
algorithm
if and algorithm the becomes
=
only if gln) ftn)) = faster than the A. When input becomes larger than
Transitivity: An) e (gn)) faster, hence we can say that for the larger input size the algorithm A has the advantage
and gln)= 0 (h(n) imply
=
An) 0 (g(n)) fn) =e (h(n)) over the algorithm B. For example,
and g(n) O (h(n)
=
=
fn) 2 gln)) and
= imply ftn) =0 h{n) For the inputs of size 100, the algorithm A is twice as fast as the algorithm B, and for
g{n) 2 (h(n) = inputs of size 1000, the algorithm A is 20 times as fast as algorithm B.
An)= (gtn))
o imply fn) 2 (h(n) =
fn)= o gn))
and gin) =o (h(n) imply fn) o (h{n) = 2.8.1. Comparing Different Running Times Using Limits
Transpose symmetry: and gln) o (hin) imply fn) =
(h{n)) = o For comparing the orders of growth of two algorithms with running times Ta (n) and
An))- Og(n)) if and 7, (n) respectively, we can compute the limit as:
fn)) olgn) if and only if g(n)
=
=
2 (fln))
only if g(n) = o
fln))
 S6
has a smaller order of growth than m
37
0 implies that 7,(n) has the same order of growth ne Growth of Functions
lim ) c>oimplies that T,(n)
has a larger order of growth than T. lim82 n)
RTn) implies that 7(n)
n (Vn)
comparison between the orders of omo
By using this limit-based approach 1s more efficient tha rele. loB)
This limit based appr0ach
only for large input sizes. it takes advantage or computing the limits h appm lim
based on the definitions because
rule as: LHospital ath 2Nn
lim n
lim 2 log2 e lim = 0
Ifwe consider the running times of two algorithms as 7 , (n) = 15n +20 and n ,
20 and T, n) The limit is equal to zero, hence, we can say that the function log2n has a smaller order
can easily say that T, has smaller order of growth than T =
of growth than vn
eoRTANT POUNTS
EXAMPLE 2.20. Compare the orders of growth of two algorithms with running time
1. Ogtn))is a class of functions fn) that grows less fast than g{n) n! and 3
2 gta) is a class offanetions
fin) that grows at same rate as gin). Solution. Let, T.n)= nl and 7,(n) = 3*
3. gn) isa dass of functions fn) that grows faster than or atleast as fast as eol We can compare the functions by using limits as
lim n )
EXAMPLE 2.18. Let us consider two lim
algorithm with running time h n (n -
1) andn n*1,) n 3
compare the orders ofgrowth.
but by Stirling's approximation
Solution: Let, T(n)= nn- 1), T, n) =n2
nl= 2 r
By using limits we can compare the functions as
T(n) nn-1)
alim 7,(n)
lim lim
n 2 3
hm
3
the lim 2 7
limit is equal to positive constant. Hence,
growth.
a
the both functions have the same
oru
= lim 2 m
EXAMPLE 2.19. Compare the orders of n
Solution. Let,
growth of log, n and yn.
Tn) log,n and T,(n) vn
= the limit is equal to, so we can say that the function n! grows very fast.
=
We can
compare the functions by ASYMPTOTIC EFFICIENCY CLASSES
using-limits as 2.9.
lim lim g2n The classes of efficiency are useful to categorize the algorithms. Two algorithms having the
n 1,(n) same order of growth belongs to the same efticiency class. Some classes of efticiency are
given below:
 38
Bxample
Description
Asymptotic motation
binary search Growth of Functions 39
n)
logarithm sequential search
linear Oln)
merge sort
lg n) lim ta)° lim-
insertion sort n n*»
quadratic OUn)
cubic On) product of two matrices of order n
processing all subsets
exponential O(2) lim
of a set with n elements
factorial OCn!) processing all permutations of order n lim | 1 + .
The best algorighms are that having a logarithmic order
of growth.
On tthe
wth. On
the exponential algorithms can be applied only 1or problems of small sizes. other hand 1 which is a constant so.
We can write (n + a) = e(n°).
EXAMPLE 2.23. Explain why the statement, "The running time of algorithm is at
sOLVED EXAMPLES least O(n)," is meaningless. (UPTU MCA 2006)
Solution: The running time On) is the worst-case running time.
EXAMPLE 2.21. Let ftn) and g(n) be asymptotically non-negative functions
the basic definition of &notation, prove that mar fn), g(n) = 0 (ffn) +gln), Usin,
O-notation describes an upper bound, ie., we use O-notation to
bound the worst-case
running time of an algorithm; therefore O(n) means "atmost" c.n* and the statement" the
(UPTU MCA 2003) running timeof algorithm A is
On)saying that T(n) is asymptotically greater than
at least
Solution: By -definition we know that or equal to all those functions that are O(n*).
So we can say that the statement "The running time of algorithm A is at least O(n )" is
0sC, n)) + g(n)) S max (fn), gln) s C, An) + gln)) for all n 2 n,
meaningless.
0sC, A)+ gtn)) Smax (fn), g(n)
0s max fn)), gtn)) s C, (fn) + gln)) EXAMPLE 2.24. 2*1 = O(2")?Is 2 = O (2n)? (UPTU MCA 2006)
. Solution: (0) 2n*1 = On)
Let we choose constant C = then Now by definition of O-notation we have
2n* Sc.2" for any constant c and all n 2no
0sn) +gtn)) S max
(n), gtn)) will be holds because the maximum is
alwaps
2ns 2.2
this inequality is hold for C2 2 and ng 21
greater than the average
of fin) and gln). Now, if we choose hence, 2 l =O(2") is true
C 1 then the second inequality will be
because the maximum must be smaller than the holds of
0 S max (fln), gln)) S 1.(ftn) +i we can also prove this by using the limit of functions.
(D and (I) are holds so we can
sum fn)
write, max(f{n), g{n)) 0 fin)andgln). Since, both inequai
=
+ g{n). lim (n)
, Where o s c
EXAMPLE 2.22. Show n g(n)
that for any real constant a and b, where b >0, (n =An'
+az n+1 2.2"
lim lim
2 =2
Solution. Let (UPTUB.Tech 2005) (constant)
n)= (n +a n 2n n
Hence, 2n +l = O(2") i s true
sn) = n*
by 6, notation definition, we (i) 2 = O(2")
have
Now by definition of O-notation we have
im n) 22n S c.2"
n g(n) Wnere o <c < For any constant c and all n
2no
 Design and Analysis of Algorithms
40
Growth of Functions 41
92n
c2
(UPTU B. Tech 2000)
i=1
C27
Here the f fa fa.) is an infinite family of functions. Such that
The value of constant c is very according to the value of n, so inequality 2" S c.2", does Solution:
not hold because no constant is greater than all 2", fn)= Olr) then we can say f(n)sc.n
we can also prove this by using the limit functions as
2n) = fn) + f,n) +fn) +. f,n)
L=l
n
lim g(n)
-d c where o Sc<o s Cn + Cn + Cn +.. Cn
s C(n +n +n+... n)
lim im 2 2 2" s C.(n.n)
2 s C.n2 O(n ) Ans.
therefore, above does not hold.
EXAMPLE 2.25. Prove that o(g(n)) n o(g(n)) is the empty set. (UPTU MCA 2006)
EXAMPLE 2.28. For the following program gives Big O analysis of the running timne
(in terms of n)
Solution: By the definition of little o, we have (i) For (i = 0: i < n ; i ++)
lim n ) Alil =+
g(n) = 0 ...) (ii) For ( i = 0: i<n;i++)
and by the definition of little-omega, we have For G i ; j < n:j ++)
For (k =j; k <n;k + 1)
lim n )
n Sn) S++
from equation (i) and (ii) it is clear that fin) cannot both O and o. Hence, Olgln)n (ii) For (i = 0; i <n*n; i++)
olgin)) is the empty set. Alil = i (UPTU MCA 2003)
Solution: (i) For (i = 0: i < n; i + ) > n +1 times
EXAMPLE 2.26. For each ofthe following statements decide in always true, never
true, or sometimes true for non-negative function f ) , g1.). If it is always true or
it is Ali= +; n times
never true. Ezplain why-
Here, let us assume that each statement take unitary cost. Hence, tota cost is
(i) fin)+ gin) = 0 (mar fn)), g(n)
Ttn)= (n +1) x 1+n x1
(ii) fn) O (g(n)) andg(n) 0 (fn). = (n +1)+ (n)
Solution: (i) Always true: Let ftn) is asymptotically greater than gin), ie., fn)> gin),
then fin) +gn) gives 2n +1
a polynomials whose degree will be equal to fn) degree. Now,
Tn) = On)
0S max
fn), sin)) sfn) +gtn) se
hence, An) +gln) = 6(max fln), gln).
max
(fn), g(n)), is hold for e =2
(iü)For (i = 0: i < n;i + ) - n + 1 times
For G =i;j<n:j ++) atmost n x (n + 1) times
(ii) Never true
Proof by counter example. Let fin) n and
gln) = =
n2 For (k =j; k <n;k+ +) atmost n x n x (n + 1) times
n2 is S++;
upperbound on gln) while the reverse is
an
not true.
Hence the total cost is
EXAMPLE 2.27. 1f,
1 f.) is a n infinite family offunctions such that
i s a function from natural number to natural numbers such that f (n) = O ) each function Ttn) =(n + 1) x 1+n x
(n + 1) x 1+n xn x (n + 1)x 1
then
=
n+1+n + n +n3+ n2
42 Design and Analysis of Algorithms Growth of Functions
n2, then
= n3 +2n2+2" +1 fAn) = 33n3 + 4n, gin) =
Ttn) = Otn3) lim ) lim 3 3 n 4 n
n > 8(n)
(ii)For(i =0:i <n xn;i ++) > n? + 1 times
Ali= i n2 timnes lim n(33n +4
Hence, the total cost is
Tn) =(n2+ 1) x 1+nx 1 lim 33 n +4 =
7-
n2+1+n2
=2 n2+1 Hence, 33n3+ 4n = 2(n2) is True.
ii) n! Otn")
Tn)= On*). =
fn) = n!
EXAMPLE 2.29. Given fin) = log,n g(n) = n show which function is asymptoticating gn) = n
faster? (UPTU MCA 2006)
Solution: An) = log, gin) = vn Now, lim = lim
n g{n) n |
We can compare the functions by usinglimits as
= lim 2(n-1)(n-2)...3.2.11
n n
lim
n Bn)
im
n* Vn
.3.2.1
lim
(log, n) im - = 1 (constant)
n (Vn)
Hence, n! = O(n") is True.
= lim loe) (ii) 10n2+9 = OCn2)
fn) = 10n2+9, gin) = n2
2n
n Now, lim @ lim 10n+9
2. loga lim =0 n g(n)
Zn n
the limit is equal to zero, hence, we can say that the function logEn has smaller order of
growth than vn. Therefore, the function fln) logn is faster than vn.
EXAMPLE 2.30. Show that the following equations are correct

=
.
lim
(i) 33n3 + 4n = Mn)
(ii) n! = ( n " )
=lim
10 10 constant
hence, 10n2+9 =O(n2) is True.
(iii) 10n+9 = O(n)
(iv) 6n/(log n + 1) = Ofn) (UPTU MCA 200) 6n
(iu) (log
ae n+1) OCns)
Solution: () An) =
Qgln)) if lim f(n)
n~ g{n) *C
6n3
An)= dog n +1) 8n) =n3
where, 0 <e So
33n3+ 4n? =
Qgln2))
 and AnalySIS of AlgoW
Design =
44 Growth ofFunctions 45
63
fn) h m Oftn)) = n2 then
lim n(logn+1)Xn then
now, n 8n) n2+ 2n 1-n2
fAn)-OAn)) = +
= 0 = 2n + 1
dog n +1) so 2n + 1+6(n2)
definition of
O-notation hence, An)- Ofn)) * 0ftn))
now by
(iv) fln) -gin) * O(max(fin), gn)) is False.
lim =
c where o Sce
n * g\n) that lgfnl) =Ofn lgn).
EXAMPLE 2.33. Prove
that
Solution: By definition of O-notation we know
that
so, we can say
n True.
0sfn) segin)
* On") is .0)
log n +1) 0slgtn) s e.(n lgn)
now, lg(n!l) =
lg (n.(n -1).(n-2).. 1)
Example 2.31. List the following functions in increasing order ofgrowth. Iftno
have the order
same of growth states the fact. functions slg (n.n.n..n)
and 3" slg(n)
log(n), nl, n'Nog(n), n.2", (log)logn (UPTU MCA 20 n lgn
Solution: The list of functions in increasing order of growth is as follows:
now from equation (1), we have
1. logtn) forc21 and n20.
0slgtn!) e nlgn
2. (log n)og n
Prove o r disprove
EXAMPLE 2.34. Let ffn) and g(n) be asymptotically positive functions.
n each of the following conjectures.
33. Jog (a) fon) = O(g(n)) implies g(n) =O(fn).
4. n.27 (bfn) + g(n) = e (min) (fln), g(n)).
5.3 (e) fin)= O(g(n)) implies ls(fin)) O(lg(g(n)), = where lg(g(gn)21 and fn) 2 1
6. n! for all sufficiently large n.
(d) fin) = O(g(n)) implies 2fn) =O (28m)).
EXAMPLE 2.32. ffn) and g(n) be
asymptotically positive functions write true or false (e) fp = Oftn)P).
an
appropriate, against each of the following: ( fn)= O(g(n)) implies g(n) = Mfin)).
)fin) Og(n)) implies g('n) 2(fn))
=
=
(g) fn) = e (fln/2).
(ii) fin) =
ftn))
(ii) ffn) O(fn)) 0
(fin))
-
=
(h) fn)= offin)) =
6 (fin).
Solution: (a) fin) = Ogin) implies gln) = Ofn) False
iv) fn)- g(n) 0 (mas =
(f(n), g(n))» (UPTU MCA 2009 Let fn) =n and gln) = n2.
Solution: (i) fn) O(gln)) =
implies gln) =2(fln)) is True n is an upper bound on n, but n is not an upper bound on n
By transpose symmetry,
fAn) Olgln)) iff =
6) fn)+gn) = 6 (min) (ftn). g{n)M. False
gn) = fn)) Let fn)= n, gn) = n?. So for fn) + gn).min n ) , gin)) = n.
ii) fn) =
0fn)) is True
(i) fn) Thus, n+n2 (n) does not hold.
=
-
Offn) 0 (fn) is False
=
where lglg(n)) 2 0 and fn) z 1 for all
proof by counter example (e) fn) Og(n) implies 1g(fn)) Olgte(n),
= =
sufficiently large n. True
Let
fln) =
n +2n +1 An) = Ogn)) means Inp C such that 0 s fn) sc igtn) for all n>ng ..1)
 46 +getn))
for all n >
n 47
0 s lg ( n ) lg c,
s
Taking Ig give t
)$2lg (g(n).
lgg(n)) s2 l a t Growth of Functions
0 s lgfn)) lg c
s
For sufficiently
large n in (2), False
2t) =O(28n) Solution: We know that
(d)fn)=Ogtn))implies
2n and gin) =n.,
2th = 4n e O(2") n! =
n.(n -1). (n -2).(n -3)...3.2.1
Let ffn) =
. 3.2.1
e) fn) OGn)) False
- nn7
Let fn) be now,
fn) = " 1 - H 1 - -
Qfn)) True
Ogln) implies gn)
=
gn) = 2
D An) =
and
fn) = Og(n)) iffg(n) 2 n ) ) now by definition of o-notation, we have
By transpose symmetry,
lim n )
n ) -P a l e n g{n)
Let
fn)= 2", 2n e O.
n! = » (27).
n
.
im
o
2
Hence,
e (fAn) True.
(h)fAn)+olfn)) =
O(n).
EXAMPLE 2.37. If fn) =
. a , n +a, then provethat fn) =
EXAMPLE 2.35. Prove that nl = ofn")
Solution: An) =
m n" +an +
G
Solution: We know that
n! = n.(n -1).(n-2).(n - 3)... 3.2.1
1)- sn la;|n'-m
i =0
therefore,
sn l V n>1
and &n) = n"
now by definition of o-notation, Hence, fn)= O(n).
we have
(A, in table
lim 0
EXAMPLE 2.38. Indicate, for each pair ofexpressions B)
A is O, o, 2 o or e of B. Assume that k 2 1, e > 0, and c > 1 are constants. Your
the belou, whether
n g(n) answer should be in the form of the table with "yes" or "no" written in each box.
.2.3
n"
(a) lg n n
(6)n
nSinn
(e)n
Hence, d2 2n/2
nl = o(n").
EXAMPLE 2.36. Prove that
n! =
o(2") lg(n)|lgn")
 48
Solution
YES YES No No No
Growth of Functions 49
now by equation (1) and (2), we have
YES YES| No No| No
(n) Cfn)
Tn) Cafn)
Sinn No No No No
n No
T,(n)
2n/2 No No YES YES| No
C
T(n)
YES No YES| No YES
Hence 4 = O(1).
T,(n)
This condition is true only in some cases but is is not universally true.
Ig(n! IE n")YES No YES No YES T n ) = OT,(n))
(iii)
EXAMPLE 2.39. Suppose T,(n) =
Ofn)) and T,(n) Now by definition of O-notation we have
are true? Justify your answers.
=
Ofn)) which of the fola.
llouiy Tn)s C,T,(n)
)T,n)+T,n)= Ofn) Now by equation (2)
1 = O(1) Tn)s CC, fn))
T,(n) Tn) s CCafn))
T,(n)s Cfn)
(iii) T,(n) O(T, (n))
=
(G.G.S.LP.U - B.Tech. 20 Tn) Ofn)) this is the given
= relation which is already true.
Solution:
()7,(n)+T,n) = Oftn)) Hence, Tn) O(T,(n)) is holds.
=
By the definition of EXAMPLE 2.40. Find the G-notation for
O-notation, we have
Tn)= Oun)) if (G.G.S.1.P.U. B.Tech. 2008)
7,tn) s C, An)
and Solution: fn) =
Tn) = O (qn) if
the from
Tn)s C, fn)
equation (1) and (2), we have
Tn)+Tn)s CAn)
Tn)+ TAn)s (C +C,)C, fn)
+
C+Ca Cthen fn)
Let,
Tn)+Tn)s Cfn)) fn)sCn3 for C
s 1.n3
1
Tn)+Tgn) 0 ftn)) is True.
=
Let gn) = n3
ii)
T(n) An)s Cgln)
TnOu)
therefore, An) (gln)
An) eln).
 50
Show that
log () is Ofn log n). Growth of Functions 51
EXAMPLE2.41. i=1
Solution: We know that
Solution: Given that 1. Og(n)) is a class of functions fün) that grows less fast than gtn).
2. eg(n)) is a class of funetions fn) that grows at same rate as gn).
fn)= 2log () and gin)
=n
log n
i=1 3. 2gn)) is a class of functions fn) that grows faster or atleast as fast as gln).
of O-notation, we have () fn) = 3x" = 7 which is independent of n i.e., x work as a constant so gn) = 1.
then by the definition Now comparing order of growth of two functions
=0slog (6)Seq n logn
i-1 lim4
no
3a 1+7=3x2 + 7 (constant)
= log 1 + log 2 + log log + 3 + 4
log n Se, n log n . .
i.e., Ar) has same order of growth as gn) so fin) = 0(g(n)) = An) =e(1)
0+ log 2+ log 3 + l0E 4 + . + log n Scn
=
log n (ii) fn) = 2n + 5n + 3
=
log 2+ log3 + log 4 t
log n
.
log n Se, n
gn) = 2
=
log 2, 3, 4...n Sc, n logn Now comparing order of growth of two functionsas
=
log (n 1)! Se,n logn
-
s log n! sc, n log n lim f) lim n2" * lim1a = 1(constant)
ngn) no
Now by stirlings approximation: i.e., fn) has same order of growth as gln) so
fn) 0g(n)) > f\n) = 0(20).
n!
(iii) fn)=
nt lo = 12 +2+ 33 + 42+. n2
n (n +1)(2n +1)
i.e., log n = n
log=n log n -
n
log e 2n+3n +n
Now from equation (1), we have fn) =
6
Sn log n-n
log eSe n log n gn)= n3
will be hold
for n2 n=1 by choosing c, =1 ie.,
Now comparing order of growth of two functions as
log
i=1
(i)- 0 n log m) lim) lim 2n +3n+n)
6.n
EXAMPLE 2.42. Determine
the
asymptotie order of the following
(i) fn)
3+6 =
functtions
(ii) fin) =
2" + 6n +3
(constant)
i=1 ie, n) has same order of growth as g{n), so
(iv) n) 5 = eg(n))
An) = fn) 0(nS) =
(v) fn)= x+7
01e"
Tech,
(UPTU, B.
 52
fn)= 6
(iv) stn)= 1 Growth of Functions 53
Now,
limL n )
n) n
7
=
5 ie.,An)
has sam
growth as
gn) s0 2Vnlog n'.n012
fn)= e(1) 216logn'L.Vni.n
+ 7
n° logn.log log
fn) n
n)n®2".(n+ 4)
=
(u)
gin) = n
6.
Rank the following functions order or growth:
- ln 2 (constant) ng n
a) 1
Now, lim
=
Ig(lg n) 28" n! (lg n) n3lgn lgn!) 22
as g(n) so
i.e., An) has same order of growth ln In n g*n n.2" n'ogn Inn1 2lgn (lg n)l8n e" 4sn (n + 1)! Vlgn
0gn)) > fn) =
etn).
fn) =
Iglg n)
22lgn n 2n n lgn 22+ 1
notation in the studyof-r. What is the smallest value of n such that an algorithm whose running time is 100 i runs
asymptotic 6.
EXAMPLE 2.43.Why do we use
algorithm? Epl
in brief various asymptotic notations and give theirsignificance
faster than an algorithm whose running time is 2*" on the same machine?
7. For each function fn) and time t in the following table, determine the largest size n of a problem
that can be solved in time t, assuming that the algorithm to solve the problem takes fn)
(UPTU, B-Tech, 2011 microseconds.
Solution: Refer Section 2.5.
second minute hour day month year century|
EXERCISES
. Suppose program A takes 2/1000 units of time and program B takes 1000n? n
values of n does program A take less time than units, Far sh
program B?
2. Some of the following statements are true and some are false. Which ngn
are which?
n
2n
= o(1)
( +1) (6) l l = O(1)
n
(c) (log log x)2 otr3 log x)
= 8. Express the function n/1000 100n2- 100n + 3 in terms of + notation.
(d) log
(e) sin r =
Q(1)
+1 =Mlog log ) 9. We can extend our notation to the case of two parameters n and m that can go to infinity
A cos xlr = O(1) independently at different rates. For a given function gln, m), we denote by
0gn,m)) the set offunctions.
O(1) 1 0gn,m) = {fn, m): there exist positive constants c, ng and mo Such that
3 Prove by
0
(h)5 O1) 0sfn, m) sceg n, m) for all n 2n and m
2m,
induction that the ith Fibonacci Give, corresponding definitions for a (gln, m)) and 0 (g{n, m)).
number satisfies the
equality.
V5 , where is the
golden ratio and ois its
4Arrange the conjugage.
list them following so that each functions
one is
in
increasing order of their rates of growth, for large That .
little oh of its ofgrow
successor.