Chapter 2

Fundamentals of the Analysis of Algorithm Efficiency

Establishing Order of Growth using Limits

Examples
Establishing order of growth using limits

0 order of growth of T(n) < order of growth of g(n)

lim T(n)/g(n) = c > 0 order of growth of T(n) = order of growth of g(n)

n→∞
∞ order of growth of T(n) > order of growth of g(n)

Examples:
• 10n vs. n2

• n(n+1)/2 vs. n2

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-1
Establishing order of growth using limits
Examples:
• 10n vs. n2

limn 10n / n2 = limn 10n / n2 = limn 10 / n = 0

Therefore, the first function 10n has smaller order of growth than n2

• n(n+1)/2 vs. n2

limn n(n+1) /2 = limn n(n+1) = limn ( ½ + 1/2n) = 1/2

n2 2 n2

Therefore the first function n(n+1) has the same order of growth with n2

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-2
L’Hôpital’s rule and Stirling’s formula

L’Hôpital’s rule: If limn f(n) = limn g(n) =  and

• the derivatives f´, g´ exist, then

lim f(n) lim f ´(n)

=
n g(n) n g ´(n)
Example: log n vs. n
limn log n = limn n = 

limn log n = limn d(log n) = limn (1/n) = limn 1/n = 0

 n d(n) 1

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-3
Orders of growth of some important functions

 All logarithmic functions loga n belong to the same class

(log n) no matter what the logarithm’s base a > 1 is

 All polynomials of the same degree k belong to the same class:

aknk + ak-1nk-1 + … + a0  (nk)

 Exponential functions an have different orders of growth for

different a’s

 order log n < order n (>0) < order an < order n! < order nn

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-4
Basic asymptotic efficiency classes
1 constant

log n logarithmic

n linear

n log n n-log-n

n2 quadratic

n3 cubic

2n exponential

n! factorial

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-5
Time efficiency of nonrecursive algorithms
General Plan for Analysis

 Decide on parameter n indicating input size

 Identify algorithm’s basic operation

 Determine worst, average, and best cases for input of size n

 Set up a sum for the number of times the basic operation is

executed

 Simplify the sum using standard formulas and rules (see

Appendix A)

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-6
Useful summation formulas and rules
liu1 = 1+1+…+1 = u - l + 1
In particular, liu1 = n - 1 + 1 = n  (n)

1in i = 1+2+…+n = n(n+1)/2  n2/2  (n2)

1in i2 = 12+22+…+n2 = n(n+1)(2n+1)/6  n3/3  (n3)

0in ai = 1 + a +…+ an = (an+1 - 1)/(a - 1) for any a  1

In particular, 0in 2i = 20 + 21 +…+ 2n = 2n+1 - 1  (2n )

(ai ± bi ) = ai ± bi cai = cai liuai = limai + m+1iuai

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-7
Example 1: Maximum element

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-8
Maximum element analysis

 The obvious measure of an input’s size here is

the number of elements in the array, i.e. n

 The operations that are going to be executed

most often are in the algorithm’s for loop. The
basic operation is the comparison:
 A[i] > maxval


Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-9
Maximum element analysis

 Since the number of comparisons will be the

same for all arrays of size n, there is no need to
distinguish among the worst, average and best
cases here.
 Let C(n) denote the number of times this
comparison is executed and try to find a
formula expressing it as a function of size n.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-10
Maximum element analysis
 The algorithm makes 1 comparison on each
execution of the loop, which is repeated for
each value of the loop’s variable I within the
bound 1and n-1 (inclusively).
 Therefore we get the following sum for C(n).
 In particular,
 C(n) = 1in-1 1
 = (n – 1) -1 + 1
 = n-1  (n) .

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-11
Example 2: Element uniqueness problem

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-12
Element Uniqueness : Worst-case

 The algorithm’s input size is n

 The algorithm’s basic operation is comparison

between A[i] and A[j] which appeared in the
inner double loop

 The worst-case efficiency of the algorithm is

given by:

 Cworst(n) = 0i n-2 i+1jn-1 1

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-13
Element Uniqueness : Worst-case

 The algorithm’s worst-case

 Cworst(n) = 0i n-2 i+1jn-1 1
 Cworst(n) = 0i n-2 [ (n-1)- (i+1) + 1 ]
 Cworst(n) = 0i n-2 (n-1-i)
 Cworst(n) = 0i n-2 (n-1) - 0i n-2 i
 = (n-2+1)(n-1) – {0 + 1 + 2 + . . . + (n-3) + (n-2) }
 Cworst(n) = (n-1)(n-1) - (n-2)(n-1)
 2
 Cworst(n) = (n-1)n  (n2)
 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-14
Element Uniqueness : Best-case

 The algorithm’s basic operation is comparison

between A[i] and A[j]

 The best-case efficiency of the algorithm is

when the first two elements are the same.

 The number of comparison would be 1

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-15
Example 3: Matrix multiplication

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-16
Matrix multiplication : Worst-case

 The algorithm’s worst-case

 Cworst(n) = 0i n-1 0jn-1 0kn-1 1
 Cworst(n) = 0i n-1 0jn-1 [ (n-1) - 0 + 1 ]
 Cworst(n) = 0i n-1 0jn-1 n
 Cworst(n) = n 0i n-1 0jn-1 1
 Cworst(n) = n 0i n-1 [ (n-1) - 0 + 1 ]
 Cworst(n) = n 0i n-1 n
 Cworst(n) = n2 0i n-1 1
 Cworst(n) = n2 [ (n-1) - 0 + 1 ]
 Cworst(n) = (n2)n  (n3)
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-17