Fundamentals of The Analysis of Algorithm Efficiency
Fundamentals of The Analysis of Algorithm Efficiency
Approaches:
• theoretical analysis
• empirical analysis
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-2
Theoretical analysis of time efficiency
Time efficiency is analyzed by determining the number of
repetitions of the basic operation as a function of input size
T(n) ≈ copC(n)
running time execution time Number of times
for basic operation basic operation is
executed
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-3
Input size and basic operation examples
Visiting a vertex or
Typical graph problem #vertices and/or edges
traversing an edge
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-4
Empirical analysis of time efficiency
Select a specific (typical) sample of inputs
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-5
Best-case, average-case, worst-case
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-6
Example: Sequential search
Worst case
Best case
Average case
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-7
Types of formulas for basic operation’s count
Exact formula
e.g., C(n) = n(n-1)/2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-8
Order of growth
Most important: Order of growth within a constant multiple
as n→∞
Example:
• How much faster will algorithm run on computer that is
twice as fast?
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-9
Values of some important functions as n
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-10
Asymptotic order of growth
A way of comparing functions that ignores constant factors and
small input sizes
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-11
Big-oh
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-12
Big-omega
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-13
Big-theta
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-14
Establishing order of growth using the definition
Examples:
10n is O(n2)
5n+20 is O(n)
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-15
Some properties of asymptotic order of growth
f(n) O(f(n))
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-16
Establishing order of growth using limits
Examples:
• 10n vs. n2
• n(n+1)/2 vs. n2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-17
L’Hôpital’s rule and Stirling’s formula
L’Hôpital’s rule: If limn f(n) = limn g(n) = and
the derivatives f´, g´ exist, then
Stirling’s formula: n! (2n)1/2 (n/e)n
lim f(n) lim f ´(n)
=
n g(n) n g ´(n)
Example: log n vs. n
Example: 2n vs. n!
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-18
Orders of growth of some important functions
All logarithmic functions loga n belong to the same class
(log n) no matter what the logarithm’s base a > 1 is
order log n < order n (>0) < order an < order n! < order nn
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-19
Basic asymptotic efficiency classes
1 constant
log n logarithmic
n linear
n log n n-log-n
n2 quadratic
n3 cubic
2n exponential
n! factorial
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-20
Time efficiency of nonrecursive algorithms
General Plan for Analysis
Decide on parameter n indicating input size
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-21
Useful summation formulas and rules
liu1 = 1+1+…+1 = u - l + 1
In particular, liu1 = n - 1 + 1 = n (n)
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-22
Example 1: Maximum element
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-23
Example 2: Element uniqueness problem
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-24
Example 3: Matrix multiplication
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-25
Example 4: Gaussian elimination
Algorithm GaussianElimination(A[0..n-1,0..n])
//Implements Gaussian elimination of an n-by-(n+1) matrix A
for i 0 to n - 2 do
for j i + 1 to n - 1 do
for k i to n do
A[j,k] A[j,k] - A[i,k] A[j,i] / A[i,i]
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-26
Example 5: Counting binary digits
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-27
Plan for Analysis of Recursive Algorithms
Decide on a parameter indicating an input’s size.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-28
Example 1: Recursive evaluation of n!
Definition: n ! = 1 2 … (n-1) n for n ≥ 1 and 0! = 1
Size:
Basic operation:
Recurrence relation:
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-29
Solving the recurrence for M(n)
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-30
Example 2: The Tower of Hanoi Puzzle
1 3
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-31
Solving recurrence for number of moves
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-32
Tree of calls for the Tower of Hanoi Puzzle
n
n-1 n-1
1 1 1 1 1 1 1 1
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-33
Example 3: Counting #bits
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-34
Fibonacci numbers
The Fibonacci numbers:
0, 1, 1, 2, 3, 5, 8, 13, 21, …
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-35
Solving aX(n) + bX(n-1) + cX(n-2) = 0
Set up the characteristic equation (quadratic)
ar2 + br + c = 0
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-36
Application to the Fibonacci numbers
Characteristic equation:
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-37
Computing Fibonacci numbers
1. Definition-based recursive algorithm
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 2 2-38