Chapter 6

Algorithm Analysis

Bernard Chen
Spring 2006
Why Algorithm analysis
 Generally, we use a computer because
we need to process a large amount of
data. When we run a program on large
amounts of input, besides to make sure
the program is correct, we must be
certain that the program terminates
within a reasonable amount of time.
6.1 What is Algorithm Analysis?
 Algorithm: A clearly specified finite set
of instructions a computer follows to
solve a problem.
 Algorithm analysis: a process of
determining the amount of time,
resource, etc. required when executing
an algorithm.
Big Oh Notation
 Big Oh notation is used to capture the
most dominant term in a function, and
to represent the growth rate.
 Also called asymptotic upper bound.

Ex: 100n3 + 30000n =>O(n3)

100n3 + 2n5+ 30000n =>O(n5)
Upper and lower bounds of a
function
Functions in order of increasing
growth rate
Function Name
C Constant
LogN Logarithmic
Log2N Log-squared
N Linear
NlogN NlogN
N2 Quaratic
N3 Cubic
2n Exponential
Functions in order of
increasing growth rate
6.2 Examples of Algorithm
Running Times
 Min element in an array :O(n)
 Closest points in the plane, ie. Smallest
distance pairs: n(n-1)/2 => O(n2)
 Colinear points in the plane, ie. 3 points
on a straight line: n(n-1)(n-2)/6 =>
O(n3)
6.3 The Max. Contiguous
Subsequence
 Given (possibly negative) integers A1,
A2, .., An, find (and identify the
sequence corresponding to) the max.
value of sum of Ak where k = i -> j.
The max. contiguous sequence sum is
zero if all the integer are negative.
 {-2, 11, -4, 13, -5, 2} =>20
 {1, -3, 4, -2, -1, 6} => 7
 Brute Force Algorithm O(n3)
template <class Comparable>
Comparable maxSubSum(const vector<Comparable> a,
int & seqStart, int & seqEnd){
int n = a.size();
Comparable maxSum = 0;
for(int i = 0; i < n; i++){ // for each possible start point
for(int j = i; j < n; j++){ // for each possible end point
Comparable thisSum = 0;
for(int k = i; k <= j; k++)
thisSum += a[k];//dominant term
if( thisSum > maxSum){
maxSum = thisSum;
seqStart = i;
seqEnd = j;
}
}
}
return maxSum;
} //A cubic maximum contiguous subsequence sum algorithm
O(n3) Algorithm Analysis
 We do not need precise calculations for
a Big-Oh estimate. In many cases, we
can use the simple rule of multiplying
the size of all the nested loops
O(N2) algorithm
 An improved algorithm makes use of the fact
that

 If we have already calculated the sum for the

subsequence i, …, j-1. Then we need only
one more addition to get the sum for the
subsequence i, …, j. However, the cubic
algorithm throws away this information.
 If we use this observation, we obtain an
improved algorithm with the running time
O(N2).
O(N2) Algorithm cont.
template <class Comparable>
Comparable maxSubsequenceSum(const vector<Comparable>& a,
int & seqStart, int &seqEnd){
int n = a.size();
Comparable maxSum = 0;
for( int i = 0; i < n; i++){
Comparable thisSum = 0;
for( int j = i; j < n; j++){
thisSum += a[j];
if( thisSum > maxSum){
maxSum = thisSum;
seqStart = i;
seqEnd = j;
}
}
}
return maxSum;
}//figure 6.5
O(N) Algorithm
template <class Comparable>
Comparable maxSubsequenceSum(const vector<Comparable>& a,
int & seqStart, int &seqEnd){
int n = a.size();
Comparable thisSum = 0, maxSum = 0;
int i=0;
for( int j = 0; j < n; j++){
thisSum += a[j];
if( thisSum > maxSum){
maxSum = thisSum;
seqStart = i;
seqEnd = j;
}else if( thisSum < 0) {
i = j + 1;
thisSum = 0;
}
}
return maxSum;
}//figure 6.8
6.4 General Big-Oh Rules
•Def: (Big-Oh) T(n) is O(F(n)) if there are positive constants c and
n0 such that
T(n)<= cF(n) when n >= n0

•Def: (Big-Omega) T(n) is Ω(F(n)) if there are positive constant c

and n0 such that
T(n) >= cF(n) when n >= n0

•Def: (Big-Theta) T(n) is Θ(F(n)) if and only if

T(n) = O(F(n)) and T(n) = Ω(F(n)) 

•Def: (Little-Oh) T(n) = o(F(n)) if and only if

T(n) = O(F(n)) and T(n) != Θ (F(n))
Figure 6.9
Mathematical Expression Relative Rates of Growth

T(n) = O(F(n)) Growth of T(n) <= growth of

F(n)

T(n) = Ω(F(n)) Growth of T(n) >= growth of

F(n)

T(n) = Θ(F(n)) Growth of T(n) = growth of F(n)

T(n) = o(F(n)) Growth of T(n) < growth of F(n)

Various growth rates
T ( n)   (1) : Constant time
T ( n)   (log log n) : As fast as constant time
T ( n)   (log n) : logarithmic time
T ( n)   ((log n) ) : polylogarithmic time
k

T ( n)   ( n) : linear time
T ( n)   ( n log n) : famous for sorting
T ( n)   ( n ) : qualratic time
2

T ( n)   ( n ) : polynomial time
k

T ( n)   ( 2 ), T ( n)   ( n ), T ( n)   ( n!) : exponential time;

n n

Practical for small values of n (e.g., n = 10 or n = 20)

Worst-case vs. Average-case
 A worst-case bound is a guarantee over
all inputs of size N.
 In an average-case bound, the running
time is measured as an average over all
of the possible inputs of size N.
 We will mainly focus on worst-case
analysis, but sometimes it is useful to
do average one.
6.6 Static Searching problem
 Static Searching Problem
Given an integer X and an array A,
return the position of X in A or an
indication that it is not present. If X
occurs more than once, return any
occurrence. The array A is never
altered.
Cont.
 Sequential search: =>O(n)
 Binary search (sorted data): => O(logn)
 Interpolation search (data must be uniform
distributed): making guesses and search
=>O(n) in worse case, but better than binary
search on average Big-Oh performance,
(impractical in general).
Sequential Search
 A sequential search steps through the data
sequentially until an match is found.
 A sequential search is useful when the array
is not sorted.
 A sequential search is linear O(n) (i.e.
proportional to the size of input)
 Unsuccessful search --- n times
 Successful search (worst) --- n times
 Successful search (average) --- n/2 times
Binary Search
 If the array has been sorted, we can use
binary search, which is performed from the
middle of the array rather than the end.
 We keep track of low_end and high_end,
which delimit the portion of the array in
which an item, if present, must reside.
 If low_end is larger than high_end, we know
the item is not present.
Binary Search 3-ways
comparisons
template < class Comparable>
int binarySearch(const vector<Comparable>& a, const Comparable & x){
int low = 0;
int high = a.size() – 1;
int mid;
while(low < high) {
mid = (low + high) / 2;
if(a[mid] < x)
low = mid + 1;
else if( a[mid] > x)
high = mid - 1;
else
return mid;
}
return NOT_FOUND; // NOT_FOUND = -1
}//figure 6.11 binary search using three-ways comparisons
Binary Search 2-ways
comparisons
template < class Comparable>
int binarySearch(const vector<Comparable>& a,
const Comparable & x){
int low, mid;
int high = a.size() – 1;
while(low < high) {
mid = (low + high) / 2;
if(a[mid] < x)
low = mid + 1;
else
high = mid;
}
return (low == high && a[low] == x) ? low: NOT_FOUND;
}//figure 6.12 binary search using two ways comparisons
6.7 Checking an Algorithm
Analysis
 If it is possible, write codes to test your
algorithm for various large n.
6.8 Limitations of Big-Oh
Analysis
 Big-Oh is an estimate tool for algorithm
analysis. It ignores the costs of memory
access, data movements, memory
allocation, etc. => hard to have a
precise analysis.
Ex: 2nlogn vs. 1000n. Which is faster?
=> it depends on n
Common errors (Page 222)
 For nested loops, the total time is
effected by the product of the loop size,
for consecutive loops, it is not.
 Do not write expressions such as
O(2N2) or O(N2+2). Only the dominant
term, with the leading constant
removed is needed.
 More errors on page 222..
Summary
 Introduced some estimate tools for
algorithm analysis.
 Introduced binary search.
In Class exercises
 Q6.14
 Q6.15
Answers