Week 5 Lecture Algorithm Analysis

Analysis Tools
&
Asymptotic Notations
Data structures & Algorithms

Program Performance
 Program performance is the amount of computer
memory and time needed to run a program.
 How is it determined?
1. Analytically
 performance analysis
2. Experimentally
 performance measurement
Criteria for Measurement
 Space
– amount of memory program occupies
– usually measured in bytes, KB or MB
 Time
– execution time
– usually measured by the number of executions
Complexity
 Complexity is the resources required by the
algorithms, most commonly known as the time
how many steps it needs and the space how much
memory it takes
 Running Time
 Running time — the actual time spent in execution
of an algorithm.
 It depends on a number of factors
– Inputs data
– The hardware and software environment.
Analysis of Algorithms
 There are often many different algorithms which can be
used to solve the same problem. Thus, it makes sense to
develop techniques that allow us to:
– compare different algorithms with respect to their “efficiency”
– choose the most efficient algorithm for the problem
 The efficiency of any algorithmic solution to a problem is a
measure of bye the:
– Time efficiency: the time it takes to execute.
– Space efficiency: the space (primary or secondary memory) it uses.
 We will focus on an algorithm’s efficiency with respect to
time.
Components of Program Space
 Data space
– very much dependent on the computer architecture and
compiler
– The magnitude of the data that a program works with is
another factor
char 1 float 4
short 2 double8
int 2 long double 10
long 4 pointer 2
Unit: bytes
Components of Program Space
 Data space
– Choosing a “smaller” data type has an effect on the
overall space usage of the program.
– Choosing the correct type is especially important when
working with arrays.
– How many bytes of memory are allocated with each of

the following declarations?
double a[100];
int maze[rows][cols];
Time Complexity
 Timecomplexity is the amount of computer time a
program needs to run.
 Why do we care about time complexity?

– Some computers require upper limits for program exe-
cution times.
– Some programs require a real-time response.
– If there are many solutions to a problem, typically we’d
like to choose the quickest.
Machine independence
 The evaluation of efficiency should be as machine
independent as possible.
 It is not useful to measure how fast the algorithm runs as this
depends on which particular computer, OS, programming
language, compiler, and kind of inputs are used in testing
 Instead,
– we count the number of basic operations the algorithm performs.
– we calculate how this number depends on the size of the input.
 A basic operation is an operation which takes a constant
amount of time to execute.
 Hence, the efficiency of an algorithm is the number of basic
operations it performs.
Example of Basic Operations
 Arithmetic operations: *, /, %, +, -
 Assignment statements of simple data types.
 Reading of primitive types.
 Indexing into an array
 Following an object reference
 Returning from a method
 writing of a primitive types
 Simple conditional tests: if (x < 12) ...
 a method's return statement
 We consider an operation such as ++ , += , and *= as consisting of two
basic operations.
Example
 Given an algorithm which finds the maximum value in an array, count
the number of primitive operations executed in this algorithm .
 The number of primitive operations t(n) executed
by algorithm arrayMax is:
 Best case
t(n) = 2+1+n+(2+2)*(n-1)+1 = 5n At least
 Worst case
t(n) = 2+1+n+(2+2+2)*(n-1)+1 = 7n-2 At most
Best, Average and Worst
 Average-Case Analysis—expresses the running time of an algorithm
as an
 average taken over all possible inputs.
 Best-Case Analysis — the shortest running time of an algorithm.
 Worst-Case Analysis — the longest running time of an algorithm.
 We are usually interested in the worst case complexity

Other way of writing the above algorithm
Result of algorithms analysis will enable to:
 determine the running time of a program as a function of its
inputs.
 determine the total or maximum memory space needed for
program data.
 determine the total size of the program code.
 determine whether the program correctly computes the
desired result.
 determine the complexity of the program--e.g., how easy is
it to read, understand, and modify.
 determine the strongest of the program--e.g., how well
does it deal with unexpected or erroneous inputs?
Loop Example
 Find the exact number of basic operations in the following program
fragment:
double x, y;
x = 2.5 ; y = 3.0; 2
for(int i = 0; i <= n; i++) 1 n+1,
2*n
{
a[i] = x * y; 2*n
x = 2.5 * x; 2*n
y = y + a[i]; 3*n
}
The total number of basic operations is 6 * n + 2 * n + (n + 1) + 3
= 10n + 4
 Suppose n is a multiple of 2. Determine the number of basic operations performed by of
the method myMethod():
static int myMethod(int n) static int helper(int n){

{
int sum = 0;
int sum = 0;
for(int i = 1; i < n; i = i * 2)
for(int i = 1; i <= n; i++)
sum = sum + i + helper(i); sum = sum + i;
return sum; return sum;
} }
for(int i = 1; i < n; i = i * 2)
sum = sum + i + helper(i); is log2n
Therefore the number of basic operations is:
1 + 1 + ( log2n) + log2n [2 + 4 + 1 + 1 + (n + 1) + n[2 + 2] + 1] + 1
= 2 + log2n + log2n [10 + 5n] + 1
= 5 n log2n + 11 log2n + 3
Operation Count
 Worst case count = maximum count

 Best case count = minimum count
 Average count
Step Count
 The operation-count method omits accounting for
the time spent on all but the chosen operation
 The step-count method count for all the time spent
in all parts of the program
 A program step is loosely defined to be a syntacti-
cally or semantically meaningful segment of a pro-
gram for which the execution time is independent
of the instance characteristics.
– 100 adds, 100 subtracts, 1000 multiples can be counted
as one step.
– However, n adds cannot be counted as one step.
Asymptotic Complexity
 Two important reasons to determine operation
and step counts
1. To compare the time complexities of two programs that
compute the same function
2. To predict the growth in run time as the instance char-
acteristic changes
 Neither of the two yield a very accurate measure
– Operation counts: focus on “key” operations and ig-
nore all others
– Step counts: the notion of a step is itself inexact
 Asymptotic complexity provides meaningful
statements about the time and space complexi-
ties of a program
Complexity Example
 Two programs have complexities c1n2 + c2n and
c3n, respectively
 The program with complexity c3n will be faster
than the one with complexity c1n2 + c2n for suffi-
ciently large values of n
 For small values of n, either program could be
faster  depends on the values of c1, c2 and c3
 If c1 = 1, c2 = 2, c3 = 100, then c1n2 + c2n ≤ c3n for
n ≤ 98 and c1n2 + c2n > c3n for n > 98
 What if c1 = 1, c2 = 2, and c3 = 3?
Big Oh (O) Notation
 Definition of “Big-O”
Big-Oh notation represents the growth rate of an algorithm.
Big-Oh notation shows the comparison result of the growth
rate of two function.
Let f(n) and g(n) be functions mapping nonnegative
integers to real numbers.
We say that f(n) is O(g(n)) if there is a real constant c>0 and
an integer constant n0 >= 1 such that
f(n)<=cg(n) for every integer n>= n0
Big Oh Examples
Justify 7n-2 is O(n).
Justification: we need to find a real constant c>0 and an integer
n0>=1 such that:
7n-2<=cn for every n>=n0.
We can choose c=7, n0=1 and then we have
7n-2<7n when n>=1
Thus 7n-2 is O(n).
 Justify 20n3 + 5n2 + 6 O(n3 )
Justification:
We chose c=31, n0=1 and then we have
20n3 + 5n2 + 6 ≤ 31n3 when n>=1
Thus 20n3 + 5n2 + 6 is O(n3 )
Common Growth Rate Functions
1 (constant): growth is independent of the problem
size n.
 log N (logarithmic): growth increases slowly com-
2
pared to the problem size (binary search)
 N (linear): directly proportional to the size of the
problem.
 N * log N (n log n): typical of some divide and
2
conquer approaches (merge sort)
 N2 (quadratic): typical in nested loops
 N3 (cubic): more nested loops
 2N (exponential): growth is extremely rapid and
possibly impractical.
Practical Complexities
logn n nlogn n2 n3 2n
0 1 0 1 1 2
1 2 2 4 8 4
2 4 8 16 64 16
3 8 24 64 512 256
4 16 64 256 4096 65536
5 32 160 1024 32768 4294967296
Overly complex programs may not be practical given the
computing power of the system.
READ Chapter 2 of the texbook
Proving Big-O Complexity
To prove that f(n) is O(g(n)) we find any pair of values n0 and c that
satisfy:
f(n) ≤ c * g(n) for  n  n0
Note: The pair (n0, c) is not unique. If such a pair exists then there is an
infinite number of such pairs.
Example: Prove that f(n) = 3n2 + 5 is O(n2)

We try to find some values of n and c by solving the following inequal-
ity:
3n2 + 5  cn2 OR 3 + 5/n2  c
(By putting different values for n, we get corresponding values for c)
 4 3 2 1 n0
3 3.3125 3.55 4.25 8 c

Week 5 Lecture Algorithm Analysis

Uploaded by

Copyright:

Available Formats

Week 5 Lecture Algorithm Analysis

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Week 5 Lecture Algorithm Analysis

Uploaded by

Copyright:

Available Formats

Analysis Tools

Data structures & Algorithms

– How many bytes of memory are allocated with each of

 Why do we care about time complexity?

 We are usually interested in the worst case complexity

static int myMethod(int n) static int helper(int n){

 Worst case count = maximum count

Example: Prove that f(n) = 3n2 + 5 is O(n2)

You might also like