7/15/15

Big-O Performance Analysis

Execution Time Factors

•  Computer:
–  CPU speed, amount of memory, etc.
•  Compiler:
–  Efficiency of code generation.
•  Data:
–  Number of items to be processed.
–  Initial ordering (e.g., random, sorted, reversed)
•  Algorithm:
–  E.g., linear vs. binary search.
2

1
 7/15/15

Are Algorithms Important?

•  The fastest algorithm
for 100 items may not
be the fastest for
O(log N)
10,000 items!
Execution Time

O(N)

•  Algorithm choice is
O(N2)
more important than
any other factor!

Number of Data Items

Counting the instructions

public void SelectionSort ( int [ ] num ){
int i, j, first, temp;
for ( i = num.length - 1; i > 0; i - - )
{
first = 0; //initialize to subscript of first element
for(j = 1; j <= i; j ++) //locate smallest element between positions 1 and i.
{
i times if( num[ j ] < num[ first ] ) n times
first = j;
}
temp = num[ first ]; //swap smallest found with element in position i.
1 time num[ first ] = num[ i ];
num[ i ] = temp;
}
} 4*(n-1) + 2*(n-1) + 4 * (n-2) + 2 * (n-2)+ 4 + 2*1 =
4(n-1) + 2((n-1)+(n-2)+(n-3)…1) = 4(n-1) * 2 n(n-1)/2
=4(n-1) + n2 - n= n2 + 3n - 4
4

2
 7/15/15

What is Big-O?
•  Big-O characterizes algorithm performance.

•  Big-O describes how execution time grows as

the number of data items increase.

•  Big-O is a function with parameter N, where N

represents the number of items.

Predicting Execution Time

•  If a program takes
10ms to process one log10 N 3 x 10ms .03 sec
item, how long will it
take for 1000 items? N 103 x 10ms 10 sec

•  (time for 1 item) x (Big- N log10 N 103 x 3 x 10ms 30 sec

O( ) time complexity of
N items) N2 106 x 10ms 16 min

N3 109 x 10ms 12 days

3
 7/15/15

Complexity

• In general, we are not so much interested in the

time and space complexity for small inputs.

• For example, while the difference in time

complexity between linear and binary search is
meaningless for a sequence with n = 10, it is
gigantic for n = 230.

Complexity

• For example, let us assume two algorithms A and B

that solve the same class of problems.
• The time complexity of A is 5,000n, the one for B is
⎡1.1n⎤ for an input with n elements.
• For n = 10, A requires 50,000 steps, but B only 3,
so B seems to be superior to A.
• For n = 1000, however, A requires 5,000,000 steps,
while B requires 2.5⋅1041 steps.

4
 7/15/15

Complexity

• This means that algorithm B cannot be used for

large inputs, while algorithm A is still feasible.

• So what is important is the growth of the

complexity functions.

• The growth of time and space complexity with

increasing input size n is a suitable measure for the
comparison of algorithms.

Complexity
•  Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

n 5,000n ⎡1.1n⎤

10 50,000 3

100 500,000 13,781

1,000 5,000,000 2.5⋅1041

1,000,000 5⋅109 4.8⋅1041392

10

5
 7/15/15

The Growth of Functions

• The growth of functions is usually described using
the big-O notation.

• Definition: Let f and g be functions from the

integers or the real numbers to the real numbers.
• We say that f(x) is O(g(x)) if there are constants C
and k such that
• |f(x)| ≤ C|g(x)|
• whenever x > k.

11

The Growth of Functions

• When we analyze the growth of complexity

functions, f(x) and g(x) are always positive.
• Therefore, we can simplify the big-O requirement to
• f(x) ≤ C⋅g(x) whenever x > k.

• If we want to show that f(x) is O(g(x)), we only need

to find one pair (C, k) (which is never unique).

12

6
 7/15/15

The Growth of Functions

• The idea behind the big-O notation is to establish an
upper boundary for the growth of a function f(x) for
large x.
• This boundary is specified by a function g(x) that is
usually much simpler than f(x).
• We accept the constant C in the requirement
• f(x) ≤ C⋅g(x) whenever x > k,
• because C does not grow with x.
• We are only interested in large x, so it is OK if
f(x) > C⋅g(x) for x ≤ k.

13

What is Big-O
f(n) = O(g(n)) iff ∃ positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n) ∀ n ≥ n0.

14

7
 7/15/15

Big-O Example

Prove f(x)=O(n4)

15

The Growth of Functions

• Example:
• Show that f(x) = x2 + 2x + 1 is O(x2).

• For x > 1 we have:

• x2 + 2x + 1 ≤ x2 + 2x2 + x2
• ⇒ x2 + 2x + 1 ≤ 4x2
• Therefore, for C = 4 and k = 1:
• f(x) ≤ Cx2 whenever x > k.

• ⇒ f(x) is O(x2).

16

8
 7/15/15

Common Growth Rates

Big-O Characterization Example

O(1) constant Adding to the front of a linked list

O(log N) log Binary search

O(N) linear Linear search

O(N log N) n-log-n Binary merge sort

O(N2) quadratic Bubble Sort

O(N3) cubic Simultaneous linear equations

O(2N) exponential The Towers of Hanoi problem

17

Common Growth Rates

18

9
 7/15/15

The Growth of Functions

• Question: If f(x) is O(x2), is it also O(x3)?

• Yes. x3 grows faster than x2, so x3 grows also faster

than f(x).

• Therefore, we always have to find the smallest

simple function g(x) for which f(x) is O(g(x)).

19

The Growth of Functions

• “Popular” functions g(n) are
• n log n, 1, 2n, n2, n!, n, n3, log n

• Listed from slowest to fastest growth:

•  1
•  log n
•  n
•  n log n
•  n2
•  n3
•  2n
•  n!
20

10
 7/15/15

The Growth of Functions

• A problem that can be solved with polynomial worst-

case complexity is called tractable.

• Problems of higher complexity are called

intractable.

• Problems that no algorithm can solve are called

unsolvable.

21

Determining Big-O: Repetition

for (i = 1; i <= n; i++)

executed {
n times m=m+2; constant time
}

Total time = (a constant c) * n = cn = O(N)

Ignore multiplicative constants (e.g., “c”).

22

11
 7/15/15

Determining Big-O: Repetition

for (i = 1; i <= n; i++)

outer loop {
executed for (j = 1; j <= n; j++) inner loop
n times { executed
k = k+1 ; n times
}
} constant time

Total time = c * n * n * = cn2 = O(N2)

23

Determining Big-O: Repetition

for (i = 1; i <= n; i++)

outer loop {
executed for (j = 1; j <= 100; j++) inner loop
n times { executed
k = k+1 ; 100 times
}
} constant time

Total time = c * 100 * n * = 100cn = O(N)

24

12
 7/15/15

Determining Big-O: Sequence

constant time (c0) x = x +1;
for (i=1; i<=n; i++)
{ executed
constant time (c1) m = m + 2; n times
}
for (i=1; i<=n; i++)
{
outer loop for (j=1; j<=n; j++) inner loop
executed { executed
n times k = k+1;
n times
}
} constant time (c2)

Total time = c0 + c1n + c2n2 = O(N2)

Only dominant term is used
25

Determining Big-O: Selection

test + worst-case(then, else)

test: if (depth( ) != otherStack.depth( ) )

constant (c0) {
return false; then part:
} constant (c1)
else
{
for (int n = 0; n < depth( ); n++)
{
else part:
another if : if (!list[n].equals(otherStack.list[n])) (c2 + c3) * n
test (c2) return false;
+ }
then (c3) }
Total time = c0 + Worst-Case(c1, (c2 + c3) * n) = O(N)
Total time = c0 + Worst-Case(then, else)
Total time = c0 + Worst-Case(c1, else)
26

13