RAIK 283: Data Structures &
Algorithms
Analysis of Algorithms
Dr. Ying Lu
ylu@cse.unl.edu
August 28, 2012
http://www.cse.unl.edu/~ylu/raik283/
1
Design and Analysis of Algorithms Chapter 2.1
RAIK 283: Data Structures &
Algorithms
Giving credit where credit is due:
• Most of the lecture notes are based on the slides
from the Textbook’s companion website
http://www.aw-bc.com/info/levitin
• Several slides are from Jeff Edmonds of the
York University
• I have modified them and added new slides
2
Design and Analysis of Algorithms Chapter 2.1
The Time Complexity of an
Algorithm
The Time Complexity of an
Algorithm
Specifies how the running time depends
on the size of the input
Purpose
5
Design and Analysis of Algorithms Chapter 2.1
Purpose
To estimate how long a program will run.
To estimate the largest input that can reasonably be
given to the program.
To compare the efficiency of different algorithms.
To help focus on the parts of code that are executed
the largest number of times.
To choose an algorithm for an application.
6
Design and Analysis of Algorithms Chapter 2.1
Purpose (Example)
Suppose a machine that performs a million floating-
point operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?
• 1) If the algorithm requires n2 such operations:
7
Design and Analysis of Algorithms Chapter 2.1
Purpose (Example)
Suppose a machine that performs a million floating-
point operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?
• 1) If the algorithm requires n2 such operations:
– 0.0025 second
8
Design and Analysis of Algorithms Chapter 2.1
Purpose (Example)
Suppose a machine that performs a million floating-
point operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?
• 1) If the algorithm requires n2 such operations:
– 0.0025 second
• 2) If the algorithm requires 2n such operations:
– A) Takes a similar amount of time (t < 1 sec)
– B) Takes a little bit longer time (1 sec < t < 1 year)
– C) Takes a much longer time (1 year < t)
9
Design and Analysis of Algorithms Chapter 2.1
Purpose (Example)
Suppose a machine that performs a million floating-
point operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?
• 1) If the algorithm requires n2 such operations:
– 0.0025 second
• 2) If the algorithm requires 2n such operations:
– over 35 years!
10
Design and Analysis of Algorithms Chapter 2.1
Time Complexity Is a Function
Specifies how the running time depends on the size of the
input.
A function mapping
“size” of input
“time” T(n) executed .
11
Design and Analysis of Algorithms Chapter 2.1
Definition of Time?
12
Design and Analysis of Algorithms Chapter 2.1
Definition of Time
# of seconds (machine, implementation dependent).
# lines of code executed.
# of times a specific operation is performed (e.g.,
addition).
13
Design and Analysis of Algorithms Chapter 2.1
Theoretical analysis of time
efficiency
Time efficiency is analyzed by determining the number of
repetitions of the basic operation as a function of input size
Basic operation: the operation that contributes most
towards the running time of the algorithm.
input size
T(n) ≈ copC(n)
running time execution time Number of times
for basic operation basic operation is
executed
14
Design and Analysis of Algorithms Chapter 2.1
Input size and basic operation
examples
Problem Input size measure Basic operation
Search for key in a list of
n items
Multiply two matrices of
floating point numbers
Compute an
Graph problem
15
Design and Analysis of Algorithms Chapter 2.1
Input size and basic operation
examples
Problem Input size measure Basic operation
Search for key in a list of Number of items in the
Key comparison
n items list: n
Multiply two matrices of
floating point numbers
Compute an
Graph problem
16
Design and Analysis of Algorithms Chapter 2.1
Input size and basic operation
examples
Problem Input size measure Basic operation
Search for key in a list of Number of items in the
Key comparison
n items list: n
Multiply two matrices of Floating point
Dimensions of matrices
floating point numbers multiplication
Compute an
Graph problem
17
Design and Analysis of Algorithms Chapter 2.1
Input size and basic operation
examples
Problem Input size measure Basic operation
Search for key in list of
Number of items in list n Key comparison
n items
Multiply two matrices of Floating point
Dimensions of matrices
floating point numbers multiplication
Floating point
Compute an n
multiplication
Graph problem
18
Design and Analysis of Algorithms Chapter 2.1
Input size and basic operation
examples
Problem Input size measure Basic operation
Search for key in list of
Number of items in list n Key comparison
n items
Multiply two matrices of Floating point
Dimensions of matrices
floating point numbers multiplication
Floating point
Compute an n
multiplication
Visiting a vertex or
Graph problem #vertices and/or edges
traversing an edge
19
Design and Analysis of Algorithms Chapter 2.1
Theoretical analysis of time
efficiency
Time efficiency is analyzed by determining the
number of repetitions of the basic operation as a
function of input size
20
Design and Analysis of Algorithms Chapter 2.1
Which Input of size n?
Efficiency also depends on the particular input
For instance: search a key in a list of n letters
• Problem input: a list of n letters
• How many different inputs?
21
Design and Analysis of Algorithms Chapter 2.1
Which Input of size n?
Efficiency also depends on the particular input
For instance: search a key in a list of n letters
There are 26n inputs of size n.
Which do we consider
for the time efficiency C(n)?
22
Design and Analysis of Algorithms Chapter 2.1
Best-case, average-case, worst-
case
Worst case: W(n) – maximum over inputs of size n
Best case: B(n) – minimum over inputs of size n
Average case: A(n) – “average” over inputs of size n
• NOT the average of worst and best case
• Under some assumption about the probability distribution of all
possible inputs of size n, calculate the weighted sum of expected C(n)
(numbers of basic operation repetitions) over all possible inputs of size
n.
23
Design and Analysis of Algorithms Chapter 2.1
Example: Sequential search
Problem: Given a list of n elements and a search key K, find
an element equal to K, if any.
Algorithm: Scan the list and compare its successive
elements with K until either a matching element is found
(successful search) or the list is exhausted (unsuccessful
search)
Worst case
Best case
Average case
24
Design and Analysis of Algorithms Chapter 2.1
An example
Compute gcd(m, n) by applying the algorithm based on
checking consecutive integers from min(m, n) down to
gcd(m, n)
Input size?
Best case?
Worst case?
Average case?
25
Design and Analysis of Algorithms Chapter 2.1
Types of formulas for basic
operation count
Exact formula
e.g., C(n) = n(n-1)/2
Formula indicating order of growth with specific
multiplicative constant
e.g., C(n) ≈ 0.5 n2
Formula indicating order of growth with unknown
multiplicative constant
e.g., C(n) ≈ cn2
26
Design and Analysis of Algorithms Chapter 2.1
Types of formulas for basic
operation count
Exact formula
e.g., C(n) = n(n-1)/2
Formula indicating order of growth with specific
multiplicative constant
e.g., C(n) ≈ 0.5 n2
Formula indicating order of growth with unknown
multiplicative constant
e.g., C(n) ≈ cn2
Most important: Order of growth within a constant
multiple as n→∞
27
Design and Analysis of Algorithms Chapter 2.1
Asymptotic growth rate
A way of comparing functions that ignores constant factors
and small input sizes
O(g(n)):
Θ (g(n)):
Ω(g(n)):
28
Design and Analysis of Algorithms Chapter 2.1
Asymptotic growth rate
A way of comparing functions that ignores constant factors
and small input sizes
O(g(n)): class of functions f(n) that grow no faster than g(n)
Θ (g(n)): class of functions f(n) that grow at same rate as g(n)
Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)
29
Design and Analysis of Algorithms Chapter 2.1
Table 2.1
30
Design and Analysis of Algorithms Chapter 2.1
Classifying Functions
Giving an idea of how fast a function
grows without going into too much detail.
Which are more alike?
32
Design and Analysis of Algorithms Chapter 2.1
Which are more alike?
Mammals
33
Design and Analysis of Algorithms Chapter 2.1
Which are more alike?
34
Design and Analysis of Algorithms Chapter 2.1
Which are more alike?
Dogs
35
Design and Analysis of Algorithms Chapter 2.1
Classifying Animals
Vertebrates
Fish
Birds
Mammals
Reptiles
Dogs
Giraffe
36
Design and Analysis of Algorithms Chapter 2.1
Which are more alike?
n1000 n2 2n
37
Design and Analysis of Algorithms Chapter 2.1
Which are more alike?
n1000 n2 2n
Polynomials
38
Design and Analysis of Algorithms Chapter 2.1
Which are more alike?
1000n2 3n2 2n3
39
Design and Analysis of Algorithms Chapter 2.1
Which are more alike?
1000n2 3n2 2n3
Quadratic
40
Design and Analysis of Algorithms Chapter 2.1
Classifying Functions?
Functions
41
Design and Analysis of Algorithms Chapter 2.1
Classifying Functions
Functions
Poly Logarithmic
Exponential
Constant
Logarithmic
Polynomial
Factorial
5 5 log n (log n)5 n5 25n n!
42
Design and Analysis of Algorithms Chapter 2.1
Classifying Functions?
Polynomial
43
Design and Analysis of Algorithms Chapter 2.1
Classifying Functions
Polynomial
Linear
Quadratic
?
Cubic
5n 5n2 5n3 5n4
44
Design and Analysis of Algorithms Chapter 2.1
Logarithmic
log n = # digits to write n
10
Differ only by a
log n = # bits to write n
2 multiplicative
= 3.32 log10n constant
log(n1000) = 1000 log(n)
45
Design and Analysis of Algorithms Chapter 2.1
Poly Logarithmic
(log n)5 = log5 n
46
Design and Analysis of Algorithms Chapter 2.1
Which grows faster?
log1000 n n0.001
47
Design and Analysis of Algorithms Chapter 2.1
Poly Logarithmic <<
Polynomial
log1000 n << n0.001
For sufficiently large n
48
Design and Analysis of Algorithms Chapter 2.1
Which grows faster?
10000 n 0.0001 n2
49
Design and Analysis of Algorithms Chapter 2.1
Linear << Quadratic
10000 n << 0.0001 n2
For sufficiently large n
50
Design and Analysis of Algorithms Chapter 2.1
Which grows faster?
n1000 20.001 n
51
Design and Analysis of Algorithms Chapter 2.1
Polynomial << Exponential
n1000 << 20.001 n
For sufficiently large n
52
Design and Analysis of Algorithms Chapter 2.1
Ordering Functions
Functions
Poly Logarithmic
Exponential
Constant
Logarithmic
Polynomial
Factorial
<< << << << <<
5 << 5 log n (log n)5 n5 25n n!
<< << << <<
53
Design and Analysis of Algorithms Chapter 2.1
Which Functions are Constant?
• 5
• 1,000,000,000,000
• 0.0000000000001
• -5
• 0
• 8 + sin(n)
54
Design and Analysis of Algorithms Chapter 2.1
Which Functions are Constant?
Yes • 5
Yes • 1,000,000,000,000
Yes • 0.0000000000001
Yes • -5
Yes • 0
No • 8 + sin(n)
55
Design and Analysis of Algorithms Chapter 2.1
Which Functions are “Constant”?
The running time of the algorithm is a “Constant”
if it does not depend significantly
on the size of the input.
• 5
• 1,000,000,000,000
• 0.0000000000001
• -5
• 0
• 8 + sin(n)
56
Design and Analysis of Algorithms Chapter 2.1
Which Functions are “Constant”?
The running time of the algorithm is a “Constant”
It does not depend significantly
on the size of the input.
Yes • 5
Yes • 1,000,000,000,000 9
Yes • 0.0000000000001
No • 7
-5
No • 0 Lie in between
Yes • 8 + sin(n)
57
Design and Analysis of Algorithms Chapter 2.1
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
58
Design and Analysis of Algorithms Chapter 2.1
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2 Lie in between
• 5n2 + 3n + 2log n
59
Design and Analysis of Algorithms Chapter 2.1
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
Ignore low-order terms
Ignore multiplicative constants.
Ignore "small" values of n.
Write θ(n2).
60
Design and Analysis of Algorithms Chapter 2.1
Examples
f(n) g(n) O(g(n))? Ω(g(n))? Θ(g(n))?
1) ln2n ne
2) nk cn
3) nsinn
n
4) 2n 2n/2
5) nlgc clgn
6) lg(n!) lg(nn)
61
Design and Analysis of Algorithms Chapter 2.1
