Advanced Analysis of Algorithms

(2.2,3.1)

Introduction
Proof By Induction
Asymptotic notation
 The Course
• Textbook: Introduction to Algorithms,
Cormen, Leiserson, Rivest, Stein
 Objective of This Course
Major objective of this course is:
• Design and analysis of modern algorithms
• Different variants
• Accuracy
• Efficiency
• Comparing efficiencies
• Motivation thinking new algorithms
• Advanced designing techniques
• Real world problems will be taken as examples
• To create feelings about usefulness of this course
 What is Algorithm?
• A computer algorithm is a detailed step-by-step method for
solving a problem by using a computer.
• An algorithm is a sequence of unambiguous instructions for
solving a problem in a finite amount of time.
• An Algorithm is well defined computational procedure that
takes some value, or set of values, as input and produces some
value, or set of values as output.
• More generally, an Algorithm is any well defined computational
procedure that takes collection of elements as input and
produces a collection of elements as output.

Input output
Algorithm
 Popular Algorithms, Factors of Dependence
• Most basic and popular algorithms are
 Sorting algorithms
 Searching algorithms
Which algorithm is best?
• Mainly, it depends upon various factors, for example in
case of sorting
 The number of items to be sorted
 The extent to which the items are already sorted
 Possible restrictions on the item values
 The kind of storage device to be used etc.
 One Problem, Many Algorithms
Problem
• The statement of the problem specifies, in general terms,
the desired input/output relationship.
Algorithm
• The algorithm describes a specific computational
procedure for achieving input/output relationship.
Example
• One might need to sort a sequence of numbers into non-
decreasing order.
Algorithms
• Various algorithms e.g. merge sort, quick sort, heap sorts
etc.
 Important Designing Techniques
• Brute Force
 Straightforward, naive approach
 Mostly expensive
• Divide-and-Conquer
 Divide into smaller sub-problems
• Iterative Improvement
 Improve one change at a time
• Decrease-and-Conquer
 Decrease instance size
• Transform-and-Conquer
 Modify problem first and then solve it
• Space and Time Tradeoffs
 Use more space now to save time later
 Some of the Important Designing Techniques
• Greedy Approach
 Locally optimal decisions, can not change once made.
 Efficient
 Easy to implement
 The solution is expected to be optimal
 Every problem may not have greedy solution
• Dynamic programming
 Decompose into sub-problems like divide and conquer
 Sub-problems are dependant
 Record results of smaller sub-problems
 Re-use it for further occurrence
 Mostly reduces complexity exponential to polynomial
 Problem Solving Phases
• Analysis
 How does system work?
 Breaking a system down to known components
 How components (processes) relate to each other
 Breaking a process down to known functions

• Synthesis
 Building tools
 Building functions with supporting tools
 Composing functions to form a process
 How components should be put together?
 Final solution
 Problem Solving Process
• Problem
• Strategy
• Algorithm
 Input
 Output
 Steps
• Analysis
 Correctness
 Time & Space
 Optimality
• Implementation
• Verification
 Model of Computation (Assumptions)
• Design assumption
 Level of abstraction which meets our requirements
 Neither more nor less e.g. [0, 1] infinite continuous interval
• Analysis independent of the variations in
 Machine
 Operating system
 Programming languages
 Compiler etc.
• Low-level details will not be considered
• Our model will be an abstraction of a standard generic
single-processor machine, called a random access
machine or RAM.
 Model of Computation (Assumptions)
• A RAM is assumed to be an idealized machine
 Infinitely large random-access memory
 Instructions execute sequentially
• Every instruction is in fact a basic operation on two values in
the machines memory which takes unit time.
• These might be characters or integers.
• Example of basic operations include
 Assigning a value to a variable
 Arithmetic operation (+, - , × , /) on integers
 Performing any comparison e.g. a < b
 Boolean operations
 Accessing an element of an array.
 RAM model

– All memory equally expensive to access

– No concurrent operations:
• instructions are executed one after another, with no
concurrent operations.
– All reasonable instructions take unit time
• Except, of course, function calls
– Strictly speaking, one should precisely define the
instructions of the RAM model and their costs.
• To do so, however, would be tedious and would yield
little insight into algorithm design and analysis.
 Model of Computation (Assumptions)
• In theoretical analysis, computational complexity
 Estimated in asymptotic sense, i.e.
 Estimating for large inputs
• Big O, Omega, Theta etc. notations are used to compute
the complexity
• Asymptotic notations are used because different
implementations of algorithm may differ in efficiency
• Efficiencies of two given algorithm are related
 By a constant multiplicative factor
 Called hidden constant.
 Drawbacks in Model of Computation
First poor assumption
• We assumed that each basic operation takes constant
time, i.e. model allows
 Adding
 Multiplying
 Comparing etc.
two numbers of any length in constant time
• Addition of two numbers takes a unit time!
 Not good because numbers may be arbitrarily
• Addition and multiplication both take unit time!
 Again very bad assumption
 Model of Computation not so Bad
Finally what about Our Model?
• But with all these weaknesses, our model is not so bad
because we have to give the
 Comparison not the absolute analysis of any algorithm.
 We have to deal with large inputs not with the small size
• Model seems to work well describing computational
power of modern nonparallel machines

Can we do Exact Measure of Efficiency ?

• Exact, not asymptotic, measure of efficiency can be
sometimes computed but it usually requires certain
assumptions concerning implementation
 Review: Induction
• Suppose
 S(k) is true for fixed constant k
o Often k = 0
 S(n)  S(n+1) for all n >= k
• Then S(n) is true for all n >= k
 Proof By Induction
• Claim:S(n) is true for all n >= k
• Basis:
 Show formula is true when n = k
• Inductive hypothesis:
 Assume formula is true for an arbitrary n
• Step:
 Show that formula is then true for n+1
 What is Mathematical Induction?
Let P(n) be the predicate defined for any positive
integers n, and let n0 be a fixed integer. Suppose the
following two statements are true
1. P(n0) is true.
2. For any positive integers k, k  n0, if P(k) is true
then P(k+1)is true.

If both of the above statements are true then the

statement:
 n  N, such that n  n0, P(n) is also true
 Induction Example:
• Prove 1 + 2 + 3 + … + n = n(n+1) / 2
 Basis:
o If n = 0, then 0 = 0(0+1) / 2
 Inductive hypothesis:
o Assume 1 + 2 + 3 + … + n = n(n+1) / 2
 Step (show true for n+1):
1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1)
= n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2
= (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2
 Induction Example:

• Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for all a 

1
 Basis: show that a0 = (a0+1 - 1)/(a - 1)
a0 = 1 = (a1 - 1)/(a - 1)
 Inductive hypothesis:
o Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1)
 Step (show true for n+1):
a0 + a1 + … + an+1 = a0 + a1 + … + an + an+1
= (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1)
Another Easy Example
Reasoning
Use Mathematical InductionoftoProof
prove that sum of the
first n odd positive integers is n2.
Proof
• Let P(n) denote the proposition that
• Basis step : P(1) is true , since 1 = 12 n (2i  1) n 2
• Inductive step : Let P(k) is true for a positive
i 1
integer k,
i.e., 1+3+5+…+(2k-1) = k2
• Note that: 1+3+5+…+(2k-1)+(2k+1) = k2+2k+1= (k+1)2
∴ P(k+1) true, by induction, P(n) is true for all n  Z+
Another Proof
n n

 (
i 1
2i  1) 2  i
i 1
 n n ( n  1)  n n 2
Proving Inequalities
• Use mathematical
Reasoning ofprove
Induction to Proofthat the inequality
n < 2n for all n  Z+
Proof
• Let P(n) be the proposition that n < 2n
• Basis step : P(1) is true since 1 < 21 .
• Inductive step :
Assume that P(n) is true for a positive integer n = k,
i.e., k < 2k.
• Now consider for P(k+1) :

∴ P(k+1) is true.
Since, k + 1 < 2k + 1  2k + 2k = 2.2k = 2k + 1

It proves that P(n) is true for all n  Z+.

Example : Harmonic Numbers
The harmonic numbers Hk, k = 1, 2, 3, …, are
defined by H 1  1  1  ...  1
k
2 3 k
n
Use mathematical induction to show that H 2n 1 
2
whenever n is a nonnegative integer.
Proof
Let P(n) be the proposition that H 2n 1  n / 2
Basis step :
P(0) is true, since, H 20 H1 1 1  0 / 2 1
Inductive step
Assume that P(k) is true for some k, H 2k 1  k / 2
Example : Harmonic Numbers
Now consider
1 1 1 1 1 1
H 2k 1 1      k  k  k    k 1
2 3 2 2 1 2  2 2 2 k  2 k
1 1 1
H 2 k  k  k    k 1
2 1 2  2 2
k 1 1 1
(1  )  k  k    k 1
2 2 1 2  2 2
k 1 1 1
(1  )  k k
 k k
  k
2 2 2 2 2 2  2k
k 2k k 1 k 1
(1  )  k k 1   1 
2 2 2 2 2 2
∴P(k+1) is true.
Hence the statement is true for all n  Z+.
More Complicated Example
1
Problem
Let f0  x   , and fn 1 f0  fn , n 0.
2 x
• Find an expression for fn and prove it by induction.
Solution
1
Since f 0  and f n 1 f o  f 0 therefore
2 x
1 1 2 x
f1 ( x) f 0  f 0 (x) f 0 ( ) 
2 x 1 3  2x
2
2 x
2 x 1 3  2x
And, f 2 ( x) f 0  f1 (x) f 0 ( ) 
3  2x 2 x 4  3x
2
3  2x
More Complicated Example
3  2x
And, f 3 ( x) f 0  f 2 (x) f 0 ( )
4  3x
1 4  3x
 
3  2x 5  4x
2
4  3x
And so on

n  (n  1) x
f n ( x) f 0  f n -1 (x) f 0 ( )
(n  1)  nx
1 (n  1)  nx
 
n  (n  1) x (n  2)  (n  1) x
2
(n  1)  nx
More Complicated Example
Now generalized function is

(n  1)  nx
f n ( x) 
(n  2)  (n  1) x
Now we prove this guess by mathematical Induction
Basis case: take n = 0
1
f0  , which is true
2 x
Inductive hypothesis: assume that statement is true n = k
(k  1)  kx
f k ( x) 
(k  2)  (k  1) x
More Complicated Example
Claim: Now we have to prove that statement is true n = k + 1
(k  1  1)  (k  1) x (k  2)  (k  1) x
f k 1 ( x)  
(k  1  2)  (k  1  1) x (k  3)  (k  2) x

By definition: f n 1  f 0  f n  f k 1  f 0  f k  k 0

(k  1)  kx 1
f k 1 ( x) f 0 ( )
(k  2)  (k  1) x (k  1)  kx
2
(k  2)  (k  1) x

(k  2)  (k  1) x
After simplifica tion, f k 1 ( x)  , proved.
(k  3)  (k  2) x
 Asymptotic Performance
• In this course, we care most about asymptotic
performance
 How does the algorithm behave as the problem
size gets very large?
o Running time
o Memory/storage requirements
o Bandwidth/power requirements/logic gates/etc.
 Asymptotic Notation
• By now you should have an intuitive feel for
asymptotic (big-O) notation:
 What does O(n) running time mean? O(n2)?
O(n lg n)?
 How does asymptotic running time relate to
asymptotic memory usage?
• Our first task is to define this notation more
formally and completely
 Analysis
• Best case
• Worst case
 Provides an upper bound on running time
 An absolute guarantee
• Average case
 Provides the expected running time
 Very useful, but treat with care: what is “average”?
o Random (equally likely) inputs
o Real-life inputs
 An Example: Insertion Sort
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
30 10 40 20 i =  j =  key = 
A[j] =  A[j+1] = 
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
30 10 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
30 30 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
30 30 40 20 i=2 j=1 key = 10
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
30 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
30 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=2 j=0 key = 10
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 10
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 40
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=3 j=0 key = 40
A[j] =  A[j+1] = 10
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=3 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 40
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 20 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 40 i=4 j=3 key = 20
A[j] = 40 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 40 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 40
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 30 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 30 40 i=4 j=2 key = 20
A[j] = 30 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 30 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 30
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 20 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 An Example: Insertion Sort
10 20 30 40 i=4 j=1 key = 20
A[j] = 10 A[j+1] = 20
1 2 3 4
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
} Done!
 Insertion Sort
InsertionSort(A, n) { What is the precondition
for i = 2 to n { for this loop?
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}
 Insertion Sort
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
} How many times will
} this loop execute?
 Best Case
• Even for inputs of a given size, an algorithm’s running time
may depend on which input of that size is given (i.e. how they
are arranged)
• For example, in INSERTION-SORT, the best case occurs if
the array is already sorted
• For each j = 2, 3, . . . , n, we then find that A[i ] ≤ key in line 5
when i has its initial value of j − 1.
• Thus t j = 1 for j = 2, 3, . . . , n, and the best-case running time
is (i.e. condition will be false in this loop and it will take
constant time i.e. 1) so
Best Case
Worst case
 Upper Bound Notation
• We say InsertionSort’s run time is O(n2)
 Properly we should say run time is in O(n2)
 Read O as “Big-O” (you’ll also hear it as “order”)
• In general a function
 f(n) is O(g(n)) if there exist positive constants c
and n0 such that f(n)  c  g(n) for all n  n0
• Formally
 O(g(n)) = { f(n):  positive constants c and n0 such
that f(n)  c  g(n)  n  n0
 Insertion Sort Is O(n2)
• Proof
 Suppose runtime is an2 + bn + c
o If any of a, b, and c are less than 0 replace the constant
with its absolute value
 an2 + bn + c  (a + b + c)n2 + (a + b + c)n + (a + b +
c)
  3(a + b + c)n2 for n  1
 Let c’ = 3(a + b + c) and let n0 = 1
• Question
 Is InsertionSort O(n3)?
 Is InsertionSort O(n)?
 Lower Bound Notation
• We say InsertionSort’s run time is (n)
• In general a function
 f(n) is (g(n)) if  positive constants c and n0 such
that 0  cg(n)  f(n)  n  n0
 Asymptotic Tight Bound
• A function f(n) is (g(n)) if  positive
constants c1, c2, and n0 such that

c1 g(n)  f(n)  c2 g(n)  n  n0

• Theorem
 f(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n))
 Other Asymptotic Notations
• A function f(n) is o(g(n)) if  positive
constants c and n0 such that
f(n) < c g(n)  n  n0
• A function f(n) is (g(n)) if  positive
constants c and n0 such that
c g(n) < f(n)  n  n0
• Intuitively,
o() is like <  () is like >  () is like =
 O() is like   () is like 