Scribd
Upload
183 views28 pages

Data Structures and Algorithms: Asymptotic Notations

The document discusses algorithm analysis. It explains that analysis determines an algorithm's properties like time and space complexity. Asymptotic notations like Big-O, Omega, and Theta are used to analyze algorithms since they ignore constants and lower order terms and focus on relative growth rates. Common functions like logarithms and polynomials are classified based on their growth rates. Rules for combining complexities using sums and products are presented. Examples demonstrate how to determine the complexity class of algorithms.

Uploaded by

elemaniaq
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
183 views28 pages

Data Structures and Algorithms: Asymptotic Notations

The document discusses algorithm analysis. It explains that analysis determines an algorithm's properties like time and space complexity. Asymptotic notations like Big-O, Omega, and Theta are used to analyze algorithms since they ignore constants and lower order terms and focus on relative growth rates. Common functions like logarithms and polynomials are classified based on their growth rates. Rules for combining complexities using sums and products are presented. Examples demonstrate how to determine the complexity class of algorithms.

Uploaded by

elemaniaq
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

Bilgisayar Mühendisliği Bölümü

DATA STRUCTURES AND


ALGORITHMS

Asymptotic Notations

GIT – Computer Engineering Department


The Process of Algorithm Development
 Design
– divide&conquer, greedy, dynamic programming
 Validation
– check whether it is correct
 Analysis
– determine the properties of algorithm
 Implementation
 Testing
– check whether it works for all possible cases

GIT – Computer Engineering Department 2


Analysis of Algorithm
 Analysis investigates
– What are the properties of the algorithm?
• in terms of time and space
– How good is the algorithm ?
• according to its properties
– How it compares with others?
• not always exact
– Is it the best that can be done?
• difficult !

GIT – Computer Engineering Department 3


Mathematical Background
 Assume the running times of two algorthms are
calculated:

For input size N


Running time of Algorithm A = TA(N) = 1000 N
Running time of Algorithm B = TB(N) = N2

Which one is faster ?

GIT – Computer Engineering Department 4


Mathematical Background
If the unit of running time of algorithms A and B is µsec
N TA TB
10 10-2 sec 10-4 sec
100 10-1 sec 10-2 sec
1000 1 sec 1 sec
10000 10 sec 100 sec
100000 100 sec 10000 sec

So which algorithm is faster ?


GIT – Computer Engineering Department 5
Mathematical Background

If N<1000 TA(N) > TB(N)


o/w TB(N) > TA(N)

Compare their relative growth ?


GIT – Computer Engineering Department 6
Mathematical Background
 Is it always possible to have definite results?
NO !
The running times of algorithms can change
because of the platform, the properties of the
computer, etc.

We use asymptotic notations (O, Ω, θ, o)


• compare relative growth
– No constants
– No lower order terms
• compare only algorithms

GIT – Computer Engineering Department 7


Big Oh Notation (O)
Provides an “upper bound” for the function f
 Definition :
T(N) = O (f(N)) if there are positive constants c
and n0 such that
T(N) ≤ cf(N) when N ≥ n0

– T(N) grows no faster than f(N)


– growth rate of T(N) is less than or equal to growth rate
of f(N) for large N
– f(N) is an upper bound on T(N)
• not fully correct !

GIT – Computer Engineering Department 8


Big Oh Notation (O)
 Analysis of Algorithm A

TA (N) = 1000 N = O(N)


1000 N ≤ cN ∀N ≥ n0

if c= 2000 and n0 = 1 for all N

TA (N) = 1000 N = O(N) is right

GIT – Computer Engineering Department 9


Examples

 7n+5 = O(n)
for c=8 and n0 =5
7n+5 ≤ 8n n>5 = n0

 7n+5 = O(n2)
for c=7 and n0=2
7n+5 ≤ 7n2 n≥n0

 7n2+3n = O(n) ?

GIT – Computer Engineering Department 10


Advantages of O Notation
 It is possible to compare of two algorithms
with running times
 Constants can be ignored.
– Units are not important
O(7n2) = O(n2)
 Lower order terms are ignored
– Compare relative growth only
O(n3+7n2+3) = O(n3)

GIT – Computer Engineering Department 11


Big Oh Notation (O)

Running Times of Algorithm A and B

TA(N) = 1000 N = O(N)


TB(N) = N2 = O(N2)

A is asymptotically faster than B !

GIT – Computer Engineering Department 12


Omega Notation (Ω)
 Definition :
T(N) = Ω (f(N)) if there are positive
constants c and n0 such that T(N) ≥ c f(N)
when N≥ n0

– T(N) grows no slower than f(N)


– growth rate of T(N) is greater than or equal to
growth rate of f(N) for large N
– f(N) is a lower bound on T(N)
• not fully correct !

GIT – Computer Engineering Department 13


Omega Notation
Example:
 n 1/2 = Ω (lg n) .
for c = 1 and n0 = 16
Let n > 16
c*(lg n) ≤ n 1/2

GIT – Computer Engineering Department 14


Omega Notation
 Theorem:
f(N) = O(g(n)) <=> g(n) = Ω(f(N))
Proof:
f(N) ≤ c1g(n) <=> g(n) ≥ c2f(N)
divide the left side with c1

1/c1f(N) ≤ g(n) <=> g(n) ≥ c2f(N)


if we choose c2 as 1/c1 then theorem is right.

GIT – Computer Engineering Department 15


Omega Notation
 7n2 + 3n + 5 = O(n4)
 7n2 + 3n + 5 = O(n3)
 7n2 + 3n + 5 = O(n2)
 7n2 + 3n + 5 = Ω(n2)
 7n2 + 3n + 5 = Ω(n)
 7n2 + 3n + 5 = Ω(1)

n2 and 7n2 + 3n + 5 grows at the same rate


7n2 + 3n + 5 = O(n2) = Ω(n2) = θ (n2)

GIT – Computer Engineering Department 16


Theta Notation (θ)
 Definition :
T(N) = θ (h(N)) if and only if
T(N) = O(h(N)) and T(N) = Ω(h(N))

– T(N) grows as fast as h(N)


– growth rate of T(N) and h(N) are equal for
large N
– h(N) is a tight bound on T(N)
• not fully correct !

GIT – Computer Engineering Department 17


Theta Notation (θ)
 Example :
T(N) = 3N2

T(N) = O(N4)
T(N) = O(N3)
T(N) = θ(N2)  best

GIT – Computer Engineering Department 18


Little O Notation (o)
 Definition :
T(N) = o(p(N)) if
T(N) = O(p(N)) and T(N)≠θ(p(N))

– p(N) grows strictly faster than T(N)


– growth rate of T(N) is less than the growth
rate of p(N) for large N
– p(N) is an upperbound on T(N) (but not tight)
• not fully correct !

GIT – Computer Engineering Department 19


Little O Notation (o)
 Example :

T(N) = 3N2

T(N) = o(N4)
T(N) = o(N3)
T(N) = θ(N2)

GIT – Computer Engineering Department 20


RULES
 RULE 1:

if T1(N) = O(f(N)) and T2(N) = O(g(N)) then

a) T1(N) + T2(N) = max (O(f(N)), O(g(N)))


b) T1(N) * T2(N) = O(f(N) * g(N))

You can prove these ?


Is it true for θ notation ?
What about Ω notation?

GIT – Computer Engineering Department 21


RULES
 RULE 2:

if T(N) is a polynomial of degree k

T(N) = akNk + ak-1Nk-1 + … + a1N + a0


then
T(N) = θ(Nk)

GIT – Computer Engineering Department 22


RULES
 RULE 3:

logk N = o(N) for any constant k

logarithm grows very slowly !

GIT – Computer Engineering Department 23


Some Common Functions
 c = o (log N) => c=O(log N) but
c≠Ω(logN)
 log N = o(log2 N)
 log2 N = o(N)
 N = o(N log N)
 N = o (N2)
 N2 = o (N3)
 N3 = o (2N)

GIT – Computer Engineering Department 24


Example
 T(N) = 4N2

• T(N) = O(2N2)
correct but bad style
T(N) = O(N2)
drop the constants
• T(N) = O(N2+N)
correct but bad style
T(N) = O(N2)
ignore low order terms

GIT – Computer Engineering Department 25


Another Way to Compute Growth Rates

f(N )
lim =0 ⇒ f ( N ) = o( g ( N ))
N →∞ g ( N )

=c≠0 ⇒ f ( N ) = θ ( g ( N ))
=∞ ⇒ g ( N ) = o( f ( N ))
= oscilate ⇒ there is no relation

GIT – Computer Engineering Department 26


Example

 f(N) = 7N2 g(N) = N2 + N

GIT – Computer Engineering Department 27


Example

 f(N) = N logN g(N) = N1.5

compare logN with N0.5


compare log2N with N
compare log2N with o(N)

N logN = o(N1.5)

GIT – Computer Engineering Department 28

You might also like