Scribd
Upload
25 views29 pages

Order of Growth

The document discusses the fundamentals of algorithmic efficiency, focusing on the order of growth which approximates the time required for an algorithm as input size increases. It categorizes growth orders into constant, logarithmic, linear, quadratic, and exponential, providing examples for each. Additionally, it covers asymptotic notations (O, Ω, θ) used to compare the growth rates of algorithms.

Uploaded by

slokesh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
25 views29 pages

Order of Growth

The document discusses the fundamentals of algorithmic efficiency, focusing on the order of growth which approximates the time required for an algorithm as input size increases. It categorizes growth orders into constant, logarithmic, linear, quadratic, and exponential, providing examples for each. Additionally, it covers asymptotic notations (O, Ω, θ) used to compare the growth rates of algorithms.

Uploaded by

slokesh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 29

4.

Fundamentals of the Analysis of Algorithmic Efficiency


Orders of growth

• The order of growth of an algorithm is an approximation of the time


required to run a computer program as the input size increases.

• The order of growth ignores the constant factor needed for fixed
operations and focuses instead on the operations that increase
proportional to input size.
Orders of growth
Order Big-Theta Example

Constant Θ(1) Indexing an item in a list

Logarithmic Θ(lg N) Repeatedly halving a number

Linear Θ(n) Summing a list

Quadratic Θ(n^2) Summing each pair of numbers in a


list

Exponential Θ(2^n) Visiting each node in a binary tree


Orders of growth (Constant)
List size Steps
• When an algorithm has a constant 1 1
order of growth, it means that it 10 1
always takes a fixed number of steps, 100 1
no matter how large the input size 1000 1
increases.

• As an example, consider accessing the


first element of a list:

first_post = posts[0]
Orders of growth (Logarithmic)
List size Steps
• When analgorithm has a logarithmic order of 1 1
growth, it increases proportionally to the 10 4
logarithm of the input size. 100 7
1000 10
• The binary search algorithm is an example of
an algorithm that runs in logarithmic time.
Orders of growth (Linear time)
List size Steps
• When an algorithm has a linear order of growth, its
1 1
number of steps increases in direct proportion to the
input size. 10 10

• The aptly-named linear search algorithm runs in linear 100 100


time. 1000 1000
Orders of growth (Quadratic time)
• When an algorithm has a quadratic order of growth, its steps List size Steps
increase in proportion to the input size squared. 1 1
10 45
• Several list sorting algorithms run in quadratic time, like
selection sort. That algorithm starts from the front of the list, 100 4950
then keeps finding the next smallest value in the list and 1000 499500
swapping it with the current value.
Orders of growth (Exponential
time) Digits Steps
• When an algorithm has a superpolynomial order of growth,
1 10
its number of steps increases faster than a polynomial
2 100
function of the input size.
3 1000
• An algorithm often requires superpolynomial time when it
4 10000
must look at every permutation of values. For example,
consider an algorithm that generates all possible numerical 5 100000
passwords for a given password length.
Orders of growth (All together now)
Engineered for

5. Asymptotic Notations and its Properties


Tomorrow

•
To compare two algorithms with running times f(n) and g(n), we need a rough measure that
characterizes how fast each function grows.
•
Hint: use rate of growth
•
Compare functions in the limit, that is, asymptotically!
(i.e., for large values of n)
Engineered for
Tomorrow

ASYMPTOTIC NOTATION
•
O notation: asymptotic “less than”:


f(n)=O(g(n)) implies: f(n) “≤” g(n)

•
Ω notation: asymptotic “greater than”:


f(n)= Ω (g(n)) implies: f(n) “≥” g(n)

•
θ notation: asymptotic “equality”:


f(n)= θ (g(n)) implies: f(n) “=” g(n)

19
Engineered for
Tomorrow

ASYMPTOTIC NOTATIONS
O-notation

12/8/13
Engineered for
Tomorrow

Examples

f(n)=3n2+n
= 3n2+n2 n2 ≤ cn2 ; c ≥ 1 ; c = 1 and n0= 1
=4n2


f(n)<=c*g(n)

3n2+n<=4n2=o(n2)

Where n>=n0 and n=1


Engineered for
Tomorrow

ASYMPTOTIC NOTATIONS (CONT.)


Ω - notation
•

Ω(g(n)) is the set of functions with


larger or same order of growth as
g(n)

12/8/13
Engineered for
Tomorrow

Examples

f(n)=3n2+n
= 3n2+n
=3n2


f(n)>=c*g(n)

3n2+n>=3n2= Ω(n2)

Where n<=n0 and n=1


Engineered for
Tomorrow

ASYMPTOTIC NOTATIONS (CONT.)


•
θ -notation

θ(g(n)) is the set of functions with


the same order of growth as g(n)
Engineered for
Tomorrow

Examples
C2g(n)<=f(n)<=c1g(n) for all n>=n0
3n2+n<=f(n)<=3n2+n2
3n2<=f(n)<=4n2
Where c2=3, c1=4 and n=1
Therefore, 3n2+n ∈ θ(n2)
Establishing order of growth using limits
0 order of growth of T(n) < order of growth of g(n)

c > 0 order of growth of T(n) = order of growth of g(n)


lim T(n)/g(n)
n→∞ =
∞ order of growth of T(n) > order of growth of g(n)

Examples:
• 10n vs. n2

• n(n+1)/2 vs. n2
L-Hopital’s rule and Stirling’s formula
L’Hôpital’s rule: If limn f(n) = limn g(n) =  and
the derivatives f´, g´ exist, then

1/2 f(n) n f ´(n)


Stirling’s formula: n!  (2n)
lim (n/e) lim
n
g(n) = n
g ´(n)

Example: log n vs. n

Example: 2n vs. n!
Orders of growth of some important functions
• All logarithmic functions loga n belong to the same class
(log n) no matter what the logarithm’s base a > 1 is

• All polynomials of the same degree k belong to the same class: aknk + ak-1nk-1 + … +
a0  (nk)
• Exponential functions an have different orders of growth for different a’s

• order log n < order n (>0) < order an < order n! < order nn
Basic asymptotic efficiency classes
1 constant

log n logarithmic

n linear

n log n n-log-n

n2 quadratic

n3 cubic

2n exponential

n! factorial

You might also like