Design and Analysis Algorithm

Drs. Achmad Ridok M.Kom

Fitra A. Bachtiar, S.T., M. Eng
Imam Cholissodin, S.Si., M.Kom
Aryo Pinandito, MT

Pertemuan 04
Contents
2
Asymptotic Notation
XE3024uAZ
3
Contents
Asymptotic Notations:
O (big oh)
O (big omega)
O (big theta)
Basic Efficiency Classes
XE3024uAZ
4
In the following discussion
t(n) & g(n): any nonnegative functions defined
on the set of natural numbers

t(n) an algorithms running time
Usually indicated by its basic operation count C(n)
g(n) some simple function to compare the
count with
XE3024uAZ
5
O(g(n)): Informally
O(g(n)) is a set of all functions with a smaller or
same order of growth as g(n)
Examples:
n e O(n
2
); 100n + 5 e O(n
2
)
n (n-1) e O(n
2
)
n
3
e O(n
2
); 0.0001 n
3
e O(n
2
); n
4
+n+1 e O(n
2
)

XE3024uAZ
6
O(g(n)): Informally
O(g(n)) is a set of all functions with a larger or
same order of growth as g(n)
Examples:
n
3
e O(n
2
)
n (n-1) e O(n
2
)
100n + 5 e O(n
2
)

XE3024uAZ
7
O(g(n)): Informally
O(g(n)) is a set of all functions with a same
order of growth as g(n)
Examples:
an
2
+bn+c; a>0 e O(n
2
); n
2
+sin n e O(n
2
)
n (n-1) e O(n
2
); n
2
+log n e O(n
2
)
100n + 5 e O(n
2
); n
3
e O(n
2
)
=
XE3024uAZ
8
O-notation: Formally
DEF1: A function t(n) is said to be in O(g(n)),
denoted t(n) e O(g(n)), if t(n) is bounded above
by some constant multiple of g(n) for all large n

i.e. there exist some positive constant c and
some nonnegative integer n
0
, such that
t(n) cg(n) for all n n
0
XE3024uAZ
9
t(n) e O(g(n)): Illustration
n
n
0
d
o
e
s
n
'
t

m
a
t
t
e
r
t(n)
cg(n)
XE3024uAZ
10
Proving Example: 100n + 5 e
O(n
2
)
Remember DEF1: find c and n
0
, such that t(n)
cg(n) for all n n
0


100n + 5 100n + n (for all n 5) = 101n
101n
2
c=101, n
0
=5
100n + 5 100n + 5n (for all n 1) = 105n
105n
2
c=105, n
0
=1


XE3024uAZ
11
O-notation: Formally
DEF2: A function t(n) is said to be in O(g(n)),
denoted t(n) e O(g(n)), if t(n) is bounded below
by some constant multiple of g(n) for all large n

i.e. there exist some positive constant c and
some nonnegative integer n
0
, such that
t(n) cg(n) for all n n
0
XE3024uAZ
12
t(n) e O(g(n)): Illustration
n
n
0
d
o
e
s
n
'
t

m
a
t
t
e
r
t(n)
cg(n)
XE3024uAZ
13
Proving Example: n
3
e O(n
2
)
Remember DEF2: find c and n
0
, such that t(n)
cg(n) for all n n
0


n
3
n
2
(for all n 0) c=1, n
0
=0


XE3024uAZ
14
O-notation: Formally
DEF3: A function t(n) is said to be in O(g(n)),
denoted t(n) e O(g(n)), if t(n) is bounded both
above and below by some constant multiple of
g(n) for all large n

i.e there exist some positive constant c
1
and c
2

and some nonnegative integer n
0
, such that
c
2
g(n) t(n) c
1
g(n) for all n n
0
XE3024uAZ
15
t(n) e O(g(n)): Illustration
n
n
0
d
o
e
s
n
'
t

m
a
t
t
e
r
t(n)
c
1
g(n)
c
2
g(n)
XE3024uAZ
16
Proving Example: n(n-1)

e
O(n
2
)
Remember DEF3: find c
1
and c
2
and some
nonnegative integer n
0
, such that
c
2
g(n) t(n) c
1
g(n) for all n n
0


The upper bound: n(n-1) = n
2
n n
2
(for all n 0)
The lower bound: n(n-1) = n
2
n n
2

- n n (for all n 2) = n
2

c
1
= , c
2
= , n
0
= 2
XE3024uAZ
17
Useful Property
Theorem:
If t
1
(n) e O(g
1
(n)) and t
2
(n) e O(g
2
(n)), then t
1
(n)
+ t
2
(n) e O(max{g
1
(n), g
2
(n)})

The analogous assertions are true for the O and
O notations as well
XE3024uAZ
18
Example
Alg to check whether an array has identical
elements:
1. Sort the array
2. Scan the sorted array to check its consecutive
elements for equality
(1) = n(n-1) comparison O(n
2
)
(2) = n-1 comparison O(n)
The efficiency of (1)+(2) = O(max{n
2
,n}) =
O(n
2
)
XE3024uAZ
19
Using Limits for Comparing
OoG
A convenient method for comparing order of
growth of two specific functions
Three principal cases:



The first two cases t(n) e O(g(n)); the last two
cases t(n) e O(g(n)); the second case alone
t(n) e O(g(n))

g(n) OoG than larger a has that t(n) implies
g(n) as OoG same the has that t(n) implies c
g(n) OoG than smaller a has that t(n) implies 0
) (
) (
lim
n g
n t
n
XE3024uAZ
20
Limit-based: why convenient?
It can take advantage of the powerful calculus
techniques developed for computing limits, such
as
LHopitals rule

Stirlings formula
) ( '
) ( '
lim
) (
) (
lim
n g
n t
n g
n t
n n
=
( ) n of value large for 2 !
n
e
n
n n t ~
XE3024uAZ
21
Example (1)
Compare OoG of n(n-1) and n
2
.


The limit = c n(n-1) e O (n
2
)

Compare OoG of log
2
n and n


The limit = 0 log
2
n has smaller order of n
( )
2
1
1 lim
2
1
lim
2
1 ) 1 (
lim
1
2
2
2
2
1
= =

n
n n n
n
n n
n
n n
0 lim log 2
) (log
lim
)' (
)' (log
lim
log
lim
2
2
1
1
2 2 2
= = = =

n
n
e
e
n
n
n
n
n
n
n
n n n
XE3024uAZ
22
Example (2)
Compare OoG of n! and 2
n
.



The limit = n! e O(2
n
)

( )
=
|
.
|

\
|
= = =

n
n
n n
n
n
n
n
e
n
n
n
n
e
n
n
e
n
n
n n
2
2 lim
2
2 lim
2
2
lim
2
!
lim t t
t
Asymptotic Notation
Think of n as the number of records we wish to
sort with an algorithm that takes f(n) to run. How
long will it take to sort n records?
What if n is big?
We are interested in the range of a function as n
gets large.
Will f stay bounded?
Will f grow linearly?
Will f grow exponentially?
Our goal is to find out just how fast f grows with
respect to n.
Asymptotic Notation






Misalkan:
T(n) = 5n
2
+ 6n + 25
T(n) proporsional untuk ordo n
2
untuk data yang sangat
besar.

24
Memperkirakan formula untuk run-time
Indikasi kinerja algoritma
(untuk jumlah data yang sangat besar)

Asymptotic Notation
Indikator efisiensi algoritma bedasar pada OoG
pada basic operation suatu algoritma.
Penggunaan notasi sebagai pembanding
urutan OoG:
O (big oh)
(big omega)
(big theta)
t(n) : algoritma running time (diindikasikan
dengan basic operation count (C(n))
g(n) : simple function to compare the count
25
Classifying functions by their
Asymptotic Growth Rates (1/2)
asymptotic growth rate, asymptotic order, or
order of functions
Comparing and classifying functions that ignores
constant factors and small inputs.

O(g(n)), Big-Oh of g of n, the Asymptotic Upper
Bound;
O(g(n)), Omega of g of n, the Asymptotic Lower
Bound.
O(g(n)), Theta of g of n, the Asymptotic Tight
Bound; and

Example
Example: f(n) = n
2
- 5n + 13.
The constant 13 doesn't change as n grows,
so it is not crucial. The low order term, -5n,
doesn't have much effect on f compared to
the quadratic term, n
2
.
We will show that f(n) = O(n
2
) .

Q: What does it mean to say f(n) = O(g(n)) ?
A: Intuitively, it means that function f is the
same order of magnitude as g.
Example (cont.)
Q: What does it mean to say f
1
(n) = O(1)?
A: f
1
(n) = O(1) means after a few n, f
1
is
bounded above & below by a constant.
Q: What does it mean to say f
2
(n) = O(n log n)?
A: f
2
(n) = O(n log n) means that after a few n, f
2

is bounded above and below by a constant
times n log n. In other words, f
2
is the same
order of magnitude as n log n.
More generally, f(n) = O(g(n)) means that f(n) is
a member of O(g(n)) where O(g(n)) is a set of
functions of the same order of magnitude.
Big-Oh
The O symbol was introduced in 1927 to
indicate relative growth of two functions based
on asymptotic behavior of the functions now
used to classify functions and families of
functions
Upper Bound Notation
We say Insertion Sorts run time is O(n
2
)
Properly we should say run time is in O(n
2
)
Read O as Big-O (youll also hear it as order)

In general a function
f(n) is O(g(n)) if - positive constants c and n
0
such
that f(n) s c g(n) n > n
0

e.g. if f(n)=1000n and g(n)=n
2
, n
0
> 1000 and c
= 1 then f(n
0
) < 1.g(n
0
) and we say that f(n) =
O(g(n))

Asymptotic Upper Bound
f(n)
g(n)
c g(n)
f(n) s c g(n) for all n > n
0

g(n) is called an
asymptotic upper bound of f(n).
We write f(n)=O(g(n))
It reads f(n) is big oh of g(n).

n
0
Big-Oh, the Asymptotic Upper Bound
This is the most popular notation for run time
since we're usually looking for worst case time.
If Running Time of Algorithm X is O(n
2
) , then
for any input the running time of algorithm X is
at most a quadratic function, for sufficiently
large n.

e.g. 2n
2
= O(n
3
) .
From the definition using c = 1 and n
0
= 2. O(n
2
)
is tighter than O(n
3
).
6
g(n)
f(n)
for all n>6, g(n) > 1 f(n).
Thus the function f is in the
big-O of g.
that is, f(n) in O(g(n)).
Example 1
g(n)
f(n)
5
There exists a n
0
=5 s.t. for all
n>n
0
, f(n) < 1 g(n).
Thus, f(n) is in O(g(n)).
Example 2
There exists a n
0
=5, c=3.5, s.t.
for all n>n
0
, f(n) < c h(n).
Thus, f(n) is in O(h(n)).
5
h(n)
f(n)
3.5 h(n)
Example 3
Example of Asymptotic Upper Bound
f(n)=3n
2
+5
g(n)=n
2
4g(n)=4n
2
4 g(n) = 4n
2

= 3n
2
+ n
2

> 3n
2
+ 9 for all n > 3
> 3n
2
+ 5
= f(n)
Thus, f(n)=O(g(n)).

3

Exercise on O-notation
Show that 3n
2
+2n+5 = O(n
2
)

10 n
2
= 3n
2
+ 2n
2
+ 5n
2
> 3n
2
+ 2n + 5 for n > 1

c = 10, n
0
= 1
Classification of Function : BIG O (1/2)
A function f(n) is said to be of at most logarithmic
growth if f(n) = O(log n)
A function f(n) is said to be of at most quadratic
growth if f(n) = O(n
2
)
A function f(n) is said to be of at most polynomial
growth if f(n) = O(n
k
), for some natural number k > 1
A function f(n) is said to be of at most exponential
growth if there is a constant c, such that f(n) = O(c
n
),
and c > 1
A function f(n) is said to be of at most factorial growth
if f(n) = O(n!).
Classification of Function : BIG O (2/2)
A function f(n) is said to have constant running
time if the size of the input n has no effect on
the running time of the algorithm (e.g.,
assignment of a value to a variable). The
equation for this algorithm is f(n) = c
Other logarithmic classifications:
f(n) = O(n log n)
f(n) = O(log log n)
Review Tugas n!
Menghitung kompleksitas pada Faktorial
Function Faktorial (input n : integer) integer
{menghasilkan nilai n!, n 0}
Algoritma
If n=0 then
Return 1
Else
Return n*faktorial (n-1)
Endif
Kompleksitas waktu :
untuk kasus basis, tidak ada operasi perkalian (0)
untuk kasus rekurens, kompleksitas waktu diukur dari jumlah perkalian (1)
ditambah kompleksitas waktu untuk faktorial (n-1)


Review Tugas n! (Lanjutan)
Kompleksitas waktu n! :
T(n)=1+T(n-1)
=T(n)=1+1+T(n-2)=2+T(n-2)
=T(n)=2+1+T(n-3)=3+T(n-3)
=
=
= n+T(0)
= n + 0
Jadi T(n) = n
T(n) O(n)

Lower Bound Notation
We say InsertionSorts run time is O(n)
In general a function
f(n) is O(g(n)) if - positive constants c and n
0
such
that 0 s cg(n) s f(n) n > n
0

Proof:
Suppose run time is an + b
Assume a and b are positive (what if b is negative?)
an s an + b
Big O Asymptotic Lower Bound
f(n)
c g(n)
f(n) > c g(n) for all n > n
0

g(n) is called an
asymptotic lower bound of f(n).
We write f(n)=O(g(n))
It reads f(n) is omega of g(n).

n
0
Example of Asymptotic Lower Bound
f(n)=n
2
/2-7
c g(n)=n
2
/4

g(n)=n
2
g(n)/4 = n
2
/4
= n
2
/2 n
2
/4
s n
2
/2 9 for all n > 6
< n
2
/2 7
Thus, f(n)= O(g(n)).

6

g(n)=n
2
Example: Big Omega
Example: n
1/2
= O( log n) .
Use the definition with c = 1 and n
0
= 16.

Checks OK.
Let n 16 : n
1/2
(1) log n
if and only if n = ( log n )
2
by squaring both sides.
This is an example of polynomial vs. log.
Big Theta Notation
Definition: Two functions f and g are said to be
of equal growth, f = Big Theta(g) if and only if
both
f=O(g) and g = O(f).

Definition: f(n) = O(g(n)) means - positive
constants c
1
, c
2
, and n
0
such that
c
1
g(n) s f(n) s c
2
g(n) n > n
0

If f(n) = O(g(n)) and f(n) = O(g(n)) then f(n) = O(g(n))

(e.g. f(n) = n
2
and g(n) = 2n
2
)
Theta, the Asymptotic Tight Bound
Theta means that f is bounded above and below
by g; BigTheta implies the "best fit".
f(n) does not have to be linear itself in order to
be of linear growth; it just has to be between two
linear functions,
Asymptotically Tight Bound
f(n)
c
1
g(n)
f(n) = O(g(n)) and f(n) = O(g(n))
g(n) is called an
asymptotically tight bound of f(n).
We write f(n)=O(g(n))
It reads f(n) is theta of g(n).

n
0
c
2
g(n)
Other Asymptotic Notations
A function f(n) is o(g(n)) if - positive constants c
and n
0
such that
f(n) < c g(n) n > n
0

A function f(n) is e(g(n)) if - positive constants c
and n
0
such that
c g(n) < f(n) n > n
0

Intuitively,

o() is like <
O() is like s

e() is like >
O() is like >

O() is like =
Examples
1. 2n
3
+ 3n
2
+ n = 2n
3
+ 3n
2
+ O(n)
= 2n
3
+ O( n
2
+ n) = 2n
3
+ O( n
2
)
= O(n
3
) = O(n
4
)

2. 2n
3
+ 3n
2
+ n = 2n
3
+ 3n
2
+ O(n)
= 2n
3
+ O(n
2
+ n)
= 2n
3
+ O(n
2
) = O(n
3
)
Example (cont.)
n
3
= 50
3
* 729 3
n
= 3
50
* 729
n = n = log
3
(729 * 3
50
)
n = n = log
3
(729) + log
3
3
50

n = 50 * 9 n = 6 + log
3
3
50

n = 50 * 9 = 450 n = 6 + 50 = 56

Improvement: problem size increased by 9 times for n
3

algorithm but only a slight improvement in problem size
(+6) for exponential algorithm.
3 3
729 * 50
More Examples
(a) 0.5n
2
- 5n + 2 = ( n
2
).
Let c = 0.25 and n
0
= 25.
0.5 n
2
- 5n + 2 = 0.25( n
2
) for all n = 25

(b) 0.5 n
2
- 5n + 2 = O( n
2
).
Let c = 0.5 and n
0
= 1.
0.5( n
2
) = 0.5 n
2
- 5n + 2 for all n = 1

(c) 0.5 n
2
- 5n + 2 = ( n
2
)
from (a) and (b) above.
Use n
0
= 25, c
1
= 0.25, c
2
= 0.5 in the definition.
More Examples
(d) 6 * 2
n
+ n
2
= O(2
n
).
Let c = 7 and n
0
= 4.
Note that 2
n
= n
2
for n = 4. Not a tight upper bound, but
it's true.

(e) 10 n
2
+ 2 = O(n
4
).
There's nothing wrong with this, but usually we try to get
the closest g(n). Better is to use O(n
2
).
Practical Complexity t < 250

0
250
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
f(n) = n
f(n) = log(n)
f(n) = n log(n)
f(n) = n^2
f(n) = n^3
f(n) = 2^n
Practical Complexity t < 500

0
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
f(n) = n
f(n) = log(n)
f(n) = n log(n)
f(n) = n^2
f(n) = n^3
f(n) = 2^n
Practical Complexity t < 1000

0
1000
1 3 5 7 9 11 13 15 17 19
f(n) = n
f(n) = log(n)
f(n) = n log(n)
f(n) = n^2
f(n) = n^3
f(n) = 2^n
Practical Complexity t < 5000

0
1000
2000
3000
4000
5000
1 3 5 7 9 11 13 15 17 19
f(n) = n
f(n) = log(n)
f(n) = n log(n)
f(n) = n^2
f(n) = n^3
f(n) = 2^n
XE3024uAZ
58
Tugas (1)
1. True or false:
a. n(n+1)/2 e O(n
3
)
b. n(n+1)/2 e O(n
2
)
c. n(n+1)/2 e O(n
3
)
d. n(n+1)/2 e O(n)
2. Indicate the class O(g(n)):
a. (n
2
+1)
10
b. (10n
2
+7n+3)

c. 2n log (n+2)
2
+(n+2)
2
log (n/2)
Tugas 1 : O-notation
3. Tentukan OoG dari masing-masing soal
a. f1(n) = 10 n + 25 n
2
b. f2(n) = 20 n log n + 5 n
c. f3(n) = 12 n log n + 0.05 n
2
d. f4(n) = n
1/2
+ 3 n log n

4.. True/false ?
(a) 0.25n
2
- 5n + 2 = ( n
2
).
(b) 0.25n
2
- 5n + 2 = O( n
2
).
(c) 0.25n
2
- 5n + 2 = ( n
2
).

O(n
2
)
O(n log n)
O(n
2
)
O(n log n)

Tugas Kelompok
1. Kerjakan soal di hal 29 no 2.2-1 sd. 2.2-4
2. Tugas 2 s.d Tugas 6 di slide ini
3. Pengumpulan :
1. Tulis dikertas folio bergaris
2. Dikumpulkan minggu depan di kelas
3. KODE TUGAS :
DAA_A_1_1 (MT DAA, kelas A, Kelompok1, tugas
ke-1)
DAA_D_5_1



Tugas 2

Tugas 3

Tugas 4

XE3024uAZ
64
Tugas 5
5. Prove that every polynomial
p(n) = a
k
n
k
+ a
k-1
n
k-1
+ + a
0
with a
k
> 0
belongs to O(n
k
)

6. Prove that exponential functions a
n
have
different orders of growth for different values
of base a > 0

Tugas 6: Examples (cont.)
7. Suppose a program P is O(n
3
), and a program Q
is O(3
n
), and that currently both can solve
problems of size 50 in 1 hour. If the programs are
run on another system that executes exactly 729
times as fast as the original system, what size
problems will they be able to solve?
Click to edit subtitle style