Algorithm Efficiency Analysis

Data Structures and Algorithm

Objectives
At the end of the class, students are
expected to be able to do the following:

n Know how to measure algorithm

efficiency.
n Know the meaning of big O notation and
determine the notation in algorithm
analysis.
Introduction

Algorithm analysis
Study the efficiency of algorithms when
the input size grow based on the number
of steps, the amount of computer time and
space.
Analysis of algorithms
n Isa major field that provides tools for evaluating
the efficiency of different solutions
What is an efficient algorithm?
n Faster is better (Time)
 How do you measure time? Wall clock? Computer
clock?
n Less space demanding is better
 But if you need to get data out of main memory it
takes time
Analysis of algorithms
Algorithm analysis should be independent of :
n Specific implementations and coding tricks
(programming language, control statements –
Pascal, C, C++, Java)
n Specific Computers (hw chip, OS, clock speed)
n Particular sets of data (string, int, float)

But size of data should matter

Worse Case/Best Case/Average Case
For a particular problem size, we may be interested in:
n Worst-case efficiency: Longest running time for any
input of size n
A determination of the maximum amount of time that an
algorithm requires to solve problems of size n
n Best-case efficiency: Shortest running time for any
input of size n
A determination of the minimum amount of time that an
algorithm requires to solve problems of size n
n Average-case efficiency: Average running time for all
inputs of size n
A determination of the average amount of time that an
algorithm requires to solve problems of size n
Example
Algorithm: sequential search of n elements
n Best-case: We get lucky and find the
target in the first place we look. C(n) = 1
n Worst-case: We look at every element
before finding (or not finding) the target.
C(n) = n
n Average-case: Depends on the probability
(p) that the target will be found.
Introduction
Complexity time can be represented by
Big ‘O’ notation
n Big ‘O’ notation is denoted as : O(acc)
whereby:
O - order
acc - class of algorithm complexity
n Big O Notation Example:
O(1),O(logxn),O(n),O(n logxn),O(n2)
Big O Notation
Notation being used to show the complexity time of
algorithms.
Notation Execution time / number of step
Constant function, independent of input size, n
O(1) Example: Finding the first element of a list.
Problem complexity increases slowly as the problem size
O(logxn) increases.
Squaring the problem size only doubles the time.
Characteristic: Solve a problem by splitting into constant
fractions of the problem (e.g., throw away ½ at each step)
Problem complexity increases linearly with
O(n) the size of the input, n
Example: counting the elements in a list.
Big O Notation
Log-linear increase - Problem complexity increases a little
O(n logxn) faster than n
Characteristic: Divide problem into sub problems that are
solved the same way.
Example: Merge sort
Quadratic increase.
O(n2) Problem complexity increases fairly fast, but
still manageable
Characteristic: Two nested loops of size n
Cubic increase.
O(n3)
Practical for small input size, n.
Exponential increase - Increase too rapidly to be practical
O(2n) Problem complexity increases very fast
Generally unmanageable for any meaningful n
Example: Find all subsets of a set of n elements
Order-of-Magnitude Analysis and Big O
Notation

A comparison of growth-rate functions: (a) in tabular form

11
Order-of-Magnitude Analysis and
Big O Notation

A comparison of growth-rate functions: in graphical form

12
Order of increasing
complexity
O(1) < O(logxn) < O(n) < O(n log2n) < O(n2) < O(n3) < O(2n)

Notasi n=8 n = 16 n = 32
O(log2n) 3 4 5
O(n) 8 16 32
O(n log2n) 24 64 160
O(n2) 64 256 1024
O(n3) 512 4096 32768
O(2n) 256 65536 4294967296
Big O Notation
n Example of algorithm (only for cout operation):

notation code

int counter = 1;
O(1)
cout << "Arahan cout kali ke " << counter << "\n";

int counter = 1; int i = 0;

O(logxn)
for (i = x; i <= n; i = i * x) { // x must be > than 1
cout << "Arahan cout kali ke " << counter << "\n";
counter++;
}
Big O Notation

int counter = 1; int i = 0;

O(n)
for (i = 1; i <= n; i++) {
cout << "Arahan cout kali ke " << counter << "\n";
counter++;
}
int counter = 1; int i = 0; int j = 1;
O(n
logxn) for (i = x; i <= n; i = i * x) { // x must be > than 1
while (j <= n) {
cout << "Arahan cout kali ke " << counter << "\n";
counter++; j++;
}
}
Big O Notation

int counter = 1;
O(n2)
int i = 0;
int j = 0;

for (i = 1; i <= n; i++) {

for (j = 1; j <= n; j++) {
cout << "Arahan cout kali ke " <<
counter << "\n";
counter++;
}
}
Big O Notation

int counter = 1;
O(n3) int i = 0;
int j = 0;
int k = 0;

for (i = 1; i <= n; i++) {

for (j = 1; j <= n; j++) {
for (j = 1; j <= n; j++) {
cout << "Arahan cout kali ke " <<
counter << "\n";
counter++;
}
}
}
Big O Notation

int counter = 1;
O(2n) int i = 1;
int j = 1;

while (i <= n) {
j = j * 2;
i++;
}

for (i = 1; i <= j; i++) {

cout << "Arahan cout kali ke " << counter
<< "\n";
counter++;
}
Determine the complexity time
of algorithm
Can be determined
n theoretically – by calculation
n practically – by experiment or
implementation
Determine the complexity time
of algorithm - practically
 Implement the algorithms in any programming
language and run the programs
 Depend on the compiler, computer, data input
and programming style.
Determine the complexity time
of algorithm - theoretically
n Thecomplexity time is related to the
number of steps /operations.
n Complexity time can be determined by
1. Count the number of steps and then find the
class of complexity.
Or
2. Find the complexity time for each steps and
then count the total.
Determine the number of steps
n Thefollowing algorithm is categorized as
O(n).
int counter = 1;
int i = 0;
for (i = 1; i <= n; i++) {
cout << "Arahan cout kali ke " << counter << "\n";
counter++;
}
Determine the number of steps
Num statements
int counter = 1;
1
int i = 0;
2
i=1
3
i <= n
4
i++
5
cout << "Arahan cout kali ke " << counter << "\n"
6
counter++
7
Determine the number of steps
n Statement3, 4 & 5 are the loop control
and can be assumed as one statement.
Num Statements

int counter = 1;
1
int i = 0;
2
i = 1; i <= n; i++
3
cout << "Arahan cout kali ke " << counter << "\n"
6
counter++
7
Determine the number of steps-
summation series
n statement 3, 6 & 7 are in the repetition
structure.
n It can be expressed by summation series
n
∑ f(i)= f(1) + f(2) + . . . + f(n) = n
i=1
Where
f(i) – statement executed in the loop
Determine the number of steps-
summation series
n example:- if n = 5, i = 1

5
∑ f(i)= f(1) + f(2) + f(3) + f(4) + f(5) = 5
i=1

The statement that represented by f(i) will be repeated

5 times
Determine the number of steps-
summation series
n example:- if n = 5, i = 3

5
∑ f(i)= f(3) + f(4) + f(5) = 3
i=3

The statement that represented by f(i) will be repeated

3 time
Determine the number of steps-
summation series
n Example: if n = 1, i = 1

1
∑ f(i)= f(1) = 1
i=1

The statement that represented by f(i) will be executed

only once.
Determine the number of steps
statements Number of steps
int counter = 1;
1
∑ f(i)= 1
i=1
int i = 0;
1
∑ f(i)= 1
i=1
i = 1; i= n; i++ n
∑ f(i)= n
i=1
cout << "Arahan cout kali ke " << counter << n 1
"\n" ∑ f(i). ∑ f(i) = n . 1 = n
i=1 i=1
counter++ n 1
∑ f(i). ∑ f(i)= n . 1 = n
i=1 i=1
Determine the number of steps

n Total steps:

1 + 1 + n + n + n = 2 + 3n

Consider the largest factor.

n Algorithm complexity can be categorized
as O(n)
 Algorithm Number of Steps
void sample4 ( ) 0
{ 0
for (int a=2; a<=n; a++) n-2+1=n-1
cout << “ Contoh kira langkah “; (n-1).1=n-1
} 0
Total Steps 2(n-1)

Total steps =2(n-1), Complexity Time = O (n)

Algorithm Number of steps
void sample5 ( ) 0
{ 0
for (int a=1; a<=n-1; a++) n-1-1+1=n-1
cout << “ Contoh kira langkah “; (n-1).1=n-1
} 0
Total steps 2(n-1)

31
 Continued…

Algorithm Number of Steps

void sample6 ( ) 0
{ 0
for (int a=1; a<=n; a++) n-1+1=n
for (int b=1; b<=n; b++) n.(n-1+1)=n.n
cout << “ Contoh kira langkah “; n.n.1=n.n
} 0
Total Steps n+2n2

Total Steps =n+2n2 ;

Complexity Time = O (n2)

32
 Continued……
Algorithm Number of Steps
void sample7 ( ) 0
{ 0
for (int a=1; a<=n; a++) n-1+1=n
for (int b=1; b<=a; b++) n.(n+1)/2
cout << “ Contoh kira langkah “; n.(n+1)/2
} 0
Total steps 2n+n2

Total steps =2n+n2

Complexity Time= O (n2)

33
 Continued…
n n
∑ ∑1 =
a=1 b=1
=n( 1 + 2 + 3 + 4 +…+n)

= n (n+1)
2
= n2 + n
2

To get n.(n+1)/2,

we used summation series as shown above:

34
Determine the number of steps
- exercise
Count the number of steps and find the
Big ‘O’ notation for the following algorithm
int counter = 1;
int i = 0;
int j = 1;

for (i = 3; i <= n; i = i * 3) {
while (j <= n) {
cout << "Arahan cout kali ke " << counter << "\n";
counter++;
j++;
}
}
Determine the number of steps
- solution
statements Number of steps

int counter = 1;
1
∑ f(i)= 1
int i = 0; i=1
1
∑ f(i)= 1
int j = 1; i=1
1
∑ f(i)= 1
i=1
i = 3; i <= n; i = i * 3
n
∑ f(i)= f(3) + f(9) + f(27) + … + f(n) = log3n
i=3
j <= n
n n
∑ f(i). ∑ f(i)= log3n . n
i=3 j=1
 Determine the number of steps
- solution
cout << "Arahan cout kali ke " n n 1
<< counter ∑ f(i). ∑ f(i). ∑ f(i)= log3n . n . 1
i=3 j=1 i=1
<< "\n";
n n 1
counter++; ∑ f(i). ∑ f(i). ∑ f(i)= log3n . n . 1
i=3 j=1 i=1
n n 1
j++; ∑ f(i). ∑ f(i). ∑ f(i)= log3n . n . 1
i=3 j=1 i=1
Determine the number of steps
- solution
Total steps:

=> 1 + 1+ 1 + log3n + log3n . n + log3n . n . 1 + log3n . n . 1 + log3n . n . 1

=> 3 + log3n + log3n . n + log3n . n + log3n . n + log3n . n

=> 3 + log3n + 4n log3n

Determine the number of steps :
solution 3 + log3n + 4n log3n
n Consider the largest factor
(4n log3n)
n and remove the coefficient
(n log3n)
n In asymptotic classification, the base of the log can be
omitted as shown in this formula:
logan = logbn / logba
n Thus, log3n = log2n / log23 = log2n / 1.58…
n Remove the coefficient 1/1.58..
n So we get the complexity time of the algorithm is
O(n log2n)
 Continued…
Algorithm No. of Steps

void sample8 ( ) 0
{ 0
int n, x, i=1; 1
while (i<=n) n
{ 0
x++; n.1 = n
i++; n.1 = n
} 0
} 0
Number of Steps 1 + 3n

Consider the largest factor : 3n

and remove the coefficient : O(n)

40
 Continued…
Algorithm Number of Steps

void sample9 ( ) 0
{ 0
int n, x, i=1; 1
while (i<=n) 1+log2n
{ 0
x++; 1+log2n
i=i*2; 1+log2n
} 0
} 0
Jumlah langkah 1+ 3(1+log2n)

•Notasi O = O (log2n)

41
 Continued…
Algorithm Number of Steps

void sample10 ( ) 0
{ 0
int n, x, i=1; 1
while (i<=n) 1+log4n
{ 0
x++; 1+log4n
i=i*4; 1+log4n
} 0
} 0
Number of steps 3(1+log4n)

•While loop iterate from i=1 until i=n ; i increment 4 times at each
iteration
• Number of iteration for while loop = ( 1 + log4n )

42
 Continued…
Number of steps =3(1+log4n)

n Using asimtotik O, where base for the alogarithm is ignored

:

logan = logbn/logba

Then;

log4n=log2n/log24=(log2n)/2

 Complexity Time = O (log2n)

43