Chapter 2

Introduction
Algorithm
Analysis

1
 Content
This Chapter Covers:

Data Structure
Primitive and Non-primitive Data structures
Algorithm Analysis:
Empirical and Theoretical analysis
Best case, Worst Case, Average Case Analysis
Order of Magnitude: Big-O, Ω, Θ, little-o and Little-Omega

2
 Data Structure
A data structure is a way of storing data in a computer so that it
can be used efficiently.
Organizing or handling of data in computer memory and the
operation that may be performed upon them.
Consider as an example books in a library. It can be stored in a
shelf arbitrarily or using some defined orders such as sorted by
Title, Author and so on.
It can be said that the study of data structures involves the
storage, retrieval and manipulation of information.
In other words, the possible logical arrangement of data items to
3
 Operations on data Structures
Abstract Data Type(ADT)
The abstract datatype is consists of data type to be stored and operation
supported to them.

contains set of value and operation

The various operations that can be performed over Data Structures are
Create
Delete
Access/retrieve/ display/view
Update
Searching
Storing 4
 Continued…
Primitive Data Structures
Integers, Real Numbers, Characters, Logical, Pointers,…
Non-Primitive Data Structure
Non-Primitive data structures are those that can be
obtained from the primitive data structures.
If the allocation of memory is sequential or continuous
then such a data structure is called a non-primitive linear
structure.
Example: arrays, stacks, queues, linked lists, etc.
If the allocation of memory is not sequential but random
or discontinuous then such a data structure is called a
Non-primitive non-linear structure.
• Example: Trees, Graphs, etc.
5
 Continued…
Primitive Data Structures vs Non-Primitive Data Structure
Primitive data structure that stores the data of only one type
Non-primitive data structure is a type of data structure which is a
user-defined that stores the data of different types in a single
entity.

6
 Algorithm Analysis
Algorithm is a step by step procedure to solve a problem.
E.g. Student department placement, Student Registration,
etc., all need an algorithm to follow.

The purpose of an algorithm is to accept input values, to

change a value hold by a data structure, to re-organize the
data structure itself (e.g. sorting), to display the content of
the data structure, and so on.

7
 Algorithm Analysis
Let's understand the properties of a good algorithm:

Definiteness/Unambiguity: Each step must be clearly defined, having

one and only one interpretation. At each point in computation,
one should be able to tell exactly what happens next.
Feasibility : It must be possible to perform each instruction.
Correctness : It must compute correct answer for all possible legal
inputs.
Language Independence: It must not depend on any one programming
language.
Effectiveness: It must be possible to perform each step exactly
and in a finite
amount of time
Efficiency: It must solve with the least amount of computational
resources 8
 Algorithm Analysis
Algorithm Efficiency
The efficiency of an algorithm is mainly defined by two factors i.e. space and time.
A good algorithm is one that is taking less time and less space, but this is not possible all
the time. If you want to reduce the time, then space might increase. Similarly, if you want
to reduce the space, then the time may increase.
Space Complexity
Space Complexity of an algorithm denotes the total space used or needed by the algorithm
for its working, for various input sizes.
Time Complexity
The time complexity is the number of operations an algorithm performs to complete its
task with respect to input size
• Input Size: Input size is defined as total number of elements present in the input. For a
given problem we characterize the input size n appropriately.

9
 Algorithm Analysis Concepts
 Algorithm analysis is concerned with determining the
efficiency of algorithms in the case of computational
resource.
It’s the resource requirement of an algorithm to solve
a specific problem
Memory usage
Running time
 Why need algorithm analysis ?

How we measure the efficiency/performance of

algorithms?

Two approaches:-
10
Empirical (Computational) Analysis
1. Empirical (Computational) Analysis
Here the total running time of the program is considered.
Final-initial time

It uses the system time to calculate the running time and

it can’t be used for measuring efficiency of algorithms.
(actual clock time)

Drawback of empirical analysis

Its difficult to use actual clock because clock time varies

based on :
Processor speed 11
 2. Theoretical (Asymptotic Complexity) Analysis
Determining the quantity of resource required using
mathematical concepts.

Analyze an algorithm according to the number of basic

operation (time unit)

We use theoretical approach to determine the efficiency of

algorithm

 The number of operation is not varies under different condition

 Helps to determine the complexity of algorithm

Complexity analysis :- systematic study of the cost of

12
computation when run program
How do we estimate the complexity of algorithms?

1. Algorithm analysis – Find T(n)- helps to determine the

complexity of an algorithm

2. Order of Magnitude – the rate at which the storage or

time grows as function of problem size (function

(n)).

13
 Algorithm Analysis Rule:
1. Assume an arbitrary time unit
2. Execution of one of the following operations takes time
unit 1
Assignment statement Eg. Sum=0;
Single I/O statement;. E.g., cin>>sum; , cout<<sum;
Single Boolean statement. E.g. !done
Single arithmetic. E.g., a+b
Function return. E.g., return(sum);
3. selection statement
Time for condition evaluation + the maximum time of its
clauses
4. Loop statement
1 + n+1 + n (initialization time + checking + update)

14
Estimation of Complexity of Algorithm
1. Calculate T(n) for the following
k=0;
Cout<<“enter an integer”;
Cin>>n;
For (i=0;i<n;i++)
K++
T(n) = 1+1+1+ (1+n+1+ n +n)
=5+3n

15
Estimation of Complexity of Algorithm
2. i=0;
While (i<n){
x++;
i++;
}
J=1;
While(j<=10){
x++;
j++
}
T(n)=1+n+1+n+n+1+11+10+10
=3n+34

16
Estimation of Complexity of Algorithm
3. for(i=1;i<=n;i++)
for(j=1;j<=n; j++)
k++;
T(n)=1+n+1+n+n(1+n+1+n+n)=3n2+4n+2

17
Estimation of Complexity of Algorithm
4. Sum=0;
if(test==1)
{
for (i=1;i<=n;i++)
sum=sum+i
}
else
{
cout<<sum;
}
T(n)=1+1+(1+n+1+n+n+n,1)= 4n+4

18
Estimation of Complexity of Algorithm
5.
Sample code  1 for the assignment.
int partial_sum= 0;
 1 assignment, n+1tests, and
for (int i = 1; i <=
n increments.
n;i++)
 n loops of 4 units for an
partial_sum=
assignment, an addition,
partial_sum+ (i * i
and two multiplications.
* i);
 1 for the return statement.
return partial_sum;
 T (n) = 1+(1+n+1+n)
+4n+1 =
6n+4

19
 Exercise
Calculate T(n) for the following codes
A. sum=0;
for(i=1;i<=n;i++)
for(j=1;j<=m;j++)
Sum++;

Two algorithms for computing the Factorial

Which one is best?
2)
1) int factorial (int n)
int factorial (int n) {
{
if (n<=1) return 1; else
if (n < = 1) return 1;
{
else fact = 1;
return n * factorial(n-1); for (k=2; k<=n; k++) fact
20
*= k;
 Formal Approach
In the above examples we have seen that analysis so complex.
 However, it can be simplified by using some formal
approach in which case we can ignore initializations, loop
control, and book keeping.

21
 Algorithm Analysis Categories
Algorithm must be examined under different situations to
correctly determine their efficiency for accurate
comparisons.
Best Case Analysis: Assumes that data are arranged in the
most advantageous order. It also assumes the minimum
input size.
Minimum number of operation performed
E.g.
 For sorting – the best case is if the data are arranged in
the required order.
 For searching – the required item is found at the first
position.
Note: Best Case computes the lower boundary of T(n). 22
 Average case analysis
Assumes that data are found in random order.
It also assumes random or average input size.
E.g.
• For sorting – data are in random order.
• For searching – the required item is found at any
position or missing.
It computes optimal bound of T(n)
It also causes average number of execution.
Best case and average case can not be used to estimate
(determine) complexity of algorithms
Worst case is the best to determine the complexity of
algorithms.

23
 Worst case analysis
Assumes that data are arranged in the disadvantageous
order.
It also assumes that the input size is infinite.
Eg.
• For sorting – data are arranged in opposite required
order
• For searching – the required item is found at the end
of the item or the item is missing
It computes the upper bound of T(n) and causes
maximum number of execution.

24
 Order Of Magnitude
Order Of Magnitude refers to the rate at which the storage or
time grows as function of problem size (function (n)).
It is expressed in terms of its relationship to some known
functions – asymptotic analysis.
Asymptotic Notation is used to describe the running time of
an algorithm - how much time an algorithm takes with a
given input.
Types of Asymptotic Notations
1. Big-O Notation: The Big O notation defines an upper
bound of an algorithm, it bounds a function only from
above. (worst case analysis)
2. Big – Omega (Ω): Just as Big O notation provides an
asymptotic upper bound on a function, Ω notation
provides an asymptotic lower bound.(best case analysis
3. Big –Theta (Θ): The theta notation bounds a function
from above and below, so it defines exact asymptotic 25
 1. Big-O Notation
Definition :The function F(n) is O(g(n)) if there exist
constants c and k such that F(n) ≤ c.g(n) for all n ≥ k.

f(n) = O(g(n)) f(n) is asymptotically less than or equal to

g(n).

As n increases, F(n) grows no faster than g(n) or in the

long run (for large n) F grows at most as fast as g(n)

It computes the tight upper bound of F(n)

Describes the worst-case analysis.

Properties of Big-O Notation
 Examples
1 Find g(n) =n show that F(n) = O(g(n)) for F(n) = 3n+5

Solution: we must show that constants c and k such that

10n+5≤ c.n for n>=k
c=15, K=1
10n+5<=15n for n>=1(subtract 10n both sides)
5<=5n for n>=1
1<=n for N>=1

10n+5 = O(g(n)) for k=1,c=15

 Exercise
1. F(n)=5n+6, g(n)=n show that F(n)=O(g())

2. f(n) = 3n2+4n+1. g(n)=n2 Show that f(n)=O(g(n))

 2. Big – Omega
Definition : The function F(n) is Ω(g(n)) if there exist
Constants c and K such that
F(n) ≥ c.g(n) for all n ≥ K.
As n increases F(n) grows no slower than g(n) or in the
long run (for large n) F grows at least as fast as g(n)
It computes the tight lower bound of F(n).
Describes the best-case analysis
 Examples
Find g(n) such that F(n) = Ω(g(n))
for F(n) = 3n+5
g(n) = n as 3n+5>=c.n for c=1

Ex f(n) =n2 , g(n)=n, show that F(n)= Ω(g(n))

 3. Big Theta Notation
Definition: The function F(n) is Θ(g(n)) if there exist
constants c1, c2 and K such that c1.g(n) ≤ F(n) ≤ c2.g(n)
for all n ≥ K.
As n increases, F(n) grows as fast as g(n)
It computes the tight optimal bound of F(n).
Describes the average case analysis.
Example:
• Find g(n) such that F(n) = Θ(g(n)) for F(n)=2n2.
• Solution: c1n2 ≤ 2n2 ≤ c2n2
• G(n)=n2 because for c1=1 and c2=3,
• n2 ≤ 2n2 ≤ 3n2 for all n.
F(n)=3n+1?
Asymptotic Notation
Reading Assignment
Little-o Notation

Little-Omega
 Exercise
Find T(n) of the following algorithm
1. for(i=1;i<=n;i++)
Sum=sum+i;
For (i=1;i<=n;i++)
For(j=1;j<=m;j++)
Sum++;
 …
2. if(k==1)
{For (i=1;i<=100;i++)
For (j=1;j<=1000;j++)
Cout<<I
}Else if(k==2)
{
For(i=1;i<=n;i=i*2)
Cout<<i;
}Else
{
{for (i=1;i<=n;i++)
Sum++;
}
 …
3. for (i=1; i<=n;i++)
{
If (i<10)
For(j=1;j<=n;j=j*2)
Cout<<j;
Else
Cout<<i;
}
 …
4. for (int i=1; i<=N; ++i)
for (int j=i; j>0; --j)
{} // do nothing
5. for (int i=1; i<N; i=i*2)
{}
6. for (int i=0; i<N; ++i)
for (int j=i; j>0; j=j/2)
{}
 …
7. for (int i=1; i<N; ++i)
for (int j=0; j<N; j+=i)
{}
8. int i=0, sum=0;
while (sum < N)
sum = sum + i++;
9. for (int i=1; i<N; i=i*2)
for (int j=0; j<i; ++j)
{}
 …
10. i=1;
while (i<n)
{
i=i*2;
}
11. for ( int i = 0; i < n; i++ )
for ( int j = 0; j <= n - i; j++ )
int a = i;
for (int i = 0; i < n; i ++ )
for (int j = 0; j < n*n*n ; j++ )
sum++;
 ….
12. void Test( int N){
for(int i =0 ; i < N ; i= i*4){
for(int j = N ; j > 2; j = j/3){
cout<<“I have no idea what this does”;
}
}
}
13. for (int i = 0; i < n; i++)
sum++;
for(int j = 0; j < n; j++)
sum++;
 …

???
End!!!!!!!!

42