07/05/2024

WOLLO UNIVERSITY
KOMBOLCHA INSTITUTES OF TECHNOLOGY
School of ECEG

Data Structures and Algorithms

Chapter 2
Algorithm and Algorithm Analysis

Tegegn.A.
[email protected]

Outline
Introduction to Algorithm
Properties of Algorithm
Analysis of Algorithm

5/7/2024 2

1
 07/05/2024

What is an algorithm?
 An algorithm is a finite set of well-defined instructions to be followed for solving a
problem.

Problem

Algorithm

Input Computer Output

5/7/2024 3

Properties of an algorithm
Every algorithm must satisfy the following criteria:
 Finiteness: Algorithm must complete after a finite number of steps.
 Definiteness: Each step must be clearly defined, having one and only one
interpretation.
 Sequence: Each step must have a unique defined preceding and succeeding step. The
first step (start step) and last step (halt step) must be clearly noted.
 Feasibility: It must be possible to perform each instruction.

5/7/2024 4

2
 07/05/2024

…cont’d
 Correctness: It must compute correct answer for all possible legal inputs.
 Language Independence: It must not depend on any one programming language.
 Completeness: It must solve the problem completely.
 Effectiveness: It must be possible to perform each step exactly and in a finite amount
of time. Steps are sufficiently simple and basic
 Efficiency: It must solve with the least amount of computational resources such as
time and space.
 Generality: Algorithm should be valid on all possible inputs.
 Input/Output: There must be a specified number of input values, and one or more
result values.

5/7/2024 5

Algorithm Design and Analysis Process

5/7/2024 6

3
 07/05/2024

Algorithm Analysis Concepts

Algorithm analysis refers to the process of determining how much
computing time and storage that algorithms will require.
In other words, it’s a process of predicting the resource requirement of
algorithms in a given environment.
The main resources are:
 Computational Time (Compile Time and Running Time)
 Memory Usage
 Communication Bandwidth

5/7/2024 7

..cont’d
Analysis of algorithm is important for the following reasons
 Analysis is more reliable than the experimental testing
 Analysis helps to select better algorithm
 Analysis predicts performance
 Analysis identifies the scope of improvement of algorithm

5/7/2024 8

4
 07/05/2024

Approaches to measure the efficiency of algorithms

There are two approaches to measure the efficiency of algorithms:
1. Empirical: Programming competing algorithms and trying them on different
instances.
2. Theoretical: Determining the quantity of resources required mathematically
(Execution time, memory space, etc.) needed by each algorithm.

5/7/2024 9

Complexity Analysis of Algorithms

Complexity Analysis is the systematic study of the cost of computation,
measured either in time units or in operations performed, or in the amount
of storage space required.
The time and space are two major measures of the efficiency of an
algorithm.
The complexity of an algorithm is the function which gives the running time
and/or space in terms of the input size.

5/7/2024 10

5
 07/05/2024

…cont’d
Complexity analysis involves two distinct phases:
Algorithm Analysis: Analysis of the algorithm or data structure to produce
a function T (n) that describes the algorithm in terms of the operations
performed in order to measure the complexity of the algorithm.
Order of Magnitude (Asymptotic) Analysis: Analysis of the function T
(n) to determine the general complexity category to which it belongs.

5/7/2024 11

…cont’d
The factor of time is more important than that of space, so efficiency
considerations usually focus on the amount of time elapsed which
processing data.

5/7/2024 12

6
 07/05/2024

Space Complexity
The space complexity of an algorithm is the amount of memory it needs to
run to completion.
An algorithm generally requires space for following components :
 Instruction Space: Its the space required to store the executable version of the
program. This space is fixed, but varies depending upon the number of lines of code
in the program.
 Data Space: Its the space required to store all the constants and variables(including
temporary variables) value.
 Environment Space: Its the space required to store the environment information
needed to resume the suspended function.

5/7/2024 13

….cont’d
Space complexity includes both Auxiliary space and space used by input.
Auxiliary Space is the extra space or temporary space used by an
algorithm.
Space Complexity of an algorithm is total space taken by the algorithm
with respect to the input size.

5/7/2024 14

7
 07/05/2024

…cont’d
Generally, the space needed by any algorithm is seen to be the sum of the
following two components.
1. Fixed part:
 This is independent of the characteristics of the input and output.
 This part includes the space for the code, space for simple variables, space for the
fixed size component variables and constants and so on.
2. Variable part:
 This part consists of the space needed by component variables whose size is
dependent on the particular problem instance being solved, the space needed by
referenced variables and the recursion stack space.
5/7/2024 15

…cont’d
So, the space requirement S(P) of any algorithm P may be written as
S(P) = c + SP
Where SP is the instance characteristics and c is a constant.
Space complexity must be taken seriously for multiuser systems and in
situations where limited memory is available.

5/7/2024 16

8
 07/05/2024

Calculating the Space Complexity

For calculating the space complexity, we need to know the value of memory
used by different type of datatype variables.
// n is the length of array a[]
int sum(int a[], int n) {
int x = 0;
for(int i = 0; i < n; i++)
{
x = x + a[i];
}
return(x);
}

total memory requirement will be (4n + 12)

5/7/2024 17

Time Complexity
The time complexity of an algorithm is the amount of computer time it
needs to run to completion.
The time taken by a program is the sum of the compile time and run time.
The compile time does not depend on the instance characteristics.

5/7/2024 18

9
 07/05/2024

…cont’d
The run time may depend on:
 Specific Processor speed
 Type of compiler used
 Current Processor Load
 Specific data for a particular run of the program (Logical size of the program)
 Input Size
 Input Properties
 Available memory.
 Operating Environment
5/7/2024 19

Analysis Rules
1. We assume an arbitrary time unit.
2. Execution of one of the following operations takes time 1:
 Assignment Operation
 Single Input/Output Operation
 Single Boolean Operations
 Single Arithmetic Operations
 Function Return

5/7/2024 20

10
 07/05/2024

…cont’d
3. Running time of a selection statement (if, switch) is the time for the
condition evaluation + the maximum of the running times for the
individual clauses in the selection.
4. Loops: Running time for a loop is equal to the running time for the
statements inside the loop * number of iterations.
 The total running time of a statement inside a group of nested loops is the running
time of the statements multiplied by the product of the sizes of all the loops.
 For nested loops, analyze inside out.
Always assume that the loop executes the maximum number of iterations
possible.
5/7/2024 21

…cont’d
5. Running time of a function call is 1 for setup + the time for any parameter
calculations + the time required for the execution of the function body.
Example1:
int count(){
1 for the assignment statement: int k=0
int k=0;
cout<< “Enter an integer”; 1 for the output statement.
cin>>n; 1 for the input statement.
for (i=0;i<n;i++)
In the for loop:
k=k+1; 1 assignment, n+1 tests, and n increments.
return 0; n loops of 2 units for an assignment, and an addition.
} 1 for the return statement.

5/7/2024 T (n)= 1+1+1+(1+n+1+n)+2n+1 = 4n+6 = O(n) 22

11
 07/05/2024

…cont’d
while (i<n){
Example2:
x++;
void func()
i++;
{
}
int x=0;
while (j<n)
int i=0;
{
int j=1;
j++;
cout<< “Enter an Integer value”;
}
cin>>n;
}

T (n)= 1+1+1+1+1+n+1+2n+n+n-1 = 5n+5

5/7/2024 23

Formal Approach to Analysis

In the above examples we have seen that analysis so complex.
However, it can be simplified by using some formal approach in which case
we can ignore initializations, loop control, and book keeping.
For Loops: Formally
 In general, a for loop translates to a summation.
 The index and bounds of the summation are the same as the index and bounds of the
for loop.
N


f o r ( in t i = 1 ; i < = N ; i+ + ) {
s u m = s u m + i; 1  N
}
i 1
5/7/2024 24

12
 07/05/2024

…cont’d
Suppose we count the number of additions that are done.
There is 1 addition per iteration of the loop, hence N additions in total.
Nested Loops: Formally
 Nested for loops translate into multiple summations, one for each for loop.
fo r (in t i = 1 ; i < = N ; i+ + ) {
fo r (in t j = 1 ; j < = M ; j+ + ) { N M N

}
su m = s u m + i+ j; 
i 1 j 1
2   2M
i 1
 2 MN
}

Again, count the number of additions. The outer summation is for the outer
for loop.
5/7/2024 25

…cont’d
Consecutive Statements: Formally
 Add the running times of the separate blocks of your code
f o r ( in t i = 1 ; i < = N ; i+ + ) {
s u m = s u m + i;
}  N   N N 
  1     2  N  2 N 2
f o r ( in t i = 1 ; i < = N ; i+ + ) {
f o r ( in t j = 1 ; j < = N ; j + + ) {  i 1   i 1 j 1 
s u m = s u m + i+ j ;
}
}

5/7/2024 26

13
 07/05/2024

…cont’d
Conditionals: Formally
 If (test) s1 else s2: Compute the maximum of the running time for s1 and s2.
if (te s t = = 1 ) {
fo r (in t i = 1 ; i < = N ; i+ + ) {  N N N 
s u m = s u m + i; max   1 ,   2  
}}  i1 i1 j 1 
 
e ls e f o r ( in t i = 1 ; i < = N ; i+ + ) {
fo r (in t j = 1 ; j < = N ; j+ + ) { max N , 2 N 2
 2N 2
s u m = s u m + i+ j;
}}

5/7/2024 27

Example
Suppose we have hardware capable of executing 106 instructions per
second.
How long would it take to execute an algorithm whose complexity function
was:
T (n) = 2n2 on an input size of n=108?

Running time =2*1016/106 = 2*1010

5/7/2024 28

14
 07/05/2024

Asymptotic Analysis
Asymptotic analysis is concerned with how the running time of an
algorithm increases with the size of the input in the limit, as the size of the
input increases without bound.
Generally, the complexity is investigated in three cases.
1. Best Case − Minimum time required for program execution.
2. Average Case − Average time required for program execution.
3. Worst Case − Maximum time required for program execution.

5/7/2024 29

Asymptotic Notations
Following are the commonly used asymptotic notations to calculate the
running time complexity of an algorithm.
 Big-Oh Notation (O)
 Big-Omega Notation (W)
 Theta Notation (θ)

5/7/2024 30

15
 07/05/2024

Big-Oh notation,(O)
The notation Ο(n) is the formal way to express the upper bound of an
algorithm's running time.

5/7/2024 31

…cont’d
 Formal Definition:
 f (n) = O (g (n)) if there exist c, k ∊ ℛ+ such that for all n≥ k, f (n) ≤ c.g (n).
 Examples: The following points are facts that you can use for Big-Oh problems:
 1<=n for all n>=1
 n<=n2 for all n>=1
 2n <=n! for all n>=4
 log2n<=n for all n>=2
 n<=nlog2n for all n>=2

5/7/2024 32

16
 07/05/2024

…cont’d
f(n)=10n+5 and g(n)=n.
Show that f(n) is O(g(n)).
To show that f(n) is O(g(n)) we must show that constants c and k such that
f(n) <=c.g(n) for all n>=k
Or 10n+5<=c.n for all n>=k
Try c=15. Then we need to show that 10n+5<=15n
Solving for n we get: 5<5n or 1<=n.
So, f(n) =10n+5 <=15.g(n) for all n>=1.
(c=15,k=1).
5/7/2024 33

…cont’d
f(n) = 3n2 +4n+1.
Show that f(n)=O(n2).
4n <=4n2 for all n>=1 and 1<=n2 for all n>=1
3n2 +4n+1<=3n2+4n2+n2 for all n>=1
3n2 +4n+1 <=8n2 for all n>=1
So we have shown that f(n)<=8n2 for all n>=1
Therefore, f (n) is O(n2) (c=8,k=1)

5/7/2024 34

17
 07/05/2024

Omega Notation, Ω
The notation Ω(n) is the formal way to express the lower bound of an
algorithm's running time.

5/7/2024 35

…cont’d
Formal Definition:
 A function f(n) is Ω ( g (n)) if there exist constants c and k ∊ ℛ+ such that
 f(n) >=c. g(n) for all n>=k.
 f(n)>= Ω ( g (n)) means that f(n) is greater than or equal to some constant multiple of
g(n) for all values of n greater than or equal to some k.

5/7/2024 36

18
 07/05/2024

…cont’d
Examples:
1. If f(n)=3n + n = Ω(n) as 3n+3 >=n for all n >=1.
2. If f(n) =n2, then f(n)= Ω( n)
In simple terms, f(n)= Ω( g (n)) means that the growth rate of f(n) is greater
than or equal to g(n).

5/7/2024 37

Theta Notation Θ
It gives bound for a function f(n) to within a constant factor.
Formal Definition:
A function f (n) is Θ (g(n)) if it is both O( g(n) ) and Ω ( g(n) ).
 In other words, there exist constants c1, c2, and k >0 such that c1.g (n)<=f(n)<=c2. g(n) for all n >= k
If f(n) = Θ (g(n)) then g(n) is both upper and lower bound on f(n) ( g(n) is
an asymptotically tight bound for f(n)). i.e., the best and worst cases require
the same amount of time within a constant factor.
In simple terms, f(n)= Θ (g(n)) means that f(n) and g(n) have the same rate
of growth.
5/7/2024 38

19
 07/05/2024

…cont’d

5/7/2024 39

…cont’d
Example :
1. 3n+2 >=3n for all n>=1 and 3n+2<=4n for all n >=2 so C1 =3 and C2 = 4
and n0 = 2
⸫3n+2 = Θ(n)

2. If f(n)=2n+1, then f(n) = Θ(n)

5/7/2024 40

20
 07/05/2024

Review Questions
1. What is algorithm analysis?
2. Explain the difference between algorithm analysis approach
3. Write the importance of algorithm analysis?
4. Discuss complexity analysis phases with example.
5. Prove that running time T(n) = n3 + 20n + 1 is O(n3)

5/7/2024 41

…cont’d
6. Find the time complexity of the following algorithm?
1) 2)
void selection_sort(int list[]){ int max(int n)
int temp; {
int i,j, smallest;
int a=1;
for(i=0;i<list.length;i++){
if ( a< n) {
smallest=i;
a= n*( a‐1)+( n‐1)*( a‐2) 7
for(j=i+1;j<list.length;j++){
} else{
if(list[j]<list[smallest])
a++;
smallest=j;
}
}//end of inner loop
return a;
temp=list[smallest];
}
list[smallest]=list[i];
list[i]=temp;
} //end of outer loop
5/7/2024
}//end of selection 42

21
 07/05/2024

End of Ch.2
Questions, Ambiguities, Doubts, … ???

5/7/2024 43

22