Course Title: Data Structure & Algorithm.

Credit Hour: 3 hrs.

Course Code: CoSc2091.
ECTS: 5 [2 Lecture hours and 3 Lab hours]
Lecture Schedule: Every ____________

Bedasa Wayessa
[email protected]

Data Structures and Algorithms - CoSc2091 1

Introduction to Data Structure and
Algorithms

Part II: Algorithm Concepts

Bedasa Wayessa
[email protected]

Data Structures and Algorithms - CoSc2091 2

Outline
 Algorithms
 Properties of an algorithm
 Algorithm Analysis Concepts
 Complexity Analysis
 Asymptotic Analysis

Data Structures and Algorithms - CoSc2091 3

 Introduction to Algorithms
 Definition
 An algorithm is a step-by-step procedure to solve a problem.
 It is a well-organized, pre-arranged, and defined computational
module that receives some values or set of values as input and
provides a single or set of values as output.
 An algorithm is said to be accurate and truthful only when it
provides the exact wanted output.
 Efficiency
 The efficiency of an algorithm depends on the time and space
complexities.
 The complexity of an algorithm is the function which gives the
running time and/or space in terms of the input size.

Data Structures and Algorithms - CoSc2091 4

 Introduction to Algorithms
 Algorithm Analysis Concepts
– Time complexity: The amount of time taken by an algorithm to
run as a function of the size of its input.
– Space complexity: The amount of memory used by an algorithm
to run as a function of the size of its input.
– Asymptotic Analysis
– Asymptotic analysis is a technique used to analyze the performance
of an algorithm as the input size becomes very large.
– It involves analyzing the behavior of the algorithm as the input size
approaches infinity.
– The goal is to determine how the running time or space
requirements of the algorithm grow as the input size increases.

Data Structures and Algorithms - CoSc2091 5

 Introduction to Algorithms
 Steps Required to Develop an Algorithm:
– Finding a method for solving a problem.
• Every step of an algorithm should be defined in a precise and in a
clear manner.
• Pseudo code is also used to describe an algorithm.
– The next step is to validate the algorithm.
• This step includes all the steps in our algorithm and should be
done manually by giving the required input, perform the required
steps including in our algorithm and should get the required
amount of output in a finite amount of time.
– Finally implement the algorithm in terms of programming language.

Data Structures and Algorithms - CoSc2091 6

 Introduction to Algorithms
• How to write Algorithm
– Note that an algorithm can be described in many ways:
• Natural Language
• Pseudo code
• Programming Language
• Algorithmic Notations:
– While writing the algorithm the comments are provided with in [ ].
– The assignment should use the symbol “: =” instead of “=”
– For Input use Read : variable name
– For output use write : message/variable name
 The control structures can also be allowed to use inside an algorithm
but their way of approaching will be some what different as.
Data Structures and Algorithms - CoSc2091 7
 Introduction to Algorithms: Example
 Finding the maximum element problem
 Input: an array A storing n integers.
 Output: the maximum element in A.
 An algorithm for finding the maximum element
Algorithm arrayMax(A, n):
Input: An array A storing n ≥ 1 integers.
Output: The maximum element in A.
currMax ← A[0]
Designing Algorithm
for i ← 1 to n−1 do
– Read : two number inputs
if currMax < A[i] then
– Process: numbers using the
currMax ← A[i] + operator
return currMax – Write: the result

Data Structures and Algorithms - CoSc2091 8

 Introduction to Algorithms
• Quiz:
1. What is the main difference between an algorithm and a program?
2. What will it be If a program is running much more slowly than the
algorithm analysis suggests?
• Key:
– It is a well-organized, pre-arranged, and defined computational module that
receives some values or set of values as input and provides a single or set
of values as output.
– A program, on the other hand, is a set of instructions for the computer
to follow.
– It is a sequence of instructions written in a specific programming language
that tells the computer what to do.
– A program can implement one or more algorithms, or it may be so simple
that we don’t have to use an algorithm

Data Structures and Algorithms - CoSc2091 9

 Properties of Algorithm
A. Input/Output specified
– Every algorithm should have a specified number (Zero or more)
input values (or quantities) which are externally supplied.
– We must specify : The type of data and The amount of data
– Every algorithm should have one or more output procedures.
B. Finiteness (It must terminate)
– Every valid algorithm should have completed after a finite number of
steps.
C. Correctness:
– It must compute correct answer for all possible legal inputs.
– It is most important criteria to evaluate an algorithm.

Data Structures and Algorithms - CoSc2091 10

 Properties of Algorithm
D. Definiteness (Absence of Ambiguity)
– Each step must be clearly defined, having one and only one interpretation.
– At each point in computation, one should be able to tell exactly what
happens next.
– Definiteness ensures that if the algorithm is performed at different times
or by different agents (people or computers) using the same data, the
output will be the same.
E Effectiveness:
– It must be possible to perform each step exactly and in a finite amount
of time.
– It must be possible to perform each instruction i.e. it means that they are
doable.

Data Structures and Algorithms - CoSc2091 11

 Properties of Algorithm
F. Efficiency : Algorithm must solve the problem with the least amount
of computational resources, Such as time and space.
G. Sequential:
– Each step must have a unique defined preceding and succeeding step.
– The first step (start step) and last step (halt step) must be clearly noted.
H. Simplicity:
– We would like our algorithms to be easy to understand and easy to
implement. - There often a tradeoff between efficiency and simplicity.
K. Completeness: It must solve the problem completely.
L. Language independence:
– It must not depend on any one programming language.

Data Structures and Algorithms - CoSc2091 12

Algorithm Analysis Concepts

Data Structures and Algorithms - CoSc2091 13

 Introduction to Algorithm Analysis
 Why do we bother to analyze an algorithm?
– For some of us analyzing algorithms is an intellectual activity that is fun.
– Another reason is the challenge of being able to predict the future and
even though we are narrowing our predictions to algorithms, it is
gratifying when we succeed.
– A 3rd reason is because computer science attracts many people who
enjoy being efficiency experts.
 Analyzing algorithms gives these people a chance to exhibit their
skills by devising new ways of doing the same task even faster.
 This tendency has a large payoff in computing where time means money
and efficiency saves dollars.

Data Structures and Algorithms - CoSc2091 14

 Introduction to Algorithm Analysis
 An algorithm is a clearly specified set of simple instructions to be
followed to solve a problem.
 Once an algorithm is given for a problem and decided (somehow) to be
correct, an important step is to determine how much in the way of
resources, such as time or space, the algorithm will require.
 An algorithm that solves a problem but requires a year is hardly of any
use.
 Likewise, an algorithm that requires thousands of gigabytes of main
memory is not (currently) useful on most machines.
 Each of our algorithms will involve a particular data structure.

Data Structures and Algorithms - CoSc2091 15

 Introduction to Algorithm Analysis
• What to Analyze?
– It is difficult to use actual clock-time to measure efficiency of algorithm.
– clock-time can vary based on many things:
• Specific processor speed
• Current processor load
• Specific data for a particular run of the program
• Input Size and Input Properties
• Operating Environment
– So, analyze an algorithm based on the number of operations required,
rather than absolute amount of time involved
• algorithm’s efficiency changes according to the size of the input

Data Structures and Algorithms - CoSc2091 16

 Introduction to Algorithm Analysis
 What to Analyze?
int largest(int A[], int n) {
int currlarge = 0;
for (int i = 0; i < n; i++)
if (A[currlarge] < A[i])
return currlarge;
}
 The most important resource to analyze is generally the running time.
 Other main factors are the algorithm used and the input to the
algorithm.
 Typically, the size of the input is the main consideration.

Data Structures and Algorithms - CoSc2091 17

 Introduction to Algorithm Analysis
 Why is it necessary to analyze an algorithm?
– An algorithm analysis measures the efficiency of the algorithm.
– The efficiency of an algorithm can be checked by:
1. The correctness of an algorithm.
2. The implementation of an algorithm.
3. The simplicity of an algorithm.
4. The execution time and memory space requirements of an
algorithm.
5. The new ways of doing the same task even faster.

Data Structures and Algorithms - CoSc2091 18

 Complexity Analysis
 In order to solve a problem there are many possible algorithms that
differ in efficiency.
 To compare the efficiency of algorithms, a measure of the degree of
difficulty of an algorithm called computational complexity.
 Complexity in algorithms refers to the amount of resources (such as
time or memory) required to solve a problem or perform a task.
 Complexity analysis is the systematic study of the cost of a
computation, measured either in time units or in operations
performed, or in the amount of storage space required.
 The goal is to have a meaningful measure that permits comparison of
algorithms and/or implementations independent of operating platform.

Data Structures and Algorithms - CoSc2091 19

 Complexity Analysis
• The complexity of an algorithm M is the function f(n) which gives the
running time and/ for storage space requirement of algorithm in
terms of the size n of the input data.
• The complexity of an algorithm is normally expressed using the Big O
notation

Data Structures and Algorithms - CoSc2091 20

 Complexity Analysis: Space Complexity
 The space complexity of a program is the amount of memory it needs
to run to completion.
 The space needed by a program is the sum of the following
components:
– A Fixed space requirements: part that includes space for the code:
– space for simple variables and fixed size component variables, space
for constants, etc.
– A Variable space requirements: that includes the space needed by:
– component variables whose size is dependent on the particular
problem instance being solved, and the stack space used by recursive
procedures.
– S(P) = c + Sp(I)
Data Structures and Algorithms - CoSc2091 21
 Complexity Analysis: Space Complexity
 Example 1: We want to add a list of numbers (Program 1).
– Although the output is a simple value, the input includes an array.
– Therefore, the variable space requirement depends on how the array is
passed into the function.
– C passes all parameters by value.
– When an array is passed as an argument to a function, C interprets it as
passing the address of the first element of the array.
– C does not copy the array.

– Therefore, Ssum(n)= 0.
float sum(float list[], int n) {
float tempsum = 0;
for (int i =0; i < n; i++)
tempsum += list[i];
return tempsum;
}
Data Structures and Algorithms - CoSc2091 22
 Complexity Analysis:Time Complexity
 Time complexity is a function describing the amount of time an
algorithm takes in-terms of the amount of input to the algorithm.
 It is the amount of computer time it needs to run to completion.
 The time is measured by counting the number of key operations:
– in sorting and
– searching algorithms, for example, the number of comparisons.
 It is most important one which does not use real time.
 The time complexity is of two types such as
 Compilation time and Runtime
 Compilation Time: the amount of time taken by the compiler to
compile an algorithm.

Data Structures and Algorithms - CoSc2091 23

 Complexity Analysis:Time Complexity
 During compilation time it does not calculate for the executable
statements, it calculates only the declaration statements and checks for
any syntax and semantic errors.
 The run time depends on the size of an algorithm.
 If the number of instructions in an algorithm is large, then the run time is
also large, and if the number of instructions in an algorithm is small, then
the time for executing the program is also small.
 The runtime is calculated for executable statements and not for
declaration statements.

Data Structures and Algorithms - CoSc2091 24

 Complexity Analysis:Time Complexity
 Basics of Time Complexity in Algorithms
Notation Name Running Time
Execution time is independent of the size of
O(1) Constant Time
the input
Execution time grows linearly with the size of
O(n) Linear Time
the input
Execution time grows quadratically with the
O(n2) Quadratic Time
size of the input
Logarithmic Execution time grows logarithmically with the
O(log n)
Time size of the input
Execution time grows exponentially with the
O(𝟐𝟐𝒏𝒏) Exponential
size of the input
 The goal with time complexity analysis is to achieve a low time
complexity for better performance.
 The lower the time complexity, the faster the algorithm executes.
Data Structures and Algorithms - CoSc2091 25
 Complexity Analysis:Time Complexity
 Measures of Times:
 Suppose space is fixed for one algorithm then only run time will be
considered for obtaining the complexity of algorithm, these are:
 Best case
 Worst case
 Average case
 In order to determine the running time of an algorithm it is possible to
define three functions Tbest(n), Tavg(n) and Tworst(n) as the
best, the average and the worst case running time of the algorithm
respectively.

Data Structures and Algorithms - CoSc2091 26

 Complexity Analysis:Time Complexity
 Best case:
 the best case is when the number of steps is the smallest.
 Best case takes shortest time to execute, as it causes the algorithms
to do the least amount of work.
 It is the amount of time the algorithm takes on the smallest possible
set of inputs.
 It cheats with a slow algorithm that works fast on some input.
 It compute the lower bound of T(n) Beast case is the cheapest
among all inputs of size n.

Data Structures and Algorithms - CoSc2091 27

 Complexity Analysis:Time Complexity
 Average case:
 The average case falls between these extremes.
 Analyzing the average case behavior algorithm is a little bit complex
than the best case and worst case.
 The amount of time the algorithm takes on an average set of inputs
 T(n) = expected time(mean time) of algorithm over all in puts of size n
 It compute the optimal (the one which is found most of the time)
bound of T(n), it also need assumption of statistical distribution of
inputs.
 Gives the average performance of an algorithm.

Data Structures and Algorithms - CoSc2091 28

 Complexity Analysis:Time Complexity
 Worst case:
 The worst case is when an algorithm requires a maximum number of
steps.
 It is the amount of time algorithm takes on the worst possible set of
inputs T(n) = maximum time of algorithm on any input of size n
 It compute the upper bound of T(n) , the longest running time for any
input size n.
 We are interested in the worst-case time, since it provides a bound
for all input – this is called the “Big-Oh” estimate.

Data Structures and Algorithms - CoSc2091 29

 Complexity Analysis:Time Complexity
 For example:
 Consider the problem of searching an array containing n integers to find
the one with a particular value K (assume that K appears exactly once in
the array).
 The sequential search algorithm begins at the first position in the array
and looks at each value in turn until K is found.
• Once K is found, the algorithm stops.
– The first integer in the array could have value K – Best Case
– If the last position in the array contains K – Worst Case
– The algorithm examines about n/2 values to find K – Average Case

Data Structures and Algorithms - CoSc2091 30

 Complexity Analysis:Time Complexity
 For example:
 Consider the problem of searching an array containing n integers to find
the one with a particular value K (assume that K appears exactly once in
the array).
 The sequential search algorithm begins at the first position in the array
and looks at each value in turn until K is found.
• Once K is found, the algorithm stops.
– The first integer in the array could have value K – Best Case
– If the last position in the array contains K – Worst Case
– The algorithm examines about n/2 values to find K – Average Case

Data Structures and Algorithms - CoSc2091 31

 How to Determine Complexities
 To calculate the time complexity for any algorithm, we can follow
these steps:
1. Identify the input size.
2. Identify the number of basic operations performed by the
algorithm.
3. Determine how many times each operation will be executed.
4. Express the total number of operations as a function of the input
size.
5. Simplify the function using Big O notation.

Data Structures and Algorithms - CoSc2091 32

 How to Determine Complexities
 In general, how can you determine the running time of a piece of code?

the frequency count

x  x+y
this statement is 1.

for i  1 to n do
x  x+y
this statement is n.
repeat

for i  1 to n do
for j  1 to n do
x  x +y this statement is n2.
repeat
repeat

Data Structures and Algorithms - CoSc2091 33

 How to Determine Complexities
 In general, how can you determine the running time of a piece of code?
 The answer is that it depends on what kinds of statements are used.
 General Rules for Estimation
 Execution of one of the following operations takes time 1:
– Assignment statement
– Single read/write statement (I/0)
– Single Boolean operations, numeric comparisons
– Single arithmetic operation
– Function return
– array index operations, pointer dereferences

Data Structures and Algorithms - CoSc2091 34

 How to Determine Complexities
1. Sequence of statements
statement 1;
statement 2;
...
statement k;
 The total time is found by adding the times for all statements:
total time = time(statement 1) + time (statement 2)
+ ... + time(statement k).
 If each statement is “simple” (only involves basic operations) then
the time for each statement is constant and the total time is also
constant: O(1).

Data Structures and Algorithms - CoSc2091 35

 How to Determine Complexities
2. If-then-else statements
 Running time of a selection statement (if, switch) is the
 time for the condition evaluation + the maximum of the running times for the
individual clauses in the selection.
if (cond) {
sequence of statements 1
}else {
sequence of statements 2
}
 Here, either sequence 1 will execute, or sequence 2 will execute.
 Therefore, the worst-case time is the slowest of the two possibilities:
max(time(sequence 1), time(sequence 2)).
 For example, if sequence 1 is O(N) and sequence 2 is O(1) the
worst-case time for the whole if-then-else statement would be O(N).
Data Structures and Algorithms - CoSc2091 36
 How to Determine Complexities
2. If-then-else statements
Time Units to Compute
 Simple Example ---------------------------------------------
if (x > 5) { 1 for the if comparison statement:
for (int i = 0; i <= n; i++){ x>5
cout<<i; In the for loop:
} 1 assignment, n+1 tests, and n
}else { increments.
cout<<"Hello, "<<endl; n loops of 1 units
} 1 for an output
 Time = 1+max(n,1) = 1+n = O(n) 1 for the else statement.
-----------------------------------------------
T (n)= 1+ max(n,1) = n +1 = O(n)

Data Structures and Algorithms - CoSc2091 37

 How to Determine Complexities
3. for loops statements
– Loop execution time is the sum, over the number of times the loop is
executed, of the body time + time for the loop check and update
operations, + time for the loop setup.
– Always assume that the loop executes the maximum number of
iterations possible
for (i = 0; i < N; i++) {
sequence of statements
}
– The loop executes N times, so the sequence of statements also
executes N times.
– Since we assume the statements are O(1), the total time for the for
loop is N * O(1), which is O(N) overall.
Data Structures and Algorithms - CoSc2091 38
 How to Determine Complexities
4. Nested loops statements
for (i = 0; i <= N; i++) {
Outer Loops executes n times

for (j = 0; j <= M; j++) { Inner Loops executes n times

sequence of statements
}
}
• Let see how it works: assume N & M is equal value
I=1 I=2 I=3 I=4 …
J = 1 to n J = 1 to n J = 1 to n J = 1 to n

– Total time = n * n = O(n2)

• Let see how it works: assume N & M is Not equal value
– Total number of iterations is (N+1)(M+1).

Data Structures and Algorithms - CoSc2091 39

 How to Determine Complexities
4. Nested loops statements
for (i = 0; i < N; i++) {
for (j = 0; j < M; j++) {
sequence of statements
}
}
– The outer loop executes N times.
– Every time the outer loop executes, the inner loop executes M times.
– As a result, the stmt in the inner loop execute a total of N * M times.
– Thus, the complexity is O(N * M).
– In a common special case where the stopping condition of the inner loop
is j < N instead of j < M (i.e., the inner loop also executes N times), the
total complexity for the two loops is O(N2).

Data Structures and Algorithms - CoSc2091 40

 How to Determine Complexities
• Exercise
• Determine the run time equation and complexity of the
following code segments.
for(int i =1; i <= n; i++){
for (int j = 1; j <= i; j++){
sum++;
}
}
What is the value of the sum if n = 20?
I=1 I=2 I=3 … I=n
J = 1 to 1 J = 1 to 2 J = 1 to 3 … J = 1 to n

• The total sum can be calculated as 1 + 2 + 3 + ... + n

• Can be expressed as (n * (n + 1)) / 2 = (20 * (20 + 1)) / 2 = 210.

Data Structures and Algorithms - CoSc2091 41
 How to Determine Complexities
5. Statements with method calls:
– Let’s suppose that we have the following segment of code:
for (int count = 0; count < n; count++) {
printsum(count); //method called
}

– Loop rule: find the order of the loop by multiplying the order of
the body of the loop by the number of times the loop will execute.
– In this case, however, the body of the loop is a method call.
– Therefore, we must determine the order of the method before we
can determine the order of the code segment.
– Let’s suppose that the purpose of the method is to print the sum
of the integers from 1 to n each time it is called.
Data Structures and Algorithms - CoSc2091 42
 How to Determine Complexities
5. Statements with method calls:
– Let’s suppose that we have the following segment of code:
for (int count = 0; count < n; count++) {
printsum(count); //method called
}
void printsum(int count) {
int sum = 0;
for (int i = 1; i < count; i++){
sum += i;
}
cout<<sum;
}
– What is the time complexity of this printsum method?
– The loop, on the other hand, is O(n), and thus the method itself is O(n).

Data Structures and Algorithms - CoSc2091 43

 How to Determine Complexities
5. Statements with method calls:
– Now, to compute the time complexity of the original loop that
called the method:
– we simply multiply the complexity of the method, which is the
body of the loop, by the number of times the loop will execute.
– Our result, then, is O(n2) using this implementation of the
printsum method.
– However: if we use n(n + 1)/2, to calculate the sum of the
numbers from 1 to n. void printsum(int count) {
sum = count*(count+1)/2;
cout<<sum;
}
Data Structures and Algorithms - CoSc2091 44
 How to Determine Complexities
5. Statements with method calls:
– Now the time complexity of the printsum method is made up of an
assignment statement that is O(1) and a print statement that is also O(1).
– The result of this change is that the time complexity of the printsum
method is now O(1), which means that the loop that calls this method
now goes from being O(n2) to being O(n).
– There is one additional issue to deal with when analyzing the time
complexity of method calls, and that is recursion—the situation when a
method calls itself.
– Running time of a function call is 1 for setup plus the time for
any parameter calculations plus the time required for the
execution of the function body.
Data Structures and Algorithms - CoSc2091 45
 How to Determine Complexities
 A Simple Example 1

int total(int n){ Time Units to Compute

int sum = 0; -------------------------------------------------
1 for the assignment statement:
for(int i=1;i<=n;i++) int sum=0
In the for loop:
sum = sum + 1; 1 assignment, n+1 tests, and n increments.
return sum; n loops of 2 units
1 for an assignment, and
} 1 for an addition.
1 for the return statement.
-----------------------------------------------------------
T (n)= 1+ (1+n+1+n)+2n+1 = 4n+4 = O(n)

Data Structures and Algorithms - CoSc2091 46

 How to Determine Complexities
 A Simple Example 2
Time Units to Compute
-------------------------------------------------
int sum(int n) { 0 for the declarations:
int partialSum; int partialSum;
1 for the assignment statement:
partialSum = 0; partialSum = 0;
for(int i = 1; i <= n; ++i ) In the for loop:
1 assignment, n+1 tests, and n increments.
{ n loops of 4 units
partialSum += i * i * i; 2 for multiplications
1 for one addition, and
} 1 for assignment) and
return partialSum; 1 for the return statement.
---------------------------------------------
} 0 + 1+(1+n+1+n)+4n+1 = 6n+4 is O(N)

Data Structures and Algorithms - CoSc2091 47

 How to Determine Complexities
 A Simple Example 3
Time Units to Compute
-------------------------------------------------
int count(){ 1 for the assignment statement:
int k = 0; int k=0
1 for the output statement.
cout<< “Enter an integer”; 1 for the input statement.
cin>>n; In the for loop:
1 assignment, n+1 tests, and n increments.
for(i = 0; i < n; i++) n loops of 2 units
{ 1 for an assignment, and
1 for an addition.
k = k + 1; 1 for the return statement.
} -----------------------------------
T (n)= 1+1+1+(1+n+1+n)+2n+1 = 4n+6 = O(n)
return 0;
}

Data Structures and Algorithms - CoSc2091 48

 How to Determine Complexities
 A Simple Example 4
void func() { Time Units to Compute
-------------------------------------------------
int x = i = 0; 1 for the first assignment statement: x=0;
int j = 1; 1 for the second assignment statement: i=0;
1 for the third assignment statement: j=1;
cout<< “Enter value”; 1 for the output statement.
cin>>n; 1 for the input statement.
In the first while loop:
while (i < n){ n+1 tests
x++; n loops of 2 units for the two increment
(addition) operations
i++; In the second while loop:
} while (j < n){ n tests
n-1 increments
j++; -----------------------------------
T (n)= 1+1+1+1+1+n+1+2n+n+n-1 = 5n+5 = O(n)
}
}

Data Structures and Algorithms - CoSc2091 49

 How to Determine Complexities
 A Simple Example 5
 Example 1:Swap two numbers

void swap(x, y) {
int temp;
t = x; 1 f(n) = 3
x = y; 1 sum -constant running time.
y = temp; 1
=>f(n)=O(1)
}

Data Structures and Algorithms - CoSc2091 50

 Exercises
 Determine the run time equation and complexity of each of the
following code segments.
for (i = 0; i<n; i++)
for (j = 0; j<n; j++)
A. sum = sum + i + j;

for(int i = 1; i <= n; i++)

for (int j = 1; j <= i; j++)
B. sum++;

What is the value of the sum if n=20?

1. [True/False] The time complexity of a loop is found by multiplying the

complexity of the body of the loop by how many times the loop will
execute.

Data Structures and Algorithms - CoSc2091 51

 Exercises
 Determine the run time equation and complexity of each of the
following code segments.

for (i = 0; i <= N; i++) {

A. for (j = 0; j <= N; j++) {
cout<< “Hello”;
break;
}
}

Data Structures and Algorithms - CoSc2091 52

 Comparing Algorithm Growth rates…

1<logn < n < nlogn < 𝒏𝒏𝟐𝟐 <n 𝒏𝒏𝟑𝟑 <…< 𝟐𝟐𝒏𝒏 < 𝟑𝟑𝒏𝒏 <…< 𝒏𝒏𝒏𝒏
Exponential

Data Structures and Algorithms - CoSc2091 53

Comparing Algorithm Growth rates…

Data Structures and Algorithms - CoSc2091 54

 Asymptotic Analysis
• Asymptotic analysis is concerned with how the running time of an
algorithm increases with the size of the input in the limit, as the size of
input increases without bound, (estimating the rate of function growth).
• There are five notations used for specifying asymptotic complexity
(estimating a rate of running time function growth)
– Big – oh notation ( O)
– Big – Omega Notation Ω)
– Theta Notation (Θ )
– Little 0 Notation (o)
– Little omega Notation (ω )

Data Structures and Algorithms - CoSc2091 55

 Big–oh notation ( O)
• Big-O notation is a way to analyze and express the upper bound or
worst-case behavior of a function.
• The main idea is to focus on the dominant or largest term in the expression to
determine how the function behaves as the input size grows.
• In Big-O notation, only the most significant term is considered, while
less significant terms are ignored.
• For example, in a function T(n) = n2 - n, the term n is insignificant
compared to n2, so the n term is disregarded when determining the
overall behavior of the function.
• Informally it is defined as follow:
– If f is given function, and if some function g is an upper bound for f,
then we say that f is big O of g.

Data Structures and Algorithms - CoSc2091 56

 Formal definition of Big-Oh
 We define O(g) to be the set of functions f such that for some pair of
non negative constants C and k, f(n) ≤ Cg(n) for all n ≥ k
 Equivalently can be defined as follow:
 f(n) is O(g(n)) if there exist positive number C and k such that
f(n) ≤ Cg(n) for all n ≥ k or If f is O(g):
𝒇𝒇(𝒏𝒏) • f is order g
 if 𝐥𝐥𝐥𝐥𝐥𝐥 ≈ 𝟎𝟎 or constant not infinite.
𝒏𝒏→∞ 𝒈𝒈(𝒏𝒏)
• g is an upper bound
on the value of f
• f grows at most as
fast as g
• f εO(g)
• f = O(g)
Data Structures and Algorithms - CoSc2091 57
 Big-Oh: Example 1
1. What is the worst-case complexity of the each of the following code
fragments? Two loops in a row:
for (i = 0; i < N; i++) {
sequence of statements
}for (j = 0; j < M; j++) {
sequence of statements
}
 Answer: The first loop is O(N) and the second loop is O(M).
– Since you do not know which is bigger, you say this is O(N+M).
– This can also be written as O(max(N, M)).
– If the second loop goes to N instead of M the complexity is O(N).
– O(N+M) becomes O(2N) and when you drop the constant it is O(N).
– O(max(N, M)) becomes O(max(N, N)) which is O(N).

Data Structures and Algorithms - CoSc2091 58

 Big-Oh: Example 2
• Show that 3x2+25 is O(x2).
– To show that 3x2+25 is O(x2), we need to find two constants C
and k such that:
• |3x2 + 25| <= C|x2| for all x > k
– Let’s assume that C = 4 and k = 1. Then, we have:
• |3x2 + 25| <= 4|x2| for all x > 1
– We can simplify the inequality as follows:
• 3x2 + 25 <= 4x2 for all x > 1
– Subtracting 3x2 from both sides, we get:
• 25 <= x2 for all x > 1
– Or this also works: |3x2 + 25| <= |3x2 + 25x2|

Data Structures and Algorithms - CoSc2091 59

 Big-Oh: Example 3
 Show that f(n) is O(g(n))
• f(n) = 10n + 5
• g(n) = n
• To show f(n) ε O(g(n)) we have to find C and k such that C ≥ 0 and k ≥
0 such that f(n) ≤ Cg(n) for all n ≥ k
= ( 10n + 5) ≤ Cn for all n ≥ k
• Let’s fix n to be 15, then we need to show that 10n+5≤15n for all n
greater than some constant k
10n + 5 ≤ 15n = 5 ≤ 5n = 1 ≤ n
• So f(n) = 10n + 5 ≤ 15g(n) for all n≥1
• Therefore f(n) = O(n)

Data Structures and Algorithms - CoSc2091 60

 Big-Oh: Example 4
 f(n) = 3n2 + 4n + 1 show that f(n)= O(n2 )
= 4n ≤ 4n2 for all n>1
= 3n2 + 4n ≤ 4n2 +3n2 for all n>1
= 3n2 +4n+1 ≤ 4n2 +3n2 +n2 for all n>1
= f(n) = 8n2
f(n) ≤8 (n2 ) for all n > 1
Therefore f(n) ε O(n2 )
• Show that 3x2 + 25 is O(x2).

Data Structures and Algorithms - CoSc2091 61

 Big Omega
 Definition:
 Just as O-notation provides an asymptotic upper bound on a
function, Ω notation provides an asymptotic lower bound.
 f(n) = 𝜴𝜴(g(n)), (read as "f of n equals omega of g(n)") if and only if
there exist positive constants c and 𝒏𝒏𝟎𝟎 such that for all n > 𝒏𝒏𝟎𝟎 ,
𝒇𝒇 𝒏𝒏 ≥ 𝒄𝒄 𝒈𝒈 𝒏𝒏 .
 specifically the big-Omega notation, if we say that f(n) = Ω(g(n)),
it means that f(n) grows at least as fast as a constant multiple of g(n) for
sufficiently large values of n.
 Formally, f(n)=Ω(g(n)) if there exist positive constants c and k

such that f(n) ≥ c ⋅ g(n) for all n ≥ k.

Data Structures and Algorithms - CoSc2091 62

 Big Omega
 Example
 Let's consider two functions: f(n) = 2n + 5 and g(n) = n.
 We want to show that f(n) = Ω(g(n)), which means that f(n)
grows at least as fast as g(n) for sufficiently large values of n.
 Solution:
 find constants c and k such that f(n) ≥ c⋅g(n) for all n ≥ k.
 Taking c = 1 and k = 5, we can see that for all 5n ≥ 5:
= Now, let's check if f(n) ≥ c * g(n) for n ≥ 5:
= 2n + 5 ≥ 1 * n (for n ≥ 5)
= 2n + 5 ≥ n
= n ≥ 5 (which is true for n ≥ 5)
 This means that for all values n ≥ 5, the f(n) is at least as large as n.
Data Structures and Algorithms - CoSc2091 63
 Theta Notation
• Definition:
• Theta notation is one of the three asymptotic notations used to
describe the running time of an algorithm when the input size
becomes large.
• It provides both upper and lower bounds for a function, considering
constant factors.
• In other words, it characterizes the average time complexity of an
algorithm.
• Formally, a function f(n) is said to be Θ(g(n)) if and only if there
exist positive constants c1, c2, and k such that for all n ≥ k:
c1​
⋅g(n) ≤ f(n) ≤ c2​
⋅g(n)

Data Structures and Algorithms - CoSc2091 64

 Theta Notation
• Definition:
• In simpler terms, Θ(g(n)) represents a tight bound on the growth rate
of f(n), where g(n) serves as both an upper and lower bound for
f(n) up to a constant factor.
• Suppose we have a function f(n) = 3n2 + 2n + 1 and another
function g(n)=n2. We want to show that f(n) = Θ(g(n)).
• First, we need to find constants c1, c2, and k such that:
c1​
⋅g(n) ≤ f(n) ≤ c2​
⋅g(n)
• Let's choose c1 = 1, c2 = 4, and k=1. Now, for all n ≥ 1:
1⋅n2 ≤3n2 + 2n + 1 ≤ 4⋅n2
• So, f(n) = Θ(g(n)) with c1​
=1, c2​
=4, and k=1.

Data Structures and Algorithms - CoSc2091 65

 Little-o Notation (o):
• Definition:
• Little o notation characterizes an upper bound that is not tight.
• In other words, it gives a loose upper estimate for the growth of a
function (f(n)).
• The notation f(n) = o(g(n)) is read as "f(n) is little-o of g(n)." This
means that f(n) grows strictly slower than g(n).
• More formally, f(n) = o(g(n)) if, for every positive constant c>0,
there exists a constant k such that:
• 0 ≤ f(n) < c⋅g(n)
• In other words, the growth rate of f(n) is negligible compared to the
growth rate of g(n).
• The function f(n) grows much slower than g(n).

Data Structures and Algorithms - CoSc2091 66

 Little-o Notation (o):
• Definition:
• Little-o Notation (o):
• If f(n) = 2n and g(n)=n2, then f(n) = o(g(n)), because 2n grows much
slower than n2.
• In mathematical terms, for any constant c>0, you can find a k such that
0 ≤ 2n < c⋅n2 for all n ≥ k.

Data Structures and Algorithms - CoSc2091 67

 Little omega Notation (ω):
• Definition:
• Little omega notation describes a loose lower bound for the
growth of a function (f(n)).
• The notation f(n) = ω(g(n)) is read as "f(n) is little-omega of g(n)."
This means that f(n) grows strictly faster than g(n).
• Formally, f(n) = ω(g(n)) if, for every positive constant c>0, there
exists a constant k such that:
0 ≤ c⋅g(n) < f(n)
• In this case, the growth rate of f(n) is significantly faster than the growth
rate of g(n).

Data Structures and Algorithms - CoSc2091 68

 Little omega Notation (ω):
• Definition:
• Little-omega Notation (ω):
• If f(n) = n2 and g(n) = logn, then f(n) = ω(g(n)), because n2
grows much faster than logn.
• For any constant c > 0, there exists a k such that
0 ≤ c⋅logn < n2 for all n≥k.
• In summary, little-o notation (o) is used to describe functions that
grow strictly slower than another function,
• while little-omega notation (ω) is used to describe functions that
grow strictly faster than another function.

Data Structures and Algorithms - CoSc2091 69

 Reading Assignment
• Big Omega
• Theta Notation
• Little oh
• Little Omega

Data Structures and Algorithms - CoSc2091 70

 End of Chapter 1

Next: Simple Sorting and Searching

Algorithms

Data Structures and Algorithms - CoSc2091 71