Design and Analysis of

Algorithms
COMPILED BY: GELANA. A
 Contents to be covered

 Introduction to Algorithm analysis

 Asymptotic Notations
 Analysis of Algorithm
 Review of elementary Data Structures
 Heaps
 Hashing
CHAPTER 1
INTRODUCTION AND ELEMENTARY
DATA STRUCTURE
Overview of Data Structure and Algorithm

 An algorithm is a step-by-step procedure or formula for solving a problem or

performing a task.
 It consists of a sequence of instructions that are executed in a specific order to achieve
a desired outcome.
 Algorithms are fundamental to computer science and programming, as they determine
how efficiently a problem can be solved.
 Common examples include search algorithms (like binary search), sorting algorithms
(like quicksort), and algorithms for graph traversal (like depth-first search).
 A data structure is a specialized format for organizing, processing, and storing data in
a computer. It defines the way data is arranged and accessed, allowing for efficient data
management and manipulation.
 Common examples include arrays, linked lists, stacks, queues, trees, and graphs. Each
data structure is optimized for specific types of operations and use cases.
 Cont.

 Data structures model the static part of the world. They are
unchanging while the world is changing.
 In order to model the dynamic part of the world we need to work
with algorithms.
 Algorithms are the dynamic part of a program’s world model
 An algorithm transforms data structures from one state to another
state in two ways:
1. An algorithm may change the value held by a data structure or
2. An algorithm may change the data structure itself
Cont…

 The quality of a data structure is related to its ability to

successfully model the characteristics of the world.
 Similarly, the quality of an algorithm is related to its ability to
successfully simulate the changes in the world.
 However, independent of any particular world model, the
quality of data structure and algorithms is determined by their
ability to work together well.
 Generally speaking, correct data structures lead to simple and
efficient algorithms and correct algorithms lead to accurate and
efficient data structures.
.

Algorithm vs program
Properties of an algorithm

 Finiteness: Algorithm must complete after a finite number of steps.

 Definiteness: Each step must be clearly defined, having one and
only one interpretation. At each point in computation, one should
be able to tell exactly what happens next.
 Sequence: Each step must have a unique defined preceding and
succeeding step. The first step (start step) and last step (halt step)
must be clearly noted.
 Feasibility: It must be possible to perform each instruction.
 Correctness: It must compute correct answer for all possible legal
inputs.
 Language Independence: It must not depend on any one
programming language.
Cont..

 Completeness: It must solve the problem completely.

 Effectiveness: It must be possible to perform each step exactly and
in a finite amount of time.
 Efficiency: It must solve with the least amount of computational
resources such as time and space.
 Generality: Algorithm should be valid on all possible inputs.
 Input/Output: There must be a specified number of input values,
and one or more result values.
Algorithm Specification

 Algorithm can be described in two ways. And they are; Pseudo-

code and flowchart.

1. FLOWCHART
▪ Flowchart is graphic representation of an algorithm.
▪ Easy to understand the logic of complicated and lengthy problems
 Example: Algorithm to find the average of three numbers
2. PSEUDO-CODE METHOD:

 Pseudo code is a type of structured English that is used to specify an

algorithm.
 This cannot be compiled and not executed.
 Example: Pseudo-code to find the average of three numbers
1. START
2. READ values of X,Y,Z
3. COMPUTE S=X+Y+Z
4. COMPUTE A=S/3
5. WRITE the value of A
6. STOP
Algorithm Analysis Concepts

 Algorithm analysis refers to the process of determining the amount of computing time
and storage space required by different algorithms.
 In other words, it’s a process of predicting the resource requirement of algorithms in a
given environment.
 In order to solve a problem, there are many possible algorithms.
 One has to be able to choose the best algorithm for the problem at hand using some
scientific method.
 To classify some data structures and algorithms as good, we need precise ways of
analyzing them in terms of resource requirement. The main resources are:
1. Running Time
2. Memory Usage
3. Communication Bandwidth
Cont…

 Running time is usually treated as the most important since computational time is the
most precious resource in most problem domains.
 There are two approaches to measure the efficiency of algorithms:
1. Empirical: Programming competing algorithms and trying them on different
instances.
2. Theoretical: Determining the quantity of resources required mathematically
(Execution time, memory space, etc.) needed by each algorithm.
 However, it is difficult to use actual clock-time as a consistent measure of an
algorithm’s efficiency, because clock-time can vary based on many things. For
example,
1. Specific processor speed
2. Current processor load
Cont…

3. Specific data for a particular run of the program

• Input Size
• Input Properties
4. Operating Environment
 Accordingly, we can analyze an algorithm according to the
number of operations required, rather than according to an
absolute amount of time involved. This can show how an
algorithm’s efficiency changes according to the size of the
input
Importance of algorithm analysis

 There are many algorithms to solve a problem

 Requirement is to find the most efficient algorithm to
solve the problem. The efficiency is in terms of times and
memory space consumption.
 Through algorithm analysis we can select the most
efficient algorithm out of all possible algorithms to solve a
problem.
Example
Example for more integer.. 100 let
say
Complexity Analysis

 Complexity Analysis is the systematic study of the cost of

computation, measured either in time units or in operations
performed, or in the amount of storage space required.
 The goal is to have a meaningful measure that permits comparison
of algorithms independent of operating platform.
 There are two things to consider:
1. Time Complexity: Determine the approximate number of
operations required to solve a problem of size n.
2. Space Complexity: Determine the approximate memory required
to solve a problem of size n.
Complexity analysis involves two distinct
phases:
➢ Algorithm Analysis: Analysis of the algorithm or data structure to
produce a function T (n) that describes the algorithm in terms of the
operations performed in order to measure the complexity of the
algorithm.
➢ Order of Magnitude Analysis: Analysis of the function T (n) to
determine the general complexity category to which it belongs.
➢ There is no generally accepted set of rules for algorithm analysis.
However, an exact count of operations is commonly used.
Analysis Rules:

1. We assume an arbitrary time unit.

2. Execution of one of the following operations takes time 1:
• Assignment Operation
• Single Input/Output Operation
• Single Boolean Operations
• Single Arithmetic Operations
• Function Return
 3. Running time of a selection statement (if, switch) is the time for
the condition evaluation + the maximum of the running times for the
individual clauses in the selection.
Cont…
4. Loops: Running time for a loop is equal to the running time for the
statements inside the loop * number of iterations.
 The total running time of a statement inside a group of nested loops is the
running time of the statements multiplied by the product of the sizes of all
the loops.
 For nested loops, analyze inside out.
 Always assume that the loop executes the maximum number of iterations
possible.
5. Running time of a function call is 1 for setup + the time for any parameter
calculations + the time required for the execution of the function body.
Example 1

int count(){
int k=0;
cout<< “Enter an integer”;
cin>>n;
for (i=0;i<n; i++)
k=k+1;
return 0;
}
Time Units to Compute
 1 for the assignment statement: int k=0
 1 for the output statement.
 1 for the input statement.
 In the for loop:
 1 assignment, n+1 tests, and n increments.
 n loops of 2 units for an assignment, and an addition.
 1 for the return statement.
 -------------------------------------------------------------------
 T (n)= 1+1+1+(1+n+1+n)+2n+1 = 4n+6 = O(n)
Example 2
 int total(int n)
 {
 int sum=0;
 for (int i=1;i<=n;i++)
 sum=sum+1;
 return sum;
 }
Time Units to Compute
 1 for the assignment statement: int sum=0
In the for loop:
 1 assignment, n+1 tests, and n increments.
 n loops of 2 units for an assignment, and an addition.
 1 for the return statement.
 T (n)= 1+ (1+n+1+n)+2n+1 = 4n+4 = O(n)
Example 3
int sum (int n)
{
int partial_sum = 0;
for (int i = 1; i <= n; i++)
partial_sum = partial_sum +(i * i * i);
return partial_sum;
}
Time Units to Compute
 1 for the assignment.
 1 assignment, n+1 tests, and n increments.
 n loops of 4 units for an assignment, an addition, and two
multiplications.
 1 for the return statement.
 -------------------------------------------------------------------
 T (n)= 1+(1+n+1+n)+4n+1 = 6n+4 = O(n)
Formal Approach to Analysis
 In the above examples we have seen that analysis is a bit complex.
However, it can be simplified by using some formal approach in
which case we can ignore initializations, loop control, and book
keeping.
for Loops: Formally
 In general, a for loop translates to a summation. The index and
bounds of the summation are the same as the index and bounds of
the for loop.

for (int i = 1; i <= N; i++) {

sum = sum+i; }

 Suppose we count the number of additions that are done. There is 1

addition per iteration of the loop, hence N additions in total.
Nested Loops: Formally
 Nested for loops translate into multiple summations, one for each for
loop.
for (int i = 1; i <= N; i++) {
for (int j = 1; j <= M; j++) {
sum = sum+i+j;
}
}
Again, count the number of additions. The outer summation is for
the outer for loop.
Consecutive Statements: Formally
 Add the running times of the separate blocks of your code
for (int i = 1; i <= N; i++) {
sum = sum+i; }
for (int i = 1; i <= N; i++) {
for (int j = 1; j <= N; j++) {
sum = sum+i+j; } }
Conditionals: Formally
 If (test) s1 else s2: Compute the maximum of the running time for s1 and
s2.
if (test == 1) {
for (int i = 1; i <= N; i++) {
sum = sum+i;
}}
else
for (int i = 1; i <= N; i++) {
for (int j = 1; j <= N; j++) {
sum = sum+i+j;
}}
Measures of Times

 In order to determine the running time of an algorithm it is possible to

define three functions T best(n), T avg(n) and T worst(n) as the best, the
average and the worst-case running time of the algorithm respectively.
 Average Case (T avg): The amount of time the algorithm takes on an
"average" set of inputs.
 Worst Case (T worst): The amount of time the algorithm takes on the worst
possible set of inputs.
 Best Case (T best): The amount of time the algorithm takes on the smallest
possible set of inputs.
 We are interested in the worst-case time, since it provides a bound for all
input – this is called the “Big-Oh” estimate.
Asymptotic Analysis

 Asymptotic analysis is concerned with how the running time of an

algorithm increases with the size of the input in the limit, as the size
of the input increases without bound.
Big Oh(O)notation:

 Big O notation specifically describes worst case scenario.

 The big oh notation describes an upper bound on the asymptotic
growth rate of the function f.
 f(n) = O(g(n)) (read as “f of n is big oh of g of n”)
 Definition: Assuming f(n) and g(n) are non-negative functions
 f(n) = O(g(n)) iff f(n) <= c.g(n) for all values of n where n>=n0 and c
and n0 are constants.
 This is to mean O(g(n)) is asymptotically bigger than f(n)
Cont..

 The definition states that the function f(n) is at most c times the
function g(n) except when n is smaller than n0.
 In other words, f(n) grows slower than or same rate as” g(n).
 Examples
–f(n) = 3n+2
3n + 2 <= 4n, for all n >= 2, 3n + 2 = O(n)
–f(n) = 10n2+4n+2
10n2 +4n+2 <= 11n2, for all n >= 5,
10n2 +4n+2 = O(n2)
Omega(Ω)notation:

 The omega notation describes a lower bound on the asymptotic

growth rate of the function f.
 Definition:[Omega]
 f(n) = Ω(g(n)) (read as “f of n is omega of g of n”) iff there exist
positive constants c and n0 such that f(n)>=cg(n) for all n, n>n0.
 The definition states that the function f(n) is at least c times the
function g(n) except when n is smaller than n0.
 In other words, f(n)grows faster than or same rate as g(n).
Cont.…

Examples
–f(n) = 3n+2
3n + 2 >= 3n, for all n >= 1, 3n + 2 = Ω (n)
–f(n) = 10n2+4n+2
10n2 +4n+2 >= n2 , for all n >= 1, 10n2+4n+2 = Ω (n2)
Theta(Θ)notation:

 The Theta notation describes a tight bound on the asymptotic growth

rate of the function f.
 Definition:[Theta]
 f(n) = Θ(g(n)) (read as “f of n is theta of g of n”) iff there exist
positive constants c1, c2,and n0 such that c1g(n) <f(n) <c2g(n)for all
n, n>n0.
Cont.

 The definition states that the function f(n) lies between c1 times the
function g(n) and c2 times the function g(n) except when n is
smaller than n0.
 In other words, f(n)grows same rate as g(n).
 Example
 f(n) = 3n+2
 3n <= 3n + 2 <= 4n, for all n >= 2, 3n + 2 = Θ(n)
 f(n) = 10n2+4n+2
 n2<= 10n2+4n+2 <= 11n2, for all n >= 5, 10n2+4n+2 = Θ(n2)
Heap Sort

 Heap sort operates by first converting the list in to a heap tree. Heap
tree is a binary tree in which each node has a value greater than both
its children (if any). It uses a process called "adjust to accomplish its
task (building a heap tree) whenever a value is larger than its parent.
The time complexity of heap sort is O(nlogn).
Hashing

 A hash table is a data structure that stores data and allows insertions,
lookups, and deletions to be performed in constant, O(1), time.
 Hash tables provide a means for rapid searching for a record with a
given key value and adapts well to the insertion and deletion of
records.
 It is a technique used for performing insertions, deletions and finds
in constant time.
 The aim of hashing is to map an extremely large key space onto a
reasonable small range of integers such that it is unlikely that two
keys are mapped onto the same integer.
.

END OF CHAPTER 1